Load Balancing With Apache Tomcat

Recently, we have to make our application work with more than one node which is currently using Tomcat as a server. We tried scalability using Apache Web Server and mod_jk with Tomcat. And it worked well for us. At the end we ended up using Amazon AWS inbuilt clustering support for our application. But whatever homework I have done for Tomcat clustering, I am logging here with simple steps.

mod_jk Tomcat Connector:

mod_jk is the connector used to connect the Tomcat servlet container with web servers such as Apache using the AJP protocol. Here is how this mechanism works:

A web server is responsible for managing client HTTP requests. When these requests come, the web server needs also to perform the following:

  • Load the servlet container connector library and initialize it.
  • It has to identify the URL and decide certain request belongs to a servlet, if so then give those requests to connector.

The connector needs to know what requests it is going to serve, and to where to direct these requests (based on configuration we provided).

Now, let’s configure mod_jk with Apache Web server. So, this is what you need to download:

  1. Apache Web Server 2.2.21 from http://httpd.apache.org/download.cgi.
  2. Apache Tomcat 6.0.20 from http://tomcat.apache.org/download-60.cgi.
  3. Mod_jk Tomcat connector from http://tomcat.apache.org/download-connectors.cgi

Now let’s start by installing Tomcat first.

  1. Extract the Tomcat zip you have downloaded. Hereafter, the directory you extracted to will be referred to as TOMCAT_1_HOME.
  2. Test Tomcat to see that it works. Go to TOMCAT_1_HOME\bin and run startup.bat.
  3. Open up your browser and access http://localhost:8080/
  4. If you see the default page, then Tomcat Instance 1 is working fine. Shut down Tomcat.

That’s all for the first Tomcat instance. Now, for the second.

  1.  Make a directory called TOMCAT_2_HOME and extract Tomcat.zip.
  2. Open up TOMCAT_2_HOME \conf\server.xml in a text editor. We’ve got to change the port numbers so that they don’t conflict with the first instance.
  3. Change <Server port=”8005″ shutdown=”SHUTDOWN”>  to  <Server port=”7005″ shutdown=”SHUTDOWN”>.
  4. Change <Connector port=”8080″ maxHttpHeaderSize=”8192″… to <Connector port=”7070″ maxHttpHeaderSize=”8192″…
  5. Change <Connector port=”8009″ enableLookups=”false” redirectPort=”8443″ protocol=”AJP/1.3″ /> to  <Connector port=”7009″ enableLookups=”false” redirectPort=”8443″ protocol=”AJP/1.3″ />
  6. Go to bin directory of tomcat2 and start the second tomcat using startup.bat.
  7. Test it out by pointing your browser to http://localhost:7070/
  8. Your second tomcat instance is now ready to be used.

Next, let’s set up the Apache HTTP Server. It’s very simple…

  1. Run the installer you downloaded. The standard install will do.
  2. Open the Apache Server Monitor and start the web server if it’s not already running.
  3. Point your browser to http://localhost/ to verify that Apache is running on port 80.
  4. Stop Apache.

Finally, we reach mod_ JK. Let’s set it up first just to delegate requests to the two Tomcat instances, and we’ll load balance it a bit later.

  1. Get mod_jk.so file from your downloaded mod_jk archive and copy it to the modules directory in your Apache installation.
  2. Open up httpd.conf in the conf directory of your Apache installation in a text edit, and
    add the following line at the end of the set of LoadModule statements:

                                  LoadModule jk_module modules/mod_jk.so 

       3.  Create a file called workers.properies in the conf directory. Add these lines to it:

                                worker.list=worker1,worker2

                                worker.worker1.port=8009

                                worker.worker1.host=localhost

                                worker.worker1.type=ajp13

                                worker.worker2.port=7009

                                worker.worker2.host=localhost #you can also specify other machine address or name

                                worker.worker2.type=ajp13

       4.  This file defines which workers Apache can delegate to. We’ve listed worker1 and worker2 to correspond to our two tomcat instances.

       5.   Specify the worker properties in httpd.conf:

             Add these lines just after the LoadModule definitions-

# Path to workers.properties

JkWorkersFile c:/apache2.2/conf/workers.properties

# Path to jk logs

JkLogFile c:/apache2.2/mod_jk.log

# Jk log level [debug/error/info] JkLogLevel info

# Jk log format

JkLogStampFormat “[%a %b %d %H:%M:%S %Y] “

# JkOptions for forwarding

JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat set the request format

JkRequestLogFormat “%w %V %T”

JkMount /jsp-examples/* worker1

JkMount /tomcat-docs/* worker2

Defining these tells Apache where to look for definitions of its workers and tells it that any requests for the jsp-examples context should be handed off to the Tomcat instance represented by worker 1, and any requests for tomcat-docs context should be handed off to Tomcat Instance 2, represented by worker 2.

Edit the server.xml for Tomcat 1 and Tomcat 2 and add a jvmRoute attribute to the Engine element:

<Engine name=”Catalina” defaultHost=”localhost” jvmRoute=”worker1″>

                      For the first instance and

<Engine name=”Catalina” defaultHost=”localhost” jvmRoute=”worker2″>

                     For the second.

       6.  Start Tomcat 1 and 2. Start up the Apache webserver. Point your browser to http://localhost/jsp-examples/ and then to http://localhost/tomcat-docs. You should see the respective pages load. To distinguish which Tomcat is serving you the page, the easiest thing to do is edit the index page in the tomcat-docs and jsp-examples of Tomcat 2 and change the title for example. Then you can verify that tomcat-docs is being served only by the second instance.

             At this point, your apache serve is ready to route certain requests to specific Tomcat instance. But we need to set tomcat for load balancing and for failover mechanism. If Tomcat 1 is crashed for whatever reason, Apache will automatically keep delegating to Tomcat 2 so your application remains accessible.

Load balancing is a simple configuration. First shut down your Tomcat instances and Apache as well.

  • Open workers.properties.

Edit it so it looks like this (changed lines in bold)-

#worker.list=worker1,worker2

 worker.list=balancer

worker.worker1.port=8009

worker.worker1.host=localhost

worker.worker1.type=ajp13

worker.worker1.lbfactor=1

worker.worker2.port=7009

worker.worker2.host=localhost

worker.worker2.type=ajp13

worker.worker2.lbfactor=1

worker.balancer.type=lb

worker.balancer.balance_workers=worker1,worker2

 worker.balancer.method=B

# specifies whether requests with SESSION ID’s should be routed back to the same #Tomcat worker.

worker.balancer. sticky_session =True

              We’ve changed the worker list to a single worker called balancer, and specified that the worker type of balancer is ‘lb‘ or load      balancer. The workers it manages are worker1 and worker2 (these do not need to appear in the workers list). And finally, we set the balance method to ‘B’ or balance by busy factor. Apache will delegate the next request to the Tomcat instance which is least busy. Please note that there are some other options for method- check out the Apache/Tomcat documentation which lists out options for workers properties to help you decide the best method for your type of application.

             You can also see that we have kept session stickiness to true. That means, the tomcat instance who first initiated the session will receive all rest of the request from that session. This is one way of managing sessions across the cluster. There are some other ways like persistence sessions etc. to achieve the same.

  • Open httpd.conf and comment out the previous JkMount directives. Replace them with these:

JkMount /tomcat-docs/* balancer

JkMount /jsp-examples/* balancer

                          We’ve just pointed Apache to a single worker- the balancer.

  • Start-up both Tomcats and Apache. Access http://localhost/jsp-examples. You will either be served by Tomcat 1 or Tomcat 2. To prove that both are capable of serving, shut down the first instance and refresh your browser. You should be served by instance two.

Conclusions

This solution provides high scalability, high availability, and good load balancing capabilities that are comparable with any other software solution. And it is pretty easy as well!

About these ads
    • Paco
    • January 29th, 2012

    Great post!!!
    Easy to understand and easy to confirure/use.

  1. Great post. We did the same thing on an old project I was on and I wish we had this guide to help us.

    • Vijay Anand
    • June 26th, 2012

    wonderful post!!! This was exactly was I was looking for.

  2. Great post ! will follow up the above instructions and see how things are giong :)
    very helpful & straight forward post :)

  3. Everyone loves what you guys tend to be up too.
    This type of clever work and coverage! Keep up the wonderful
    works guys I’ve added you guys to our blogroll.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: