This article explains you on how to configure a cluster using multiple tomcat instances and perform a session replication between them. We will be using the in-memory session replication in this article. For this demo article I have chosen RHEL 5.4 and Tomcat 6.
In order to configure clustering in tomcat. We follow the sequence,
1. Multicast?
2. Test Whether Your Kernel supports Multicast.
3. Configure Apache Http Server with mod_jk.
4. Configure Multiple Instances of Tomcat.
5. Configure Clustering in Tomcat.
6. Make your application distributable.
7. Session Replication Rules
8. Deploy the sample application.
9. Test.
10. Basic Issues.
1. Multicast?
If we have information that should be sent to various hosts on the internet or LAN, then it is called multicasting (UDP).
This is the opposite of unicast (TCP) where there will be once sender and only one receiver.
As we know, the IP address are divided into classes based on the high order of 32 bits, like
0 31 Address Range:
0| Class A Address 0.0.0.0 - 127.255.255.255
|1 0| Class B Address 128.0.0.0 - 191.255.255.255
|1 1 0| Class C Address 192.0.0.0 - 223.255.255.255
|1 1 1 0| MULTICAST Address 224.0.0.0 - 239.255.255.255
|1 1 1 1 0| Reserved 240.0.0.0 - 247.255.255.255
Class D addresses (224.0.0.0) are said to Used for multicast. The net mask is a 32 bit number with
network part being filled with all “1” and host part with “0”.
2. Test Whether Your Kernel supports Multicast.
Once we are aware of multi casting, we need to make sure that our kernel (or network interface) supports multicasting. In order to find this, run
/sbin/ifconfig
And output may be something like this
eth0 Link encap:Ethernet HWaddr 00:21:9B:F3:48:4D
inet addr:xxx.xx.xxx.xx Bcast:183.82.191.255 Mask:255.255.192.0
inet6 addr: fe80::221:9bff:fef3:484d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:23231 errors:0 dropped:0 overruns:0 frame:0
TX packets:30417 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:19979434 (19.0 MiB) TX bytes:4609420 (4.3 MiB)
Interrupt:169
Note the MULTICAST attribute in the third line of the eth0 properties. If this is not present, it's possible that your kernel has not been compiled with multicast support.
Once we are sure that our kernel supports multicasting the next step is to enable multicast routing and this can be done by,
route add 224.0.0.0 netmask 240.0.0.0 dev eth0
Once this is done. You can make sure to ping the ip address, like
ping 224.0.0.8
You can also make sure about the multicasting using
[root@vx111a bin]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
183.82.128.0 0.0.0.0 255.255.192.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
224.0.0.0 0.0.0.0 240.0.0.0 U 0 0 0 eth0
0.0.0.0 183.82.128.1 0.0.0.0 UG 0 0 0 eth0
Or
[root@vx111a bin]# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
183.82.128.0 0.0.0.0 255.255.192.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
224.0.0.0 0.0.0.0 240.0.0.0 U 0 0 0 eth0
0.0.0.0 183.82.128.1 0.0.0.0 UG 0 0 0 eth0
Or
[root@vx111a bin]# netstat -g
IPv6/IPv4 Group Memberships
Interface RefCnt Group
--------------- ------ ---------------------
lo 1 all-systems.mcast.net
eth0 2 228.0.0.8
eth0 1 224.0.0.251
eth0 1 all-systems.mcast.net
lo 1 ff02::1
eth0 1 ff02::fb
eth0 1 ff02::1:fff3:484d
eth0 1 ff02::1
In future if you want to disable multicast, just drop the IP from Routes Table:
route -v delete -inet 228.0.0.4
3. Configure Apache Http Server with mod_jk.
The 3 step is to configure http Server with mod_jk.The article can be found here.
The configuration that I done on my desktop is,
conf/worker.properties
#
#Worker.Properties
#
worker.list=web1, web2, loadbalancer
#
#First Server Details
#
worker.web1.port=11009
worker.web1.host=183.82.144.26 (make sure you change your ip here)
worker.web1.type=ajp13
worker.web1.lbfactor=100
#
#Second Server Details
#
worker.web1.port=12009
worker.web1.host=183.82.137.30 (make sure you change your ip here)
worker.web1.type=ajp13
worker.web1.lbfactor=100
#
#LoadBalancer Worker
#
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=web1, web2
worker. loadbalancer.sticky_session=1
#
#End Of the Properties File
#
conf/ mod_jk.conf
#
# Mod_Jk Configuration
#
LoadModule jk_module modules/mod_jk.so
<IfModule mod_jk.c>
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel error
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"
JkMount /*.jsp loadbalancer
JkMount /servlet/* loadbalancer
</IfModule>
#
# Done
#
In httpd.conf, add the following line at the bottom,
Include conf/mod_jk.conf
By this the configuration on the Http Server is done. We have now configured http server with mod_jk (load balancing support)
4. Configure Multiple Instances of Tomcat.
The next step is configuring tomcat instances. The article for configuring multiple instances of tomcat can be found here or.
Download the Tomcat to a location
Export the CATALINA_HOME and CATALINA_BASE.Add these to the /root/.bashrc
export CATALINA_HOME=/usr/jboss-ews-1.0/tomcat6;
export CATALINA_BASE=/usr/jboss-ews-1.0/tomcat6;
Once added, use source command to update, source /root/.bashrc
Now create 2 directories in other location (I have create tmc1 and tmc2 in /usr/local)
Once both locations are created, copy the directories bin, conf, lib, logs, temp, webapps, and work.
In bin directory, remove all the files except, startup.sh and shutdown.sh (if windows keep their respective bat files)
Clean startup.sh and shutdown.sh and update with the following
startup.sh
export CATALINA_BASE=/usr/local/tmc2
export CATALINA_HOME=/usr/jboss-ews-1.0/tomcat6
/usr/jboss-ews-1.0/tomcat6/bin/startup.sh
And shutdown.sh
export CATALINA_BASE=/usr/local/tmc2
export CATALINA_HOME=/usr/jboss-ews-1.0/tomcat6
/usr/jboss-ews-1.0/tomcat6/bin/shutdown.sh
Once these changes are done, go to the conf location and modify the server.xml file according to your port needs. The 2 main ports to change in server.xml are shutdown port and http port. It would be good if we change all ports.
Now we have 2 instances of tomcat running. Now for the Clustering we need to modify the ports like
These are the changes that I did in the both tomcat instances
Shutdown Http Connector jvmRoute
Tomcat 1 8005 8080 11009 web1
Tomcat 2 8105 8180 12009 web2
These changes are to be done in server.xml files in Tomcat (1,2)/conf/.These changes should be same as we configured in worker.properties file(port, jvmRoute…).
The JvmRoute attribute element of the Engine element allows the load balancer to match request to the exact jvm responsible for updating the session. This is done by appending the name of the jvm to the JSESSIONID of the request and matching this against the worker name provided in the worker.properties file
By this we are done with configuring multiple instances of tomcat. The next step is to configure cluster
5. Configure Clustering in Tomcat.
For configuring cluster we need to add the Cluster element in the Engine element like
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.8"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.css;.*\.txt;"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
This element is to be add under the
<Engine name="Catalina" defaultHost="localhost" jvmRoute="web1">
Add the cluster element to tomcat{1,2}/conf/server.xml file and restart the tomcat instances.
Since we are going for a In-Memory session replication, We need to configure 2 elemets
Cluster: this element is responsible for the actual session replication which includes sending the new session information to the group, taking incoming session information and also managing group’s membership.
Replication: The other element is the replication element which helps in reducing the session data by filtering certain request from session replication
SimpleTcpCluster is the implementation available in tomcat for In-Memory replication. This uses apache tribes for communication. This uses heartbeat mechanism to determine the group memberships.
The SimpleTcpCluster provides us with 2 managers
DeltaManager : replicate sessions across all tomcat instances
BackupManager : replicate session from master to a backup server
We can configure a manager depending on our needs.
SimpleTcpCluster uses Apache Tribes for communication between the servers or groups. Membership is established and maintained by Apache Tribles.It also handles server crash and recovery. This also offers guaranteed message delivery between members. This also carries the task of replicating the session data to all members.
6. Make Your Applications Distributable
In order to work with your application in a clustered environment, our applications need to add <distributable/> element to the web.xml file.
Your other option is to simply add the "distributable" attribute to the relevant application's Context element, as follows:
<Context distributable="true">
7. Session Replication Rules
Session attributes must implement java.io.Serializable.
HttpSession.setaAttribute() must be called any time changes are made to an object that belongs to a session, so that the session replicator can distribute the changes across the cluster.
The sessions must not be so big that they overload the server with traffic when copied.
Deploy the sample application.
Now the Cluster configuration is done, we need to deploy a sample application.
Create a web application. Download the following jsp file from here and add to the web application. Make sure you add <distributable/> element to the web.xml file
7. Test.
Test the application using, http://localhost:<port>/<context>/session.jsp
Stop one Tomcat server and access the url , it should still show the updated counter value.
Check the jsp file for more.
8. Basic Issues.
Some Tomcat instances are missing with tribes.jar file. Download catalina-tribes.jar file from internet.
No comments :
Post a Comment