OK, I'm sure some of you guys can weigh in better on this than me, but: If he already has two "intelligent" load-balancers terminating the SSL, couldn't he simplify the configuration a good bit by removing the Apache servers all together? That is, couldn't he just run Tomcat as a stand-alone cluster with HTTP/HTTPS connectors instead of AJP?
-- Jeff > -----Original Message----- > From: Richard Levy [mailto:rich....@gmail.com] > Sent: Tuesday, March 29, 2011 5:07 AM > To: users@tomcat.apache.org > Subject: Simulating HTTPS in terminated SSL/Apache 2/Tomcat 6 cluster > > Hi all, > > First of all, sorry for the long post, but the reasoning and > architecture requires explanation to get the full understanding of the > situation. > > We are currently migrating our website from Oracle 10g to Apache > 2/Tomcat 6.0.29. The site itself is a set of pages where customers > select what they want to purchase, then once they're happy, it > switches to SSL to gather their customer details and take payments. > > The current setup involves two intelligent load-balancers that > terminates SSL then hands over to Oracle 10g. The 10g stack has > custom Oracle versions of Apache which have configuration options not > found in standard Apache and appear to have made configuration > simpler. For instance, there is a "simulateHttps" directive that you > can give to a virtual host in Oracle Apache which does not exist in > the standard version. Using that option, we simply have 2 virtual > hosts defined in Apache, one for http traffic and one for https > traffic (with simulateHttps set to on). They both forward to OC4J and > it works fine. > > With the new architecture we have two load-balancer which round-robin > to two Apache servers. These servers connect to two Tomcat servers > which are configured with a mod_jk loadbalancer using session > affinity. Through config we have separated HTTP & HTTPS traffic. We > have done this because the application needs to identify when it is > secure, which we do by simulating HTTP using settings in the connector > in the Tomcat server.xml. This is detailed further down. > > Obviously we want to keep the same functionality on the Apache/Tomcat > stack, and also retain the termination of SSL at the load-balancer, > but are having trouble when the application switches to HTTPS. It > works fine if we have a single Tomcat instance running, but once the > 2nd Tomcat instance is enabled, moving to HTTPS fails because a new > session is created. > > Our configuration is as follows: > > Apache: > > We have 2 Apache servers running - they are served in a round-robin > style by the load-balancer. They recieve 2 types of traffic, > non-secure and secure, but the secure traffic is NOT encrypted (SSL > terminated at the load-balancer). > > ---- httpd.conf ---- > > JkWorkersFile conf/workers.properties > > Listen 5555 > Listen 6666 > NameVirtualHost *:5555 > NameVirtualHost *:6666 > > <VirtualHost *:5555> > ServerName http://www.mysite.com:80 > JkMount /* loadbalancer > </VirtualHost> > > <VirtualHost *:6666> > ServerName https://www.mysite.com:443 > JkMount /* loadbalancerSSL > </VirtualHost> > > ---- end httpd.conf ---- > > We are using mod_jk to connect to Tomcat > > ---- workers.properties ---- > > worker.list = loadbalancer, loadbalancerSSL, status > worker.tomcat1.type=ajp13 > worker.tomcat1.host=192.168.1.36 > worker.tomcat1.port=8009 > worker.tomcat2.type=ajp13 > worker.tomcat2.host=192.168.1.85 > worker.tomcat2.port=8009 > > worker.tomcatSSL1.type=ajp13 > worker.tomcatSSL1.host=192.168.1.36 > worker.tomcatSSL1.port=8010 > worker.tomcatSSL2.type=ajp13 > worker.tomcatSSL2.host=192.168.1.85 > worker.tomcatSSL2.port=8010 > > worker.loadbalancer.type = lb > worker.loadbalancer.sticky_session = 1 > worker.loadbalancer.balance_workers = tomcat1,tomcat2 > worker.loadbalancer.session_cookie = JSESSIONID > > worker.loadbalancerSSL.type = lb > worker.loadbalancerSSL.sticky_session = 1 > worker.loadbalancerSSL.balance_workers = > tomcatSSL1,tomcatSSL2 > worker.loadbalancerSSL.session_cookie = JSESSIONID > > worker.status.type = status > > ---- end workers.properties ---- > > As you can see, Apache can receive on either 5555 or 6666 dependant on > the source of the traffic. If it's non-secure, it uses the > loadbalancer called "loadbalancer" which uses two Tomcats listening on > port 8009 - this is a connector in Tomcat that is not secure. For > traffic on 6666, "loadbalancerSSL" is used. This is the same two > Tomcats but on port 8010 - this is a connector that believes it is > secure. Tomcat config is detailed below. > > > Tomcat: > > We have 2 Tomcat servers. They too recieve two types of traffic, > non-secure and secure, as passed on from Apache. > > ---- server.xml Tomcat 1 ---- > > <!-- Define an AJP 1.3 Connector on port 8009 --> > <Connector port="8009" > protocol="AJP/1.3" > redirectPort="8010" > maxThreads="200"/> > > <!-- This is the secure connetor, although its not using SSL, it's > just secure traffic --> > <Connector port="8010" > protocol="AJP/1.3" > connectionTimeout="20000" > scheme="https" > SSLEnabled="true" > secure="true" > maxThreads="200" > proxyPort="443"/> > > .... > > <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1"> > > ---- end server.xml Tomcat 1 ---- > > ---- server.xml Tomcat 2 ---- > <!-- Define an AJP 1.3 Connector on port 8009 --> > <Connector port="8009" > protocol="AJP/1.3" > redirectPort="8010" > maxThreads="200"/> > > <!-- This is the secure connetor, although its not using SSL, it's > just secure traffic --> > <Connector port="8010" > protocol="AJP/1.3" > connectionTimeout="20000" > scheme="https" > SSLEnabled="true" > secure="true" > maxThreads="200" > proxyPort="443"/> > > .... > > <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2"> > > ---- end server.xml Tomcat 2 ---- > > As can be seen, the only difference between the two Tomcat configs is > the jvmRoute parameter on the Engine config. > The connector on 8009 is a vanilla connector configuration. On 8010, > SSL related parameters are set so that it believes it is secure. > > What's happening: > > If I have a single Tomcat running, I can progress all the way through > the site, go secure, retain the session and get to the end of the > process. As soon as the 2nd Tomcat in introduced, when it goes > secure, the site returns to the first page. I believe this is because > sticky sessions fails and the requests ends up on a separate server, > hence creating a new session. The application detects this and sends > the user back to the first page in the process (it needs the data in > the session at this point and as there is none, redirects to the > start). > > I think that sticky sessions fails because the process behind it uses > the jvmRoute value from the Tomcat server.xml to append to the end of > the JSESSIONID cookie. When the app goes secure and Apache receives > traffic on 6666, it starts to look for tomcatSSL[1/2] but none of the > tomcats use that jvmRoute and hence it will not be part of the > JSESSIONID, so it will not stick to the same server. The reason that > it works with one server is because it only has a single server to > send it to, therefore will always find the session. > > If this is the case, we may have approached this incorrectly. > Separating non-SSL and SSL traffic all the way down to Tomcat may not > be the answer and we may need to use something in Apache to modify > request and response, in the same way that I assume "simulateHttps" > worked. > > Has anyone else setup a site using this sort of architecture? I can't > imagine that this is uncommon in the enterprise? > > Any help is greatly appreciated! > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > __________________________________________________________________________ Confidentiality Notice: This Transmission (including any attachments) may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this transmission in error, please immediately reply to the sender or telephone (512) 343-9100 and delete this transmission from your system. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
