request.getServerPort with RemoteIpValve
Hi, May I ask for your opinion on the following case: When RemoteIpValve is enabled getServerPort returns inconsistent value depending on the presence of protocolHeader (“x-forwarded-proto” by default) Here is a short example 1) no x-forwarded-proto curl -H "Host: myserverhost:32279" -H "x-forwarded-for: 10.0.0.10" http://10.19.82.245:8080/remoteip.jsp getRemoteAddr 10.0.0.10 getRemoteHost 10.0.0.10 getServerPort 32279 getScheme http 2) with x-forwarded-proto curl -H "Host: myserverhost:32279" -H "x-forwarded-for: 10.0.0.10" -H "x-forwarded-proto: https" http://10.19.82.245:8080/remoteip.jsp getRemoteAddr 10.0.0.10 getRemoteHost 10.0.0.10 getServerPort 443 getScheme https Hence when we have the presence of x-forwarded-proto we default to 443 otherwise we respect the value in the Host header. Looking in the source code of the RemoteIpValve when setPort is invoked if the protocolHeader is present and defaults to 443 if portHeader is not present as in the case above but should it be as in the case 1) Best regards Niki
Re: Coverity static analysis scanning
Niki Dokovski | @nickytd On 26.08.2014, at 22:56, Mark Thomas wrote: > On 26/08/2014 20:49, Niki Dokovski wrote: >> The observer role does not see actual ‘defects’. Only general statistic is >> shown on the project dashboard. > > About as much use as a chocolate teapot then. > > I've upgraded all non-committers to contributor / viewer. Any better? Much better. Now the actual issues are visible. > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Coverity static analysis scanning
Niki Dokovski | @nickytd On 26.08.2014, at 22:26, Niki Dokovski wrote: > > On 26.08.2014, at 22:19, Mark Thomas wrote: > >> On 26/08/2014 18:02, Filip Hanik wrote: >>> hook me up >> >> Done (and for everyone else who has asked). > > Thanks Mark. > >> >> It appears I now have admin karma for the Coverity's project for >> scanning Tomcat. >> >> The approach I am taking is Tomcat committers get the Maintainer/Owner >> role and everyone else gets the Observer role. The idea being that if >> someone contributes enough that we want to change their role we probably >> should make them a committer :) The observer role does not see actual ‘defects’. Only general statistic is shown on the project dashboard. >> >> Mark >> >> >>> >>> On Tuesday, August 26, 2014, Mark Thomas wrote: >>> >>>> All, >>>> >>>> I have been pinged off-list by Coverity to say that they have set up >>>> Tomcat with a free account with their static code analysis service. >>>> >>>> I think I have the ability to send invitations so if anyone wants to >>>> take a look at the results, just reply here. >>>> >>>> I have taken a quick look and they do appear to have found some valid >>>> threading issues. There are ~350 issues in total and I don't yet have a >>>> feel for the false positive rate. >>>> >>>> Mark >>>> >>>> - >>>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: dev-h...@tomcat.apache.org >>>> >>>> >>> >> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Coverity static analysis scanning
On 26.08.2014, at 22:19, Mark Thomas wrote: > On 26/08/2014 18:02, Filip Hanik wrote: >> hook me up > > Done (and for everyone else who has asked). Thanks Mark. > > It appears I now have admin karma for the Coverity's project for > scanning Tomcat. > > The approach I am taking is Tomcat committers get the Maintainer/Owner > role and everyone else gets the Observer role. The idea being that if > someone contributes enough that we want to change their role we probably > should make them a committer :) > > Mark > > >> >> On Tuesday, August 26, 2014, Mark Thomas wrote: >> >>> All, >>> >>> I have been pinged off-list by Coverity to say that they have set up >>> Tomcat with a free account with their static code analysis service. >>> >>> I think I have the ability to send invitations so if anyone wants to >>> take a look at the results, just reply here. >>> >>> I have taken a quick look and they do appear to have found some valid >>> threading issues. There are ~350 issues in total and I don't yet have a >>> feel for the false positive rate. >>> >>> Mark >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: dev-h...@tomcat.apache.org >>> >>> >> > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Coverity static analysis scanning
Hi Mark, On Tue, Aug 26, 2014 at 12:20 PM, Mark Thomas wrote: > All, > > I have been pinged off-list by Coverity to say that they have set up > Tomcat with a free account with their static code analysis service. > > I think I have the ability to send invitations so if anyone wants to > take a look at the results, just reply here. > I'm interested. > > I have taken a quick look and they do appear to have found some valid > threading issues. There are ~350 issues in total and I don't yet have a > feel for the false positive rate. > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: [VOTE] Release Apache Tomcat 8.0.1
On Thu, Jan 30, 2014 at 12:32 PM, Rémy Maucherat wrote: > 2014-01-30 Mark Thomas : > > > The proposed 8.0.1 release is: > > [ ] Broken - do not release > > [ ] Alpha - go ahead and release as 8.0.1 (alpha) > > [X] Beta - go ahead and release as 8.0.1 (beta) > > [ ] Stable - go ahead and release as 8.0.1 (stable) > > > > Rémy > +1 (non-binding) [X] Beta - go ahead and release as 8.0.1 (beta) websocket implementation checked with applications that use the new specifications features.
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 4:02 PM, Romain Manni-Bucau wrote: > +1 > Romain Manni-Bucau > Twitter: @rmannibucau > Blog: http://rmannibucau.wordpress.com/ > LinkedIn: http://fr.linkedin.com/in/rmannibucau > Github: https://github.com/rmannibucau > > > > 2013/10/29 Mark Thomas : > > On 29/10/2013 13:41, Romain Manni-Bucau wrote: > >> Yes but if it runs in a JavaEE container (J2EE doesn't exist anymore > >> if i'm not mistaken) > > > > Indeed but it is quicker to type J2EE and everyone knows what I mean. > > > >> or at least a servlet container what is the minimum required? > > > > As the specification is currently written I'd say it requires that any > > implementation in a JavaEE web container requires at least Servlet 3.0 > > as it requires the use of the scanning mechanism introduced in 3.0. > > > > Some EG members did express a desire to implement it on Servlet 2.5 > > containers. I don't know if anyone has or, if they have, how they > > addressed the scanning issue. > > > > If Tomcat's JSR-356 implementation was back-ported to 6.0.x (I have no > > desire to do that) then I guess the scanning would have to be > > back-ported in some form as well. I suspect it could get messy quite > > quickly. The connector code would certainly be painful to back-port as > > none of the refactoring that significantly reduced the duplication > > exists in 6.0.x. > > > > If there is a desire to implement JSR-356 in Servlet 2.5 or earlier, I'd > > suggest changing section 6.2 so it applies to Servlet 3.0 and later > > containers and changing section 6.3 so it applies to Servlet 2.5 or > > earlier containers and any other non-JavaEE container. > +1 Spec clarification would help in cases like this one. > > > Mark > > > >> Romain Manni-Bucau > >> Twitter: @rmannibucau > >> Blog: http://rmannibucau.wordpress.com/ > >> LinkedIn: http://fr.linkedin.com/in/rmannibucau > >> Github: https://github.com/rmannibucau > >> > >> > >> > >> 2013/10/29 Mark Thomas : > >>> On 29/10/2013 13:24, Niki Dokovski wrote: > >>>> On Tue, Oct 29, 2013 at 3:08 PM, Mark Thomas > wrote: > >>>> > >>>>> On 29/10/2013 13:01, Niki Dokovski wrote: > >>>>>> On Tue, Oct 29, 2013 at 2:51 PM, Mark Thomas > wrote: > >>>>>> > >>>>>>> On 29/10/2013 12:41, Niki Dokovski wrote: > >>>>>>> > >>>>>>>> WebSocket container can be used in Java SE env only but for a > standard > >>>>>>> JSR > >>>>>>>> 356 compliance implementation an existing servlet container is > needed. > >>>>>>> > >>>>>>> No it is not. > >>>>>>> > >>>>>>>> Basically you are correct if you don't refer to JSR 356. But my > >>>>> question > >>>>>>>> was related to improving the spec, triggered by Romain's question. > >>>>>>> > >>>>>>> Wrong again. > >>>>>>> > >>>>>>> You can implement a specification compliant JSR 356 server > container > >>>>>>> without implementing any other J2EE specs. This was an explicit > design > >>>>>>> decision made by the JSR 356 EG and explains, for example, why the > >>>>>>> HttpSession instance is passed as Object rather than as > >>>>>>> javax.servlet.http.HttpSession. > >>>>>>> > >>>>>>> I still do not see where, how or why there is a specification issue > >>>>> here. > >>>>>>> > >>>>>> > >>>>>> The simple fact that you cannot pass the TCK, hence claim > compatibility > >>>>>> proves the other way. > >>>>> > >>>>> If the TCK requires that the WebSocket implementation be part of a > J2EE > >>>>> container then that is an issue with the TCK. I only had access to a > >>>>> draft of the TCK and there were much more fundamental issues with it > at > >>>>> that stage. > >>>>> > >>>> > >>>> IMHO Those fundamental issues still exist then. Which still lead to a > need > >>>> of clarity on EG, which was my original question. :) > >>> > >>> I still don't see the need. Section 6.3 makes it very clear it is not > >>> required to run in a J2EE container. > >>> > >>> Mark > >>> > >>> > >>> - > >>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > >>> For additional commands, e-mail: dev-h...@tomcat.apache.org > >>> > >> > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: dev-h...@tomcat.apache.org > >> > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 3:41 PM, Romain Manni-Bucau wrote: > Yes but if it runs in a JavaEE container (J2EE doesn't exist anymore > if i'm not mistaken) or at least a servlet container what is the > minimum required? > A beer or two ... or more are needed here to resolve this dispute :) > Romain Manni-Bucau > Twitter: @rmannibucau > Blog: http://rmannibucau.wordpress.com/ > LinkedIn: http://fr.linkedin.com/in/rmannibucau > Github: https://github.com/rmannibucau > > > > 2013/10/29 Mark Thomas : > > On 29/10/2013 13:24, Niki Dokovski wrote: > >> On Tue, Oct 29, 2013 at 3:08 PM, Mark Thomas wrote: > >> > >>> On 29/10/2013 13:01, Niki Dokovski wrote: > >>>> On Tue, Oct 29, 2013 at 2:51 PM, Mark Thomas > wrote: > >>>> > >>>>> On 29/10/2013 12:41, Niki Dokovski wrote: > >>>>> > >>>>>> WebSocket container can be used in Java SE env only but for a > standard > >>>>> JSR > >>>>>> 356 compliance implementation an existing servlet container is > needed. > >>>>> > >>>>> No it is not. > >>>>> > >>>>>> Basically you are correct if you don't refer to JSR 356. But my > >>> question > >>>>>> was related to improving the spec, triggered by Romain's question. > >>>>> > >>>>> Wrong again. > >>>>> > >>>>> You can implement a specification compliant JSR 356 server container > >>>>> without implementing any other J2EE specs. This was an explicit > design > >>>>> decision made by the JSR 356 EG and explains, for example, why the > >>>>> HttpSession instance is passed as Object rather than as > >>>>> javax.servlet.http.HttpSession. > >>>>> > >>>>> I still do not see where, how or why there is a specification issue > >>> here. > >>>>> > >>>> > >>>> The simple fact that you cannot pass the TCK, hence claim > compatibility > >>>> proves the other way. > >>> > >>> If the TCK requires that the WebSocket implementation be part of a J2EE > >>> container then that is an issue with the TCK. I only had access to a > >>> draft of the TCK and there were much more fundamental issues with it at > >>> that stage. > >>> > >> > >> IMHO Those fundamental issues still exist then. Which still lead to a > need > >> of clarity on EG, which was my original question. :) > > > > I still don't see the need. Section 6.3 makes it very clear it is not > > required to run in a J2EE container. > > > > Mark > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 3:08 PM, Mark Thomas wrote: > On 29/10/2013 13:01, Niki Dokovski wrote: > > On Tue, Oct 29, 2013 at 2:51 PM, Mark Thomas wrote: > > > >> On 29/10/2013 12:41, Niki Dokovski wrote: > >> > >>> WebSocket container can be used in Java SE env only but for a standard > >> JSR > >>> 356 compliance implementation an existing servlet container is needed. > >> > >> No it is not. > >> > >>> Basically you are correct if you don't refer to JSR 356. But my > question > >>> was related to improving the spec, triggered by Romain's question. > >> > >> Wrong again. > >> > >> You can implement a specification compliant JSR 356 server container > >> without implementing any other J2EE specs. This was an explicit design > >> decision made by the JSR 356 EG and explains, for example, why the > >> HttpSession instance is passed as Object rather than as > >> javax.servlet.http.HttpSession. > >> > >> I still do not see where, how or why there is a specification issue > here. > >> > > > > The simple fact that you cannot pass the TCK, hence claim compatibility > > proves the other way. > > If the TCK requires that the WebSocket implementation be part of a J2EE > container then that is an issue with the TCK. I only had access to a > draft of the TCK and there were much more fundamental issues with it at > that stage. > IMHO Those fundamental issues still exist then. Which still lead to a need of clarity on EG, which was my original question. :) cheers Niki > The intention of the WebSocket EG was that a specification compliant > WebSocket server container could be implemented on a stand-along basis > without having to implement any other J2EE specs. I don't believe there > is anything in the WebSocket spec that contradicts that. For example, > see section 6.3 which explicitly references implementations that do not > include a Servlet container. > > > The dependency is more hidden to me. Mark, I > > understand your point by saying ok when implementing the websocket > > container I don't use servlet spec APIs. The javax.servlet APIs are not > > used in the implementation, are they? > > In Tomcat's implementation, yes they are because that was how I opted to > implement it. My intention was to produce a container neutral WebSocket > implementation. > > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 2:51 PM, Mark Thomas wrote: > On 29/10/2013 12:41, Niki Dokovski wrote: > > > WebSocket container can be used in Java SE env only but for a standard > JSR > > 356 compliance implementation an existing servlet container is needed. > > No it is not. > > > Basically you are correct if you don't refer to JSR 356. But my question > > was related to improving the spec, triggered by Romain's question. > > Wrong again. > > You can implement a specification compliant JSR 356 server container > without implementing any other J2EE specs. This was an explicit design > decision made by the JSR 356 EG and explains, for example, why the > HttpSession instance is passed as Object rather than as > javax.servlet.http.HttpSession. > > I still do not see where, how or why there is a specification issue here. > The simple fact that you cannot pass the TCK, hence claim compatibility proves the other way. The dependency is more hidden to me. Mark, I understand your point by saying ok when implementing the websocket container I don't use servlet spec APIs. The javax.servlet APIs are not used in the implementation, are they? > > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 1:59 PM, Mark Thomas wrote: > On 29/10/2013 11:46, Niki Dokovski wrote: > > On Tue, Oct 29, 2013 at 1:40 PM, Mark Thomas wrote: > > > >> On 29/10/2013 11:17, Niki Dokovski wrote: > >> > >>> I agree with Mark and looked further in the spec looking for a list of > >>> dependent specifications. Actually the only dependency I found is > >> described > >>> in 7.1.1 Websocket Endpoints and Dependency Injection and it deals with > >> CDI > >>> and no exact version of the Servlet specification is defined. Perhaps > >> it's > >>> no big deal but it's always good to avoid gray areas. Mark did you > >>> discussed this in EG? What about having this as an improvement for the > >> next > >>> version of the spec? > >> > >> Huh? > >> > >> The Java WebSocket spec was explicitly designed not to have dependency > >> on any Servlet specification. There is no requirement for a WebSocket > >> implementation to be part of a J2EE container. > >> > > > > Java EE 7 Full Profile or Web profile compliant implementations should > > provide JSR 356 defined container. It is listed at least in Web Profile > > Specification "WP 2.1 Required Components". Or I'm missing something? > > Java EE requires WebSocket. WebSocket does not require Java EE. It is perfectly acceptable to develop a stand-alone Java WebSocket container > that implements no other J2EE specs. > WebSocket container can be used in Java SE env only but for a standard JSR 356 compliance implementation an existing servlet container is needed. Basically you are correct if you don't refer to JSR 356. But my question was related to improving the spec, triggered by Romain's question. > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
On Tue, Oct 29, 2013 at 1:40 PM, Mark Thomas wrote: > On 29/10/2013 11:17, Niki Dokovski wrote: > > > I agree with Mark and looked further in the spec looking for a list of > > dependent specifications. Actually the only dependency I found is > described > > in 7.1.1 Websocket Endpoints and Dependency Injection and it deals with > CDI > > and no exact version of the Servlet specification is defined. Perhaps > it's > > no big deal but it's always good to avoid gray areas. Mark did you > > discussed this in EG? What about having this as an improvement for the > next > > version of the spec? > > Huh? > > The Java WebSocket spec was explicitly designed not to have dependency > on any Servlet specification. There is no requirement for a WebSocket > implementation to be part of a J2EE container. > Java EE 7 Full Profile or Web profile compliant implementations should provide JSR 356 defined container. It is listed at least in Web Profile Specification "WP 2.1 Required Components". Or I'm missing something? cheers Niki > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: tomcat7-websocket web-fragment is wrong?
Mark, On Tue, Oct 29, 2013 at 12:57 PM, Romain Manni-Bucau wrote: > Thanks for your speed Mark! > > I'm hacking a temporary fix in TomEE waiting for the next release. > Romain Manni-Bucau > Twitter: @rmannibucau > Blog: http://rmannibucau.wordpress.com/ > LinkedIn: http://fr.linkedin.com/in/rmannibucau > Github: https://github.com/rmannibucau > > > > 2013/10/29 Mark Thomas : > > On 29/10/2013 10:53, Mark Thomas wrote: > >> On 29/10/2013 10:41, Romain Manni-Bucau wrote: > >>> Hi > >>> > >>> some TomEE users noticed they have issues with web-fragment xsd. > >>> > >>> The point is tomcat7-websocket jar brings a web-fragment based on > >>> servlet 3.1 xsd. Is it intended? It means you can't validate the xml > >>> descriptors if this jar is present. > >> > >> That would be a bug. > > > > Which should be fixed now. The fix will be in 7.0.48 onwards. > I agree with Mark and looked further in the spec looking for a list of dependent specifications. Actually the only dependency I found is described in 7.1.1 Websocket Endpoints and Dependency Injection and it deals with CDI and no exact version of the Servlet specification is defined. Perhaps it's no big deal but it's always good to avoid gray areas. Mark did you discussed this in EG? What about having this as an improvement for the next version of the spec? > > > Mark > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: websocket connection (at startup) between 2 webapps in same Tomcat 7.0.47 instance hangs indefinitely
On Sat, Oct 19, 2013 at 4:26 PM, Bob DeRemer wrote: > > > > -Original Message- > > From: Niki Dokovski [mailto:nick...@gmail.com] > > Sent: Saturday, October 19, 2013 9:24 AM > > To: Tomcat Developers List > > Subject: Re: websocket connection (at startup) between 2 webapps in same > > Tomcat 7.0.47 instance hangs indefinitely > > > > On Sat, Oct 19, 2013 at 4:20 PM, Niki Dokovski > wrote: > > > > > > > > > > > > > > On Sat, Oct 19, 2013 at 12:09 AM, Bob DeRemer > > wrote: > > > > > >> Hi Guys, > > >> > > >> ** ** > > >> > > >> In our implementation, we have a gateway app that uses jsr websockets > > >> to communication with our main application server. In a small > > >> system, we want to run them both on a single Tomcat instance using > > >> the same Tomcat NIO connector, but directing to different respective > > >> WS paths. This works fine if you deploy the MAIN first, then the GW > > >> - so that MAIN is already up and running. If you restart Tomcat when > > >> both webapps are deployed - and the GW > > >> (client) starts first, it hangs indefinitely in the following code > > >> trying to establish a WS connection: > > >> > > > > > > Hi Bob, > > > Do you use the latest implementation? In your case, if got it > > > correctly, we have following: > > > > > This was with the latest TAGGED code (7.0.47), not trunk. If this was a > known issue that's fixed in TRUNK, let me know and I'll test it out. > > Thx - bob > I tested with a sample app that has an annotated server endpoint,an annotated client endpoint and a context listener. In the listener, I obtain instance of ServerContainer and try to connect to the server endpoint. In both implementations (trunk and tc7.0.x) I got "javax.websocket.DeploymentException: The HTTP response from the server [HTTP/1.1 404 Not Found] did not permit the HTTP upgrade to WebSocket" as expected. However, the response processing is a blocking operation and can lead to the wait you describe. Can you isolate the case with a test app? Obviously I'm missing something. > > > > > > > > > > 1. a socket listening - Hence connections are processed > > > > > 2. missing application - Hence 404 Not Found status code is > expected I > > expect to have 404 response code on initial upgrade request > > > > cheers > > Niki > > > > > > (sorry for previous partial response, hit send by mistake) > > > > > > > > > >> > > >> ** ** > > >> > > >> Is this a bug or a known limitation when a client/server in the same > > >> webapp try to connect at startup? > > >> > > >> ** ** > > >> > > >> Thanks > > >> > > >> ** ** > > >> > > >> "localhost-startStop-1" daemon prio=6 tid=0x0ef0f800 > > >> nid=0x1624 waiting on condition [0x1046e000] > > >> > > >>java.lang.Thread.State: WAITING (parking) > > >> > > >>at sun.misc.Unsafe.park(Native Method) > > >> > > >>- parking to wait for <0x0007d6d98b18> (a > > >> java.util.concurrent.CountDownLatch$Sync) > > >> > > >>at > > >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > > >> > > >>at > > >> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInt > > >> errupt(AbstractQueuedSynchronizer.java:834) > > >> > > >> > > >>at > > >> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared > > >> Interruptibly(AbstractQueuedSynchronizer.java:994) > > >> > > >> > > >>at > > >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedIn > > >> terruptibly(AbstractQueuedSynchronizer.java:1303) > > >> > > >> > > >>at > > >> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)*** > > >> * > > >> > > >>at sun.nio.ch.PendingFuture.get(PendingFuture.java:180) > > >> > > >>at > > >> > > org.apache.tomcat.websocket.WsWebSocketContainer.processResponse(WsWe
Re: websocket connection (at startup) between 2 webapps in same Tomcat 7.0.47 instance hangs indefinitely
On Sat, Oct 19, 2013 at 4:20 PM, Niki Dokovski wrote: > > > > On Sat, Oct 19, 2013 at 12:09 AM, Bob DeRemer > wrote: > >> Hi Guys, >> >> ** ** >> >> In our implementation, we have a gateway app that uses jsr websockets to >> communication with our main application server. In a small system, we want >> to run them both on a single Tomcat instance using the same Tomcat NIO >> connector, but directing to different respective WS paths. This works fine >> if you deploy the MAIN first, then the GW – so that MAIN is already up and >> running. If you restart Tomcat when both webapps are deployed – and the GW >> (client) starts first, it hangs indefinitely in the following code trying >> to establish a WS connection: >> > > Hi Bob, > Do you use the latest implementation? In your case, if got it correctly, > we have following: > > 1. a socket listening - Hence connections are processed > 2. missing application - Hence 404 Not Found status code is expected I expect to have 404 response code on initial upgrade request cheers Niki (sorry for previous partial response, hit send by mistake) > >> >> ** ** >> >> Is this a bug or a known limitation when a client/server in the same >> webapp try to connect at startup? >> >> ** ** >> >> Thanks >> >> ** ** >> >> "localhost-startStop-1" daemon prio=6 tid=0x0ef0f800 nid=0x1624 >> waiting on condition [0x1046e000] >> >>java.lang.Thread.State: WAITING (parking) >> >>at sun.misc.Unsafe.park(Native Method) >> >>- parking to wait for <0x0007d6d98b18> (a >> java.util.concurrent.CountDownLatch$Sync) >> >>at >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) >> >>at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) >> >> >>at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) >> >> >>at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) >> >> >>at >> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236) >> >>at sun.nio.ch.PendingFuture.get(PendingFuture.java:180) >> >>at >> org.apache.tomcat.websocket.WsWebSocketContainer.processResponse(WsWebSocketContainer.java:568) >> >> >>at >> org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:317) >> >> >>at >> com.thingworx.core.communication.channels.jsr356.client.Jsr356ClientChannel.connect(Jsr356ClientChannel.java:57) >> >> >>at >> com.thingworx.core.communication.endpoints.CommunicationEndpoint.connect(CommunicationEndpoint.java:186) >> >> >>at >> com.thingworx.core.communication.CommunicationSubsystem.startSubsystem(CommunicationSubsystem.java:88) >> >> >>at >> com.thingworx.core.subsystems.SubsystemBase.start(SubsystemBase.java:48)* >> *** >> >>at >> com.thingworx.apiserver.APIServerManager.startSubsystem(APIServerManager.java:92) >> >> >>at >> com.thingworx.core.subsystems.SubsystemBase.start(SubsystemBase.java:48)* >> *** >> >>at >> com.thingworx.apiserver.Bootstrapper.contextInitialized(Bootstrapper.java:57) >> >> >>at >> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4939) >> >> >>at >> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5434) >> >> >>- locked <0x0007da3e0308> (a >> org.apache.catalina.core.StandardContext) >> >>at >> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) >> >>- locked <0x0007da3e0308> (a >> org.apache.catalina.core.StandardContext) >> >>at >> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559) >> >> >>at >> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549) >> >> >>at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> >>at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> >>at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> >>at java.lang.Thread.run(Thread.java:744) >> >> ** ** >> >>Locked ownable synchronizers: >> >>- <0x0007da3a7ab0> (a >> java.util.concurrent.ThreadPoolExecutor$Worker) >> >> ** ** >> >> ** ** >> >> ** ** >> >> *Bob DeRemer* >> >> *Senior Director, Architecture and Development* >> >> ** ** >> >> [image: Description: Description: Description: Description: >> cid:image001.png@01CBE3DE.51A12030] >> >> http://www.thingworx.com >> >> Skype: bob.deremer.thingworx >> >> O: 610.594.6200 x812 >> >> M: 717.881.3986 >> >> ** ** >> > >
Re: websocket connection (at startup) between 2 webapps in same Tomcat 7.0.47 instance hangs indefinitely
On Sat, Oct 19, 2013 at 12:09 AM, Bob DeRemer wrote: > Hi Guys, > > ** ** > > In our implementation, we have a gateway app that uses jsr websockets to > communication with our main application server. In a small system, we want > to run them both on a single Tomcat instance using the same Tomcat NIO > connector, but directing to different respective WS paths. This works fine > if you deploy the MAIN first, then the GW – so that MAIN is already up and > running. If you restart Tomcat when both webapps are deployed – and the GW > (client) starts first, it hangs indefinitely in the following code trying > to establish a WS connection: > Hi Bob, Do you use the latest implementation? In your case, if got it correctly, we have following: a socket listening and missing context > > > ** ** > > Is this a bug or a known limitation when a client/server in the same > webapp try to connect at startup? > > ** ** > > Thanks > > ** ** > > "localhost-startStop-1" daemon prio=6 tid=0x0ef0f800 nid=0x1624 > waiting on condition [0x1046e000] > >java.lang.Thread.State: WAITING (parking) > >at sun.misc.Unsafe.park(Native Method) > >- parking to wait for <0x0007d6d98b18> (a > java.util.concurrent.CountDownLatch$Sync) > >at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > > >at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) > > >at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) > > >at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) > > >at > java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236) > >at sun.nio.ch.PendingFuture.get(PendingFuture.java:180) > >at > org.apache.tomcat.websocket.WsWebSocketContainer.processResponse(WsWebSocketContainer.java:568) > > >at > org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:317) > > >at > com.thingworx.core.communication.channels.jsr356.client.Jsr356ClientChannel.connect(Jsr356ClientChannel.java:57) > > >at > com.thingworx.core.communication.endpoints.CommunicationEndpoint.connect(CommunicationEndpoint.java:186) > > >at > com.thingworx.core.communication.CommunicationSubsystem.startSubsystem(CommunicationSubsystem.java:88) > > >at > com.thingworx.core.subsystems.SubsystemBase.start(SubsystemBase.java:48)** > ** > >at > com.thingworx.apiserver.APIServerManager.startSubsystem(APIServerManager.java:92) > > >at > com.thingworx.core.subsystems.SubsystemBase.start(SubsystemBase.java:48)** > ** > >at > com.thingworx.apiserver.Bootstrapper.contextInitialized(Bootstrapper.java:57) > > >at > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4939) > > >at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5434) > > >- locked <0x0007da3e0308> (a > org.apache.catalina.core.StandardContext) > >at > org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) > >- locked <0x0007da3e0308> (a > org.apache.catalina.core.StandardContext) > >at > org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559) > > >at > org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549) > > >at java.util.concurrent.FutureTask.run(FutureTask.java:262) > >at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > >at java.lang.Thread.run(Thread.java:744) > > ** ** > >Locked ownable synchronizers: > >- <0x0007da3a7ab0> (a > java.util.concurrent.ThreadPoolExecutor$Worker) > > ** ** > > ** ** > > ** ** > > *Bob DeRemer* > > *Senior Director, Architecture and Development* > > ** ** > > [image: Description: Description: Description: Description: > cid:image001.png@01CBE3DE.51A12030] > > http://www.thingworx.com > > Skype: bob.deremer.thingworx > > O: 610.594.6200 x812 > > M: 717.881.3986 > > ** ** >
Re: setting handler in AbstractEndpont
On Tue, Oct 8, 2013 at 9:52 PM, Mark Thomas wrote: > On 08/10/2013 19:46, Niki Dokovski wrote: > > Hi, > > Is there a good reason why AbstractEndpoint in coyote does not have an > > association with Handler through an abstract getter and setter? All > > concrete implementations of AbstractEndpoint > > [AprEndpoint,JIoEndpoint,NioEndpoint] provide a getter and a setter of a > > AbstractEndpoint.Handler > > No they don't. The getters and setters use the endpoint specific Handler. > Yep that's true. Just spotted that in all of the Protocols implementations [Http11AprProtocol, Http11NioProtocol and Http11Protocol] there is duplication of the initialization of the handler in the constructors, so I though of some potential options for cleanup. What I am doing is to go through the coyote code and try to get a better understanding of it by providing yet another "dummy" protocol implementation. :) > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
setting handler in AbstractEndpont
Hi, Is there a good reason why AbstractEndpoint in coyote does not have an association with Handler through an abstract getter and setter? All concrete implementations of AbstractEndpoint [AprEndpoint,JIoEndpoint,NioEndpoint] provide a getter and a setter of a AbstractEndpoint.Handler Cheers Niki
Re: 8.0.x / 7.0.x progress
On Mon, Oct 7, 2013 at 6:34 PM, Konstantin Preißer wrote: > > > > -Original Message- > > From: Konstantin Preißer [mailto:kpreis...@apache.org] > > Sent: Monday, October 7, 2013 4:10 PM > > To: 'Tomcat Developers List' > > Subject: RE: 8.0.x / 7.0.x progress > > > > Hi Mark, > > > > > -Original Message- > > > From: Mark Thomas [mailto:ma...@apache.org] > > > Sent: Monday, October 7, 2013 3:53 PM > > > To: Tomcat Developers List > > > Subject: Re: 8.0.x / 7.0.x progress > > > > > > > > > To me this reads that by default (value = -1), there is no limit when > > > processing whole messages, so I think Tomcat should handle such large > > > messages when not using methods to read partial messages. > > > > > > > > Am I missing something? > > > > > > DoS via a single large message that triggers an OOME. > > > > Yes, that can happen if there is no value specified for the maximum > message > > size. (I thought it would be the application's responsibility so set a > reasonable > > limit there, e.g. with the maxMessageSize attribute). > > > > But what I meant was, that the javadoc specifies that "-1" is the > default value > > which means that there is no limit when receiving the message (as a > whole), > > and the ChatAnnotation does not specify a value in its OnMessage > > annotation. So Tomcat does not seem to implement this default value. > > > > Also, when I change the value to something like this: > > > > @OnMessage(maxMessageSize = 1000L) > > > > so that Tomcat should be able to receive 10 MB messages, but it still > does not > > receive the 1 characters string message. > > Sorry - I think I missed something regarding the buffer size in the > session. > > When setting session.setMaxTextMessageBufferSize(100); (e.g. in onOpen > method), then Tomcat does indeed receive messages with such size, and it > calls the @OnMessage method > I also noticed that when using @OnMessage with maxMessageSize that is > lower than the one set in session.setMaxTextMessageBufferSize, the message > will be rejected. > So it seems that first the limit from > session.setMaxTextMessageBufferSize(...) is applied, and then the limit > from @OnMessage(maxMessageSize = ...) is applied. Is this correct? > Looking into the javadoc of those methods and the implementation this seems to be the correct behavior. The message is rejected simply because the buffer on the session has more data than the concrete message handler can process: hence it is rejected as the message will be too big for processing. Implementation details can be found at org.apache.tomcat.websocket.WsFrameBase > > Note that for the BIO connector, this does not seem to work - there Tomcat > does not call the OnMessage method when such large message is received (and > I also was not able to receive partial messages), whereas for NIO it works. > IMHO this particular behavior shouldn't be influenced by NIO and BIO connectors. > > Regards, > Konstantin Preißer > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: 8.0.x / 7.0.x progress
On Mon, Oct 7, 2013 at 5:39 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Konstantin, > > On 10/7/13 10:24 AM, Konstantin Preißer wrote: > >> -Original Message- > >> From: Christopher Schultz [mailto:ch...@christopherschultz.net] > >> Sent: Monday, October 7, 2013 3:29 PM > >> To: Tomcat Developers List > >> Subject: Re: 8.0.x / 7.0.x progress > >> > >> Konstantin, > >> > >> > >> Note that it's not Tomcat sending the ACKs, but the TCP/IP stack in the > >> OS running underneath the JVM. I wanted to point that out because it > >> means that Tomcat may be entirely unaware that data exists for reading > >> for some reason. If Tomcat itself were sending the ACKs, it would mean > >> that Tomcat was at least conceptually aware of the data, but refusing to > >> read it for some reason. > > > > Yes, that's correct. I guess there is some buffering in the various > > layers (TCP/IP stack of OS, Sockets in the JVM etc) so that when sending > > data, the client receive ACKS with a new window size even if the remote > > java code might not be reading the data. > > Exactly. > > > However, the buffering will not be endlessly so at some point TCP > > will advertise a window of 0 if the java code does not read from the > > Socket (at least this will be the case with blocking IO - though I do > > not exactly know how the non-blocking IO is implemented so I may as > > well be wrong here). This would mean that Firefox will have to wait > > until there is more window available to continue sending data. > > I'm not sure how big the average TCP/IP receive buffer size is. I'd > suspect a few MBs at least. It's bufferbloat at its finest ;) > The SO_RCVBUF and SO_SNDBUF are system dependent socket options and typically are in range of 8Kb up to 60Kb. Those are socket level options though. BTW I was able to reproduce the "hanging" problem described above. I haven't looked deeper for now but it seems that both sockets are closed cleanly after the "suspended" browser is resumed. Hence normal TCP close sequence is initiated and normal two way FIN, ACK sequences are communicated between the sockets. > > > In this case, I guess to verify that the remote TCP endpoint is > > actually reading the data, one would have to send much bigger > > messages to make sure the buffer of the remote endpoint runs out of > > space if it is not reading from the connection. I can try that if I > > have some time left. > > Just add a few zeros to your loop limit ;) > > -chris > >
ws package naming
Hi, just out of curiosity and a bit of fun... While I was going through the tomcat source code I spotted missing naming convention for the websocket implementation. If "catalina", "jasper", "coyote" have their dedicated responsibilities and frequently used as a reference point, the websocket container package is just org.apache.tomcat.websocket Do you have any ideas for having a name there? For the sake of "keeping the various containers named" :) cheers Niki
Re: 8.0.x / 7.0.x progress
On Sat, Oct 5, 2013 at 4:56 PM, Konstantin Preißer wrote: > Hi Niki, > thank you for your reply. > > > -Original Message- > > From: Niki Dokovski [mailto:nick...@gmail.com] > > Sent: Saturday, October 5, 2013 7:47 AM > > To: Tomcat Developers List > > Subject: Re: 8.0.x / 7.0.x progress > > > > I have installed Wireshark now and can confirm that Firefox (after it > is > > > resumed) still sends data over the TCP connection which Tomcat seems > not > > to > > > be able to read. > > > > > > > Are there ACKs for the TCP packets? > > I'll try to reproduce the case. > > Yes, I verified with Wireshark that Tomcat ACKs these packets correctly, > but does not seem to be able to process them. I continued to send data from > Firefox (SEQ went up to 61373) to ensure that the ACKs were not just > resulting from buffering somewhere in the Windows or Java network stack, > and I can confirm that Tomcat still ACKed these SEQ numbers. > > Note, that when using the BIO connector, everything works fine - the > problems only appear with NIO or APR connector. > > > > > > > B) For BIO connector: > > > > > I noticed that on Tomcat with BIO connector, when using a > > > > RemoteEndpoint.Async to asynchronously send data over the > > WebSocket > > > > connection, sendText(String, SendHandler) (or similar methods) will > > > block if > > > > the Remote endpoint does not read data from the connection, whereas > > for > > > > NIO and APR connector this method will always return immediately. > > > > > Is it intended that for the BIO connector those methods are > blocking? > > > As > > > > the javadoc says, "Initiates the asynchronous transmission of a text > > > message. > > > > This method returns before the message is transmitted.", I would have > > > > expected that e.g. another Thread is used to write in blocking mode, > so > > > that > > > > the sendText() method can return immediately. > > > > > > > > You can't do non-blocking IO with the BIO connector. All > communication > > > > with BIO is blocking. This is working as designed. > > > > > > OK, but my understanding was that there is a difference between the > > terms > > > "synchronous/asynchronous" and "blocking/non-blocking" (but maybe the > > > meaning differs from programming language to programming language). > > > > > > > An excellent and detailed explanation on this topic can be found in > "UNIX > > Network Programing" R. Stevens Vol 1 3td ed. p154-p160 > > Thank you for that information. > However, currently I don't have that book (and I'm not very familiar with > UNIX as I'm a windows guy ;) ), but please let me try to illustrate my use > case with the Snake example. > > > The SnakeTimer class has a broadcast() method that broadcasts a message to > all connected clients by looping over them and calling > snake.sendMessage(...), which in turn will call > session.getBasicRemote().sendText(msg) to send the message. As the methods > of RemoteEndpoint.Basic are blocking, this means that if some client > establishes a Websocket connection and stops to read from it, then after > some time the broadcast() method will hang waiting for sendText() to > complete, which will block until that specific client continues to read > data. This means that messages cannot be broadcasted to all other clients, > so for them all snakes stop moving. > > A simple way that comes in my mind to solve this would be to use an > additional thread for each connected client. This thread would take message > from a Queue and send them to the client while the SnakeTimer's broadcast() > would add a messages to the queue of each client. Then, if some client's > sendText() method blocks (because the client does not read data from the > connection), it does not interfere with the other clients, so they still > would see the snakes moving. > > However, this is costly because it would require an additional thread per > Websocket connection. > > So, another way would be to look at RemoteEndpoint.Async that can send > messages asynchronously. The javadoc (from Oracle) of > Async#sendText(String, SendHandler) says, "Initiates the asynchronous > transmission of a text message. This method returns before the message is > transmitted." > For me, this looks like I can call this method and be sure that it does > not block when some client does not read data from the websocket
Re: 8.0.x / 7.0.x progress
On Sat, Oct 5, 2013 at 2:18 AM, Konstantin Preißer wrote: > Hi Mark, > > > -Original Message- > > From: Mark Thomas [mailto:ma...@apache.org] > > Sent: Friday, October 4, 2013 11:58 PM > > To: Tomcat Developers List > > Subject: Re: 8.0.x / 7.0.x progress > > > > On 04/10/2013 22:49, Konstantin Preißer wrote: > > > Hi Mark, > > > > > > while I was developing and experimenting with a WebSocket application > > (that I think could be added to Tomcat 8's WebSocket examples), I think I > > found some more possible issues with the Websocket handling. One of > > these can be seen with the Snake example. > > > > > > > > > A) For NIO and APR connector: > > > It seems that when a client establishes a Websocket connection and then > > stops to read data from it (but doesn't close the connection, so that > writing > > data to the underlying TCP connection will be blocked), and then after > some > > time continues to read data from the Websocket connection, then Tomcat > > seems not to be able to read data from that client any more (and doesn't > > notice when the connection has been closed), but it can still read from > it. > > > > I think you have written read when you meant write somewhere in the > > previous sentence. > > Sorry, yes. I'll try again: > > It seems that when a client establishes a Websocket connection and then > stops to read data from it (but doesn't close the connection, so that at > Tomcat's side writing data to the underlying TCP connection will be > blocked), and then after some time the client continues to read data from > the Websocket connection, then Tomcat seems not to be able to read data > from that client any more (and doesn't notice when the connection has been > closed), but it can still write data to the client. > > > > > To reproduce: > > > 1) Start Tomcat (current trunk) on Windows 64-bit with Java7 64-bit and > > either NIO or APR connector. > > > 2) Open two instances of Firefox (that may or may not be on the same > > machine) and open the snake example. On both instances, press up or down > > key so that both snakes begin moving. > > > 3) Suspend one of the two Firefox processes (say Firefox B). This can > be > > done with "Process Explorer" [1] tool by right-clicking on the > firefox.exe > > entry and select "Suspend". You can see that Firefox B does not respond > any > > more, but on Firefox A the snakes continue to move. > > > 4) After some time, you can see that on Firefox A the snakes suddenly > stop > > moving. This is correct because the current code uses > RemoteEndpoint.Basic > > that may block on write() methods. > > > 5) Now resume Firefox B with Process Explorer. You can see that on both > > Firefoxes the snakes will continue to move. This means that both > Firefoxes > > are able to receive data from the Websocket connection. > > > 6) When you try to change the direction of the snake in Firefox A, > > everything works. However on Firefox B, the snake will not change its > > direction. This means that while Tomcat continues to send data to this > > Websocket connection, it cannot receive from it any more. > > > 7) If you close Firefox B, then the corresponding snake will not > disappear > > (so it seems Tomcat doesn't notice that the connection closed). > > > > It would be worth taking a look with something like Wireshark to see if > > the data is being sent to Tomcat. > > I have installed Wireshark now and can confirm that Firefox (after it is > resumed) still sends data over the TCP connection which Tomcat seems not to > be able to read. > Are there ACKs for the TCP packets? I'll try to reproduce the case. > > > > > B) For BIO connector: > > > I noticed that on Tomcat with BIO connector, when using a > > RemoteEndpoint.Async to asynchronously send data over the WebSocket > > connection, sendText(String, SendHandler) (or similar methods) will > block if > > the Remote endpoint does not read data from the connection, whereas for > > NIO and APR connector this method will always return immediately. > > > Is it intended that for the BIO connector those methods are blocking? > As > > the javadoc says, "Initiates the asynchronous transmission of a text > message. > > This method returns before the message is transmitted.", I would have > > expected that e.g. another Thread is used to write in blocking mode, so > that > > the sendText() method can return immediately. > > > > You can't do non-blocking IO with the BIO connector. All communication > > with BIO is blocking. This is working as designed. > > OK, but my understanding was that there is a difference between the terms > "synchronous/asynchronous" and "blocking/non-blocking" (but maybe the > meaning differs from programming language to programming language). > An excellent and detailed explanation on this topic can be found in "UNIX Network Programing" R. Stevens Vol 1 3td ed. p154-p160 > > For example, in .Net applications (C#) you can do Stream operations (read, > write) in a synchronous and asynchron
Re: Back-porting JSR-356 Progress update
On Thu, Aug 22, 2013 at 5:47 PM, Mark Thomas wrote: > On 22/08/2013 15:30, Niki Dokovski wrote: > > On Thu, Aug 22, 2013 at 1:08 AM, Mark Thomas wrote: > > > >> Investigating some Gump and BuildBot failures has identified a large > >> block of functionality that still remains to be back-ported to 7.0.x - > >> namely the changes to Coyote that allowed upgraded connections to > >> support concurrent read and write. > >> > >> It isn't as simple as copy and paste as there has been some additional > >> refactoring particularly in the endpoints we probably don't want to > >> backport. > >> > >> Fixing this is top of my TODO list followed by getting Gump and Buidlbot > >> passing for 7.0.x and 8.0.x and then another 8.0.0 RC.# > >> > >> Mark > >> > > > > tomcat-7.0.43-dev revision 1516420 passes JSR 356 TCKs > > Excellent. Thanks for that update. With which connector(s)? > > (He asks knowing that the APR implementation is very likely to fail as > it needs the concurrent read/write changes back-ported from trunk). > Default without any additional configuration. > > I also want to run Autobahn against it but I have a few issues to fix > first. > > Cheers, > > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Back-porting JSR-356 Progress update
On Thu, Aug 22, 2013 at 1:08 AM, Mark Thomas wrote: > Investigating some Gump and BuildBot failures has identified a large > block of functionality that still remains to be back-ported to 7.0.x - > namely the changes to Coyote that allowed upgraded connections to > support concurrent read and write. > > It isn't as simple as copy and paste as there has been some additional > refactoring particularly in the endpoints we probably don't want to > backport. > > Fixing this is top of my TODO list followed by getting Gump and Buidlbot > passing for 7.0.x and 8.0.x and then another 8.0.0 RC.# > > Mark > tomcat-7.0.43-dev revision 1516420 passes JSR 356 TCKs Niki > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Back-porting JSR-356 implementation to 7.0.x
On Fri, Aug 16, 2013 at 11:01 AM, Mark Thomas wrote: > On 16/08/2013 08:16, Niki Dokovski wrote: > > On Fri, Aug 16, 2013 at 9:54 AM, Mark Thomas wrote: > > > >> Christopher Schultz wrote: > >>> Mark, > >>> > >>> On 8/15/13 6:47 PM, Mark Thomas wrote: > >>>> This isn't going to be quite as simple as I first thought. > >>>> > >>>> The WebSocket client API requires Java SE 7 or later. > >>>> The WebSocket server API requires Java EE 6 or later. > >>>> Java EE 6 requires Java 6 or later. > >>> > >>> Clarification: are you saying that Java EE requires that it be allowed > >>> to run on JVMs as low as Java 6? > >> > >> Java EE 6 is the version Tomcat 7 uses. > >> > >>>> The WebSocket server API depends on the WebSocket client API. > >>>> > >>>> The WebSocket client implementation makes extensive use of new Java 7 > >>>> non-blocking IO features. > >>> > >>> Are any of these possible (albeit with some back-flips) with Java 6? > >> > >> The first thing I thought of. It is possible but would require a > selector, > >> poller and thread pool implementation plus associated plumbing. Java 7 > >> looks like the better option at this point. > >> > >>> Perhaps one class (Java7NioShims) could be replaced with another > >>> implementation (Java6NioShim) at runtime depending upon the current > >>> JVM? > >>> I know nothing about the code involved... just stabbing in the dark. > >>> > >>>> My conclusion from the above is that the back-port is going to > >>> require > >>>> Java 7. That begs the question how to do that while keeping the main > >>>> build Java 6 based. > >>>> > >>>> My (untested) plan is as follows: > >>>> - Create a WebSocket module > >>>> - Back-port (i.e. copy) the trunk code to that module > >>>> - Build just that module with Java 7 > >>>> - Make the minimum changes necessary to get it to work > >>>> - Modify the back-ported SCI so it only adds the filter if Java 7 is > >>>> detected (going to need to ensure the SCI is executable on Java 6) > >>>> - Ship Tomcat 7 with the WebSocket JARs > >>> > >>> I'm not sure there's a better way, but the whole > >>> compile-one-module-with-a-higher-version thing has been a small thorn > >>> in > >>> the past (IIRC, Tomcat 6 had to be built with Java 6, but could be run > >>> with Java 5) causing a mild amount of confusion. If this could be > >>> avoided, I think it might be best. > >> > >> I know. I'm trying to find the minimum effort, minimum risk, minimum > >> hassle solution but there isn't a perfect solution that I can see. > >> > >>> Maybe we could package JSR-356-for-Tomcat7 as an optional module that > >>> has a Java 7 prerequisite? > >> > >> I'd really like to ship JSR-356 suport as standard. > >> > > > > How about making Java 7 hard requirement for releases that include JSR > 356 > > implementation? Is this OK? > > No. Tomcat 7 needs to run on Java 6. I think it is doable with a little > hoop jumping. Lets see what a first solution looks like and worry about > better solutions if we need one. > OK. The implementation in Tomcat 8 is well tested and we need to do the same for Tomcat 7. niki > > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Back-porting JSR-356 implementation to 7.0.x
On Fri, Aug 16, 2013 at 9:54 AM, Mark Thomas wrote: > Christopher Schultz wrote: > >Mark, > > > >On 8/15/13 6:47 PM, Mark Thomas wrote: > >> This isn't going to be quite as simple as I first thought. > >> > >> The WebSocket client API requires Java SE 7 or later. > >> The WebSocket server API requires Java EE 6 or later. > >> Java EE 6 requires Java 6 or later. > > > >Clarification: are you saying that Java EE requires that it be allowed > >to run on JVMs as low as Java 6? > > Java EE 6 is the version Tomcat 7 uses. > > >> The WebSocket server API depends on the WebSocket client API. > >> > >> The WebSocket client implementation makes extensive use of new Java 7 > >> non-blocking IO features. > > > >Are any of these possible (albeit with some back-flips) with Java 6? > > The first thing I thought of. It is possible but would require a selector, > poller and thread pool implementation plus associated plumbing. Java 7 > looks like the better option at this point. > > >Perhaps one class (Java7NioShims) could be replaced with another > >implementation (Java6NioShim) at runtime depending upon the current > >JVM? > >I know nothing about the code involved... just stabbing in the dark. > > > >> My conclusion from the above is that the back-port is going to > >require > >> Java 7. That begs the question how to do that while keeping the main > >> build Java 6 based. > >> > >> My (untested) plan is as follows: > >> - Create a WebSocket module > >> - Back-port (i.e. copy) the trunk code to that module > >> - Build just that module with Java 7 > >> - Make the minimum changes necessary to get it to work > >> - Modify the back-ported SCI so it only adds the filter if Java 7 is > >> detected (going to need to ensure the SCI is executable on Java 6) > >> - Ship Tomcat 7 with the WebSocket JARs > > > >I'm not sure there's a better way, but the whole > >compile-one-module-with-a-higher-version thing has been a small thorn > >in > >the past (IIRC, Tomcat 6 had to be built with Java 6, but could be run > >with Java 5) causing a mild amount of confusion. If this could be > >avoided, I think it might be best. > > I know. I'm trying to find the minimum effort, minimum risk, minimum > hassle solution but there isn't a perfect solution that I can see. > > >Maybe we could package JSR-356-for-Tomcat7 as an optional module that > >has a Java 7 prerequisite? > > I'd really like to ship JSR-356 suport as standard. > How about making Java 7 hard requirement for releases that include JSR 356 implementation? Is this OK? It's preferable to keep the implementations in Tomcat 7 and Tomcat 8 as close as possible. > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Back-porting JSR-356 implementation to 7.0.x
On Fri, Aug 16, 2013 at 4:34 AM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Mark, > > On 8/15/13 6:47 PM, Mark Thomas wrote: > > This isn't going to be quite as simple as I first thought. > > > > The WebSocket client API requires Java SE 7 or later. > > The WebSocket server API requires Java EE 6 or later. > > Java EE 6 requires Java 6 or later. > > Clarification: are you saying that Java EE requires that it be allowed > to run on JVMs as low as Java 6? > > > The WebSocket server API depends on the WebSocket client API. > > > > The WebSocket client implementation makes extensive use of new Java 7 > > non-blocking IO features. > > Are any of these possible (albeit with some back-flips) with Java 6? > Perhaps one class (Java7NioShims) could be replaced with another > implementation (Java6NioShim) at runtime depending upon the current JVM? > I know nothing about the code involved... just stabbing in the dark. > > > My conclusion from the above is that the back-port is going to require > > Java 7. That begs the question how to do that while keeping the main > > build Java 6 based. > > > > My (untested) plan is as follows: > > - Create a WebSocket module > > - Back-port (i.e. copy) the trunk code to that module > > - Build just that module with Java 7 > > - Make the minimum changes necessary to get it to work > > - Modify the back-ported SCI so it only adds the filter if Java 7 is > > detected (going to need to ensure the SCI is executable on Java 6) > > - Ship Tomcat 7 with the WebSocket JARs > > I'm not sure there's a better way, but the whole > compile-one-module-with-a-higher-version thing has been a small thorn in > the past (IIRC, Tomcat 6 had to be built with Java 6, but could be run > with Java 5) causing a mild amount of confusion. If this could be > avoided, I think it might be best. > > Maybe we could package JSR-356-for-Tomcat7 as an optional module that > has a Java 7 prerequisite? > +1 for making it optional and Java 7 dependent in Tomcat 7 Keep it simple if possibble Java 7 is slowly adopted in productive environments anyway. One reason is EOL http://www.oracle.com/technetwork/java/eol-135779.html > -chris > >
Re: WebSocket: to flip or not the ByteBuffer during sendBinary in RemoteEndpoint(s)
On Fri, Jun 28, 2013 at 3:57 PM, Niki Dokovski wrote: > > > > On Fri, Jun 28, 2013 at 3:44 PM, Mark Thomas wrote: > >> On 28/06/2013 12:47, Niki Dokovski wrote: >> > Hi folks, >> > while playing around with tyrus and tomcat implementation of websocket I >> > spotted a difference in the way sendBinary is actually implemented. In >> > short: tyrus uses bytebuffer.array(), hence there is no change in >> buffer's >> > position while we end with channel write operation that does this. >> Neither >> > the spec nor the javadoc detail that but the result is that one >> application >> > can run perfectly on one of the implementations and could cause problem >> on >> > the other. Shall we contact the EG for clarification on this matter? >> >> No need. The EG has already stated its view (well, the EG lead did and >> no-one disagreed) >> >> >> Since the spec does not say anything about re-using ByteBuffers and they >> are mutable objects, I would expect the conventional developer practice >> to be to use a new one each time. >> >> > > Thanks for sharing. This is an assumption about what is "conventional > developer practice" :) > >> >> > Opinions? >> >> I agree with the EG lead. Client's should not be making any assumptions >> about what the implementation will or won't do with a ByteBuffer. >> >> If you want to argue for a specific behaviour, open an issue against the >> spec. >> >> I'll ask for a clarification and let's see. The effect is that because of > this we can lose portability. > https://java.net/jira/browse/WEBSOCKET_SPEC-209 > > > >> Mark >> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: dev-h...@tomcat.apache.org >> >> >
Re: WebSocket: to flip or not the ByteBuffer during sendBinary in RemoteEndpoint(s)
On Fri, Jun 28, 2013 at 3:44 PM, Mark Thomas wrote: > On 28/06/2013 12:47, Niki Dokovski wrote: > > Hi folks, > > while playing around with tyrus and tomcat implementation of websocket I > > spotted a difference in the way sendBinary is actually implemented. In > > short: tyrus uses bytebuffer.array(), hence there is no change in > buffer's > > position while we end with channel write operation that does this. > Neither > > the spec nor the javadoc detail that but the result is that one > application > > can run perfectly on one of the implementations and could cause problem > on > > the other. Shall we contact the EG for clarification on this matter? > > No need. The EG has already stated its view (well, the EG lead did and > no-one disagreed) > > > Since the spec does not say anything about re-using ByteBuffers and they > are mutable objects, I would expect the conventional developer practice > to be to use a new one each time. > > Thanks for sharing. This is an assumption about what is "conventional developer practice" :) > > > Opinions? > > I agree with the EG lead. Client's should not be making any assumptions > about what the implementation will or won't do with a ByteBuffer. > > If you want to argue for a specific behaviour, open an issue against the > spec. > > I'll ask for a clarification and let's see. The effect is that because of this we can lose portability. > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
WebSocket: to flip or not the ByteBuffer during sendBinary in RemoteEndpoint(s)
Hi folks, while playing around with tyrus and tomcat implementation of websocket I spotted a difference in the way sendBinary is actually implemented. In short: tyrus uses bytebuffer.array(), hence there is no change in buffer's position while we end with channel write operation that does this. Neither the spec nor the javadoc detail that but the result is that one application can run perfectly on one of the implementations and could cause problem on the other. Shall we contact the EG for clarification on this matter? Opinions? cheers niki
Re: [Bug 55006] Add http proxy support for ClientEndpoint using system properties
On Tue, Jun 4, 2013 at 12:01 AM, Rossen Stoyanchev < rstoyanc...@gopivotal.com> wrote: > On Mon, Jun 3, 2013 at 10:27 AM, Niki Dokovski wrote: > > > On Mon, Jun 3, 2013 at 3:44 PM, Niki Dokovski > wrote:Further > > on, if you choose to use connectToServer call that has the > > ClientEndpointConfig param you are losing the annotation approach as the > > first param of this call has to be an instance of an Endpoint. Hence your > > class can not be just annotated. hmmm Mark where can we bring this > > discussion? > > > This is not very obvious from the API but ClientEndpointConfig (and > ServerEndpointConfig) are not meant to be used with annotated endpoints. In > other words, I think the methods accepting those types are for use with > type-based endpoints (i.e. javax.websocket.Endpoint). So as far as I can > see user properties can be updated before the session starts only for > type-based endpoints. > Exactly, What is the rationale behind this? Still an annotated endpoint can supply javax.websocket.ClientEndpointConfig.Configurator but has no access to the user properties from there. May be the javax.websocket.ClientEndpointConfig.Configurator can be extended to provide access to the map. cheers > > Rossen >
Re: [Bug 55006] Add http proxy support for ClientEndpoint using system properties
On Mon, Jun 3, 2013 at 3:44 PM, Niki Dokovski wrote: > > > > On Mon, Jun 3, 2013 at 3:13 PM, wrote: > >> https://issues.apache.org/bugzilla/show_bug.cgi?id=55006 >> >> --- Comment #3 from Konstantin Kolinko --- >> (In reply to Mark Thomas from comment #2) >> > Generally, I dislike using system properties for any kind of >> configuration. >> >> I do as well. >> >> How about using ClientEndpointConfig.getUserProperties() to pass those? >> > > +1 > In general, I'm fine with any proposal that removes the limitation. Just > hope that JSR 356 EG won't wait for Java EE 8 to come up with an update. > Meanwhile people can use current patch. Or we can craft something based on > user configurations in ClientEndpoint leveraging Mark's ProxySelector. I'll > give it a try... > > cheers > At first look, It turned to be a bit challenging of how to get to the modifiable user properties map if you work only with annotated class. Just created another bug for the spec. (The javadoc in one of the connectToServer methods should be changed as it states that the supplied object should be annotated with ServerEndpoint where it actually should be annotated with ClientEndpoint. https://java.net/jira/browse/WEBSOCKET_SPEC-206 Further on, if you choose to use connectToServer call that has the ClientEndpointConfig param you are losing the annotation approach as the first param of this call has to be an instance of an Endpoint. Hence your class can not be just annotated. hmmm Mark where can we bring this discussion? > >> -- >> You are receiving this mail because: >> You are the assignee for the bug. >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: dev-h...@tomcat.apache.org >> >> >
Re: [Bug 55006] Add http proxy support for ClientEndpoint using system properties
On Mon, Jun 3, 2013 at 3:13 PM, wrote: > https://issues.apache.org/bugzilla/show_bug.cgi?id=55006 > > --- Comment #3 from Konstantin Kolinko --- > (In reply to Mark Thomas from comment #2) > > Generally, I dislike using system properties for any kind of > configuration. > > I do as well. > > How about using ClientEndpointConfig.getUserProperties() to pass those? > +1 In general, I'm fine with any proposal that removes the limitation. Just hope that JSR 356 EG won't wait for Java EE 8 to come up with an update. Meanwhile people can use current patch. Or we can craft something based on user configurations in ClientEndpoint leveraging Mark's ProxySelector. I'll give it a try... cheers > > -- > You are receiving this mail because: > You are the assignee for the bug. > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Wed, May 22, 2013 at 1:48 PM, Konstantin Kolinko wrote: > 2013/5/22 Niki Dokovski : > > > > > > > > On Tue, May 21, 2013 at 3:15 PM, Niki Dokovski > wrote: > >> > >> > >> > >> > >> On Mon, May 20, 2013 at 5:09 PM, Niki Dokovski > wrote: > >>> > >>> > >>> > >>> > >>> On Mon, May 20, 2013 at 3:45 PM, Niki Dokovski > wrote: > >>>> > >>>> > >>>> > >>>> > >>>> On Fri, May 17, 2013 at 3:53 PM, Niki Dokovski > >>>> wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Fri, May 17, 2013 at 3:35 PM, Mark Thomas > wrote: > >>>>>> > >>>>>> On 17/05/2013 13:07, Niki Dokovski wrote: > >>>>>> > Hi, > >>>>>> > > >>>>>> > Developing some sample apps with websockets and tomcat 8 revealed > >>>>>> > two > >>>>>> > issues I’d like to discuss here. > >>>>>> > > >>>>>> > 1. There is no proxy support in the implementation of > >>>>>> > WebSocketContainer.connectToServer calls. While playing around > with > >>>>>> > the > >>>>>> > code it was relatively easy to add this in the implementation with > >>>>>> > simple > >>>>>> > consideration of http.proxyHost and Port system properties; > >>>>>> > replacing > >>>>>> > host and port values which were obtained from path method > parameter > >>>>>> > and > >>>>>> > making initial HTTP CONNECT message to the proxy. In this case, do > >>>>>> > you > >>>>>> > think we can add proxy support here? I was thinking how to provide > >>>>>> > some > >>>>>> > kind of automation test as well, but no luck so far. > >>>>>> > >>>>>> Personally I dislike the global nature system properties for > >>>>>> configuration unless they are absolutely the only option. > >>>>>> > >>>>>> Since the WebSocket API has no way of setting a proxy at the moment, > >>>>>> system properties might be the only choice :( > >>>>>> > >>>>>> This is an issue you should raise with the WebSocket EG group. I > >>>>>> suggest > >>>>>> opening a Jira to request proxy support in a future version. > >>>>>> https://java.net/jira/browse/WEBSOCKET_SPEC > >>>>> > >>>>> > >>>>> Here is the issue: > >>>>> https://java.net/jira/browse/WEBSOCKET_SPEC-202 > >>>>> > >>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > 2. So when I got my proxy tunneling in place (with the > CONNECT > >>>>>> > http > >>>>>> > request) I hit a tougher problem with SSL handshake implementation > >>>>>> > in > >>>>>> > WebSocketSslHandshakeThread. The SSL handshake was successful > while > >>>>>> > I was > >>>>>> > stepping with the debugger through the code and failed when I run > >>>>>> > the > >>>>>> > test. It should be some timing issue. This bit needs more > debugging. > >>>>>> > Have > >>>>>> > you seen this before? > >>>>>> > >>>>>> No, but the SSL support for WebSocket clients hasn't been heavily > used > >>>>>> yet so there could still be some bugs to iron out. If you haven't > >>>>>> already, take a look at > >>>>>> TestWsWebSocketContainer.testConnectToServerEndpointSSL() > >>>>>> > >>>>>> Mark > >>>>>> > >>>>>> > >>>>> > >>>>> Thanks. I'll take a look at it. > >>>>> > >>>> > >>>> > >>>> Doing some debugging in the implementation of > >>>> AsyncChannelWeapperSecure.WebSocketSslHandshakeThread
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Tue, May 21, 2013 at 3:15 PM, Niki Dokovski wrote: > > > > On Mon, May 20, 2013 at 5:09 PM, Niki Dokovski wrote: > >> >> >> >> On Mon, May 20, 2013 at 3:45 PM, Niki Dokovski wrote: >> >>> >>> >>> >>> On Fri, May 17, 2013 at 3:53 PM, Niki Dokovski wrote: >>> >>>> >>>> >>>> >>>> On Fri, May 17, 2013 at 3:35 PM, Mark Thomas wrote: >>>> >>>>> On 17/05/2013 13:07, Niki Dokovski wrote: >>>>> > Hi, >>>>> > >>>>> > Developing some sample apps with websockets and tomcat 8 revealed two >>>>> > issues I’d like to discuss here. >>>>> > >>>>> > 1. There is no proxy support in the implementation of >>>>> > WebSocketContainer.connectToServer calls. While playing around with >>>>> the >>>>> > code it was relatively easy to add this in the implementation with >>>>> simple >>>>> > consideration of http.proxyHost and Port system properties; >>>>> replacing >>>>> > host and port values which were obtained from path method parameter >>>>> and >>>>> > making initial HTTP CONNECT message to the proxy. In this case, do >>>>> you >>>>> > think we can add proxy support here? I was thinking how to provide >>>>> some >>>>> > kind of automation test as well, but no luck so far. >>>>> >>>>> Personally I dislike the global nature system properties for >>>>> configuration unless they are absolutely the only option. >>>>> >>>>> Since the WebSocket API has no way of setting a proxy at the moment, >>>>> system properties might be the only choice :( >>>>> >>>>> This is an issue you should raise with the WebSocket EG group. I >>>>> suggest >>>>> opening a Jira to request proxy support in a future version. >>>>> https://java.net/jira/browse/WEBSOCKET_SPEC >>>> >>>> >>>> Here is the issue: >>>> https://java.net/jira/browse/WEBSOCKET_SPEC-202 >>>> >>>> >>>> >>>>> >>>>> >>>>> >>>>> > 2. So when I got my proxy tunneling in place (with the CONNECT >>>>> http >>>>> > request) I hit a tougher problem with SSL handshake implementation in >>>>> > WebSocketSslHandshakeThread. The SSL handshake was successful while >>>>> I was >>>>> > stepping with the debugger through the code and failed when I run >>>>> the >>>>> > test. It should be some timing issue. This bit needs more debugging. >>>>> Have >>>>> > you seen this before? >>>>> >>>>> No, but the SSL support for WebSocket clients hasn't been heavily used >>>>> yet so there could still be some bugs to iron out. If you haven't >>>>> already, take a look at >>>>> TestWsWebSocketContainer.testConnectToServerEndpointSSL() >>>>> >>>>> Mark >>>>> >>>>> >>>>> >>>> Thanks. I'll take a look at it. >>>> >>>> >>> >>> Doing some debugging in the implementation of >>> AsyncChannelWeapperSecure.WebSocketSslHandshakeThread run() >>> revealed that by having small timeout (thread sleep) in "NEED_WRAP" >>> case, allows SSL handshake to complete successfully. >>> (This is still the case when there is a HTTP Proxy tunnel established >>> between Endpoints) This is not the solution for the problem, of course. >>> Here I attach some sysout logs with working and non-working flows. >>> 1. working example: >>> after sending initial message sleep the executing thread in "need_wrap" >>> case for little and you get following sequence >>> write: 154 >>> write done: true >>> NEED_UNWRAP >>> read: 2357 >>> read done: true >>> >>> SSL Handshake is OK >>> >>> >>> 2. non working example >>> write: 154 >>> write done: true >>> NEED_UNWRAP >>> read: 1460 << much less bytes to consider >>> read done: true >>> ... >>> Exception >>> >>> >>> Any comments? >>> >> >> getting closer as the error condition is caused by BUFFER_UNDERFLOW >> state. Will try to read again in this condition. Basically there is no >> enough bytes to construct the message in SSLEngine. >> > > OK that does the trick. Here is a bug on the issue > https://issues.apache.org/bugzilla/show_bug.cgi?id=54997 > I'll try to polish a bit and send a small code snippet solving > BUFFER_UNDERFLOW case for discussion. > > cheers > Niki > Here are two patches for http proxy support (WsWebSocketContainer) and buffer underflow (AsyncChannelWrapperSecure) fix respectively. I'd like to ask for your comments. cheers Niki > > >> >>> >>> >>> >>> >>> >>>> - >>>>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >>>>> For additional commands, e-mail: dev-h...@tomcat.apache.org >>>>> >>>>> >>>> >>> >> > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Mon, May 20, 2013 at 5:09 PM, Niki Dokovski wrote: > > > > On Mon, May 20, 2013 at 3:45 PM, Niki Dokovski wrote: > >> >> >> >> On Fri, May 17, 2013 at 3:53 PM, Niki Dokovski wrote: >> >>> >>> >>> >>> On Fri, May 17, 2013 at 3:35 PM, Mark Thomas wrote: >>> >>>> On 17/05/2013 13:07, Niki Dokovski wrote: >>>> > Hi, >>>> > >>>> > Developing some sample apps with websockets and tomcat 8 revealed two >>>> > issues I’d like to discuss here. >>>> > >>>> > 1. There is no proxy support in the implementation of >>>> > WebSocketContainer.connectToServer calls. While playing around with >>>> the >>>> > code it was relatively easy to add this in the implementation with >>>> simple >>>> > consideration of http.proxyHost and Port system properties; >>>> replacing >>>> > host and port values which were obtained from path method parameter >>>> and >>>> > making initial HTTP CONNECT message to the proxy. In this case, do you >>>> > think we can add proxy support here? I was thinking how to provide >>>> some >>>> > kind of automation test as well, but no luck so far. >>>> >>>> Personally I dislike the global nature system properties for >>>> configuration unless they are absolutely the only option. >>>> >>>> Since the WebSocket API has no way of setting a proxy at the moment, >>>> system properties might be the only choice :( >>>> >>>> This is an issue you should raise with the WebSocket EG group. I suggest >>>> opening a Jira to request proxy support in a future version. >>>> https://java.net/jira/browse/WEBSOCKET_SPEC >>> >>> >>> Here is the issue: >>> https://java.net/jira/browse/WEBSOCKET_SPEC-202 >>> >>> >>> >>>> >>>> >>>> >>>> > 2. So when I got my proxy tunneling in place (with the CONNECT >>>> http >>>> > request) I hit a tougher problem with SSL handshake implementation in >>>> > WebSocketSslHandshakeThread. The SSL handshake was successful while I >>>> was >>>> > stepping with the debugger through the code and failed when I run the >>>> > test. It should be some timing issue. This bit needs more debugging. >>>> Have >>>> > you seen this before? >>>> >>>> No, but the SSL support for WebSocket clients hasn't been heavily used >>>> yet so there could still be some bugs to iron out. If you haven't >>>> already, take a look at >>>> TestWsWebSocketContainer.testConnectToServerEndpointSSL() >>>> >>>> Mark >>>> >>>> >>>> >>> Thanks. I'll take a look at it. >>> >>> >> >> Doing some debugging in the implementation of >> AsyncChannelWeapperSecure.WebSocketSslHandshakeThread run() >> revealed that by having small timeout (thread sleep) in "NEED_WRAP" case, >> allows SSL handshake to complete successfully. >> (This is still the case when there is a HTTP Proxy tunnel established >> between Endpoints) This is not the solution for the problem, of course. >> Here I attach some sysout logs with working and non-working flows. >> 1. working example: >> after sending initial message sleep the executing thread in "need_wrap" >> case for little and you get following sequence >> write: 154 >> write done: true >> NEED_UNWRAP >> read: 2357 >> read done: true >> >> SSL Handshake is OK >> >> >> 2. non working example >> write: 154 >> write done: true >> NEED_UNWRAP >> read: 1460 << much less bytes to consider >> read done: true >> ... >> Exception >> >> >> Any comments? >> > > getting closer as the error condition is caused by BUFFER_UNDERFLOW state. > Will try to read again in this condition. Basically there is no enough > bytes to construct the message in SSLEngine. > OK that does the trick. Here is a bug on the issue https://issues.apache.org/bugzilla/show_bug.cgi?id=54997 I'll try to polish a bit and send a small code snippet solving BUFFER_UNDERFLOW case for discussion. cheers Niki > >> >> >> >> >> >>> - >>>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: dev-h...@tomcat.apache.org >>>> >>>> >>> >> >
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Mon, May 20, 2013 at 3:45 PM, Niki Dokovski wrote: > > > > On Fri, May 17, 2013 at 3:53 PM, Niki Dokovski wrote: > >> >> >> >> On Fri, May 17, 2013 at 3:35 PM, Mark Thomas wrote: >> >>> On 17/05/2013 13:07, Niki Dokovski wrote: >>> > Hi, >>> > >>> > Developing some sample apps with websockets and tomcat 8 revealed two >>> > issues I’d like to discuss here. >>> > >>> > 1. There is no proxy support in the implementation of >>> > WebSocketContainer.connectToServer calls. While playing around with the >>> > code it was relatively easy to add this in the implementation with >>> simple >>> > consideration of http.proxyHost and Port system properties; >>> replacing >>> > host and port values which were obtained from path method parameter and >>> > making initial HTTP CONNECT message to the proxy. In this case, do you >>> > think we can add proxy support here? I was thinking how to provide some >>> > kind of automation test as well, but no luck so far. >>> >>> Personally I dislike the global nature system properties for >>> configuration unless they are absolutely the only option. >>> >>> Since the WebSocket API has no way of setting a proxy at the moment, >>> system properties might be the only choice :( >>> >>> This is an issue you should raise with the WebSocket EG group. I suggest >>> opening a Jira to request proxy support in a future version. >>> https://java.net/jira/browse/WEBSOCKET_SPEC >> >> >> Here is the issue: >> https://java.net/jira/browse/WEBSOCKET_SPEC-202 >> >> >> >>> >>> >>> >>> > 2. So when I got my proxy tunneling in place (with the CONNECT >>> http >>> > request) I hit a tougher problem with SSL handshake implementation in >>> > WebSocketSslHandshakeThread. The SSL handshake was successful while I >>> was >>> > stepping with the debugger through the code and failed when I run the >>> > test. It should be some timing issue. This bit needs more debugging. >>> Have >>> > you seen this before? >>> >>> No, but the SSL support for WebSocket clients hasn't been heavily used >>> yet so there could still be some bugs to iron out. If you haven't >>> already, take a look at >>> TestWsWebSocketContainer.testConnectToServerEndpointSSL() >>> >>> Mark >>> >>> >>> >> Thanks. I'll take a look at it. >> >> > > Doing some debugging in the implementation of > AsyncChannelWeapperSecure.WebSocketSslHandshakeThread run() > revealed that by having small timeout (thread sleep) in "NEED_WRAP" case, > allows SSL handshake to complete successfully. > (This is still the case when there is a HTTP Proxy tunnel established > between Endpoints) This is not the solution for the problem, of course. > Here I attach some sysout logs with working and non-working flows. > 1. working example: > after sending initial message sleep the executing thread in "need_wrap" > case for little and you get following sequence > write: 154 > write done: true > NEED_UNWRAP > read: 2357 > read done: true > > SSL Handshake is OK > > > 2. non working example > write: 154 > write done: true > NEED_UNWRAP > read: 1460 << much less bytes to consider > read done: true > ... > Exception > > > Any comments? > getting closer as the error condition is caused by BUFFER_UNDERFLOW state. Will try to read again in this condition. Basically there is no enough bytes to construct the message in SSLEngine. > > > > > > >> - >>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: dev-h...@tomcat.apache.org >>> >>> >> >
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Fri, May 17, 2013 at 3:53 PM, Niki Dokovski wrote: > > > > On Fri, May 17, 2013 at 3:35 PM, Mark Thomas wrote: > >> On 17/05/2013 13:07, Niki Dokovski wrote: >> > Hi, >> > >> > Developing some sample apps with websockets and tomcat 8 revealed two >> > issues I’d like to discuss here. >> > >> > 1. There is no proxy support in the implementation of >> > WebSocketContainer.connectToServer calls. While playing around with the >> > code it was relatively easy to add this in the implementation with >> simple >> > consideration of http.proxyHost and Port system properties; >> replacing >> > host and port values which were obtained from path method parameter and >> > making initial HTTP CONNECT message to the proxy. In this case, do you >> > think we can add proxy support here? I was thinking how to provide some >> > kind of automation test as well, but no luck so far. >> >> Personally I dislike the global nature system properties for >> configuration unless they are absolutely the only option. >> >> Since the WebSocket API has no way of setting a proxy at the moment, >> system properties might be the only choice :( >> >> This is an issue you should raise with the WebSocket EG group. I suggest >> opening a Jira to request proxy support in a future version. >> https://java.net/jira/browse/WEBSOCKET_SPEC > > > Here is the issue: > https://java.net/jira/browse/WEBSOCKET_SPEC-202 > > > >> >> >> >> > 2. So when I got my proxy tunneling in place (with the CONNECT >> http >> > request) I hit a tougher problem with SSL handshake implementation in >> > WebSocketSslHandshakeThread. The SSL handshake was successful while I >> was >> > stepping with the debugger through the code and failed when I run the >> > test. It should be some timing issue. This bit needs more debugging. >> Have >> > you seen this before? >> >> No, but the SSL support for WebSocket clients hasn't been heavily used >> yet so there could still be some bugs to iron out. If you haven't >> already, take a look at >> TestWsWebSocketContainer.testConnectToServerEndpointSSL() >> >> Mark >> >> >> > Thanks. I'll take a look at it. > > Doing some debugging in the implementation of AsyncChannelWeapperSecure.WebSocketSslHandshakeThread run() revealed that by having small timeout (thread sleep) in "NEED_WRAP" case, allows SSL handshake to complete successfully. (This is still the case when there is a HTTP Proxy tunnel established between Endpoints) This is not the solution for the problem, of course. Here I attach some sysout logs with working and non-working flows. 1. working example: after sending initial message sleep the executing thread in "need_wrap" case for little and you get following sequence write: 154 write done: true NEED_UNWRAP read: 2357 read done: true SSL Handshake is OK 2. non working example write: 154 write done: true NEED_UNWRAP read: 1460 << much less bytes to consider read done: true ... Exception Any comments? > - >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: dev-h...@tomcat.apache.org >> >> > written: 147 HTTP/1.1 200 Connection established write: 154 write done: true NEED_UNWRAP read: 2357 read done: true NEED_TASK NEED_UNWRAP NEED_TASK NEED_UNWRAP NEED_TASK NEED_WRAP write: 267 write done: true NEED_WRAP write: 6 write done: true NEED_WRAP write: 41 write done: true NEED_UNWRAP read: 47 read done: true NEED_UNWRAP FINISHED session with id 0 is open pong: clientwritten: 147 HTTP/1.1 200 Connection established write: 154 write done: true NEED_UNWRAP read: 1460 read done: true NEED_TASK NEED_UNWRAP javax.websocket.DeploymentException: The HTTP request to initiate the WebSocket conenction failed at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:390) at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:179) at ws.client.Client.main(Client.java:23) Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLException: TODO at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WrapperFuture.get(AsyncChannelWrapperSecure.java:488) at org.apache.tomcat.websocket.WsWebSocketContainer.connectToServer(WsWebSocketContainer.java:357) ... 2 more Caused by: javax.net.ssl.SSLException: TODO at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.checkResult(AsyncChannelWrapperSecure.java:418) at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$WebSocketSslHandshakeThread.run(AsyncChannelWrapperSecure.java:376) WebSocket session is null - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: WebSockets in Tomcat 8 (proxy support and ssl handshake)
On Fri, May 17, 2013 at 3:35 PM, Mark Thomas wrote: > On 17/05/2013 13:07, Niki Dokovski wrote: > > Hi, > > > > Developing some sample apps with websockets and tomcat 8 revealed two > > issues I’d like to discuss here. > > > > 1. There is no proxy support in the implementation of > > WebSocketContainer.connectToServer calls. While playing around with the > > code it was relatively easy to add this in the implementation with simple > > consideration of http.proxyHost and Port system properties; replacing > > host and port values which were obtained from path method parameter and > > making initial HTTP CONNECT message to the proxy. In this case, do you > > think we can add proxy support here? I was thinking how to provide some > > kind of automation test as well, but no luck so far. > > Personally I dislike the global nature system properties for > configuration unless they are absolutely the only option. > > Since the WebSocket API has no way of setting a proxy at the moment, > system properties might be the only choice :( > > This is an issue you should raise with the WebSocket EG group. I suggest > opening a Jira to request proxy support in a future version. > https://java.net/jira/browse/WEBSOCKET_SPEC Here is the issue: https://java.net/jira/browse/WEBSOCKET_SPEC-202 > > > > > 2. So when I got my proxy tunneling in place (with the CONNECT http > > request) I hit a tougher problem with SSL handshake implementation in > > WebSocketSslHandshakeThread. The SSL handshake was successful while I was > > stepping with the debugger through the code and failed when I run the > > test. It should be some timing issue. This bit needs more debugging. Have > > you seen this before? > > No, but the SSL support for WebSocket clients hasn't been heavily used > yet so there could still be some bugs to iron out. If you haven't > already, take a look at > TestWsWebSocketContainer.testConnectToServerEndpointSSL() > > Mark > > > Thanks. I'll take a look at it. > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
WebSockets in Tomcat 8 (proxy support and ssl handshake)
Hi, Developing some sample apps with websockets and tomcat 8 revealed two issues I’d like to discuss here. 1. There is no proxy support in the implementation of WebSocketContainer.connectToServer calls. While playing around with the code it was relatively easy to add this in the implementation with simple consideration of http.proxyHost and Port system properties; replacing host and port values which were obtained from path method parameter and making initial HTTP CONNECT message to the proxy. In this case, do you think we can add proxy support here? I was thinking how to provide some kind of automation test as well, but no luck so far. 2. So when I got my proxy tunneling in place (with the CONNECT http request) I hit a tougher problem with SSL handshake implementation in WebSocketSslHandshakeThread. The SSL handshake was successful while I was stepping with the debugger through the code and failed when I run the test. It should be some timing issue. This bit needs more debugging. Have you seen this before? Regards Nikolai
Re: websocket client doesn't exit
Hi, I found that when the NIO connector is used the problem is gone. With protocol="org.apache.coyote.http11.Http11NioProtocol" in server.xml the example is working just fine. The wrong content of inputBuffer in WsFrameBase causes the observed behavior and I haven't tracked it further to the default connector in tomcat 8. Best Regards Nikolai On Tue, May 14, 2013 at 1:40 PM, Niki Dokovski wrote: > Hello, > here is a small example based on JSR 356 defined APIs and implementation > in tomcat 8 resulting in non exiting websocket client request. There is > nothing special on both endpoints: On @ServerEndpoint there is simple > OnMessage processing and on client side there is PongMessage handler. The > client opens connection and sends ping message looking for the pong return. > After executing 3,4 times the client (it is standalone client) hangs in > constant executing of WsWebSocketContainer.processResponse .. > > where the buffer from which is read contains the same message of "The > client frame set the reserved bits to [4] which was not supported by this > endpoint" > > This happens on connectToServer call. > > Here I attach the war and client code which reproduce the problem. The JDK > is Orcl JDK1.7_21. > > Could you please take a look at it and comment the observed behavior? > Best regards > Nikolai > > PS. I'm still digging into the implementation... > > >
Fwd: websocket client doesn't exit
-- Forwarded message -- From: Niki Dokovski Date: Tue, May 14, 2013 at 1:40 PM Subject: websocket client doesn't exit To: dev@tomcat.apache.org Hello, here is a small example based on JSR 356 defined APIs and implementation in tomcat 8 resulting in non exiting websocket client request. There is nothing special on both endpoints: On @ServerEndpoint there is simple OnMessage processing and on client side there is PongMessage handler. The client opens connection and sends ping message looking for the pong return. After executing 3,4 times the client (it is standalone client) hangs in constant executing of WsWebSocketContainer.processResponse .. where the buffer from which is read contains the same message of "The client frame set the reserved bits to [4] which was not supported by this endpoint" This happens on connectToServer call. Here I attach the war and client code which reproduce the problem. The JDK is Orcl JDK1.7_21. Could you please take a look at it and comment the observed behavior? Best regards Nikolai PS. I'm still digging into the implementation... client.jar Description: application/java-archive - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
