Re: Why are Tomcat's threads more costly than background threads?
well, that's no more a technical question ;) Romain Manni-Bucau Twitter: @rmannibucau Blog: http://rmannibucau.wordpress.com/ LinkedIn: http://fr.linkedin.com/in/rmannibucau Github: https://github.com/rmannibucau 2012/12/17 Howard W. Smith, Jr. : > Well, I had that when my app was running via GlassFish. I still have that > code ready to use but websockets and tomee getting it all done for me, > quite nicely. :-) > On Dec 17, 2012 2:59 AM, "Romain Manni-Bucau" > wrote: > >> Yes and no, it makes the connection being from the client why is better for >> it and it works - when websockets can have issues with proxies and so on. >> Websocket are nice when you have a real protocol...just create one and ask >> the client to ask for an update with a timer, that's already better and you >> keep the full connected advantages >> Le 17 déc. 2012 08:54, "Howard W. Smith, Jr." a >> écrit : >> >> > Polling is too much network traffic in my opinion. Websocket is instant >> > messaging. No need to add polling code and xhtml here there for different >> > type of messages coming from server and no need of an ajax request to >> > server throughout entire session just to get an update from the server. >> > On Dec 17, 2012 2:47 AM, "Romain Manni-Bucau" >> > wrote: >> > >> > > Side note: if your need is pushing regularly to the client just having >> > some >> > > client polling can make it far easier to manage on both sides... >> > > Le 17 déc. 2012 08:33, "Howard W. Smith, Jr." >> a >> > > écrit : >> > > >> > > > Almost forgot...PrimeFaces peer/friend of mine informed me that JMS >> is >> > > good >> > > > for running two instances of an app, similar to load balancer. >> > > > >> > > > Prior to using TomEE, I really wanted to use PrimeFaces Push (powered >> > by >> > > > Atmosphere/websocket), and at that time, I was thinking that I could >> > > > possibly run Tomcat or TomEE just to push messages to the clients of >> my >> > > app >> > > > running on Glassfish, since Glassfish had issues with PrimeFaces Push >> > > > (Atmosphere). I think he said that tomEE and Glassfish could do that, >> > but >> > > > they would have to communicate via JMS or the two servlets would have >> > to >> > > > serve requests/responses, appropriately, via JMS. >> > > > >> > > > I am probably not stating the above correctly as he said, but that's >> > how >> > > I >> > > > understood what he told me. >> > > > >> > > > Now that I'm only using tomEE, there is no need to do such a thing. >> > > Loving >> > > > TomEE!!! >> > > > >> > > > >> > > > >> > > > On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < >> > > > smithh032...@gmail.com> wrote: >> > > > >> > > > > Very interesting discussion and responses here! Anthony motivates >> me >> > to >> > > > > jump on the bandwagon of using JMS/message-driven-beans, and at the >> > > same >> > > > > time, Romain tells me, not much need (for local) in a Java EE 6 >> > > > > application. :) >> > > > > >> > > > > Either way, i have not ruled out JMS. I have done some strategic >> > coding >> > > > on >> > > > > my part to avoid @Schedule or use of timer service...using Date >> class >> > > to >> > > > > postpone pushing data to google calendar, and only pushing when >> there >> > > is >> > > > a >> > > > > new date to update...application-wide. At first, i was pushing data >> > to >> > > > > google calendar after 'every' update request... sending over 1,000 >> > > > requests >> > > > > to google calendar sometimes or daily (adding and deleting events >> on >> > > > google >> > > > > calendar to keep google calendar in sync with the database), but I >> > cut >> > > > that >> > > > > back (or decreased the # of requests) with the new code I developed >> > 2+ >> > > > > months ago. just makes for a faster user experience since they >> > > requested >> > > > to >> > > > > have data updated via ajax on change/selection of this field, and >> > that >> > > > > field. >> > > > > >> > > > > Right now, JMS would be similar to a nice science project for me... >> > > just >> > > > > to see if I can do it and the success of it. >> > > > > >> > > > > >> > > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau < >> > > > rmannibu...@gmail.com >> > > > > > wrote: >> > > > > >> > > > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it >> on >> > > the >> > > > >> road qince javaee 6, no more. >> > > > >> >> > > > >> It works but i dont find it effective when it stays local >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >>
Re: Why are Tomcat's threads more costly than background threads?
Well, I had that when my app was running via GlassFish. I still have that code ready to use but websockets and tomee getting it all done for me, quite nicely. :-) On Dec 17, 2012 2:59 AM, "Romain Manni-Bucau" wrote: > Yes and no, it makes the connection being from the client why is better for > it and it works - when websockets can have issues with proxies and so on. > Websocket are nice when you have a real protocol...just create one and ask > the client to ask for an update with a timer, that's already better and you > keep the full connected advantages > Le 17 déc. 2012 08:54, "Howard W. Smith, Jr." a > écrit : > > > Polling is too much network traffic in my opinion. Websocket is instant > > messaging. No need to add polling code and xhtml here there for different > > type of messages coming from server and no need of an ajax request to > > server throughout entire session just to get an update from the server. > > On Dec 17, 2012 2:47 AM, "Romain Manni-Bucau" > > wrote: > > > > > Side note: if your need is pushing regularly to the client just having > > some > > > client polling can make it far easier to manage on both sides... > > > Le 17 déc. 2012 08:33, "Howard W. Smith, Jr." > a > > > écrit : > > > > > > > Almost forgot...PrimeFaces peer/friend of mine informed me that JMS > is > > > good > > > > for running two instances of an app, similar to load balancer. > > > > > > > > Prior to using TomEE, I really wanted to use PrimeFaces Push (powered > > by > > > > Atmosphere/websocket), and at that time, I was thinking that I could > > > > possibly run Tomcat or TomEE just to push messages to the clients of > my > > > app > > > > running on Glassfish, since Glassfish had issues with PrimeFaces Push > > > > (Atmosphere). I think he said that tomEE and Glassfish could do that, > > but > > > > they would have to communicate via JMS or the two servlets would have > > to > > > > serve requests/responses, appropriately, via JMS. > > > > > > > > I am probably not stating the above correctly as he said, but that's > > how > > > I > > > > understood what he told me. > > > > > > > > Now that I'm only using tomEE, there is no need to do such a thing. > > > Loving > > > > TomEE!!! > > > > > > > > > > > > > > > > On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < > > > > smithh032...@gmail.com> wrote: > > > > > > > > > Very interesting discussion and responses here! Anthony motivates > me > > to > > > > > jump on the bandwagon of using JMS/message-driven-beans, and at the > > > same > > > > > time, Romain tells me, not much need (for local) in a Java EE 6 > > > > > application. :) > > > > > > > > > > Either way, i have not ruled out JMS. I have done some strategic > > coding > > > > on > > > > > my part to avoid @Schedule or use of timer service...using Date > class > > > to > > > > > postpone pushing data to google calendar, and only pushing when > there > > > is > > > > a > > > > > new date to update...application-wide. At first, i was pushing data > > to > > > > > google calendar after 'every' update request... sending over 1,000 > > > > requests > > > > > to google calendar sometimes or daily (adding and deleting events > on > > > > google > > > > > calendar to keep google calendar in sync with the database), but I > > cut > > > > that > > > > > back (or decreased the # of requests) with the new code I developed > > 2+ > > > > > months ago. just makes for a faster user experience since they > > > requested > > > > to > > > > > have data updated via ajax on change/selection of this field, and > > that > > > > > field. > > > > > > > > > > Right now, JMS would be similar to a nice science project for me... > > > just > > > > > to see if I can do it and the success of it. > > > > > > > > > > > > > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau < > > > > rmannibu...@gmail.com > > > > > > wrote: > > > > > > > > > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it > on > > > the > > > > >> road qince javaee 6, no more. > > > > >> > > > > >> It works but i dont find it effective when it stays local > > > > >> > > > > > > > > > > > > > > > > > > > >
Re: Why are Tomcat's threads more costly than background threads?
Yes and no, it makes the connection being from the client why is better for it and it works - when websockets can have issues with proxies and so on. Websocket are nice when you have a real protocol...just create one and ask the client to ask for an update with a timer, that's already better and you keep the full connected advantages Le 17 déc. 2012 08:54, "Howard W. Smith, Jr." a écrit : > Polling is too much network traffic in my opinion. Websocket is instant > messaging. No need to add polling code and xhtml here there for different > type of messages coming from server and no need of an ajax request to > server throughout entire session just to get an update from the server. > On Dec 17, 2012 2:47 AM, "Romain Manni-Bucau" > wrote: > > > Side note: if your need is pushing regularly to the client just having > some > > client polling can make it far easier to manage on both sides... > > Le 17 déc. 2012 08:33, "Howard W. Smith, Jr." a > > écrit : > > > > > Almost forgot...PrimeFaces peer/friend of mine informed me that JMS is > > good > > > for running two instances of an app, similar to load balancer. > > > > > > Prior to using TomEE, I really wanted to use PrimeFaces Push (powered > by > > > Atmosphere/websocket), and at that time, I was thinking that I could > > > possibly run Tomcat or TomEE just to push messages to the clients of my > > app > > > running on Glassfish, since Glassfish had issues with PrimeFaces Push > > > (Atmosphere). I think he said that tomEE and Glassfish could do that, > but > > > they would have to communicate via JMS or the two servlets would have > to > > > serve requests/responses, appropriately, via JMS. > > > > > > I am probably not stating the above correctly as he said, but that's > how > > I > > > understood what he told me. > > > > > > Now that I'm only using tomEE, there is no need to do such a thing. > > Loving > > > TomEE!!! > > > > > > > > > > > > On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < > > > smithh032...@gmail.com> wrote: > > > > > > > Very interesting discussion and responses here! Anthony motivates me > to > > > > jump on the bandwagon of using JMS/message-driven-beans, and at the > > same > > > > time, Romain tells me, not much need (for local) in a Java EE 6 > > > > application. :) > > > > > > > > Either way, i have not ruled out JMS. I have done some strategic > coding > > > on > > > > my part to avoid @Schedule or use of timer service...using Date class > > to > > > > postpone pushing data to google calendar, and only pushing when there > > is > > > a > > > > new date to update...application-wide. At first, i was pushing data > to > > > > google calendar after 'every' update request... sending over 1,000 > > > requests > > > > to google calendar sometimes or daily (adding and deleting events on > > > google > > > > calendar to keep google calendar in sync with the database), but I > cut > > > that > > > > back (or decreased the # of requests) with the new code I developed > 2+ > > > > months ago. just makes for a faster user experience since they > > requested > > > to > > > > have data updated via ajax on change/selection of this field, and > that > > > > field. > > > > > > > > Right now, JMS would be similar to a nice science project for me... > > just > > > > to see if I can do it and the success of it. > > > > > > > > > > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau < > > > rmannibu...@gmail.com > > > > > wrote: > > > > > > > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on > > the > > > >> road qince javaee 6, no more. > > > >> > > > >> It works but i dont find it effective when it stays local > > > >> > > > > > > > > > > > > > >
Re: Why are Tomcat's threads more costly than background threads?
Polling is too much network traffic in my opinion. Websocket is instant messaging. No need to add polling code and xhtml here there for different type of messages coming from server and no need of an ajax request to server throughout entire session just to get an update from the server. On Dec 17, 2012 2:47 AM, "Romain Manni-Bucau" wrote: > Side note: if your need is pushing regularly to the client just having some > client polling can make it far easier to manage on both sides... > Le 17 déc. 2012 08:33, "Howard W. Smith, Jr." a > écrit : > > > Almost forgot...PrimeFaces peer/friend of mine informed me that JMS is > good > > for running two instances of an app, similar to load balancer. > > > > Prior to using TomEE, I really wanted to use PrimeFaces Push (powered by > > Atmosphere/websocket), and at that time, I was thinking that I could > > possibly run Tomcat or TomEE just to push messages to the clients of my > app > > running on Glassfish, since Glassfish had issues with PrimeFaces Push > > (Atmosphere). I think he said that tomEE and Glassfish could do that, but > > they would have to communicate via JMS or the two servlets would have to > > serve requests/responses, appropriately, via JMS. > > > > I am probably not stating the above correctly as he said, but that's how > I > > understood what he told me. > > > > Now that I'm only using tomEE, there is no need to do such a thing. > Loving > > TomEE!!! > > > > > > > > On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < > > smithh032...@gmail.com> wrote: > > > > > Very interesting discussion and responses here! Anthony motivates me to > > > jump on the bandwagon of using JMS/message-driven-beans, and at the > same > > > time, Romain tells me, not much need (for local) in a Java EE 6 > > > application. :) > > > > > > Either way, i have not ruled out JMS. I have done some strategic coding > > on > > > my part to avoid @Schedule or use of timer service...using Date class > to > > > postpone pushing data to google calendar, and only pushing when there > is > > a > > > new date to update...application-wide. At first, i was pushing data to > > > google calendar after 'every' update request... sending over 1,000 > > requests > > > to google calendar sometimes or daily (adding and deleting events on > > google > > > calendar to keep google calendar in sync with the database), but I cut > > that > > > back (or decreased the # of requests) with the new code I developed 2+ > > > months ago. just makes for a faster user experience since they > requested > > to > > > have data updated via ajax on change/selection of this field, and that > > > field. > > > > > > Right now, JMS would be similar to a nice science project for me... > just > > > to see if I can do it and the success of it. > > > > > > > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau < > > rmannibu...@gmail.com > > > > wrote: > > > > > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on > the > > >> road qince javaee 6, no more. > > >> > > >> It works but i dont find it effective when it stays local > > >> > > > > > > > > >
Re: Why are Tomcat's threads more costly than background threads?
Side note: if your need is pushing regularly to the client just having some client polling can make it far easier to manage on both sides... Le 17 déc. 2012 08:33, "Howard W. Smith, Jr." a écrit : > Almost forgot...PrimeFaces peer/friend of mine informed me that JMS is good > for running two instances of an app, similar to load balancer. > > Prior to using TomEE, I really wanted to use PrimeFaces Push (powered by > Atmosphere/websocket), and at that time, I was thinking that I could > possibly run Tomcat or TomEE just to push messages to the clients of my app > running on Glassfish, since Glassfish had issues with PrimeFaces Push > (Atmosphere). I think he said that tomEE and Glassfish could do that, but > they would have to communicate via JMS or the two servlets would have to > serve requests/responses, appropriately, via JMS. > > I am probably not stating the above correctly as he said, but that's how I > understood what he told me. > > Now that I'm only using tomEE, there is no need to do such a thing. Loving > TomEE!!! > > > > On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > > > Very interesting discussion and responses here! Anthony motivates me to > > jump on the bandwagon of using JMS/message-driven-beans, and at the same > > time, Romain tells me, not much need (for local) in a Java EE 6 > > application. :) > > > > Either way, i have not ruled out JMS. I have done some strategic coding > on > > my part to avoid @Schedule or use of timer service...using Date class to > > postpone pushing data to google calendar, and only pushing when there is > a > > new date to update...application-wide. At first, i was pushing data to > > google calendar after 'every' update request... sending over 1,000 > requests > > to google calendar sometimes or daily (adding and deleting events on > google > > calendar to keep google calendar in sync with the database), but I cut > that > > back (or decreased the # of requests) with the new code I developed 2+ > > months ago. just makes for a faster user experience since they requested > to > > have data updated via ajax on change/selection of this field, and that > > field. > > > > Right now, JMS would be similar to a nice science project for me... just > > to see if I can do it and the success of it. > > > > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau < > rmannibu...@gmail.com > > > wrote: > > > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on the > >> road qince javaee 6, no more. > >> > >> It works but i dont find it effective when it stays local > >> > > > > >
Re: Why are Tomcat's threads more costly than background threads?
Almost forgot...PrimeFaces peer/friend of mine informed me that JMS is good for running two instances of an app, similar to load balancer. Prior to using TomEE, I really wanted to use PrimeFaces Push (powered by Atmosphere/websocket), and at that time, I was thinking that I could possibly run Tomcat or TomEE just to push messages to the clients of my app running on Glassfish, since Glassfish had issues with PrimeFaces Push (Atmosphere). I think he said that tomEE and Glassfish could do that, but they would have to communicate via JMS or the two servlets would have to serve requests/responses, appropriately, via JMS. I am probably not stating the above correctly as he said, but that's how I understood what he told me. Now that I'm only using tomEE, there is no need to do such a thing. Loving TomEE!!! On Mon, Dec 17, 2012 at 2:26 AM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > Very interesting discussion and responses here! Anthony motivates me to > jump on the bandwagon of using JMS/message-driven-beans, and at the same > time, Romain tells me, not much need (for local) in a Java EE 6 > application. :) > > Either way, i have not ruled out JMS. I have done some strategic coding on > my part to avoid @Schedule or use of timer service...using Date class to > postpone pushing data to google calendar, and only pushing when there is a > new date to update...application-wide. At first, i was pushing data to > google calendar after 'every' update request... sending over 1,000 requests > to google calendar sometimes or daily (adding and deleting events on google > calendar to keep google calendar in sync with the database), but I cut that > back (or decreased the # of requests) with the new code I developed 2+ > months ago. just makes for a faster user experience since they requested to > have data updated via ajax on change/selection of this field, and that > field. > > Right now, JMS would be similar to a nice science project for me... just > to see if I can do it and the success of it. > > > On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau > wrote: > >> I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on the >> road qince javaee 6, no more. >> >> It works but i dont find it effective when it stays local >> > >
Re: Why are Tomcat's threads more costly than background threads?
Very interesting discussion and responses here! Anthony motivates me to jump on the bandwagon of using JMS/message-driven-beans, and at the same time, Romain tells me, not much need (for local) in a Java EE 6 application. :) Either way, i have not ruled out JMS. I have done some strategic coding on my part to avoid @Schedule or use of timer service...using Date class to postpone pushing data to google calendar, and only pushing when there is a new date to update...application-wide. At first, i was pushing data to google calendar after 'every' update request... sending over 1,000 requests to google calendar sometimes or daily (adding and deleting events on google calendar to keep google calendar in sync with the database), but I cut that back (or decreased the # of requests) with the new code I developed 2+ months ago. just makes for a faster user experience since they requested to have data updated via ajax on change/selection of this field, and that field. Right now, JMS would be similar to a nice science project for me... just to see if I can do it and the success of it. On Mon, Dec 17, 2012 at 1:57 AM, Romain Manni-Bucau wrote: > I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on the > road qince javaee 6, no more. > > It works but i dont find it effective when it stays local >
Re: Why are Tomcat's threads more costly than background threads?
I used it a lot in javaee 5 (openejb 3) but i'm happy to leave it on the road qince javaee 6, no more. It works but i dont find it effective when it stays local @others: am i alone? Le 17 déc. 2012 06:53, "Anthony Fryer" a écrit : > Yes i've done this before a few times and had really good results using > weblogic. I must say i have an element of doubt about it with Romain not > liking it though. I've used ActiveMQ before and not encountered issues but > these guys are the tomee devs and I'd go with what they say. > > > > -- > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659718.html > Sent from the OpenEJB User mailing list archive at Nabble.com. >
Re: Why are Tomcat's threads more costly than background threads?
Yes i've done this before a few times and had really good results using weblogic. I must say i have an element of doubt about it with Romain not liking it though. I've used ActiveMQ before and not encountered issues but these guys are the tomee devs and I'd go with what they say. -- View this message in context: http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659718.html Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: Why are Tomcat's threads more costly than background threads?
Ok so what i'm hearing is that you're against the idea because you don't trust the implementation of JMS included in tomee? -- View this message in context: http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659713.html Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: Why are Tomcat's threads more costly than background threads?
well i don't like it because for asynchronouism you now have some tools in JavaEE. then i agree the retry mecanism can be interesting but 1) did you already look what does the container (AMQ here) when you send/receive a message? i wouldn't try to get any perf with such a lot of job 2) AMQ blocks easily and it is as easy to get twice the same message or lost it so i wouldn't bet on it don't take it bad, AMQ is fine in general but got so much issues that i wouldnt use it when i stay in the same JVM @Asynchronous should be interesting that's said when a server is really used async is useless (work has to be done :p) Romain Manni-Bucau Twitter: @rmannibucau Blog: http://rmannibucau.wordpress.com/ LinkedIn: http://fr.linkedin.com/in/rmannibucau Github: https://github.com/rmannibucau 2012/12/17 Howard W. Smith, Jr. : > Good points Anthony and thanks for sharing all that. Stephen, right now, I > assume the same as what you stated. My experience with entirely @stateless > solution, IMHO, got me nowhere because @stateless seems to be application > scope and the database got locked application scope/wide, so that is why I > reverted to user session being responsible for the update. JMS seems a bit > @stateless in its nature to me. I honestly believe that entirely @stateless > can perform narrower scope transactions. Maybe one day I can achieve such a > thing. :-) > On Dec 16, 2012 7:07 PM, "Anthony Fryer" wrote: > >> Romain, you're opinion means alot so i'm wondering why you are against JMS >> in >> this case. This kind of pattern happens alot in my experience. In this >> case a timer job executes that finds a list of jobs to execute, in this >> case >> read a list of emails and for each email insert a row into a database. >> >> In my experience, the timer job that executes wants to finish as quickly as >> possible so you don't get overlapping timer executions and also to minimize >> resource utilization. Putting a jms message on a queue is generally alot >> quicker than performing an actual database insert. In addition JMS gives >> you guaranteed message delivery, so you know that the message is going to >> get processed and not get lost. It is part of the JMS spec that i believe >> is very applicable in this case. >> >> As for the transactional semantics that apply to the Message Driven Bean >> that performs the actual work of inserting the record into the database, >> again i think it works well in this case. If the MDB is executing within a >> container managed transaction, then only if the database insert succeeds >> and >> the transaction commits will the message be taken off the queue. This >> means >> if the database insert fails, you haven't lost the message and can either >> retry or configure the JMS to place the message on an error queue that can >> be processed in a different way. >> >> How would you handle a database insert failure in the current solution? >> >> Also I don't think it matters if the MDB that performs the message >> processing is local or not. Its irrelevant really. I wouldn't be using >> the >> JMS just for asynchronous calls. I'd be using it for the guarenteed >> message >> delivery and processing, performance AND asynchronous processing. >> >> >> >> >> >> >> -- >> View this message in context: >> http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659710.html >> Sent from the OpenEJB User mailing list archive at Nabble.com. >>
Re: Why are Tomcat's threads more costly than background threads?
Good points Anthony and thanks for sharing all that. Stephen, right now, I assume the same as what you stated. My experience with entirely @stateless solution, IMHO, got me nowhere because @stateless seems to be application scope and the database got locked application scope/wide, so that is why I reverted to user session being responsible for the update. JMS seems a bit @stateless in its nature to me. I honestly believe that entirely @stateless can perform narrower scope transactions. Maybe one day I can achieve such a thing. :-) On Dec 16, 2012 7:07 PM, "Anthony Fryer" wrote: > Romain, you're opinion means alot so i'm wondering why you are against JMS > in > this case. This kind of pattern happens alot in my experience. In this > case a timer job executes that finds a list of jobs to execute, in this > case > read a list of emails and for each email insert a row into a database. > > In my experience, the timer job that executes wants to finish as quickly as > possible so you don't get overlapping timer executions and also to minimize > resource utilization. Putting a jms message on a queue is generally alot > quicker than performing an actual database insert. In addition JMS gives > you guaranteed message delivery, so you know that the message is going to > get processed and not get lost. It is part of the JMS spec that i believe > is very applicable in this case. > > As for the transactional semantics that apply to the Message Driven Bean > that performs the actual work of inserting the record into the database, > again i think it works well in this case. If the MDB is executing within a > container managed transaction, then only if the database insert succeeds > and > the transaction commits will the message be taken off the queue. This > means > if the database insert fails, you haven't lost the message and can either > retry or configure the JMS to place the message on an error queue that can > be processed in a different way. > > How would you handle a database insert failure in the current solution? > > Also I don't think it matters if the MDB that performs the message > processing is local or not. Its irrelevant really. I wouldn't be using > the > JMS just for asynchronous calls. I'd be using it for the guarenteed > message > delivery and processing, performance AND asynchronous processing. > > > > > > > -- > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659710.html > Sent from the OpenEJB User mailing list archive at Nabble.com. >
Re: Why are Tomcat's threads more costly than background threads?
Romain, you're opinion means alot so i'm wondering why you are against JMS in this case. This kind of pattern happens alot in my experience. In this case a timer job executes that finds a list of jobs to execute, in this case read a list of emails and for each email insert a row into a database. In my experience, the timer job that executes wants to finish as quickly as possible so you don't get overlapping timer executions and also to minimize resource utilization. Putting a jms message on a queue is generally alot quicker than performing an actual database insert. In addition JMS gives you guaranteed message delivery, so you know that the message is going to get processed and not get lost. It is part of the JMS spec that i believe is very applicable in this case. As for the transactional semantics that apply to the Message Driven Bean that performs the actual work of inserting the record into the database, again i think it works well in this case. If the MDB is executing within a container managed transaction, then only if the database insert succeeds and the transaction commits will the message be taken off the queue. This means if the database insert fails, you haven't lost the message and can either retry or configure the JMS to place the message on an error queue that can be processed in a different way. How would you handle a database insert failure in the current solution? Also I don't think it matters if the MDB that performs the message processing is local or not. Its irrelevant really. I wouldn't be using the JMS just for asynchronous calls. I'd be using it for the guarenteed message delivery and processing, performance AND asynchronous processing. -- View this message in context: http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659710.html Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: Why are Tomcat's threads more costly than background threads?
Need to take care around transaction symmantics with JMS too. On Sunday, 16 December 2012, Anthony Fryer wrote: > Another change you could make to improve the performance of the method > invoked by the timer, is instead of performing the actual database insert > for each email in that method, instead write a message to a JMS queue. > Then > create a message driven bean that processes the JMS messages and performs > the actual database insert. > > > > -- > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659697.html > Sent from the OpenEJB User mailing list archive at Nabble.com. >
Re: Why are Tomcat's threads more costly than background threads?
I have never used JMS/message-driven-beans as of yet, but considering it... On Sun, Dec 16, 2012 at 8:35 AM, Romain Manni-Bucau wrote: > i'm quite again using JMS for local asynchronism generally > > not the goal of JMS IMO > > Romain Manni-Bucau > Twitter: @rmannibucau > Blog: http://rmannibucau.wordpress.com/ > LinkedIn: http://fr.linkedin.com/in/rmannibucau > Github: https://github.com/rmannibucau > > > > 2012/12/16 Anthony Fryer : > > Another change you could make to improve the performance of the method > > invoked by the timer, is instead of performing the actual database insert > > for each email in that method, instead write a message to a JMS queue. > Then > > create a message driven bean that processes the JMS messages and performs > > the actual database insert. > > > > > > > > -- > > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659697.html > > Sent from the OpenEJB User mailing list archive at Nabble.com. >
Re: Why are Tomcat's threads more costly than background threads?
Anthony, thanks for the recommendation. I will have to give that a try. On Dec 16, 2012 8:33 AM, "Anthony Fryer" wrote: > Another change you could make to improve the performance of the method > invoked by the timer, is instead of performing the actual database insert > for each email in that method, instead write a message to a JMS queue. > Then > create a message driven bean that processes the JMS messages and performs > the actual database insert. > > > > -- > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659697.html > Sent from the OpenEJB User mailing list archive at Nabble.com. >
Re: Why are Tomcat's threads more costly than background threads?
i'm quite again using JMS for local asynchronism generally not the goal of JMS IMO Romain Manni-Bucau Twitter: @rmannibucau Blog: http://rmannibucau.wordpress.com/ LinkedIn: http://fr.linkedin.com/in/rmannibucau Github: https://github.com/rmannibucau 2012/12/16 Anthony Fryer : > Another change you could make to improve the performance of the method > invoked by the timer, is instead of performing the actual database insert > for each email in that method, instead write a message to a JMS queue. Then > create a message driven bean that processes the JMS messages and performs > the actual database insert. > > > > -- > View this message in context: > http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659697.html > Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: Why are Tomcat's threads more costly than background threads?
Another change you could make to improve the performance of the method invoked by the timer, is instead of performing the actual database insert for each email in that method, instead write a message to a JMS queue. Then create a message driven bean that processes the JMS messages and performs the actual database insert. -- View this message in context: http://openejb.979440.n4.nabble.com/Re-Why-are-Tomcat-s-threads-more-costly-than-background-threads-tp4659429p4659697.html Sent from the OpenEJB User mailing list archive at Nabble.com.
Re: Why are Tomcat's threads more costly than background threads?
Interesting response and lesson learned. Thanks David. On Sat, Dec 15, 2012 at 5:45 PM, David Blevins wrote: > Looks like you got it working, which is fantastic! > > Just as an FYI to make sure no false lessons are learned, the > @javax.ejb.Lock annotation only affects access to the @Singleton. Under > the covers the @Lock is really just syntactic sugar for a > ReentrantReadWriteLock[1]. The container creates one for every @Singleton > class and only the container can see it or use it. Before letting a call > reach the Singleton, the container effectively invokes: > > theSingletonInstancesLock.readLock().tryLock(accessTimeout.time(), > accessTimeout.unit()) > > Where `theSingletonInstancesLock` is a simple private final variable held > by a private scoped class (to really ensure no one else can use it). The > @Lock(WRITE) is a synonym for `writeLock()` and `@Lock(READ)` is a synonym > for `readLock()`. The accessTimeout.time()/unit() are supplied via a > @javax.ejb.AccessTimeout annotation on the method or bean class. > > In short, not at all related to Derby's locking. If there were issues, it > mgiht have been correlated but definitely not caused by it -- not in a > direct sense. > > -David > > [1] > http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html >
Re: Why are Tomcat's threads more costly than background threads?
On Dec 14, 2012, at 2:30 PM, "Howard W. Smith, Jr." wrote: > I will let you know, if I use Singleton and @Lock(WRITE). On the current > database (Apache Derby), the @Lock(WRITE) always results in a 'locked' > exception which tells me that @Lock(WRITE) does not work well with the > Apache Derby database. > > It would be nice to know why all you recommended to me...does not work with > Apache Derby; Looks like you got it working, which is fantastic! Just as an FYI to make sure no false lessons are learned, the @javax.ejb.Lock annotation only affects access to the @Singleton. Under the covers the @Lock is really just syntactic sugar for a ReentrantReadWriteLock[1]. The container creates one for every @Singleton class and only the container can see it or use it. Before letting a call reach the Singleton, the container effectively invokes: theSingletonInstancesLock.readLock().tryLock(accessTimeout.time(), accessTimeout.unit()) Where `theSingletonInstancesLock` is a simple private final variable held by a private scoped class (to really ensure no one else can use it). The @Lock(WRITE) is a synonym for `writeLock()` and `@Lock(READ)` is a synonym for `readLock()`. The accessTimeout.time()/unit() are supplied via a @javax.ejb.AccessTimeout annotation on the method or bean class. In short, not at all related to Derby's locking. If there were issues, it mgiht have been correlated but definitely not caused by it -- not in a direct sense. -David [1] http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html
Re: Why are Tomcat's threads more costly than background threads?
David, just wanted to let you know that I did what you recommended below, and it's working great (no errors; will continue to monitor server log). I had to make a few other code changes to this piece of software, and did some testing as well. Feel free to see my latest test results... once I have the data from each email in memory/POJO, then it is updating data at an average of 4 to 6 seconds (2 airport shuttles per email). https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13532966 Thanks, Howard On Fri, Dec 14, 2012 at 3:47 PM, David Blevins wrote: > Now that things are working, I suggest converting your @Schedule bean from > @Stateless to @Singleton for the reasons mentioned before. If you someday > (knock on wood) get totally nailed with a ton of emails/orders, the > @Singleton with @Lock(WRITE) will prevent any situations where your timers > start to overlap due to backlog and you accidentally process emails twice > or more.
Re: Why are Tomcat's threads more costly than background threads?
Thanks David. Before you introduced me to the @Lock(WRITE), I already came up with a Boolean variable that is updated on @ApplicationScoped bean, every time @schedule is triggered 'and' whenever a user clicks the commandLink to check/get emails. The Boolean variable is checked whenever @schedule or enduser is checking emails...if @schedule or enduser is 'already' checking emails, then 'return' from the method. The @Schedule is 'not' responsible for retrieving/downloading the data from the email server. The enduser is 100% responsible for downloading the data from the email server...via the commandLink. The @schedule is only used for providing a notification to endusers...that airport shuttle requests are available to download. I will let you know, if I use Singleton and @Lock(WRITE). On the current database (Apache Derby), the @Lock(WRITE) always results in a 'locked' exception which tells me that @Lock(WRITE) does not work well with the Apache Derby database. It would be nice to know why all you recommended to me...does not work with Apache Derby; I did read that row-level or table-level locking is (or can be) enabled/disabled throughout the database. All I know...i have 'no' issues updating the database when enduser initiates database updates via managed-bean-to-stateless-ejb approach. since I've spent too much time on this and since I have something working-as-designed, I'm going to leave this alone for now and move on to the next task...which is very similar to this task...another set of emails to retrieve from the email server, but a different set of data. A lot of my code can be reused, but still need to write separate software for those emails. Looking forward to it...i'm loving me some Java!!! It will be interesting, when i have time/bandwidth to migrate to another database...as others, here, have suggested to me. :) On Fri, Dec 14, 2012 at 3:47 PM, David Blevins wrote: > Now that things are working, I suggest converting your @Schedule bean from > @Stateless to @Singleton for the reasons mentioned before. If you someday > (knock on wood) get totally nailed with a ton of emails/orders, the > @Singleton with @Lock(WRITE) will prevent any situations where your timers > start to overlap due to backlog and you accidentally process emails twice > or more.
Re: Why are Tomcat's threads more costly than background threads?
On Dec 14, 2012, at 9:38 AM, "Howard W. Smith, Jr." wrote: > David (and any/all interested), > > Please click URL below to see my latest test results (along with code > changes). It took about 12 seconds to perform the update on the Windows > Server 2003 32bit 4GB RAM. > > https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13532480 > > > I know, I know, I need to consider moving away from Apache Derby database > to a 'real database', but I hope to do that, ASAP. > > Thanks for all the responses/recommendations/etc... Absolutely amazing. This is fantastic!! Now that things are working, I suggest converting your @Schedule bean from @Stateless to @Singleton for the reasons mentioned before. If you someday (knock on wood) get totally nailed with a ton of emails/orders, the @Singleton with @Lock(WRITE) will prevent any situations where your timers start to overlap due to backlog and you accidentally process emails twice or more. -David
Re: Why are Tomcat's threads more costly than background threads?
David (and any/all interested), Please click URL below to see my latest test results (along with code changes). It took about 12 seconds to perform the update on the Windows Server 2003 32bit 4GB RAM. https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13532480 I know, I know, I need to consider moving away from Apache Derby database to a 'real database', but I hope to do that, ASAP. Thanks for all the responses/recommendations/etc... Howard On Thu, Dec 13, 2012 at 11:09 PM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > David, I corrected that as you stated, and I made sure the code was as you > recommended, and I've updated the JIRA with test results. Windows Server > 2008 64bit is still fast, Windows SErver 2003 32bit is 'still' slow and > locking the database.
Re: Why are Tomcat's threads more costly than background threads?
David, I corrected that as you stated, and I made sure the code was as you recommended, and I've updated the JIRA with test results. Windows Server 2008 64bit is still fast, Windows SErver 2003 32bit is 'still' slow and locking the database. On Thu, Dec 13, 2012 at 7:02 PM, David Blevins wrote: > My bad, should be @TransactionAttribute(NOT_SUPPORTED) -- even hard for > me to remember the right names :) > > In case you're curious, reading for another day: > > http://tomee.apache.org/transaction-annotations.html >
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 2:41 PM, "Howard W. Smith, Jr." wrote: > Question... > > My last/latest implementation/version of AirportShuttleProcessingBean is a > @Stateless EJB that calls many other @Stateless EJB (DAO). Should I keep > PersistentContext in AirportShuttleProcessingBean, or can > AirportShuttleProcessingBean call any/all @Stateless EJB (DAO) for shortest > transaction possiblel? I hear that a lot here in Apache/TomEE/Tomcat > land... shortest transactions are best. The rule of thumb is that you want the transaction to be as short as possible while also not jeopardizing data integrity/consistency. As a small example, say you had to process an order from a customer that didn't yet exist. Ideally, you would add the new customer information in one transaction, then add the order in another transaction. The scope starts to become "too short", when we get into say processing the order itself with several small transactions. You wouldn't want to insert a new bare "Order" as one transaction, then add each "Item" in an order as separate transactions. The result of a failure to complete is a half processed order which is clearly a non-recoverable situation. One somewhat unrelated note, I did notice several flush() commands in your code. All data is automatically flushed at the end of the transaction, so there should be no need for this can and in fact calling it excessively can cause performance issues. It can mean the difference between one trip to the database and one hundred trips to the database. Having said that, *do not* remove them now. Wait till we are clearly at a state where we have successfully split the transactions and things work. Once we reach that milestone and all works and code is checked in, then give the "remove excess flush() calls" optimization a try. -David
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 3:57 PM, "Howard W. Smith, Jr." wrote: > David, I did as you recommended. I got an exception related to the ejbTimer > and JTA. please see my latest 2 post on the JIRA. My bad, should be @TransactionAttribute(NOT_SUPPORTED) -- even hard for me to remember the right names :) In case you're curious, reading for another day: http://tomee.apache.org/transaction-annotations.html -David
Re: Why are Tomcat's threads more costly than background threads?
David, I did as you recommended. I got an exception related to the ejbTimer and JTA. please see my latest 2 post on the JIRA. https://issues.apache.org/jira/browse/OPENEJB-1968 Need to leave my desk for little bit. Thanks. On Thu, Dec 13, 2012 at 4:06 PM, David Blevins wrote: > If you're willing to go back to the original version of the code and do > *only* the suggestions given, I think we can get this easily sorted out > today.
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 2:01 PM, "Howard W. Smith, Jr." wrote: > Thanks for the response. Guess what... the 2 beans along with the names and > all... is exactly what I did 'yesterday', the only thing I did 'not' do was > mark each of the beans according to what you mentioned. Right, the annotations are a critical part. Splitting the code and not changing the TransactionAttribute annotations as noted would essentially still result in one big transaction (i.e. no change). Educational point; when no @TransactionAttribute is specified on an EJB, it defaults to @TransactionAttribute(REQUIRED) REQUIRED basically means "join the existing transaction if possible, otherwise create one". So EmailSessionBean would create a transaction in the @Schedule method and every method invocation on AirportShuttleRequestProcessorBean would just join the existing transaction. The result: - one big transaction that locks all tables and doesn't let them go till getEmails completes - if email 1 from Joe fails, all email processing will fail and rollback when getEmail completes So aside from a performance problem, there's a very real business logic problem. Splitting them *and* adjusting the transaction settings will fix the business problem and likely help with the performance problem. The big difference of 2 seconds on 64-bit vs several minutes on a 32-bit is alarming and very likely a system issue of some kind. That said, going down this path of shrinking transaction scope will help zero in on where the problem occurs. -David
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 1:53 PM, David Blevins wrote: > > On Dec 13, 2012, at 1:17 PM, "Howard W. Smith, Jr." > wrote: > >> David, >> >> 1. definitely not using @Asynchronous; that was no good at all >> >> 2. Actually, it was 2 seconds on the Windows Server 2008 64-bit 16GB RAM... >> always >> >> 3. Always 5 to 10 minutes to process 1 or 2 emails on Windows Server 2003 >> 32-bit 4GB RAM > > Good info thanks. > >> 4. As noted on the JIRA, I tried the TransactionAttribute NEVER and >> REQUIRES_NEW on @Stateless EJB that was all invoked via @Schedule, and same >> test results, which are The EJB timer rolls back the transaction, and >> every database operation that the EmailStatelessBean completed...prior to >> @Schedule rolling back the transaction. > > Right, that suggestion was never implemented as it was intended. Changing > the TransactionAttribute alone will do nothing. What we need is to 1) > prevent a transaction from being started and 2) creating a new and smaller > transaction scope through a small code rework. > > The ApplicationScoped check backfired because it has the effect of locking > tables for much longer. We want to process each "email" in an individual > transaction and move away from one big transaction. This is really the most > critical step to take, no other steps or workarounds matter at this phase of > the problem. > > Let's try this: > > - Go back to the very original 5-10 minutes 32-bit code > > - Split EmailSessionBean into two beans > > - EmailSessionBean > - will be marked @TransactionAttribute(NEVER) > - will no longer have an @PersistenceContext EntityManager reference > - AirportShuttleRequestProcessorBean > - will be marked @TransactionAttribute(REQUIRES_NEW) > - will have only one public method, `createOrderForAirportShuttle` > - will do all persistence processing > > The EmailSessionBean will have an @EJB reference to the new > AirportShuttleRequestProcessorBean. Effectively, we're splitting the code > that does the contacting the of the email server from the code that contacts > the database and we're creating several much smaller transactions. And one more tweak. Let's change the EmailSessionBean to type @Singleton and add @Lock(WRITE) to our `getEmails()` method. We're doing this to ensure that two `getEmail()` invocations cannot possibly happen at the same time. This is not necessarily the case now, but this can easily occur if the `getEmails()` method takes longer than the configured time of every 10 minutes. It can also occur if the time gets shorted, to say 2 minutes. With switching our EmailSessionBean bean to type @Singleton and ensuring @Lock(WRITE) on the `getEmails()` method, once things are working well we could effectively dramatically reduce the delay between firings to something far more aggressive than 10 minutes (say 2 or 5 minutes) and still be safe. The combination of @Singleton/@Lock(WRITE) guarantees that method will only ever be active one-at-a-time in the entire application. -David
Re: Why are Tomcat's threads more costly than background threads?
David, Thanks for the response. Guess what... the 2 beans along with the names and all... is exactly what I did 'yesterday', the only thing I did 'not' do was mark each of the beans according to what you mentioned. I will try that, ASAP might not be able to report about it until a little later. I want the @Schedule approach along with a user-initiated approach. I'm coding the user initiated approach right now. The original approach will definitely serve for background processing. Thanks, Howard On Thu, Dec 13, 2012 at 4:53 PM, David Blevins wrote: > Let's try this: > > - Go back to the very original 5-10 minutes 32-bit code > > - Split EmailSessionBean into two beans > > - EmailSessionBean > - will be marked @TransactionAttribute(NEVER) > - will no longer have an @PersistenceContext EntityManager > reference > - AirportShuttleRequestProcessorBean > - will be marked @TransactionAttribute(REQUIRES_NEW) > - will have only one public method, `createOrderForAirportShuttle` > - will do all persistence processing > > The EmailSessionBean will have an @EJB reference to the new > AirportShuttleRequestProcessorBean. Effectively, we're splitting the code > that does the contacting the of the email server from the code that > contacts the database and we're creating several much smaller transactions. >
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 1:17 PM, "Howard W. Smith, Jr." wrote: > David, > > 1. definitely not using @Asynchronous; that was no good at all > > 2. Actually, it was 2 seconds on the Windows Server 2008 64-bit 16GB RAM... > always > > 3. Always 5 to 10 minutes to process 1 or 2 emails on Windows Server 2003 > 32-bit 4GB RAM Good info thanks. > 4. As noted on the JIRA, I tried the TransactionAttribute NEVER and > REQUIRES_NEW on @Stateless EJB that was all invoked via @Schedule, and same > test results, which are The EJB timer rolls back the transaction, and > every database operation that the EmailStatelessBean completed...prior to > @Schedule rolling back the transaction. Right, that suggestion was never implemented as it was intended. Changing the TransactionAttribute alone will do nothing. What we need is to 1) prevent a transaction from being started and 2) creating a new and smaller transaction scope through a small code rework. The ApplicationScoped check backfired because it has the effect of locking tables for much longer. We want to process each "email" in an individual transaction and move away from one big transaction. This is really the most critical step to take, no other steps or workarounds matter at this phase of the problem. Let's try this: - Go back to the very original 5-10 minutes 32-bit code - Split EmailSessionBean into two beans - EmailSessionBean - will be marked @TransactionAttribute(NEVER) - will no longer have an @PersistenceContext EntityManager reference - AirportShuttleRequestProcessorBean - will be marked @TransactionAttribute(REQUIRES_NEW) - will have only one public method, `createOrderForAirportShuttle` - will do all persistence processing The EmailSessionBean will have an @EJB reference to the new AirportShuttleRequestProcessorBean. Effectively, we're splitting the code that does the contacting the of the email server from the code that contacts the database and we're creating several much smaller transactions. -David
Re: Why are Tomcat's threads more costly than background threads?
David, 1. definitely not using @Asynchronous; that was no good at all 2. Actually, it was 2 seconds on the Windows Server 2008 64-bit 16GB RAM... always 3. Always 5 to 10 minutes to process 1 or 2 emails on Windows Server 2003 32-bit 4GB RAM 4. As noted on the JIRA, I tried the TransactionAttribute NEVER and REQUIRES_NEW on @Stateless EJB that was all invoked via @Schedule, and same test results, which are The EJB timer rolls back the transaction, and every database operation that the EmailStatelessBean completed...prior to @Schedule rolling back the transaction. On Thu, Dec 13, 2012 at 4:06 PM, David Blevins wrote: > > David, > > > > Yes, please refer to the following (and see what I wrote at the beginning > > of that comment), thanks. > > > > https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13531442 > > > > So first note, the @TransactionAttribute(value = > TransactionAttributeType.NEVER) and @TransactionAttribute(value = > TransactionAttributeType.REQUIRES_NEW) were two aspects of the same > suggestion. > > Second note is, were I in your shoes, the very first thing I'd do is > revert all the @ApplicationScoped and @Asychronous stuff and go back to the > code that only took 2 minutes -- I seem to recall that before it jumped to > 30 minutes there was a version that ran faster (2 minute range).
Re: Why are Tomcat's threads more costly than background threads?
On Dec 13, 2012, at 12:27 PM, "Howard W. Smith, Jr." wrote: > David, > > Yes, please refer to the following (and see what I wrote at the beginning > of that comment), thanks. > > https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13531442 > So first note, the @TransactionAttribute(value = TransactionAttributeType.NEVER) and @TransactionAttribute(value = TransactionAttributeType.REQUIRES_NEW) were two aspects of the same suggestion. Second note is, were I in your shoes, the very first thing I'd do is revert all the @ApplicationScoped and @Asychronous stuff and go back to the code that only took 2 minutes -- I seem to recall that before it jumped to 30 minutes there was a version that ran faster (2 minute range). Third, all versions of the code posted have the same flaw as the original code -- there is only one transaction for processing all emails. This is the fundamental problem and there is no possible solution that involves *adding* code and not changing this one fundamental flaw. This is why when you added the code that waited for a safe time to do the processing it made things worse -- it was too late a transaction was already started and the database tables already locked. We can definitely get this working better, but the problem is there is simply too much code happing in one transaction. No other fix will help. If you're willing to go back to the original version of the code and do *only* the suggestions given, I think we can get this easily sorted out today. Thoughts? -David
Re: Why are Tomcat's threads more costly than background threads?
David, Yes, please refer to the following (and see what I wrote at the beginning of that comment), thanks. https://issues.apache.org/jira/browse/OPENEJB-1968#comment-13531442 On Thu, Dec 13, 2012 at 3:11 PM, David Blevins wrote: > The description of EmailStatelessBean still looking beans and failing > rollback would imply no transactional semantics have actually changed. Is > it possible you can paste that bean?
Re: Why are Tomcat's threads more costly than background threads?
On Dec 11, 2012, at 7:10 PM, "Howard W. Smith, Jr." wrote: > Shaking my head... test results were not good at all. > > 1. @StatelessEJB EmailStatelessBean has @Schedule getEmails() > 2. @EmailStatelessBean has multiple @EJB references to @Stateless > (sessionfacade) classes that are the DAO classes > 3. EmailStatelessBean gets invoked when triggered by @Schedule > 4. EmailStatelessBean execution locks the tables, still, and locks up the > entire app, and seems to take longer...since endusers (myself) are making > requests against database (and web app). > 5. Last but not least, EmailStatelessBean failed to commit the changes; > transaction rolled back for EmailStatelessBean as well as enduser requests. The description of EmailStatelessBean still looking beans and failing rollback would imply no transactional semantics have actually changed. Is it possible you can paste that bean? -David
Re: Why are Tomcat's threads more costly than background threads?
okay, thanks. I was aware of that already. ;) On Wed, Dec 12, 2012 at 4:03 AM, Jean-Louis MONTEIRO wrote: > Finder methods are usually methods to retrieve an entity or a set of > entities. > Basically, it means Qurey.getSingleResult() or getResultList(). > > By default any method is REQUIRED in EJBs so that means a transaction will > be created whereas it's not required as per the spec. > When you just want to browse data, you actually don't need a transaction. > > A special attention must be of course placed on JPA relationships and more > especially LAZY tagged ones. > As soon as the transaction finished or as soon as they are retrieved when > no transaction is used, they are in an unmanaged state which means that you > cannot navigate through lazy relationships. > > So before retrieving an object graph to browse it in the front layer, you > basically need to know what you want to display in order to pre load all > relationships (using a left join fetch or so query). > > Jean-Louis > > > > > > 2012/12/12 Howard W. Smith, Jr. > > > Jean-Louis, please clarify 'finder methods' and/provide a short example, > > thanks. > > > > > > On Wed, Dec 12, 2012 at 3:08 AM, Jean-Louis MONTEIRO > >wrote: > > > > > I tend to recommend to set any finder methods to SUPPORT which is far > > > better in terms of performances and allow reusability of finder methods > > in > > > business methods. > > > > > > JLouis > > > > > > > > > 2012/12/11 David Blevins > > > > > > > Setting the LockModeType is good. Understand, though, that all the > > > values > > > > in LockModeType are not guaranteed to be supported in all databases. > > > > > > > > It doesn't address the core problem in that the transaction scope is > > > > likely too broad. As a general rule *always* ensure your > transactions > > > are > > > > as *small* as possible. In order to be very conscious about it it > can > > > help > > > > to actually document the code and justify why two or more operations > > need > > > > to be in the same transaction. It'll force you to think about > > > transactions > > > > more deliberately and ask yourself other questions like "where are my > > > > transactions starting and stopping in the first place". > > > > > > > > In this particular situation, with default settings, you'll get one > > > > transaction started for you when the @Schedule method fires. So > > > > essentially, you're batch processing unrelated data in one > transaction. > > > If > > > > Joe and Jane both send email with data, you're processing Jane's data > > in > > > > Joe's transaction. As as well your email fetching time is getting > > > included > > > > in your transaction time. If your email server hangs, your > transaction > > > > hangs. > > > > > > > > So instead of: > > > > > > > > // TRANSACTION START > > > > @Schedule(...) > > > > public void processPendingEmail() { > > > > > > > > List emails = fetchEmail(); > > > > for (Email email : emails) { > > > >// parse email and insert records > > > > } > > > > } > > > > // TRANSACTION END > > > > > > > > You want: > > > > > > > > @Schedule(...) > > > > public void processPendingEmail() { > > > > > > > > List emails = fetchEmail(); > > > > for (Email email : emails) { > > > >// TRANSACTION START > > > >// parse email and insert records > > > >// TRANSACTION END > > > > } > > > > } > > > > > > > > This can be as easy as marking the `processPendingEmail` as > > > > `@TransactionAttribute(NEVER)` then moving the insert logic to > another > > > EJB > > > > method. Just make sure that if you move that logic to another method > > of > > > > the same EJB that you don't invoke that method with a `this` > reference > > or > > > > you won't get container-managed transactions on that method call. > (see > > > this > > > > reference for why): > > > > > > > > > > > > > > > > > > http://stackoverflow.com/questions/3381002/how-is-the-return-value-of-sessioncontext-getbusinessobject-different-from-th/3384128#3384128 > > > > > > > > Hope this helps! > > > > > > > > > > > > -David > > > > > > > > On Dec 11, 2012, at 12:21 PM, "Howard W. Smith, Jr." < > > > > smithh032...@gmail.com> wrote: > > > > > > > > > Prior to adding this @Stateless EJB (that's invoked via @Schedule > > every > > > > 10 > > > > > minutes), my JSF web app has been running 100% okay without any > > > deadlocks > > > > > or anything else like that. > > > > > > > > > > So, is it okay if I just add optimistic locking, em.lock(entity, > > > > > LockType.Optimistic) before any/all em.persist(...) that I have in > > the > > > > > @Stateless EJB? > > > > > > > > > > JSF web app is working okay, because the JSF components call > multiple > > > > > @Stateless EJB that usually do any of the following, SELECT, > UPDATE, > > > > > DELETE, on one table at time. > > > > > > > > > > > > > > > > > > > > On Tue, Dec 11, 2012 at 2:35 PM, How
Re: Why are Tomcat's threads more costly than background threads?
Finder methods are usually methods to retrieve an entity or a set of entities. Basically, it means Qurey.getSingleResult() or getResultList(). By default any method is REQUIRED in EJBs so that means a transaction will be created whereas it's not required as per the spec. When you just want to browse data, you actually don't need a transaction. A special attention must be of course placed on JPA relationships and more especially LAZY tagged ones. As soon as the transaction finished or as soon as they are retrieved when no transaction is used, they are in an unmanaged state which means that you cannot navigate through lazy relationships. So before retrieving an object graph to browse it in the front layer, you basically need to know what you want to display in order to pre load all relationships (using a left join fetch or so query). Jean-Louis 2012/12/12 Howard W. Smith, Jr. > Jean-Louis, please clarify 'finder methods' and/provide a short example, > thanks. > > > On Wed, Dec 12, 2012 at 3:08 AM, Jean-Louis MONTEIRO >wrote: > > > I tend to recommend to set any finder methods to SUPPORT which is far > > better in terms of performances and allow reusability of finder methods > in > > business methods. > > > > JLouis > > > > > > 2012/12/11 David Blevins > > > > > Setting the LockModeType is good. Understand, though, that all the > > values > > > in LockModeType are not guaranteed to be supported in all databases. > > > > > > It doesn't address the core problem in that the transaction scope is > > > likely too broad. As a general rule *always* ensure your transactions > > are > > > as *small* as possible. In order to be very conscious about it it can > > help > > > to actually document the code and justify why two or more operations > need > > > to be in the same transaction. It'll force you to think about > > transactions > > > more deliberately and ask yourself other questions like "where are my > > > transactions starting and stopping in the first place". > > > > > > In this particular situation, with default settings, you'll get one > > > transaction started for you when the @Schedule method fires. So > > > essentially, you're batch processing unrelated data in one transaction. > > If > > > Joe and Jane both send email with data, you're processing Jane's data > in > > > Joe's transaction. As as well your email fetching time is getting > > included > > > in your transaction time. If your email server hangs, your transaction > > > hangs. > > > > > > So instead of: > > > > > > // TRANSACTION START > > > @Schedule(...) > > > public void processPendingEmail() { > > > > > > List emails = fetchEmail(); > > > for (Email email : emails) { > > >// parse email and insert records > > > } > > > } > > > // TRANSACTION END > > > > > > You want: > > > > > > @Schedule(...) > > > public void processPendingEmail() { > > > > > > List emails = fetchEmail(); > > > for (Email email : emails) { > > >// TRANSACTION START > > >// parse email and insert records > > >// TRANSACTION END > > > } > > > } > > > > > > This can be as easy as marking the `processPendingEmail` as > > > `@TransactionAttribute(NEVER)` then moving the insert logic to another > > EJB > > > method. Just make sure that if you move that logic to another method > of > > > the same EJB that you don't invoke that method with a `this` reference > or > > > you won't get container-managed transactions on that method call. (see > > this > > > reference for why): > > > > > > > > > > > > http://stackoverflow.com/questions/3381002/how-is-the-return-value-of-sessioncontext-getbusinessobject-different-from-th/3384128#3384128 > > > > > > Hope this helps! > > > > > > > > > -David > > > > > > On Dec 11, 2012, at 12:21 PM, "Howard W. Smith, Jr." < > > > smithh032...@gmail.com> wrote: > > > > > > > Prior to adding this @Stateless EJB (that's invoked via @Schedule > every > > > 10 > > > > minutes), my JSF web app has been running 100% okay without any > > deadlocks > > > > or anything else like that. > > > > > > > > So, is it okay if I just add optimistic locking, em.lock(entity, > > > > LockType.Optimistic) before any/all em.persist(...) that I have in > the > > > > @Stateless EJB? > > > > > > > > JSF web app is working okay, because the JSF components call multiple > > > > @Stateless EJB that usually do any of the following, SELECT, UPDATE, > > > > DELETE, on one table at time. > > > > > > > > > > > > > > > > On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > > > > smithh032...@gmail.com> wrote: > > > > > > > >> You're absolutely right, David. Thanks for the response. > > > >> > > > >> It is a transaction issue and the @Stateless EJB is holding a > > > transaction > > > >> open and writing into 6 to 10 tables in a single database, which is > > also > > > >> the database accessed by JSF components (by endusers). The > @Stateles
Re: Why are Tomcat's threads more costly than background threads?
Jean-Louis, please clarify 'finder methods' and/provide a short example, thanks. On Wed, Dec 12, 2012 at 3:08 AM, Jean-Louis MONTEIRO wrote: > I tend to recommend to set any finder methods to SUPPORT which is far > better in terms of performances and allow reusability of finder methods in > business methods. > > JLouis > > > 2012/12/11 David Blevins > > > Setting the LockModeType is good. Understand, though, that all the > values > > in LockModeType are not guaranteed to be supported in all databases. > > > > It doesn't address the core problem in that the transaction scope is > > likely too broad. As a general rule *always* ensure your transactions > are > > as *small* as possible. In order to be very conscious about it it can > help > > to actually document the code and justify why two or more operations need > > to be in the same transaction. It'll force you to think about > transactions > > more deliberately and ask yourself other questions like "where are my > > transactions starting and stopping in the first place". > > > > In this particular situation, with default settings, you'll get one > > transaction started for you when the @Schedule method fires. So > > essentially, you're batch processing unrelated data in one transaction. > If > > Joe and Jane both send email with data, you're processing Jane's data in > > Joe's transaction. As as well your email fetching time is getting > included > > in your transaction time. If your email server hangs, your transaction > > hangs. > > > > So instead of: > > > > // TRANSACTION START > > @Schedule(...) > > public void processPendingEmail() { > > > > List emails = fetchEmail(); > > for (Email email : emails) { > >// parse email and insert records > > } > > } > > // TRANSACTION END > > > > You want: > > > > @Schedule(...) > > public void processPendingEmail() { > > > > List emails = fetchEmail(); > > for (Email email : emails) { > >// TRANSACTION START > >// parse email and insert records > >// TRANSACTION END > > } > > } > > > > This can be as easy as marking the `processPendingEmail` as > > `@TransactionAttribute(NEVER)` then moving the insert logic to another > EJB > > method. Just make sure that if you move that logic to another method of > > the same EJB that you don't invoke that method with a `this` reference or > > you won't get container-managed transactions on that method call. (see > this > > reference for why): > > > > > > > http://stackoverflow.com/questions/3381002/how-is-the-return-value-of-sessioncontext-getbusinessobject-different-from-th/3384128#3384128 > > > > Hope this helps! > > > > > > -David > > > > On Dec 11, 2012, at 12:21 PM, "Howard W. Smith, Jr." < > > smithh032...@gmail.com> wrote: > > > > > Prior to adding this @Stateless EJB (that's invoked via @Schedule every > > 10 > > > minutes), my JSF web app has been running 100% okay without any > deadlocks > > > or anything else like that. > > > > > > So, is it okay if I just add optimistic locking, em.lock(entity, > > > LockType.Optimistic) before any/all em.persist(...) that I have in the > > > @Stateless EJB? > > > > > > JSF web app is working okay, because the JSF components call multiple > > > @Stateless EJB that usually do any of the following, SELECT, UPDATE, > > > DELETE, on one table at time. > > > > > > > > > > > > On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > > > smithh032...@gmail.com> wrote: > > > > > >> You're absolutely right, David. Thanks for the response. > > >> > > >> It is a transaction issue and the @Stateless EJB is holding a > > transaction > > >> open and writing into 6 to 10 tables in a single database, which is > also > > >> the database accessed by JSF components (by endusers). The @Stateless > > EJB > > >> is on a timer ( @Schedule), checks email server for incoming requests > > from > > >> customers, and takes that data and inserts it into database, so > endusers > > >> don't have to manually enter the data (that they receive from these > > >> formmail results). > > >> > > >> Below is the exception that I saw maybe an hour or two ago, and I > > >> confirmed that it's a locking/transaction issue, and from the > > stacktrace, > > >> it seems as though the JSF components cannot access the table, that is > > >> evidently locked by @Stateless EJB. The @Stateless @EJB updated all > > tables > > >> on my faster/dev/test server on average of 1 to 2 seconds, and/so in > my > > >> opinion, the data is not that much at all, maybe a few kbytes of data > > being > > >> inserted into the database. Anyway, I've been reading up on > pessismistic > > >> and optimistic locking, and that's where I was in my reading before I > > >> noticed your email. I really don't know what the database or JPA is > > >> defaulting to, because honestly, I have no code that specifies any > > locks. > > >> Recently, I only added query hint (read
Re: Why are Tomcat's threads more costly than background threads?
I tend to recommend to set any finder methods to SUPPORT which is far better in terms of performances and allow reusability of finder methods in business methods. JLouis 2012/12/11 David Blevins > Setting the LockModeType is good. Understand, though, that all the values > in LockModeType are not guaranteed to be supported in all databases. > > It doesn't address the core problem in that the transaction scope is > likely too broad. As a general rule *always* ensure your transactions are > as *small* as possible. In order to be very conscious about it it can help > to actually document the code and justify why two or more operations need > to be in the same transaction. It'll force you to think about transactions > more deliberately and ask yourself other questions like "where are my > transactions starting and stopping in the first place". > > In this particular situation, with default settings, you'll get one > transaction started for you when the @Schedule method fires. So > essentially, you're batch processing unrelated data in one transaction. If > Joe and Jane both send email with data, you're processing Jane's data in > Joe's transaction. As as well your email fetching time is getting included > in your transaction time. If your email server hangs, your transaction > hangs. > > So instead of: > > // TRANSACTION START > @Schedule(...) > public void processPendingEmail() { > > List emails = fetchEmail(); > for (Email email : emails) { >// parse email and insert records > } > } > // TRANSACTION END > > You want: > > @Schedule(...) > public void processPendingEmail() { > > List emails = fetchEmail(); > for (Email email : emails) { >// TRANSACTION START >// parse email and insert records >// TRANSACTION END > } > } > > This can be as easy as marking the `processPendingEmail` as > `@TransactionAttribute(NEVER)` then moving the insert logic to another EJB > method. Just make sure that if you move that logic to another method of > the same EJB that you don't invoke that method with a `this` reference or > you won't get container-managed transactions on that method call. (see this > reference for why): > > > http://stackoverflow.com/questions/3381002/how-is-the-return-value-of-sessioncontext-getbusinessobject-different-from-th/3384128#3384128 > > Hope this helps! > > > -David > > On Dec 11, 2012, at 12:21 PM, "Howard W. Smith, Jr." < > smithh032...@gmail.com> wrote: > > > Prior to adding this @Stateless EJB (that's invoked via @Schedule every > 10 > > minutes), my JSF web app has been running 100% okay without any deadlocks > > or anything else like that. > > > > So, is it okay if I just add optimistic locking, em.lock(entity, > > LockType.Optimistic) before any/all em.persist(...) that I have in the > > @Stateless EJB? > > > > JSF web app is working okay, because the JSF components call multiple > > @Stateless EJB that usually do any of the following, SELECT, UPDATE, > > DELETE, on one table at time. > > > > > > > > On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > > smithh032...@gmail.com> wrote: > > > >> You're absolutely right, David. Thanks for the response. > >> > >> It is a transaction issue and the @Stateless EJB is holding a > transaction > >> open and writing into 6 to 10 tables in a single database, which is also > >> the database accessed by JSF components (by endusers). The @Stateless > EJB > >> is on a timer ( @Schedule), checks email server for incoming requests > from > >> customers, and takes that data and inserts it into database, so endusers > >> don't have to manually enter the data (that they receive from these > >> formmail results). > >> > >> Below is the exception that I saw maybe an hour or two ago, and I > >> confirmed that it's a locking/transaction issue, and from the > stacktrace, > >> it seems as though the JSF components cannot access the table, that is > >> evidently locked by @Stateless EJB. The @Stateless @EJB updated all > tables > >> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my > >> opinion, the data is not that much at all, maybe a few kbytes of data > being > >> inserted into the database. Anyway, I've been reading up on pessismistic > >> and optimistic locking, and that's where I was in my reading before I > >> noticed your email. I really don't know what the database or JPA is > >> defaulting to, because honestly, I have no code that specifies any > locks. > >> Recently, I only added query hint (readonly) to JPA queries. > Personally, I > >> feel as though I can use 'no locking' for these inserts, since this is > >> brand new data, but the JSF components may be locking the entites/tables > >> during simple queries (SELECT statements). Feel free to change this > >> email/topic to a more suitable topic/subject, if necessary. :) > >> > >> > >> > >> > > -- Jean-Louis
Re: Why are Tomcat's threads more costly than background threads?
Also, are there any issues that I need to be made aware of...if @Stateless EJB invokes a method on CDI @ApplicationScoped bean... @Stateless EJB injects the CDI @ApplicationScoped bean via @Inject. Is this a good practice or not? I think I read in Java EE 6 tutorial that EJB timers cannot be used on stateful session beans, so I would assume it's not an option to add @Schedule to method on the CDI @ApplicationScoped bean. On Tue, Dec 11, 2012 at 10:10 PM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > Shaking my head... test results were not good at all. > > 1. @StatelessEJB EmailStatelessBean has @Schedule getEmails() > 2. @EmailStatelessBean has multiple @EJB references to @Stateless > (sessionfacade) classes that are the DAO classes > 3. EmailStatelessBean gets invoked when triggered by @Schedule > 4. EmailStatelessBean execution locks the tables, still, and locks up the > entire app, and seems to take longer...since endusers (myself) are making > requests against database (and web app). > 5. Last but not least, EmailStatelessBean failed to commit the changes; > transaction rolled back for EmailStatelessBean as well as enduser requests. > > So, my workaround options include the following: > > 1. EmailStatelessBean, when invoked by @Schedule method, will check the > CDI @ApplicationScoped bean (ApplicationScopeBean), and see if any endusers > are logged in; i developed a sessionInfo class that is updated everytime > enduser does a HTTP request; if any endusers are logged in, then I can use > Atmosphere (PrimeFaces Push) to push a message to the endusers, notifying > them to 'please logout, as there are customer requests pending that need to > be retrieved and inserted into the web app's database', and > EmailStatelessBean will 'return' (or exit) and not perform the operation. > This way, everytime @Schedule invokes the bean method, it will continue to > check for an available time to perform the > get-emails-and-insert-into-database operation. Also, I'll set when > EmailStatelessBean begins the operation, then I'll set flag on > @ApplicationScoped bean (insertingCustomerRequestsIntoDatabase = true); > this flag will be checked when endusers attempt to login web app, and > FacesMessage will be displayed, Please login a few minutes later as system > is inserting customer requests into database. > > 2. Another option would be to send a message to all clients via Atmosphere > (PrimeFaces Push), and the UI will be locked immediately, and user will be > required to wait. > > Honestly, I don't like either of these approaches, but I think the > endusers will accept the 1st option (above). If I go with 1st approach, > then I may revert back to single transaction, which will complete the > entire operation much faster than multiple transactions (via multiple EJB > DAO classes). > > As you can see, I don't have enough experience locking database for such a > batch operation as this. I know the endusers want the customer requests to > show up into the database 'immediately', and they want to know when it > happens. Right now, this is my only option until I redesign the business > website to interface directly with the web app. right now, the web app is > 'only' used by 'personnel' (the owners of the business...which is my > family). hahaha :) > > They love the app (honestly, they liked the speed and reliability of the > web app when it was on Glassfish), but I'm doing all i can to win them over > with TomEE. :) > > Your thoughts, please. > > Thanks, > Howard > > > > On Tue, Dec 11, 2012 at 9:23 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> Done. testing now and watching JVM. I need to read that article you >> shared, so i can learn a bit about heap dump. I see the heap dump button >> there, but never used it 'yet'. :) >> >> >> On Tue, Dec 11, 2012 at 3:53 PM, Howard W. Smith, Jr. < >> smithh032...@gmail.com> wrote: >> >>> I think I'm going to avoid the single transaction (via single @Stateless >>> EJB) approach, and use other @Stateless EJBs to make the database table >>> updates. This approach works everywhere else throughout the web app. >>> >>> I'll just keep the @Stateless EJB for the purpose of being invoked by >>> @Schedule, and it will be the 'driver', and call the >>> already-defined-and-already-used other @Stateless EJBs that contain >>> Persistent context and entity manager to update tables, so this clearly >>> will be a multiple transaction commit, which should 'not' lock any tables. >>> :) >>> >>> >>> >>> On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < >>> smithh032...@gmail.com> wrote: >>> Prior to adding this @Stateless EJB (that's invoked via @Schedule every 10 minutes), my JSF web app has been running 100% okay without any deadlocks or anything else like that. So, is it okay if I just add optimistic locking, em.lock(entity, LockType.Optimistic) before any/all em.persist(...) that I have in the @Stateless EJB? JSF web
Re: Why are Tomcat's threads more costly than background threads?
Shaking my head... test results were not good at all. 1. @StatelessEJB EmailStatelessBean has @Schedule getEmails() 2. @EmailStatelessBean has multiple @EJB references to @Stateless (sessionfacade) classes that are the DAO classes 3. EmailStatelessBean gets invoked when triggered by @Schedule 4. EmailStatelessBean execution locks the tables, still, and locks up the entire app, and seems to take longer...since endusers (myself) are making requests against database (and web app). 5. Last but not least, EmailStatelessBean failed to commit the changes; transaction rolled back for EmailStatelessBean as well as enduser requests. So, my workaround options include the following: 1. EmailStatelessBean, when invoked by @Schedule method, will check the CDI @ApplicationScoped bean (ApplicationScopeBean), and see if any endusers are logged in; i developed a sessionInfo class that is updated everytime enduser does a HTTP request; if any endusers are logged in, then I can use Atmosphere (PrimeFaces Push) to push a message to the endusers, notifying them to 'please logout, as there are customer requests pending that need to be retrieved and inserted into the web app's database', and EmailStatelessBean will 'return' (or exit) and not perform the operation. This way, everytime @Schedule invokes the bean method, it will continue to check for an available time to perform the get-emails-and-insert-into-database operation. Also, I'll set when EmailStatelessBean begins the operation, then I'll set flag on @ApplicationScoped bean (insertingCustomerRequestsIntoDatabase = true); this flag will be checked when endusers attempt to login web app, and FacesMessage will be displayed, Please login a few minutes later as system is inserting customer requests into database. 2. Another option would be to send a message to all clients via Atmosphere (PrimeFaces Push), and the UI will be locked immediately, and user will be required to wait. Honestly, I don't like either of these approaches, but I think the endusers will accept the 1st option (above). If I go with 1st approach, then I may revert back to single transaction, which will complete the entire operation much faster than multiple transactions (via multiple EJB DAO classes). As you can see, I don't have enough experience locking database for such a batch operation as this. I know the endusers want the customer requests to show up into the database 'immediately', and they want to know when it happens. Right now, this is my only option until I redesign the business website to interface directly with the web app. right now, the web app is 'only' used by 'personnel' (the owners of the business...which is my family). hahaha :) They love the app (honestly, they liked the speed and reliability of the web app when it was on Glassfish), but I'm doing all i can to win them over with TomEE. :) Your thoughts, please. Thanks, Howard On Tue, Dec 11, 2012 at 9:23 PM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > Done. testing now and watching JVM. I need to read that article you > shared, so i can learn a bit about heap dump. I see the heap dump button > there, but never used it 'yet'. :) > > > On Tue, Dec 11, 2012 at 3:53 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> I think I'm going to avoid the single transaction (via single @Stateless >> EJB) approach, and use other @Stateless EJBs to make the database table >> updates. This approach works everywhere else throughout the web app. >> >> I'll just keep the @Stateless EJB for the purpose of being invoked by >> @Schedule, and it will be the 'driver', and call the >> already-defined-and-already-used other @Stateless EJBs that contain >> Persistent context and entity manager to update tables, so this clearly >> will be a multiple transaction commit, which should 'not' lock any tables. >> :) >> >> >> >> On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < >> smithh032...@gmail.com> wrote: >> >>> Prior to adding this @Stateless EJB (that's invoked via @Schedule every >>> 10 minutes), my JSF web app has been running 100% okay without any >>> deadlocks or anything else like that. >>> >>> So, is it okay if I just add optimistic locking, em.lock(entity, >>> LockType.Optimistic) before any/all em.persist(...) that I have in the >>> @Stateless EJB? >>> >>> JSF web app is working okay, because the JSF components call multiple >>> @Stateless EJB that usually do any of the following, SELECT, UPDATE, >>> DELETE, on one table at time. >>> >>> >>> >>> On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < >>> smithh032...@gmail.com> wrote: >>> You're absolutely right, David. Thanks for the response. It is a transaction issue and the @Stateless EJB is holding a transaction open and writing into 6 to 10 tables in a single database, which is also the database accessed by JSF components (by endusers). The @Stateless EJB is on a timer ( @Schedule), checks email server for incoming requests
Re: Why are Tomcat's threads more costly than background threads?
Done. testing now and watching JVM. I need to read that article you shared, so i can learn a bit about heap dump. I see the heap dump button there, but never used it 'yet'. :) On Tue, Dec 11, 2012 at 3:53 PM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > I think I'm going to avoid the single transaction (via single @Stateless > EJB) approach, and use other @Stateless EJBs to make the database table > updates. This approach works everywhere else throughout the web app. > > I'll just keep the @Stateless EJB for the purpose of being invoked by > @Schedule, and it will be the 'driver', and call the > already-defined-and-already-used other @Stateless EJBs that contain > Persistent context and entity manager to update tables, so this clearly > will be a multiple transaction commit, which should 'not' lock any tables. > :) > > > > On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> Prior to adding this @Stateless EJB (that's invoked via @Schedule every >> 10 minutes), my JSF web app has been running 100% okay without any >> deadlocks or anything else like that. >> >> So, is it okay if I just add optimistic locking, em.lock(entity, >> LockType.Optimistic) before any/all em.persist(...) that I have in the >> @Stateless EJB? >> >> JSF web app is working okay, because the JSF components call multiple >> @Stateless EJB that usually do any of the following, SELECT, UPDATE, >> DELETE, on one table at time. >> >> >> >> On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < >> smithh032...@gmail.com> wrote: >> >>> You're absolutely right, David. Thanks for the response. >>> >>> It is a transaction issue and the @Stateless EJB is holding a >>> transaction open and writing into 6 to 10 tables in a single database, >>> which is also the database accessed by JSF components (by endusers). The >>> @Stateless EJB is on a timer ( @Schedule), checks email server for incoming >>> requests from customers, and takes that data and inserts it into database, >>> so endusers don't have to manually enter the data (that they receive from >>> these formmail results). >>> >>> Below is the exception that I saw maybe an hour or two ago, and I >>> confirmed that it's a locking/transaction issue, and from the stacktrace, >>> it seems as though the JSF components cannot access the table, that is >>> evidently locked by @Stateless EJB. The @Stateless @EJB updated all tables >>> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my >>> opinion, the data is not that much at all, maybe a few kbytes of data being >>> inserted into the database. Anyway, I've been reading up on pessismistic >>> and optimistic locking, and that's where I was in my reading before I >>> noticed your email. I really don't know what the database or JPA is >>> defaulting to, because honestly, I have no code that specifies any locks. >>> Recently, I only added query hint (readonly) to JPA queries. Personally, I >>> feel as though I can use 'no locking' for these inserts, since this is >>> brand new data, but the JSF components may be locking the entites/tables >>> during simple queries (SELECT statements). Feel free to change this >>> email/topic to a more suitable topic/subject, if necessary. :) >>> >>> >>> >>> >
Re: Why are Tomcat's threads more costly than background threads?
Yes, that is funny. I was away from my desk, and saw that your previous email, and I was already on the same page as you, but your (previous) email was confirmation to me that that is what I needed to do. All around the same time, I sent an email to the endusers and explained the problem and what I was planning to do to resolve the issue. Now, time to get to work and get this done. Thanks again for the help, assistance, and responses! On Tue, Dec 11, 2012 at 4:26 PM, David Blevins wrote: > Funny, I just sent that suggestion :) But it took me too long to type it > -- didn't see your response came in before I finished :) > > -David > > On Dec 11, 2012, at 12:53 PM, "Howard W. Smith, Jr." < > smithh032...@gmail.com> wrote: > > > I think I'm going to avoid the single transaction (via single @Stateless > > EJB) approach, and use other @Stateless EJBs to make the database table > > updates. This approach works everywhere else throughout the web app. > > > > I'll just keep the @Stateless EJB for the purpose of being invoked by > > @Schedule, and it will be the 'driver', and call the > > already-defined-and-already-used other @Stateless EJBs that contain > > Persistent context and entity manager to update tables, so this clearly > > will be a multiple transaction commit, which should 'not' lock any > tables. > > :) > > > > > > > > On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < > > smithh032...@gmail.com> wrote: > > > >> Prior to adding this @Stateless EJB (that's invoked via @Schedule every > 10 > >> minutes), my JSF web app has been running 100% okay without any > deadlocks > >> or anything else like that. > >> > >> So, is it okay if I just add optimistic locking, em.lock(entity, > >> LockType.Optimistic) before any/all em.persist(...) that I have in the > >> @Stateless EJB? > >> > >> JSF web app is working okay, because the JSF components call multiple > >> @Stateless EJB that usually do any of the following, SELECT, UPDATE, > >> DELETE, on one table at time. > >> > >> > >> > >> On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > >> smithh032...@gmail.com> wrote: > >> > >>> You're absolutely right, David. Thanks for the response. > >>> > >>> It is a transaction issue and the @Stateless EJB is holding a > transaction > >>> open and writing into 6 to 10 tables in a single database, which is > also > >>> the database accessed by JSF components (by endusers). The @Stateless > EJB > >>> is on a timer ( @Schedule), checks email server for incoming requests > from > >>> customers, and takes that data and inserts it into database, so > endusers > >>> don't have to manually enter the data (that they receive from these > >>> formmail results). > >>> > >>> Below is the exception that I saw maybe an hour or two ago, and I > >>> confirmed that it's a locking/transaction issue, and from the > stacktrace, > >>> it seems as though the JSF components cannot access the table, that is > >>> evidently locked by @Stateless EJB. The @Stateless @EJB updated all > tables > >>> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my > >>> opinion, the data is not that much at all, maybe a few kbytes of data > being > >>> inserted into the database. Anyway, I've been reading up on > pessismistic > >>> and optimistic locking, and that's where I was in my reading before I > >>> noticed your email. I really don't know what the database or JPA is > >>> defaulting to, because honestly, I have no code that specifies any > locks. > >>> Recently, I only added query hint (readonly) to JPA queries. > Personally, I > >>> feel as though I can use 'no locking' for these inserts, since this is > >>> brand new data, but the JSF components may be locking the > entites/tables > >>> during simple queries (SELECT statements). Feel free to change this > >>> email/topic to a more suitable topic/subject, if necessary. :) > >>> > >>> > >>> > >>> > >
Re: Why are Tomcat's threads more costly than background threads?
Funny, I just sent that suggestion :) But it took me too long to type it -- didn't see your response came in before I finished :) -David On Dec 11, 2012, at 12:53 PM, "Howard W. Smith, Jr." wrote: > I think I'm going to avoid the single transaction (via single @Stateless > EJB) approach, and use other @Stateless EJBs to make the database table > updates. This approach works everywhere else throughout the web app. > > I'll just keep the @Stateless EJB for the purpose of being invoked by > @Schedule, and it will be the 'driver', and call the > already-defined-and-already-used other @Stateless EJBs that contain > Persistent context and entity manager to update tables, so this clearly > will be a multiple transaction commit, which should 'not' lock any tables. > :) > > > > On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> Prior to adding this @Stateless EJB (that's invoked via @Schedule every 10 >> minutes), my JSF web app has been running 100% okay without any deadlocks >> or anything else like that. >> >> So, is it okay if I just add optimistic locking, em.lock(entity, >> LockType.Optimistic) before any/all em.persist(...) that I have in the >> @Stateless EJB? >> >> JSF web app is working okay, because the JSF components call multiple >> @Stateless EJB that usually do any of the following, SELECT, UPDATE, >> DELETE, on one table at time. >> >> >> >> On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < >> smithh032...@gmail.com> wrote: >> >>> You're absolutely right, David. Thanks for the response. >>> >>> It is a transaction issue and the @Stateless EJB is holding a transaction >>> open and writing into 6 to 10 tables in a single database, which is also >>> the database accessed by JSF components (by endusers). The @Stateless EJB >>> is on a timer ( @Schedule), checks email server for incoming requests from >>> customers, and takes that data and inserts it into database, so endusers >>> don't have to manually enter the data (that they receive from these >>> formmail results). >>> >>> Below is the exception that I saw maybe an hour or two ago, and I >>> confirmed that it's a locking/transaction issue, and from the stacktrace, >>> it seems as though the JSF components cannot access the table, that is >>> evidently locked by @Stateless EJB. The @Stateless @EJB updated all tables >>> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my >>> opinion, the data is not that much at all, maybe a few kbytes of data being >>> inserted into the database. Anyway, I've been reading up on pessismistic >>> and optimistic locking, and that's where I was in my reading before I >>> noticed your email. I really don't know what the database or JPA is >>> defaulting to, because honestly, I have no code that specifies any locks. >>> Recently, I only added query hint (readonly) to JPA queries. Personally, I >>> feel as though I can use 'no locking' for these inserts, since this is >>> brand new data, but the JSF components may be locking the entites/tables >>> during simple queries (SELECT statements). Feel free to change this >>> email/topic to a more suitable topic/subject, if necessary. :) >>> >>> >>> >>>
Re: Why are Tomcat's threads more costly than background threads?
Setting the LockModeType is good. Understand, though, that all the values in LockModeType are not guaranteed to be supported in all databases. It doesn't address the core problem in that the transaction scope is likely too broad. As a general rule *always* ensure your transactions are as *small* as possible. In order to be very conscious about it it can help to actually document the code and justify why two or more operations need to be in the same transaction. It'll force you to think about transactions more deliberately and ask yourself other questions like "where are my transactions starting and stopping in the first place". In this particular situation, with default settings, you'll get one transaction started for you when the @Schedule method fires. So essentially, you're batch processing unrelated data in one transaction. If Joe and Jane both send email with data, you're processing Jane's data in Joe's transaction. As as well your email fetching time is getting included in your transaction time. If your email server hangs, your transaction hangs. So instead of: // TRANSACTION START @Schedule(...) public void processPendingEmail() { List emails = fetchEmail(); for (Email email : emails) { // parse email and insert records } } // TRANSACTION END You want: @Schedule(...) public void processPendingEmail() { List emails = fetchEmail(); for (Email email : emails) { // TRANSACTION START // parse email and insert records // TRANSACTION END } } This can be as easy as marking the `processPendingEmail` as `@TransactionAttribute(NEVER)` then moving the insert logic to another EJB method. Just make sure that if you move that logic to another method of the same EJB that you don't invoke that method with a `this` reference or you won't get container-managed transactions on that method call. (see this reference for why): http://stackoverflow.com/questions/3381002/how-is-the-return-value-of-sessioncontext-getbusinessobject-different-from-th/3384128#3384128 Hope this helps! -David On Dec 11, 2012, at 12:21 PM, "Howard W. Smith, Jr." wrote: > Prior to adding this @Stateless EJB (that's invoked via @Schedule every 10 > minutes), my JSF web app has been running 100% okay without any deadlocks > or anything else like that. > > So, is it okay if I just add optimistic locking, em.lock(entity, > LockType.Optimistic) before any/all em.persist(...) that I have in the > @Stateless EJB? > > JSF web app is working okay, because the JSF components call multiple > @Stateless EJB that usually do any of the following, SELECT, UPDATE, > DELETE, on one table at time. > > > > On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> You're absolutely right, David. Thanks for the response. >> >> It is a transaction issue and the @Stateless EJB is holding a transaction >> open and writing into 6 to 10 tables in a single database, which is also >> the database accessed by JSF components (by endusers). The @Stateless EJB >> is on a timer ( @Schedule), checks email server for incoming requests from >> customers, and takes that data and inserts it into database, so endusers >> don't have to manually enter the data (that they receive from these >> formmail results). >> >> Below is the exception that I saw maybe an hour or two ago, and I >> confirmed that it's a locking/transaction issue, and from the stacktrace, >> it seems as though the JSF components cannot access the table, that is >> evidently locked by @Stateless EJB. The @Stateless @EJB updated all tables >> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my >> opinion, the data is not that much at all, maybe a few kbytes of data being >> inserted into the database. Anyway, I've been reading up on pessismistic >> and optimistic locking, and that's where I was in my reading before I >> noticed your email. I really don't know what the database or JPA is >> defaulting to, because honestly, I have no code that specifies any locks. >> Recently, I only added query hint (readonly) to JPA queries. Personally, I >> feel as though I can use 'no locking' for these inserts, since this is >> brand new data, but the JSF components may be locking the entites/tables >> during simple queries (SELECT statements). Feel free to change this >> email/topic to a more suitable topic/subject, if necessary. :) >> >> >> >>
Re: Why are Tomcat's threads more costly than background threads?
I think I'm going to avoid the single transaction (via single @Stateless EJB) approach, and use other @Stateless EJBs to make the database table updates. This approach works everywhere else throughout the web app. I'll just keep the @Stateless EJB for the purpose of being invoked by @Schedule, and it will be the 'driver', and call the already-defined-and-already-used other @Stateless EJBs that contain Persistent context and entity manager to update tables, so this clearly will be a multiple transaction commit, which should 'not' lock any tables. :) On Tue, Dec 11, 2012 at 3:21 PM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > Prior to adding this @Stateless EJB (that's invoked via @Schedule every 10 > minutes), my JSF web app has been running 100% okay without any deadlocks > or anything else like that. > > So, is it okay if I just add optimistic locking, em.lock(entity, > LockType.Optimistic) before any/all em.persist(...) that I have in the > @Stateless EJB? > > JSF web app is working okay, because the JSF components call multiple > @Stateless EJB that usually do any of the following, SELECT, UPDATE, > DELETE, on one table at time. > > > > On Tue, Dec 11, 2012 at 2:35 PM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> You're absolutely right, David. Thanks for the response. >> >> It is a transaction issue and the @Stateless EJB is holding a transaction >> open and writing into 6 to 10 tables in a single database, which is also >> the database accessed by JSF components (by endusers). The @Stateless EJB >> is on a timer ( @Schedule), checks email server for incoming requests from >> customers, and takes that data and inserts it into database, so endusers >> don't have to manually enter the data (that they receive from these >> formmail results). >> >> Below is the exception that I saw maybe an hour or two ago, and I >> confirmed that it's a locking/transaction issue, and from the stacktrace, >> it seems as though the JSF components cannot access the table, that is >> evidently locked by @Stateless EJB. The @Stateless @EJB updated all tables >> on my faster/dev/test server on average of 1 to 2 seconds, and/so in my >> opinion, the data is not that much at all, maybe a few kbytes of data being >> inserted into the database. Anyway, I've been reading up on pessismistic >> and optimistic locking, and that's where I was in my reading before I >> noticed your email. I really don't know what the database or JPA is >> defaulting to, because honestly, I have no code that specifies any locks. >> Recently, I only added query hint (readonly) to JPA queries. Personally, I >> feel as though I can use 'no locking' for these inserts, since this is >> brand new data, but the JSF components may be locking the entites/tables >> during simple queries (SELECT statements). Feel free to change this >> email/topic to a more suitable topic/subject, if necessary. :) >> >> >> >>
Re: Why are Tomcat's threads more costly than background threads?
You're absolutely right, David. Thanks for the response. It is a transaction issue and the @Stateless EJB is holding a transaction open and writing into 6 to 10 tables in a single database, which is also the database accessed by JSF components (by endusers). The @Stateless EJB is on a timer ( @Schedule), checks email server for incoming requests from customers, and takes that data and inserts it into database, so endusers don't have to manually enter the data (that they receive from these formmail results). Below is the exception that I saw maybe an hour or two ago, and I confirmed that it's a locking/transaction issue, and from the stacktrace, it seems as though the JSF components cannot access the table, that is evidently locked by @Stateless EJB. The @Stateless @EJB updated all tables on my faster/dev/test server on average of 1 to 2 seconds, and/so in my opinion, the data is not that much at all, maybe a few kbytes of data being inserted into the database. Anyway, I've been reading up on pessismistic and optimistic locking, and that's where I was in my reading before I noticed your email. I really don't know what the database or JPA is defaulting to, because honestly, I have no code that specifies any locks. Recently, I only added query hint (readonly) to JPA queries. Personally, I feel as though I can use 'no locking' for these inserts, since this is brand new data, but the JSF components may be locking the entites/tables during simple queries (SELECT statements). Feel free to change this email/topic to a more suitable topic/subject, if necessary. :) enCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) ... 126 more javax.ejb.EJBException: The bean encountered a non-application exception; nested exception is: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.2.v2025-r10461): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: java.sql.SQLTransactionRollbackException: A lock could not be obtained within the time requested Error Code: 3 Call: SELECT t0.ORDER_ID, t0.BALANCE, t0.CANCELLED, t0.CANCELLED_DATE, t0.CHANGED_DT, t0.CONFIRMED, t0.CONFIRMED_DATE, t0.CONTRACT_SENT, t0.CONTRACT_SENT_DATE, t0.CREATED_BY, t0.CREATED_DT, t0.DEPOSIT, t0.DEPOSIT_PAID, t0.DEPOSIT_PAID_DATE, t0.DISCOUNT, t0.DISCOUNT_RATE, t0.INTRASTATE, t0.INVOICE_SENT, t0.INVOICE_SENT_DATE, t0.PAID_IN_FULL, t0.PAID_IN_FULL_DATE, t0.PASSENGERS, t0.QUOTE_SENT, t0.QUOTE_SENT_DATE, t0.REPORT_DATE_TIME, t0.RETURN_DATE_TIME, t0.TOTAL_COST, t0.TRIP_DATE_TIME, t0.BANK_FOR_BALANCE_ID, t0.BANK_FOR_DEPOSIT_ID, t0.CUSTOMER_ID, t0.DETAILS_ID, t0.METHOD_OF_PAYMENT_FOR_BALANCE_ID, t0.METHOD_OF_PAYMENT_FOR_DEPOSIT_ID, t0.NOTES_ID, t0.ROUTES_ID, t0.TEMPLATE_ID FROM ORDERS t0 WHERE (t0.TRIP_DATE_TIME BETWEEN {ts '2012-12-06 00:00:00.0'} AND {ts '2012-12-06 23:59:59.0'}) OR (t0.RETURN_DATE_TIME BETWEEN {ts '2012-12-06 00:00:00.0'} AND {ts '2012-12-06 23:59:59.0'})) OR ({ts '2012-12-06 00:00:00.0'} BETWEEN t0.TRIP_DATE_TIME AND t0.RETURN_DATE_TIME)) OR ({ts '2012-12-06 23:59:59.0'} BETWEEN t0.TRIP_DATE_TIME AND t0.RETURN_DATE_TIME)) AND NOT EXISTS (SELECT 1 FROM ORDER_TEMPLATE t2, ORDER_TEMPLATE t1 WHERE ((t1.TEMPLATE_ID = t2.TEMPLATE_ID) AND (t2.TEMPLATE_ID = t0.TEMPLATE_ID))) ) ORDER BY t0.REPORT_DATE_TIME ASC, t0.TRIP_DATE_TIME ASC Query: ReadAllQuery(name="Orders.findAllBookedTrips" referenceClass=Orders sql="SELECT t0.ORDER_ID, t0.BALANCE, t0.CANCELLED, t0.CANCELLED_DATE, t0.CHANGED_DT, t0.CONFIRMED, t0.CONFIRMED_DATE, t0.CONTRACT_SENT, t0.CONTRACT_SENT_DATE, t0.CREATED_BY, t0.CREATED_DT, t0.DEPOSIT, t0.DEPOSIT_PAID, t0.DEPOSIT_PAID_DATE, t0.DISCOUNT, t0.DISCOUNT_RATE, t0.INTRASTATE, t0.INVOICE_SENT, t0.INVOICE_SENT_DATE, t0.PAID_IN_FULL, t0.PAID_IN_FULL_DATE, t0.PASSENGERS, t0.QUOTE_SENT, t0.QUOTE_SENT_DATE, t0.REPORT_DATE_TIME, t0.RETURN_DATE_TIME, t0.TOTAL_COST, t0.TRIP_DATE_TIME, t0.BANK_FOR_BALANCE_ID, t0.BANK_FOR_DEPOSIT_ID, t0.CUSTOMER_ID, t0.DETAILS_ID, t0.METHOD_OF_PAYMENT_FOR_BALANCE_ID, t0.METHOD_OF_PAYMENT_FOR_DEPOSIT_ID, t0.NOTES_ID, t0.ROUTES_ID, t0.TEMPLATE_ID FROM ORDERS t0 WHERE (t0.TRIP_DATE_TIME BETWEEN ? AND ?) OR (t0.RETURN_DATE_TIME BETWEEN ? AND ?)) OR (? BETWEEN t0.TRIP_DATE_TIME AND t0.RETURN_DATE_TIME)) OR (? BETWEEN t0.TRIP_DATE_TIME AND t0.RETURN_DATE_TIME)) AND NOT EXISTS (SELECT ? FROM ORDER_TEMPLATE t2, ORDER_TEMPLATE t1 WHERE ((t1.TEMPLATE_ID = t2.TEMPLATE_ID) AND (t2.TEMPLATE_ID = t0.TEMPLATE_ID))) ) ORDER BY t0.REPORT_DATE_TIME ASC, t0.TRIP_DATE_TIME ASC") at org.apache.openejb.core.ivm.BaseEjbProxyHandler.convertException(BaseEjbProxyHandler.java:363) at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:283) at jpa.session.OrdersFacade$LocalBeanProxy.filterBy(jpa/session/OrdersFacade.java) at jsf.orders.pf_OrdersController.fi
Re: Why are Tomcat's threads more costly than background threads?
First a tip. Next time the server locks up, immediately do a thread dump. - http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstack.html Second, it sounds like this locking issue isn't a "threading" issue, but a transaction issue. Accessing all your databases in the same JTA transaction is not advised unless you're doing something like withdrawing $100 from database A and putting it in database B. This kind of work is fine to do, but you want it to be as focused and razor sharp as possible -- only do it with the understanding you're locking the referenced tables/rows on both databases while that transaction is open. This can be tragic if pessimistic locking is used, less of a problem with more optimistic locking options. These options are all database specific. Again, this kind of thing is fine if the task requires it -- there's no other reliable way to solve the "transfer data from one database to the next problem" other then to enroll them both in a JTA transaction -- but such a thing must be done with full understanding of the cost. What kind of work are you attempting to do with this job that accesses all your databases? -David On Dec 11, 2012, at 8:40 AM, "Howard W. Smith, Jr." wrote: > Okay, TomEE committers/users, I need your help/advice on this one. Please > read what I wrote in the email below. > > I am considering to use the @Asynchronous on one of the methods on > @Stateless EJB to complete this task, but I think I still may need to do > something on the Persistence end to avoid locking up the entire app, > database, etc... > > I guess this my first time experiencing a DEADLOCK, but I could be wrong. :) > > > > On Tue, Dec 11, 2012 at 11:01 AM, Howard W. Smith, Jr. < > smithh032...@gmail.com> wrote: > >> Wow, i'm reading this now, because I just experienced an issue on my >> production server that is TomEE 1.5.1 (Tomcat 7.0.34), and the whole server >> locked up all because I had a @Stateless EJB inserting data into multiple >> tables in the database, because @Schedule timed event triggered the EJB to >> check email server for incoming (customer) requests, and it literally took >> down the server. I was on it as well as few other endusers, and then one >> enduser captured a LOCK error and the screen capture (photo/pic) had an >> error message that showed a long SQL query with datatable table and column >> names t0..., t0... >> >> What I primarily saw was the word 'lock' at the top of that, and we >> definitely experienced a lockup. I'm about to check server logs and read >> this article. >> >> The @Stateless EJB had one transaction (entitymanager / persistence >> context) that made database updates to multiple tables in the database. I >> am only using entitymanager.persist(), flush(), and few lookups/queries >> during that process. >> >> But other endusers (including myself) could not do simple queries against >> the database at all. Most of my queries contain query hints (readonly, >> statement caching). >> >> Also, I never seen this behavior at all, but this is first time I added >> @Stateless EJB along with @Schedule that does database updates 'during >> business hours'. I thought this would be a no-brainer, but I guess it's >> not. Again, the server is TomEE 1.5.1 (tomcat 7.0.34). >> >> Any advise, then please let me know. Onto reading this post now. Thanks. :) >> >> >> >> On Tue, Dec 11, 2012 at 10:49 AM, Julien Martin wrote: >> >>> Thank you very much for this exhaustive reply Christopher. >>> >>> 2012/12/11 Christopher Schultz >>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Julien, Warning: this is long. Like, André-or-Mark-Eggers long. On 12/11/12 7:30 AM, Julien Martin wrote: > I am in reference to the following blog entry: Blog > entry< >>> http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support > > > about Spring MVC 3.2 asynchronous support. > > I understand Tomcat uses a thread pool in order to serve http/web > requests. Furthermore, the above article seems to indicate that > Spring MVC asynchronous support relieves Tomcat's thread pool and > allows for better concurrency in the webapp by using background > threads for "heavy-lift" operations. I believe you are misinterpreting what that post has to say. It's not that a "background" thread itself is more efficient, it's that processing that does not need to communicate with the client can be de-coupled from the request-processing thread-pool that exists for that purpose. An example - right from the blog post - will make much more sense than what I wrote above. Let's take the example of sending an email message. First, some assumptions: 1. Sending an email takes a long time (say, 5 seconds) 2. The client does not need confirmation that the email has been sent
Re: Why are Tomcat's threads more costly than background threads?
Okay, TomEE committers/users, I need your help/advice on this one. Please read what I wrote in the email below. I am considering to use the @Asynchronous on one of the methods on @Stateless EJB to complete this task, but I think I still may need to do something on the Persistence end to avoid locking up the entire app, database, etc... I guess this my first time experiencing a DEADLOCK, but I could be wrong. :) On Tue, Dec 11, 2012 at 11:01 AM, Howard W. Smith, Jr. < smithh032...@gmail.com> wrote: > Wow, i'm reading this now, because I just experienced an issue on my > production server that is TomEE 1.5.1 (Tomcat 7.0.34), and the whole server > locked up all because I had a @Stateless EJB inserting data into multiple > tables in the database, because @Schedule timed event triggered the EJB to > check email server for incoming (customer) requests, and it literally took > down the server. I was on it as well as few other endusers, and then one > enduser captured a LOCK error and the screen capture (photo/pic) had an > error message that showed a long SQL query with datatable table and column > names t0..., t0... > > What I primarily saw was the word 'lock' at the top of that, and we > definitely experienced a lockup. I'm about to check server logs and read > this article. > > The @Stateless EJB had one transaction (entitymanager / persistence > context) that made database updates to multiple tables in the database. I > am only using entitymanager.persist(), flush(), and few lookups/queries > during that process. > > But other endusers (including myself) could not do simple queries against > the database at all. Most of my queries contain query hints (readonly, > statement caching). > > Also, I never seen this behavior at all, but this is first time I added > @Stateless EJB along with @Schedule that does database updates 'during > business hours'. I thought this would be a no-brainer, but I guess it's > not. Again, the server is TomEE 1.5.1 (tomcat 7.0.34). > > Any advise, then please let me know. Onto reading this post now. Thanks. :) > > > > On Tue, Dec 11, 2012 at 10:49 AM, Julien Martin wrote: > >> Thank you very much for this exhaustive reply Christopher. >> >> 2012/12/11 Christopher Schultz >> >> > -BEGIN PGP SIGNED MESSAGE- >> > Hash: SHA256 >> > >> > Julien, >> > >> > Warning: this is long. Like, André-or-Mark-Eggers long. >> > >> > On 12/11/12 7:30 AM, Julien Martin wrote: >> > > I am in reference to the following blog entry: Blog >> > > entry< >> > >> http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support >> > > >> > > >> > > >> > about Spring MVC 3.2 asynchronous support. >> > > >> > > I understand Tomcat uses a thread pool in order to serve http/web >> > > requests. Furthermore, the above article seems to indicate that >> > > Spring MVC asynchronous support relieves Tomcat's thread pool and >> > > allows for better concurrency in the webapp by using background >> > > threads for "heavy-lift" operations. >> > >> > I believe you are misinterpreting what that post has to say. It's not >> > that a "background" thread itself is more efficient, it's that >> > processing that does not need to communicate with the client can be >> > de-coupled from the request-processing thread-pool that exists for >> > that purpose. >> > >> > An example - right from the blog post - will make much more sense than >> > what I wrote above. Let's take the example of sending an email >> > message. First, some assumptions: >> > >> > 1. Sending an email takes a long time (say, 5 seconds) >> > 2. The client does not need confirmation that the email has been sent >> > >> > If your were to write a "classic" servlet, it would look something >> > like this: >> > >> > doPost() { >> > validateOrder(); >> > >> > queueOrder(); >> > >> > sendOrderConfirmation(); // This is the email >> > >> > response.sendRedirect("/order_complete.jsp"); >> > } >> > >> > Let's say that validation takes 500ms, queuing takes 800ms, and >> > emailing (as above) takes 5000ms. That means that the request, from >> > the client perspective, takes 6300ms (6.3 sec). That's a noticeable >> delay. >> > >> > Also, during that whole time, a single request-processing thread (from >> > Tomcat's thread pool) is tied-up, meaning that no other requests can >> > be processed by that same thread. >> > >> > If you have a thread pool of size=1 (foolish, yet instructive), it >> > means you can only process a single transaction every 6.3 seconds. >> > >> > Lets re-write the servlet with a background thread -- no >> > "asynchronous" stuff from the servlet API, but just with a simple >> > background thread: >> > >> > doPost() { >> > validateOrder(); >> > >> > queueOrder(); >> > >> > (new Thread() { >> > public void run() { >> > sendOrderConfirmation(); >> > } >> > }).start(); >> > >> > response.sendRedirect("order_complete.jsp"); >> > } >> > >> > So, now the email is being sent by a backg
Re: Why are Tomcat's threads more costly than background threads?
Wow, i'm reading this now, because I just experienced an issue on my production server that is TomEE 1.5.1 (Tomcat 7.0.34), and the whole server locked up all because I had a @Stateless EJB inserting data into multiple tables in the database, because @Schedule timed event triggered the EJB to check email server for incoming (customer) requests, and it literally took down the server. I was on it as well as few other endusers, and then one enduser captured a LOCK error and the screen capture (photo/pic) had an error message that showed a long SQL query with datatable table and column names t0..., t0... What I primarily saw was the word 'lock' at the top of that, and we definitely experienced a lockup. I'm about to check server logs and read this article. The @Stateless EJB had one transaction (entitymanager / persistence context) that made database updates to multiple tables in the database. I am only using entitymanager.persist(), flush(), and few lookups/queries during that process. But other endusers (including myself) could not do simple queries against the database at all. Most of my queries contain query hints (readonly, statement caching). Also, I never seen this behavior at all, but this is first time I added @Stateless EJB along with @Schedule that does database updates 'during business hours'. I thought this would be a no-brainer, but I guess it's not. Again, the server is TomEE 1.5.1 (tomcat 7.0.34). Any advise, then please let me know. Onto reading this post now. Thanks. :) On Tue, Dec 11, 2012 at 10:49 AM, Julien Martin wrote: > Thank you very much for this exhaustive reply Christopher. > > 2012/12/11 Christopher Schultz > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA256 > > > > Julien, > > > > Warning: this is long. Like, André-or-Mark-Eggers long. > > > > On 12/11/12 7:30 AM, Julien Martin wrote: > > > I am in reference to the following blog entry: Blog > > > entry< > > > http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support > > > > > > > > > > > about Spring MVC 3.2 asynchronous support. > > > > > > I understand Tomcat uses a thread pool in order to serve http/web > > > requests. Furthermore, the above article seems to indicate that > > > Spring MVC asynchronous support relieves Tomcat's thread pool and > > > allows for better concurrency in the webapp by using background > > > threads for "heavy-lift" operations. > > > > I believe you are misinterpreting what that post has to say. It's not > > that a "background" thread itself is more efficient, it's that > > processing that does not need to communicate with the client can be > > de-coupled from the request-processing thread-pool that exists for > > that purpose. > > > > An example - right from the blog post - will make much more sense than > > what I wrote above. Let's take the example of sending an email > > message. First, some assumptions: > > > > 1. Sending an email takes a long time (say, 5 seconds) > > 2. The client does not need confirmation that the email has been sent > > > > If your were to write a "classic" servlet, it would look something > > like this: > > > > doPost() { > > validateOrder(); > > > > queueOrder(); > > > > sendOrderConfirmation(); // This is the email > > > > response.sendRedirect("/order_complete.jsp"); > > } > > > > Let's say that validation takes 500ms, queuing takes 800ms, and > > emailing (as above) takes 5000ms. That means that the request, from > > the client perspective, takes 6300ms (6.3 sec). That's a noticeable > delay. > > > > Also, during that whole time, a single request-processing thread (from > > Tomcat's thread pool) is tied-up, meaning that no other requests can > > be processed by that same thread. > > > > If you have a thread pool of size=1 (foolish, yet instructive), it > > means you can only process a single transaction every 6.3 seconds. > > > > Lets re-write the servlet with a background thread -- no > > "asynchronous" stuff from the servlet API, but just with a simple > > background thread: > > > > doPost() { > > validateOrder(); > > > > queueOrder(); > > > > (new Thread() { > > public void run() { > > sendOrderConfirmation(); > > } > > }).start(); > > > > response.sendRedirect("order_complete.jsp"); > > } > > > > So, now the email is being sent by a background thread: the response > > returns to the client after 1.3 seconds which is a significant > > improvement. Now, we can handle a request once every 1.3 seconds with > > a request-processing thread-pool of size=1. > > > > Note that a better implementation of the above would be to use a > > thread pool for this sort of thing instead of creating a new thread > > for every request. This is what Spring provides. It's not that Spring > > can do a better job of thread management, it's that Tomcat's thread > > pool is special: it's the only one that can actually dispatch client > > requests. Off-loading onto another thread pool for background > > pr