Re: Connection pooling again
Hmm. I think the answer here is one of resource starvation - the resource in this case being the CPU. It seems that core of your test is 'Select * from table1' Where is the database? Is it another application on the same box? What mechanism do you use to connect between the database and the app? (a pipe? tcp?) How big is 'table1'? How long does that select statement take? Why should anything else happen in the system until that select has finished? Here's a guess. Look at a resource monitor and you will see that when you start your test the CPU goes to 100% and stays there. I suspect that this is a case that you are queuing on processing the select statement in the database. From a systems point of view the Java JVM on windows and linux is a *cooperative* multithreaded system. Connection pooling has nothing to do with concurrency. Its as someone said earlier a performance optimization to avoid the overhead of recreating the datasource each time you need it. It you want to write a test which shows the connection pool being used CONCURRENTLY then you must write a concurrent test! For example, open the connection, sleep for a while, do something, sleep again and do something else and then close the connection. Turn on logging for the pool and watch concurrent requests being processed. Alan Chaney [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daad, [EMAIL PROTECTED] wrote: | No, my test is done with threads that connect using sockets. Server | listen with ServerSocket and 30 Threads are started connecting to it. Er... did you write your own server, or are you using Tomcat? I'm using tomcat. I'm sorry... all the evidence you have presented does not support your conclusions. There are other ways to get a log file looking like the one you posted. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkf9PnsACgkQ9CaO5/Lv0PBAPACfR8qL7tD33mFTWCdY99D6p8ic shoAn3v0C49JvF2muOefUF5Q+QbBhVdD =kLLa -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] !DSPAM:47fddbcd240221497319589! - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Daad, > > [EMAIL PROTECTED] wrote: > | No, my test is done with threads that connect using sockets. Server > | listen with ServerSocket and 30 Threads are started connecting to it. > > Er... did you write your own server, or are you using Tomcat? > I'm using tomcat. > I'm sorry... all the evidence you have presented does not support your > conclusions. There are other ways to get a log file looking like the one > you posted. > > - -chris > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkf9PnsACgkQ9CaO5/Lv0PBAPACfR8qL7tD33mFTWCdY99D6p8ic > shoAn3v0C49JvF2muOefUF5Q+QbBhVdD > =kLLa > -END PGP SIGNATURE- > > - > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
> I haven't looked at the source, but I'm willing to bet there is at least > one sync block -- the pooling implementation would have to perform a > brief sync when it borrows a connection object from the pool. In > addition, if this is from a fresh startup of tomcat without a minIdle > setting (default=0) in your Resource definition (I didn't see one in > your original post), each parallel call to getConnection() will mean > creating a new connection to the db because all previous connections are > busy and none are being reused. > > I'd be curious what this would look like with: > a) a minIdle setting to establish a pool of connections on startup and I tried that option and the behaviour doesn't change.. > b) if the threads kept going over a few iterations releasing and > reacquiring connections. Would the time keep going up or would it drop > as connections start getting reused? > > --David > > [EMAIL PROTECTED] wrote: > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA1 > >> > >> Daad, > >> > >> [EMAIL PROTECTED] wrote: > >> | i think i don't get the utility of a connection pooling, since i have > >> this situation: 30 threads try to perform at same time a db access with > >> the call: > >> | > >> | new Database().doSomething() > >> > >> Are you sure you're using your Database class in this way? I wouldn't be > >> surprised if you are creating a single Database object and using it > >> repeatedly. > >> > > > > Yes i'm sure, i'm using (new Database()).doSomething() always. > > > > > >> Are your threads actually running in parallel? > >> > >> > > > > That's my question. > > > > > >> | [wed, 09 apr 2008 00:55:03] INFO Database -> doSomething: 836 > >> | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 1444 > >> | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 2054 > >> | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 2417 > >> | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 3060 > >> | [wed, 09 apr 2008 00:55:06] INFO Database -> doSomething: 3647 > >> | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4279 > >> | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4967 > >> | [wed, 09 apr 2008 00:55:08] INFO Database -> doSomething: 5592 > >> | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 6208 > >> | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 7026 > >> | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 7455 > >> | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 8087 > >> | [wed, 09 apr 2008 00:55:11] INFO Database -> doSomething: 8705 > >> | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 9318 > >> | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 10040 > >> | [wed, 09 apr 2008 00:55:13] INFO Database -> doSomething: 11146 > >> | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 11702 > >> | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 12033 > >> | [wed, 09 apr 2008 00:55:15] INFO Database -> doSomething: 12658 > >> | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13279 > >> | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13897 > >> | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 14523 > >> | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 15139 > >> | [wed, 09 apr 2008 00:55:18] INFO Database -> doSomething: 15759 > >> | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 16411 > >> | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 17056 > >> | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 17672 > >> | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 18292 > >> > >> Every single operation takes more time than the previous operation. That > >> smells of a problem with resetting a counter. Isn't it interesting that > >> the logging statements show that all 30 threads completed within 18 > >> seconds of each other, and each thread says that it took up to 18 > >> seconds to complete? If these processes were sharing a single > >> connection, and they were taking 18 seconds to complete, the total time > >> would be 9 minutes, not 18 seconds. > >> > >> > > > > As you can see from the code i pasted, start time is taken in > > getConnection() method and end time is taken in closeResources() method. So > > from the logs i understand that all 30 threads arrive in getConnection() at > > same time (so all "start" variable of each Database instance is set to same > > timestamp (about)), but then seems that operation get enqueued, since the > > first thread that is able to quit needs 836ms, while the last one 18292ms. > > That means the second thread that can perform the query waited that first > > thread complete the query. So, even if the second thread arrived to > > getConnection() at same time of the first, the second took almost double > > time because waited that the first complete the task. That's what i > > understood from logs... I didn't get what you said about 9
Re: Connection pooling again
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daad, [EMAIL PROTECTED] wrote: | No, my test is done with threads that connect using sockets. Server | listen with ServerSocket and 30 Threads are started connecting to it. Er... did you write your own server, or are you using Tomcat? I'm sorry... all the evidence you have presented does not support your conclusions. There are other ways to get a log file looking like the one you posted. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkf9PnsACgkQ9CaO5/Lv0PBAPACfR8qL7tD33mFTWCdY99D6p8ic shoAn3v0C49JvF2muOefUF5Q+QbBhVdD =kLLa -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
I haven't looked at the source, but I'm willing to bet there is at least one sync block -- the pooling implementation would have to perform a brief sync when it borrows a connection object from the pool. In addition, if this is from a fresh startup of tomcat without a minIdle setting (default=0) in your Resource definition (I didn't see one in your original post), each parallel call to getConnection() will mean creating a new connection to the db because all previous connections are busy and none are being reused. I'd be curious what this would look like with: a) a minIdle setting to establish a pool of connections on startup and b) if the threads kept going over a few iterations releasing and reacquiring connections. Would the time keep going up or would it drop as connections start getting reused? --David [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daad, [EMAIL PROTECTED] wrote: | i think i don't get the utility of a connection pooling, since i have this situation: 30 threads try to perform at same time a db access with the call: | | new Database().doSomething() Are you sure you're using your Database class in this way? I wouldn't be surprised if you are creating a single Database object and using it repeatedly. Yes i'm sure, i'm using (new Database()).doSomething() always. Are your threads actually running in parallel? That's my question. | [wed, 09 apr 2008 00:55:03] INFO Database -> doSomething: 836 | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 1444 | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 2054 | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 2417 | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 3060 | [wed, 09 apr 2008 00:55:06] INFO Database -> doSomething: 3647 | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4279 | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4967 | [wed, 09 apr 2008 00:55:08] INFO Database -> doSomething: 5592 | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 6208 | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 7026 | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 7455 | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 8087 | [wed, 09 apr 2008 00:55:11] INFO Database -> doSomething: 8705 | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 9318 | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 10040 | [wed, 09 apr 2008 00:55:13] INFO Database -> doSomething: 11146 | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 11702 | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 12033 | [wed, 09 apr 2008 00:55:15] INFO Database -> doSomething: 12658 | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13279 | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13897 | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 14523 | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 15139 | [wed, 09 apr 2008 00:55:18] INFO Database -> doSomething: 15759 | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 16411 | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 17056 | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 17672 | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 18292 Every single operation takes more time than the previous operation. That smells of a problem with resetting a counter. Isn't it interesting that the logging statements show that all 30 threads completed within 18 seconds of each other, and each thread says that it took up to 18 seconds to complete? If these processes were sharing a single connection, and they were taking 18 seconds to complete, the total time would be 9 minutes, not 18 seconds. As you can see from the code i pasted, start time is taken in getConnection() method and end time is taken in closeResources() method. So from the logs i understand that all 30 threads arrive in getConnection() at same time (so all "start" variable of each Database instance is set to same timestamp (about)), but then seems that operation get enqueued, since the first thread that is able to quit needs 836ms, while the last one 18292ms. That means the second thread that can perform the query waited that first thread complete the query. So, even if the second thread arrived to getConnection() at same time of the first, the second took almost double time because waited that the first complete the task. That's what i understood from logs... I didn't get what you said about 9 minutes. If the operation takes 600-800ms (as first thread demonstrate), 30 x 600 = 18000ms. So with one connection, if those 30 threads are enqueued, the total time is 18 seconds, that is what happens. So i still think that one connection is shared only =\ If anybod
Re: Connection pooling again
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Daad, > > [EMAIL PROTECTED] wrote: > | i think i don't get the utility of a connection pooling, since i have > this situation: 30 threads try to perform at same time a db access with > the call: > | > | new Database().doSomething() > > Are you sure you're using your Database class in this way? I wouldn't be > surprised if you are creating a single Database object and using it > repeatedly. Yes i'm sure, i'm using (new Database()).doSomething() always. > > Are your threads actually running in parallel? > That's my question. > | [wed, 09 apr 2008 00:55:03] INFODatabase -> doSomething: 836 > | [wed, 09 apr 2008 00:55:04] INFODatabase -> doSomething: 1444 > | [wed, 09 apr 2008 00:55:04] INFODatabase -> doSomething: 2054 > | [wed, 09 apr 2008 00:55:05] INFODatabase -> doSomething: 2417 > | [wed, 09 apr 2008 00:55:05] INFODatabase -> doSomething: 3060 > | [wed, 09 apr 2008 00:55:06] INFODatabase -> doSomething: 3647 > | [wed, 09 apr 2008 00:55:07] INFODatabase -> doSomething: 4279 > | [wed, 09 apr 2008 00:55:07] INFODatabase -> doSomething: 4967 > | [wed, 09 apr 2008 00:55:08] INFODatabase -> doSomething: 5592 > | [wed, 09 apr 2008 00:55:09] INFODatabase -> doSomething: 6208 > | [wed, 09 apr 2008 00:55:09] INFODatabase -> doSomething: 7026 > | [wed, 09 apr 2008 00:55:10] INFODatabase -> doSomething: 7455 > | [wed, 09 apr 2008 00:55:10] INFODatabase -> doSomething: 8087 > | [wed, 09 apr 2008 00:55:11] INFODatabase -> doSomething: 8705 > | [wed, 09 apr 2008 00:55:12] INFODatabase -> doSomething: 9318 > | [wed, 09 apr 2008 00:55:12] INFODatabase -> doSomething: 10040 > | [wed, 09 apr 2008 00:55:13] INFODatabase -> doSomething: 11146 > | [wed, 09 apr 2008 00:55:14] INFODatabase -> doSomething: 11702 > | [wed, 09 apr 2008 00:55:14] INFODatabase -> doSomething: 12033 > | [wed, 09 apr 2008 00:55:15] INFODatabase -> doSomething: 12658 > | [wed, 09 apr 2008 00:55:16] INFODatabase -> doSomething: 13279 > | [wed, 09 apr 2008 00:55:16] INFODatabase -> doSomething: 13897 > | [wed, 09 apr 2008 00:55:17] INFODatabase -> doSomething: 14523 > | [wed, 09 apr 2008 00:55:17] INFODatabase -> doSomething: 15139 > | [wed, 09 apr 2008 00:55:18] INFODatabase -> doSomething: 15759 > | [wed, 09 apr 2008 00:55:19] INFODatabase -> doSomething: 16411 > | [wed, 09 apr 2008 00:55:19] INFODatabase -> doSomething: 17056 > | [wed, 09 apr 2008 00:55:20] INFODatabase -> doSomething: 17672 > | [wed, 09 apr 2008 00:55:20] INFODatabase -> doSomething: 18292 > > Every single operation takes more time than the previous operation. That > smells of a problem with resetting a counter. Isn't it interesting that > the logging statements show that all 30 threads completed within 18 > seconds of each other, and each thread says that it took up to 18 > seconds to complete? If these processes were sharing a single > connection, and they were taking 18 seconds to complete, the total time > would be 9 minutes, not 18 seconds. > As you can see from the code i pasted, start time is taken in getConnection() method and end time is taken in closeResources() method. So from the logs i understand that all 30 threads arrive in getConnection() at same time (so all "start" variable of each Database instance is set to same timestamp (about)), but then seems that operation get enqueued, since the first thread that is able to quit needs 836ms, while the last one 18292ms. That means the second thread that can perform the query waited that first thread complete the query. So, even if the second thread arrived to getConnection() at same time of the first, the second took almost double time because waited that the first complete the task. That's what i understood from logs... I didn't get what you said about 9 minutes. If the operation takes 600-800ms (as first thread demonstrate), 30 x 600 = 18000ms. So with one connection, if those 30 threads are enqueued, the total time is 18 seconds, that is what happens. So i still think that one connection is shared only =\ If anybody know what to check on Mysql application to see if there is a sort of limitation on number of concurrent queries, please let me know. > Even if you had resource contention (which you shouldn't have with 600 > shared connections), I would expect that some numbers would be low and > others were high -- essentially randomly distributed. > > It's pretty clear to me that something else is going on... I'm guessing > that the connection pool (which one are you using?) is working just > fine, but either your "load test" code or your server-side database use > is being incorrectly used and/or incorrectly instrumented. > I'm using mysql-connector-java-5.1.5-bin.jar > | so i can't understand the meaning of maxActive parameter.. i thought > | that could be at most 600 concurrent connection. > > ~From the documentation: "The maximum nu
Re: Connection pooling again
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Daad, > > [EMAIL PROTECTED] wrote: > | I missed some messages seems.. So is the enqueuing thing a bad > | behaviour of pooling or is it just normal? > > I suspect you have a problem with your code, somewhere. Are you creating > a new Database object every time? Do you have any synchronization or any > kind? Yes, i create Database object each time, and no, i removed synchronization blocks from the piece of code that call the doSomething() method. As you can see from logs i posted, i think that every thread enter in getConnection() at same time, and then they enqueue running one per time. So if there is no synchronization on Database class, i think there is no synchronized block problem, isn't it? > > - -chris > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkf84p0ACgkQ9CaO5/Lv0PDIYwCcD0xlEpOMqSDHCz5zFUXMy6sV > HCkAn2HNwdYkggG6o1flpgKZ6X/RD7qd > =aqt8 > -END PGP SIGNATURE- > > - > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
> > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > > Subject: RE: Connection pooling again > > > > what should i ask to check to the db admin? > > Start with: does the db limit the number of connections from a given > client (which in this case, is the box Tomcat is running on)? Is there a specific option to check this thing? more precise i can be with db admin, faster we get the answer.. > > > How? Should i use Jconsole.exe program? > > The jstack program is better, since it gets all threads at once. AFAIK, > you can only do them one at a time with jconsole. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Subject: RE: Connection pooling again > > what should i ask to check to the db admin? Start with: does the db limit the number of connections from a given client (which in this case, is the box Tomcat is running on)? > How? Should i use Jconsole.exe program? The jstack program is better, since it gets all threads at once. AFAIK, you can only do them one at a time with jconsole. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
> Your queueing problem most likely has nothing to do with the pooling > capability but rather with your app, your database, or your test driver. > > 1) If the test driver is browser based (e.g., Java script), you're > limited to two connections to a given server. If your test driver is > using some generic HTTP software, that may also have the two connection > limit built in, but it's less likely. > No, my test is done with threads that connect using sockets. Server listen with ServerSocket and 30 Threads are started connecting to it. > 2) Does your database actually support multiple connections? Are you > running into locking in the data base that's forcing serialization of > requests? > Since i'm not the owner of the db part, how can i check it? what should i ask to check to the db admin? > 3) Does your app use some sort of locking that is essentially making > everything go through one request at a time? > No, synchronized block are removed from that part, so there should not be any concurrency limitation on that side > Take a thread dump during the 18-second window and see where the threads > are. > How? Should i use Jconsole.exe program? > - Chuck > thanks - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Subject: Re: Connection pooling again > > So is the enqueuing thing a bad behaviour of pooling or is it > just normal? Your queueing problem most likely has nothing to do with the pooling capability but rather with your app, your database, or your test driver. 1) If the test driver is browser based (e.g., Java script), you're limited to two connections to a given server. If your test driver is using some generic HTTP software, that may also have the two connection limit built in, but it's less likely. 2) Does your database actually support multiple connections? Are you running into locking in the data base that's forcing serialization of requests? 3) Does your app use some sort of locking that is essentially making everything go through one request at a time? Take a thread dump during the 18-second window and see where the threads are. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daad, [EMAIL PROTECTED] wrote: | I missed some messages seems.. So is the enqueuing thing a bad | behaviour of pooling or is it just normal? I suspect you have a problem with your code, somewhere. Are you creating a new Database object every time? Do you have any synchronization or any kind? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkf84p0ACgkQ9CaO5/Lv0PDIYwCcD0xlEpOMqSDHCz5zFUXMy6sV HCkAn2HNwdYkggG6o1flpgKZ6X/RD7qd =aqt8 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
I missed some messages seems..
So is the enqueuing thing a bad behaviour of pooling or is it just normal? I
don't know anymore what to expect from db pooling at this point :(
I started a forum thread there
http://forum.java.sun.com/thread.jspa?threadID=5274763&tstart=0 too, if anybody
want to partecipate... thanks
-- Initial Header ---
>From : "Christopher Schultz" [EMAIL PROTECTED]
To : "Tomcat Users List" users@tomcat.apache.org
Cc :
Date : Wed, 09 Apr 2008 10:58:58 -0400
Subject : Re: Connection pooling again
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Mikolaj,
>
> Mikolaj Rydzewski wrote:
> | JNDI lookups are expensive.
>
> No, they're not. We're not talking about using a remote JNDI server or
> AD or anything like that. This is all local and the lookups are very fast.
>
> To convince yourself, run this simple JSP:
>
> <%@ page
> ~import="javax.naming.*"
> %>
> <%
> ~int iterations = 1;
> ~long start = System.currentTimeMillis();
>
> ~for(int i=0; i ~{
> ~Context ctx = new InitialContext();
> ~ctx.lookup("java:comp/env/jdbc/diagnosis");
> ~}
>
> ~long elapsed = System.currentTimeMillis() - start;
> %>
> <%= iterations %> lookups took <%= elapsed %>ms, average = <%=
> (double)elapsed / (double)iterations %>ms.
>
> On my dev machine (AMD Athlon XP 1700+, default 64MB heap), these
> statistics are printed by this JSP:
>
> 1 lookups took 442ms, average = 0.0442ms.
>
> That's pretty fast. Database queries are slow, not JNDI lookups.
>
> | You should get DataSource object from JNDI
> | first, and then pass it to other threads. Every thread should just call
> | DataSource.getConnection().
>
> That's a bad idea. The reason you store the DataSource in JNDI is so
> that it is universally available and replaceable. Caching the DataSource
> somewhere else is counter-productive.
>
> - -chris
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkf82bIACgkQ9CaO5/Lv0PC5jACgkAWrgv4G1nbbxm8SGUJxnHsa
> 4fUAoMLU6W4lGlO5khaI2xwfbWLMpvk8
> =j/sF
> -END PGP SIGNATURE-
>
> -
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
Christopher Schultz wrote: Mikolaj Rydzewski wrote: | You should get DataSource object from JNDI | first, and then pass it to other threads. Every thread should just call | DataSource.getConnection(). That's a bad idea. The reason you store the DataSource in JNDI is so that it is universally available and replaceable. Caching the DataSource somewhere else is counter-productive. I'd rather get such objects from JNDI during application startup, and implement functionality to 'reload' them on demand. In general you don't know whether JNDI call will be local or remote. -- Mikolaj Rydzewski <[EMAIL PROTECTED]> - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mikolaj, Mikolaj Rydzewski wrote: | JNDI lookups are expensive. No, they're not. We're not talking about using a remote JNDI server or AD or anything like that. This is all local and the lookups are very fast. To convince yourself, run this simple JSP: <%@ page ~import="javax.naming.*" %> <% ~int iterations = 1; ~long start = System.currentTimeMillis(); ~for(int i=0; i <%= iterations %> lookups took <%= elapsed %>ms, average = <%= (double)elapsed / (double)iterations %>ms. On my dev machine (AMD Athlon XP 1700+, default 64MB heap), these statistics are printed by this JSP: 1 lookups took 442ms, average = 0.0442ms. That's pretty fast. Database queries are slow, not JNDI lookups. | You should get DataSource object from JNDI | first, and then pass it to other threads. Every thread should just call | DataSource.getConnection(). That's a bad idea. The reason you store the DataSource in JNDI is so that it is universally available and replaceable. Caching the DataSource somewhere else is counter-productive. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkf82bIACgkQ9CaO5/Lv0PC5jACgkAWrgv4G1nbbxm8SGUJxnHsa 4fUAoMLU6W4lGlO5khaI2xwfbWLMpvk8 =j/sF -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
[EMAIL PROTECTED] wrote: i think i don't get the utility of a connection pooling, since i have this situation: 30 threads try to perform at same time a db access with the call: new Database().doSomething() JNDI lookups are expensive. You should get DataSource object from JNDI first, and then pass it to other threads. Every thread should just call DataSource.getConnection(). -- Mikolaj Rydzewski <[EMAIL PROTECTED]> - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Connection pooling again
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daad, [EMAIL PROTECTED] wrote: | i think i don't get the utility of a connection pooling, since i have this situation: 30 threads try to perform at same time a db access with the call: | | new Database().doSomething() Are you sure you're using your Database class in this way? I wouldn't be surprised if you are creating a single Database object and using it repeatedly. Are your threads actually running in parallel? | [wed, 09 apr 2008 00:55:03] INFO Database -> doSomething: 836 | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 1444 | [wed, 09 apr 2008 00:55:04] INFO Database -> doSomething: 2054 | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 2417 | [wed, 09 apr 2008 00:55:05] INFO Database -> doSomething: 3060 | [wed, 09 apr 2008 00:55:06] INFO Database -> doSomething: 3647 | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4279 | [wed, 09 apr 2008 00:55:07] INFO Database -> doSomething: 4967 | [wed, 09 apr 2008 00:55:08] INFO Database -> doSomething: 5592 | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 6208 | [wed, 09 apr 2008 00:55:09] INFO Database -> doSomething: 7026 | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 7455 | [wed, 09 apr 2008 00:55:10] INFO Database -> doSomething: 8087 | [wed, 09 apr 2008 00:55:11] INFO Database -> doSomething: 8705 | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 9318 | [wed, 09 apr 2008 00:55:12] INFO Database -> doSomething: 10040 | [wed, 09 apr 2008 00:55:13] INFO Database -> doSomething: 11146 | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 11702 | [wed, 09 apr 2008 00:55:14] INFO Database -> doSomething: 12033 | [wed, 09 apr 2008 00:55:15] INFO Database -> doSomething: 12658 | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13279 | [wed, 09 apr 2008 00:55:16] INFO Database -> doSomething: 13897 | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 14523 | [wed, 09 apr 2008 00:55:17] INFO Database -> doSomething: 15139 | [wed, 09 apr 2008 00:55:18] INFO Database -> doSomething: 15759 | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 16411 | [wed, 09 apr 2008 00:55:19] INFO Database -> doSomething: 17056 | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 17672 | [wed, 09 apr 2008 00:55:20] INFO Database -> doSomething: 18292 Every single operation takes more time than the previous operation. That smells of a problem with resetting a counter. Isn't it interesting that the logging statements show that all 30 threads completed within 18 seconds of each other, and each thread says that it took up to 18 seconds to complete? If these processes were sharing a single connection, and they were taking 18 seconds to complete, the total time would be 9 minutes, not 18 seconds. Even if you had resource contention (which you shouldn't have with 600 shared connections), I would expect that some numbers would be low and others were high -- essentially randomly distributed. It's pretty clear to me that something else is going on... I'm guessing that the connection pool (which one are you using?) is working just fine, but either your "load test" code or your server-side database use is being incorrectly used and/or incorrectly instrumented. | so i can't understand the meaning of maxActive parameter.. i thought | that could be at most 600 concurrent connection. ~From the documentation: "The maximum number of active connections that can be allocated from this pool at the same time, or negative for no limit." Your expectation that 600 connections should be available is correct. Although, 600 connections in a connection pool is a /lot/ of connections. ;) - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkf8yAkACgkQ9CaO5/Lv0PAGtgCeL5O0jUrxfZZZM/3ix3pbmrWV kgcAnR1wlDWpS9ez9FE/uNHzKptqTCmc =mSOU -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
It's not a matter of web connection, the server listen with serverSocket on a port, so i made a test application that connect with 30 threads. What i can't understand is: connection pooling shouldn't perform 'x' concurrent connections? Why instead enqueue it? Is it because i do from same ip (mine)? -- Initial Header --- >From : "Caldarale, Charles R" [EMAIL PROTECTED] To : "Tomcat Users List" users@tomcat.apache.org Cc : Date : Wed, 9 Apr 2008 06:18:40 -0500 Subject : RE: Connection pooling again > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > > Subject: Connection pooling again > > > > i think i don't get the utility of a connection pooling, > > since i have this situation: 30 threads try to perform at > > same time a db access with the call: > > Are you running the 30 client requests from a single browser? If so, > you are likely hitting the HTTP RFC recommended limit of two concurrent > connections to a given server. You can tweak browser settings to raise > the limit, or try it with a more appropriate testing tool. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Connection pooling again
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Subject: Connection pooling again > > i think i don't get the utility of a connection pooling, > since i have this situation: 30 threads try to perform at > same time a db access with the call: Are you running the 30 client requests from a single browser? If so, you are likely hitting the HTTP RFC recommended limit of two concurrent connections to a given server. You can tweak browser settings to raise the limit, or try it with a more appropriate testing tool. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
