RE: ** JVM and Processes
Randy, Thanks for the advice. Could you be a little more specific, though, about how to use green threads instead of native threads and possibly differences between the two? Thanks. - Adam At 10:59 AM 6/1/2001 -0400, you wrote: Don't use ps - these are actually threads. ps is showing them as processes because that is what it does. If you use green thread (as opposed to the native threads you are using now), the display will go away, but you will experience a slowdown (how much depends on your operating system and other activity on the system). Randy -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, June 01, 2001 10:37 AM To: [EMAIL PROTECTED] Subject: ** JVM and Processes Hi, For a particular web server we are running with Tomcat 3.1, we are having an issue with the java servlets that are running. What appears to be happening is that each time a servlet is called from the web site, a new process is created to run the java program. When I view processes with ps ax, I see dozens of instances of: /usr/java/jdk1.3/bin/i386/native_threads/java It was briefly stated in Java Servlet Programming by Hunter Crawford, (c) Oreilly that 'most servlet containers execute all servlets in a single JVM ... the exception being high-end containers that support execution across multiple backend servers...' We are only using 1 web server with an average weekly load of a couple of hundred visitors. Any ideas as to why we would be seeing so many identical processes and if so, how to modify that? Thanks in advance. -Adam
RE: ** JVM and Processes
Title: RE: ** JVM and Processes http://java.sun.com/products/jdk/1.1/packs/native-threads/README -Message d'origine- De: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Date: vendredi 1 juin 2001 17:46 À: [EMAIL PROTECTED] Objet: RE: ** JVM and Processes Randy, Thanks for the advice. Could you be a little more specific, though, about how to use green threads instead of native threads and possibly differences between the two? Thanks. - Adam At 10:59 AM 6/1/2001 -0400, you wrote: Don't use ps - these are actually threads. ps is showing them as processes because that is what it does. If you use green thread (as opposed to the native threads you are using now), the display will go away, but you will experience a slowdown (how much depends on your operating system and other activity on the system). Randy -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, June 01, 2001 10:37 AM To: [EMAIL PROTECTED] Subject: ** JVM and Processes Hi, For a particular web server we are running with Tomcat 3.1, we are having an issue with the java servlets that are running. What appears to be happening is that each time a servlet is called from the web site, a new process is created to run the java program. When I view processes with ps ax, I see dozens of instances of: /usr/java/jdk1.3/bin/i386/native_threads/java It was briefly stated in Java Servlet Programming by Hunter Crawford, (c) Oreilly that 'most servlet containers execute all servlets in a single JVM ... the exception being high-end containers that support execution across multiple backend servers...' We are only using 1 web server with an average weekly load of a couple of hundred visitors. Any ideas as to why we would be seeing so many identical processes and if so, how to modify that? Thanks in advance. -Adam
Re: ** JVM and Processes
Title: RE: ** JVM and Processes
My understanding of green vs. native threads is as
follows:
With native threads, an actual system thread is
created when a Java thread is created.
On linux a system thread takes the form of another
process, but one that shares memory
etc. with another process. This is why if you
create a program that allocates 100 megs of memory,
then spins off 10 threads, it looks like 10
processes each taking up 100 megs of memory, when in
fact the amount of memory is 100 megs + 10*overhead
for each thread (not much more than 100 megs).
On WIN32 systems, threads do not show up as
separate processes, they are separate threads of execution
inside the same process (essentially the same as
the Linux implementation with differences too subtle to care about)
Green threads on the other hand use timers,
signals, setjmp etc. voodoo to "simulate" threads within one
process.
Essentially taking over the scheduling from the
kernel.
I believe the command-line option for green threads
is simply "-green" as in
java -green MyThreaddedApp
If you have a multi-cpu system, green threads will
only take advantage of one cpu, whereas native threads
will use all the cpus on your system (that's the
theory anyway)
I've heard of problems with blocking I/O with green
threads, but have no first hand knowledge.
Hope this helps.
-Mike Jennings
- Original Message -
From:
BARRAUD Valérie
To: '[EMAIL PROTECTED]'
Sent: Friday, June 01, 2001 9:01 AM
Subject: RE: ** JVM and Processes
http://java.sun.com/products/jdk/1.1/packs/native-threads/README
-Message d'origine- De: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]]
Date: vendredi 1 juin 2001 17:46 À: [EMAIL PROTECTED]
Objet: RE: ** JVM and Processes
Randy,
Thanks for the advice. Could you be a little more
specific, though, about how to use green
threads instead of native threads and possibly differences between the two? Thanks.
- Adam
At 10:59 AM 6/1/2001 -0400, you wrote:
Don't use ps - these are
actually threads. ps is showing them as processes because that is what it does. If you use green
thread (as opposed to the native
threads you are using now), the display will go away, but you
will experience a slowdown (how much depends
on your operating system and other
activity on the system).
Randy-Original
Message- From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent:
Friday, June 01, 2001 10:37 AM
To: [EMAIL PROTECTED] Subject: ** JVM and ProcessesHi,
For a particular web server we
are running with Tomcat 3.1, we
are having an issue with the
java servlets that are running. What appears to be happening is that each time a servlet is called
from the web site, a new
process is created to run the java
program. When I view processes
with "ps ax", I see dozens of
instances of:
/usr/java/jdk1.3/bin/i386/native_threads/java It was
briefly stated in Java Servlet Programming by Hunter Crawford, (c) Oreilly that 'most servlet containers execute all servlets
in a single JVM ... the exception being high-end containers that
support execution across
multiple backend servers...'
We are only using 1 web server with an average weekly load
of a couple of hundred visitors. Any ideas as to
why we would be seeing so many identical processes and if
so, how to modify that?
Thanks in advance.
-Adam
Re: ** JVM and Processes
When Java first came to the Linux platform (via the Blackdown port), green-threads were the only option. Native threads took a little longer to implement, but are a much better option for the reasons listed in the previous message. So, I would recommend avoiding green-threads unless you have a specific reason for using them. A lot of people freak out when they see the number of processes being reported by ps or top, without realizing that these are merely threads and not full-blown processes. If you have a lightly loaded Tomcat, you can tune down the number of threads being spawned by using the max_threads, max_spare_threads, and min_spare_threads parameters of the PoolTCPConnector in your server.xml file. For an example of this, take a look at the tomcat user's guide: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/index.html Do a find in your web browser for max_threads. I use this to limit the number of ajp12 threads and maximize ajp13 threads -- because I'm using ajp13 for my servlets and ajp12 only for startup/shutdown of Tomcat. Conversely, if you have a heavily loaded Tomcat, you should use these parameters to increase performance. Thanks, --jeff - Original Message - From: Michael Jennings [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 9:32 AM Subject: Re: ** JVM and Processes RE: ** JVM and ProcessesMy understanding of green vs. native threads is as follows: With native threads, an actual system thread is created when a Java thread is created. On linux a system thread takes the form of another process, but one that shares memory etc. with another process. This is why if you create a program that allocates 100 megs of memory, then spins off 10 threads, it looks like 10 processes each taking up 100 megs of memory, when in fact the amount of memory is 100 megs + 10*overhead for each thread (not much more than 100 megs). On WIN32 systems, threads do not show up as separate processes, they are separate threads of execution inside the same process (essentially the same as the Linux implementation with differences too subtle to care about) Green threads on the other hand use timers, signals, setjmp etc. voodoo to simulate threads within one process. Essentially taking over the scheduling from the kernel. I believe the command-line option for green threads is simply -green as in java -green MyThreaddedApp If you have a multi-cpu system, green threads will only take advantage of one cpu, whereas native threads will use all the cpus on your system (that's the theory anyway) I've heard of problems with blocking I/O with green threads, but have no first hand knowledge. Hope this helps. -Mike Jennings - Original Message - From: BARRAUD Valérie To: '[EMAIL PROTECTED]' Sent: Friday, June 01, 2001 9:01 AM Subject: RE: ** JVM and Processes http://java.sun.com/products/jdk/1.1/packs/native-threads/README -Message d'origine- De: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Date: vendredi 1 juin 2001 17:46 À: [EMAIL PROTECTED] Objet: RE: ** JVM and Processes Randy, Thanks for the advice. Could you be a little more specific, though, about how to use green threads instead of native threads and possibly differences between the two? Thanks. - Adam At 10:59 AM 6/1/2001 -0400, you wrote: Don't use ps - these are actually threads. ps is showing them as processes because that is what it does. If you use green thread (as opposed to the native threads you are using now), the display will go away, but you will experience a slowdown (how much depends on your operating system and other activity on the system). Randy -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, June 01, 2001 10:37 AM To: [EMAIL PROTECTED] Subject: ** JVM and Processes Hi, For a particular web server we are running with Tomcat 3.1, we are having an issue with the java servlets that are running. What appears to be happening is that each time a servlet is called from the web site, a new process is created to run the java program. When I view processes with ps ax, I see dozens of instances of: /usr/java/jdk1.3/bin/i386/native_threads/java It was briefly stated in Java Servlet Programming by Hunter Crawford, (c) Oreilly that 'most servlet containers execute all servlets in a single JVM ... the exception being high-end containers that support execution across multiple backend servers...' We are only using 1 web server with an average weekly load of a couple of hundred visitors. Any ideas as to why we would be seeing so many identical processes and if so, how to modify that? Thanks in advance. -Adam
Re: ** JVM and Processes
Jeff, Thanks a bunch. Your answer appears to be the best so far. I have implemented the PoolTCPConnector in the server xml file and it appears to be limiting the number of threads as it should. However, something that has been happening (even before switching to PoolTCPConnector) is that when running multiple java servlets the threads stay alive long after they should have died or been garbage collected. Even after a long wait, the only way (apparently) to get rid of them is to go through and kill them one at a time. Is there a setting somewhere that is telling the java threads to stay alive indefinitely? Thanks for your help, - Adam At 10:34 AM 6/1/2001 -0700, you wrote: When Java first came to the Linux platform (via the Blackdown port), green-threads were the only option. Native threads took a little longer to implement, but are a much better option for the reasons listed in the previous message. So, I would recommend avoiding green-threads unless you have a specific reason for using them. A lot of people freak out when they see the number of processes being reported by ps or top, without realizing that these are merely threads and not full-blown processes. If you have a lightly loaded Tomcat, you can tune down the number of threads being spawned by using the max_threads, max_spare_threads, and min_spare_threads parameters of the PoolTCPConnector in your server.xml file. For an example of this, take a look at the tomcat user's guide: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/index.html Do a find in your web browser for max_threads. I use this to limit the number of ajp12 threads and maximize ajp13 threads -- because I'm using ajp13 for my servlets and ajp12 only for startup/shutdown of Tomcat. Conversely, if you have a heavily loaded Tomcat, you should use these parameters to increase performance. Thanks, --jeff - Original Message - From: Michael Jennings [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 9:32 AM Subject: Re: ** JVM and Processes RE: ** JVM and ProcessesMy understanding of green vs. native threads is as follows: With native threads, an actual system thread is created when a Java thread is created. On linux a system thread takes the form of another process, but one that shares memory etc. with another process. This is why if you create a program that allocates 100 megs of memory, then spins off 10 threads, it looks like 10 processes each taking up 100 megs of memory, when in fact the amount of memory is 100 megs + 10*overhead for each thread (not much more than 100 megs). On WIN32 systems, threads do not show up as separate processes, they are separate threads of execution inside the same process (essentially the same as the Linux implementation with differences too subtle to care about) Green threads on the other hand use timers, signals, setjmp etc. voodoo to simulate threads within one process. Essentially taking over the scheduling from the kernel. I believe the command-line option for green threads is simply -green as in java -green MyThreaddedApp If you have a multi-cpu system, green threads will only take advantage of one cpu, whereas native threads will use all the cpus on your system (that's the theory anyway) I've heard of problems with blocking I/O with green threads, but have no first hand knowledge. Hope this helps. -Mike Jennings - Original Message - From: BARRAUD Valérie To: '[EMAIL PROTECTED]' Sent: Friday, June 01, 2001 9:01 AM Subject: RE: ** JVM and Processes http://java.sun.com/products/jdk/1.1/packs/native-threads/README -Message d'origine- De: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Date: vendredi 1 juin 2001 17:46 À: [EMAIL PROTECTED] Objet: RE: ** JVM and Processes Randy, Thanks for the advice. Could you be a little more specific, though, about how to use green threads instead of native threads and possibly differences between the two? Thanks. - Adam At 10:59 AM 6/1/2001 -0400, you wrote: Don't use ps - these are actually threads. ps is showing them as processes because that is what it does. If you use green thread (as opposed to the native threads you are using now), the display will go away, but you will experience a slowdown (how much depends on your operating system and other activity on the system). Randy -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, June 01, 2001 10:37 AM To: [EMAIL PROTECTED] Subject: ** JVM and Processes Hi, For a particular web server we are running with Tomcat 3.1, we are having an issue with the java servlets that are running. What appears to be happening is that each time a servlet is called from the web site, a new process is created to run the java program. When I view processes with ps ax, I see
Re: ** JVM and Processes
Hi Adam, No, the garbage collector runs as a low priority background process and, on a lightly loaded server, may never get called because the server's not using enough resources to warrant it. I really wouldn't worry about it too much and I would definitely avoid killing threads individually, especially since you're now utilizing a Pool connector. (you don't want to kill threads that are marked as available in the pool...) The min_spare_threads and max_spare_threads settings are supposed to take care of cleaning up any extra unused threads that are laying around. I think the best benefit you could do yourself would be to upgrade your 3.1 version of Tomcat to the newly released 3.2.2 final to take advantage of upgrades and bug fixes. Thanks, --jeff - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 11:03 AM Subject: Re: ** JVM and Processes Jeff, Thanks a bunch. Your answer appears to be the best so far. I have implemented the PoolTCPConnector in the server xml file and it appears to be limiting the number of threads as it should. However, something that has been happening (even before switching to PoolTCPConnector) is that when running multiple java servlets the threads stay alive long after they should have died or been garbage collected. Even after a long wait, the only way (apparently) to get rid of them is to go through and kill them one at a time. Is there a setting somewhere that is telling the java threads to stay alive indefinitely? Thanks for your help, - Adam At 10:34 AM 6/1/2001 -0700, you wrote: When Java first came to the Linux platform (via the Blackdown port), green-threads were the only option. Native threads took a little longer to implement, but are a much better option for the reasons listed in the previous message. So, I would recommend avoiding green-threads unless you have a specific reason for using them. A lot of people freak out when they see the number of processes being reported by ps or top, without realizing that these are merely threads and not full-blown processes. If you have a lightly loaded Tomcat, you can tune down the number of threads being spawned by using the max_threads, max_spare_threads, and min_spare_threads parameters of the PoolTCPConnector in your server.xml file. For an example of this, take a look at the tomcat user's guide: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/index.html Do a find in your web browser for max_threads. I use this to limit the number of ajp12 threads and maximize ajp13 threads -- because I'm using ajp13 for my servlets and ajp12 only for startup/shutdown of Tomcat. Conversely, if you have a heavily loaded Tomcat, you should use these parameters to increase performance. Thanks, --jeff - Original Message - From: Michael Jennings [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 9:32 AM Subject: Re: ** JVM and Processes RE: ** JVM and ProcessesMy understanding of green vs. native threads is as follows: With native threads, an actual system thread is created when a Java thread is created. On linux a system thread takes the form of another process, but one that shares memory etc. with another process. This is why if you create a program that allocates 100 megs of memory, then spins off 10 threads, it looks like 10 processes each taking up 100 megs of memory, when in fact the amount of memory is 100 megs + 10*overhead for each thread (not much more than 100 megs). On WIN32 systems, threads do not show up as separate processes, they are separate threads of execution inside the same process (essentially the same as the Linux implementation with differences too subtle to care about) Green threads on the other hand use timers, signals, setjmp etc. voodoo to simulate threads within one process. Essentially taking over the scheduling from the kernel. I believe the command-line option for green threads is simply -green as in java -green MyThreaddedApp If you have a multi-cpu system, green threads will only take advantage of one cpu, whereas native threads will use all the cpus on your system (that's the theory anyway) I've heard of problems with blocking I/O with green threads, but have no first hand knowledge. Hope this helps. -Mike Jennings - Original Message - From: BARRAUD Valérie To: '[EMAIL PROTECTED]' Sent: Friday, June 01, 2001 9:01 AM Subject: RE: ** JVM and Processes http://java.sun.com/products/jdk/1.1/packs/native-threads/README -Message d'origine- De: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Date: vendredi 1 juin 2001 17:46 À: [EMAIL PROTECTED] Objet: RE: ** JVM and Processes Randy, Thanks for the advice. Could you be a little more specific, though, about how to use green threads instead of native threads and possibly differences between
Re: ** JVM and Processes
Using the Pool connector and the min_spare_threads,max_spare_threads, and max_threads, I set max_threads to 30 just to test it. Once I restart the server.xml file, if I wait a little while (after some people have visited the site and used some of the servlets) more than 30 threads appear (listed as previously mentioned: /usr/java/jdk1.3/bin/i386/native_threads/java ). Sometimes as many as 80 or 90 appear after a while. It appears the max_threads variable has no affect and is useless. Can you shed light on the issue? - Thanks, Adam At 11:57 AM 6/1/2001 -0700, you wrote: Hi Adam, No, the garbage collector runs as a low priority background process and, on a lightly loaded server, may never get called because the server's not using enough resources to warrant it. I really wouldn't worry about it too much and I would definitely avoid killing threads individually, especially since you're now utilizing a Pool connector. (you don't want to kill threads that are marked as available in the pool...) The min_spare_threads and max_spare_threads settings are supposed to take care of cleaning up any extra unused threads that are laying around. I think the best benefit you could do yourself would be to upgrade your 3.1 version of Tomcat to the newly released 3.2.2 final to take advantage of upgrades and bug fixes. Thanks, --jeff - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 11:03 AM Subject: Re: ** JVM and Processes Jeff, Thanks a bunch. Your answer appears to be the best so far. I have implemented the PoolTCPConnector in the server xml file and it appears to be limiting the number of threads as it should. However, something that has been happening (even before switching to PoolTCPConnector) is that when running multiple java servlets the threads stay alive long after they should have died or been garbage collected. Even after a long wait, the only way (apparently) to get rid of them is to go through and kill them one at a time. Is there a setting somewhere that is telling the java threads to stay alive indefinitely? Thanks for your help, - Adam At 10:34 AM 6/1/2001 -0700, you wrote: When Java first came to the Linux platform (via the Blackdown port), green-threads were the only option. Native threads took a little longer to implement, but are a much better option for the reasons listed in the previous message. So, I would recommend avoiding green-threads unless you have a specific reason for using them. A lot of people freak out when they see the number of processes being reported by ps or top, without realizing that these are merely threads and not full-blown processes. If you have a lightly loaded Tomcat, you can tune down the number of threads being spawned by using the max_threads, max_spare_threads, and min_spare_threads parameters of the PoolTCPConnector in your server.xml file. For an example of this, take a look at the tomcat user's guide: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/index.html Do a find in your web browser for max_threads. I use this to limit the number of ajp12 threads and maximize ajp13 threads -- because I'm using ajp13 for my servlets and ajp12 only for startup/shutdown of Tomcat. Conversely, if you have a heavily loaded Tomcat, you should use these parameters to increase performance. Thanks, --jeff - Original Message - From: Michael Jennings [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, June 01, 2001 9:32 AM Subject: Re: ** JVM and Processes RE: ** JVM and ProcessesMy understanding of green vs. native threads is as follows: With native threads, an actual system thread is created when a Java thread is created. On linux a system thread takes the form of another process, but one that shares memory etc. with another process. This is why if you create a program that allocates 100 megs of memory, then spins off 10 threads, it looks like 10 processes each taking up 100 megs of memory, when in fact the amount of memory is 100 megs + 10*overhead for each thread (not much more than 100 megs). On WIN32 systems, threads do not show up as separate processes, they are separate threads of execution inside the same process (essentially the same as the Linux implementation with differences too subtle to care about) Green threads on the other hand use timers, signals, setjmp etc. voodoo to simulate threads within one process. Essentially taking over the scheduling from the kernel. I believe the command-line option for green threads is simply -green as in java -green MyThreaddedApp If you have a multi-cpu system, green threads will only take advantage of one cpu, whereas native threads will use all the cpus on your system (that's the theory anyway) I've heard of problems with blocking I/O with green threads, but have no first hand knowledge. Hope this helps. -Mike Jennings - Original Message - From
