RE: Master + slave + job JVM memory tuning...
Have enough RAM on the slave that you can run your jobs plus Jenkins + some for overhead. (e.g. 1GB of Jenkins slave, + #executors * ( 2GB for Maven +2GB for unit tests in forked VM) + OS overhead) Although this amount is mostly an overkill setting it too close to the threshold will slow your jobs down due to paging at best, or mean you get intermittent failures due to OOM at worst – (memory is generally cheaper that human time taken to investigate issues!) (our maven jobs can be very memory hungry so your figures will vary) (Note I recommend a single executor per slave) The master depends more on how many jobs / reports per jobs / job history you have etc – we use 10GB (but not tomcat). If you’re in the UK and there is any spaces left I will cover some things like this next week at the Jenkins User Event. /James From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Jeff Sent: 04 January 2013 21:56 To: jenkinsci-users@googlegroups.com Subject: Master + slave + job JVM memory tuning... I'm trying to optimize how much memory I allocate to the master application (aka - tomcat), slave processes, maven tasks, etc. and going crazy. I've been unsuccessful in finding a guideline to determine how much memory I ought to allocate or if I should just let the defaults work. All machines are single processor, dual core with 4 GB ram. I've typically set tomcat to use 2048MB + 512MB permgen space and set my maven process to use 384MB max. Is this reasonable? My slaves are set to run 4 executors. Jobs use either svn or git to check out code and Maven to build with a SONAR post-build analysis. We also do Maven releases at times. How much does the core slave process need? Is there a rule-of-thumb based on the number of executors? Any perspective/considerations/resources/links/pointers are much appreciated. -- Jeff Vincent predato...@gmail.commailto:predato...@gmail.com See my LinkedIn profile at: http://www.linkedin.com/in/rjeffreyvincent I ♥ DropBoxhttp://db.tt/9O6LfBX !! ** This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmas...@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00 **
RE: Master + slave + job JVM memory tuning...
Thanks...I unfortunately can't be in the UK for the event. So if the amount of RAM is fixed (4GB), what is a good rule of thumb for determining what to allocate to the slave process, how many executors it can reasonably support, etc? I realize there are a lot of other variables but any starting point and things to consider or factor in that may not be obvious could help. Or is it just a matter or monitoring and tuning? On Jan 7, 2013 2:31 AM, Nord, James jn...@nds.com wrote: Have enough RAM on the slave that you can run your jobs plus Jenkins + some for overhead. (e.g. 1GB of Jenkins slave, + #executors * ( 2GB for Maven +2GB for unit tests in forked VM) + OS overhead) Although this amount is mostly an overkill setting it too close to the threshold will slow your jobs down due to paging at best, or mean you get intermittent failures due to OOM at worst – (memory is generally cheaper that human time taken to investigate issues!) (our maven jobs can be very memory hungry so your figures will vary) (Note I recommend a single executor per slave) The master depends more on how many jobs / reports per jobs / job history you have etc – we use 10GB (but not tomcat). If you’re in the UK and there is any spaces left I will cover some things like this next week at the Jenkins User Event. /James *From:* jenkinsci-users@googlegroups.com [mailto: jenkinsci-users@googlegroups.com] *On Behalf Of *Jeff *Sent:* 04 January 2013 21:56 *To:* jenkinsci-users@googlegroups.com *Subject:* Master + slave + job JVM memory tuning... I'm trying to optimize how much memory I allocate to the master application (aka - tomcat), slave processes, maven tasks, etc. and going crazy. I've been unsuccessful in finding a guideline to determine how much memory I ought to allocate or if I should just let the defaults work. All machines are single processor, dual core with 4 GB ram. I've typically set tomcat to use 2048MB + 512MB permgen space and set my maven process to use 384MB max. Is this reasonable? My slaves are set to run 4 executors. Jobs use either svn or git to check out code and Maven to build with a SONAR post-build analysis. We also do Maven releases at times. How much does the core slave process need? Is there a rule-of-thumb based on the number of executors? Any perspective/considerations/resources/links/pointers are much appreciated. -- Jeff Vincent predato...@gmail.com See my LinkedIn profile at: http://www.linkedin.com/in/rjeffreyvincent I ♥ DropBox http://db.tt/9O6LfBX !! -- ** This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmas...@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00 **
Re: Master + slave + job JVM memory tuning...
I would never advise more than one executor per slave :-o At some point there will be unit test failures in a project as it has badly written unit tests that blindly assume they can use port 12345 (and it will be in use for a different job - and then fight each other). And it will be the CI systems fault and come at a time that the most awkward for you (the project is trying to release, and you are about to leave for the weekend!) The exception to this would be where the jenkins job is acting more as a controller and the work is being performed on some other system. /James On Monday, 7 January 2013 18:36:58 UTC, Jeff Vincent wrote: Thanks...I unfortunately can't be in the UK for the event. So if the amount of RAM is fixed (4GB), what is a good rule of thumb for determining what to allocate to the slave process, how many executors it can reasonably support, etc? I realize there are a lot of other variables but any starting point and things to consider or factor in that may not be obvious could help. Or is it just a matter or monitoring and tuning? On Jan 7, 2013 2:31 AM, Nord, James jn...@nds.com javascript: wrote: Have enough RAM on the slave that you can run your jobs plus Jenkins + some for overhead. (e.g. 1GB of Jenkins slave, + #executors * ( 2GB for Maven +2GB for unit tests in forked VM) + OS overhead) Although this amount is mostly an overkill setting it too close to the threshold will slow your jobs down due to paging at best, or mean you get intermittent failures due to OOM at worst – (memory is generally cheaper that human time taken to investigate issues!) (our maven jobs can be very memory hungry so your figures will vary) (Note I recommend a single executor per slave) The master depends more on how many jobs / reports per jobs / job history you have etc – we use 10GB (but not tomcat). If you’re in the UK and there is any spaces left I will cover some things like this next week at the Jenkins User Event. /James *From:* jenkins...@googlegroups.com javascript: [mailto: jenkins...@googlegroups.com javascript:] *On Behalf Of *Jeff *Sent:* 04 January 2013 21:56 *To:* jenkins...@googlegroups.com javascript: *Subject:* Master + slave + job JVM memory tuning... I'm trying to optimize how much memory I allocate to the master application (aka - tomcat), slave processes, maven tasks, etc. and going crazy. I've been unsuccessful in finding a guideline to determine how much memory I ought to allocate or if I should just let the defaults work. All machines are single processor, dual core with 4 GB ram. I've typically set tomcat to use 2048MB + 512MB permgen space and set my maven process to use 384MB max. Is this reasonable? My slaves are set to run 4 executors. Jobs use either svn or git to check out code and Maven to build with a SONAR post-build analysis. We also do Maven releases at times. How much does the core slave process need? Is there a rule-of-thumb based on the number of executors? Any perspective/considerations/resources/links/pointers are much appreciated. -- Jeff Vincent preda...@gmail.com javascript: See my LinkedIn profile at: http://www.linkedin.com/in/rjeffreyvincent I ♥ DropBox http://db.tt/9O6LfBX !! -- ** This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postm...@nds.com javascript: and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00 **
