Re: CI improvment part 1
I'm waking up this old thread. I was reviewing some jenkins jobs and took a closer look at simulatorfast, We currently have a jenkins job running every 20mins for the simulatorfast, That seams great and seams to provide a good coverage and is perform quite fast, although it's currently running on the pytest branch which is now fairly outdated. I've tried a merge into master and endup with a bunch of conflict. Considering the plus value of using pytest, Would it be possible to update and merge pytest branch so we could use it as CI tests? Regards, On Wed, Dec 3, 2014 at 5:13 PM, Edison Su edison...@citrix.com wrote: haven't have time to work on it since last update. -Original Message- From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Sent: Tuesday, December 02, 2014 9:53 PM To: dev@cloudstack.apache.org; Edison Su Subject: Re: CI improvment part 1 Hi Edison, Just curious, any update on the pytest effort? Thanks. On Friday 26 September 2014 10:13 AM, Edison Su wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back- compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest- xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system. -- Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 8826230892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and- build// CSForge - rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack- software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack- infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack- training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
RE: CI improvment part 1
haven't have time to work on it since last update. -Original Message- From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Sent: Tuesday, December 02, 2014 9:53 PM To: dev@cloudstack.apache.org; Edison Su Subject: Re: CI improvment part 1 Hi Edison, Just curious, any update on the pytest effort? Thanks. On Friday 26 September 2014 10:13 AM, Edison Su wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back- compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest- xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system. -- Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 8826230892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and- build// CSForge - rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack- software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack- infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack- training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: CI improvment part 1
Hi Edison, Just curious, any update on the pytest effort? Thanks. On Friday 26 September 2014 10:13 AM, Edison Su wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back-compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest-xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system. -- Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 8826230892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: CI improvment part 1
Hey Edison, Nice job! Speeding up the simulator is an important step :-) We should probably try to get the new setup working with Travis so we can run it on every commit. I don;t think we nee gerrit yet (who other discussion) but if we get this testing every github pull request and every newly created hotfix/bugfix/feature branch we have the same effect right. Cheers, Hugo On 26 sep. 2014, at 06:43, Edison Su edison...@citrix.com wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back-compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest-xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system.
Re: CI improvment part 1
On Sep 26, 2014, at 6:43 AM, Edison Su edison...@citrix.com wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back-compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. So that is *8* to *10* times faster! Wow :-). Very cool, that’s worth a lot if you ask me! cheers, Leo
Re: CI improvment part 1
Great effort Edison, much needed! How soon would it be in a shape to merge in master and have some wiki pages on it so we can write tests using pytest than nose? It usually takes 12-15 mins on Travis to build CloudStack and we split tests, so on Travis we would be able to run all simulator/integration tests in less than 20 mins. On 26-Sep-2014, at 6:43 am, Edison Su edison...@citrix.com wrote: Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back-compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest-xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system. Regards, Rohit Yadav Software Architect, ShapeBlue M. +41 779015219 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
CI improvment part 1
Highlights the improvements on marvin I made in these days on pytest branch: 1. Switch test framework from nose to pytest, while also maintain the back-compatibility for existing test cases. pytest is more flexible than nose, more enjoyable to work with. 2. Speed up test speed a LOT: It only takes 250-300 seconds for a single run for all the test cases under test/integration/smoke folder, while it usually takes 40 minutes to run before. The speed up is coming from running test cases in parallel. pytest-xdist is modified to fix the issue: https://bitbucket.org/hpk42/pytest/issue/175/way-to-control-how-pytest-xdist-runs-tests. we have a lot of existing test cases have to run in sequence per test class. 3. Add a new Jenkins job at http://jenkins.buildacloud.org/job/fastsimulatorbuild/, which only takes 22 minutes for a full cycle buildtest. A new VM is created on digitalocean: 4G memory, 40G SSD, and added into Jenkins.buildacloud.org. SSD is life saver, it really makes big difference during build/test. A new docker image is created, which has all the tools installed in order to build and test: e.g. java/mysql/python/git/ installed. A new Jenkins job is created to build and test periodically in every 20 minutes. So far, the Jenkins job works very well, if everybody ok with what I am doing, I want to merge pytest branch into master later. With a quick CI in hand, we then can start to think about how to add gerrit into our system.
