[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes TOXENV User-defined Axix Change By: Marc Abramowitz (21/Nov/13 6:32 PM) Attachment: TOXENV.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz
updated JENKINS-20706
Support multi-configuration jobs with user-defined axes
Change By:
Marc Abramowitz
(21/Nov/13 6:29 PM)
Description:
LovetheCoberturaplugin!ImusingitwithPythonbuildjobsthataresetupasmulti-configurationjobs;thereisauser-definedaxisdefinedforTOXENVwhichletsmeuseaPythontoolcalledtoxtorunforeachbuild,asub-build(forgivemeifthisisthewrongterminology)foreachofseveralPythonversions--e.g.:py26,py27,py33,andpypy.EachofthesePythonversionsisasub-buildofthebuild.Eachofthosesub-buildsgeneratesitsowncoverage.xmlfile.
HereishowIsetupthesemulti-configurationjobs:
http://tox.readthedocs.org/en/latest/example/jenkins.html
TheCoberturapluginkindofworkswiththissetupinasomewhatincompleteway.IfIgototheprojectpage,Idontseethecoveragegraph.IfIclickonCoverageReport,thenIgetbackthenodatapage.Icanonlyviewcoveragebydrillingintoeachoftheindividualenvironments.Forexample,Icanclickonthemostrecentbuildandthenclickthepy26sub-buildandthenforthatIcanseeacoveragereport.WhatIwouldreallylikeistoalsobeabletoseesomecoverageinformationfortheoverallbuild--eitherby:1.(simpler)havingthepluginpickoneofthesub-buildsasrepresentativeandshowingthatsub-buildscoverageasthecoveragefortheoverallbuild.Attheverysimplest,itcouldsimplypickanyoneofthesub-buildsasthecoverageissometimesslightlydifferentforthesub-builds,butapproximatelythesame,soshowinganyoneisbetterthannothing.Fancierwouldbetoallowtheusertospecifyinthepluginconfigurationthatoneofthesub-buildsistheonetouse.Forexample,sinceIdeploytopy26,thatoneismostimportanttome,soImightpickthepy26coverageasbeingtheonetousefortheoverallbuild.2.(fancier)dosomemergingofthecoverageacrossthesub-builds--e.g.:ifalineiscoveredinanyofthesub-buildsthenitisconsideredcovered.Thisseemstricky.Anotherwaytohandlethisistoleaveituptotheuser.IbelievethatthePythoncoveragetoolhasamergesub-commandthatletsyoumergemultiplerunsintoone(Ihaventusedityetthough).Itmightbepossibleformetomanuallyrunthisinmybuild.Ifyoudonthavetimetoactivelyworkonthisissue,Iamopentoworkingonitmyself,especiallyifyoucangivemealittleguidanceofwheretolookandwhattheoverallapproachmightbe.IvealreadyclonedthegitrepoandsetupNetbeansandIhavemessedaroundalittlewithsettingbreakpointsanddebuggingbutIhaventbeenabletofigureoutyethowtogetittowork.IammoreofaPythonguythanaJenkinsguy,butifyougivemesomepointers,Imightbeabletofindmywayandpossiblysendyouapullrequest.IthinkthemainproblemthatIdiscoveredwhiledebuggingisthatin{{CoberturaProjectAction.getLastResultBuild}}whentheplugincalls{{getAction}}onmybuild,itgets{{null}}(insteadofa{{CoberturaBuildAction}}),soitdoesntconsiderthebuildtohavecoverage.SoevenifIattempttomanuallycopyacoverage.xmlintothebuilddirectory,thepluginstilldoesntthinkthatthebuildhascoveragebecauseitcalls{{getAction}}andgets{{null}}insteadofa{{CoberturaBuildAction}}.Iamguessingthatatsomepoint,thepluginregistersthisaction(couldntfindwhere)andinthemulti-configjobcase,itprobablyregisterstheactiononthesub-jobsbutnottheparentjob.Justaguess.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes Note that coverage is shown for TOXENV=py26 "sub-build" of build #25. Coverage is not shown for build #25 overall. Change By: Marc Abramowitz (21/Nov/13 6:43 PM) Attachment: TOXENV=py26coverageresults.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz
updated JENKINS-20706
Support multi-configuration jobs with user-defined axes
Change By:
Marc Abramowitz
(21/Nov/13 6:40 PM)
Description:
LovetheCoberturaplugin!ImusingitwithPythonbuildjobsthataresetupasmulti-configurationjobs;thereisauser-definedaxisdefinedforTOXENVwhichletsmeuseaPythontoolcalledtoxtorunforeachbuild,asub-build(forgivemeifthisisthewrongterminology)foreachofseveralPythonversions--e.g.:py26,py27,py33,andpypy.EachofthesePythonversionsisasub-buildofthebuild.Eachofthosesub-buildsgeneratesitsowncoverage.xmlfile.HereishowIsetupthesemulti-configurationjobs:http://tox.readthedocs.org/en/latest/example/jenkins.htmlTheCoberturapluginkindofworkswiththissetupinasomewhatincompleteway.IfIgototheprojectpage,Idontseethecoveragegraph
(butI*do*seeJUnittestresultssoIguesstheJUnitpluginhassupportformulti-configurationbuilds)
.IfIclickonCoverageReport,thenIgetbackthenodatapage.Icanonlyviewcoveragebydrillingintoeachoftheindividualenvironments.Forexample,Icanclickonthemostrecentbuildandthenclickthepy26sub-buildandthenforthatIcanseeacoveragereport.WhatIwouldreallylikeistoalsobeabletoseesomecoverageinformationfortheoverallbuild--eitherby:1.(simpler)havingthepluginpickoneofthesub-buildsasrepresentativeandshowingthatsub-buildscoverageasthecoveragefortheoverallbuild.Attheverysimplest,itcouldsimplypickanyoneofthesub-buildsasthecoverageissometimesslightlydifferentforthesub-builds,butapproximatelythesame,soshowinganyoneisbetterthannothing.Fancierwouldbetoallowtheusertospecifyinthepluginconfigurationthatoneofthesub-buildsistheonetouse.Forexample,sinceIdeploytopy26,thatoneismostimportanttome,soImightpickthepy26coverageasbeingtheonetousefortheoverallbuild.2.(fancier)dosomemergingofthecoverageacrossthesub-builds--e.g.:ifalineiscoveredinanyofthesub-buildsthenitisconsideredcovered.Thisseemstricky.Anotherwaytohandlethisistoleaveituptotheuser.IbelievethatthePythoncoveragetoolhasamergesub-commandthatletsyoumergemultiplerunsintoone(Ihaventusedityetthough).Itmightbepossibleformetomanuallyrunthisinmybuild.Ifyoudonthavetimetoactivelyworkonthisissue,Iamopentoworkingonitmyself,especiallyifyoucangivemealittleguidanceofwheretolookandwhattheoverallapproachmightbe.IvealreadyclonedthegitrepoandsetupNetbeansandIhavemessedaroundalittlewithsettingbreakpointsanddebuggingbutIhaventbeenabletofigureoutyethowtogetittowork.IammoreofaPythonguythanaJenkinsguy,butifyougivemesomepointers,Imightbeabletofindmywayandpossiblysendyouapullrequest.IthinkthemainproblemthatIdiscoveredwhiledebuggingisthatin{{CoberturaProjectAction.getLastResultBuild}}whentheplugincalls{{getAction}}onmybuild,itgets{{null}}(insteadofa{{CoberturaBuildAction}}),soitdoesntconsiderthebuildtohavecoverage.SoevenifIattempttomanuallycopyacoverage.xmlintothebuilddirectory,thepluginstilldoesntthinkthatthebuildhascoveragebecauseitcalls{{getAction}}andgets{{null}}insteadofa{{CoberturaBuildAction}}.Iamguessingthatatsomepoint,thepluginregistersthisaction(couldntfindwhere)andinthemulti-configjobcase,itprobablyregisterstheactiononthesub-jobsbutnottheparentjob.Justaguess.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes Post build actions for Cobertura and JUnit Change By: Marc Abramowitz (21/Nov/13 6:36 PM) Attachment: Postbuildactions.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes Coverage is not shown for build #25 overall. Note that coverage is shown for TOXENV=py26 "sub-build" of build #25. Change By: Marc Abramowitz (21/Nov/13 6:45 PM) Attachment: build25.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes Project page - note that JUnit test results are shown but not Cobertura Change By: Marc Abramowitz (21/Nov/13 6:39 PM) Attachment: projectpage.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz updated JENKINS-20706 Support multi-configuration jobs with user-defined axes "Execute Python Script" build step for tox Change By: Marc Abramowitz (21/Nov/13 6:34 PM) Attachment: ExecutePythonScript.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [cobertura] (JENKINS-20706) Support multi-configuration jobs with user-defined axes
Marc Abramowitz created JENKINS-20706 Support multi-configuration jobs with user-defined axes Issue Type: New Feature Assignee: stephenconnolly Components: cobertura Created: 21/Nov/13 6:26 PM Description: Love the Cobertura plugin! I'm using it with Python build jobs that are set up as multi-configuration jobs; there is a user-defined axis defined for TOXENV which lets me use a Python tool called tox to run for each build, a "sub-build" (forgive me if this is the wrong terminology) for each of several Python versions e.g.: py26, py27, py33, and pypy. Each of these Python versions is a "sub-build" of the build. Each of those sub-builds generates its own coverage.xml file. The Cobertura plugin kind of works with this setup in a somewhat incomplete way. If I go to the project page, I don't see the coverage graph. If I click on "Coverage Report", then I get back the "nodata" page. I can only view coverage by drilling into each of the individual environments. For example, I can click on the most recent build and then click the "py26" sub-build and then for that I can see a coverage report. What I would really like is to also be able to see some coverage information for the overall build either by: 1. (simpler) having the plugin pick one of the sub-builds as representative and showing that sub-build's coverage as the coverage for the overall build. At the very simplest, it could simply pick any one of the sub-builds as the coverage is sometimes slightly different for the sub-builds, but approximately the same, so showing any one is better than nothing. Fancier would be to allow the user to specify in the plugin configuration that one of the sub-builds is the one to use. For example, since I deploy to py26, that one is most important to me, so I might pick the py26 coverage as being the one to use for the overall build. 2. (fancier) do some merging of the coverage across the sub-builds e.g.: if a line is covered in any of the sub-builds then it is considered covered. This seems tricky. Another way to handle this is to leave it up to the user. I believe that the Python "coverage" tool has a merge sub-command that lets you merge multiple runs into one (I haven't used it yet though). It might be possible for me to manually run this in my build. If you don't have time to actively work on this issue, I am open to working on it myself, especially if you can give me a little guidance of where to look and what the overall approach might be. I've already cloned the git repo and set up Netbeans and I have messed around a little with setting breakpoints and debugging but I haven't been able to figure out yet how to get it to work. I am more of a Python guy than a Jenkins guy, but if you give me some pointers, I might be able to find my way and possibly send you a pull request. I think the main problem that I discovered while debugging is that in CoberturaProjectAction.getLastResultBuild when the plugin calls getAction on my build, it gets null (instead of a CoberturaBuildAction), so it doesn't consider the build to have coverage. So even if I attempt to manually copy a coverage.xml into the build directory, the plugin still doesn't think that the build has coverage because it calls getAction and gets null instead of a CoberturaBuildAction. I am guessing that at some point, the plugin registers this action (couldn't find where) and in the multi-config job case, it probably registers the action on the sub-jobs but not the parent job. Just a guess. Project: Jenkins Priority: Major Reporter: Marc Abramowitz
