[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title Jesse Glick commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow Do we need to break up ./gradlew commands between separate stages? Currently I am afraid you do. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.161194.1426173157000.8147.1567779181504%40Atlassian.JIRA.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title Mark Han commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow I think there's still a problem where running a simple `./gradlew test` merges all tests into one section, even though there can be multiple flavors of tests run. Not sure if this is the case if we explicitly break up the flavors e.g. `./gradlew testDebug` Do we need to break up `./gradlew` commands between separate stages? Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.161194.1426173157000.7464.1567741451672%40Atlassian.JIRA.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title Nicholas Klopfer-Webber commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow We had the same problem, but managed to work it out. When we started using a Jenkinsfile we copied this example, however this bypasses the code that keeps each run of tests independent. step([$class : 'JUnitResultArchiver', testResults: '**/build/test-results/**/*.xml']) What we now use and works is: junit testResults: '**/build/test-results/**/*.xml' Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title trejkaz commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow We don't use an XSLT yet because we're using Gradle's reports, which are already in the right format. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title Sid S commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow trejkaz, could you share your XSLT file or link to it? Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title
Sid S edited a comment on JENKINS-27395
Re: distinguish test reports filed from different parts of workflow
Was able to get this working as follows Documenting for future users 1. Switched from nunit3.x to mstest's .trx (xml) test output. In dotnet this simply meant swapping the logger to the mstest one even though we're still using nunit as the actual test framework i.e. {code :java } dotnet test --logger "trx;LogFileName=resultfile.trx"{code}2. Directly used the xsl file published by the mstest folks ([here|https://github.com/jenkinsci/mstest-plugin/blob/master/src/main/resources/hudson/plugins/mstest/mstest-to-junit.xsl]) to transform .trx to junit compatible .xml files. In powershell is this a simple{code :java }$Xslt = New-Object System.Xml.Xsl.XslCompiledTransform$Xslt.Load($xsltFilePath)$Xslt.Transform($xmlInputFilePath, $xmlOutputFilePath){code}3. Use a Jenkinsfile like the one below. In the traditional Jenkins UI, the stage names are get prefixed, to so tests with same namespace are now seen differently. BlueOcean UI still shows them as indistinguishable / identical duplicates {code :java }pipeline { agent {node { label 'master'} } stages {stage('variant-1') { steps {// build steps } post {always {junit 'results/*.xml'} }}stage('variant-2') { steps {// build steps } post {always {junit 'results/*.xml'} }} }}{code}
Add Comment
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
--
You received this message because you are subscribed to th
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title trejkaz commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow Something doesn't quite add up here. Our build already has stage names, but we still don't get the tests distinguished between the different stages. The only workaround idea I had so far was to use XSLT to add the prefix into the actual XML files. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title
Sid S commented on JENKINS-27395
Re: distinguish test reports filed from different parts of workflow
Was able to get this working as follows 1. Switched from nunit3.x to mstest's .trx (xml) test output. In dotnet this simply meant swapping the logger to the mstest one even though we're still using nunit as the actual test framework i.e.
dotnet test --logger "trx;LogFileName=resultfile.trx"
2. Directly used the xsl file published by the mstest folks (here) to transform .trx to junit compatible .xml files. In powershell is this a simple
$Xslt = New-Object System.Xml.Xsl.XslCompiledTransform
$Xslt.Load($xsltFilePath)
$Xslt.Transform($xmlInputFilePath, $xmlOutputFilePath)
3. Use a Jenkinsfile like the one below. In the traditional Jenkins UI, the stage names are prefixed, to tests with same namespace are now seen differently. BlueOcean UI still shows them as indistinguishable / identical duplicates
pipeline {
agent {
node {
label 'master'
}
}
stages {
stage('variant-1') {
steps {
// build steps
}
post {
always {
junit 'results/*.xml'
}
}
}
stage('variant-2') {
steps {
// build steps
}
post {
always {
junit 'results/*.xml'
}
}
}
}
}
Add Comment
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title
Sid S edited a comment on JENKINS-27395
Re: distinguish test reports filed from different parts of workflow
How do you actually consume this in a Jenkinsfile? Our tests have the same name/namespaces across different variants and each variant gets it's own build stage (it's serial for unrelated reasons below). Right now tests across multiple runs/stages are indistinguishable as identical duplicates. Currently we're using the NUnitPublisher which converts nunit3 test XMLs to junit format. We could switch from NUnitPublisher to mstest but it's unclear if we need the test transforming plugins to also be updated OR if the junit changes are sufficient (since it can pull stage names directly).Jenkins 2.124Blue Ocean 1.5.0 JUnit 1.24 Nunit 0.23 This is the pipeline being used and it doesn't work i.e. {code}pipeline { agent {node { label 'master'} } stages {stage('Build variant1') { // build steps here}stage('Publish variant 1 tests') { steps {step([$class: 'NUnitPublisher', testResultsPattern: 'results/variant1/*.xml', debug: false, keepJUnitReports: true, skipJUnitArchiver:false, failIfNoResults: true]) }}stage('Build variant2') { // build steps here}stage('Publish variant 2 tests') { steps {step([$class: 'NUnitPublisher', testResultsPattern: 'results/variant2/*.xml', debug: false, keepJUnitReports: true, skipJUnitArchiver:false, failIfNoResults: true]) }} }}{code}
Add Comment
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title
Sid S commented on JENKINS-27395
Re: distinguish test reports filed from different parts of workflow
How do you actually consume this in a Jenkinsfile? Our tests have the same name/namespaces across different variants and each variant gets it's own build stage (it's serial for unrelated reasons below). Right now tests across multiple runs/stages are indistinguishable as identical duplicates. Currently we're using the NUnitPublisher which converts nunit3 test XMLs to junit format. We could switch from NUnitPublisher to mstest but it's unclear if we need the test transforming plugins to also be updated OR if the junit changes are sufficient (since it can pull stage names directly). Jenkins 2.124 Blue Ocean 1.5.0 This is the pipeline being used and it doesn't work i.e.
pipeline {
agent {
node {
label 'master'
}
}
stages {
stage('Build variant1') {
// build steps here
}
stage('Publish variant 1 tests') {
steps {
step([$class: 'NUnitPublisher', testResultsPattern: 'results/variant1/*.xml', debug: false, keepJUnitReports: true, skipJUnitArchiver:false, failIfNoResults: true])
}
}
stage('Build variant2') {
// build steps here
}
stage('Publish variant 2 tests') {
steps {
step([$class: 'NUnitPublisher', testResultsPattern: 'results/variant2/*.xml', debug: false, keepJUnitReports: true, skipJUnitArchiver:false, failIfNoResults: true])
}
}
}
}
Add Comment
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title SCM/JIRA link daemon commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow Code changed in jenkins User: Cyrille Le Clerc Path: jenkins-plugin/pom.xml jenkins-plugin/src/main/java/org/jenkinsci/plugins/pipeline/maven/publishers/JunitTestsPublisher.java jenkins-plugin/src/test/java/org/jenkinsci/plugins/pipeline/maven/WithMavenStepOnMasterTest.java http://jenkins-ci.org/commit/pipeline-maven-plugin/31c1156f199515f907154938ed8e3bf5a14431c8 Log: JENKINS-27395 temporarily disable bump to the jenkins-junit-pipeline 1.23 due to additional dependencies. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title SCM/JIRA link daemon commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow Code changed in jenkins User: Cyrille Le Clerc Path: jenkins-plugin/pom.xml jenkins-plugin/src/main/java/org/jenkinsci/plugins/pipeline/maven/publishers/JunitTestsPublisher.java jenkins-plugin/src/test/java/org/jenkinsci/plugins/pipeline/maven/WithMavenStepOnMasterTest.java http://jenkins-ci.org/commit/pipeline-maven-plugin/7224db1606420368db3f54b02a79f784c9202665 Log: JENKINS-27395 Record flow node and enclosing stage/branch for tests #82 Merge pull request #82 from abayer/jenkins-27395 Compare: https://github.com/jenkinsci/pipeline-maven-plugin/compare/15990d246db2...7224db160642 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-27395) distinguish test reports filed from different parts of workflow
Title: Message Title SCM/JIRA link daemon commented on JENKINS-27395 Re: distinguish test reports filed from different parts of workflow Code changed in jenkins User: Andrew Bayer Path: jenkins-plugin/pom.xml jenkins-plugin/src/main/java/org/jenkinsci/plugins/pipeline/maven/publishers/JunitTestsPublisher.java jenkins-plugin/src/test/java/org/jenkinsci/plugins/pipeline/maven/WithMavenStepOnMasterTest.java http://jenkins-ci.org/commit/pipeline-maven-plugin/aeb7d81775305282afcaa0464c4bf72c1faf3ca9 Log: JENKINS-27395 Record flow node and enclosing stage/branch for tests Downstream of https://github.com/jenkinsci/junit-plugin/pull/76. This records the `FlowNode#id` for the step that's actually recording the test results and the enclosing stage(s)/branch(es) of that `FlowNode` with each suite, along with the names of the enclosing stage(s)/branch(es) on each `CaseResult` for display purposes. This is accomplished by getting the `FlowNode` via `context.get(FlowNode.class)`, and then passing the configured `JUnitResultArchiver`, the node's ID, the enclosing stage/branch IDs, and the run (which is used to get the `Run#getExternalizableId()` for internal purposes) to the new `JUnitResultArchiver#parseAndAttach` static method. That method does the rest of the work. Once this is all released, the classic test results pages will display the test name with the enclosing stage/branch names prepended (if there are test results from multiple stages/branches in the full test result - i.e., only if relevant). Blue Ocean has a PR to do the same that's pending. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this me
