[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita closed JENKINS-17960 as Fixed Indicate if tests dont go to plan Fixed in 1.18 Change By: Bruno P. Kinoshita (13/Mar/14 5:29 PM) Status: Resolved Closed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe
commented on JENKINS-17960
Indicate if tests dont go to plan
The logic for this feature should really be very simple.
if (planExists numberAtEndOfPlan != numberOfLastTest) {
log message that tests didn't go to plan
if (status == SUCCESS) then status = UNSTABLE
}
The checkbox should not cause anything else to happen.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan I liked this simpler logic. Going to commit now but will be able to release another alpha version only tonight. Hope you can give it another shot this week. I'd like to cut a new release by weekend. Thanks James! This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
SCM/JIRA link daemon commented on JENKINS-17960 Indicate if tests dont go to plan Code changed in jenkins User: Bruno P. Kinoshita Path: src/main/java/org/tap4j/plugin/TapPublisher.java http://jenkins-ci.org/commit/tap-plugin/275c599945025e3467bb42e9d82d3089ee25dc2b Log: JENKINS-17960 Simplify logic This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe commented on JENKINS-17960 Indicate if tests dont go to plan Does that commit also address the problem of all the results being lost if the tests don't go to plan? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe
edited a comment on JENKINS-17960
Indicate if tests dont go to plan
The logic for this feature should really be very simple.
if (planExists numberAtEndOfPlan != numberOfLastTest) {
log message that tests didn't go to plan;
if (status == SUCCESS) then status = UNSTABLE;
}
The checkbox should not cause anything else to happen.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Not that one, I believe it was fixed in https://github.com/jenkinsci/tap-plugin/commit/9e60ea571d359a0c445a33f791e62ca6c5f87479#diff-7ebebde3074c0bb507b4983802c5803cL149, already released in the experimental update center. This new commit only simplifies the logic as you suggested. Let me know if I missed anything. Probably I can push more commits today before releasing 1.18-2-alpha tonight. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe
edited a comment on JENKINS-17960
Indicate if tests dont go to plan
The logic for this feature should really be very simple.
if (planExists numberAtEndOfPlan != numberOfLastTest) {
log message that tests didn't go to plan
if (status == SUCCESS) then status = UNSTABLE
}
The checkbox should not cause anything else to happen.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe
edited a comment on JENKINS-17960
Indicate if tests dont go to plan
The logic for this feature should really be very simple.
if (planExists numberAtEndOfPlan != numberOfLastTest) {
log message that tests didn't go to plan
if (status == SUCCESS) then status = UNSTABLE
}
The checkbox should not cause anything else to happen.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan By the way, I'm not exactly comparing the number of the last test, but comparing if the total of tests is equal to the expected number in the test plan. Do you see any problem in doing like that? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe commented on JENKINS-17960 Indicate if tests dont go to plan That looks like it might work now I'd need something in the next couple of hours in order to test it before Monday. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita updated JENKINS-17960 Indicate if tests dont go to plan TAP Plug-in snapshot 2014-03-12 HPI file Change By: Bruno P. Kinoshita (12/Mar/14 3:01 PM) Attachment: tap-snapshot-20140312.hpi This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Due to firewall policies in the office where I am atm I can't release an alpha version to the experimental update center I've attached a snapshot HPI with the latest code though, don't know if you can install it in some environment and try it. If so let me know if that works and feel free to update this issue. JENKINS-17960 is the only pending issue I'm waiting to cut a new release. Next release will focus in JVM crashes and memory management, and the following will be code review/smaller improvements. Thanks! Bruno This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Yay, thanks for testing it so quickly James. Cutting a new release this night, 1.18 should be available in the update center within hours. Thanks again Bruno This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe resolved JENKINS-17960 as Fixed Indicate if tests dont go to plan This looks like it now works as I would expect. Change By: James Howe (12/Mar/14 3:58 PM) Status: Open Resolved Resolution: Fixed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe commented on JENKINS-17960 Indicate if tests dont go to plan I'm also seeing in version 1.17 that if the plan doesn't match then all the results are discarded. It should report them just as for any other build. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe commented on JENKINS-17960 Indicate if tests dont go to plan On the other hand, if you planned 100 test cases, but only 90 were execute, and the others were skipped, then with this new change your build will be marked as unstable because the number of test cases doesn't match the plan. That's not following the TAP standard. Skipped tests should count. If Passed + Failed + Skipped = Planned then do nothing, otherwise make a Stable job Unstable. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Starting a dev cycle on tap plugin looking forward to working on this issue again James. Hope you have some spare time and patience to go through this issue again. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Tests planned: 100, Tests run: 90, Tests passed: 90, Status: Unstable Tests planned: 100, Tests run: 110, Tests passed: 110, Status: Unstable Tests planned: 100, Tests run: 100, Tests passed: 100, Status: Stable Tis indeed not so hard to do. However, should you have any failure, and the job is configured to fail the build in case of failures, your build will be marked as failure, instead of unstable. On the other hand, if you planned 100 test cases, but only 90 were execute, and the others were skipped, then with this new change your build will be marked as unstable because the number of test cases doesn't match the plan. WDYT? Will release an alpha version to the alpha update site. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
SCM/JIRA link daemon commented on JENKINS-17960 Indicate if tests dont go to plan Code changed in jenkins User: Bruno P. Kinoshita Path: src/main/java/org/tap4j/plugin/TapParser.java src/main/java/org/tap4j/plugin/TapPublisher.java src/main/java/org/tap4j/plugin/TapResult.java src/main/java/org/tap4j/plugin/model/TapStreamResult.java src/main/java/org/tap4j/plugin/model/TapTestResultResult.java src/test/java/org/tap4j/plugin/issue16964/TestIssue16964.java http://jenkins-ci.org/commit/tap-plugin/9e60ea571d359a0c445a33f791e62ca6c5f87479 Log: JENKINS-17960 Indicate if tests don't go to plan This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan 1.18-alpha released to http://updates.jenkins-ci.org/update-center.json Give it a shot and let me know if that is in the right direction. You'll see a message in the console output, but nothing in the UI yet (will work on this if you confirm we are in the right direction). Thanks! Bruno This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita updated JENKINS-17960 Indicate if tests dont go to plan Change By: Bruno P. Kinoshita (05/Mar/14 12:32 AM) Status: Reopened Open This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
SCM/JIRA link daemon commented on JENKINS-17960 Indicate if tests dont go to plan Code changed in jenkins User: Bruno P. Kinoshita Path: src/main/java/org/tap4j/plugin/TapParser.java src/main/java/org/tap4j/plugin/TapPublisher.java src/main/resources/org/tap4j/plugin/TapPublisher/config.jelly http://jenkins-ci.org/commit/tap-plugin/2127053964d7d4b0b8a3038658ef231a0a214e31 Log: FIXED JENKINS-17960 FIXED JENKINS-17781 Adding option to validate number of planned tests and number of tests executed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
SCM/JIRA link daemon commented on JENKINS-17960 Indicate if tests dont go to plan Code changed in jenkins User: Bruno P. Kinoshita Path: src/test/java/org/tap4j/plugin/issue16647/TestIssue16647.java src/test/java/org/tap4j/plugin/issue16964/TestIssue16964.java http://jenkins-ci.org/commit/tap-plugin/c52f18350d32a93e5242e872eb8c24f756544b7d Log: JENKINS-17960 JENKINS-17781 Fixing tests This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita resolved JENKINS-17960 as Fixed Indicate if tests dont go to plan Change By: Bruno P. Kinoshita (26/Jul/13 8:25 PM) Status: InProgress Resolved Resolution: Fixed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita closed JENKINS-17960 as Fixed Indicate if tests dont go to plan Fixed in 1.13 Change By: Bruno P. Kinoshita (27/Jul/13 1:55 AM) Status: Resolved Closed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita updated JENKINS-17960 Indicate if tests dont go to plan Change By: Bruno P. Kinoshita (24/Jul/13 12:42 PM) Labels: 1.13 This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita commented on JENKINS-17960 Indicate if tests dont go to plan Hi James, Good idea. We can include a new checkbox in the job configuration, so we'll keep the previous behaviour while we include this new feature. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
Bruno P. Kinoshita started work on JENKINS-17960 Indicate if tests dont go to plan Change By: Bruno P. Kinoshita (11/Jul/13 6:40 PM) Status: Open InProgress This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [tap] (JENKINS-17960) Indicate if tests don't go to plan
James Howe created JENKINS-17960 Indicate if tests dont go to plan Issue Type: New Feature Assignee: Bruno P. Kinoshita Components: tap Created: 15/May/13 10:02 AM Description: One of the major features of TAP is that you have a test plan. If my TAP starts 1..100, but only 5 results are recorded before something broke, then I want to know about it. Likewise, if 105 results are recorded, I want some kind of error indication. I suggest counting this as 95 skipped tests by default, possibly with an config option to treat them as failed instead. Project: Jenkins Priority: Critical Reporter: James Howe This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
