How to distinct errors and failures in tests

[Nick N]

Hey, guys!

I have a freestyle project. It builds an app and run tests (python,
unittest). It is quite straightforward if return codes are 0 then build is
successful, otherwise it is failed. Earlier I returned amount of errors and
failures as a returncode to see tests results on the dashboard (passed or
failed). But test results may contain failures and errors, and I want to
know if there were errors on tests or they all finished, but found some
bugs. Actually I need to get last build (via jenkins api) without errors,
no matter if there were failures.

I've tried to look at "lastStableBuild", but it seems it sticks to
"lastSuccessfulBuild".
I've tried to find information about build statuses but had no success.
Particularly what are criteria of each of them? Maybe it is completely
wrong approach and there's a better way?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CACA70iDR0X9RTWv2051vTuO3pC5odS10kssRYLQe6%2B1A2DjGDg%40mail.gmail.com.

Re: How to distinct errors and failures in tests

[Ullrich Hafner]

Typically you do not set the status of a build by setting the return code. Your 
build should never alter the return code just because the tests are failing. So 
I would suggest that your build always returns 0 if the build is successful 
(ignoring the test results). The JUnit plugin will do the build status handling 
for you: if there are test failures, then the JUnit plugin will set the build 
status to unstable. If not, the status remains successful. You can then 
navigate with lastSuccessful or lastStable to the last build without 
testfailures.

BTW: I don’t think that there is a way to differentiate between test errors and 
failures in Jenkins yet.

> Am 21.04.2021 um 11:26 schrieb Nick N :
> 
> Hey, guys!
> 
> I have a freestyle project. It builds an app and run tests (python, 
> unittest). It is quite straightforward if return codes are 0 then build is 
> successful, otherwise it is failed. Earlier I returned amount of errors and 
> failures as a returncode to see tests results on the dashboard (passed or 
> failed). But test results may contain failures and errors, and I want to know 
> if there were errors on tests or they all finished, but found some bugs. 
> Actually I need to get last build (via jenkins api) without errors, no matter 
> if there were failures.
> 
> I've tried to look at "lastStableBuild", but it seems it sticks to 
> "lastSuccessfulBuild". I've tried to find information about build statuses 
> but had no success. Particularly what are criteria of each of them? Maybe it 
> is completely wrong approach and there's a better way?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-users+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-users/CACA70iDR0X9RTWv2051vTuO3pC5odS10kssRYLQe6%2B1A2DjGDg%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/61D82A4F-D2EF-4D49-909C-D0D55BC4A556%40gmail.com.

Re: How to distinct errors and failures in tests

[nikola...@gmail.com]

Thank you for the answer. If I don't mistake, JUnit plugin just publishes 
the report, but unittest can't produce JUint-compatible xml. Anyway, we 
agree to consider builds with failed tests like failed. Tests generate some 
artifacts, and I want to know all artifacts exist before trying to download 
them.

среда, 21 апреля 2021 г. в 15:03:35 UTC+3, ullrich...@gmail.com: 

> Typically you do not set the status of a build by setting the return code. 
> Your build should never alter the return code just because the tests are 
> failing. So I would suggest that your build always returns 0 if the build 
> is successful (ignoring the test results). The JUnit plugin will do the 
> build status handling for you: if there are test failures, then the JUnit 
> plugin will set the build status to unstable. If not, the status remains 
> successful. You can then navigate with lastSuccessful or lastStable to the 
> last build without testfailures.
>
> BTW: I don’t think that there is a way to differentiate between test 
> errors and failures in Jenkins yet.
>
> Am 21.04.2021 um 11:26 schrieb Nick N :
>
> Hey, guys!
>
> I have a freestyle project. It builds an app and run tests (python, 
> unittest). It is quite straightforward if return codes are 0 then build is 
> successful, otherwise it is failed. Earlier I returned amount of errors and 
> failures as a returncode to see tests results on the dashboard (passed or 
> failed). But test results may contain failures and errors, and I want to 
> know if there were errors on tests or they all finished, but found some 
> bugs. Actually I need to get last build (via jenkins api) without errors, 
> no matter if there were failures.
>
> I've tried to look at "lastStableBuild", but it seems it sticks to 
> "lastSuccessfulBuild". 
> I've tried to find information about build statuses but had no success. 
> Particularly what are criteria of each of them? Maybe it is completely 
> wrong approach and there's a better way?
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-users/CACA70iDR0X9RTWv2051vTuO3pC5odS10kssRYLQe6%2B1A2DjGDg%40mail.gmail.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0d1f597e-ae2d-44d6-b53f-85217e5e1ec7n%40googlegroups.com.

ANN - DevOps World 2021 CFP is Open

[Alyssa Tong]

Hi all,

DevOps World 2021  is now accepting speaking
proposals, and will end on May 20. The conference will be virtual on Sept
28 - 30.  Jenkins plans to have workshops, Ask the Experts, contributor
summit at this event. More details to come as we get closer to September.

In the meantime, we'd love to receive your Jenkins, DevOps stories. We're
looking for:

   - tips & tricks
   - best practices
   - war stories
   - how you overcame barriers and challenges

Thank you,
alyssa & Oleg

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAC9wNayyoR97%2BaAx8H_yzgRwRQcPEPyf0aKb-YDYcWEM_xmWXA%40mail.gmail.com.