Re: Next Jenkins weekly

2020-04-27 Thread Olblak 
> It makes sense to me that release runs the the tests on the artifacts, 
Me too as long as they don't increase the release time too much.
If we move those tests to ci.jenkins.io, it becomes more monitoring as we'll 
run tests on public artifacts and so it will make more sense to use Datadog to 
create alerts.etc.

> I agree that it should be a formal on-rotation assignment shared among 
> Jenkins core maintainers and, maybe, the release team.
> Bot roles are documented here: 
> https://github.com/jenkinsci/jenkins/blob/master/docs/MAINTAINERS.adoc#roles
> 
> One problem is that the "Release team" members are not formalized at the 
> moment, and we could clarify that.

Thanks for the documented roles, I didn't know that document exists and I will 
add a link to the jenkins-infra/release readme.
The release team would be the perfect team to have ownership on weekly 
releases, again it doesn't mean they are alone on this, just that they are 
responsible that the full process is going well and they know who to contact in 
case of emergency.

On Sat, Apr 25, 2020, at 10:00 AM, Oleg Nenashev wrote:
> Hi all,
> 
>> Olivier: While we agreed to switch weekly versions to the new release 
>> environment, we didn't clarify who should be responsible for those weekly 
>> releases and when we trigger them. 
>> Mark: I think you're proposing a "build master" role for the weekly build. 
>> That role would investigate build failures, raise issues, and monitor their 
>> resolution. I like that idea. It might fit as a role that could be taken by 
>> one of many people, overseen by the Release Officer.
> 
> I agree that it should be a formal on-rotation assignment shared among 
> Jenkins core maintainers and, maybe, the release team.
> Bot roles are documented here: 
> https://github.com/jenkinsci/jenkins/blob/master/docs/MAINTAINERS.adoc#roles
> 
> One problem is that the "Release team" members are not formalized at the 
> moment, and we could clarify that.
> We have a number of usual suspects (Oliver Gondza for LTS, Daniel Beck for 
> Security, Mark and me for weekly), but IMO it makes sense to make it more 
> formal and maybe to add more contributors.
> 
> Best regards,
> Oleg
> 
> On Friday, April 24, 2020 at 11:43:37 PM UTC+2, Mark Waite wrote:
>> 
>> 
>> On Fri, Apr 24, 2020 at 3:21 PM Tim Jacomb  wrote:
>>> Couldn’t multiple instances run it?
>> 
>> Yes, multiple instances could run it
>> 
>>> It makes sense to me that release runs the the tests on the artifacts, 
>> 
>> The plan is that the package step will run tests on the packaged components. 
>> There are docker based tests that I've created which are intended to run 
>> within the package job. They check that the packages install from the files 
>> that the build generated (deb, rpm, rpm for SUSE, eventually MSI). What they 
>> don't test is that the install works from the Jenkins package repository. In 
>> the case of Debian and Fedora, there are additional checks applied to the 
>> installer when it is downloaded from a package repository.
>> 
>> The acceptance test jobs exercise those additional checks already. Those 
>> acceptance tests could also be executed inside the release.ci..jenkins.io CI 
>> server as well. It seemed like duplication of effort to run them both on 
>> release.ci.jenkins.io and ci.jenkins.io.
>> 
>>> 
>>> And also makes sense that ci.jenkins.io has tests that it’s working 
>>> (possibly it could have simpler tests and doesn’t need to run the full 
>>> acceptance tests set)
>>> 
>> 
>> When I say "acceptance tests" in this case, I mean the existing small tests 
>> which perform an installation from the official package repository. These 
>> are not tests from the acceptance test harness. They are installation smoke 
>> tests.
>> 
>>> 
>>> Thanks
>>> Tim
>>> 
>>> On Fri, 24 Apr 2020 at 21:08, Mark Waite  wrote:
 
 
 On Fri, Apr 24, 2020 at 12:52 PM Jesse Glick  wrote:
> On Fri, Apr 24, 2020 at 12:19 PM Mark Waite  wrote:
>  > The test results aggregator plugin might help us combine test results 
> from multiple jobs into a single summary job.
> 
>  Or just combine them into one job.
> 
 
 That is certainly an option and is worth discussion.
 
 The acceptance test jobs run on ci.jenkins.io because they don't need the 
 added security provided by release.ci.jenkins.io. They are using the 
 publicly available Debian, CentOS, and OpenSUSE images to install the 
 latest weekly Jenkins from the public repositories. They are part of a 
 repository that is also used to check that the ci.jenkins.io 
 infrastructure is providing the expected agent labels and is executing 
 jobs promptly.
 
 The build and package jobs run on release.ci.jenkins.io because they need 
 credentials and permissions that we'd rather not include in ci.jenkins.io .
 
 We could move the acceptance test jobs to release.ci.jenkins.io but then 
 they are no longer able to help

Re: Next Jenkins weekly

2020-04-25 Thread Oleg Nenashev 
Hi all,

Olivier: While we agreed to switch weekly versions to the new release 
> environment, we didn't clarify who should be responsible for those weekly 
> releases and when we trigger them. 

Mark: I think you're proposing a "build master" role for the weekly build.  
> That role would investigate build failures, raise issues, and monitor their 
> resolution.  I like that idea.  It might fit as a role that could be taken 
> by one of many people, overseen by the Release Officer.
>

I agree that it should be a formal on-rotation assignment shared among 
Jenkins core maintainers and, maybe, the release team.
Bot roles are documented here: 
https://github.com/jenkinsci/jenkins/blob/master/docs/MAINTAINERS.adoc#roles

One problem is that the "Release team" members are not formalized at the 
moment, and we could clarify that.
We have a number of usual suspects (Oliver Gondza for LTS, Daniel Beck for 
Security, Mark and me for weekly), but IMO it makes sense to make it more 
formal and maybe to add more contributors.

Best regards,
Oleg

On Friday, April 24, 2020 at 11:43:37 PM UTC+2, Mark Waite wrote:
>
>
>
> On Fri, Apr 24, 2020 at 3:21 PM Tim Jacomb  > wrote:
>
>> Couldn’t multiple instances run it?
>>
>
> Yes, multiple instances could run it
>  
>
>> It makes sense to me that release runs the the tests on the artifacts, 
>>
>
> The plan is that the package step will run tests on the packaged 
> components.  There are docker based tests that I've created which are 
> intended to run within the package job.  They check that the packages 
> install from the files that the build generated (deb, rpm, rpm for SUSE, 
> eventually MSI).  What they don't test is that the install works from the 
> Jenkins package repository.  In the case of Debian and Fedora, there are 
> additional checks applied to the installer when it is downloaded from a 
> package repository.
>
> The acceptance test jobs exercise those additional checks already.  Those 
> acceptance tests could also be executed inside the release.ci..jenkins.io 
> CI server as well.  It seemed like duplication of effort to run them both 
> on release.ci.jenkins.io and ci.jenkins.io.
>  
>
>>
>> And also makes sense that ci.jenkins.io has tests that it’s working 
>> (possibly it could have simpler tests and doesn’t need to run the full 
>> acceptance tests set)
>>
>>
> When I say "acceptance tests" in this case, I mean the existing small 
> tests which perform an installation from the official package repository.  
> These are not tests from the acceptance test harness.  They are 
> installation smoke tests.
>  
>
>> Thanks
>> Tim
>>
>> On Fri, 24 Apr 2020 at 21:08, Mark Waite > > wrote:
>>
>>>
>>>
>>> On Fri, Apr 24, 2020 at 12:52 PM Jesse Glick >> > wrote:
>>>
 On Fri, Apr 24, 2020 at 12:19 PM Mark Waite >>> > wrote:
 > The test results aggregator plugin might help us combine test results 
 from multiple jobs into a single summary job.

 Or just combine them into one job.


>>> That is certainly an option and is worth discussion.
>>>
>>> The acceptance test jobs run on ci.jenkins.io because they don't need 
>>> the added security provided by release.ci.jenkins.io.  They are using 
>>> the publicly available Debian, CentOS, and OpenSUSE images to install the 
>>> latest weekly Jenkins from the public repositories.  They are part of a 
>>> repository that is also used to check that the ci.jenkins.io 
>>> infrastructure is providing the expected agent labels and is executing jobs 
>>> promptly.
>>>
>>> The build and package jobs run on release.ci.jenkins.io because they 
>>> need credentials and permissions that we'd rather not include in 
>>> ci.jenkins.io .
>>>
>>> We could move the acceptance test jobs to release.ci.jenkins.io but 
>>> then they are no longer able to help us confirm the health of 
>>> ci.jenkins.io.
>>>
>>> Mark Waite
>>>  
>>>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "Jenkins Developers" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to jenkin...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0BbRGafPyrCDukMdt8SQGdo4X2W8pgNOEzxHcbsqhE6A%40mail.gmail.com
 .

>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkin...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFCuA9LmG7BW0sdDvrYinO9T8jGeYptwBnSwLFu69QuAQ%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins

Re: Next Jenkins weekly

2020-04-24 Thread Mark Waite 
On Fri, Apr 24, 2020 at 3:21 PM Tim Jacomb  wrote:

> Couldn’t multiple instances run it?
>

Yes, multiple instances could run it


> It makes sense to me that release runs the the tests on the artifacts,
>

The plan is that the package step will run tests on the packaged
components.  There are docker based tests that I've created which are
intended to run within the package job.  They check that the packages
install from the files that the build generated (deb, rpm, rpm for SUSE,
eventually MSI).  What they don't test is that the install works from the
Jenkins package repository.  In the case of Debian and Fedora, there are
additional checks applied to the installer when it is downloaded from a
package repository.

The acceptance test jobs exercise those additional checks already.  Those
acceptance tests could also be executed inside the release.ci..jenkins.io
CI server as well.  It seemed like duplication of effort to run them both
on release.ci.jenkins.io and ci.jenkins.io.


>
> And also makes sense that ci.jenkins.io has tests that it’s working
> (possibly it could have simpler tests and doesn’t need to run the full
> acceptance tests set)
>
>
When I say "acceptance tests" in this case, I mean the existing small tests
which perform an installation from the official package repository.  These
are not tests from the acceptance test harness.  They are installation
smoke tests.


> Thanks
> Tim
>
> On Fri, 24 Apr 2020 at 21:08, Mark Waite 
> wrote:
>
>>
>>
>> On Fri, Apr 24, 2020 at 12:52 PM Jesse Glick 
>> wrote:
>>
>>> On Fri, Apr 24, 2020 at 12:19 PM Mark Waite 
>>> wrote:
>>> > The test results aggregator plugin might help us combine test results
>>> from multiple jobs into a single summary job.
>>>
>>> Or just combine them into one job.
>>>
>>>
>> That is certainly an option and is worth discussion.
>>
>> The acceptance test jobs run on ci.jenkins.io because they don't need
>> the added security provided by release.ci.jenkins.io.  They are using
>> the publicly available Debian, CentOS, and OpenSUSE images to install the
>> latest weekly Jenkins from the public repositories.  They are part of a
>> repository that is also used to check that the ci.jenkins.io
>> infrastructure is providing the expected agent labels and is executing jobs
>> promptly.
>>
>> The build and package jobs run on release.ci.jenkins.io because they
>> need credentials and permissions that we'd rather not include in
>> ci.jenkins.io .
>>
>> We could move the acceptance test jobs to release.ci.jenkins.io but then
>> they are no longer able to help us confirm the health of ci.jenkins.io.
>>
>> Mark Waite
>>
>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to jenkinsci-dev+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0BbRGafPyrCDukMdt8SQGdo4X2W8pgNOEzxHcbsqhE6A%40mail.gmail.com
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFCuA9LmG7BW0sdDvrYinO9T8jGeYptwBnSwLFu69QuAQ%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BidMfABzCzVPs4RX2XRZWseEhzGTsPZ%2BUDQANyaJZ7kc%3Dw%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtHx7%3DFH%2Bp08tdjUyW3hXjWUU1Lr0FhJs-660S2820QZeg%40mail.gmail.com.

Re: Next Jenkins weekly

2020-04-24 Thread Tim Jacomb 
Couldn’t multiple instances run it?
It makes sense to me that release runs the the tests on the artifacts,

And also makes sense that ci.jenkins.io has tests that it’s working
(possibly it could have simpler tests and doesn’t need to run the full
acceptance tests set)

Thanks
Tim

On Fri, 24 Apr 2020 at 21:08, Mark Waite  wrote:

>
>
> On Fri, Apr 24, 2020 at 12:52 PM Jesse Glick  wrote:
>
>> On Fri, Apr 24, 2020 at 12:19 PM Mark Waite 
>> wrote:
>> > The test results aggregator plugin might help us combine test results
>> from multiple jobs into a single summary job.
>>
>> Or just combine them into one job.
>>
>>
> That is certainly an option and is worth discussion.
>
> The acceptance test jobs run on ci.jenkins.io because they don't need the
> added security provided by release.ci.jenkins.io.  They are using the
> publicly available Debian, CentOS, and OpenSUSE images to install the
> latest weekly Jenkins from the public repositories.  They are part of a
> repository that is also used to check that the ci.jenkins.io
> infrastructure is providing the expected agent labels and is executing jobs
> promptly.
>
> The build and package jobs run on release.ci.jenkins.io because they need
> credentials and permissions that we'd rather not include in ci.jenkins.io
> .
>
> We could move the acceptance test jobs to release.ci.jenkins.io but then
> they are no longer able to help us confirm the health of ci.jenkins.io.
>
> Mark Waite
>
>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0BbRGafPyrCDukMdt8SQGdo4X2W8pgNOEzxHcbsqhE6A%40mail.gmail.com
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFCuA9LmG7BW0sdDvrYinO9T8jGeYptwBnSwLFu69QuAQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BidMfABzCzVPs4RX2XRZWseEhzGTsPZ%2BUDQANyaJZ7kc%3Dw%40mail.gmail.com.

Re: Next Jenkins weekly

2020-04-24 Thread Mark Waite 
On Fri, Apr 24, 2020 at 12:52 PM Jesse Glick  wrote:

> On Fri, Apr 24, 2020 at 12:19 PM Mark Waite 
> wrote:
> > The test results aggregator plugin might help us combine test results
> from multiple jobs into a single summary job.
>
> Or just combine them into one job.
>
>
That is certainly an option and is worth discussion.

The acceptance test jobs run on ci.jenkins.io because they don't need the
added security provided by release.ci.jenkins.io.  They are using the
publicly available Debian, CentOS, and OpenSUSE images to install the
latest weekly Jenkins from the public repositories.  They are part of a
repository that is also used to check that the ci.jenkins.io infrastructure
is providing the expected agent labels and is executing jobs promptly.

The build and package jobs run on release.ci.jenkins.io because they need
credentials and permissions that we'd rather not include in ci.jenkins.io .

We could move the acceptance test jobs to release.ci.jenkins.io but then
they are no longer able to help us confirm the health of ci.jenkins.io.

Mark Waite


> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0BbRGafPyrCDukMdt8SQGdo4X2W8pgNOEzxHcbsqhE6A%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFCuA9LmG7BW0sdDvrYinO9T8jGeYptwBnSwLFu69QuAQ%40mail.gmail.com.

Re: Next Jenkins weekly

2020-04-24 Thread Jesse Glick 
On Fri, Apr 24, 2020 at 12:19 PM Mark Waite  wrote:
> The test results aggregator plugin might help us combine test results from 
> multiple jobs into a single summary job.

Or just combine them into one job.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0BbRGafPyrCDukMdt8SQGdo4X2W8pgNOEzxHcbsqhE6A%40mail.gmail.com.

Re: Next Jenkins weekly

2020-04-24 Thread Mark Waite 
On Fri, Apr 24, 2020 at 9:12 AM Olblak  wrote:

> Hi,
>
> While we agreed to switch weekly versions to the new release environment,
> we didn't clarify who should be responsible for those weekly releases and
> when we trigger them.
> As long as the process is not 100% stabilized I think it makes sense for
> me to trigger them on  Monday morning UTC and then when 100% confident we
> can use cronjob for it but I think we'll still need someone responsible for
> them.
>
> Any thoughts?
>
>
I think we should have you run the release Monday morning UTC while we're
stabilizing and we should continue that pattern even after we switch to a
scheduled job to do the build.

I think you're proposing a "build master" role for the weekly build.  That
role would investigate build failures, raise issues, and monitor their
resolution.  I like that idea.  It might fit as a role that could be taken
by one of many people, overseen by the Release Officer.

I'd be willing to take that role, but we'd need to shift the release time
to UTC midday rather than UTC morning so that I would not need to be
watching a build in the middle of my night.

Would you be willing to consider the idea of a DataDog dashboard or a
Jenkins job which shows the overall health of the weekly build?  That
dashboard or Jenkins job would show the results from:

   - Weekly build release job
   - Weekly build package job
   - Install latest Debian package
   

   job
   - Install latest Red Hat package
   

   job
   - Install latest OpenSUSE package job (future)
   - Install latest Windows package job (future)

The test results aggregator plugin
 might help us combine
test results from multiple jobs into a single summary job.


>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/1aa27bc3-a3ff-4f3c-bf71-d13e260dbe80%40www.fastmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtHW2rJikBWOhfzsoUb2r%2BehJ6ZQPa0EV1oEbEw46Qa3Sw%40mail.gmail.com.