Re: [hibernate-dev] Jenkins job priorities

[Davide D'Alto]

Well done, thanks a lot.


On Fri, Jan 12, 2018 at 8:12 AM, Yoann Rodiere  wrote:
> Quick update: the priority plugin seems to be working fine, and I disabled
> the Heavy Job plugin. It turns out the Heavy Job plugin was preventing the
> Amazon EC2 plugin to spin up new slaves, probably because the Amazon EC2
> plugin only saw two empty slots on an existing slave and couldn't
> understand that the waiting jobs couldn't be ran with only two slots.
> Consequently, the Amazon EC2 plugin now spins up lots of instances, with a
> limit of 5. In order to avoid a big hit on the budget, Sanne reduced the
> idle timeout to 30 minutes. Please allow 2 minutes for the slave to boot if
> there is no slave up when you start your job.
>
> So now we have working Amazon EC2 plugin, ensuring new slaves will be spun
> up if there are waiting jobs, and a priority queue, ensuring release/PR
> jobs will be ran first in the (hopefully unlikely) event a lot of jobs are
> waiting in the queue.
> It looks like a reasonable setup, so let's see how it goes for the next
> releases and discuss it afterwards.
>
>
> On Wed, 10 Jan 2018 at 17:46 Steve Ebersole  wrote:
>
>> I know ;)
>>
>> Anyway I do agree that any release jobs should be given the highest
>> priority in the job queue
>>
>> On Wed, Jan 10, 2018, 10:29 AM Guillaume Smet 
>> wrote:
>>
>> > On Wed, Jan 10, 2018 at 5:00 PM, Steve Ebersole 
>> > wrote:
>> >
>> >> And in advance I say I would not be cool with you killing my jobs for
>> >> your job to run
>> >>
>> >
>> > Yeah, that was my understanding.
>> >
>> > I don't expect anyone to be cool with it.
>> >
>> >
>> ___
>> hibernate-dev mailing list
>> hibernate-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>
>
> --
> Yoann Rodiere
> yo...@hibernate.org / yrodi...@redhat.com
> Software Engineer
> Hibernate NoORM team
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Yoann Rodiere]

Quick update: the priority plugin seems to be working fine, and I disabled
the Heavy Job plugin. It turns out the Heavy Job plugin was preventing the
Amazon EC2 plugin to spin up new slaves, probably because the Amazon EC2
plugin only saw two empty slots on an existing slave and couldn't
understand that the waiting jobs couldn't be ran with only two slots.
Consequently, the Amazon EC2 plugin now spins up lots of instances, with a
limit of 5. In order to avoid a big hit on the budget, Sanne reduced the
idle timeout to 30 minutes. Please allow 2 minutes for the slave to boot if
there is no slave up when you start your job.

So now we have working Amazon EC2 plugin, ensuring new slaves will be spun
up if there are waiting jobs, and a priority queue, ensuring release/PR
jobs will be ran first in the (hopefully unlikely) event a lot of jobs are
waiting in the queue.
It looks like a reasonable setup, so let's see how it goes for the next
releases and discuss it afterwards.


On Wed, 10 Jan 2018 at 17:46 Steve Ebersole  wrote:

> I know ;)
>
> Anyway I do agree that any release jobs should be given the highest
> priority in the job queue
>
> On Wed, Jan 10, 2018, 10:29 AM Guillaume Smet 
> wrote:
>
> > On Wed, Jan 10, 2018 at 5:00 PM, Steve Ebersole 
> > wrote:
> >
> >> And in advance I say I would not be cool with you killing my jobs for
> >> your job to run
> >>
> >
> > Yeah, that was my understanding.
> >
> > I don't expect anyone to be cool with it.
> >
> >
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>


-- 
Yoann Rodiere
yo...@hibernate.org / yrodi...@redhat.com
Software Engineer
Hibernate NoORM team
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Steve Ebersole]

I know ;)

Anyway I do agree that any release jobs should be given the highest
priority in the job queue

On Wed, Jan 10, 2018, 10:29 AM Guillaume Smet 
wrote:

> On Wed, Jan 10, 2018 at 5:00 PM, Steve Ebersole 
> wrote:
>
>> And in advance I say I would not be cool with you killing my jobs for
>> your job to run
>>
>
> Yeah, that was my understanding.
>
> I don't expect anyone to be cool with it.
>
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Guillaume Smet]

On Wed, Jan 10, 2018 at 5:00 PM, Steve Ebersole  wrote:

> And in advance I say I would not be cool with you killing my jobs for your
> job to run
>

Yeah, that was my understanding.

I don't expect anyone to be cool with it.
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Steve Ebersole]

And in advance I say I would not be cool with you killing my jobs for your
job to run

On Wed, Jan 10, 2018 at 9:52 AM Sanne Grinovero  wrote:

> On 10 January 2018 at 11:33, Guillaume Smet 
> wrote:
> > On Wed, Jan 10, 2018 at 12:08 PM, Sanne Grinovero 
> > wrote:
> >>
> >> Some of our test suites used to take 2 hours to run (even 5 days some
> >> years ago); now you say waiting 20 minutes is not good enough? You'll
> >> have to optimise our code better :P
> >
> >
> > What I'm saying is that in the current setup, I don't wait at all when I
> > have something to release.
> >
> > All is passed in parallel to the currently running jobs.
> >
> > And it works well.
>
> I'm confused now. AFAIK this has never been the case? I understand
> that the release process itself runs without running the tests, but
> I'd still run the tests by triggering a full build before.
> You made the example of the TCK and various tests; to run them you'd
> not be allowed to run them in parallel with other builds, so you
> wanted to release and the jobs happened to be building ORM and all its
> RDBMS, you'd have had to wait for a couple hours.
>
> >
> >>
> >> It's very easy to spin up extra nodes; my recommendation is that when
> >> you know you're about to release [for example approximately one hour
> >> in advance while you might be double-checking JIRA state and such
> >> things] hit that manual scale-up button and have CI "warmed up" with
> >> one or two extra nodes.
> >>
> >> By the time you need to trigger the release job you'll have the build
> >> queue flushed, the priority plugin helping you out, and still
> >> additional extra slaves running to run it all in parallel.
> >>
> >> And of course for many releases we don't care for an extra 30 minutes
> >> so you're free to skip this all if it's not important; incidentally
> >> for "work in progress" milestones like the module packs which we
> >> recently re-released several times while polishing up the PR I've been
> >> releasing from my local machine; it's good to have CI automate things
> >> but I don't think we should get in a position to require 100%
> >> availability from CI: practice releases locally sometimes.
> >
> >
> > Well, the ultimate goal of releasing on CI is to have consistent releases
> > and an automated process.
> >
> > I really don't want to build a release locally and be at risk of doing
> > something wrong.
> >
> > That's the main purpose of an automated process and having a stable
> machine
> > doing it.
> >
> >>
> >> Let's not forget that many Apache projects take a week or two to
> >> perform a release, we all know of other projects needing months, so by
> >> the law of diminishing returns I don't think we should invest much
> >> more of out time to shave on the 10 minutes.. just spin up some extra
> >> nodes :)
> >
> >
> > What I'm saying is that the current setup is working very well for
> releases
> > and the proposed setup won't work as well.
> >
> > You can find all sorts of workarounds but it won't work as well and be as
> > practical as it used to be. Yeah, you can think of starting another node
> 1
> > hour before doing your release and hope it will still be there and you
> won't
> > have another project's commit triggering 4 jobs just before you start.
> Sure.
> > But I'm pretty sure it's going to be a pain.
>
> Still I don't really understand if you're having a better idea. In a
> nutshell these jobs need resources, if they are busy you either add
> more resources, or change priorities, or you wait. That's the three
> aspects you can play with "safely".
>
> Then there's the option of playing with parallelism, but it's really
> dangerous: it risks failing both your release and causing failures in
> the other tests which are hard to expliain, cause confusion among us
> all, and ultimately lead to have to repeat all involved jobs so
> consuming unnecessarily more resources and time.
> In many cases parallelism isn't even an option, for examplethe ORM
> builds consume most system memory so you just can't run additional
> JVMs to run the TCK or similar jobs; if it was safe, I would be using
> smaller machines.
>
> > I'm probably the one doing releases the most frequently with HV, that's
> why
> > I am vocal about it.
> >
> > And maybe I'm the only one but, when I'm working on a release, I don't
> like
> > to do stuff in parallel because I don't want to forget something or make
> a
> > mistake. So I'm fully focused on it. Waiting 20 minutes before having my
> job
> > running will be a complete waste of time. And if it has to happen more
> than
> > one time on a given release time, I can predict I will get grumpy :).
> >
> > That being said, if you have nothing against me cancelling the running
> jobs
> > because they are in the way, we can do that. But I'm not sure people will
> > like it very much.
>
> Just make sure you ask for permissions, but yea we've done that
> previously, 

Re: [hibernate-dev] Jenkins job priorities

[Sanne Grinovero]

On 10 January 2018 at 11:33, Guillaume Smet  wrote:
> On Wed, Jan 10, 2018 at 12:08 PM, Sanne Grinovero 
> wrote:
>>
>> Some of our test suites used to take 2 hours to run (even 5 days some
>> years ago); now you say waiting 20 minutes is not good enough? You'll
>> have to optimise our code better :P
>
>
> What I'm saying is that in the current setup, I don't wait at all when I
> have something to release.
>
> All is passed in parallel to the currently running jobs.
>
> And it works well.

I'm confused now. AFAIK this has never been the case? I understand
that the release process itself runs without running the tests, but
I'd still run the tests by triggering a full build before.
You made the example of the TCK and various tests; to run them you'd
not be allowed to run them in parallel with other builds, so you
wanted to release and the jobs happened to be building ORM and all its
RDBMS, you'd have had to wait for a couple hours.

>
>>
>> It's very easy to spin up extra nodes; my recommendation is that when
>> you know you're about to release [for example approximately one hour
>> in advance while you might be double-checking JIRA state and such
>> things] hit that manual scale-up button and have CI "warmed up" with
>> one or two extra nodes.
>>
>> By the time you need to trigger the release job you'll have the build
>> queue flushed, the priority plugin helping you out, and still
>> additional extra slaves running to run it all in parallel.
>>
>> And of course for many releases we don't care for an extra 30 minutes
>> so you're free to skip this all if it's not important; incidentally
>> for "work in progress" milestones like the module packs which we
>> recently re-released several times while polishing up the PR I've been
>> releasing from my local machine; it's good to have CI automate things
>> but I don't think we should get in a position to require 100%
>> availability from CI: practice releases locally sometimes.
>
>
> Well, the ultimate goal of releasing on CI is to have consistent releases
> and an automated process.
>
> I really don't want to build a release locally and be at risk of doing
> something wrong.
>
> That's the main purpose of an automated process and having a stable machine
> doing it.
>
>>
>> Let's not forget that many Apache projects take a week or two to
>> perform a release, we all know of other projects needing months, so by
>> the law of diminishing returns I don't think we should invest much
>> more of out time to shave on the 10 minutes.. just spin up some extra
>> nodes :)
>
>
> What I'm saying is that the current setup is working very well for releases
> and the proposed setup won't work as well.
>
> You can find all sorts of workarounds but it won't work as well and be as
> practical as it used to be. Yeah, you can think of starting another node 1
> hour before doing your release and hope it will still be there and you won't
> have another project's commit triggering 4 jobs just before you start. Sure.
> But I'm pretty sure it's going to be a pain.

Still I don't really understand if you're having a better idea. In a
nutshell these jobs need resources, if they are busy you either add
more resources, or change priorities, or you wait. That's the three
aspects you can play with "safely".

Then there's the option of playing with parallelism, but it's really
dangerous: it risks failing both your release and causing failures in
the other tests which are hard to expliain, cause confusion among us
all, and ultimately lead to have to repeat all involved jobs so
consuming unnecessarily more resources and time.
In many cases parallelism isn't even an option, for examplethe ORM
builds consume most system memory so you just can't run additional
JVMs to run the TCK or similar jobs; if it was safe, I would be using
smaller machines.

> I'm probably the one doing releases the most frequently with HV, that's why
> I am vocal about it.
>
> And maybe I'm the only one but, when I'm working on a release, I don't like
> to do stuff in parallel because I don't want to forget something or make a
> mistake. So I'm fully focused on it. Waiting 20 minutes before having my job
> running will be a complete waste of time. And if it has to happen more than
> one time on a given release time, I can predict I will get grumpy :).
>
> That being said, if you have nothing against me cancelling the running jobs
> because they are in the way, we can do that. But I'm not sure people will
> like it very much.

Just make sure you ask for permissions, but yea we've done that
previously, hopefully won't be needed often, but it's always an
option.

>
> --
> Guillaume
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Guillaume Smet]

On Wed, Jan 10, 2018 at 12:08 PM, Sanne Grinovero 
wrote:

> Some of our test suites used to take 2 hours to run (even 5 days some
> years ago); now you say waiting 20 minutes is not good enough? You'll
> have to optimise our code better :P
>

What I'm saying is that in the current setup, I don't wait at all when I
have something to release.

All is passed in parallel to the currently running jobs.

And it works well.


> It's very easy to spin up extra nodes; my recommendation is that when
> you know you're about to release [for example approximately one hour
> in advance while you might be double-checking JIRA state and such
> things] hit that manual scale-up button and have CI "warmed up" with
> one or two extra nodes.
>
> By the time you need to trigger the release job you'll have the build
> queue flushed, the priority plugin helping you out, and still
> additional extra slaves running to run it all in parallel.
>
> And of course for many releases we don't care for an extra 30 minutes
> so you're free to skip this all if it's not important; incidentally
> for "work in progress" milestones like the module packs which we
> recently re-released several times while polishing up the PR I've been
> releasing from my local machine; it's good to have CI automate things
> but I don't think we should get in a position to require 100%
> availability from CI: practice releases locally sometimes.
>

Well, the ultimate goal of releasing on CI is to have consistent releases
and an automated process.

I really don't want to build a release locally and be at risk of doing
something wrong.

That's the main purpose of an automated process and having a stable machine
doing it.


> Let's not forget that many Apache projects take a week or two to
> perform a release, we all know of other projects needing months, so by
> the law of diminishing returns I don't think we should invest much
> more of out time to shave on the 10 minutes.. just spin up some extra
> nodes :)
>

What I'm saying is that the current setup is working very well for releases
and the proposed setup won't work as well.

You can find all sorts of workarounds but it won't work as well and be as
practical as it used to be. Yeah, you can think of starting another node 1
hour before doing your release and hope it will still be there and you
won't have another project's commit triggering 4 jobs just before you
start. Sure. But I'm pretty sure it's going to be a pain.

I'm probably the one doing releases the most frequently with HV, that's why
I am vocal about it.

And maybe I'm the only one but, when I'm working on a release, I don't like
to do stuff in parallel because I don't want to forget something or make a
mistake. So I'm fully focused on it. Waiting 20 minutes before having my
job running will be a complete waste of time. And if it has to happen more
than one time on a given release time, I can predict I will get grumpy :).

That being said, if you have nothing against me cancelling the running jobs
because they are in the way, we can do that. But I'm not sure people will
like it very much.

-- 
Guillaume
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Davide D'Alto]

> Let's not forget that many Apache projects take a week or two to
> perform a release, we all know of other projects needing months, so by
> the law of diminishing returns I don't think we should invest much
> more of out time to shave on the 10 minutes.. just spin up some extra
> nodes :)

+1



On Wed, Jan 10, 2018 at 11:08 AM, Sanne Grinovero  wrote:
> On 10 January 2018 at 10:25, Guillaume Smet  wrote:
>> Hi,
>>
>> On Wed, Jan 10, 2018 at 11:06 AM, Yoann Rodiere  wrote:
>>>
>>> I hope we will be able to use this priority feature instead of the Heavy
>>> Job plugin (which allows to assign weights to jobs), and avoid concurrent
>>> builds completely. With the current setup, someone releasing his/her
>>> project will only have to wait for the currently executing build to finish,
>>> and will get the highest priority on the release builds. Maybe this is
>>> enough? If you disagree, please raise your concerns now: I will disable the
>>> Heavy Job plugin soon and set each slave to only offer one execution slot.
>
> Thanks Yoann! that sounds great.
>
>>>
>>
>> I'm not really convinced by this solution. Some jobs still take quite a lot
>> of time and having to wait 20 minutes for each job I would trigger is a bit
>> annoying.
>>
>> If it was for only one job, it would be acceptable, but let's take the
>> worst case of a coordinated HV release :
>> - TCK release
>> - API release
>> - HV release
>> - website
>> - blog
>>
>> I won't have to wait for each of them as some of them will be grouped by
>> the prioritization but I'm pretty sure I will have to wait for several of
>> them.
>>
>> So, I'm +1 on having this plugin as it seems to be helpful on its own but
>> I'm -1 on considering it is a solution to the "let's roll a release" thing.
>
> Some of our test suites used to take 2 hours to run (even 5 days some
> years ago); now you say waiting 20 minutes is not good enough? You'll
> have to optimise our code better :P
>
> It's very easy to spin up extra nodes; my recommendation is that when
> you know you're about to release [for example approximately one hour
> in advance while you might be double-checking JIRA state and such
> things] hit that manual scale-up button and have CI "warmed up" with
> one or two extra nodes.
>
> By the time you need to trigger the release job you'll have the build
> queue flushed, the priority plugin helping you out, and still
> additional extra slaves running to run it all in parallel.
>
> And of course for many releases we don't care for an extra 30 minutes
> so you're free to skip this all if it's not important; incidentally
> for "work in progress" milestones like the module packs which we
> recently re-released several times while polishing up the PR I've been
> releasing from my local machine; it's good to have CI automate things
> but I don't think we should get in a position to require 100%
> availability from CI: practice releases locally sometimes.
>
> If we really wanted to invest more in it (both time and budget),
> there's the option of spinning up new containers for each job as soon
> as you need one but I feel like we've spent too much time on CI
> already; such technology is maturing so my take is let it mature a bit
> more, and in 6 months we'll do another step of improvement; jumping on
> those things makes us otherwise the beta testers and steals critical
> time from our own projects.
> Let's not forget that many Apache projects take a week or two to
> perform a release, we all know of other projects needing months, so by
> the law of diminishing returns I don't think we should invest much
> more of out time to shave on the 10 minutes.. just spin up some extra
> nodes :)
>
> Thanks,
> Sanne
>
>>
>> --
>> Guillaume
>> ___
>> hibernate-dev mailing list
>> hibernate-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Sanne Grinovero]

On 10 January 2018 at 10:25, Guillaume Smet  wrote:
> Hi,
>
> On Wed, Jan 10, 2018 at 11:06 AM, Yoann Rodiere  wrote:
>>
>> I hope we will be able to use this priority feature instead of the Heavy
>> Job plugin (which allows to assign weights to jobs), and avoid concurrent
>> builds completely. With the current setup, someone releasing his/her
>> project will only have to wait for the currently executing build to finish,
>> and will get the highest priority on the release builds. Maybe this is
>> enough? If you disagree, please raise your concerns now: I will disable the
>> Heavy Job plugin soon and set each slave to only offer one execution slot.

Thanks Yoann! that sounds great.

>>
>
> I'm not really convinced by this solution. Some jobs still take quite a lot
> of time and having to wait 20 minutes for each job I would trigger is a bit
> annoying.
>
> If it was for only one job, it would be acceptable, but let's take the
> worst case of a coordinated HV release :
> - TCK release
> - API release
> - HV release
> - website
> - blog
>
> I won't have to wait for each of them as some of them will be grouped by
> the prioritization but I'm pretty sure I will have to wait for several of
> them.
>
> So, I'm +1 on having this plugin as it seems to be helpful on its own but
> I'm -1 on considering it is a solution to the "let's roll a release" thing.

Some of our test suites used to take 2 hours to run (even 5 days some
years ago); now you say waiting 20 minutes is not good enough? You'll
have to optimise our code better :P

It's very easy to spin up extra nodes; my recommendation is that when
you know you're about to release [for example approximately one hour
in advance while you might be double-checking JIRA state and such
things] hit that manual scale-up button and have CI "warmed up" with
one or two extra nodes.

By the time you need to trigger the release job you'll have the build
queue flushed, the priority plugin helping you out, and still
additional extra slaves running to run it all in parallel.

And of course for many releases we don't care for an extra 30 minutes
so you're free to skip this all if it's not important; incidentally
for "work in progress" milestones like the module packs which we
recently re-released several times while polishing up the PR I've been
releasing from my local machine; it's good to have CI automate things
but I don't think we should get in a position to require 100%
availability from CI: practice releases locally sometimes.

If we really wanted to invest more in it (both time and budget),
there's the option of spinning up new containers for each job as soon
as you need one but I feel like we've spent too much time on CI
already; such technology is maturing so my take is let it mature a bit
more, and in 6 months we'll do another step of improvement; jumping on
those things makes us otherwise the beta testers and steals critical
time from our own projects.
Let's not forget that many Apache projects take a week or two to
perform a release, we all know of other projects needing months, so by
the law of diminishing returns I don't think we should invest much
more of out time to shave on the 10 minutes.. just spin up some extra
nodes :)

Thanks,
Sanne

>
> --
> Guillaume
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Jenkins job priorities

[Guillaume Smet]

Hi,

On Wed, Jan 10, 2018 at 11:06 AM, Yoann Rodiere  wrote:
>
> I hope we will be able to use this priority feature instead of the Heavy
> Job plugin (which allows to assign weights to jobs), and avoid concurrent
> builds completely. With the current setup, someone releasing his/her
> project will only have to wait for the currently executing build to finish,
> and will get the highest priority on the release builds. Maybe this is
> enough? If you disagree, please raise your concerns now: I will disable the
> Heavy Job plugin soon and set each slave to only offer one execution slot.
>

I'm not really convinced by this solution. Some jobs still take quite a lot
of time and having to wait 20 minutes for each job I would trigger is a bit
annoying.

If it was for only one job, it would be acceptable, but let's take the
worst case of a coordinated HV release :
- TCK release
- API release
- HV release
- website
- blog

I won't have to wait for each of them as some of them will be grouped by
the prioritization but I'm pretty sure I will have to wait for several of
them.

So, I'm +1 on having this plugin as it seems to be helpful on its own but
I'm -1 on considering it is a solution to the "let's roll a release" thing.

-- 
Guillaume
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

[hibernate-dev] Jenkins job priorities

[Yoann Rodiere]

Hello,

TL;DR: I installed a plugin to prioritize Jenkins jobs, please let me know
if you notice anything wrong. Also, I will remove the Heavy Job plugin
soon, let me know if you're not okay with that.

I recently raised the issue on HipChat that some Jenkins builds are
triggered in batch, something like 4 or 5 at a time. Since builds are
executed in the order they are requested, this forces the next requested
builds to wait for more than one hour before being executed, regardless of
their urgency.
One example of such batch is whenever something is pushed to Hibernate ORM
master (or Search master, probably): one build is triggered for tests
against H2, another for tests against PostgreSQL, another for tests against
MariaDB, and so on.

It turns out there is a solution for this problem: the PrioritySorter
plugin. I installed the plugin on CI and configured it to give higher
priority to the following builds:

   - Builds triggered by users (highest priority)
   - Release builds (builds in the "Release" view)
   - Website builds (builds in the "Website" view)
   - PR builds (builds in the "PR" view)

In practice, such builds will be moved to the front of the queue whenever
they are triggered, resulting in reduced waiting times.

I hope we will be able to use this priority feature instead of the Heavy
Job plugin (which allows to assign weights to jobs), and avoid concurrent
builds completely. With the current setup, someone releasing his/her
project will only have to wait for the currently executing build to finish,
and will get the highest priority on the release builds. Maybe this is
enough? If you disagree, please raise your concerns now: I will disable the
Heavy Job plugin soon and set each slave to only offer one execution slot.

Please let me know if you notice anything wrong. I tested the plugin on a
local Jenkins instance, but who knows...

Yoann


-- 
Yoann Rodiere
yo...@hibernate.org / yrodi...@redhat.com
Software Engineer
Hibernate NoORM team
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev