Re: Pipeline: Building downstream dependencies

2016-06-06 Thread Sverre Moe
Apparently there are Flyweight and Heavyweight Executors.
Flyweight runs on the master, while the node step runs on the slave 
matching the label.

> When you run a node step:
>
>- 
>
>A regular heavyweight executor is allocated on a node (usually a 
>slave) matching the label expression, as soon as one is available. This 
>executor represents the real work being done on the node.
>- 
>
>If you start a second build of the Pipeline while the first is still 
>paused with the one available executor, you will see both Pipeline builds 
>running on master. But only the first will have grabbed the one available 
>executor on the slave; the other part of jobname #11 will be shown in 
> Build 
>Queue (1). (shortly after, the console log for the second build will 
>note that it is still waiting for an available executor).
>
>
Even the master node has limited executors. What I need is to schedule 
builds on a flyweight executor since it does not need an actual workspace 
to perform.

Using this will likely allocate a heavyweight executor.
node("master") {
   
}


fredag 3. juni 2016 10.39.48 UTC+2 skrev Sverre Moe følgende:
>
> My first predicament:
> In Multi-configuration build, one single executor can contain multiple 
> parent build, but that single executor can only contain one 
> configuration/axis/slave build at a time.
>
> Using Pipeline I am not sure how to get it to work like this.
> After my Build stage I try to schedule build of all the dependencies and 
> it will wait until they finish. But the project already holds the executor.
>
> ProjectA schedules ProjectB. ProjectA waits for ProjectB, ProjectB cannot 
> start because ProjectA holds the executor on the same slave node. So they 
> are both locked...
> build '../ProjectB/master'
>
>> Still waiting to schedule task
>> Waiting for next available executor on master-sles11-x86_64
>>
>
> How can I do the same in Pipeline as I do with Multi-configuration parent. 
> The parent node is is selected from the list of nodes the project is 
> building on. I could probably solve this by making the Jenkins master as 
> the parent node while building dependencies. Triggering other projects to 
> build does not necessary need to be done on one the build nodes.
>
> It would not make a good approach for me to increase the number of 
> executors on the slave nodes. Since the build is removing/installing rpm 
> packages in the build stage it could cause havoc having two builds doing 
> this interchangeably. Each build must have the slave node system to itself 
> while building.
>
>
> My second predicament:
> Each of my projects has an RPM spec file. Its BuildRequires dependencies 
> are declared in this file. When building I acquire a list of all the 
> upstream build dependencies which is parsed from this file. This is used to 
> install all needed libraries before building. However I also need to build 
> all the downstream dependencies, all the other projects that contain this 
> project in their RPM spec file.
>
> Using pipeline it is as easy to just call
> def buildResult = build dependency+"/"+BRANCH_NAME
> However I am not sure how to lookup these downstream dependencies. I would 
> need to check out each and every project within Jenkins, parse their spec 
> file and compute the dependencies. That is the only approach I could see 
> before me, but perhaps not the best approach. It seems like a heavy 
> operation to perform for each build to checkout 50+ projects.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/7e03efde-7736-4ffe-8b14-f37547a1322c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Pipeline: Building downstream dependencies

2016-06-03 Thread Sverre Moe
My first predicament:
In Multi-configuration build, one single executor can contain multiple 
parent build, but that single executor can only contain one 
configuration/axis/slave build at a time.

Using Pipeline I am not sure how to get it to work like this.
After my Build stage I try to schedule build of all the dependencies and it 
will wait until they finish. But the project already holds the executor.

ProjectA schedules ProjectB. ProjectA waits for ProjectB, ProjectB cannot 
start because ProjectA holds the executor on the same slave node. So they 
are both locked...
build '../ProjectB/master'

> Still waiting to schedule task
> Waiting for next available executor on master-sles11-x86_64
>

How can I do the same in Pipeline as I do with Multi-configuration parent. 
The parent node is is selected from the list of nodes the project is 
building on. I could probably solve this by making the Jenkins master as 
the parent node while building dependencies. Triggering other projects to 
build does not necessary need to be done on one the build nodes.

It would not make a good approach for me to increase the number of 
executors on the slave nodes. Since the build is removing/installing rpm 
packages in the build stage it could cause havoc having two builds doing 
this interchangeably. Each build must have the slave node system to itself 
while building.


My second predicament:
Each of my projects has an RPM spec file. Its BuildRequires dependencies 
are declared in this file. When building I acquire a list of all the 
upstream build dependencies which is parsed from this file. This is used to 
install all needed libraries before building. However I also need to build 
all the downstream dependencies, all the other projects that contain this 
project in their RPM spec file.

Using pipeline it is as easy to just call
def buildResult = build dependency+"/"+BRANCH_NAME
However I am not sure how to lookup these downstream dependencies. I would 
need to check out each and every project within Jenkins, parse their spec 
file and compute the dependencies. That is the only approach I could see 
before me, but perhaps not the best approach. It seems like a heavy 
operation to perform for each build to checkout 50+ projects.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/c5e83459-78be-411f-a67b-cb4908823185%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.