Re: Approaches for sharing workspace in a pipeline

[Martin d'Anjou]

Producer code:
String root = "/opt/martin"
String workspace = "${root}/${env.JOB_NAME}/${env.BUILD_NUMBER}"
node() {
ws(workspace) {
writeFile file: 'file.txt', text: "1234"
}
}

Consumer code:
String root = "/opt/martin"
String folder = env.JOB_NAME.split('/')[0..-2].join('/')
String workspace = "${root}/${folder}/producer/${PRODUCER_BUILD_NUMBER}"
node() {
ws(workspace) {
String text = readFile 'file.txt'
echo text
}
}

The consumer has an input parameter called PRODUCER_BUILD_NUMBER, and that 
is of course annoying for users to type it in. What I want for this 
parameter is to give users the option to choose a producer build number, or 
if they leave it empty, the most recent successful producer build number is 
used. Finding this most recent producer build number programmatically is a 
bit of a challenge currently but in the past with freestyle jobs I have 
done it with curl. With the Workflow I would prefer to use a plugin based 
on a groovy RESTClient or the apache fluent api. If I understood how to 
return an object to the workflow (as opposed to perform a build action) 
perhaps I would be able to write a decent REST API plugin but I don't write 
plugins for a living so it would take me a lot of time to learn to do this. 
I am also aware of Jenkins Workflow - Creating a Class to Wrap Access to a 
Secured HTTP Endpoint 
,
 
but it uses the Global Library which is not a good long term solution yet 
as explained in JENKINS-31155 
.

Concurrent consumer builds: the workspace has to be copied before the build 
jumps into action, unless you are certain that no files will collide on 
write. For me copying the workspace is still cheaper than re-creating it 
from scratch, so I will go for a full copy. Code is TBD but the easiest is 
probably a simple unix copy inside an "sh" build step.

Martin

On Wednesday, December 30, 2015 at 5:24:26 AM UTC-5, Thomas Goeppel wrote:
>
> Hi John,
>
> your approach sounds interesting. Could you please share some Workflow 
> code that shows how you compute the Workspace location, use ws(), and pass 
> workspace locations to subsequent jobs? 
>
> When accessing workspaces in Workflow you're also facing the problem of 
> concurrent runs. How do you deal with that?
>
> Thanks in advance!
>
> Greetings,
> Thomas
>  
>
> On Wednesday, December 30, 2015 at 3:11:46 AM UTC+1, Martin d'Anjou wrote:
>>
>> On Wednesday, December 23, 2015 at 9:45:57 PM UTC-5, John D. Ament wrote:
>>>
>>> Hi,
>>>
>>> I was wondering if anyone had any best practices or tips to share on 
>>> have a common workspace for a pipeline job.
>>>
>>
>> I use Workflow job definitions, and I compute the location of the 
>> workspace in each job of the build chain before the actual build actions 
>> take place, using the ws() built step. Subsequent jobs in the build chain 
>> use the same workspace (the location can be computed). Subsequent jobs need 
>> not to be in the same workflow definition, nor in a formal pipeline (I do 
>> not use triggers nor pipeline plugins). I have enough input parameters for 
>> the subsequent jobs in the chain to locate the workspace of the previous 
>> job. I do not move nor archive data (I have thousands of files and 
>> gigabytes of data over NFS so I don't want to spend time moving it or 
>> archiving/extracting it).
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/df3d3729-603b-4e7b-acd2-228f94e20290%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Baptiste Mathus]

Would be great if y'all around could share the cool things you've done with
workflow in that examples/tutorial repo:
https://github.com/jenkinsci/workflow-examples

Thanks!

2015-12-30 11:26 GMT+01:00 Thomas Goeppel :

> Oh sorry, wrong addressee
>
> %s/John/Martin/ :-)
>
>
> On Wednesday, December 30, 2015 at 11:24:26 AM UTC+1, Thomas Goeppel wrote:
>>
>> Hi John,
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/53be2f41-9a45-4af5-a952-c49f53ef94b1%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Baptiste  MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS478cwFaLvdC0sanySC2EqVUOfJqyw16LiX3y9OX1kOGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Thomas Goeppel]

Oh sorry, wrong addressee  

%s/John/Martin/ :-)


On Wednesday, December 30, 2015 at 11:24:26 AM UTC+1, Thomas Goeppel wrote:
>
> Hi John,
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/53be2f41-9a45-4af5-a952-c49f53ef94b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Thomas Goeppel]

Hi John,

your approach sounds interesting. Could you please share some Workflow code 
that shows how you compute the Workspace location, use ws(), and pass 
workspace locations to subsequent jobs? 

When accessing workspaces in Workflow you're also facing the problem of 
concurrent runs. How do you deal with that?

Thanks in advance!

Greetings,
Thomas
 

On Wednesday, December 30, 2015 at 3:11:46 AM UTC+1, Martin d'Anjou wrote:
>
> On Wednesday, December 23, 2015 at 9:45:57 PM UTC-5, John D. Ament wrote:
>>
>> Hi,
>>
>> I was wondering if anyone had any best practices or tips to share on have 
>> a common workspace for a pipeline job.
>>
>
> I use Workflow job definitions, and I compute the location of the 
> workspace in each job of the build chain before the actual build actions 
> take place, using the ws() built step. Subsequent jobs in the build chain 
> use the same workspace (the location can be computed). Subsequent jobs need 
> not to be in the same workflow definition, nor in a formal pipeline (I do 
> not use triggers nor pipeline plugins). I have enough input parameters for 
> the subsequent jobs in the chain to locate the workspace of the previous 
> job. I do not move nor archive data (I have thousands of files and 
> gigabytes of data over NFS so I don't want to spend time moving it or 
> archiving/extracting it).
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/8724e85d-36e2-4e1d-a920-f9047587aada%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Martin d'Anjou]

On Wednesday, December 23, 2015 at 9:45:57 PM UTC-5, John D. Ament wrote:
>
> Hi,
>
> I was wondering if anyone had any best practices or tips to share on have 
> a common workspace for a pipeline job.
>

I use Workflow job definitions, and I compute the location of the workspace 
in each job of the build chain before the actual build actions take place, 
using the ws() built step. Subsequent jobs in the build chain use the same 
workspace (the location can be computed). Subsequent jobs need not to be in 
the same workflow definition, nor in a formal pipeline (I do not use 
triggers nor pipeline plugins). I have enough input parameters for the 
subsequent jobs in the chain to locate the workspace of the previous job. I 
do not move nor archive data (I have thousands of files and gigabytes of 
data over NFS so I don't want to spend time moving it or 
archiving/extracting it).

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/f5e82eba-3ac5-488e-9638-087d363bc199%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Brian Ray]

Ah yes, that help seems a little misleading with the wording "May take a 
*mapping* parameter ..."

On Sunday, December 27, 2015 at 8:02:36 PM UTC-8, John D. Ament wrote:
>
> Ok, so I think I got something.  If I use "**/*" on the unarchive step, I 
> get everything.  Not quite what I expected, but it works.
>
>
>
> On Thursday, December 24, 2015 at 1:20:16 PM UTC-5, John D. Ament wrote:
>>
>> Creating the archive doesn't seem to be an issue.
>>
>> Its the unarchive step where things don't quite work for me.  If I read 
>> this info, it implies that the mapping step is not required.  
>> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/resources/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStep/config.jelly#L31
>>
>> However, if mapping isn't set, an exception gets thrown: 
>> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L39
>>
>> And in fact, the expanding of the archive is keyed off of this mapping 
>> config.  
>> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L42
>>
>> It looks like it may be straight forward enough to add an unarchive all 
>> option, and if that works I may try that out instead.
>>
>> John
>>
>> On Thursday, December 24, 2015 at 12:55:30 PM UTC-5, Brian Ray wrote:
>>>
>>> Isn't the "get everything" Ant regex something like  or ***/**?
>>>
>>> If you continue looking at workflow, also check out the *stash* and 
>>> *unstash 
>>> *steps. Similar purpose and syntax but more applicable to intermediate 
>>> stages where you don't need to retain the artifacts. You can also refer to 
>>> the batch of *stash*ed artifacts by an arbitrary logical name for 
>>> downstream *unstash*ing.
>>>
>>> On Thursday, December 24, 2015 at 6:53:56 AM UTC-8, John D. Ament wrote:

 Hi,

 I definitely thought about workflow.  It looks promising.

 One question though - I can't seem to archive/unarchive everything.  It 
 looks like to use unarchiver you need to know the paths that will be 
 exposed, and instead I'd like to just get everything, including class 
 files.  Is that possible?

 John

 On Thursday, December 24, 2015 at 1:49:01 AM UTC-5, Baptiste Mathus 
 wrote:
>
> Hi John,
>
> Not sure what you call a pipeline job, do you mean 'workflow job'? or 
> do you use the term in a generic way and actually have many (freestyle) 
> jobs you're coordinating?
>
> If the latter, then it really seems like a use case for a workflow job 
> (using the workflow plugin). Using/archiving etc. artifacts and being 
> able 
> to share the ws during the build is gonna be both more natural and more 
> maintainable (and more robust because of the durability).
>
> My 2 cents
> Le 24 déc. 2015 3:46 AM, "John D. Ament"  a 
> écrit :
>
>> Hi,
>>
>> I was wondering if anyone had any best practices or tips to share on 
>> have a common workspace for a pipeline job.
>>
>> Basically, I have a series of pipeline jobs and I want them to have a 
>> single workspace for the duration of the job chain.  I compile the 
>> artifacts once, running unit tests, followed by a suite of integration 
>> and 
>> BDD tests.  It's a fairly complicated build, including generating an app 
>> server and minifying a lot of javascript for our UI.  Some of these 
>> steps 
>> are pretty long, and in total we have 4 pipeline steps.  I figure by 
>> doing 
>> this once, I would cut out about 40 minutes of rebuild time in my 
>> pipeline.
>>
>> One idea I had was to use the clone workspace plugin to copy them, 
>> https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
>> but it seems like this isn't pipeline sensitive since each step in the 
>> pipeline should be building the same commit.  I also thought about 
>> copying 
>> artifacts, but it seems like its a huge number of artifacts.  Could I 
>> build 
>> a zip with the contents?
>>
>> Any thoughts?
>>
>> Thanks!
>>
>> John
>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to jenkinsci-use...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

--

Re: Approaches for sharing workspace in a pipeline

[John D. Ament]

Ok, so I think I got something.  If I use "**/*" on the unarchive step, I 
get everything.  Not quite what I expected, but it works.



On Thursday, December 24, 2015 at 1:20:16 PM UTC-5, John D. Ament wrote:
>
> Creating the archive doesn't seem to be an issue.
>
> Its the unarchive step where things don't quite work for me.  If I read 
> this info, it implies that the mapping step is not required.  
> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/resources/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStep/config.jelly#L31
>
> However, if mapping isn't set, an exception gets thrown: 
> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L39
>
> And in fact, the expanding of the archive is keyed off of this mapping 
> config.  
> https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L42
>
> It looks like it may be straight forward enough to add an unarchive all 
> option, and if that works I may try that out instead.
>
> John
>
> On Thursday, December 24, 2015 at 12:55:30 PM UTC-5, Brian Ray wrote:
>>
>> Isn't the "get everything" Ant regex something like  or ***/**?
>>
>> If you continue looking at workflow, also check out the *stash* and *unstash 
>> *steps. Similar purpose and syntax but more applicable to intermediate 
>> stages where you don't need to retain the artifacts. You can also refer to 
>> the batch of *stash*ed artifacts by an arbitrary logical name for 
>> downstream *unstash*ing.
>>
>> On Thursday, December 24, 2015 at 6:53:56 AM UTC-8, John D. Ament wrote:
>>>
>>> Hi,
>>>
>>> I definitely thought about workflow.  It looks promising.
>>>
>>> One question though - I can't seem to archive/unarchive everything.  It 
>>> looks like to use unarchiver you need to know the paths that will be 
>>> exposed, and instead I'd like to just get everything, including class 
>>> files.  Is that possible?
>>>
>>> John
>>>
>>> On Thursday, December 24, 2015 at 1:49:01 AM UTC-5, Baptiste Mathus 
>>> wrote:

 Hi John,

 Not sure what you call a pipeline job, do you mean 'workflow job'? or 
 do you use the term in a generic way and actually have many (freestyle) 
 jobs you're coordinating?

 If the latter, then it really seems like a use case for a workflow job 
 (using the workflow plugin). Using/archiving etc. artifacts and being able 
 to share the ws during the build is gonna be both more natural and more 
 maintainable (and more robust because of the durability).

 My 2 cents
 Le 24 déc. 2015 3:46 AM, "John D. Ament"  a 
 écrit :

> Hi,
>
> I was wondering if anyone had any best practices or tips to share on 
> have a common workspace for a pipeline job.
>
> Basically, I have a series of pipeline jobs and I want them to have a 
> single workspace for the duration of the job chain.  I compile the 
> artifacts once, running unit tests, followed by a suite of integration 
> and 
> BDD tests.  It's a fairly complicated build, including generating an app 
> server and minifying a lot of javascript for our UI.  Some of these steps 
> are pretty long, and in total we have 4 pipeline steps.  I figure by 
> doing 
> this once, I would cut out about 40 minutes of rebuild time in my 
> pipeline.
>
> One idea I had was to use the clone workspace plugin to copy them, 
> https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
> but it seems like this isn't pipeline sensitive since each step in the 
> pipeline should be building the same commit.  I also thought about 
> copying 
> artifacts, but it seems like its a huge number of artifacts.  Could I 
> build 
> a zip with the contents?
>
> Any thoughts?
>
> Thanks!
>
> John
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/

Re: Approaches for sharing workspace in a pipeline

[John D. Ament]

Creating the archive doesn't seem to be an issue.

Its the unarchive step where things don't quite work for me.  If I read 
this info, it implies that the mapping step is not required. 
 
https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/resources/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStep/config.jelly#L31

However, if mapping isn't set, an exception gets 
thrown: 
https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L39

And in fact, the expanding of the archive is keyed off of this mapping 
config. 
 
https://github.com/jenkinsci/workflow-plugin/blob/master/basic-steps/src/main/java/org/jenkinsci/plugins/workflow/steps/ArtifactUnarchiverStepExecution.java#L42

It looks like it may be straight forward enough to add an unarchive all 
option, and if that works I may try that out instead.

John

On Thursday, December 24, 2015 at 12:55:30 PM UTC-5, Brian Ray wrote:
>
> Isn't the "get everything" Ant regex something like  or ***/**?
>
> If you continue looking at workflow, also check out the *stash* and *unstash 
> *steps. Similar purpose and syntax but more applicable to intermediate 
> stages where you don't need to retain the artifacts. You can also refer to 
> the batch of *stash*ed artifacts by an arbitrary logical name for 
> downstream *unstash*ing.
>
> On Thursday, December 24, 2015 at 6:53:56 AM UTC-8, John D. Ament wrote:
>>
>> Hi,
>>
>> I definitely thought about workflow.  It looks promising.
>>
>> One question though - I can't seem to archive/unarchive everything.  It 
>> looks like to use unarchiver you need to know the paths that will be 
>> exposed, and instead I'd like to just get everything, including class 
>> files.  Is that possible?
>>
>> John
>>
>> On Thursday, December 24, 2015 at 1:49:01 AM UTC-5, Baptiste Mathus wrote:
>>>
>>> Hi John,
>>>
>>> Not sure what you call a pipeline job, do you mean 'workflow job'? or do 
>>> you use the term in a generic way and actually have many (freestyle) jobs 
>>> you're coordinating?
>>>
>>> If the latter, then it really seems like a use case for a workflow job 
>>> (using the workflow plugin). Using/archiving etc. artifacts and being able 
>>> to share the ws during the build is gonna be both more natural and more 
>>> maintainable (and more robust because of the durability).
>>>
>>> My 2 cents
>>> Le 24 déc. 2015 3:46 AM, "John D. Ament"  a écrit :
>>>
 Hi,

 I was wondering if anyone had any best practices or tips to share on 
 have a common workspace for a pipeline job.

 Basically, I have a series of pipeline jobs and I want them to have a 
 single workspace for the duration of the job chain.  I compile the 
 artifacts once, running unit tests, followed by a suite of integration and 
 BDD tests.  It's a fairly complicated build, including generating an app 
 server and minifying a lot of javascript for our UI.  Some of these steps 
 are pretty long, and in total we have 4 pipeline steps.  I figure by doing 
 this once, I would cut out about 40 minutes of rebuild time in my pipeline.

 One idea I had was to use the clone workspace plugin to copy them, 
 https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
 but it seems like this isn't pipeline sensitive since each step in the 
 pipeline should be building the same commit.  I also thought about copying 
 artifacts, but it seems like its a huge number of artifacts.  Could I 
 build 
 a zip with the contents?

 Any thoughts?

 Thanks!

 John

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Jenkins Users" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to jenkinsci-use...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/3a074a74-c6e4-4354-a75e-4b286ebff524%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Brian Ray]

Isn't the "get everything" Ant regex something like  or ***/**?

If you continue looking at workflow, also check out the *stash* and *unstash 
*steps. Similar purpose and syntax but more applicable to intermediate 
stages where you don't need to retain the artifacts. You can also refer to 
the batch of *stash*ed artifacts by an arbitrary logical name for 
downstream *unstash*ing.

On Thursday, December 24, 2015 at 6:53:56 AM UTC-8, John D. Ament wrote:
>
> Hi,
>
> I definitely thought about workflow.  It looks promising.
>
> One question though - I can't seem to archive/unarchive everything.  It 
> looks like to use unarchiver you need to know the paths that will be 
> exposed, and instead I'd like to just get everything, including class 
> files.  Is that possible?
>
> John
>
> On Thursday, December 24, 2015 at 1:49:01 AM UTC-5, Baptiste Mathus wrote:
>>
>> Hi John,
>>
>> Not sure what you call a pipeline job, do you mean 'workflow job'? or do 
>> you use the term in a generic way and actually have many (freestyle) jobs 
>> you're coordinating?
>>
>> If the latter, then it really seems like a use case for a workflow job 
>> (using the workflow plugin). Using/archiving etc. artifacts and being able 
>> to share the ws during the build is gonna be both more natural and more 
>> maintainable (and more robust because of the durability).
>>
>> My 2 cents
>> Le 24 déc. 2015 3:46 AM, "John D. Ament"  a écrit :
>>
>>> Hi,
>>>
>>> I was wondering if anyone had any best practices or tips to share on 
>>> have a common workspace for a pipeline job.
>>>
>>> Basically, I have a series of pipeline jobs and I want them to have a 
>>> single workspace for the duration of the job chain.  I compile the 
>>> artifacts once, running unit tests, followed by a suite of integration and 
>>> BDD tests.  It's a fairly complicated build, including generating an app 
>>> server and minifying a lot of javascript for our UI.  Some of these steps 
>>> are pretty long, and in total we have 4 pipeline steps.  I figure by doing 
>>> this once, I would cut out about 40 minutes of rebuild time in my pipeline.
>>>
>>> One idea I had was to use the clone workspace plugin to copy them, 
>>> https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
>>> but it seems like this isn't pipeline sensitive since each step in the 
>>> pipeline should be building the same commit.  I also thought about copying 
>>> artifacts, but it seems like its a huge number of artifacts.  Could I build 
>>> a zip with the contents?
>>>
>>> Any thoughts?
>>>
>>> Thanks!
>>>
>>> John
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkinsci-use...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/efb5de9c-5c33-46dd-a443-4761652f7c20%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[John D. Ament]

Hi,

I definitely thought about workflow.  It looks promising.

One question though - I can't seem to archive/unarchive everything.  It 
looks like to use unarchiver you need to know the paths that will be 
exposed, and instead I'd like to just get everything, including class 
files.  Is that possible?

John

On Thursday, December 24, 2015 at 1:49:01 AM UTC-5, Baptiste Mathus wrote:
>
> Hi John,
>
> Not sure what you call a pipeline job, do you mean 'workflow job'? or do 
> you use the term in a generic way and actually have many (freestyle) jobs 
> you're coordinating?
>
> If the latter, then it really seems like a use case for a workflow job 
> (using the workflow plugin). Using/archiving etc. artifacts and being able 
> to share the ws during the build is gonna be both more natural and more 
> maintainable (and more robust because of the durability).
>
> My 2 cents
> Le 24 déc. 2015 3:46 AM, "John D. Ament"  > a écrit :
>
>> Hi,
>>
>> I was wondering if anyone had any best practices or tips to share on have 
>> a common workspace for a pipeline job.
>>
>> Basically, I have a series of pipeline jobs and I want them to have a 
>> single workspace for the duration of the job chain.  I compile the 
>> artifacts once, running unit tests, followed by a suite of integration and 
>> BDD tests.  It's a fairly complicated build, including generating an app 
>> server and minifying a lot of javascript for our UI.  Some of these steps 
>> are pretty long, and in total we have 4 pipeline steps.  I figure by doing 
>> this once, I would cut out about 40 minutes of rebuild time in my pipeline.
>>
>> One idea I had was to use the clone workspace plugin to copy them, 
>> https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
>> but it seems like this isn't pipeline sensitive since each step in the 
>> pipeline should be building the same commit.  I also thought about copying 
>> artifacts, but it seems like its a huge number of artifacts.  Could I build 
>> a zip with the contents?
>>
>> Any thoughts?
>>
>> Thanks!
>>
>> John
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-use...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/84538971-9e6d-4fc0-9f5d-beee2e68262b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Approaches for sharing workspace in a pipeline

[Baptiste Mathus]

Hi John,

Not sure what you call a pipeline job, do you mean 'workflow job'? or do
you use the term in a generic way and actually have many (freestyle) jobs
you're coordinating?

If the latter, then it really seems like a use case for a workflow job
(using the workflow plugin). Using/archiving etc. artifacts and being able
to share the ws during the build is gonna be both more natural and more
maintainable (and more robust because of the durability).

My 2 cents
Le 24 déc. 2015 3:46 AM, "John D. Ament"  a écrit :

> Hi,
>
> I was wondering if anyone had any best practices or tips to share on have
> a common workspace for a pipeline job.
>
> Basically, I have a series of pipeline jobs and I want them to have a
> single workspace for the duration of the job chain.  I compile the
> artifacts once, running unit tests, followed by a suite of integration and
> BDD tests.  It's a fairly complicated build, including generating an app
> server and minifying a lot of javascript for our UI.  Some of these steps
> are pretty long, and in total we have 4 pipeline steps.  I figure by doing
> this once, I would cut out about 40 minutes of rebuild time in my pipeline.
>
> One idea I had was to use the clone workspace plugin to copy them,
> https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin,
> but it seems like this isn't pipeline sensitive since each step in the
> pipeline should be building the same commit.  I also thought about copying
> artifacts, but it seems like its a huge number of artifacts.  Could I build
> a zip with the contents?
>
> Any thoughts?
>
> Thanks!
>
> John
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS51_Avn9Vuzq%3Dpi0uZX0N191Jxr-YZz44W61-tFkJwBbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Approaches for sharing workspace in a pipeline

[John D. Ament]

Hi,

I was wondering if anyone had any best practices or tips to share on have a 
common workspace for a pipeline job.

Basically, I have a series of pipeline jobs and I want them to have a 
single workspace for the duration of the job chain.  I compile the 
artifacts once, running unit tests, followed by a suite of integration and 
BDD tests.  It's a fairly complicated build, including generating an app 
server and minifying a lot of javascript for our UI.  Some of these steps 
are pretty long, and in total we have 4 pipeline steps.  I figure by doing 
this once, I would cut out about 40 minutes of rebuild time in my pipeline.

One idea I had was to use the clone workspace plugin to copy 
them, https://wiki.jenkins-ci.org/display/JENKINS/Clone+Workspace+SCM+Plugin, 
but it seems like this isn't pipeline sensitive since each step in the 
pipeline should be building the same commit.  I also thought about copying 
artifacts, but it seems like its a huge number of artifacts.  Could I build 
a zip with the contents?

Any thoughts?

Thanks!

John

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/3c25d084-56c8-4242-a6e0-a0b347ea0af8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.