Re: Multiple pipelines in Jenkinsfile
So you have a git repository with pipeline (as script) and want to run them all?
Its possible, but Its somewhat a messy syntax.
# global variable
def Myclosure
Node {
# has to be in a node
Checkout scripts
Myclosure = # load scripts and call them
}
# call the closure.
Myclosure()
See triggering manual load
https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md
I dont think its possible to execute plain pipeline scripts. You always have to
define a function (closure) thats stored and later invoked
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/fa58662c-1b83-42f0-b3e1-8d011e6f3a81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Multiple pipelines in Jenkinsfile
So you have a git repository with pipeline (as script) and want to run them all?
Its possible, but Its somewhat a messy syntax.
# global variable
def Myclosure
Node {
# has to be in a node
Checkout scripts
Myclosure = # load scripts and call them
}
# call the closure.
Myclosure()
See triggering manual load
https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md
I dont think its possible to execute plain pipeline scripts. You always have to
define a function (closure) thats stored and later invoked
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/facc30f9-1133-4c57-8ba4-29ae2787f60e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Multiple pipelines in Jenkinsfile
Sorry, its not really clear to me what you expect. 1 Build = one Pipeline (execution) This Pipeline can be flexible and seperated horizontally with stages, and vertically with parallel execution. Further, what will be ultimately run can be scripted from multiple scripts (some could be "pipelines" on its own). Maybe you just mean the graphical representation? Am Sonntag, 29. Mai 2016 19:47:40 UTC+2 schrieb Bartłomiej Sacharski: > > I'm really hyped about the Jenkinsfiles - they make it much much easier to > document and preserve project configuration. > However, all the examples that I've seen seem to use single pipeline. > I've tried to define different stages in separate node blocks, however > they still were seen as a single pipeline. > > Is it possible to define multiple pipelines in a single Jenkinsfile? Or > maybe there's undocumented functionality for .jenkinsfile extension to > handle such cases? > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/bcbd232b-7d4f-480b-9539-33514723c233%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Pipelines, iterating maps and more headaches
Nope, I initially used groovysh for rapid testing, until I found out good
Groovy in Groovy is not necessarily good Groovy in Jenkins. Then I created
a job with a Pipeline Script an ran that violently.
AFAIK, the replay feature isnt mentioned in any of the documents
from https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin, you
should really mention it there to get people using it (I doubt many people
will dig through your blog if they want to start with pipelines)
Am Donnerstag, 28. April 2016 00:37:11 UTC+2 schrieb Patrick Wolf:
>
> Did make use of the "Replay" feature at all to troubleshoot your script,
> Norbert? Just curious.
>
> https://jenkins.io/blog/2016/04/14/replay-with-pipeline/
>
>
>
> On Wed, Apr 27, 2016 at 3:34 PM, Brian Ray > wrote:
>
>> Nice hackaround. I will try that in the uncooperative parts of my script.
>>
>> Ah yes, *def x* vs *x*. On the face of it the two declarations should be
>> identical--roughly typing *x* as an *Object*. But there are different
>> scoping implications
>> <http://stackoverflow.com/questions/15619216/groovy-scope-how-to-access-script-variable-in-a-method>
>>
>> for each.
>>
>>
>> On Wednesday, April 27, 2016 at 3:00:43 PM UTC-7, Norbert Lange wrote:
>>>
>>> Hi Brian,
>>>
>>> every single method in the "final foo" - including the String
>>> Constructor requires approval. I was hoping for a proper subset that would
>>> work within the sandbox.
>>> What I ended with (several dozen "Builds" later ) is using a helper
>>> function squeezing the map into a list, seems the most sane aproach so far
>>> =(
>>>
>>> Btw, whats the difference of using "def x" vs "x"?
>>>
>>> node {
>>>
>>> for (it2 in mapToList(depmap)) {
>>> name = it2[0]
>>> revision = it2[1]
>>> }
>>> }
>>> @NonCPS
>>> def mapToList(depmap) {
>>> def dlist = []
>>> for (entry in depmap) {
>>> dlist.add([entry.key, entry.value])
>>> }
>>> dlist
>>> }
>>>
>>>
>>>
>>>
>>> Am Mittwoch, 27. April 2016 18:26:51 UTC+2 schrieb Brian Ray:
>>>>
>>>> FWIW I recently replaced several C-style loops with *for ( x in y )*
>>>> for iterating over both lists and maps in CPS code and for the most part
>>>> conversion went fine. There were a couple of CPS sections where I could
>>>> not
>>>> use that construct and had to fall back on the C-loops and further do a
>>>> torturous cast to avoid a serialization error getting keys and values from
>>>> the map. I want to say *Set>* caused the exception
>>>> because *AbstractMap.SimpleEntry* and *.SimpleImmutableEntry* are
>>>> serializable, while *Set *is not, per the JDK.
>>>>
>>>> for ( int i = 0; i < myMap.size(); i++ ) {
>>>>
>>>> // hacktacular String() cloning to avoid NotSerializableException;
>>>> also
>>>> // hacktacular Map > Set > Array morph to enable C-style looping
>>>> final foo = new String( myMap.entrySet().toArray()[i].value )
>>>>
>>>> // do stuff with foo...
>>>>
>>>> }
>>>>
>>>>
>>>> Nonetheless as mentioned in another part of the script I had no problem
>>>> using the shorter alternative, nor accessing keys and values using
>>>> *myMap.key* and *myMap.value*. Not sure what the difference is with my
>>>> more stubborn loop.
>>>>
>>>> On Wednesday, April 27, 2016 at 8:21:45 AM UTC-7, Norbert Lange wrote:
>>>>>
>>>>>
>>>>>
>>>>> Am Mittwoch, 27. April 2016 16:41:40 UTC+2 schrieb Jesse Glick:
>>>>>>
>>>>>> On Tuesday, April 26, 2016 at 7:18:55 PM UTC-4, Norbert Lange wrote:
>>>>>>>
>>>>>>> There seem to be some arcane rules on how to iterate over some
>>>>>>> builtin Groovy/Java Types within a sandbox. I haven`t found a way
>>>>>>> that
>>>>>>> works without manually allowing the function.
>>>>>>
>>>>>>
>>>>>> Which methods did you need to approve? We can easily add them to the
>>>>>> default whitelist in the Script Security plugin. But anyway
>>>>>>
>>>>>&g
Re: Pipelines, iterating maps and more headaches
Hi Brian,
every single method in the "final foo" - including the String
Constructor requires approval. I was hoping for a proper subset that would
work within the sandbox.
What I ended with (several dozen "Builds" later ) is using a helper
function squeezing the map into a list, seems the most sane aproach so far
=(
Btw, whats the difference of using "def x" vs "x"?
node {
for (it2 in mapToList(depmap)) {
name = it2[0]
revision = it2[1]
}
}
@NonCPS
def mapToList(depmap) {
def dlist = []
for (entry in depmap) {
dlist.add([entry.key, entry.value])
}
dlist
}
Am Mittwoch, 27. April 2016 18:26:51 UTC+2 schrieb Brian Ray:
>
> FWIW I recently replaced several C-style loops with *for ( x in y )* for
> iterating over both lists and maps in CPS code and for the most part
> conversion went fine. There were a couple of CPS sections where I could not
> use that construct and had to fall back on the C-loops and further do a
> torturous cast to avoid a serialization error getting keys and values from
> the map. I want to say *Set>* caused the exception because
> *AbstractMap.SimpleEntry* and *.SimpleImmutableEntry* are serializable,
> while *Set *is not, per the JDK.
>
> for ( int i = 0; i < myMap.size(); i++ ) {
>
> // hacktacular String() cloning to avoid NotSerializableException; also
> // hacktacular Map > Set > Array morph to enable C-style looping
> final foo = new String( myMap.entrySet().toArray()[i].value )
>
> // do stuff with foo...
>
> }
>
>
> Nonetheless as mentioned in another part of the script I had no problem
> using the shorter alternative, nor accessing keys and values using
> *myMap.key* and *myMap.value*. Not sure what the difference is with my
> more stubborn loop.
>
> On Wednesday, April 27, 2016 at 8:21:45 AM UTC-7, Norbert Lange wrote:
>>
>>
>>
>> Am Mittwoch, 27. April 2016 16:41:40 UTC+2 schrieb Jesse Glick:
>>>
>>> On Tuesday, April 26, 2016 at 7:18:55 PM UTC-4, Norbert Lange wrote:
>>>>
>>>> There seem to be some arcane rules on how to iterate over some
>>>> builtin Groovy/Java Types within a sandbox. I haven`t found a way that
>>>> works without manually allowing the function.
>>>
>>>
>>> Which methods did you need to approve? We can easily add them to the
>>> default whitelist in the Script Security plugin. But anyway
>>>
>>> The map`s each (at least)
>>
>>
>>>
>>>> 2) Serialization issues for iterators.
>>>>
>>>
>>> `for (x : list) {…}` works as of `workflow-cps` 2.x. Other iterators do
>>> not yet work (outside a `@NonCPS` method). Probably fixing them is not
>>> hard, just have not gotten to it yet
>>>
>>
>> Yes, these issues I can very likely work around. For someone who is new
>> to Groovy and Jenkins sandboxing, a list of preferred methods would be very
>> welcome (the examples from the workflow libs are rather simple). There are
>> atleast 3 different ways to iterate over containers, and several variations
>> of those for maps.
>>
>>
>>>
>>>
>>>>
>>>> take the createDList
>>>> which seems to execute the code differently (throws errrors, need to
>>>> define a explicit variable)
>>>>
>>>
>>> Not sure what this is about. If you find something you think should work
>>> which does not work in a minimal reproducible script, please file a bug
>>> report for it.
>>>
>> Where? Is that a feature-not-bug of Groovy, an issue in Jenkins or some
>> Plugin? I was hoping for some feedback as I am not proficient in either of
>> those to pinpoint issues.
>> The code above should be able to explain the issue, the exact same method
>> body in the node scope works fine, the call will result in some message
>> about "it not defined". Similary there seems some issues with name clashes
>> (if variables in functions are named like those in the node scope), but it
>> mightve been some flukes during trial-and-error
>>
>>
>>>
>>>
>>>>
>>>> Are variables from Closures global?
>>>>
>>>
>>> Local `def` variables in a closure? Not sure what you are referring to
>>> here.
>>>
>> See last point, and the code were I can access the it variable after the
>> closure were its used (Noted with "// Weird !" )
>>
>>>
>>> The main problem you are presumably hitting is th
Re: Pipelines, iterating maps and more headaches
Am Mittwoch, 27. April 2016 16:41:40 UTC+2 schrieb Jesse Glick:
>
> On Tuesday, April 26, 2016 at 7:18:55 PM UTC-4, Norbert Lange wrote:
>>
>> There seem to be some arcane rules on how to iterate over some
>> builtin Groovy/Java Types within a sandbox. I haven`t found a way that
>> works without manually allowing the function.
>
>
> Which methods did you need to approve? We can easily add them to the
> default whitelist in the Script Security plugin. But anyway
>
> The map`s each (at least)
>
>> 2) Serialization issues for iterators.
>>
>
> `for (x : list) {…}` works as of `workflow-cps` 2.x. Other iterators do
> not yet work (outside a `@NonCPS` method). Probably fixing them is not
> hard, just have not gotten to it yet
>
Yes, these issues I can very likely work around. For someone who is new to
Groovy and Jenkins sandboxing, a list of preferred methods would be very
welcome (the examples from the workflow libs are rather simple). There are
atleast 3 different ways to iterate over containers, and several variations
of those for maps.
>
>
>>
>> take the createDList
>> which seems to execute the code differently (throws errrors, need to
>> define a explicit variable)
>>
>
> Not sure what this is about. If you find something you think should work
> which does not work in a minimal reproducible script, please file a bug
> report for it.
>
Where? Is that a feature-not-bug of Groovy, an issue in Jenkins or some
Plugin? I was hoping for some feedback as I am not proficient in either of
those to pinpoint issues.
The code above should be able to explain the issue, the exact same method
body in the node scope works fine, the call will result in some message
about "it not defined". Similary there seems some issues with name clashes
(if variables in functions are named like those in the node scope), but it
mightve been some flukes during trial-and-error
>
>
>>
>> Are variables from Closures global?
>>
>
> Local `def` variables in a closure? Not sure what you are referring to
> here.
>
See last point, and the code were I can access the it variable after the
closure were its used (Noted with "// Weird !" )
>
> The main problem you are presumably hitting is the well-known
> JENKINS-26481 <https://issues.jenkins-ci.org/browse/JENKINS-26481>. We
> are working on a fix, but in the meantime, do not use any method built into
> Groovy which takes a closure argument, such as `list.each {x -> …}` or
> `someText.eachLine {line -> …}`. Rather use a Java-style loop. (To be on
> the safe side, also avoid iterators, meaning use a C- or JDK 1.4-style loop
> with an index.)
>
Could you please post me the preferable code for iterating a map in this
way? (Not sure I fully understand the bug )
> Incidentally `it` does not currently work in closures as noted in
> JENKINS-33468
> <https://www.google.com/url?q=https%3A%2F%2Fissues.jenkins-ci.org%2Fbrowse%2FJENKINS-33468&sa=D&sntz=1&usg=AFQjCNHCpckU-LtRXqq1a2tEPCzNrYhPTw>;
>
> use an explicit parameter name instead.
>
That seems to be one of the issues I am fighting with, and might be that
the supposed name-clashes came from unfocused variations of the code.
Strangely it does seem to somewhat work in the node body?
Kind Regards,
Norbert
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/c8a00b42-49e3-4152-a624-ff8bb0b0518a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Pipelines, iterating maps and more headaches
Hello,
I trying to get comfortable with Pipelines, so far its a rather
unpleasant experience since I cant even get a simple script going.
1) There seem to be some arcane rules on how to iterate over some
builtin Groovy/Java Types within a sandbox. I haven`t found a way that
works without manually allowing the function.
2) Serialization issues for iterators.
3) Language rules seem inconsistent, for example take the createDList
which seems to execute the code differently (throws errrors, need to
define a explicit variable)
4) Are variables from Closures global??? Is this a Groovy thing or a bug???
There seem to be scarce information on how to write useful simple yet
non-trivial pipeline scripts, even the most basic builtin types can`t
be used freely.
I dont know how to use maps, or if there is a prefered way (which
doesnt needs approval). Layering out code into seperate functions to
not step into serialization issues often results in the very same
lines not working, it even appears there are clashes with variable
names across all scopes?. I hope you got some feedback or pointers on
the issues above.
Kind Regards,
Norbert
Example Pipeline Code follows.
This should fetch some additional modules (the url-mapping shouldnt be
coded in the script and come from a configuration file later), a file
.builddepencies should define name and potentially revision for a
project, should be able to override these (thus a map seems
fitting).All repositories should be checked out from the pipeline
script, remaining build can (and will) be run from shell scripts.
---
// Gonna be a separate, global function when its grown up
def repomap = [
'myscripts': 'ssh://git@localhost:10022/scripts.git',
'myprojekt': 'ssh://git@localhost:10022/myprojekt.git'
]
node {
// Mark the code checkout 'stage'
stage 'Checkout'
// Get some code from a GitHub repository
//git branch: 'master', url: repomap['myprojekt']
sh('echo "fake checkout: ' + repomap['myprojekt'] + '"')
sh('echo "myscripts=somebranch" > .builddepencies')
// Get name of needed modules
def depmap = [:]
readFile('.builddepencies').eachLine {
def m = (it =~ /([^=]*)=?(.*)/)
depmap[m[0][1].trim()]=m[0][2].trim()
m = null
}
// Just a check so it works (after script approval )
depmap.each {
println 'map: ' + it.key + '=' + it.value
}
// Weird !
println 'it still defined: ' + it.key
// Weirder, no name-scopes in functions?, why is this not working ??
createDList (depmap)
depmap.each {
name = it.key
revision = it.value
it = null // Ok, I could find my way around serialization issues..
dir (name) {
//git url: repomap[name], changelog: false, poll: false
sh('echo "fake checkout: ' + repomap[name] + '"')
}
}
println 'end'
}
@NonCPS
def createDList(depmap) {
// Need to change the iterator name, then this works?
depmap.each {
println 'map: ' + it.key + '=' + it.value
}
}
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CADYdroP99RG%2BcShBvVTbsCrrSs1oFDxRo%3DiYsRqTKcevhciBRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
