The following example should work (I did not test it), I my case I have
used maps like the “result” variable, that it is a simple map not
synchronized and stores data from all task but it is not read from the
different tasks. The other two cases “data” and “mapSync” uses concurrent
classes, they are thread safe and synchronized so you can share data across
tasks, I dunno is they are in the allow list for pipeline, if not you have
to approve it use in pipelines in the Jenkins config. Finally, the last
part of the pipeline uses nested parallel task, from my experience is not a
good idea, the parallel explosion of task is a little incontrolable and
there are other solution like launch a job from those task and inside that
job launch parallel task, in this way you only have 1 parallel level that
is easy to control and understand when something goes wrong.
import groovy.transform.Field
import java.util.concurrent.atomic.AtomicInteger
import java.util.concurrent.ConcurrentHashMap
@Field def results = [:]
@Field AtomicInteger data = new AtomicInteger()
@Filed ConcurrentHashMap mapSync = new ConcurrentHashMap<String,Integer>()
pipeline{
agent any
stages
{
stage(‘Parallel BuiLd’) {
steps {
script {
def i = 0
def builds = [:]
mapSync[“odd”] = 0
mapSync[“even”] = 0
stash name: ‘src’, include: ‘src/**’
//generate 1000 parallel block
for (i = 0; i<1000; i++) {
// make the Map of Closure
builds[“$i”] = {
results[“$i”] = 1
data++
if(i%2==0){
mapSync[“odd”] = mapSync[“odd”]++
} else {
mapSync[“even”] = mapSync[“even”]++
}
}
}
parallel builds
println results.toString()
println data
println mapSync
}
}
}
}
}
El sábado, 30 de octubre de 2021 a las 18:44:57 UTC+2, abstrakta escribió:
> Thanks for your reply.
> So parallel directive is like spawning some Java threads?Do you have any
> pipeline code can demo this thread safety issue and how to fix it
> using Java types that are thread safe ?
> I guess that some directive "stash unstash archive" should be thread
> safety internally.Because I find some articles use parallel unstash in
> different slave node without thread protection.Is my guess correct?I can't
> find any other articles that discuss this parallel thread safety issue.
> ---Original---
> *From:* "Ivan Fernandez Calvo"<[email protected]>
> *Date:* Sun, Oct 31, 2021 00:03 AM
> *To:* "Jenkins Users"<[email protected]>;
> *Subject:* Re: thread safety of scripted pipeline parallel and usage of
> nested parallel
>
> No, if you plan to use shared variables across parallel steps you should
> use Java types that are thread safe, if not you will have random/weird
> results. I have several pipelines that uses a map to store results in
> parallel steps
>
> El sábado, 30 de octubre de 2021 a las 14:18:29 UTC+2, abstrakta escribió:
>
>> Hi, Jenkins friends.I wish that I'm in the right place that post these
>> Jenkins usage question.
>>
>> I find that the Scriped Pipeline parallel works like threads.Unlike the
>> Declarative Pipeline parallel,the Scriped Pipeline parallel just use one
>> executor.Closure in parallel parameters works like a thread.
>> My question is:
>>
>> 1.Does Jenkins garantee the data thread safety of parallel closure
>> internally?
>>
>> 2.Does I need to care about the thread safety of the commands that
>> executes in scriped parallel closure?
>>
>> 3.Is there any limit of usage in the commands that executes in
>> parallel?Can I use nested scripted parallel? Why the documentation of
>> Declarative Pipeline parallel in Pipeline Syntax reference says that "Note
>> that a stage must have one and only one of steps, stages, parallel, or
>> matrix. It is not possible to nest a parallel or matrix block within a
>> stage directive if that stage directive is nested within a parallel or
>> matrix block itself."
>>
>> I test some nested Pipeline code that might cause thread race condition
>> many times.Jekins always give the right answer that shared data is modified
>> correctly.Is this thread safety garanteed in the design of Jenkins parallel
>> directive?
>>
>> Pipeline code like this:
>>
>> pipeline{
>> agent any
>> stages
>> {
>> stage('Parallel BuiLd') {
>> steps {
>> script {
>> def i = 0
>> def data = 0
>> def builds = [:]
>> stash name: 'src', include: 'src/**'
>> //generate 1000 parallel block
>> for (i = 0; i<1000; i++) {
>> // make the Map of Closure
>> builds["$i"] = {
>> //modify shared data, need thread mutex lock?
>> data++
>> //unstash or other command, need thread mutex
>> lock?
>> unstah name: 'src'
>> def tests = [:]
>> // ... generate tests Map
>> // Can I use nested parallel?
>> parallel tests
>> }
>> }
>> parallel builds
>> println data //It does always print 1000
>> }
>> }
>> }
>> }
>> }
>>
>> The variable data is always modified to 1000. So Jenkins garantee the
>> thread safety of parallel?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/0712b8c5-42dc-439f-a017-2a5ca45ad1e9n%40googlegroups.com
>
> <https://groups.google.com/d/msgid/jenkinsci-users/0712b8c5-42dc-439f-a017-2a5ca45ad1e9n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/fed01b0e-a04c-448b-9f32-3d2b3ffd6c40n%40googlegroups.com.
