Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

So the repo has been moved by Kohsuke.
Nicolas also reverted the README so it is no longer deprecated. Now we are 
good, I believe

BR, Oleg


On Thursday, March 1, 2018 at 8:23:15 PM UTC+1, Kohsuke Kawaguchi wrote:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some 
> Jenkins users and I felt something like this might move the needle for them 
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be 
> important for you, does it miss any key points for you? If you mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/deaf7b53-7f85-4455-8777-6e4709b261fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[nicolas de loof]

Le jeu. 4 oct. 2018 à 08:04, Oleg Nenashev  a
écrit :

> Hi Nicolas,
>
> I am ready to execute on the request. Please confirm that all the data is
> backed up in your fork.
>

Confirmed
Please do


>
> and fork https://github.com/kohsuke/jenkinsfile-runner
>> 
>> as a replacement.
>>
>
> No fork, please, it will only contribute to the confusion.
> My proposal is to move the project then. I will work with Kohsuke to do
> that once the target repo is deleted
>

+1


>
> BR, Oleg
>
> On Tuesday, October 2, 2018 at 5:45:38 PM UTC+2, nicolas de loof wrote:
>>
>> all those changes are related to my version's README.md
>> ,
>> no impact on KK's repository
>>
>> Le mar. 2 oct. 2018 à 17:33, Jesse Glick  a écrit :
>>
>>> On Tue, Oct 2, 2018 at 10:13 AM nicolas de loof
>>>  wrote:
>>> > DELETE the jenkinsci/jenkinsfile-runner repository
>>>
>>> What about
>>>
>>>
>>> https://github.com/ndeloof/jenkinsfile-runner/compare/master...jenkinsci:master
>>>
>>> ? That should be upstreamed first, no?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to jenkinsci-de...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0rfAkiB8vMAczAApGQm-EVGK3_SFsPiR6sDLz0RreSOw%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> --
>> Nicolas De Loof
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/b6870795-6ef3-419d-9c37-793248006c33%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Nicolas De Loof

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznNOWAJo_jiqM0hJhOC%2BmbYVmnaBK1C9qFDCTi1h5QFUQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

Hi Nicolas,

I am ready to execute on the request. Please confirm that all the data is 
backed up in your fork.

and fork https://github.com/kohsuke/jenkinsfile-runner 
> 
>  
> as a replacement.
>

No fork, please, it will only contribute to the confusion.
My proposal is to move the project then. I will work with Kohsuke to do 
that once the target repo is deleted

BR, Oleg

On Tuesday, October 2, 2018 at 5:45:38 PM UTC+2, nicolas de loof wrote:
>
> all those changes are related to my version's README.md 
> ,
>  
> no impact on KK's repository 
>
> Le mar. 2 oct. 2018 à 17:33, Jesse Glick  > a écrit :
>
>> On Tue, Oct 2, 2018 at 10:13 AM nicolas de loof
>> > wrote:
>> > DELETE the jenkinsci/jenkinsfile-runner repository
>>
>> What about
>>
>>
>> https://github.com/ndeloof/jenkinsfile-runner/compare/master...jenkinsci:master
>>
>> ? That should be upstreamed first, no?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-de...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0rfAkiB8vMAczAApGQm-EVGK3_SFsPiR6sDLz0RreSOw%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> Nicolas De Loof
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/b6870795-6ef3-419d-9c37-793248006c33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[nicolas de loof]

all those changes are related to my version's README.md
,
no impact on KK's repository

Le mar. 2 oct. 2018 à 17:33, Jesse Glick  a écrit :

> On Tue, Oct 2, 2018 at 10:13 AM nicolas de loof
>  wrote:
> > DELETE the jenkinsci/jenkinsfile-runner repository
>
> What about
>
>
> https://github.com/ndeloof/jenkinsfile-runner/compare/master...jenkinsci:master
>
> ? That should be upstreamed first, no?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0rfAkiB8vMAczAApGQm-EVGK3_SFsPiR6sDLz0RreSOw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Nicolas De Loof

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzmhsk7L04Rch2y5SieunEbCF4Bjuk9KMoaajZjZUgYWDA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

On Tue, Oct 2, 2018 at 10:13 AM nicolas de loof
 wrote:
> DELETE the jenkinsci/jenkinsfile-runner repository

What about

https://github.com/ndeloof/jenkinsfile-runner/compare/master...jenkinsci:master

? That should be upstreamed first, no?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0rfAkiB8vMAczAApGQm-EVGK3_SFsPiR6sDLz0RreSOw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[James Rawlings]

The Jenkins X project is using custom war packager along with 
https://github.com/kohsuke/jenkinsfile-runner 
 so this get’s a +1 from us.

Thanks, James.

> On 2 Oct 2018, at 15:13, nicolas de loof  wrote:
> 
> 
> Based on the fact custom-war-packager already support KK's approach 
> I'm asking jenkinsci organisation owners to DELETE the 
> jenkinsci/jenkinsfile-runner repository, and fork 
> https://github.com/kohsuke/jenkinsfile-runner as a replacement.
> 
> My own version (https://github.com/ndeloof/jenkinsfile-runner) will stay 
> there for those interested in trying to reconcile both.
> 
> 
> 
> Le jeudi 6 septembre 2018 17:10:52 UTC+2, Liam Newman a écrit :
> Oleg,
> +1 for cleanup, definitely.  The path you propose overall sounds good to me. 
> 
> On Thursday, September 6, 2018 at 6:40:23 AM UTC-7, Oleg Nenashev wrote:
> Obviously, step (5) is a bad idea then. Could we just have 2 repos without 
> "fork" relations then?
> If the codebase is not shared, it would help.
> 
> As you can guess I have my strong opinion about classloader voodoo in KK's 
> approach to a jenkinsfile-runner CLI.
> Fine for PoC IMHO. It improves performance, and it may be useful in cases 
> when we want to have fast builds with really short initialization times. I 
> was doing some performance hacks for similar purposes a while ago (actually 
> some even weirder hacks). Obviously, a final implementation should not be 
> based on Jenkins Test Harness and JenkinsRule, some Jenkins core patches will 
> likely be needed.
> 
> 
> On Thu, Sep 6, 2018 at 3:31 PM nicolas de loof > 
> wrote:
> please note ndeloof/jenkinsfile-runner is a complete different design vs KK 
> initial prototype
> they don't share architecture nor git history (only few java classes)
> 
> so you'll have to choose one approach over the other
> 
> As you can guess I have my strong opinion about classloader voodoo in KK's 
> approach to a jenkinsfile-runner CLI.
> 
> Le jeu. 6 sept. 2018 à 15:27, Baptiste Mathus > a écrit :
> +1 for cleaning up. Like we recommend for plugins, it would be nice that 
> canonical repo location is more clearly the one under jenkinsci org
> 
> Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev > a écrit :
> Hi all,
> 
> It seems we have a splitbrain issue in this project.
> Currently there is the following fork tree:
> https://github.com/kohsuke/jenkinsfile-runner 
> 
> https://github.com/ndeloof/jenkinsfile-runner 
> 
> https://github.com/jenkinsci/jenkinsfile-runner 
> 
> Jenkinsfile Runner in Kohsuke's repo has the most number of stars + there is 
> a number of issues and pull requests. It makes the jenkinsci repo 
> unsearchable, and it can also confuse users looking for a place to report 
> issues / propose patches.
> 
> I propose to do the following:
> Remove https://github.com/jenkinsci/jenkinsfile-runner 
>  and evacute the branches 
> somewhere
> Move https://github.com/kohsuke/jenkinsfile-runner 
>  to jenkinsci org (yes, move, 
> not fork)
> Rename the current master branch to "kohsuke-master"
> Reintegrate the current https://github.com/jenkinsci/jenkinsfile-runner 
>  state as a master branch
> See if "master" and "kohsuke-master" can be merged
> WDYT?
> 
> BR, Oleg
> 
> On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi wrote:
> Ouch, that's a shame. It looked like an interesting project, I hope my 
> writing to you didn't trigger that.
> 
> You say "stress builds and also to burn in our build agents" -- can you 
> elaborate on that? It sounds like you are trying to warm up a cache or 
> something, but I'm not sure what that means in the context of builds.
> 
> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw > wrote:
> Hi Kohsuke,
> 
> Thanks for getting in touch. I've had to remove that repo temporarily at the 
> request of my former employer. It looks like we are trying to solve the same 
> problem. I wanted to use jenkinless to stress builds and also to burn in our 
> build agents. It's more of a convenience tool. Running the slave using docker 
> in docker worked really well. I might try and turn this into an executable 
> that can be stored in the repo alongside code. 
> 
> Tom
> 
> 
> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi, > wrote:
> Hi, Thomas,
> 
> Tyler passed me a link to your project 
> https://github.com/tomwillfixit/jenkinless 
> , which is in a similar space 
> with my project of the day called Jenkinsfile Runner.
> 
> I haven't studied your project carefully yet, but I already see some 
> interesting ingredients like memcached that I have no idea what you use it 
> for :-)   I'd love to hear from you the philosophy & use 

Re: POTD: Jenkinsfile Runner

[nicolas de loof]

Based on the fact custom-war-packager already support KK's approach 
I'm asking jenkinsci organisation owners to DELETE the 
jenkinsci/jenkinsfile-runner repository, and 
fork https://github.com/kohsuke/jenkinsfile-runner as a replacement.

My own version (https://github.com/ndeloof/jenkinsfile-runner) will stay 
there for those interested in trying to reconcile both.



Le jeudi 6 septembre 2018 17:10:52 UTC+2, Liam Newman a écrit :
>
> Oleg,
> +1 for cleanup, definitely.  The path you propose overall sounds good to 
> me. 
>
> On Thursday, September 6, 2018 at 6:40:23 AM UTC-7, Oleg Nenashev wrote:
>>
>> Obviously, step (5) is a bad idea then. Could we just have 2 repos 
>> without "fork" relations then?
>> If the codebase is not shared, it would help.
>>
>> As you can guess I have my strong opinion about classloader voodoo in 
>>> KK's approach to a jenkinsfile-runner CLI.
>>>
>> Fine for PoC IMHO. It improves performance, and it may be useful in cases 
>> when we want to have fast builds with really short initialization times. I 
>> was doing some performance hacks for similar purposes a while ago (actually 
>> some even weirder hacks). Obviously, a final implementation should not be 
>> based on Jenkins Test Harness and JenkinsRule, some Jenkins core patches 
>> will likely be needed.
>>
>>
>> On Thu, Sep 6, 2018 at 3:31 PM nicolas de loof  
>> wrote:
>>
>>> please note ndeloof/jenkinsfile-runner is a complete different design vs 
>>> KK initial prototype
>>> they don't share architecture nor git history (only few java classes)
>>>
>>> so you'll have to choose one approach over the other
>>>
>>> As you can guess I have my strong opinion about classloader voodoo in 
>>> KK's approach to a jenkinsfile-runner CLI.
>>>
>>> Le jeu. 6 sept. 2018 à 15:27, Baptiste Mathus  a 
>>> écrit :
>>>
 +1 for cleaning up. Like we recommend for plugins, it would be nice 
 that canonical repo location is more clearly the one under jenkinsci org

 Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev  a 
 écrit :

> Hi all,
>
> It seems we have a splitbrain issue in this project.
> Currently there is the following fork tree:
>
>- https://github.com/kohsuke/jenkinsfile-runner
>   - https://github.com/ndeloof/jenkinsfile-runner
>  - https://github.com/jenkinsci/jenkinsfile-runner
>  
> Jenkinsfile Runner in Kohsuke's repo has the most number of stars + 
> there is a number of issues and pull requests. It makes the jenkinsci 
> repo 
> unsearchable, and it can also confuse users looking for a place to report 
> issues / propose patches.
>
> I propose to do the following:
>
>1. Remove https://github.com/jenkinsci/jenkinsfile-runner and 
>evacute the branches somewhere
>2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci 
>org (yes, move, not fork)
>3. Rename the current master branch to "kohsuke-master"
>4. Reintegrate the current 
>https://github.com/jenkinsci/jenkinsfile-runner state as a master 
>branch
>5. See if "master" and "kohsuke-master" can be merged
>
> WDYT?
>
> BR, Oleg
>
> On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi 
> wrote:
>>
>> Ouch, that's a shame. It looked like an interesting project, I hope 
>> my writing to you didn't trigger that.
>>
>> You say "stress builds and also to burn in our build agents" -- can 
>> you elaborate on that? It sounds like you are trying to warm up a cache 
>> or 
>> something, but I'm not sure what that means in the context of builds.
>>
>> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:
>>
>>> Hi Kohsuke,
>>>
>>> Thanks for getting in touch. I've had to remove that repo 
>>> temporarily at the request of my former employer. It looks like we are 
>>> trying to solve the same problem. I wanted to use jenkinless to stress 
>>> builds and also to burn in our build agents. It's more of a convenience 
>>> tool. Running the slave using docker in docker worked really well. I 
>>> might 
>>> try and turn this into an executable that can be stored in the repo 
>>> alongside code. 
>>>
>>> Tom
>>>
>>>
>>> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  
>>> wrote:
>>>
 Hi, Thomas,

 Tyler passed me a link to your project 
 https://github.com/tomwillfixit/jenkinless, which is in a similar 
 space with my project of the day called Jenkinsfile Runner.

 I haven't studied your project carefully yet, but I already see 
 some interesting ingredients like memcached that I have no idea what 
 you 
 use it for :-)   I'd love to hear from you the philosophy & use cases 
 that 
 led to it. And I'd also love to hear what you think of Jenkinsfile 

Re: POTD: Jenkinsfile Runner

[Liam Newman]

Oleg,
+1 for cleanup, definitely.  The path you propose overall sounds good to 
me. 

On Thursday, September 6, 2018 at 6:40:23 AM UTC-7, Oleg Nenashev wrote:
>
> Obviously, step (5) is a bad idea then. Could we just have 2 repos without 
> "fork" relations then?
> If the codebase is not shared, it would help.
>
> As you can guess I have my strong opinion about classloader voodoo in KK's 
>> approach to a jenkinsfile-runner CLI.
>>
> Fine for PoC IMHO. It improves performance, and it may be useful in cases 
> when we want to have fast builds with really short initialization times. I 
> was doing some performance hacks for similar purposes a while ago (actually 
> some even weirder hacks). Obviously, a final implementation should not be 
> based on Jenkins Test Harness and JenkinsRule, some Jenkins core patches 
> will likely be needed.
>
>
> On Thu, Sep 6, 2018 at 3:31 PM nicolas de loof  > wrote:
>
>> please note ndeloof/jenkinsfile-runner is a complete different design vs 
>> KK initial prototype
>> they don't share architecture nor git history (only few java classes)
>>
>> so you'll have to choose one approach over the other
>>
>> As you can guess I have my strong opinion about classloader voodoo in 
>> KK's approach to a jenkinsfile-runner CLI.
>>
>> Le jeu. 6 sept. 2018 à 15:27, Baptiste Mathus > > a écrit :
>>
>>> +1 for cleaning up. Like we recommend for plugins, it would be nice that 
>>> canonical repo location is more clearly the one under jenkinsci org
>>>
>>> Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev >> > a écrit :
>>>
 Hi all,

 It seems we have a splitbrain issue in this project.
 Currently there is the following fork tree:

- https://github.com/kohsuke/jenkinsfile-runner
   - https://github.com/ndeloof/jenkinsfile-runner
  - https://github.com/jenkinsci/jenkinsfile-runner
  
 Jenkinsfile Runner in Kohsuke's repo has the most number of stars + 
 there is a number of issues and pull requests. It makes the jenkinsci repo 
 unsearchable, and it can also confuse users looking for a place to report 
 issues / propose patches.

 I propose to do the following:

1. Remove https://github.com/jenkinsci/jenkinsfile-runner and 
evacute the branches somewhere
2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci 
org (yes, move, not fork)
3. Rename the current master branch to "kohsuke-master"
4. Reintegrate the current 
https://github.com/jenkinsci/jenkinsfile-runner state as a master 
branch
5. See if "master" and "kohsuke-master" can be merged

 WDYT?

 BR, Oleg

 On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi 
 wrote:
>
> Ouch, that's a shame. It looked like an interesting project, I hope my 
> writing to you didn't trigger that.
>
> You say "stress builds and also to burn in our build agents" -- can 
> you elaborate on that? It sounds like you are trying to warm up a cache 
> or 
> something, but I'm not sure what that means in the context of builds.
>
> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:
>
>> Hi Kohsuke,
>>
>> Thanks for getting in touch. I've had to remove that repo temporarily 
>> at the request of my former employer. It looks like we are trying to 
>> solve 
>> the same problem. I wanted to use jenkinless to stress builds and also 
>> to 
>> burn in our build agents. It's more of a convenience tool. Running the 
>> slave using docker in docker worked really well. I might try and turn 
>> this 
>> into an executable that can be stored in the repo alongside code. 
>>
>> Tom
>>
>>
>> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  
>> wrote:
>>
>>> Hi, Thomas,
>>>
>>> Tyler passed me a link to your project 
>>> https://github.com/tomwillfixit/jenkinless, which is in a similar 
>>> space with my project of the day called Jenkinsfile Runner.
>>>
>>> I haven't studied your project carefully yet, but I already see some 
>>> interesting ingredients like memcached that I have no idea what you use 
>>> it 
>>> for :-)   I'd love to hear from you the philosophy & use cases that led 
>>> to 
>>> it. And I'd also love to hear what you think of Jenkinsfile Runner. I 
>>> think 
>>> we have similar interests here, are there any opportunity to 
>>> collaborate?
>>>
>>> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi  
>>> wrote:
>>>
 And of course I forgot to have the link to the project! 
 https://github.com/kohsuke/jenkinsfile-runner 


 On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi  
 wrote:

> Jenkinsfile Runner is an experiment to package Jenkins pipeline 
> execution as a command line tool. The intend use 

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

Obviously, step (5) is a bad idea then. Could we just have 2 repos without
"fork" relations then?
If the codebase is not shared, it would help.

As you can guess I have my strong opinion about classloader voodoo in KK's
> approach to a jenkinsfile-runner CLI.
>
Fine for PoC IMHO. It improves performance, and it may be useful in cases
when we want to have fast builds with really short initialization times. I
was doing some performance hacks for similar purposes a while ago (actually
some even weirder hacks). Obviously, a final implementation should not be
based on Jenkins Test Harness and JenkinsRule, some Jenkins core patches
will likely be needed.


On Thu, Sep 6, 2018 at 3:31 PM nicolas de loof 
wrote:

> please note ndeloof/jenkinsfile-runner is a complete different design vs
> KK initial prototype
> they don't share architecture nor git history (only few java classes)
>
> so you'll have to choose one approach over the other
>
> As you can guess I have my strong opinion about classloader voodoo in KK's
> approach to a jenkinsfile-runner CLI.
>
> Le jeu. 6 sept. 2018 à 15:27, Baptiste Mathus  a écrit :
>
>> +1 for cleaning up. Like we recommend for plugins, it would be nice that
>> canonical repo location is more clearly the one under jenkinsci org
>>
>> Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev  a
>> écrit :
>>
>>> Hi all,
>>>
>>> It seems we have a splitbrain issue in this project.
>>> Currently there is the following fork tree:
>>>
>>>- https://github.com/kohsuke/jenkinsfile-runner
>>>   - https://github.com/ndeloof/jenkinsfile-runner
>>>  - https://github.com/jenkinsci/jenkinsfile-runner
>>>
>>> Jenkinsfile Runner in Kohsuke's repo has the most number of stars +
>>> there is a number of issues and pull requests. It makes the jenkinsci repo
>>> unsearchable, and it can also confuse users looking for a place to report
>>> issues / propose patches.
>>>
>>> I propose to do the following:
>>>
>>>1. Remove https://github.com/jenkinsci/jenkinsfile-runner and
>>>evacute the branches somewhere
>>>2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci
>>>org (yes, move, not fork)
>>>3. Rename the current master branch to "kohsuke-master"
>>>4. Reintegrate the current
>>>https://github.com/jenkinsci/jenkinsfile-runner state as a master
>>>branch
>>>5. See if "master" and "kohsuke-master" can be merged
>>>
>>> WDYT?
>>>
>>> BR, Oleg
>>>
>>> On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi wrote:

 Ouch, that's a shame. It looked like an interesting project, I hope my
 writing to you didn't trigger that.

 You say "stress builds and also to burn in our build agents" -- can you
 elaborate on that? It sounds like you are trying to warm up a cache or
 something, but I'm not sure what that means in the context of builds.

 On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:

> Hi Kohsuke,
>
> Thanks for getting in touch. I've had to remove that repo temporarily
> at the request of my former employer. It looks like we are trying to solve
> the same problem. I wanted to use jenkinless to stress builds and also to
> burn in our build agents. It's more of a convenience tool. Running the
> slave using docker in docker worked really well. I might try and turn this
> into an executable that can be stored in the repo alongside code.
>
> Tom
>
>
> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  wrote:
>
>> Hi, Thomas,
>>
>> Tyler passed me a link to your project
>> https://github.com/tomwillfixit/jenkinless, which is in a similar
>> space with my project of the day called Jenkinsfile Runner.
>>
>> I haven't studied your project carefully yet, but I already see some
>> interesting ingredients like memcached that I have no idea what you use 
>> it
>> for :-)   I'd love to hear from you the philosophy & use cases that led 
>> to
>> it. And I'd also love to hear what you think of Jenkinsfile Runner. I 
>> think
>> we have similar interests here, are there any opportunity to collaborate?
>>
>> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi 
>> wrote:
>>
>>> And of course I forgot to have the link to the project!
>>> https://github.com/kohsuke/jenkinsfile-runner
>>>
>>>
>>> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi 
>>> wrote:
>>>
 Jenkinsfile Runner is an experiment to package Jenkins pipeline
 execution as a command line tool. The intend use cases include:

- Use Jenkins in Function-as-a-Service context
- Assist editing Jenkinsfile locally
- Integration test shared libraries

 Over the past year, I've done some deep-dive 1:1 conversations with
 some Jenkins users and I felt something like this might move the 
 needle for
 them in an important way.

Re: POTD: Jenkinsfile Runner

[nicolas de loof]

please note ndeloof/jenkinsfile-runner is a complete different design vs KK
initial prototype
they don't share architecture nor git history (only few java classes)

so you'll have to choose one approach over the other

As you can guess I have my strong opinion about classloader voodoo in KK's
approach to a jenkinsfile-runner CLI.

Le jeu. 6 sept. 2018 à 15:27, Baptiste Mathus  a écrit :

> +1 for cleaning up. Like we recommend for plugins, it would be nice that
> canonical repo location is more clearly the one under jenkinsci org
>
> Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev  a
> écrit :
>
>> Hi all,
>>
>> It seems we have a splitbrain issue in this project.
>> Currently there is the following fork tree:
>>
>>- https://github.com/kohsuke/jenkinsfile-runner
>>   - https://github.com/ndeloof/jenkinsfile-runner
>>  - https://github.com/jenkinsci/jenkinsfile-runner
>>
>> Jenkinsfile Runner in Kohsuke's repo has the most number of stars + there
>> is a number of issues and pull requests. It makes the jenkinsci repo
>> unsearchable, and it can also confuse users looking for a place to report
>> issues / propose patches.
>>
>> I propose to do the following:
>>
>>1. Remove https://github.com/jenkinsci/jenkinsfile-runner and evacute
>>the branches somewhere
>>2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci
>>org (yes, move, not fork)
>>3. Rename the current master branch to "kohsuke-master"
>>4. Reintegrate the current
>>https://github.com/jenkinsci/jenkinsfile-runner state as a master
>>branch
>>5. See if "master" and "kohsuke-master" can be merged
>>
>> WDYT?
>>
>> BR, Oleg
>>
>> On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi wrote:
>>>
>>> Ouch, that's a shame. It looked like an interesting project, I hope my
>>> writing to you didn't trigger that.
>>>
>>> You say "stress builds and also to burn in our build agents" -- can you
>>> elaborate on that? It sounds like you are trying to warm up a cache or
>>> something, but I'm not sure what that means in the context of builds.
>>>
>>> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:
>>>
 Hi Kohsuke,

 Thanks for getting in touch. I've had to remove that repo temporarily
 at the request of my former employer. It looks like we are trying to solve
 the same problem. I wanted to use jenkinless to stress builds and also to
 burn in our build agents. It's more of a convenience tool. Running the
 slave using docker in docker worked really well. I might try and turn this
 into an executable that can be stored in the repo alongside code.

 Tom


 On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  wrote:

> Hi, Thomas,
>
> Tyler passed me a link to your project
> https://github.com/tomwillfixit/jenkinless, which is in a similar
> space with my project of the day called Jenkinsfile Runner.
>
> I haven't studied your project carefully yet, but I already see some
> interesting ingredients like memcached that I have no idea what you use it
> for :-)   I'd love to hear from you the philosophy & use cases that led to
> it. And I'd also love to hear what you think of Jenkinsfile Runner. I 
> think
> we have similar interests here, are there any opportunity to collaborate?
>
> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi 
> wrote:
>
>> And of course I forgot to have the link to the project!
>> https://github.com/kohsuke/jenkinsfile-runner
>>
>>
>> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi 
>> wrote:
>>
>>> Jenkinsfile Runner is an experiment to package Jenkins pipeline
>>> execution as a command line tool. The intend use cases include:
>>>
>>>- Use Jenkins in Function-as-a-Service context
>>>- Assist editing Jenkinsfile locally
>>>- Integration test shared libraries
>>>
>>> Over the past year, I've done some deep-dive 1:1 conversations with
>>> some Jenkins users and I felt something like this might move the needle 
>>> for
>>> them in an important way.
>>>
>>> I'd love to hear any reactions on your side. Could something like
>>> this be important for you, does it miss any key points for you? If you
>>> mentally picture a perfect version of this, what would that do, and how
>>> would you use?
>>>
>>> Let me know!
>>>
>>> --
>>> Kohsuke Kawaguchi
>>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> Kohsuke Kawaguchi
>
 --
>>> Kohsuke Kawaguchi
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> 

Re: POTD: Jenkinsfile Runner

[Baptiste Mathus]

+1 for cleaning up. Like we recommend for plugins, it would be nice that
canonical repo location is more clearly the one under jenkinsci org

Le jeu. 6 sept. 2018 à 15:24, Oleg Nenashev  a
écrit :

> Hi all,
>
> It seems we have a splitbrain issue in this project.
> Currently there is the following fork tree:
>
>- https://github.com/kohsuke/jenkinsfile-runner
>   - https://github.com/ndeloof/jenkinsfile-runner
>  - https://github.com/jenkinsci/jenkinsfile-runner
>
> Jenkinsfile Runner in Kohsuke's repo has the most number of stars + there
> is a number of issues and pull requests. It makes the jenkinsci repo
> unsearchable, and it can also confuse users looking for a place to report
> issues / propose patches.
>
> I propose to do the following:
>
>1. Remove https://github.com/jenkinsci/jenkinsfile-runner and evacute
>the branches somewhere
>2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci org
>(yes, move, not fork)
>3. Rename the current master branch to "kohsuke-master"
>4. Reintegrate the current
>https://github.com/jenkinsci/jenkinsfile-runner state as a master
>branch
>5. See if "master" and "kohsuke-master" can be merged
>
> WDYT?
>
> BR, Oleg
>
> On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi wrote:
>>
>> Ouch, that's a shame. It looked like an interesting project, I hope my
>> writing to you didn't trigger that.
>>
>> You say "stress builds and also to burn in our build agents" -- can you
>> elaborate on that? It sounds like you are trying to warm up a cache or
>> something, but I'm not sure what that means in the context of builds.
>>
>> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:
>>
>>> Hi Kohsuke,
>>>
>>> Thanks for getting in touch. I've had to remove that repo temporarily at
>>> the request of my former employer. It looks like we are trying to solve the
>>> same problem. I wanted to use jenkinless to stress builds and also to burn
>>> in our build agents. It's more of a convenience tool. Running the slave
>>> using docker in docker worked really well. I might try and turn this into
>>> an executable that can be stored in the repo alongside code.
>>>
>>> Tom
>>>
>>>
>>> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  wrote:
>>>
 Hi, Thomas,

 Tyler passed me a link to your project
 https://github.com/tomwillfixit/jenkinless, which is in a similar
 space with my project of the day called Jenkinsfile Runner.

 I haven't studied your project carefully yet, but I already see some
 interesting ingredients like memcached that I have no idea what you use it
 for :-)   I'd love to hear from you the philosophy & use cases that led to
 it. And I'd also love to hear what you think of Jenkinsfile Runner. I think
 we have similar interests here, are there any opportunity to collaborate?

 On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi 
 wrote:

> And of course I forgot to have the link to the project!
> https://github.com/kohsuke/jenkinsfile-runner
>
>
> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi 
> wrote:
>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline
>> execution as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with
>> some Jenkins users and I felt something like this might move the needle 
>> for
>> them in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like
>> this be important for you, does it miss any key points for you? If you
>> mentally picture a perfect version of this, what would that do, and how
>> would you use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> Kohsuke Kawaguchi
>
 --
 Kohsuke Kawaguchi

>>> --
>> Kohsuke Kawaguchi
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/452ee425-972d-4563-920e-cbb6644bacab%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

Hi all,

It seems we have a splitbrain issue in this project.
Currently there is the following fork tree:

   - https://github.com/kohsuke/jenkinsfile-runner
  - https://github.com/ndeloof/jenkinsfile-runner
 - https://github.com/jenkinsci/jenkinsfile-runner
 
Jenkinsfile Runner in Kohsuke's repo has the most number of stars + there 
is a number of issues and pull requests. It makes the jenkinsci repo 
unsearchable, and it can also confuse users looking for a place to report 
issues / propose patches.

I propose to do the following:

   1. Remove https://github.com/jenkinsci/jenkinsfile-runner and evacute 
   the branches somewhere
   2. Move https://github.com/kohsuke/jenkinsfile-runner to jenkinsci org 
   (yes, move, not fork)
   3. Rename the current master branch to "kohsuke-master"
   4. Reintegrate the current 
   https://github.com/jenkinsci/jenkinsfile-runner state as a master branch
   5. See if "master" and "kohsuke-master" can be merged
   
WDYT?

BR, Oleg

On Wednesday, April 4, 2018 at 9:30:15 PM UTC+2, Kohsuke Kawaguchi wrote:
>
> Ouch, that's a shame. It looked like an interesting project, I hope my 
> writing to you didn't trigger that.
>
> You say "stress builds and also to burn in our build agents" -- can you 
> elaborate on that? It sounds like you are trying to warm up a cache or 
> something, but I'm not sure what that means in the context of builds.
>
> On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  > wrote:
>
>> Hi Kohsuke,
>>
>> Thanks for getting in touch. I've had to remove that repo temporarily at 
>> the request of my former employer. It looks like we are trying to solve the 
>> same problem. I wanted to use jenkinless to stress builds and also to burn 
>> in our build agents. It's more of a convenience tool. Running the slave 
>> using docker in docker worked really well. I might try and turn this into 
>> an executable that can be stored in the repo alongside code. 
>>
>> Tom
>>
>>
>> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi, > > wrote:
>>
>>> Hi, Thomas,
>>>
>>> Tyler passed me a link to your project 
>>> https://github.com/tomwillfixit/jenkinless, which is in a similar space 
>>> with my project of the day called Jenkinsfile Runner.
>>>
>>> I haven't studied your project carefully yet, but I already see some 
>>> interesting ingredients like memcached that I have no idea what you use it 
>>> for :-)   I'd love to hear from you the philosophy & use cases that led to 
>>> it. And I'd also love to hear what you think of Jenkinsfile Runner. I think 
>>> we have similar interests here, are there any opportunity to collaborate?
>>>
>>> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi >> > wrote:
>>>
 And of course I forgot to have the link to the project! 
 https://github.com/kohsuke/jenkinsfile-runner 


 On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi >>> > wrote:

> Jenkinsfile Runner is an experiment to package Jenkins pipeline 
> execution as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with 
> some Jenkins users and I felt something like this might move the needle 
> for 
> them in an important way.
>
> I'd love to hear any reactions on your side. Could something like this 
> be important for you, does it miss any key points for you? If you 
> mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>
 -- 
 Kohsuke Kawaguchi

>>> -- 
>>> Kohsuke Kawaguchi
>>>
>> -- 
> Kohsuke Kawaguchi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/452ee425-972d-4563-920e-cbb6644bacab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Ouch, that's a shame. It looked like an interesting project, I hope my
writing to you didn't trigger that.

You say "stress builds and also to burn in our build agents" -- can you
elaborate on that? It sounds like you are trying to warm up a cache or
something, but I'm not sure what that means in the context of builds.

On Wed, Apr 4, 2018 at 11:15 AM Tom Shaw  wrote:

> Hi Kohsuke,
>
> Thanks for getting in touch. I've had to remove that repo temporarily at
> the request of my former employer. It looks like we are trying to solve the
> same problem. I wanted to use jenkinless to stress builds and also to burn
> in our build agents. It's more of a convenience tool. Running the slave
> using docker in docker worked really well. I might try and turn this into
> an executable that can be stored in the repo alongside code.
>
> Tom
>
>
> On Wed, 4 Apr 2018, 00:19 Kohsuke Kawaguchi,  wrote:
>
>> Hi, Thomas,
>>
>> Tyler passed me a link to your project
>> https://github.com/tomwillfixit/jenkinless, which is in a similar space
>> with my project of the day called Jenkinsfile Runner.
>>
>> I haven't studied your project carefully yet, but I already see some
>> interesting ingredients like memcached that I have no idea what you use it
>> for :-)   I'd love to hear from you the philosophy & use cases that led to
>> it. And I'd also love to hear what you think of Jenkinsfile Runner. I think
>> we have similar interests here, are there any opportunity to collaborate?
>>
>> On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi  wrote:
>>
>>> And of course I forgot to have the link to the project!
>>> https://github.com/kohsuke/jenkinsfile-runner
>>>
>>>
>>> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi 
>>> wrote:
>>>
 Jenkinsfile Runner is an experiment to package Jenkins pipeline
 execution as a command line tool. The intend use cases include:

- Use Jenkins in Function-as-a-Service context
- Assist editing Jenkinsfile locally
- Integration test shared libraries

 Over the past year, I've done some deep-dive 1:1 conversations with
 some Jenkins users and I felt something like this might move the needle for
 them in an important way.

 I'd love to hear any reactions on your side. Could something like this
 be important for you, does it miss any key points for you? If you mentally
 picture a perfect version of this, what would that do, and how would you
 use?

 Let me know!

 --
 Kohsuke Kawaguchi

>>> --
>>> Kohsuke Kawaguchi
>>>
>> --
>> Kohsuke Kawaguchi
>>
> --
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4w8VgdCKyp2Gz9ZXGjP1AN88Qb-NvWR3dCyP-p8dVkDsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Hi, Thomas,

Tyler passed me a link to your project
https://github.com/tomwillfixit/jenkinless, which is in a similar space
with my project of the day called Jenkinsfile Runner.

I haven't studied your project carefully yet, but I already see some
interesting ingredients like memcached that I have no idea what you use it
for :-)   I'd love to hear from you the philosophy & use cases that led to
it. And I'd also love to hear what you think of Jenkinsfile Runner. I think
we have similar interests here, are there any opportunity to collaborate?

On Thu, Mar 1, 2018 at 11:23 AM Kohsuke Kawaguchi  wrote:

> And of course I forgot to have the link to the project!
> https://github.com/kohsuke/jenkinsfile-runner
>
>
> On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi  wrote:
>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> Kohsuke Kawaguchi
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4wKRJ9irATTo45XM5AfvZcm2tGaxCKjFL5oNbH_Bo3MMA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

On Sat, Mar 24, 2018 at 12:53 PM,   wrote:
> This might have something to do with this warning, issued right at the start
> of the pipeline.
>
> jenkins.security.ClassFilterImpl#isBlacklisted: rejecting 
> org.codehaus.groovy.runtime.powerassert.PowerAssertionError according to 
> standard blacklist; see https://jenkins.io/redirect/class-filter/

Sounds like JENKINS-41751. Make sure you have updated `workflow-api`
to 2.26. Any other discussion of this should take place in a JIRA
issue with complete steps to reproduce from scratch using a fully
up-to-date installation.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0y91PMquYu9CpQ137kOoO_rXJxCsxtCtin6nGsaw1%2B_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

KK wrote:
> I wonder if it'd be nice if Jenkinsfile Runner can do something like the
> permissive mode of SELinux, where it runs your pipeline entirely and spit
> out the approvals it needed?

https://plugins.jenkins.io/permissive-script-security

> so instead of invisible WorkflowJob, we create invisible WorkflowRun, right?

Yes, that seems to be the safest approach.

> I was thinking you'd just put assert statements into Jenkinsfile and that 
> should already work. Granted, I haven't tried it so maybe it doesn't work.

Groovy “power-asserts” work in Pipeline script as far as I know.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3e92S1hvsAZBEMeh3%3DgWFSun%2Bc0SPYZb3irezbxywn3g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

It's pretty cool that it actually works, huh?

On Mon, Mar 12, 2018 at 10:57 PM Michael Neale  wrote:

> As an example of how to use (abuse?) this:
>
> I was able to take a container with the set of "recommended" plugins, and
> run it on the "codeship pro" service (which uses docker):
> https://github.com/michaelneale/codeship-jenkinsfile
>
>
>
> On Friday, March 2, 2018 at 6:23:15 AM UTC+11, Kohsuke Kawaguchi wrote:
>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/e1613f77-8bb0-4d88-91a3-e2dea440444c%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4ymaQQT_FnCJMsbVTBAH31TR99wHpJwnypkXro9A1npXg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

On Sun, Mar 11, 2018 at 6:43 AM  wrote:

> From a user perspective, I like your idea of a
> "pipeline.sharedlibrary.test" step and/or a "run-pipeline" CLI command.
> They would definitely solve configuration issues with plugins, docker, etc.
> Now, Jesse mentioned some implementation challenges in his post from Wed,
> 07 Mar 2018 13:04:46
> ,
> that I can't comment on, because I'm not that deep into Jenkins internals
> (yet?).
>
> In the meantime, Jesse also provided a workaround for loading a "local"
> shared lib . With this I
> implemented my first (very simple) integration test
> 
> and run it successfully from the Jenkinsfile of the shared lib
> 
> 
>

Nice!

You meant asserts should already work? How would you do those in that
> scenario?
>

I was thinking you'd just put assert statements into Jenkinsfile and that
should already work. Granted, I haven't tried it so maybe it doesn't work.

This approach has of course the limitations we talked about, regarding the
> Jenkins configuration .
> In addition these tests would only be run on Jenkins and not when running
> the build locally with Maven.
>

I'm not sure if I follow you here. The two approaches that I was thinking
about were:

   - "Jenkinsfile Runner" that runs embedded Jenkins locally and runs
   Jenkinsfile in there
   - "run-pipeline CLI command" that submits Jenkinsfile into an existing
   Jenkins server and run it there and send the result back

"pipeline.sharedlibrary.test" is just a pipeline step that runs tests in
some form. In each of the above two versions we can think of their
corresponding "pipeline.sharedlibrary.test" step.


> So right now, I think a better approach would be to provide a Java library
> that allows for running Jenkinsfiles.
> It could be used to implement e.g. JUnit tests that are run by build tools
> such as maven locally (e.g. with the failsafe plugin) and in the same way
> in the Jenkinsfile.
> This library could abstract from the concrete execution with could either
> be
> - Jenkinsfile runner with local plugin folder
> - Jenkinsfile runner in Docker
> - "pipeline.sharedlibrary.test" step
> - a "run-pipeline" CLI command.
>

I was kind of trying to avoid doing 4 different things that each has
different unique trade-offs that only Jenkins experts would understand.

You have mentioned two primary use cases for you, pipeline development and
shared library development. I think we can find one solution that works for
those two cases. I just need to think about this.

The downside would be that such an implementation wouldn't be trivial. And,
> unfortunately, I can't invest to much of my free time right now.
>

Your feedback has been very helpful, and I appreciate the time you are
investing in those.


>
> Am Dienstag, 6. März 2018 22:23:26 UTC+1 schrieb Kohsuke Kawaguchi:
>>
>>
>>
>> On Sun, Mar 4, 2018 at 8:13 AM  wrote:
>>
>>> I think Jenkinsfile Runner brings a lot of opportunities for pipeline
>>> developers. The most obvious ones to me are
>>>
>>>1. Pipeline development (Jenkinsfile)
>>>2. Shared library development
>>>
>>> *Pipeline development*
>>>
>>> Right now (as described by others in this thread) pipeline development
>>> is either a loop of committing / fixing pipelines on production Jenkins,
>>> using pipeline replay on production Jenkins or setting up a local instance
>>> manually.
>>>
>>> With Jenkinsfile Runner we can get faster feedback without polluting our
>>> commit or Jenkins build history and don't have to set up a local instance
>>> manually.
>>>
>> Right. I think we all get this basic picture. Details are where things
>> get interesting!
>>
>> *Shared library development*
>>>
>>> Shared library development right now works much in the same as pipeline
>>> development, except that you have the library code and another (often
>>> production) Jenkinsfile to maintain, in order to try out (as opposed to
>>> automatically test) your Jenkinsfile.
>>> For shared libraries, we thankfully already have JenkinsPipelineUnit,
>>> that makes it easier to implement some tests. However, (as also mentioned
>>> by others in this thread) this is unit testing only. It mocks most of our
>>> environment. Often, green unit tests mean nothing for productive use of our
>>> share library. I even gave up test-driven development for shared libraries,
>>> in favor of 90s try-and-error-style programming. Because most of the time
>>> when trying the library with green unit tests in production, it turns out
>>> that the real Jenkins environment has some restriction that is beyond the
>>> scope of JenkinsPieplineUnit 

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

On Wed, Mar 7, 2018 at 1:04 PM Jesse Glick  wrote:

> On Tue, Mar 6, 2018 at 4:23 PM, Kohsuke Kawaguchi  wrote:
> > I have another idea. Instead of
> > running Jenkinsfile in this CLI and try to emulate your Jenkins instance
> as
> > closely as possible (in terms of configuration), we could just run
> > Jenkinsfile in the current Jenkins, in a place that nobody else can see.
>
> In principle this is possible, by defining a new implementation of
> `FlowExecutionOwner` that is not based on a `Run`. We have in the past
> discussed the possibility of a Pipeline equivalent to the `/script`
> console.
>

Thanks for the tip. I was actually thinking about creating a full blown
WorkflowJob that's just so happen to be invisible to anyone. Do you think
that would work better/worse?

I doubt this would be a very satisfactory solution to the use case at
> hand, though. Any step which required a `Run` context (either
> mandatory, as in `StepDescriptor.getRequiredContext`, or optionally,
> by checking for null from `StepContext.get(Run.class)`) would either
> fail or behave in some reduced capacity. If you took care to only
> write `Jenkinsfile`s that just used `node` and `sh` and the like and
> none of these features, then fine. But a lot of common functionality
> does assume it has a `Run` context: anything that looks up
> folder-scoped environment variables or credentials; `junit`,
> `milestone`, `lock`, `stash`; everything based on `SimpleBuildStep` or
> `SimpleBuildWrapper`…a lot. You could create a temporary `WorkflowJob`
> in the same folder and hack around with access control so that it is
> only visible to the invoking user, which would let most of these
> things work (probably with some specialized support for branch
> projects), but this seems like it is asking for trouble.
>

IIUC, creating invisible WorkflowJob would bypass these problems.

I think it would be far more practical to work with the existing
> Replay feature, which was designed for precisely this use case. If the
> main objection to using Replay is that you do not want these
> experimental builds to “pollute” the general build history, then we
> can do some minor UI work (for example in Blue Ocean) to hide these
> build records by default. There is already a plan to do something very
> similar for restarted stages (JENKINS-48643). We could even stream the
> build log to the CLI command (JENKINS-33438) and then add an option to
> delete the build as soon as it completes—a very simple and safe
> feature.
>

Ah, so instead of invisible WorkflowJob, we create invisible WorkflowRun,
right?
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zn1XtZrYY5O6CVe_Kzk%3DL%3Dga%2BBUE_ZvHmtMVYAFES3RQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Sorry for the bit of delay to come back to this,

On Wed, Mar 7, 2018 at 5:46 AM Bill Dennis  wrote:

> Kohsuke  - some more answers to your questions inline below.
>
> Also I thought of these other things on my walk to the office this morning
> that might be of interest:
>
> - Parameters to jobs : most of my pipeline jobs have parameters defined
> inside. Would the Jenkinsfile Runner have a way to pass in parameters so it
> doesn't just use default values?
>

Ah, good catch, this is actually pretty easy to do. Tracking it as #13
.

- Environment values for jobs: For our pipeline automation deployments, we
> set "env" values (key value pairs as strings) at the gobal of folder level
> (using Jenkins Enterprise Folders plugin) and these define some environment
> for jobs (things like REST endpoints to some RESTful services we use in the
> jobs, for example). With the Jenkinsfile Runner have a way to supply this?
>

The global settings, I put that into the class of problems to be solved by
config-as-code, along with plugin installations.

The folder level things, I haven't thought about. Solving the general case
of this is bit involving.

- Script approvals : I have mentioned this below, but some way to define
> script approvals or get the required approvals would help possibly. I know
> if you are hitting script approvals, then probably trying to do too much in
> the pipeline code but it is easy to hit this with simple stuff like Java
> date functions or JSON parsing.
>

I guess the idea here is that you want Jenkinsfile Runner to run the
pipeline with exactly the same script approval settings.

I wonder if it'd be nice if Jenkinsfile Runner can do something like the
permissive mode of SELinux
,
where it runs your pipeline entirely and spit out the approvals it needed?
That way you don't need to do any trial & error on production Jenkins.


- Jenkinsfile As a Service idea (JFAAS?): you mentioned this and I really
> like this idea. The possibility to externally run a Jenkinsfile supplied
> and get back some results like build artifacts and console output without
> having any jobs or builds appear on the Jenkins master or needing a job
> configuration. The reason is we tend to think about loading all our
> automation into Jenkins. But our world is not all inside Jenkins and we
> have other automation solutions that can have extensions and customisation.
> An example is some of our custom services we build and some off the shelf
> stuff like the Serena Business Process management server we have. But
> Jenkins is really good and doing some automation, tooling and spinning /
> scaling up agent workers in Docker etc, so it would be good to use that
> from these other services by having them send a Jenkinsfile they own over
> and getting some results back somehow. Maybe this is something different
> from Jenkinsfile Runner though.
>

Yeah, as you nailed it, JFaaS idea is around being able to submit the work
and spin up the process to execute it, and store the result somewhere,
which you can later browse. There's a bit of similarity to build publisher
plugin .

Obviously, Jenkinsfile Runner alone is not going to be able to do that, but
the point of it is so that we can talk about it more concretely, like we
are doing it here.

I'm not sure if I fully grok the portion where you say "But our world is
not all inside Jenkins and we have other automation solutions that can have
extensions and customisation. An example is ... Serena Business Process
management server."

Can you help me understand how those tools and your custom stuff work
together in your environment? If you need to get to sensitive stuff, I'm
happy to jump onto a call, especially given that you appear to be a
CloudBees customer :-)


>
> Best regards,
> --Bill
>
> On Tuesday, 6 March 2018 21:33:37 UTC, Kohsuke Kawaguchi wrote:
>
>>
>> On Tue, Mar 6, 2018 at 2:57 AM Bill Dennis  wrote:
>>
>>> Thanks Kohsuke, I tried to give some answers to your questions inline
>>> below, if I didn't mess up the reply..
>>>
>>> Bill
>>>
>>> On Friday, 2 March 2018 17:57:24 UTC, Kohsuke Kawaguchi wrote:
>>>


 On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis  wrote:

> Hello Kohsuke -
>
> I am a developer using Jenkins pipeline quite lot where I work.
>
> We are using Jenkins pipelines in two scenarios:
>
>- For CI building and testing some of our internal components
>(what Jenkins is traditionally used for)
>- For running / orchestrating complex automation processes (so
>Jenkins is talking to some external systems using SOAP / REST etc) via
>tooling and even directly via REST using plugins.
>
> I have mostly used JenkinsPipelineUnit 

Re: POTD: Jenkinsfile Runner

[Michael Neale]

As an example of how to use (abuse?) this: 

I was able to take a container with the set of "recommended" plugins, and 
run it on the "codeship pro" service (which uses docker): 
https://github.com/michaelneale/codeship-jenkinsfile



On Friday, March 2, 2018 at 6:23:15 AM UTC+11, Kohsuke Kawaguchi wrote:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some 
> Jenkins users and I felt something like this might move the needle for them 
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be 
> important for you, does it miss any key points for you? If you mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/e1613f77-8bb0-4d88-91a3-e2dea440444c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[johannes]

>From a user perspective, I like your idea of a 
"pipeline.sharedlibrary.test" step and/or a "run-pipeline" CLI command. 
They would definitely solve configuration issues with plugins, docker, etc.
Now, Jesse mentioned some implementation challenges in his post from Wed, 
07 Mar 2018 13:04:46 
, 
that I can't comment on, because I'm not that deep into Jenkins internals 
(yet?).

In the meantime, Jesse also provided a workaround for loading a "local" 
shared lib . With this I 
implemented my first (very simple) integration test 

 
and run it successfully from the Jenkinsfile of the shared lib 

 


You meant asserts should already work? How would you do those in that 
scenario?

This approach has of course the limitations we talked about, regarding the 
Jenkins configuration .
In addition these tests would only be run on Jenkins and not when running 
the build locally with Maven.

So right now, I think a better approach would be to provide a Java library 
that allows for running Jenkinsfiles.
It could be used to implement e.g. JUnit tests that are run by build tools 
such as maven locally (e.g. with the failsafe plugin) and in the same way 
in the Jenkinsfile.
This library could abstract from the concrete execution with could either be
- Jenkinsfile runner with local plugin folder
- Jenkinsfile runner in Docker
- "pipeline.sharedlibrary.test" step
- a "run-pipeline" CLI command.

The downside would be that such an implementation wouldn't be trivial. And, 
unfortunately, I can't invest to much of my free time right now.

Am Dienstag, 6. März 2018 22:23:26 UTC+1 schrieb Kohsuke Kawaguchi:
>
>
>
> On Sun, Mar 4, 2018 at 8:13 AM  
> wrote:
>
>> I think Jenkinsfile Runner brings a lot of opportunities for pipeline 
>> developers. The most obvious ones to me are
>>
>>1. Pipeline development (Jenkinsfile)
>>2. Shared library development
>>
>> *Pipeline development*
>>
>> Right now (as described by others in this thread) pipeline development is 
>> either a loop of committing / fixing pipelines on production Jenkins, using 
>> pipeline replay on production Jenkins or setting up a local instance 
>> manually.
>>
>> With Jenkinsfile Runner we can get faster feedback without polluting our 
>> commit or Jenkins build history and don't have to set up a local instance 
>> manually.
>>
> Right. I think we all get this basic picture. Details are where things get 
> interesting!
>
> *Shared library development*
>>
>> Shared library development right now works much in the same as pipeline 
>> development, except that you have the library code and another (often 
>> production) Jenkinsfile to maintain, in order to try out (as opposed to 
>> automatically test) your Jenkinsfile.
>> For shared libraries, we thankfully already have JenkinsPipelineUnit, 
>> that makes it easier to implement some tests. However, (as also mentioned 
>> by others in this thread) this is unit testing only. It mocks most of our 
>> environment. Often, green unit tests mean nothing for productive use of our 
>> share library. I even gave up test-driven development for shared libraries, 
>> in favor of 90s try-and-error-style programming. Because most of the time 
>> when trying the library with green unit tests in production, it turns out 
>> that the real Jenkins environment has some restriction that is beyond the 
>> scope of JenkinsPieplineUnit (e.g. sandboxing). 
>>
> Worst thing about the current state is that we don't have reliable 
>> regression tests. A change in shared library with green unit tests is far 
>> from convincing me that the library will continue to work in production.
>>
>> With Jenkinsfile Runner we could write small Jenkinsfiles within the 
>> shared library repo's test folder and run them from the Jenkinsfile of the 
>> shared lib. Pretty much in the same way as we use Maven Invoker Plugin (as 
>> mentioned by Jesse) when developing maven plugins.
>>
>
> OK, thank you, this is really helpful. I think maven invoker plugin 
> analogy is a great one for somebody like me. So from this perspective, 
> mocking isn't really high on the priority. 
>  
>
>>
>> *A first approach to shared library integration testing with Jenkinsfile 
>> Runner*
>> My naive first approach was to build a Docker Image that contains 
>> Jenkinsfile Runner and all default plugins.
>>
>> docker run -v~/src/it/myfunction:/workspace schnatterer/jenkinsfile-
>> runner:1.0-SNAPSHOT-2.108 /workspace
>>
>> runs the ~/foo/Jenkinsfile using Jenkinsfile Runner with all default 
>> plugins of Jenkins 2.108.
>>
>> My idea was to eventually do the same in Jenkinsfile of the shared lib 
>> 

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

On Tue, Mar 6, 2018 at 4:23 PM, Kohsuke Kawaguchi  wrote:
> I have another idea. Instead of
> running Jenkinsfile in this CLI and try to emulate your Jenkins instance as
> closely as possible (in terms of configuration), we could just run
> Jenkinsfile in the current Jenkins, in a place that nobody else can see.

In principle this is possible, by defining a new implementation of
`FlowExecutionOwner` that is not based on a `Run`. We have in the past
discussed the possibility of a Pipeline equivalent to the `/script`
console.

I doubt this would be a very satisfactory solution to the use case at
hand, though. Any step which required a `Run` context (either
mandatory, as in `StepDescriptor.getRequiredContext`, or optionally,
by checking for null from `StepContext.get(Run.class)`) would either
fail or behave in some reduced capacity. If you took care to only
write `Jenkinsfile`s that just used `node` and `sh` and the like and
none of these features, then fine. But a lot of common functionality
does assume it has a `Run` context: anything that looks up
folder-scoped environment variables or credentials; `junit`,
`milestone`, `lock`, `stash`; everything based on `SimpleBuildStep` or
`SimpleBuildWrapper`…a lot. You could create a temporary `WorkflowJob`
in the same folder and hack around with access control so that it is
only visible to the invoking user, which would let most of these
things work (probably with some specialized support for branch
projects), but this seems like it is asking for trouble.

I think it would be far more practical to work with the existing
Replay feature, which was designed for precisely this use case. If the
main objection to using Replay is that you do not want these
experimental builds to “pollute” the general build history, then we
can do some minor UI work (for example in Blue Ocean) to hide these
build records by default. There is already a plan to do something very
similar for restarted stages (JENKINS-48643). We could even stream the
build log to the CLI command (JENKINS-33438) and then add an option to
delete the build as soon as it completes—a very simple and safe
feature.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2xLC0nVJ4okTQ03nYsLtW0HoaJX0RK%2B-BYLhg%2Ba7f6qw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Bill Dennis]

Kohsuke  - some more answers to your questions inline below.

Also I thought of these other things on my walk to the office this morning 
that might be of interest:

- Parameters to jobs : most of my pipeline jobs have parameters defined 
inside. Would the Jenkinsfile Runner have a way to pass in parameters so it 
doesn't just use default values?

- Environment values for jobs: For our pipeline automation deployments, we 
set "env" values (key value pairs as strings) at the gobal of folder level 
(using Jenkins Enterprise Folders plugin) and these define some environment 
for jobs (things like REST endpoints to some RESTful services we use in the 
jobs, for example). With the Jenkinsfile Runner have a way to supply this?

- Script approvals : I have mentioned this below, but some way to define 
script approvals or get the required approvals would help possibly. I know 
if you are hitting script approvals, then probably trying to do too much in 
the pipeline code but it is easy to hit this with simple stuff like Java 
date functions or JSON parsing.

- Jenkinsfile As a Service idea (JFAAS?): you mentioned this and I really 
like this idea. The possibility to externally run a Jenkinsfile supplied 
and get back some results like build artifacts and console output without 
having any jobs or builds appear on the Jenkins master or needing a job 
configuration. The reason is we tend to think about loading all our 
automation into Jenkins. But our world is not all inside Jenkins and we 
have other automation solutions that can have extensions and customisation. 
An example is some of our custom services we build and some off the shelf 
stuff like the Serena Business Process management server we have. But 
Jenkins is really good and doing some automation, tooling and spinning / 
scaling up agent workers in Docker etc, so it would be good to use that 
from these other services by having them send a Jenkinsfile they own over 
and getting some results back somehow. Maybe this is something different 
from Jenkinsfile Runner though.

Best regards,
--Bill

On Tuesday, 6 March 2018 21:33:37 UTC, Kohsuke Kawaguchi wrote:
>
>
> On Tue, Mar 6, 2018 at 2:57 AM Bill Dennis  > wrote:
>
>> Thanks Kohsuke, I tried to give some answers to your questions inline 
>> below, if I didn't mess up the reply..
>>
>> Bill
>>
>> On Friday, 2 March 2018 17:57:24 UTC, Kohsuke Kawaguchi wrote:
>>
>>>
>>>
>>> On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis  wrote:
>>>
 Hello Kohsuke -

 I am a developer using Jenkins pipeline quite lot where I work.

 We are using Jenkins pipelines in two scenarios:

- For CI building and testing some of our internal components (what 
Jenkins is traditionally used for)
- For running / orchestrating complex automation processes (so 
Jenkins is talking to some external systems using SOAP / REST etc) via 
tooling and even directly via REST using plugins.

 I have mostly used JenkinsPipelineUnit for testing / validation of the 
 pipelines and I have been looking into the direct / live approach that 
 Oleg 
 demonstrated (running Jenkins locally in Docker and getting the pipeline 
 being developed direct from a host file system volume mount).

 I think Jenkinsfile Runner would be really useful for developers who 
 don't need or want the overhead of developing tests with 
 JenkinsPipelineUnit. I have worked with some developers wanting to develop 
 Jenkinsfiles for their CI process and the main problem is knowing if the 
 Jenkinsfile will work when they commit it to the repo. They go round this 
 loop of commit / fix running in the production Jenkins or using the 
 Jenkins 
 pipeline "replay" feature. It can be a painful process if you are not 
 familiar with Jenkins pipeline and Groovy syntax!

>>>
>>> This kind of context is really helpful. Thank you!
>>>
>>  
>> Happy to feedback. Thanks for Jenkins and pipeline as code, it helped me 
>> deliver some projects with Jenkins in a way that I thought would not be 
>> possible a few years ago.
>>
>
> My pleasure, and really the portion I can take credit for is getting 
> smaller every day!
>  
>
>>
>>
>>> I think some things to consider are:

 - How does the Jenkins Runner replicate the agents / slaves identifiers 
 on the target Jenkins?
 - How to deal with tooling on the target Jenkins (custom tools, JDKs, 
 Gradle, etc)?

>>>
>>> Right, I guess your point is that Jenkinsfile Runner should aim to run 
>>> Jenkinsfile in much more realistic setup, and that doesn't stop at using 
>>> real Jenkins and real Pipeline plugins, but it also needs to include other 
>>> configurations of Jenkins. I think Jesse made a similar observation. I have 
>>> a few thoughts:
>>>
>>>- Configuration-as-code 
>>> could 
>>>play a 

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

On Tue, Mar 6, 2018 at 2:57 AM Bill Dennis  wrote:

> Thanks Kohsuke, I tried to give some answers to your questions inline
> below, if I didn't mess up the reply..
>
> Bill
>
> On Friday, 2 March 2018 17:57:24 UTC, Kohsuke Kawaguchi wrote:
>
>>
>>
>> On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis  wrote:
>>
>>> Hello Kohsuke -
>>>
>>> I am a developer using Jenkins pipeline quite lot where I work.
>>>
>>> We are using Jenkins pipelines in two scenarios:
>>>
>>>- For CI building and testing some of our internal components (what
>>>Jenkins is traditionally used for)
>>>- For running / orchestrating complex automation processes (so
>>>Jenkins is talking to some external systems using SOAP / REST etc) via
>>>tooling and even directly via REST using plugins.
>>>
>>> I have mostly used JenkinsPipelineUnit for testing / validation of the
>>> pipelines and I have been looking into the direct / live approach that Oleg
>>> demonstrated (running Jenkins locally in Docker and getting the pipeline
>>> being developed direct from a host file system volume mount).
>>>
>>> I think Jenkinsfile Runner would be really useful for developers who
>>> don't need or want the overhead of developing tests with
>>> JenkinsPipelineUnit. I have worked with some developers wanting to develop
>>> Jenkinsfiles for their CI process and the main problem is knowing if the
>>> Jenkinsfile will work when they commit it to the repo. They go round this
>>> loop of commit / fix running in the production Jenkins or using the Jenkins
>>> pipeline "replay" feature. It can be a painful process if you are not
>>> familiar with Jenkins pipeline and Groovy syntax!
>>>
>>
>> This kind of context is really helpful. Thank you!
>>
>
> Happy to feedback. Thanks for Jenkins and pipeline as code, it helped me
> deliver some projects with Jenkins in a way that I thought would not be
> possible a few years ago.
>

My pleasure, and really the portion I can take credit for is getting
smaller every day!


>
>
>> I think some things to consider are:
>>>
>>> - How does the Jenkins Runner replicate the agents / slaves identifiers
>>> on the target Jenkins?
>>> - How to deal with tooling on the target Jenkins (custom tools, JDKs,
>>> Gradle, etc)?
>>>
>>
>> Right, I guess your point is that Jenkinsfile Runner should aim to run
>> Jenkinsfile in much more realistic setup, and that doesn't stop at using
>> real Jenkins and real Pipeline plugins, but it also needs to include other
>> configurations of Jenkins. I think Jesse made a similar observation. I have
>> a few thoughts:
>>
>>- Configuration-as-code
>> could
>>play a role here in terms of letting people define the configuration of
>>Jenkins once and use it both in production and in setup like Jenkinsfile
>>Runner
>>- I'm a fan of making Jenkinsfile itself more portable. For example,
>>if people are already in the mode of using docker images to run builds in,
>>then more of the toolings would be packaged in there, and it should allow
>>Jenkinsfile Runner to run your project in much the same way as your
>>production Jenkins. I'm curious how much of this is already reality vs
>>ideal that people are working toward.
>>
>> Yes, all of this. I have often thought that we need something like
> declarative pipeline for the configuration of Jenkins as code instead of
> going into all those web config pages. Jenkins master as Docker container
> seems good. In our environment we are not currently using Docker but I have
> seen that that is the way to go. Getting a larger organisation to adopt the
> right technology and the associated costs of that is the challenge, so we
> remain using traditional Jenkins slaves and tooling methods. Hopefully
> Dockerised soon. We do use Jenkins Enterprise from CloudBees.
>

"something like declarative pipeline for the configuration of Jenkins as
code instead of going into all those web config pages" --- please do check
out the linked configuration-as-code effort. I think you'll love it.

And thanks for the explanation of the state of your current Jenkins master.

I think the perfect Jenkinsfile Runner for me would provide:
>>>
>>> - Somehow capture the plugins, tooling and agents on our production
>>> Jenkins
>>> - Validate the Jenkinsfile pipeline syntax
>>>
>>
>> I think this is already happening as a result of actually running the
>> pipeline -- one of the virtue of actually using the real pipeline plugins
>> to run!
>>
>>
>>> - Validate the Jenkinsfile against the plugins and agents / tooling
>>> (fail if it refers to some tool or agent not configured for example).
>>> - Run the Jenkinfile in some sort of "no-op" mode : what would it do if
>>> I ran it, without actually doing anything
>>>
>>
>> This one is interesting. I assumed JenkinsPipelineUnit does this pretty
>> well, though. Can you tell me more about this? I'm imagining 

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Oleg gave us the fix, which I merged to the master just now. I think
that'll fix the problem.

On Tue, Mar 6, 2018 at 1:22 AM Michael Neale  wrote:

> Trying this out, looks like I am hitting JEP-200:
>
> https://jenkins.io/redirect/class-filter/
>
> Need to dig in further (I thought I tried the same version of Jenkins as
> you). Anyone else seen this?
>
>
>
> java.lang.UnsupportedOperationException: Refusing to marshal
> io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security reasons;
> see https://jenkins.io/redirect/class-filter/
> at
> hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:543)
> at
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
> at
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
> at
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
> at
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
> at
> com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
> at
> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)
> at
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
> at
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
> at
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
> at
> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
> at
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
> Caused: java.lang.RuntimeException: Failed to serialize
> hudson.model.Actionable#actions for class
> org.jenkinsci.plugins.workflow.job.WorkflowRun
>
>
> On Friday, March 2, 2018 at 6:23:15 AM UTC+11, Kohsuke Kawaguchi wrote:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let me know!
>>
>> --
>> Kohsuke Kawaguchi
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/3836b845-5e9f-44d6-883d-10caf7e78832%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4xs-Y3WQ365CXpeORMyo%2BL-uMScPof2FqMCzv%3DxWx7eqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

On Sun, Mar 4, 2018 at 8:13 AM  wrote:

> I think Jenkinsfile Runner brings a lot of opportunities for pipeline
> developers. The most obvious ones to me are
>
>1. Pipeline development (Jenkinsfile)
>2. Shared library development
>
> *Pipeline development*
>
> Right now (as described by others in this thread) pipeline development is
> either a loop of committing / fixing pipelines on production Jenkins, using
> pipeline replay on production Jenkins or setting up a local instance
> manually.
>
> With Jenkinsfile Runner we can get faster feedback without polluting our
> commit or Jenkins build history and don't have to set up a local instance
> manually.
>
Right. I think we all get this basic picture. Details are where things get
interesting!

*Shared library development*
>
> Shared library development right now works much in the same as pipeline
> development, except that you have the library code and another (often
> production) Jenkinsfile to maintain, in order to try out (as opposed to
> automatically test) your Jenkinsfile.
> For shared libraries, we thankfully already have JenkinsPipelineUnit, that
> makes it easier to implement some tests. However, (as also mentioned by
> others in this thread) this is unit testing only. It mocks most of our
> environment. Often, green unit tests mean nothing for productive use of our
> share library. I even gave up test-driven development for shared libraries,
> in favor of 90s try-and-error-style programming. Because most of the time
> when trying the library with green unit tests in production, it turns out
> that the real Jenkins environment has some restriction that is beyond the
> scope of JenkinsPieplineUnit (e.g. sandboxing).
>
Worst thing about the current state is that we don't have reliable
> regression tests. A change in shared library with green unit tests is far
> from convincing me that the library will continue to work in production.
>
> With Jenkinsfile Runner we could write small Jenkinsfiles within the
> shared library repo's test folder and run them from the Jenkinsfile of the
> shared lib. Pretty much in the same way as we use Maven Invoker Plugin (as
> mentioned by Jesse) when developing maven plugins.
>

OK, thank you, this is really helpful. I think maven invoker plugin analogy
is a great one for somebody like me. So from this perspective, mocking
isn't really high on the priority.


>
> *A first approach to shared library integration testing with Jenkinsfile
> Runner*
> My naive first approach was to build a Docker Image that contains
> Jenkinsfile Runner and all default plugins.
>
> docker run -v~/src/it/myfunction:/workspace schnatterer/jenkinsfile-runner
> :1.0-SNAPSHOT-2.108 /workspace
>
> runs the ~/foo/Jenkinsfile using Jenkinsfile Runner with all default
> plugins of Jenkins 2.108.
>
> My idea was to eventually do the same in Jenkinsfile of the shared lib
> like so (not tested)
> docker.image('schnatterer/jenkinsfile-runner:1.0-SNAPSHOT-2.108').inside {
>
> sh 'jenkinsfile-runner /app/jenkins /app/plugins src/it/myfunction'
> }
> or
> sh 'docker run --rm -v $(pwd):$(pwd) src/it/myfunction'
>

So if this is the intended use case, then I have another idea. Instead of
running Jenkinsfile in this CLI and try to emulate your Jenkins instance as
closely as possible (in terms of configuration), we could just run
Jenkinsfile in the current Jenkins, in a place that nobody else can see.

For example, packaged as CLI, you'd run it like:

   $ java -jar jenkins-cli.jar -s https://jenkins.example.com/ run-pipeline
./src/it/myfunction

Or from within Jenkinsfile, could be something like:

  pipeline.sharedlibrary.test './src/it/myfunction'

... and this would completely bypass the challenge of trying to replicate
the Jenkins configuration.



> It turned out that there a two major problems:
>
>1. There's no way to add non-default Jenkins plugins.
>My local test for cloudogu/ces-build-lib
> failed because there was
>no GitHub Groovy Libraries plugin.
>Here, my hope is that Configuration-as-code plugin might help automate
>loading plugins.
>
> Yeah, I'm really looking forward for CaC project to fill this gap, and I
know they are working on it.


>
>1. There's still no way to load a "local" shared library from the same
>repo. So, we still would have to find a way to configure the shared library
>in our Jenkinsfile Runner.
>Loading local shared libraries has already been discussed here
>
>and there .
>
> Good point.


> Once those issues are solved, we'll have a very basic way of automating
> integration tests for shared libraries by executing IT Jenkinsfiles from
> the shared libraries pipeline and failing the build if the IT fails.
>

That'd be really cool, isn't 

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Yeah, there are many possible ways to go about something like this,
including what you described. That's why I'm trying to hear from Bill what
his world looks like. I can use some concrete data points like that.

On Fri, Mar 2, 2018 at 11:08 AM Jesse Glick  wrote:

> On Fri, Mar 2, 2018 at 12:56 PM, Kohsuke Kawaguchi  wrote:
> > well, though. Can you tell me more about this? I'm imagining you'd want
> to
> > be able to selectively mock out some steps (e.g., when Jenkinsfile gets
> to
> > sh "./deploy.sh" don't actually do it and pretend that it succeeded)
>
> One suggestion alluded to in JENKINS-33925 was to have a globally
> recognized “dry-run” flag (akin to `Main.isUnitTest` I suppose) that
> could be checked from various features in core or plugins, so that for
> example the `mail` step would know to just print out the mail it
> _would_ have sent without actually contacting an SMTP server.
>
> This would not help directly with your example above—since Jenkins
> would have no way of knowing whether `deploy.sh` actually had any
> externally visible effects, or was just a build command operating
> locally in the workspace—but perhaps in dry-run mode a special
> environment variable could be set for the whole build which would be
> visible to external processes, so your own script could include
> something like
>
> if [ "$DRY_RUN" -eq true ]
> then
> echo "Would be deploying to ${SERVER}:"
> ls -lR
> exit
> fi
> # else proceed
>
> Not as flexible as the fine-grained mocking available (as I recall) in
> JenkinsPipelineUnit, but perhaps sufficient for many use cases.
>
> > This got me thinking that maybe all I needed was a Jenkins CLI command
> that
> > behind the scene creates a temporary/hidden job on the target Jenkins
> master
> > and run the Pipeline. IOW, keep the same development flow as Jenkinsfile
> > Runner today, but don't run Jenkins locally, just do it on your actual
> > Jenkins.
>
> Already exists (though it uses your real job). See for example
>
> https://jenkins.ci.cloudbees.com/cli/command/replay-pipeline
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr03Mg5r5i-XxMi8TZH3uUE7ER1mox76VuA2mX-OiKciZA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4xBAfGRMXX8yiryE%2B3zv5cdKAefa3L53jb6egjRCBZ14w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

https://github.com/kohsuke/jenkinsfile-runner/pull/8

On Tue, Mar 6, 2018 at 8:13 PM, Oleg Nenashev 
wrote:

> I will create a pull request, stay tuned
>
> On Tuesday, March 6, 2018 at 7:32:21 PM UTC+1, joha...@schnatterer.info
> wrote:
>>
>> Same exception here: java.lang.UnsupportedOperationException: Refusing
>> to marshal io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for
>> security reasons;
>>
>> I created an issue with full stacktrace
>> .
>> Jesse or Oleg, maybe you could elaborate there how we could fix this.
>> Which Jar needs the manifest, which library needs to be whitelisted and
>> how can we achieve this?
>>
>> Cheers,
>> Johannes
>>
>> Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>>>
>>> Jenkinsfile Runner is an experiment to package Jenkins pipeline
>>> execution as a command line tool. The intend use cases include:
>>>
>>>- Use Jenkins in Function-as-a-Service context
>>>- Assist editing Jenkinsfile locally
>>>- Integration test shared libraries
>>>
>>> Over the past year, I've done some deep-dive 1:1 conversations with some
>>> Jenkins users and I felt something like this might move the needle for them
>>> in an important way.
>>>
>>> I'd love to hear any reactions on your side. Could something like this
>>> be important for you, does it miss any key points for you? If you mentally
>>> picture a perfect version of this, what would that do, and how would you
>>> use?
>>>
>>> Let me know!
>>>
>>> --
>>> Kohsuke Kawaguchi
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/jenkinsci-dev/gjz3CDhi-kk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/8ad1c473-5292-4a38-ab6b-ab78c4371092%
> 40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLBA%2B0k%2BXsPW57nJExWuEpao6dEs_h7oSVJ%3DHG5qQyWSRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

I will create a pull request, stay tuned

On Tuesday, March 6, 2018 at 7:32:21 PM UTC+1, joha...@schnatterer.info 
wrote:
>
> Same exception here: java.lang.UnsupportedOperationException: Refusing to 
> marshal io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security 
> reasons; 
>
> I created an issue with full stacktrace 
> .
> Jesse or Oleg, maybe you could elaborate there how we could fix this.
> Which Jar needs the manifest, which library needs to be whitelisted and 
> how can we achieve this?
>
> Cheers,
> Johannes
>
> Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some 
>> Jenkins users and I felt something like this might move the needle for them 
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be 
>> important for you, does it miss any key points for you? If you mentally 
>> picture a perfect version of this, what would that do, and how would you 
>> use?
>>
>> Let me know!
>>
>> -- 
>> Kohsuke Kawaguchi
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/8ad1c473-5292-4a38-ab6b-ab78c4371092%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[johannes]

Same exception here: java.lang.UnsupportedOperationException: Refusing to 
marshal io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security 
reasons; 

I created an issue with full stacktrace 
.
Jesse or Oleg, maybe you could elaborate there how we could fix this.
Which Jar needs the manifest, which library needs to be whitelisted and how 
can we achieve this?

Cheers,
Johannes

Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some 
> Jenkins users and I felt something like this might move the needle for them 
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be 
> important for you, does it miss any key points for you? If you mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/aef469a7-5b3f-4fb1-9ccf-12a11a3d1be5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Oleg Nenashev]

Yeah, the entire library needs to me whitelisted (or packaged as Jenkins
module).
It's actually a quick-win to get it fixed as long as it is a development
tool.

BR, Oleg


On Tue, Mar 6, 2018 at 5:00 PM, Bill Dennis  wrote:

> I too am seeing the JEP-200 issue with these exceptions packaging to a
> Docker image and running with that using Docker for Windows:
>
> java.lang.UnsupportedOperationException: Refusing to marshal io.jenkins.
> jenkinsfile.runner.FileSystemSCM for security reasons; see https://
> jenkins.io/redirect/class-filter/
> java.lang.UnsupportedOperationException: Refusing to marshal io.jenkins.
> jenkinsfile.runner.SetJenkinsfileLocation for security reasons; see https:
> //jenkins.io/redirect/class-filter/
>
> I'd like to try out the Jenkinsfile Runner which looks like a great idea
> but I don't have time to figure this out right now (busy writing a
> plugin..).
>
> --Bill
>
>
> On Tuesday, 6 March 2018 09:22:53 UTC, Michael Neale wrote:
>>
>> Trying this out, looks like I am hitting JEP-200:
>>
>> https://jenkins.io/redirect/class-filter/
>>
>> Need to dig in further (I thought I tried the same version of Jenkins as
>> you). Anyone else seen this?
>>
>>
>>
>> java.lang.UnsupportedOperationException: Refusing to marshal
>> io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security
>> reasons; see https://jenkins.io/redirect/class-filter/
>> at hudson.util.XStream2$BlacklistedTypesConverter.marshal(
>> XStream2.java:543)
>> at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.co
>> nvert(AbstractReferenceMarshaller.java:69)
>> at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(
>> TreeMarshaller.java:58)
>> at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(
>> TreeMarshaller.java:43)
>> at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.
>> convertAnother(AbstractReferenceMarshaller.java:88)
>> at com.thoughtworks.xstream.converters.collections.AbstractColl
>> ectionConverter.writeItem(AbstractCollectionConverter.java:64)
>> at com.thoughtworks.xstream.converters.collections.CollectionCo
>> nverter.marshal(CollectionConverter.java:74)
>> at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.co
>> nvert(AbstractReferenceMarshaller.java:69)
>> at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(
>> TreeMarshaller.java:58)
>> at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.
>> convertAnother(AbstractReferenceMarshaller.java:84)
>> at hudson.util.RobustReflectionConverter.marshallField(RobustRe
>> flectionConverter.java:265)
>> at hudson.util.RobustReflectionConverter$2.writeField(RobustRef
>> lectionConverter.java:252)
>> Caused: java.lang.RuntimeException: Failed to serialize
>> hudson.model.Actionable#actions for class org.jenkinsci.plugins.workflow
>> .job.WorkflowRun
>>
>>
>> On Friday, March 2, 2018 at 6:23:15 AM UTC+11, Kohsuke Kawaguchi wrote:
>>>
>>> Jenkinsfile Runner is an experiment to package Jenkins pipeline
>>> execution as a command line tool. The intend use cases include:
>>>
>>>- Use Jenkins in Function-as-a-Service context
>>>- Assist editing Jenkinsfile locally
>>>- Integration test shared libraries
>>>
>>> Over the past year, I've done some deep-dive 1:1 conversations with some
>>> Jenkins users and I felt something like this might move the needle for them
>>> in an important way.
>>>
>>> I'd love to hear any reactions on your side. Could something like this
>>> be important for you, does it miss any key points for you? If you mentally
>>> picture a perfect version of this, what would that do, and how would you
>>> use?
>>>
>>> Let me know!
>>>
>>> --
>>> Kohsuke Kawaguchi
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/jenkinsci-dev/gjz3CDhi-kk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/da0e3018-9206-43a3-b814-4a65a73d5c17%
> 40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLC6%3DyuTyakmF_cMryuJNa68bLLQO%3DrrR9VU%3D5gmAmfksg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Bill Dennis]

I too am seeing the JEP-200 issue with these exceptions packaging to a 
Docker image and running with that using Docker for Windows:

java.lang.UnsupportedOperationException: Refusing to marshal io.jenkins.
jenkinsfile.runner.FileSystemSCM for security reasons; see https:
//jenkins.io/redirect/class-filter/
java.lang.UnsupportedOperationException: Refusing to marshal io.jenkins.
jenkinsfile.runner.SetJenkinsfileLocation for security reasons; see https:
//jenkins.io/redirect/class-filter/

I'd like to try out the Jenkinsfile Runner which looks like a great idea 
but I don't have time to figure this out right now (busy writing a 
plugin..).

--Bill


On Tuesday, 6 March 2018 09:22:53 UTC, Michael Neale wrote:
>
> Trying this out, looks like I am hitting JEP-200: 
>
> https://jenkins.io/redirect/class-filter/
>
> Need to dig in further (I thought I tried the same version of Jenkins as 
> you). Anyone else seen this? 
>
>
>
> java.lang.UnsupportedOperationException: Refusing to marshal 
> io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security reasons; 
> see https://jenkins.io/redirect/class-filter/
> at 
> hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:543)
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
> at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
> at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
> at 
> com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
> at 
> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
> at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
> at 
> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
> at 
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
> Caused: java.lang.RuntimeException: Failed to serialize 
> hudson.model.Actionable#actions for class 
> org.jenkinsci.plugins.workflow.job.WorkflowRun
>
>
> On Friday, March 2, 2018 at 6:23:15 AM UTC+11, Kohsuke Kawaguchi wrote:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some 
>> Jenkins users and I felt something like this might move the needle for them 
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be 
>> important for you, does it miss any key points for you? If you mentally 
>> picture a perfect version of this, what would that do, and how would you 
>> use?
>>
>> Let me know!
>>
>> -- 
>> Kohsuke Kawaguchi
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/da0e3018-9206-43a3-b814-4a65a73d5c17%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

On Tue, Mar 6, 2018 at 4:22 AM, Michael Neale  wrote:
> java.lang.UnsupportedOperationException: Refusing to marshal 
> io.jenkins.jenkinsfile.runner.SetJenkinsfileLocation for security reasons; 
> see https://jenkins.io/redirect/class-filter/

As suggested in that link, the JAR probably just needs a
`Jenkins-ClassFilter-Whitelisted: true` manifest header.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2dw%3DKoXwWyu-tYubOgN8YBg6dCZzNSsCnHFhcuyTkprA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Bill Dennis]

Thanks Kohsuke, I tried to give some answers to your questions inline 
below, if I didn't mess up the reply..

Bill

On Friday, 2 March 2018 17:57:24 UTC, Kohsuke Kawaguchi wrote:
>
>
>
> On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis  > wrote:
>
>> Hello Kohsuke -
>>
>> I am a developer using Jenkins pipeline quite lot where I work.
>>
>> We are using Jenkins pipelines in two scenarios:
>>
>>- For CI building and testing some of our internal components (what 
>>Jenkins is traditionally used for)
>>- For running / orchestrating complex automation processes (so 
>>Jenkins is talking to some external systems using SOAP / REST etc) via 
>>tooling and even directly via REST using plugins.
>>
>> I have mostly used JenkinsPipelineUnit for testing / validation of the 
>> pipelines and I have been looking into the direct / live approach that Oleg 
>> demonstrated (running Jenkins locally in Docker and getting the pipeline 
>> being developed direct from a host file system volume mount).
>>
>> I think Jenkinsfile Runner would be really useful for developers who 
>> don't need or want the overhead of developing tests with 
>> JenkinsPipelineUnit. I have worked with some developers wanting to develop 
>> Jenkinsfiles for their CI process and the main problem is knowing if the 
>> Jenkinsfile will work when they commit it to the repo. They go round this 
>> loop of commit / fix running in the production Jenkins or using the Jenkins 
>> pipeline "replay" feature. It can be a painful process if you are not 
>> familiar with Jenkins pipeline and Groovy syntax!
>>
>
> This kind of context is really helpful. Thank you!
>
 
Happy to feedback. Thanks for Jenkins and pipeline as code, it helped me 
deliver some projects with Jenkins in a way that I thought would not be 
possible a few years ago.


> I think some things to consider are:
>>
>> - How does the Jenkins Runner replicate the agents / slaves identifiers 
>> on the target Jenkins?
>> - How to deal with tooling on the target Jenkins (custom tools, JDKs, 
>> Gradle, etc)?
>>
>
> Right, I guess your point is that Jenkinsfile Runner should aim to run 
> Jenkinsfile in much more realistic setup, and that doesn't stop at using 
> real Jenkins and real Pipeline plugins, but it also needs to include other 
> configurations of Jenkins. I think Jesse made a similar observation. I have 
> a few thoughts:
>
>- Configuration-as-code 
> could play 
>a role here in terms of letting people define the configuration of Jenkins 
>once and use it both in production and in setup like Jenkinsfile Runner
>- I'm a fan of making Jenkinsfile itself more portable. For example, 
>if people are already in the mode of using docker images to run builds in, 
>then more of the toolings would be packaged in there, and it should allow 
>Jenkinsfile Runner to run your project in much the same way as your 
>production Jenkins. I'm curious how much of this is already reality vs 
>ideal that people are working toward.
>
> Yes, all of this. I have often thought that we need something like 
declarative pipeline for the configuration of Jenkins as code instead of 
going into all those web config pages. Jenkins master as Docker container 
seems good. In our environment we are not currently using Docker but I have 
seen that that is the way to go. Getting a larger organisation to adopt the 
right technology and the associated costs of that is the challenge, so we 
remain using traditional Jenkins slaves and tooling methods. Hopefully 
Dockerised soon. We do use Jenkins Enterprise from CloudBees.


> I think the perfect Jenkinsfile Runner for me would provide:
>>
>> - Somehow capture the plugins, tooling and agents on our production 
>> Jenkins
>> - Validate the Jenkinsfile pipeline syntax
>>
>
> I think this is already happening as a result of actually running the 
> pipeline -- one of the virtue of actually using the real pipeline plugins 
> to run!
>  
>
>> - Validate the Jenkinsfile against the plugins and agents / tooling (fail 
>> if it refers to some tool or agent not configured for example).
>> - Run the Jenkinfile in some sort of "no-op" mode : what would it do if I 
>> ran it, without actually doing anything
>>
>
> This one is interesting. I assumed JenkinsPipelineUnit does this pretty 
> well, though. Can you tell me more about this? I'm imagining you'd want to 
> be able to selectively mock out some steps (e.g., when Jenkinsfile gets to 
> sh "./deploy.sh" don't actually do it and pretend that it succeeded) but 
> more details would be helpful.
>

Yes, in JenkinsPipelineUnit I pretty much mock out everything like calls to 
tools. Because pipeline is based on Groovy I thought I could do some unit 
tests using Spock and Groovy for pipelines but then I discovered it was 
already done and shared as JenkinsPipelineUnit, so hat tip to Ozan there. 
So the Spock unit tests 

Re: POTD: Jenkinsfile Runner

[johannes]

I think Jenkinsfile Runner brings a lot of opportunities for pipeline 
developers. The most obvious ones to me are

   1. Pipeline development (Jenkinsfile)
   2. Shared library development

*Pipeline development*

Right now (as described by others in this thread) pipeline development is 
either a loop of committing / fixing pipelines on production Jenkins, using 
pipeline replay on production Jenkins or setting up a local instance 
manually.

With Jenkinsfile Runner we can get faster feedback without polluting our 
commit or Jenkins build history and don't have to set up a local instance 
manually.


*Shared library development*

Shared library development right now works much in the same as pipeline 
development, except that you have the library code and another (often 
production) Jenkinsfile to maintain, in order to try out (as opposed to 
automatically test) your Jenkinsfile.
For shared libraries, we thankfully already have JenkinsPipelineUnit, that 
makes it easier to implement some tests. However, (as also mentioned by 
others in this thread) this is unit testing only. It mocks most of our 
environment. Often, green unit tests mean nothing for productive use of our 
share library. I even gave up test-driven development for shared libraries, 
in favor of 90s try-and-error-style programming. Because most of the time 
when trying the library with green unit tests in production, it turns out 
that the real Jenkins environment has some restriction that is beyond the 
scope of JenkinsPieplineUnit (e.g. sandboxing).
Worst thing about the current state is that we don't have reliable 
regression tests. A change in shared library with green unit tests is far 
from convincing me that the library will continue to work in production.

With Jenkinsfile Runner we could write small Jenkinsfiles within the shared 
library repo's test folder and run them from the Jenkinsfile of the shared 
lib. Pretty much in the same way as we use Maven Invoker Plugin (as 
mentioned by Jesse) when developing maven plugins.

*A first approach to shared library integration testing with Jenkinsfile 
Runner*
My naive first approach was to build a Docker Image that contains 
Jenkinsfile Runner and all default plugins.

docker run -v~/src/it/myfunction:/workspace schnatterer/jenkinsfile-runner:
1.0-SNAPSHOT-2.108 /workspace

runs the ~/foo/Jenkinsfile using Jenkinsfile Runner with all default 
plugins of Jenkins 2.108.

My idea was to eventually do the same in Jenkinsfile of the shared lib like 
so (not tested)
docker.image('schnatterer/jenkinsfile-runner:1.0-SNAPSHOT-2.108').inside {

sh 'jenkinsfile-runner /app/jenkins /app/plugins src/it/myfunction'
}
or
sh 'docker run --rm -v $(pwd):$(pwd) src/it/myfunction'

It turned out that there a two major problems:

   1. There's no way to add non-default Jenkins plugins.
   My local test for cloudogu/ces-build-lib 
    failed because there was no 
   GitHub Groovy Libraries plugin.
   Here, my hope is that Configuration-as-code plugin might help automate 
   loading plugins.
   2. There's still no way to load a "local" shared library from the same 
   repo. So, we still would have to find a way to configure the shared library 
   in our Jenkinsfile Runner.
   Loading local shared libraries has already been discussed here 
    
   and there .

Once those issues are solved, we'll have a very basic way of automating 
integration tests for shared libraries by executing IT Jenkinsfiles from 
the shared libraries pipeline and failing the build if the IT fails.


Of course, this would be very basic testing. For more sophistiated testing 
we would want to


   - trigger the ITs from maven or gradle,
   - use asserts,
   - get the results as JUnit XML.

So, yes, we're not there yet. But we now have a foundation to build all 
this upon.

Thanks for that & best regards,

Johannes

Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some 
> Jenkins users and I felt something like this might move the needle for them 
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be 
> important for you, does it miss any key points for you? If you mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>


Am Donnerstag, 1. März 2018 20:23:15 UTC+1 schrieb Kohsuke Kawaguchi:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line 

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

On Fri, Mar 2, 2018 at 12:56 PM, Kohsuke Kawaguchi  wrote:
> well, though. Can you tell me more about this? I'm imagining you'd want to
> be able to selectively mock out some steps (e.g., when Jenkinsfile gets to
> sh "./deploy.sh" don't actually do it and pretend that it succeeded)

One suggestion alluded to in JENKINS-33925 was to have a globally
recognized “dry-run” flag (akin to `Main.isUnitTest` I suppose) that
could be checked from various features in core or plugins, so that for
example the `mail` step would know to just print out the mail it
_would_ have sent without actually contacting an SMTP server.

This would not help directly with your example above—since Jenkins
would have no way of knowing whether `deploy.sh` actually had any
externally visible effects, or was just a build command operating
locally in the workspace—but perhaps in dry-run mode a special
environment variable could be set for the whole build which would be
visible to external processes, so your own script could include
something like

if [ "$DRY_RUN" -eq true ]
then
echo "Would be deploying to ${SERVER}:"
ls -lR
exit
fi
# else proceed

Not as flexible as the fine-grained mocking available (as I recall) in
JenkinsPipelineUnit, but perhaps sufficient for many use cases.

> This got me thinking that maybe all I needed was a Jenkins CLI command that
> behind the scene creates a temporary/hidden job on the target Jenkins master
> and run the Pipeline. IOW, keep the same development flow as Jenkinsfile
> Runner today, but don't run Jenkins locally, just do it on your actual
> Jenkins.

Already exists (though it uses your real job). See for example

https://jenkins.ci.cloudbees.com/cli/command/replay-pipeline

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr03Mg5r5i-XxMi8TZH3uUE7ER1mox76VuA2mX-OiKciZA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

On Fri, Mar 2, 2018 at 9:26 AM Bill Dennis  wrote:

> Hello Kohsuke -
>
> I am a developer using Jenkins pipeline quite lot where I work.
>
> We are using Jenkins pipelines in two scenarios:
>
>- For CI building and testing some of our internal components (what
>Jenkins is traditionally used for)
>- For running / orchestrating complex automation processes (so Jenkins
>is talking to some external systems using SOAP / REST etc) via tooling and
>even directly via REST using plugins.
>
> I have mostly used JenkinsPipelineUnit for testing / validation of the
> pipelines and I have been looking into the direct / live approach that Oleg
> demonstrated (running Jenkins locally in Docker and getting the pipeline
> being developed direct from a host file system volume mount).
>
> I think Jenkinsfile Runner would be really useful for developers who don't
> need or want the overhead of developing tests with JenkinsPipelineUnit. I
> have worked with some developers wanting to develop Jenkinsfiles for their
> CI process and the main problem is knowing if the Jenkinsfile will work
> when they commit it to the repo. They go round this loop of commit / fix
> running in the production Jenkins or using the Jenkins pipeline "replay"
> feature. It can be a painful process if you are not familiar with Jenkins
> pipeline and Groovy syntax!
>

This kind of context is really helpful. Thank you!

I think some things to consider are:
>
> - How does the Jenkins Runner replicate the agents / slaves identifiers on
> the target Jenkins?
> - How to deal with tooling on the target Jenkins (custom tools, JDKs,
> Gradle, etc)?
>

Right, I guess your point is that Jenkinsfile Runner should aim to run
Jenkinsfile in much more realistic setup, and that doesn't stop at using
real Jenkins and real Pipeline plugins, but it also needs to include other
configurations of Jenkins. I think Jesse made a similar observation. I have
a few thoughts:

   - Configuration-as-code
    could play a
   role here in terms of letting people define the configuration of Jenkins
   once and use it both in production and in setup like Jenkinsfile Runner
   - I'm a fan of making Jenkinsfile itself more portable. For example, if
   people are already in the mode of using docker images to run builds in,
   then more of the toolings would be packaged in there, and it should allow
   Jenkinsfile Runner to run your project in much the same way as your
   production Jenkins. I'm curious how much of this is already reality vs
   ideal that people are working toward.


I think the perfect Jenkinsfile Runner for me would provide:
>
> - Somehow capture the plugins, tooling and agents on our production Jenkins
> - Validate the Jenkinsfile pipeline syntax
>

I think this is already happening as a result of actually running the
pipeline -- one of the virtue of actually using the real pipeline plugins
to run!


> - Validate the Jenkinsfile against the plugins and agents / tooling (fail
> if it refers to some tool or agent not configured for example).
> - Run the Jenkinfile in some sort of "no-op" mode : what would it do if I
> ran it, without actually doing anything
>

This one is interesting. I assumed JenkinsPipelineUnit does this pretty
well, though. Can you tell me more about this? I'm imagining you'd want to
be able to selectively mock out some steps (e.g., when Jenkinsfile gets to
sh "./deploy.sh" don't actually do it and pretend that it succeeded) but
more details would be helpful.


> - Actually run the Jenkinsfile locally so I can know it works completely
> before committing to source control.
>

Yeah, this was the first goal for me.


> - Run the Jenkinsfile on the target Jenkins master server using the
> resources of that server (so know it works on the server).
>

This got me thinking that maybe all I needed was a Jenkins CLI command that
behind the scene creates a temporary/hidden job on the target Jenkins
master and run the Pipeline. IOW, keep the same development flow as
Jenkinsfile Runner today, but don't run Jenkins locally, just do it on your
actual Jenkins.


>
> Hope that helps!
>
> Bill Dennis
>
>
> On Thursday, 1 March 2018 19:23:15 UTC, Kohsuke Kawaguchi wrote:
>>
>> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
>> as a command line tool. The intend use cases include:
>>
>>- Use Jenkins in Function-as-a-Service context
>>- Assist editing Jenkinsfile locally
>>- Integration test shared libraries
>>
>> Over the past year, I've done some deep-dive 1:1 conversations with some
>> Jenkins users and I felt something like this might move the needle for them
>> in an important way.
>>
>> I'd love to hear any reactions on your side. Could something like this be
>> important for you, does it miss any key points for you? If you mentally
>> picture a perfect version of this, what would that do, and how would you
>> use?
>>
>> Let 

Re: POTD: Jenkinsfile Runner

[Bill Dennis]

Hello Kohsuke -

I am a developer using Jenkins pipeline quite lot where I work.

We are using Jenkins pipelines in two scenarios:

   - For CI building and testing some of our internal components (what 
   Jenkins is traditionally used for)
   - For running / orchestrating complex automation processes (so Jenkins 
   is talking to some external systems using SOAP / REST etc) via tooling and 
   even directly via REST using plugins.

I have mostly used JenkinsPipelineUnit for testing / validation of the 
pipelines and I have been looking into the direct / live approach that Oleg 
demonstrated (running Jenkins locally in Docker and getting the pipeline 
being developed direct from a host file system volume mount).

I think Jenkinsfile Runner would be really useful for developers who don't 
need or want the overhead of developing tests with JenkinsPipelineUnit. I 
have worked with some developers wanting to develop Jenkinsfiles for their 
CI process and the main problem is knowing if the Jenkinsfile will work 
when they commit it to the repo. They go round this loop of commit / fix 
running in the production Jenkins or using the Jenkins pipeline "replay" 
feature. It can be a painful process if you are not familiar with Jenkins 
pipeline and Groovy syntax!

I think some things to consider are:

- How does the Jenkins Runner replicate the agents / slaves identifiers on 
the target Jenkins?
- How to deal with tooling on the target Jenkins (custom tools, JDKs, 
Gradle, etc)?

I think the perfect Jenkinsfile Runner for me would provide:

- Somehow capture the plugins, tooling and agents on our production Jenkins
- Validate the Jenkinsfile pipeline syntax
- Validate the Jenkinsfile against the plugins and agents / tooling (fail 
if it refers to some tool or agent not configured for example).
- Run the Jenkinfile in some sort of "no-op" mode : what would it do if I 
ran it, without actually doing anything
- Actually run the Jenkinsfile locally so I can know it works completely 
before committing to source control.
- Run the Jenkinsfile on the target Jenkins master server using the 
resources of that server (so know it works on the server).

Hope that helps!

Bill Dennis


On Thursday, 1 March 2018 19:23:15 UTC, Kohsuke Kawaguchi wrote:
>
> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution 
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some 
> Jenkins users and I felt something like this might move the needle for them 
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be 
> important for you, does it miss any key points for you? If you mentally 
> picture a perfect version of this, what would that do, and how would you 
> use?
>
> Let me know!
>
> -- 
> Kohsuke Kawaguchi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/b475ecc2-2807-47c7-809f-5e6c1a51d2e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Jesse Glick]

For the third use case, what might be useful is an archetype or two analogous to

https://github.com/jenkinsci/archetypes/tree/master/scripted-pipeline/src/main/resources/archetype-resources
https://github.com/jenkinsci/archetypes/tree/master/global-shared-library/src/main/resources/archetype-resources

which use the Docker packaging and take care of all the boilerplate
needed to have a Maven module which, when “built”, runs the specified
`Jenkinsfile` or library against some test inputs in a well-controlled
environment, and presumably passes or fails according to some
conditions. (For example, Groovy validation scripts, akin to Maven
Invoker?) I think the hard part would be stubbing out side effects you
do not want—something handled easily by JenkinsPipelineUnit, at the
cost of running in an unrealistic environment.


For anyone interested in this general topic,

https://issues.jenkins-ci.org/browse/JENKINS-33925

has some longstanding discussion and ideas. (And, wow, 85 votes! Alas
there are no clear acceptance criteria for closing it.)

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3fDig6_-X-Pjnm1jRgv3RFZnC59ezcVpuSsjU4Wrw9sQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

And of course I forgot to have the link to the project!
https://github.com/kohsuke/jenkinsfile-runner

On Thu, Mar 1, 2018 at 11:22 AM Kohsuke Kawaguchi  wrote:

> Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
> as a command line tool. The intend use cases include:
>
>- Use Jenkins in Function-as-a-Service context
>- Assist editing Jenkinsfile locally
>- Integration test shared libraries
>
> Over the past year, I've done some deep-dive 1:1 conversations with some
> Jenkins users and I felt something like this might move the needle for them
> in an important way.
>
> I'd love to hear any reactions on your side. Could something like this be
> important for you, does it miss any key points for you? If you mentally
> picture a perfect version of this, what would that do, and how would you
> use?
>
> Let me know!
>
> --
> Kohsuke Kawaguchi
>
-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4y9e33M%3D%2Bd7umCtauMSCSaH1TkJM_UYhnSqHdH_G3VrHw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

POTD: Jenkinsfile Runner

[Kohsuke Kawaguchi]

Jenkinsfile Runner is an experiment to package Jenkins pipeline execution
as a command line tool. The intend use cases include:

   - Use Jenkins in Function-as-a-Service context
   - Assist editing Jenkinsfile locally
   - Integration test shared libraries

Over the past year, I've done some deep-dive 1:1 conversations with some
Jenkins users and I felt something like this might move the needle for them
in an important way.

I'd love to hear any reactions on your side. Could something like this be
important for you, does it miss any key points for you? If you mentally
picture a perfect version of this, what would that do, and how would you
use?

Let me know!

-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4xMgdsJRdjTMH-U%3DRUbvopDqiAm4o3r9AjdbjuPTmFnow%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.