Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-06-03 Thread Oleg Nenashev 
Hi all,

Just a quick summary from the yesterday's governance meeting:

   - We approve Elastic sponsorship and further collaboration on OpenTelemetry 
   for ci.jenkins.io. The Jenkins infra team is clear to proceed 
   - FTR we do not think the governance meeting approval is needed in this 
   case. Jenkins Infra officer can represent the Jenkins community when it 
   comes to infra matters. 
   - Taking the security and data privacy considerations, we recommend to 
   not connect other Jenkins instances in our infra until there is an approval 
   from the Jenkins Security Officer
   
And thanks again to all contributors!

Best regards,
Oleg Nenashev


On Thursday, June 3, 2021 at 4:30:05 PM UTC+2 Cyrille Le Clerc wrote:

> Hello Ulrich,
>
> We documented all the data we collect on h the README.md 
>  
> page of the plugin. 
>
> We capture :
> * the details collected on pipeline execution as span attributes (see list 
> here )
> * metrics on job executions, build queue, agent lifecycle, security events 
> (see list here 
> 
> ) 
>
> Note that we don't capture logs for the moment but we are interested in 
> covering this in the future.
>
> Cyrille
>
>
>
>
> On Monday, May 31, 2021 at 11:13:41 PM UTC+2 ullrich...@gmail.com wrote:
>
>> Can you elaborate what information you want to gather? What build metrics 
>> and traces? 
>>
>> Am 31.05.2021 um 11:47 schrieb Damien Duportal :
>>
>> Good point Oleg, thanks a lot for this reminder!
>>
>> Any test on infra.ci.jenkins.io must be done on a private APM instance, 
>> to ensure no sensitive data is leaked. It means that we should only do it 
>> for testing purpose, and the feedback to the community will need a special 
>> attention on the data presented.
>>
>> Damien
>>
>> Le 28 mai 2021 à 12:58, Oleg Nenashev  a écrit :
>>
>> Connecting ci.jenkins.io looks perfectly fine to me. There is no 
>> sensitive data on this instance. For other Jenkins instances (infra, 
>> release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
>> Officer is required IMHO
>>
>> On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:
>>
>>> Sounds great
>>>
>>> On Fri, 28 May 2021 at 09:30, Damien Duportal  
>>> wrote:
>>>
 [Cross posted on jenkins-dev and jenkins-infra]

 Hello dear community,

 During the last Jenkins infrastructure weekly meeting (2021-05-25 - 
 Notes -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), 
 Victor Martinez and Iván Fernández from Elastic did a demonstration of the 
 opentelemetry plugin used with an Elastic APM platform.
 The goal was to demonstrate how the build metrics could be used with 
 distributed tracing: please check the notes, video and 
 https://plugins.jenkins.io/opentelemetry/ for more details.


 Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the 
 Jenkins project by providing an APM instance that could be used along with 
 opentelemetry plugin, and a bit support to get it configured and 
 maintained.
 Their interest is to study what are our (Jenkins community) usages (and 
 obviously mentioning that we are using it).

 The proposal from the Infrastructure team is to provide this service 
 for the 2 following use cases:

- For users of ci.jenkins.io (core and plugins contributors) to get 
build metrics and traces of their projects. It could help a lot for 
performances tracking, and much more as "Jenkins Pipeline level metrics"
- For infrastructure maintainers of ci.jenkins.io to track the 
agents allocations and their errors, to improve the QoS of the build 
 farm, 
as the pure infrastructure metrics is not enough because issues come 
 from 
different areas (infra, Cloud providers, Jenkins configuration, 
 Pipeline 
usages, etc.)


 It could be a great opportunity for whoever interested in better 
 understanding Jenkins behavior. As we'll try to identify typical use cases 
 that we want to better understand and then build dashboards using metric.
 The challenge we face on the Jenkins infrastructure is that it's pretty 
 easy to know if Jenkins is running. But not so much if it's working in an 
 efficient way.

 The plan would be the following:

- Start by a PoC on a private Jenkins instance  (e.g. 
infra.ci.jenkins.io)
   - Install the plugin
   - Check APM instance access with Elastic
   - Configure the plugin to point to the APM
   - Bring back the conclusion to the community (by email here, 
   blog post, meetup, whatever)
- Once the configuration is straightforward, deploying to 
c

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-06-03 Thread 'Cyrille Le Clerc' via Jenkins Developers 
Hello Ulrich,

We documented all the data we collect on h the README.md 
 
page of the plugin. 

We capture :
* the details collected on pipeline execution as span attributes (see list 
here )
* metrics on job executions, build queue, agent lifecycle, security events 
(see list here 

) 

Note that we don't capture logs for the moment but we are interested in 
covering this in the future.

Cyrille




On Monday, May 31, 2021 at 11:13:41 PM UTC+2 ullrich...@gmail.com wrote:

> Can you elaborate what information you want to gather? What build metrics 
> and traces? 
>
> Am 31.05.2021 um 11:47 schrieb Damien Duportal :
>
> Good point Oleg, thanks a lot for this reminder!
>
> Any test on infra.ci.jenkins.io must be done on a private APM instance, 
> to ensure no sensitive data is leaked. It means that we should only do it 
> for testing purpose, and the feedback to the community will need a special 
> attention on the data presented.
>
> Damien
>
> Le 28 mai 2021 à 12:58, Oleg Nenashev  a écrit :
>
> Connecting ci.jenkins.io looks perfectly fine to me. There is no 
> sensitive data on this instance. For other Jenkins instances (infra, 
> release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
> Officer is required IMHO
>
> On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:
>
>> Sounds great
>>
>> On Fri, 28 May 2021 at 09:30, Damien Duportal  
>> wrote:
>>
>>> [Cross posted on jenkins-dev and jenkins-infra]
>>>
>>> Hello dear community,
>>>
>>> During the last Jenkins infrastructure weekly meeting (2021-05-25 - 
>>> Notes -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), 
>>> Victor Martinez and Iván Fernández from Elastic did a demonstration of the 
>>> opentelemetry plugin used with an Elastic APM platform.
>>> The goal was to demonstrate how the build metrics could be used with 
>>> distributed tracing: please check the notes, video and 
>>> https://plugins.jenkins.io/opentelemetry/ for more details.
>>>
>>>
>>> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the 
>>> Jenkins project by providing an APM instance that could be used along with 
>>> opentelemetry plugin, and a bit support to get it configured and maintained.
>>> Their interest is to study what are our (Jenkins community) usages (and 
>>> obviously mentioning that we are using it).
>>>
>>> The proposal from the Infrastructure team is to provide this service for 
>>> the 2 following use cases:
>>>
>>>- For users of ci.jenkins.io (core and plugins contributors) to get 
>>>build metrics and traces of their projects. It could help a lot for 
>>>performances tracking, and much more as "Jenkins Pipeline level metrics"
>>>- For infrastructure maintainers of ci.jenkins.io to track the 
>>>agents allocations and their errors, to improve the QoS of the build 
>>> farm, 
>>>as the pure infrastructure metrics is not enough because issues come 
>>> from 
>>>different areas (infra, Cloud providers, Jenkins configuration, Pipeline 
>>>usages, etc.)
>>>
>>>
>>> It could be a great opportunity for whoever interested in better 
>>> understanding Jenkins behavior. As we'll try to identify typical use cases 
>>> that we want to better understand and then build dashboards using metric.
>>> The challenge we face on the Jenkins infrastructure is that it's pretty 
>>> easy to know if Jenkins is running. But not so much if it's working in an 
>>> efficient way.
>>>
>>> The plan would be the following:
>>>
>>>- Start by a PoC on a private Jenkins instance  (e.g. 
>>>infra.ci.jenkins.io)
>>>   - Install the plugin
>>>   - Check APM instance access with Elastic
>>>   - Configure the plugin to point to the APM
>>>   - Bring back the conclusion to the community (by email here, blog 
>>>   post, meetup, whatever)
>>>- Once the configuration is straightforward, deploying to 
>>>ci.jenkins.io, targeting as much publicly available dashboards and 
>>>data as possible, as service for the ci.jenkins.io users.
>>>
>>> What do you think about this topic, what would be your thoughts or red 
>>> flags?
>>>
>>> Cheers,
>>>
>>> For the Infra. team,
>>> Damien DUPORTAL
>>>
>>> PS: by default, without any blocker or red flags, we (the infra.team) 
>>> will proceed with the infra.ci setup described in this email the 1st of 
>>> June.
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Infrastructure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkins-infr...@googlegroups.com.
>>> To view this discussion on the web, visit 
>>> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
>>>

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-31 Thread Ullrich Hafner 
Can you elaborate what information you want to gather? What build metrics and 
traces? 

> Am 31.05.2021 um 11:47 schrieb Damien Duportal :
> 
> Good point Oleg, thanks a lot for this reminder!
> 
> Any test on infra.ci.jenkins.io  must be done on 
> a private APM instance, to ensure no sensitive data is leaked. It means that 
> we should only do it for testing purpose, and the feedback to the community 
> will need a special attention on the data presented.
> 
> Damien
> 
>> Le 28 mai 2021 à 12:58, Oleg Nenashev > > a écrit :
>> 
>> Connecting ci.jenkins.io  looks perfectly fine to me. 
>> There is no sensitive data on this instance. For other Jenkins instances 
>> (infra, release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
>> Officer is required IMHO
>> 
>> On Fri, May 28, 2021, 10:40 Tim Jacomb > > wrote:
>> Sounds great
>> 
>> On Fri, 28 May 2021 at 09:30, Damien Duportal > > wrote:
>> [Cross posted on jenkins-dev and jenkins-infra]
>> 
>> Hello dear community,
>> 
>> During the last Jenkins infrastructure weekly meeting (2021-05-25 - Notes 
>> -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw 
>> ), Victor Martinez and Iván 
>> Fernández from Elastic did a demonstration of the opentelemetry plugin used 
>> with an Elastic APM platform.
>> The goal was to demonstrate how the build metrics could be used with 
>> distributed tracing: please check the notes, video and 
>> https://plugins.jenkins.io/opentelemetry/ 
>>  for more details.
>> 
>> 
>> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the Jenkins 
>> project by providing an APM instance that could be used along with 
>> opentelemetry plugin, and a bit support to get it configured and maintained.
>> Their interest is to study what are our (Jenkins community) usages (and 
>> obviously mentioning that we are using it).
>> 
>> The proposal from the Infrastructure team is to provide this service for the 
>> 2 following use cases:
>> For users of ci.jenkins.io  (core and plugins 
>> contributors) to get build metrics and traces of their projects. It could 
>> help a lot for performances tracking, and much more as "Jenkins Pipeline 
>> level metrics"
>> For infrastructure maintainers of ci.jenkins.io  to 
>> track the agents allocations and their errors, to improve the QoS of the 
>> build farm, as the pure infrastructure metrics is not enough because issues 
>> come from different areas (infra, Cloud providers, Jenkins configuration, 
>> Pipeline usages, etc.)
>> 
>> It could be a great opportunity for whoever interested in better 
>> understanding Jenkins behavior. As we'll try to identify typical use cases 
>> that we want to better understand and then build dashboards using metric.
>> The challenge we face on the Jenkins infrastructure is that it's pretty easy 
>> to know if Jenkins is running. But not so much if it's working in an 
>> efficient way.
>> 
>> The plan would be the following:
>> Start by a PoC on a private Jenkins instance  (e.g. infra.ci.jenkins.io 
>> )
>> Install the plugin
>> Check APM instance access with Elastic
>> Configure the plugin to point to the APM
>> Bring back the conclusion to the community (by email here, blog post, 
>> meetup, whatever)
>> Once the configuration is straightforward, deploying to ci.jenkins.io 
>> , targeting as much publicly available dashboards and 
>> data as possible, as service for the ci.jenkins.io  
>> users.
>> What do you think about this topic, what would be your thoughts or red flags?
>> 
>> Cheers,
>> 
>> For the Infra. team,
>> Damien DUPORTAL
>> 
>> PS: by default, without any blocker or red flags, we (the infra.team) will 
>> proceed with the infra.ci  setup described in this email 
>> the 1st of June.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Infrastructure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkins-infra+unsubscr...@googlegroups.com 
>> .
>> To view this discussion on the web, visit 
>> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
>>  
>> .
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Infrastructure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkins-infra+unsubscr...@googlegroups.com 
>>

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-31 Thread Damien Duportal 
Good point Oleg, thanks a lot for this reminder!

Any test on infra.ci.jenkins.io  must be done on a 
private APM instance, to ensure no sensitive data is leaked. It means that we 
should only do it for testing purpose, and the feedback to the community will 
need a special attention on the data presented.

Damien

> Le 28 mai 2021 à 12:58, Oleg Nenashev  a écrit :
> 
> Connecting ci.jenkins.io  looks perfectly fine to me. 
> There is no sensitive data on this instance. For other Jenkins instances 
> (infra, release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
> Officer is required IMHO
> 
> On Fri, May 28, 2021, 10:40 Tim Jacomb  > wrote:
> Sounds great
> 
> On Fri, 28 May 2021 at 09:30, Damien Duportal  > wrote:
> [Cross posted on jenkins-dev and jenkins-infra]
> 
> Hello dear community,
> 
> During the last Jenkins infrastructure weekly meeting (2021-05-25 - Notes 
> -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw 
> ), Victor Martinez and Iván 
> Fernández from Elastic did a demonstration of the opentelemetry plugin used 
> with an Elastic APM platform.
> The goal was to demonstrate how the build metrics could be used with 
> distributed tracing: please check the notes, video and 
> https://plugins.jenkins.io/opentelemetry/ 
>  for more details.
> 
> 
> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the Jenkins 
> project by providing an APM instance that could be used along with 
> opentelemetry plugin, and a bit support to get it configured and maintained.
> Their interest is to study what are our (Jenkins community) usages (and 
> obviously mentioning that we are using it).
> 
> The proposal from the Infrastructure team is to provide this service for the 
> 2 following use cases:
> For users of ci.jenkins.io  (core and plugins 
> contributors) to get build metrics and traces of their projects. It could 
> help a lot for performances tracking, and much more as "Jenkins Pipeline 
> level metrics"
> For infrastructure maintainers of ci.jenkins.io  to 
> track the agents allocations and their errors, to improve the QoS of the 
> build farm, as the pure infrastructure metrics is not enough because issues 
> come from different areas (infra, Cloud providers, Jenkins configuration, 
> Pipeline usages, etc.)
> 
> It could be a great opportunity for whoever interested in better 
> understanding Jenkins behavior. As we'll try to identify typical use cases 
> that we want to better understand and then build dashboards using metric.
> The challenge we face on the Jenkins infrastructure is that it's pretty easy 
> to know if Jenkins is running. But not so much if it's working in an 
> efficient way.
> 
> The plan would be the following:
> Start by a PoC on a private Jenkins instance  (e.g. infra.ci.jenkins.io 
> )
> Install the plugin
> Check APM instance access with Elastic
> Configure the plugin to point to the APM
> Bring back the conclusion to the community (by email here, blog post, meetup, 
> whatever)
> Once the configuration is straightforward, deploying to ci.jenkins.io 
> , targeting as much publicly available dashboards and 
> data as possible, as service for the ci.jenkins.io  
> users.
> What do you think about this topic, what would be your thoughts or red flags?
> 
> Cheers,
> 
> For the Infra. team,
> Damien DUPORTAL
> 
> PS: by default, without any blocker or red flags, we (the infra.team) will 
> proceed with the infra.ci  setup described in this email 
> the 1st of June.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkins-infra+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web, visit 
> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
>  
> .
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkins-infra+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web, visit 
> https://groups.google.com/d/msgid/jenkins-infra/CAH-3Bie3U4UOxmSLpex8sSw48FQeVGwJjXP4KLQTPWBAon%3DS1Q%40mail.gmail.com
>  
>

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-30 Thread Mark Waite 
+1 from me for sponsorship and evaluation

On Sat, May 29, 2021 at 11:28 PM Oleg Nenashev 
wrote:

> Hi all,
>
> I have added this topic to the next Governance Meeting agenda:
> https://docs.google.com/document/d/11Nr8QpqYgBiZjORplL_3Zkwys2qK1vEvK-NYyYa4rzg/edit#heading=h.1gtco63t6ztr
> Casting explicit +1 for proceeding with sponsorship and evaluation
>
> Best regards,
> Oleg Nenashev
>
> On Friday, May 28, 2021 at 3:35:54 PM UTC+2 Cyrille Le Clerc wrote:
>
>> Thank you Damien.
>>
>> Elastic is very excited to collaborate with the Jenkins community on
>> better observability for Jenkins and for CI/CD at large.
>> Please note that these observability are vendor neutral, there is no
>> vendor lock-in and what we will accomplish with the Elastic Observability
>> backend will be achievable with other observability solution that also
>> integrate with OpenTelemetry such as Jaeger and Prometheus.
>>
>> Victor, Ivan and I will do our best to offer an unprecedented User
>> Experience with Elastic Observability there will be no lock in, it will
>> alway be possible to switch to other observability backends :-)
>>
>> Cyrille
>>
>> On Friday, May 28, 2021 at 12:59:06 PM UTC+2 Oleg Nenashev wrote:
>>
>>> Connecting ci.jenkins.io looks perfectly fine to me. There is no
>>> sensitive data on this instance. For other Jenkins instances (infra,
>>> release, trusted-ci, cert, ...) a sign-off from the Jenkins Security
>>> Officer is required IMHO
>>>
>>> On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:
>>>
 Sounds great

 On Fri, 28 May 2021 at 09:30, Damien Duportal 
 wrote:

> [Cross posted on jenkins-dev and jenkins-infra]
>
> Hello dear community,
>
> During the last Jenkins infrastructure weekly meeting (2021-05-25 -
> Notes -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw),
> Victor Martinez and Iván Fernández from Elastic did a demonstration of the
> opentelemetry plugin used with an Elastic APM platform.
> The goal was to demonstrate how the build metrics could be used with
> distributed tracing: please check the notes, video and
> https://plugins.jenkins.io/opentelemetry/ for more details.
>
>
> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the
> Jenkins project by providing an APM instance that could be used along with
> opentelemetry plugin, and a bit support to get it configured and 
> maintained.
> Their interest is to study what are our (Jenkins community) usages
> (and obviously mentioning that we are using it).
>
> The proposal from the Infrastructure team is to provide this service
> for the 2 following use cases:
>
>- For users of ci.jenkins.io (core and plugins contributors) to
>get build metrics and traces of their projects. It could help a lot for
>performances tracking, and much more as "Jenkins Pipeline level 
> metrics"
>- For infrastructure maintainers of ci.jenkins.io to track the
>agents allocations and their errors, to improve the QoS of the build 
> farm,
>as the pure infrastructure metrics is not enough because issues come 
> from
>different areas (infra, Cloud providers, Jenkins configuration, 
> Pipeline
>usages, etc.)
>
>
> It could be a great opportunity for whoever interested in better
> understanding Jenkins behavior. As we'll try to identify typical use cases
> that we want to better understand and then build dashboards using metric.
> The challenge we face on the Jenkins infrastructure is that it's
> pretty easy to know if Jenkins is running. But not so much if it's working
> in an efficient way.
>
> The plan would be the following:
>
>- Start by a PoC on a private Jenkins instance  (e.g.
>infra.ci.jenkins.io)
>   - Install the plugin
>   - Check APM instance access with Elastic
>   - Configure the plugin to point to the APM
>   - Bring back the conclusion to the community (by email here,
>   blog post, meetup, whatever)
>- Once the configuration is straightforward, deploying to
>ci.jenkins.io, targeting as much publicly available dashboards and
>data as possible, as service for the ci.jenkins.io users.
>
> What do you think about this topic, what would be your thoughts or red
> flags?
>
> Cheers,
>
> For the Infra. team,
> Damien DUPORTAL
>
> PS: by default, without any blocker or red flags, we (the infra.team)
> will proceed with the infra.ci setup described in this email the 1st
> of June.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkins-infr...@googlegroups.com.
> To view this discussion on the web, visit
>>

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-29 Thread Oleg Nenashev 
Hi all,

I have added this topic to the next Governance Meeting 
agenda: 
https://docs.google.com/document/d/11Nr8QpqYgBiZjORplL_3Zkwys2qK1vEvK-NYyYa4rzg/edit#heading=h.1gtco63t6ztr
Casting explicit +1 for proceeding with sponsorship and evaluation

Best regards,
Oleg Nenashev

On Friday, May 28, 2021 at 3:35:54 PM UTC+2 Cyrille Le Clerc wrote:

> Thank you Damien.
>
> Elastic is very excited to collaborate with the Jenkins community on 
> better observability for Jenkins and for CI/CD at large.
> Please note that these observability are vendor neutral, there is no 
> vendor lock-in and what we will accomplish with the Elastic Observability 
> backend will be achievable with other observability solution that also 
> integrate with OpenTelemetry such as Jaeger and Prometheus. 
>
> Victor, Ivan and I will do our best to offer an unprecedented User 
> Experience with Elastic Observability there will be no lock in, it will 
> alway be possible to switch to other observability backends :-) 
>
> Cyrille
>
> On Friday, May 28, 2021 at 12:59:06 PM UTC+2 Oleg Nenashev wrote:
>
>> Connecting ci.jenkins.io looks perfectly fine to me. There is no 
>> sensitive data on this instance. For other Jenkins instances (infra, 
>> release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
>> Officer is required IMHO
>>
>> On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:
>>
>>> Sounds great
>>>
>>> On Fri, 28 May 2021 at 09:30, Damien Duportal  
>>> wrote:
>>>
 [Cross posted on jenkins-dev and jenkins-infra]

 Hello dear community,

 During the last Jenkins infrastructure weekly meeting (2021-05-25 - 
 Notes -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), 
 Victor Martinez and Iván Fernández from Elastic did a demonstration of the 
 opentelemetry plugin used with an Elastic APM platform.
 The goal was to demonstrate how the build metrics could be used with 
 distributed tracing: please check the notes, video and 
 https://plugins.jenkins.io/opentelemetry/ for more details.


 Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the 
 Jenkins project by providing an APM instance that could be used along with 
 opentelemetry plugin, and a bit support to get it configured and 
 maintained.
 Their interest is to study what are our (Jenkins community) usages (and 
 obviously mentioning that we are using it).

 The proposal from the Infrastructure team is to provide this service 
 for the 2 following use cases:

- For users of ci.jenkins.io (core and plugins contributors) to get 
build metrics and traces of their projects. It could help a lot for 
performances tracking, and much more as "Jenkins Pipeline level metrics"
- For infrastructure maintainers of ci.jenkins.io to track the 
agents allocations and their errors, to improve the QoS of the build 
 farm, 
as the pure infrastructure metrics is not enough because issues come 
 from 
different areas (infra, Cloud providers, Jenkins configuration, 
 Pipeline 
usages, etc.)


 It could be a great opportunity for whoever interested in better 
 understanding Jenkins behavior. As we'll try to identify typical use cases 
 that we want to better understand and then build dashboards using metric.
 The challenge we face on the Jenkins infrastructure is that it's pretty 
 easy to know if Jenkins is running. But not so much if it's working in an 
 efficient way.

 The plan would be the following:

- Start by a PoC on a private Jenkins instance  (e.g. 
infra.ci.jenkins.io)
   - Install the plugin
   - Check APM instance access with Elastic
   - Configure the plugin to point to the APM
   - Bring back the conclusion to the community (by email here, 
   blog post, meetup, whatever)
- Once the configuration is straightforward, deploying to 
ci.jenkins.io, targeting as much publicly available dashboards and 
data as possible, as service for the ci.jenkins.io users.

 What do you think about this topic, what would be your thoughts or red 
 flags?

 Cheers,

 For the Infra. team,
 Damien DUPORTAL

 PS: by default, without any blocker or red flags, we (the infra.team) 
 will proceed with the infra.ci setup described in this email the 1st 
 of June.

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Jenkins Infrastructure" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to jenkins-infr...@googlegroups.com.
 To view this discussion on the web, visit 
 https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-28 Thread 'Cyrille Le Clerc' via Jenkins Developers 
Thank you Damien.

Elastic is very excited to collaborate with the Jenkins community on better 
observability for Jenkins and for CI/CD at large.
Please note that these observability are vendor neutral, there is no vendor 
lock-in and what we will accomplish with the Elastic Observability backend 
will be achievable with other observability solution that also integrate 
with OpenTelemetry such as Jaeger and Prometheus. 

Victor, Ivan and I will do our best to offer an unprecedented User 
Experience with Elastic Observability there will be no lock in, it will 
alway be possible to switch to other observability backends :-) 

Cyrille

On Friday, May 28, 2021 at 12:59:06 PM UTC+2 Oleg Nenashev wrote:

> Connecting ci.jenkins.io looks perfectly fine to me. There is no 
> sensitive data on this instance. For other Jenkins instances (infra, 
> release, trusted-ci, cert, ...) a sign-off from the Jenkins Security 
> Officer is required IMHO
>
> On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:
>
>> Sounds great
>>
>> On Fri, 28 May 2021 at 09:30, Damien Duportal  
>> wrote:
>>
>>> [Cross posted on jenkins-dev and jenkins-infra]
>>>
>>> Hello dear community,
>>>
>>> During the last Jenkins infrastructure weekly meeting (2021-05-25 - 
>>> Notes -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), 
>>> Victor Martinez and Iván Fernández from Elastic did a demonstration of the 
>>> opentelemetry plugin used with an Elastic APM platform.
>>> The goal was to demonstrate how the build metrics could be used with 
>>> distributed tracing: please check the notes, video and 
>>> https://plugins.jenkins.io/opentelemetry/ for more details.
>>>
>>>
>>> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the 
>>> Jenkins project by providing an APM instance that could be used along with 
>>> opentelemetry plugin, and a bit support to get it configured and maintained.
>>> Their interest is to study what are our (Jenkins community) usages (and 
>>> obviously mentioning that we are using it).
>>>
>>> The proposal from the Infrastructure team is to provide this service for 
>>> the 2 following use cases:
>>>
>>>- For users of ci.jenkins.io (core and plugins contributors) to get 
>>>build metrics and traces of their projects. It could help a lot for 
>>>performances tracking, and much more as "Jenkins Pipeline level metrics"
>>>- For infrastructure maintainers of ci.jenkins.io to track the 
>>>agents allocations and their errors, to improve the QoS of the build 
>>> farm, 
>>>as the pure infrastructure metrics is not enough because issues come 
>>> from 
>>>different areas (infra, Cloud providers, Jenkins configuration, Pipeline 
>>>usages, etc.)
>>>
>>>
>>> It could be a great opportunity for whoever interested in better 
>>> understanding Jenkins behavior. As we'll try to identify typical use cases 
>>> that we want to better understand and then build dashboards using metric.
>>> The challenge we face on the Jenkins infrastructure is that it's pretty 
>>> easy to know if Jenkins is running. But not so much if it's working in an 
>>> efficient way.
>>>
>>> The plan would be the following:
>>>
>>>- Start by a PoC on a private Jenkins instance  (e.g. 
>>>infra.ci.jenkins.io)
>>>   - Install the plugin
>>>   - Check APM instance access with Elastic
>>>   - Configure the plugin to point to the APM
>>>   - Bring back the conclusion to the community (by email here, blog 
>>>   post, meetup, whatever)
>>>- Once the configuration is straightforward, deploying to 
>>>ci.jenkins.io, targeting as much publicly available dashboards and 
>>>data as possible, as service for the ci.jenkins.io users.
>>>
>>> What do you think about this topic, what would be your thoughts or red 
>>> flags?
>>>
>>> Cheers,
>>>
>>> For the Infra. team,
>>> Damien DUPORTAL
>>>
>>> PS: by default, without any blocker or red flags, we (the infra.team) 
>>> will proceed with the infra.ci setup described in this email the 1st of 
>>> June.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Infrastructure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkins-infr...@googlegroups.com.
>>> To view this discussion on the web, visit 
>>> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Infrastructure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkins-infr...@googlegroups.com.
>>
> To view this discussion on the web, visit 
>> https://groups.google.com/d/msgid/jenkins-infra/CAH-3Bie3U4UOxmSLpex8sSw48FQeVGwJjXP4KLQTPWBAon%3DS1Q%40mail

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-28 Thread Oleg Nenashev 
Connecting ci.jenkins.io looks perfectly fine to me. There is no sensitive
data on this instance. For other Jenkins instances (infra, release,
trusted-ci, cert, ...) a sign-off from the Jenkins Security Officer is
required IMHO

On Fri, May 28, 2021, 10:40 Tim Jacomb  wrote:

> Sounds great
>
> On Fri, 28 May 2021 at 09:30, Damien Duportal 
> wrote:
>
>> [Cross posted on jenkins-dev and jenkins-infra]
>>
>> Hello dear community,
>>
>> During the last Jenkins infrastructure weekly meeting (2021-05-25 - Notes
>> -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), Victor
>> Martinez and Iván Fernández from Elastic did a demonstration of the
>> opentelemetry plugin used with an Elastic APM platform.
>> The goal was to demonstrate how the build metrics could be used with
>> distributed tracing: please check the notes, video and
>> https://plugins.jenkins.io/opentelemetry/ for more details.
>>
>>
>> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the
>> Jenkins project by providing an APM instance that could be used along with
>> opentelemetry plugin, and a bit support to get it configured and maintained.
>> Their interest is to study what are our (Jenkins community) usages (and
>> obviously mentioning that we are using it).
>>
>> The proposal from the Infrastructure team is to provide this service for
>> the 2 following use cases:
>>
>>- For users of ci.jenkins.io (core and plugins contributors) to get
>>build metrics and traces of their projects. It could help a lot for
>>performances tracking, and much more as "Jenkins Pipeline level metrics"
>>- For infrastructure maintainers of ci.jenkins.io to track the agents
>>allocations and their errors, to improve the QoS of the build farm, as the
>>pure infrastructure metrics is not enough because issues come from
>>different areas (infra, Cloud providers, Jenkins configuration, Pipeline
>>usages, etc.)
>>
>>
>> It could be a great opportunity for whoever interested in better
>> understanding Jenkins behavior. As we'll try to identify typical use cases
>> that we want to better understand and then build dashboards using metric.
>> The challenge we face on the Jenkins infrastructure is that it's pretty
>> easy to know if Jenkins is running. But not so much if it's working in an
>> efficient way.
>>
>> The plan would be the following:
>>
>>- Start by a PoC on a private Jenkins instance  (e.g.
>>infra.ci.jenkins.io)
>>   - Install the plugin
>>   - Check APM instance access with Elastic
>>   - Configure the plugin to point to the APM
>>   - Bring back the conclusion to the community (by email here, blog
>>   post, meetup, whatever)
>>- Once the configuration is straightforward, deploying to
>>ci.jenkins.io, targeting as much publicly available dashboards and
>>data as possible, as service for the ci.jenkins.io users.
>>
>> What do you think about this topic, what would be your thoughts or red
>> flags?
>>
>> Cheers,
>>
>> For the Infra. team,
>> Damien DUPORTAL
>>
>> PS: by default, without any blocker or red flags, we (the infra.team)
>> will proceed with the infra.ci setup described in this email the 1st of
>> June.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Infrastructure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkins-infra+unsubscr...@googlegroups.com.
>> To view this discussion on the web, visit
>> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkins-infra+unsubscr...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/jenkins-infra/CAH-3Bie3U4UOxmSLpex8sSw48FQeVGwJjXP4KLQTPWBAon%3DS1Q%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLBCAAJGP2p3oAudZztVQ2yT8sp5Zi9m%3DmMHC3REr%3DYNcA%40mail.gmail.com.

Re: [jenkins-infra] Elastic Sponsorship for OpenTelemetry usage for community Jenkins instances

2021-05-28 Thread Tim Jacomb 
Sounds great

On Fri, 28 May 2021 at 09:30, Damien Duportal 
wrote:

> [Cross posted on jenkins-dev and jenkins-infra]
>
> Hello dear community,
>
> During the last Jenkins infrastructure weekly meeting (2021-05-25 - Notes
> -with recording link: https://hackmd.io/sBywc0IfRraU93aNETVwkw), Victor
> Martinez and Iván Fernández from Elastic did a demonstration of the
> opentelemetry plugin used with an Elastic APM platform.
> The goal was to demonstrate how the build metrics could be used with
> distributed tracing: please check the notes, video and
> https://plugins.jenkins.io/opentelemetry/ for more details.
>
>
> Victor and Ivan's employer, Elastic Inc. is proposing to sponsor the
> Jenkins project by providing an APM instance that could be used along with
> opentelemetry plugin, and a bit support to get it configured and maintained.
> Their interest is to study what are our (Jenkins community) usages (and
> obviously mentioning that we are using it).
>
> The proposal from the Infrastructure team is to provide this service for
> the 2 following use cases:
>
>- For users of ci.jenkins.io (core and plugins contributors) to get
>build metrics and traces of their projects. It could help a lot for
>performances tracking, and much more as "Jenkins Pipeline level metrics"
>- For infrastructure maintainers of ci.jenkins.io to track the agents
>allocations and their errors, to improve the QoS of the build farm, as the
>pure infrastructure metrics is not enough because issues come from
>different areas (infra, Cloud providers, Jenkins configuration, Pipeline
>usages, etc.)
>
>
> It could be a great opportunity for whoever interested in better
> understanding Jenkins behavior. As we'll try to identify typical use cases
> that we want to better understand and then build dashboards using metric.
> The challenge we face on the Jenkins infrastructure is that it's pretty
> easy to know if Jenkins is running. But not so much if it's working in an
> efficient way.
>
> The plan would be the following:
>
>- Start by a PoC on a private Jenkins instance  (e.g.
>infra.ci.jenkins.io)
>   - Install the plugin
>   - Check APM instance access with Elastic
>   - Configure the plugin to point to the APM
>   - Bring back the conclusion to the community (by email here, blog
>   post, meetup, whatever)
>- Once the configuration is straightforward, deploying to ci.jenkins.io,
>targeting as much publicly available dashboards and data as possible, as
>service for the ci.jenkins.io users.
>
> What do you think about this topic, what would be your thoughts or red
> flags?
>
> Cheers,
>
> For the Infra. team,
> Damien DUPORTAL
>
> PS: by default, without any blocker or red flags, we (the infra.team) will
> proceed with the infra.ci setup described in this email the 1st of June.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Infrastructure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkins-infra+unsubscr...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/jenkins-infra/0A4790E7-DF29-4C36-AAF3-320763D230B9%40gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3Bie3U4UOxmSLpex8sSw48FQeVGwJjXP4KLQTPWBAon%3DS1Q%40mail.gmail.com.