[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu resolved as Fixed Merged as part of https://github.com/jenkinsci/logstash-plugin/pull/75 Jenkins / JENKINS-51029 Option to ignore SSL certificate errors Change By: Karthik Duddu Status: In Progress Resolved Resolution: Fixed Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu started work on JENKINS-51029 Change By: Karthik Duddu Status: Open In Progress Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50730) NoClassDefFound errors in Cloud Slaves
Title: Message Title Karthik Duddu commented on JENKINS-50730 Re: NoClassDefFound errors in Cloud Slaves Oleg Nenashev Jeff Thompson I'd be happy to help in resolving the issue, but I'm not very familiar with the remoting codebase, and I'm not too sure what I'm supposed to be looking out for - I've rebuilt Jenkins/remoting with extremely verbose logging for `JarLoaderImpl` and Checksum classes, but that hasn't really helped as I got inundated with information. Also, one of the biggest problems with this issue is that it isn't consistently reproducible (at least in our case), which makes it difficult to test out too many theories. Do you guys have any initial ideas or starting points that we can work off of? Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu commented on JENKINS-51029 Re: Option to ignore SSL certificate errors Markus Winter I think we might be using different terminology to address the same idea So when a client asks for a secure connection, the server responds with both its public key and its X.509 certificate chain - the client then responds with a randomly-generated shared-session key encrypted using the server's public key if it trusts that the server is who it claims to be (future communication occurs using the shared key). The points I'm trying to make are: at no point do eavesdroppers have access to actual communication contents; i.e. the connection after the handshake remains encrypted with the shared session key even if the X.509 chain cannot be validated by the client (the identity cannot be verified) Self-signed certificates, by definition, have the endpoint claiming to be itself on its own authority and not on the authority of a (trusted) CA. This means that the client has the server's public key, but has no way of knowing if the public key actually belongs to who the server claims to be. This allows MitM attacks by proxy or impersonation (https://stackoverflow.com/a/3865294/9366691 sketches out a scenario) As long as the endpoint validity cannot be verified, it doesn't matter if we ignore the cert (which essentially means trust the cert presented implicitly), or store it as a credential, since in either case eavesdropping is prevented even if MitM isn't. Also note that I'm not proposing that we ignore errors in all SSL connections that Jenkins might be making; by restricting ourselves to the plugin's connection to ES, we're effectively accepting whatever certificate the ES server (whose location is configured in the plugin by us anyway) presents without interfering with other connections. To me, this seems sufficient for testing purposes. To your other point, I agree that people might want to use certs that aren't CA validated in their ELK setups (in which case, they might be better off installing their company authority's cert in `/etc/ssl/certs` or some other system location, but I don't think it's our job/duty to police the way certs are used ). To be clear, I'm not opposed to the idea of configuring the plugin to accept cert credentials for the server, just that I don't see too much value in it. If Rusty Gerard, Jakub Bochenski and others feel that this is a good idea, then maybe this is the way to go. Add Comment
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu edited a comment on JENKINS-51029 Re: Option to ignore SSL certificate errors [~jbochenski] I believe it should be possible, but I don't think that's a great idea (at least in my context) - I think it's a greater security risk to add a _temporary_ cert to the JVM's keystore (which could potentially affect other Java services running on the box) as compared to modifying just the SSLContext of the connection to ES, especially since I don't administer the machines running our Jenkins instances. Also, in order to apply this fix, would people running containerized masters have to rebuild their application image?As [~mwinter69] suggests, we could possibly include an option to upload the server's cert as a Jenkins credential and put this in the SSLContext. Please correct me if I'm wrong, but I don't see the point of this - : if the certificate is self-signed, we might as well ignore it (for reasons below); - uploading a self-signed cert won't help if a malicious entity is proxying/impersonating the connection, as we still won't be able to verify it 's identity; if certificate isn't self-signed , well, this issue probably won't arise in the first place. I'm not sure if eavesdropping/MitM is an issue here - Also, ignoring verification of the endpoint's identity doesn't imply that communication will be unencrypted ([https://stackoverflow.com/a/3865055/9366691)|https://stackoverflow.com/a/3865055/9366691).] ; if a malicious entity is proxying the connection then uploading a self-signed cert won't help , as we still won so eavesdropping shouldn 't be able to verify the entity a problem anyway . Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu commented on JENKINS-51029 Re: Option to ignore SSL certificate errors Jakub Bochenski I believe it should be possible, but I don't think that's a great idea (at least in my context) - I think it's a greater security risk to add a temporary cert to the JVM's keystore (which could potentially affect other Java services running on the box) as compared to modifying just the SSLContext of the connection to ES, especially since I don't administer the machines running our Jenkins instances. Also, in order to apply this fix, would people running containerized masters have to rebuild their application image? As Markus Winter suggests, we could possibly include an option to upload the server's cert as a Jenkins credential and put this in the SSLContext. Please correct me if I'm wrong, but I don't see the point of this - if the certificate is self-signed, we might as well ignore it (for reasons below); if it isn't, well, this issue probably won't arise in the first place. I'm not sure if eavesdropping/MitM is an issue here - ignoring verification of the endpoint's identity doesn't imply that communication will be unencrypted (https://stackoverflow.com/a/3865055/9366691); if a malicious entity is proxying the connection then uploading a self-signed cert won't help, as we still won't be able to verify the entity. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. Fo
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu commented on JENKINS-51029 Re: Option to ignore SSL certificate errors If this is deemed appropriate for the plugin, I'd be happy to work on this issue Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-51029) Option to ignore SSL certificate errors
Title: Message Title Karthik Duddu created an issue Jenkins / JENKINS-51029 Option to ignore SSL certificate errors Issue Type: Improvement Assignee: Jakub Bochenski Components: logstash-plugin Created: 2018-04-26 18:37 Priority: Minor Reporter: Karthik Duddu We're currently testing this plugin for sending logs to a test Elasticsearch instance, which hasn't had a proxy certificate authenticated by our CA installed yet. Currently, the plugin fails to send any data at all as it's unable to verify the certificate returned by ES. It would be helpful (especially in test/dev scenarios) to be able to ignore SSL certificate errors while sending logs to ES. Add Comment
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu updated an issue Jenkins / JENKINS-50735 Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Change By: Karthik Duddu At my company, we use Jenkins to run builds with lots of parallel tasks using the Pipeline suite of plugins , where and the slaves for each task are provisioned from a private Kubernetes cluster. We have a very specific problem with these provisioned slaves: we'd like to reduce the time overhead of a Kubernetes slave to match that of a physical slave (or get as close as possible). Since our slave container itself has a non-trivial start-up time (after provisioning, but before registering with the Jenkins master), we're thinking of maintaining a Kubernetes deployment of 'ready' slaves that register themselves with the master, and then are removed from the deployment when they're assigned a job; the rest of the lifecycle remains the same (that is, the slaves are still used only once). This ensures that we have a continuous supply of ready slaves, and we can also use pool size auto-scaling to keep up with load.We've tried this out internally by modified the Kubernetes plugin a little to be able to support this system, and are reasonably satisfied with the results. I have a couple of questions with regard to this:1. Is there a better way to reduce overhead? In our case, overhead essentially comprises of provisioning request time + pod scheduling time + container setup + slave connect-back + pipeline setup time .2. Does this use-case fall within the realm of the Kubernetes plugin, or is it better off developed as a plugin dependent on this one? Add Comment
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu edited a comment on JENKINS-50735 Re: Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Perhaps I should've explained a little more: we're using the Pipeline plugin suite of Jenkins, and there's an inital start-up cost associated with shell step of the pipeline. The breakdown of time consumed came about to roughly the following:Breakdown of execution time taken by different steps of a simple `echo hello` shell step, when run in a pipeline with an un-modified version of the Kubernetes plugin:|Order of execution|Operation|Time (in secs)|| | | ||0|Job start request|1||1|Job start|3||2|Request for pod|10||3|k8s scheduling + startup|5||4|Slave registration and info exchange|5||5|Agent Connection|20||6|Script setup|51||7|Script execution|8||8|Job end|0| We've already set the over provisioning flags while starting up the instance. Using a deployment allows us to eliminate steps 2-6 (just after connection, we run a small script on the slave to complete the start-up time), and provides an overhead roughly similar to that of physical slaves. Also, as you mentioned, we don't want to reuse slaves, which kind of eliminates "Time in minutes to retain slave when idle" as an option. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu edited a comment on JENKINS-50735 Re: Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Perhaps I should've explained a little more: we're using the Pipeline plugin suite of Jenkins, and there's an inital start-up cost associated with shell step of the pipeline. The breakdown of time consumed came about to roughly the following:Breakdown of execution time taken by different steps of a simple `echo hello` shell step, when run in a pipeline with an un-modified version of the Kubernetes plugin: |Order of execution|Operation|Time (in secs)|| | | ||0|Job start request|1||1|Job start|3||2|Request for pod|10||3|k8s scheduling + startup|5||4|Slave registration and info exchange|5||5|Agent Connection|20||6|Script setup|51||7|Script execution|8||8|Job end|0| We've already set the over provisioning flags while starting up the instance. Using a deployment allows us to eliminate steps 2-6 (just after connection, we run a small script on the slave to complete the start-up time), and provides an overhead roughly similar to that of physical slaves. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu edited a comment on JENKINS-50735 Re: Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Perhaps I should've explained a little more: we're using the Pipeline plugin suite of Jenkins, and there's an inital start-up cost associated with shell step of the pipeline. The breakdown of time consumed came about to roughly the following:Breakdown of execution time taken by different steps of a simple `echo hello` shell step, when run in a pipeline with an un-modified version of the Kubernetes plugin: |Order of execution|Operation|Time (in secs)|| | | ||0|Job start request|1||1|Job start|3||2|Request for pod|10||3|k8s scheduling + startup|5||4|Slave registration and info exchange|5||5|Agent Connection|20||6|Script setup|51||7|Script execution|8||8|Job end|0| We've already set the over provisioning flags while starting up the instance. Using a deployment allows us to eliminate steps 2-6 (just after connection, we run a small script on the slave to complete the start-up time), and provides an overhead roughly similar to that of physical slaves. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu commented on JENKINS-50735 Re: Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Perhaps I should've explained a little more: we're using the Pipeline plugin suite of Jenkins, and there's an inital start-up cost associated with shell step of the pipeline. The breakdown of time consumed came about to roughly the following: Breakdown of execution time taken by different steps of a simple `echo hello` shell step, when run in a pipeline with an un-modified version of the Kubernetes plugin: Order of execution Operation Time (in secs) 0 Job start request 1 1 Job start 3 2 Request for pod 10 3 k8s scheduling + startup 5 4 Slave registration and info exchange 5 5 Agent Connection
[JIRA] (JENKINS-50735) Using pods in a Kubernetes deployment as slaves via Kubernetes plugin
Title: Message Title Karthik Duddu created an issue Jenkins / JENKINS-50735 Using pods in a Kubernetes deployment as slaves via Kubernetes plugin Issue Type: New Feature Assignee: Carlos Sanchez Components: kubernetes-plugin Created: 2018-04-11 11:42 Environment: Jenkins v2.89.2 Kubernetes plugin v1.3.3 Priority: Major Reporter: Karthik Duddu At my company, we use Jenkins to run builds with lots of parallel tasks, where the slaves for each task are provisioned from a private Kubernetes cluster. We have a very specific problem with these provisioned slaves: we'd like to reduce the time overhead of a Kubernetes slave to match that of a physical slave (or get as close as possible). Since our slave container itself has a non-trivial start-up time (after provisioning, but before registering with the Jenkins master), we're thinking of maintaining a Kubernetes deployment of 'ready' slaves that register themselves with the master, and then are removed from the deployment when they're assigned a job; the rest of the lifecycle remains the same (that is, the slaves are still used only once). This ensures that we have a continuous supply of ready slaves, and we can also use pool size auto-scaling to keep up with load. We've tried this out internally by modified the Kubernetes plugin a little to be able to support this system, and are reasonably satisfied with the results. I have a couple of questions with regard to this: 1. Is there a better way to reduce overhead? In our case, overhead essentially comprises of provisioning request time + pod scheduling time + container setup + slave connect-back. 2. Does this use-case fall within the realm of the Kubernetes plugin, or is it better off developed as a plugin dependent on this one?
[JIRA] (JENKINS-50730) NoClassDefFound errors in Cloud Slaves
Title: Message Title Karthik Duddu assigned an issue to Oleg Nenashev Jenkins / JENKINS-50730 NoClassDefFound errors in Cloud Slaves Change By: Karthik Duddu Assignee: Carlos Sanchez Oleg Nenashev Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50730) NoClassDefFound errors in Cloud Slaves
Title: Message Title Karthik Duddu created an issue Jenkins / JENKINS-50730 NoClassDefFound errors in Cloud Slaves Issue Type: Bug Assignee: Carlos Sanchez Components: kubernetes-plugin, remoting Created: 2018-04-11 08:56 Environment: Jenkins v2.89.2 Kubernetes Plugin v1.3.3 Priority: Major Reporter: Karthik Duddu While provisioning slaves from a private Kubernetes instance, we've found that a lot of slaves terminate with the following (or similar) stack trace on the slave's side: INFO: Setting up slave: kube1-medium-r9zf4 Apr 10, 2018 11:02:05 AM hudson.remoting.jnlp.Main$CuiListener INFO: Jenkins agent is running in headless mode. Apr 10, 2018 11:02:05 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /home//workDir/remoting as a remoting work directory Apr 10, 2018 11:02:06 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server ... Apr 10, 2018 11:02:06 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve INFO: Remoting server accepts the following protocols: [JNLP4-connect, CLI2-connect, JNLP-connect, Ping, CLI-connect, JNLP2-connect] Apr 10, 2018 11:02:06 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Agent discovery successful <...> pr 10, 2018 11:02:06 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Apr 10, 2018 11:02:06 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to Apr 10, 2018 11:02:06 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Trying protocol: JNLP4-connect Apr 10, 2018 11:02:07 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Remote identity confirmed:
