Re: Securely obtain the Jenkins package and public key

[Marius Gedminas]

On Thu, Jan 16, 2014 at 09:33:34AM -0800, Kohsuke Kawaguchi wrote:
> On 01/16/2014 06:19 AM, Marius Gedminas wrote:
> >What about packaging?  The Debian packages available from upstream start
> >the web server on the public IP right from the post-inst script, before
> >you get a chance to set up any security.  This allows remote code
> >execution as user jenkins.
> >
> >(The Jenkins package in the Ubuntu archive is configured to start the
> >web server on localhost only, which sounds like a sane mitigation
> >strategy for me.)
> 
> Yeah, this is a good idea.

I filed https://issues.jenkins-ci.org/browse/JENKINS-21417 for this.

> Perhaps a slight variation of it is to keep listening to these ports
> like we do today but to show the message saying non-local access is
> blocked until you acknowledge what you are doing?
> 
> That way, it would work across platforms and it's less confusing to
> those who are less experienced in the system administration?
> 
> Something like that can be packaged as a plugin, which makes the
> integration process easy --- you can just go create code yourself
> and other people can try/use it, and we'd only have to bundle it in
> the core.

Hm.  Yes, a page explaining what to do would be more user-friendly.
E.g. it could tell the admin how to use SSH port-forwarding to connect
to the Jenkins and get full access to the configuration pages, or what
config files to edit to set a password.

(I'm not volunteering to implement this.  My familiarity with Java ended
with a brief university course in 2000.)

> >Should I file a bug?  (I have filed a few bugs about packaging issues
> >(one involving data loss), but haven't seen any response at all after
> >many months, which made me stop caring.  I have some small expertise
> >with Debian packaging, and I'm willing to post patches, if I think they
> >will not be silently ignored for months.)
> 
> OK, our apologies. Do you know which ones they are? We want to take a look.

https://issues.jenkins-ci.org/browse/JENKINS-18797 and
https://issues.jenkins-ci.org/browse/JENKINS-18798 were the most
annoying, with the 1st one clobbering my configuration (luckily, I had a
backup).

https://issues.jenkins-ci.org/browse/JENKINS-19329 is related to 18798.

Perhaps I picked the wrong component?  It was the only one that
mentioned Debian packaging.

Marius Gedminas
-- 
6 out of 7 dwarves are not Happy.


signature.asc
Description: Digital signature

Re: Securely obtain the Jenkins package and public key

[Kohsuke Kawaguchi]

On 01/16/2014 06:19 AM, Marius Gedminas wrote:

On Wed, Jan 15, 2014 at 06:57:07PM -0800, Kohsuke Kawaguchi wrote:

OK, that's embarassing.

Indeed we haven't figured out how to communicate security problems to
plugin developers. Sometimes it's not obvious who to talk to, and even when
it is obvious, we haven't configured JIRA to let us grant read access on
issue-by-issue basis.

Issues not getting a timely enough attention is unfortunate, but aside from
trying to add more people to the jenkinsci-cert group (which we are always
trying), I'm not sure how to resolve that.

Daniel, given the level of activity you commit in the core, I feel like you
could help us fixing those issues, in addition to finding them.


As an outsider, I'm glad to see you care about security.


What about packaging?  The Debian packages available from upstream start
the web server on the public IP right from the post-inst script, before
you get a chance to set up any security.  This allows remote code
execution as user jenkins.

(The Jenkins package in the Ubuntu archive is configured to start the
web server on localhost only, which sounds like a sane mitigation
strategy for me.)


Yeah, this is a good idea.

Perhaps a slight variation of it is to keep listening to these ports 
like we do today but to show the message saying non-local access is 
blocked until you acknowledge what you are doing?


That way, it would work across platforms and it's less confusing to 
those who are less experienced in the system administration?


Something like that can be packaged as a plugin, which makes the 
integration process easy --- you can just go create code yourself and 
other people can try/use it, and we'd only have to bundle it in the core.



Should I file a bug?  (I have filed a few bugs about packaging issues
(one involving data loss), but haven't seen any response at all after
many months, which made me stop caring.  I have some small expertise
with Debian packaging, and I'm willing to post patches, if I think they
will not be silently ignored for months.)


OK, our apologies. Do you know which ones they are? We want to take a look.

--
Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/
Try Jenkins Enterprise, our professional version of Jenkins

--
You received this message because you are subscribed to the Google Groups "Jenkins 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Marius Gedminas]

On Wed, Jan 15, 2014 at 06:57:07PM -0800, Kohsuke Kawaguchi wrote:
> OK, that's embarassing.
> 
> Indeed we haven't figured out how to communicate security problems to
> plugin developers. Sometimes it's not obvious who to talk to, and even when
> it is obvious, we haven't configured JIRA to let us grant read access on
> issue-by-issue basis.
> 
> Issues not getting a timely enough attention is unfortunate, but aside from
> trying to add more people to the jenkinsci-cert group (which we are always
> trying), I'm not sure how to resolve that.
> 
> Daniel, given the level of activity you commit in the core, I feel like you
> could help us fixing those issues, in addition to finding them.

As an outsider, I'm glad to see you care about security.


What about packaging?  The Debian packages available from upstream start
the web server on the public IP right from the post-inst script, before
you get a chance to set up any security.  This allows remote code
execution as user jenkins.

(The Jenkins package in the Ubuntu archive is configured to start the
web server on localhost only, which sounds like a sane mitigation
strategy for me.)

Should I file a bug?  (I have filed a few bugs about packaging issues
(one involving data loss), but haven't seen any response at all after
many months, which made me stop caring.  I have some small expertise
with Debian packaging, and I'm willing to post patches, if I think they
will not be silently ignored for months.)

Marius Gedminas
-- 
Open Source Software: There are days when I can't figure out whether I'm living
in a Socialist utopia or a Libertarian one.
-- Alex Future Bokov


signature.asc
Description: Digital signature

Re: Securely obtain the Jenkins package and public key

[Kohsuke Kawaguchi]

OK, that's embarassing.

Indeed we haven't figured out how to communicate security problems to
plugin developers. Sometimes it's not obvious who to talk to, and even when
it is obvious, we haven't configured JIRA to let us grant read access on
issue-by-issue basis.

Issues not getting a timely enough attention is unfortunate, but aside from
trying to add more people to the jenkinsci-cert group (which we are always
trying), I'm not sure how to resolve that.

Daniel, given the level of activity you commit in the core, I feel like you
could help us fixing those issues, in addition to finding them.



2014/1/10 Daniel Beck 

> On 10.01.2014, at 18:11, teilo  wrote:
>
> > Have you helped to improve this situation by actually reporting them via
> the proper channels?
>
> Yes. That's why I consider the resolution process to be broken. The
> "proper channels" don't work.
>
> The first security issue I reported was SECURITY-35 in email-ext
> (installed on 30% of all instances) which I re-filed publicly as
> JENKINS-15213 after getting no response for three months. The email-ext
> author informed me he didn't receive any information from those with access
> to the private issue tracker and quickly fixed the problem. Another five
> months later, a response to SECURITY-35 arrived, explaining that, because
> the process was broken, some issues were overlooked.
>
> Then there's the ongoing SECURITY-87: I reported that anyone can trivially
> DoS any Jenkins instance (including those where anonymous has no
> permissions) on 13 Aug 2013. AFAICT the problem persists. Sure, it's not
> privilege escalation, but still annoying if you're running a public
> instance.
>
> Another example of a security issue in current LTS is JENKINS-20800, which
> I originally reported to Cloudbees Enterprise support in a non-security
> context (so it was filed publicly). I only later found it to be trivially
> exploitable on any Jenkins instance by anyone. Four weeks ago, the fix was
> backported early to LTS, likely because I asked for it on the dev list. But
> 1.532.2 still doesn't even have an RC. Should I have reported it separately
> as a security issue? Maybe, but the developers were aware of this issue,
> and by then I'd mostly given up on "proper channels".
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Kohsuke Kawaguchi]

I've modified the server infrastructure so that the PGP public key is
available at https://jenkins-ci.org/jenkins-ci.org.key. These keys are used
to sign native packages. In the next release I'll update the those package
documentaion to refer to this key location in HTTPS.

Aside from that, regardless of the packaging, the war file is signed with
my X509 certificate. That certificate is itself signed by CoMoDo, so you
can establish a known trust chain back up to the well-trusted root CAs.




2014/1/8 Abhijith Chandrashekar 

> Hello all,
>
> I work with a tech company where we're trying to establish a pristine
> build environment for all of our products. As part of this, we are looking
> to create a Jenkins CI server from scratch using the most secure methods
> possible. This would be on an underlying CentOS 6.2 machine. From reading
> the guide on installing Jenkins on CentOS/RedHat I see that the package and
> the key are both obtained over http as -
>
> wget -O /etc/yum.repos.d/jenkins.repo
> http://pkg.jenkins-ci.org/redhat/jenkins.repo
>
> and
>
> rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
>
> This raises possibilities of a Man-in-the-middle attack compromising the
> integrity of the repo or the key or both. To avoid this, is there a way to
> obtain the package and the key securely? This could either be over HTTPS,
> SFTP or by exchanging PGP keys with the owner and then transporting it over
> email.
>
> If there's a better place to post this question, please inform.
>
> Thanks,
> Abhijith
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Kohsuke Kawaguchi

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Richard Mortimer]

Hi Abhijith,

I think you need to read about chains of trust. Everything that you 
suggest below is at best hiding what you are downloading from an 
observer. It doesn't stop man in the middle attacks or guarantee that 
the contents were not corrupted during transit.


As James suggested all you need to do is convince yourself that 
Kohsuke's public PGP key is actually his. Once you have done that you 
can verify that the signed release is actually what Kohsuke meant to 
release.


How paranoid you are is really up to you. You could just trust the key 
that is publicly posted by Kohsuke, attempt to verify the key via a web 
of trust, or even try to convince Kohsuke to meet you in person to 
verify the key.


Really I suspect that most of that is just overkill. You just need to 
ensure that you have a repeatable build environment where you have a 
reasonable level of trust that the software you downloaded is fit for 
purpose. Many Linux OS distributions (Debian for example) have all of 
the individual packages are linked back to a widely published central 
key allowing you to detect any tampering in transit.


The same applies to any commercial arrangements with appropriate 
software suppliers. You would need to get & trust a public key from them 
to allow you to verify that you receive exactly what was intended to be 
sent.


Regards

Richard


On 15/01/2014 22:27, abhijith chandrashekar wrote:

Thanks Tielo. Although I do think downloading sources and inspecting
them would not only be overkill, but also not a foolproof way of
ensuring security. What if the source files are mangled during download?

The only few ways I can think of are

1. to get the binaries and keys/hashes over PGP email from somebody on
the inside
2. obtain it over HTTPS on their website
3. using SFTP or
4. by having someone on the inside ship a DVD that contains the binary +
public key.

I'm trying other avenues (CloudBees, for example) but there do not seem
to be any provisions to do this.

Thanks,
Abhijith



On Mon, Jan 13, 2014 at 1:46 AM, teilo mailto:teilo+goo...@teilo.net>> wrote:


On Sunday, 12 January 2014 22:20:17 UTC, Abhijith Chandrashekar wrote:

 > Of course, you'd need a secure way to make sure it's actually
his signature, but that should be easier than changing the
entire distribution chain.

That's exactly the problem. Any ideas on how I can do that?

Thanks,
Abhijith


http://kohsuke.org/about/pgp/

But if you are that security paranoid then you should download the
sources, inspect them (and the history)  and then compile them
yourself every release (like you do for all the plugins right!?).

/James

--
You received this message because you are subscribed to a topic in
the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/jenkinsci-users/3O8vpxrWZH8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
jenkinsci-users+unsubscr...@googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.


--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


--
You received this message because you are subscribed to the Google Groups "Jenkins 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[abhijith chandrashekar]

Thanks Tielo. Although I do think downloading sources and inspecting them
would not only be overkill, but also not a foolproof way of ensuring
security. What if the source files are mangled during download?

The only few ways I can think of are

1. to get the binaries and keys/hashes over PGP email from somebody on the
inside
2. obtain it over HTTPS on their website
3. using SFTP or
4. by having someone on the inside ship a DVD that contains the binary +
public key.

I'm trying other avenues (CloudBees, for example) but there do not seem to
be any provisions to do this.

Thanks,
Abhijith



On Mon, Jan 13, 2014 at 1:46 AM, teilo  wrote:

>
> On Sunday, 12 January 2014 22:20:17 UTC, Abhijith Chandrashekar wrote:
>>
>> > Of course, you'd need a secure way to make sure it's actually his
>> signature, but that should be easier than changing the entire distribution
>> chain.
>>
>> That's exactly the problem. Any ideas on how I can do that?
>>
>> Thanks,
>> Abhijith
>>
>
> http://kohsuke.org/about/pgp/
>
> But if you are that security paranoid then you should download the
> sources, inspect them (and the history)  and then compile them yourself
> every release (like you do for all the plugins right!?).
>
> /James
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Jenkins Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/jenkinsci-users/3O8vpxrWZH8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> jenkinsci-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[teilo]

On Sunday, 12 January 2014 22:20:17 UTC, Abhijith Chandrashekar wrote:
>
> > Of course, you'd need a secure way to make sure it's actually his 
> signature, but that should be easier than changing the entire distribution 
> chain.
>
> That's exactly the problem. Any ideas on how I can do that?
>
> Thanks,
> Abhijith
>

http://kohsuke.org/about/pgp/

But if you are that security paranoid then you should download the sources, 
inspect them (and the history)  and then compile them yourself every 
release (like you do for all the plugins right!?).

/James 

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[abhijith chandrashekar]

> Of course, you'd need a secure way to make sure it's actually his
signature, but that should be easier than changing the entire distribution
chain.

That's exactly the problem. Any ideas on how I can do that?

Thanks,
Abhijith



On Sat, Jan 11, 2014 at 1:12 AM, Daniel Beck  wrote:

> On 08.01.2014, at 23:08, Abhijith Chandrashekar <
> abhijith.chandrashe...@gmail.com> wrote:
>
> > This raises possibilities of a Man-in-the-middle attack compromising the
> integrity of the repo or the key or both.
>
> The war packages themselves are signed by Kohsuke. You can use the tool
> 'jarsigner' to verify.
>
> Of course, you'd need a secure way to make sure it's actually his
> signature, but that should be easier than changing the entire distribution
> chain.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Jenkins Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/jenkinsci-users/3O8vpxrWZH8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> jenkinsci-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Daniel Beck]

On 08.01.2014, at 23:08, Abhijith Chandrashekar 
 wrote:

> This raises possibilities of a Man-in-the-middle attack compromising the 
> integrity of the repo or the key or both.

The war packages themselves are signed by Kohsuke. You can use the tool 
'jarsigner' to verify.

Of course, you'd need a secure way to make sure it's actually his signature, 
but that should be easier than changing the entire distribution chain.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Daniel Beck]

On 10.01.2014, at 18:11, teilo  wrote:

> Have you helped to improve this situation by actually reporting them via the 
> proper channels?

Yes. That's why I consider the resolution process to be broken. The "proper 
channels" don't work.

The first security issue I reported was SECURITY-35 in email-ext (installed on 
30% of all instances) which I re-filed publicly as JENKINS-15213 after getting 
no response for three months. The email-ext author informed me he didn't 
receive any information from those with access to the private issue tracker and 
quickly fixed the problem. Another five months later, a response to SECURITY-35 
arrived, explaining that, because the process was broken, some issues were 
overlooked.

Then there's the ongoing SECURITY-87: I reported that anyone can trivially DoS 
any Jenkins instance (including those where anonymous has no permissions) on 13 
Aug 2013. AFAICT the problem persists. Sure, it's not privilege escalation, but 
still annoying if you're running a public instance.

Another example of a security issue in current LTS is JENKINS-20800, which I 
originally reported to Cloudbees Enterprise support in a non-security context 
(so it was filed publicly). I only later found it to be trivially exploitable 
on any Jenkins instance by anyone. Four weeks ago, the fix was backported early 
to LTS, likely because I asked for it on the dev list. But 1.532.2 still 
doesn't even have an RC. Should I have reported it separately as a security 
issue? Maybe, but the developers were aware of this issue, and by then I'd 
mostly given up on "proper channels".


-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[teilo]

>
>
> > an interesting target for attacks 
>
> Jenkins security is a joke. You can find security issues without trying, 
> even in core. And the process to resolve them seems to be really broken. 
>
>
Have you helped to improve this situation by actually reporting them via 
the proper channels?

https://wiki.jenkins-ci.org/display/JENKINS/Security+Advisories

 /James 

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Daniel Beck]

On 10.01.2014, at 16:56, Johannes Wienke  
wrote:

> an interesting target for attacks

Jenkins security is a joke. You can find security issues without trying, even 
in core. And the process to resolve them seems to be really broken.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Securely obtain the Jenkins package and public key

[Johannes Wienke]

Hi,

I'd like to underline this issue. With the increasing use of Jenkins, it
might actually become an interesting target for attacks, as in some
environments the jenkins installation is tighly integrated into the
system infrastructure, e.g. generating binary packages for linux
distributions etc.

Cheers,
Johannes

On 01/08/2014 11:08 PM, Abhijith Chandrashekar wrote:
> I work with a tech company where we're trying to establish a pristine build 
> environment for all of our products. As part of this, we are looking to 
> create a Jenkins CI server from scratch using the most secure methods 
> possible. This would be on an underlying CentOS 6.2 machine. From reading 
> the guide on installing Jenkins on CentOS/RedHat I see that the package and 
> the key are both obtained over http as - 
> 
> wget -O /etc/yum.repos.d/jenkins.repo 
> http://pkg.jenkins-ci.org/redhat/jenkins.repo
> 
> and 
> 
> rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
> 
> This raises possibilities of a Man-in-the-middle attack compromising the 
> integrity of the repo or the key or both. To avoid this, is there a way to 
> obtain the package and the key securely? This could either be over HTTPS, 
> SFTP or by exchanging PGP keys with the owner and then transporting it over 
> email.
> 
> If there's a better place to post this question, please inform.
> 
> Thanks,
> Abhijith
> 




signature.asc
Description: OpenPGP digital signature

Securely obtain the Jenkins package and public key

[Abhijith Chandrashekar]

Hello all,

I work with a tech company where we're trying to establish a pristine build 
environment for all of our products. As part of this, we are looking to 
create a Jenkins CI server from scratch using the most secure methods 
possible. This would be on an underlying CentOS 6.2 machine. From reading 
the guide on installing Jenkins on CentOS/RedHat I see that the package and 
the key are both obtained over http as - 

wget -O /etc/yum.repos.d/jenkins.repo 
http://pkg.jenkins-ci.org/redhat/jenkins.repo

and 

rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key

This raises possibilities of a Man-in-the-middle attack compromising the 
integrity of the repo or the key or both. To avoid this, is there a way to 
obtain the package and the key securely? This could either be over HTTPS, 
SFTP or by exchanging PGP keys with the owner and then transporting it over 
email.

If there's a better place to post this question, please inform.

Thanks,
Abhijith

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.