RE: failover for maven repository?
On Wed, 2004-06-09 at 10:48, Julian C. Dunn wrote: > > > If you would like to expand on that and make a little doc with some > > references I will happily add it to the wiki material that exists there > > already. > > Sure. My document is attached as a text file. Cool, thanks. It's in the wiki now: http://docs.codehaus.org/display/MAVEN/Repository+-+Security+by+julian+c+dunn -- jvz. Jason van Zyl [EMAIL PROTECTED] http://maven.apache.org First, the taking in of scattered particulars under one Idea, so that everyone understands what is being talked about ... Second, the separation of the Idea into parts, by dividing it at the joints, as nature directs, not breaking any limb in half as a bad carver might. -- Plato, Phaedrus (Notes on the Synthesis of Form by C. Alexander) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Wed, 2004-06-09 at 02:10, Jason van Zyl wrote: > On Wed, 2004-06-09 at 00:11, Julian C. Dunn wrote: > > > Well, we all "hope" that nobody mucks up the repository, but that only > > gets you so far -- all you have to do is to ask the Debian or FSF > > maintainers whose sites got cracked how far "hope" gets you. > > Yes, I'm well aware. It's not like I'm floating along in la-la land. But > I don't have infinite time and I tackle what I can as does anyone else > contributing to maven. Understood. I just took exception to your remark that security is a "bogey man" when, in fact, there have been demonstrated instances of people (script kiddies and otherwise) breaking into repository servers and wreaking all manner of havoc. > > In my mind, the correct approach as suggested by Casey, Pryce et al. is to > > store the MD5 or SHA1 checksums offline, i.e. not in the same place the > > JARs themselves, and then to to transfer those securely. This is basically > > the approach used by the FreeBSD ports system or NetBSD pkgsrc. The actual > > transfer of the JARs need not be secure as long as the checksums are > > trustworthy. > > If you would like to expand on that and make a little doc with some > references I will happily add it to the wiki material that exists there > already. Sure. My document is attached as a text file. - Julian -- Julian C. Dunn, B.A.Sc. <[EMAIL PROTECTED]> Software Developer, CBC New Media Operations Office: 2C310-I * Tel.: (416) 205-3311 x5592 A number of key security issues have been identified in the way Maven resolves and retrieves dependencies. These issues have been identified in documents written by Nat Pryce and John Casey: http://docs.codehaus.org/display/MAVEN/Repository+-+Security http://docs.codehaus.org/display/MAVEN/Repository+-+Security+by+nat+pryce Casey proposes to tighten up the repository upload procedure, which is a good first step. However, signing all artifacts (and in particular, the ongoing workload of needing to distribute derivative certificates) may prove to be too onerous a procedure. Pryce proposes an reasonable solution from a security perspective: storing JAR signatures in the repository as well. Presumably users would then obtain the public keys of the signers through some non-Maven means. However, whether this would actually happen is a matter of discussion. My suspicion is that most people (aside from the truly paranoid) don't even do this for the code they download from the main Apache repository, e.g httpd. A solution that I propose is to adopt the model taken by both the FreeBSD and NetBSD projects for their software packaging. In the "ports" (FreeBSD) or "pkgsrc" (NetBSD) systems, the checksums and sizes of the source tarballs are stored in a separate location, i.e. in the ports/pkgsrc CVS repository itself. Access to CVS is tightly controlled, just as it would be for the actual code itself. The checksums and sizes are transferred to the users' machines by a secure mechanism, i.e. CVS over SSH or cvsup. Adapting the model to the Maven project would entail the following: 1. Store checksums and sizes of JARs/distributions at Apache.org. Apply the same control to the checksum repo as would be kept for actual CVS commits to the code. 2. When Maven requires an artifact, it securely (e.g. over HTTPS) retrieves the size/checksum information from Apache.org. 3. It then retrieves the artifact from ibiblio. If ibiblio has been compromised, the checksum/size will not match and Maven would refuse to continue. References: --- [1] Using FreeBSD Ports: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports-using.html [2] NetBSD Pkgsrc: http://www.netbsd.org/Documentation/pkgsrc/ [3] NetBSD Pkgsrc info about 'distinfo' where checksum/size info is stored: http://www.netbsd.org/Documentation/pkgsrc/components.html#components.distinfo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Wed, 2004-06-09 at 00:11, Julian C. Dunn wrote: > Well, we all "hope" that nobody mucks up the repository, but that only > gets you so far -- all you have to do is to ask the Debian or FSF > maintainers whose sites got cracked how far "hope" gets you. Yes, I'm well aware. It's not like I'm floating along in la-la land. But I don't have infinite time and I tackle what I can as does anyone else contributing to maven. > I would > rather that the Maven community take proactive steps to rectify this, > rather than getting egg on our collective faces when the repo does get > mangled, either by accident or on purpose. Rest assured, we have. Again things are tackled as deemed fit and many people like to comment but when it comes to stepping up to the plate to actually doing any work expectations fall much shorter than desired. > I'll give you an example of a case where even "accidental" repo mangling > has caused us grief: commons-configuration. The JAR that is up there on > ibiblio labelled 1.0-dev doesn't contain the same code as the current one > (also labelled 1.0-dev) which you can download off the Jakarta site. I had > a developer run across this just today: when he ran his code against what > he thought was the "correct" 1.0-dev JAR but was in fact the old one from > ibiblio, the code blew up, predictably. Yes, these things happen and is mostly due in part to some lax upload procedures and people using -dev extensions as -SNAPSHOT extension which they were warned about eons ago. This package comes from authors originally from turbine where the -dev preference exists. You were screwed by that practice not by maven inherently. We are trying to close up the gaps wrt to distribution but whenever there is a gap you can rest assured that someone will find it and you got bit in the ass. > In my mind, the correct approach as suggested by Casey, Pryce et al. is to > store the MD5 or SHA1 checksums offline, i.e. not in the same place the > JARs themselves, and then to to transfer those securely. This is basically > the approach used by the FreeBSD ports system or NetBSD pkgsrc. The actual > transfer of the JARs need not be secure as long as the checksums are > trustworthy. If you would like to expand on that and make a little doc with some references I will happily add it to the wiki material that exists there already. > - Julian -- jvz. Jason van Zyl [EMAIL PROTECTED] http://maven.apache.org happiness is like a butterfly: the more you chase it, the more it will elude you, but if you turn your attention to other things, it will come and sit softly on your shoulder ... -- Thoreau - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Tue, 8 Jun 2004, Jason van Zyl wrote: > > On Tue, 2004-06-08 at 22:59, Julian C. Dunn wrote: > > > > > I must admit that I share their concern; I'm curious to know whether the > > > security implications of this have been discussed at all. > > > > Many times, we have use cases, and the upload process will become more > > rigourous over time. We've also had a couple more complete proposals > > submitted: one by Nat Pryce and one by John Casey > > For reference: > > http://docs.codehaus.org/display/MAVEN/Repository+-+Security > > http://docs.codehaus.org/display/MAVEN/Repository+-+Security+by+nat+pryce Those articles pretty much reflect my (and my sysadmins') concerns, thank you. > Some may consider it negligence but I considered convenience to be the > overriding concern. I realize security is an issue, but I feel it's > become a bit a boogey man. Anything is possible and maybe there is some > really, really bored guy with nothing better to do then muck up the > works for everyone but I'm really hoping that doesn't happen. But in m2 > we will have options for the paranoid and the upload process will be > easier and more secure. Well, we all "hope" that nobody mucks up the repository, but that only gets you so far -- all you have to do is to ask the Debian or FSF maintainers whose sites got cracked how far "hope" gets you. I would rather that the Maven community take proactive steps to rectify this, rather than getting egg on our collective faces when the repo does get mangled, either by accident or on purpose. I'll give you an example of a case where even "accidental" repo mangling has caused us grief: commons-configuration. The JAR that is up there on ibiblio labelled 1.0-dev doesn't contain the same code as the current one (also labelled 1.0-dev) which you can download off the Jakarta site. I had a developer run across this just today: when he ran his code against what he thought was the "correct" 1.0-dev JAR but was in fact the old one from ibiblio, the code blew up, predictably. In my mind, the correct approach as suggested by Casey, Pryce et al. is to store the MD5 or SHA1 checksums offline, i.e. not in the same place the JARs themselves, and then to to transfer those securely. This is basically the approach used by the FreeBSD ports system or NetBSD pkgsrc. The actual transfer of the JARs need not be secure as long as the checksums are trustworthy. - Julian -- Julian C. Dunn <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> Software Developer, CBC.ca Production & Operations Office: 2C310-I * Tel.: (416)-205-5592 PGP Key: 0xDA6A5B30 [7DCD A0C3 8B6F 6A76 F4CD 9F9B F941 A1B2 DA6A 5B30] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Tue, 2004-06-08 at 23:10, Jason van Zyl wrote: > On Tue, 2004-06-08 at 22:59, Julian C. Dunn wrote: > > > I must admit that I share their concern; I'm curious to know whether the > > security implications of this have been discussed at all. > > Many times, we have use cases, and the upload process will become more > rigourous over time. We've also had a couple more complete proposals > submitted: one by Nat Pryce and one by John Casey For reference: http://docs.codehaus.org/display/MAVEN/Repository+-+Security http://docs.codehaus.org/display/MAVEN/Repository+-+Security+by+nat+pryce Wagon is being employed in m2 so things like using encrypted transfer will become possible. Some may consider it negligence but I considered convenience to be the overriding concern. I realize security is an issue, but I feel it's become a bit a boogey man. Anything is possible and maybe there is some really, really bored guy with nothing better to do then muck up the works for everyone but I'm really hoping that doesn't happen. But in m2 we will have options for the paranoid and the upload process will be easier and more secure. I am no security expert and I don't think we have one on the team here but pick away at the proposals, badger security folks you know to look things over. We are will to implement anything deemed reasonable and sane. -- jvz. Jason van Zyl [EMAIL PROTECTED] http://maven.apache.org happiness is like a butterfly: the more you chase it, the more it will elude you, but if you turn your attention to other things, it will come and sit softly on your shoulder ... -- Thoreau - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Tue, 2004-06-08 at 22:59, Julian C. Dunn wrote: > I must admit that I share their concern; I'm curious to know whether the > security implications of this have been discussed at all. Many times, we have use cases, and the upload process will become more rigourous over time. We've also had a couple more complete proposals submitted: one by Nat Pryce and one by John Casey -- jvz. Jason van Zyl [EMAIL PROTECTED] http://maven.apache.org happiness is like a butterfly: the more you chase it, the more it will elude you, but if you turn your attention to other things, it will come and sit softly on your shoulder ... -- Thoreau - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Tue, 8 Jun 2004, Jerome Lacoste wrote: > On Tue, 2004-06-08 at 10:11, Maczka Michal wrote: > > you can use: > > > > maven.repo.remote=http://repo1,http://repo2,file://repo3,http://www.ibiblio. > > org/maven > > Let me add that by precaution, you should probably always have a local repository > around containing all your projects dependencies. > I heard people say you shouldn't, but: > - you may lose your connection > - you may need to build from a place where you don't have a connection > - it's always good to not be dependent on somebody else when you have to build some > sotware. That's definitely my philosophy here; my sysadmins also add one more reason to that list, and it is - downloading unverified JARs from an Internet website and putting them blindly in your software is a bad idea I must admit that I share their concern; I'm curious to know whether the security implications of this have been discussed at all. - Julian -- Julian C. Dunn <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> Software Developer, CBC.ca Production & Operations Office: 2C310-I * Tel.: (416)-205-5592 PGP Key: 0xDA6A5B30 [7DCD A0C3 8B6F 6A76 F4CD 9F9B F941 A1B2 DA6A 5B30] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
On Tue, 2004-06-08 at 10:11, Maczka Michal wrote: > you can use: > > maven.repo.remote=http://repo1,http://repo2,file://repo3,http://www.ibiblio. > org/maven Let me add that by precaution, you should probably always have a local repository around containing all your projects dependencies. I heard people say you shouldn't, but: - you may lose your connection - you may need to build from a place where you don't have a connection - it's always good to not be dependent on somebody else when you have to build some sotware. I also believe it should work with ftp:// URLs, because the HttpUtils class used in maven 1.0 for downloading the artifacts works with them according to my tests. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
you can use: maven.repo.remote=http://repo1,http://repo2,file://repo3,http://www.ibiblio. org/maven and so on Michal > -Original Message- > From: Julian C. Dunn [mailto:[EMAIL PROTECTED] > Sent: Tuesday, June 08, 2004 3:51 PM > To: Maven Users List > Subject: failover for maven repository? > > > Hello, > > I'm wondering if there is any way to get Maven to fail over to one or > more repositories when ibiblio.org is unavailable, or does not contain > the JARs that I'm looking for. > > It seems like the "maven.repo.remote" knob is an > all-or-nothing deal; if > I want to pull certain JARs from my local repository, I have to mirror > *all* of the ibiblio JARs I want. > > Similarly, if I don't set "maven.repo.remote" to anything, I can't > override one particular JAR by pointing it to a local > repository; i.e. a > maven.jar.foo property can only refer to a local filesystem > path, not an > internal repository URL. > > Is this a limitation of Maven, or of my understanding of how to > configure it? > > - Julian > > P.S. Anyone know why xalan 2.6.0 is missing from ibiblio? > > -- > Julian C. Dunn, B.A.Sc. <[EMAIL PROTECTED]> > Software Developer, CBC New Media Operations > Office: 2C310-I * Tel.: (416) 205-3311 x5592 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: failover for maven repository?
Just put your repo before the ibiblio one and you'll pull from there instead, and only go to ibiblio for the onese that are missing.. Eric > -Original Message- > From: Julian C. Dunn [mailto:[EMAIL PROTECTED] > Sent: Tuesday, June 08, 2004 3:51 PM > To: Maven Users List > Subject: failover for maven repository? > > > Hello, > > I'm wondering if there is any way to get Maven to fail over to one or > more repositories when ibiblio.org is unavailable, or does not contain > the JARs that I'm looking for. > > It seems like the "maven.repo.remote" knob is an all-or-nothing deal; if > I want to pull certain JARs from my local repository, I have to mirror > *all* of the ibiblio JARs I want. > > Similarly, if I don't set "maven.repo.remote" to anything, I can't > override one particular JAR by pointing it to a local repository; i.e. a > maven.jar.foo property can only refer to a local filesystem path, not an > internal repository URL. > > Is this a limitation of Maven, or of my understanding of how to > configure it? > > - Julian > > P.S. Anyone know why xalan 2.6.0 is missing from ibiblio? > > -- > Julian C. Dunn, B.A.Sc. <[EMAIL PROTECTED]> > Software Developer, CBC New Media Operations > Office: 2C310-I * Tel.: (416) 205-3311 x5592 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
