Re: /usr/share/java as maven repository?
2007/12/29, Michael Koch [EMAIL PROTECTED]: On Sat, Dec 29, 2007 at 11:35:54PM +0100, Marcus Better wrote: Dalibor Topic wrote: I'm not sure if our ideas are really opposed: Certainly not, I'm all for doing rebuilds of reverse dependencies if possible. So do you have any plans on how to implement this? Should the rebuilding be done before uploading a new version, or will that take too long? Or should we just have an ongoing archive rebuild as a QA effort? I would vote for doing this before the upload to the archive. I have a buildd here configured to be able to rebuild all reverse dependencies of a package on request. I'm happy to to do this for everyone who asks. Unfortunately I cant put that machine available for everyone. Its a 4 way machine with a daily updated sid-mirror on disk. It can build amd64 and i386 currently. I work on being able to build for other archs too. This is excellent! ;-) Happy New Year! :-D -- Arnaud Vandyck ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Dalibor Topic wrote: I'm not sure if our ideas are really opposed: Certainly not, I'm all for doing rebuilds of reverse dependencies if possible. So do you have any plans on how to implement this? Should the rebuilding be done before uploading a new version, or will that take too long? Or should we just have an ongoing archive rebuild as a QA effort? You seem to be arguing for packagers to be careful about documented ABI/API changes in the code and to check for them manually, if necessary. Yes, but first we need a way to specify dependencies in terms of ABI like what is done with C libraries. Cheers, Marcus ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
On Sat, Dec 29, 2007 at 11:35:54PM +0100, Marcus Better wrote: Dalibor Topic wrote: I'm not sure if our ideas are really opposed: Certainly not, I'm all for doing rebuilds of reverse dependencies if possible. So do you have any plans on how to implement this? Should the rebuilding be done before uploading a new version, or will that take too long? Or should we just have an ongoing archive rebuild as a QA effort? I would vote for doing this before the upload to the archive. I have a buildd here configured to be able to rebuild all reverse dependencies of a package on request. I'm happy to to do this for everyone who asks. Unfortunately I cant put that machine available for everyone. Its a 4 way machine with a daily updated sid-mirror on disk. It can build amd64 and i386 currently. I work on being able to build for other archs too. Cheers, Michael ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Marcus Better wrote: Most of the Debian library packaging guide [1] applies to Java libraries too. It seems to me that this problem has been ignored for Java libs just because Java doesn't automatically force us to deal with it, like SONAMEs do to some extent. I'm not sure if our ideas are really opposed: I'm talking about detecting ABI problems automatically within the distribution through rebuilding the transitive chain of reverse dependencies of an updated package (i.e. detecting FTBFS problems caused by the update) and running their tests to potentially detect breaking API changes automatically. You seem to be arguing for packagers to be careful about documented ABI/API changes in the code and to check for them manually, if necessary. I believe those approaches are complementary. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
On Sat, December 22, 2007 21:03, Marcus Better wrote: Paul Cager wrote: E.g. the POM could require JUnit-3.8.1 but Debian has packaged version 3.8.2. In this case we would set up a link in the temporary Maven repository: debian/.m2/repository/.../junit-3.8.1.jar - /usr/share/java/junit.jar - /usr/share/java/junit-3.8.2.jar We should check that the Debian version is not *older* than the version required by the POM, but we are still relying on releases being backwards compatible. Any comments? This is a recipe for disaster since the dependencies will likely break ABI at some point. Then our package will FTBFS if we are lucky, or build and run with the wrong version and give very interesting bugs later on. (On the other hand it's similar to what we already do...) Yes, I would generally agree with that. [...] In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. Now there I'm afraid I have to disagree. Newer library versions that are not backwards-compatible will affect packages built using ant in just the same way as packages built using Maven. I'm not sure what advantage we would gain by precluding the use of Maven in package builds - or have I misunderstood you? Thanks, Paul ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Marcus Better wrote: Dalibor Topic wrote: It seems to me that we must keep track of ABI compatibility. This is a bit complicated, but it's our job if we want to be a well-integrated distribution. In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. There are two sides to that: ABI compatibility tells us if class A will continue to link to class B. I had in mind a stricter definition that takes semantics into account. For example if a method breaks its contract by changing its behaviour then it may be a new ABI, even if the method signature is the same. Is it possible to derive such information automatically? If it isn't, I don't see how that would be feasible without reading and comparing different versions of a package with each other, which, at least for something like Eclipse, is a tall order on any packager due to the sheer size of the source code. I think the stricter criterium (builds runs tests successfully) is sufficient, and does not require maintaining ABI information. Then we cannot prevent a library from being upgraded to an incompatible version, as in the Tomcat breakage that was reported to the list just the other day. That should prove that we do need ABI information. In theory, uploading the package that broke Tomcat could have triggered a series of rebuilds of packages depending on it, along with their test suites, and chances are it would have been caught that way. I have to admit that I must have missed an explanation of what was wrong with tomcat 5.5 and commons-io in detail other than it required an older version, so it's not clear to me whether that is a binary compatibility requirement mismatch that could have been caught by a tomcat5.5 rebuild, or something else. Also there are packages that don't have test suites, or that don't run them at build time. Assuming one rebuilds the world sitting on top of the uploaded package with the uploaded package installed as a 'QA' measure, there is a very good chance to catch ABI breakage that way (i.e. binary compatibility issues preventing rebuilds of packages depending on the updated package, and resulting in fun linkage exceptions at runtime). Incompatible semantic changes could be caught by the package's own test suite, or by one of the test suites of packages depending on it. If they aren't, well, that's what bugs.debian.org is for. ;) It'd be great if more packages ran their test suites as part of the build process, it would give us more confidence that we we're shipping works sufficiently well for the configuration it is built for. I don't think we can catch all possible issues that way (i.e. rebuild reverse-deps with updated package before upload and run their test suites), but it could both make having explicit ABI information unnecessary (as build errors would be triggered by the rebuild), and reduce 'version drift' issues by catching them as soon as they are about to be introduced. How we deal with version drift issues, by patching a package using the drifting one if it's a one line change, or introducing different versions of it into the archives, is something that I assume the packagers decide on a case by case basis. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Paul Cager wrote: In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. Now there I'm afraid I have to disagree. Newer library versions that are not backwards-compatible will affect packages built using ant in just the same way as packages built using Maven. You are right, but I'm afraid that we will build some Maven infrastructure that will lock us into this bad situation, or need to be replaced later. It will perhaps be fine if the Maven builds use versions explicitly specified by the maintainer, just like Ant builds, but any automatic version guessing that doesn't take ABI into account would be dangerous. Cheers, Marcus ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Dalibor Topic wrote: I had in mind a stricter definition that takes semantics into account. For example if a method breaks its contract by changing its behaviour then it may be a new ABI, even if the method signature is the same. Is it possible to derive such information automatically? No, that's what makes library packaging so tricky. If it isn't, I don't see how that would be feasible without reading and comparing different versions of a package with each other, I've occasionally done precisely that for C libraries. However responsible upstream developers would document library API/ABI changes in the changelog. If they don't then one must check the code for changes, or manage the risk of introducing bugs. Most of the Debian library packaging guide [1] applies to Java libraries too. It seems to me that this problem has been ignored for Java libs just because Java doesn't automatically force us to deal with it, like SONAMEs do to some extent. Cheers, Marcus [1] http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Marcus Better wrote: Paul Cager wrote: E.g. the POM could require JUnit-3.8.1 but Debian has packaged version 3.8.2. In this case we would set up a link in the temporary Maven repository: debian/.m2/repository/.../junit-3.8.1.jar - /usr/share/java/junit.jar - /usr/share/java/junit-3.8.2.jar We should check that the Debian version is not *older* than the version required by the POM, but we are still relying on releases being backwards compatible. Any comments? This is a recipe for disaster since the dependencies will likely break ABI at some point. Then our package will FTBFS if we are lucky, or build and run with the wrong version and give very interesting bugs later on. (On the other hand it's similar to what we already do...) It seems to me that we must keep track of ABI compatibility. This is a bit complicated, but it's our job if we want to be a well-integrated distribution. In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. There are two sides to that: ABI compatibility tells us if class A will continue to link to class B. It does not tell us anything about classes using reflection, for example. So it's not a substitute for rebuilding and rerunning the test suite, just a quick way of confirming that a combination of classes would cause trouble. I think the stricter criterium (builds runs tests successfully) is sufficient, and does not require maintaining ABI information. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Dalibor Topic wrote: It seems to me that we must keep track of ABI compatibility. This is a bit complicated, but it's our job if we want to be a well-integrated distribution. In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. There are two sides to that: ABI compatibility tells us if class A will continue to link to class B. I had in mind a stricter definition that takes semantics into account. For example if a method breaks its contract by changing its behaviour then it may be a new ABI, even if the method signature is the same. I think the stricter criterium (builds runs tests successfully) is sufficient, and does not require maintaining ABI information. Then we cannot prevent a library from being upgraded to an incompatible version, as in the Tomcat breakage that was reported to the list just the other day. That should prove that we do need ABI information. Also there are packages that don't have test suites, or that don't run them at build time. Cheers, Marcus ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
[EMAIL PROTECTED] wrote: I totally agree with all your points. It would however be great to have this as a goal in mind and work towards it in parallel as you suggested. We are thinking along the same lines. Great. Do you want to take a shot at making Wagon grok Debian packages / metadata? cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
On Saturday December 22 2007, Dalibor Topic wrote: [EMAIL PROTECTED] wrote: I totally agree with all your points. It would however be great to have this as a goal in mind and work towards it in parallel as you suggested. We are thinking along the same lines. Great. Do you want to take a shot at making Wagon grok Debian packages / metadata? I would love to be able to do that. However with 2 babies in the house I can barely find the time to read and post on the mailinglists, provide ideas at least and run VIJUG. At this stage in my life I am beyond maxed out.. manfred ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Manfred Moser wrote: On Saturday December 22 2007, Dalibor Topic wrote: [EMAIL PROTECTED] wrote: I totally agree with all your points. It would however be great to have this as a goal in mind and work towards it in parallel as you suggested. We are thinking along the same lines. Great. Do you want to take a shot at making Wagon grok Debian packages / metadata? I would love to be able to do that. However with 2 babies in the house I can barely find the time to read and post on the mailinglists, provide ideas at least and run VIJUG. At this stage in my life I am beyond maxed out.. No worries, I just figured that someone 'coming out' as a Maven user might be a good person to ask. ;) cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Paul Cager wrote: E.g. the POM could require JUnit-3.8.1 but Debian has packaged version 3.8.2. In this case we would set up a link in the temporary Maven repository: debian/.m2/repository/.../junit-3.8.1.jar - /usr/share/java/junit.jar - /usr/share/java/junit-3.8.2.jar We should check that the Debian version is not *older* than the version required by the POM, but we are still relying on releases being backwards compatible. Any comments? This is a recipe for disaster since the dependencies will likely break ABI at some point. Then our package will FTBFS if we are lucky, or build and run with the wrong version and give very interesting bugs later on. (On the other hand it's similar to what we already do...) It seems to me that we must keep track of ABI compatibility. This is a bit complicated, but it's our job if we want to be a well-integrated distribution. In other words we should try to solve the library versioning problem first, and only then try to build packages with Maven. Cheers, Marcus ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
On Wed, December 19, 2007 17:52, Dalibor Topic wrote: Arnaud Vandyck wrote: Many thanks for the clarifications. So if we are talking about refactoring /usr/share/java, let's do it in some sort of Debian way. We'll write a wrapper around different build system to let them understand our layout. (is it what you meant?) I hope that wrapping the build systems themselves won't be necessary. It may be easier to 'setUp' and 'tearDown' virtual repositories based on our own binaries in /usr/share/java and automatically generated metadata from debian's metadata about the packages using some kind of a debhelper_maven script or something like that. I'd imagine something like this: 1. for the package build, you'd call debhelper_maven 2. (setUp) it would look at the dependency information of the package, and for each dependency generate a POM file on the fly from the debian package metadata, 3. (build) then call maven with the freshly generated pom repository to do its build work, 4. (tearDown) then tear down the POM repository, i.e. remove POMs, unregister them from Maven, etc. That sounds good. There's still a few things bothering me - anyone got any ideas on the following? Offline Mode + Repository = When building a Debian package I expect we'll be running Maven with the offline flag, so I believe we'll have to set up the temporary repository in the local cache, e.g. debian/.m2/repository (and provide a localRepository setting). Or is there a better way? Versions The POMs will refer to an explicit version of a dependency, e.g. junit-3.8.1, and its unlikely we'd have exactly the right version in /usr/share/java (the Maven build of jetty, for instance, downloaded two different versions of junit). I guess we will have to fake it and pretend our jar is the correct version. E.g. the POM could require JUnit-3.8.1 but Debian has packaged version 3.8.2. In this case we would set up a link in the temporary Maven repository: debian/.m2/repository/.../junit-3.8.1.jar - /usr/share/java/junit.jar - /usr/share/java/junit-3.8.2.jar We should check that the Debian version is not *older* than the version required by the POM, but we are still relying on releases being backwards compatible. Any comments? Mapping Maven Artifacts to Our Jar Names Any ideas how we'd generate the list of jars to copy into our temporary repository? I think most of the time a simple naming convention is good enough (e.g. artifact junit maps onto /usr/share/java/junit.jar). But I think we also need to allow the packager to override the default - for example if Debian has packaged two ABI-incompatible versions of a jar, or used non-standard jar names. I guess that *indirect* dependencies wouldn't be set up in the temporary maven repository - we'd just add them to the classpath as we do for ant builds. Would this work? I can't see an alternative here, as we wouldn't have the dependencies' POMs. The No-Dependencies Alternative = Alternatively we could add *all* required jars to the classpath (same as we do now for ant builds), and remove all dependencies from the POM(s). I think that would work, but haven't tested it. Instead of modifying the POM(s) we could write a Maven plugin so that Maven ignores any dependencies - I'm not sure how easy that would be. Then I believe we would only need to put the plugins (e.g. maven-compiler-plugin) in the temporary repository. Replace maven POM above with a different system, but the steps would be virtually the same. Pros: * builds are reproducible consistent within the Debian system * Debian's metadata is used throughout the build system, i.e. no metadata skew * could be separated from user's own build system repository, so no 'your jars are messing with my jars' conflicts Cons: * someone would need to implement it and see if the pros actually hold. :) cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Hi! I have been following this thread with much interest since I am using Maven commercially on a daily basis and have been doing so for a while. Here are some ideas regarding Pauls questions.. Quoting Paul Cager [EMAIL PROTECTED]: That sounds good. There's still a few things bothering me - anyone got any ideas on the following? Offline Mode + Repository = When building a Debian package I expect we'll be running Maven with the offline flag, so I believe we'll have to set up the temporary repository in the local cache, e.g. debian/.m2/repository (and provide a localRepository setting). Or is there a better way? That sounds about right. I would suggest to make the repository more of a permanent thing in some sort of maven-build-essentials package or whatever you want to call it. With transitive dependencies the temporary haul up and tear down process will be a significant overhead. Versions The POMs will refer to an explicit version of a dependency, e.g. junit-3.8.1, and its unlikely we'd have exactly the right version in /usr/share/java (the Maven build of jetty, for instance, downloaded two different versions of junit). I guess we will have to fake it and pretend our jar is the correct version. E.g. the POM could require JUnit-3.8.1 but Debian has packaged version 3.8.2. In this case we would set up a link in the temporary Maven repository: debian/.m2/repository/.../junit-3.8.1.jar - /usr/share/java/junit.jar - /usr/share/java/junit-3.8.2.jar We should check that the Debian version is not *older* than the version required by the POM, but we are still relying on releases being backwards compatible. Any comments? I think we will have to provide links for all versions that supported packages contain. These links will have to include not only the jar files, but more importantly the pom files. Ideally it will also include the packaged src files. These can be very helpful when actually coding with a maven project in an IDE allowing debugging and source code reading in the whole chain including your dependencies. Mapping Maven Artifacts to Our Jar Names Any ideas how we'd generate the list of jars to copy into our temporary repository? I think most of the time a simple naming convention is good enough (e.g. artifact junit maps onto /usr/share/java/junit.jar). But I think we also need to allow the packager to override the default - for example if Debian has packaged two ABI-incompatible versions of a jar, or used non-standard jar names. To sort that out it would probably better to have a more permanent maven repository as a meta package that is a build dependency for every package that uses maven for its build. It would contain multiple versions of jars and so on. I guess that *indirect* dependencies wouldn't be set up in the temporary maven repository - we'd just add them to the classpath as we do for ant builds. Would this work? I can't see an alternative here, as we wouldn't have the dependencies' POMs. You could do that. Maven allows you to create a list of all dependencies including transitive ones, but you need to use the dependencies and have their poms available. The No-Dependencies Alternative = Alternatively we could add *all* required jars to the classpath (same as we do now for ant builds), and remove all dependencies from the POM(s). I think that would work, but haven't tested it. Instead of modifying the POM(s) we could write a Maven plugin so that Maven ignores any dependencies - I'm not sure how easy that would be. Then I believe we would only need to put the plugins (e.g. maven-compiler-plugin) in the temporary repository. That sounds like a very bad idea. Maven has all the power to work through the transitive dependencies and conflicts of version allowing the developer to override and exclude stuff in the pom. Removing that power and writing a new system that does the same again sounds like a waste of time to me. The same point as above also applies to Dalibors suggestion bwlow. Replace maven POM above with a different system, but the steps would be virtually the same. Pros: * builds are reproducible consistent within the Debian system * Debian's metadata is used throughout the build system, i.e. no metadata skew * could be separated from user's own build system repository, so no 'your jars are messing with my jars' conflicts Cons: * someone would need to implement it and see if the pros actually hold. :) I think using Maven with a Debian internal and cleaned up repository that only contains debian compliant packages and the fixes to make the build of any debian package work is the better approach. It is also more useful for the bigger community since this debian repo could be used as a clean minimum repo. Many other public repos are quite a mess with lots of
Re: /usr/share/java as maven repository?
[EMAIL PROTECTED] wrote: I think using Maven with a Debian internal and cleaned up repository that only contains debian compliant packages and the fixes to make the build of any debian package work is the better approach. It is also more useful for the bigger community since this debian repo could be used as a clean minimum repo. Many other public repos are quite a mess with lots of historic rubbish in there (like broken spring packages, wrong and moved groupids and so on). That's a good goal to strive for in the long run, but currently, I don't think that it's possible to meet it. One could, in parallel, work on using the setUp tools to generate the POMs for all kinds of libraries packaged by Debian from Debian's metadata. That could also require dealing with Wagon to teach it to read deb files, for example. But I'd see the two activities as separate, for now, as the immediate problem of 'how do we build packages inside Debian within Debian using maven 2' is of a lot smaller scope than 'how do we turn the Debian achives into a repo useful for general maven2 users'. I believe solving the latter will require a lot of communication effort, for example, between Debian Java packagers, and Maven archive maintainers, as it can become a 'power' struggle over whose repo is the 'default' one for Debian users, etc. And that kind of potential political problem is best solved by waiting it out until we've got some experience solving the more immediate ones. ;) cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Quoting Dalibor Topic [EMAIL PROTECTED]: [EMAIL PROTECTED] wrote: I think using Maven with a Debian internal and cleaned up repository that only contains debian compliant packages and the fixes to make the build of any debian package work is the better approach. It is also more useful for the bigger community since this debian repo could be used as a clean minimum repo. Many other public repos are quite a mess with lots of historic rubbish in there (like broken spring packages, wrong and moved groupids and so on). That's a good goal to strive for in the long run, but currently, I don't think that it's possible to meet it. One could, in parallel, work on using the setUp tools to generate the POMs for all kinds of libraries packaged by Debian from Debian's metadata. That could also require dealing with Wagon to teach it to read deb files, for example. But I'd see the two activities as separate, for now, as the immediate problem of 'how do we build packages inside Debian within Debian using maven 2' is of a lot smaller scope than 'how do we turn the Debian achives into a repo useful for general maven2 users'. I believe solving the latter will require a lot of communication effort, for example, between Debian Java packagers, and Maven archive maintainers, as it can become a 'power' struggle over whose repo is the 'default' one for Debian users, etc. And that kind of potential political problem is best solved by waiting it out until we've got some experience solving the more immediate ones. ;) I totally agree with all your points. It would however be great to have this as a goal in mind and work towards it in parallel as you suggested. We are thinking along the same lines. manfred ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
2007/12/18, Marcus Better [EMAIL PROTECTED]: Arnaud Vandyck wrote: if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar If it's meant to be run in postinst then Maven might not be installed yet. Yes, I was thinking about putting the code in postinst... But I wonder how we are going to handle versioned dependencies. Maven projects tend to specify exact version numbers, right? That could be a real headache. I don't think we can ignore the version number, but we cannot use it as a hard dependency either. There is a -SNAPSHOT special versionning that could be used. But My first thought was to install the jar so: duplicate the jar with the exact version number: If you install liblog4j1.2-java 1.2.13-5, then, you'll have: /usr/share/java/log4j-1.2.jar - log4j-1.2.13 /usr/share/java/log4j-1.2.13.jar + /var/lib/maven/repository/log4j/log4j/1.2.13/log4j-1.2.13.jar (+sha, pom, etc) if you install liblog4j1.2-java 1.2.15-2, no more 1.2.13 in /usr/share/java but: /usr/share/java/log4j-1.2.jar - log4j-1.2.15 /usr/share/java/log4j-1.2.15.jar + /var/lib/maven/repository/log4j/log4j/1.2.13/log4j-1.2.13.jar (+sha, pom, etc) /var/lib/maven/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar (+sha, pom, etc) The idea is to provide a local debian repo for maven. But maybe it's not the best idea! Other comments will be in other responses. -- Arnaud Vandyck ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
2007/12/18, Paul Cager [EMAIL PROTECTED]: Arnaud Vandyck wrote: if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar The install:install-file plugin does almost what Arnaud suggests: http://maven.apache.org/guides/mini/guide-installing-3rd-party-jars.html I know this plugin and that's why I proposed that solution ;-) I think if there is a correct settings.xml, we could do that. -- Arnaud Vandyck ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
2007/12/19, Dalibor Topic [EMAIL PROTECTED]: Paul Cager wrote: The install:install-file plugin does almost what Arnaud suggests: [...] But I wonder just how important it is to have a system-wide local repo? It would certainly reduce the number of Jars an end-user needed to download (clearly a good thing), but it seems like a lot of work (for us). I assume it's going to be a necessity for us for reproducible offline maven builds to inform maven about our own jars. That's the idea. -- Arnaud Vandyck ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Am Mittwoch, 19. Dezember 2007 11:01:11 schrieb Arnaud Vandyck: 1° To build software in Debian: we need to tell maven how and where to find jars (and aliases: I think of other version numbers) 2° For our users: here, there are three ways of doing it: a) open a hole to possibly non-free software on a local maven repo; b) install the debian java packages in a local repo (with or without symlinks); c) no site-wide repo, then every user install libs in there home. 3° If we choose 2°c, we can also create a Debian Maven repository. We simply organize our libs in a way maven can understand, but those jars would be uploaded by DD's and extracted from the Debian packages. I think 1° is a short term must have as I think more and more projects will be built with maven and we could add maven plugin to do debian specific tasks so everybody could win here. Some example: slf4j uses maven as build system. According to their website (slf4j.org) projects like Apache Directory, Apache MINA, hibernate etc. use slf4j. If we want to have more java software in the debian repo this will soon be a show stopper. About 2°, we have to decide what we wanna do. I'm not that familiar with maven repositories but it should be possible to create a lokal maven repo from *.deb files. As no user should have write access rights to it software not yet packaged by debian will be downloaded to the users lokal repo in .m2. To simplify things a bit I would suggest to convert /usr/share/java to something maven understands as repository instead of creating another repo. A DD maintained maven repo would be something that could fire usage of maven in debian but I think resources will be a great problem. Regards Stephan Wienczny ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Arnaud Vandyck wrote: To simplify things a bit I would suggest to convert /usr/share/java to something maven understands as repository instead of creating another repo. Maybe we can think about refactoring /usr/share/java, but why do we use the maven layout. I did not fully understand the remark from Dalibor about the undirection comments, but I understood don't bind to a specific way. Dalibor, was it what you explained (and I tried to understand)? There are several different build systems for java applications that support some form of a repository concept. They are also all kind of different in what the expected format of the repository is: Maven 1 Maven 2 Ivy bnd/OBR Raven JSR 277 Picking one over the other is hopeless, as we'll likely encounter applications and libraries using any of those in the future, in the same way how people just didn't stay happy with make/ant/etc. Such programs will likely depend on the same jars, just 'wrapped' suitably to make them 'visible' for each build system in the way it likes to see its repository, as poms, gems, ivy poms, bundles, jam files, etc. So I think the economical way forward is to add a layer of indirection between what the build tools see as their image of the repository formed by packages in /usr/share/java and the actual /usr/share/java binaries. That would let us support any current or future build system with any idea of what its particular repository layout should be, without having to pick one as a default, and having to switch to another default later, decoupling what we do inside /usr/share/java from a particular repository layout chosen by a build tool. I'd suggest solving the first problem on your list first, and only that problem for the next release. Once we have some experience with using maven to satisfy build requirements, I think we'll be able to make a more informed decision which of the three ways forward on the second problem to pick. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Many thanks for the clarifications. So if we are talking about refactoring /usr/share/java, let's do it in some sort of Debian way. We'll write a wrapper around different build system to let them understand our layout. (is it what you meant?) ;-) 2007/12/19, Dalibor Topic [EMAIL PROTECTED]: Arnaud Vandyck wrote: To simplify things a bit I would suggest to convert /usr/share/java to something maven understands as repository instead of creating another repo. Maybe we can think about refactoring /usr/share/java, but why do we use the maven layout. I did not fully understand the remark from Dalibor about the undirection comments, but I understood don't bind to a specific way. Dalibor, was it what you explained (and I tried to understand)? There are several different build systems for java applications that support some form of a repository concept. They are also all kind of different in what the expected format of the repository is: Maven 1 Maven 2 Ivy bnd/OBR Raven JSR 277 Picking one over the other is hopeless, as we'll likely encounter applications and libraries using any of those in the future, in the same way how people just didn't stay happy with make/ant/etc. Such programs will likely depend on the same jars, just 'wrapped' suitably to make them 'visible' for each build system in the way it likes to see its repository, as poms, gems, ivy poms, bundles, jam files, etc. So I think the economical way forward is to add a layer of indirection between what the build tools see as their image of the repository formed by packages in /usr/share/java and the actual /usr/share/java binaries. That would let us support any current or future build system with any idea of what its particular repository layout should be, without having to pick one as a default, and having to switch to another default later, decoupling what we do inside /usr/share/java from a particular repository layout chosen by a build tool. I'd suggest solving the first problem on your list first, and only that problem for the next release. Once we have some experience with using maven to satisfy build requirements, I think we'll be able to make a more informed decision which of the three ways forward on the second problem to pick. cheers, dalibor topic -- Arnaud Vandyck ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Arnaud Vandyck wrote: Many thanks for the clarifications. So if we are talking about refactoring /usr/share/java, let's do it in some sort of Debian way. We'll write a wrapper around different build system to let them understand our layout. (is it what you meant?) I hope that wrapping the build systems themselves won't be necessary. It may be easier to 'setUp' and 'tearDown' virtual repositories based on our own binaries in /usr/share/java and automatically generated metadata from debian's metadata about the packages using some kind of a debhelper_maven script or something like that. I'd imagine something like this: 1. for the package build, you'd call debhelper_maven 2. (setUp) it would look at the dependency information of the package, and for each dependency generate a POM file on the fly from the debian package metadata, 3. (build) then call maven with the freshly generated pom repository to do its build work, 4. (tearDown) then tear down the POM repository, i.e. remove POMs, unregister them from Maven, etc. Replace maven POM above with a different system, but the steps would be virtually the same. Pros: * builds are reproducible consistent within the Debian system * Debian's metadata is used throughout the build system, i.e. no metadata skew * could be separated from user's own build system repository, so no 'your jars are messing with my jars' conflicts Cons: * someone would need to implement it and see if the pros actually hold. :) cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Arnaud Vandyck wrote: What about a maven plugin that leave the jar in /usr/share/java, but register the jar. if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar and we could have maven to use /var/lib/maven2 or /var/lib/m2 then, when you install a lib*-java, it's present for global maven. I think it'd be possible to do that if a /etc/maven2/settings.xml tell where the local repo is A layer of indirection makes sense, as we will likely see more 'repository systems' in the next year or two, with Java 7 modules, for example. It's highly unlikely that their developers will be able to agree on anything, in particular what a repository should look like, so indirection lets us support several of them without having to pick favorites. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Arnaud Vandyck wrote: What about a maven plugin that leave the jar in /usr/share/java, but register the jar. if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar If it's meant to be run in postinst then Maven might not be installed yet. But I wonder how we are going to handle versioned dependencies. Maven projects tend to specify exact version numbers, right? That could be a real headache. I don't think we can ignore the version number, but we cannot use it as a hard dependency either. Cheers, Marcus ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Marcus Better wrote: Arnaud Vandyck wrote: What about a maven plugin that leave the jar in /usr/share/java, but register the jar. if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar If it's meant to be run in postinst then Maven might not be installed yet. But I wonder how we are going to handle versioned dependencies. Maven projects tend to specify exact version numbers, right? That could be a real headache. I don't think we can ignore the version number, but we cannot use it as a hard dependency either. Cheers, Marcus The install:install-file plugin does almost what Arnaud suggests: http://maven.apache.org/guides/mini/guide-installing-3rd-party-jars.html mvn install:install-file -Dfile=path-to-file -DgroupId=group-id \ -DartifactId=artifact-id -Dversion=version \ -Dpackaging=packaging But I wonder just how important it is to have a system-wide local repo? It would certainly reduce the number of Jars an end-user needed to download (clearly a good thing), but it seems like a lot of work (for us). ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Paul Cager wrote: Marcus Better wrote: Arnaud Vandyck wrote: What about a maven plugin that leave the jar in /usr/share/java, but register the jar. if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar If it's meant to be run in postinst then Maven might not be installed yet. But I wonder how we are going to handle versioned dependencies. Maven projects tend to specify exact version numbers, right? That could be a real headache. I don't think we can ignore the version number, but we cannot use it as a hard dependency either. Cheers, Marcus The install:install-file plugin does almost what Arnaud suggests: http://maven.apache.org/guides/mini/guide-installing-3rd-party-jars.html mvn install:install-file -Dfile=path-to-file -DgroupId=group-id \ -DartifactId=artifact-id -Dversion=version \ -Dpackaging=packaging But I wonder just how important it is to have a system-wide local repo? It would certainly reduce the number of Jars an end-user needed to download (clearly a good thing), but it seems like a lot of work (for us). I assume it's going to be a necessity for us for reproducible offline maven builds to inform maven about our own jars. cheers, dalibor topic ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
Dalibor Topic wrote: Paul Cager wrote: Marcus Better wrote: Arnaud Vandyck wrote: What about a maven plugin that leave the jar in /usr/share/java, but register the jar. if mvn present: mvn install -DgroupId=... -DversionId=... -DartifactId=... /usr/share/java/my.jar If it's meant to be run in postinst then Maven might not be installed yet. But I wonder how we are going to handle versioned dependencies. Maven projects tend to specify exact version numbers, right? That could be a real headache. I don't think we can ignore the version number, but we cannot use it as a hard dependency either. Cheers, Marcus The install:install-file plugin does almost what Arnaud suggests: http://maven.apache.org/guides/mini/guide-installing-3rd-party-jars.html mvn install:install-file -Dfile=path-to-file -DgroupId=group-id \ -DartifactId=artifact-id -Dversion=version \ -Dpackaging=packaging But I wonder just how important it is to have a system-wide local repo? It would certainly reduce the number of Jars an end-user needed to download (clearly a good thing), but it seems like a lot of work (for us). I assume it's going to be a necessity for us for reproducible offline maven builds to inform maven about our own jars. cheers, dalibor topic Yes, I'd agree with that. When using maven as part of a Debian package build, you'd need to use install:install-file or similar to create a (presumably temporary) maven repo from the jars in /usr/share/java. Or, more efficiently, we could just set up symlinks. What I'm not so sure about is exporting the jars in /usr/share/java as a Maven repo to *end users*. I can see it would be a useful way to reduce network traffic (especially on multi-user systems), but it seems like a lot of additional complexity and work for a relatively small gain. The system admin could always set up a Maven-Proxy (http://maven-proxy.codehaus.org/). Just my opinion, of course, but I think there's a lot of things that should be higher priority. Thanks, Paul ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
Re: /usr/share/java as maven repository?
On Wed, Sep 05, 2007 at 02:06:04AM +0200, Steffen Moeller wrote: Dear all, would it be reasonable to transform the /usr/share/java directory into something that may also function as an internal Maven repository? http://maven.apache.org/guides/introduction/introduction-to-repositories.html It would allow us to store multiple versions of a jar on the same machine and at the same time and thus avoid redundant downloads when compiling/starting software. The ambiguity for which jar's version to take when there is no version specified could be organised by something analogous to Debian's alternative system. I think using /usr/share/java directly as a maven repo would bring us into a big mess. We should use something like /usr/share/maven/repo and write a policy how to make all java libraries provide entries for the repo (where to put the pom.xml and jar, put a link into the repo that points to /usr/share/java/some.jar, etc.). Cheers, Michael ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
/usr/share/java as maven repository?
Dear all, would it be reasonable to transform the /usr/share/java directory into something that may also function as an internal Maven repository? http://maven.apache.org/guides/introduction/introduction-to-repositories.html It would allow us to store multiple versions of a jar on the same machine and at the same time and thus avoid redundant downloads when compiling/starting software. The ambiguity for which jar's version to take when there is no version specified could be organised by something analogous to Debian's alternative system. Regards, Steffen ___ pkg-java-maintainers mailing list pkg-java-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/pkg-java-maintainers
