Re: a cleaned up central repository?
What about creating a new legacy repository for deprecated artifacts? Stuff that we don't want in the main repository could be moved to a new legacy repo. This way we effectively deprecate these artifacts, but it does not require any POM or metadata changes. Anyone that needs to use the deprecated artifacts, could then just add the legacy repo to their repo manager or a profile in settings.xml. Jason van Zyl wrote: On 2009-09-25, at 5:13 AM, Petar Tahchiev wrote: Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. Exactly, this is all part of providing better information over time. There is no big bang cleanup that makes it all better. That is just a pipe dream. We just have to incrementally and diligently clean up what we have. Maven Central is a great resource and it needs some TLC but not cleansing. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : +1 to adding deprecation metadata. -1 to having a plugin... the deprecation warnings should come from maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support Also if we are adding deprecation metadata, there are a number of other little bits of metadata that should be added, e.g. version comparison scheme (since OSGI rules are different from Maven 2.x rules, we will need a way to flag which rules apply... I see this as being a rule applied to a groupId or at best a groupId:artifactId... if you want to change your version comparison rule halfway through, change the groupId or the artifactId) -Stephen 2009/9/25 Anders Kristian Andersen : I think it is TOTAL important we keep the contract: *** This is a long standing rule that we do not remove or change the contents of central *** Therefore a way out could be to start making deprecated tags in the directories. Then we could come up with a *** maven-deprected-dependency-plugin *** that could warn users against various bad / deprecated / moved artifacts. Consider a director with pom.xml and other files. adding a file called deprecated.xml could deprecate the directory / parts of the files etc. then running mvn deprected-dependency:check would list usage This could be combined with options / tools in archiva could help too. best regards Anders Kristian Andersen On 25/09/2009, at 06.11, Brian Fox wrote: This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Ma
Re: a cleaned up central repository?
if we have a second repo at all that means that anyone not using a repository manager will hit both repos for every artifact (which is bad) IMHO, central is what it is, all we can do is add metadata to help paint over the crayon scribbles on the wall from when the children were growing up ;-) -Stephen Sent from my [rhymes with tryPod] ;-) On 1 Oct 2009, at 18:39, Paul Gier wrote: What about creating a new legacy repository for deprecated artifacts? Stuff that we don't want in the main repository could be moved to a new legacy repo. This way we effectively deprecate these artifacts, but it does not require any POM or metadata changes. Anyone that needs to use the deprecated artifacts, could then just add the legacy repo to their repo manager or a profile in settings.xml. Jason van Zyl wrote: On 2009-09-25, at 5:13 AM, Petar Tahchiev wrote: Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. Exactly, this is all part of providing better information over time. There is no big bang cleanup that makes it all better. That is just a pipe dream. We just have to incrementally and diligently clean up what we have. Maven Central is a great resource and it needs some TLC but not cleansing. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : +1 to adding deprecation metadata. -1 to having a plugin... the deprecation warnings should come from maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support Also if we are adding deprecation metadata, there are a number of other little bits of metadata that should be added, e.g. version comparison scheme (since OSGI rules are different from Maven 2.x rules, we will need a way to flag which rules apply... I see this as being a rule applied to a groupId or at best a groupId:artifactId... if you want to change your version comparison rule halfway through, change the groupId or the artifactId) -Stephen 2009/9/25 Anders Kristian Andersen >: I think it is TOTAL important we keep the contract: *** This is a long standing rule that we do not remove or change the contents of central *** Therefore a way out could be to start making deprecated tags in the directories. Then we could come up with a *** maven-deprected-dependency- plugin *** that could warn users against various bad / deprecated / moved artifacts. Consider a director with pom.xml and other files. adding a file called deprecated.xml could deprecate the directory / parts of the files etc. then running mvn deprected-dependency:check would list usage This could be combined with options / tools in archiva could help too. best regards Anders Kristian Andersen On 25/09/2009, at 06.11, Brian Fox wrote: This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version
Re: a cleaned up central repository?
http://www.mail-archive.com/us...@maven.apache.org/msg102285.html According to current policies, fixing corruptions on Central is not an option. Change policies is not an option. (Change-able policy is not a policy). Option1: Hide the corruption (through added metadata) Option2: Build a new Central which is clean On Thu, Oct 1, 2009 at 1:20 PM, Stephen Connolly wrote: > if we have a second repo at all that means that anyone not using a > repository manager will hit both repos for every artifact (which is bad) > > IMHO, central is what it is, all we can do is add metadata to help paint > over the crayon scribbles on the wall from when the children were growing up > ;-) > > -Stephen > > Sent from my [rhymes with tryPod] ;-) > > On 1 Oct 2009, at 18:39, Paul Gier wrote: > >> What about creating a new legacy repository for deprecated artifacts? >> Stuff that we don't want in the main repository could be moved to a new >> legacy repo. This way we effectively deprecate these artifacts, but it does >> not require any POM or metadata changes. Anyone that needs to use the >> deprecated artifacts, could then just add the legacy repo to their repo >> manager or a profile in settings.xml. >> >> Jason van Zyl wrote: >>> >>> On 2009-09-25, at 5:13 AM, Petar Tahchiev wrote: Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. >>> >>> Exactly, this is all part of providing better information over time. >>> There is no big bang cleanup that makes it all better. That is just a pipe >>> dream. We just have to incrementally and diligently clean up what we have. >>> Maven Central is a great resource and it needs some TLC but not cleansing. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : > > +1 to adding deprecation metadata. > > -1 to having a plugin... the deprecation warnings should come from > maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support > > Also if we are adding deprecation metadata, there are a number of > other little bits of metadata that should be added, e.g. version > comparison scheme (since OSGI rules are different from Maven 2.x > rules, we will need a way to flag which rules apply... I see this as > being a rule applied to a groupId or at best a groupId:artifactId... > if you want to change your version comparison rule halfway through, > change the groupId or the artifactId) > > -Stephen > > 2009/9/25 Anders Kristian Andersen > : >> >> I think it is TOTAL important we keep the contract: >> *** This is a long standing rule that we do not remove or change the >> contents of central *** >> >> Therefore a way out could be to start making deprecated tags in the >> directories. >> Then we could come up with a *** maven-deprected-dependency-plugin >> *** that >> could warn users against various bad / deprecated / moved artifacts. >> >> Consider a director with pom.xml and other files. >> >> adding a file called deprecated.xml could deprecate the directory / >> parts of >> the files etc. >> >> then running >> >> mvn deprected-dependency:check >> would list usage >> >> This could be combined with options / tools in archiva could help too. >> >> best regards >> Anders Kristian Andersen >> >> >> >> >> On 25/09/2009, at 06.11, Brian Fox wrote: >> >>> This has already been done once in history, between M1 and M2 and >>> look >>> how we still have that mess to deal with all the time. Doing this >>> again serves no one well, making sure new data coming in is clean is >>> more productive for everyone. Who would _want_ to deploy their stuff >>> to the "old" repo? No one. The hurdle to get to the new repo would be >>> the same as putting the checks on the current repo for all new >>> artifacts. >>> >>> On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter >>> wrote: Moving over to dev... So here's a t
Re: a cleaned up central repository?
You know where Option1 will drive us? When the added metadata which hides current corruption will become corrupt, we need another layer. At the end, it will look like a big onion. :) Where will Option2 will get us? The new repo will get corrupted again. Unless the policy of repo-ing something will get rewritten, like this: only source code can be uploaded in packages to a public repo. Compile can only take place locally when you are checking out something or after (lazy checkout). - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
I would like to see some votes: 1. Big Rotten Onion 2. Starting Over After Writing New Policies On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz wrote: > You know where Option1 will drive us? > When the added metadata which hides current corruption will become > corrupt, we need another layer. > At the end, it will look like a big onion. :) > > Where will Option2 will get us? > The new repo will get corrupted again. > Unless the policy of repo-ing something will get rewritten, like this: > only source code can be uploaded in packages to a public repo. > Compile can only take place locally when you are checking out > something or after (lazy checkout). > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Isn't that funny: http://ask.metafilter.com/68789/Why-do-rotten-onions-smell-good On Thu, Oct 1, 2009 at 2:13 PM, Albert Kurucz wrote: > I would like to see some votes: > > 1. Big Rotten Onion > 2. Starting Over After Writing New Policies > > On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz wrote: >> You know where Option1 will drive us? >> When the added metadata which hides current corruption will become >> corrupt, we need another layer. >> At the end, it will look like a big onion. :) >> >> Where will Option2 will get us? >> The new repo will get corrupted again. >> Unless the policy of repo-ing something will get rewritten, like this: >> only source code can be uploaded in packages to a public repo. >> Compile can only take place locally when you are checking out >> something or after (lazy checkout). >> > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
My vote is on Option2: http://xircles.codehaus.org/projects/pinin On Thu, Oct 1, 2009 at 2:43 PM, Albert Kurucz wrote: > Isn't that funny: > http://ask.metafilter.com/68789/Why-do-rotten-onions-smell-good > > > On Thu, Oct 1, 2009 at 2:13 PM, Albert Kurucz wrote: >> I would like to see some votes: >> >> 1. Big Rotten Onion >> 2. Starting Over After Writing New Policies >> >> On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz >> wrote: >>> You know where Option1 will drive us? >>> When the added metadata which hides current corruption will become >>> corrupt, we need another layer. >>> At the end, it will look like a big onion. :) >>> >>> Where will Option2 will get us? >>> The new repo will get corrupted again. >>> Unless the policy of repo-ing something will get rewritten, like this: >>> only source code can be uploaded in packages to a public repo. >>> Compile can only take place locally when you are checking out >>> something or after (lazy checkout). >>> >> > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Albert, Clearly you seem to have plenty of enthusiasm for helping to provide better metadata, and I don't want to discourage that. Providing and hosting a repository containing 90 thousand files that serves greater than 12TB of data a month is not as easy as you might imagine. Starting over with a new repository is not the answer here. Getting 90k artifacts vetted and cleaned is a significant undertaking. Frankly many of the artifacts in there are old, underused versions and spending effort on those will have much less immediate impact than stopping new artifacts getting in that have bad (or missing) data. We have chosen to attack this problem by raising the barrier to entry for new artifacts. This work started months ago and we'll be able to put something in place in the next few weeks. This will be done in an automated fashion, starting with artifacts that are uploaded manually. Then we will apply the same rules (automatically) to anything coming in via rsync. Besides improving and vetting the data coming in, this more automated process is designed at drastically improving the turnaround time to get new artifacts and sync's configured. If you really want to assist here, I can see several ways you personally can assist this process: 1) Contribute _code_ that validates the various conditions you think are important to validate. We already have an interface developed that I can point you at if you're interested. These rules will help in many ways because it will help check new artifacts, check old artifacts, and allow people to use them with their repository manager internally if they choose. 2) Provide a detailed list of artifacts and metadata you consider broken. We can sit around and theorize about how things could be better, but first having a concrete list of broken poms and other data will help us focus on the most prominent problems first. The MEV project in Jira is where we collect these. I don't care much if you produce one jira or a jira with a huge list, it's the gathering of the list that is most important. 3) When these artifacts are identified, work with the teams producing these poms to educate them on the proper pom constructs to eliminate them. Generally the teams don't produce bad data on purpose so some education is required. 4) Assuming we have identified a significant number of the problems from work done in #2, we would then need concrete proposals for how to fix this without breaking people's builds. Proposals can be posted on the MAVENUSER wiki space for futher discussion and refinement. 5) Assuming the proposals gather momentum, someone would need to code the proposals 6) Then assistance would be needed to actually cleanup the metadata in the context of the implementation. Things get done around here by people actually stepping in to get them done. You're enthusiastic and we'd be glad to accept help in the areas above. I think further theorizing at this point is going to get diminishing returns, and I personally think attempting to fork an entire repository is not going to help the users as much as even items #1,2,3 above. Brian Fox Apache Maven PMC Chair-Elect On Thu, Oct 1, 2009 at 12:13 PM, Albert Kurucz wrote: > I would like to see some votes: > > 1. Big Rotten Onion > 2. Starting Over After Writing New Policies > > On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz wrote: >> You know where Option1 will drive us? >> When the added metadata which hides current corruption will become >> corrupt, we need another layer. >> At the end, it will look like a big onion. :) >> >> Where will Option2 will get us? >> The new repo will get corrupted again. >> Unless the policy of repo-ing something will get rewritten, like this: >> only source code can be uploaded in packages to a public repo. >> Compile can only take place locally when you are checking out >> something or after (lazy checkout). >> > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
On 02/10/2009, at 9:51 AM, Brian Fox wrote: We have chosen to attack this problem by raising the barrier to entry for new artifacts. This work started months ago and we'll be able to put something in place in the next few weeks. This will be done in an automated fashion, starting with artifacts that are uploaded manually. Then we will apply the same rules (automatically) to anything coming in via rsync. Besides improving and vetting the data coming in, this more automated process is designed at drastically improving the turnaround time to get new artifacts and sync's configured. Brian, do you want expand on this for the rest of us? What are the rules, how will it work, and how can we use it? - Brett - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
> > Brian, do you want expand on this for the rest of us? What are the rules, > how will it work, and how can we use it? > The details belong in another thread, but we'll be able to implement these rules for projects using repository.apache.org and oss.sonatype.org. This isn't really new information, we discussed this many months ago, it was affectionately referred to as the "wendy" plugin. ;-) - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Why would everyone need to use both repos? If the legacy repository is done correctly, the vast majority of users would never need to hit it, or even know about it at all. Note that this idea is different than creating a new clean repository. This would be a new repository just for the garbage. Most artifacts would stay where they are. Just as an example, in the jboss repo I found someone had mixed up the main jar with the javadoc jar. So Maven was trying to use the javadocs on the classpath. Stuff like this is unusable, and should be removed from the repository IMO. Stephen Connolly wrote: if we have a second repo at all that means that anyone not using a repository manager will hit both repos for every artifact (which is bad) IMHO, central is what it is, all we can do is add metadata to help paint over the crayon scribbles on the wall from when the children were growing up ;-) -Stephen Sent from my [rhymes with tryPod] ;-) On 1 Oct 2009, at 18:39, Paul Gier wrote: What about creating a new legacy repository for deprecated artifacts? Stuff that we don't want in the main repository could be moved to a new legacy repo. This way we effectively deprecate these artifacts, but it does not require any POM or metadata changes. Anyone that needs to use the deprecated artifacts, could then just add the legacy repo to their repo manager or a profile in settings.xml. Jason van Zyl wrote: On 2009-09-25, at 5:13 AM, Petar Tahchiev wrote: Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. Exactly, this is all part of providing better information over time. There is no big bang cleanup that makes it all better. That is just a pipe dream. We just have to incrementally and diligently clean up what we have. Maven Central is a great resource and it needs some TLC but not cleansing. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : +1 to adding deprecation metadata. -1 to having a plugin... the deprecation warnings should come from maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support Also if we are adding deprecation metadata, there are a number of other little bits of metadata that should be added, e.g. version comparison scheme (since OSGI rules are different from Maven 2.x rules, we will need a way to flag which rules apply... I see this as being a rule applied to a groupId or at best a groupId:artifactId... if you want to change your version comparison rule halfway through, change the groupId or the artifactId) -Stephen 2009/9/25 Anders Kristian Andersen : I think it is TOTAL important we keep the contract: *** This is a long standing rule that we do not remove or change the contents of central *** Therefore a way out could be to start making deprecated tags in the directories. Then we could come up with a *** maven-deprected-dependency-plugin *** that could warn users against various bad / deprecated / moved artifacts. Consider a director with pom.xml and other files. adding a file called deprecated.xml could deprecate the directory / parts of the files etc. then running mvn deprected-dependency:check would list usage This could be combined with options / tools in archiva could help too. best regards Anders Kristian Andersen On 25/09/2009, at 06.11, Brian Fox wrote: This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo
Re: a cleaned up central repository?
On Fri, Oct 2, 2009 at 5:57 AM, Paul Gier wrote: > Why would everyone need to use both repos? If the legacy repository is done > correctly, the vast majority of users would never need to hit it, or even > know about it at all. > > Note that this idea is different than creating a new clean repository. This > would be a new repository just for the garbage. Most artifacts would stay > where they are. > > Just as an example, in the jboss repo I found someone had mixed up the main > jar with the javadoc jar. So Maven was trying to use the javadocs on the > classpath. Stuff like this is unusable, and should be removed from the > repository IMO. > Something like that likely would, but we don't control the jboss repo. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Brian Fox wrote: On Fri, Oct 2, 2009 at 5:57 AM, Paul Gier wrote: Why would everyone need to use both repos? If the legacy repository is done correctly, the vast majority of users would never need to hit it, or even know about it at all. Note that this idea is different than creating a new clean repository. This would be a new repository just for the garbage. Most artifacts would stay where they are. Just as an example, in the jboss repo I found someone had mixed up the main jar with the javadoc jar. So Maven was trying to use the javadocs on the classpath. Stuff like this is unusable, and should be removed from the repository IMO. Something like that likely would, but we don't control the jboss repo. I removed that one from the JBoss repo already. I was just using it as an example. My point is that there is stuff in central that isn't being used and could safely be moved to a legacy repository as part of the cleanup effort. I agree with Brian's other comment though that the most improvement will probably be gained from preventing new bad stuff going in. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
On 2009-10-02, at 6:35 AM, Paul Gier wrote: Brian Fox wrote: On Fri, Oct 2, 2009 at 5:57 AM, Paul Gier wrote: Why would everyone need to use both repos? If the legacy repository is done correctly, the vast majority of users would never need to hit it, or even know about it at all. Note that this idea is different than creating a new clean repository. This would be a new repository just for the garbage. Most artifacts would stay where they are. Just as an example, in the jboss repo I found someone had mixed up the main jar with the javadoc jar. So Maven was trying to use the javadocs on the classpath. Stuff like this is unusable, and should be removed from the repository IMO. Something like that likely would, but we don't control the jboss repo. I removed that one from the JBoss repo already. I was just using it as an example. My point is that there is stuff in central that isn't being used and could safely be moved to a legacy repository as part of the cleanup effort. I agree with Brian's other comment though that the most improvement will probably be gained from preventing new bad stuff going in. That is simply the only way it will work without wreaking havoc. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org Thanks, Jason -- Jason van Zyl Founder, Apache Maven http://twitter.com/jvanzyl -- - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Brian, 1. Thanks for your invitation to contribute. I would like to do that, but within my limited free time I can promise to contribute my thought to MEV, after crystallizing them a bit. I am working on it. 2. > Provide a detailed list of artifacts and metadata you consider broken. I think this approach will not work. You should really work on providing list of artifacts which are not broken (Certified good list), and having a Maven client which is able to use this list (or multiple lists) for filtering and searching. 3. > When these artifacts are identified, work with the teams producing > these poms to educate them on the proper pom constructs to eliminate > them. If you have a list (or multiple lists which classify them by testing different qualities) of good artifacts, teams will try to deliver their artifacts to these lists by educating themselves. Trying to police these teams, you will just receive resistance. Regards, Albert On Thu, Oct 1, 2009 at 6:51 PM, Brian Fox wrote: > Albert, > Clearly you seem to have plenty of enthusiasm for helping to provide > better metadata, and I don't want to discourage that. > > Providing and hosting a repository containing 90 thousand files that > serves greater than 12TB of data a month is not as easy as you might > imagine. Starting over with a new repository is not the answer here. > > Getting 90k artifacts vetted and cleaned is a significant undertaking. > Frankly many of the artifacts in there are old, underused versions and > spending effort on those will have much less immediate impact than > stopping new artifacts getting in that have bad (or missing) data. We > have chosen to attack this problem by raising the barrier to entry for > new artifacts. This work started months ago and we'll be able to put > something in place in the next few weeks. > > This will be done in an automated fashion, starting with artifacts > that are uploaded manually. Then we will apply the same rules > (automatically) to anything coming in via rsync. Besides improving and > vetting the data coming in, this more automated process is designed at > drastically improving the turnaround time to get new artifacts and > sync's configured. > > If you really want to assist here, I can see several ways you > personally can assist this process: > > 1) Contribute _code_ that validates the various conditions you think > are important to validate. We already have an interface developed that > I can point you at if you're interested. These rules will help in many > ways because it will help check new artifacts, check old artifacts, > and allow people to use them with their repository manager internally > if they choose. > > 2) Provide a detailed list of artifacts and metadata you consider > broken. We can sit around and theorize about how things could be > better, but first having a concrete list of broken poms and other data > will help us focus on the most prominent problems first. The MEV > project in Jira is where we collect these. I don't care much if you > produce one jira or a jira with a huge list, it's the gathering of the > list that is most important. > > 3) When these artifacts are identified, work with the teams producing > these poms to educate them on the proper pom constructs to eliminate > them. Generally the teams don't produce bad data on purpose so some > education is required. > > 4) Assuming we have identified a significant number of the problems > from work done in #2, we would then need concrete proposals for how to > fix this without breaking people's builds. Proposals can be posted on > the MAVENUSER wiki space for futher discussion and refinement. > > 5) Assuming the proposals gather momentum, someone would need to code > the proposals > > 6) Then assistance would be needed to actually cleanup the metadata in > the context of the implementation. > > Things get done around here by people actually stepping in to get them > done. You're enthusiastic and we'd be glad to accept help in the areas > above. I think further theorizing at this point is going to get > diminishing returns, and I personally think attempting to fork an > entire repository is not going to help the users as much as even > items #1,2,3 above. > > Brian Fox > Apache Maven PMC Chair-Elect > > > > > > On Thu, Oct 1, 2009 at 12:13 PM, Albert Kurucz > wrote: >> I would like to see some votes: >> >> 1. Big Rotten Onion >> 2. Starting Over After Writing New Policies >> >> On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz >> wrote: >>> You know where Option1 will drive us? >>> When the added metadata which hides current corruption will become >>> corrupt, we need another layer. >>> At the end, it will look like a big onion. :) >>> >>> Where will Option2 will get us? >>> The new repo will get corrupted again. >>> Unless the policy of repo-ing something will get rewritten, like this: >>> only source code can be uploaded in packages to a public repo. >>> Compile can only take place locally when you ar
Re: a cleaned up central repository?
> > 2. >> Provide a detailed list of artifacts and metadata you consider broken. > I think this approach will not work. > You should really work on providing list of artifacts which are not > broken (Certified good list), and having a Maven client which is able > to use this list (or multiple lists) for filtering and searching. Ok then, I assert they are all fine. Now you can provide a list and refute me ;-). Seriously, the definition of "broken" depends on the observer. A pom that doesn't mark something as optional only appears broken to users who don't otherwise need that dependency for example. Before we can "fix" anything "broken" we need to start by defining what you think is broken and why. > > 3. >> When these artifacts are identified, work with the teams producing >> these poms to educate them on the proper pom constructs to eliminate >> them. > If you have a list (or multiple lists which classify them by testing > different qualities) of good artifacts, teams will try to deliver > their artifacts to these lists by educating themselves. > Trying to police these teams, you will just receive resistance. Not true from experience, again this is subject to the definition of broken. > > > > Regards, > Albert > > > > > On Thu, Oct 1, 2009 at 6:51 PM, Brian Fox wrote: >> Albert, >> Clearly you seem to have plenty of enthusiasm for helping to provide >> better metadata, and I don't want to discourage that. >> >> Providing and hosting a repository containing 90 thousand files that >> serves greater than 12TB of data a month is not as easy as you might >> imagine. Starting over with a new repository is not the answer here. >> >> Getting 90k artifacts vetted and cleaned is a significant undertaking. >> Frankly many of the artifacts in there are old, underused versions and >> spending effort on those will have much less immediate impact than >> stopping new artifacts getting in that have bad (or missing) data. We >> have chosen to attack this problem by raising the barrier to entry for >> new artifacts. This work started months ago and we'll be able to put >> something in place in the next few weeks. >> >> This will be done in an automated fashion, starting with artifacts >> that are uploaded manually. Then we will apply the same rules >> (automatically) to anything coming in via rsync. Besides improving and >> vetting the data coming in, this more automated process is designed at >> drastically improving the turnaround time to get new artifacts and >> sync's configured. >> >> If you really want to assist here, I can see several ways you >> personally can assist this process: >> >> 1) Contribute _code_ that validates the various conditions you think >> are important to validate. We already have an interface developed that >> I can point you at if you're interested. These rules will help in many >> ways because it will help check new artifacts, check old artifacts, >> and allow people to use them with their repository manager internally >> if they choose. >> >> 2) Provide a detailed list of artifacts and metadata you consider >> broken. We can sit around and theorize about how things could be >> better, but first having a concrete list of broken poms and other data >> will help us focus on the most prominent problems first. The MEV >> project in Jira is where we collect these. I don't care much if you >> produce one jira or a jira with a huge list, it's the gathering of the >> list that is most important. >> >> 3) When these artifacts are identified, work with the teams producing >> these poms to educate them on the proper pom constructs to eliminate >> them. Generally the teams don't produce bad data on purpose so some >> education is required. >> >> 4) Assuming we have identified a significant number of the problems >> from work done in #2, we would then need concrete proposals for how to >> fix this without breaking people's builds. Proposals can be posted on >> the MAVENUSER wiki space for futher discussion and refinement. >> >> 5) Assuming the proposals gather momentum, someone would need to code >> the proposals >> >> 6) Then assistance would be needed to actually cleanup the metadata in >> the context of the implementation. >> >> Things get done around here by people actually stepping in to get them >> done. You're enthusiastic and we'd be glad to accept help in the areas >> above. I think further theorizing at this point is going to get >> diminishing returns, and I personally think attempting to fork an >> entire repository is not going to help the users as much as even >> items #1,2,3 above. >> >> Brian Fox >> Apache Maven PMC Chair-Elect >> >> >> >> >> >> On Thu, Oct 1, 2009 at 12:13 PM, Albert Kurucz >> wrote: >>> I would like to see some votes: >>> >>> 1. Big Rotten Onion >>> 2. Starting Over After Writing New Policies >>> >>> On Thu, Oct 1, 2009 at 2:04 PM, Albert Kurucz >>> wrote: You know where Option1 will drive us? When the added metadata which hides current corruption will
Re: a cleaned up central repository?
Brian, > Ok then, I assert they are all fine. Now you can provide a list and > refute me ;-). In this case (if they were all fine) here is your list: http://repo2.maven.org/maven2/.index/ (But unfortunately they are not all fine.) > Seriously, the definition of "broken" depends on the observer. True. This is why maybe there should be different "Good lists" and users should be allowed to choose, depending on their taste. > Before we can > "fix" anything "broken" we need to start by defining what you think is > broken and why. One of the possible Definitions of "Good list", which I would like call "Maven Central Compliance" is defined here: http://maven.apache.org/guides/mini/guide-central-repository-upload.html If artifacts are on Central which are not on this list (which list should really be realized soon), I don't mind, as long as I could search or filter by this list. You cannot objectively define what is "broken" only if you specify which Lists you are talking about. Do you mean, the "Maven Central Compliance" list? On Mon, Oct 5, 2009 at 12:38 PM, Brian Fox wrote: >> >> 2. >>> Provide a detailed list of artifacts and metadata you consider broken. >> I think this approach will not work. >> You should really work on providing list of artifacts which are not >> broken (Certified good list), and having a Maven client which is able >> to use this list (or multiple lists) for filtering and searching. > > Ok then, I assert they are all fine. Now you can provide a list and > refute me ;-). > > Seriously, the definition of "broken" depends on the observer. A pom > that doesn't mark something as optional only appears broken to users > who don't otherwise need that dependency for example. Before we can > "fix" anything "broken" we need to start by defining what you think is > broken and why. > >> >> 3. >>> When these artifacts are identified, work with the teams producing >>> these poms to educate them on the proper pom constructs to eliminate >>> them. >> If you have a list (or multiple lists which classify them by testing >> different qualities) of good artifacts, teams will try to deliver >> their artifacts to these lists by educating themselves. >> Trying to police these teams, you will just receive resistance. > > Not true from experience, again this is subject to the definition of broken. > >> >> >> >> Regards, >> Albert >> >> >> >> >> On Thu, Oct 1, 2009 at 6:51 PM, Brian Fox wrote: >>> Albert, >>> Clearly you seem to have plenty of enthusiasm for helping to provide >>> better metadata, and I don't want to discourage that. >>> >>> Providing and hosting a repository containing 90 thousand files that >>> serves greater than 12TB of data a month is not as easy as you might >>> imagine. Starting over with a new repository is not the answer here. >>> >>> Getting 90k artifacts vetted and cleaned is a significant undertaking. >>> Frankly many of the artifacts in there are old, underused versions and >>> spending effort on those will have much less immediate impact than >>> stopping new artifacts getting in that have bad (or missing) data. We >>> have chosen to attack this problem by raising the barrier to entry for >>> new artifacts. This work started months ago and we'll be able to put >>> something in place in the next few weeks. >>> >>> This will be done in an automated fashion, starting with artifacts >>> that are uploaded manually. Then we will apply the same rules >>> (automatically) to anything coming in via rsync. Besides improving and >>> vetting the data coming in, this more automated process is designed at >>> drastically improving the turnaround time to get new artifacts and >>> sync's configured. >>> >>> If you really want to assist here, I can see several ways you >>> personally can assist this process: >>> >>> 1) Contribute _code_ that validates the various conditions you think >>> are important to validate. We already have an interface developed that >>> I can point you at if you're interested. These rules will help in many >>> ways because it will help check new artifacts, check old artifacts, >>> and allow people to use them with their repository manager internally >>> if they choose. >>> >>> 2) Provide a detailed list of artifacts and metadata you consider >>> broken. We can sit around and theorize about how things could be >>> better, but first having a concrete list of broken poms and other data >>> will help us focus on the most prominent problems first. The MEV >>> project in Jira is where we collect these. I don't care much if you >>> produce one jira or a jira with a huge list, it's the gathering of the >>> list that is most important. >>> >>> 3) When these artifacts are identified, work with the teams producing >>> these poms to educate them on the proper pom constructs to eliminate >>> them. Generally the teams don't produce bad data on purpose so some >>> education is required. >>> >>> 4) Assuming we have identified a significant number of the problems >>> from work done in
Re: a cleaned up central repository?
Brian, If you start maintaining a list of violators I have zero motivation to contribute to your list, because this kind of list is useless for me. If you start maintaining a list of compliant artifacts (what many still believe Central is) that is a totally different case. On Mon, Oct 5, 2009 at 2:16 PM, Albert Kurucz wrote: > Brian, > >> Ok then, I assert they are all fine. Now you can provide a list and >> refute me ;-). > In this case (if they were all fine) here is your list: > http://repo2.maven.org/maven2/.index/ > (But unfortunately they are not all fine.) > >> Seriously, the definition of "broken" depends on the observer. > True. This is why maybe there should be different "Good lists" and > users should be allowed to choose, depending on their taste. > >> Before we can >> "fix" anything "broken" we need to start by defining what you think is >> broken and why. > > One of the possible Definitions of "Good list", which I would like > call "Maven Central Compliance" is defined here: > http://maven.apache.org/guides/mini/guide-central-repository-upload.html > If artifacts are on Central which are not on this list (which list > should really be realized soon), I don't mind, as long as I could > search or filter by this list. > You cannot objectively define what is "broken" only if you specify > which Lists you are talking about. Do you mean, the "Maven Central > Compliance" list? > > > > > On Mon, Oct 5, 2009 at 12:38 PM, Brian Fox wrote: >>> >>> 2. Provide a detailed list of artifacts and metadata you consider broken. >>> I think this approach will not work. >>> You should really work on providing list of artifacts which are not >>> broken (Certified good list), and having a Maven client which is able >>> to use this list (or multiple lists) for filtering and searching. >> >> Ok then, I assert they are all fine. Now you can provide a list and >> refute me ;-). >> >> Seriously, the definition of "broken" depends on the observer. A pom >> that doesn't mark something as optional only appears broken to users >> who don't otherwise need that dependency for example. Before we can >> "fix" anything "broken" we need to start by defining what you think is >> broken and why. >> >>> >>> 3. When these artifacts are identified, work with the teams producing these poms to educate them on the proper pom constructs to eliminate them. >>> If you have a list (or multiple lists which classify them by testing >>> different qualities) of good artifacts, teams will try to deliver >>> their artifacts to these lists by educating themselves. >>> Trying to police these teams, you will just receive resistance. >> >> Not true from experience, again this is subject to the definition of broken. >> >>> >>> >>> >>> Regards, >>> Albert >>> >>> >>> >>> >>> On Thu, Oct 1, 2009 at 6:51 PM, Brian Fox wrote: Albert, Clearly you seem to have plenty of enthusiasm for helping to provide better metadata, and I don't want to discourage that. Providing and hosting a repository containing 90 thousand files that serves greater than 12TB of data a month is not as easy as you might imagine. Starting over with a new repository is not the answer here. Getting 90k artifacts vetted and cleaned is a significant undertaking. Frankly many of the artifacts in there are old, underused versions and spending effort on those will have much less immediate impact than stopping new artifacts getting in that have bad (or missing) data. We have chosen to attack this problem by raising the barrier to entry for new artifacts. This work started months ago and we'll be able to put something in place in the next few weeks. This will be done in an automated fashion, starting with artifacts that are uploaded manually. Then we will apply the same rules (automatically) to anything coming in via rsync. Besides improving and vetting the data coming in, this more automated process is designed at drastically improving the turnaround time to get new artifacts and sync's configured. If you really want to assist here, I can see several ways you personally can assist this process: 1) Contribute _code_ that validates the various conditions you think are important to validate. We already have an interface developed that I can point you at if you're interested. These rules will help in many ways because it will help check new artifacts, check old artifacts, and allow people to use them with their repository manager internally if they choose. 2) Provide a detailed list of artifacts and metadata you consider broken. We can sit around and theorize about how things could be better, but first having a concrete list of broken poms and other data will help us focus on the most prominent problems first. The MEV project in Jira is where we collect these. I don't care much if you >>
Re: a cleaned up central repository?
On Mon, Oct 5, 2009 at 3:16 PM, Albert Kurucz wrote: > Brian, > >> Ok then, I assert they are all fine. Now you can provide a list and >> refute me ;-). > In this case (if they were all fine) here is your list: > http://repo2.maven.org/maven2/.index/ > (But unfortunately they are not all fine.) > >> Seriously, the definition of "broken" depends on the observer. > True. This is why maybe there should be different "Good lists" and > users should be allowed to choose, depending on their taste. > >> Before we can >> "fix" anything "broken" we need to start by defining what you think is >> broken and why. > > One of the possible Definitions of "Good list", which I would like > call "Maven Central Compliance" is defined here: > http://maven.apache.org/guides/mini/guide-central-repository-upload.html > If artifacts are on Central which are not on this list (which list > should really be realized soon), I don't mind, as long as I could > search or filter by this list. > You cannot objectively define what is "broken" only if you specify > which Lists you are talking about. Do you mean, the "Maven Central > Compliance" list? I assume you mean this list of requirements? There are some requirements for the minimal information in the POMs that are in the central repository. At least these must be present: modelVersion groupId artifactId packaging name version description url licenses scm url dependencies I don't think that I would consider things broken simply because the name, description, url, scm url where missing. I would be annoyed but not surprised if the license wasn't populated correctly. So if you're saying you want to exclude everything from your build simply because one of those are missing, then I think we fundamentally disagree. Yes it would be nice if those were filled in properly but none of those reduce the usefulness of users to a point where they simply should be treated like they don't exist. I consider things broken if the pom doesn't parse, the dependencies are wrong (again subject to perspective in some cases), the gav isn't correct, the checksums or signatures are broken. Otherwise from a repository perspective they are not broken. If you attempt to enumerate all the things in central that match all of those values above and build a repo of only those, it will be a nearly useless repo. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
On Mon, Oct 5, 2009 at 4:36 PM, Albert Kurucz wrote: > Brian, > > If you start maintaining a list of violators I have zero motivation to > contribute to your list, because this kind of list is useless for me. > If you start maintaining a list of compliant artifacts (what many > still believe Central is) that is a totally different case. Again, this list is meant to provide examples of what you think is broken, but i'm not particularly interested in listing all the poms missing the things like description, etc. Rather I think identifying totally bad poms, broken checksums, bad dependencies is a more useful place to start. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Not to mention that these below are "formal" requirements only. Their _presence_ is required, but nothing is said about their _content_.I can publish a POM that will _have_ dependencies section, but how do we know that the dependencies section is _correct_? Also: license in POM. What license "name" is allowed? Are they keyed by by license URL? Etc... Many of these are pretty hard to determine in "machine way" ~t~ On Tue, Oct 6, 2009 at 2:26 AM, Brian Fox wrote: > On Mon, Oct 5, 2009 at 3:16 PM, Albert Kurucz > wrote: > > Brian, > > > >> Ok then, I assert they are all fine. Now you can provide a list and > >> refute me ;-). > > In this case (if they were all fine) here is your list: > > http://repo2.maven.org/maven2/.index/ > > (But unfortunately they are not all fine.) > > > >> Seriously, the definition of "broken" depends on the observer. > > True. This is why maybe there should be different "Good lists" and > > users should be allowed to choose, depending on their taste. > > > >> Before we can > >> "fix" anything "broken" we need to start by defining what you think is > >> broken and why. > > > > One of the possible Definitions of "Good list", which I would like > > call "Maven Central Compliance" is defined here: > > http://maven.apache.org/guides/mini/guide-central-repository-upload.html > > If artifacts are on Central which are not on this list (which list > > should really be realized soon), I don't mind, as long as I could > > search or filter by this list. > > You cannot objectively define what is "broken" only if you specify > > which Lists you are talking about. Do you mean, the "Maven Central > > Compliance" list? > > I assume you mean this list of requirements? > There are some requirements for the minimal information in the POMs > that are in the central repository. At least these must be present: > > modelVersion > groupId > artifactId > packaging > name > version > description > url > licenses > scm url > dependencies > > I don't think that I would consider things broken simply because the > name, description, url, scm url where missing. I would be annoyed but > not surprised if the license wasn't populated correctly. So if you're > saying you want to exclude everything from your build simply because > one of those are missing, then I think we fundamentally disagree. Yes > it would be nice if those were filled in properly but none of those > reduce the usefulness of users to a point where they simply should be > treated like they don't exist. > > I consider things broken if the pom doesn't parse, the dependencies > are wrong (again subject to perspective in some cases), the gav isn't > correct, the checksums or signatures are broken. Otherwise from a > repository perspective they are not broken. > > If you attempt to enumerate all the things in central that match all > of those values above and build a repo of only those, it will be a > nearly useless repo. > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: a cleaned up central repository?
Changing the rules is OK. Not publishing the change of rules (policies) is not OK, because it is like getting "undocumented features" to the system. We cannot talk about "broken" until we define good, and I believed once it was defined. If you redefined good, please publish it. On Mon, Oct 5, 2009 at 7:28 PM, Brian Fox wrote: > On Mon, Oct 5, 2009 at 4:36 PM, Albert Kurucz wrote: >> Brian, >> >> If you start maintaining a list of violators I have zero motivation to >> contribute to your list, because this kind of list is useless for me. >> If you start maintaining a list of compliant artifacts (what many >> still believe Central is) that is a totally different case. > > Again, this list is meant to provide examples of what you think is > broken, but i'm not particularly interested in listing all the poms > missing the things like description, etc. Rather I think identifying > totally bad poms, broken checksums, bad dependencies is a more useful > place to start. > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Tamas, I cannot predict when, but once it will be done in a "machine way" or a mathematical/logical proof will be discovered that it is impossible. Agreed, it will not be easy. 2009/10/6 Tamás Cservenák : > Not to mention that these below are "formal" requirements only. Their > _presence_ is required, but nothing is said about their _content_.I can > publish a POM that will _have_ dependencies section, but how do we know that > the dependencies section is _correct_? > > Also: license in POM. What license "name" is allowed? Are they keyed by by > license URL? Etc... > > Many of these are pretty hard to determine in "machine way" > > ~t~ > > On Tue, Oct 6, 2009 at 2:26 AM, Brian Fox wrote: > >> On Mon, Oct 5, 2009 at 3:16 PM, Albert Kurucz >> wrote: >> > Brian, >> > >> >> Ok then, I assert they are all fine. Now you can provide a list and >> >> refute me ;-). >> > In this case (if they were all fine) here is your list: >> > http://repo2.maven.org/maven2/.index/ >> > (But unfortunately they are not all fine.) >> > >> >> Seriously, the definition of "broken" depends on the observer. >> > True. This is why maybe there should be different "Good lists" and >> > users should be allowed to choose, depending on their taste. >> > >> >> Before we can >> >> "fix" anything "broken" we need to start by defining what you think is >> >> broken and why. >> > >> > One of the possible Definitions of "Good list", which I would like >> > call "Maven Central Compliance" is defined here: >> > http://maven.apache.org/guides/mini/guide-central-repository-upload.html >> > If artifacts are on Central which are not on this list (which list >> > should really be realized soon), I don't mind, as long as I could >> > search or filter by this list. >> > You cannot objectively define what is "broken" only if you specify >> > which Lists you are talking about. Do you mean, the "Maven Central >> > Compliance" list? >> >> I assume you mean this list of requirements? >> There are some requirements for the minimal information in the POMs >> that are in the central repository. At least these must be present: >> >> modelVersion >> groupId >> artifactId >> packaging >> name >> version >> description >> url >> licenses >> scm url >> dependencies >> >> I don't think that I would consider things broken simply because the >> name, description, url, scm url where missing. I would be annoyed but >> not surprised if the license wasn't populated correctly. So if you're >> saying you want to exclude everything from your build simply because >> one of those are missing, then I think we fundamentally disagree. Yes >> it would be nice if those were filled in properly but none of those >> reduce the usefulness of users to a point where they simply should be >> treated like they don't exist. >> >> I consider things broken if the pom doesn't parse, the dependencies >> are wrong (again subject to perspective in some cases), the gav isn't >> correct, the checksums or signatures are broken. Otherwise from a >> repository perspective they are not broken. >> >> If you attempt to enumerate all the things in central that match all >> of those values above and build a repo of only those, it will be a >> nearly useless repo. >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz wrote: > Tamas, I cannot predict when, but once it will be done in a "machine > way" or a mathematical/logical proof will be discovered that it is > impossible. Agreed, it will not be easy. > This is why in suggestion 1) I said lets get some code to validate the artifacts. Having a suite of validation rules implemented hurts noone and then people can choose to use them or not, it's just like the bunch of enforcer rules we already have. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Brian, > This is why in suggestion 1) I said lets get some code to validate the > artifacts. Reading this article I thought you already have that http://www.think88.com/resources/Maven_white_paper_june_2009.pdf " Sonatype maintains a central repository with more than 90,000 artifacts, consuming more than 60 GB of storage. In addition to the artifacts themselves, the Maven Central Repository also contains a POM-file for each of the artifacts, containing the meta data for these artifacts. To protect the integrity of the repository, Sonatype checks the meta data for correctness. If the meta data is erroneous, the artifact can’t be uploaded. " On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: > On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz wrote: >> Tamas, I cannot predict when, but once it will be done in a "machine >> way" or a mathematical/logical proof will be discovered that it is >> impossible. Agreed, it will not be easy. >> > > This is why in suggestion 1) I said lets get some code to validate the > artifacts. Having a suite of validation rules implemented hurts noone > and then people can choose to use them or not, it's just like the > bunch of enforcer rules we already have. > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Brian, This is a Stalemate! I go to sleep() until that code arrives to you. If I had the time, I promise I would code it for you. On Tue, Oct 6, 2009 at 12:30 PM, Albert Kurucz wrote: > Brian, > >> This is why in suggestion 1) I said lets get some code to validate the >> artifacts. > > Reading this article I thought you already have that > > http://www.think88.com/resources/Maven_white_paper_june_2009.pdf > " > Sonatype maintains a central repository with more than 90,000 artifacts, > consuming more than 60 GB of storage. In addition to the artifacts > themselves, the > Maven Central Repository also contains a POM-file for each of the artifacts, > containing the meta data for these artifacts. To protect the integrity of the > repository, Sonatype checks the meta data for correctness. If the meta data is > erroneous, the artifact can’t be uploaded. > " > > > On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: >> On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz >> wrote: >>> Tamas, I cannot predict when, but once it will be done in a "machine >>> way" or a mathematical/logical proof will be discovered that it is >>> impossible. Agreed, it will not be easy. >>> >> >> This is why in suggestion 1) I said lets get some code to validate the >> artifacts. Having a suite of validation rules implemented hurts noone >> and then people can choose to use them or not, it's just like the >> bunch of enforcer rules we already have. >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Albert, I think you are confusing the metadata.XML files from the pom.XML files the metadata sonatype are referring to is the metametadata (ie metadata.xml files) and nit the artifact metadata (ie pom.xml files) there are places in central where the metametadata is incorrect. let's get those fixed pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the dependencies with compile scope and without optional=true in my case, it is a bad pom because on a point release started pulling in windows nt logging support, and my app breaks with that support in place... but it is still a valid pom and it is still a "correct" pom I could argue that the dependencies could be optional, others could argue that instead the whole log4j should be refactored into multiple artifacts pulling in each of the dependencies I think should be optional... none of us are correct I could argue that a pom which does not list a license is broken... others might say that code in the public domain has no license, so their pom would be incorrect to list a license I could have a closed source artifact on central, so no scm, no developers, no distMgnt, no build, no reporting... and that is still a valid pom the only metadata we can prove to be incorrect is the metametadata... and thankfully that can be reconstructed from the pom files Sent from my [rhymes with tryPod] ;-) On 6 Oct 2009, at 18:30, Albert Kurucz wrote: Brian, This is why in suggestion 1) I said lets get some code to validate the artifacts. Reading this article I thought you already have that http://www.think88.com/resources/Maven_white_paper_june_2009.pdf " Sonatype maintains a central repository with more than 90,000 artifacts, consuming more than 60 GB of storage. In addition to the artifacts themselves, the Maven Central Repository also contains a POM-file for each of the artifacts, containing the meta data for these artifacts. To protect the integrity of the repository, Sonatype checks the meta data for correctness. If the meta data is erroneous, the artifact can’t be uploaded. " On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz > wrote: Tamas, I cannot predict when, but once it will be done in a "machine way" or a mathematical/logical proof will be discovered that it is impossible. Agreed, it will not be easy. This is why in suggestion 1) I said lets get some code to validate the artifacts. Having a suite of validation rules implemented hurts noone and then people can choose to use them or not, it's just like the bunch of enforcer rules we already have. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Steven, http://lmgtfy.com/?q=maven+metametadata Found this 1st: " So he's talking about me!? Does that make me a maven? Does mavenhood explain metametametadata? Does it excuse all of its self-referential posts? Are you sick of them yet? Is this clever? Can I ask anymore questions? Um, no. " >From 2004! On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly wrote: > Albert, > > I think you are confusing the metadata.XML files from the pom.XML files > > the metadata sonatype are referring to is the metametadata (ie metadata.xml > files) and nit the artifact metadata (ie pom.xml files) > > there are places in central where the metametadata is incorrect. let's get > those fixed > > pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the > dependencies with compile scope and without optional=true > > in my case, it is a bad pom because on a point release started pulling in > windows nt logging support, and my app breaks with that support in place... > but it is still a valid pom and it is still a "correct" pom > > I could argue that the dependencies could be optional, others could argue > that instead the whole log4j should be refactored into multiple artifacts > pulling in each of the dependencies I think should be optional... none of us > are correct > > I could argue that a pom which does not list a license is broken... others > might say that code in the public domain has no license, so their pom would > be incorrect to list a license > > I could have a closed source artifact on central, so no scm, no developers, > no distMgnt, no build, no reporting... and that is still a valid pom > > the only metadata we can prove to be incorrect is the metametadata... and > thankfully that can be reconstructed from the pom files > > Sent from my [rhymes with tryPod] ;-) > > On 6 Oct 2009, at 18:30, Albert Kurucz wrote: > >> Brian, >> >>> This is why in suggestion 1) I said lets get some code to validate the >>> artifacts. >> >> Reading this article I thought you already have that >> >> http://www.think88.com/resources/Maven_white_paper_june_2009.pdf >> " >> Sonatype maintains a central repository with more than 90,000 artifacts, >> consuming more than 60 GB of storage. In addition to the artifacts >> themselves, the >> Maven Central Repository also contains a POM-file for each of the >> artifacts, >> containing the meta data for these artifacts. To protect the integrity of >> the >> repository, Sonatype checks the meta data for correctness. If the meta >> data is >> erroneous, the artifact can’t be uploaded. >> " >> >> >> On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: >>> >>> On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz >>> wrote: Tamas, I cannot predict when, but once it will be done in a "machine way" or a mathematical/logical proof will be discovered that it is impossible. Agreed, it will not be easy. >>> >>> This is why in suggestion 1) I said lets get some code to validate the >>> artifacts. Having a suite of validation rules implemented hurts noone >>> and then people can choose to use them or not, it's just like the >>> bunch of enforcer rules we already have. >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
metadata is data about data metaanalysis is an analysis of other analyses metaworry is worrying about worrying metacognition is thinking about thinking metametadata is therefore data about metadata the jar is your artifact : data the pom is data about the artifact : metadata the metadata.xml is data about the pom files : metametadata Sent from my [rhymes with tryPod] ;-) On 6 Oct 2009, at 20:02, Albert Kurucz wrote: Steven, http://lmgtfy.com/?q=maven+metametadata Found this 1st: " So he's talking about me!? Does that make me a maven? Does mavenhood explain metametametadata? Does it excuse all of its self-referential posts? Are you sick of them yet? Is this clever? Can I ask anymore questions? Um, no. " From 2004! On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly wrote: Albert, I think you are confusing the metadata.XML files from the pom.XML files the metadata sonatype are referring to is the metametadata (ie metadata.xml files) and nit the artifact metadata (ie pom.xml files) there are places in central where the metametadata is incorrect. let's get those fixed pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the dependencies with compile scope and without optional=true in my case, it is a bad pom because on a point release started pulling in windows nt logging support, and my app breaks with that support in place... but it is still a valid pom and it is still a "correct" pom I could argue that the dependencies could be optional, others could argue that instead the whole log4j should be refactored into multiple artifacts pulling in each of the dependencies I think should be optional... none of us are correct I could argue that a pom which does not list a license is broken... others might say that code in the public domain has no license, so their pom would be incorrect to list a license I could have a closed source artifact on central, so no scm, no developers, no distMgnt, no build, no reporting... and that is still a valid pom the only metadata we can prove to be incorrect is the metametadata... and thankfully that can be reconstructed from the pom files Sent from my [rhymes with tryPod] ;-) On 6 Oct 2009, at 18:30, Albert Kurucz wrote: Brian, This is why in suggestion 1) I said lets get some code to validate the artifacts. Reading this article I thought you already have that http://www.think88.com/resources/Maven_white_paper_june_2009.pdf " Sonatype maintains a central repository with more than 90,000 artifacts, consuming more than 60 GB of storage. In addition to the artifacts themselves, the Maven Central Repository also contains a POM-file for each of the artifacts, containing the meta data for these artifacts. To protect the integrity of the repository, Sonatype checks the meta data for correctness. If the meta data is erroneous, the artifact can’t be uploaded. " On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz > wrote: Tamas, I cannot predict when, but once it will be done in a "machine way" or a mathematical/logical proof will be discovered that it is impossible. Agreed, it will not be easy. This is why in suggestion 1) I said lets get some code to validate the artifacts. Having a suite of validation rules implemented hurts noone and then people can choose to use them or not, it's just like the bunch of enforcer rules we already have. --- -- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org --- -- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
Steven, It is absolute necessary to sit down and brainstorm. Thanks for the ideas! I mean it. :-) On Tue, Oct 6, 2009 at 3:22 PM, Stephen Connolly wrote: > metadata is data about data > metaanalysis is an analysis of other analyses > metaworry is worrying about worrying > metacognition is thinking about thinking > > metametadata is therefore data about metadata > > the jar is your artifact : data > the pom is data about the artifact : metadata > the metadata.xml is data about the pom files : metametadata > > Sent from my [rhymes with tryPod] ;-) > > On 6 Oct 2009, at 20:02, Albert Kurucz wrote: > >> Steven, >> >> http://lmgtfy.com/?q=maven+metametadata >> Found this 1st: >> " >> So he's talking about me!? Does that make me a maven? Does mavenhood >> explain metametametadata? Does it excuse all of its self-referential >> posts? Are you sick of them yet? Is this clever? Can I ask anymore >> questions? Um, no. >> " >> From 2004! >> >> On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly >> wrote: >>> >>> Albert, >>> >>> I think you are confusing the metadata.XML files from the pom.XML files >>> >>> the metadata sonatype are referring to is the metametadata (ie >>> metadata.xml >>> files) and nit the artifact metadata (ie pom.xml files) >>> >>> there are places in central where the metametadata is incorrect. let's >>> get >>> those fixed >>> >>> pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the >>> dependencies with compile scope and without optional=true >>> >>> in my case, it is a bad pom because on a point release started pulling in >>> windows nt logging support, and my app breaks with that support in >>> place... >>> but it is still a valid pom and it is still a "correct" pom >>> >>> I could argue that the dependencies could be optional, others could argue >>> that instead the whole log4j should be refactored into multiple artifacts >>> pulling in each of the dependencies I think should be optional... none of >>> us >>> are correct >>> >>> I could argue that a pom which does not list a license is broken... >>> others >>> might say that code in the public domain has no license, so their pom >>> would >>> be incorrect to list a license >>> >>> I could have a closed source artifact on central, so no scm, no >>> developers, >>> no distMgnt, no build, no reporting... and that is still a valid pom >>> >>> the only metadata we can prove to be incorrect is the metametadata... and >>> thankfully that can be reconstructed from the pom files >>> >>> Sent from my [rhymes with tryPod] ;-) >>> >>> On 6 Oct 2009, at 18:30, Albert Kurucz wrote: >>> Brian, > This is why in suggestion 1) I said lets get some code to validate the > artifacts. Reading this article I thought you already have that http://www.think88.com/resources/Maven_white_paper_june_2009.pdf " Sonatype maintains a central repository with more than 90,000 artifacts, consuming more than 60 GB of storage. In addition to the artifacts themselves, the Maven Central Repository also contains a POM-file for each of the artifacts, containing the meta data for these artifacts. To protect the integrity of the repository, Sonatype checks the meta data for correctness. If the meta data is erroneous, the artifact can’t be uploaded. " On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: > > On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz > wrote: >> >> Tamas, I cannot predict when, but once it will be done in a "machine >> way" or a mathematical/logical proof will be discovered that it is >> impossible. Agreed, it will not be easy. >> > > This is why in suggestion 1) I said lets get some code to validate the > artifacts. Having a suite of validation rules implemented hurts noone > and then people can choose to use them or not, it's just like the > bunch of enforcer rules we already have. > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h.
Re: a cleaned up central repository?
Sorry, could not stand it: the deprecation data about "broken" artifacts described in metametadata : metametametadata :D ~t~ On Tue, Oct 6, 2009 at 10:22 PM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > metadata is data about data > metaanalysis is an analysis of other analyses > metaworry is worrying about worrying > metacognition is thinking about thinking > > metametadata is therefore data about metadata > > the jar is your artifact : data > the pom is data about the artifact : metadata > the metadata.xml is data about the pom files : metametadata > > Sent from my [rhymes with tryPod] ;-) > > On 6 Oct 2009, at 20:02, Albert Kurucz wrote: > > Steven, >> >> http://lmgtfy.com/?q=maven+metametadata >> Found this 1st: >> " >> So he's talking about me!? Does that make me a maven? Does mavenhood >> explain metametametadata? Does it excuse all of its self-referential >> posts? Are you sick of them yet? Is this clever? Can I ask anymore >> questions? Um, no. >> " >> From 2004! >> >> On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly >> wrote: >> >>> Albert, >>> >>> I think you are confusing the metadata.XML files from the pom.XML files >>> >>> the metadata sonatype are referring to is the metametadata (ie >>> metadata.xml >>> files) and nit the artifact metadata (ie pom.xml files) >>> >>> there are places in central where the metametadata is incorrect. let's >>> get >>> those fixed >>> >>> pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the >>> dependencies with compile scope and without optional=true >>> >>> in my case, it is a bad pom because on a point release started pulling in >>> windows nt logging support, and my app breaks with that support in >>> place... >>> but it is still a valid pom and it is still a "correct" pom >>> >>> I could argue that the dependencies could be optional, others could argue >>> that instead the whole log4j should be refactored into multiple artifacts >>> pulling in each of the dependencies I think should be optional... none of >>> us >>> are correct >>> >>> I could argue that a pom which does not list a license is broken... >>> others >>> might say that code in the public domain has no license, so their pom >>> would >>> be incorrect to list a license >>> >>> I could have a closed source artifact on central, so no scm, no >>> developers, >>> no distMgnt, no build, no reporting... and that is still a valid pom >>> >>> the only metadata we can prove to be incorrect is the metametadata... and >>> thankfully that can be reconstructed from the pom files >>> >>> Sent from my [rhymes with tryPod] ;-) >>> >>> On 6 Oct 2009, at 18:30, Albert Kurucz wrote: >>> >>> Brian, This is why in suggestion 1) I said lets get some code to validate the > artifacts. > Reading this article I thought you already have that http://www.think88.com/resources/Maven_white_paper_june_2009.pdf " Sonatype maintains a central repository with more than 90,000 artifacts, consuming more than 60 GB of storage. In addition to the artifacts themselves, the Maven Central Repository also contains a POM-file for each of the artifacts, containing the meta data for these artifacts. To protect the integrity of the repository, Sonatype checks the meta data for correctness. If the meta data is erroneous, the artifact can’t be uploaded. " On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: > > On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz > > wrote: > >> >> Tamas, I cannot predict when, but once it will be done in a "machine >> way" or a mathematical/logical proof will be discovered that it is >> impossible. Agreed, it will not be easy. >> >> > This is why in suggestion 1) I said lets get some code to validate the > artifacts. Having a suite of validation rules implemented hurts noone > and then people can choose to use them or not, it's just like the > bunch of enforcer rules we already have. > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> >>> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> > - > To unsu
Re: a cleaned up central repository?
Should we call a software, which links to other software (has dependencies), metasoftware? 2009/10/7 Tamás Cservenák : > Sorry, could not stand it: > the deprecation data about "broken" artifacts described in metametadata : > metametametadata :D > > ~t~ > > On Tue, Oct 6, 2009 at 10:22 PM, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > >> metadata is data about data >> metaanalysis is an analysis of other analyses >> metaworry is worrying about worrying >> metacognition is thinking about thinking >> >> metametadata is therefore data about metadata >> >> the jar is your artifact : data >> the pom is data about the artifact : metadata >> the metadata.xml is data about the pom files : metametadata >> >> Sent from my [rhymes with tryPod] ;-) >> >> On 6 Oct 2009, at 20:02, Albert Kurucz wrote: >> >> Steven, >>> >>> http://lmgtfy.com/?q=maven+metametadata >>> Found this 1st: >>> " >>> So he's talking about me!? Does that make me a maven? Does mavenhood >>> explain metametametadata? Does it excuse all of its self-referential >>> posts? Are you sick of them yet? Is this clever? Can I ask anymore >>> questions? Um, no. >>> " >>> From 2004! >>> >>> On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly >>> wrote: >>> Albert, I think you are confusing the metadata.XML files from the pom.XML files the metadata sonatype are referring to is the metametadata (ie metadata.xml files) and nit the artifact metadata (ie pom.xml files) there are places in central where the metametadata is incorrect. let's get those fixed pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the dependencies with compile scope and without optional=true in my case, it is a bad pom because on a point release started pulling in windows nt logging support, and my app breaks with that support in place... but it is still a valid pom and it is still a "correct" pom I could argue that the dependencies could be optional, others could argue that instead the whole log4j should be refactored into multiple artifacts pulling in each of the dependencies I think should be optional... none of us are correct I could argue that a pom which does not list a license is broken... others might say that code in the public domain has no license, so their pom would be incorrect to list a license I could have a closed source artifact on central, so no scm, no developers, no distMgnt, no build, no reporting... and that is still a valid pom the only metadata we can prove to be incorrect is the metametadata... and thankfully that can be reconstructed from the pom files Sent from my [rhymes with tryPod] ;-) On 6 Oct 2009, at 18:30, Albert Kurucz wrote: Brian, > > This is why in suggestion 1) I said lets get some code to validate the >> artifacts. >> > > Reading this article I thought you already have that > > http://www.think88.com/resources/Maven_white_paper_june_2009.pdf > " > Sonatype maintains a central repository with more than 90,000 artifacts, > consuming more than 60 GB of storage. In addition to the artifacts > themselves, the > Maven Central Repository also contains a POM-file for each of the > artifacts, > containing the meta data for these artifacts. To protect the integrity > of > the > repository, Sonatype checks the meta data for correctness. If the meta > data is > erroneous, the artifact can’t be uploaded. > " > > > On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox wrote: > >> >> On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz > > >> wrote: >> >>> >>> Tamas, I cannot predict when, but once it will be done in a "machine >>> way" or a mathematical/logical proof will be discovered that it is >>> impossible. Agreed, it will not be easy. >>> >>> >> This is why in suggestion 1) I said lets get some code to validate the >> artifacts. Having a suite of validation rules implemented hurts noone >> and then people can choose to use them or not, it's just like the >> bunch of enforcer rules we already have. >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> >> >> > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org >>> ---
Re: a cleaned up central repository?
No because metasoftware would be software about software (which makes no sense) 2009/10/8 Albert Kurucz > Should we call a software, which links to other software (has > dependencies), metasoftware? > > 2009/10/7 Tamás Cservenák : > > Sorry, could not stand it: > > the deprecation data about "broken" artifacts described in metametadata : > > metametametadata :D > > > > ~t~ > > > > On Tue, Oct 6, 2009 at 10:22 PM, Stephen Connolly < > > stephen.alan.conno...@gmail.com> wrote: > > > >> metadata is data about data > >> metaanalysis is an analysis of other analyses > >> metaworry is worrying about worrying > >> metacognition is thinking about thinking > >> > >> metametadata is therefore data about metadata > >> > >> the jar is your artifact : data > >> the pom is data about the artifact : metadata > >> the metadata.xml is data about the pom files : metametadata > >> > >> Sent from my [rhymes with tryPod] ;-) > >> > >> On 6 Oct 2009, at 20:02, Albert Kurucz wrote: > >> > >> Steven, > >>> > >>> http://lmgtfy.com/?q=maven+metametadata > >>> Found this 1st: > >>> " > >>> So he's talking about me!? Does that make me a maven? Does mavenhood > >>> explain metametametadata? Does it excuse all of its self-referential > >>> posts? Are you sick of them yet? Is this clever? Can I ask anymore > >>> questions? Um, no. > >>> " > >>> From 2004! > >>> > >>> On Tue, Oct 6, 2009 at 1:06 PM, Stephen Connolly > >>> wrote: > >>> > Albert, > > I think you are confusing the metadata.XML files from the pom.XML > files > > the metadata sonatype are referring to is the metametadata (ie > metadata.xml > files) and nit the artifact metadata (ie pom.xml files) > > there are places in central where the metametadata is incorrect. let's > get > those fixed > > pom's are more subjective: is log4j 1.2.14 a bad pom? it lists all the > dependencies with compile scope and without optional=true > > in my case, it is a bad pom because on a point release started pulling > in > windows nt logging support, and my app breaks with that support in > place... > but it is still a valid pom and it is still a "correct" pom > > I could argue that the dependencies could be optional, others could > argue > that instead the whole log4j should be refactored into multiple > artifacts > pulling in each of the dependencies I think should be optional... none > of > us > are correct > > I could argue that a pom which does not list a license is broken... > others > might say that code in the public domain has no license, so their pom > would > be incorrect to list a license > > I could have a closed source artifact on central, so no scm, no > developers, > no distMgnt, no build, no reporting... and that is still a valid pom > > the only metadata we can prove to be incorrect is the metametadata... > and > thankfully that can be reconstructed from the pom files > > Sent from my [rhymes with tryPod] ;-) > > On 6 Oct 2009, at 18:30, Albert Kurucz > wrote: > > Brian, > > > > This is why in suggestion 1) I said lets get some code to validate > the > >> artifacts. > >> > > > > Reading this article I thought you already have that > > > > http://www.think88.com/resources/Maven_white_paper_june_2009.pdf > > " > > Sonatype maintains a central repository with more than 90,000 > artifacts, > > consuming more than 60 GB of storage. In addition to the artifacts > > themselves, the > > Maven Central Repository also contains a POM-file for each of the > > artifacts, > > containing the meta data for these artifacts. To protect the > integrity > > of > > the > > repository, Sonatype checks the meta data for correctness. If the > meta > > data is > > erroneous, the artifact can’t be uploaded. > > " > > > > > > On Tue, Oct 6, 2009 at 12:19 PM, Brian Fox > wrote: > > > >> > >> On Tue, Oct 6, 2009 at 9:30 AM, Albert Kurucz < > albert.kur...@gmail.com > >> > > >> wrote: > >> > >>> > >>> Tamas, I cannot predict when, but once it will be done in a > "machine > >>> way" or a mathematical/logical proof will be discovered that it is > >>> impossible. Agreed, it will not be easy. > >>> > >>> > >> This is why in suggestion 1) I said lets get some code to validate > the > >> artifacts. Having a suite of validation rules implemented hurts > noone > >> and then people can choose to use them or not, it's just like the > >> bunch of enforcer rules we already have. > >> > >> > - > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > >> > >> > >
Re: a cleaned up central repository?
Stephen Connolly wrote at Donnerstag, 8. Oktober 2009 15:56: > No because metasoftware would be software about software (which makes no > sense) Actually it does, it's called UML :)) - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
UML is a diagram about software 2009/10/8 Jörg Schaible > Stephen Connolly wrote at Donnerstag, 8. Oktober 2009 15:56: > > > No because metasoftware would be software about software (which makes no > > sense) > > Actually it does, it's called UML :)) > > - Jörg > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: a cleaned up central repository?
Modello? Or any source/code generator? :D ~t~ On Thu, Oct 8, 2009 at 5:05 PM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > UML is a diagram about software > > 2009/10/8 Jörg Schaible > > > Stephen Connolly wrote at Donnerstag, 8. Oktober 2009 15:56: > > > > > No because metasoftware would be software about software (which makes > no > > > sense) > > > > Actually it does, it's called UML :)) > > > > - Jörg > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > >
Re: a cleaned up central repository?
If you don't like the term metasoftware, why would you like the metadata? Data has software dependencies (or data dependencies) just like software. Data is software. More about this: http://www.dans.knaw.nl/nl/over_dans/factsheets/mixed_future_proof/ If it looks like a duck... 2009/10/8 Tamás Cservenák : > Modello? Or any source/code generator? > :D > > ~t~ > > On Thu, Oct 8, 2009 at 5:05 PM, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > >> UML is a diagram about software >> >> 2009/10/8 Jörg Schaible >> >> > Stephen Connolly wrote at Donnerstag, 8. Oktober 2009 15:56: >> > >> > > No because metasoftware would be software about software (which makes >> no >> > > sense) >> > >> > Actually it does, it's called UML :)) >> > >> > - Jörg >> > >> > >> > - >> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> > For additional commands, e-mail: dev-h...@maven.apache.org >> > >> > >> > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository?
As fun as this discussion is, I think perhaps it has outlived not only the subject line, but this list :) - Brett On 09/10/2009, at 2:32 AM, Albert Kurucz wrote: If you don't like the term metasoftware, why would you like the metadata? Data has software dependencies (or data dependencies) just like software. Data is software. More about this: http://www.dans.knaw.nl/nl/over_dans/factsheets/mixed_future_proof/ If it looks like a duck... 2009/10/8 Tamás Cservenák : Modello? Or any source/code generator? :D ~t~ On Thu, Oct 8, 2009 at 5:05 PM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: UML is a diagram about software 2009/10/8 Jörg Schaible Stephen Connolly wrote at Donnerstag, 8. Oktober 2009 15:56: No because metasoftware would be software about software (which makes no sense) Actually it does, it's called UML :)) - Jörg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Maven built projects can enter, maybe even more restriction: only the designated Continuous Integration server can upload to it. This pure Maven repo would not be able to compete with Maven Central regarding size or the number of artifacts, but some OSS developers might prefer to use from and supply to this one instead of the big and ugly. On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: On Thu, Sep 24, 2009 at 3:16 PM, Albert Kurucz > wrote: Requirements for the POMs are defined as: http://maven.apache.org/guides/mini/guide-central-repository-upload.html I call the artifact corrupt (regarding Maven Central Compliance) if the POM of the artifact does not fulfills the above requirements. There are corrupt ones have made it to the Central, because the guard was sleeping. Correct, but changing them is not an option because it will destabilize builds. This is a long standing rule that we do not remove or change the contents of central. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
On 2009-09-24, at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? What's matters is improvement of the process going forward. As people move forward with improved submissions from projects then the older submissions of dubious quality will naturally fall out of use. Improving the process and helping projects do the right thing is necessary. Creating another repository isn't really going change the reality that people are going to use a mixture of old and new submissions over a long period of time. You can't just up and change the old content because people depend on it, and you can't just make a rift between the old and new. In much the same way we just have to deal with the shit that exists in Maven 2.x and make it work in Maven 3.x we have to deal with the shit in the repository and make it work with newer submissions that we consciously try to improve. Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Maven built projects can enter, maybe even more restriction: only the designated Continuous Integration server can upload to it. This pure Maven repo would not be able to compete with Maven Central regarding size or the number of artifacts, but some OSS developers might prefer to use from and supply to this one instead of the big and ugly. On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: On Thu, Sep 24, 2009 at 3:16 PM, Albert Kurucz > wrote: Requirements for the POMs are defined as: http://maven.apache.org/guides/mini/guide-central-repository-upload.html I call the artifact corrupt (regarding Maven Central Compliance) if the POM of the artifact does not fulfills the above requirements. There are corrupt ones have made it to the Central, because the guard was sleeping. Correct, but changing them is not an option because it will destabilize builds. This is a long standing rule that we do not remove or change the contents of central. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org Thanks, Jason -- Jason van Zyl Founder, Apache Maven http://twitter.com/jvanzyl -- - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: > Moving over to dev... > > So here's a thought - why don't we create a "new" central repository? > > - a new repository with strict acceptance rules regarding POMs, signatures, > ownership, etc. > - if there's a new metadata format needed (recently discussed), this would > use it > - validated artifacts could be moved over and requests to the old rewritten > (in the same way we maintained the maven1 repo) > - default Maven can ship with both repositories enabled, but a "best > practice" would be to turn old central off (or better, use a repository > manager that doesn't access it / only access it for acceptable artifacts) > > The main issue is finding a way to overcome confusion when an artifact is > changed - you want "old" builds to keep using the same one it always did, > but new builds to use the new one (and cope with potential revision of > metadata without breaking builds). This is the sort of thing that could be > built into Maven in a new version and the new repo format only accessible > from that version. > > Thoughts? > > Cheers, > Brett > > On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: > >> Jason and Brian, thanks for the explanations. >> Understood, the policy of not removing anything from Maven Central >> serves a purpose. >> >> I wish there would be another publicly Maven repository, which is >> maintained with rules enforced. This repo could even have a rule >> (additional to the old and unenforced rules) that only Maven built >> projects can enter, maybe even more restriction: only the designated >> Continuous Integration server can upload to it. >> This pure Maven repo would not be able to compete with Maven Central >> regarding size or the number of artifacts, but some OSS developers >> might prefer to use from and supply to this one instead of the big and >> ugly. >> >> On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: >>> >>> On Thu, Sep 24, 2009 at 3:16 PM, Albert Kurucz >>> wrote: Requirements for the POMs are defined as: http://maven.apache.org/guides/mini/guide-central-repository-upload.html I call the artifact corrupt (regarding Maven Central Compliance) if the POM of the artifact does not fulfills the above requirements. There are corrupt ones have made it to the Central, because the guard was sleeping. >>> >>> Correct, but changing them is not an option because it will >>> destabilize builds. This is a long standing rule that we do not remove >>> or change the contents of central. >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org >>> For additional commands, e-mail: users-h...@maven.apache.org >>> >>> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org >> For additional commands, e-mail: users-h...@maven.apache.org >> > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
I think it is TOTAL important we keep the contract: *** This is a long standing rule that we do not remove or change the contents of central *** Therefore a way out could be to start making deprecated tags in the directories. Then we could come up with a *** maven-deprected-dependency-plugin *** that could warn users against various bad / deprecated / moved artifacts. Consider a director with pom.xml and other files. adding a file called deprecated.xml could deprecate the directory / parts of the files etc. then running mvn deprected-dependency:check would list usage This could be combined with options / tools in archiva could help too. best regards Anders Kristian Andersen On 25/09/2009, at 06.11, Brian Fox wrote: This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Maven built projects can enter, maybe even more restriction: only the designated Continuous Integration server can upload to it. This pure Maven repo would not be able to compete with Maven Central regarding size or the number of artifacts, but some OSS developers might prefer to use from and supply to this one instead of the big and ugly. On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: On Thu, Sep 24, 2009 at 3:16 PM, Albert Kurucz > wrote: Requirements for the POMs are defined as: http://maven.apache.org/guides/mini/guide-central-repository-upload.html I call the artifact corrupt (regarding Maven Central Compliance) if the POM of the artifact does not fulfills the above requirements. There are corrupt ones have made it to the Central, because the guard was sleeping. Correct, but changing them is not an option because it will destabilize builds. This is a long standing rule that we do not remove or change the contents of central. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
+1 to adding deprecation metadata. -1 to having a plugin... the deprecation warnings should come from maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support Also if we are adding deprecation metadata, there are a number of other little bits of metadata that should be added, e.g. version comparison scheme (since OSGI rules are different from Maven 2.x rules, we will need a way to flag which rules apply... I see this as being a rule applied to a groupId or at best a groupId:artifactId... if you want to change your version comparison rule halfway through, change the groupId or the artifactId) -Stephen 2009/9/25 Anders Kristian Andersen : > I think it is TOTAL important we keep the contract: > *** This is a long standing rule that we do not remove or change the > contents of central *** > > Therefore a way out could be to start making deprecated tags in the > directories. > Then we could come up with a *** maven-deprected-dependency-plugin *** that > could warn users against various bad / deprecated / moved artifacts. > > Consider a director with pom.xml and other files. > > adding a file called deprecated.xml could deprecate the directory / parts of > the files etc. > > then running > > mvn deprected-dependency:check > would list usage > > This could be combined with options / tools in archiva could help too. > > best regards > Anders Kristian Andersen > > > > > On 25/09/2009, at 06.11, Brian Fox wrote: > >> This has already been done once in history, between M1 and M2 and look >> how we still have that mess to deal with all the time. Doing this >> again serves no one well, making sure new data coming in is clean is >> more productive for everyone. Who would _want_ to deploy their stuff >> to the "old" repo? No one. The hurdle to get to the new repo would be >> the same as putting the checks on the current repo for all new >> artifacts. >> >> On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: >>> >>> Moving over to dev... >>> >>> So here's a thought - why don't we create a "new" central repository? >>> >>> - a new repository with strict acceptance rules regarding POMs, >>> signatures, >>> ownership, etc. >>> - if there's a new metadata format needed (recently discussed), this >>> would >>> use it >>> - validated artifacts could be moved over and requests to the old >>> rewritten >>> (in the same way we maintained the maven1 repo) >>> - default Maven can ship with both repositories enabled, but a "best >>> practice" would be to turn old central off (or better, use a repository >>> manager that doesn't access it / only access it for acceptable artifacts) >>> >>> The main issue is finding a way to overcome confusion when an artifact is >>> changed - you want "old" builds to keep using the same one it always did, >>> but new builds to use the new one (and cope with potential revision of >>> metadata without breaking builds). This is the sort of thing that could >>> be >>> built into Maven in a new version and the new repo format only accessible >>> from that version. >>> >>> Thoughts? >>> >>> Cheers, >>> Brett >>> >>> On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: >>> Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Maven built projects can enter, maybe even more restriction: only the designated Continuous Integration server can upload to it. This pure Maven repo would not be able to compete with Maven Central regarding size or the number of artifacts, but some OSS developers might prefer to use from and supply to this one instead of the big and ugly. On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: > > On Thu, Sep 24, 2009 at 3:16 PM, Albert Kurucz > > wrote: >> >> Requirements for the POMs are defined as: >> >> http://maven.apache.org/guides/mini/guide-central-repository-upload.html >> I call the artifact corrupt (regarding Maven Central Compliance) if >> the POM of the artifact does not fulfills the above requirements. >> There are corrupt ones have made it to the Central, because the guard >> was sleeping. >> > > Correct, but changing them is not an option because it will > destabilize builds. This is a long standing rule that we do not remove > or change the contents of central. > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : > +1 to adding deprecation metadata. > > -1 to having a plugin... the deprecation warnings should come from > maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support > > Also if we are adding deprecation metadata, there are a number of > other little bits of metadata that should be added, e.g. version > comparison scheme (since OSGI rules are different from Maven 2.x > rules, we will need a way to flag which rules apply... I see this as > being a rule applied to a groupId or at best a groupId:artifactId... > if you want to change your version comparison rule halfway through, > change the groupId or the artifactId) > > -Stephen > > 2009/9/25 Anders Kristian Andersen : >> I think it is TOTAL important we keep the contract: >> *** This is a long standing rule that we do not remove or change the >> contents of central *** >> >> Therefore a way out could be to start making deprecated tags in the >> directories. >> Then we could come up with a *** maven-deprected-dependency-plugin *** that >> could warn users against various bad / deprecated / moved artifacts. >> >> Consider a director with pom.xml and other files. >> >> adding a file called deprecated.xml could deprecate the directory / parts of >> the files etc. >> >> then running >> >> mvn deprected-dependency:check >> would list usage >> >> This could be combined with options / tools in archiva could help too. >> >> best regards >> Anders Kristian Andersen >> >> >> >> >> On 25/09/2009, at 06.11, Brian Fox wrote: >> >>> This has already been done once in history, between M1 and M2 and look >>> how we still have that mess to deal with all the time. Doing this >>> again serves no one well, making sure new data coming in is clean is >>> more productive for everyone. Who would _want_ to deploy their stuff >>> to the "old" repo? No one. The hurdle to get to the new repo would be >>> the same as putting the checks on the current repo for all new >>> artifacts. >>> >>> On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: > Jason and Brian, thanks for the explanations. > Understood, the policy of not removing anything from Maven Central > serves a purpose. > > I wish there would be another publicly Maven repository, which is > maintained with rules enforced. This repo could even have a rule > (additional to the old and unenforced rules) that only Maven built > projects can enter, maybe even more restriction: only the designated > Continuous Integration server can upload to it. > This pure Maven repo would not be able to compete with Maven Central > regarding size or the number of artifacts, but some OSS developers > might prefer to use from and supply to this one instead of the big and > ugly. > > On Thu, Sep 24, 2009 at 6:44 PM, Brian Fox wrote: >> >> On Thu, Sep 24, 2
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
On 2009-09-25, at 5:13 AM, Petar Tahchiev wrote: Hi all, I also like the idea about deprecating artifacts, or whole directory structure. A step forward would be to create a maven-dev-approved repositories or artifacts. Exactly, this is all part of providing better information over time. There is no big bang cleanup that makes it all better. That is just a pipe dream. We just have to incrementally and diligently clean up what we have. Maven Central is a great resource and it needs some TLC but not cleansing. Imagine you are using an outside repository that is really messed up (or worse - anyone can put anything any time). What maven the team could do is setup a ground base of rules, so that if you want your repository to be maven-dev-compliant you must follow them. Later if you are using artifacts from repositories that are not maven-dev-compliant, there could be a message warning you. To me it doesn't really make sense to create a new repository without improving the process of using the repositories. Otherwise, as Brian pointed, we will end up with a new repository, which has the same data, and no one is using the old repo. Cheers, Petar. Because without it doesn't really make sense creating new repository and 2009/9/25 Stephen Connolly : +1 to adding deprecation metadata. -1 to having a plugin... the deprecation warnings should come from maven core itself... ok we can add a plugin for [2.0.x,2.2.1] support Also if we are adding deprecation metadata, there are a number of other little bits of metadata that should be added, e.g. version comparison scheme (since OSGI rules are different from Maven 2.x rules, we will need a way to flag which rules apply... I see this as being a rule applied to a groupId or at best a groupId:artifactId... if you want to change your version comparison rule halfway through, change the groupId or the artifactId) -Stephen 2009/9/25 Anders Kristian Andersen >: I think it is TOTAL important we keep the contract: *** This is a long standing rule that we do not remove or change the contents of central *** Therefore a way out could be to start making deprecated tags in the directories. Then we could come up with a *** maven-deprected-dependency- plugin *** that could warn users against various bad / deprecated / moved artifacts. Consider a director with pom.xml and other files. adding a file called deprecated.xml could deprecate the directory / parts of the files etc. then running mvn deprected-dependency:check would list usage This could be combined with options / tools in archiva could help too. best regards Anders Kristian Andersen On 25/09/2009, at 06.11, Brian Fox wrote: This has already been done once in history, between M1 and M2 and look how we still have that mess to deal with all the time. Doing this again serves no one well, making sure new data coming in is clean is more productive for everyone. Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. On Thu, Sep 24, 2009 at 8:37 PM, Brett Porter wrote: Moving over to dev... So here's a thought - why don't we create a "new" central repository? - a new repository with strict acceptance rules regarding POMs, signatures, ownership, etc. - if there's a new metadata format needed (recently discussed), this would use it - validated artifacts could be moved over and requests to the old rewritten (in the same way we maintained the maven1 repo) - default Maven can ship with both repositories enabled, but a "best practice" would be to turn old central off (or better, use a repository manager that doesn't access it / only access it for acceptable artifacts) The main issue is finding a way to overcome confusion when an artifact is changed - you want "old" builds to keep using the same one it always did, but new builds to use the new one (and cope with potential revision of metadata without breaking builds). This is the sort of thing that could be built into Maven in a new version and the new repo format only accessible from that version. Thoughts? Cheers, Brett On 25/09/2009, at 12:52 PM, Albert Kurucz wrote: Jason and Brian, thanks for the explanations. Understood, the policy of not removing anything from Maven Central serves a purpose. I wish there would be another publicly Maven repository, which is maintained with rules enforced. This repo could even have a rule (additional to the old and unenforced rules) that only Maven built projects can enter, maybe even more restriction: only the designated Continuous Integration server can upload to it. This pure Maven repo would not be able to compete with Maven Central regarding size or the number of artifacts, but some OSS developers might prefer to use from and supply to this one instead of the big and ugly. On Thu, Sep 24, 2009 at 6:44 PM, Bri
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
Ok, I did a poor job of expressing what I was getting at. On 25/09/2009, at 2:02 PM, Jason van Zyl wrote: What's matters is improvement of the process going forward. Agreed, I see that as a pre-requisite. As people move forward with improved submissions from projects then the older submissions of dubious quality will naturally fall out of use. That may not happen in any sort of feasible timeframe. There is some very old stuff out there that isn't getting new releases in a hurry. As you traverse a dependency tree, it's nearly impossible not to hit the rough spots like early commons-logging releases, velocity vs velocity-dep, plexus:plexus-utils vs org.codehaus.plexus:plexus-utils and so on. On 25/09/2009, at 2:11 PM, Brian Fox wrote: Who would _want_ to deploy their stuff to the "old" repo? No one. The hurdle to get to the new repo would be the same as putting the checks on the current repo for all new artifacts. I agree - I'm not saying we allow anyone to deploy to the "old" one - but it would be nice to have a way in Maven to say "only use stuff that meets the criteria", and let that drive further clean up efforts. Maybe it's not a separate repository, but just a trigger to only accept artifacts "marked" in some way. But I don't think just doing new stuff is enough - we need a way to reliably correct existing artifacts, and prepare for the future when the bar moves again. This is the steps I saw towards it: 1) defined rules for what an "acceptable" artifact is 2) gating central with those rules 3) extensible repository metadata support in Maven (this keeps coming up for a variety of reasons) 4) the ability to revise metadata without impacting historical builds (SCMs move, people make mistakes with dependency scopes, we need to rewrite locations, deprecation, rules for "acceptable" change, etc). 5) a way to identify all the artifacts that are marked "acceptable" to have a cleaner build Does that make any sense? - Brett - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
> 1) defined rules for what an "acceptable" artifact is > 2) gating central with those rules Any time when I referred to http://maven.apache.org/guides/mini/guide-central-repository-upload.html as a requirement, I always had a bad feeling. Why? Requirement is what you can verify against, but this mini howto is not a requirement then. Rules need to be defined in software. Could someone write a unit test? > 3) extensible repository metadata support in Maven (this keeps coming up for > a variety of reasons) Extending metadata is good as long as creating/using that metadata is optional, like at #5. > 4) the ability to revise metadata without impacting historical builds (SCMs > move, people make mistakes with dependency scopes, we need to rewrite > locations, deprecation, rules for "acceptable" change, etc). Going back in history of multiple projects cannot be done reliable based on SCMs. It could be done only if attached sources are full and buildable. This would make the repo a kind of SCM: diff only between releases, details hidden, but OK. > 5) a way to identify all the artifacts that are marked "acceptable" to have > a cleaner build How do you mark "acceptable" when it is not always means the same for everyone? I have proposed in the original thread a solution, based on multiple index files, which index files would represent the output of CAs. If I configure in my Maven settings file (by URL) which index file I want to use, my Maven should be blind to any other artifacts on the repo. Allowing AND and OR operations on this cert lists would be a bonus. It is interesting, that the Central itself could store these (cert list) index files packaged in artifacts. Some index files should be for the public some for limited use. As a client of Central give me a choice which one I sign up to. Acceptance criteria cannot be the same for everyone. On Wed, Sep 30, 2009 at 12:20 AM, Brett Porter wrote: > Ok, I did a poor job of expressing what I was getting at. > > On 25/09/2009, at 2:02 PM, Jason van Zyl wrote: > >> >> What's matters is improvement of the process going forward. > > Agreed, I see that as a pre-requisite. > >> As people move forward with improved submissions from projects then the >> older submissions of dubious quality will naturally fall out of use. > > That may not happen in any sort of feasible timeframe. There is some very > old stuff out there that isn't getting new releases in a hurry. As you > traverse a dependency tree, it's nearly impossible not to hit the rough > spots like early commons-logging releases, velocity vs velocity-dep, > plexus:plexus-utils vs org.codehaus.plexus:plexus-utils and so on. > > On 25/09/2009, at 2:11 PM, Brian Fox wrote: > >> Who would _want_ to deploy their stuff >> to the "old" repo? No one. The hurdle to get to the new repo would be >> the same as putting the checks on the current repo for all new >> artifacts. > > I agree - I'm not saying we allow anyone to deploy to the "old" one - but it > would be nice to have a way in Maven to say "only use stuff that meets the > criteria", and let that drive further clean up efforts. > > Maybe it's not a separate repository, but just a trigger to only accept > artifacts "marked" in some way. > > But I don't think just doing new stuff is enough - we need a way to reliably > correct existing artifacts, and prepare for the future when the bar moves > again. > > This is the steps I saw towards it: > 1) defined rules for what an "acceptable" artifact is > 2) gating central with those rules > 3) extensible repository metadata support in Maven (this keeps coming up for > a variety of reasons) > 4) the ability to revise metadata without impacting historical builds (SCMs > move, people make mistakes with dependency scopes, we need to rewrite > locations, deprecation, rules for "acceptable" change, etc). > 5) a way to identify all the artifacts that are marked "acceptable" to have > a cleaner build > > Does that make any sense? > > - Brett > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
> I have proposed in the original thread a solution, based on multiple > index files, which index files would represent the output of CAs. If I > configure in my Maven settings file (by URL) which index file I want > to use, my Maven should be blind to any other artifacts on the repo. > Allowing AND and OR operations on this cert lists would be a bonus. Who exactly is setting up, managing, and paying for the ongoing operation of these CAs? You didn't address this in the original thread, and right now I'm not aware of any existing organizations that would fill this void. Especially not for free. Without address that aspect, the rest of the discussion is not worth much IMO. Wayne - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
Wayne, Who paid Linus to create version 1.0 of the Linux Kernel? If somebody can create a Maven repo crawler as a hobby project, then somebody will. The first Cert List will probably be just the result of some simple tests, not much. But many will follow. We will have to protect somehow our bandwidth very soon! On Thu, Oct 1, 2009 at 4:08 PM, Wayne Fay wrote: >> I have proposed in the original thread a solution, based on multiple >> index files, which index files would represent the output of CAs. If I >> configure in my Maven settings file (by URL) which index file I want >> to use, my Maven should be blind to any other artifacts on the repo. >> Allowing AND and OR operations on this cert lists would be a bonus. > > Who exactly is setting up, managing, and paying for the ongoing > operation of these CAs? You didn't address this in the original > thread, and right now I'm not aware of any existing organizations that > would fill this void. Especially not for free. > > Without address that aspect, the rest of the discussion is not worth much IMO. > > Wayne > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: a cleaned up central repository? (was: Maven Central Repository - Cleanup Efforts)
On Thu, Oct 1, 2009 at 2:24 PM, Albert Kurucz wrote: > Wayne, Who paid Linus to create version 1.0 of the Linux Kernel? > If somebody can create a Maven repo crawler as a hobby project, then > somebody will. The Nexus Indexer already runs on Central, directly on the machine. Crawling the repository will get you banned in no time, I unfortunately have to do this all the time as the bandwidth is certainly not free. See the other thread for how you can help us improve the data, and lets keep the discussion there. - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
