Hi, i created some ant scripts to do so using tycho based build. see [1]
however, i also discovered that earlier builds using buckminster had incorrectly pointing "p2.mirrorsURL" to a wrong location[2]. i don't know if other projects are affected, but Acceleo, ATL, EMF Compare and EEF were impacted.
hope this will decrease the amount of p2 requests, cheers, Stéphane. [1]http://sbouchet-eef.blogspot.com/2012/03/p2-repositories-we-can-do-better-eef.html [2]https://bugs.eclipse.org/bugs/show_bug.cgi?id=314388#c69 Le 02/03/2012 07:05, David M Williams a écrit :
That is, we can do better at educating people how to make a good p2 repository! We have not done that education well. Hopefully this note, with its links, can help provide the education you need .... then tell your friends! ... tell your family! :) This information actually applies to all projects and all releases at Eclipse, not just the yearly train, but I am sending to cross-project list, thinking that will educate most projects as a first step. This should be considered a requirement of correct and effective use of the Eclipse Foundation Infrastructure so mentors and PMCs should make sure their projects know about it and making effective use of the infrastructure. I think everyone knows about pack200, and while sometimes a pain, it can help reduce bandwidth for faster installs, and I think most projects do a good job of using that. And that is one of the hardest things to do! ... which makes me think the current situation is more about education than amount of effort. There are two things which, as a whole, we are not doing well. A. Use p2.mirrorsURL property in artifact repositories (artifacts.jar/xml files). See bug 368826. There are many reasons for not using it ... such as lack of tools, but another reason is that some of us (e.g. me) thought it was "common knowledge" that you should use it, but of course it is not to projects that are new to Eclipse. In response, I have added a little documentation to the main IT Doc [2], beefed up and corrected some of the community maintained page that describes how to use p2.mirrorsURL [3] and provided some documentation on how to use a custom Eclipse Application to set, change, or remove the p2.mirrorsURL, courtesy of the Webtools Releng project [4]. While an "unofficial" tool, I think most can take advantage of it, as is. Just practice on some test repos first! :) But this type of tool can make it easier to fix existing artifact repositories without hand-editing a 2 Megabyte XML file. But again, practice on some test repos first! You can make a copy of (only) the artifacts.jar file to some /temp directory your local machine and "pretend" that is the artifact repository, to see if it makes the changes as you intend. Then use your scripts on production machine, being sure to copy original artifacts jar to a safe location first in case you need to revert, and be sure to test its really working, afterwards, as is well described on the p2.mirrorsURL wiki[3]. B. Use p2.index at repository locations. p2 likes metafiles for its metafiles! As if content, artifacts, composites, jars and xml were not enough, a while back p2 found the need to use another file, called p2.index to help it figure out a few ambiguous cases. Since few people need this functionality, it has gone ignored for a long time but it is finally becoming noticeable, as Eclipse gets larger and more popular (and, as a victim of its own success, as "p2 update" is working well enough many people use that now, instead of downloading a fresh zip file!). The "lucky accident" is that providing the p2.index file can save p2 a few needless "round trips" to the download server, while it figures out what is there, and what it should use, even if not truly needed to resolve ambiguity. The long gory history is documented in bug 347448 [5]. In response, I have added a little documentation to the main IT Doc [6] and beefed up and corrected the p2.index wiki [7]. The good news is that it is easy to do. For 99% of you (I am guessing) it is simply a matter of copying one of two files to your repository locations ... you do need to pick which goes where (and most of you will need both, one at composite sites, the other at simple sites). The two to pick from (for these frequent, common cases) are documented on the wiki [7] (if you find other common, or, even tricky cases, please add them to the wiki). This is especially important to have for "composite sites" since p2 normally checks for that last, but if the p2.index says that location is a composite site, then that saves p2 several needless "peek attempts" at download server. Note, recent versions of b3 aggregator [8] produce these p2.index files automatically, though the default Equinox p2 publisher does not (bug 302909 [9]) (I do not know about other p2 publishers, if there are any). While all these issues and p2 in general can be made better, faster, easier, and some day even mow your lawn (over time, if someone commits to it) we are where we are, right now. So these are two things we can/should/need to do right now to improve the situation: add p2.index files to repository directories, and add p2.mirrorsURL property to artifact repositories. I know, this is a long note. And even tons more information linked to go off and read for details. Keep this note for reference; bookmark the links. Discuss it with your team. I suspect only one person per project needs to understand it enough to act one it, but the main point is that there are things that can be done right now (over the new few weeks, month) to improve our p2 repositories (even old, existing ones -- the stats say even helios repositories are still getting LOTS of use!) and then we would of course clearly want to continue to do these good practices for Juno and Kepler. Maybe one person per project could spend one day on it within the next 2 to 4 weeks? (And, yes, I know that is asking for a lot of total effort ... but, you will learn a lot of cool things in the process! ... and cleaning up and learning these things now will help us be ready for the Juno release! Doing these things will give our Eclipse users a little better update and install experience. To be clear and honest, this is not gong to be like a 50% improvement, or anything ... I suspect it won't be noticeable most of the time for most users ... but, some users will notice it a lot! if they can get mirrors closer to their corner of the world (via p2.mirrorsURL property), and it will be noticeable by many people during those "peak times" then download.eclipse.org gets flooded by huge numbers of tiny requests for files that will never exist (such as "peeking" for content.jar, when p2 could look more directly for compositeContent.jar, if p2.index was there). [If our webmasters didn't know about p2, they'd probably think it as a DOS attack :) just kidding, I've no idea what a DOS attack looks like, and am pretty sure our IT infrastructure is responding well.] Three years ago, who would have thought p2 would be a victim of its own success?! :) I am sure it can be even more successful and more "enterprise ready", so if anyone is interested, get involved! provide some patches! make some tools! As always, feel free to ask if questions ... and feel free to correct any mistakes I make in my statements or my wiki updates. Much thanks for your time and efforts. p2.mirrorsURL related: [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=368826 [2] http://wiki.eclipse.org/IT_Infrastructure_Doc#Enable_mirrors_.2F_use_mirrorsURL_for_my_p2_repo.3F [3] http://wiki.eclipse.org/Equinox/p2/p2.mirrorsURL [4] http://wiki.eclipse.org/WTP/Releng/Tools/addRepoProperties p2.index related: [5] https://bugs.eclipse.org/bugs/show_bug.cgi?id=347448 [6] http://wiki.eclipse.org/IT_Infrastructure_Doc#Include_a_p2.index_file_at_p2_repository_site.3F [7] http://wiki.eclipse.org/Equinox/p2/p2_index [8] http://wiki.eclipse.org/Eclipse_b3/aggregator/manual [9] https://bugs.eclipse.org/bugs/show_bug.cgi?id=302909 _______________________________________________ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
<<attachment: stephane_bouchet.vcf>>
_______________________________________________ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
