You guys weren't very clear which repository are reviewing for the donation.
Initially I converted releases/ but then I released I have some commits of mine in there so I couldn't push it to github. So I did a fresh conversion for main-silver/ which I assumed is the repository Oracle will donate. I will re-do releases/ but it will take a bit. --emi On Tue, Dec 20, 2016 at 3:31 PM, Jaroslav Tulach <[email protected] > wrote: > On pátek 9. prosince 2016 19:05:48 CET Emilian Bold wrote: > > Martin, I have just pushed https://github.com/emilianbold/main-silver > You > > may experiment with that. > > Hello Emilian, > I managed to fork & use your repository and everything seems great. I have > a > functional job that executes > > $ ant build-platform > > for each pull request. I plan to add a call to "ant test-platform" once it > is > stable enough[1]. Great work! I believe we shall use your Git repository > as a > base (somehow) when donating code to Apache once my Oracle peers finish > review > of the code to donate. > > > To https://github.com/emilianbold/main-silver.git > > * [new branch] master -> master > > Branch master set up to track remote branch master from origin. > > Could you synchronize https://hg.netbeans.org/releases/ instead? It > contains > history of all the NetBeans releases (in branches like release82, etc.) > and it > is the repository that is currently under the review. Btw. The releases > repository contains everything that is available in the main-silver - just > more. > > It would be fantastic, if you could create a complete mirror of the > releases > repository. Thanks again for your great work! > > -jt > > [1] http://deadlock.netbeans.org/job/prototypes-MavenDownload269264/ - > still > 12 test failures remaining > > > git push -u origin master > > Counting objects: 3951610, done. > > Delta compression using up to 8 threads. > > Compressing objects: 100% (732965/732965), done. > > Writing objects: 100% (3951610/3951610), 674.94 MiB | 717.00 KiB/s, done. > > Total 3951610 (delta 2068729), reused 3951610 (delta 2068729) > > remote: Resolving deltas: 100% (2068729/2068729), done. > > remote: Checking connectivity: 3951610, done. > > remote: warning: GH001: Large files detected. You may want to try Git > Large > > File Storage - https://git-lfs.github.com. > > remote: warning: See http://git.io/iEPt8g for more information. > > remote: warning: File dlight.util/test/manual/DLight_Simple_Tests/core > is > > 51.88 MB; this is larger than GitHub's recommended maximum file size of > > 50.00 MB > > To https://github.com/emilianbold/main-silver.git > > * [new branch] master -> master > > Branch master set up to track remote branch master from origin. > > > > > > --emi > > > > On Wed, Dec 7, 2016 at 4:42 PM, Martin Balin <[email protected]> > > > > wrote: > > > Hello Emilian, > > > I'm working at Oracle on NetBeans development and we would like to > start > > > fixing build scripts to use Git instead of HG. > > > This could be done earlier on your Git repo if you agree to as it will > > > take time. Does not need to wait for final official donation of > sources. > > > Can you please send me the URL,... > > > Thank you Martin Balin > > > > > > On 24.11.2016 20:07, Emilian Bold wrote: > > >> At under 1GB the repository size is not an issue anymore. > > >> > > >> It's sad to see we will still have migration problems due to legal > > >> considerations. > > >> > > >> Could you provide an estimate how long it would take to verify and > > >> whitelist the entire codebase Oracle plans on donating? > > >> > > >> It's unclear to me how history would be preserved with an incremental > > >> approach. > > >> > > >> I would prefer we migrate the whole thing in one piece with history > and > > >> all. > > >> > > >> > > >> --emi > > >> > > >> On Thu, Nov 24, 2016 at 5:22 PM, Jaroslav Tulach < > > >> [email protected] > > >> > > >>> wrote: > > >>> Emilian, Jan, Mark, great work. > > >>> > > >>> Smooth migration from Hg to Git is essential for successful > migration to > > >>> Apache. Thanks a lot for investigating how to do that. > > >>> > > >>> My plan (as described in another email) is to prepare the code > donation > > >>> in > > >>> Hg > > >>> and update it incrementally with code integrated into Hg. > > >>> > > >>> Are your conversions methods ready for incremental updates or do they > > >>> only > > >>> work as a one-time batch conversion? > > >>> > > >>> -jt > > >>> > > >>> On čtvrtek 24. listopadu 2016 10:41:50 CET Jan Lahoda wrote: > > >>>> Interesting. I tried "git gc --aggressive" on the Mark's converted > > >>>> repository, and the result is: > > >>>> netbeans-import/.git$ du -hs . > > >>>> 792M . > > >>>> > > >>>> The original was: > > >>>> netbeans-import.git $ du -hs . > > >>>> 3,5G . > > >>>> > > >>>> (IIRC Mark was converting http://hg.netbeans.org/main, not > releases, so > > >>> > > >>> the > > >>> > > >>>> repository is a little bit smaller than the releases one.) > > >>>> > > >>>> I tried: > > >>>> $ git log -p | sha1sum > > >>>> > > >>>> on both repositories, and the hashes appear to be the same. I also > > >>>> tried > > >>> > > >>> to > > >>> > > >>>> clone the gc-ed repository using git clone --bare --no-local, and > the > > >>>> resulting repository is still about the same size. So, this seems > good > > >>>> to > > >>>> me, unless there is some downside I don't know about. > > >>>> > > >>>> Jan > > >>>> > > >>>> > > >>>> On Wed, Nov 23, 2016 at 8:26 PM, Emilian Bold < > [email protected]> > > >>>> > > >>>> wrote: > > >>>>> Actually I don't believe the data loss is that large. (There may > also > > >>>> > > >>>> be > > >>>> > > >>>> mercurial commits that are intentionally ignored by the conversion > > >>>> > > >>>> script, > > >>>> > > >>>> like commits that only add tags?) > > >>>> > > >>>>> hg log | grep '^changeset:' | wc -l > > >>>>> > > >>>>> 313209 > > >>>>> > > >>>>> git log | grep '^commit ' | wc -l > > >>>>> > > >>>>> 301478 > > >>>>> > > >>>>> So there is a difference of 11731 commits (about 4%) but those > > >>>>> couldn't > > >>>>> have such a large impact on repository size. > > >>>>> > > >>>>> I hope somebody else is willing to work with me on this so we > document > > >>>>> everything and do a reproducible repository conversion. > > >>>>> > > >>>>> > > >>>>> > > >>>>> --emi > > >>>>> > > >>>>> On Wed, Nov 23, 2016 at 9:10 PM, Emilian Bold < > [email protected]> > > >>>>> > > >>>>> wrote: > > >>>>>> Well, I dunno what black magic `gc --aggressive` does but the > > >>>>> > > >>>>> repository > > >>>> > > >>>> is 0.85GB now! > > >>>> > > >>>>>> I also ran `git reflog expire` first but it didn't change the > size at > > >>>>> > > >>>>> all. > > >>>>> > > >>>>> One thing to keep in mind is that I used --force although I had 6 > > >>>>> > > >>>>>> commits > > >>>>>> with the warning "repository has at least one unnamed head". Which > > >>>>> > > >>>>> were > > >>>> > > >>>> probably all close branch commits (hg commit --close-branch). > > >>>> > > >>>>>> So I might have have data loss(!) since I believe I read > > >>>>> > > >>>>> hg-fast-export.sh > > >>>>> > > >>>>> picks only one unnamed head as the migration winner. I wonder if > the > > >>>>> > > >>>>> gc > > >>>> > > >>>> command didn't just purge a lot of valid commits from such an > unnamed > > >>>> > > >>>>> head > > >>>>> > > >>>>> and that's why the repository became so small. > > >>>>> > > >>>>>> Could somebody else try a test repository conversion and validate > my > > >>>>>> numbers? > > >>>>>> > > >>>>>> git gc --aggressive --prune=now > > >>>>>> Counting objects: 4085031, done. > > >>>>>> Delta compression using up to 8 threads. > > >>>>>> Compressing objects: 100% (2909203/2909203), done. > > >>>>>> Writing objects: 100% (4085031/4085031), done. > > >>>>>> Total 4085031 (delta 2150468), reused 1585934 (delta 0) > > >>>>>> Checking connectivity: 4085031, done. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> --emi > > >>>>>> > > >>>>>> On Wed, Nov 23, 2016 at 7:59 PM, Paul Merlin < > [email protected]> > > >>>>>> > > >>>>>> wrote: > > >>>>>>> Hi Emilian, > > >>>>>>> > > >>>>>>> I see hg-fast-export.sh finished at some point. > > >>>>>>> > > >>>>>>>> As expected though, git does not have any of the disk space > gains. > > >>>>>>>> The > > >>>>>>>> converted git releases/ repository is 3.6GB. > > >>>>>>> > > >>>>>>> Just a thought. > > >>>>>>> Did you try some git cleanups after the conversion? > > >>>>>>> > > >>>>>>> git reflog expire --expire=now --all > > >>>>>>> git gc --aggressive --prune=now > > >>>>>>> > > >>>>>>> Cheers > > >>>>>>> > > >>>>>>> In case these statistics mean something: > > >>>>>>>> git-fast-import statistics: > > >>>>>>>> ------------------------------------------------------------ > > >>>>>>> > > >>>>>>> --------- > > >>>> > > >>>> Alloc'd objects: 4090000 > > >>>> > > >>>>>>>> Total objects: 4085509 ( 40220100 duplicates > > >>>>>>>> > > >>>>>>> ) > > >>>>>>> > > >>>> blobs : 1036365 ( 28386238 duplicates 858087 > > >>>>>>> > > >>>>>>> deltas > > >>>> > > >>>> of > > >>>> > > >>>>> 969684 attempts) > > >>>>> > > >>>>>>>> trees : 2735935 ( 11833862 duplicates 1370606 > > >>>>>>> > > >>>>>>> deltas > > >>>> > > >>>> of > > >>>> > > >>>>> 2613480 attempts) > > >>>>> > > >>>>>>>> commits: 313209 ( 0 duplicates 0 > > >>>>>>> > > >>>>>>> deltas > > >>>> > > >>>> of > > >>>> > > >>>>> 0 attempts) > > >>>>> > > >>>>>>>> tags : 0 ( 0 duplicates 0 > > >>>>>>> > > >>>>>>> deltas > > >>>> > > >>>> of > > >>>> > > >>>>> 0 attempts) > > >>>>>>>> > > >>>>>>>> Total branches: 1283 ( 346 loads ) > > >>>>>>>> > > >>>>>>>> marks: 1048576 ( 313209 unique ) > > >>>>>>>> atoms: 124011 > > >>>>>>>> > > >>>>>>>> Memory total: 218429 KiB > > >>>>>>>> > > >>>>>>>> pools: 26711 KiB > > >>>>>>>> > > >>>>>>>> objects: 191718 KiB > > >>>>>>>> > > >>>>>>>> ------------------------------------------------------------ > > >>>>>>> > > >>>>>>> --------- > > >>>> > > >>>> pack_report: getpagesize() = 4096 > > >>>> > > >>>>>>>> pack_report: core.packedGitWindowSize = 1073741824 > > >>>>>>>> pack_report: core.packedGitLimit = 8589934592 > > >>>>>>>> pack_report: pack_used_ctr = 39000045 > > >>>>>>>> pack_report: pack_mmap_calls = 733040 > > >>>>>>>> pack_report: pack_open_windows = 4 / 7 > > >>>>>>>> pack_report: pack_mapped = 4280730006 / 6950823920 > > >>>>>>>> ------------------------------------------------------------ > > >>>>>>> > > >>>>>>> --------- > > >>>>>>> > > >>>>>>>> --emi > > >>>>>>>> > > >>>>>>>> On Fri, Nov 18, 2016 at 1:32 PM, Emilian Bold < > > >>>>>>> > > >>>>>>> [email protected] > > >>>> > > >>>> wrote: > > >>>>>>>>> A releases/ clone which on my system takes 3.8GB is reduced to > > >>>>>>>> > > >>>>>>>> 1.6GB > > >>>> > > >>>> with > > >>>> > > >>>>>>> the generaldelta and aggressivemergedeltas flags (took about 14 > > >>>>>>> > > >>>>>>>> hours). > > >>>>> > > >>>>> Pretty impressive! > > >>>>> > > >>>>>>>>> Converting to git with hg-fast-export.sh complains that > > >>>>>>>> > > >>>>>>>> "repository > > >>>> > > >>>> has at > > >>>> > > >>>>>>> least one unnamed head" for about 6 revisions. With --force I'm > > >>>>>>> > > >>>>>>>> able > > >>>> > > >>>> to > > >>>> > > >>>>> start the conversion but it hasn't finished yet. > > >>>>> > > >>>>>>>>> The git conversion is about 35% done and already using 1.3GB. > > >>>>>>>>> > > >>>>>>>>> So... I assume it's going to need just like the original > > >>>>>>>> > > >>>>>>>> repository > > >>>> > > >>>> about > > >>>> > > >>>>>>> 3.8GB. > > >>>>>>> > > >>>>>>>>> I wonder if git has similar space-saving tricks? > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> --emi > > >>>>>>>>> > > >>>>>>>>> On Thu, Nov 17, 2016 at 8:46 AM, Emilian Bold < > > >>>>>>>> > > >>>>>>>> [email protected]> > > >>>>> > > >>>>> wrote: > > >>>>>>>>>> Forgot about this. I've just started the Mercurial repository > > >>>>>>>>> > > >>>>>>>>> conversion > > >>>>>>> > > >>>>>>> which will take a few hours. > > >>>>>>> > > >>>>>>>>>> Will report tomorrow or when it's done. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> --emi > > >>>>>>>>>> > > >>>>>>>>>> On Wed, Nov 16, 2016 at 11:18 PM, cowwoc < > > >>>>>>>>> > > >>>>>>>>> [email protected]> > > >>>> > > >>>> wrote: > > >>>>>>>> Hi Emilian, > > >>>>>>>> > > >>>>>>>>>>> Any update on this? > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks, > > >>>>>>>>>>> Gili > > >>>>>>>>>>> > > >>>>>>>>>>> On 2016-11-11 01:33 (-0500), Emilian Bold <[email protected]> > > >>>>>>>>>> > > >>>>>>>>>> wrote: > > >>>> Thank you for following through with this after we talked on > > >>>> > > >>>>>>>>>>>> IRC.> > > >>>>>>>>>>>> > > >>>>>>>>>>>> I will check later the size reduction for the releases/ > repo.> > > >
