Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On Tue, Mar 27, 2018 at 3:10 PM, Nish Aravamudan wrote: > On Fri, Mar 16, 2018 at 11:39 AM, Nish Aravamudan > wrote: >> On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan >> wrote: >>> On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote: On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan wrote: >> >> >> Once that is done, we will start the 'keep-up' script again, which will spend some time catching up the repositories based upon Launchpad publishes since the mass reimport. >>> >>> This has been started and is churning through roughly 11 days of >>> publishing backlog. >>> Finally, we will then look to bump our phasing to 2% (of main), to approximate the disk usage as the phasing increases. >>> >>> I am looking into this step now. >> >> The keep-up script ... well, caught up, and we successfully increased >> our phasing of main to 2%. We are now increasing again to 10%. > > We successfully went through 10% and 20% of main, and have updated the > default Git repository alias for imported packages to point to the > imported repositories (so if you clone them, we should see minimal > on-disk usage in Launchpad). > > We are now starting a 50% of main run, which I expect to take a while. I'm excited to announce that we have completed our first 100% phasing of main round of imports. We have a handful of failures that are being monitored in https://bugs.launchpad.net/usd-importer/+bug/1754898, but for those packages not in our blacklist, we expect all of them to be imported and kept current now. They are also setup to be the default Git repository for the corresponding source package. Thanks! -Nish > > Thanks, > Nish -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On Fri, Mar 16, 2018 at 11:39 AM, Nish Aravamudan wrote: > On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan > wrote: >> On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote: >>> On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan >>> wrote: > > > >>> Once that is done, we will start the 'keep-up' script again, which >>> will spend some time catching up the repositories based upon Launchpad >>> publishes since the mass reimport. >> >> This has been started and is churning through roughly 11 days of >> publishing backlog. >> >>> Finally, we will then look to bump our phasing to 2% (of main), to >>> approximate the disk usage as the phasing increases. >> >> I am looking into this step now. > > The keep-up script ... well, caught up, and we successfully increased > our phasing of main to 2%. We are now increasing again to 10%. We successfully went through 10% and 20% of main, and have updated the default Git repository alias for imported packages to point to the imported repositories (so if you clone them, we should see minimal on-disk usage in Launchpad). We are now starting a 50% of main run, which I expect to take a while. Thanks, Nish -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On Thu, Mar 15, 2018 at 2:45 PM, Nish Aravamudan wrote: > On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote: >> On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan >> wrote: >> Once that is done, we will start the 'keep-up' script again, which >> will spend some time catching up the repositories based upon Launchpad >> publishes since the mass reimport. > > This has been started and is churning through roughly 11 days of > publishing backlog. > >> Finally, we will then look to bump our phasing to 2% (of main), to >> approximate the disk usage as the phasing increases. > > I am looking into this step now. The keep-up script ... well, caught up, and we successfully increased our phasing of main to 2%. We are now increasing again to 10%. Thanks, Nish -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On 12.03.2018 [11:59:11 -0700], Nish Aravamudan wrote: > On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan > wrote: > > On Thu, Mar 1, 2018 at 5:20 PM, Nish Aravamudan > > wrote: > >> On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote: > >>> Hello all! > >>> > >>> I am happy to announce the release of git-ubuntu 0.7.1 to all channels > >>> today. > >> > >> I encountered some unfortunate issues with snapcraft that hit > >> xenial-updates just as I was building (LP: #1752481) and then realized > >> we needed to snap the archive keyrings in order to verify recent archive > >> files (LP: #1752656). I then made a typo fixing the latter, so we're now > >> at 0.7.3. This should build successfully in all channels shortly. > >> > >> Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek > >> for the keyring suggestion. > > > > Well, it didn't quite go as planned. I found another bug (so we're at > > 0.7.4) and also the snapcraft fix didn't resolve all the issues. So > > I'm temporarily using a PPA that reverts the xenial-updates of > > snapcraft back to 2.35. > > > > Kyle Fazzari is working on a proper fix, but if it's not able to get > > done soon, we might back out the snapcraft SRU in xenial-updates. > > For completeness, we ended up backing out the snapcraft SRU via a > re-upload of 2.35 to xenial-updates. The snapcraft team is working on > fixes to the bugs found. > > Meanwhile, a mass reimport ran last week and the existing whitelist + > 1% of main was reimported. This resulted in 905 successful imports and > 5 failed imports. The latter 5 will presumably be broken for a while, > as we focus on upping our phasing before resolving those issues (LP: > #1754898). I have removed the repositories for the failure cases. They would not update with the normal 'keep the repositories current' anyways, and we would rather have consistent repository contents. > I am working right now on an issue with our default repository > repointing script. This was PEBKAC. > Additionally, roughly 150 manual imports remain in the > ~usd-import-team space, but those source packages were never added to > the whitelist. They need to be reimported, as well (and added to the > whitelist). I am waiting to ensure that there are no pending MPs for > those source packages before reimporting. This was completed this morning. > Once that is done, we will start the 'keep-up' script again, which > will spend some time catching up the repositories based upon Launchpad > publishes since the mass reimport. This has been started and is churning through roughly 11 days of publishing backlog. > Finally, we will then look to bump our phasing to 2% (of main), to > approximate the disk usage as the phasing increases. I am looking into this step now. Thanks, Nish -- Nishanth Aravamudan Ubuntu Server Canonical Ltd -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On Fri, Mar 2, 2018 at 2:00 PM, Nish Aravamudan wrote: > On Thu, Mar 1, 2018 at 5:20 PM, Nish Aravamudan > wrote: >> On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote: >>> Hello all! >>> >>> I am happy to announce the release of git-ubuntu 0.7.1 to all channels >>> today. >> >> I encountered some unfortunate issues with snapcraft that hit >> xenial-updates just as I was building (LP: #1752481) and then realized >> we needed to snap the archive keyrings in order to verify recent archive >> files (LP: #1752656). I then made a typo fixing the latter, so we're now >> at 0.7.3. This should build successfully in all channels shortly. >> >> Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek >> for the keyring suggestion. > > Well, it didn't quite go as planned. I found another bug (so we're at > 0.7.4) and also the snapcraft fix didn't resolve all the issues. So > I'm temporarily using a PPA that reverts the xenial-updates of > snapcraft back to 2.35. > > Kyle Fazzari is working on a proper fix, but if it's not able to get > done soon, we might back out the snapcraft SRU in xenial-updates. For completeness, we ended up backing out the snapcraft SRU via a re-upload of 2.35 to xenial-updates. The snapcraft team is working on fixes to the bugs found. Meanwhile, a mass reimport ran last week and the existing whitelist + 1% of main was reimported. This resulted in 905 successful imports and 5 failed imports. The latter 5 will presumably be broken for a while, as we focus on upping our phasing before resolving those issues (LP: #1754898). I am working right now on an issue with our default repository repointing script. Additionally, roughly 150 manual imports remain in the ~usd-import-team space, but those source packages were never added to the whitelist. They need to be reimported, as well (and added to the whitelist). I am waiting to ensure that there are no pending MPs for those source packages before reimporting. Once that is done, we will start the 'keep-up' script again, which will spend some time catching up the repositories based upon Launchpad publishes since the mass reimport. Finally, we will then look to bump our phasing to 2% (of main), to approximate the disk usage as the phasing increases. Thanks for your patience as we continue to work through this process. -Nish -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
Re: git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On Thu, Mar 1, 2018 at 5:20 PM, Nish Aravamudan wrote: > On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote: >> Hello all! >> >> I am happy to announce the release of git-ubuntu 0.7.1 to all channels >> today. > > I encountered some unfortunate issues with snapcraft that hit > xenial-updates just as I was building (LP: #1752481) and then realized > we needed to snap the archive keyrings in order to verify recent archive > files (LP: #1752656). I then made a typo fixing the latter, so we're now > at 0.7.3. This should build successfully in all channels shortly. > > Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek > for the keyring suggestion. Well, it didn't quite go as planned. I found another bug (so we're at 0.7.4) and also the snapcraft fix didn't resolve all the issues. So I'm temporarily using a PPA that reverts the xenial-updates of snapcraft back to 2.35. Kyle Fazzari is working on a proper fix, but if it's not able to get done soon, we might back out the snapcraft SRU in xenial-updates. -Nish -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
git-ubuntu 0.7.3 in edge/candidate/stable channels [Was Re: git-ubuntu 0.7.1 in edge/candidate/stable channels]
On 28.02.2018 [21:33:38 -0800], Nish Aravamudan wrote: > Hello all! > > I am happy to announce the release of git-ubuntu 0.7.1 to all channels > today. I encountered some unfortunate issues with snapcraft that hit xenial-updates just as I was building (LP: #1752481) and then realized we needed to snap the archive keyrings in order to verify recent archive files (LP: #1752656). I then made a typo fixing the latter, so we're now at 0.7.3. This should build successfully in all channels shortly. Thanks to Sergio Schvezov for the snapcraft assist and Steve Langasek for the keyring suggestion. Thanks, Nish -- Nishanth Aravamudan Ubuntu Server Canonical Ltd -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
git-ubuntu 0.7.1 in edge/candidate/stable channels
Hello all! I am happy to announce the release of git-ubuntu 0.7.1 to all channels today. This is a massive update to our code as well as a refocus on the import side of things. While regressions in other subcommands is not intentional, it is certainly possible right now -- our goal is to get all of main imported, by slowly ramping our phasing (we're aiming for 10% first, to see how it affects Launchpad, etc.). To that end, I'm not going to provide highlights this time around. Almost all of the code has been changed since the last tagged release! We are working on improving code quality, by adding unit tests (thanks to Robie for extensive work in this area). I am about to kick off a reimport of all source packages in our whitelist and 1% of main. Then the default Git repositories for the working set of source packages will be updated to point at our repositories. Finally, I will restart our keep-up job with those same parameters. I will send follow-up e-mails as we get through each of those steps. Because of this reimport, if you have pending MPs against previously imported repositories, they will need to be resubmitted (our importer code should put a comment in appropriately!). What you will need to do (roughly) is: git tag git fetch pkg (which should force update) git rebase --onto HEAD We are effectively transplanting changes we had relative to to now be based upon the updated pkg/ branch. (Note that you do not need to create a tag for this, I just find it handy to give descriptive names for commits; you can just use the commit hash). If you have any questions about this, please let me know. Thanks! -Nish -- Nishanth Aravamudan Ubuntu Server Canonical Ltd -- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
