Re: General mechanism to supply "rich history" to git-ubuntu
On Tue, Feb 2, 2021 at 4:56 PM Robie Basak wrote: > > On Tue, Feb 02, 2021 at 10:48:07AM -0500, Scott Moser wrote: > > I think this is great. I think it is a game-changer for ubuntu development. > > Thank you. > > Thanks! I have discussed this with you before and also like it very much - thanks for all your work on this! > > In your example above, the 'refs/heads/test' has to exist (and to have the > > referenced commit) in order for the importer to find it. > > > > So the uploader would then have to not delete or force-push that reference > > until some point in the future, right? > > > > My only concern here is that I already have way too many branches > > that I've pushed. This will just leave more manual maintenance for me > > to not clean up these branches until the upload that references > > them is landed. > > > > Did I undestand that correctly? > > Correct. At the time the importer runs, it needs to be able to fetch a > ref from somewhere. If you push to a personal branch and use that ref, > then it'll need to exist until git-ubuntu has caught up with your > upload. Right now the time varies, is commonly an hour or two, but > sometimes has been a week. Over time, I hope for us to get more reliable > with it (this requires infrastructure, monitoring, etc). > > I'm not really sure how to avoid this requirement. > > Another approach you could take is to maintain a team repository and > branch somewhere. Then all your uploads could be "from" a single main > branch, which would then persist by its nature. The following just came to my mind, I haven't thought it through for hours, so I'm sharing it with some disclaimer attached :-) I think there is a chance if someone has a "next" or "for-upload" branch that it might be recycled and no longer match on import. But IMHO we could synthesize a branch name in the wrapper and push to that one. Example: 1. user works on his branch called "cool-new-feature" (that would change) 2. user decides to upload, and calls the wrapper 3. the wrapper synthesizes a new branch name like --<$date/time> => git-ubuntu-upload-cool-new-feature-03012021085107 and push this one. 4. the synthesized name is referenced in the tags and used by the importer It is unlikely (unless intentional) that this branch is force pushed before the import happens. But it has the downside of a potential proliferation of pushed branches, but TBH providing a "remove all merged branches" command should not be too hard. If we spin this forward (And ACS for it work out) we could push it to a special namespace in the actual per package git ubuntu repositories. And maybe add some UUID to the synthesized branch name (for multi-user de-collision). The ACS would need to allow that no "normal" user can force push, but nearly everyone (with a LP user) can push their branches. In that repository, the importer could do auto-remove-after-import and proliferation would not happen in the users repository anymore. > I guess you could sort of do that with a personal repository, too, if > you wanted. Maintain a "for-upload" branch, always push there, and then > forget about it until you reuse it next time. > -- > ubuntu-devel mailing list > ubuntu-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
[ubuntu-studio-devel] Fwd: Call for testing: 20.04.2 release candidate images ready!
FYI all, this is for Focal (Xfce Desktop) -- Forwarded Message -- Subject: Call for testing: 20.04.2 release candidate images ready! Date: Tuesday, February 2, 2021, 10:19:03 AM PST From: Lukasz Zemczak To: ubuntu-devel , ubuntu-release , Ubuntu Quality Hello everyone! I'm only sending this e-mail now as previously we were still resolving some unexpected issues regarding the .2 point-release. After a rough start, we now have a *hopefully* stable set of 20.04.2 release candidate images built and published on the .2 milestone: http://iso.qa.ubuntu.com/qatracker/milestones/420/builds Please pick your favorite flavor and start testing! And be sure to report your results on the isotracker. As with every recent release, we have a discourse thread for tracking progress which we try to keep up to date as much as possible: https://discourse.ubuntu.com/t/focal-fossa-20-04-2-lts-point-release-status-tracking/ Best regards, -- Łukasz 'sil2100' Zemczak Foundations Team lukasz.zemc...@canonical.com www.canonical.com -- ubuntu-devel mailing list ubuntu-de...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ ubuntu-devel - signature.asc Description: This is a digitally signed message part. -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
Call for testing: 20.04.2 release candidate images ready!
Hello everyone! I'm only sending this e-mail now as previously we were still resolving some unexpected issues regarding the .2 point-release. After a rough start, we now have a *hopefully* stable set of 20.04.2 release candidate images built and published on the .2 milestone: http://iso.qa.ubuntu.com/qatracker/milestones/420/builds Please pick your favorite flavor and start testing! And be sure to report your results on the isotracker. As with every recent release, we have a discourse thread for tracking progress which we try to keep up to date as much as possible: https://discourse.ubuntu.com/t/focal-fossa-20-04-2-lts-point-release-status-tracking/ Best regards, -- Łukasz 'sil2100' Zemczak Foundations Team lukasz.zemc...@canonical.com www.canonical.com -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: General mechanism to supply "rich history" to git-ubuntu
On Tue, Feb 02, 2021 at 10:48:07AM -0500, Scott Moser wrote: > I think this is great. I think it is a game-changer for ubuntu development. > Thank you. Thanks! > In your example above, the 'refs/heads/test' has to exist (and to have the > referenced commit) in order for the importer to find it. > > So the uploader would then have to not delete or force-push that reference > until some point in the future, right? > > My only concern here is that I already have way too many branches > that I've pushed. This will just leave more manual maintenance for me > to not clean up these branches until the upload that references > them is landed. > > Did I undestand that correctly? Correct. At the time the importer runs, it needs to be able to fetch a ref from somewhere. If you push to a personal branch and use that ref, then it'll need to exist until git-ubuntu has caught up with your upload. Right now the time varies, is commonly an hour or two, but sometimes has been a week. Over time, I hope for us to get more reliable with it (this requires infrastructure, monitoring, etc). I'm not really sure how to avoid this requirement. Another approach you could take is to maintain a team repository and branch somewhere. Then all your uploads could be "from" a single main branch, which would then persist by its nature. I guess you could sort of do that with a personal repository, too, if you wanted. Maintain a "for-upload" branch, always push there, and then forget about it until you reuse it next time. signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: General mechanism to supply "rich history" to git-ubuntu
I think this is great. I think it is a game-changer for ubuntu development. Thank you. One thought inline. > > 5) If the commits pass sanity checks (eg. the final commit matches the > upload exactly), then it uses the commits provided instead of > synthesizing its own. > > What goes into the changes file is three fields. Example: > https://launchpadlibrarian.net/516799033/hello_2.10-2ubuntu3~ppa1_source.changes > > Vcs-Git: https://git.launchpad.net/~racb/ubuntu/+source/hello > Vcs-Git-Commit: 4511fdfc01cbfd5bc351e1da294d6acb44e8a4a2 > Vcs-Git-Refs: refs/heads/test > > We need the Refs field because git is designed not to be able to fetch a > commit by hash, but by a ref that can reach it only. So Vcs-Git-Refs > must specify what ref(s), when fetched will make the commit given to be > reachable. In practice this could just be the branch name prefixed by > 'refs/heads/' as in this example. In your example above, the 'refs/heads/test' has to exist (and to have the referenced commit) in order for the importer to find it. So the uploader would then have to not delete or force-push that reference until some point in the future, right? My only concern here is that I already have way too many branches that I've pushed. This will just leave more manual maintenance for me to not clean up these branches until the upload that references them is landed. Did I undestand that correctly? -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
General mechanism to supply "rich history" to git-ubuntu
Dear Ubuntu developers, I have an experimental branch ready[0] that would allow any Ubuntu uploader to provide git commits to the git-ubuntu importer, such that it would use those commits instead of synthesizing its own. This way, the git-ubuntu will always reflect "the truth" of Launchpad publications, but also include individual git commits leading to each upload where the uploaders have provided them. git-ubuntu already supports this through use of "upload tags", but this doesn't scale, so this experimental branch is an effort to make it better and available to everyone. Question: before I land this branch, I'd like to make sure that there aren't any issues with the "spec", as follows. For (developer) users, you'd run something like: dpkg-buildpackage $(git ubuntu push-for-upload) The push-for-upload[1] command would push your branch to Launchpad and also output some -Dfield=value arguments that would get passed through to dpkg-genchanges. Further/better wrappers could come later - especially for newcomers where I'd like to wrap away the dpkg-buildpackage stuff. Technically, the mechanism is: 1) The uploader pushes their commits somewhere. 2) The uploader includes a reference to the commits in the changes file. 3) The uploader dputs as normal. 4) When git-ubuntu sees the upload, it pulls the commits from the repository listed in the changes file. 5) If the commits pass sanity checks (eg. the final commit matches the upload exactly), then it uses the commits provided instead of synthesizing its own. What goes into the changes file is three fields. Example: https://launchpadlibrarian.net/516799033/hello_2.10-2ubuntu3~ppa1_source.changes Vcs-Git: https://git.launchpad.net/~racb/ubuntu/+source/hello Vcs-Git-Commit: 4511fdfc01cbfd5bc351e1da294d6acb44e8a4a2 Vcs-Git-Refs: refs/heads/test We need the Refs field because git is designed not to be able to fetch a commit by hash, but by a ref that can reach it only. So Vcs-Git-Refs must specify what ref(s), when fetched will make the commit given to be reachable. In practice this could just be the branch name prefixed by 'refs/heads/' as in this example. Some thoughts: I used Vcs-Git since this is already well-defined for debian/control files. For now, my branch restricts the acceptable Vcs-Git URL to Launchpad and https, because this is the MVP for this use case and the more general untrusted case is harder to correctly validate the input from. Notably there's a Dgit field defined by Debian Policy against dsc files, which is used for a very similar purpose[2]. I chose the changes file because it seemed to me that this metadata is more to do with the action of uploading - especially since there's no guarantee that the git repository, refs and commit will continue to be available in the future. Long term, the correct repository and commit to be associated with the source package would be the git-ubuntu repository into which it gets imported, not the uploader's repository. But the Dgit field goes in the dsc file. Maybe it makes more sense there for Dgit, because it's better defined in Dgit-land that the commit specified will be expected to exist indefinitely. I've used generic field names because it made sense to me for the changes file to be able to generally specify "this git commit is the one that I'm actually uploading here". But I haven't conveyed what the uploader "intends" to be done with the field. It might not be right in the general case for git-ubuntu to always assume that the uploader wants the commit supplied to be incorporated. So do I need another field to specify intent, or is that over-engineering it? The general field spec might be something to collaborate with Debian on, and get into the spec, same as Dgit. The way I designed it, it expands and could incorporate/replace the Dgit field. But questions on dsc vs. changes and the intent, as I mention above, would probably need to be resolved first. And perhaps this is more than is practical right now anyway. Feedback appreciated. Robie [0] https://code.launchpad.net/~racb/usd-importer/+git/usd-importer/+merge/396756 [1] Not yet implemented. I'm not sure of the best name for this subcommand; suggestions appreciated. [2] https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-dgit signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Phased updates in APT in 21.04
(this is a copy of https://discourse.ubuntu.com/t/phased-updates-in-apt-in-21-04/20345 for everyone's attention) Starting with this development release, with APT 2.1.16 to be precise, APT now implements phased updates. Previously, only update-manager implemented phased updates, and this was only used on desktops - the implementation in APT means this works on servers, raspberry pis, and containers, too. This means that some updates will be hold back on some machines while they are being phased. This is being decided by a value derived from the machine-id, the package name, and package version. # To control phased updates in APT: * To have all your machines phase the same, set the `APT::Machine-ID`option to a UUID like /etc/machine-id (the format is not being checked right now, but that might change); or point `Dir::Etc::machine-id` to another machine-id file. * To always include phased updates, set `APT::Get::Always-Include-Phased-Updates` to true. This is the old behavior. * To never include phased updates, set `APT::Get::Never-Include-Phased-Updates` to false. Note that this does not apply to fresh package installs. APT also respects the options from update-manager about phased updates which are similar to the ones above. This also applies to any tool using the APT library, such as aptitude and packagekit. # Special notes: * To avoid breaking existing build chroots, chroots are excluded from this change for now (as are systems without an `/etc/machine-id`). This may change in 21.10. * The `/etc/machine-id` file is looked up relatively to `Dir::Etc` setting, which is `/etc/apt`. So if you point `Dir` or `Dir::Etc` elsewhere, you might have to copy `machine-id` to the right place or set that option to `/etc/machine-id`, otherwise phasing will be disabled, and phased updates always included. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
