Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi, On 06-06-2024 8:54 a.m., Helmut Grohne wrote: I think this is fine. Doing final checks then uploading it all. For the record, the set of base-files, bash, dash, glibc and util-linux migrated yesterday in the 16:00 UTC britney2 run after some hinting from the Release Team side (me). Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Aurelien, Trimmed Cc list for glibc matters. On Wed, Jun 05, 2024 at 11:16:50PM +0200, Aurelien Jarno wrote: > Oops, I was convinced that #1071462 was filled with severity serious. > Nevermind. It migrated anyway. :) > Ok, a simultaneous experimental NMU sounds good. Ok, will do with a bit of delay. > Note however that people often downgrade glibc when they have suspicions > (like in the fakeroot case above), or even the hard way with dpkg once > their system is broken. Therefore we should at least test that case to > see how much breakage to expect, and also to easily spot patterns where > a system went through a glibc downgrade possibly followed by an upgrade. Thanks for the pointer. I tested a downgrade. It seems to work without loosing files, but two protective diversions remain in place. One is issued by libc6 against itself (such that it is immune to its own diversion) and the other is issued on behalf of base-files and properly owned by base-files after upgrading base-files. So the downgrade situation is not exactly the situation before the downgrade, but close enough, it works and after upgrading again it's all as it should be. I think this is fine. Doing final checks then uploading it all. Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Helmut On 2024-06-05 14:21, Helmut Grohne wrote: > Hi Aurelien, > > On Tue, Jun 04, 2024 at 06:58:00PM +0200, Aurelien Jarno wrote: > > It would be really great if glibc can migrate before as it fixes an RC > > bug. That said there are suspicions that it introduced bug #1072521, so > > it might be worth investigating before it gets pushed into testing. > > Unfortunately, on my side, I am unable to reproduce the issue. > > I looked and couldn't spot the RC bug you are referring to. The highest > severity one I could spot is the systemd/debianutils/sbuild where > telinit kills the build, but that isn't RC. Am I missing something? Oops, I was convinced that #1071462 was filled with severity serious. Nevermind. > I also looked at #1072521 and am fairly certain that it is not a glibc i> regression. For sure, #1072521 is not trivially reproducible. In fact, I > am not aware of anyone other than the submitter reproducing it. Beyond > that, it is very likely that the submitter uses a non-default file > descriptor soft limit (even though raising it does not reproduce the > problem). > > In general, I agree with glibc migrating before doing the synced upload > and that's also what Paul asked for. From my side the urgency here is > risk management. Doing it later makes it harder for me to provide > resources to fix fallout and I'm trying to find a good balance. > Given that both util-linux and glibc need to migrate, deferring to > Thursday (still leaves time until the Sunday cron) or even Monday looks > most reasonable to me. Ok, thanks. > > Please go ahead with a NMU. I wonder about experimental, do we want to > > do the changes at the same time, or a bit after? Said otherwise is > > moving files from /usr to /lib supported? > > I don't want to support such moves in any way. Hence, I think the best > option would be to simultaneously NMU experimental. dash is affected in > the same way. Ok, a simultaneous experimental NMU sounds good. Note however that people often downgrade glibc when they have suspicions (like in the fakeroot case above), or even the hard way with dpkg once their system is broken. Therefore we should at least test that case to see how much breakage to expect, and also to easily spot patterns where a system went through a glibc downgrade possibly followed by an upgrade. But that's not a reason to block the upload. Regards Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://aurel32.net
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Aurelien, On Tue, Jun 04, 2024 at 06:58:00PM +0200, Aurelien Jarno wrote: > It would be really great if glibc can migrate before as it fixes an RC > bug. That said there are suspicions that it introduced bug #1072521, so > it might be worth investigating before it gets pushed into testing. > Unfortunately, on my side, I am unable to reproduce the issue. I looked and couldn't spot the RC bug you are referring to. The highest severity one I could spot is the systemd/debianutils/sbuild where telinit kills the build, but that isn't RC. Am I missing something? I also looked at #1072521 and am fairly certain that it is not a glibc regression. For sure, #1072521 is not trivially reproducible. In fact, I am not aware of anyone other than the submitter reproducing it. Beyond that, it is very likely that the submitter uses a non-default file descriptor soft limit (even though raising it does not reproduce the problem). In general, I agree with glibc migrating before doing the synced upload and that's also what Paul asked for. From my side the urgency here is risk management. Doing it later makes it harder for me to provide resources to fix fallout and I'm trying to find a good balance. Given that both util-linux and glibc need to migrate, deferring to Thursday (still leaves time until the Sunday cron) or even Monday looks most reasonable to me. > Please go ahead with a NMU. I wonder about experimental, do we want to > do the changes at the same time, or a bit after? Said otherwise is > moving files from /usr to /lib supported? I don't want to support such moves in any way. Hence, I think the best option would be to simultaneously NMU experimental. dash is affected in the same way. Admittedly, I'm not too worried about experimental upgrades failing. We have a number of packages that move files the wrong way in experimental and it doesn't seem to bother people (nor me). Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi, On 2024-06-03 23:10, Helmut Grohne wrote: > On Wed, May 29, 2024 at 03:14:59PM +0200, Helmut Grohne wrote: > > Since noble includes these changes and I'd get this done sooner rather > > than later, how about moving forward with June 5th after 22:30 UTC (such > > that all buildds have regenerated their chroots before the upload)? > > I got vaguely positive feedback from Paul Gevers on this date. Hence, I > plan to upload after the June 5th mirror push and allocate time for > handling unexpected fallout. > > dash, base-files and bash are fully migrated at the time of this > writing. glibc migrated -11 and -12 still has 5 autopkgtest regressions. > util-linux migrated -6, -7 has a piuparts regression and -8 hopefully > gets tested soon. I hope that both migrate before the planned upload and > will consult with the release team on whether to bump back or go ahead. It would be really great if glibc can migrate before as it fixes an RC bug. That said there are suspicions that it introduced bug #1072521, so it might be worth investigating before it gets pushed into testing. Unfortunately, on my side, I am unable to reproduce the issue. > I have rebased and retested the patches in > https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. > > Andrew, Aurelien, Chris, Matthias, Santiago: Any objections? For glibc no objection. > You may send me signed uploads (.dsc + source chanages) and I will > otherwise move ahead with regular NMUs. If you send signed changes, I > recommend encrypting them using my gpg key. Please go ahead with a NMU. I wonder about experimental, do we want to do the changes at the same time, or a bit after? Said otherwise is moving files from /usr to /lib supported? Regards Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://aurel32.net
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
El 3/6/24 a las 23:10, Helmut Grohne escribió: On Wed, May 29, 2024 at 03:14:59PM +0200, Helmut Grohne wrote: Since noble includes these changes and I'd get this done sooner rather than later, how about moving forward with June 5th after 22:30 UTC (such that all buildds have regenerated their chroots before the upload)? I got vaguely positive feedback from Paul Gevers on this date. Hence, I plan to upload after the June 5th mirror push and allocate time for handling unexpected fallout. dash, base-files and bash are fully migrated at the time of this writing. glibc migrated -11 and -12 still has 5 autopkgtest regressions. util-linux migrated -6, -7 has a piuparts regression and -8 hopefully gets tested soon. I hope that both migrate before the planned upload and will consult with the release team on whether to bump back or go ahead. I have rebased and retested the patches in https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. Andrew, Aurelien, Chris, Matthias, Santiago: Any objections? For base-files, please use branch "wip-202406" here: git clone https://salsa.debian.org/sanvila/base-files-not-yet/ (I've just made such branch the default) and upload as is. This will avoid sending signed files. Thanks.
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
* Helmut Grohne [240603 23:11]: > On Wed, May 29, 2024 at 03:14:59PM +0200, Helmut Grohne wrote: > > Since noble includes these changes and I'd get this done sooner rather > > than later, how about moving forward with June 5th after 22:30 UTC (such > > that all buildds have regenerated their chroots before the upload)? > > I got vaguely positive feedback from Paul Gevers on this date. Hence, I > plan to upload after the June 5th mirror push and allocate time for > handling unexpected fallout. > > dash, base-files and bash are fully migrated at the time of this > writing. glibc migrated -11 and -12 still has 5 autopkgtest regressions. > util-linux migrated -6, -7 has a piuparts regression and -8 hopefully > gets tested soon. I hope that both migrate before the planned upload and > will consult with the release team on whether to bump back or go ahead. > > I have rebased and retested the patches in > https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. > > Andrew, Aurelien, Chris, Matthias, Santiago: Any objections? For util-linux: no objections. > You may send me signed uploads (.dsc + source chanages) and I will > otherwise move ahead with regular NMUs. If you send signed changes, I > recommend encrypting them using my gpg key. Please NMU util-linux, to cut short any waiting time. Chris signature.asc Description: PGP signature
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
On Wed, May 29, 2024 at 03:14:59PM +0200, Helmut Grohne wrote: > Since noble includes these changes and I'd get this done sooner rather > than later, how about moving forward with June 5th after 22:30 UTC (such > that all buildds have regenerated their chroots before the upload)? I got vaguely positive feedback from Paul Gevers on this date. Hence, I plan to upload after the June 5th mirror push and allocate time for handling unexpected fallout. dash, base-files and bash are fully migrated at the time of this writing. glibc migrated -11 and -12 still has 5 autopkgtest regressions. util-linux migrated -6, -7 has a piuparts regression and -8 hopefully gets tested soon. I hope that both migrate before the planned upload and will consult with the release team on whether to bump back or go ahead. I have rebased and retested the patches in https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. Andrew, Aurelien, Chris, Matthias, Santiago: Any objections? You may send me signed uploads (.dsc + source chanages) and I will otherwise move ahead with regular NMUs. If you send signed changes, I recommend encrypting them using my gpg key. Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi release team, On Wed, Mar 13, 2024 at 10:55:09AM +0100, Helmut Grohne wrote: > I propose April 11th on the condition that all relevant packages > (including cryptsetup and e2fsprogs) have migrated. I'll also check with > Aurelien on feasibility after Easter. Stuff wasn't ready back then and we kept bumping the /usr-move back. I think we need to fix dates soon. During the past months, I reserved time for the essential fallout and I cannot promise that in second-half-June, July and first-half-August. Meanwhile all the relevant stuff has migrated including the glibc upstream release. So we basically have the following options: * Pick a date before June 7th and go ahead soon. * Go for later and I'll handle the fallout best-effort. (i.e. similar t64 fallout) * Pick a date August 12th or later. Since noble includes these changes and I'd get this done sooner rather than later, how about moving forward with June 5th after 22:30 UTC (such that all buildds have regenerated their chroots before the upload)? There also is little to be done from the release team's side beyond installing a block for base-files, bash, dash, glibc, util-linux and then lifting the block once all of them are ready turn that block into a force hint such that they really migrate together. (There is no dependency relation between them ensuring concurrent migration as that would make upgrades more difficult.) Other than the bootstrap matter, we're down to 350 affected packages and dumat is finding one to five issues per week most of which are undeclared file conflicts. As a result of the dumat work, I haven't seen an actual end user report about a /usr-move problem in quite a while. A list of affected packages is at https://subdivi.de/~helmut/usrmove.ddlist (not entirely current). I'll propose a MBF for the remaining no-change sourceful uploads as well as the remaining "Build-Depends: dh-sequence-movetousr" uploads. All other moves already have bugs. I also plan to bump the severity of these bugs to important soon. Do you also agree with bumping these bugs to serious severity once their number goes below 200? Keep in mind that they are all actionable in the sense that one of the following applies: * No-change sourceful upload fixes * An upload adding "Build-Depends: dh-sequence-movetousr" fixes * The attached patch fixes * Rarely maintainer feedback has been requested Chris has already performed a number of NMUs and we'll need to do some more to eventually get us down to 0 aliased packages. A while ago I removed 10 affected source packages that were FTBFSing for two stable releases already. Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi, On Sat, Mar 09, 2024 at 09:50:11PM +0100, Sebastian Ramacher wrote: > > I'd now like to coordinate a time of upload. Given that chroots are > > rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning > > for the actual upload. If I unexpectedly break stuff, I still have a few > > days to fix. How about March 21st? > > If and only if time64_t is done by then. Adding more changes where > transition has to be coordinated to the ongoing mega transition does not > help. Aurelien also said that glibc doesn't really build at this time. Furthermore, cryptsetup needs to migrate to testing before the upload and cryptsetup -> openssl -> dpkg is also entangled in time64. I propose April 11th on the condition that all relevant packages (including cryptsetup and e2fsprogs) have migrated. I'll also check with Aurelien on feasibility after Easter. Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
On Sat, Mar 09, 2024 at 09:39:50PM +0100, Helmut Grohne wrote: > Hi release team and essential maintainers, > > On Mon, Sep 04, 2023 at 10:33:54PM +0200, Helmut Grohne wrote: > > Once these issues have been resolved, we can move most files except for > > a small set of essential packages. For those, a coordinated upload > > moving their files will be needed as will be an upload of base-files > > adding the aliasing symlinks there. > > We're well into this now. Most of the essential set is moved and I've > most of the remaining pieces. I hope that within one week, we're left > with only: > - base-files > - bash > - dash > - glibc > - util-linux > > Patches for these have been prepared. The current patches are available > from https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. These > changes have been uploaded to Ubuntu noble already and feedback has been > incorporated. In particular, base-files will now divert to dot files to > avoid cluttering the / view during the transition and base-files will > remove unnecessary diversions (those where it ships symlinks). > > I'd now like to coordinate a time of upload. Given that chroots are > rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning > for the actual upload. If I unexpectedly break stuff, I still have a few > days to fix. How about March 21st? > > Once this has uploaded, we need to ensure that these packages migrate > together. Also note that dash's autopkgtest will fail unless it uses the > updated base-files, but it cannot depend on base-files. If you prefer, I > can mark the relevant test case as flaky and unmark it in a second > upload. Having a temporary migration block on these packages would also > be a good idea. > > Once agreed, I shall announce this change to d-d-a as I cannot fully > rule out it being disruptive despite the extensive testing performed. > > > We probably have to use NMUs to convert remaining packages at this > > point. Once everything is moved, we may think we're done, but we're not. > > Speaking of the rest of packages. At the time of this writing, the > numbers are: > * 224 source packages can be moved via dh-sequence-movetousr. > * 191 source packages have a dep17 usertagged bug report (most with >patches). > * 141 source packages can be moved with a no-change sourceful upload. >This is about Arch:all packages as we already binNMUed the others. > * 35 source packages cannot be analyzed, because they FTBFS (reported). > * A 1-digit number of packages (e.g. the bootstrap ones above) needs >special handling, but this is communicated and monitored. > > I hope that these numbers go down moving forward (especially the patches > one). At some point, I want to mass-NMU the remaining packages. > > > As packages are restructured throughout the release cycle, they may > > introduce file loss scenarios. Continued monitoring for problems until > > trixie is released is crucial. > > The biggest chunk of findings was due to time64. I think the reports are > timely and actionable. Generally, I hope that we'll run into less > fallout moving forward as the "big" stuff is being handled. One > exception here is that time64 has introduced a pile of "risky replaces". > These are not accounted as buggy in the above numbers but need to be > addressed before we can mass-NMU. That'll happen once the dust settles > on time64. > > Any objections so far? I know my objection is meaningless, and that I will end up dying on this hill. But, I do have an objection. It has come to my attention that the big idea behind it is due to, "compatability." Look, / is for unice universal tools that are found in every unice environment and to service boot up. That's its intention. /usr is for distrobution specific tooling. You are ALREADY doing all the work of putting everything into /usr anyways. I see no reason why the source code of unice universal tools should end up in /usr. Please provide a way for a local admin to break usr-merge. > > Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi On 2024-03-09 21:39:50 +0100, Helmut Grohne wrote: > Hi release team and essential maintainers, > > On Mon, Sep 04, 2023 at 10:33:54PM +0200, Helmut Grohne wrote: > > Once these issues have been resolved, we can move most files except for > > a small set of essential packages. For those, a coordinated upload > > moving their files will be needed as will be an upload of base-files > > adding the aliasing symlinks there. > > We're well into this now. Most of the essential set is moved and I've > most of the remaining pieces. I hope that within one week, we're left > with only: > - base-files > - bash > - dash > - glibc > - util-linux > > Patches for these have been prepared. The current patches are available > from https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. These > changes have been uploaded to Ubuntu noble already and feedback has been > incorporated. In particular, base-files will now divert to dot files to > avoid cluttering the / view during the transition and base-files will > remove unnecessary diversions (those where it ships symlinks). > > I'd now like to coordinate a time of upload. Given that chroots are > rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning > for the actual upload. If I unexpectedly break stuff, I still have a few > days to fix. How about March 21st? If and only if time64_t is done by then. Adding more changes where transition has to be coordinated to the ongoing mega transition does not help. Cheers -- Sebastian Ramacher
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi release team and essential maintainers, On Mon, Sep 04, 2023 at 10:33:54PM +0200, Helmut Grohne wrote: > Once these issues have been resolved, we can move most files except for > a small set of essential packages. For those, a coordinated upload > moving their files will be needed as will be an upload of base-files > adding the aliasing symlinks there. We're well into this now. Most of the essential set is moved and I've most of the remaining pieces. I hope that within one week, we're left with only: - base-files - bash - dash - glibc - util-linux Patches for these have been prepared. The current patches are available from https://salsa.debian.org/helmutg/bootstrap-usrmerge-demo. These changes have been uploaded to Ubuntu noble already and feedback has been incorporated. In particular, base-files will now divert to dot files to avoid cluttering the / view during the transition and base-files will remove unnecessary diversions (those where it ships symlinks). I'd now like to coordinate a time of upload. Given that chroots are rebuilt in Wednesday and Sunday, I suggest we pick a Thursday morning for the actual upload. If I unexpectedly break stuff, I still have a few days to fix. How about March 21st? Once this has uploaded, we need to ensure that these packages migrate together. Also note that dash's autopkgtest will fail unless it uses the updated base-files, but it cannot depend on base-files. If you prefer, I can mark the relevant test case as flaky and unmark it in a second upload. Having a temporary migration block on these packages would also be a good idea. Once agreed, I shall announce this change to d-d-a as I cannot fully rule out it being disruptive despite the extensive testing performed. > We probably have to use NMUs to convert remaining packages at this > point. Once everything is moved, we may think we're done, but we're not. Speaking of the rest of packages. At the time of this writing, the numbers are: * 224 source packages can be moved via dh-sequence-movetousr. * 191 source packages have a dep17 usertagged bug report (most with patches). * 141 source packages can be moved with a no-change sourceful upload. This is about Arch:all packages as we already binNMUed the others. * 35 source packages cannot be analyzed, because they FTBFS (reported). * A 1-digit number of packages (e.g. the bootstrap ones above) needs special handling, but this is communicated and monitored. I hope that these numbers go down moving forward (especially the patches one). At some point, I want to mass-NMU the remaining packages. > As packages are restructured throughout the release cycle, they may > introduce file loss scenarios. Continued monitoring for problems until > trixie is released is crucial. The biggest chunk of findings was due to time64. I think the reports are timely and actionable. Generally, I hope that we'll run into less fallout moving forward as the "big" stuff is being handled. One exception here is that time64 has introduced a pile of "risky replaces". These are not accounted as buggy in the above numbers but need to be addressed before we can mass-NMU. That'll happen once the dust settles on time64. Any objections so far? Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Sebastian, On Tue, Sep 12, 2023 at 09:24:45AM +0200, Sebastian Ramacher wrote: > thanks for the detailed explanation. thanks for following up in detail. > On 2023-09-04 22:33:54 +0200, Helmut Grohne wrote: > Skipping all the technical details, as I would not classify this as a > transition where the release team needs to be involved. We can of course > help with the rebuilds of the affected packages, but this change does > not require us to check for outdated binary packages in testing or to > make sure that everything migrates together. It certainly is not a usual transition and also will not migrate together. In that sense, I agree. Yet, I think this is something that the release team wants to know is going on and consider how it interacts with other transitions (e.g. time64). In essence, I think we agree. :) > For the systemd change, the following steps should be enough in general: > > 1. debhelper with support for service files in /usr migrates to testing. done > 2. Rebuild packages which currently have their service files. A guinea pig package "location" has been uploaded with support from Jochen. I don't have numbers on the number of packages that manually install to /lib, but it is many. So this will likely require collective effort rather than me sending patches. > 3. debhelper installing service files to /usr per default migrates to testing. I'm unsure when to do this as I see a moderate risk here. The dumat report currently looks promising, but we also might run into a stream of RC bugs. > 4. Rebuild all packages with service files. I note that the deadline for this practically is trixie and that this doesn't really block any other part of the transition. > For packages where these steps are not enough, bugs are filed along > while preparing 1. and 3. This will include all packages which include > service files in Arch: all packages as we cannot binNMU them. > > Depending on the number of packages in step 4., this would ideally not > be done during the time_t transition to avoid any surprises. I'd hope half of the packages that could be binNMUed have a maintainer upload in the relevant time frame. That also has the benefit of spreading any bad effects over time rather than breaking the archive at once. Regarding the time64 transition, I expect bad interaction. Essentially, time64 will restructure a large amount of packages (moving files from one binary package to another retaining the name on 64bit architectures and thus employing Replaces). As we combine this with mass-moving files from / to /usr, we get exactly that DEP17-P1 scenario that the moratorium was meant to prevent. This interaction is not avoidable by clever timing, because it affects upgrades from bookworm to trixie. I've talked to Steve already to make him aware. The current plan is to just handle this on an individual level and applying the proposed mitigations to issues detected by dumat. > The other / to /usr file moves will need uploads anyway. So we cannot > help here (except for monitoring the status of the bugs during the > freeze). Regarding the move on d-i's side, please coordinate this change > with Cyril. From your explanation it is however not clear to me whether > we are blocked on finishing the /usr-merge change in the main archive by > d-i or not. Let me clarify that in filing this transition bug, my intention was not to get assistance from the release team, but making you aware of what is going on and giving you a reasonable chance to object. The d-i part is rather critical from my point of view. Since d-i still is unmerged, moving files from / to /usr in udebs is likely to break d-i. This explicitly does not cover systemd units though when systemd drops support for split-/usr (next release), d-i will likely be broken anyway. So until d-i is fixed, the remaining moratorium should cover at least all source packages that build udebs. Some of this may be overly cautious, but I'd rather be cautious than temporarily break the archive and that way cause lots of work to unsuspecting developers. Developer time is a scarce resource and some of what we're up to has a risk of consuming lots of that. In any case, I take that at least Paul and Sebastian do not veto moving forward with this. Cyril will probably take some time, but will unlikely comment non-udeb aspects. No clue about Graham. I therefore intend to slowly move forward with the actual conversion in a way that causes least possible disruption and will keep you updated when I expect non-trivial disruption. Helmut
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Helmut thanks for the detailed explanation. On 2023-09-04 22:33:54 +0200, Helmut Grohne wrote: > Unless there is disagreement, I intend to move forward with moving > systemd units on the grounds that the moratorium only is a > recommendation and this transition request. Skipping all the technical details, as I would not classify this as a transition where the release team needs to be involved. We can of course help with the rebuilds of the affected packages, but this change does not require us to check for outdated binary packages in testing or to make sure that everything migrates together. For the systemd change, the following steps should be enough in general: 1. debhelper with support for service files in /usr migrates to testing. 2. Rebuild packages which currently have their service files. 3. debhelper installing service files to /usr per default migrates to testing. 4. Rebuild all packages with service files. For packages where these steps are not enough, bugs are filed along while preparing 1. and 3. This will include all packages which include service files in Arch: all packages as we cannot binNMU them. Depending on the number of packages in step 4., this would ideally not be done during the time_t transition to avoid any surprises. The other / to /usr file moves will need uploads anyway. So we cannot help here (except for monitoring the status of the bugs during the freeze). Regarding the move on d-i's side, please coordinate this change with Cyril. From your explanation it is however not clear to me whether we are blocked on finishing the /usr-merge change in the main archive by d-i or not. Cheers -- Sebastian Ramacher
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Hi Helmut, On 04/09/2023 22.33, Helmut Grohne wrote: If you disagree with this and do not want to get involved, please just close this transition request. Given the impact (size) of this transition, I'd like to also see Graham, Cyril (for d-i) and Sebastian acknowledge this request. I agree with it. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1051237: transition: move files from / to /usr to finalize /usr-merge
Package: release.debian.org User: release.debian@packages.debian.org Usertags: transition Dear release team, you may be aware from d-devel@l.d.o discussions that things got rolling regarding a way forward from our current /usr-merge state. The results of various discussions have been recorded in DEP17[1]. The rough idea is that files in all packages move from / to /usr. I think this change qualifies as a transition and should be coordinated with you. If you disagree with this and do not want to get involved, please just close this transition request. A major blocker regarding this finalization is the file move moratorium. I note that this is given as advice/recommendation rather than being mandatory (thanks Ansgar). However, the release team also indicated enforcement[2] of it. The current best idea is to progressively lift the moratorium by documenting the current state of lifting at https://wiki.debian.org/UsrMerge (not yet documented there) and then officially repealing the moratorium while recommending to follow those guides. As we move files, we will run into issues such as lost files. Some of the issues are automatically identified by the Debian Usr Merge Analysis Tool https://salsa.debian.org/helmutg/dumat. I'm regularly updating the output https://subdivi.de/~helmut/dumat.yaml. The intention is to automatically file RC bugs for some classes of issues (e.g file conflicts), but at the time of this writing I'm still filing bugs manually. The exact way of lifting the moratorium is not clear at this time. It shall be adapted depending on the amount of (expected and unexpected) issues we face. In total, there are around 2000 affected binary packages. More than half of them is only affected due to installing systemd units. As such moving those units from / to /usr is the first step. We first move those where upstream build systems install them and debhelper is prepared to recognize units below /usr. Once that works reasonably well, dh_installsystemd shall be changed to install to /usr. This change will cause latent issues. When packages are restructured or renamed the famous file loss scenario may happen. The dumat service shall run for the entire release period and detect such issues before they hit testing. Moving beyond this step requires more preparation. While systemd still deals nicely with users in / or /usr, other tools may not and our buildds are still unmerged. We'll have to wait until DSA has updated debootstrap to SPU request from Simon McVittie. Also moving files may affect udebs and the debian-installer is still unmerged. The relevant MR addressing this needs to be merged. Once these issues have been resolved, we can move most files except for a small set of essential packages. For those, a coordinated upload moving their files will be needed as will be an upload of base-files adding the aliasing symlinks there. We probably have to use NMUs to convert remaining packages at this point. Once everything is moved, we may think we're done, but we're not. As packages are restructured throughout the release cycle, they may introduce file loss scenarios. Continued monitoring for problems until trixie is released is crucial. As problems are located, context-dependent mitigations from DEP17 are to be applied. We'll recommend that maintainers upload restructuring changes to experimental first and given quick bug filing that should reduce the number of issues in unstable and keep most issues out of testing. In any case, you can only call this transition bug finished once trixie is released. For the purpose of keeping bugs out of testing, I intend to file all relevant bugs (such as file conflicts, file loss, directory loss, ineffective diversions, missing trigger invocations, etc.) at RC severity. I hope this all makes sense to you. Let me know if you disagree about any of this. Quite probably I am missing some important aspects here. Unless there is disagreement, I intend to move forward with moving systemd units on the grounds that the moratorium only is a recommendation and this transition request. Helmut [1] https://salsa.debian.org/dep-team/deps/-/merge_requests/5 rendered version at https://subdivi.de/~helmut/dep17.html [2] https://lists.debian.org/debian-devel-announce/2022/07/msg2.html
