Alexis Ballier posted on Tue, 05 Apr 2016 12:10:51 +0200 as excerpted: > On Tuesday, April 5, 2016 3:19:59 AM CEST, William Hubbs wrote: > [...] >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: >> >> /bin->usr/bin /lib->usr/lib /lib32->usr/lib32 /lib64->usr/lib64 > > (OT: maybe it'd be a good oportunity to kill SYMLINK_LIB too :p) > >> /sbin->usr/bin /usr/sbin->bin >> >> Once that is in place in a new baselayout, I think portage's colission >> detection would be able to catch files that had the same names and were >> originally in different paths when building the new stages. > > > I think that prior to that we have to ensure that packages with intra > collisions are not merged: What happens with current coreutils ebuilds > that install, e.g., /bin/seq and a /usr/bin/seq symlink to it ? > I haven't looked at the actual code, thus I can only assume there are no > guarantees, which is definitely bad.
As I said in the other thread, I'm running merged /usr and bin/sbin here, except that I merged them the other way, with /usr -> . so everything in /usr is now in /. Portage has long "just worked" in that regard, tho I've no idea whether the other PMs do. Portage has enough intelligence to avoid replacing a file with a symlink pointing to it (and thus to itself once the replacement is done), regardless of which way the directory symlinks point. As such, coreutils "just works". If the two would end up in the same canonical location, the file wins and the symlink isn't installed. There are a few individual package bugs, including one open right now where the gcc-config ebuild does an unconditional rm -f of any old versions found in its old sbin location, even when it removes the executable it just installed into the bin location, because they're the same canonical location. (Bug number for that and other bugs in the reply on the other thread.) And cmake can get mixed up in some instances so a few packages (baloo) have problems with some cmake versions. But the bugs aren't with portage, they're with the ebuild or the upstream sources, and the number of them I've run into in the two years plus I've been running merged can fit on one hand. Certainly, they're few enough to deal with on a case-by-case basis. >> I put some thought also in how to nigrate live systems, and I'm not >> sure what the best way to do that is. I wrote a script, which would do >> it in theory, but I haven't tested because I only have one system and >> if it breaks it the only way back would be to reinstall. > > > Does it behave properly wrt portage's way of tracking of package files? > I remember that modifying files owned by portage used to cause issues. What I did for my migration was simply move everything from /usr to / and create the /usr -> . symlink. I did that with mc, and kept it running just in case I ended up not being able to start something, until I had the symlink in place and had tested starting various things, including X/ KDE, so I knew it was working. Similarly for the sbin -> bin moves and symlinks. The moves worked fine, and with the directory symlinks replacing the old dirs, everything else, including portage on updates after that, worked just fine. There are a couple things that behave slightly differently regarding packages, that one needs to be aware of, but in general it just works. Those couple things are: 1) Unless one is sure of the actual install path used and uses it, equery belongs and I assume q and similar tools with the same query, need the bare file name, not the full path, because you might use the wrong one. For instance, /bin/grep, /sbin/grep, /usr/sbin/grep, and /usr/bin/grep, are all the same file due to directory level symlinks. However, if you try equery belongs with all four paths, only one of them will return the hit you're looking for. Easily solved by simply doing equery belongs grep (no path), and letting equery supply the installed path in its results. That's actually how I find out which path a file was actually installed to, these days, as well. =:^) 2) revdep-rebuild will, in its default configuration, end up processing files using all paths. So grep, to use the same example as above, will be processed four times, one each for /bin and /sbin, /usr/bin and /usr/ sbin. While it's possible to reconfigure revdep-rebuild to ignore three of those paths and only check one of them (and similarly, ignore one of /lib64 and /usr/lib64, etc), doing so will result in revdep-rebuild complaining about unowned broken binaries if they're installed using a different path than the one it processed. That's not a big problem, because equery belongs <file> (without the path) will tell you what owns it as well as the installed path it used, and then that package can be remerged manually, if needed. So with revdep-rebuild, it's a tradeoff. You can either have it taking 4X as long to resolve executables and 2X as long to resolve libs, but handle the rebuilds automatically when it's done, or shorten its processing time by only processing one of the paths, and then have to figure out what owns some of the files it complains about and remerge them manually. (FWIW, I chose the short but often manual method when I first setup the merge, as the revdep-rebuild shell script was slow and processing things four times took way too long. But the newer python-based revdep-rebuild is far faster, and I've been going to switch back to letting it process all the paths now so it handles all its rebuilds automatically once again. But I've yet to get around to switching the config around, so it's still using the short but often semi-manual method.) > Also, I think your script may not work: > > # copy binaries for dir in /bin /sbin /usr/sbin; do > run_command cp -a $dir/* /usr/bin > done > > ---> Here it is important to ensure nothing /usr/bin/ is a symlink to > /bin, > otherwise this would just copy, e.g., /bin/seq onto /bin/seq Very good point! When I did it the migration here, I did the move (in mc) skipping files that existed in the new location. There were twenty or so skipped, IIRC, few enough to manually investigate. From memory, all of them were symlinks in one place or the other, so I could delete the symlinks and move the rest of the files over, without hitting skips the second time. Obviously a naive move will replace some of the files with dead (once replaced) symlinks, so valid point raised. =:^) > # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do > run_command rm -rf $dir > > ---> where are the 'ln' and 'rm' taken from after this step ? > If this fails here, you're also leaving the system in a broken state. > > run_command ln -s usr/bin $dir done In my case I was using the mc binary, which continued to run after the transfer, so it wasn't an issue. But using the individual ln and rm binaries, while they'll still be on the path, you may need to run hash -r in the shell so it forgets the old location and checks the path again. Similar thing for the libs, since the lib cache will be screwed after the move, until the symlink has been created so the old paths work again, at least. In my case I was using the mc binary which continued to run and thus could be used to create the symlink, but for one-shot executables like ln, that could be an issue. One way around the problem would be to create a few static-linked executables for the purpose, and ship them in a tarball that's unpacked to an unchanging tmp location for the run, so they can be called from there to finish up regardless of whether the dynamically linked normal executables can still be invoked. Smart use of the shell's builtin read, echo and redirection could probably do some of it too, but can ln be emulated using shell builtins? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
