Re: [gentoo-dev] usr merge
On Sunday, April 10, 2016 10:04:42 AM James Le Cuirot wrote: > On Sun, 10 Apr 2016 02:09:35 +0200 > > "J. Roeleveld" wrote: > > I actually write my own initramfs because neither dracut not > > genkernel end up with a convenient boot system. > > > > I have 2 disks, both encrypted. > > I prefer only to enter the decryption password once. Both Dracut and > > Genkernel insist on asking for the password/key for every single disk. > > Dracut on RHEL actually handles this out of the box. Might be worth > finding out how. Might have even been fixed in a more recent version of Dracut. I just have passed the point where I am interested in it enough to try it. The initramfs I use gets embedded into the kernel and doesn't need any kernel parameters to work. It does what it needs to do with minimal work. The simplicity should also make it faster than the scripts generated by either Dracut or genkernel. (As they need to parse the kernel cmdline and try to figure out static details on the fly) -- Joost signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [RFC] New eclass: mate
On 11/04/16 06:09, NP-Hardass wrote: > Greetings all, > > As all potential new eclasses are supposed to be discussed here, I > thought I'd file a message and see if anyone had anything to > contribute on the matter. > > I'm in the midst of a major version bump for the entirety of the MATE > desktop environment, consisting of 40-50 packages. There is a huge > amount of repetition in my ebuilds, and a lot of things are formulaic > (SRC_URI, HOMEPAGE, EGIT_REPO_URI, inherits, src_prepare, etc). As > such, I think that moving all of that to an eclass would greatly > simplify my life and my ebuilds, so I thought I'd look into creating > an eclass. > > Any opinions either way? Thanks in advance. > > I say go for it .. post up any milestones for review, of course... :] signature.asc Description: OpenPGP digital signature
[gentoo-dev] [RFC] New eclass: mate
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Greetings all, As all potential new eclasses are supposed to be discussed here, I thought I'd file a message and see if anyone had anything to contribute on the matter. I'm in the midst of a major version bump for the entirety of the MATE desktop environment, consisting of 40-50 packages. There is a huge amount of repetition in my ebuilds, and a lot of things are formulaic (SRC_URI, HOMEPAGE, EGIT_REPO_URI, inherits, src_prepare, etc). As such, I think that moving all of that to an eclass would greatly simplify my life and my ebuilds, so I thought I'd look into creating an eclass. Any opinions either way? Thanks in advance. - -- NP-Hardass -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCAAGBQJXCzGLAAoJEBzZQR2yrxj7knsP/AyP3yREWCEuw/k2IbAw987y QM78PN1u1IYA1YtP1IDFyOd91aGuIqwtURdoY6zq7ExBA8/5nftMHalgz5kfUyFn lXhe5ZYsnFUAMIJHrsdECFdRbK7RfYYraEZSRyPs4kZJjUrg4PLiFuJIy8ITzq5p Ul7hD8pFIEMpgEmWly/cJ5G7Eatl8A8OgJW9LAYOxDDFdNX1xvzP0VQhRE9lBE7u SiVPI+iTwH5yIM4KDpKPGvsCOR82ijj661IMFa/sJdZdwWPmtf8LRQb12CMk/Yjr YheUgamZ88KC4zd55mLotZGi7g71Lof7IRByzOO5VsWRpW5h9vm+ivVLLI72fj9W 88lmmvJ6f/Pw1w8WXeX/D8wN4PE6veD8Y/O/P3czIi1zXQM+IRZhJmoxpCiQng93 fXdLd2h0o2ntx1qsubIHU/fiakz8RZtsO0zQbYdrvC+YV2+IJTuKfco8YyWgiNul PsWuVCcM8cjVVIbk10Ny93MLEkhM2DvUsu35imFfJzIU1eCFo9PtD3YEOzqStNuB uNeOZi5PAhWllJCVgNkO2a0hXpA/N24a0TUh5CFS2y/rZ7BH3ydGVg8D0nkwx4dV 2f6/ydR2cNmLckPVdfrXljFgE8vDLrobZaBWvyxbQqyslLbb684Js5KjH9T1fgPt 7ibouuaYZ80U+FiL4mtu =/7NT -END PGP SIGNATURE-
Re: [gentoo-dev] usr merge
On 04/10/2016 08:14, Rich Freeman wrote: > On Sun, Apr 10, 2016 at 7:55 AM, Joshua Kinard wrote: >> >> Create like, a table on the Wiki or some kind of metadata property >> per-package >> that can contain a boolean or tri-state flag indicating whether it works or >> doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. >> Something. >> > > I'm sure there will be a tracker for packages that don't work on a > merged /usr. (We are already on a split /usr.) > > Honestly, I'm still not quite sure why we're even having this > discussion. I don't think anybody actually intends to make any > changes at all. If they do, they should issue some kind of plan and > indicate what they're looking for from everybody else. Agreed. A plan is most definitely needed if we ever choose to pursue a policy of only supporting non-split-/usr installs, especially if we want to handle cases like mine where we try to make migration of existing installs possible or not. [snip] > It seems to me that we're just having a general discussion about the > pros/cons of a /usr merge. That is nice, but people are getting > worked up because they think that somehow whoever "loses" this > "discussion" will get something shoved down their throats or won't be > able to have something nice. "General discussion" -- hah! Maybe it's the way Thunderbird is displaying it, but I have five distinct, top-level threads in my gentoo-dev folder for this discussion. I think we left "general" back there after we broke through the plaid barrier. That said, I don't really think there are any pros or cons of split or merged. Largely, what's being discussed is the after-effects of what once was a common approach to filesystem layout. Myself, I only went the split-usr route because of habit, itself which started because when I set up my first Gentoo system. I studied both the then-Gentoo Security and Debian Security manuals, which indicated a split-/usr layout had certain advantages in that you could limit capabilities via mount options. Mainly, /usr didn't need devices, so "nodev" was common there. Along with /var having "noexec" and "nodev". I've pretty much stuck to that layout approach since then out of habit. Certainly, I've got a few VMs where I have just /, /boot, and /tmp as my only partitions, and there's no real noticeable difference except what's in /etc/fstab. --- I think where the problem ultimately arises is a subtle conflict between filesystem semantics and system-design philosophy against the Linux kernel's device architecture and management. It's long regarded that /bin and /sbin contain system-level binaries, and /usr/bin and /usr/sbin being for almost everything else. A.k.a., a two-level binary installation hierarchy (and the BSD's extend this with a third level under /usr/local). >From the kernel angle, you have a monolithic kernel design where device drivers run in kernel space most of the time. This worked okay with traditional, static devices that didn't change a whole lot and whose resources could be determined at boot-time, before userspace is brought up. But once the Linux ecosystem needed devices that can come online from userspace or need their resource determination to be dynamic (e.g., for hotplugging), we went to the approach that the kernel needs to get out of managing devices altogether, and thus came up with udev for device management from userspace. Since udev is supposed to run from userspace, but kinda needs to interface with the kernel a lot, the split between what's system-specific and what's not clashes with the two-level file system layout. This wasn't anything new...this conflict has existed elsewhere for a long time (namely in X11), but it came to a head when udev maintainers (and later, systemd maintainers) questioned the approach, and largely decided it wasn't worth it, and a merged filesystem, with /usr not separate, was simpler and more elegant. --- But really, does it matter where the binaries all live? It's just a design decision. Every OS had a different approach, such as NetWare running virtually everything out of SYS:SYSTEM, and Windows out of C:\WINDOWS. Heck, for the longest time, you could *not* install Windows on anything other than the first partition of the first drive, because software literally hardcoded filespec strings as "C:\WINDOWS\...". And even why C:, the third letter of the alphabet, for the first fixed disk? Because A: and B: were reserved for systems that needed two floppy drives. Yay MS-DOS! If it were possible to give every binary and file out there a unique name, you wouldn't even need directories. You could have a totally-flat namespace with all files of any kind under /, and let security models handle who sees/accesses what. But in that setup, would you even need "/"? > Almost every big change that has become popular in Gentoo just started > out as another alternative, and support grew organically. I don't > really see the need to
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2016-04-10 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2016-04-10 23:59 UTC. Removals: dev-db/etcdctl20160410-06:06 zmedicod6a77a1 dev-lang/go-bootstrap 20160410-17:53 williamh 75bb881 dev-perl/math-pari20160407-07:33 dilfridge 014964d lxqt-base/liblxqt-mount 20160410-16:42 kensington f8da31f lxqt-base/lxqt-config-randr 20160410-16:42 kensington b23d09c net-im/qwit 20160410-16:39 kensington a6cd2cf Additions: app-admin/cli53 20160409-14:59 floppym11b2509 app-admin/github-backup-utils 20160406-05:48 wizardedit 4cf236a app-crypt/stoken 20160407-16:52 prometheanfire 1c6d473 app-editors/atom 20160406-11:06 cynede 10167cf app-misc/tails-installer 20160408-04:03 wizardedit 0135da7 app-text/lesspipe 20160406-22:05 vapier 23f0a89 dev-java/jetty-alpn-api 20160405-21:09 chewi 4e79621 dev-java/jetty-npn-api20160405-20:45 chewi 8d21163 dev-java/maven-hawtjni-plugin 20160404-21:43 chewi c7e010f dev-java/netty-tcnative 20160404-22:55 chewi ac4ca41 dev-perl/File-ShareDir-ProjectDistDir 20160403-19:29 dilfridge eb318cd dev-perl/Math-Pari20160407-07:16 dilfridge 48ef171 dev-perl/Path-FindDev 20160403-19:26 dilfridge be120c4 dev-perl/Path-IsDev 20160403-19:22 dilfridge a1f82d0 dev-perl/Time-TZOffset20160405-15:24 dilfridge 20d1e2e dev-python/bibtexparser 20160410-12:57 soap 1850fe6 dev-python/healpy 20160407-00:32 bicatali 7ff820b dev-python/python-spidermonkey20160406-21:36 wizardedit 0fe9479 dev-python/typing 20160410-22:47 alunduil a6195c6 dev-util/gcovr20160407-02:17 ottxor 85877ac dev-util/ply 20160410-03:03 zmedicode8f6e9 dev-vcs/git-remote-hg 20160406-22:00 wizardedit e57a2d5 kde-apps/kdesdk-thumbnailers 20160407-12:06 kensington 8971a31 kde-frameworks/kactivities-stats 20160410-11:33 johu 0fdba17 media-gfx/xdot20160407-06:12 wizardedit cc361de sci-astronomy/healpix 20160407-00:14 bicatali 5407c47 sci-misc/cdfplayer20160409-20:06 dilfridge 7e522f4 sys-apps/frandom 20160406-18:10 zerochaos 694a094 www-apps/piwigo 20160405-08:05 voyageur 2d97cd3 -- Robin Hugh Johnson Gentoo Linux Developer E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 Removed Packages: dev-lang/go-bootstrap,removed,williamh,20160410-17:53,75bb881 lxqt-base/lxqt-config-randr,removed,kensington,20160410-16:42,b23d09c lxqt-base/liblxqt-mount,removed,kensington,20160410-16:42,f8da31f net-im/qwit,removed,kensington,20160410-16:39,a6cd2cf dev-db/etcdctl,removed,zmedico,20160410-06:06,d6a77a1 dev-perl/math-pari,removed,dilfridge,20160407-07:33,014964d Added Packages: dev-python/typing,added,alunduil,20160410-22:47,a6195c6 dev-python/bibtexparser,added,soap,20160410-12:57,1850fe6 kde-frameworks/kactivities-stats,added,johu,20160410-11:33,0fdba17 dev-util/ply,added,zmedico,20160410-03:03,de8f6e9 sci-misc/cdfplayer,added,dilfridge,20160409-20:06,7e522f4 app-admin/cli53,added,floppym,20160409-14:59,11b2509 app-misc/tails-installer,added,wizardedit,20160408-04:03,0135da7 dev-perl/Math-Pari,added,dilfridge,20160407-07:16,48ef171 dev-perl/Time-TZOffset,added,dilfridge,20160405-15:24,20d1e2e dev-perl/File-ShareDir-ProjectDistDir,added,dilfridge,20160403-19:29,eb318cd dev-perl/Path-FindDev,added,dilfridge,20160403-19:26,be120c4 dev-perl/Path-IsDev,added,dilfridge,20160403-19:22,a1f82d0 dev-java/jetty-alpn-api,added,chewi,20160405-21:09,4e79621 dev-java/jetty-npn-api,added,chewi,20160405-20:45,8d21163 dev-java/netty-tcnative,added,chewi,20160404-22:55,ac4ca41 dev-java/maven-hawtjni-plugin,added,chewi,20160404-21:43,c7e010f app-crypt/stoken,added,prometheanfire,20160407-16:52,1c6d473 kde-apps/kdesdk-thumbnailers,added,kensington,20160407-12:06,8971a31 media-gfx/xdot,added,wizardedit,20160407-06:12,cc361de dev-util/gcovr,added,ottxor,20160407-02:17,85877ac dev-python/healpy,added,bicatali,20160407-00:32,7ff820b sci-astronomy/healpix,added,bicatali,20160407-00:14,5407c47 app-text/lesspipe,added,vapier,20160406-22:05,23f0a89 dev-vcs/git-remote-hg,added,wizardedit,20160406-22:00,e57a2d5 dev-python/python-spidermonkey,added,wizardedit,20160406-21:36,0fe9479 sys-apps/frandom,added,zerochaos,20160406-18:10,694a094 app-editors/atom,added,cynede,20160406-11:06,10167cf app-admin/github-backup-utils
Re: [gentoo-dev] CVS headers in ebuilds
On Sun, 10 Apr 2016 18:21:44 -0400 Michael Orlitzky wrote: > On 04/10/2016 05:36 PM, Gordon Pettey wrote: > > Or you could just use a branching workflow like Git has great > > support for, and create your overlay as a branch of the main tree > > you're copying ebuilds from. With recent versions, you can even > > have checkouts of different branches from the same tree in > > different directories, so you're not duplicating the the whole > > repository. Then you could just git diff from the master branch, > > and no need for preserving CVS misbehavior. > > I suggested this on the bug, but it's not so clear-cut. With a branch, > you have a million other ebuilds sitting around in "your overlay" that > you don't care about. I'm also not sure how the metadata updates would > work; like, if it would be efficient to pull the rsync metadata and > then run `emerge --metadata` or something like that. It wouldn't be > fair to say "just use a branch!" if that's going to mean that > somebody's day-to-day work gets a lot slower. Would a sparse checkout help here? I've used this feature in a different context. You could add the packages you do care about to the sparse checkout list. http://stackoverflow.com/questions/600079/is-there-any-way-to-clone-a-git-repositorys-sub-directory-only/13738951#13738951 -- James Le Cuirot (chewi) Gentoo Linux Developer pgprLYJIC7D0v.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] CVS headers in ebuilds
On 04/10/2016 05:36 PM, Gordon Pettey wrote: > Or you could just use a branching workflow like Git has great support > for, and create your overlay as a branch of the main tree you're copying > ebuilds from. With recent versions, you can even have checkouts of > different branches from the same tree in different directories, so > you're not duplicating the the whole repository. Then you could just git > diff from the master branch, and no need for preserving CVS misbehavior. > I suggested this on the bug, but it's not so clear-cut. With a branch, you have a million other ebuilds sitting around in "your overlay" that you don't care about. I'm also not sure how the metadata updates would work; like, if it would be efficient to pull the rsync metadata and then run `emerge --metadata` or something like that. It wouldn't be fair to say "just use a branch!" if that's going to mean that somebody's day-to-day work gets a lot slower. I would also feel a little guilty saying that the supported way of extending/modifying the tree (overlays) won't work and you have to branch off our VCS. I promised to think about it, but haven't had much inspiration. Here's the best I've got so far. 1. Use git for your ::gentoo tree. 2. Before every sync (git pull), save the current HEAD: OLD=$(git rev-parse --verify HEAD) 3. git pull 4. Save the new HEAD: NEW=$(git rev-parse --verify HEAD) 5. Search through the log (between the old HEAD and the new one) for any files that were modified: git log $OLD..$NEW --format='' --name-status \ | grep '^M.*ebuild' \ | cut -f2 \ | sort \ | uniq That gives you a nice list at least, and you can compare it to the files present in your overlay. If you add another | xargs git log --stat $OLD..$NEW onto the end of that last command, you can see statistics on what changes were made. (Holy crap do we have a lot of people editing dependencies without making revision bumps.)
Re: [gentoo-dev] CVS headers in ebuilds
Or you could just use a branching workflow like Git has great support for, and create your overlay as a branch of the main tree you're copying ebuilds from. With recent versions, you can even have checkouts of different branches from the same tree in different directories, so you're not duplicating the the whole repository. Then you could just git diff from the master branch, and no need for preserving CVS misbehavior. On Sun, Apr 10, 2016 at 12:28 PM, Robin H. Johnson wrote: > On Sun, Apr 10, 2016 at 06:16:05PM +0200, Ulrich Mueller wrote: > > Why would you need $Id$ feature for this? "git ls-files -s" gives you > > the hash of the blob as well, is more efficient than grep, and even > > works recursively on a directory tree. > > > >$ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild > >100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0 > www-client/seamonkey/seamonkey-2.40.ebuild > Your ls-files doesn't let you track what blob the modified ebuild came > from, when it's copied out of the git repo where expansion was > happening. > > If the $Id$ is expanded in rsync, or your local environment, then you > copy the file with the expanded $Id$ to an overlay (or another Git > environment without expansion enabled), you have preserved the $Id$. > > Now the user edits it in their overlay, and since the original $Id$ is > preserved, when you ask on bugzilla, please submit it as a diff; they > can simply do: > # diff <(git show $IdHash) $OVERLAY/pn/pv.ebuild > > > -- > Robin Hugh Johnson > Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee > E-Mail : robb...@gentoo.org > GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 > >
[gentoo-dev] [warning] the bug queue has 85 bugs
Our bug queue has 85 bugs! If you have some spare time, please help assign/sort a few bugs. To view the bug queue, click here: http://bit.ly/m8PQS5 Thanks!
[gentoo-dev] Re: usr merge
On Fri, 8 Apr 2016 21:18:37 -0400 waltd...@waltdnes.org wrote: > On Fri, Apr 08, 2016 at 04:30:04PM -0400, Rich Freeman wrote > > > Half the reason we don't officially support running without /usr > > mounted during early boot is that if we actually put everything in / > > that could conceivably be needed during early boot we'd end up with > > everything there. Bluetooth keyboards is a common example. The > > console should work during early boot, right? > > Seriously... how many people run Bluetooth keyboards on Gentoo > anyways? The sarcasm and the rhetorical questions make it tough to tell -- are you trying here to make an argument that Gentoo should commit to supporting boot without /usr mounted early?
Re: [gentoo-dev] usr merge
On Fri, Apr 08, 2016 at 09:18:37PM -0400, waltd...@waltdnes.org wrote: > On Fri, Apr 08, 2016 at 04:30:04PM -0400, Rich Freeman wrote > > Half the reason we don't officially support running without /usr > > mounted during early boot is that if we actually put everything in / > > that could conceivably be needed during early boot we'd end up with > > everything there. Bluetooth keyboards is a common example. The > > console should work during early boot, right? > Seriously... how many people run Bluetooth keyboards on Gentoo anyways? I had a BT keyboard on my last Gentoo-based mediacentre box so that I could sit on the couch and still use it... -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] CVS headers in ebuilds
On Sun, Apr 10, 2016 at 06:16:05PM +0200, Ulrich Mueller wrote: > Why would you need $Id$ feature for this? "git ls-files -s" gives you > the hash of the blob as well, is more efficient than grep, and even > works recursively on a directory tree. > >$ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild >100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0 > www-client/seamonkey/seamonkey-2.40.ebuild Your ls-files doesn't let you track what blob the modified ebuild came from, when it's copied out of the git repo where expansion was happening. If the $Id$ is expanded in rsync, or your local environment, then you copy the file with the expanded $Id$ to an overlay (or another Git environment without expansion enabled), you have preserved the $Id$. Now the user edits it in their overlay, and since the original $Id$ is preserved, when you ask on bugzilla, please submit it as a diff; they can simply do: # diff <(git show $IdHash) $OVERLAY/pn/pv.ebuild -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] CVS headers in ebuilds
> On Sat, 9 Apr 2016, Lars Wendler wrote: >>> > Yes, I still use these lines to check for ebuild changes between >>> > portage and my personal overlay. So please keep this line. > Enable the ident feature for *.ebuild files in git: > $ cat ~/gentoo/.git/info/attributes > *.ebuild ident > Now re-checkout every ebuild you wanna track. > $ git checkout -- > ~/gentoo/www-client/seamonkey/seamonkey-2.40.ebuild > Once you have done that those ebuilds will have some hash in the > $Id$ field: > $ grep '$Id' ~/gentoo/www-client/seamonkey/seamonkey-2.40.ebuild > # $Id: 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 $ > The same hash is in each corresponding ebuild in my personal overlay > as well. Occasionally I run a script to compare ebuilds from my > overlay with the one from the git tree. When the hash is different > something in the gentoo ebuild has changed and I can decide if I > want to apply these changes to ebuilds in my overlay as well. Why would you need $Id$ feature for this? "git ls-files -s" gives you the hash of the blob as well, is more efficient than grep, and even works recursively on a directory tree. $ git ls-files -s -- www-client/seamonkey/seamonkey-2.40.ebuild 100644 5ecd7709c6c8a316d9f005b4e4a0a54da81eb048 0 www-client/seamonkey/seamonkey-2.40.ebuild Ulrich pgpOFBY7awgmg.pgp Description: PGP signature
Re: [gentoo-dev] usr merge
On 4/10/16 8:14 AM, Rich Freeman wrote: > > Honestly, I'm still not quite sure why we're even having this > discussion. I don't think anybody actually intends to make any > changes at all. If they do, they should issue some kind of plan and > indicate what they're looking for from everybody else. > Because William started this thread, http://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg73010.html and because the passive voice in the sentence "I thought that since the usr merge is coming up again" begs the question: Who is bringing it up and why? So ... questioning minds want to know. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] usr merge
On 4/10/16 7:55 AM, Joshua Kinard wrote: > On 04/04/2016 21:19, William Hubbs wrote: >> All, >> >> I thought that since the usr merge is coming up again, and since I lost >> track of the message where it was brought up, I would open a >> new thread to discuss it. Why is this coming up? What problem(s) are we trying to solve with the usr merge. I'm still not clear on this. @William can you please answer this? >> >> When it came up before, some were saying that the /usr merge violates >> the fhs. I don't remember the specifics of what the claim was at the >> time, (I'm sure someone will point it out if it is still a concern). >> >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: If you give me a prototype baselayout (or I can write one myself but I'd rather see what you have in mind) I could test some catalyst runs and see. I could put these on the /experimental for others to look at. >> >> /bin->usr/bin >> /lib->usr/lib >> /lib32->usr/lib32 >> /lib64->usr/lib64 >> /sbin->usr/bin >> /usr/sbin->bin >> >> Once that is in place in a new baselayout, I think portage's colission >> detection would be able to catch files that had the same names and were >> originally in different paths when building the new stages. >> >> I put some thought also in how to nigrate live systems, and I'm not sure >> what the best way to do that is. I wrote a script, which would do it in >> theory, but I haven't tested because I only have one system and if >> it breaks it the only way back would be to reinstall. >> >> The script is attached. >> >> >> Thoughts on any of this? >> >> William > > I looked at Thunderbird, at my folder labeled "gentoo-dev", and it had "666" > unread messages. I should've done the smart thing and closed my mail program. > But n, I had to look inside the folder. I am now regretting this > decision. Did you round off to the nearest evil? > > *sigh*, I can see the thread has gone clongie 'round the blonger, so all I'll > have to say is we should still try to maintain the choice for users. But, in > order to evaluate what amount of effort is needed to maintain that choice, we > need to know what packages break on such a setup, what the level of effort > needed to fix them is, and do those fixes impact the non-split crowd. I tested and it will affect systems using RBAC. So if we force people to migrate, then users of hardened-sources using RBAC will have to update their policy file. To be clear, I'm not against moving forwards with this, but we can't these people hanging because hardened-sources+RBAC is one reason people in the industry consider gentoo at all. I'm willing to help with backwards compat. > > Create like, a table on the Wiki or some kind of metadata property per-package > that can contain a boolean or tri-state flag indicating whether it works or > doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. > Something. +1 > > Once we know this, then we can work out what's the minimum system that can be > successfully run on split-usr. Then, knowing *that*, we can see if that > system > can be supported by our @system target or some minimal subset of @world. As > new package versions come out of upstream, we update this metadata with > changes > to the split-usr status, and this then provides a history of the more or less > amount of difficulty needed to maintain support for split-usr, and *then*, we > can make an objective decision to continue supporting or not supporting the > capability. > > As for me, I am flat out ruling out a full-reinstall of all my systems. I > have > fixed disk partition layouts on all of them that cannot be re-arranged unless > I > tar up each filesystem and temporarily move it off, then rebuild the MD-RAID > and reformat the filesystems. I am simply not going to do that on my many SGI > systems, and whatever facet of upstream, whether it's some core GNU package or > RH itself, can go pound sand for all I care. I'll go back to a static /dev > and > I'll manually mknod any missing devices if I have to. Not just you, but many sys admins out there in the real world are in similar situation. Again we can move forward on this but not without backwards compat planning. > > You know it's getting ridiculous when you can maintain a Windows/NTFS > partition > layout easier than a Linux one. > Has it come to that? -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] usr merge
On Sun, Apr 10, 2016 at 7:55 AM, Joshua Kinard wrote:
>
> Create like, a table on the Wiki or some kind of metadata property per-package
> that can contain a boolean or tri-state flag indicating whether it works or
> doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie.
> Something.
>
I'm sure there will be a tracker for packages that don't work on a
merged /usr. (We are already on a split /usr.)
Honestly, I'm still not quite sure why we're even having this
discussion. I don't think anybody actually intends to make any
changes at all. If they do, they should issue some kind of plan and
indicate what they're looking for from everybody else.
Such as: "Hello, I help maintain baselayout and I intend to change
/(s)lib and /(s)bin and /usr/sbin into symlinks to /usr/bin and move
all the files into those directories there. To test this out now
please do xyz, and report any bugs against tracker #123456."
Or: "Hello, I help maintain baselayout and I just introduced a new USE
flag which does ... I think it is something you should try out. Bugs
can be reported at..."
Or: "Hello, I think the baselayout maintainers are idiots and I just
introduced librelayout which does ... You should definitely switch
because only losers run with a split /usr. Bugs can be reported at...
Oh, and my fancy librelayout doesn't need gen_usr_ldscript so I
select_one_of('won't lift a finger to keep it working', 'will just
laugh at the folks who are wasting their time keeping it working')."
It seems to me that we're just having a general discussion about the
pros/cons of a /usr merge. That is nice, but people are getting
worked up because they think that somehow whoever "loses" this
"discussion" will get something shoved down their throats or won't be
able to have something nice.
Almost every big change that has become popular in Gentoo just started
out as another alternative, and support grew organically. I don't
really see the need to reach some kind of consensus here. I'd love to
have an option of a /usr merge and a migration path. I'd love to see
it as the default, but that is a different discussion, and if it is
optional then it is also a less contentious discussion whichever way
it goes.
--
Rich
Re: [gentoo-dev] usr merge
On 04/04/2016 21:19, William Hubbs wrote: > All, > > I thought that since the usr merge is coming up again, and since I lost > track of the message where it was brought up, I would open a > new thread to discuss it. > > When it came up before, some were saying that the /usr merge violates > the fhs. I don't remember the specifics of what the claim was at the > time, (I'm sure someone will point it out if it is still a concern). > > I don't think creating usr merged stages would be that difficult. I > think it would just be a matter of creating a new version of baselayout > that puts these symlinks in place: > > /bin->usr/bin > /lib->usr/lib > /lib32->usr/lib32 > /lib64->usr/lib64 > /sbin->usr/bin > /usr/sbin->bin > > Once that is in place in a new baselayout, I think portage's colission > detection would be able to catch files that had the same names and were > originally in different paths when building the new stages. > > I put some thought also in how to nigrate live systems, and I'm not sure > what the best way to do that is. I wrote a script, which would do it in > theory, but I haven't tested because I only have one system and if > it breaks it the only way back would be to reinstall. > > The script is attached. > > > Thoughts on any of this? > > William I looked at Thunderbird, at my folder labeled "gentoo-dev", and it had "666" unread messages. I should've done the smart thing and closed my mail program. But n, I had to look inside the folder. I am now regretting this decision. *sigh*, I can see the thread has gone clongie 'round the blonger, so all I'll have to say is we should still try to maintain the choice for users. But, in order to evaluate what amount of effort is needed to maintain that choice, we need to know what packages break on such a setup, what the level of effort needed to fix them is, and do those fixes impact the non-split crowd. Create like, a table on the Wiki or some kind of metadata property per-package that can contain a boolean or tri-state flag indicating whether it works or doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. Something. Once we know this, then we can work out what's the minimum system that can be successfully run on split-usr. Then, knowing *that*, we can see if that system can be supported by our @system target or some minimal subset of @world. As new package versions come out of upstream, we update this metadata with changes to the split-usr status, and this then provides a history of the more or less amount of difficulty needed to maintain support for split-usr, and *then*, we can make an objective decision to continue supporting or not supporting the capability. As for me, I am flat out ruling out a full-reinstall of all my systems. I have fixed disk partition layouts on all of them that cannot be re-arranged unless I tar up each filesystem and temporarily move it off, then rebuild the MD-RAID and reformat the filesystems. I am simply not going to do that on my many SGI systems, and whatever facet of upstream, whether it's some core GNU package or RH itself, can go pound sand for all I care. I'll go back to a static /dev and I'll manually mknod any missing devices if I have to. You know it's getting ridiculous when you can maintain a Windows/NTFS partition layout easier than a Linux one. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org 6144R/F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] Re: usr merge
On Sun, Apr 10, 2016 at 5:37 AM, Duncan <1i5t5.dun...@cox.net> wrote: > > Tho with the initr*, I did go the dracut route myself. But I'm still not > entirely convinced that I wouldn't have been better off rolling my own, > as I'm still not entirely comfortable with the level to which I > understand, or more accurately don't understand, dracut. Tho I do > understand it well enough to have cut it down to the modules I need, > only, but I still don't understand the scripts at the level I know I > would had I created them myself... > Well, by this logic you ought to be writing your own kernel as well. How else are you going to deal with a kernel panic? :) Dracut is pretty straightforward though and fairly well-documented. It runs through a series of stages like detecting devices, mounting filesystems, and so on. Plugins can run at any stage, and they also can add files to the initramfs at time of creation. The advantage of using something standard is that somebody else has probably already thought of all the edge cases already, and they'll fix the bugs as well. If you contribute your plugin upstream they'll take care of it for you as well. So, you can just put one line in a config file to add /usr/bin/btrfs to your initramfs and the built-in code will figure out that you'll need /usr/lib64/liblzo2.so.2 installed there as well. Of course, that is just an example as somebody else has already written the btrfs module. I was running into some kind of strange md-raid behavior ages ago and worked around it by writing a quick module: https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/ That one is a bit of a hack - I suspect there was some underlying bug in udev or something preventing the raid from being set up by the normal module. I suspect that module would no longer be needed (I've since moved on to btrfs). However, it demonstrated how dracut plugins work - you just define hooks that get run at the appropriate phase. -- Rich
[gentoo-dev] Re: usr merge
Rich Freeman posted on Sat, 09 Apr 2016 21:07:46 -0400 as excerpted: > On Sat, Apr 9, 2016 at 8:09 PM, J. Roeleveld wrote: >> >> I actually write my own initramfs because neither dracut not genkernel >> end up with a convenient boot system. >> >> I have 2 disks, both encrypted. >> I prefer only to enter the decryption password once. Both Dracut and >> Genkernel insist on asking for the password/key for every single disk. >> >> > You can of course roll your own, but I imagine that it would be more > straightforward to just write your own dracut plugin. They're basically > just scripts that run at whatever boot stage you define. > You might also just be able to modify the existing plugin. The problem with writing your own plugin, is that then you have to understand /both/ what you want to do, and the system you're writing a plugin for, in sufficient depth to actually do so. Whereas if you write the functionality directly, you only have to, by the end, understand the functionality you're implementing, nothing else. True, you'll probably be implementing something a bit broader than the plugin, and will probably learn a bit more about the topic as you do it, but if that was the intent in the first place, you'll definitely have achieved it when you're done, while if you write the plugin, you'll only have a so-so knowledge of two things, instead of a more in depth knowledge of the one you were actually interested in. This is actually one of the problems I had trying to learn firewalling on Linux, back when I switched from MS. I knew the general concepts, and indeed had played with them on my DSL router, back in the day, well enough to be the reference person from which many others learned. But I just couldn't get the hang of the various firewall helper scripts, shorewall and the like, because they were simply too far from the action, and I would have had to effectively learn both them and Linux IPTables, particularly when the helper script didn't cover what I wanted to do in its simplified model. When I finally gave up on the so-called helper scripts and decided to try IPTables directly, it was *MUCH* simpler, because now I was just learning ONE thing! Similarly, I learned kernel suspend/hibernate and resume by reading the kernel docs and writing my own script. Once I was part way done, I was able to actually understand the complex suspend-tools or whatever script far better, and could actually borrow a few tricks from it, whereas when I started I tried reading it and it was just trying to do far too much fancy stuff, far too far from the real action, for me to properly master the real kernel action that the fancy script covered. Grub2 same story. I ended up learning the advanced scripting stuff and doing my own thing, eventually install.masking the normal user-side config stuff, which wouldn't let me do what I wanted to do, without having to learn BOTH the advanced scripting and the user-side layout, because the userside stuff was too dumbed down to expose what I needed to do what I wanted to do. Tho with the initr*, I did go the dracut route myself. But I'm still not entirely convinced that I wouldn't have been better off rolling my own, as I'm still not entirely comfortable with the level to which I understand, or more accurately don't understand, dracut. Tho I do understand it well enough to have cut it down to the modules I need, only, but I still don't understand the scripts at the level I know I would had I created them myself... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: usr merge
Dale posted on Sat, 09 Apr 2016 13:42:37 -0500 as excerpted: > James Le Cuirot wrote: >> On Sat, 9 Apr 2016 12:09:38 -0400 waltd...@waltdnes.org wrote: >> I never really got the mentality that using an initramfs is a burden. >>> One more piece of software that can go wrong. You have to >>> maintain+configure it; e.g. sync software and library versions with >>> what's on the rest of the system. >> Errm, have you ever actually used dracut? >> >> dracut --kver 4.5 >> >> Wow, that was hard! It requires zero configuration and that's true even >> if you've got LVM on top of LUKS on top of RAID or something equally >> complex. If you're already running that kernel version, you don't even >> need to specify it. >> > FYI. I've had those to fail too. As Walt said, just one more thing to > fail. And more to the point, if all you know about dracut is dracut --kver 4.5, then you're not going to be able to _fix_ those failures. Some years ago I tried lvm2 as well as mdraid. I quickly ejected lvm2 from my system and future plans (keeping mdraid), because it was simply too complex for me as an admin to be confident in my ability to work with it without fat-fingering something, under the extreme pressure of a disaster recovery situation, possibly without proper access to manpages and other documentation due to the disaster recovery I was working thru. Waltdnes has a point, the same point I learned then. As a responsible admin, if the system's too complex to be understood well enough to be confident in one's ability to restore in a disaster recovery situation with limited or no access to manpages and similar documentation, it's too complex. A reasonable system is one you understand well enough to confidently manage it in those sorts of situations. Otherwise it's simply too complex for your skill level as an admin. And understanding an initr* well enough to confidently deal with a disaster recovery situation, possibly/likely without access to documentation (it's a disaster recovery, after all!) is definitely *well* beyond "dracut --kver 4.5" level, so he's very right to be worried about it. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-dev] usr merge
On Sun, 10 Apr 2016 02:09:35 +0200 "J. Roeleveld" wrote: > I actually write my own initramfs because neither dracut not > genkernel end up with a convenient boot system. > > I have 2 disks, both encrypted. > I prefer only to enter the decryption password once. Both Dracut and > Genkernel insist on asking for the password/key for every single disk. Dracut on RHEL actually handles this out of the box. Might be worth finding out how. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpwkCLInTFjv.pgp Description: OpenPGP digital signature
[gentoo-dev] Re: usr merge
Nicolas Sebrecht posted on Sat, 09 Apr 2016 14:44:25 +0200 as excerpted: > On Fri, Apr 08, 2016 at 07:58:35AM +, Duncan wrote: > >> > I would also re-iterate, as I'm sure you're aware .. there ARE >> > differences between sbin and bin .. unless of course you spend all >> > your time in a Rooted VM where it doesn't matter if you accidentally >> > trash your system. Some of us maintain a sensible user/superuser >> > distinction for a variety of reasons, and simplifying your filesystem >> > to suit some particular package style doesn't really sound like good >> > reasoning for causing a lot of headaches for maintainers and a distro >> > overall. >> >> But... the real important distinction in terms of user vs. superuser >> executables is file ownership and permissions, not the directory >> they're in. > > No. With a lightweight / I can install systems with two root filesystems > that I rsync once I'm sure there's no regression. If one won't boot > after an upgrade, I can just reboot and select the other root filesystem > in grub. > > This is much more easy than anything else. Actually I do precisely that, with / itself, which here includes pretty much everything the package manager installs (with the exception of a few things in /var that need to be writable in normal operation, that are symlinked to /home/var as my / is normally read-only mounted), including the package database itself, so everything stays in sync with the package database tracking it, on the same filesystem. =:^) $ df / Filesystem 1M-blocks Used Available Use% Mounted on /dev/sda5 8192M 2819M 5132M 36% / That's the working-copy. It's actually a two-device btrfs raid1, 8 GiB per device (8 GiB partition on each of two SSDs). That already gives me device redundancy. The first backup is another 8 GiB, again actually two-device btrfs raid1, 8 GiB per device, on another partition on each of those ssds. That gives me fat-finger and broken update redundancy. The second backup is an 8 GiB reiserfs on spinning rust, giving me both filesystem type redundancy because btrfs is still stabilizing, and a second backup in case disaster strikes when I'm actually updating the primary backup, taking out both it and the working copy. All three are independently bootable from grub2 as installed on all three devices, using the working copy /boot on one of the ssds, the primary backup /boot on the other ssd, or the secondary backup /boot on the spinning rust. /home similarly has a working copy raid1 btrfs on the ssds, a primary backup on the ssds, and a reiserfs secondary backup on spinning rust. There are further backups on USB tho I don't keep them as current so if I actually had to fall back to them I'd have some work ahead of me. Actually, I don't even have to switch to the grub2 commandline to switch between the three, either. They're all selectable directly from my customized grub2 menu, as is init=/bin/bash, systemd emergency and rescue mode, etc. Of course I can go grub commandline if needed, but it's not needed for those entries as they're already available in the menu I've setup. And of course the grub installation and corresponding /boot I use is selectable directly from the BIOS. What's nice about this is that the 8 GiB root is plenty big enough to hold the entire working system, including all manpages, the X and KDE install, etc, so not only do I have full documentation to work with while I'm recovering my broken root, but I have a full X and kde-plasma, which with /home or one of its backups gives me a fully customized kde install as well. So I can load up X/kde-plasma, and fire up youtube for full- monitor viewing say some club music videos to keep me awake on the 42- inch, while I work from one of the backups to recover the working copy with multiple konsole terminals on the 48-inch below it, and the system performance graphs display on the 21-inch off to the side. Try doing all /that/ in recovery mode from your "lightweight" / backup. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Re: usr merge
Luca Barbato posted on Sat, 09 Apr 2016 15:03:15 +0200 as excerpted:
> On 09/04/16 14:37, Rich Freeman wrote:
>> I've certainly haven't had many problems with dracut. When it fails it
>> is usually because I'm doing something ELSE that is off-the-wall and it
>> just doesn't have a plugin for it yet. (And in those cases it isn't
>> like the kernel tends to get it right without an initramfs.)
>>
>> I'd certainly want to test it on a merged /usr, but I'd be surprised if
>> it doesn't work, since it was designed to run on distros that are using
>> a merged /usr.
>
> I think that should be the first thing to do not the last one =)
FWIW, dracut works just fine with a "reverse-merged" /usr (usr -> .), as
well as the bin/sbin merge. And if it works with that, it'll certainly
work with (normal) merged usr, as AFAIK upstream's fedora/rh sponsored.
>> In an ideal world, you might argue that / should just be a tmpfs or
>> something almost as ephemeral. It is just a place you hang everything
>> else off of.
> That would be the core concept, but then you can just not have /bin
> /sbin /lib
That's in the context of (forward) /usr merge, which would make all those
symlinks to the appropriate /usr location anyway. Those symlinks could
of course be created dynamically, as could the various mountpoint
directories.
Of course /etc would have to be dynamically mounted in that scenario as
well, but that's not a big issue as long as there's an initr*
I actually thought about doing a tmpfs-based / here, or effectively just
never doing the pivotroot off the initramfs, with everything else
dynamically mounted over top the initramfs dirs, but decided to go a
different way instead, putting (almost) everything installed by the PM
on /, and doing a reverse-/usr-merge, with /usr -> . , with / then ro-
mounted by default. Seemed simpler for what I wanted to do than the
tmpfs or stay-on-initramfs / route.
>> The thing I like about the merge is that it basically puts all your
>> distro-supplied stuff in one place. /usr basically becomes the OS
>> minus state. If things started out that way and you just had a short
>> stub loader that gets things initialized, and I were arguing that
>> instead of that little initialization stub you should break up /usr so
>> that the root count mount /usr, would that sound all that compelling? I
>> think having it all in one mountpoint seems a lot more compelling.
>
> you cannot ever have everything in 1 mount point, you just move the
> problem somewhere else you notice less (initramfs), but the problem
> remains and either is solved or not.
>
> having everything in /usr and then copy it over ${somewhere} is there,
> it can be debated if /bin or initramfs is the best place to put it.
I suppose many of us have made that point at least to ourselves, at some
point. =:^)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
