Re: [gentoo-dev] Patching vanilla-sources with genpatches in catalyst
>From: alexmcwhir...@triadic.us >Sent: Saturday, September 17, 2016 3:48 PM >To: gentoo-dev@lists.gentoo.org >Subject: [gentoo-dev] Patching vanilla-sources with genpatches in catalyst > >For my sparc64 port, i need a livecd with kernel 4.8-rc6 that is patched >to work on a livecd. I'm not going to expect gentoo-sources to be What patches were you thinking are needed for a live CD? I would expect that either a vanilla or gentoo kernel would handle being on a live CD without problems. What is giving you trouble that you are aiming for 4.8-rc6, I have not seen many problems. Though the most recent gentoo live CD will hang one of my machines due to radeon driver in the 3.14.14 kernel not accessing I/O correctly ( it did not use the access functions just dereferenced a pointer). The only other quirk I remember off the top of my head is the 2nd required initramfs that the netboot kernel target needs, but even that works with a empty file. >released for this kernel, and hacking up the kernel eclass for 4.8-rc6 >doesn't seem like the right way to go. So i was looking for some input >on the best way to do this. > >The first thing that comes to mind is to make a vanilla-sources ebuild >for 4.8-rc6 and use that in the spec file. Then after doing a Are you following some set of instructions on building stages and a live CD? >livecd-stage1 put the genpatches in /etc/portage/patches/... and let the >epatch_user interface take care of the rest. > >Does anyone have any better ideas? > Jim McMechan
Re: [gentoo-dev] Dropping ia64/ppc/sparc profiles to dev/exp
> > > The easiest solution is for the arch team to remove keywords until they > > > have a reasonable response time again. And if the arch team doesn't do > > > that by itself, well, ... > > > > > > Having one-man teams block everybody else hurts Gentoo as a whole. > > > > We have appropriate hardware if people wanna do the work, jut go & make > > things better :), I do not think someone from existing arch teams has > > something against that > > I'm sorry, but I can think of about a dozen things where my Gentoo time is > spent in a more useful way. > > (With useful being not so much defined with what I personally find fun, but > with > what brings Gentoo and its userbase forward.) > > Prove to me that more than 10 persons are interested in Gentoo on any of sh, > s390, m68k, ia64, sparc. well I currently have Gentoo up on bunch of arm v5,v6,v7 1 arm64 5 mips 1 mips64 2 ppc 1 ppc64 2 sparc (64bit) bunch of x86 bunch of amd64 and am still playing with 1 sh3 So how many do I count as 1,2,3-12? ;) Silly Notes: While I personally considered IA64 useless shortly after arrival, Intel has just (5/11/17) announced another bump in that family the 9700 Series. SPARC hardware is stilling being sold by both Fujistu and Oracle, e.g. the M12 series released in April, but it is mostly sold to those already invested in it. Looking I also saw updates in January for the SPARC32:LEON4 processor The OpenSPARC.net SPARC64:T1 & T2 seem less active, may just be my lack of google-fu PowerPC is still running along, mostly embedded for the 32bit version, and IBM still likes to sell big iron with 64bit Power processors. SuperH is even having a tiny bit of a renaissance, what with the J-core.org people building a VHDL open source version for FPGA/ASIC Please note: that I am not objecting to having arch keywords on only the base @system packages for the minor archs, hopefully it will save a small mountain of effort. Would it be worthwhile to have an other/minor/general keyword shared among the arches? e.g. have keywords always try their arch keyword first and if it is not present look for other/minor/general keyword? sort of like */~*/-* except if your arch is mentioned use that instead. It could make things easier many packages /should/ not care about which arch it is exactly. I could see it as trying a stable request: 3 arches send yes and dead silence from everybody else... So after a reasonable time mark the 3 that replied and let everyone who did not follow the general keyword until they reply. My current interpretation: amd64 recent desktops/laptops x86 most older desktops/laptops arm most cellphones/tablets arm64 upcoming cellphones/tablets e.g. raspberry pi3. not yet common running 64bit. mips gobs of routers & embedded systems. everything else niche markets <- Hi, I fit in here a lot. Jim McMechan
Re: [gentoo-dev] Call for help in testing - sparc + gnutls-3.5
Hi, Matt I have been helping Alon with testing on one of my sparc boxes, what is broken? is there a log of #gentoo-sparc or a gentoo-sparc mailing list somewhere? on a side note where should I ask about a respin of the iso, the old kernel will crash if a ATI graphics card is present. Thank you, Jim McMechan From: Matt Turner Sent: Saturday, July 15, 2017 12:04 PM To: gentoo development; Gentoo infra Subject: Re: [gentoo-dev] Call for help in testing - sparc + gnutls-3.5 On Sat, Jul 15, 2017 at 11:33 AM, Alon Bar-Lev wrote: > Hello, > > I am looking for someone that is using gentoo on sparc and is willing to > help out to resolve an issue[1] of gnutls-3.5 with sparc so that we can drop > gnutls-3.3 from tree. > > I tried to create a bootable sparc qemu gentoo image and failed, so need > someone with a live system. > > Regards, > Alon > > [1] https://bugs.gentoo.org/show_bug.cgi?id=613418 I don't think our sparc box has been fixed (down since March). Otherwise I would continue helping. I'll ask in #gentoo-sparc for help.
Re: [gentoo-dev] Reviving the Sandbox project
On Fri, Sep 22, 2017 at 5:27 AM, Rich Freeman wrote: >On Fri, Sep 22, 2017 at 7:38 AM, Sergei Trofimovich wrote: >> >> Some other distros try harder to isolate build environment either >> through chroot and/or private mount/user/network namespace that >> contains only explicitly specified files in build environment. >> >> That would require more cooperation from package manager to fetch >> list of all visible depends. >> > >I definitely think this is something that would be VERY useful to >have, because it makes build-time dependency issues almost impossible. > >However, you don't need that complete solution just to have a sandbox. >You could simply create a container with the entire contents of the >main filesystem, but read-only, with the exception of the build area. >That would replicate the functionality of the current sandbox and >would be easier than building out just the files specified in the >dependencies. > >The main issue I see with making it dependency aware is performance. >You would need to walk all the dependencies and their run-time >dependencies, and the system set, and then individually bind-mount the >files that belong to them read-only into the container. That isn't >necessarily difficult, but I imagine that it would be slow. Walking >the dependencies obviously already happens during resolution so that >could probably be cached. You could also cache the list of files for >the system set, and you could even maintain a snapshot of these that >is used as the base of the container (somebody would need to work out >the savings of doing this vs the cost of updating it every time a >system set package changes). > >However, the main thing I wanted to point out is that you don't need a >dependency-aware solution to just replace the existing sandbox. > >> I like clear sandbox error reporting. > >As far as error reporting goes, you would get kernel-level errors like >attempting to write to a read-only bind mount, or trying to read a >file that doesn't exist. To a portage dev it should be pretty obvious >what is going on. You could add canned text to the portage error >output at the end. I'm not sure how visible the problems would be to >portage except to the degree that the build system makes them visible >- it would just see make terminate with a non-zero return. > >I would think that containers would be almost completely bug-free >though. The only thing I could see as an issue is some build system >that relies on IPC with the host, network access, etc. Namespaces >themselves are very robust, and the kernel already looks at every >process as belonging to a set of namespaces even in the default case >where you only have one set of namespaces on the system, so they don't >involve different code paths/etc. > >-- >Rich Another possibility, could be to have the sandbox be an overlayfs not of the build directory but of the entire system starting at "/" and stick that into the container, only CLONE_NEWNS looks to be needed. A container with all writes going to the upper layer of the overlay could be safe against even #RM -RF / and other disasters, while still tracking what is happening during the build. Should performance be too much of a problem having a bind mount of the build directory on top of the overlay should give normal performance to everything that is obeying good practice. After the build completes the directory that was mounted as upper could be scanned to find any wayward writes that had occurred... This would not require LD_PRELOAD or ptrace eliminating most of the "high magic" currently used. Just yesterday I had a lot of ptrace failures while trying something odd with ROOT= and custom USE Jim McMechan
[gentoo-dev] An example overlayfs sandbox test
Hello, I thought a example of how a overlay sandbox could work was in order. ### # load the overlayfs filesystem for this test modprobe overlay # make the directories for the test mkdir -p /var/tmp/upper /var/tmp/work /mnt/gentoo # now create a separate mount namespace non-persistent unshare -m bash # setup the overlay mount -toverlay -oupperdir=/var/tmp/upper/,workdir=/var/tmp/work/,lowerdir=/ overlay /mnt/gentoo/ # since I don't care about protecting /var/tmp/portage # put the original on top of the overlay for better performance maybe? mount -o bind /var/tmp/portage /mnt/gentoo/var/tmp/portage # then like the handbook cd /mnt/gentoo mount -t proc proc proc mount --rbind /sys sys mount --rbind /dev dev #finally change into the protected sandbox chroot . bash # mess up the system exit # the chroot exit # the unshare ### done. This version allows the sandbox to work with the special files in /dev, /proc, /sys other options are available for example a second separate dev/pts and dev/shm submounts When you exit the chroot and then the unshare, the /var/tmp/upper directory will contain all the changes made while in the chroot. Enjoy, Jim McMechan
Re: [gentoo-dev] An example overlayfs sandbox test
On Fri, Sep 22, 2017 at 5:18 PM, Rich Freeman wrote: >On Fri, Sep 22, 2017 at 4:43 PM, James McMechan > wrote: >> >> # now create a separate mount namespace non-persistent >> unshare -m bash >> > >If you're going to go to the trouble to set up a container, you might >as well add some more isolation: > >unshare --mount --net --pid --uts --cgroup --fork --ipc --mount-proc bash > >I'm not sure how much of a hassle mapping a uid namespace would be or >if it would really add anything, especially if this chroots to portage >right away. > >-- >Rich Well mostly it was an example, I am not actually very good at containers. the more stuff is isolated the more it needs to be setup. The mount namespace is the whole point of the example I would not want to change the networking, it should already be working and I would be better served by not messing with it. portage should not care about the --pid --uts(hostname/domainname) --cgroup or --ipc The --mount-proc is not really helpful as I immediately remount the entire "/" filesystem at /mnt/gentoo and chroot into it after custom setup of proc sys and dev Now I could see a use for --map-root-user --user, then portage could run as root in the container with the least danger by being user portage:portage outside. Enjoy Jim McMechan
Re: [gentoo-dev] An example overlayfs sandbox test
On Fri, Sep 22, 2017 at 7:26 PM, Rich Freeman wrote: >On Fri, Sep 22, 2017 at 6:29 PM, James McMechan >wrote: >> On Fri, Sep 22, 2017 at 5:18 PM, Rich Freeman wrote: >>>On Fri, Sep 22, 2017 at 4:43 PM, James McMechan >>> wrote: >>>> >>>> # now create a separate mount namespace non-persistent >>>> unshare -m bash >>>> >>> >>>If you're going to go to the trouble to set up a container, you might >>>as well add some more isolation: >>> >>>unshare --mount --net --pid --uts --cgroup --fork --ipc --mount-proc bash >>> >> >> I would not want to change the networking, it should already be working >> and I would be better served by not messing with it. >> > >Well, that's the point. You don't want networking to work during the >build phases. Maybe you'd want it for the test phase. In any case, >you would definitely want control over that in the ebuild. Random >build systems shouldn't be talking to the internet, if for no other >reason than to avoid it fetching stuff to install that bypasses the >integrity checks. > >If you create a new net namespace by default it won't have any >interfaces other than lo. Err, perhaps I was thinking of doing the switch too early, I had been thinking of making sure networking works for the fetch phase... >> The --mount-proc is not really helpful as I immediately remount the entire >> "/" filesystem at /mnt/gentoo and chroot into it after custom setup of proc >> sys and dev >> > >As long as it doesn't see the host /proc then you're fine. You just >wouldn't want to have it mounted into the container. I am pretty sure a /proc needs to be mounted inside of any container if you want builds to work. mount, ps, and a lot of other stuff does not work without it. I think you mean that only the container /proc is mounted. >> Now I could see a use for --map-root-user --user, then portage could run as >> root in the container with the least danger by being user portage:portage >> outside. >> > >Certainly, but that takes a bit more work, and to be honest I've never >actually bothered to get it working using unshare. It probably isn't >too difficult. You were right, I just tried it and it seemed easy enough. >The options I listed basically "just work" without any real additional effort. I like the lack of effort part ;) I want the computer doing the work not me. >So, we're drifting in topic, but as long as we're coming up with >nice-to-have utilities it would be lovely if our install CDs had >something similar to systemd-nspawn to set up a container instead of a >chroot for performing the install. If nothing else it would make >mount cleanup easier when you're done. I imagine it would just be a >bit of shell scripting with util-linux on the CD - while nspawn is >bundled with systemd you don't need any of its fancier features for >doing an install. Ok, but what is the advantage? the mounts disappearing when you exit the container, and anything else? >Back on topic - none of this stuff will work on FreeBSD, which might >be an issue for those running Gentoo on that kernel. Ditto for Prefix >I suppose. I suspect that jails/etc would also do the job but you'd >need some arch-dependent code to set up the container. Just about all >of these tricks are involving non-POSIX functionality. Actually, I'm >not sure if even the current LD_PRELOAD approach is completely >portable, though it has the advantage of being entirely in userspace. I don't see any BSD stuff on the download page, so it slips my mind. I have not used any of the BSD derived systems in quite a few years. A quick glance shows they are running things like docker so container like functionality exists, and I seem to remember that Linux was late to the overlay stuff, the freeBSD mount_unionfs says it went in as mount_null in 4.4BSD according to the man page so that is present also. Regarding prefix, or maybe RAP does sandbox even work there? It looks like prefix uses the host system's library... would either method sandbox currently uses work? RAP appeared to have it's own C library so it should be possible there I do not remember if sandbox worked under RAP it the whole project felt very experimental when I tried it. I think non-linux systems did adopt LD_PRELOAD and ptrace some, I do not think either was in any of the POSIX versions though I did not look much at the last version, it seemed mostly irrelevant by then. Enjoy, Jim McMechan
