Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 21 January 2015 at 22:44, Rich Freeman wrote: > On Wed, Jan 21, 2015 at 9:00 AM, Sam Bishop wrote: >> >> I don't see why it can't be all the combinations, the issue is >> storage, and the storage costs could be a lot lower than expected >> given how hard it is to guess. > > I don't believe that binpkg filenames contain the use flag settings, > and I'm not sure that given multiple copies of a binpkg with different > filenames portage goes through them and figures out which ones are > which. This isn't an area I have looked into seriously. However, it > obviously would be a blocker for getting what you propose to work, > even theoretically. > I'll quote from the binpkg docs: >> Next to these, portage will check if the binary package is built using the >> same USE flags as expected on the client. If a package is built with a >> different USE flag combination, portage will either ignore the binary >> package (and use source-based build) or fail, depending on the options >> passed on to emerge So I'm fairly sure that implies they can coexist based on the directory structure. - http://wiki.gentoo.org/wiki/Binary_package_guide#The_PKGDIR_layout One big concern would be having a HUGE Packages metadata file and making the look up too slow. I'm not sure how big that file could get before things became an issue. http://wiki.gentoo.org/wiki/Binary_package_guide#Pulling_packages_from_a_binary_package_host > > I don't really see the value in having EVERY combination of use flags > on call though. > > Practically speaking I doubt this could be done. You're talking about > a LOT of combinations. > > However, I think it would be very useful to have a binpkg repository > all the same. Perhaps have one for each of a few common profiles with > the default flags. That alone would be a significant undertaking. > > Just about everybody who has talked about running Gentoo in a > datacenter has set up a binpkg repository. They may very well deviate > from the default USE flags, but for the most part they try to keep > their systems identical. They would build updates as binpkg, install > to a test system, and after testing deploy them to production and that > would of course go quickly. > > I have a script I use to build binpkg nightly for the day's updates. > That lets me review updates and deploy them quickly. Any rebuilds/etc > still take time, but the bulk of my updates are very fast this way > with minimal time spent staring at the screen. This would be another > route to take if your really did need highly varied deployments. > > -- > Rich >
Re: [gentoo-user] Picking dom0 OS for new XEN server
On 22. jan. 2015 08:49, Tomas Mozes wrote:
On 2015-01-22 08:32, Håkon Alstadheim wrote:
Pondering dom0 os for new Xen server (Xeon e5 2620 v3 cpu) . Server is
for home use. (routing/firewall; dns/dhcp, mail servers, etc. etc;
linux gui desktop; linux media-server; windows as separate domains).
I have basic skills in debian, and gentoo. A bit rusty on the
rpm-based distros. Totally unfamiliar with *BSD. Total newbie in XEN
So, given this list is bound to be a bit biased, : why would I pick
gentoo for dom0, and what version of XEN would I use ?
Picking gentoo for dom0 is probably the same question as "why would I
use gentoo instead of other linux distribution". Me personally have
very good experience in running xen dom0 machines on gentoo. If you
separate your dom0 machine and the services are in domUs, then the
dom0 is very small, clean and easy to maintain (for years). Ask on the
debian list and I'm sure they will answer the same way so it's your
choice ;)
If it's a new server you may try the new 4.5 release. I don't know how
stable it is, I just started my first VM on 4.5 today, but I'm running
4.3 in production and 4.4 in pre-production. Since you don't need to
migrate in production, it's not a problem I believe.
Arye you using the 4.5 ebuilds from portage ?
-
Debian Jessie tells me (as an example):
# apt-cache search xen-hyp
xen-hypervisor-4.4-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.0-amd64 - The Xen Hypervisor on AMD64
xen-hypervisor-4.0-i386 - The Xen Hypervisor on i386
xen-hypervisor-4.1-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.1-i386 - Xen Hypervisor on i386
# apt-cache policy xen-hypervisor-4.4-amd64
xen-hypervisor-4.4-amd64:
Installed: (none)
Candidate: 4.4.1-6
Version table:
4.4.1-6 0
990 http://ftp.se.debian.org/debian/ jessie/main amd64 Packages
500 http://ftp.se.debian.org/debian/ unstable/main amd64 Packages
--
My gentoo box tells me:
# eix -e xen
* app-emulation/xen
Available versions: 4.2.5-r3^t ~4.2.5-r4^t *4.3.3-r3^t
~*4.3.3-r4^t ~*4.4.1-r4^t ~*4.4.1-r5^t ~*4.5.0^t {custom-cflags debug
efi flask pae xsm}
Homepage:http://xen.org/
Description: The Xen virtual machine monitor
--
So it seems that if I decide xen 4.5, gentoo might be less hassle ?
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 22 January 2015 at 01:54, Andreas K. Huettel wrote: > Am Mittwoch 21 Januar 2015, 20:36:55 schrieb Sam Bishop: >> So I've been thinking crazy thoughts. >> >> Theoretically it can't be that hard to do a complete package binhost for >> gentoo. >> >> To be clear, when i say complete, Im referring to building, all >> versions of all ebuilds marked stable or unstable on amd64, with every >> combination of use flags. > > Not enough. You will also have to build against every combination of > dependency subslots. > > e.g., different poppler, boost, icu, perl and many more versions... > > Which makes the task near impossible. > Not impossible, just more computationally demanding and requiring more storage. As I mentioned in another post, its not the task of building and storing all these I think will be the problem. Its can portage/emerge handle this? Is the current implementation of binhost inadequate to deal with such a massive binhost, would it require new utilities or code or a new version of the binhost metadata format. These are the kinds of things I feel make it challenging, not the simple demand for compute and storage. Those are a rather moot point when S3 is pennies a gigabyte and an AWS spot instance powered compile farm can be obtained relatively cheaply and if gifted to the Gentoo community even run at discount prices. > -- > Andreas K. Huettel > Gentoo Linux developer > kde, council > >
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 21 January 2015 at 23:53, Alec Ten Harmsel wrote: > I actually had kind of a cool idea while walking to the bus stop this > morning; a JIT Portage server that builds packages on demand. This would > require: > > * Writing a portage server > * Patching portage to connect to said server > Or... Before integrating it into portage, it could be a wrapper, lets call it 'premerge' for the sake of example. Calling premerge www-client/firefox-35.0[pulseaudio] would unpack the arguments, work out the relevant metadata, perhaps by parsing the output of emerge -pv, and then fetch the binaries from the big storage pool they live in, put them in the correct place for portage to find them, and then call portage in such a way it can find the prebuilt binary version we just provided it. If emerge itself is incapable of handling such a large binary prebuilt collection of packages, then I'm likely to explore this route for a while. > Basically, `emerge ` would send a message to the server "I need > www-client/firefox-35.0[pulseaudio]". The server would return the > tarball if already built, otherwise build it and then return it. This > would be reasonably complex to implement in practice, but it would let > everybody using the same binhost to run their own custom USE flags. > > Re more accurate numbers: dev-java/icedtea. Let's pretend building this > takes ~5 minutes (this is faster than my desktop can do it in RAM with 6 > hyper-threaded cores). There are 13 USE flags that are configurable if > you're using HotSpot; we'll ignore JamVM and CACAO. On a single server, > this would take nearly a month (28.44 days, exactly). > > Alec >
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On Thu, 22 Jan 2015 16:43:32 +0800, Sam Bishop wrote: > I'll quote from the binpkg docs: > >> Next to these, portage will check if the binary package is built > >> using the same USE flags as expected on the client. If a package is > >> built with a different USE flag combination, portage will either > >> ignore the binary package (and use source-based build) or fail, > >> depending on the options passed on to emerge > > So I'm fairly sure that implies they can coexist based on the > directory structure. - > http://wiki.gentoo.org/wiki/Binary_package_guide#The_PKGDIR_layout The package name is the same as the ebuild name but with a .tbz2 extension, so how could portage cope with multiple variants with different USE flags when there is only one name? There can be only one package per ebuild and either the USE flags match exactly or they do not. You could get away with this with a limited set of profiles by having a different $PKGDIR for each profile but to do it with random combinations would require some sort of middleware to handle the requests and place the specified packages where portage expects to find them. I think the check for USE flags is done using the IUSE and USE settings in the package metadata, so even if a USE flag you don't use is added to an ebuild, the package will no longer match. ISTR having to hack metadata in /var/db in the past to avoid a rebuild of *Office. -- Neil Bothwick When companies ship Styrofoam, what do they pack it in? pgppgIhzUwDYu.pgp Description: OpenPGP digital signature
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 22 January 2015 at 17:00, Neil Bothwick wrote: > On Thu, 22 Jan 2015 16:43:32 +0800, Sam Bishop wrote: > >> I'll quote from the binpkg docs: >> >> Next to these, portage will check if the binary package is built >> >> using the same USE flags as expected on the client. If a package is >> >> built with a different USE flag combination, portage will either >> >> ignore the binary package (and use source-based build) or fail, >> >> depending on the options passed on to emerge >> >> So I'm fairly sure that implies they can coexist based on the >> directory structure. - >> http://wiki.gentoo.org/wiki/Binary_package_guide#The_PKGDIR_layout > > The package name is the same as the ebuild name but with a .tbz2 > extension, so how could portage cope with multiple variants with > different USE flags when there is only one name? There can be only one > package per ebuild and either the USE flags match exactly or they do not. > > You could get away with this with a limited set of profiles by having a > different $PKGDIR for each profile but to do it with random combinations > would require some sort of middleware to handle the requests and place > the specified packages where portage expects to find them. > > I think the check for USE flags is done using the IUSE and USE settings > in the package metadata, so even if a USE flag you don't use is added to > an ebuild, the package will no longer match. ISTR having to hack metadata > in /var/db in the past to avoid a rebuild of *Office. > Thank you kindly Neil. You rephrasing what was right in front of my face in the docs finally lead to the lightbulb going off. Happens to all of us I suppose. The pkdir layout diagram isn't implying multiple versions of a single package, it is referring to multiple packages with a numeric shorthand. So this would require middleware, wrappers, or improvements to portage to cope with having overlapping packages like this. So interim functionality could be achieved with separate bin hosts directories for each of the baseline profiles with their default use flags. Once the infrastructure was stable then work could be undertaken to build some kind of wrapper, or enhancement to portage. > > -- > Neil Bothwick > > When companies ship Styrofoam, what do they pack it in?
Re: [gentoo-user] Openrc now on Arch LInux
2015-01-22 0:32 GMT+02:00 James : > It seems Openrc is spreading? > > > https://wiki.archlinux.org/index.php/OpenRC > > Anyone tested openrc on Arch linux? I have read in several places > that Arch linux has moved to systemd, exclusively? Did you read the web page's beginning ? :) " Note: Arch uses systemd by default. If you use OpenRC, please mention so while asking for help."
Re: [gentoo-user] Picking dom0 OS for new XEN server
On 2015-01-22 09:46, Håkon Alstadheim wrote:
On 22. jan. 2015 08:49, Tomas Mozes wrote:
On 2015-01-22 08:32, Håkon Alstadheim wrote:
Pondering dom0 os for new Xen server (Xeon e5 2620 v3 cpu) . Server
is
for home use. (routing/firewall; dns/dhcp, mail servers, etc. etc;
linux gui desktop; linux media-server; windows as separate domains).
I have basic skills in debian, and gentoo. A bit rusty on the
rpm-based distros. Totally unfamiliar with *BSD. Total newbie in XEN
So, given this list is bound to be a bit biased, : why would I pick
gentoo for dom0, and what version of XEN would I use ?
Picking gentoo for dom0 is probably the same question as "why would I
use gentoo instead of other linux distribution". Me personally have
very good experience in running xen dom0 machines on gentoo. If you
separate your dom0 machine and the services are in domUs, then the
dom0 is very small, clean and easy to maintain (for years). Ask on the
debian list and I'm sure they will answer the same way so it's your
choice ;)
If it's a new server you may try the new 4.5 release. I don't know how
stable it is, I just started my first VM on 4.5 today, but I'm running
4.3 in production and 4.4 in pre-production. Since you don't need to
migrate in production, it's not a problem I believe.
Arye you using the 4.5 ebuilds from portage ?
-
Debian Jessie tells me (as an example):
# apt-cache search xen-hyp
xen-hypervisor-4.4-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.0-amd64 - The Xen Hypervisor on AMD64
xen-hypervisor-4.0-i386 - The Xen Hypervisor on i386
xen-hypervisor-4.1-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.1-i386 - Xen Hypervisor on i386
# apt-cache policy xen-hypervisor-4.4-amd64
xen-hypervisor-4.4-amd64:
Installed: (none)
Candidate: 4.4.1-6
Version table:
4.4.1-6 0
990 http://ftp.se.debian.org/debian/ jessie/main amd64 Packages
500 http://ftp.se.debian.org/debian/ unstable/main amd64
Packages
--
My gentoo box tells me:
# eix -e xen
* app-emulation/xen
Available versions: 4.2.5-r3^t ~4.2.5-r4^t *4.3.3-r3^t
~*4.3.3-r4^t ~*4.4.1-r4^t ~*4.4.1-r5^t ~*4.5.0^t {custom-cflags debug
efi flask pae xsm}
Homepage:http://xen.org/
Description: The Xen virtual machine monitor
--
So it seems that if I decide xen 4.5, gentoo might be less hassle ?
Yes, 4.5 from portage. Only if you use EFI you need a patch that is not
in portage at the moment: https://bugs.gentoo.org/show_bug.cgi?id=534570
If you want debian, version 4.4 will be also great and they will supply
the packages soon I believe. I think the version (4.3, 4.4 or 4.5) is
not the crucial factor, each of them will work, only if you need the
newer features.
[gentoo-user] Re: Openrc now on Arch LInux
Rich Freeman gentoo.org> writes: > > Anyone tested openrc on Arch linux? I have read in several places > > that Arch linux has moved to systemd, exclusively? > Don't believe everything you read. You'll probably also read that > Gentoo doesn't use systemd. :) https://wiki.archlinux.org/index.php/OpenRC James
[gentoo-user] Re: Openrc now on Arch LInux
taozhijiang tp-link.com.cn> writes: >> https://wiki.archlinux.org/index.php/OpenRC >> Anyone tested openrc on Arch linux? I have read in several places >> that Arch linux has moved to systemd, exclusively? > why ask this question in Gentoo user land? Openrc is developed by Gentoo folks. [1] Some debian activity around openrc lately (Devuan) and others. As codes spread to multiple operating systems, the codes usually improves and newfeatures get added and the code becomes more 'robust' imho. I like openrc and researching about other usages (embedded linux too) is a curiosity I find enlightening. Openrc is part of gentoo, although it can and is being used elsewhere in other distros. [1] http://www.gentoo.org/proj/en/base/openrc/ James
[gentoo-user] Re: Openrc now on Arch LInux
Emre Eryilmaz piesso.com> writes: > > Anyone tested openrc on Arch linux? > Did you read the web page's beginning ? :) > " Note: Arch uses systemd by default. If you use OpenRC, please > mention so while asking for help." I think that stating what codes you are using, that might affect helps one seeks, is pretty much accepted netiquette, ymmv. AS to the robustness of Openrc on Arch; well that is why I'm asking here if anyone has experiences with Openrc on Arch (actually testing Openrc on Arch Linux) and not just conjecture or attitude about it. Just (test) experiences, ok? James
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 22 January 2015 7:20:07 PM AEST, Sam Bishop wrote: >On 22 January 2015 at 17:00, Neil Bothwick wrote: >> On Thu, 22 Jan 2015 16:43:32 +0800, Sam Bishop wrote: >> >>> I'll quote from the binpkg docs: >>> >> Next to these, portage will check if the binary package is built >>> >> using the same USE flags as expected on the client. If a package >is >>> >> built with a different USE flag combination, portage will either >>> >> ignore the binary package (and use source-based build) or fail, >>> >> depending on the options passed on to emerge >>> >>> So I'm fairly sure that implies they can coexist based on the >>> directory structure. - >>> http://wiki.gentoo.org/wiki/Binary_package_guide#The_PKGDIR_layout >> >> The package name is the same as the ebuild name but with a .tbz2 >> extension, so how could portage cope with multiple variants with >> different USE flags when there is only one name? There can be only >one >> package per ebuild and either the USE flags match exactly or they do >not. >> >> You could get away with this with a limited set of profiles by having >a >> different $PKGDIR for each profile but to do it with random >combinations >> would require some sort of middleware to handle the requests and >place >> the specified packages where portage expects to find them. >> >> I think the check for USE flags is done using the IUSE and USE >settings >> in the package metadata, so even if a USE flag you don't use is added >to >> an ebuild, the package will no longer match. ISTR having to hack >metadata >> in /var/db in the past to avoid a rebuild of *Office. >> > >Thank you kindly Neil. You rephrasing what was right in front of my >face in the docs finally lead to the lightbulb going off. Happens to >all of us I suppose. The pkdir layout diagram isn't implying multiple >versions of a single package, it is referring to multiple packages >with a numeric shorthand. So this would require middleware, wrappers, >or improvements to portage to cope with having overlapping packages >like this. So interim functionality could be achieved with separate >bin hosts directories for each of the baseline profiles with their >default use flags. Once the infrastructure was stable then work could >be undertaken to build some kind of wrapper, or enhancement to >portage. There was a discussion recently on the portage-dev list regarding storing multiple versions with different use flags in a pkgdir. There's an open bug in bugzilla too, I believe, but I cannot find the reference right now; if I can I'll follow up. I think the summary was that the Packages file is able to index multiple versions of a package, but the tooling to create and manage packages needs some improvement. (Don't quote me on that though!) > >> >> -- >> Neil Bothwick >> >> When companies ship Styrofoam, what do they pack it in? -- :b
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 22 January 2015 8:50:29 PM AEST, Bruce Schultz wrote: > > >On 22 January 2015 7:20:07 PM AEST, Sam Bishop >wrote: >>On 22 January 2015 at 17:00, Neil Bothwick wrote: >>> On Thu, 22 Jan 2015 16:43:32 +0800, Sam Bishop wrote: >>> I'll quote from the binpkg docs: >> Next to these, portage will check if the binary package is built >> using the same USE flags as expected on the client. If a package >>is >> built with a different USE flag combination, portage will either >> ignore the binary package (and use source-based build) or fail, >> depending on the options passed on to emerge So I'm fairly sure that implies they can coexist based on the directory structure. - http://wiki.gentoo.org/wiki/Binary_package_guide#The_PKGDIR_layout >>> >>> The package name is the same as the ebuild name but with a .tbz2 >>> extension, so how could portage cope with multiple variants with >>> different USE flags when there is only one name? There can be only >>one >>> package per ebuild and either the USE flags match exactly or they do >>not. >>> >>> You could get away with this with a limited set of profiles by >having >>a >>> different $PKGDIR for each profile but to do it with random >>combinations >>> would require some sort of middleware to handle the requests and >>place >>> the specified packages where portage expects to find them. >>> >>> I think the check for USE flags is done using the IUSE and USE >>settings >>> in the package metadata, so even if a USE flag you don't use is >added >>to >>> an ebuild, the package will no longer match. ISTR having to hack >>metadata >>> in /var/db in the past to avoid a rebuild of *Office. >>> >> >>Thank you kindly Neil. You rephrasing what was right in front of my >>face in the docs finally lead to the lightbulb going off. Happens to >>all of us I suppose. The pkdir layout diagram isn't implying multiple >>versions of a single package, it is referring to multiple packages >>with a numeric shorthand. So this would require middleware, wrappers, >>or improvements to portage to cope with having overlapping packages >>like this. So interim functionality could be achieved with separate >>bin hosts directories for each of the baseline profiles with their >>default use flags. Once the infrastructure was stable then work could >>be undertaken to build some kind of wrapper, or enhancement to >>portage. > >There was a discussion recently on the portage-dev list regarding >storing multiple versions with different use flags in a pkgdir. There's >an open bug in bugzilla too, I believe, but I cannot find the reference >right now; if I can I'll follow up. > >I think the summary was that the Packages file is able to index >multiple versions of a package, but the tooling to create and manage >packages needs some improvement. (Don't quote me on that though!) Found it http://thread.gmane.org/gmane.linux.gentoo.portage.devel/5031 https://bugs.gentoo.org/show_bug.cgi?id=150031 -- :b
Re: Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
Am Donnerstag 22 Januar 2015, 16:50:45 schrieb Sam Bishop: > On 22 January 2015 at 01:54, Andreas K. Huettel wrote: > > Am Mittwoch 21 Januar 2015, 20:36:55 schrieb Sam Bishop: > >> So I've been thinking crazy thoughts. > >> > >> Theoretically it can't be that hard to do a complete package binhost for > >> gentoo. > >> > >> To be clear, when i say complete, Im referring to building, all > >> versions of all ebuilds marked stable or unstable on amd64, with every > >> combination of use flags. > > > > Not enough. You will also have to build against every combination of > > dependency subslots. > > > > e.g., different poppler, boost, icu, perl and many more versions... > > > > Which makes the task near impossible. > > Not impossible, just more computationally demanding and requiring more > storage. Well, exponential increase is exponential increase. * A libreoffice binary package with debug information has roughly 800Mbyte size * 2 libreoffice versions in the tree * libreoffice links against poppler, icu, boost (among other things) * poppler: 5 subslots, icu (soon) 3 subslots, boost 5 subslots in tree -> 75 combinations * libreoffice has 22 useflags and 4 extensions, plus three supported python variants -> 29 switches * REQUIRED_USE limits your combinations, let's conservatively guess 25 independent switches -> 2^25=33554432 use combinations Which ends up with roughly 2 Exabyte (10^9 GByte) of storage for all packages. -- Andreas K. Huettel Gentoo Linux developer kde, council
Re: [gentoo-user] Download of source for file-5.22 blocked by firewall?
On 1/21/2015 4:45 PM, Neil Bothwick wrote: > On Wed, 21 Jan 2015 11:58:05 -0500, Tanstaafl wrote: > >> Changed mirror setting in make.conf to: >> >> http://www.gtlib.gatech.edu/pub/gentoo/ >> >> and all is well now... >> >> Guess there is a problem with mirror.datapipe? > > That's why I have several mirrors defined, portage will try them in turn > so it doesn't matter if one fails. Understood, but this is the first time this has happened to me that I can recall in the last 10 years or so...
Re: [gentoo-user] Re: Openrc now on Arch LInux
2015-01-22 4:23 GMT-06:00 James : > Emre Eryilmaz piesso.com> writes: > > >> > Anyone tested openrc on Arch linux? > >> Did you read the web page's beginning ? :) >> " Note: Arch uses systemd by default. If you use OpenRC, please >> mention so while asking for help." > > > I think that stating what codes you are using, that might affect > helps one seeks, is pretty much accepted netiquette, ymmv. > > > AS to the robustness of Openrc on Arch; well that is why I'm asking > here if anyone has experiences with Openrc on Arch (actually testing Openrc > on Arch Linux) and not just conjecture or attitude about it. > > Just (test) experiences, ok? > > Systemd is the only officially 'supported' init system in ArchLinux. other init systems are from the AUR[1]. I was an Arch user a while ago, as an Arch user you would want as less as possible stuff from AUR, many times it means poorly written PKBUILD's[2] and not maintained, It's common that upgrading from the main repos would break some AUR stuff, then anyone can upload anything to the AUR repo. It's a bit like using a random not well maintained overlay in gentoo, often not in-sync with the portage tree, it might work or it might not even build. If you read the wiki page you posted you will notice it's mainly two users(a manjaro developer is one of them), installing openrc in different ways[3][4], don't expect robustness when they can't even join forces to maintain a single effort, unnecessarily duplicating work, and one of those deviates from the Gentoo way. For a package to be moved into the main repos it needs to have many votes in the AUR, if you look at the votes of those, the non-gentoo way[3] hast 17 and the gentoo way[4] has 3, not really popular considering there are packages with 1K+ votes, and are still in the AUR. You should ask via Arch or Manjaro channels for testing experience, or try manjaro yourself (OpenRC isn't their default either but they have prebuilt packages installable with pacman[5]). Even better search for artoo(the Manjaro developer[6] porting it to that distro) and apg, they are doing the work, I doubt you will get the responses you want from this list. [1] https://wiki.archlinux.org/index.php/Arch_User_Repository [2] https://wiki.archlinux.org/index.php/PKGBUILD [3] https://aur.archlinux.org/packages/openrc/ [4] https://aur.archlinux.org/packages/openrc-core/ [5] https://wiki.manjaro.org/index.php?title=OpenRC,_an_alternative_to_systemd [6] http://manjaro.github.io/team/
Re: Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
2015-01-22 6:11 GMT-06:00 Andreas K. Huettel : > Am Donnerstag 22 Januar 2015, 16:50:45 schrieb Sam Bishop: >> On 22 January 2015 at 01:54, Andreas K. Huettel > wrote: >> > Am Mittwoch 21 Januar 2015, 20:36:55 schrieb Sam Bishop: >> >> So I've been thinking crazy thoughts. >> >> >> >> Theoretically it can't be that hard to do a complete package binhost for >> >> gentoo. >> >> >> >> To be clear, when i say complete, Im referring to building, all >> >> versions of all ebuilds marked stable or unstable on amd64, with every >> >> combination of use flags. >> > >> > Not enough. You will also have to build against every combination of >> > dependency subslots. >> > >> > e.g., different poppler, boost, icu, perl and many more versions... >> > >> > Which makes the task near impossible. >> >> Not impossible, just more computationally demanding and requiring more >> storage. > > Well, exponential increase is exponential increase. > > * A libreoffice binary package with debug information has roughly 800Mbyte > size > * 2 libreoffice versions in the tree > * libreoffice links against poppler, icu, boost (among other things) > * poppler: 5 subslots, icu (soon) 3 subslots, boost 5 subslots in tree -> 75 > combinations > * libreoffice has 22 useflags and 4 extensions, plus three supported python > variants -> 29 switches > * REQUIRED_USE limits your combinations, let's conservatively guess 25 > independent switches -> 2^25=33554432 use combinations > Based on this. If it would take 1 minute(being more than optimistic) to build libreoffice: 33554432 builds * 1min = 63 years building If one would want to build that in a day it would be needed to rent 23301 super fast boxes. and have them heating all day long, leaving the storage problem aside, just for libreoffice, if we think now about firefox, chromium and the webkit packages, I think that makes for a good analogy of hell, and a terrible waste of resources.
Re: [gentoo-user] Re: Get off my lawn?
On Tue, Jan 20, 2015 at 2:58 PM, Marc Stürmer wrote: > Zitat von Tom H : >> >> Lennart claims that the embedded world loves systemd. I suspect that, >> as in other corners of the Linux world, there are lovers and haters of >> systemd. > > Embedded systems also quite often means low on resources, CPU power, memory, > space. > > If you are using hard space constrained systems, the sheer size of systemd > in the file system can be a valid reason not to use it at all. > > So it does depend on the type of embedded system you are looking at. Sure. My point was that anyone can claim that systemd is (un)popular in the embedded space. Samsung's starting to release Tizen-driven phones, TVs, white goods, etc. Tizen uses systemd and, given the size of Samsung, the number of systemd embedded devices is going to skyrocket in the next few years. Samsung wouldn't have chosen systemd for Tizen if it were too resource hungry for its use case. There might be devices where systemd's too fat to be wedged in but it's unfortunately going to be difficult to know whether this is really the case or whether that determination's shaded by an anti-systemd bias. :(
Re: [gentoo-user] Re: Get off my lawn?
On Thu, Jan 22, 2015 at 1:06 PM, Tom H wrote: > > Samsung's starting to release Tizen-driven phones, TVs, white goods, > etc. Tizen uses systemd and, given the size of Samsung, the number of > systemd embedded devices is going to skyrocket in the next few years. > Samsung wouldn't have chosen systemd for Tizen if it were too resource > hungry for its use case. > Embedded is a pretty broad term, and it impacts all aspects of a device's design. You can't really put a smartphone and a microwave in the same category. Phones actually have plenty of storage, RAM, and CPU by most embedded standards. The main issue is battery use, which is mostly about ensuring that your software isn't constantly waking up the CPU. If systemd is well-behaved in this regard I'd expect it to work on a phone just fine. The thing is that most devices that couldn't run systemd would probably be hard-pressed to run any kind of generic linux distro in any case. They might not even run linux, or if they did it might be a super-stripped-down build with an embedded initramfs containing nothing but a single executable built in C which runs as PID 1 (no need for even filesystem support, let alone stuff like /proc and so on). I'm genuinely curious as to how systemd and competing solutions are adopted in the embedded world, including phones but especially getting beyond this (huge) niche. I'm also curious as to where ChromeOS ends up going. It is based on Gentoo, but runs Upstart (which isn't used by just about anybody else now, and which isn't even in Gentoo's portage). -- Rich
Re: [gentoo-user] Re: Get off my lawn?
On Thu, Jan 22, 2015 at 1:16 PM, Rich Freeman wrote: > On Thu, Jan 22, 2015 at 1:06 PM, Tom H wrote: >> Samsung's starting to release Tizen-driven phones, TVs, white goods, >> etc. Tizen uses systemd and, given the size of Samsung, the number of >> systemd embedded devices is going to skyrocket in the next few years. >> Samsung wouldn't have chosen systemd for Tizen if it were too resource >> hungry for its use case. >> > > Embedded is a pretty broad term, and it impacts all aspects of a > device's design. You can't really put a smartphone and a microwave in > the same category. > > Phones actually have plenty of storage, RAM, and CPU by most embedded > standards. The main issue is battery use, which is mostly about > ensuring that your software isn't constantly waking up the CPU. If > systemd is well-behaved in this regard I'd expect it to work on a > phone just fine. > > The thing is that most devices that couldn't run systemd would > probably be hard-pressed to run any kind of generic linux distro in > any case. They might not even run linux, or if they did it might be a > super-stripped-down build with an embedded initramfs containing > nothing but a single executable built in C which runs as PID 1 (no > need for even filesystem support, let alone stuff like /proc and so > on). ACK to all the above! > I'm genuinely curious as to how systemd and competing solutions are > adopted in the embedded world, including phones but especially getting > beyond this (huge) niche. Same here. I'd really like to see whether systemd'll be used beyond Tizen/Sailfish/UbuntuTouch. > I'm also curious as to where ChromeOS ends up going. It is based on > Gentoo, but runs Upstart (which isn't used by just about anybody else > now, and which isn't even in Gentoo's portage). I'm also curious about the future ChromeOS init. Upstart is, sadly, walking dead (IIUC Ubuntu'll stop using it in 2019 once 14.04 is EOLd). It's going to be systemd or Android init, isn't it? AIUI Google wants to have Android and ChromeOS converge somewhat so it's more likely to be Android init. Speculation! :)
Re: [gentoo-user] Re: Get off my lawn?
On 19.01.2015 22:49, Stefan G. Weichinger wrote: > I learned my first steps with ansible around these ansible-playbook(s): > > https://github.com/jameskyle/ansible-gentoo Here my changes in a fork of the mentioned ansible-role: https://github.com/stefangweichinger/ansible-gentoo Maybe someone is interested to join in and improve this. Stefan
Re: [gentoo-user] Re: Get off my lawn?
On Thu, Jan 22, 2015 at 1:36 PM, Tom H wrote: > > I'm also curious about the future ChromeOS init. Upstart is, sadly, > walking dead (IIUC Ubuntu'll stop using it in 2019 once 14.04 is > EOLd). It's going to be systemd or Android init, isn't it? AIUI Google > wants to have Android and ChromeOS converge somewhat so it's more > likely to be Android init. Speculation! :) > Interesting, I hadn't thought about Android init. Neither ChromeOS nor Android support user-supplied daemons or anything else traditional along those lines (anything running in the background is run at a higher level in the framework). However, I think a key difference here is suspend/hibernate. Android doesn't do that, and ChromeOS does. Android goes into lower-power mode all the time, but I don't think that is the same as a traditional desktop sleep mode, and Android definitely doesn't do suspend-to-disk. ChromeOS tends to hide this stuff from the user, but I believe it does both. That seems likely to greatly favor an event-driven init, though the fact that you aren't running tons of arbitrary daemons might help to mitigate that need. -- Rich
Re: [gentoo-user] Re: Get off my lawn?
Hello Rich, and everybody else, Happy New Year! On Thu, Jan 22, 2015 at 01:16:53PM -0500, Rich Freeman wrote: > On Thu, Jan 22, 2015 at 1:06 PM, Tom H wrote: > > Samsung's starting to release Tizen-driven phones, TVs, white goods, > > etc. Tizen uses systemd and, given the size of Samsung, the number of > > systemd embedded devices is going to skyrocket in the next few years. > > Samsung wouldn't have chosen systemd for Tizen if it were too resource > > hungry for its use case. > Embedded is a pretty broad term, and it impacts all aspects of a > device's design. You can't really put a smartphone and a microwave in > the same category. > Phones actually have plenty of storage, RAM, and CPU by most embedded > standards. The main issue is battery use, which is mostly about > ensuring that your software isn't constantly waking up the CPU. If > systemd is well-behaved in this regard I'd expect it to work on a > phone just fine. > The thing is that most devices that couldn't run systemd would > probably be hard-pressed to run any kind of generic linux distro in > any case. They might not even run linux, or if they did it might be a > super-stripped-down build with an embedded initramfs containing > nothing but a single executable built in C which runs as PID 1 (no > need for even filesystem support, let alone stuff like /proc and so > on). > I'm genuinely curious as to how systemd and competing solutions are > adopted in the embedded world, including phones but especially getting > beyond this (huge) niche. Just as a data point, the last project I worked on was an automotive system, whose controller was a 32-bit Power PC with 768k/64k code/data flash, and 64k RAM. It did not run systemd! Rather than Linux, it ran Autosar (a special, and somewhat wierd, OS specially for automotive products). The sensors in the system were even more constrained, using a special low-power processor with 16k flash and 1.5k RAM. They didn't run Linux either! In fact, they didn't have an OS - they were coded directly to the metal, in a single-tasked loop. My impression is that the embedded world is split roughly equally between large systems (like smart phones or infotainment systems where RAM is measured in gigabytes, and full scale OSs are used) and small systems (such as my automotive system, microwave ovens, TV zappers, elevator controllers, where special OSs, if any, are used, and RAM is measured in kilobytes). > I'm also curious as to where ChromeOS ends up going. It is based on > Gentoo, but runs Upstart (which isn't used by just about anybody else > now, and which isn't even in Gentoo's portage). > -- > Rich -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] Re: Get off my lawn?
On Thu, Jan 22, 2015 at 3:42 PM, Alan Mackenzie wrote: > Just as a data point, the last project I worked on was an automotive > system, whose controller was a 32-bit Power PC with 768k/64k code/data > flash, and 64k RAM. It did not run systemd! Rather than Linux, it ran > Autosar (a special, and somewhat wierd, OS specially for automotive > products). I wonder how small linux can actually get in such a world, assuming you still needed the multitasking, drivers, etc (which would be the main benefits of running an OS vs just embedding a single program written in C that directly talks to the hardware). You can trim a lot of stuff out of linux that we all take for granted, but I'm not sure if you can really get it into the 100kb range. I couldn't really find hard numbers anywhere for the actual minimum RAM requirements for a linux that contains the minimum features needed to provide a bit of hardware support and run init with almost nothing else exposed but the system call interface (no need for /proc, devfs, /sys, and so on). It sounds like you're still talking hundreds of kilobytes to 1-2MB of RAM use in most cases. So, dedicated embedded kernels are likely to stay around for a while to come. -- Rich
Re: [gentoo-user] Open Question: The feasibility of a complete portage binhost
On 22/01/15 15:46, Jc García wrote: > 2015-01-22 6:11 GMT-06:00 Andreas K. Huettel : >> Am Donnerstag 22 Januar 2015, 16:50:45 schrieb Sam Bishop: >>> On 22 January 2015 at 01:54, Andreas K. Huettel >> wrote: Am Mittwoch 21 Januar 2015, 20:36:55 schrieb Sam Bishop: > So I've been thinking crazy thoughts. > > Theoretically it can't be that hard to do a complete package binhost for > gentoo. > > To be clear, when i say complete, Im referring to building, all > versions of all ebuilds marked stable or unstable on amd64, with every > combination of use flags. Not enough. You will also have to build against every combination of dependency subslots. e.g., different poppler, boost, icu, perl and many more versions... Which makes the task near impossible. >>> Not impossible, just more computationally demanding and requiring more >>> storage. >> Well, exponential increase is exponential increase. >> >> * A libreoffice binary package with debug information has roughly 800Mbyte >> size >> * 2 libreoffice versions in the tree >> * libreoffice links against poppler, icu, boost (among other things) >> * poppler: 5 subslots, icu (soon) 3 subslots, boost 5 subslots in tree -> 75 >> combinations >> * libreoffice has 22 useflags and 4 extensions, plus three supported python >> variants -> 29 switches >> * REQUIRED_USE limits your combinations, let's conservatively guess 25 >> independent switches -> 2^25=33554432 use combinations >> > Based on this. > If it would take 1 minute(being more than optimistic) to build libreoffice: > 33554432 builds * 1min = 63 years building > > If one would want to build that in a day it would be needed to rent > 23301 super fast boxes. and have them heating all day long, leaving > the storage problem aside, just for libreoffice, if we think now about > firefox, chromium and the webkit packages, I think that makes for a > good analogy of hell, and a terrible waste of resources. > My 2c what if instead of one person does all the compiling and storage, we have "cc-emerge" which would stand for "cloud contributor emerge" it would be a wrapper / slightly modified emerge, to always build packages, but have a postinstall hook which then bundles the package with "builtwith.ini" which would have parsable detail, because on top of the slots, and the uses and the 32/64 bit versions, you also have pluggable compilers and even CHOST are different ok so once the build package is bundled with the builtwith.ini it would then be sent up to AWS for further analysis. this would allow some interesting feedback: 1. most popular compilers 2. most common use flags for packages. 3. if no one is using a specific use flag then why bother having it in communal binhost ? i'm not saying that folks using something odd should then be expunged, but it would possibly give devs some interesting feedback. it might also help to streamline tinderboxing as you could compare your compiled version with the communal version also it would tap into (voluntarily of course) our collective compiling time too. with the billions and growing number of gentoo users ;) we should be able to crank out a communal binhost. once that is there and it can be queried and indexed, we could have some fun ensuring that all builds are built the same across the board we could also then have cc-emerge do a lookup to see if someone else already compiled it and choose to download it, or only download if 100 folks have also compiled it and all checksums are the same for the 100 folks that compiled it the same (same hardware, same use flags etc etc)
[gentoo-user] Re: Openrc now on Arch LInux
Jc García gmail.com> writes: > I doubt you will get the responses you want from this list. But, I did; Your response was excellent. I asked here because OpenRC users with Gentoo, often have a deep understanding of OpenRC, as we have been through several migrations over the years. I certainly appreciate your explanation and links. James
Re: [gentoo-user] Re: Get off my lawn?
On Thu, Jan 22, 2015 at 04:28:26PM -0500, Rich Freeman wrote > > I wonder how small linux can actually get in such a world, assuming > you still needed the multitasking, drivers, etc (which would be the > main benefits of running an OS vs just embedding a single program > written in C that directly talks to the hardware). You can trim a lot > of stuff out of linux that we all take for granted, but I'm not sure > if you can really get it into the 100kb range. *BSD would be a better candidate, in terms of a smaller kernel to start with. And I'm sure that legal would be a lot happier about not having to supply source code. -- Walter Dnes I don't run "desktop environments"; I run useful applications
Re: [gentoo-user] Openrc now on Arch LInux
AFAIR OpenRC exists in Arch repo for a long time. I mean, even after changing to systemd by default you could install and use OpenRC as init. 22.01.2015 1:32, James пишет: It seems Openrc is spreading? https://wiki.archlinux.org/index.php/OpenRC Anyone tested openrc on Arch linux? I have read in several places that Arch linux has moved to systemd, exclusively? James -- Regards, R.H.
