Re: [oi-dev] Desktop Illumos Still Matters
Nick Zivkovic wrote: > On Thu, Sep 6, 2012 at 4:35 AM, Joerg Schilling > wrote: > > "Andrew M. Hettinger" wrote: > > > >> That said, I think what Nick was talking about was not a build failure, but > >> common issues IPS itself can have. I've seen a few (rare) times where it > >> will just spit out a call-stack. Haveing a goto page of potental problems > >> and fixes/workarounds would help. > > > > Apart from the uncomodities with IPS, there is a real bug in OI: > > > > There is no way to mark a zone as "native" in OI and a native zone will be > > modified on upgrades even though this is wrong. > > Hi Joerg. > > If I understand you correctly, when doing an image-update with IPS, > IPS will try to forcefully update all (ipkg-brand) NGZs? > > I don't see this in the documentation anywhere. Or did you have > something else in mind? Correct and this is the problem together with the fact that Sun did remove the support files for zonecfg and zoneadm that are needed to mark a zone to be a native Solaris zone. As a result, all zones are of type ipkg brand even when you install a natize zone that runs e.g. SchilliX. I recommend to add this changeset: http://hg.berlios.de/repos/schillix-on/rev/b43ccebc5075 to fix the problem. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Thu, Sep 6, 2012 at 4:35 AM, Joerg Schilling wrote: > "Andrew M. Hettinger" wrote: > >> That said, I think what Nick was talking about was not a build failure, but >> common issues IPS itself can have. I've seen a few (rare) times where it >> will just spit out a call-stack. Haveing a goto page of potental problems >> and fixes/workarounds would help. > > Apart from the uncomodities with IPS, there is a real bug in OI: > > There is no way to mark a zone as "native" in OI and a native zone will be > modified on upgrades even though this is wrong. Hi Joerg. If I understand you correctly, when doing an image-update with IPS, IPS will try to forcefully update all (ipkg-brand) NGZs? I don't see this in the documentation anywhere. Or did you have something else in mind? Thanks. > > BTW: SchilliX-ON recently fixed the problem with the missing files that are > the > reason for nor being able to mark a zone as "native". > > > > Jörg > > -- > EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin >j...@cs.tu-berlin.de(uni) >joerg.schill...@fokus.fraunhofer.de (work) Blog: > http://schily.blogspot.com/ > URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
"Andrew M. Hettinger" wrote: > That said, I think what Nick was talking about was not a build failure, but > common issues IPS itself can have. I've seen a few (rare) times where it > will just spit out a call-stack. Haveing a goto page of potental problems > and fixes/workarounds would help. Apart from the uncomodities with IPS, there is a real bug in OI: There is no way to mark a zone as "native" in OI and a native zone will be modified on upgrades even though this is wrong. BTW: SchilliX-ON recently fixed the problem with the missing files that are the reason for nor being able to mark a zone as "native". Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Francois Dion wrote on 09/04/2012 07:29:09 PM: > > On Tue, Sep 4, 2012 at 5:38 PM, Nick Zivkovic wrote: > >>> 2) document every single IPS failure and either fix the > >>> packages or the IPS code (depend on what caused the failure), and > >> > >> First thought here is that it needs to be in the bug tracker, but that may > >> not be easily accessible either. Maybe a sub-page on the wiki? > > > > Either should be fine. FreeBSD records their ports build failures on > > their wiki. I think Gentoo recorded this on a bug tracker. Wiki is > > probably easiest. > > Jenkins can automate all of that. For $JOB, I manage various products, > millions of lines of code in total with it. The nice thing is that it > will "blame" whoever breaks the build. It also provides an easy to > read dashboard, and notify only on status change, not on every build > that fails etc. Plus it can post to a URL, so a few lines of python > code with web.py and you have an interface to a wiki or bug tracker. > > Francois > I spent some time thinking about this today, and you are right. A Jenkins server which with a plugin to publish via IPS to a sandbox server (or use the build system's IPS publishing, where available), and one to make a zone on the build server to work in would not only do everything I want, but exceed it. I guess what I care about is less the build system, and more psudo-CI. I would also need a way to promote builds, once working (and again once tested, sandbox->testing->stable, which means I really don't care much about blame. This worked well for SJ). I think I'll fire up a couple of VMs and get this working. I have used Jenkins before, it's really an awesome bit of software. For the life of me I don't know why I didn't realize Kohsuke Kawaguchi had already done most of the work to make me happy again! Guess I just got myself stuck in the mindset that we have too many build systems and didn't realize that that isn't what I am really frustrated by at all. That said, I think what Nick was talking about was not a build failure, but common issues IPS itself can have. I've seen a few (rare) times where it will just spit out a call-stack. Haveing a goto page of potental problems and fixes/workarounds would help. Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356(int'l)___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Joerg Schilling wrote: Andrew Gabriel wrote: Nick Zivkovic wrote: Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms of their purpose. For example we have /usr/X11. According to `man filesystem` /opt is meant to hold add-on/third-party software. /opt was meant for unbundled software. Ideally, it should be empty immediately following a full install of a distro, as everything is by definition bundled. I don't think Solaris ever quite got that right, but it was almost there. Everything you install after that (which isn't part of the distro) should be in /opt (and /etc/opt and /var/opt), but a lot of 3rd-party software developers got that wrong too. There was a nice talk from Steve Bourne at the Sun User Group meeting in December 1990. He explained that first /usr/bin was hijacked by the system and people started with /usr/local. Then external sources hijacked /usr/local and as a result a FHS summit with most UNIX vendors decided to usr /opt. I went to an AT&T presentation on the upcoming SVR4 for UNIX source porters, which was probably 1989 or early 1990, where they went into this. One of the key reasons for having it outside /usr was that an upgrade required blowing away all of /usr and reinstalling (there was no real upgrade capability in their original SRV4.0 other than a reinstall), and they didn't expect you would want to lose all your 3rd-party/unbundled products. It still wasn't very well thought out compared with where we are today, but it was vastly better than SVR3.2 which is what commercial organisations were running at the time. This was also why SVR4 moved home directories out of /usr, leaving us with the irony of a /usr which no longer contained the users. We are now nearly 25 years after that /opt decision and still not everybody got it. -- Andrew ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alasdair Lumsden wrote: > On 05/09/2012 21:58, Joerg Schilling wrote: > > Alasdair Lumsden wrote: > > It seems that you missunderstand the problem. > > > > The main issue is that the build system linked against /usr instead of > > linking > > against something like: /home/user/proto/usr > > userland-gate still links against /usr - it has a per-package proto area > rather than a build-system wide proto area. Then the biggest problem has not been solved. The "concept" of "flagdays" is wrong, it does not allow reliable upgrades as everytime, when more than a single leaf project in such a consolidation is updated, an unknown number of build/install cycles must follow until no binaries change from the next build cycle. A clean build system would have a own global proto area. With this concept, you would still need to have the right compile order that depends on the link dependencies. > "You need to comment out line 71 of the file > /usr/include/net-snmp/net-snmp-config.h > do that it then looks this way: > > /*#define HAVE_CPP_UNDERBAR_FUNCTION_DEFINED 1*/ > > This is needed as the sunstudio-12 compiler and gcc-3.4.3 do not support > __FUNCTION__" > > That's an autoconf problem, not a problem with the build system. If you > build software with a new compiler, autoconf will detect its new features. It is a problem that is based on the original software, I mentioned it because Sun claimed that all include files in /usr/include are non-dynamic and independent from compiler versions. This was done in a discussion where Sun claimed that dynamic configuration results are unacceptable dependencies for the ONNV compilation. I should note that the file net-snmp-config.h _is_ such a ONNV dependency. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:58, Joerg Schilling wrote: Alasdair Lumsden wrote: It seems that you missunderstand the problem. The main issue is that the build system linked against /usr instead of linking against something like: /home/user/proto/usr userland-gate still links against /usr - it has a per-package proto area rather than a build-system wide proto area. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... That is not an issue with userland-gate or oi-build. You do have to install gmake, bash and a bunch of dependencies, but they're all available in the package repo. It is not a real issue with SchilliX either but for a looking at a self-contained system it would. - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Can you supply an example? I'm not sure I understand this complaint. Check e.g. the notes about sfw in: ftp://ftp.berlios.de/pub/schillix/AN-0.8 "You need to comment out line 71 of the file /usr/include/net-snmp/net-snmp-config.h do that it then looks this way: /*#define HAVE_CPP_UNDERBAR_FUNCTION_DEFINED 1*/ This is needed as the sunstudio-12 compiler and gcc-3.4.3 do not support __FUNCTION__" That's an autoconf problem, not a problem with the build system. If you build software with a new compiler, autoconf will detect its new features. To work around it, the net-snmp build recipe can modify the generated header prior to packaging it. However, typically a distribution will ship with a particular supported compiler version, and the headers of the software on the system will match that version. It is unreasonable to expect an older compiler to link against all software delivered by the system when the system uses a newer compiler. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alan Coopersmith wrote: > >> Which is why it was completely thrown out and Userland started with a > >> new design from scratch. > > > > But as this did not exist before Spring 2010, I asume that the new system > > is > > not able to create native Svr4 packages. > > Correct. Userland was designed from the ground up for IPS, since that was the > only packaging system in use when it was created. So there would be a need to add the missing code. The important question here would be whether this is an open project that accepts enhancements to support the native packaging system and that would support to then change the related meta data the same way the IPS meta data is changed. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:36, Andrew Stormont wrote: These sorts of scripts are just broken. What it really should do is check the value of CC before adding any compiler specific flags. Patching it to do that would be my preferred way of solving the problem. That works too. The thing is they're pretty dumb in operation - they pick up the compiler environment, which in the case of mysql was optimised for the database server. Client libraries really won't benefit from the optimisations. We could store the Sun Studio optimisations, and expose them with CC is detected as Studio, but gcc users miss out on them. So for equality it seems easier just to strip them. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, 5 Sep 2012, Andrew Stormont wrote: The only thing you really need for extensions to build is the -I bit. The rest is Sun Studio pedantry. These sorts of scripts are just broken. What it really should do is check the value of CC before adding any compiler specific flags. Patching it to do that would be my preferred way of solving the problem. Things get complicated because it might not even be possible to combine code compiled by two different C compilers. Some compiler-specific options might be needed in order to obtain special compilation modes and secret library dependencies (especially for threads and OpenMP). There are also explicit library dependencies to get right. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alasdair Lumsden wrote: > On 05/09/2012 19:49, Joerg Schilling wrote: > > The buildsystem for sfw is a nightmare: > > > > - It only works if the whole set of tools has already been > > installed in /usr on the compiling system before with exactly > > the same version as the one that is going to be compiled. > > > > This causes that you need an unknown number of iteration to > > compile and install on the build machine before you are able > > to compile everything at all. > > > > You need at least one additional install/compile cycle before you > > can be sure that the compile/link results will no longer change. > > It sounds like what you want is a completely self contained build > system, like pkgsrc, which bootstraps itself, requires only a compiler, > knows how to build things in the correct order, and installs everything > to a custom destination prefix. > > That approach is fine for building 3rd party software, but not for > maintaining system software which ships to /usr. It seems that you missunderstand the problem. The main issue is that the build system linked against /usr instead of linking against something like: /home/user/proto/usr > > - It may be that you would need to manually install at least older > > versions of strategic tools before you may start to compile at all. > > The programs in question are gmake, bash, gm4, ... > > That is not an issue with userland-gate or oi-build. You do have to > install gmake, bash and a bunch of dependencies, but they're all > available in the package repo. It is not a real issue with SchilliX either but for a looking at a self-contained system it would. > > - It installs unmodified autoconf results in /usr/include with the > > effect that you cannot compile depending other software using > > an older version of the compilers than the one you used to compile > > sfw. > > Can you supply an example? I'm not sure I understand this complaint. Check e.g. the notes about sfw in: ftp://ftp.berlios.de/pub/schillix/AN-0.8 Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 5 Sep 2012, at 21:33, Alasdair Lumsden wrote: > On 05/09/2012 21:22, Bob Friesenhahn wrote: >> What do you suggest as a better replacement for this? > > Oh it's easy - you strip most of them out after the file is generated. Very > easy to do with a post-install sed rule in the build recipe. > > The bulk of them are pointless optimisations that aren't really relevant when > compiling an extension. The main LDFLAGS to preserve are -L/-R and -l, and > for CFLAGS its -D and -I. > > As an example, mysql_config spits out this for CFLAGS: > > -I/usr/mysql/5.1/include/mysql -xprefetch=auto -xprefetch_level=3 -mt > -fns=no -fsimple=1 -xbuiltin=%all -xlibmil -xlibmopt -xnorunpath > -DHAVE_RWLOCK_T -DUNIV_SOLARIS > > The only thing you really need for extensions to build is the -I bit. The > rest is Sun Studio pedantry. These sorts of scripts are just broken. What it really should do is check the value of CC before adding any compiler specific flags. Patching it to do that would be my preferred way of solving the problem. Andrew S. > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:22, Bob Friesenhahn wrote: What do you suggest as a better replacement for this? Oh it's easy - you strip most of them out after the file is generated. Very easy to do with a post-install sed rule in the build recipe. The bulk of them are pointless optimisations that aren't really relevant when compiling an extension. The main LDFLAGS to preserve are -L/-R and -l, and for CFLAGS its -D and -I. As an example, mysql_config spits out this for CFLAGS: -I/usr/mysql/5.1/include/mysql -xprefetch=auto -xprefetch_level=3 -mt -fns=no -fsimple=1 -xbuiltin=%all -xlibmil -xlibmopt -xnorunpath -DHAVE_RWLOCK_T -DUNIV_SOLARIS The only thing you really need for extensions to build is the -I bit. The rest is Sun Studio pedantry. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, 5 Sep 2012, Alasdair Lumsden wrote: I do find it highly annoying that a lot of software jots down the compiler flags it was built with and stores them in a [softwarename]_config file, such as mysql_config, which is used by extensions to get the CFLAGS/LDFLAGS/etc. On OSOL/OI these are often Sun Studio flags that aren't compatible with gcc. What do you suggest as a better replacement for this? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:12, Alan Coopersmith wrote: Correct. Userland was designed from the ground up for IPS, since that was the only packaging system in use when it was created. Nexenta enhanced their fork of userland to support generating .deb packages, so adding SVR4 probably wouldn't be too hard. Why you'd want to is another matter. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 5/12 01:09 PM, joerg.schill...@fokus.fraunhofer.de wrote: > Alan Coopersmith wrote: > >> On 09/ 5/12 11:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: >>> I asume that what you call "userland" is the successor for "sfw". >> >> Yes. >> >>> The buildsystem for sfw is a nightmare >> >> Which is why it was completely thrown out and Userland started with a >> new design from scratch. > > But as this did not exist before Spring 2010, I asume that the new system is > not able to create native Svr4 packages. Correct. Userland was designed from the ground up for IPS, since that was the only packaging system in use when it was created. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alan Coopersmith wrote: > On 09/ 5/12 11:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: > > I asume that what you call "userland" is the successor for "sfw". > > Yes. > > > The buildsystem for sfw is a nightmare > > Which is why it was completely thrown out and Userland started with a > new design from scratch. But as this did not exist before Spring 2010, I asume that the new system is not able to create native Svr4 packages. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 19:49, Joerg Schilling wrote: The buildsystem for sfw is a nightmare: - It only works if the whole set of tools has already been installed in /usr on the compiling system before with exactly the same version as the one that is going to be compiled. This causes that you need an unknown number of iteration to compile and install on the build machine before you are able to compile everything at all. You need at least one additional install/compile cycle before you can be sure that the compile/link results will no longer change. It sounds like what you want is a completely self contained build system, like pkgsrc, which bootstraps itself, requires only a compiler, knows how to build things in the correct order, and installs everything to a custom destination prefix. That approach is fine for building 3rd party software, but not for maintaining system software which ships to /usr. Why? Because even in a minimal zone, bootstrapping involves overwriting things in /usr that are already maintained by the packaging system. You could build software and upgrade to those packages as you go, but that's very hard to do especially when refactoring packages. If you instead tried to install everything to a custom destination prefix, you couldn't then just package it up and install it to /usr, because lots of software embeds their build prefix in the binary. If you built stuff with /foo as your prefix, then packaged it and delivered it to /usr, /usr/bin/someprogram would be looking for /foo/etc/someconfigfile.conf It's far easier just to use a build zone and install the dependencies you need, and ensure your build zone is running the latest bits from the package repository. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... That is not an issue with userland-gate or oi-build. You do have to install gmake, bash and a bunch of dependencies, but they're all available in the package repo. - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Can you supply an example? I'm not sure I understand this complaint. I do find it highly annoying that a lot of software jots down the compiler flags it was built with and stores them in a [softwarename]_config file, such as mysql_config, which is used by extensions to get the CFLAGS/LDFLAGS/etc. On OSOL/OI these are often Sun Studio flags that aren't compatible with gcc. You then end up in a situation where if you're doing "gem install somelibrary" or "pecl install somelibrary" with gcc as your compiler, it chokes when linking against a system program that supplies Sun Studio CFLAGS/LDFLAGS. SFW was a nightmare for many reasons, but not the reasons you listed. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 2:43 PM, Alan Coopersmith wrote: > On 09/ 5/12 10:55 AM, Andrew Gabriel wrote: >> Nick Zivkovic wrote: >>> Basically, I'm asking if it is better to have one convention >>> (everything in /usr/$consolidation) instead of two (some things in >>> /usr/$consolidation and others in /opt/$consolidation)? >>> >> >> There's never been any rule about consolidations being funneled into specific >> directories. It may be that it makes sense in a few specific cases because of >> functional groupings, but not universally. > > In fact, we've been going the other way for years, moving away from > /usr/$subsystem directories that impose meaningless boundaries in the way of > users. > > In the last OpenSolaris builds released (snv_130 & later), OpenIndiana, and > Solaris 11 you should find that /usr/X11 is simply a bunch of backwards > compatibility symlinks. Most of the X11 libraries were simply reached via > /usr/lib since Solaris 2.6, and the rest of the X11 files moved to /usr/bin, > /usr/share, /usr/lib, etc. in those Nevada builds. Thanks for clearing this up, Alan. Besides, these boundaries are better enforced through NG zones than through the filesystem heirarchy. > > -- > -Alan Coopersmith- alan.coopersm...@oracle.com > Oracle Solaris Engineering - http://blogs.oracle.com/alanc > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 5/12 10:55 AM, Andrew Gabriel wrote: > Nick Zivkovic wrote: >> Basically, I'm asking if it is better to have one convention >> (everything in /usr/$consolidation) instead of two (some things in >> /usr/$consolidation and others in /opt/$consolidation)? >> > > There's never been any rule about consolidations being funneled into specific > directories. It may be that it makes sense in a few specific cases because of > functional groupings, but not universally. In fact, we've been going the other way for years, moving away from /usr/$subsystem directories that impose meaningless boundaries in the way of users. In the last OpenSolaris builds released (snv_130 & later), OpenIndiana, and Solaris 11 you should find that /usr/X11 is simply a bunch of backwards compatibility symlinks. Most of the X11 libraries were simply reached via /usr/lib since Solaris 2.6, and the rest of the X11 files moved to /usr/bin, /usr/share, /usr/lib, etc. in those Nevada builds. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 5/12 11:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: > I asume that what you call "userland" is the successor for "sfw". Yes. > The buildsystem for sfw is a nightmare Which is why it was completely thrown out and Userland started with a new design from scratch. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Bob Friesenhahn wrote: > On Wed, 5 Sep 2012, Joerg Schilling wrote: > > > > As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to > > no > > longer depending on sbrk(2)) and if you use it to interpret autoconf > > scripts, > > this is 3x faster than bash. > > This sounds great. How does its performance compare with 'dash'? > > Are the various issues described in the GNU Autoconf portability notes > (http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Portable-Shell.html#Portable-Shell) > > avoided/fixed by this shell? SunOS's stagnant /bin/sh has been a > continuing issue with POSIX shell script portability. As a result, > Autoconf configure scripts typically elect to re-run themselves with > bash (the World's Slowest Shell) on Solaris sytems. If this shell is > indeed good enough for Autoconf configure scripts, then it would be > good to submit an Autoconf patch so that future configure scripts know > how to detect and use it. Given the fact that GNU autoconf has been more or less destroyed after release 2.13, so I personally base my work on an extremely enhanced gnu autoconf 2.13. The timing tests I did run, have been run with my enhanced autoconf-2.13 Autoconf 2.13 works with all known shells - I have no idea why the FSF stopped to support this. I suspect that this is just a bash marketing action. The problem why newer GNU autoconf versions are so slow may be that they call a bew bash for each single test unless /bin/sh is bash - what you don't like to have on a POSIX compliant system. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Gabriel wrote: > Nick Zivkovic wrote: > > Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms > > of their purpose. > > > > For example we have /usr/X11. According to `man filesystem` /opt is > > meant to hold add-on/third-party software. > > > > /opt was meant for unbundled software. Ideally, it should be empty > immediately following a full install of a distro, as everything is by > definition bundled. I don't think Solaris ever quite got that right, but > it was almost there. Everything you install after that (which isn't part > of the distro) should be in /opt (and /etc/opt and /var/opt), but a lot > of 3rd-party software developers got that wrong too. There was a nice talk from Steve Bourne at the Sun User Group meeting in December 1990. He explained that first /usr/bin was hijacked by the system and people started with /usr/local. Then external sources hijacked /usr/local and as a result a FHS summit with most UNIX vendors decided to usr /opt. We are now nearly 25 years after that /opt decision and still not everybody got it. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Stormont wrote: > > On 5 Sep 2012, at 18:04, Nick Zivkovic wrote: > > > I think that Andrew want to use a unified build system, instead of the > > loose confederation of radically different systems that's currently in > > use. > > > > I agree. A unified build system is necessary. The only question is: > > what should it be? > > > > Makefile-based, like ports/pkgsrc/oi-build? > > specfile-based? > > tcl-base like macports? > > shell-based like Gentoo's and Exherbo's? > > python-based like conary? > > Userland already has a perfectly good build system. I don't understand what > you're trying to accomplish here. I asume that what you call "userland" is the successor for "sfw". The buildsystem for sfw is a nightmare: - It only works if the whole set of tools has already been installed in /usr on the compiling system before with exactly the same version as the one that is going to be compiled. This causes that you need an unknown number of iteration to compile and install on the build machine before you are able to compile everything at all. You need at least one additional install/compile cycle before you can be sure that the compile/link results will no longer change. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Are these problems still in effect? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, 5 Sep 2012, Joerg Schilling wrote: As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to no longer depending on sbrk(2)) and if you use it to interpret autoconf scripts, this is 3x faster than bash. This sounds great. How does its performance compare with 'dash'? Are the various issues described in the GNU Autoconf portability notes (http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Portable-Shell.html#Portable-Shell) avoided/fixed by this shell? SunOS's stagnant /bin/sh has been a continuing issue with POSIX shell script portability. As a result, Autoconf configure scripts typically elect to re-run themselves with bash (the World's Slowest Shell) on Solaris sytems. If this shell is indeed good enough for Autoconf configure scripts, then it would be good to submit an Autoconf patch so that future configure scripts know how to detect and use it. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 12:56 PM, Alasdair Lumsden wrote: > Hi Nick, > > > On 05/09/2012 18:49, Nick Zivkovic wrote: >> >> Someone thought it would be a good idea to have a unified build system >> across consolidations. >> >> I think it's a pretty good idea to standardize on one build system. >> >> I'm merely asking which one would be preferred by the community >> (assuming the community would be willing to standardize). > > > The decision was already made by the core OI devs to use the userland-gate > format. That's the future unified build system. So the choice doesn't really > have to be made again - it's why "oi-build" is in userland-gate format. > > Cheers, > > Alasdair Oh, ok. I'm still catching up on what's been going on here, while I was away in my cave. > > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Hi Nick, On 05/09/2012 18:49, Nick Zivkovic wrote: Someone thought it would be a good idea to have a unified build system across consolidations. I think it's a pretty good idea to standardize on one build system. I'm merely asking which one would be preferred by the community (assuming the community would be willing to standardize). The decision was already made by the core OI devs to use the userland-gate format. That's the future unified build system. So the choice doesn't really have to be made again - it's why "oi-build" is in userland-gate format. Cheers, Alasdair ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Nick Zivkovic wrote: Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms of their purpose. For example we have /usr/X11. According to `man filesystem` /opt is meant to hold add-on/third-party software. /opt was meant for unbundled software. Ideally, it should be empty immediately following a full install of a distro, as everything is by definition bundled. I don't think Solaris ever quite got that right, but it was almost there. Everything you install after that (which isn't part of the distro) should be in /opt (and /etc/opt and /var/opt), but a lot of 3rd-party software developers got that wrong too. If that's the case shouldn't X11 be in /opt/X11? Or should the convention be updated, so that we store the bundles or consolidation in /usr as is already being done? If sub-directories of /usr are separate datasets (like /usr/X11 is rpool/X11), that should make migration easier. Basically, I'm asking if it is better to have one convention (everything in /usr/$consolidation) instead of two (some things in /usr/$consolidation and others in /opt/$consolidation)? There's never been any rule about consolidations being funneled into specific directories. It may be that it makes sense in a few specific cases because of functional groupings, but not universally. -- Andrew ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Someone thought it would be a good idea to have a unified build system across consolidations. I think it's a pretty good idea to standardize on one build system. I'm merely asking which one would be preferred by the community (assuming the community would be willing to standardize). On Wed, Sep 5, 2012 at 12:31 PM, Andrew Stormont wrote: > > On 5 Sep 2012, at 18:04, Nick Zivkovic wrote: > >> I think that Andrew want to use a unified build system, instead of the >> loose confederation of radically different systems that's currently in >> use. >> >> I agree. A unified build system is necessary. The only question is: >> what should it be? >> >> Makefile-based, like ports/pkgsrc/oi-build? >> specfile-based? >> tcl-base like macports? >> shell-based like Gentoo's and Exherbo's? >> python-based like conary? > > Userland already has a perfectly good build system. I don't understand what > you're trying to accomplish here. > > Andrew S. > >> >> I'm fine with any of the above (as well as any that I've not mentioned). >> >> As long as we end up with a standardized build system so that >> contributors can cross-fertilize consolidations instead of confining >> themselves to just one. >> >> What do existing OI-contributors think? >> >> Anyone know what the most technically-superior or technically-advanced >> build system is? >> >> On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger >> wrote: >>> >>> Andrew Hettinger >>> http://Prominic.NET || ahettin...@prominic.net >>> Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) >>> Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) >>> >>> >>> Alasdair Lumsden wrote on 09/04/2012 05:39:58 PM: >>> On 04/09/2012 22:42, mag...@yonderway.com wrote: > On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" > wrote: > >> One of the biggest issues here isn't that packages are particularly >> HARD >> to >> make with IPS (they aren't). It's that there are about three different >> approaches to it, and we need to get that standardized. Many of the >> packages are tied up in the consolidations, which DO seem to have a >> high >> barrier to entry. > > So what are the cliff's notes to the three different approaches, and is > one > of those approaches going to have a lower barrier to entry with still > high-quality results? I'm thinking along the lines of a third party > repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as "JDS" (Java Desktop System, renamed to just "desktop" on Solaris 11), "SFW" (Sun Freeware, replaced with "userland" in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a "giant leap forward" replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide "lockstep" upgrades between known working version sets of software. "entire" is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" versi
Re: [oi-dev] Desktop Illumos Still Matters
On 5 Sep 2012, at 18:04, Nick Zivkovic wrote: > I think that Andrew want to use a unified build system, instead of the > loose confederation of radically different systems that's currently in > use. > > I agree. A unified build system is necessary. The only question is: > what should it be? > > Makefile-based, like ports/pkgsrc/oi-build? > specfile-based? > tcl-base like macports? > shell-based like Gentoo's and Exherbo's? > python-based like conary? Userland already has a perfectly good build system. I don't understand what you're trying to accomplish here. Andrew S. > > I'm fine with any of the above (as well as any that I've not mentioned). > > As long as we end up with a standardized build system so that > contributors can cross-fertilize consolidations instead of confining > themselves to just one. > > What do existing OI-contributors think? > > Anyone know what the most technically-superior or technically-advanced > build system is? > > On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger > wrote: >> >> Andrew Hettinger >> http://Prominic.NET || ahettin...@prominic.net >> Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) >> Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) >> >> >> Alasdair Lumsden wrote on 09/04/2012 05:39:58 PM: >> >>> On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" wrote: > One of the biggest issues here isn't that packages are particularly > HARD > to > make with IPS (they aren't). It's that there are about three different > approaches to it, and we need to get that standardized. Many of the > packages are tied up in the consolidations, which DO seem to have a > high > barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. >>> >>> I think there's a bit of confusion surrounding the terms involved. >>> >>> A consolidation is just a logical grouping of packages, such as "JDS" >>> (Java Desktop System, renamed to just "desktop" on Solaris 11), "SFW" >>> (Sun Freeware, replaced with "userland" in Solaris 11), etc. >>> >>> The big issue is that all the consolidations use different build >>> systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, >>> a collection of 3rd party package recipes). SFW used a horrible Makefile >>> based system. Userland is an excellent replacement for SFW, and uses >>> Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. >>> >>> I was attempting (with some others) to get OI updated in a "giant leap >>> forward" replacing SFW with userland-gate (renamed to oi-build, and >>> later illumos-userland after Nexenta started meddling). The idea was >>> that we would try to focus on one single build system, the userland-gate >>> style, which is the best of the lot. New software would go in there, and >>> if we needed to update something in another consolidation, we would >>> instead re-implement the recipe for it in userland-gate format. >>> >>> Unfortunately my approach with /experimental was quite ambitious and >>> didn't quite work how we wanted. >>> >>> Jon Tibble is continuing to release new OpenIndiana builds (such as >>> prestable 6, released *today* - >>> http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is >>> advocating we do move to a single build system based on userland-gate, >>> but more slowly and in a more controlled fashion. >>> >>> oi_151a actually already has a mini userland-gate/oi-build, which you >>> can see here: >>> >>> https://hg.openindiana.org/sustaining/oi_151a/oi-build/ >>> >>> It's used to deliver some additional software and some bits from other >>> consolidations have been moved there. >>> >>> It is probably where people should focus their efforts moving forward. >>> >>> Incorporations are probably what people are bitching about, which are >>> there to provide "lockstep" upgrades between known working version sets >>> of software. "entire" is the best known incorporation, which with each >>> release locks all system software at a particular version. >>> >>> Incorporations are needed to prevent your system getting shafted. Lets >>> say you are on oi_148, and in oi_151a we introduced some new software >>> called "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" >>> version 1.0. Without incorporations, if you "pkg install foo" it will >>> upgrade "bar" no questions asked. Any software linked against bar >>> probably just stopped working with libbar.so.1 changed to libbar.so.2. >>> >>> Incorporations present a challenge when you're developing software, >>> because they stop you installing new versions of things. The way to get >>> around this is to have "empty" incorporations on your development >
Re: [oi-dev] Desktop Illumos Still Matters
I think that Andrew want to use a unified build system, instead of the loose confederation of radically different systems that's currently in use. I agree. A unified build system is necessary. The only question is: what should it be? Makefile-based, like ports/pkgsrc/oi-build? specfile-based? tcl-base like macports? shell-based like Gentoo's and Exherbo's? python-based like conary? I'm fine with any of the above (as well as any that I've not mentioned). As long as we end up with a standardized build system so that contributors can cross-fertilize consolidations instead of confining themselves to just one. What do existing OI-contributors think? Anyone know what the most technically-superior or technically-advanced build system is? On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger wrote: > > Andrew Hettinger > http://Prominic.NET || ahettin...@prominic.net > Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) > Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) > > > Alasdair Lumsden wrote on 09/04/2012 05:39:58 PM: > >> On 04/09/2012 22:42, mag...@yonderway.com wrote: >> > On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" >> > wrote: >> > >> >> One of the biggest issues here isn't that packages are particularly >> >> HARD >> >> to >> >> make with IPS (they aren't). It's that there are about three different >> >> approaches to it, and we need to get that standardized. Many of the >> >> packages are tied up in the consolidations, which DO seem to have a >> >> high >> >> barrier to entry. >> > >> > So what are the cliff's notes to the three different approaches, and is >> > one >> > of those approaches going to have a lower barrier to entry with still >> > high-quality results? I'm thinking along the lines of a third party >> > repo. >> >> I think there's a bit of confusion surrounding the terms involved. >> >> A consolidation is just a logical grouping of packages, such as "JDS" >> (Java Desktop System, renamed to just "desktop" on Solaris 11), "SFW" >> (Sun Freeware, replaced with "userland" in Solaris 11), etc. >> >> The big issue is that all the consolidations use different build >> systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, >> a collection of 3rd party package recipes). SFW used a horrible Makefile >> based system. Userland is an excellent replacement for SFW, and uses >> Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. >> >> I was attempting (with some others) to get OI updated in a "giant leap >> forward" replacing SFW with userland-gate (renamed to oi-build, and >> later illumos-userland after Nexenta started meddling). The idea was >> that we would try to focus on one single build system, the userland-gate >> style, which is the best of the lot. New software would go in there, and >> if we needed to update something in another consolidation, we would >> instead re-implement the recipe for it in userland-gate format. >> >> Unfortunately my approach with /experimental was quite ambitious and >> didn't quite work how we wanted. >> >> Jon Tibble is continuing to release new OpenIndiana builds (such as >> prestable 6, released *today* - >> http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is >> advocating we do move to a single build system based on userland-gate, >> but more slowly and in a more controlled fashion. >> >> oi_151a actually already has a mini userland-gate/oi-build, which you >> can see here: >> >> https://hg.openindiana.org/sustaining/oi_151a/oi-build/ >> >> It's used to deliver some additional software and some bits from other >> consolidations have been moved there. >> >> It is probably where people should focus their efforts moving forward. >> >> Incorporations are probably what people are bitching about, which are >> there to provide "lockstep" upgrades between known working version sets >> of software. "entire" is the best known incorporation, which with each >> release locks all system software at a particular version. >> >> Incorporations are needed to prevent your system getting shafted. Lets >> say you are on oi_148, and in oi_151a we introduced some new software >> called "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" >> version 1.0. Without incorporations, if you "pkg install foo" it will >> upgrade "bar" no questions asked. Any software linked against bar >> probably just stopped working with libbar.so.1 changed to libbar.so.2. >> >> Incorporations present a challenge when you're developing software, >> because they stop you installing new versions of things. The way to get >> around this is to have "empty" incorporations on your development >> system, that way you are free to shaft your own install if you want to. >> It's like taking your seatbelt off. >> >> Incorporations map to consolidations, so SFW, JDS, etc each have their >> own incorporation. This means the incorporations have to be updated when >> you move software from one consolidation
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356(int'l) Alasdair Lumsden wrote on 09/04/2012 05:39:58 PM: > On 04/09/2012 22:42, mag...@yonderway.com wrote: > > On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" > > wrote: > > > >> One of the biggest issues here isn't that packages are particularly HARD > >> to > >> make with IPS (they aren't). It's that there are about three different > >> approaches to it, and we need to get that standardized. Many of the > >> packages are tied up in the consolidations, which DO seem to have a high > >> barrier to entry. > > > > So what are the cliff's notes to the three different approaches, and is one > > of those approaches going to have a lower barrier to entry with still > > high-quality results? I'm thinking along the lines of a third party repo. > > I think there's a bit of confusion surrounding the terms involved. > > A consolidation is just a logical grouping of packages, such as "JDS" > (Java Desktop System, renamed to just "desktop" on Solaris 11), "SFW" > (Sun Freeware, replaced with "userland" in Solaris 11), etc. > > The big issue is that all the consolidations use different build > systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, > a collection of 3rd party package recipes). SFW used a horrible Makefile > based system. Userland is an excellent replacement for SFW, and uses > Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. > > I was attempting (with some others) to get OI updated in a "giant leap > forward" replacing SFW with userland-gate (renamed to oi-build, and > later illumos-userland after Nexenta started meddling). The idea was > that we would try to focus on one single build system, the userland-gate > style, which is the best of the lot. New software would go in there, and > if we needed to update something in another consolidation, we would > instead re-implement the recipe for it in userland-gate format. > > Unfortunately my approach with /experimental was quite ambitious and > didn't quite work how we wanted. > > Jon Tibble is continuing to release new OpenIndiana builds (such as > prestable 6, released *today* - > http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is > advocating we do move to a single build system based on userland-gate, > but more slowly and in a more controlled fashion. > > oi_151a actually already has a mini userland-gate/oi-build, which you > can see here: > > https://hg.openindiana.org/sustaining/oi_151a/oi-build/ > > It's used to deliver some additional software and some bits from other > consolidations have been moved there. > > It is probably where people should focus their efforts moving forward. > > Incorporations are probably what people are bitching about, which are > there to provide "lockstep" upgrades between known working version sets > of software. "entire" is the best known incorporation, which with each > release locks all system software at a particular version. > > Incorporations are needed to prevent your system getting shafted. Lets > say you are on oi_148, and in oi_151a we introduced some new software > called "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" > version 1.0. Without incorporations, if you "pkg install foo" it will > upgrade "bar" no questions asked. Any software linked against bar > probably just stopped working with libbar.so.1 changed to libbar.so.2. > > Incorporations present a challenge when you're developing software, > because they stop you installing new versions of things. The way to get > around this is to have "empty" incorporations on your development > system, that way you are free to shaft your own install if you want to. > It's like taking your seatbelt off. > > Incorporations map to consolidations, so SFW, JDS, etc each have their > own incorporation. This means the incorporations have to be updated when > you move software from one consolidation to another, eg from JDS to > oi-build. > > Hope this makes sense. > > Alasdair > I used terminology sloppily, thank you for clarifying for everyone. Source Juicer used the same RPM style spec file that SFE uses, with a web frontend to automatically handle submitting and building packages. What it lacked as a simple way to promote a package once it had been testing for a while. And the process for getting that done that was always a thorn in all of our sides. As I recall, for someone on the outside, it was "badger the right people in sun until you where annoying enough that they'll do promotions just to get you out of their hair". Even for all it's problems, it was a really good system, which (atleast for those of us who knew about it) strongly encouraged contribution. I feel that as long as there are so many differing build systems, people will tend to confine themselves to one of them, a
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 4:18 AM, Jim Klimov wrote: > 2012-09-04 22:25, Andrew M. Hettinger пишет: > >> was kept in /bin and /sbin that did not depend on anything. This was >> done to allow you to NFS mount everything else. IIRC the decision was >> made to go ahead and make them dynamicly linked because noone NFS mounts >> them anymore anyway, and it meant we had to keep both a simplified and >> full version of the programs. I think this will encounter many similar >> issues as that. If we are going back down this road, we could consider >> just delivering a /bin and /sbin we can use as you propose. It would >> have the advantage of bringing back this lost (albit rarely used) >> functionality. >> > > Well, as a little offtopic from desktopness, I have long ago posted an > illumos RFE 829 to (again) support separatable "/usr" datasets, allowing > one to compress much of the rootfs among other benefits of hierarchical > environment (quotas, different storage and stuff). > > I've recently redone this on my laptop with no problems, following my > own logs on wiki and bugtracker; the only substantial blocker was and is > the "/sbin/sh" being a symlink to "../usr/bin/ksh" or somesuch. System > fails to boot itself when /usr is separate. Replacing this symlink with > a hard copy of /usr/bin/bash (as /sbin/sh) does not break the system as > much as I can see (several machines doing this for several months now) > and allows to split /usr off into its own compressed dataset. > > There were some other nuances discussed in the RFE, like additions to > the SMF service which mounts minimal root environment, and problems > with beadm/zfsclone not setting compression attributes on clones, > but I won't bother you here with those ;) > > I just wanted to stress that this is not a feature only useful for > diskless NFS boots, but also on a PC or a laptop or a VM, especially > with limited HDD or SSD space and/or IOPS/bandwidth (less reads = > faster boots). Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms of their purpose. For example we have /usr/X11. According to `man filesystem` /opt is meant to hold add-on/third-party software. If that's the case shouldn't X11 be in /opt/X11? Or should the convention be updated, so that we store the bundles or consolidation in /usr as is already being done? If sub-directories of /usr are separate datasets (like /usr/X11 is rpool/X11), that should make migration easier. Basically, I'm asking if it is better to have one convention (everything in /usr/$consolidation) instead of two (some things in /usr/$consolidation and others in /opt/$consolidation)? Also, for the SMF nits you ran into, _I think_ we can probably update the manifests so that SMF doesn't try to start, for example, gdm/X11 until it mounts rpool/X11 onto /usr/X11. Thanks. > > //Jim Klimov > > > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Tue, Sep 4, 2012 at 5:39 PM, Alasdair Lumsden wrote: > Hi All, > > > On 04/09/2012 22:42, mag...@yonderway.com wrote: >> >> On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" >> wrote: >> >>> One of the biggest issues here isn't that packages are particularly HARD >>> to >>> make with IPS (they aren't). It's that there are about three different >>> approaches to it, and we need to get that standardized. Many of the >>> packages are tied up in the consolidations, which DO seem to have a high >>> barrier to entry. >> >> >> So what are the cliff's notes to the three different approaches, and is >> one >> of those approaches going to have a lower barrier to entry with still >> high-quality results? I'm thinking along the lines of a third party repo. > > > I think there's a bit of confusion surrounding the terms involved. > > A consolidation is just a logical grouping of packages, such as "JDS" (Java > Desktop System, renamed to just "desktop" on Solaris 11), "SFW" (Sun > Freeware, replaced with "userland" in Solaris 11), etc. > > The big issue is that all the consolidations use different build systems. > JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection > of 3rd party package recipes). SFW used a horrible Makefile based system. > Userland is an excellent replacement for SFW, and uses Makefiles but in a > way very similar to the FreeBSD ports tree or pkgsrc. > > I was attempting (with some others) to get OI updated in a "giant leap > forward" replacing SFW with userland-gate (renamed to oi-build, and later > illumos-userland after Nexenta started meddling). The idea was that we would > try to focus on one single build system, the userland-gate style, which is > the best of the lot. New software would go in there, and if we needed to > update something in another consolidation, we would instead re-implement the > recipe for it in userland-gate format. > > Unfortunately my approach with /experimental was quite ambitious and didn't > quite work how we wanted. > > Jon Tibble is continuing to release new OpenIndiana builds (such as > prestable 6, released *today* - > http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is > advocating we do move to a single build system based on userland-gate, but > more slowly and in a more controlled fashion. > > oi_151a actually already has a mini userland-gate/oi-build, which you can > see here: > > https://hg.openindiana.org/sustaining/oi_151a/oi-build/ > > It's used to deliver some additional software and some bits from other > consolidations have been moved there. > > It is probably where people should focus their efforts moving forward. > > Incorporations are probably what people are bitching about, which are there > to provide "lockstep" upgrades between known working version sets of > software. "entire" is the best known incorporation, which with each release > locks all system software at a particular version. > > Incorporations are needed to prevent your system getting shafted. Lets say > you are on oi_148, and in oi_151a we introduced some new software called > "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" version > 1.0. Without incorporations, if you "pkg install foo" it will upgrade "bar" > no questions asked. Any software linked against bar probably just stopped > working with libbar.so.1 changed to libbar.so.2. > > Incorporations present a challenge when you're developing software, because > they stop you installing new versions of things. The way to get around this > is to have "empty" incorporations on your development system, that way you > are free to shaft your own install if you want to. It's like taking your > seatbelt off. Well, incorporations sound like a great feature, imho. I guess the only problem is when two packages have mutually exclusive dependencies (foo depends on libbar.so.1, and baz depends on libbar.so.2). But even then, one can probably avoid this by using NGZ's, if the foo package isn't updated to use libbar.so.2, or if that software is no longer maintained by the original author. > > Incorporations map to consolidations, so SFW, JDS, etc each have their own > incorporation. This means the incorporations have to be updated when you > move software from one consolidation to another, eg from JDS to oi-build. > > Hope this makes sense. > > Alasdair > > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 01:30, Alasdair Lumsden wrote: I've actually realised it would be really useful if there was a single document explaining all this stuff, a kind of "In the beginning there was..." style walk through of how things came to be. I'll try to write one over the next few weeks and put it on the wiki, as it would probably help new developers get their head around things. That would be very helpful. When I attended the userland hackathon I was quite confused about all of the different build systems and their history. Thanks for your patience and support there :-). I should dig up my notes and scripts for creating a build environment. Henk ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Jonathan Adams wrote: > > > > ftp://ftp.berlios.de/pub/schily/ > > > > do you have a patch/diffs to source supplied elsewhere? Is this > project stored in a git repository, or even in an SCCS tar ball > separate to the Schillix-ON project? P.S. if you like to check the latest man page, have a look at: http://cdrecord.berlios.de/private/man/sh.1.html I append the output of "sccs -R log" that contasins a list of commit messages. Note that these messages are in German. Tue Jul 3 23:45:16 2012 joerg * sh.1 1.34 Umbau auf .TP Mon Jul 2 23:13:39 2012 joerg * cmd.c 1.22 for i; do mit Semikolon erlauben Mon Jul 2 23:06:56 2012 joerg * sh.1 1.33 -option -> \-option Mon Jul 2 22:59:55 2012 joerg * sh.1 1.32 Weitgehender Umbau von \fB auf .B ... Sun Jul 1 15:09:47 2012 joerg * sh.1 1.31 Copyright Header repariert Sun Jun 10 22:28:23 2012 joerg * word.c 1.23 Bei rekursiven aliasen ist standin->fnxt nicht im Bereich von standin->fbuf[] Sun Jun 10 17:55:01 2012 joerg * Makefile 1.25 abbrev.c nach Vorne wegen der Abhaengigkeiten Sun Jun 10 17:32:23 2012 joerg * args.c 1.21 * sh.1 1.30 * cmd.c 1.21 * MKLINKS 1.4 * Makefile 1.24 * version.h 1.6 * name.c 1.22 * bltin.c 1.27 * word.c 1.22 * fault.c 1.18 * main.c 1.20 * defs.h 1.45 ENV=, /etc/sh.shrc, $HOME/.shrc, alias, unalias neu Sun Jun 10 16:57:41 2012 joerg * msg.c 1.18 Neue Strings fuer alias/unalias Sun Jun 10 16:54:01 2012 joerg * alias.c 1.1 date and time created 12/06/10 16:54:01 by joerg Tue Jun 5 23:38:33 2012 joerg * service.c 1.23 * io.c 1.17 fstat() & S_ISDIR() neu Mon Jun 4 21:42:35 2012 joerg * args.c 1.20 * defs.h 1.44 set -o neu Mon May 21 23:54:00 2012 joerg * defs.h 1.43 * msg.c 1.17 * bltin.c 1.26 * sh.1 1.29 dosh Kommando neu Sat May 19 15:21:45 2012 joerg * sh.1 1.28 alloc built-in dokumentiert Neue Schily Kommandos mit Hinweis versehen Thu May 17 20:24:55 2012 joerg * sh.1 1.27 Option -c beschreibt nun dass das naechste Argument nach dem Kommando-String $0 ist Tue May 15 23:34:00 2012 joerg * stak.c 2.7 Memory Leak in "repeat(1)" verhindern durch erweitertes savstak() und tdystak() Sun May 13 20:20:53 2012 joerg * bltin.c 1.25 savstak() / tdystak() um execexp() fuer SYSREPEAT neu - wird auch im sbrk Shell benoetigt Sun May 13 20:19:19 2012 joerg * stak.c 2.6 Tracemem = 0, Stackdebug = 0, -> Tracemem = TRACEMEM, Stackdebug = STACKDEBUG, Sat May 12 23:53:34 2012 joerg * jobs.c 1.25 * expand.c 1.14 * macro.c 1.17 * pwd.c 1.14 * fault.c 1.17 * service.c 1.22 * string.c 1.15 * cmd.c 1.20 * word.c 1.21 * xec.c 1.21 * bltin.c 1.24 * name.c 1.21 * hashserv.c 1.13 lint Warnungen beseitigt Sat May 12 14:13:33 2012 joerg * bltin.c 1.23 * print.c 1.17 times(1) Ausgabe ist nun POSIX kompatibel Fri May 11 20:54:01 2012 joerg * hash.h 1.6 Eingerueckt Fri May 11 20:50:39 2012 joerg * io.c 1.16 * string.c 1.14 * jobs.c 1.24 * xec.c 1.20 * test.c 1.13 * fault.c 1.16 * defs.c 1.9 * msg.c 1.16 * expand.c 1.13 * func.c 1.11 * name.c 1.20 * word.c 1.20 * service.c 1.21 * cmd.c 1.19 * error.c 1.9 * hashserv.c 1.12 * main.c 1.19 * defs.h 1.42 * macro.c 1.16 * args.c 1.19 * hash.c 1.10 * ctype.c 1.4 * bltin.c 1.22 * pwd.c 1.13 Cstyle Fri May 11 00:33:36 2012 joerg * msg.c 1.15 "history" an richtige Stelle in builtin-Tabelle bewegt Thu May 10 23:58:38 2012 joerg * defs.h 1.41 * bltin.c 1.21 * pwd.c 1.12 Dir nicht mehr ausgeben wenn pushd ueber CDPATH erfolgt Thu May 10 23:58:34 2012 joerg * version.h 1.5 Neue Version mit pushd/popd/dirs Thu May 10 23:26:14 2012 joerg * bltin.c 1.20 switch bei SYSCD korrekt eingerueckt Thu May 10 23:19:58 2012 joerg * pwd.c 1.11 * bltin.c 1.19 * msg.c 1.14 * defs.h 1.40 pushd/popd/dirs neu Thu May 10 20:07:33 2012 joerg * sh.1 1.26 .ne 2 -> .ne 3 Wed May 9 20:25:00 2012 joerg * sh.1 1.25 OLDPWD, pushd, popd, dirs neu Tue May 8 20:02:00 2012 joerg * msg.c 1.13 Spezielle Kommandos markiert Tue May 8 20:00:48 2012 joerg * func.c 1.10 Funktionen mit "case" korrekt ausgeb
Re: [oi-dev] Desktop Illumos Still Matters
Jonathan Adams wrote: > On 5 September 2012 11:14, Joerg Schilling > wrote: > > Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in > > SchilliX-ON since a longer time. Before I introduce my fix, I will first > > replace the unmaintained Bourne Shell from Sun sources by the current > > maintained one which you can find in: > > > > ftp://ftp.berlios.de/pub/schily/ > > > > do you have a patch/diffs to source supplied elsewhere? Is this > project stored in a git repository, or even in an SCCS tar ball > separate to the Schillix-ON project? The Bourne Shell enhancement project started in November 2006 and the first action was to transform the unmaintained code into modern code that makes use of C prototypes to allow for better code review. These changes alone will prevent you from being able to do anything useful by comparing older versions of the source from Sun with recent ones as half of the code did change even though I did not yet re-indent the code to match cstyle. The project itself is implemented using SCCS, you may know that I am also enhancing SCCS towards a network aware distributed suystem that handles changesets. This SCCS will soon appear in SchillIX-ON as I believe that everything that was traditionally inside the UNIX sources should become part of ONNV again. > Are you offering these changes as an update to the Illumos bash > project to be upstreamed, will you be working with these changes in > the community? What is the Illumos bsh project? I tried to get interaction with the OpenSolaris community before to no avail. Later, Illumos signalled not to be interested in collaboration with non-Nexenta people. The discussions I had, have been made with the Bourne Shell "expert" Sven Maschek, see http://www.in-ulm.de/~mascheck/bourne/ for a list of features and Bugs I fixed. I plan to rename usr/src/cmd/sh into usr/src/cmd/osh and to add the current state of the Bourne Shell into SchilliX-ON as usr/src/cmd/sh. > is it possible to exclude your added functionality that was not in the > original shell so that we can stay compatible with all older POSIX > compliant systems that don't have this additional functionality? There are #ifdefs for the editor, but not for the rest of the enhancements. Please note that the Bourne Shell is not in POSIX and it is questuonable whether a reduced functionality will make sense. Switching off new builtins would be easy, but things like support for "set -o" is spread over the code. On the other side, there is a #ifdef ARGS_RIGHT_TO_LEFT to switch back to the old incorrect macro evaluation order that is a result from the fact that Steve Bourne may not yet have known how to effiviently add to the end of a list in 1977. There is also no #ifdef to switch on the security bugs introduced by Sun when implementing a kernel based ofexec(). Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 5 September 2012 11:14, Joerg Schilling wrote: > Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in > SchilliX-ON since a longer time. Before I introduce my fix, I will first > replace the unmaintained Bourne Shell from Sun sources by the current > maintained one which you can find in: > > ftp://ftp.berlios.de/pub/schily/ > do you have a patch/diffs to source supplied elsewhere? Is this project stored in a git repository, or even in an SCCS tar ball separate to the Schillix-ON project? Are you offering these changes as an update to the Illumos bash project to be upstreamed, will you be working with these changes in the community? is it possible to exclude your added functionality that was not in the original shell so that we can stay compatible with all older POSIX compliant systems that don't have this additional functionality? Jon ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Jim Klimov wrote: > I've recently redone this on my laptop with no problems, following my > own logs on wiki and bugtracker; the only substantial blocker was and is > the "/sbin/sh" being a symlink to "../usr/bin/ksh" or somesuch. System > fails to boot itself when /usr is separate. Replacing this symlink with > a hard copy of /usr/bin/bash (as /sbin/sh) does not break the system as > much as I can see (several machines doing this for several months now) > and allows to split /usr off into its own compressed dataset. I thought thet IPS is too dumb to support this. Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in SchilliX-ON since a longer time. Before I introduce my fix, I will first replace the unmaintained Bourne Shell from Sun sources by the current maintained one which you can find in: ftp://ftp.berlios.de/pub/schily/ This Bourne Shell - fixes all Bourne Shell bugs that have been documented since the late 1980 but never fixed by AT&T or Sun. - Was converted from using sbrk(2) to malloc(3) to achieve portability and allows to use libc routines that depend on malloc(). - fixes some previouly unknown bugs in string handling that have been a result from removing a dependency on SIGSEV to auto-expand memory for automatic string management as implemented in the original version from 1977. - Adds the Commandline History editor I developed in 1982 and implemented in 1984 for my "bsh". - Adds the advanced alias mechanism from my "bsh" that is better than the alias implementation from ksh. - Adds a new builtin "dosh" that allows to write intrinsic "shell scripts" via aliases. - Adds pushd/popd/dirs builtins - Adds a mapper for e.g. cursor keys that works reliably in contrast to what is known from ksh and bash that sometimes do not map strings and thus insert raw key strings. The combination of editable history, aliases and pushd/popd makes it nice for interactive work. I plan to install this shell (that is still _much_ smaller than ksh) as /sbin/sh and to fix the 6 SMF scripts that have been modified to only work with ksh. As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to no longer depending on sbrk(2)) and if you use it to interpret autoconf scripts, this is 3x faster than bash. I hope that other OpenSolaris distros will follow SchilliX-ON and SchilliX with this enhancement. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
2012-09-04 22:25, Andrew M. Hettinger пишет: was kept in /bin and /sbin that did not depend on anything. This was done to allow you to NFS mount everything else. IIRC the decision was made to go ahead and make them dynamicly linked because noone NFS mounts them anymore anyway, and it meant we had to keep both a simplified and full version of the programs. I think this will encounter many similar issues as that. If we are going back down this road, we could consider just delivering a /bin and /sbin we can use as you propose. It would have the advantage of bringing back this lost (albit rarely used) functionality. Well, as a little offtopic from desktopness, I have long ago posted an illumos RFE 829 to (again) support separatable "/usr" datasets, allowing one to compress much of the rootfs among other benefits of hierarchical environment (quotas, different storage and stuff). I've recently redone this on my laptop with no problems, following my own logs on wiki and bugtracker; the only substantial blocker was and is the "/sbin/sh" being a symlink to "../usr/bin/ksh" or somesuch. System fails to boot itself when /usr is separate. Replacing this symlink with a hard copy of /usr/bin/bash (as /sbin/sh) does not break the system as much as I can see (several machines doing this for several months now) and allows to split /usr off into its own compressed dataset. There were some other nuances discussed in the RFE, like additions to the SMF service which mounts minimal root environment, and problems with beadm/zfsclone not setting compression attributes on clones, but I won't bother you here with those ;) I just wanted to stress that this is not a feature only useful for diskless NFS boots, but also on a PC or a laptop or a VM, especially with limited HDD or SSD space and/or IOPS/bandwidth (less reads = faster boots). //Jim Klimov ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Tue, Sep 4, 2012 at 5:38 PM, Nick Zivkovic wrote: >>> 2) document every single IPS failure and either fix the >>> packages or the IPS code (depend on what caused the failure), and >> >> First thought here is that it needs to be in the bug tracker, but that may >> not be easily accessible either. Maybe a sub-page on the wiki? > > Either should be fine. FreeBSD records their ports build failures on > their wiki. I think Gentoo recorded this on a bug tracker. Wiki is > probably easiest. Jenkins can automate all of that. For $JOB, I manage various products, millions of lines of code in total with it. The nice thing is that it will "blame" whoever breaks the build. It also provides an easy to read dashboard, and notify only on status change, not on every build that fails etc. Plus it can post to a URL, so a few lines of python code with web.py and you have an interface to a wiki or bug tracker. Francois ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
I've actually realised it would be really useful if there was a single document explaining all this stuff, a kind of "In the beginning there was..." style walk through of how things came to be. I'll try to write one over the next few weeks and put it on the wiki, as it would probably help new developers get their head around things. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Hi All, On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as "JDS" (Java Desktop System, renamed to just "desktop" on Solaris 11), "SFW" (Sun Freeware, replaced with "userland" in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a "giant leap forward" replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide "lockstep" upgrades between known working version sets of software. "entire" is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called "foo". Foo depends on version 2.0 of "bar". oi_148 delivers "bar" version 1.0. Without incorporations, if you "pkg install foo" it will upgrade "bar" no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop you installing new versions of things. The way to get around this is to have "empty" incorporations on your development system, that way you are free to shaft your own install if you want to. It's like taking your seatbelt off. Incorporations map to consolidations, so SFW, JDS, etc each have their own incorporation. This means the incorporations have to be updated when you move software from one consolidation to another, eg from JDS to oi-build. Hope this makes sense. Alasdair ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Tue, 4 Sep 2012 13:25:39 -0500, "Andrew M. Hettinger" wrote: > One of the biggest issues here isn't that packages are particularly HARD > to > make with IPS (they aren't). It's that there are about three different > approaches to it, and we need to get that standardized. Many of the > packages are tied up in the consolidations, which DO seem to have a high > barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Tue, Sep 4, 2012 at 1:25 PM, Andrew M. Hettinger wrote: > My thoughts. Remember, they are probably only worth what you paid for them! > ;p > > Nick Zivkovic wrote on 09/01/2012 10:42:14 AM: > > >> >> Yes. I am more interested in contributing drivers and the like. As far >> as packages go, to be honest, I've experienced torture at the hands of >> IPS (though that could very easily be my fault), and am reluctant go >> near it. For example I tried an image-update and it failed. So I am >> stuck on OI-147 until I backup-reinstall-import to OI-151a. >> >> I think packages are a high priority, but not as high as making sure >> the latest illumos-gate can build and run on a modern desktop. For >> example, I can't get SmartOS running on a thinkpad or my desktop >> computer. Somewhere in June 2012, a bug was introduced that prevents >> the illumos kernel from booting. If I had been building and testing >> the latest source, that bug could probably have been caught before it >> got buried in a mountain of commits. Now, I image, it is like finding >> a needle in a haystack. >> >> I am willing to assist with packages, but my time is limited, and I >> think it is more important to direct my effort to building >> illumos-gate and writing drivers. Also, making packages is still a >> black art to me, and wouldn't know where to start. > > One of the biggest issues here isn't that packages are particularly HARD to > make with IPS (they aren't). It's that there are about three different > approaches to it, and we need to get that standardized. Many of the packages > are tied up in the consolidations, which DO seem to have a high barrier to > entry. I considered putting together a source-juicer-like self-service > system for building packages. If I can get the time, I'll revisit that. It > would make my (and everyone else's, I think) life easier. Ok this sounds very useful. I will investigate consolidations, and see what can be done to lower that barrier. > > >> But since we are already on the topic of packages, Adam, do you think >> there is a way to make it less painful, more consistent? I'm _not_ >> talking about extreme measures like changing from IPS to >> [DEB/RPM/SVR/etc]. I'm wondering if we could 1) make IPS easier to use >> by documenting stuff in an easily accessible way [the man pages aren't >> very helpful] > > The wiki would be an ideal place for this to happen. Frankly, I haven't see > much trouble with the man pages from a user perspective, but from the > developer's side, it could definitely use some work. Much of this was > documented in the OS.o site, but we need to not depend on that. > > >> 2) document every single IPS failure and either fix the >> packages or the IPS code (depend on what caused the failure), and > > First thought here is that it needs to be in the bug tracker, but that may > not be easily accessible either. Maybe a sub-page on the wiki? Either should be fine. FreeBSD records their ports build failures on their wiki. I think Gentoo recorded this on a bug tracker. Wiki is probably easiest. > > >> 3) >> have IPS install all userland libs to a zfs dataset named rpool/ips or >> rpool/pkgs; this way, we can zfs-send these datasets, and snap-shot >> them, and clone them, without pulling in the rest of the file-system >> heirarchy. This would make my bitterness toward IPS reduce >> significantly. This way, you can migrate different user-land configs >> between systems. Also, an easy way to do updates across a multitude of >> systems. One can share their binaries and packages via zfs-send, >> because they won't destroy an existing system's /usr /bin and so >> forth. Also, OI would benefit tremendously from offering pre-made NG >> zones on the web-site, available for downloading and running. In fact, >> we could use Zones as a delivery mechanism for things like an Illumos >> build-environment. An NG zone can contain a working and sandboxed >> version of firefox. Zones are a great technology that can make the >> system more attractive amateur power users who may become programmers >> some day (like I did). Multiple ways of sharing pre-compiled binaries >> can only help OI and Illumos. In fact I can see people sharing >> datasets with packages via bit-torrent. Plus, incremental send/recv is >> a huge benefit. > > Back in the bad-old-days, (if memory serves) a basic copy of userland was > kept in /bin and /sbin that did not depend on anything. This was done to > allow you to NFS mount everything else. IIRC the decision was made to go > ahead and make them dynamicly linked because noone NFS mounts them anymore > anyway, and it meant we had to keep both a simplified and full version of > the programs. I think this will encounter many similar issues as that. If we > are going back down this road, we could consider just delivering a /bin and > /sbin we can use as you propose. It would have the advantage of bringing > back this lost (albit rarely used) functionality. > > That said, there is nothi
Re: [oi-dev] Desktop Illumos Still Matters
My thoughts. Remember, they are probably only worth what you paid for them! ;p Nick Zivkovic wrote on 09/01/2012 10:42:14 AM: > > Yes. I am more interested in contributing drivers and the like. As far > as packages go, to be honest, I've experienced torture at the hands of > IPS (though that could very easily be my fault), and am reluctant go > near it. For example I tried an image-update and it failed. So I am > stuck on OI-147 until I backup-reinstall-import to OI-151a. > > I think packages are a high priority, but not as high as making sure > the latest illumos-gate can build and run on a modern desktop. For > example, I can't get SmartOS running on a thinkpad or my desktop > computer. Somewhere in June 2012, a bug was introduced that prevents > the illumos kernel from booting. If I had been building and testing > the latest source, that bug could probably have been caught before it > got buried in a mountain of commits. Now, I image, it is like finding > a needle in a haystack. > > I am willing to assist with packages, but my time is limited, and I > think it is more important to direct my effort to building > illumos-gate and writing drivers. Also, making packages is still a > black art to me, and wouldn't know where to start. One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. I considered putting together a source-juicer-like self-service system for building packages. If I can get the time, I'll revisit that. It would make my (and everyone else's, I think) life easier. > But since we are already on the topic of packages, Adam, do you think > there is a way to make it less painful, more consistent? I'm _not_ > talking about extreme measures like changing from IPS to > [DEB/RPM/SVR/etc]. I'm wondering if we could 1) make IPS easier to use > by documenting stuff in an easily accessible way [the man pages aren't > very helpful] The wiki would be an ideal place for this to happen. Frankly, I haven't see much trouble with the man pages from a user perspective, but from the developer's side, it could definitely use some work. Much of this was documented in the OS.o site, but we need to not depend on that. > 2) document every single IPS failure and either fix the > packages or the IPS code (depend on what caused the failure), and First thought here is that it needs to be in the bug tracker, but that may not be easily accessible either. Maybe a sub-page on the wiki? > 3) > have IPS install all userland libs to a zfs dataset named rpool/ips or > rpool/pkgs; this way, we can zfs-send these datasets, and snap-shot > them, and clone them, without pulling in the rest of the file-system > heirarchy. This would make my bitterness toward IPS reduce > significantly. This way, you can migrate different user-land configs > between systems. Also, an easy way to do updates across a multitude of > systems. One can share their binaries and packages via zfs-send, > because they won't destroy an existing system's /usr /bin and so > forth. Also, OI would benefit tremendously from offering pre-made NG > zones on the web-site, available for downloading and running. In fact, > we could use Zones as a delivery mechanism for things like an Illumos > build-environment. An NG zone can contain a working and sandboxed > version of firefox. Zones are a great technology that can make the > system more attractive amateur power users who may become programmers > some day (like I did). Multiple ways of sharing pre-compiled binaries > can only help OI and Illumos. In fact I can see people sharing > datasets with packages via bit-torrent. Plus, incremental send/recv is > a huge benefit. Back in the bad-old-days, (if memory serves) a basic copy of userland was kept in /bin and /sbin that did not depend on anything. This was done to allow you to NFS mount everything else. IIRC the decision was made to go ahead and make them dynamicly linked because noone NFS mounts them anymore anyway, and it meant we had to keep both a simplified and full version of the programs. I think this will encounter many similar issues as that. If we are going back down this road, we could consider just delivering a /bin and /sbin we can use as you propose. It would have the advantage of bringing back this lost (albit rarely used) functionality. That said, there is nothing stopping anyone from delivering a basic userland in /rpool/pkgs, although I would suggest using an alternate mount point in /usr (/usr/zdu or some-such? solaris has a long history of delivering alternate userlands in filesystems off of /usr). > We might even be able to integrate a window manager (like i3 or dwm) > so that switching virtual desktop, actually switches to another zone. Can you do this in zones? Frankly, all my other zones h
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 2/12 10:47 AM, Nick Zivkovic wrote: > On Sun, Sep 2, 2012 at 11:17 AM, Magnus wrote: >> On Sep 2, 2012, at 11:18 AM, Alan Coopersmith wrote: >>> On 09/ 2/12 07:00 AM, Adam Števko wrote: IPS is documented in the official IPS Developer Guide located somewhere in the OpenSolaris/Oracle page. I went it through lately and I find it a good source for learning to work with IPS, in general. >>> >>> http://hub.opensolaris.org/bin/download/Project+pkg/files/ipsdevguide.pdf >> >> FWIW, I think this sort of response is a sign of ill health for Illumos. >> We're still pointing to Oracle for our own documentation. And, as we should >> know by now, Oracle has no problems withdrawing things from public view with >> no warning. > > This documentation can probably be rewritten on the OI/Illumos wikis. > I will do this. The sources to that version of the developer guide are available, so you don't need to rewrite from scratch: https://hg.openindiana.org/upstream/oracle/pkg-gate/file/tip/doc/dev-guide (Later versions are instead in the internal doc publishing system, so the sources aren't readily available under a suitable license, but then, they document post-175 features that aren't in the IPS version that I believe OI is using.) -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
> OpenIndiana aims to be a general-purpose traditional distribution usable on > server or desktop, not a hypervisor > (although kvm and qemu packages can be found in the repositories). I don't see why OpenIndiana can't be _both_. This is software after all. More choices means more users means potentially more contributors. I don't see why we should support only one way of sharing and installing packages. This is orthogonal to the NGZ appliances I mentioned earlier. Either way, I'll test/implement my ideas, and make them publicly available. On Sun, Sep 2, 2012 at 11:17 AM, Magnus wrote: > > On Sep 2, 2012, at 11:18 AM, Alan Coopersmith wrote: > >> On 09/ 2/12 07:00 AM, Adam Števko wrote: >>> IPS is documented in the official IPS Developer Guide located somewhere in >>> the OpenSolaris/Oracle page. I went it through lately and I find it a good >>> source for learning to work with IPS, in general. >> >> http://hub.opensolaris.org/bin/download/Project+pkg/files/ipsdevguide.pdf > > FWIW, I think this sort of response is a sign of ill health for Illumos. > We're still pointing to Oracle for our own documentation. And, as we should > know by now, Oracle has no problems withdrawing things from public view with > no warning. Agreed. Oracle is not a friend of Illumos. In fact Oracle is probably Illumos's biggest nemesis (however inept and impotent they may be). This documentation can probably be rewritten on the OI/Illumos wikis. I will do this. > > Additionally, I see us debating about putting a lot of work into supporting a > small fringe of users (desktop) while nobody is really talking about > modernizing hardware support for ubiquitous 4K sector hard disks (and I mean > beyond crude hacks). > > We're missing the big important stuff. Well the desktop is important to me and others. So _I_ will personally put a significant amount of work into things like having a modern Xorg, USB drivers, WiFi drivers, and so on. That's what open source is all about. I am not saying that engineers that are preoccupied with other things that are actually relevant to their companies' business plans, should do desktop work pro-bono, at no benefit to themselves. People should scratch their own itches. I stand by my claim that having Illumos work _adequately_ on the desktop, can mean fresh talent for the Illumos community. Much how the Linux desktop helps recruit potential contributors. This is, imho, very important for Illumos. I see no logical reason why Illumos can't be an amazing desktop system given that it has best of breed technologies. It won't be able to inter-operate with proprietary formats and protocols (skype, MS Office, etc), but that's why KVM is an excellent technology for a desktop OS. I think community contributions to desktop technologies should be encouraged by the Illumos devs even if they have no interest in writing the code themselves. I am not saying that Illumos can dethrone dominant desktop OS's; we can't, nor should we, cater to non-techies. I am saying that it can grab a fragment of those desktop power-users (those that run linux, for example, but would like to use something better, because it comes at no personal cost to them, but can help them significantly). > > > ___ > oi-dev mailing list > oi-dev@openindiana.org > http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Sep 2, 2012, at 11:18 AM, Alan Coopersmith wrote: > On 09/ 2/12 07:00 AM, Adam Števko wrote: >> IPS is documented in the official IPS Developer Guide located somewhere in >> the OpenSolaris/Oracle page. I went it through lately and I find it a good >> source for learning to work with IPS, in general. > > http://hub.opensolaris.org/bin/download/Project+pkg/files/ipsdevguide.pdf FWIW, I think this sort of response is a sign of ill health for Illumos. We're still pointing to Oracle for our own documentation. And, as we should know by now, Oracle has no problems withdrawing things from public view with no warning. Additionally, I see us debating about putting a lot of work into supporting a small fringe of users (desktop) while nobody is really talking about modernizing hardware support for ubiquitous 4K sector hard disks (and I mean beyond crude hacks). We're missing the big important stuff. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 2/12 07:00 AM, Adam Števko wrote: > IPS is documented in the official IPS Developer Guide located somewhere in > the OpenSolaris/Oracle page. I went it through lately and I find it a good > source for learning to work with IPS, in general. http://hub.opensolaris.org/bin/download/Project+pkg/files/ipsdevguide.pdf -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Hi Nick, On Sep 1, 2012, at 5:42 PM, Nick Zivkovic wrote: > Yes. I am more interested in contributing drivers and the like. As far > as packages go, to be honest, I've experienced torture at the hands of > IPS (though that could very easily be my fault), and am reluctant go > near it. For example I tried an image-update and it failed. So I am > stuck on OI-147 until I backup-reinstall-import to OI-151a. > > I think packages are a high priority, but not as high as making sure > the latest illumos-gate can build and run on a modern desktop. For > example, I can't get SmartOS running on a thinkpad or my desktop > computer. Somewhere in June 2012, a bug was introduced that prevents > the illumos kernel from booting. If I had been building and testing > the latest source, that bug could probably have been caught before it > got buried in a mountain of commits. Now, I image, it is like finding > a needle in a haystack. I couldn't agree more that having great hardware support is very important. Although, packages available in the repository should have a high-priority. If you enjoy hacking on illumos-gate then just do it, everyone using it will benefit from it. Also, If you use OpenIndiana with software that is not available via repositories and you are willing to maintain those things, feel free to do so. Any additional questions about packaging/developing will be gladly answered. > I am willing to assist with packages, but my time is limited, and I > think it is more important to direct my effort to building > illumos-gate and writing drivers. Also, making packages is still a > black art to me, and wouldn't know where to start. Everything related to packaging can be found at http://wiki.illumos.org/display/illumos/illumos-userland > > But since we are already on the topic of packages, Adam, do you think > there is a way to make it less painful, more consistent? I'm _not_ > talking about extreme measures like changing from IPS to > [DEB/RPM/SVR/etc]. I'm wondering if we could 1) make IPS easier to use > by documenting stuff in an easily accessible way [the man pages aren't > very helpful] 2) document every single IPS failure and either fix the > packages or the IPS code (depend on what caused the failure), IPS is documented in the official IPS Developer Guide located somewhere in the OpenSolaris/Oracle page. I went it through lately and I find it a good source for learning to work with IPS, in general. The error documentation could be doable, although I am not sure if it will benefit us that much. Further discussions are needed for this imho. > and 3) > have IPS install all userland libs to a zfs dataset named rpool/ips or > rpool/pkgs; this way, we can zfs-send these datasets, and snap-shot > them, and clone them, without pulling in the rest of the file-system > heirarchy. This would make my bitterness toward IPS reduce > significantly. This way, you can migrate different user-land configs > between systems. Also, an easy way to do updates across a multitude of > systems. One can share their binaries and packages via zfs-send, > because they won't destroy an existing system's /usr /bin and so > forth. Also, OI would benefit tremendously from offering pre-made NG > zones on the web-site, available for downloading and running. In fact, > we could use Zones as a delivery mechanism for things like an Illumos > build-environment. An NG zone can contain a working and sandboxed > version of firefox. Zones are a great technology that can make the > system more attractive amateur power users who may become programmers > some day (like I did). Multiple ways of sharing pre-compiled binaries > can only help OI and Illumos. In fact I can see people sharing > datasets with packages via bit-torrent. Plus, incremental send/recv is > a huge benefit. OpenIndiana aims to be a general-purpose traditional distribution usable on server or desktop, not a hypervisor (although kvm and qemu packages can be found in the repositories). The zone thingie is a great idea imo. Delivering prebuilt zone usable for packaging and Illumos development could ease certain things for developers and help them concentrate more on the development. But again, this can be delivered also as a meta package available via IPS. > We might even be able to integrate a window manager (like i3 or dwm) > so that switching virtual desktop, actually switches to another zone. > > What kind of changes to IPS are OI willing to accept? I am willing to > test and improve a lot of code. As I said, I dislike IPS. But I am > willing to help make it better and more usable. I am not sure. People more involved with IPS should comment on this. > Also, a major problem with IPS is that Sun encouraged people to use it > to _consume_ packages, but they didn't encourage people to _create_ > packages. We need a self-fueling ecosystem of packages. There is already a oi-build, which needs more packages and updates. oi-build is based on Oracle's userland
Re: [oi-dev] Desktop Illumos Still Matters
Yes. I am more interested in contributing drivers and the like. As far as packages go, to be honest, I've experienced torture at the hands of IPS (though that could very easily be my fault), and am reluctant go near it. For example I tried an image-update and it failed. So I am stuck on OI-147 until I backup-reinstall-import to OI-151a. I think packages are a high priority, but not as high as making sure the latest illumos-gate can build and run on a modern desktop. For example, I can't get SmartOS running on a thinkpad or my desktop computer. Somewhere in June 2012, a bug was introduced that prevents the illumos kernel from booting. If I had been building and testing the latest source, that bug could probably have been caught before it got buried in a mountain of commits. Now, I image, it is like finding a needle in a haystack. I am willing to assist with packages, but my time is limited, and I think it is more important to direct my effort to building illumos-gate and writing drivers. Also, making packages is still a black art to me, and wouldn't know where to start. But since we are already on the topic of packages, Adam, do you think there is a way to make it less painful, more consistent? I'm _not_ talking about extreme measures like changing from IPS to [DEB/RPM/SVR/etc]. I'm wondering if we could 1) make IPS easier to use by documenting stuff in an easily accessible way [the man pages aren't very helpful] 2) document every single IPS failure and either fix the packages or the IPS code (depend on what caused the failure), and 3) have IPS install all userland libs to a zfs dataset named rpool/ips or rpool/pkgs; this way, we can zfs-send these datasets, and snap-shot them, and clone them, without pulling in the rest of the file-system heirarchy. This would make my bitterness toward IPS reduce significantly. This way, you can migrate different user-land configs between systems. Also, an easy way to do updates across a multitude of systems. One can share their binaries and packages via zfs-send, because they won't destroy an existing system's /usr /bin and so forth. Also, OI would benefit tremendously from offering pre-made NG zones on the web-site, available for downloading and running. In fact, we could use Zones as a delivery mechanism for things like an Illumos build-environment. An NG zone can contain a working and sandboxed version of firefox. Zones are a great technology that can make the system more attractive amateur power users who may become programmers some day (like I did). Multiple ways of sharing pre-compiled binaries can only help OI and Illumos. In fact I can see people sharing datasets with packages via bit-torrent. Plus, incremental send/recv is a huge benefit. We might even be able to integrate a window manager (like i3 or dwm) so that switching virtual desktop, actually switches to another zone. What kind of changes to IPS are OI willing to accept? I am willing to test and improve a lot of code. As I said, I dislike IPS. But I am willing to help make it better and more usable. Also, a major problem with IPS is that Sun encouraged people to use it to _consume_ packages, but they didn't encourage people to _create_ packages. We need a self-fueling ecosystem of packages. I also think that SmartOS's diskless boot model is great. I think that booting from disk is great too. Shouldn't OI support both? I'm willing to contribute to this. I know these ideas come from SmartOS to some extent, but they are great ideas that could make OI better! Making a new distribution is one way to try to make things better. But I think a metamorphosis in the OI distro will be more effective. I want the many Illumos distros to be held up as an example of triumphant collaboration, 5 years from now. But that will happen only if we avoid going down the path of NIH-inspired suicide. So, in short I am willing to contribute, to OpenIndiana and Illumos. I will get OI-151 installed today or tomorrow. I will try to build illumos-gate, and will report back with any problems. I would appreciate any pointers on making new packages. Is it possible to make a new zone without an internet connection? Where can I find the OI plans for future IPS features and improvements. Also, I don't know if this is available in your repos, but if not, I am going to port and package the i3 window manager for OI, if I have trouble I'll let you guys know. I am going to see what I can do about pre-built NG zones. I will try to find resources about NG zones, making new brands, modifying existing brands, etc. Also, I recommend updating the mission statement on the web page. It is "coming soon", and not very inspiring. I recommend something along the lines of "making cutting edge technology available to power users on the desktop..." and then advertise the technologies. Trust me, it is the power users, not the simple desktop users that you want. Basically an incubator for future illumos devs, and a platform for those who like to
