Re: [lfs-dev] Once more: Package Management

2012-06-03 Thread Pierre Labastie
Le 02/06/2012 18:00, Jeremy Huntwork a écrit :
 I'm going to start by jumping on the parser, since it will be 
 necessary right away for any of this to work. My initial thoughts are 
 to build one parser that can accept different output filters, for 
 example, outputting to PKGBUILD files, or rpm spec files, or even, the 
 jhalfs-type build scriptlets. Any thoughts on that? JH 
Thinking a little more about it, I think I might have done a small part 
of the work, which might be interesting to you.
For the BLFS part of jhalfs, I needed to generate a list of the 
packages, with the version and the dependencies. I chose to have it in 
xml. A typical entry is:
package
nameaccountsservice/name
version0.6.15/version
dependency status=required name=dbus-glib type=ref/
dependency status=required name=polkit type=ref/
dependency status=optional name=gobject-introspection type=ref/
dependency status=optional name=xmlto type=ref/
/package

You could add a few tags, like tarball, scriptlet (maybe that one 
splitted into configure, make, install, ...), and so on and you would be 
done.

The parser for this file is BLFS/xsl/gen_pkg_list.xsl, in the project 
svn://svn.linuxfromscratch.org/ALFS/jhalfs/branches/ablfs. It uses 
another generated file for the special cases, generated by 
BLFS/gen-special.sh. It is for the BLFS book. I think it should be easy 
to adapt it to LFS.

Regards
Pierre
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-06-02 Thread Jeremy Huntwork
On 6/1/12 9:31 AM, James Robertson wrote:
 Sure.  Lets take sudo as an example.  What could be considered a simple
 program has 8 optional dependencies with 4 being off book. I think we
 ignore those 4 and worry only for the 4 in book optional
 dependencies.  The build instructions in the main BLFS book explicitly
 use two without configure parameters to get around no pam or an mta.  So
 I am thinking we have a few versions of this package.

   sudo-1.8.4p4-book
   sudo-1.8.4p4-pam-sendmail
   sudo-1.8.4p4-pam-sendmail-kerberos-ldap

 The first notates that it is a package right from the book.  The second
 one adds pam and leverages a local sendmail and the third has all the
 in book optional deps.  The second and third packages would been a
 pre- script to check for a valid sendmail command before installation as
 well.

 The spec file for each of these will be slightly different and certainly
 the run time deps will be as well.

 Lets use shadow as another example.  Since shadow is also in LFS, there
 is a real need for more than one version.

   shadow-4.1.5-lfsbook
   shadow-4.1.5-blfsbook-pam-cracklib

 or something like that.

I have to admit that I didn't like this at first, because something like 
this would likely confuse auto-installers. For example, if we used 
pacman to auto-install a full base system to use as development - which 
version of shadow do we use? But after thinking on it some more, I think 
there are likely various ways to work around this and your multi-package 
method does fit the BLFS model more closely.

 We you thinking of providing a pre-packaged kernel?  That is going to be
 an interesting one as we all love to customize our kernels.  Present
 company included.

Yeah, not sure about this one. For pure (B)LFS development purposes, a 
packaged kernel isn't necessary at all. Perhaps we leave this as a 
'build it and package it yourself' option for anyone that wants to 
actually use the binaries as a working system. I suppose we could still 
provide a template for it and let people generate their own config - the 
default perhaps being a very general purpose nearly-all-modules-enabled 
config.

 I have been wanting to get back to contributing for some time and this
 seems to be a great place to do it.

I'm going to start by jumping on the parser, since it will be necessary 
right away for any of this to work. My initial thoughts are to build one 
parser that can accept different output filters, for example, outputting 
to PKGBUILD files, or rpm spec files, or even, the jhalfs-type build 
scriptlets.

Any thoughts on that?

JH


-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-06-02 Thread Pierre Labastie
Le 02/06/2012 18:00, Jeremy Huntwork a écrit :
 I'm going to start by jumping on the parser, since it will be necessary
 right away for any of this to work. My initial thoughts are to build one
 parser that can accept different output filters, for example, outputting
 to PKGBUILD files, or rpm spec files, or even, the jhalfs-type build
 scriptlets.

I can tell you the difficulties I had when trying to improve the BLFS 
part of jhalfs. You might be facing some of them too. For example, it is 
usually easy to find the name and the version of a package, by cutting 
the full name at a dash preceding a numeric character, but not always, 
because some version strings contain alphabetic characters and package 
names contain dashes. I had to examine package.ent to find all the 
version strings and take some actions for the cases where the version 
strings contain letters.
Also, there are some varations in the book on the way download addresses 
are given. For example, I have not been able to find a way to automate 
the download of all the tarballs in openjdk.

Regards
Pierre
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-06-01 Thread James Robertson
On Thu, May 31, 2012 at 5:40 PM, Jeremy Huntwork 
jhuntw...@lightcubesolutions.com wrote:

 On 5/31/12 4:41 PM, James Robertson wrote:

  1. Adding PM is NOT a replacement for the books.  It should also be
  noted and clear that the purpose of this effort is not to turn (B)LFS
  into a binary distro.  It is and always should be a cookbook so those
  that want can still roll their own.  I really think that this effort
  should be a book development tool and cookbook for those wanting to
  learn about package management.  I think the current books stay as is
  and PM is another off-shoot like cLFS.  The books should stay
  independent as they have different goals.  I am thinking that we attempt
  to leverage the (B)LFS book sources in some manner (maybe combine them
  into a super book) and then add the PM stuff to each page (build
  steps).  If we go this route, we won't confuse goals and won't make the
  main books too hard to read.  Remember we get new users all the time.
  Also by creating an off-shoot version of each book, we can go hog wild
  with educational text about package management strategies, procedures,
  etc and that text stays out of the main books (again because it is not
  the main book's goal).

 Hmm, possibly. I'm wary of new projects/books (given their life span)
 but I also see the value in what you're saying about keeping the book
 unchanged.


I understand.  I think the value is greater however than attempting to put
what all I can see us wanting to put into this into the main books.


  Questions about the binary repository came up.  I too have some
  questions about that.  What is the definition of 'official'?  Who/What
  is 'official'?  Who is going to vet submitted binaries?  What is the
  standard we are going to follow?  I would assume binaries compiled with
  commands right out of the books with no extra optimization flags and
  such, but that should be agreed upon.  Yet another good reason for PM to
  be a separate book.  We can have a whole chapter documenting this if we
  need to.

 Yep, all good questions. I don't know the answer yet. Perhaps the answer
 is to do like I suggested in another message and show a proof of concept
 first, then we can tackle the hard decisions as the concept gains ground.


Yes, I definitely think a poc is in order.  Like many things, if you build
it, they will come plays out nicely.  I honestly think that a combined
super book that leverages the build page source from each book is the
best approach.  That way we don't have to re-type (CnP) the main book
source steps into the PM cookbook and then can add PM specific items like
you suggested we add into the main books there.  I am not sure how to do
this exactly, but if we can figure it out it would make development of the
book easier.  We could then take some advantage of a single book and insert
optional steps in from BLFS that make sense and also not have the direct
break between books as the purpose is really to work with both to aid
development.



  The separate work things also aid #3 above as well.  We can document the
  standard workflow, tool use, etc there.  Things like what you need to
  get started building a book development environment, a reminder of
  selection criteria used for the PM tool of our choice (just to help us
  remember why a certain tool was picked as our memory fades), etc.  One
  tool I think we will need is a spec file generator.  This goes back to
  my thought about putting xml tags in the book source that is parsed out.

 Yep, the parser/generator will need to be one of the first things tackled.

  WRT BLFS book development specifically.  Lots of commentary about
  dependencies - both build and run-time ones.  Part of the PM project
  would be to standardize how we create packages.  My thinking is to
  create a package for the required deps and the come up with a way to
  generate other versions of a package with the run time deps built
  in.   Keeping PM in a separate book will give us this option to document
  and explain.

 Not sure I'm following this exactly. Do you mind being a little more
 specific or provide an example?


Sure.  Lets take sudo as an example.  What could be considered a simple
program has 8 optional dependencies with 4 being off book. I think we
ignore those 4 and worry only for the 4 in book optional dependencies.
The build instructions in the main BLFS book explicitly use two without
configure parameters to get around no pam or an mta.  So I am thinking we
have a few versions of this package.

 sudo-1.8.4p4-book
 sudo-1.8.4p4-pam-sendmail
 sudo-1.8.4p4-pam-sendmail-kerberos-ldap

The first notates that it is a package right from the book.  The second one
adds pam and leverages a local sendmail and the third has all the in book
optional deps.  The second and third packages would been a pre- script to
check for a valid sendmail command before installation as well.

The spec file for each of these will be slightly different and 

Re: [lfs-dev] Once more: Package Management

2012-05-31 Thread James Robertson
On Sat, May 19, 2012 at 8:26 AM, Jeremy Huntwork 
jhuntw...@lightcubesolutions.com wrote:

 I've been holding back bringing this up on-list for a while because I
 intended to do the bulk of the work and then present a working system to
 the community for comment and review. I still intend to do that, but
 given some recent discussions, I think the time is right to bring this
 up and see where it goes.


I have been watching this thread and it seems to have gone a bit dormant so
maybe now is a good time to add my thoughts.  First off -  Jeremy, your
contributions to this project continue to amaze me.  Keep it up buddy.


 (Sorry for the cross posting, but since it involves both projects, I
 wanted to make sure all saw it - if possible, please reply on lfs-dev.)

 Proposal:

 1. Adjust LFS/BLFS to auto-generate build recipes for individual
 packages that a packaging tool can use to create binary packages with
 meta information included such as dependency tracking.

 2. Store 'official' copies of those binary packages in a developer
 package repository so that when developing (primarily BLFS) a dev can
 automatically pull and install into a build environment the dependencies
 he needs to build and create an individual package.

 3. Create a standard workflow and tools whereby a developer can
 accomplish #2 and edit the book accordingly in an efficient, reliable way.


OK so first let's make sure we are clear on goals, roles and
responsibilities here.  The thread seemed to float around a bit with some
concerns about losing the roll your own control that the books give us.

1. Adding PM is NOT a replacement for the books.  It should also be noted
and clear that the purpose of this effort is not to turn (B)LFS into a
binary distro.  It is and always should be a cookbook so those that want
can still roll their own.  I really think that this effort should be a book
development tool and cookbook for those wanting to learn about package
management.  I think the current books stay as is and PM is another
off-shoot like cLFS.  The books should stay independent as they have
different goals.  I am thinking that we attempt to leverage the (B)LFS book
sources in some manner (maybe combine them into a super book) and then add
the PM stuff to each page (build steps).  If we go this route, we won't
confuse goals and won't make the main books too hard to read.  Remember we
get new users all the time.  Also by creating an off-shoot version of each
book, we can go hog wild with educational text about package management
strategies, procedures, etc and that text stays out of the main books
(again because it is not the main book's goal).

2.  PM is for book developers for the most part.  Support for the binary
packages and repository is to aid book developers. Non-book developers that
want to use the packages for their own purposes can do so at any time, but
the *-dev community does not support them for that purpose.

If we keep these things separate, we can add value for those that are
interested and reduce confusion and angst for those that are not.  I am
also thinking that if we leverage an extension of the main book sources
for the PM branch, then we can leverage jhalfs (after an enhancement) or
another parser to pulls the PM commands out of the book for rapid re-use.

Questions about the binary repository came up.  I too have some questions
about that.  What is the definition of 'official'?  Who/What is
'official'?  Who is going to vet submitted binaries?  What is the standard
we are going to follow?  I would assume binaries compiled with commands
right out of the books with no extra optimization flags and such, but that
should be agreed upon.  Yet another good reason for PM to be a separate
book.  We can have a whole chapter documenting this if we need to.

The separate work things also aid #3 above as well.  We can document the
standard workflow, tool use, etc there.  Things like what you need to get
started building a book development environment, a reminder of selection
criteria used for the PM tool of our choice (just to help us remember why a
certain tool was picked as our memory fades), etc.  One tool I think we
will need is a spec file generator.  This goes back to my thought about
putting xml tags in the book source that is parsed out.



 Rationale:

 (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a
 huge undertaking - and it's a difficult beast to maintain. In order to
 create a stable reference page in BLFS an editor has to have spent the
 time to build all of LFS, tweak it, go through current BLFS, manually
 install dependency packages and then carefully document the package he
 builds. No two developers are guaranteed to have the same environment,
 either, so accuracy and stability becomes an issue.


WRT BLFS book development specifically.  Lots of commentary about
dependencies - both build and run-time ones.  Part of the PM project would
be to standardize how we create packages.  My 

Re: [lfs-dev] Once more: Package Management

2012-05-31 Thread Jeremy Huntwork
On 5/31/12 4:41 PM, James Robertson wrote:
 I have been watching this thread and it seems to have gone a bit dormant
 so maybe now is a good time to add my thoughts.  First off -  Jeremy,
 your contributions to this project continue to amaze me.  Keep it up buddy.

Thanks James, :)

 1. Adding PM is NOT a replacement for the books.  It should also be
 noted and clear that the purpose of this effort is not to turn (B)LFS
 into a binary distro.  It is and always should be a cookbook so those
 that want can still roll their own.  I really think that this effort
 should be a book development tool and cookbook for those wanting to
 learn about package management.  I think the current books stay as is
 and PM is another off-shoot like cLFS.  The books should stay
 independent as they have different goals.  I am thinking that we attempt
 to leverage the (B)LFS book sources in some manner (maybe combine them
 into a super book) and then add the PM stuff to each page (build
 steps).  If we go this route, we won't confuse goals and won't make the
 main books too hard to read.  Remember we get new users all the time.
 Also by creating an off-shoot version of each book, we can go hog wild
 with educational text about package management strategies, procedures,
 etc and that text stays out of the main books (again because it is not
 the main book's goal).

Hmm, possibly. I'm wary of new projects/books (given their life span) 
but I also see the value in what you're saying about keeping the book 
unchanged.

 Questions about the binary repository came up.  I too have some
 questions about that.  What is the definition of 'official'?  Who/What
 is 'official'?  Who is going to vet submitted binaries?  What is the
 standard we are going to follow?  I would assume binaries compiled with
 commands right out of the books with no extra optimization flags and
 such, but that should be agreed upon.  Yet another good reason for PM to
 be a separate book.  We can have a whole chapter documenting this if we
 need to.

Yep, all good questions. I don't know the answer yet. Perhaps the answer 
is to do like I suggested in another message and show a proof of concept 
first, then we can tackle the hard decisions as the concept gains ground.

 The separate work things also aid #3 above as well.  We can document the
 standard workflow, tool use, etc there.  Things like what you need to
 get started building a book development environment, a reminder of
 selection criteria used for the PM tool of our choice (just to help us
 remember why a certain tool was picked as our memory fades), etc.  One
 tool I think we will need is a spec file generator.  This goes back to
 my thought about putting xml tags in the book source that is parsed out.

Yep, the parser/generator will need to be one of the first things tackled.

 WRT BLFS book development specifically.  Lots of commentary about
 dependencies - both build and run-time ones.  Part of the PM project
 would be to standardize how we create packages.  My thinking is to
 create a package for the required deps and the come up with a way to
 generate other versions of a package with the run time deps built
 in.   Keeping PM in a separate book will give us this option to document
 and explain.

Not sure I'm following this exactly. Do you mind being a little more 
specific or provide an example?

 I would like to contribute to this one, but won't be able to until this
 fall when I can get some other things off my plate.  Never seems to be
 enough time to do what I want to do.

Nice! Totally understood. We're all busy... some perhaps more than 
others. :) I look forward to working with you on it.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-22 Thread Pierre Labastie
Le 22/05/2012 01:58, Armin K. a écrit :
 Also, I see you mention package managers ... For me, Debian's dpkg is
 the hardest one. debian/rules file uses Makefile syntax which I am not
 familiar with. Red Hat's rpm uses some kind of spec file which doesn't
 seem that hard to understand, but still it's medium-hard for newbies and
 for me, too. Gentoo's portage is difficult like hell, I never even
 bothered to look at one of it's files. Archlinux's pacman seems most
 easiest. It uses bash syntax, it's just like LFS input commands are
 pasted into that file. So, if you people decide to make book generate
 spec files, go for pacman, it will be the most easiest to implement.

Hi,
It depends on what you want to do with a package manager. I would 
separate two main uses :
(a) Building from sources using a spec file.
(b) Packaging and installing.

As far as (B)LFS is concerned, (a) involves some kind of rewriting of 
the book instructions to make a spec file. I do not know whether it is 
the purpose of (B)LFS to show how to do that. Among the PM I know 
(mainly dpkg and pacman), pacman is by far the easiest for this purpose. 
The spec files used by Bent linux were very simple too, but the project 
seems to have been stale for a few years. I agree that dpkg is more work 
to learn, but at least it is Makefile instructions, not a completely new 
language as rpm.

If the only aim is to do (b), and I would believe that it is what is 
needed by (B)LFS, dpkg is by far the simplest: it amounts to build per 
the book instructions adding DESTDIR to the install commands, making a 
very simple description file (sample below), and running dpkg-deb then 
dpkg -i. Even Pacman is slightly more complicated, because (a) and (b) 
are not well separated. The advantage is that the PM warns you about 
overwritten files, and you can uninstall with just one command.

Sample of the description file for gcc in chapter 6 of LFS:

-
cd $DESTDIR

mkdir -v DEBIAN

cat  DEBIAN/control  EOF

Package: gcc-lfs

Version: 4.4.3

Architecture: i386

Maintainer: Pierre Labastie lnim...@club-internet.fr

Depends: mpfr

Provides: gcc, g++

Description: GNU compiler collection (C and C++)

This package is the LFS installation of the GNU compiler collection, 
which contains only

C and C++. You need to install gcc-blfs for the other compilers.

EOF

--
Only five lines are mandatory:
Package, Version, Architecture, Maintainer, Description. Depends and 
Provides are of course rather useful. It can be made (slightly or much) 
more sophisticated, regarding the post installation instructions and the 
management of config files. This is the same for other PMs. Those 
sophistications are not really needed for (B)LFS, IMHO.

Regards
Pierre



-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-22 Thread Jeremy Huntwork
On 5/20/12 7:09 PM, Ken Moffat wrote:
   The more I think about this, the less happy I am.  Point 2 doesn't
 really help editing BLFS as far as I can see (upgrading a package
 usually needs several builds - typically, for me, a first to see if
 it actually works when I use it, then others to get nice clean
 measurements, check what is in the DESTDIR (or INSTALLROOT), and
 review options for the optional dependencies and any testsuite.

Actually, from my experience a packaging tool eases a lot of the pain 
inherent in the process you've described. Auto-fetching of sources 
(including dependencies), auto-preparation of a clean build tree, 
post-build reports of what files have been included in the package and 
what (if any) auto dependencies have been discovered.

Re-doing a build if issues or errors are found (including, if necessary, 
setting up an entire chroot system as well) becomes a matter of one 
command, instead of several.

   Then there is the question of dependencies - in BLFS we don't
 normally tell people how to use optional deps.  Sometimes they are
 picked up automatically, but many times you have to pass a switch to
 get them used.  The instructions in BLFS are hopefully correct, but
 they don't suit everyone.

   I'm also wary of standard workflows - my own LFS build is different
 enough (nothing updated in /sources, because that is an nfs mount on
 my systems, and with efforts to remove many of the static libraries)
 that I expect pain.  And that's just for LFS.  For BLFS, I suspect
 this sort of change will actually increase my workload and therefore
 reduce my contributions.

This is one of those scenarios where I think it's difficult to see how a 
thing will be useful or beneficial until after you've actually tried it 
out. I'm not guaranteeing you'll think it better, more useful or simpler 
- though my belief is that it does actually simplify the process of 
correctly building and documenting the build of an individual package, 
once the tools are available and the infrastructure in place.

   The intention is good, but I'm not at all convinced that the plan
 will help.

Perhaps it really is something that needs to been seen and experienced. 
I can do as I planned and document the procedure to build in PM with LFS 
and set up the basic infrastructure required (a web server with a public 
binary repo). Once that's in place, anyone who wants to test out the new 
workflow can try it and offer thoughts, comments, suggestions. Then 
perhaps a decision can be reached as to whether it be actually 
implemented for either LFS or BLFS. This way, too, at least the 
documentation will have been done.

Anyone interested in helping with the above, or should I just report 
back here when I've accomplished this on my own?

   Also, can I really trust whoever is permitted to edit a book (I'm
 assuming that part of the aim is to get more people editing in BLFS
 ?) to have an uncompromised system, and to not want to upload
 compromised binaries ?  For the xml in the book, and for patches, we
 can look at what is being changed.  For a binary, how do we know
 what was done to it ?  Distros have build machines with restricted
 access and some degree of security.  Is LFS going to need signed
 binaries and a ring of trust ?

This is a great question, and I'm still not sure the best way to answer 
it. Perhaps it should be tabled and reconsidered if and when either book 
looks to actually adopt the workflow as 'official'.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-21 Thread Armin K.
On 05/22/2012 01:27 AM, DJ Lucas wrote:

 It omits the core point of LFS...education. I dislike providing binary
 packages to users, but a working example of how to create those binary
 packages is something we've lacked for a long time, plus getting those
 packages that don't honor DESTDIR explained with a working example
 chroot...I just can't (or possibly don't want to) see a downside as far
 as education or documentation is concerned. Yes, it'll undoubtedly be a
 bit hairy at first. As far as choosing what PM to use, I guess it is no
 different than making the decision to use vim over emacs (an equally
 nasty holy war). As long as concepts learned are applicable to other
 PMs, then it is all good. I suspect a good deal could be taken from
 other consumers of pacman (that is what Arch uses, right?).

 -- DJ


I don't get what's the whole thing about, but I think that packaging can 
be documented very well. I have installed about 1000 packages using 
DESTDIR or equivalent method, writing base INSTALL files which contain 
just cp -r commands and some triggers, like adding an user, chowning, 
chmoding, running gtk-update-icon-cache, man-db update, installing info 
files and such. It's not hard work and it can be done if we just put 
some effort. Also, if this is to happen, I suggest we don't make any 
changes to the current BLFS since Bruce has planed release for 
September, and it isn't enough time for the goal. I am in process of 
fully documenting build process from cd sourcedir to after make install 
DESTDIR, including install triggers, post make install stuff like 
stripping, creating aditional files, symlinks and such.

Also, I see you mention package managers ... For me, Debian's dpkg is 
the hardest one. debian/rules file uses Makefile syntax which I am not 
familiar with. Red Hat's rpm uses some kind of spec file which doesn't 
seem that hard to understand, but still it's medium-hard for newbies and 
for me, too. Gentoo's portage is difficult like hell, I never even 
bothered to look at one of it's files. Archlinux's pacman seems most 
easiest. It uses bash syntax, it's just like LFS input commands are 
pasted into that file. So, if you people decide to make book generate 
spec files, go for pacman, it will be the most easiest to implement.

Sure there are more, but theese are most common ones mentioned.
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Bryan Kadzban
FWIW...

DJ Lucas wrote:
 Fortunately, that is not a deal breaker for me if the 
 readers get the same treatment (which seems to be the case), but this 
 does hard code optional dependencies for the pre-packaged installations. 
 This is both good and bad. From a development standpoint, it won't take 
 me a week to build a fairly standard system to test a simple package 
 upgrade, but that still means manual (or maybe only partially automated) 
 builds for omitting optional deps in the build process up to the point 
 that I need.

This exact reason, for the record, is why I really dislike binary
distros.  I *never* choose the same set of dependencies that are
optional in the source, as the distro does.  And because when they ran
./configure, they added a --with-foo flag, the package compiled with
-lfoo, meaning the binary version of the package now has a hardcoded
requirement for libfoo.so.N or whatever it is.

For example, python and bluez.  I don't want an entire bluetooth stack.
 But some distros' python packages require that stack in order to run.
For another example, gtk+ (both 2 and 3), or chrome, or firefox, and
CUPS: I don't want an entire printserver stack (since I only have a
single printer, and only use it from one machine).  Or xine-lib and ...
well, any half of its dependencies, depending on what you're going to
use it to play.  :-)

Now I should say that I don't think this necessarily applies to BLFS,
except inasmuch as testing with or without some of the optional
dependencies might require changes to the package in question.  (If the
CUPS-less version of FF needs some other configuration different from
the FF-with-CUPS binary, for instance.  I don't think that's the case
today at least.)  OTOH when building a system I can probably figure that
out, too.

 My hope is that build order is still a manual process where the user 
 determines build order herself. Dependency checking is done only at 
 build time and that optional deps remain optional. If there will be 
 automation, how do we determine what optional deps to pull in?

Perhaps it makes more sense to just keep ALFS the way it is, and not
build packages?  I *know* that I won't be building any package-ish thing
for my systems at all; I'm still using an oldish version of MSB's
package-user setup, so none of this applies.

In other words, I think it'd help to only use packages to simplify
(mostly BLFS) testing, but make them semi-public for people who really
want them.  Don't use them at all in the actual build instructions (what
would be the point?  :-) ), but generate the spec files, or whatever
equivalent, from the book XML.  (This last bit might be either hard or
impossible; I don't know.)



signature.asc
Description: OpenPGP digital signature
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Bruce Dubbs
Bryan Kadzban wrote:
 FWIW...
 
 DJ Lucas wrote:
 Fortunately, that is not a deal breaker for me if the 
 readers get the same treatment (which seems to be the case), but this 
 does hard code optional dependencies for the pre-packaged installations. 
 This is both good and bad. From a development standpoint, it won't take 
 me a week to build a fairly standard system to test a simple package 
 upgrade, but that still means manual (or maybe only partially automated) 
 builds for omitting optional deps in the build process up to the point 
 that I need.
 
 This exact reason, for the record, is why I really dislike binary
 distros.  I *never* choose the same set of dependencies that are
 optional in the source, as the distro does.  And because when they ran
 ./configure, they added a --with-foo flag, the package compiled with
 -lfoo, meaning the binary version of the package now has a hardcoded
 requirement for libfoo.so.N or whatever it is.

I agree with this.  I am updating vim in BLFS to add current patches and 
do not like all the xorg dependencies in vim.  Others may want gvim.

There are a lot of decisions that must be made in BLFS about 
dependencies.  It's difficult to provide a package manager that does not 
take away the user's choices.

   -- Bruce
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Jeremy Huntwork
On 5/20/12 2:18 PM, Bryan Kadzban wrote:
 In other words, I think it'd help to only use packages to simplify
 (mostly BLFS) testing, but make them semi-public for people who really
 want them.  Don't use them at all in the actual build instructions (what
 would be the point?  :-) ), but generate the spec files, or whatever
 equivalent, from the book XML.  (This last bit might be either hard or
 impossible; I don't know.)

Yes - I believe you have hit exactly what I'm trying to propose. The 
binaries (and binary repository) aren't advertised in any way for 
general use - they aren't even mentioned in the build instructions.

For example, consider this as a very simplified version of a package page:

Build package Blah:
Manually:
  ./configure ...
  make

(Optionally, via package tool):
  # Grab generated spec file here...
  ./makepkg


And then the spec file, would have references to the dependency packages 
in BLFS, calling them exactly how they've been referenced in the book's 
source. I suppose here the 'optional' ones could be enabled or disabled 
by a reader. As would devs - although I think for dev purposes it might 
be useful to have built binaries with all optional dependencies enabled.

Behind the scenes, a set of tools and a defined workflow can help the 
devs to share binaries already produced via the spec files - but those 
are never explicitly shared or brought to the attention of the reader.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Qrux

On May 20, 2012, at 1:58 PM, Jeremy Huntwork wrote:

 On 5/20/12 3:10 PM, Bruce Dubbs wrote:
 This exact reason, for the record, is why I really dislike binary
 distros.  I *never* choose the same set of dependencies that are
 optional in the source, as the distro does.  And because when they ran
 ./configure, they added a --with-foo flag, the package compiled with
 -lfoo, meaning the binary version of the package now has a hardcoded
 requirement for libfoo.so.N or whatever it is.
 
 I agree with this.  I am updating vim in BLFS to add current patches and
 do not like all the xorg dependencies in vim.  Others may want gvim.
 
 There are a lot of decisions that must be made in BLFS about
 dependencies.  It's difficult to provide a package manager that does not
 take away the user's choices.

Couldn't agree more.

For instance, SuSE wants Xorg for python, which I only needed for Xen.

 I think perhaps the point is being missed here.

Couldn't agree more with this, too.

I don't feel it's conflicting to have a system where you can take LFS a a base, 
and then build on top of it.

This situation seems to really speak to git, where it's easier to branch 
off--and to decentralize, because any project based on LFS obviously still 
needs the trunk, but would allow addons--stuff like JH's sysroot build and 
also the current package management concepts--to get explored, without always 
sounding the alarm bells of whether or not it's violating the book.

 So there is no threat here to what LFS or BLFS currently is. I 
 absolutely agree that choice of optional dependencies should be left 
 completely to user discretion.
 
 A decision about how to build a binary (and provide a spec file) for use 
 by other developers should be based completely, then, on what is useful 
 for developing BLFS.

Totally agree here.  I think people are seeing LFS addons as being in 
conflict with the book, and that's probably not the case.

The issue seems to be that there's no way for people to build addons with 
either centralized control (official repo signoff) or distributed development 
(fork it, because it's not a big deal to fork).

Q


-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Ken Moffat
On Sat, May 19, 2012 at 09:26:31AM -0400, Jeremy Huntwork wrote:
 
 Proposal:
 
 1. Adjust LFS/BLFS to auto-generate build recipes for individual 
 packages that a packaging tool can use to create binary packages with 
 meta information included such as dependency tracking.
 
 2. Store 'official' copies of those binary packages in a developer 
 package repository so that when developing (primarily BLFS) a dev can 
 automatically pull and install into a build environment the dependencies 
 he needs to build and create an individual package.
 
 3. Create a standard workflow and tools whereby a developer can 
 accomplish #2 and edit the book accordingly in an efficient, reliable way.
 
 The more I think about this, the less happy I am.  Point 2 doesn't
really help editing BLFS as far as I can see (upgrading a package
usually needs several builds - typically, for me, a first to see if
it actually works when I use it, then others to get nice clean
measurements, check what is in the DESTDIR (or INSTALLROOT), and
review options for the optional dependencies and any testsuite.

 OK, for a few packages I will build them without being able to
confirm that they still work (e.g. mutt in the recent tagging), but
in general the absence of required dependencies is the least of the
issues - and anyway, sometimes the dependencies need to be upgraded.

 Then there is the question of dependencies - in BLFS we don't
normally tell people how to use optional deps.  Sometimes they are
picked up automatically, but many times you have to pass a switch to
get them used.  The instructions in BLFS are hopefully correct, but
they don't suit everyone.

 I'm also wary of standard workflows - my own LFS build is different
enough (nothing updated in /sources, because that is an nfs mount on
my systems, and with efforts to remove many of the static libraries)
that I expect pain.  And that's just for LFS.  For BLFS, I suspect
this sort of change will actually increase my workload and therefore
reduce my contributions.

 Rationale:
 
 (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a 
 huge undertaking - and it's a difficult beast to maintain. In order to 
 create a stable reference page in BLFS an editor has to have spent the 
 time to build all of LFS, tweak it, go through current BLFS, manually 
 install dependency packages and then carefully document the package he 
 builds. No two developers are guaranteed to have the same environment, 
 either, so accuracy and stability becomes an issue.
 
 Indeed.  For BLFS, I'm typically now building on both the last LFS
release, and also on a more recent svn version to make sure that
when I say it builds and works with 7.1 it does, and to detect if
any change in a newer LFS package has broken something along the way
(nothing recent, but I can remember pain in a package from a grep
update: something to do with manpages in a docbook package, I think,
which only bit me some time later because I hadn't been building
whichever package it was, and also problems caused by newer kernel
headers).

 The intention is good, but I'm not at all convinced that the plan
will help.

 Also, can I really trust whoever is permitted to edit a book (I'm
assuming that part of the aim is to get more people editing in BLFS
?) to have an uncompromised system, and to not want to upload
compromised binaries ?  For the xml in the book, and for patches, we
can look at what is being changed.  For a binary, how do we know
what was done to it ?  Distros have build machines with restricted
access and some degree of security.  Is LFS going to need signed
binaries and a ring of trust ?

 If I upload an unsigned binary package, the only way you can verify
it is by following the build instructions and seeing if you get the
same result.  I gave up 'farce' testing (seeing if binaries were the
same after an LFS system built itself) because there were too many
inexplicable differences, probably caused by randomisation of
addresses.  Posts in the last few months have suggested that this
problem is no longer present, but don't understimate the difficulties
of trying to see if my binary build matches yours using the same
instructions and the same versions of everything.

ĸen
-- 
das eine Mal als Tragödie, das andere Mal als Farce
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Baho Utot
On 05/20/2012 07:09 PM, Ken Moffat wrote:

[putolin]

   The more I think about this, the less happy I am.  Point 2 doesn't
 really help editing BLFS as far as I can see (upgrading a package
 usually needs several builds - typically, for me, a first to see if
 it actually works when I use it, then others to get nice clean
 measurements, check what is in the DESTDIR (or INSTALLROOT), and
 review options for the optional dependencies and any testsuite.

   OK, for a few packages I will build them without being able to
 confirm that they still work (e.g. mutt in the recent tagging), but
 in general the absence of required dependencies is the least of the
 issues - and anyway, sometimes the dependencies need to be upgraded.

   Then there is the question of dependencies - in BLFS we don't
 normally tell people how to use optional deps.  Sometimes they are
 picked up automatically, but many times you have to pass a switch to
 get them used.  The instructions in BLFS are hopefully correct, but

Just use the dependencies that are required that's in the book.  You as 
a user/builder can add any additional one you would like.

 they don't suit everyone.

   I'm also wary of standard workflows - my own LFS build is different
 enough (nothing updated in /sources, because that is an nfs mount on
 my systems, and with efforts to remove many of the static libraries)
 that I expect pain.  And that's just for LFS.  For BLFS, I suspect
 this sort of change will actually increase my workload and therefore
 reduce my contributions.

 Rationale:

 (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a
 huge undertaking - and it's a difficult beast to maintain. In order to
 create a stable reference page in BLFS an editor has to have spent the
 time to build all of LFS, tweak it, go through current BLFS, manually
 install dependency packages and then carefully document the package he
 builds. No two developers are guaranteed to have the same environment,
 either, so accuracy and stability becomes an issue.

   Indeed.  For BLFS, I'm typically now building on both the last LFS
 release, and also on a more recent svn version to make sure that
 when I say it builds and works with 7.1 it does, and to detect if
 any change in a newer LFS package has broken something along the way
 (nothing recent, but I can remember pain in a package from a grep
 update: something to do with manpages in a docbook package, I think,
 which only bit me some time later because I hadn't been building
 whichever package it was, and also problems caused by newer kernel
 headers).

pacman package management also has a set of tools to check the resulting 
package for corectness based upon LSB.  It will also detect permission 
problems and RPATH issues.


   The intention is good, but I'm not at all convinced that the plan
 will help.

   Also, can I really trust whoever is permitted to edit a book (I'm
 assuming that part of the aim is to get more people editing in BLFS
 ?) to have an uncompromised system, and to not want to upload
 compromised binaries ?  For the xml in the book, and for patches, we
 can look at what is being changed.  For a binary, how do we know
 what was done to it ?  Distros have build machines with restricted
 access and some degree of security.  Is LFS going to need signed
 binaries and a ring of trust ?

One doesn't need to fetch a binary, just the PKGBUILD file and then you 
can build it

   If I upload an unsigned binary package, the only way you can verify
 it is by following the build instructions and seeing if you get the
 same result.  I gave up 'farce' testing (seeing if binaries were the
 same after an LFS system built itself) because there were too many
 inexplicable differences, probably caused by randomisation of
 addresses.  Posts in the last few months have suggested that this
 problem is no longer present, but don't understimate the difficulties
 of trying to see if my binary build matches yours using the same
 instructions and the same versions of everything.

 ĸen

I have placed my build of LFS 6.8 using Arch linux pacman package 
manager onto github.

The URL: github.com/baho-utot/LFS-pacman

Have a look at it if you like.

I don't follow the recent comments at all.
Pacman packeage manager has some great features that makes rooling your 
own easier.
For example if I get the PKGBUILD for a package that is in BLFS from 
someone then I am not duplicating my effort and I have a mecanisim that 
gives me a package that will work.

Package management is also learned so why not include something in the book?

I see using pacman package management as a plus.


-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Jeremy Huntwork
On 5/20/12 5:34 PM, Bruce Dubbs wrote:
 OK, then what's wrong with a tarball of binaries that we have created
 for this purpose?  There could be a tarball of the base LFS system and
 then additional tarballs for certain packages or groups (e.g. xorg) of
 packages.

This method does not collect and keep package dependency. Using a 
manager like pacman, ff you have a repository with dependency baked in, 
when you go to build an individual tool you can automatically discover 
and download/install into your build environment all required packages - 
and only the required packages.

Also, for a reader who is serious about making a distro, creating a 
giant tarball of the LFS base does not teach any good practices for 
creating and deploying a distribution of their own.

Another benefit of a package manager is that changelog information can 
be related to individual packages and can be easier to track changes - 
again, this is less important in LFS, but could be useful in BLFS.

If it's simplicity that you like, one advantage of pacman is that the 
packages themselves are actually just tar archives containing the files 
and metadata.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-20 Thread Jeremy Huntwork
On 5/20/12 7:09 PM, Ken Moffat wrote:

[snip - a number of good, thoughtful questions]

I'm going to have to let your questions brew for a while before I can 
reply to them. Perhaps someone else will have an opinion regarding them 
in the meantime...

JH


-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread DJ Lucas
On 05/19/2012 08:26 AM, Jeremy Huntwork wrote:
 I've been holding back bringing this up on-list for a while because I
 intended to do the bulk of the work and then present a working system to
 the community for comment and review. I still intend to do that, but
 given some recent discussions, I think the time is right to bring this
 up and see where it goes.

 (Sorry for the cross posting, but since it involves both projects, I
 wanted to make sure all saw it - if possible, please reply on lfs-dev.)

 Proposal:

 1. Adjust LFS/BLFS to auto-generate build recipes for individual
 packages that a packaging tool can use to create binary packages with
 meta information included such as dependency tracking.

 2. Store 'official' copies of those binary packages in a developer
 package repository so that when developing (primarily BLFS) a dev can
 automatically pull and install into a build environment the dependencies
 he needs to build and create an individual package.

 3. Create a standard workflow and tools whereby a developer can
 accomplish #2 and edit the book accordingly in an efficient, reliable way.

 Rationale:

 (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a
 huge undertaking - and it's a difficult beast to maintain. In order to
 create a stable reference page in BLFS an editor has to have spent the
 time to build all of LFS, tweak it, go through current BLFS, manually
 install dependency packages and then carefully document the package he
 builds. No two developers are guaranteed to have the same environment,
 either, so accuracy and stability becomes an issue.

 The same is true of the LiveCD project, and is the main reason why it
 sits dead today. It is difficult to maintain when there are no packaged
 binaries to draw from and the entire thing is a scripted source build.

 Let me just say now that because (B)LFS is primarily (and probably
 should always be) about educating, I don't think we should require
 end-users to use package management. Mainly the proposal is intended to
 assist in development. However, if we have taken the time to incorporate
 PM in our dev workflow, we can make the option of building via PM tools
 available to readers if they wish to use it for themselves and build
 their own package repository.

 Details:

 (The following details assume pacman is the package manager chosen, but
 it could conceivably be another, such as rpm5.)

 1. The end of LFS chapter 5 is modified to (optionally) build the
 packaging tool and any dependencies it has.

 2. LFS chapter 6 is modified so that for each package a build recipe
 (e.g. PKGBUILD file) is generated from the book's source. A reader is
 given the option of carrying out the build manually or via the packaging
 tool.

 3. LFS devs create official copies of the binary packages and install
 them to a semi-public package repository

 4. BLFS is modified to also generate build recipes (PKGBUILD files) from
 its source

 5. As BLFS devs work on individual packages, they can use the defined
 workflow and tools to generate a chroot environment with only the
 packages they need for build-time dependencies - they can then both
 produce a binary version of the package as well as document what they've
 done, the end product being a BLFS page which will generate/correspond
 to the PKGBUILD file.

 6. BLFS dev updates the BLFS binary package repository with the
 'official' package and other devs can now draw from and use those when
 working on their respective package.

 There are, I'm sure, both positives and negatives to this proposal, and
 I'd like to hear everyone's thoughts.

 I intended to do all the development in the jh branch, but if there are
 more parties interested in helping this development, then I'm also open
 to sharing the workload and perhaps creating an environment where this
 can be done together.

 JH

I do believe this was consistent with Gerard's vision a few years back. 
I have (as I'm sure, most other long timers have) been using a 
home-grown system for some time. In my own case, that involves manually 
editing my scripts on every pass to avoid if-then constructs, and I have 
actually kept it that way so that I have no surprises. I do this because 
I am forced to reevaluate if a package build breaks and that forces me 
to keep in touch with the readers' experience, at least somewhat.

The immediate downside of your proposal for me, is that the optional 
dependencies must be accounted for in some cases by individual configure 
flags, which results in a logical if-then in the build description file, 
and sometimes even compound flags which result in nested if-then 
constructs. Fortunately, that is not a deal breaker for me if the 
readers get the same treatment (which seems to be the case), but this 
does hard code optional dependencies for the pre-packaged installations. 
This is both good and bad. From a development standpoint, it won't take 
me a week to build a fairly standard system to test 

Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Baho Utot
On 05/19/2012 09:26 AM, Jeremy Huntwork wrote:
 I've been holding back bringing this up on-list for a while because I
 intended to do the bulk of the work and then present a working system to
 the community for comment and review. I still intend to do that, but
 given some recent discussions, I think the time is right to bring this
 up and see where it goes.

 (Sorry for the cross posting, but since it involves both projects, I
 wanted to make sure all saw it - if possible, please reply on lfs-dev.)

 Proposal:

 1. Adjust LFS/BLFS to auto-generate build recipes for individual
 packages that a packaging tool can use to create binary packages with
 meta information included such as dependency tracking.

 2. Store 'official' copies of those binary packages in a developer
 package repository so that when developing (primarily BLFS) a dev can
 automatically pull and install into a build environment the dependencies
 he needs to build and create an individual package.

 3. Create a standard workflow and tools whereby a developer can
 accomplish #2 and edit the book accordingly in an efficient, reliable way.

 Rationale:

 (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a
 huge undertaking - and it's a difficult beast to maintain. In order to
 create a stable reference page in BLFS an editor has to have spent the
 time to build all of LFS, tweak it, go through current BLFS, manually
 install dependency packages and then carefully document the package he
 builds. No two developers are guaranteed to have the same environment,
 either, so accuracy and stability becomes an issue.

 The same is true of the LiveCD project, and is the main reason why it
 sits dead today. It is difficult to maintain when there are no packaged
 binaries to draw from and the entire thing is a scripted source build.

 Let me just say now that because (B)LFS is primarily (and probably
 should always be) about educating, I don't think we should require
 end-users to use package management. Mainly the proposal is intended to
 assist in development. However, if we have taken the time to incorporate
 PM in our dev workflow, we can make the option of building via PM tools
 available to readers if they wish to use it for themselves and build
 their own package repository.

 Details:

 (The following details assume pacman is the package manager chosen, but
 it could conceivably be another, such as rpm5.)

 1. The end of LFS chapter 5 is modified to (optionally) build the
 packaging tool and any dependencies it has.

 2. LFS chapter 6 is modified so that for each package a build recipe
 (e.g. PKGBUILD file) is generated from the book's source. A reader is
 given the option of carrying out the build manually or via the packaging
 tool.

 3. LFS devs create official copies of the binary packages and install
 them to a semi-public package repository

 4. BLFS is modified to also generate build recipes (PKGBUILD files) from
 its source

 5. As BLFS devs work on individual packages, they can use the defined
 workflow and tools to generate a chroot environment with only the
 packages they need for build-time dependencies - they can then both
 produce a binary version of the package as well as document what they've
 done, the end product being a BLFS page which will generate/correspond
 to the PKGBUILD file.

 6. BLFS dev updates the BLFS binary package repository with the
 'official' package and other devs can now draw from and use those when
 working on their respective package.

 There are, I'm sure, both positives and negatives to this proposal, and
 I'd like to hear everyone's thoughts.

 I intended to do all the development in the jh branch, but if there are
 more parties interested in helping this development, then I'm also open
 to sharing the workload and perhaps creating an environment where this
 can be done together.

 JH

I have in the past worked on LFS-6.8 and have a completed pacman build 
for it.  I wanted to build a desktop system from LFS/BLFS but it was too 
much work for me.   I have not gone further because BLFS is a beast as 
you say.  I completed a server using LFS/BLFS that handles mail web and 
news services.

Sharing the work using pacman would be great,  maybe we can exchange notes?


-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On 5/19/12 1:15 PM, DJ Lucas wrote:
 What separates LFS from say Arch, Gentoo, T2... at that point? No mile
 long USE flags or complex switching scripts I presume, but I know little
 about the other two. I've included some of their work in BLFS in the
 past, but that is about it.

Well, the way I saw this working out in my head - we don't really 
advertise the binary package repository, although it would be available 
for anyone to use. Hence, semi-public. The focus would still be on the 
book and letting a user choose her own path. The optional PM parts would 
serve as a reference for how to do something similar either with the 
same packaging tool, or one of the user's choice.

And yet, in the background, what I'll call the distro-y bits would be 
fully functional, driving consistency in the book(s) as well as accuracy 
since it should serve to help identify dependencies and installed 
resources. If anyone chose to actually use the created and shared 
binaries as a full-fledged distro, they absolutely should be able to, 
with the knowledge that we are not providing any guarantees as to 
usability, reliability, or support, etc.

 My hope is that build order is still a manual process where the user
 determines build order herself. Dependency checking is done only at
 build time and that optional deps remain optional. If there will be
 automation, how do we determine what optional deps to pull in?

I think that can still work. The main point of the distro-y bits is to 
make development easier, as well as provide actual documentation on how 
to make a binary package distro - not just a system built from source. 
There's a fair bit more to it than what LFS or the hints provide.

 What benefits can we expect for users who have already done the leg work
 to use other packaging methods?
 For instance, with what is put into the book, could a logical parser be
 made to genarate a spec file? How about dpkg?

If we did it correctly, I think this should be possible. It would 
definitely be nice to keep this as a goal. However we do it, it would be 
wise to make the spec generator modular enough to accept different 
output filters.

 While I admit that the .d/ directory reduces this concern immensely,
 what happens to configuration files that must be modified by additional
 installations? Do we keep diffs indefinitely, or do we create a comment
 marker for each package's additions, modify the package to support some
 sort of include directive? I can see scripting getting somewhat
 difficult here, not impossible, but difficult.

There's two features of most mature package managers that can help with 
this. First, config files can be identified in the spec so that when the 
version installed on the system is different than the one in the package 
being installed, a backup of one or the other is made, and the PM tells 
you what it did. So this helps preserve user modifications.

Second, for config files which we might reasonably expect to be modified 
by various installed system packages (e.g., /etc/passwd, /etc/group), 
those modifications can be done through pre- or post-install scripts, as 
well as pre- or post-uninstall scripts.

And yes, using .d directories where supported does help simplify things.

 If it is to be done, it should be designed with all goals in mind from
 the start. I have long suggested that LFS and BLFS move to installations
 from DESTDIR (or equivalent) as this does about 60% of building your src
 rpm without defining a particular package manager. The other 40% is just
 transposing what is in the book to spec, and could be automated, but
 even this has had lackluster support in the past despite the obvious
 educational benefits.

Absolutely. Looking forward to hearing more thoughts.

JH

-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On 5/19/12 1:21 PM, Baho Utot wrote:
 I have in the past worked on LFS-6.8 and have a completed pacman build
 for it.  I wanted to build a desktop system from LFS/BLFS but it was too
 much work for me.   I have not gone further because BLFS is a beast as
 you say.  I completed a server using LFS/BLFS that handles mail web and
 news services.

Nice!

 Sharing the work using pacman would be great,  maybe we can exchange notes?

Assuming this proposal gets some forward momentum, it's still not 100% 
decided that pacman is the package manager that would be used, although 
I would probably vote that direction.

But if it is pacman, by all means, we can exchange notes. :)

Thanks,

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On 5/19/12 1:15 PM, DJ Lucas wrote:
 My hope is that build order is still a manual process where the user
 determines build order herself. Dependency checking is done only at
 build time and that optional deps remain optional. If there will be
 automation, how do we determine what optional deps to pull in?

Sorry, I meant to answer this question better in the last reply.

I think I would aim to keep the BLFS book as close as it is today in 
terms of allowing a someone to 'choose your own path'. Where that gets 
interesting is how to fit that into a new workflow with shared binary 
packages. From what I can see, there's three options:

1. Make the policy to not build the official binaries with _any_ of the 
optional dependencies present.

2. Make the policy for to the dev to build with _all_ of the optional 
dependencies listed present.

3. Choose some 'sane' set of defaults to build against, but mark that 
certain ones are not strictly required.

This also gets into the realm of the difference between run-time and 
build-time dependencies, as well as the reasons for splitting some 
packages into sub packages - but I'll leave those for perhaps a little 
later when this topic has gotten a bit more circulation. :)

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Bruce Dubbs
DJ Lucas wrote:

 I have long suggested that LFS and BLFS move to installations 
 from DESTDIR (or equivalent) 

I do use DESTDIR to check the installation of most packages, but there 
are some where it just doesn't work.  Actually, any package that decides 
to do a chown or use the -g or -o options of install is a problem, but 
several just ignore DESTDIR completely.

   -- Bruce
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page



Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Bruce Dubbs
Jeremy Huntwork wrote:

 Well, the way I saw this working out in my head - we don't really 
 advertise the binary package repository, although it would be available 
 for anyone to use. Hence, semi-public. The focus would still be on the 
 book and letting a user choose her own path. The optional PM parts would 
 serve as a reference for how to do something similar either with the 
 same packaging tool, or one of the user's choice.

It's easy enough to create a tarball of binaries for a specific 
architecture (686, x86_64, etc) and extract that to an empty partition. 
  A rebuild of the kernel, setting up grub, and a script to handle some 
specific things (fstab, ip address, etc) would be easy enough.

But then, I don't think of that as LFS.

   -- Bruce

-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On May 19, 2012, at 5:04 PM, Bruce Dubbs bruce.du...@gmail.com wrote:

 It's easy to create a tarball of binaries for a specific
 architecture (686, x86_64, etc) and extract that to an empty partition.
  A rebuild of the kernel, setting up grub, and a script to handle some
 specific things (fstab, ip address, etc) would be easy enough.

 But then, I don't think of that as LFS.


And it misses nearly all of the goals I'm hoping to hit with the proposal.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On May 19, 2012, at 5:03 PM, Bruce Dubbs bruce.du...@gmail.com wrote:

 DJ Lucas wrote:

 I have long suggested that LFS and BLFS move to installations
 from DESTDIR (or equivalent)

 I do use DESTDIR to check the installation of most packages, but there
 are some where it just doesn't work.  Actually, any package that decides
 to do a chown or use the -g or -o options of install is a problem, but
 several just ignore DESTDIR completely.

Not necessarily a problem. There's various ways to address this,
depending on which PM you use. Typically you'd be building an
individual package on a chroot or fake root env where you have privs
to do this very type of thing.

JH
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread lfs-dev . neophyte_rep
On Sat, May 19, 2012 at 12:27 PM,  jhuntw...@lightcubesolutions.com wrote:
 On 5/19/12 1:21 PM, Baho Utot wrote:
 I have in the past worked on LFS-6.8 and have a completed pacman build
 for it.

 Nice!

 Sharing the work using pacman would be great,  maybe we can exchange notes?

 Assuming this proposal gets some forward momentum, it's still not 100%
 decided that pacman is the package manager that would be used, although
 I would probably vote that direction.

 But if it is pacman, by all means, we can exchange notes. :)


The pacman reference here is to https://wiki.archlinux.org/index.php/Pacman
not http://pacman.com/en/
Correct?

Just wanted to make that clear for those who read this posting later.

-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Bruce Dubbs
Jeremy Huntwork wrote:
 On May 19, 2012, at 5:03 PM, Bruce Dubbs bruce.du...@gmail.com wrote:
 
 DJ Lucas wrote:

 I have long suggested that LFS and BLFS move to installations
 from DESTDIR (or equivalent)
 I do use DESTDIR to check the installation of most packages, but there
 are some where it just doesn't work.  Actually, any package that decides
 to do a chown or use the -g or -o options of install is a problem, but
 several just ignore DESTDIR completely.
 
 Not necessarily a problem. There's various ways to address this,
 depending on which PM you use. Typically you'd be building an
 individual package on a chroot or fake root env where you have privs
 to do this very type of thing.

Certainly you have the privs, but I would like to have the capability of 
doing make DESTDIR=$DEST without being root.  I don't particularly like 
the developer saying you have to be root to run install.  It's my 
system, not the developer's.

   -- Bruce
-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page


Re: [lfs-dev] Once more: Package Management

2012-05-19 Thread Jeremy Huntwork
On 5/19/12 7:29 PM, Bruce Dubbs wrote:
 Certainly you have the privs, but I would like to have the capability of
 doing make DESTDIR=$DEST without being root.  I don't particularly like
 the developer saying you have to be root to run install.  It's my
 system, not the developer's.

I believe that pacman handles that with fakeroot, when not running 
makepkg as root.

JH

-- 
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page