Re: Farce differences [ was Re: Is 6.3 ready for release? ]
On Tue, Aug 07, 2007 at 11:41:22AM -0700, Dan Nicholson wrote: > > I applaud your efforts to debug these differences, but I see nothing. > Usually, the only way I've ever solved these things is at the source > level. E.g., stop the build at perl round 2 and try to figure out > what's different before generating the binaries. > Regarding perl, I think it was ok on my previous pair of builds. Certainly I now know that libc -sometimes- differs, but I still think that is either randomization or undiscovered builder error. Thanks for the suggestion, but I don't see me doing another two-pass build any time soon. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Farce differences [ was Re: Is 6.3 ready for release? ]
On 8/4/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > On Thu, Aug 02, 2007 at 11:34:23PM +0100, Ken Moffat wrote: > > > > Anybody who follows my ramblings on clfs-dev will know that using > > an earlier kernel than the headers, particularly an -rc kernel, is > > "NOT a Good Idea" (TM) and I'm lucky I wasn't using glibc-2.6. > > > > I'll try to do a fresh by-the-book pair of builds (without > > non-toolchain tests) if I've got time. > > > I rather wish I hadn't bothered! This was 2007-07-31 with > glibc-2.5.1. The following files differed: > > /lib/libc-2.5.1.so > /usr/bin/perl (also flagged as perl5.8.8 which is a hard link) > /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1 - generally expected > /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1plus - generally expected > /usr/lib/libstdc++.la - known difference > /usr/lib/libsupc++.la - known difference > /usr/share/doc/groff/1.18.1.4/meintro.ps impenetrable difference in numbers > /usr/share/doc/groff/1.18.1.4/meref.ps impenetrable difference in numbers > /usr/share/info/coreutils.info - known problem, second build used precompiled > info Using jhalfs and farce (no compiler options) I got good results: $ cat iteration-1_V_iteration-2/farce-differ /etc/blkid.tab.old /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1 /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1plus /usr/lib/libstdc++.la /usr/lib/libsupc++.la /usr/lib/locale/locale-archive /usr/share/info/coreutils.info For some reason, the TIME changed for all my volumes in blkid.tab.old. Don't know why. I'm also not sure about locale-archive, but I'm not worried about it and I recall Greg used to see this issue, too. > Of these, the libtool archives add, or rather repeat, directories > in the first inplace build - libtool is like that, not a significant > issue. cc1 and cc1plus seem to differ for everybody, they should > probably be treated as 'expected to differ'. Yeah, cc1 and cc1plus will differ unless you strip the objects before they're linked. I.e., LDFLAGS=-s or CFLAGS!=-g. The reason is that essentially a hashsum is taken of the objects to be linked into cc1. This is then compiled into cc1. If the objects contain debugging symbols (the default), the hashsum will be different between the first and second passes since the location of the crt files changes. This hashsum can't be stripped out later because it's part of the program data, not in a .debug section or something. > The postscript files are impenetrable, the differences are in long > strings of numbers - this is the first time I've run farce on this > version of groff and I suspect these numbers are calculated, and > there is a small amount of error. I'm not worried about this. I don't know why you get those and I don't. > That just leaves two somewhat important binaries. Looking at > -libc-2.5.1.so-0: file format elf32-i386 > +libc-2.5.1.so-1: file format elf32-i386 > -perl-0: file format elf32-i386 > +perl-1: file format elf32-i386 I applaud your efforts to debug these differences, but I see nothing. Usually, the only way I've ever solved these things is at the source level. E.g., stop the build at perl round 2 and try to figure out what's different before generating the binaries. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Ken Moffat wrote: > If you don't want me to change these figures, as the release > manager all you have to do is NAK them. I don't mind you changing the numbers, but I'd like to see more values to validate them. One machine and specific configuration is not enough to override what is in the book now. I'd like to see at least three sets of data to feel confident that the data is as consistent as possible. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Aug 06, 2007 at 06:33:55PM -0500, Bruce Dubbs wrote: > > Your values don't seem too much out of line. Your speeds seem to > generally (not always) run a bit faster and use more disk space. It > could be a difference in the way you measure, the amount of memory you > have, the amount of cache, the disk sector sizes, disk cache, disk > speed, etc. > I measure time in whole seconds (from $SECONDS) from configure to the end of install (so, not including untarring and removing, even though everybody has to do that). Space is measured by df -k with a grep for the rootfs (the build is logged, but to a different filesystem) - before untarring, and after the install. The builds were on an athlon64 'winchester' 2GHz using ondemand cpufreq - it has 1GiB of memory, but only around 900MiB available because I'm not using CONFIG_HIMEM. The disk is a common or garden SATA 7200rpm from 2 or 3 years ago, with "ordinary" cache size - I think the disk sector size is the same as everybody else's. My experience shows that the biggest influence on SBUs is the host - if you can find something with an old (fast) compiler (even gcc-3.3 probably counts as a fast compiler now), the initial SBU will be a lot less, and therefore everything else will use more SBUs. In this case, the host was LFS-svn from 2006-12-09 using a 2.6.22.1 kernel, with gcc-4.1.1. The SBU was 178 seconds. Filesystem ext3, using the defaults in mke2fs -j. If you don't want me to change these figures, as the release manager all you have to do is NAK them. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Dan Nicholson wrote:
> On Sat, Aug 04, 2007 at 04:04:17PM -0400, Bryan Kadzban wrote:
>> (if normal "read(0, ...);" works, why doesn't "test -r > device>"?)
>
> Why do you think read(0,...) works?
I assumed that the normal C-program method for reading standard input
(that is, fgets(..., stdin);) was equivalent to read(0, ...);, with the
addition of possible line buffering. Perhaps it isn't? Let me do some
testing...
OK, testing shows that calling read(0, ...); works, even when /dev/stdin
"test"s not readable. The program I used is attached. I su'ed to root,
then to another nonprivileged user, then ran "[ -r /dev/stdin ] ; echo
$?" and got 1. Then I ran this program, and it echo'ed my input, so its
read succeeded.
> I don't see anything in the tests that tries to read from stdin.
Right, it doesn't actually try to perform the read (if it did, then that
would be a better test of what I think it's trying to look for). It
just looks to see whether /dev/stdin's permissions mask (which it gets
from stat) allows reading. IMO that check is a poor one -- it should
just try to do the read, because there are cases where the test will
fail but reading is still possible.
> An alternative to using /dev/null is to use /dev/tty. This is what's
> done in the perl tests (not by us), but I guess we can't guarantee
> that'll be readable either.
Well, /dev/tty is "the process's controlling TTY", and its permissions
(AFAIK) should always be 0666. So that sounds like it might work better
too.
On your patch: Looks fine to me, unless we want to use /dev/tty.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGt8W+S5vET1Wea5wRAxLPAJ944U6ypnq/szhHbORXzwplyV10HwCdHNzy
n9AUy3lIebsKc2w3Hl3COVU=
=570e
-END PGP SIGNATURE-
#include
#include
int main()
{
ssize_t bytes;
char buf[1024];
bytes = read(0, buf, sizeof(buf));
if(bytes < 0) {
perror("read");
return 1;
}
if(bytes >= 1024) bytes = 1023;
buf[bytes] = 0;
printf("Success: %s", buf);
return 0;
}
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Ken Moffat wrote: > As a non-statistician, what does the standard deviation mean in > this context ? e.g. diffutils chapter 5 SBU Average 0.1 SBU Std > Dev 0.6. I could understand 0.1 plus or minus 0.1 (well, any bigger > number plus or minus, 0.1 minus anything is technically below the > minimum), but I can't make any sense of '0.6'. > > Those figures look somewhat old, too (the presence of MAKEDEV). > > Anyway, for 6.3 I suppose we should continue to provide some > figures. In on-technical terms, standard deviation is a measure of dispersion of the values. About 68% of all data will be within one standard deviation of the average. About 95% of all data will be within two standard deviations of the average. This assumes the data has "normal" variations. I used the standard deviation to just provide more information than just the arithmetic mean of the data input. This is not true if some LFSers run tests and others do not when collecting their data. On the other hand, if the standard deviations are relatively small, the user can be assured that the times are reasonably consistent. > 1. kernel headers - call it 304 MB and blame it on the new safer way > of installing. (reminder: chapter 5 too) > 2. glibc - I install _all_ locales, so I won't object that I'm using a > little more space, but I manage to do that in 14.3 SBU instead of > 19.5. Or maybe I'm missing the point about standard deviations ? > 3. gcc - for me, it takes 25.7 SBU not 22. > 4. db - 88MB rather than 77MB. > 5. e2fsprogs uses 46.9MB for me, not 31.2MB. > 6. coreutils takes 0.6 SBU, not 1.0 SBU - for consistency with item > 12 below, I think this is 70.7 or 71MB depending on how we are > currently rounding, otherwise I wouldn't get worked up about 1MB in > 70. > 7. perl takes 1.2 SBU for me rather than 1.5, which is just about > enough of a difference to notice. > 8. for me, bzip2 needs 6.4MB not 5.3MB - maybe somebody didn't > count the docs ? > 9. gettext for me took 1.3 SBU not 1.0 (again, just about enough to > notice the difference), and more importantly it needed 83.5MB > instead of 65MB. > 10. I'll treat my space difference on gzip as splitting hairs, > unless pressed (3.3MB for 2.2MB) > 11. inetutils for me needed 12.1 MB, not 8.9MB, and if I'm going to > change the figure it used 0.3 SBU for me, not 0.2. > 12. shadow took 0.5 SBU for me, not 0.3SBU, and if I'm going to > change it I think the space is 21.6MB (or 22MB if rounded?) Your values don't seem too much out of line. Your speeds seem to generally (not always) run a bit faster and use more disk space. It could be a difference in the way you measure, the amount of memory you have, the amount of cache, the disk sector sizes, disk cache, disk speed, etc. > For 7.0, I'll be happy to round future changes to 0.5 SBU, and go > with x86 space usage (nearest 1MB/10MB, or nearest 100MB for gcc and > glibc ?), but I don't think this thread is the right place to discuss > that. Perhaps we could develop a way to populate the SBU database with data extracted from jhalfs logs. I haven't looked into that though. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Wed, Aug 01, 2007 at 01:22:01PM -0500, Bruce Dubbs wrote: > Jeremy Huntwork wrote: > > M.Canales.es wrote: > >> If we are happy with big figures, thus use the same values for all archs, > >> If > >> we want something accurate for each arch, remove the info from the book a > >> create a web page to place jhalfs reports. > > > > I like this option. Perhaps provide *very* rough estimates for the SBU > > (round to the nearest 1/2 SBU or so, based probably on x86) in the book > > and a store of SBU reports elsewhere on the web. > > http://www.linuxfromscratch.org/~sbu/ > > -- Bruce As a non-statistician, what does the standard deviation mean in this context ? e.g. diffutils chapter 5 SBU Average 0.1 SBU Std Dev 0.6. I could understand 0.1 plus or minus 0.1 (well, any bigger number plus or minus, 0.1 minus anything is technically below the minimum), but I can't make any sense of '0.6'. Those figures look somewhat old, too (the presence of MAKEDEV). Anyway, for 6.3 I suppose we should continue to provide some figures. To my surprise, my chapter 5 script runs tests wherever possible (well, I'm sure that was a good idea at one time), so I'm not suggesting any changes there other than for the kernel headers. For chapter 6, my figures diverge from the book's figures on the following (this is only running tests on the toolchain - I'm ignoring my results on module-init-tools where I also run tests). 1. kernel headers - call it 304 MB and blame it on the new safer way of installing. (reminder: chapter 5 too) 2. glibc - I install _all_ locales, so I won't object that I'm using a little more space, but I manage to do that in 14.3 SBU instead of 19.5. Or maybe I'm missing the point about standard deviations ? 3. gcc - for me, it takes 25.7 SBU not 22. 4. db - 88MB rather than 77MB. 5. e2fsprogs uses 46.9MB for me, not 31.2MB. 6. coreutils takes 0.6 SBU, not 1.0 SBU - for consistency with item 12 below, I think this is 70.7 or 71MB depending on how we are currently rounding, otherwise I wouldn't get worked up about 1MB in 70. 7. perl takes 1.2 SBU for me rather than 1.5, which is just about enough of a difference to notice. 8. for me, bzip2 needs 6.4MB not 5.3MB - maybe somebody didn't count the docs ? 9. gettext for me took 1.3 SBU not 1.0 (again, just about enough to notice the difference), and more importantly it needed 83.5MB instead of 65MB. 10. I'll treat my space difference on gzip as splitting hairs, unless pressed (3.3MB for 2.2MB) 11. inetutils for me needed 12.1 MB, not 8.9MB, and if I'm going to change the figure it used 0.3 SBU for me, not 0.2. 12. shadow took 0.5 SBU for me, not 0.3SBU, and if I'm going to change it I think the space is 21.6MB (or 22MB if rounded?) So, apart from item 11, I intend to change these unless anybody cares to contest my figures. I'll wait at least 24 hours. For 7.0, I'll be happy to round future changes to 0.5 SBU, and go with x86 space usage (nearest 1MB/10MB, or nearest 100MB for gcc and glibc ?), but I don't think this thread is the right place to discuss that. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Aug 06, 2007 at 02:18:08PM -0700, Dan Nicholson wrote:
> On 8/6/07, Ken Moffat <[EMAIL PROTECTED]> wrote:
> > >
> > OK, but that doesn't explain why it fails for me :)
>
> Nope. But the second part I wrote might help. For example:
>
> make -j1 test &> test.log
> for t in src/testdir/*.failed; do
> echo ${t%.failed} failed
> cat -t $t
> done >> test.log
>
Maybe I'm being even more dense than usual, but isn't that only
going to tell me which test failed ? I already got a message saying
'test3 FAILED' just before 'ALL DONE'. That is the test to see if
cindent works, but I can't begin to find any output from it in the
log - there are a lot of lines containing only 'OK^M^M', but I can't
see any 'FAIL' except for 'FAILED' in the text of the commands and at
the end. OTOH, If I search for 'cindent' I can see the following
impenetrable chunk -
rm -rf X* test.ok viminfo
rm -rf test3.failed test.ok test.out X* viminfo
cp test3.ok test.ok
# Sleep a moment to avoid that the xterm title is messed up
../vim -u unix.vim -U NONE --noplugin -s dotest.in test3.in
Vim: Warning: Output is not to a terminal
^[[?1h^[=^[[1;40r^[[m^[[H^[[2J^[[40;1H"test3.in" 1320 lines, 13734
characters^[[1;1H/* vim: set cin ts=4 sw=4 : */^M
^M
Test for 'cindent'^M
^M
STARTTEST^M
:so small.vim^M
:set nocompatible viminfo+=nviminfo^M
:edit^[[16C" read modeline^M
/start of AUTO^M
=/end of AUTO^M
ENDTEST^M
^M
/* start of AUTO matically checked vim: set ts=4 : */^M
{^[[15;9Hif (test)^[[16;17Hcmd1;^[[17;9Hcmd2;^M
}^M
^M
{^[[21;9Hif (test)^[[22;17Hcmd1;^[[23;9Helse^[[24;17Hcmd2;^M
}^M
^M
{^[[28;9Hif (test)^[[29;9H{^[[30;17Hcmd1;^[[31;17Hcmd2;^[[32;9H}^M
}^M
^M
{^[[36;9Hif
(test)^[[37;9H{^[[38;17Hcmd1;^[[39;17Helse^[[1;1H^[[40;1H^[[K^[[40;1H:set
cp^M^[[1;1H^[[40;1H^[[K^[[40;1H:map dotest
/^STARTTEST^^H^[[1m^M^[[mj:set ff=unix
cpo-=A^^H^[[1m^M^[[m:.,/ENDTEST/-1w! Xdotest^^H^[[1m^M^[[m:set ff&
cpo+=A^^H^[[1m^M^[[mnj0:so! X^M^M
^[[39;100Hd^[[40;1Hotest^^H^[[1m^M^[[mdotest^M^[[1;1H^[[L^[[1;1H/*
vim: set cin ts=4 sw=4 :
*/^[[40;1H^[[K^[[1;1H^[[40;1H/^STARTTEST^M^[[5;1H^M
^[[40;1H^[[K^[[40;1H:set ff=unix
cpo-=A^M^[[6;1H^[[40;1H^[[K^[[40;1H:.,/ENDTEST/-1w!
Xdotest^M"Xdotest" ^[[40;11H^[[K^[[40;11H[New File] 5 lines, 116
characters written^[[6;1H^[[40;1H^[[K^[[40;1H:set ff&
cpo+=A^M^[[6;1H^[[40;1H/ENDTEST^[[40;10H^[[K^[[40;1H^[[11;1H^M
^[[40;1H^[[K^[[40;1H:so! Xdotest^M^[[12;1H^[[40;1H^[[K^[[40;1H:so
small.vim^M^[[12;1H^[[40;1H^[[K^[[40;1H:set nocompatible
viminfo+=nviminfo^M^[[12;1H^[[40;1H^[[K^[[40;1H:edit
" read modeline^M"test3.in"^[[40;22H^[[K^[[40;12H1320L,
13734C^[[12;1H^[[40;1H^[[K^[[40;1H/start of
AUTO^M^[[13;4H^[[40;1H^[[K^[[40;1H/end of AUTO^M789 lines to
indent...^M750^H^H0^M65^H0^M55^H0^M45^H0^M35^H0^M25^H0^M15^H0^M50
lines to indent... ^M790 lines indented
^[[40;20H^[[K^[[13;1H^[[40;1H^[[K^[[40;1H/^STARTTEST^M^[[m^[[H^[[2J^[[1;27H}^[[2;9H}^M
}^M
^M
main()^M
{^[[7;9H(void) MyFancyFuasdfadsfnction(^[[8;25Hargument);^M
}^M
^M
main()^M
{^[[13;9Hcharfoo[] = "/*";^[[14;9H/* as^[[15;12Hdf
*/^[[16;9Hhello^M
}^M
/* end of AUTO */^M
^M
STARTTEST^M
:set tw=0 wm=60 columns=80 noai fo=croq^M
/serious/e^M
a about life, the universe, and the rest^[[1m^[^[[m^M
ENDTEST^M
(ok ,that last part _must_ be the start of the next test).
Maybe it fails because output is not to a terminal, but I don't
remember seeing it before July.
ĸen
--
das eine Mal als Tragödie, das andere Mal als Farce
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/6/07, Ken Moffat <[EMAIL PROTECTED]> wrote:
> On Mon, Aug 06, 2007 at 02:04:48PM -0700, Dan Nicholson wrote:
> > On 8/6/07, Ken Moffat <[EMAIL PROTECTED]> wrote:
> > > On Mon, Aug 06, 2007 at 12:06:55PM -0700, Dan Nicholson wrote:
> > > >
> > > > A couple things I figured out about the vim tests. First, the test
> > > > hang was due to bad make dependencies when running in parallel.
> > >
> > > The package itself is set up to run in parallel ? I don't see
> > > that.
> >
> > No, I run with MAKEFLAGS=-j3. This breaks when Makefiles don't have
> > well constructed dependencies. In the case of vim's tests, it seems
> > that only one test can be active at a time or they may hang.
> >
> OK, but that doesn't explain why it fails for me :)
Nope. But the second part I wrote might help. For example:
make -j1 test &> test.log
for t in src/testdir/*.failed; do
echo ${t%.failed} failed
cat -t $t
done >> test.log
--
Dan
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Aug 06, 2007 at 02:04:48PM -0700, Dan Nicholson wrote: > On 8/6/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > On Mon, Aug 06, 2007 at 12:06:55PM -0700, Dan Nicholson wrote: > > > > > > A couple things I figured out about the vim tests. First, the test > > > hang was due to bad make dependencies when running in parallel. > > > > The package itself is set up to run in parallel ? I don't see > > that. > > No, I run with MAKEFLAGS=-j3. This breaks when Makefiles don't have > well constructed dependencies. In the case of vim's tests, it seems > that only one test can be active at a time or they may hang. > OK, but that doesn't explain why it fails for me :) ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/6/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > On Mon, Aug 06, 2007 at 12:06:55PM -0700, Dan Nicholson wrote: > > > > A couple things I figured out about the vim tests. First, the test > > hang was due to bad make dependencies when running in parallel. > > The package itself is set up to run in parallel ? I don't see > that. No, I run with MAKEFLAGS=-j3. This breaks when Makefiles don't have well constructed dependencies. In the case of vim's tests, it seems that only one test can be active at a time or they may hang. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Aug 06, 2007 at 12:06:55PM -0700, Dan Nicholson wrote: > > A couple things I figured out about the vim tests. First, the test > hang was due to bad make dependencies when running in parallel. The package itself is set up to run in parallel ? I don't see that. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/1/07, Dan Nicholson <[EMAIL PROTECTED]> wrote: > On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > > And the vim test failure is totally impenetrable. > > One of the vim tests hung on me, and trying to decipher the output > only resulted in garbage in the terminal. I think I might just stop > running this thing. A couple things I figured out about the vim tests. First, the test hang was due to bad make dependencies when running in parallel. Second, failures will be indicated (I think) by presence of a test*.failed files. Basically, I think this could be used for success/failure: ls src/testdir/*.failed 2>/dev/null && echo failed -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Sat, Aug 04, 2007 at 04:04:17PM -0400, Bryan Kadzban wrote: > Ken Moffat wrote: > > But how do we explain it in the text ? "This test won't work when > > run like this, so we make it succeed by doing nothing useful" might > > raise a lot of questions about the point of testsuites in the minds > > of some of our readers ;) > > True -- how about: > > One of the tests requires the ability to read the device file behind > stdin. By default this file can't be read in the current environment, > since it's owned by another user. Make the test use an alternate device > for its stdin, which is always readable: Below is a patch that gives the most minimal explanation I can think of. If you guys feel like fleshing out the explanation, my hat goes off to you. I understand where you're coming from, but it could get a little long winded to explain that the tests are not being run as the developer expected and need to be worked-around. > That wording should be a bit less questionable. Unfortunately there's > still an issue with why is the test trying to read the device when it > isn't always readable (if normal "read(0, ...);" works, why doesn't > "test -r "?), but maybe we can avoid that question. Why do you think read(0,...) works? I don't see anything in the tests that tries to read from stdin. An alternative to using /dev/null is to use /dev/tty. This is what's done in the perl tests (not by us), but I guess we can't guarantee that'll be readable either. -- Dan diff --git a/BOOK/chapter01/changelog.xml b/BOOK/chapter01/changelog.xml index 0abd7d1..4416e3e 100644 --- a/BOOK/chapter01/changelog.xml +++ b/BOOK/chapter01/changelog.xml @@ -40,6 +40,11 @@ 2007-08-06 + [dnicholson] - Redirected /dev/null to + stdin when running the Bash testsuite to prevent errors with + terminal permissions. + + [dnicholson] - Fixed a typo and clarified text on the Perl page. Reported by Shawn. diff --git a/BOOK/chapter06/bash.xml b/BOOK/chapter06/bash.xml index ba08418..767993a 100644 --- a/BOOK/chapter06/bash.xml +++ b/BOOK/chapter06/bash.xml @@ -81,9 +81,10 @@ sed -i "s|htmldir = @htmldir@|htmldir = /usr/share/doc/bash-&bash-version;|" \ chown -Rv nobody ./ Now, run the tests as the nobody user: +class="username">nobody user, ensuring that the standard +input device is readable: -su-tools nobody -s /bin/bash -c "make tests" +su-tools nobody -s /bin/bash -c "make tests"
Re: Is 6.3 ready for release?
Hello all, Ok got the latest 6.3 build done with tests: Nothing out of the normal and tar passed all tests. Here are some of my results. I won't post those that past all tests, with normal skipped tests. === binutils Summary === # of expected passes35 === ld Summary === # of expected passes272 # of expected failures 4 === gas Summary === # of expected passes149 ../gcc-4.1.2/contrib/test_summary cat <<'EOF' | LAST_UPDATED: Obtained from SVN: tags/gcc_4_1_2_release revision 121944 Native configuration is i686-pc-linux-gnu === g++ tests === Running target unix XPASS: g++.dg/tree-ssa/pr14814.C scan-tree-dump-times &this 0 XPASS: g++.old-deja/g++.other/init5.C execution test === g++ Summary === # of expected passes12408 # of unexpected successes 2 # of expected failures 66 # of unsupported tests 69 /sources/gcc-build/gcc/testsuite/g++/../../g++ version 4.1.2 === gcc tests === Running target unix XPASS: gcc.dg/cpp/cmdlne-dI-M.c scan-file (^|n)cmdlne-dI-M.*:[^\\\ \n]*cmdlne-dI-M.c XPASS: gcc.dg/cpp/cmdlne-dM-M.c scan-file (^|n)cmdlne-dM-M[^n] *:[^n]*cmdlne-dM-M.c === gcc Summary === # of expected passes38985 # of unexpected successes 2 # of expected failures 99 # of untested testcases 28 # of unsupported tests 246 /sources/gcc-build/gcc/xgcc version 4.1.2 === libmudflap tests === Running target unix === libmudflap Summary === # of expected passes1799 === libstdc++ tests === Running target unix XPASS: 26_numerics/cmath/c99_classification_macros_c.cc (test for excess errors) === libstdc++ Summary === # of expected passes3398 # of unexpected successes 1 # of expected failures 12 # of unsupported tests 324 Compiler version: 4.1.2 Platform: i686-pc-linux-gnu configure flags: --prefix=/usr --libexecdir=/usr/lib --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu -- enable-languages=c,c++ EOF Mail -s "Results for 4.1.2 testsuite on i686-pc-linux-gnu" gcc- [EMAIL PROTECTED] && mv /sources/gcc-build/./gcc/testsuite/g++/g++.sum /sources/gcc- build/./gcc/testsuite/g++/g++.sum.sent && mv /sources/gcc-build/./gcc/testsuite/gcc/gcc.sum /sources/gcc- build/./gcc/testsuite/gcc/gcc.sum.sent && mv /sources/gcc-build/./i686-pc-linux-gnu/libmudflap/testsuite/ libmudflap.sum /sources/gcc-build/./i686-pc-linux-gnu/libmudflap/ testsuite/libmudflap.sum.sent && mv /sources/gcc-build/./i686-pc-linux-gnu/libstdc++-v3/testsuite/ libstdc++.sum /sources/gcc-build/./i686-pc-linux-gnu/libstdc++-v3/ testsuite/libstdc++.sum.sent && mv /sources/gcc-build/./gcc/testsuite/g++/g++.log /sources/gcc- build/./gcc/testsuite/g++/g++.log.sent && mv /sources/gcc-build/./gcc/testsuite/gcc/gcc.log /sources/gcc- build/./gcc/testsuite/gcc/gcc.log.sent && mv /sources/gcc-build/./i686-pc-linux-gnu/libmudflap/testsuite/ libmudflap.log /sources/gcc-build/./i686-pc-linux-gnu/libmudflap/ testsuite/libmudflap.log.sent && mv /sources/gcc-build/./i686-pc-linux-gnu/libstdc++-v3/testsuite/ libstdc++.log /sources/gcc-build/./i686-pc-linux-gnu/libstdc++-v3/ testsuite/libstdc++.log.sent && true Chap 6 tests: glibc-2.5.1 make[2]: [/sources/glibc-build/posix/annexc.out] Error 1 (ignored) make[2]: *** [/sources/glibc-build/nptl/tst-cancel1.out] Error 1 ## ## ## GNU tar 1.18 test suite. ## ## ## 1: tar version ok 2: decompressing from stdin ok 3: mixing optionsok 4: interspersed options ok 5: files-from: empty entries ok 6: files-from: 0-separated file without -0 ok 7: tar --index-file=FILE --file=-ok 8: tar cvf - ok 9: appendok 10: appending files with long names ok 11: append vs. create ok 12: exclude ok 13: deleting a member after a big one ok 14: deleting a member from stdin archive ok 15: deleting members with long names ok 16: deleting a large last member ok 17: deleting non-existing member ok 18: extract over an existing directoryok 19: extracting symlinks over an existing file ok 20: extraction loops ok 21: extract + fnmatch ok 22: extracting selected members from pax ok 23: mode of extracted directories ok 24: extracting symlinks to a read-only dir
Farce differences [ was Re: Is 6.3 ready for release? ]
On Thu, Aug 02, 2007 at 11:34:23PM +0100, Ken Moffat wrote: > > Anybody who follows my ramblings on clfs-dev will know that using > an earlier kernel than the headers, particularly an -rc kernel, is > "NOT a Good Idea" (TM) and I'm lucky I wasn't using glibc-2.6. > > I'll try to do a fresh by-the-book pair of builds (without > non-toolchain tests) if I've got time. > I rather wish I hadn't bothered! This was 2007-07-31 with glibc-2.5.1. The following files differed: /lib/libc-2.5.1.so /usr/bin/perl (also flagged as perl5.8.8 which is a hard link) /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1 - generally expected /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/cc1plus - generally expected /usr/lib/libstdc++.la - known difference /usr/lib/libsupc++.la - known difference /usr/share/doc/groff/1.18.1.4/meintro.ps impenetrable difference in numbers /usr/share/doc/groff/1.18.1.4/meref.ps impenetrable difference in numbers /usr/share/info/coreutils.info - known problem, second build used precompiled info Of these, the libtool archives add, or rather repeat, directories in the first inplace build - libtool is like that, not a significant issue. cc1 and cc1plus seem to differ for everybody, they should probably be treated as 'expected to differ'. The postscript files are impenetrable, the differences are in long strings of numbers - this is the first time I've run farce on this version of groff and I suspect these numbers are calculated, and there is a small amount of error. I'm not worried about this. That just leaves two somewhat important binaries. Looking at 'farce-extras' didn't show anything recognisable, so I can't get away with changing one of the regexes to allow a different date/time/version string. Time for strip --strip-all, but that just seemed to show binary differences and strings moving slightly, as on my previous build (UTF-8, for those wondering why this version of groff is new to me, and unreliable because one kernel was unreliable). This time, I made sure I was using 2.6.22.1 for both builds. On to objdump -d followed by diff -u. For libc-2.5.1.so there are various changes in addresses, enough to make me wonder if address randomization is playing a part here: -libc-2.5.1.so-0: file format elf32-i386 +libc-2.5.1.so-1: file format elf32-i386 Disassembly of section .plt: @@ -310,7 +310,7 @@ 158c6: 8d 76 00lea0x0(%esi),%esi 158c9: 8d bc 27 00 00 00 00lea0x0(%edi),%edi 158d0: 55 push %ebp - 158d1: b8 e2 02 00 00 mov$0x2e2,%eax + 158d1: b8 de 02 00 00 mov$0x2de,%eax 158d6: 89 e5 mov%esp,%ebp 158d8: 53 push %ebx 158d9: e8 42 fd ff ff call 15620 <[EMAIL PROTECTED]> @@ -482,7 +482,7 @@ 15adc: 5d pop%ebp 15add: c3 ret 15ade: 89 f6 mov%esi,%esi - 15ae0: 8d 83 cc 9a fe ff lea0xfffe9acc(%ebx),%eax + 15ae0: 8d 83 ac 9a fe ff lea0xfffe9aac(%ebx),%eax 15ae6: 89 04 24mov%eax,(%esp) 15ae9: e8 82 89 04 00 call 5e470 <__libc_fatal> 15aee: 89 f6 mov%esi,%esi @@ -1157,7 +1157,7 @@ 1621f: 89 44 24 08 mov%eax,0x8(%esp) 16223: 8d 83 77 5f fe ff lea0xfffe5f77(%ebx),%eax 16229: 89 44 24 04 mov%eax,0x4(%esp) - 1622d: 8d 83 08 9b fe ff lea0xfffe9b08(%ebx),%eax + 1622d: 8d 83 e8 9a fe ff lea0xfffe9ae8(%ebx),%eax 16233: 89 04 24mov%eax,(%esp) 16236: e8 85 bb 00 00 call 21dc0 <__assert_fail> 1623b: 8b 83 54 ff ff ff mov0xff54(%ebx),%eax @@ -1985,7 +1985,7 @@ 16c59: 89 44 24 08 mov%eax,0x8(%esp) 16c5d: 8d 83 8c 5f fe ff lea0xfffe5f8c(%ebx),%eax 16c63: 89 44 24 04 mov%eax,0x4(%esp) - 16c67: 8d 83 2c 9b fe ff lea0xfffe9b2c(%ebx),%eax + 16c67: 8d 83 0c 9b fe ff lea0xfffe9b0c(%ebx),%eax (stopped there, possibly there are other types of changes, but I'm already out of my depth). Perl is much nastier - the file sizes reported by 'ls -l' differ even after 'strip --strip-all' : I know 'ls -l' is not totally reliable for this, but the disassembly seems to indicate it has been built differently: --- objdump-d-perl-02007-08-04 23:23:45.0 +0100 +++ objdump-d-perl-12007-08-04 23:23:52.0 +0100 @@ -1,5 +1,5 @@ -perl-0: file format elf32-i386 +perl-1: file format elf32-i386 Disassembly of section .init: @@ -7,9 +7,9 @@ 805d804: 55 push %ebp 805d805: 89 e5 mov%esp,%ebp 805d807: 83 ec 08sub$0x8,%esp - 805d80a: e8 25
Re: Is 6.3 ready for release?
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Ken Moffat wrote: > But how do we explain it in the text ? "This test won't work when > run like this, so we make it succeed by doing nothing useful" might > raise a lot of questions about the point of testsuites in the minds > of some of our readers ;) True -- how about: One of the tests requires the ability to read the device file behind stdin. By default this file can't be read in the current environment, since it's owned by another user. Make the test use an alternate device for its stdin, which is always readable: That wording should be a bit less questionable. Unfortunately there's still an issue with why is the test trying to read the device when it isn't always readable (if normal "read(0, ...);" works, why doesn't "test -r "?), but maybe we can avoid that question. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGtNvAS5vET1Wea5wRAys5AKDbvJonZEA0HJXvK9oF0OM7cUu8WgCgxYAy w8JEONhMegkY09OXS7UBybg= =PrQw -END PGP SIGNATURE- -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Sat, Aug 04, 2007 at 09:34:15AM -0700, Dan Nicholson wrote: > > I believe I found a way to "fix" the error which doesn't require > changing the permissions of the host's pseudo-terms: tie stdin to > /dev/null when running the tests. Basically, this prevents the stdin > test from doing anything useful. If you look closely, though, the > stdout and stderr tests aren't doing anything either, though, because > they've been tied to a log file. So, I say we just go all the way and > make the terminal tests useless (especially since we're running them > through su, which is probably not the intended way). > > su-tools nobody -s /bin/bash -c "make tests" That feels much safer than using chown on /dev/stdin. But how do we explain it in the text ? "This test won't work when run like this, so we make it succeed by doing nothing useful" might raise a lot of questions about the point of testsuites in the minds of some of our readers ;) ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Aug 4, 2007, at 10:54 AM, Dan Nicholson wrote: > On 8/4/07, William Harrington <[EMAIL PROTECTED]> wrote: >> >> I'm going to do a 6.3-rc1 build today and see what the tests >> output for me on my p4 2.8 northwood system using an intel D845PESV >> and LFS 6.1 and a 2.6.22.1 kernel. > > FYI, there are a few post-rc1 commits here: > > http://www.linuxfromscratch.org/lfs/view/6.3-branch/ > > The only real build change is glibc-2.5.1. Thanks Dan, I'll build with the glibc and lfs-bootscripts that were in the ChangeLog. Sincerely, William -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/4/07, Bryan Kadzban <[EMAIL PROTECTED]> wrote: > (Replying via mutt since the PSU in my main machine (the one that has > Thunderbird installed) died last night: the RMA is in progress, but > it'll be a few days...) > > On Sat, Aug 04, 2007 at 04:42:51PM +0100, Ken Moffat wrote: > > > so the nobody user won't be able to read these devices. Not sure how > > > you would work around that, unless you use login instead of su to > > > start the nobody user doing the testing (which will change ownership > > > of /dev/pts/x and hence the tests will pass) > > > > > A little bit of testing (after building to the end of chapter 6 > > earlier, I've gone back into chroot to play with this). It looks as > > if chown /dev/stdin *might* work (I'm on an xterm): > > > > root in chroot /# chown nobody /dev/stdin > > root in chroot /# su-tools nobody -s /bin/bash > > bash: /dev/null/.bashrc: Not a directory > > nobody in chroot /$ ls -l /dev/stdin > > lrwxrwxrwx 1 root root 15 Aug 4 15:51 /dev/stdin -> /proc/self/fd/0 > > nobody in chroot /$ ls -l /dev/pts > > total 0 > > crw--w 1 kentty 136, 0 Aug 4 16:22 0 > > crw--w 1 kentty 136, 1 Aug 4 16:01 1 > > crw--w 1 kentty 136, 2 Aug 4 16:30 2 > > crw--w 1 nobody tty 136, 3 Aug 4 16:32 3 > > crw--w 1 kentty 136, 4 Aug 4 16:30 4 > > nobody in chroot /$ test -r /dev/stdin ; echo $? > > 0 > > nobody in chroot /$ > > > > This seems too good to be true. We are running as root, so I guess > > we can happily continue to read and write to this pts dev after the > > tests are finished. If nobody pokes a hole in this or beats me to it, > > I'll start another build, but probably not before tomorrow. > > Seems like it should work to me. There is one thing we might want to be > careful of: We may not want to allow some random host user to access the > pseudo-term device after the tests are done. However, this is a > separate devpts mount from the host's /dev/pts, so I'm not sure if the > devices are accessible from the host. I believe I found a way to "fix" the error which doesn't require changing the permissions of the host's pseudo-terms: tie stdin to /dev/null when running the tests. Basically, this prevents the stdin test from doing anything useful. If you look closely, though, the stdout and stderr tests aren't doing anything either, though, because they've been tied to a log file. So, I say we just go all the way and make the terminal tests useless (especially since we're running them through su, which is probably not the intended way). su-tools nobody -s /bin/bash -c "make tests" 0 -158c158 -< 1 -> 0 run-tilde run-tilde2 run-trap -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
(Replying via mutt since the PSU in my main machine (the one that has Thunderbird installed) died last night: the RMA is in progress, but it'll be a few days...) On Sat, Aug 04, 2007 at 04:42:51PM +0100, Ken Moffat wrote: > > so the nobody user won't be able to read these devices. Not sure how > > you would work around that, unless you use login instead of su to > > start the nobody user doing the testing (which will change ownership > > of /dev/pts/x and hence the tests will pass) > > > A little bit of testing (after building to the end of chapter 6 > earlier, I've gone back into chroot to play with this). It looks as > if chown /dev/stdin *might* work (I'm on an xterm): > > root in chroot /# chown nobody /dev/stdin > root in chroot /# su-tools nobody -s /bin/bash > bash: /dev/null/.bashrc: Not a directory > nobody in chroot /$ ls -l /dev/stdin > lrwxrwxrwx 1 root root 15 Aug 4 15:51 /dev/stdin -> /proc/self/fd/0 > nobody in chroot /$ ls -l /dev/pts > total 0 > crw--w 1 kentty 136, 0 Aug 4 16:22 0 > crw--w 1 kentty 136, 1 Aug 4 16:01 1 > crw--w 1 kentty 136, 2 Aug 4 16:30 2 > crw--w 1 nobody tty 136, 3 Aug 4 16:32 3 > crw--w 1 kentty 136, 4 Aug 4 16:30 4 > nobody in chroot /$ test -r /dev/stdin ; echo $? > 0 > nobody in chroot /$ > > This seems too good to be true. We are running as root, so I guess > we can happily continue to read and write to this pts dev after the > tests are finished. If nobody pokes a hole in this or beats me to it, > I'll start another build, but probably not before tomorrow. Seems like it should work to me. There is one thing we might want to be careful of: We may not want to allow some random host user to access the pseudo-term device after the tests are done. However, this is a separate devpts mount from the host's /dev/pts, so I'm not sure if the devices are accessible from the host. They shouldn't be available directly, but if the same device major/minor numbers show up in a file in the host's /dev/pts directory, *and* if the chown affects both, then they may be. But I haven't tried it (and don't have a system available to do so either, see above...). Er, actually, depending on the read/execute permissions on the various directories leading up to $LFS, the random user on the host may be able to open the $LFS/dev/pts/X file directly. That wouldn't be good... (OTOH, I'm not sure how much damage may be done by allowing an untrusted user to read and write your TTY device, either. I'm assuming they can get the input that you're sending it, and I'm assuming they can print to it, but I'm not sure they can read the device's contents. And if the TTY is only being used to build LFS, it may not matter much either. The root password has already been assigned by this point, I think, so sniffing that won't be possible.) pgp0gcORxdwy2.pgp Description: PGP signature -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/4/07, William Harrington <[EMAIL PROTECTED]> wrote: > > I'm going to do a 6.3-rc1 build today and see what the tests > output for me on my p4 2.8 northwood system using an intel D845PESV > and LFS 6.1 and a 2.6.22.1 kernel. FYI, there are a few post-rc1 commits here: http://www.linuxfromscratch.org/lfs/view/6.3-branch/ The only real build change is glibc-2.5.1. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Thu, Aug 02, 2007 at 01:49:33PM +1200, Steve Crosby wrote: > On 8/2/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > On Wed, Aug 01, 2007 at 02:53:59PM -0700, Dan Nicholson wrote: > > > On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > > > > > I got those failures on a single run (using jhalfs). I'm not sure > > > what's causing the errors, but what's failing is `test -r /dev/fd/0' > > > and `test -r /dev/stdin' (look at tests/test.right for the output that > > > it's diffing to above). > > > > > > So, I suspect this has something to do with the su to the nobody user > > > and how su handles these devices. But the last time I thought about > > > this it hurt my head. It may have something even more to do with how > > > our scripts are handling the user switching. > > These files both end up being symlinks to /dev/pts/0 (or whatever pts > device you logged into) - and the perms for this are > > root:~# ls -l /dev/fd/0 > lrwx-- 1 root root 64 2007-08-02 14:30 /dev/fd/0 -> /dev/pts/0 > root:~# ls -l /dev/stdin > lrwxrwxrwx 1 root root 15 2007-08-03 02:22 /dev/stdin -> /proc/self/fd/0 > root:~# ls -l /proc/self/fd/0 > lrwx-- 1 root root 64 2007-08-02 14:30 /proc/self/fd/0 -> /dev/pts/0 > root:~# ls -l /dev/pts/0 > crw--w 1 root tty 136, 0 2007-08-02 14:30 /dev/pts/0 > > so the nobody user won't be able to read these devices. Not sure how > you would work around that, unless you use login instead of su to > start the nobody user doing the testing (which will change ownership > of /dev/pts/x and hence the tests will pass) > A little bit of testing (after building to the end of chapter 6 earlier, I've gone back into chroot to play with this). It looks as if chown /dev/stdin *might* work (I'm on an xterm): root in chroot /# chown nobody /dev/stdin root in chroot /# su-tools nobody -s /bin/bash bash: /dev/null/.bashrc: Not a directory nobody in chroot /$ ls -l /dev/stdin lrwxrwxrwx 1 root root 15 Aug 4 15:51 /dev/stdin -> /proc/self/fd/0 nobody in chroot /$ ls -l /dev/pts total 0 crw--w 1 kentty 136, 0 Aug 4 16:22 0 crw--w 1 kentty 136, 1 Aug 4 16:01 1 crw--w 1 kentty 136, 2 Aug 4 16:30 2 crw--w 1 nobody tty 136, 3 Aug 4 16:32 3 crw--w 1 kentty 136, 4 Aug 4 16:30 4 nobody in chroot /$ test -r /dev/stdin ; echo $? 0 nobody in chroot /$ This seems too good to be true. We are running as root, so I guess we can happily continue to read and write to this pts dev after the tests are finished. If nobody pokes a hole in this or beats me to it, I'll start another build, but probably not before tomorrow. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Hello all, I'm going to do a 6.3-rc1 build today and see what the tests output for me on my p4 2.8 northwood system using an intel D845PESV and LFS 6.1 and a 2.6.22.1 kernel. I normally build PCRE before grep so that will be noted as well. I'll dump all the output somewhere nice just to see if I get what everyone else is talking about. If I shouldn't build pcre before grep, let me know. Sincerely, William -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Wed, Aug 01, 2007 at 08:09:58PM +0100, Ken Moffat wrote: > > Haven't yet looked at my libc, I retested the files that were where > I thought I'd left them, but those matched up. Hmm, after stripping and then running 'cmp -bl' (which isn't ideal, you have to watch out for the bytes that _don't_ change in text) I have _one_ binary (I assume) byte different in the two /lib/libc-2.5.so files, followed by a difference in the text (what you normally get when you execute /lib/libc.so.6). Turns out I accidentally built the first pass using kernel 2.6.22-rc1, and the second two days later with the intended 2.6.22.1. Anybody who follows my ramblings on clfs-dev will know that using an earlier kernel than the headers, particularly an -rc kernel, is "NOT a Good Idea" (TM) and I'm lucky I wasn't using glibc-2.6. I'll try to do a fresh by-the-book pair of builds (without non-toolchain tests) if I've got time. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 8/2/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > On Wed, Aug 01, 2007 at 02:53:59PM -0700, Dan Nicholson wrote: > > On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > > > I got those failures on a single run (using jhalfs). I'm not sure > > what's causing the errors, but what's failing is `test -r /dev/fd/0' > > and `test -r /dev/stdin' (look at tests/test.right for the output that > > it's diffing to above). > > > > So, I suspect this has something to do with the su to the nobody user > > and how su handles these devices. But the last time I thought about > > this it hurt my head. It may have something even more to do with how > > our scripts are handling the user switching. These files both end up being symlinks to /dev/pts/0 (or whatever pts device you logged into) - and the perms for this are root:~# ls -l /dev/fd/0 lrwx-- 1 root root 64 2007-08-02 14:30 /dev/fd/0 -> /dev/pts/0 root:~# ls -l /dev/stdin lrwxrwxrwx 1 root root 15 2007-08-03 02:22 /dev/stdin -> /proc/self/fd/0 root:~# ls -l /proc/self/fd/0 lrwx-- 1 root root 64 2007-08-02 14:30 /proc/self/fd/0 -> /dev/pts/0 root:~# ls -l /dev/pts/0 crw--w 1 root tty 136, 0 2007-08-02 14:30 /dev/pts/0 so the nobody user won't be able to read these devices. Not sure how you would work around that, unless you use login instead of su to start the nobody user doing the testing (which will change ownership of /dev/pts/x and hence the tests will pass) -- -- - Steve Crosby -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Wed, Aug 01, 2007 at 02:53:59PM -0700, Dan Nicholson wrote: > On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > > > To expand on what I posted earlier about test failures: > > > > Tar is repeatedly failing '26: incremental' for me, looks like a > > regression. But nobody else has commented. > > This time I got all passes for tar. Go figure. Yeah, I think it often works. Greg pointed to failures in another test on the tar list, but I haven't seen that in my own results. > > > The bash failure I reported was a fubar in my script (trying to > > chown the test log so it was writable by the appropriate user, but > > before it was created ;). But, my second run did seem to have one > > failure - 'run-test' shows > > > > 152c152 > > < 1 > > --- > > > 0 > > 158c158 > > < 1 > > --- > > > 0 > > I got those failures on a single run (using jhalfs). I'm not sure > what's causing the errors, but what's failing is `test -r /dev/fd/0' > and `test -r /dev/stdin' (look at tests/test.right for the output that > it's diffing to above). > > So, I suspect this has something to do with the su to the nobody user > and how su handles these devices. But the last time I thought about > this it hurt my head. It may have something even more to do with how > our scripts are handling the user switching. > Hmm, it might indeed. I can't begin to get my head around the detail of *most* packages' testsuites. What I haven't done is actually confirm if the test finished with a good or bad status - will try to remember to check that tomorrow, and then see what happens if I try to su-tools and that test. > > The perl failure didn't happen on the second run, I guess it's just > > another unreliable test. > > What was the exact perl failure? (edited to suppress dots and path) ext/IO/t/io_sock Died at io_sock.t line 37, line 2. FAILED--expected 26 tests, saw 11. Somebody dug into _a_ perl failure recently and pointed out that the text had a description saying it sometimes fails. I'm not sufficiently motivated to search for the posting ;) > > > And the vim test failure is totally impenetrable. > > One of the vim tests hung on me, and trying to decipher the output > only resulted in garbage in the terminal. I think I might just stop > running this thing. > If my scripts had the ability to control tests on a per-package basis, I think I'd not run them for lots of the packages - I generally keep running tests because they usually look ok, but when a failure can be tracked down, more often than not it seems to be a problem in the testsuite. With vim, mine always writes to a log so the terminal itself isn't trashed. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > To expand on what I posted earlier about test failures: > > Tar is repeatedly failing '26: incremental' for me, looks like a > regression. But nobody else has commented. This time I got all passes for tar. Go figure. > The bash failure I reported was a fubar in my script (trying to > chown the test log so it was writable by the appropriate user, but > before it was created ;). But, my second run did seem to have one > failure - 'run-test' shows > > 152c152 > < 1 > --- > > 0 > 158c158 > < 1 > --- > > 0 I got those failures on a single run (using jhalfs). I'm not sure what's causing the errors, but what's failing is `test -r /dev/fd/0' and `test -r /dev/stdin' (look at tests/test.right for the output that it's diffing to above). So, I suspect this has something to do with the su to the nobody user and how su handles these devices. But the last time I thought about this it hurt my head. It may have something even more to do with how our scripts are handling the user switching. > The perl failure didn't happen on the second run, I guess it's just > another unreliable test. What was the exact perl failure? > And the vim test failure is totally impenetrable. One of the vim tests hung on me, and trying to decipher the output only resulted in garbage in the terminal. I think I might just stop running this thing. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Tue, Jul 31, 2007 at 03:50:53PM -0700, Dan Nicholson wrote:
> On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote:
> > And moving on to farce test results ("how repeatable is it?") -
> > apart from cc1 and cc1plus I also had a failure in libc-2.5.so (not
> > yet investigated) and in private mail from archaic I know he saw a
> > failure in nscd (I've got his files for this, but haven't investigated
> > it yet).
>
> libc.so and nscd both embed the build date.
>
> $ strings /lib/libc.so.6 | grep 2007
> Compiled on a Linux >>2.6.20.14-1<< system on 2007-07-15.
> $ strings /usr/sbin/nscd | grep 2007
> Jul 15 2007 08:44:08
>
> Although, I though farce tried to strip out the dates, so maybe it's
> something worse. If you want to be sure about the nscd one, you can
> just blow away the __DATE__ and __TIME__ macros:
>
> sed -i.bak \
> -e 's/__DATE__/"today"/' \
> -e 's/__TIME__/"now"/' nscd/nscd_stat.c
>
Sorry, I missed this part of your reply. Farce does indeed replace
known date/time patterns by tokens. For me, nscd builds repeatably
as far as farce is concerned.
After running 'strip --strip-all' on the binaries I got from
archaic, the second and third builds differed in a handful of bytes.
The first run appeared to include literals at a slightly different
position (only a byte or two different).
Haven't yet looked at my libc, I retested the files that were where
I thought I'd left them, but those matched up. In general, with old
toolchains it was quite easy to account for much of the variation.
Nowadays, I see a lot of things that just differ. I'll try to get
back to it after I've finished playing with gcc-4.2.0 x86_64 on
multilib xorg-server [ don't try this at home unless you've got at
least 2GB of memory/swap and a lot of time - 4.2.1 reportedly fixes
that problem ].
ĸen
--
das eine Mal als Tragödie, das andere Mal als Farce
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Bruce Dubbs wrote: > http://www.linuxfromscratch.org/~sbu/ :-) -- JH -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Jeremy Huntwork wrote: > M.Canales.es wrote: >> If we are happy with big figures, thus use the same values for all archs, If >> we want something accurate for each arch, remove the info from the book a >> create a web page to place jhalfs reports. > > I like this option. Perhaps provide *very* rough estimates for the SBU > (round to the nearest 1/2 SBU or so, based probably on x86) in the book > and a store of SBU reports elsewhere on the web. http://www.linuxfromscratch.org/~sbu/ -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Jeremy Huntwork wrote: > M.Canales.es wrote: >> And when x86_64 will be merged, that values will be less accurate. > > I suppose that's argument for producing a separate set of text for 64-bit. >From Section 4.5 of the book: "In general, SBUs are not entirely accurate because they depend on many factors, including the host system's version of GCC. Note that on Symmetric Multi-Processor (SMP)-based machines, SBUs are even less accurate. They are provided here to give an estimate of how long it might take to install a package, but the numbers can vary by as much as dozens of minutes in some cases." In other words, we already say they are an approximation. The text above could be changed to say: "... 64-bit platforms or Symmetric Multi-Processor (SMP)-based machines ..." -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
M.Canales.es wrote: > If we are happy with big figures, thus use the same values for all archs, If > we want something accurate for each arch, remove the info from the book a > create a web page to place jhalfs reports. I like this option. Perhaps provide *very* rough estimates for the SBU (round to the nearest 1/2 SBU or so, based probably on x86) in the book and a store of SBU reports elsewhere on the web. -- JH -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
El Miércoles, 1 de Agosto de 2007 19:43, Jeremy Huntwork escribió: > I suppose that's argument for producing a separate set of text for 64-bit. Not, if creating sepparate books I refuse to have separate diskusage/buildtime blocks and entities sets for each arch. On CLFS and HLFS that info was removed due the big amount of work involved on maintain the XML code and in keeping that values more or less accurates for each book. If we are happy with big figures, thus use the same values for all archs, If we want something accurate for each arch, remove the info from the book a create a web page to place jhalfs reports. -- Manuel Canales Esparcia Usuario de LFS nº2886: http://www.linuxfromscratch.org LFS en castellano: http://www.escomposlinux.org/lfs-es http://www.lfs-es.info TLDP-ES: http://es.tldp.org -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
M.Canales.es wrote: > And when x86_64 will be merged, that values will be less accurate. I suppose that's argument for producing a separate set of text for 64-bit. -- JH -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
El Miércoles, 1 de Agosto de 2007 00:50, Dan Nicholson escribió: > > I guess I just like to know how things are gonna shape up relatively. > I.e., gcc's gonna take a while, no sense in watching the screen. I > don't care if the numbers aren't accurate. But people using older > hardware may be more interested in the readings. I suppose one > compromise would be less precision. I.e., round to the nearest MiB or > integer SBU. And when x86_64 will be merged, that values will be less accurate. -- Manuel Canales Esparcia Usuario de LFS nº2886: http://www.linuxfromscratch.org LFS en castellano: http://www.escomposlinux.org/lfs-es http://www.lfs-es.info TLDP-ES: http://es.tldp.org -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Bruce Dubbs wrote: > Yes, that sounds fine. Let me know when you think -rc2 should be released. Just wanted to mention there is a 6.3-rc1 package tarball I generated a couple days ago, will do one for rc2 as soon as it is out as well. ftp://ftp.lfs-matrix.net/pub/lfs/lfs-packages/lfs-packages-6.3-rc1.tar Justin -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
Dan Nicholson wrote: > On 7/30/07, Dan Nicholson <[EMAIL PROTECTED]> wrote: >> On 7/30/07, Bruce Dubbs <[EMAIL PROTECTED]> wrote: >>> I put out a 6.3-rc1 a week ago and there really has been very little >>> feedback. Is it ready for final release? >> Let me spin new tarballs for lfs-bootscripts and udev-config. I'll >> make an announcement on lfs-dev and ask people to test them out. > > I took care of the udev rules and lfs-bootscripts. Next: glibc-2.5.1 > out just in time :) There's only been one real commit (a bug fix in > printf) since the last branch_update patch, so it should be > straightforward. I ran the testsuite and still had only the single > failure in nptl/tst-cancel1. Haven't bootstrapped anything. I'm > prepping a diff now. > > Give me a day or so to get that in and try to go throught Ken's > points. Then we can roll -rc2 and let it gestate a little. Sound good? Yes, that sounds fine. Let me know when you think -rc2 should be released. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Tue, Jul 31, 2007 at 03:50:53PM -0700, Dan Nicholson wrote: > On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote: > > > > Tar is repeatedly failing '26: incremental' for me, looks like a > > regression. But nobody else has commented. > > If you have any idea how to narrow this down, that'd be great. I have > hit this before myself, and it seems like a race condition, but I > can't say for sure. I haven't run the full range of tests on all the > packages in a while. At worst, we can add a note that this test fails > sometimes. > Greg's reply pointed out that both 26 and 29 have been seen to sometimes fail. He suggested it might need 'sleep' added to the testsuite (in which case, yet another broken testsuite). I'm ok with adding "tests 26 and 29 sometimes fail", but then I'm equally ok with adding a more general "occasional failures in the testsuites are probably nothing to worry about". The trouble with that get-out is its use of language: to me, the text about the ignored failure in glibc is clear, but people have problems with it. Adding a comment that says "probably ok" will be a pain when every other builder starts asking "do I need to worry?" The real problem is that test failures _sometimes_ indicate a problem. Mostly (talking generally, not x86-specific) they seem to indicate that the testsuite is not perfect. ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 7/30/07, Ken Moffat <[EMAIL PROTECTED]> wrote:
>
> Tar is repeatedly failing '26: incremental' for me, looks like a
> regression. But nobody else has commented.
If you have any idea how to narrow this down, that'd be great. I have
hit this before myself, and it seems like a race condition, but I
can't say for sure. I haven't run the full range of tests on all the
packages in a while. At worst, we can add a note that this test fails
sometimes.
> The bash failure I reported was a fubar in my script (trying to
> chown the test log so it was writable by the appropriate user, but
> before it was created ;). But, my second run did seem to have one
> failure - 'run-test' shows
>
> 152c152
> < 1
> ---
> > 0
> 158c158
> < 1
> ---
> > 0
>
> - as to what that means, your guess is as good as mine.
>
> The perl failure didn't happen on the second run, I guess it's just
> another unreliable test.
>
> And the vim test failure is totally impenetrable.
Don't know about any of those. I don't really bother reading the
results of the vim tests.
> And moving on to farce test results ("how repeatable is it?") -
> apart from cc1 and cc1plus I also had a failure in libc-2.5.so (not
> yet investigated) and in private mail from archaic I know he saw a
> failure in nscd (I've got his files for this, but haven't investigated
> it yet).
libc.so and nscd both embed the build date.
$ strings /lib/libc.so.6 | grep 2007
Compiled on a Linux >>2.6.20.14-1<< system on 2007-07-15.
$ strings /usr/sbin/nscd | grep 2007
Jul 15 2007 08:44:08
Although, I though farce tried to strip out the dates, so maybe it's
something worse. If you want to be sure about the nscd one, you can
just blow away the __DATE__ and __TIME__ macros:
sed -i.bak \
-e 's/__DATE__/"today"/' \
-e 's/__TIME__/"now"/' nscd/nscd_stat.c
> He also noted in that mail that coreutils seems to be using
> a pre-created info file in his second and third builds - (I've only
> run two, and misread the diff : coreutils.info was created by
> makeinfo version 4.9 in the first build, and 4.8 in the second).
> But, we don't expect user to build it twice, so I'm not too worried
> about that, I just add it to my "coreutils Makefile is a POS"
> thoughts :-)
That sucks, but at least it's not critical.
> Which only leaves space/SBU measurements. Because my build hasn't
> been by-the-book, and has run tests whenever possible, I'll only
> comment on those I know look wrong (chapter 6 only)
I'll try to get jhalfs to spit some numbers to me and see if they agree.
> Now, time for me to ask a question: Is it worthwhile that we
> continue to record SBUs and space ? Everybody knows that many
> packages take longer to build and use more space whenever the
> toolchain is upgraded. Is it really worthwhile to be so exact about
> the time and space. Certainly, space should be constant across
> an architecture (well, i686 anyway) for a given toolchain, but the
> timings depend greatly on amount of memory (do you hit swap?),
> memory speed (try using a via processor), and disk speed.
I guess I just like to know how things are gonna shape up relatively.
I.e., gcc's gonna take a while, no sense in watching the screen. I
don't care if the numbers aren't accurate. But people using older
hardware may be more interested in the readings. I suppose one
compromise would be less precision. I.e., round to the nearest MiB or
integer SBU.
--
Dan
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 7/30/07, Dan Nicholson <[EMAIL PROTECTED]> wrote: > On 7/30/07, Bruce Dubbs <[EMAIL PROTECTED]> wrote: > > I put out a 6.3-rc1 a week ago and there really has been very little > > feedback. Is it ready for final release? > > Let me spin new tarballs for lfs-bootscripts and udev-config. I'll > make an announcement on lfs-dev and ask people to test them out. I took care of the udev rules and lfs-bootscripts. Next: glibc-2.5.1 out just in time :) There's only been one real commit (a bug fix in printf) since the last branch_update patch, so it should be straightforward. I ran the testsuite and still had only the single failure in nptl/tst-cancel1. Haven't bootstrapped anything. I'm prepping a diff now. Give me a day or so to get that in and try to go throught Ken's points. Then we can roll -rc2 and let it gestate a little. Sound good? -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Jeremy Huntwork wrote: > 3) Section 5.7, 'Adjusting the Toolchain'. Here's where it gets a > little fun. But still, easily adapted to fit both situations. > Somewhere, at the beginning of the book, we set a LINKER (or some > such) variable, depending on arch. Easily scripted via `uname -m` for > the sake of accuracy. We also take into account that gcc will make > use of the /lib64 directory internally, so we account for that, too, > in our adjusting command. Like so: > gcc -dumpspecs | sed "s@/lib[64]*/$LINKER@/tools&@g" \ (etc.) Just a suggestion: that sed regexp will match more strings than we intend (but not in this context). This would work fine too, and would be more restrictive: gcc -dumpspecs | sed "s@/lib\(64\)\?/$LINKER@/tools&@g" \ (etc.) Not that I think it really matters -- what we have will work OK. It's just that people may inadvertently start to think that square brackets tell sed to group things together, when they really delimit character classes. Unfortunately the "make a group out of "64", and accept zero or one of them" version is several characters longer due to the backslashes. That can be shortened a bit with sed's -r option: gcc -dumpspecs | sed -r "s@/lib(64)?/$LINKER@/tools&@g" \ (etc.) but then you're typing in a new "-r" option, so it's pretty much no different in terms of amount of typing. ;-) (It may also require GNU sed, I'm not sure.) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGrm56S5vET1Wea5wRAyMoAKCwViWZgKqPjB5fnUHXKjkM49cNSwCfUQ5V /dR3X6zRrtrrl1Pic+uuSV4= =xYZ6 -END PGP SIGNATURE- -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Tar tests (was Re: Is 6.3 ready for release?)
On Tue, Jul 31, 2007 at 08:07:47AM +1000, Greg Schafer wrote: > Ken Moffat wrote: > > > Tar is repeatedly failing '26: incremental' for me, looks like a > > regression. But nobody else has commented. > > I see this intermittently. I also see 29 failing intermittently too which > I reported upstream to no response: > > http://lists.gnu.org/archive/html/bug-tar/2007-07/msg00032.html > > In fact, the upstream list has various reports of both 26 and 29 failing. > Because they appear to be intermittent failures, I believe they are > probably timing related. It's possible these tests are missing sleep > statements in key areas (there are many sleep statements within the tar > testsuite). > > Regards > Greg Thanks, Greg. One less thing to worry about for a release. Maybe I was just lucky when running tests before 6.3 and on other architectures (actually, the one I definitely remember passing was on ppc - that box is so slow it probably doesn't need any added 'sleep'). ĸen -- das eine Mal als Tragödie, das andere Mal als Farce -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Tar tests (was Re: Is 6.3 ready for release?)
On 7/30/07, Greg Schafer <[EMAIL PROTECTED]> wrote: > Ken Moffat wrote: > > > Tar is repeatedly failing '26: incremental' for me, looks like a > > regression. But nobody else has commented. > > I see this intermittently. I also see 29 failing intermittently too which > I reported upstream to no response: Yeah, I've been seeing "incremental" as well as "sorted" failing on and off over the past couple releases (I think "sorted" might be fixed now). > http://lists.gnu.org/archive/html/bug-tar/2007-07/msg00032.html > > In fact, the upstream list has various reports of both 26 and 29 failing. > Because they appear to be intermittent failures, I believe they are > probably timing related. It's possible these tests are missing sleep > statements in key areas (there are many sleep statements within the tar > testsuite). Seems pretty likely. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Tar tests (was Re: Is 6.3 ready for release?)
Ken Moffat wrote: > Tar is repeatedly failing '26: incremental' for me, looks like a > regression. But nobody else has commented. I see this intermittently. I also see 29 failing intermittently too which I reported upstream to no response: http://lists.gnu.org/archive/html/bug-tar/2007-07/msg00032.html In fact, the upstream list has various reports of both 26 and 29 failing. Because they appear to be intermittent failures, I believe they are probably timing related. It's possible these tests are missing sleep statements in key areas (there are many sleep statements within the tar testsuite). Regards Greg -- http://www.diy-linux.org/ -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Jul 30, 2007 at 01:02:29PM -0500, Bruce Dubbs wrote:
> I put out a 6.3-rc1 a week ago and there really has been very little
> feedback. Is it ready for final release?
>
> -- Bruce
Wow, a whole week. Maybe, hardly anybody has tried it because this
is the holiday season in the Northern hemisphere.
To expand on what I posted earlier about test failures:
Tar is repeatedly failing '26: incremental' for me, looks like a
regression. But nobody else has commented.
The bash failure I reported was a fubar in my script (trying to
chown the test log so it was writable by the appropriate user, but
before it was created ;). But, my second run did seem to have one
failure - 'run-test' shows
152c152
< 1
---
> 0
158c158
< 1
---
> 0
- as to what that means, your guess is as good as mine.
The perl failure didn't happen on the second run, I guess it's just
another unreliable test.
And the vim test failure is totally impenetrable.
In general, the more I run test suites, the less confidence I have
in them. Sure, sometimes they point to problems (e.g. when our
build order in clfs was wrong and the findutils tests crapped out),
but a lot of the time I wonder why we bother.
And moving on to farce test results ("how repeatable is it?") -
apart from cc1 and cc1plus I also had a failure in libc-2.5.so (not
yet investigated) and in private mail from archaic I know he saw a
failure in nscd (I've got his files for this, but haven't investigated
it yet). He also noted in that mail that coreutils seems to be using
a pre-created info file in his second and third builds - (I've only
run two, and misread the diff : coreutils.info was created by
makeinfo version 4.9 in the first build, and 4.8 in the second).
But, we don't expect user to build it twice, so I'm not too worried
about that, I just add it to my "coreutils Makefile is a POS"
thoughts :-) Nobody else has reported any difference in
libc-2.5.so so that is probably a local fubar or a shortcoming in my
farce regexps.
OTOH, nobody has yet reported on their successes or failures in
using it to build BLFS, whether for a server or for a desktop. I
was hoping to spend time looking at farce before I ran a third build
without tests, and later a by-the-book build with only toolchain
tests, but if you want to get it released I'll happily drop those.
I won't be able to finish a desktop for some time.
Which only leaves space/SBU measurements. Because my build hasn't
been by-the-book, and has run tests whenever possible, I'll only
comment on those I know look wrong (chapter 6 only)
1. Kernel headers. The time is still minimal, but I think these take
304MB, not 286MB (that should apply to chapter 5 too) - that's
because the instructions were altered to install to a subdirectory in
the source and then copy them, which is much nicer/safer but
guaranteed to take more space.
2. The coreutils time (1.0 SBU) is presumably without tests, but
mine only took 1.0 SBU with the tests.
3. If the SBU for autoconf without tests is correct, the tests take
about 4 SBU, not 3 (don't you just love newer toolchains?).
4. Similarly, automake is in excess of 13 SBU with tests, not 10.
5. My bzip2 install took 6.4 MiB not 5.3, I attribute this to the
docs which seem to be non-optional according to how the book is
written.
6. Findutils supposedly takes 13.6 MiB - I assume that is without
the tests: with the tests mine took a little more time but only 12.6
MiB.
7. Gzip for me takes 3.3 MiB not 2.2 MiB.
8. Inetutils for me takes 0.3 SBU (not 0.2) and 12.1 MiB (not 8.9).
9. Iproute2 for me takes 0.1 SBU (not 0.2) and 5.0 MiB (not 4.8).
Actually, that might be getting a bit close to splitting hairs.
10. The lfs-bootscripts use 0.6 MiB for me, not 0.4 MiB.
Now, time for me to ask a question: Is it worthwhile that we
continue to record SBUs and space ? Everybody knows that many
packages take longer to build and use more space whenever the
toolchain is upgraded. Is it really worthwhile to be so exact about
the time and space. Certainly, space should be constant across
an architecture (well, i686 anyway) for a given toolchain, but the
timings depend greatly on amount of memory (do you hit swap?),
memory speed (try using a via processor), and disk speed.
ĸen
--
das eine Mal als Tragödie, das andere Mal als Farce
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Jul 30, 2007 at 10:08:53PM +0200, M.Canales.es wrote: > IMHO, multilib build instructions will be very intrussive due that several > packages need be builded two times. If we want to add it, we will need to > render sepparate books to not mess the reader with a lot of "if ". I prefer to stay away from multilib. At least for the time being. -- JH -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
El Lunes, 30 de Julio de 2007 21:26, Jeremy Huntwork escribió: > I've been thinking about this some more recently. I really think it's > not worth the time and effort (at least not now) to add the extra > complexity to the XML/XSL to render two separate books for x86 LFS and > x86_64 LFS. The command changes are so minimal that in the end, there > would be very few differences at all. Just for the sake of review and to > show how I would resolve the differences in one book, here's what we > would need to change: Yes, the differences between a x86 build and a pure 64 libs x86_64 build are minimal. The question is, we will add only pure 64 libs x86_64? or is that a mean of POC before adding also instructions to build multilib x86_64 systems? IMHO, multilib build instructions will be very intrussive due that several packages need be builded two times. If we want to add it, we will need to render sepparate books to not mess the reader with a lot of "if ". Thus I think that if multilib support will be added, is best to have ready the profiling and rendering framework now that the changes are minimal, to let editors familiarice with the new XML tagging. > 6) Bootloader. This one is probably the biggest. Legacy grub is probably > still preferable for x86. I see 3 options here: > a. Convert to lilo for both (dependency on bin86 which requires a > patch for x86_64) > b. Convert to grub2 for both (untested by me, but I believe it works > for both. Joe Ciccone has more info.) > c. Tell users to install grub via their host system. (Most have it > these days. Wouldn't work for the 64-bit only LiveCD as of now.) d. Move bootloader compilation to Chapter 8 "Making the LFS System Bootable" discussing available alternatives (Grub, Grub2, LiLo, the host bootloader, a boot CD, etc...) -- Manuel Canales Esparcia Usuario de LFS nº2886: http://www.linuxfromscratch.org LFS en castellano: http://www.escomposlinux.org/lfs-es http://www.lfs-es.info TLDP-ES: http://es.tldp.org -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On Mon, Jul 30, 2007 at 08:03:43PM +0200, M.Canales.es wrote:
> If some bug is found later we always could do a 6.3.1 release.
>
> Plus, I would start today with the preparative to can merge the x86_64 branch
> into trunk.
I've been thinking about this some more recently. I really think it's
not worth the time and effort (at least not now) to add the extra
complexity to the XML/XSL to render two separate books for x86 LFS and
x86_64 LFS. The command changes are so minimal that in the end, there
would be very few differences at all. Just for the sake of review and to
show how I would resolve the differences in one book, here's what we
would need to change:
1) After the installation of binutils-pass1, run:
ln -sv lib /tools/lib64
This, being just a symlink has absolutely no effect on x86. We can
put a note that it can be skipped for 32-bit x86, but it hurts nothing.
2) All builds of GCC get the switch '--disable-multilib'. Again,
harmless for x86. Again, we can put a note that it could be left out, if
desired, for x86.
3) Section 5.7, 'Adjusting the Toolchain'. Here's where it gets a little
fun. But still, easily adapted to fit both situations. Somewhere, at the
beginning of the book, we set a LINKER (or some such) variable,
depending on arch. Easily scripted via `uname -m` for the sake of
accuracy. We also take into account that gcc will make use of the /lib64
directory internally, so we account for that, too, in our adjusting
command. Like so:
gcc -dumpspecs | sed "s@/lib[64]*/$LINKER@/tools&@g" \ (etc.)
4) In gcc pass 2, currently there's a sed command that deletes the
multilib spec in gcc/config/i386/t-linux64. I don't think this would
have any effect on x86, but I haven't tested. Also, there have been some
inconsistencies between what I have found wrt the purpose of this
removal and what Greg has found over at DIY. I want to do more testing.
In any case, it is one command and it doesn't seem likely to me to
affect a non-multlib, non-64 bit arch.
5) Creating Directories. Again, adding the {/usr,}/lib64 symlinks. Same
comments as in item 1.
6) Bootloader. This one is probably the biggest. Legacy grub is probably
still preferable for x86. I see 3 options here:
a. Convert to lilo for both (dependency on bin86 which requires a
patch for x86_64)
b. Convert to grub2 for both (untested by me, but I believe it works
for both. Joe Ciccone has more info.)
c. Tell users to install grub via their host system. (Most have it
these days. Wouldn't work for the 64-bit only LiveCD as of now.)
7) Last one. We would need to alter the output of the sanity tests to
accomodate both instances. Or, at least, add more descriptive notes
showing *what* exactly we are looking for and what the differences would
be with a different target triplet and dynamic linker. To me, this would
be a good thing anyway as it makes the instructions that much more
educational and less robotic.
I'll follow the decision of the community on this one, but to me, it
would be a simple thing to merge both possibilities into one set of
instructions. I believe making the LFS book just a little more
architecture agnostic does more to help teach concepts.
--
JH
--
http://linuxfromscratch.org/mailman/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
On 7/30/07, Bruce Dubbs <[EMAIL PROTECTED]> wrote: > I put out a 6.3-rc1 a week ago and there really has been very little > feedback. Is it ready for final release? We need to decide what to do about usb_device devices with linux-2.6.22. http://linuxfromscratch.org/pipermail/lfs-dev/2007-July/059794.html Manuel also asked me to document the consolelog bootscript I added to Ch.7. Oops, it looks like I never spun a new tarball for lfs-bootscripts, so no one has tested the changes I've made recently unless they manually synced. There are two changes in the udev-config rules (in addition to the first item above which hasn't been addressed) that aren't in the current tarball. Let me spin new tarballs for lfs-bootscripts and udev-config. I'll make an announcement on lfs-dev and ask people to test them out. -- Dan -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: Is 6.3 ready for release?
El Lunes, 30 de Julio de 2007 20:02, Bruce Dubbs escribió: > I put out a 6.3-rc1 a week ago and there really has been very little > feedback. Is it ready for final release? I think so, but fixing before the missing consolelog bootscript description in chapter07/bootscripts.xml. I have done a lot of builds this days with no issues. If some bug is found later we always could do a 6.3.1 release. Plus, I would start today with the preparative to can merge the x86_64 branch into trunk. -- Manuel Canales Esparcia Usuario de LFS nº2886: http://www.linuxfromscratch.org LFS en castellano: http://www.escomposlinux.org/lfs-es http://www.lfs-es.info TLDP-ES: http://es.tldp.org -- http://linuxfromscratch.org/mailman/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
