Re: [blfs-dev] Package stats

[Fernando de Oliveira]

Em 07-03-2014 17:40, Ken Moffat escreveu:
> On Thu, Mar 06, 2014 at 06:22:52PM +, Ken Moffat wrote:
>>  It is clear that my rebuild of the docs was very different from
>> Fernando's.  At the moment I'm concentrating on rebuilding
>> everything in chroot, just in case there is any breakage from
>> accidental change in gnutls.  My first attempt at LO just timed out
>> on one of the downloads, so I guess I'm hours away from completing.
>> After that, I'll take another look.
>>
>  Got back to this.  This time, I cannot get close to the figures I
> had before for rebuilding with the docs: time is now an extra +0.1
> SBU, which I think is in line with Fernando, and space has
> *increased* by 3 MB instead of reducing slightly.
> 
>  Looking back at my history, I see that I ran 'make clean' before
> the second build (the one regenerating the docs), and I suppose that
> might account for the reduced space.  But I cannot see any reason
> why the 'make' with gtk-doc took so much time on that occaasion.
> 
>  Since Fernando was upset by my measurements, on this occasion I'll
> change them.
> 
> ĸen
> 

I am never upset with what you do. You have a very good and interesting
humour.

I only replied because you mentioned, if not, I would only notice
anything when upgrading, and doubt very much if I would ask anything.

On the positive side, it is interesting to know what happened (make
clean), so I will be careful. to never do that for this purpose. Another
good side of this is that I was doubting if the names I was using were
confusing, but it seems that we can continue doing that.

So, thanks for having had the patience to re-measure.

Sorry if I seemed upset, but I was really not.

Actually, I am in debt with you, you have always been helping me a lot.
One of the last, you solved an old problem I had with the ink levels of
my printer, introduced me to escputil, and tought me to build a custom
one in my printer server (Lubuntu, not BLFS, and you helping me). It was
a long exchange of mails, and even upstream you contacted.

-- 
[]s,
Fernando
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Re: [blfs-dev] Package stats

[Bruce Dubbs]

Fernando de Oliveira wrote:

> I find "du" more accurate, although more tome consuming,

Yes, that's why I went ot df.

so have to to
> mark some instants to avoid it from interfering in the timings.
>
> On example is that df will include the log size, perhaps you think it is
> relevant.

For me the log is at /usr/src/ so it is not included.

> But all numbers we give are approximate, large error bar, so I do not
> dispute methods or numbers.

True.

> The problem there is that Ken found values very different, even one
> negative: building the API docs takes less than installing the ones in
> the tarball, is one example, and I can hardly believe this is possible:
> build taking less time than just installing the documents.

That would take some analysis.  I don't see how that could happen either.

> As I introduced many of these numbers, probably after what Ken used to
> do with ImageMagick, what I think is that being non-english speaking
> native, I am giving wrong names to what I measure. That was the reason I
> detailed how and what I measure there. It seems I need to rename in the
> book, some number I gave.
>
> What I mean is: everybody know 1 inch is different from 1 cm. But I can
> use a ruler, make a measurement and tell Ken it is 1, using a cm ruler,
> but he thinking I am using inches.
>
> He is not wrong
>
> I am not wrong

Yes, but seconds is the same for everyone.  So is bytes.  Well 
some differentiate between KiB and KB.  :)

> We are giving numbers for different things, probably my fault of not
> writing carefully what my number means.

WE are just giving the user an approximation.  When I update, I do 
change the stats, but it's rarely a significant change.

   -- Bruce




-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Re: [blfs-dev] Package stats

[Fernando de Oliveira]

Em 06-03-2014 15:22, Ken Moffat escreveu:
> On Thu, Mar 06, 2014 at 02:36:02PM -0300, Fernando de Oliveira wrote:
>> Em 06-03-2014 13:36, Bruce Dubbs escreveu:
>>>
>>>So build size is measured as df_after - df_before.  The issue to note 
>>> is that there is activity during the build that adds or deletes space on 
>>> /tmp, the size will be off.  I have /tmp on its own partition.
>>>
>  To me, that seems excessively complicated.  But each to his own.
>>
>>
>> Thanks, Bruce.
>>
>> I find "du" more accurate, although more tome consuming, so have to to
>> mark some instants to avoid it from interfering in the timings.
>>
>> On example is that df wil include the log size, perhaps you think it is
>> relevant.
> 
>  When I measure for the book, I don't usually make logs!  If I am
> making logs, I usually put them in ../ and use du for the directory
> and for the DESTDIR or equivalent directory.

One thing is that the value of du -sch id different from (du -sh) + (du
-sh), du -sch is more accurate, if more than one file/dir are measured.

>>
>> But all numbers we give are approximate, large error bar, so I do not
>> dispute methods or numbers.
> 
>  Agree, I see large variations - even in remeasuring the SBU.
>>
>> The problem there is that Ken found values very different, even one
>> negative: building the API docs takes less than installing the ones in
>> the tarball, is one example, and I can hardly believe this is possible:
>> build taking less time than just installing the documents.
>>
> 
>  Did I write that ?  I intended to say that rebuilding the API docs
> took a lot of extra time, but that the space used was 1MB less - I
> guess that the recreated docs are trimmed down.

LOL. My bad, mixed the two subjects. Sorry

> 
>> As I introduced many of these numbers, probably after what Ken used to
>> do with ImageMagick, what I think is that being non-english speaking
>> native, I am giving wrong names to what I measure. That was the reason I
>> detailed how and what I measure there. It seems I need to rename in the
>> book, some number I gave.
>>
>> What I mean is: everybody know 1 inch is different from 1 cm. But I can
>> use a ruler, make a measurement and tell Ken it is 1, using a cm ruler,
>> but he thinking I am using inches.
>>
>> He is not wrong
>>
>> I am not wrong
>>
>> We are giving numbers for different things, probably my fault of not
>> writing carefully what my number means.
>>
>  It is clear that my rebuild of the docs was very different from
> Fernando's.  At the moment I'm concentrating on rebuilding
> everything in chroot, just in case there is any breakage from
> accidental change in gnutls.  My first attempt at LO just timed out
> on one of the downloads, so I guess I'm hours away from completing.
> After that, I'll take another look.
> 
>  Unfortunately, this system with gnome packages is my only 7.5
> system with gtk-doc and p11-kit, so I can't do comparative tests on
> the other box.
> 
>  What I did notice was a *lot* of activity during the rebuilding of
> the docs (not logged, so I watched stdout scroll past, with
> references to html at one point.
> 
>  Guess I'd better create logs when I come back to this, so that I
> can summarise the difference in the builds.
> 
> ĸen
> 

Actually, I only came discussing this subject, because you mentioned in
some mail this or last week. Now, that I am trying to get done the
tickets, I lost so much time with docs and tests, that for the moment, I
will continue updating without caring about them, only in the second
round will restart. So, for a couple of packages, I already have the
complete set of numbers, but for all following ones, only the main
values of dirsizes and SBU will updated, the other ones will be kpt as are.

After what you wrote, in the ticket, I think we are measuring the same
thing, probably I committed a mistake. As I said in previous paragraph.
I don't care, for the moment. Next updates, we will have more time to do
things decently.

Cheers,

-- 
[]s,
Fernando
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Re: [blfs-dev] Package stats

[Ken Moffat]

On Thu, Mar 06, 2014 at 02:36:02PM -0300, Fernando de Oliveira wrote:
> Em 06-03-2014 13:36, Bruce Dubbs escreveu:
> > 
> >So build size is measured as df_after - df_before.  The issue to note 
> > is that there is activity during the build that adds or deletes space on 
> > /tmp, the size will be off.  I have /tmp on its own partition.
> > 
 To me, that seems excessively complicated.  But each to his own.
> 
> 
> Thanks, Bruce.
> 
> I find "du" more accurate, although more tome consuming, so have to to
> mark some instants to avoid it from interfering in the timings.
> 
> On example is that df wil include the log size, perhaps you think it is
> relevant.

 When I measure for the book, I don't usually make logs!  If I am
making logs, I usually put them in ../ and use du for the directory
and for the DESTDIR or equivalent directory.
> 
> But all numbers we give are approximate, large error bar, so I do not
> dispute methods or numbers.

 Agree, I see large variations - even in remeasuring the SBU.
> 
> The problem there is that Ken found values very different, even one
> negative: building the API docs takes less than installing the ones in
> the tarball, is one example, and I can hardly believe this is possible:
> build taking less time than just installing the documents.
> 

 Did I write that ?  I intended to say that rebuilding the API docs
took a lot of extra time, but that the space used was 1MB less - I
guess that the recreated docs are trimmed down.

> As I introduced many of these numbers, probably after what Ken used to
> do with ImageMagick, what I think is that being non-english speaking
> native, I am giving wrong names to what I measure. That was the reason I
> detailed how and what I measure there. It seems I need to rename in the
> book, some number I gave.
> 
> What I mean is: everybody know 1 inch is different from 1 cm. But I can
> use a ruler, make a measurement and tell Ken it is 1, using a cm ruler,
> but he thinking I am using inches.
> 
> He is not wrong
> 
> I am not wrong
> 
> We are giving numbers for different things, probably my fault of not
> writing carefully what my number means.
> 
 It is clear that my rebuild of the docs was very different from
Fernando's.  At the moment I'm concentrating on rebuilding
everything in chroot, just in case there is any breakage from
accidental change in gnutls.  My first attempt at LO just timed out
on one of the downloads, so I guess I'm hours away from completing.
After that, I'll take another look.

 Unfortunately, this system with gnome packages is my only 7.5
system with gtk-doc and p11-kit, so I can't do comparative tests on
the other box.

 What I did notice was a *lot* of activity during the rebuilding of
the docs (not logged, so I watched stdout scroll past, with
references to html at one point.

 Guess I'd better create logs when I come back to this, so that I
can summarise the difference in the builds.

ĸen
-- 
das eine Mal als Tragödie, dieses Mal als Farce
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

Re: [blfs-dev] Package stats

[Fernando de Oliveira]

Em 06-03-2014 13:36, Bruce Dubbs escreveu:
> There is a question in the tickets about statistics.  This is how I do 
> it.  I have a script for each package that measures time and size.
> 
> I build packages in /tmp, but really could be anywhere.
> 
> First measure the free disk space:
> 
> before=`df -k /tmp | grep / | sed -e "s/ \{2,\}/ /g" | cut -d' ' -f3`
> 
> Run the build:
> 
> TIMEFMT='%1R Elapsed Time - '
> 
> { time \
>{
>  echo Making $TITLE
>  date
>  # Instructions go here
> 
> 
>}
> } 2>&1 | tee -a $LOG
> 
> Get the statistics:
> 
> stats $LOG $DIR/$PROGRAM.tar.?z* $before
> 
> Where stats does:
> 
> function stats()
> {
>log=$1
>tarball=$2
>b4=$3
> 
>#  This changes slightly for a base LFS build
>base_sbu=118
> 
>free_now=`df -k /tmp | grep / | sed -e "s/ \{2,\}/ /g" |
>  cut -d" " -f3`
> 
>buildtime=`tail -n1 $log|cut -f1 -d" "`
>sbu=`echo "scale=3; $buildtime / $base_sbu" | bc`
> 
>psizeK=`du -k $tarball | cut -f1`
>psizeM=`echo "scale=3; $psizeK / 1024"   | bc`
> 
>bsizeK=`echo "$free_now - $b4"   | bc`
>bsizeM=`echo "scale=3; $bsizeK / 1024"   | bc`
> 
>echo "SBU=$sbu"  | tee -a $log
>echo "$psizeK $tarball size ($psizeM MB)"| tee -a $log
>echo "$bsizeK kilobytes build size ($bsizeM MB)" | tee -a $log
>(echo -n "md5sum : "; md5sum $tarball)   | tee -a $log
>(echo -n "sha1sum: "; sha1sum $tarball)  | tee -a $log
> 
>echo "`date` $tarball" >> /usr/src/packages-$(lsb_release -r|
>  cut -f2).log
> }
> 
>So build size is measured as df_after - df_before.  The issue to note 
> is that there is activity during the build that adds or deletes space on 
> /tmp, the size will be off.  I have /tmp on its own partition.
> 
>-- Bruce
> 


Thanks, Bruce.

I find "du" more accurate, although more tome consuming, so have to to
mark some instants to avoid it from interfering in the timings.

On example is that df wil include the log size, perhaps you think it is
relevant.

But all numbers we give are approximate, large error bar, so I do not
dispute methods or numbers.

The problem there is that Ken found values very different, even one
negative: building the API docs takes less than installing the ones in
the tarball, is one example, and I can hardly believe this is possible:
build taking less time than just installing the documents.

As I introduced many of these numbers, probably after what Ken used to
do with ImageMagick, what I think is that being non-english speaking
native, I am giving wrong names to what I measure. That was the reason I
detailed how and what I measure there. It seems I need to rename in the
book, some number I gave.

What I mean is: everybody know 1 inch is different from 1 cm. But I can
use a ruler, make a measurement and tell Ken it is 1, using a cm ruler,
but he thinking I am using inches.

He is not wrong

I am not wrong

We are giving numbers for different things, probably my fault of not
writing carefully what my number means.

-- 
[]s,
Fernando
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-dev
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page