Performance issue of libtool-2.4.4

[Robert Yang]

Hello libtool,

It seems that libtool (2.4.4) has increased the packages build time, after
a rough investigation, it maybe because new libtoolize needs run a m4
command:

# Save the command pipeline results for further use by callers of
# this function.
func_extract_trace_result=`$ECHO "$_G_mini" \
  |$M4 -daq --prefix $_G_m4_traces - "$@" 2>&1 1>/dev/null \
  |$SED -n -e "$_G_transform"`

This may increase the build time (just a rough guess), does any other
one also notice the performance issues, please ? I will go on
investigating on it.

Any suggestion is appreciated.

--
Thanks

Robert

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Bob Friesenhahn]

On Mon, 2 Feb 2015, Robert Yang wrote:



Hello libtool,

It seems that libtool (2.4.4) has increased the packages build time, after
a rough investigation, it maybe because new libtoolize needs run a m4
command:


Did you time 'libtoolize' and compare the timings with 2.4.4 to the 
release you were using before?


Libtoolize is not normally considered to be part of package build time 
since packages usually already come 'libtoolized' (using some random 
version of libtool).


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-02 at 08:19 -0600, Bob Friesenhahn wrote:
> On Mon, 2 Feb 2015, Robert Yang wrote:
> > It seems that libtool (2.4.4) has increased the packages build time, after
> > a rough investigation, it maybe because new libtoolize needs run a m4
> > command:
> 
> Did you time 'libtoolize' and compare the timings with 2.4.4 to the 
> release you were using before?
> 
> Libtoolize is not normally considered to be part of package build time 
> since packages usually already come 'libtoolized' (using some random 
> version of libtool).

I work with Robert as part of the Yocto Project (powered by OpenEmbedded
underneath). The system can build Linux from scratch for a variety of
different architectures. As part of the process we autoreconf the
software so that any changes we make to support new architectures and so
on are applied as standard everywhere. We do patch libtool to deal with
some sysroot issues I've mentioned here in the past for example.

With 2.4.3, we saw a build time of 1 hour 12 minutes for one of our
standard test targets. With 2.4.4, this increased to 1 hour 23 minutes
and we've confirmed it is this specific change. We're in the process of
testing 2.4.5 but looking at the changes, I'm not sure this will help.

As yet we've not gotten to the point of isolating the problem to
libtoolize (that is the next step) but there is a noticeable regression
in the speed of things :(. 

Put in other terms why would libtool need 15% of the time needed to
build a whole Linux OS?! :)

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

On 02/02/2015 10:19 PM, Bob Friesenhahn wrote:

On Mon, 2 Feb 2015, Robert Yang wrote:



Hello libtool,

It seems that libtool (2.4.4) has increased the packages build time, after
a rough investigation, it maybe because new libtoolize needs run a m4
command:


Did you time 'libtoolize' and compare the timings with 2.4.4 to the release you
were using before?

Libtoolize is not normally considered to be part of package build time since
packages usually already come 'libtoolized' (using some random version of 
libtool).

Bob


Sorry, I was wrong, tt seems that libtoolize is not the key, but libtool,
when compile cairo-1.12.18:

libtool 2.4.2   libtool 2.4.5
configure:  31s 32s
compile:54s 64s

The libtool 2.4.5 costs 10 more seconds. I'm debugging on it, any suggestion
is appreciated.

// Robert

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Bob Friesenhahn]

On Tue, 3 Feb 2015, Robert Yang wrote:


Sorry, I was wrong, tt seems that libtoolize is not the key, but libtool,
when compile cairo-1.12.18:

libtool 2.4.2   libtool 2.4.5
configure:  31s 32s
compile:54s 64s

The libtool 2.4.5 costs 10 more seconds. I'm debugging on it, any suggestion
is appreciated.


The build slowdown must not be my imagination then.  I had attributed 
the slowdown to other factors.  Recently I noticed that build times 
for my own project were significantly longer than I remember.


Previously libtool was heavily optimized to reduce the number of forks 
(execution of child process).  This optimization should still be 
present in 2.4.2.  Any additional forks will result in a slower build.


It is not necessary to libtoolize a package in order to test its build 
performance with different libtool versions.  You can specify LIBTOOL 
in the environment (see the Makefiles) such that it points to the 
uninstalled libtool in a libtool build tree.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

On 02/03/2015 10:31 PM, Bob Friesenhahn wrote:

On Tue, 3 Feb 2015, Robert Yang wrote:


Sorry, I was wrong, tt seems that libtoolize is not the key, but libtool,
when compile cairo-1.12.18:

libtool 2.4.2libtool 2.4.5
configure: 31s32s
compile:54s64s

The libtool 2.4.5 costs 10 more seconds. I'm debugging on it, any suggestion
is appreciated.


The build slowdown must not be my imagination then.  I had attributed the
slowdown to other factors.  Recently I noticed that build times for my own
project were significantly longer than I remember.

Previously libtool was heavily optimized to reduce the number of forks
(execution of child process).  This optimization should still be present in
2.4.2.  Any additional forks will result in a slower build.

It is not necessary to libtoolize a package in order to test its build
performance with different libtool versions.  You can specify LIBTOOL in the
environment (see the Makefiles) such that it points to the uninstalled libtool
in a libtool build tree.


We run autoreconf for the package automatically to make sure that we can
build most of the autotools packages successfully, and autoreconf will
run libtoolize when needed, if we don't run autoreconf, then a few of the
packages will fail to build, now we upgrade libtool from 2.4.2 to 2.4.4/2.4.5,
it takes more time to build, I will try 2.4.3.

// Robert



Bob


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

Hello,

I've found apart of the reason, for xz:
libtool 2.4.2  2.4.5(before patched)   (after patched)
make -j19s   19s11s

Remove the following lines from libtool, then the compile will be
much faster, the problem is the variable like $host, $SHELL in
the long_help_message would make shell very slow:

=== Removed lines ===
# Additional text appended to 'usage_message' in response to '--help'. 


long_help_message=$long_help_message"

MODE must be one of the following:

   clean   remove files from the build directory
   compile compile a source file into a libtool object
   execute automatically set library path, then run a program
   finish  complete the installation of libtool libraries
   install install libraries or executables
   linkcreate a library or an executable
   uninstall   remove libraries from an installed directory

MODE-ARGS vary depending on the MODE.  When passed as first option,
'--mode=MODE' may be abbreviated as 'MODE' or a unique abbreviation of that.
Try '$progname --help --mode=MODE' for a more detailed description of MODE.

When reporting a bug, please describe a test case to reproduce it and
include the following information:

   host-triplet:   $host
   shell:  $SHELL
   compiler:   $LTCC
   compiler flags: $LTCFLAGS
   linker: $LD (gnu? $with_gnu_ld)
   version:$progname (GNU libtool) 2.4.5
   automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
   autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`

Report bugs to .
GNU libtool home page: .
General help using GNU software: ."
==

I think that another 2s gaps is caused by:
func_hookable func_options
func_options ()
{
$debug_cmd

func_options_prep ${1+"$@"}
eval func_parse_options \
${func_options_prep_result+"$func_options_prep_result"}
eval func_validate_options \
${func_parse_options_result+"$func_parse_options_result"}

eval func_run_hooks func_options \
${func_validate_options_result+"$func_validate_options_result"}

# save modified positional parameters for caller
func_options_result=$func_run_hooks_result
}


mainly because of:
eval func_parse_options \
${func_options_prep_result+"$func_options_prep_result"}

I'm still trying to figure out why.

// Robert

On 02/04/2015 09:11 AM, Robert Yang wrote:



On 02/03/2015 10:31 PM, Bob Friesenhahn wrote:

On Tue, 3 Feb 2015, Robert Yang wrote:


Sorry, I was wrong, tt seems that libtoolize is not the key, but libtool,
when compile cairo-1.12.18:

libtool 2.4.2libtool 2.4.5
configure: 31s32s
compile:54s64s

The libtool 2.4.5 costs 10 more seconds. I'm debugging on it, any suggestion
is appreciated.


The build slowdown must not be my imagination then.  I had attributed the
slowdown to other factors.  Recently I noticed that build times for my own
project were significantly longer than I remember.

Previously libtool was heavily optimized to reduce the number of forks
(execution of child process).  This optimization should still be present in
2.4.2.  Any additional forks will result in a slower build.

It is not necessary to libtoolize a package in order to test its build
performance with different libtool versions.  You can specify LIBTOOL in the
environment (see the Makefiles) such that it points to the uninstalled libtool
in a libtool build tree.


We run autoreconf for the package automatically to make sure that we can
build most of the autotools packages successfully, and autoreconf will
run libtoolize when needed, if we don't run autoreconf, then a few of the
packages will fail to build, now we upgrade libtool from 2.4.2 to 2.4.4/2.4.5,
it takes more time to build, I will try 2.4.3.

// Robert



Bob


___
https://lists.gnu.org/mailman/listinfo/libtool




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Bob Friesenhahn]

On Wed, 4 Feb 2015, Robert Yang wrote:


When reporting a bug, please describe a test case to reproduce it and
include the following information:

  host-triplet:   $host
  shell:  $SHELL
  compiler:   $LTCC
  compiler flags: $LTCFLAGS
  linker: $LD (gnu? $with_gnu_ld)
  version:$progname (GNU libtool) 2.4.5
  automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
  autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`


Perhaps libtool is accidentially executing 'automake --version' and 
'autoconf --version' every time it is executed?  That would certainly 
lead to a huge slowdown.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

On 02/04/2015 10:48 PM, Bob Friesenhahn wrote:

On Wed, 4 Feb 2015, Robert Yang wrote:


When reporting a bug, please describe a test case to reproduce it and
include the following information:

  host-triplet:   $host
  shell:  $SHELL
  compiler:   $LTCC
  compiler flags: $LTCFLAGS
  linker: $LD (gnu? $with_gnu_ld)
  version:$progname (GNU libtool) 2.4.5
  automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
  autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`


Perhaps libtool is accidentially executing 'automake --version' and 'autoconf
--version' every time it is executed?  That would certainly lead to a huge
slowdown.


Thanks, yes, you are right, after remove the two lines, the time is the
same as remove the whole block (11s).

// Robert



Bob


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

Hello,

After removing the two lines, libtool 2.4.5 is a little slower than 2.4.2
when using /bin/dash, but it is much slower than in bash, here is the
data when compile xz: (make -j1)

  lt-2.4.2  lt-2.4.5(before patched)  lt-2.4.5(after patched)
dash 11s  21s13s
bash 13s  30s21s


We can see that dash is much faster than bash. (note, the data I had reported
before for lt-2.4.5 after patched is 11s, but today is 13s, I think that
it is because of the free disk space, the data is very stable in each day,
I tried them several times).

// Robert

On 02/04/2015 10:56 PM, Robert Yang wrote:



On 02/04/2015 10:48 PM, Bob Friesenhahn wrote:

On Wed, 4 Feb 2015, Robert Yang wrote:


When reporting a bug, please describe a test case to reproduce it and
include the following information:

  host-triplet:   $host
  shell:  $SHELL
  compiler:   $LTCC
  compiler flags: $LTCFLAGS
  linker: $LD (gnu? $with_gnu_ld)
  version:$progname (GNU libtool) 2.4.5
  automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
  autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`


Perhaps libtool is accidentially executing 'automake --version' and 'autoconf
--version' every time it is executed?  That would certainly lead to a huge
slowdown.


Thanks, yes, you are right, after remove the two lines, the time is the
same as remove the whole block (11s).

// Robert



Bob


___
https://lists.gnu.org/mailman/listinfo/libtool




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Bob Friesenhahn]

I am not seeing quite the difference between libtool releases that you 
are although I see a big slowdown starting with 2.4.3.  These timings 
are for optimized builds of GraphicsMagick on a 12-core GNU/Linux 
system using -j 12:


2.4.2 : 23.613
2.4.3 : 31.697
2.4.4 : 28.236
2.4.5 : 28.514

And here are timings for a non-optimize (-O0) compile with no debug 
symbols:


2.4.2 :  8.629
2.4.3 : 13.216
2.4.4 : 13.012
2.4.5 : 13.281

I see similar slowdowns on an Illumos-based system, so it is not 
specific to GNU/Linux.


It is curious that there is more impact for the optimized builds.

Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:

I am not seeing quite the difference between libtool releases that you are
although I see a big slowdown starting with 2.4.3.  These timings are for
optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 12:


I think that we can't see obviously slowdown by "make -jN",
but "make -j1" will. And bash is much slower than dash, I'm trying to
figure out why.



2.4.2 : 23.613
2.4.3 : 31.697
2.4.4 : 28.236
2.4.5 : 28.514

And here are timings for a non-optimize (-O0) compile with no debug symbols:

2.4.2 :  8.629
2.4.3 : 13.216
2.4.4 : 13.012
2.4.5 : 13.281

I see similar slowdowns on an Illumos-based system, so it is not specific to
GNU/Linux.


I'm using Ubuntu 12.04.3, we have added a few patches to libtool for cross
building, but it seems these patches won't impact the speed so much since it
works well when 2.4.2 (the same patches).

// Robert



It is curious that there is more impact for the optimized builds.

Bob


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Peter Johansson]

On 02/06/2015 02:12 PM, Bob Friesenhahn wrote:
It is curious that there is more impact for the optimized builds. 
Well, it's a constant 5s impact for both optimized and non-optimized - 
except for v2.4.3 which looks odd.


Cheers,

--
Peter Johansson


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Peter Rosin]

On 2015-02-04 15:48, Bob Friesenhahn wrote:
> On Wed, 4 Feb 2015, Robert Yang wrote:
> 
>> When reporting a bug, please describe a test case to reproduce it and
>> include the following information:
>>
>>   host-triplet:   $host
>>   shell:  $SHELL
>>   compiler:   $LTCC
>>   compiler flags: $LTCFLAGS
>>   linker: $LD (gnu? $with_gnu_ld)
>>   version:$progname (GNU libtool) 2.4.5
>>   automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
>>   autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`
> 
> Perhaps libtool is accidentially executing 'automake --version' and 'autoconf 
> --version' every time it is executed?  That would certainly lead to a huge 
> slowdown.

That's it of course, how else could the variable be assigned?

But is it even useful information? I would expect that the autofoo
versions *at the time the package was created* is what matters?
Presenting some random autofoo version from the machine that runs
the libtool script is just bogus. The relevant versions are
certainly known when automake/autoconf are executed, but ever
since the ltmain.m4sh->ltmain.sh step is gone, is there even
a natural place for libtool to get to that info?

ho hum

The autoconf version can be grabbed from first lines of the
configure script, which certainly can be made available to the
ltmain.sh->libtool step. But how to get to the relevant
automake version? If automake is even used for the package in
question...

Cheers,
Peter


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Gary V. Vaughan]

Hi Peter,

> On Feb 6, 2015, at 9:22 AM, Peter Rosin  wrote:
> 
>> On 2015-02-04 15:48, Bob Friesenhahn wrote:
>>> On Wed, 4 Feb 2015, Robert Yang wrote:
>>> 
>>> When reporting a bug, please describe a test case to reproduce it and
>>> include the following information:
>>> 
>>>  host-triplet:   $host
>>>  shell:  $SHELL
>>>  compiler:   $LTCC
>>>  compiler flags: $LTCFLAGS
>>>  linker: $LD (gnu? $with_gnu_ld)
>>>  version:$progname (GNU libtool) 2.4.5
>>>  automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
>>>  autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`
>> 
>> Perhaps libtool is accidentially executing 'automake --version' and 
>> 'autoconf --version' every time it is executed?  That would certainly lead 
>> to a huge slowdown.
> 
> That's it of course, how else could the variable be assigned?

Only when --version is being serviced.

> But is it even useful information? I would expect that the autofoo
> versions *at the time the package was created* is what matters?

The information is useful in bug reports, and our instructions for reporting a 
bug to the list explicitly ask for the output from `libtool --version` which by 
including their other autotool versions makes reproducing the reporters 
environment a lot easier :-)

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Peter Rosin]

On 2015-02-06 10:30, Gary V. Vaughan wrote:
> Hi Peter,
> 
>> On Feb 6, 2015, at 9:22 AM, Peter Rosin  wrote:
>>
>>> On 2015-02-04 15:48, Bob Friesenhahn wrote:
 On Wed, 4 Feb 2015, Robert Yang wrote:

 When reporting a bug, please describe a test case to reproduce it and
 include the following information:

  host-triplet:   $host
  shell:  $SHELL
  compiler:   $LTCC
  compiler flags: $LTCFLAGS
  linker: $LD (gnu? $with_gnu_ld)
  version:$progname (GNU libtool) 2.4.5
  automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
  autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`
>>>
>>> Perhaps libtool is accidentially executing 'automake --version' and 
>>> 'autoconf --version' every time it is executed?  That would certainly lead 
>>> to a huge slowdown.
>>
>> That's it of course, how else could the variable be assigned?
> 
> Only when --version is being serviced.

Are you saying the a script with this line in it:
foo="`($AUTOCONF --version)`"
will delay the exec of $AUTOCONF until foo is expanded?

I think not.

>> But is it even useful information? I would expect that the autofoo
>> versions *at the time the package was created* is what matters?
> 
> The information is useful in bug reports, and our instructions for reporting 
> a bug to the list explicitly ask for the output from `libtool --version` 
> which by including their other autotool versions makes reproducing the 
> reporters environment a lot easier :-)

But are the autofoo versions at libtool time really what we want
to know in bug reports? Again, I'd be much more interested in the
autofoo versions used to bootstrap the package. That might often
be the same thing, but when they are not confusion will ensue.

Cheers,
Peter


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Gary V. Vaughan]

Hi Peter,

On Feb 6, 2015, at 9:46 AM, Peter Rosin  wrote:
> On 2015-02-06 10:30, Gary V. Vaughan wrote:
>>> On Feb 6, 2015, at 9:22 AM, Peter Rosin  wrote:
>>> 
 On 2015-02-04 15:48, Bob Friesenhahn wrote:
> On Wed, 4 Feb 2015, Robert Yang wrote:
> 
> When reporting a bug, please describe a test case to reproduce it and
> include the following information:
> 
> host-triplet:   $host
> shell:  $SHELL
> compiler:   $LTCC
> compiler flags: $LTCFLAGS
> linker: $LD (gnu? $with_gnu_ld)
> version:$progname (GNU libtool) 2.4.5
> automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
> autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`
 
 Perhaps libtool is accidentially executing 'automake --version' and 
 'autoconf --version' every time it is executed?  That would certainly lead 
 to a huge slowdown.
>>> 
>>> That's it of course, how else could the variable be assigned?
>> 
>> Only when --version is being serviced.
> 
> Are you saying the a script with this line in it:
>   foo="`($AUTOCONF --version)`"
> will delay the exec of $AUTOCONF until foo is expanded?
> 
> I think not.


I mean a script with this function in it won't run '$AUTOCONF --version' until 
and unless `libtool --help` is executed from the command line:

func_help ()
{
$debug_cmd

func_usage_message
$ECHO "$long_help_message
...
   automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q` 
   
   autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q` 

  
...
"
exit 0
}

Which is what I plan to commit before the next release.

>>> But is it even useful information? I would expect that the autofoo
>>> versions *at the time the package was created* is what matters?
>> 
>> The information is useful in bug reports, and our instructions for reporting 
>> a bug to the list explicitly ask for the output from `libtool --version` 
>> which by including their other autotool versions makes reproducing the 
>> reporters environment a lot easier :-)
> 
> But are the autofoo versions at libtool time really what we want
> to know in bug reports? Again, I'd be much more interested in the
> autofoo versions used to bootstrap the package. That might often
> be the same thing, but when they are not confusion will ensue.

Please propose (or commit!) a patch that fixes the regression, and also 
precisely demonstrates what you prefer :-)

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Tom Ghyselinck]

Hi everybody,

On vr, 2015-02-06 at 10:46 +0100, Peter Rosin wrote: 
> On 2015-02-06 10:30, Gary V. Vaughan wrote:
> > Hi Peter,
> > 
> >> On Feb 6, 2015, at 9:22 AM, Peter Rosin  wrote:
> >>
> >>> On 2015-02-04 15:48, Bob Friesenhahn wrote:
>  On Wed, 4 Feb 2015, Robert Yang wrote:
> 
>  When reporting a bug, please describe a test case to reproduce it and
>  include the following information:
> 
>   host-triplet:   $host
>   shell:  $SHELL
>   compiler:   $LTCC
>   compiler flags: $LTCFLAGS
>   linker: $LD (gnu? $with_gnu_ld)
>   version:$progname (GNU libtool) 2.4.5
>   automake:   `($AUTOMAKE --version) 2>/dev/null |$SED 1q`
>   autoconf:   `($AUTOCONF --version) 2>/dev/null |$SED 1q`
> >>>
> >>> Perhaps libtool is accidentially executing 'automake --version' and 
> >>> 'autoconf --version' every time it is executed?  That would certainly 
> >>> lead to a huge slowdown.
> >>
> >> That's it of course, how else could the variable be assigned?
> > 
> > Only when --version is being serviced.
> 
> Are you saying the a script with this line in it:
>   foo="`($AUTOCONF --version)`"
> will delay the exec of $AUTOCONF until foo is expanded?
> 
> I think not.

No, shell variable expansion is not delayed.

Unlikey when used in `Makefile` with GNU make, where this *is* delayed
until full expansion.

> 
> >> But is it even useful information? I would expect that the autofoo
> >> versions *at the time the package was created* is what matters?
> > 
> > The information is useful in bug reports, and our instructions for 
> > reporting a bug to the list explicitly ask for the output from `libtool 
> > --version` which by including their other autotool versions makes 
> > reproducing the reporters environment a lot easier :-)
> 
> But are the autofoo versions at libtool time really what we want
> to know in bug reports? Again, I'd be much more interested in the
> autofoo versions used to bootstrap the package. That might often
> be the same thing, but when they are not confusion will ensue.

AFAIK: Doesn't automake --version depend on the project's configure.ac?
At least some distribution have an "automake-wrapper" which selects
the automake version based on the project settings and fall-back to
system default.

e.g. Running "automake" (i.e. "automake-wrapper") when you have
"automake-1.10", "automake-1.11" and "automake-1.12" installed:

- Suppose system-default is automake-1.11

- When executed in directory *without* `configure.ac`: automake will run
automake-1.11.

- When executed in directory *with* configure.ac:
  - When automake version is *defined* to '1.10' in `configure.ac`,
it will run "automake-1.10"
  - When automake version is *defined* to '1.12' in `configure.ac`,
it will run "automake-1.12"
  - When automake version is *not defined* in `configure.ac`,
it will run "automake-1.11"

With best regards,
Tom.


> 
> Cheers,
> Peter
> 
> 
> ___
> https://lists.gnu.org/mailman/listinfo/libtool

-- 




| tom.ghyseli...@excentis.com
|
| Tom Ghyselinck
| Senior Engineer
| Excentis N.V.
| Gildestraat 8 B-9000 Ghent, Belgium
| Tel: +32 9 269 22 91 - Fax: +32 9 329 31 74




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Bob Friesenhahn]

On Fri, 6 Feb 2015, Robert Yang wrote:




On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:

I am not seeing quite the difference between libtool releases that you are
although I see a big slowdown starting with 2.4.3.  These timings are for
optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 
12:


I think that we can't see obviously slowdown by "make -jN",
but "make -j1" will. And bash is much slower than dash, I'm trying to
figure out why.


It seems like this issue is already corrected in the source tree but 
you are correct that the issue is much more apparent with -j1.


Optimized:

  2.4.2 : 3:43.43
  2.4.5 : 4:33.11

Non-Optimized:

  2.4.2 : 1:21.21
  2.4.5 : 2:05.48

We need a test case which is executed before every major release to 
make sure that a performance regression has not been introduced.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Robert Yang]

On 02/06/2015 10:46 PM, Bob Friesenhahn wrote:

On Fri, 6 Feb 2015, Robert Yang wrote:




On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:

I am not seeing quite the difference between libtool releases that you are
although I see a big slowdown starting with 2.4.3.  These timings are for
optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 12:


I think that we can't see obviously slowdown by "make -jN",
but "make -j1" will. And bash is much slower than dash, I'm trying to
figure out why.


It seems like this issue is already corrected in the source tree but you are


Yes, I think that the git repo has fixed the problem:

commit 408cfb9c5fa8a666917167ffb806cb19deded429
Author: Gary V. Vaughan 
Date:   Fri Feb 6 12:58:34 2015 +

libtool: don't execute automake and autoconf on every invocation.

// Robert


correct that the issue is much more apparent with -j1.

Optimized:

   2.4.2 : 3:43.43
   2.4.5 : 4:33.11

Non-Optimized:

   2.4.2 : 1:21.21
   2.4.5 : 2:05.48

We need a test case which is executed before every major release to make sure
that a performance regression has not been introduced.

Bob


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 10:45 +0800, Robert Yang wrote:
> On 02/06/2015 10:46 PM, Bob Friesenhahn wrote:
> > On Fri, 6 Feb 2015, Robert Yang wrote:
> >
> >>
> >>
> >> On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:
> >>> I am not seeing quite the difference between libtool releases that you are
> >>> although I see a big slowdown starting with 2.4.3.  These timings are for
> >>> optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 
> >>> 12:
> >>
> >> I think that we can't see obviously slowdown by "make -jN",
> >> but "make -j1" will. And bash is much slower than dash, I'm trying to
> >> figure out why.
> >
> > It seems like this issue is already corrected in the source tree but you are
> 
> Yes, I think that the git repo has fixed the problem:
> 
> commit 408cfb9c5fa8a666917167ffb806cb19deded429
> Author: Gary V. Vaughan 
> Date:   Fri Feb 6 12:58:34 2015 +
> 
>  libtool: don't execute automake and autoconf on every invocation.

In an effort to get to the bottom of this I made a git bisection, timing
the performance of building xz with make -j1 using each different
libtool.

The issues come down to this commit:

http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430

libtool: rewritten over funclib.sh instead of general.m4sh.

Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
in the fix in master mentioned above, I get 27s.

So whilst things are better (thanks!), the above change is still causing
a regression in the performance somewhere else. Any ideas what else in
that rather large change may be causing this?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Peter Rosin]

On 2015-02-09 14:05, Richard Purdie wrote:
> On Mon, 2015-02-09 at 10:45 +0800, Robert Yang wrote:
>> On 02/06/2015 10:46 PM, Bob Friesenhahn wrote:
>>> On Fri, 6 Feb 2015, Robert Yang wrote:
>>>


 On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:
> I am not seeing quite the difference between libtool releases that you are
> although I see a big slowdown starting with 2.4.3.  These timings are for
> optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 
> 12:

 I think that we can't see obviously slowdown by "make -jN",
 but "make -j1" will. And bash is much slower than dash, I'm trying to
 figure out why.
>>>
>>> It seems like this issue is already corrected in the source tree but you are
>>
>> Yes, I think that the git repo has fixed the problem:
>>
>> commit 408cfb9c5fa8a666917167ffb806cb19deded429
>> Author: Gary V. Vaughan 
>> Date:   Fri Feb 6 12:58:34 2015 +
>>
>>  libtool: don't execute automake and autoconf on every invocation.
> 
> In an effort to get to the bottom of this I made a git bisection, timing
> the performance of building xz with make -j1 using each different
> libtool.
> 
> The issues come down to this commit:
> 
> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> 
> libtool: rewritten over funclib.sh instead of general.m4sh.
> 
> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> in the fix in master mentioned above, I get 27s.
> 
> So whilst things are better (thanks!), the above change is still causing
> a regression in the performance somewhere else. Any ideas what else in
> that rather large change may be causing this?

IIRC, there was prior work to make func_compile appear as early as
possible. With this change, there are a lot of functions to parse
before getting to func_compile (and all of them are not needed that
early in the script, I think, but I have not verified that). Maybe
that can explain the difference? You could try to manually move
functions that are not needed by func_compile to appear after
func_compile and see if that makes a difference.

Cheers,
Peter


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
> In an effort to get to the bottom of this I made a git bisection, timing
> the performance of building xz with make -j1 using each different
> libtool.
> 
> The issues come down to this commit:
> 
> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> 
> libtool: rewritten over funclib.sh instead of general.m4sh.
> 
> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> in the fix in master mentioned above, I get 27s.
> 
> So whilst things are better (thanks!), the above change is still causing
> a regression in the performance somewhere else. Any ideas what else in
> that rather large change may be causing this?

To further narrow this down, of the changes in the above commit, the
problem appears to be in the changes to the option parsing code. I've
included the diff below which if I apply on top of the above, I get the
speed back. I've left the func_split_short_opt/func_split_long_opt code
in there but that is worth a tiny part of the speed, the issues are
around the addition of the func_options call.

As yet I don't know enough about the code in question to know why this
is an issue but traces of libtool show a lot more looping in code to do
with argument parsing and quoting.

Cheers,

Richard



--- libtool-bad 2015-02-09 16:34:30.702068736 +
+++ libtool 2015-02-09 23:31:37.238750764 +
@@ -791,13 +791,77 @@
 fi
 }
 
+exit_status=$EXIT_SUCCESS
 
-# libtool_options_prep [ARG]...
-# -
-# Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
-$debug_mode
+# We should try to minimise forks, especially on Windows where they are
+# unreasonably slow, so skip the feature probes when bash or zsh are
+# being used:
+if test set = "${BASH_VERSION+set}$ZSH_VERSION"; then
+: ${lt_HAVE_ARITH_OP="yes"}
+: ${lt_HAVE_XSI_OPS="yes"}
+fi
+
+
+
+# lt_HAVE_XSI_OPS
+# Can be empty, in which case the shell is probed, "yes" if XSI length
+# and matching operators are useable or anything else if they do not work.
+test -z "$lt_HAVE_XSI_OPS" \
+&& (eval 'x=a/b/c;
+  test 5aa/bb/cc = "${#x}${x%%/*}${x%/*}${x#*/}${x##*/}"') 2>/dev/null \
+&& lt_HAVE_XSI_OPS=yes
+
+
+
+# If this shell supports prefix and suffix pattern removal, then
+# use them to avoid forking. Hide the definition in an eval in case
+# the shell chokes on unsupported syntax...
+if test yes = "$lt_HAVE_XSI_OPS"; then
+  # func_split_short_opt shortopt
+  # Set func_split_short_opt_name and func_split_short_opt_arg shell
+  # variables after splitting SHORTOPT after the 2nd character.
+  eval 'func_split_short_opt ()
+  {
+$debug_cmd
+
+func_split_short_opt_arg=${1#??}
+func_split_short_opt_name=${1%"$func_split_short_opt_arg"}
+  }'
+
+  # func_split_long_opt longopt
+  # Set func_split_long_opt_name and func_split_long_opt_arg shell
+  # variables after splitting LONGOPT at the `=' sign.
+  eval 'func_split_long_opt ()
+  {
+func_split_long_opt_name=${1%%=*}
+func_split_long_opt_arg=${1#*=}
+  }'
+else
+  # ...otherwise fall back to using sed.
+  func_split_short_opt ()
+  {
+my_sed_short_opt='1s/^\(..\).*$/\1/;q'
+my_sed_short_rest='1s/^..\(.*\)$/\1/;q'
+
+func_split_short_opt_name=`$ECHO "$1" | $SED "$my_sed_short_opt"`
+func_split_short_opt_arg=`$ECHO "$1" | $SED "$my_sed_short_rest"`
+  }
+
+  func_split_long_opt ()
+  {
+my_sed_long_opt='1s/^\(--[^=]*\)=.*/\1/;q'
+my_sed_long_arg='1s/^--[^=]*=//'
+
+func_split_long_opt_name=`$ECHO "$1" | $SED "$my_sed_long_opt"`
+func_split_long_opt_arg=`$ECHO "$1" | $SED "$my_sed_long_arg"`
+  }
+fi
+
+debug_cmd=${debug_cmd-':'}
+opt_warning=:
+opt_verbose=:
+opt_verbose=false
+exit_cmd=:
 
 # Option defaults:
 opt_config=false
@@ -836,26 +900,23 @@
   ;;
 esac
 
-# Pass back the list of options.
-func_quote_for_eval ${1+"$@"}
-libtool_options_prep_result=$func_quote_for_eval_result
-}
-func_add_hook func_options_prep libtool_options_prep
 
 
-# libtool_parse_options [ARG]...
-# -
-# Provide handling for libtool specific options.
-libtool_parse_options ()
+# Parse options once, thoroughly.  This comes as soon as possible in the
+# script to make things like `--version' happen as quickly as we can.
 {
-$debug_cmd
-
 # Perform our own loop to consume as many options as possible in
 # each iteration.
 while test $# -gt 0; do
   _G_opt=$1
   shift
   case $_G_opt in
+
+--debug|-x)debug_cmd='set -x'
+   func_echo "enabling shell trace mode"
+   $debug_cmd
+   ;;
+
 --dry-run|--dryrun|-n)
 opt_dry_run=:
 ;;
@@ -927,29 +988,39 @@
 func_append preserve_args " $_G_opt"
 ;;
 
-   # An option not handled by this h

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
> On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
> > In an effort to get to the bottom of this I made a git bisection, timing
> > the performance of building xz with make -j1 using each different
> > libtool.
> > 
> > The issues come down to this commit:
> > 
> > http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> > 
> > libtool: rewritten over funclib.sh instead of general.m4sh.
> > 
> > Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> > in the fix in master mentioned above, I get 27s.
> > 
> > So whilst things are better (thanks!), the above change is still causing
> > a regression in the performance somewhere else. Any ideas what else in
> > that rather large change may be causing this?
> 
> To further narrow this down, of the changes in the above commit, the
> problem appears to be in the changes to the option parsing code. I've
> included the diff below which if I apply on top of the above, I get the
> speed back. I've left the func_split_short_opt/func_split_long_opt code
> in there but that is worth a tiny part of the speed, the issues are
> around the addition of the func_options call.
> 
> As yet I don't know enough about the code in question to know why this
> is an issue but traces of libtool show a lot more looping in code to do
> with argument parsing and quoting.

To be more specific, if I take my "good" libtool and add:

func_options_prep ${1+"$@"}

it slows the build down by 0.5s on a 21s build. If I look at
func_options_prep and comment out the line:

func_run_hooks func_options_prep ${1+"$@"}

I get the 0.5s back.

In func_run_hooks, if I comment:

func_quote_for_eval ${1+"$@"}
func_run_hooks_result=$func_quote_for_eval_result

I get the 0.5s back. The issue is all the quoting of the various return
values through all this looping. It doesn't appear to be hitting the
printf/sed in func_quote_for_eval which would be an obvious slow path,
its just the shear number of loops run through with the commandline
arguments. The change adds a number of calls to func_run_hooks, not just
the single test case I have above and all combined, it slows things down
significantly.

So is there a way we can change things so its not calling
func_quote_for_eval all the time with all the looping that entails?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Gary V. Vaughan]

Hi Richard,

> On Feb 9, 2015, at 11:36 PM, Richard Purdie 
>  wrote:
> 
> On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
>> In an effort to get to the bottom of this I made a git bisection, timing
>> the performance of building xz with make -j1 using each different
>> libtool.
>> 
>> The issues come down to this commit:
>> 
>> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
>> 
>> libtool: rewritten over funclib.sh instead of general.m4sh.
>> 
>> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
>> in the fix in master mentioned above, I get 27s.
>> 
>> So whilst things are better (thanks!), the above change is still causing
>> a regression in the performance somewhere else. Any ideas what else in
>> that rather large change may be causing this?
> 
> To further narrow this down, of the changes in the above commit, the
> problem appears to be in the changes to the option parsing code. I've
> included the diff below which if I apply on top of the above, I get the
> speed back. I've left the func_split_short_opt/func_split_long_opt code
> in there but that is worth a tiny part of the speed, the issues are
> around the addition of the func_options call.

Thanks for the analysis and patch.

> As yet I don't know enough about the code in question to know why this
> is an issue but traces of libtool show a lot more looping in code to do
> with argument parsing and quoting.

For background, one of the larger changes between 2.4.2 and 2.4.3 was to
move from separately maintained options code in raw shell in libtoolize and
generated by m4 in libtool, to the unified generalized shell options parser
from bootstrap, kept in gl/build-aux/options-parser.  The benefits of this
include time saved in maintenance of only one options parsing implementation
over three separate ones (I maintain bootstrap too), and a lot more flexibility
in the shell function hooking code -- ultimately, this will enable splitting
libltdl into a separate project, which can inject ltdl specific code into
the already installed libtoolize script.

I was aware this would result in a somewhat slower libtool, though I'm
surprised that it is as large a difference as you have timed.  I'd be very
happy to find a way to patch gl/build-aux/option-parser or its clients to
regain some of that speed, but I'm extremely reluctant to revert to the
hokey mishmash of m4 and shell that 2.4.2 was using purely for the sake of
speed -- it might be that for extremely large projects you'll want to stick
with 2.4.2 until we find a way to restore the necessary speed in some future
release?

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Gary V. Vaughan]

Hi Richard,

On Feb 10, 2015, at 10:35 AM, Richard Purdie 
 wrote:
> On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
>> On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
>>> In an effort to get to the bottom of this I made a git bisection, timing
>>> the performance of building xz with make -j1 using each different
>>> libtool.
>>> 
>>> The issues come down to this commit:
>>> 
>>> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
>>> 
>>> libtool: rewritten over funclib.sh instead of general.m4sh.
>>> 
>>> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
>>> in the fix in master mentioned above, I get 27s.
>>> 
>>> So whilst things are better (thanks!), the above change is still causing
>>> a regression in the performance somewhere else. Any ideas what else in
>>> that rather large change may be causing this?
>> 
>> To further narrow this down, of the changes in the above commit, the
>> problem appears to be in the changes to the option parsing code. I've
>> included the diff below which if I apply on top of the above, I get the
>> speed back. I've left the func_split_short_opt/func_split_long_opt code
>> in there but that is worth a tiny part of the speed, the issues are
>> around the addition of the func_options call.
>> 
>> As yet I don't know enough about the code in question to know why this
>> is an issue but traces of libtool show a lot more looping in code to do
>> with argument parsing and quoting.
> 
> To be more specific, if I take my "good" libtool and add:
> 
> func_options_prep ${1+"$@"}
> 
> it slows the build down by 0.5s on a 21s build. If I look at
> func_options_prep and comment out the line:
> 
>func_run_hooks func_options_prep ${1+"$@"}
> 
> I get the 0.5s back.
> 
> In func_run_hooks, if I comment:
> 
>func_quote_for_eval ${1+"$@"}
>func_run_hooks_result=$func_quote_for_eval_result
> 
> I get the 0.5s back. The issue is all the quoting of the various return
> values through all this looping. It doesn't appear to be hitting the
> printf/sed in func_quote_for_eval which would be an obvious slow path,
> its just the shear number of loops run through with the commandline
> arguments. The change adds a number of calls to func_run_hooks, not just
> the single test case I have above and all combined, it slows things down
> significantly.
> 
> So is there a way we can change things so its not calling
> func_quote_for_eval all the time with all the looping that entails?

One possibility would be to add a post-processing script that rewrites
the hook functions used by func_options into a a single top-level blob of
sequential shell code.  I imagine that adding some carefully chosen comment
strings would make extracting the right parts in the right order relatively
straight forward... it might even be that inlining func_options_prep and
any hook functions it calls would be enough?

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Tue, 2015-02-10 at 10:53 +, Gary V. Vaughan wrote:
> On Feb 10, 2015, at 10:35 AM, Richard Purdie 
>  wrote:
> > On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
> >> On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
> >>> In an effort to get to the bottom of this I made a git bisection, timing
> >>> the performance of building xz with make -j1 using each different
> >>> libtool.
> >>> 
> >>> The issues come down to this commit:
> >>> 
> >>> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> >>> 
> >>> libtool: rewritten over funclib.sh instead of general.m4sh.
> >>> 
> >>> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> >>> in the fix in master mentioned above, I get 27s.
> >>> 
> >>> So whilst things are better (thanks!), the above change is still causing
> >>> a regression in the performance somewhere else. Any ideas what else in
> >>> that rather large change may be causing this?
> >> 
> >> To further narrow this down, of the changes in the above commit, the
> >> problem appears to be in the changes to the option parsing code. I've
> >> included the diff below which if I apply on top of the above, I get the
> >> speed back. I've left the func_split_short_opt/func_split_long_opt code
> >> in there but that is worth a tiny part of the speed, the issues are
> >> around the addition of the func_options call.
> >> 
> >> As yet I don't know enough about the code in question to know why this
> >> is an issue but traces of libtool show a lot more looping in code to do
> >> with argument parsing and quoting.
> > 
> > To be more specific, if I take my "good" libtool and add:
> > 
> > func_options_prep ${1+"$@"}
> > 
> > it slows the build down by 0.5s on a 21s build. If I look at
> > func_options_prep and comment out the line:
> > 
> >func_run_hooks func_options_prep ${1+"$@"}
> > 
> > I get the 0.5s back.
> > 
> > In func_run_hooks, if I comment:
> > 
> >func_quote_for_eval ${1+"$@"}
> >func_run_hooks_result=$func_quote_for_eval_result
> > 
> > I get the 0.5s back. The issue is all the quoting of the various return
> > values through all this looping. It doesn't appear to be hitting the
> > printf/sed in func_quote_for_eval which would be an obvious slow path,
> > its just the shear number of loops run through with the commandline
> > arguments. The change adds a number of calls to func_run_hooks, not just
> > the single test case I have above and all combined, it slows things down
> > significantly.
> > 
> > So is there a way we can change things so its not calling
> > func_quote_for_eval all the time with all the looping that entails?
> 
> One possibility would be to add a post-processing script that rewrites
> the hook functions used by func_options into a a single top-level blob of
> sequential shell code.  I imagine that adding some carefully chosen comment
> strings would make extracting the right parts in the right order relatively
> straight forward... it might even be that inlining func_options_prep and
> any hook functions it calls would be enough?

Thanks for the background info on this. I understand the need to change
and improve the software so I'm not proposing reverting, I do think
there has to be some way to get some of the speed here back though.

I have a bit of a pressing need to have a things performing as they were
and I'd prefer to stay on libtool 2.4.5 than revert back to 2.4.2 so I
cooked up the patch I've included below. This basically manually unrolls
the key problematic parts, cut and paste from options-parser. With this
applied to master, I have a build time of 22s compared to 20s or 21s
(need to go and retest as I'm forgetting numbers) with 2.4.2.

For now I'll probably merge something like this into the Yocto
Project/OpenEmbedded, the exact way to solve this longer term is TBD.

Cheers,

Richard




diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index d5cf07a..0f54303 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -342,11 +342,15 @@ _LT_EOF
 # libtool_options_prep [ARG]...
 # -
 # Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
+#libtool_options_prep ()
+#{
 $debug_mode
 
 # Option defaults:
+opt_verbose=false
+opt_warning_types=
+
+# Option defaults:
 opt_config=false
 opt_dlopen=
 opt_dry_run=false
@@ -382,19 +386,14 @@ libtool_options_prep ()
   shift; set dummy --mode uninstall ${1+"$@"}; shift
   ;;
 esac
-
-# Pass back the list of options.
-func_quote_for_eval ${1+"$@"}
-libtool_options_prep_result=$func_quote_for_eval_result
-}
-func_add_hook func_options_prep libtool_options_prep
+#}
 
 
 # libtool_parse_options [ARG]...
 # -
 # Provide handling for libtool specific options.
-libtool_parse_options ()
-{
+#libtool_parse_options ()
+#{
 $debug_cmd
 
 # Perform our own loop to consume as many options as 

Re: Performance issue of libtool-2.4.4

[Robert Yang]

Great, I verified that we nearly get the speed back:

When build xz:
  libtool-2.4.5  libtool-2.4.2
bash:14s13s
dash:12s11s

// Robert

On 02/10/2015 06:35 PM, Richard Purdie wrote:

On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:

On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:

In an effort to get to the bottom of this I made a git bisection, timing
the performance of building xz with make -j1 using each different
libtool.

The issues come down to this commit:

http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430

libtool: rewritten over funclib.sh instead of general.m4sh.

Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
in the fix in master mentioned above, I get 27s.

So whilst things are better (thanks!), the above change is still causing
a regression in the performance somewhere else. Any ideas what else in
that rather large change may be causing this?


To further narrow this down, of the changes in the above commit, the
problem appears to be in the changes to the option parsing code. I've
included the diff below which if I apply on top of the above, I get the
speed back. I've left the func_split_short_opt/func_split_long_opt code
in there but that is worth a tiny part of the speed, the issues are
around the addition of the func_options call.

As yet I don't know enough about the code in question to know why this
is an issue but traces of libtool show a lot more looping in code to do
with argument parsing and quoting.


To be more specific, if I take my "good" libtool and add:

func_options_prep ${1+"$@"}

it slows the build down by 0.5s on a 21s build. If I look at
func_options_prep and comment out the line:

 func_run_hooks func_options_prep ${1+"$@"}

I get the 0.5s back.

In func_run_hooks, if I comment:

 func_quote_for_eval ${1+"$@"}
 func_run_hooks_result=$func_quote_for_eval_result

I get the 0.5s back. The issue is all the quoting of the various return
values through all this looping. It doesn't appear to be hitting the
printf/sed in func_quote_for_eval which would be an obvious slow path,
its just the shear number of loops run through with the commandline
arguments. The change adds a number of calls to func_run_hooks, not just
the single test case I have above and all combined, it slows things down
significantly.

So is there a way we can change things so its not calling
func_quote_for_eval all the time with all the looping that entails?

Cheers,

Richard





___
https://lists.gnu.org/mailman/listinfo/libtool