Re: 2.19.47 long compile times

[Trevor Bača]

On Mon, Oct 3, 2016 at 3:38 PM, Trevor Bača  wrote:

>
>
> On Wed, Sep 14, 2016 at 3:26 PM, Carl Sorensen  wrote:
>
>> More information (this is under OSX 10.8.5 Mountain Lion):
>>
>>
>> On 9/14/16 9:16 AM, "lilypond-devel on behalf of Trevor Bača" > 
>>
>> >7. LilyPond versions 2.19.47 and 48 are "unhealthy": these two most
>> recent
>> >versions of LilyPond appear to rebuild the font cache on *EVERY* run of
>> >LilyPond, meaning that every run of LilyPond takes 40 seconds or more.
>>
>>
>> On OSX 10.8.5 Mountain Lion, command line runs are all fast.  The first
>> one after downloading 2.19.48 was slow, but no others.
>>
>> I wonder if the new security stuff on El Capitan is changing the
>> owner/group of the font cache files, and that is giving problems.
>>
>> Has anyone done an ls -l ~/lilypond-font-cache-2 to see if the owner,
>> group, and permissions are right?
>>
>
>
> With LilyPond 2.19.46 (last healthy build under OSX 10.11 / 10.12):
>
> $ ls -l ~/.lilypond-fonts.cache-2
> total 2272
> -rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
> 84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
> -rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:15
> b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
> -rw-r--r--  1 trevorbaca  staff   23944 Sep 14 10:19
> ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7
>
>
> With LilyPond 2.19.48 ("unhealthy" build under OSX 10.11 / 10.12) :
>
> $ ls -l ~/.lilypond-fonts.cache-2
> total 2272
> -rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
> 84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
> -rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:31
> b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
> -rw-r--r--  1 trevorbaca  staff   23944 Oct  3 15:31
> ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7
>
>
> So permissions and ownership settings appear to be the same between
> 2.19.46 and 2.19.48 under OSX 10.11 (and 10.12).
>
> Note that LilyPond versions up to (and including) 2.19.46 work fine under
> 10.11; it's only versions 2.19.47 and .48 that exhibit the problem. (I.e.,
> seems unrelated to OSX version and definitely related to the two most
> recent LilyPond builds.)
>


Important CLUE:

It's definitely the case that LilyPond 2.19.47 and .48 are rebuilding (one
of three) font caches at every run.

The phenomenon is testable by reading the last modification time in the
output of `ls` after successive LilyPond runs.

EXAMPLE:

$ lilypond test.ly
< snip >

$ ls -l ~/.lilypond-fonts.cache-2
total 2272
-rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:36
b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff   23944 Oct  3 15:31
ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7

$ lilypond test.ly
< snip >

$ ls -l ~/.lilypond-fonts.cache-2
total 2272
-rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:40
b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff   23944 Oct  3 15:31
ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7



I ran LilyPond 2.19.48 twice, with 4 minutes between runs.

Notice the lines highlighted in cyan:

Time of last modification of the second of the three caches in
~/.lilypond-fonts.cache-2 increments from 15:36 to 15:40. Meaning that
LilyPond 2.19.48 is definitely rebuilding a font cache prior to each run.


-- 
Trevor Bača
www.trevorbaca.com
soundcloud.com/trevorbaca
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Trevor Bača]

On Wed, Sep 14, 2016 at 3:26 PM, Carl Sorensen  wrote:

> More information (this is under OSX 10.8.5 Mountain Lion):
>
>
> On 9/14/16 9:16 AM, "lilypond-devel on behalf of Trevor Bača"  
>
> >7. LilyPond versions 2.19.47 and 48 are "unhealthy": these two most recent
> >versions of LilyPond appear to rebuild the font cache on *EVERY* run of
> >LilyPond, meaning that every run of LilyPond takes 40 seconds or more.
>
>
> On OSX 10.8.5 Mountain Lion, command line runs are all fast.  The first
> one after downloading 2.19.48 was slow, but no others.
>
> I wonder if the new security stuff on El Capitan is changing the
> owner/group of the font cache files, and that is giving problems.
>
> Has anyone done an ls -l ~/lilypond-font-cache-2 to see if the owner,
> group, and permissions are right?
>


With LilyPond 2.19.46 (last healthy build under OSX 10.11 / 10.12):

$ ls -l ~/.lilypond-fonts.cache-2
total 2272
-rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:15
b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff   23944 Sep 14 10:19
ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7


With LilyPond 2.19.48 ("unhealthy" build under OSX 10.11 / 10.12) :

$ ls -l ~/.lilypond-fonts.cache-2
total 2272
-rw-r--r--  1 trevorbaca  staff  617944 Oct  3 15:14
84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff  518000 Oct  3 15:31
b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
-rw-r--r--  1 trevorbaca  staff   23944 Oct  3 15:31
ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7


So permissions and ownership settings appear to be the same between 2.19.46
and 2.19.48 under OSX 10.11 (and 10.12).

Note that LilyPond versions up to (and including) 2.19.46 work fine under
10.11; it's only versions 2.19.47 and .48 that exhibit the problem. (I.e.,
seems unrelated to OSX version and definitely related to the two most
recent LilyPond builds.)



-- 
Trevor Bača
www.trevorbaca.com
soundcloud.com/trevorbaca
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Carl Sorensen]

More information (this is under OSX 10.8.5 Mountain Lion):


On 9/14/16 9:16 AM, "lilypond-devel on behalf of Trevor Bača" mailto:lilypond-devel-bounces+c_sorensen=byu@gnu.org>

>7. LilyPond versions 2.19.47 and 48 are "unhealthy": these two most recent
>versions of LilyPond appear to rebuild the font cache on *EVERY* run of
>LilyPond, meaning that every run of LilyPond takes 40 seconds or more.


On OSX 10.8.5 Mountain Lion, command line runs are all fast.  The first
one after downloading 2.19.48 was slow, but no others.

I wonder if the new security stuff on El Capitan is changing the
owner/group of the font cache files, and that is giving problems.

Has anyone done an ls -l ~/lilypond-font-cache-2 to see if the owner,
group, and permissions are right?

Carl






___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Trevor Bača]

On Wed, Sep 14, 2016 at 10:02 AM, Masamichi Hosoda 
wrote:

> >> Hi,
> >>
> >> The new 2.19.47 is taking about 40 seconds to compile a one-note test
> >> file.
> >> (MacOS X x86.)
> >>
> >> In my past experience, new LilyPond builds sometimes take 30 or 40
> >> seconds
> >> to compile *on the very first run after install* (presumably to update
> >> the
> >> font cache, or something similar). But 2.19.47 is taking 40 seconds
> >> *on
> >> every run*.
> >>
> >> Trevor.
> >
> >
> > There was something _similar_ reported on Windows 10, although the
> > slow-down occurred between 19.42 and 19.43.  In this case, it appeared
> > that the problem was fixed by deleting all the files in
> > c:\users\yourusername\.lilypond-fonts.cache-2.  Perhaps a similar
> > approach will work for the Mac?
> >
> > --
> > Phil Holmes
>
> If you would like to delete the fontconfig cache files in OSX,
> cache directory might be `~/.lilypond-fonts.cache-2`.
> http://lists.gnu.org/archive/html/bug-lilypond/2016-07/msg00050.html
>
> So If I understand correctly, the following command can delete them.
>
> $ rm -fr ~/.lilypond-fonts.cache-2
>

Summary: removing the files in ~/.lilypond-fonts.cache-2 does *not* fix the
problem of ultra-slow compilation times under OSX for LilyPond versions
2.19.47 and 2.19.48.

Details:

1. The directory ~/.lilypond-fonts.cache-2 does indeed exist for OSX
LilyPond installs (analogously to Windows installs).

2. The three files resident in ~/lilypond-fonts.cache-2 for LilyPond
versions 2.19.46, 47 and 48 under OSX are:

 $ ls ~/.lilypond-fonts.cache-2/
84c0f976e30e948e99073af70f4ae876-i686-apple-darwin8.cache-7
b0a71e6bf6a8a1a908413a823d76e21f-i686-apple-darwin8.cache-7
ce2f012eabd1de3f1e401def0f51e425-i686-apple-darwin8.cache-7

3. Removing the files in ~/.lilypond-fonts.cache-2 is harmless. You can do
so using ...

   rm ~/.lilypond-fonts.cache-2/*

... as the command.

4. If you remove the files in ~/.lilypond-fonts.cache-2 then LilyPond will
repopulate the directory the next time you run LilyPond. This process
("rebuilding the font cache") takes awhile. Usually something like 40
seconds.

5. Healthy versions of LilyPond rebuild the font cache on first run (after
install) only.

6. LilyPond versions 2.19.42, 43, 44, 45 and 46 under OSX are "healthy"
according to this definition. (This differs from the situation reported by
Windows users. Windows users reported a problem starting with LilyPond
version 2.19.43.)

7. LilyPond versions 2.19.47 and 48 are "unhealthy": these two most recent
versions of LilyPond appear to rebuild the font cache on *EVERY* run of
LilyPond, meaning that every run of LilyPond takes 40 seconds or more.

It is, of course, better that these problems appear in development releases
of LilyPond (as opposed to stable releases of LilyPond) since we warn users
about potential risks with development versions of the program.

But there is still a real risk with this slow-compilation problem appearing
in development releases: many experienced users follow along with the
development releases and recompile our existing, complex scores with each
new release, effectively black box testing each new development release. As
of now I'm no longer able to do this because the slowdown is so severe that
it's now impossible for me to compose (and, therefore, test) with the two
most recent development versions.

Trevor.


-- 
Trevor Bača
www.trevorbaca.com
soundcloud.com/trevorbaca
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Masamichi Hosoda]

>> Hi,
>>
>> The new 2.19.47 is taking about 40 seconds to compile a one-note test
>> file.
>> (MacOS X x86.)
>>
>> In my past experience, new LilyPond builds sometimes take 30 or 40
>> seconds
>> to compile *on the very first run after install* (presumably to update
>> the
>> font cache, or something similar). But 2.19.47 is taking 40 seconds
>> *on
>> every run*.
>>
>> Trevor.
> 
> 
> There was something _similar_ reported on Windows 10, although the
> slow-down occurred between 19.42 and 19.43.  In this case, it appeared
> that the problem was fixed by deleting all the files in
> c:\users\yourusername\.lilypond-fonts.cache-2.  Perhaps a similar
> approach will work for the Mac?
> 
> --
> Phil Holmes 

If you would like to delete the fontconfig cache files in OSX,
cache directory might be `~/.lilypond-fonts.cache-2`.
http://lists.gnu.org/archive/html/bug-lilypond/2016-07/msg00050.html

So If I understand correctly, the following command can delete them.

$ rm -fr ~/.lilypond-fonts.cache-2

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Phil Holmes]

- Original Message - 
From: "Trevor Bača" 

To: "lilypond-devel" 
Sent: Wednesday, August 31, 2016 3:34 PM
Subject: 2.19.47 long compile times



Hi,

The new 2.19.47 is taking about 40 seconds to compile a one-note test 
file.

(MacOS X x86.)

In my past experience, new LilyPond builds sometimes take 30 or 40 seconds
to compile *on the very first run after install* (presumably to update the
font cache, or something similar). But 2.19.47 is taking 40 seconds *on
every run*.

Trevor.



There was something _similar_ reported on Windows 10, although the slow-down 
occurred between 19.42 and 19.43.  In this case, it appeared that the 
problem was fixed by deleting all the files in 
c:\users\yourusername\.lilypond-fonts.cache-2.  Perhaps a similar approach 
will work for the Mac?


--
Phil Holmes 



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Trevor Bača]

On Wed, Aug 31, 2016 at 10:40 AM, David Kastrup  wrote:

> Trevor Bača  writes:
>
> > Hi,
> >
> > The new 2.19.47 is taking about 40 seconds to compile a one-note test
> file.
> > (MacOS X x86.)
> >
> > In my past experience, new LilyPond builds sometimes take 30 or 40
> seconds
> > to compile *on the very first run after install* (presumably to update
> the
> > font cache, or something similar). But 2.19.47 is taking 40 seconds *on
> > every run*.
>
> Run it under strace or whatever MacOSX has available in order to find
> out where the time is going?  I suspect it fails creating a font cache
> and builds it on every run.


Would be happy to, but I've never run strace or equivalent on the Mac. If
anyone else can do it, that would be helpful.

Just from having watched these things for a while, my guess would be the
same as yours: that the font cache is rebuilt on every run (rather than on
just the first run).

Trevor.

-- 
Trevor Bača
www.trevorbaca.com
soundcloud.com/trevorbaca
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Andrew Bernard]

Hi Trevor,

I can confirm the same behaviour on Mac OS X 10.11.6.

Running struss semi-equivalent dtruss seems like a nightmare on the mac -
all sorts of system restrictions prevent it. Not being a Mac developer  I
am clueless as to how one runs a simple trace on the mac. Anybody who knows
Mac development out there?


Andrew
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[David Kastrup]

Trevor Bača  writes:

> Hi,
>
> The new 2.19.47 is taking about 40 seconds to compile a one-note test file.
> (MacOS X x86.)
>
> In my past experience, new LilyPond builds sometimes take 30 or 40 seconds
> to compile *on the very first run after install* (presumably to update the
> font cache, or something similar). But 2.19.47 is taking 40 seconds *on
> every run*.

Run it under strace or whatever MacOSX has available in order to find
out where the time is going?  I suspect it fails creating a font cache
and builds it on every run.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Phil Holmes]

- Original Message - 
From: "Trevor Bača" 

To: "lilypond-devel" 
Sent: Wednesday, August 31, 2016 3:34 PM
Subject: 2.19.47 long compile times



Hi,

The new 2.19.47 is taking about 40 seconds to compile a one-note test 
file.

(MacOS X x86.)

In my past experience, new LilyPond builds sometimes take 30 or 40 seconds
to compile *on the very first run after install* (presumably to update the
font cache, or something similar). But 2.19.47 is taking 40 seconds *on
every run*.

Trevor.


On Windows, I get the very slow initial run (> 40 seconds) but subsequent 
runs are around 1 second, as expected.


--
Phil Holmes 



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

2.19.47 long compile times

[Trevor Bača]

Hi,

The new 2.19.47 is taking about 40 seconds to compile a one-note test file.
(MacOS X x86.)

In my past experience, new LilyPond builds sometimes take 30 or 40 seconds
to compile *on the very first run after install* (presumably to update the
font cache, or something similar). But 2.19.47 is taking 40 seconds *on
every run*.

Trevor.

-- 
Trevor Bača
www.trevorbaca.com
soundcloud.com/trevorbaca
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel