Re: [gentoo-user] Delays while building Libre Office.
Hi, Florian. On Sat, May 04, 2013 at 12:12:51PM +0200, Florian Philipp wrote: Am 02.05.2013 18:27, schrieb Alan Mackenzie: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? I'm seeing the same behavior here. libreoffice-3.6.6.2 on a mostly stable system. btrfs on lvm on dmcrypt on a hybrid disk. kernel 3.7.5 pf-sources. I don't think is has anything to do with the file system or hardware. make is eating 200M memory and uses 100% CPU, not mkdir. I guess the script is just buggy. Thanks for the report. I think the build script is buggy, too. There is something about our systems, possibly some use flag, which is driving make crazy. There doesn't appear to be any action in emerge which unpacks the build files (or even the entire source) of a package for perusal. This is a shame. At this point, I have to ask myself whether spending time trying to debug it (or even to report the bug usefully) is better than simply tolerating the long build time. It doesn't seem like there'll be an easy solution. Regards, Florian Philipp -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] Delays while building Libre Office.
On 2 May 2013, at 17:27, Alan Mackenzie wrote: ... Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. I think that at least you need to write a short shell script in which $i, $j, $k $l each number 0 - 9, and which you create 1000 or 100,000 directories in the format /1/2/3/4/5 (or /a/b/c/d/e/). I'm not sure if it's equally good to record the number of seconds since the epoc into a variable as the script launches as it i to just run the `time` command. The former would allow you to calculate how long it's taken to create a directory (or 100 of them). You can compare how long it takes to delete directories and how long it takes to create and delete empty files (or files containing a single character). I think you need to run this script on all your filesystems, and compare the results when emerge is and isn't running. Ideally you'd hope to be able to reproduce this slow directory creation behaviour using only your shell script. I think it's really easy to create such a shell script as this and to compare the results. I appreciate that adding complexities to it might be a chore, but it might be that you can reproduce the behaviour simply by creating 1000 directories with a single bash loop. I think it's pretty important to rule that out before assuming that the problem is with portage. Stroller.
Re: [gentoo-user] Delays while building Libre Office.
Am 02.05.2013 18:27, schrieb Alan Mackenzie: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? I'm seeing the same behavior here. libreoffice-3.6.6.2 on a mostly stable system. btrfs on lvm on dmcrypt on a hybrid disk. kernel 3.7.5 pf-sources. I don't think is has anything to do with the file system or hardware. make is eating 200M memory and uses 100% CPU, not mkdir. I guess the script is just buggy. Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Delays while building Libre Office.
On 05/02/2013 12:27 PM, Alan Mackenzie wrote: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? Long delays suggest a timeout of some sort. First thing I'd look at is the filesystem underneath, and the disk underneath that. Second thing I'd look at is to see if permissions checks might be bouncing through something like kerberos, samba or ldap. Do you have any single-signon things configured on that machine? signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Delays while building Libre Office.
130502 Alan Mackenzie wrote: I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. I've been using 4.0.0.1 w/o problems since 130113 : 4.0.2.2 is in testing. It took 1 h 10 m used c 3,5 GB for temporary storage. You might try it report the result. You have to be careful if you mix stable/testing for system pkgs, but using the testing version of something well-maintained, like LO or KDE, shouldn't cause problems normally. -- ,, SUPPORT ___//___, Philip Webb ELECTRIC /] [] [] [] [] []| Cities Centre, University of Toronto TRANSIT`-O--O---' purslowatchassdotutorontodotca
Re: [gentoo-user] Delays while building Libre Office.
On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote: On 05/02/2013 12:27 PM, Alan Mackenzie wrote: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? Long delays suggest a timeout of some sort. OK. As a matter of interest, some of the mkdirs executed relatively quickly - perhaps in 0.5 seconds. I never saw the screen whizzing by as I ought to have done, though. First thing I'd look at is the filesystem underneath, and the disk underneath that. My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array. In the middle of the mkdiring, I checked there were enough inodes free (there were). I've no reason to suspect the disk drives might be flaky. Second thing I'd look at is to see if permissions checks might be bouncing through something like kerberos, samba or ldap. Do you have any single-signon things configured on that machine? I've not got kerberos or samba installed. I appear to have ldap (whatever that might be ;-). ls -lurt /usr/bin/ldap* shows these binaries were last accessed (?used) on 2012-03-14. What exactly do you mean by single-signon? -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] Delays while building Libre Office.
On 05/02/2013 12:58 PM, Alan Mackenzie wrote: On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote: On 05/02/2013 12:27 PM, Alan Mackenzie wrote: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? Long delays suggest a timeout of some sort. OK. As a matter of interest, some of the mkdirs executed relatively quickly - perhaps in 0.5 seconds. I never saw the screen whizzing by as I ought to have done, though. Hm. First thing I'd look at is the filesystem underneath, and the disk underneath that. My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array. How full is the ext3 partition? What options do you have enabled on it? (e.g. dir indexing?) In the middle of the mkdiring, I checked there were enough inodes free (there were). I've no reason to suspect the disk drives might be flaky. Well, you kinda do, now; you have evidence that at least some disk access is unusually slow. Check dmesg for disk I/O errors (unlikely to be reported at this point; I'm sure you checked whether your RAID was in a degraded state), and run commanded smartctl tests on the disks. Second thing I'd look at is to see if permissions checks might be bouncing through something like kerberos, samba or ldap. Do you have any single-signon things configured on that machine? I've not got kerberos or samba installed. I appear to have ldap (whatever that might be ;-). ls -lurt /usr/bin/ldap* shows these binaries were last accessed (?used) on 2012-03-14. It would be more a question of whether they were tied into PAM. What exactly do you mean by single-signon? Well, that was a slip of the tongue. More central auth. I was wondering if there were any features installed on your system that are designed to check authorization against a server somewhere. (i.e. you can use an LDAP directory to centrally manage things like users, groups, etc.) Technically, single-signon combines authorization checks with persistent authentication checks. Examples of this include kerberos, web session cookies and some uses of OAuth; once you're authenticated, the mechanism ensures you don't need to authenticate to another server in the same auth realm so long as your existing session hasn't expired. But this is less likely to be related to your problem than something seeking to ask a server if you have authorization to access something. signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Delays while building Libre Office.
On Thu, May 02, 2013 at 01:15:58PM -0400, Michael Mol wrote: On 05/02/2013 12:58 PM, Alan Mackenzie wrote: On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote: On 05/02/2013 12:27 PM, Alan Mackenzie wrote: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? Long delays suggest a timeout of some sort. OK. As a matter of interest, some of the mkdirs executed relatively quickly - perhaps in 0.5 seconds. I never saw the screen whizzing by as I ought to have done, though. Hm. First thing I'd look at is the filesystem underneath, and the disk underneath that. My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array. How full is the ext3 partition? What options do you have enabled on it? (e.g. dir indexing?) root@acm ~ # df /var Filesystem 1K-blocksUsed Available Use% Mounted on /dev/mapper/vg-var 12385456 1959860 9796580 17% /var In the middle of the mkdiring, I checked there were enough inodes free (there were). I've no reason to suspect the disk drives might be flaky. Well, you kinda do, now; The reason I say this is that building the last ?one/two/three versions of libreoffice also gave me this grief, but I haven't noticed anything else amiss. you have evidence that at least some disk access is unusually slow. Check dmesg for disk I/O errors (unlikely to be reported at this point; Nothing awry in dmesg. I'm sure you checked whether your RAID was in a degraded state), cat /proc/mdstat shows everything in order. and run commanded smartctl tests on the disks. That I haven't done, yet. Second thing I'd look at is to see if permissions checks might be bouncing through something like kerberos, samba or ldap. Do you have any single-signon things configured on that machine? I've not got kerberos or samba installed. I appear to have ldap (whatever that might be ;-). ls -lurt /usr/bin/ldap* shows these binaries were last accessed (?used) on 2012-03-14. It would be more a question of whether they were tied into PAM. OK. I'm sadly ignorant about PAM. :-( What exactly do you mean by single-signon? Well, that was a slip of the tongue. More central auth. I was wondering if there were any features installed on your system that are designed to check authorization against a server somewhere. (i.e. you can use an LDAP directory to centrally manage things like users, groups, etc.) Not that I know of. My machine is a mere desktop connected via a router/modem to the net. I'd have no reason to install any auth stuff. Technically, single-signon combines authorization checks with persistent authentication checks. Examples of this include kerberos, web session cookies and some uses of OAuth; once you're authenticated, the mechanism ensures you don't need to authenticate to another server in the same auth realm so long as your existing session hasn't expired. But this is less likely to be related to your problem than something seeking to ask a server if you have authorization to access something. If this were the case, what would libreoffice's build need to ask that no other package stumbles over? -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] Delays while building Libre Office.
On 05/02/2013 02:47 PM, Alan Mackenzie wrote: On Thu, May 02, 2013 at 01:15:58PM -0400, Michael Mol wrote: On 05/02/2013 12:58 PM, Alan Mackenzie wrote: On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote: On 05/02/2013 12:27 PM, Alan Mackenzie wrote: Hi, Gentoo. I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my 2.6 GHz quad core Athlon 2. It used to take about an hour. Watching the build, it became evident that the first 50 minutes or so was taken up by several hundred mkdir operations (more precisely, mkdir -p long path). Some of these mkdir's would take, perhaps, a minute to execute. All the while, top showed make taking 100% of one core. There seems to be something suboptimal here. Has anybody else seen this, or does anybody have any ideas how to fix the problem? Long delays suggest a timeout of some sort. OK. As a matter of interest, some of the mkdirs executed relatively quickly - perhaps in 0.5 seconds. I never saw the screen whizzing by as I ought to have done, though. Hm. First thing I'd look at is the filesystem underneath, and the disk underneath that. My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array. How full is the ext3 partition? What options do you have enabled on it? (e.g. dir indexing?) root@acm ~ # df /var Filesystem 1K-blocksUsed Available Use% Mounted on /dev/mapper/vg-var 12385456 1959860 9796580 17% /var In the middle of the mkdiring, I checked there were enough inodes free (there were). I've no reason to suspect the disk drives might be flaky. Well, you kinda do, now; The reason I say this is that building the last ?one/two/three versions of libreoffice also gave me this grief, but I haven't noticed anything else amiss. you have evidence that at least some disk access is unusually slow. Check dmesg for disk I/O errors (unlikely to be reported at this point; Nothing awry in dmesg. I'm sure you checked whether your RAID was in a degraded state), cat /proc/mdstat shows everything in order. and run commanded smartctl tests on the disks. That I haven't done, yet. Second thing I'd look at is to see if permissions checks might be bouncing through something like kerberos, samba or ldap. Do you have any single-signon things configured on that machine? I've not got kerberos or samba installed. I appear to have ldap (whatever that might be ;-). ls -lurt /usr/bin/ldap* shows these binaries were last accessed (?used) on 2012-03-14. It would be more a question of whether they were tied into PAM. OK. I'm sadly ignorant about PAM. :-( If you've just got a single box, it's very unlikely this is your problem. What exactly do you mean by single-signon? Well, that was a slip of the tongue. More central auth. I was wondering if there were any features installed on your system that are designed to check authorization against a server somewhere. (i.e. you can use an LDAP directory to centrally manage things like users, groups, etc.) Not that I know of. My machine is a mere desktop connected via a router/modem to the net. I'd have no reason to install any auth stuff. Technically, single-signon combines authorization checks with persistent authentication checks. Examples of this include kerberos, web session cookies and some uses of OAuth; once you're authenticated, the mechanism ensures you don't need to authenticate to another server in the same auth realm so long as your existing session hasn't expired. But this is less likely to be related to your problem than something seeking to ask a server if you have authorization to access something. If this were the case, what would libreoffice's build need to ask that no other package stumbles over? My presumption there was that this was a very recent thing, and LO's time-to-build makes it easier to observe. Anyway, floor's open to anyone else who might have an idea. signature.asc Description: OpenPGP digital signature
