Re: hwcap supporting architectures?
At Thu, 27 Jan 2005 18:09:49 +0100, Bernd Eckenfels wrote: In article [EMAIL PROTECTED] you wrote: BTW: I wonder why hwcap decisions are not cached in the ld.so.cache? Why don't you check /etc/ld.so.cache? Hint: strings /etc/ld.so.cache | grep /lib/tls on i686. I know it places the libs in the cache, but it is still doing all the stats, why? Exactly. Your point is a relevant. The recent upstream glibc cvs shows it does not look up such unneeded look up libraries under the default directories - but not non-standard directories. I'll investigate more. Regards, -- gotom -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
In article [EMAIL PROTECTED] you wrote: BTW: I wonder why hwcap decisions are not cached in the ld.so.cache? Why don't you check /etc/ld.so.cache? Hint: strings /etc/ld.so.cache | grep /lib/tls on i686. I know it places the libs in the cache, but it is still doing all the stats, why? Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
At Sun, 23 Jan 2005 18:38:59 +0100, Bernd Eckenfels wrote: On Mon, Jan 24, 2005 at 01:16:12AM +0900, Junichi Uekawa wrote: Looking at the rate of hardware changes, we will ideally be wanting to add a new hwcap entry just about every year; which results roughly in x10 time penalty every 3 years. BTW: I wonder why hwcap decisions are not cached in the ld.so.cache? Why don't you check /etc/ld.so.cache? Hint: strings /etc/ld.so.cache | grep /lib/tls on i686. Note that there are LD_LIBRARY_PATH, DT_RPATH and DT_RUNPATH. Regards, -- gotom -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
On Mon, Jan 24, 2005 at 01:16:12AM +0900, Junichi Uekawa wrote: The main problem with adding hwcap is that the number of directory to be traversed doubles with every addition, which is an exponential thing; rather than something linear. Yes, this is a problem. But isn't it possible to change the implementation to make several caps depending on others? For example mmx is only available on i586 and higher. I currently have problems with the implementation anyway, I can't disable tls which makes problems on Xen, nor can I disable the usage of optimized versions for testing of the other versions. Bastian -- Where there's no emotion, there's no motive for violence. -- Spock, Dagger of the Mind, stardate 2715.1 signature.asc Description: Digital signature
Re: hwcap supporting architectures?
On Mon, Jan 24, 2005 at 09:31:30AM +0100, Bastian Blank wrote: I currently have problems with the implementation anyway, I can't disable tls which makes problems on Xen, nor can I disable the usage of optimized versions for testing of the other versions. a quick workaround is to export LD_ASSUME_KERNEL=2.4.10, which also disables tls. But I don't know what other side-effects are to be expected ? -- c u henning -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
On Wed, Jan 19, 2005 at 09:43:18AM +0100, Bernd Eckenfels wrote: I agree with you, but dont forget that this micro benchmark does not really measure the overall effect on the system (i.e. to other programs, to the number of meta data updates, cach useage) and it does not take into account slow or unreachable path components (nfs). Agreed. The intention of the benchmark is not to claim that the operation that ld.so is performing is fast; it's just to provide some quantifiable argument for or against the system calls are way too expensinve meme. *My* position is that whilst they are expensive (certainly much more expensive than a function call), they are not _that_ expensive. Before asking here I went thru several mailing list archives. What I found was a general argument along the lines of don't add _that_ hwcap because that will increase the size of the list of paths that you have to stat(2) in order to find the library and that's slow. In particular upstream doesn't seem to like the idea much. I just can't find the reason why they think the stat(2) call is _so_ expensive that it will hurt system performance. There was some mention of a glib issue with plugins. I couldn't find further data points on that... So it might be actually noticeable on normal systems (I doubt it). Just dont stop anyone volunteering from fixing it. I for one are happy if the strace of a process keeps readable in finite time, so do system administrators with auditing turned on. I can understand that. I'm just not sure what's to fix in the first place. Marcelo -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
On Wed, Jan 19, 2005 at 01:20:56PM +0900, GOTO Masanori wrote: Ugh... that pushes the burden of maitaining support for new architectures to the package. Yeah - I think it's trade off - whether we support library optimization package or we don't get a bit performance improvement. So, you are trading maintainance cost for a rather subjective speed improvent? Or should I say, preventing some performance degradation? The reason why I don't try to clear hwcap issue in documentation is: I don't want to battle to someone about this kind of issue. Buying new hardware improves performance. I don't reply any more. Whatever. If you just don't want to discuss the nice feature, I won't force you into the discussion. Buy new hardware is a argument _against_ your general statement, so I have no idea where you want to go with that. I _do_ need to point out that /etc/ld.so.nohwcap is not documented anywhere where the user can find about it: $ zgrep nohwcap /usr/share/doc/libc6/* /usr/share/doc/libc6/changelog.Debian.gz: /etc/ld.so.nohwcap is 0. But I guess that just being consistent with upstream's tradition. Trying to find documentation about LD_DEBUG (for example) is frustrating at best. ld.so(8) doesn't even mention the variable! Either you *bump* into LD_DEBUG=help by chance or go RTFS, which is, quite appropriately, also badly documented. But it's fine, don't bother to reply, you certainly have better things to do than justify backward designs and weird decisions which are probably not yours to begin with. Marcelo -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
On Tue, Jan 18, 2005 at 10:09:03PM -0600, Marcelo E. Magallon wrote: What that means is that you need to make about 2000 stat(2) calls to get _anywhere_ near what's measurable by a human and about 2 to start getting said human annoyed. I agree with you, but dont forget that this micro benchmark does not really measure the overall effect on the system (i.e. to other programs, to the number of meta data updates, cach useage) and it does not take into account slow or unreachable path components (nfs). So it might be actually noticeable on normal systems (I doubt it). Just dont stop anyone volunteering from fixing it. I for one are happy if the strace of a process keeps readable in finite time, so do system administrators with auditing turned on. Greetings Bernd -- (OO) -- [EMAIL PROTECTED] -- ( .. ) [EMAIL PROTECTED],linux.de,debian.org} http://www.eckes.org/ o--o 1024D/E383CD7E [EMAIL PROTECTED] v:+497211603874 f:+497211606754 (OO) When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl! -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
At Sun, 16 Jan 2005 20:55:28 +0100, Falk Hueffner wrote: Marcelo E. Magallon writes: On Sat, Jan 15, 2005 at 10:14:15PM +0900, GOTO Masanori wrote: occurs when you have for example an ev56 library in lib/ev56, and a ev67 CPU. Then the loader looks in lib/ev67 and then falls back to lib. Since glibc is very carefully undocumented in this area [1], I didn't want to try to change this, but rather assumed one could add a symlink. Yes, and if ev67 is instruction upper compatible with ev56 (I guess so), I think it's acceptable to add a symlink ln -sf lib/ev67/libfoo.so lib/ev56/libfoo.so. Ugh... that pushes the burden of maitaining support for new architectures to the package. Yeah - I think it's trade off - whether we support library optimization package or we don't get a bit performance improvement. Please bear with me, but I'm trying to understand the issue: is the cost of calling access(2) or stat(2) really so high? I'd consider it quite acceptable in this case. However, as I tried to express, it's not possible with glibc's current design, and I didn't feel like changing that. Note that we should keep in mind: imagine most binaries on all debian system over the world start to consume access(2)/stat(2) system call cost in each binary execution time - Many a little makes a mickle. I see for example that on start up the file /etc/ld.so.nohwcap is accessed multiple times (and it's not present, isn't that a race? what happens if the file suddenly appears in the middle of program start up? what's that file anyway, I can't find it mentioned in the documentation). It's supposed to disable the use of hwcaps. Stating it multiple times seems like a bug. Debian glibc has been applied a special patch to check /etc/ld.so.nohwcap before loading libraries each time. You can see it in debian-glibc package ldso-disable-hwcap.dpatch written by Ben and Daniel. It enables us to upgrade smoothly even if we use optimized libraries - this effort is one of debian's nice features. But the drawback is it needs to pay access(2) lookup cost as you pointed out. Checking /etc/ld.so.nohwcap each time (some binaries call multiple times) is the current patch design - I think this is safer than checking /etc/ld.so.nohwcap once in program startup time. However, yes, it affects execution time for short life programs. I think it's acceptable because it's one of debian's nice feature... Regards, -- gotom -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
At Thu, 13 Jan 2005 09:05:43 +0100, Falk Hueffner wrote: If other libraries like mesa and libssl want to use /usr/lib/ev67 and so on, we may consider to add HWCAP_IMPORTANT. This should not be needed, since the library loader also looks in a directory corresponding to the architecture name. The only problem with this Thanks for your follow up, you're right. Your patch has sysdeps/alpha/dl-procinfo.h. occurs when you have for example an ev56 library in lib/ev56, and a ev67 CPU. Then the loader looks in lib/ev67 and then falls back to lib. Since glibc is very carefully undocumented in this area [1], I didn't want to try to change this, but rather assumed one could add a symlink. Yes, and if ev67 is instruction upper compatible with ev56 (I guess so), I think it's acceptable to add a symlink ln -sf lib/ev67/libfoo.so lib/ev56/libfoo.so. Regards, -- gotom -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
On Sat, Jan 15, 2005 at 10:14:15PM +0900, GOTO Masanori wrote: occurs when you have for example an ev56 library in lib/ev56, and a ev67 CPU. Then the loader looks in lib/ev67 and then falls back to lib. Since glibc is very carefully undocumented in this area [1], I didn't want to try to change this, but rather assumed one could add a symlink. Yes, and if ev67 is instruction upper compatible with ev56 (I guess so), I think it's acceptable to add a symlink ln -sf lib/ev67/libfoo.so lib/ev56/libfoo.so. Ugh... that pushes the burden of maitaining support for new architectures to the package. Please bear with me, but I'm trying to understand the issue: is the cost of calling access(2) or stat(2) really so high? I see for example that on start up the file /etc/ld.so.nohwcap is accessed multiple times (and it's not present, isn't that a race? what happens if the file suddenly appears in the middle of program start up? what's that file anyway, I can't find it mentioned in the documentation). Marcelo -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: hwcap supporting architectures?
GOTO Masanori [EMAIL PROTECTED] wrote: Marcelo E. Magallon [EMAIL PROTECTED] wrote: Mesa upstream uses -mcpu=ev5 -mieee on alpha. Is that ok? Where does this belong into? /usr/lib/ev5? IIRC, alpha does not define any hwcaps. There's a patch for this, which works fine, but wasn't committed yet: http://lists.debian.org/debian-glibc/2004/03/msg00143.html Sensible options are ev56 and ev67; ev5 is not particularly useful, since it has the same instruction set as the baseline ev4, only different scheduling. -mieee is default anyway on Debian's gcc. Falk -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
