Re: Unfortunate dynamic linking for everything
Matthew D. Fuller [EMAIL PROTECTED] wrote: Not just the startup scripts, but ANY script. I dare say there's a long, long list of scripts that use ~-expansion, to say nothing of the homegrown ones we all have working quietly and forgotten for years. It's required for POSIX compliance. Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ IRISH SEA: SOUTHWESTERLY 5 OR 6. SHOWERS. GOOD. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Robert Watson [EMAIL PROTECTED] wrote: Someone must be using /bin/sh as a shell, because apparently someone spent a lot of time adding things like character input editing, filename completion, etc. We even use sh as the default in adduser(8). Command-line editing is required for POSIX compliance. Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ FAIR ISLE: SOUTHWESTERLY BACKING EASTERLY 5 TO 7, PERHAPS GALE 8 LATER. SHOWERS THEN RAIN. MODERATE OR GOOD. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Tony Finch wrote: Matthew D. Fuller [EMAIL PROTECTED] wrote: Not just the startup scripts, but ANY script. I dare say there's a long, long list of scripts that use ~-expansion, to say nothing of the homegrown ones we all have working quietly and forgotten for years. It's required for POSIX compliance. Tony. Ouch! Very good point, Tony. Does POSIX require that such expansion work for usernames that may not be in the current passwd file? That could dictate a lot. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
[ Lots of CC trimming ] On Sun, Nov 23, 2003 at 06:27:01PM -0500 I heard the voice of Richard Coleman, and lo! it spake thus: You would need to make sure that startup scripts never use tilde expansion. I'm not sure how common that is with RCNG. Not just the startup scripts, but ANY script. I dare say there's a long, long list of scripts that use ~-expansion, to say nothing of the homegrown ones we all have working quietly and forgotten for years. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ The only reason I'm burning my candle at both ends, is because I haven't figured out how to light the middle yet ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: David O'Brien [EMAIL PROTECTED] I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I don't know anyone that farms lama's, so there cannot be any lama farmers. computer$ grep dmlb /etc/passwd dmlb:*:1166:1166:Duncan Barclay:/home/dmlb:/bin/sh Duncan So, imagine, i'm accidentally deleted /bin with your most wanted static sh... And, of course, due to static nature of /bin/sh it was removed from /rescue? Nothing will protect you from shooting in the leg, neither static linking, nor assumption that /lib is OK. MOST people uses /bin/sh only for rc scripts (to be correct, their system uses it). David O'Brien just tried to told, that NOBODY he knows will be REALLY impacted by performance loss, caused due dynamic /bin/sh linking. You will... So, because Duncan Barclay is impacted by performance loss due dynamic /bin/sh linking, ENTIRE FreeBSD community will have troubles (at least with NSS) due to static linking... Maxim, I was merely rising to David's bait for some proof that people use /bin/sh as an interactive shell. You're correct in saying that if /bin/sh on my machines gets hosed it won't matter too much - I'll use another shell to rebuild the machine, or boot from an install CD to get a shell. But as part of me fixing the machine, I'll put /bin/sh back on. I didn't say anything about NSS and I don't intend to as I don't need it. The debate should be held between people that need the functionality but want it implemented in different ways. As to performance loss, I really don't think I'm going to notice it - the cheapest machine I can find in the UK is an Athlon 1800XP, that has a lot more grunt than I need. This may not be true for others. Duncan Sincerely, Maxim M. Kazachek mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Mon, Nov 24, 2003 at 12:14:39AM -, Duncan Barclay wrote: From: David O'Brien [EMAIL PROTECTED] I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I don't know anyone that farms lama's, so there cannot be any lama farmers. One has to make a strong statement to get the people to come out of the woodwork. Ack. computer$ grep dmlb /etc/passwd dmlb:*:1166:1166:Duncan Barclay:/home/dmlb:/bin/sh Good. Now do you need NSS support? Do the benefits of supporting NSS in /bin/sh for you out-weigh the performance issue of building it dynamically? Couldn't you just as easily use the pdksh port? The machine use I generate doesn't really require a lot of /bin/sh invocations. Either I have file servers, shell boxes, or compute engines running CPU bound jobs for half an hour upwards. Whether it takes a gnats longer to start /bin/sh doesn't really matter to me. However, NSS is likely to feature as needed sometime soon. Duncan -- -- David ([EMAIL PROTECTED]) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Mon, 24 Nov 2003, Maxim M. Kazachek wrote: MOST people uses /bin/sh only for rc scripts (to be correct, their system uses it). David O'Brien just tried to told, that NOBODY he knows will be REALLY impacted by performance loss, caused due dynamic /bin/sh linking. You will... So, because Duncan Barclay is impacted by performance loss due dynamic /bin/sh linking, ENTIRE FreeBSD community will have troubles (at least with NSS) due to static linking... Actually, you appear to be agreeing with him, not disagreeing with him. Duncan was pointing out that he *does* use /bin/sh as his shell, in response to David's suggestion that on one uses it and therefore that making it statically linked wouldn't hurt. It strikes me that this whole conversation has gotten a little confrontational... The middle ground of adding a static /sbin/sh for scripts soudds like a reasonable choice, and has precedent in other systems (Solaris). We can set the boot and periodic scripts to use that, and interactive users can keep using /bin/sh. Someone must be using /bin/sh as a shell, because apparently someone spent a lot of time adding things like character input editing, filename completion, etc. We even use sh as the default in adduser(8). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Robert Watson writes: It strikes me that this whole conversation has gotten a little confrontational... The middle ground of adding a static /sbin/sh for scripts soudds like a reasonable choice, and has precedent in other systems (Solaris). We can set the boot and periodic scripts to use that, and interactive users can keep using /bin/sh. Someone must be using /bin/sh as a shell, because apparently someone spent a lot of time adding things like character input editing, filename completion, etc. We even use sh as the default in adduser(8). Tru64 also does the /sbin/sh (static) and /bin/sh (dynamic) thing. I've never liked it because standard, portable shell scipts expect to use /bin/sh. Standard, portable shell scripts don't need tilde expansion for ldap/nss usernames. So I think the best solution (*) would be to keep /bin/sh statically linked, and build a dynamic version in /usr/bin that people can use as an interactive shell. Root's shell remains /bin/sh 1) All three (;-) interactive bourne shell users that use nss/ldap get tilde expansion. 2) 3rd party scripts, which are written to be portable and don't use tilde, don't have to pay for dynamic linking. 3) System startup scripts are faster because they use the static /bin/sh (*) The real best solution would be to backout the dynamic linking changes and put the onus on those who want/need nss/ldap to prove that the dynamic linking changes do not produce a measurable slowdown (as they were asked to do 6 months ago, and never did). If it really slows things down, then they need to fix the performance problems by either speeding up dynamic linking, or getting the features they need in another, less invasive way (as discussed previously in this thread). Drew [reply-to set to freebsd-current, as I don't want to be CC'ed on this thread forever.. ] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Mon, Nov 24, 2003 at 10:47:24AM -0500, Robert Watson wrote: It strikes me that this whole conversation has gotten a little confrontational... The middle ground of adding a static /sbin/sh for scripts soudds like a reasonable choice, and has precedent in other systems (Solaris). Time for a pdksh import! We can install it as the dynamic /bin/sh, and there is precedent in other systems (Solaris) for /bin/ksh. The issue with static /sbin/sh is that we have to fix all the base scripts (doable), and then try to educate people that they need to do the same to their scripts (very hard to do). Someone must be using /bin/sh as a shell, because apparently someone spent a lot of time adding things like character input editing, filename completion, etc. Filename complete isn't in stock /bin/sh as its a private hack someone did, it is a hack and isn't commitable. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Andrew Gallatin wrote: So I think the best solution (*) would be to keep /bin/sh statically linked, and build a dynamic version in /usr/bin that people can use as an interactive shell. Root's shell remains /bin/sh 1) All three (;-) interactive bourne shell users that use nss/ldap get tilde expansion. 2) 3rd party scripts, which are written to be portable and don't use tilde, don't have to pay for dynamic linking. 3) System startup scripts are faster because they use the static /bin/sh Are you suggesting that (t)csh also move to /usr/bin to match /usr/bin/sh? The screams caused by such a change would be deafening. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Richard Coleman writes: Are you suggesting that (t)csh also move to /usr/bin to match /usr/bin/sh? The screams caused by such a change would be deafening. Of course not. Nobody in their right mind uses csh for scripting. Drew ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Richard Coleman [EMAIL PROTECTED] writes: Are you suggesting that (t)csh also move to /usr/bin to match /usr/bin/sh? The screams caused by such a change would be deafening. Would there be any screams at all? chsh -s /bin/sh root# prevent lock-out rm -f /bin/csh /bin/tcsh echo NO_TCSH=true /etc/make.conf # avoid resurrection of evil # next make world Looks that a system without [t]csh is supported. And it works for me. -- Matthias Andree Encrypt your mail: my GnuPG key ID is 0x052E7D95 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
At 8:59 PM -0500 2003/11/24, Andrew Gallatin wrote: Of course not. Nobody in their right mind uses csh for scripting. To my great horror, csh is used in most of the DNS debugging and many of the log-processing scripts that I have inherited. One of these days, I will finally live up to my threat of importing all this functionality into other programs that use other languages (toss doc and incorporate that functionality into dnswalk, etc...), but that has not happened yet. Meanwhile, I don't know that a dynamic vs. static csh does me any measurable harm -- the delays waiting for responses from nameservers will overwhelm any local delays caused by dynamic vs. static linking. -- Brad Knowles, [EMAIL PROTECTED] They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++): a C++(+++)$ UMBSHI$ P+++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+() DI+() D+(++) G+() e++ h--- r---(+++)* z(+++) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
As I pointed out earlier, some of the heat here comes from the fact that /bin/sh is currently overloaded: * It is the default system script interpreter, used by the rc scripts and many other things. As such, it must start quickly. * It is the default user shell for many users. As such, it must support NSS. So far, I haven't seen anyone in this thread seriously argue against either of these points. I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. Richard Coleman wrote: It seems /bin/sh is the real sticking point. There is a problem here: Unix systems have historically used /bin/sh for two somewhat contradictory purposes: * the system script interpreter * as a user shell The user shell must be dynamically linked in order to support centralized administration. I personally see no way around that. Given that many users do rely on /bin/sh, it seems that /bin/sh must be dynamically linked. There are good reasons to want the system script interpreter statically linked. Maybe it's time to separate these two functions? I argue the two functions are already separated as /bin/sh as interactive shell doesn't really exist outside of single user. We should build /bin/sh static and be done with the argument. Or rather, lets find a /bin/sh interactive user and have him argue that /bin/sh needs NSS support. I dare say that will be a thread two orders of magnitude shorter than this one. -- -- David ([EMAIL PROTECTED]) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
David O'Brien wrote: We should build /bin/sh static and be done with the argument. Or rather, lets find a /bin/sh interactive user and have him argue that /bin/sh needs NSS support. I dare say that will be a thread two orders of magnitude shorter than this one. Statically linking /bin/sh wouldn't be that big a deal. It would be trivial to add a port to compile a dynamic version of sh for customers that *really* want sh. Personally, I can't remember a customer specifically requesting that. You would need to make sure that startup scripts never use tilde expansion. I'm not sure how common that is with RCNG. But it would be sorta odd to have statically linked versions of sh in both /bin and /rescue. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: David O'Brien [EMAIL PROTECTED] I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I don't know anyone that farms lama's, so there cannot be any lama farmers. computer$ grep dmlb /etc/passwd dmlb:*:1166:1166:Duncan Barclay:/home/dmlb:/bin/sh Duncan ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
So far, I haven't seen anyone in this thread seriously argue against either of these points. I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I'm one... I have to operate on many disparate systems with the same home directory some of the bourne-shell variants (e.g. HP/UX's ksh) will lock-up on my simple bourne-shell .profile. So - I stick with the plain/old/boring sh. - Dave R. - -- [EMAIL PROTECTED]Work: (919) 676-0847 Get your mainframe programming tools at http://www.dignus.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Mon, Nov 24, 2003 at 12:14:39AM -, Duncan Barclay wrote: From: David O'Brien [EMAIL PROTECTED] I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I don't know anyone that farms lama's, so there cannot be any lama farmers. One has to make a strong statement to get the people to come out of the woodwork. computer$ grep dmlb /etc/passwd dmlb:*:1166:1166:Duncan Barclay:/home/dmlb:/bin/sh Good. Now do you need NSS support? Do the benefits of supporting NSS in /bin/sh for you out-weigh the performance issue of building it dynamically? Couldn't you just as easily use the pdksh port? -- -- David ([EMAIL PROTECTED]) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Sun, Nov 23, 2003 at 06:27:01PM -0500, Richard Coleman wrote: But it would be sorta odd to have statically linked versions of sh in both /bin and /rescue. We'd remove it from /rescue if the /bin/sh one was static. :-) -- -- David ([EMAIL PROTECTED]) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Mon, 24 Nov 2003, Duncan Barclay wrote: From: David O'Brien [EMAIL PROTECTED] I'll seriously argue against the 2nd point above. I don't know of a SINGLE person that uses /bin/sh as their interactive shell when multi-user. Not ONE. Every Bourne shell'ish user I've ever met uses Bash, ATT ksh, pdksh, zsh. I don't know anyone that farms lama's, so there cannot be any lama farmers. computer$ grep dmlb /etc/passwd dmlb:*:1166:1166:Duncan Barclay:/home/dmlb:/bin/sh Duncan So, imagine, i'm accidentally deleted /bin with your most wanted static sh... And, of course, due to static nature of /bin/sh it was removed from /rescue? Nothing will protect you from shooting in the leg, neither static linking, nor assumption that /lib is OK. MOST people uses /bin/sh only for rc scripts (to be correct, their system uses it). David O'Brien just tried to told, that NOBODY he knows will be REALLY impacted by performance loss, caused due dynamic /bin/sh linking. You will... So, because Duncan Barclay is impacted by performance loss due dynamic /bin/sh linking, ENTIRE FreeBSD community will have troubles (at least with NSS) due to static linking... Sincerely, Maxim M. Kazachek mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Date: Mon, 24 Nov 2003 09:34:08 +0600 (NOVT) From: Maxim M. Kazachek [EMAIL PROTECTED] So, imagine, i'm accidentally deleted /bin with your most wanted static sh... And, of course, due to static nature of /bin/sh it was removed from /rescue? Nothing will protect you from shooting in the leg, neither static linking, nor assumption that /lib is OK. So go ahead and make /bin/sh also have a (hard) link to /rescue/sh. Then the referenced action merely decrements the link count, and the executable itself doesn't go away. Sure, you could be more imaginative with foot-shooting, but the stated problem is really easy to avoid. Peace, david (reluctant to contribute to this thread) -- David H. Wolfskill [EMAIL PROTECTED] If you want true virus-protection for your PC, install a non-Microsoft OS on it. Plausible candidates include FreeBSD, Linux, NetBSD, OpenBSD, and Solaris (in alphabetical order). ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Sun, 23 Nov 2003, David Wolfskill wrote: Date: Mon, 24 Nov 2003 09:34:08 +0600 (NOVT) From: Maxim M. Kazachek [EMAIL PROTECTED] So, imagine, i'm accidentally deleted /bin with your most wanted static sh... And, of course, due to static nature of /bin/sh it was removed from /rescue? Nothing will protect you from shooting in the leg, neither static linking, nor assumption that /lib is OK. So go ahead and make /bin/sh also have a (hard) link to /rescue/sh. Then the referenced action merely decrements the link count, and the executable itself doesn't go away. Sure, you could be more imaginative with foot-shooting, but the stated problem is really easy to avoid. Peace, david (reluctant to contribute to this thread) -- David H. Wolfskill [EMAIL PROTECTED] If you want true virus-protection for your PC, install a non-Microsoft OS on it. Plausible candidates include FreeBSD, Linux, NetBSD, OpenBSD, and Solaris (in alphabetical order). I'm a real masochist, and removed /rescue/sh /sbin/sh whatsoever. And after that will begin complain, that FreeBSD is ugly, non bullet-proof operating system... :-) But, in fact, as I can remember, all /rescue stuff is one hadlinked executable. I don't think that hardlinking /bin/sh into /rescue would be nice idea... IMHO it's not clear. Sincerely, Maxim M. Kazachek mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On 2003-11-22 at 00:39:45 Tim Kientzle wrote: Right now, /sbin/init is statically linked. Not here... I've built everything with WITH_DYNAMICROOT since the time the option was introduced, and as such: # file /sbin/init /sbin/init: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), for FreeBSD 5.0.1, dynamically linked (uses shared libs), stripped # ldd /sbin/init /sbin/init: libutil.so.3 = /lib/libutil.so.3 (0x28074000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x2807f000) libc.so.5 = /lib/libc.so.5 (0x28097000) In fact, the only statically linked executable I can currently find in my base system (= -CURRENT as of 2003-11-11) is /sbin/devd... pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
Garrett Wollman wrote: You forgot: * Allow statically-linked programs to use dynamic NSS modules by forking a (dynamically-linked) resolver process when needed. This leads to a related, but widely disparaged option: * Have a persistent NSS caching daemon with an RPC interface that all programs can access for NSS lookups. You might call such a program `nscd'. (Might as well be honest about it.) Both of these options may incidentally help to resolve threading issues in the C library (although that would not be the preferred way of doing so). Regardless of how NSS is implemented, it will be useful to have some type of caching (even if optional). On a large system, you can beat the hell out of your LDAP server otherwise. Of course, you can use a local replica of your LDAP server. But that doesn't help if are using a different database like Postgres as the backend for your nss/pam setup. But if a nscd (or equivalent) is added to FreeBSD, we need to do a better job than the Linux nscd. We had real problems with the Linux nscd in a previous project. If I remember, it assumes that the mapping between username - uid was 1-1. We added an attribute to our LDAP schema so we could specify the reverse mapping in situations where more than one username mapped to the same uid. But we could never get it to work correctly, since Linux nscd made some assumption that were difficult to change. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
* Tim Kientzle [EMAIL PROTECTED] [031121 18:40]: Leo Bicknell wrote: To boot a machine into single user mode you need a kernel, init, and /bin/sh (minimally). It would seem to me that alone is a good argument for those three things to be static. * Rewrite dlopen() to not require dynamic linking. There were some patches for this submitted at one point. As I recall, the people who looked at them were not entirely comfortable with them. (I'd be concerned about version conflict problems with this approach: what happens when a dynamically-loaded NSS module refers to a libc function? Does that get resolved to the already statically-linked version? Or does another copy of libc get dynamically linked with potential version conflicts? Does anyone know?) I personally think this is worth researching, though I have my doubts. I took a look at the glibc implementation of dlopen() breifly, since that does function from within libc.a. It appears that you *do* get more than one loaded copy of libc. The copy of dlopen() that is built when #ifndef SHARED includes a flag: __libc_multiple_libcs that is set to 1. Additionally, I was reading comments from some of the glibc developers who basically claim that dlopen() in a static binary *only* works if you dlopen() a NSS module. It isn't guaranteed to work in the general case because the static binary has no DYNAMIC elf section to resolve external references etc. I suspect this means NSS modules are limited in what they are allowed to reference and still work? I haven't looked in much detail on their implementation but it certainly looks like a hack just to make NSS work, which I don't think is a good long-term solution. --Mike pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
On Thu, Nov 20, 2003 at 09:29:30PM -0500, Richard Coleman wrote: But I've often wondered how frequently a production system has such problems. I've been a sysadmin for many years and can't remember this ever happening. It's much more common to blow a hard drive, or have flaky memory, etc. We've had a customer whose system administrator installed a root cronjob including rm -r / somedirectory/tmp (note the space) - on all four production machines as well as their model system. (The person in question is a classic example of a little knowledge is a dangerous thing). The console logs (which were on another unaffected system) made interesting reading - he had spent a significant amount of time attempting to reboot the systems before calling for help. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Thu, Nov 20, 2003 at 09:51:48PM +0100, boyd, rounin wrote: From: Peter Jeremy [EMAIL PROTECTED] Shouldn't that be 'chmod +t /bin/sh' ??? Definitely. Why waste a new bit when there's already a perfectly good one that is (or was) defined for the purpose. the 't' bit was known as the 'sticky' bit to keep frequently used, sharable (judged by a human) text segments in core. since then that bit has been overloaded to death. The purpose for setting the 't' bit was to speed up exec'ing of frequently used executables by avoiding the need to load the text segment from the filesystem. This is exactly what the suggested pre-linking achieves. As for overloading the 't' bit, I don't believe it's ever been used for anything else on executable files. I agree it grew a distinct meaning for directories but the 'x' bits have always meant different things on files and directories and the setgid bit means different things on executable and non-executable files. shared libraries have always been a mistake. That is a matter of opinion. Shared libraries have both advantages and disadvantages. Statically linking /usr/X11R6/bin demonstrates one advantage of shared libraries. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: Peter Jeremy [EMAIL PROTECTED] As for overloading the 't' bit, I don't believe it's ever been used for anything else on executable files. directories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
I got bit by this just two days ago. I have one machine that tracks -current. It upgraded to DYNAMICROOT just fine. I nfs mounted /usr/src and /usr/obj on another 5.0-release machine made the necessary adjustments, installed the kernel, rebooted, remounted and began make installworld. It failed. The only thing that saved me was being able to nfs mount /rescue from the -current machine. I then selectively copied stuff from /usr/obj into the respective places to get various 'make install' stuff to work, in order to get to a point where I could re-run make installworld. THANKFULLY mount was statically linked (and make installworld had not yet overwritten it). Yes I still could have recovered this using a cd/floppy (neither of which this headless machine has), but it would have been a serious PITA. -- othermark atkin901 at nospam dot yahoo dot com (!wired)?(coffee++):(wired); ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Thu, Nov 20, 2003 at 09:29:30PM -0500, Richard Coleman wrote: But I've often wondered how frequently a production system has such problems. I've been a sysadmin for many years and can't remember this ever happening. It's much more common to blow a hard drive, or have flaky memory, etc. During my time in an investment bank, installations were usually hosed in this way by human error (systems administrators removing a file by accident, etc) or by third party package installation. The OS in question was Solaris. BMS ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
At 8:52 PM +1100 11/20/03, Peter Jeremy wrote: On Wed, Nov 19, 2003, Lyndon Nerenberg wrote: --On Wed, Nov 19, 2003, Garance A Drosihn [EMAIL PROTECTED] wrote: have a: chflags ldcache /bin/sh Shouldn't that be 'chmod +t /bin/sh' ??? Definitely. Why waste a new bit when there's already a perfectly good one that is (or was) defined for the purpose. Two reasons I didn't suggest that: a) Well, actually it never occurred to me... b) I thought that you might want to have this an admin-only command, so nefarious users couldn't abuse it on a shared system. I'm certainly fine with it using +t. Now it's just a matter of figuring out how to implement it... :-) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Leo Bicknell wrote: To boot a machine into single user mode you need a kernel, init, and /bin/sh (minimally). It would seem to me that alone is a good argument for those three things to be static. You need a static shell, yes. That does not have to be /bin/sh. init does prompt, and /rescue/sh is (and always will be) static. As I pointed out earlier, some of the heat here comes from the fact that /bin/sh is currently overloaded: * It is the default system script interpreter, used by the rc scripts and many other things. As such, it must start quickly. * It is the default user shell for many users. As such, it must support NSS. So far, I haven't seen anyone in this thread seriously argue against either of these points. Today, these two issues are somewhat contradictory. Right now, NSS requires dlopen() which requires dynamic linking which negatively impacts startup. There have been a lot of proposed solutions: * Rewrite NSS to not require dlopen(). A lot of people have suggested this, but Jacques made an important point about long-term support. In short, this would probably just gaurantee that our NSS is perpetually out-of-date, since it would become a major headache to port existing NSS modules. * Rewrite dlopen() to not require dynamic linking. There were some patches for this submitted at one point. As I recall, the people who looked at them were not entirely comfortable with them. (I'd be concerned about version conflict problems with this approach: what happens when a dynamically-loaded NSS module refers to a libc function? Does that get resolved to the already statically-linked version? Or does another copy of libc get dynamically linked with potential version conflicts? Does anyone know?) I personally think this is worth researching, though I have my doubts. * Don't support NSS in /bin/sh. This essentially amounts to: Don't use /bin/sh as an end-user shell. Given the number of commits to improve command-line usage of /bin/sh, I think it's clear that a lot of people do rely on /bin/sh as a user shell. * Change the default script interpreter for rc and such. (e.g., use a statically-linked, optimized /sbin/sh for that purpose.) Again, not particularly attractive, and it certainly breaks a lot of administrator's assumptions. * Make dynamic linking faster. As many people have pointed out, FreeBSD's dynamic linking is slower than it should be. Advances here would benefit Mozilla, OpenOffice, KDE, Gnome, Apache, Perl, bash, TCL, Python, PHP, etc. (I believe these are all slower to start on FreeBSD than on Linux, for example.) This is a major end-user experience issue that I find much more compelling than anything mentioned so far. I personally find OpenOffice only barely usable on FreeBSD, in large part because it takes so long to open. I suspect that most of our users would gladly give up a few seconds booting (or a few minutes from portupgrade or buildworld) if they could have Mozilla and OpenOffice open a few seconds faster. Benchmarks do not necessarily reflect user experience. As many people have pointed out, there are other advantages and disadvantages to dynamic linking: library upgrades are simplified, disk requirements are reduced, memory usage is less efficient, system upgrade and repair is more complicated, etc., but I think the above summarizes the issues that people seem to really care about. Tim ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Leo Bicknell wrote: The more I think about init the more I don't like dynamic linking for it. init needs to have as few failure modes as possible. I do still think it's fine for all the other /bin and /sbin things. Right now, /sbin/init is statically linked. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, 21 Nov 2003 15:38:49 -0800, Tim Kientzle [EMAIL PROTECTED] said: There have been a lot of proposed solutions: * Rewrite NSS to not require dlopen(). * Rewrite dlopen() to not require dynamic linking. * Don't support NSS in /bin/sh. * Change the default script interpreter for rc and such. * Make dynamic linking faster. You forgot: * Allow statically-linked programs to use dynamic NSS modules by forking a (dynamically-linked) resolver process when needed. This leads to a related, but widely disparaged option: * Have a persistent NSS caching daemon with an RPC interface that all programs can access for NSS lookups. You might call such a program `nscd'. (Might as well be honest about it.) Both of these options may incidentally help to resolve threading issues in the C library (although that would not be the preferred way of doing so). -GAWollman ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Bruce Evans wrote: It obviously uses NSS. How else could it be so bloated? : $ ls -l /sbin/init -r-x-- 1 root wheel 453348 Nov 18 10:30 /sbin/init I believe it's actually DNS, not NSS. Pre-5.0, the resolver ballooned significantly. A lot of the bloat in /bin and /sbin came from the NIS functions which in turn pull in the resolver. Example: /bin/date on 5.1 is also over 450k because of a single call to getservbyname(). Removing that one call shrinks a static /bin/date to a quite reasonable size. (I seem to recall 80k when I did this experiment.) I note that /sbin/init calls getpwnam(); I expect that's where the bloat gets pulled in. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Garrett Wollman wrote: On Fri, 21 Nov 2003 15:38:49 -0800, Tim Kientzle [EMAIL PROTECTED] said: There have been a lot of proposed solutions: * Rewrite NSS to not require dlopen(). * Rewrite dlopen() to not require dynamic linking. * Don't support NSS in /bin/sh. * Change the default script interpreter for rc and such. * Make dynamic linking faster. You forgot: * Allow statically-linked programs to use dynamic NSS modules by forking a (dynamically-linked) resolver process when needed. This leads to a related, but widely disparaged option: * Have a persistent NSS caching daemon with an RPC interface that all programs can access for NSS lookups. You might call such a program `nscd'. (Might as well be honest about it.) Jacques seems to be of the opinion that the current 'nscd' is not up to the task. He would know better than I. Tim ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, 21 Nov 2003, Tim Kientzle wrote: Bruce Evans wrote: It obviously uses NSS. How else could it be so bloated? : $ ls -l /sbin/init -r-x-- 1 root wheel 453348 Nov 18 10:30 /sbin/init I believe it's actually DNS, not NSS. Pre-5.0, the resolver ballooned significantly. A lot of the bloat in /bin and /sbin came from the NIS functions which in turn pull in the resolver. Perhaps both. Example: /bin/date on 5.1 is also over 450k because of a single call to getservbyname(). Removing that one call shrinks a static /bin/date to a quite reasonable size. (I seem to recall 80k when I did this experiment.) The 2 calls to logwtmp() must also be removed, at least now. I get the following text sizes: for /bin/date: RELENG_4: 137491 -current*: 93214 (* = getservbyname() and logwtmp() calls removed) -current: 371226 (only 412492 total, not 450K yet) I note that /sbin/init calls getpwnam(); I expect that's where the bloat gets pulled in. Yes, except it's only the latest 200+K of bloat (from 413558 bytes text to 633390). Before that there was 100+K of miscellaneous bloat relative to RELENG_4 (text size 305289 there). Before that there was another 200+K of bloat from implementing history. Compiling with -DNO_HISTORY removes history support and reduces the text size to 162538 (this is without getpwnam()). Then there is another 30K of mostly non-bloat for actual changes within /bin/sh, since compiling the FreeBSD-1 /bin/sh with current libraries gives a text size of 132966. Finally, IIRC the text size of the FReeeBSD-1 /bin/sh is 70K (total size 90K), so there is another 60K of miscellaneous bloat in current libraries to increase the text size from 70K to 130K. Total text sizes for /bin/sh's internals: FreeBSD-1 sh compiled with -current's compiler: 55350 current sh compiled with -current's compiler: 87779 87:55 is about right for the increased functionality. Bruce ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: William Josephson [EMAIL PROTECTED] People at Berkeley (and elsewhere) have done user studies to try to quantify this sort of thing. It is pretty clear that with modern hardware, most failures are due to human error. That's not to say that hardware and software faults aren't real problems, too, but it is more common that someone, say, pulls the wrong drive from the RAID-5 array, resulting in an unnecessary double disk fault. that means your raid 5 is bust. i've seen raid 5 fail and it just picks another disk in the 'free' pool like nothing has happened. a study? it's bleeding obvious. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, Nov 21, 2003 at 04:52:00PM -0500, Garance A Drosihn wrote: Shouldn't that be 'chmod +t /bin/sh' ??? b) I thought that you might want to have this an admin-only command, so nefarious users couldn't abuse it on a shared system. I would make one change to your proposal: Instead of having the pre-linked image wired in RAM, I'd allow it to be paged as for an ordinary process image. This means the overhead of starting a pre-linked process would be similar to starting a statically-linked executable. Once you remove the store image in RAM only requirement, I don't see that allowing an ordinary user to set the sticky bit would allow any new attacks. The user can only set the bit on their own files. The address space overhead of holding a pre-linked image is roughly the same as executing that image so allowing a user to create sticky executables is roughly the same as allowing the user to create an additional process running that executable. I'm certainly fine with it using +t. Now it's just a matter of figuring out how to implement it... :-) I thought it was obvious so I left it as an exercise for the reader :-). My guess is that if a pre-linked image didn't exist, imgact_elf() would run as normal, transferring control to ld-elf.so. ld-elf.so would do symbol binding as per normal (probably as if with LD_BIND_NOW was set) and would then invoke a new save_image() syscall just before it transferred control to main(). This would trigger the kernel to save the whole process image as COW. This image would be associated back to the original executable using a mechanism similar to that used for sharing the text segment. Next time that executable is execve()'d, imgact_elf() would note that there was a pre-linked image available and attach to it, rather than re-mapping the executable. It would pass a flag to ld-elf.so to inform the latter that it should just start main(). A few rough edges still need to be resolved: - ld-elf.so runs as the invoking user but the above model has it telling the kernel what the process image is - which shouldn't be trusted by a different uid. The easiest solution is to make save_image() a root- only syscall so pre-binding only works if the executable has been run by root. An alternative would be to have the process images tied into the user's credentials - this would mean more significant changes because there's nothing that currently maintains information about an executable on a per-uid basis outside the process. - ld-elf.so environment (eg LD_LIBRARY_PATH, LD_PRELOAD) impact. This gets messy - it is undesirable to totally inhibit pre-linking if either is set so maybe ld-elf.so needs to verify that the pre-linked image matches the result if the environment is taken into account. - ld-elf.so needs to make sure that the libraries haven't changed since the original image was mapped - to handle (eg) libc.so being replaced. - There needs to be some way for the VM system to throw away the pre-linked images if the system runs low on RAM+swap. An alternative approach might be to have a dummy 'placeholder' process which solely exists to hold the pre-linked image. This process would be started by invoking the executable with a special LD_PRELINK_ONLY flag. When the executable was later exec'd, the kernel would fork() the dummy process giving it the credentials, pid and ppid of the process being exec'd. I'm sure deeper thought will reveal problems... Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, Nov 19, 2003 at 09:27:55PM -0800, Tim Kientzle wrote: Richard Coleman wrote: It seems /bin/sh is the real sticking point. There is a problem here: Unix systems have historically used /bin/sh for two somewhat contradictory purposes: * the system script interpreter * as a user shell The user shell must be dynamically linked in order to support centralized administration. I personally see no way around that. Given that many users do rely on /bin/sh, it seems that /bin/sh must be dynamically linked. There are good reasons to want the system script interpreter statically linked. Maybe it's time to separate these two functions? I would be content to have a static /sbin/sh that is used as the system script interpreter for rc scripts, etc. And /usr/bin/sh as a user shell? --Stijn -- I'm not under the alkafluence of inkahol that some thinkle peep I am. It's just the drunker I sit here the longer I get. pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
On Wed, Nov 19, 2003 at 11:18:47AM -0700, Lyndon Nerenberg wrote: --On Wednesday, November 19, 2003 12:30 AM -0500 Garance A Drosihn [EMAIL PROTECTED] wrote: have a: chflags ldcache /bin/sh Shouldn't that be 'chmod +t /bin/sh' ??? Definitely. Why waste a new bit when there's already a perfectly good one that is (or was) defined for the purpose. As for the implementation, I presume the desired behaviour would be to snapshot the process image (make it copy-on-write) at the point where ld-elf.so invokes main(). You'd probably want LD_BIND_NOW behaviour to minimise the runtime overheads. I don't see any need for this to need massive amounts of RAM - there's no reason why the snapshot couldn't be paged normally. I think this would be a big win compared to what we have now - the full benefits of dynamic linking remain and most of the run-time binding overheads are removed. Of course it's not perfect. The snapshot image permanently occupies virtual space (RAM/swap). And there's still the PIC overhead - especially on register-starved architectures like the i386. I wonder how difficult this would be to implement in userland only? Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Jacques A. Vidrine [EMAIL PROTECTED] wrote: Finally, if we could call `dlopen' from statically-linked binaries, this wouldn't be an issue. One of the performance problems that John Dyson mentioned (the sparse dirtying of libc's data section) would still remain, because the whole of libc has to be linked into programs that use NSS. Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ THE WASH TO NORTH FORELAND: SOUTHWEST 4 OR 5, PERHAPS 6 FOR A TIME IN EVENING IN THAMES, VEERS NORTH TO NORTHEAST 2 OR 3 FROM NORTH, BACKS NORTHWEST LATER. OCCASIONAL RAIN, CLEARS ERRATICALLY FROM NORTH, EXCEPT FAR SOUTH. MODERATE OR GOOD, LOCALLY POOR IN SOUTH. SLIGHT , LOCALLY MODERATE IN SOUTH. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Stijn Hoop wrote: On Wed, Nov 19, 2003 at 09:27:55PM -0800, Tim Kientzle wrote: Maybe it's time to separate these two functions? I would be content to have a static /sbin/sh that is used as the system script interpreter for rc scripts, etc. And /usr/bin/sh as a user shell? I was thinking /bin/sh for the user shell and /sbin/sh for the system script interpreter. There must be a /bin/sh and given that there are many user scripts that rely on it, it seems that it must be a user shell. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On 2003-11-20 at 21:51:48 boyd, rounin wrote: think about a dynamically linked init(8) ... % sudo ldd /sbin/init /sbin/init: libutil.so.3 = /lib/libutil.so.3 (0x28074000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x2807f000) libc.so.5 = /lib/libc.so.5 (0x28097000) Yes, working fine here. What should the problem be? pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
From: Dimitry Andric [EMAIL PROTECTED] % sudo ldd /sbin/init /sbin/init: libutil.so.3 = /lib/libutil.so.3 (0x28074000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x2807f000) libc.so.5 = /lib/libc.so.5 (0x28097000) Yes, working fine here. What should the problem be? the day /lib gets smashed. you're building a house of cards. once, if /etc/init and /bin/sh and some other pieces where in place a smashed file-system could be easily fixed. now you have to have 3 shared libs and a viable /lib. do you want systems that work? or houses of cards? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, Nov 21, 2003 at 12:23:16AM +0100, boyd, rounin wrote: you're building a house of cards. once, if /etc/init and /bin/sh and some other pieces where in place a smashed file-system could be easily fixed. now you have to have 3 shared libs and a viable /lib. do you want systems that work? or houses of cards? Personally, I think init should be static, and can't think of any way it would benefit from shared libraries. I'm not qualified to comment on the various things people have said about /bin/sh. (Possibly irrelevant data point: Solaris 10, if it ever flies, will supposedly have only shared libraries.) Given that you've got a knob if you really care enough to change the default, static init and /rescue should be adequate to get past all the other bickering here, so please stop it already... -- Christopher Vance ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: Christopher Vance [EMAIL PROTECTED] Personally, I think init should be static, and can't think of any way it would benefit from shared libraries. plan 9 has everything static. the kernel compiles in about 20 seconds or less -- no compression -- and you can boot it off a floppy. if i can sit in /sys/src and type: mk install and have everything re-built (and i could do it for all the supported architectures) in minutes i have eliminated unnecessary complexity. if it's not there, it can't break. btw: say hi to maltby for me. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, Nov 21, 2003 at 12:23:16AM +0100, boyd, rounin wrote: From: Dimitry Andric [EMAIL PROTECTED] % sudo ldd /sbin/init /sbin/init: libutil.so.3 = /lib/libutil.so.3 (0x28074000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x2807f000) libc.so.5 = /lib/libc.so.5 (0x28097000) Yes, working fine here. What should the problem be? the day /lib gets smashed. This is only marginally different than /sbin/init getting smashed. If the root partition develops problems, you need to restore for tape. -- Steve ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: Steve Kargl [EMAIL PROTECTED] This is only marginally different than /sbin/init getting smashed. If the root partition develops problems, you need to restore for tape. tape? who uses tape? optical, my son. brother, can you spare a TU-16? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In message: [EMAIL PROTECTED] boyd, rounin [EMAIL PROTECTED] writes: : Yes, working fine here. What should the problem be? : : the day /lib gets smashed. /rescue/sh Warner ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
boyd, rounin wrote: From: Dimitry Andric [EMAIL PROTECTED] % sudo ldd /sbin/init /sbin/init: libutil.so.3 = /lib/libutil.so.3 (0x28074000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x2807f000) libc.so.5 = /lib/libc.so.5 (0x28097000) Yes, working fine here. What should the problem be? the day /lib gets smashed. you're building a house of cards. once, if /etc/init and /bin/sh and some other pieces where in place a smashed file-system could be easily fixed. now you have to have 3 shared libs and a viable /lib. do you want systems that work? or houses of cards? I would prefer to solve this problem using a fixit floppy or cdrom anyways. I don't think that creates a house of cards. My systems work just fine. But I've often wondered how frequently a production system has such problems. I've been a sysadmin for many years and can't remember this ever happening. It's much more common to blow a hard drive, or have flaky memory, etc. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
boyd, rounin wrote: From: Christopher Vance [EMAIL PROTECTED] Personally, I think init should be static, and can't think of any way it would benefit from shared libraries. plan 9 has everything static. the kernel compiles in about 20 seconds or less -- no compression -- and you can boot it off a floppy. if i can sit in /sys/src and type: mk install and have everything re-built (and i could do it for all the supported architectures) in minutes i have eliminated unnecessary complexity. if it's not there, it can't break. btw: say hi to maltby for me. plan9 doesn't count. It's so minimalistic, it's useless. It has many beautiful and brilliant ideas. But it's not useful to many people as a production system. It's a shame, really. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
plan9 doesn't count. It's so minimalistic, it's useless. well, try to think about non-minimalism when you're trying to track down a kernel bug in a zillion SLOC ... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Fri, 21 Nov 2003 03:37:20 +0100, boyd, rounin [EMAIL PROTECTED] said: well, try to think about non-minimalism when you're trying to track down a kernel bug in a zillion SLOC ... How about trying to think about FreeBSD when posting on the FreeBSD mailing-lists. -GAWollman ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
From: Bruce M Simpson [EMAIL PROTECTED] During my time in an investment bank, installations were usually hosed in this way by human error (systems administrators removing a file by accident, etc) ... yup, it's rare i've seen flakey h/w. but i do remember one sysadmin (when i was a contract sysadmin) who on day 2 chown'd the whole source tree to himself on a development m/c. ugly. there were backups but 'that would be too costly [in time]' to do a clean restore. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In message [EMAIL PROTECTED], boyd, rounin write s: From: Bruce M Simpson [EMAIL PROTECTED] During my time in an investment bank, installations were usually hosed in this way by human error (systems administrators removing a file by accident, etc) ... yup, it's rare i've seen flakey h/w. but i do remember one sysadmin (when i was a contract sysadmin) who on day 2 chown'd the whole source tree to himself on a development m/c. ugly. there were backups but 'that would be too costly [in time]' to do a clean restore. I've seen that too. An end user got permission from management to the root account, e.g. management ordered us to give her the root pw on a Tru64 box. She chowned every file on the system to herself. Very ugly indeed. Cheers, -- Cy Schubert [EMAIL PROTECTED]http://www.komquats.com/ BC Government . FreeBSD UNIX [EMAIL PROTECTED] . [EMAIL PROTECTED] http://www.gov.bc.ca/ .http://www.FreeBSD.org/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Tue, Nov 18, 2003 at 06:26:21PM -0800, Matthew Dillon wrote: :Our rationale for encouraging Gordon is as follows: : :1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want :to upgrade from 4-STABLE. Historically in 4.x, the / partition has :been very modest in size. One just simply cannot cram the bloat that :has grown in 5.x into a 4.x partition scheme. Of course there is the :venerable 'dump - clean install - restore' scheme, but we were looking :for something a little more user-friendly. This argument would apply to very old 4.x users but not to anyone who installed it as of March 2001. I bumped the nominal size of the root partition to 128MB in 1.98.2.7 of sysinstall/label.c. Don't you think that people are able to change defaults if they think thats appropriate? Prior to that Jordan had bumped the root partition size to 100MB in 1.98.2.3 in March 2001. It was 50MB before then, which is too small even for 4.x. Hm, then why do I have still room on my 50MB root partition? $ df Filesystem 1K-blocksUsed Avail Capacity Mounted on /dev/vinum/root49583 29040 1657764%/ Thats enough for installworld (if softupdates are turned off) Gunther ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: Garrett Wollman said: On Tue, 18 Nov 2003 18:07:32 -0500 (EST), [EMAIL PROTECTED] said: If the object is to maximally 'share', The object, AIUI, is for ~username expansion to work in the shells when the user stored somewhere defined by an external NSS module. I don't believe that there is anything else in a (sane) shell that cares. It is a fallacy that EVERYTHING (e.g. libc) needs to be dynamically linked to support that feature. A single, self contained library that might even be sparse WRT memory isn't going to hurt much. It's a fallacy that ANYTHING (except nscd/lookupd) needs to be dynamically linked to support that feature :-) -- jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/ Tel +44(0)117 9287088 Fax +44 (0)117 9287112 http://ioctl.org/jan/ User interface? I hardly know 'er! ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
[cc: dropped] I suppose I should comment on this thread, since I'm closely related to at least two of the rationales mentioned for moving towards an all-dynamically-linked system. (I would prefer to stay out of this thread. In my mind we've had all these arguments in various forums months ago and the issue was put to rest.) On Tue, Nov 18, 2003 at 05:06:06PM -0700, Scott Long wrote: 2. NSS support out-of-the-box: Again, this is a user-experience issue in that forcing NSS users to recompile world was seen as a potential roadblock to them. Some followups mentioned that a different implementation of NSS would not require dynamically linked binaries. This is technically true. Certainly, I explored that avenue. But practical considerations made that alternative a poor choice. Supporting NSS really also means supporting (in as much as possible) the existing base of NSS modules. These modules were all designed to be loaded by libc and invoked in-process. (nscd doesn't do what some seem to think it does, and in practice it is not well-loved.) Existing NSS modules also shaped some other decisions that I made (such as API entry points and thread awareness or lack thereof). Breaking with the fundamental designs of NSS as found on Solaris and Linux means practically starting from scratch when `porting' NSS modules. I'm sure someone more clever and with more time could make it work out-of-process. Based on my experience, I think the result would be overengineered. *shrug* Finally, if we could call `dlopen' from statically-linked binaries, this wouldn't be an issue. I'd really like to see dlopen support for statically-linked binaries, for NSS and many other applications. 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. Actually, not only binary security updates but any security updates, or bug fixes for that matter. If there is a bug in a library, fixing that bug on your system is much more straightforward if everything is dynamically linked. It is much easier to rebuild libc or libcrypto and restart applications then to have to go through an entire `make world'. It can be hard for many administrators to avoid `make world', because it is not always easy to ascertain what applications are using what APIs and libraries. There's been a lot of talk about performance, but for my environment all the workhorse applications are already dynamically linked. I'd guess that is the case for most FreeBSD users. And of course, most FreeBSD binaries--- in the base system, in ports, and commercially available--- are already dynamically linked. For the minority of users that it actually has a performance impact (show of hands?), of course they are sophisticated enough to build the affected binaries statically. Unless we are talking about /bin/sh, they probably already have to go through special measures to get a statically linked binary. Cheers, -- Jacques Vidrine NTT/Verio SME FreeBSD UNIX Heimdal [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In a message written on Wed, Nov 19, 2003 at 08:10:59AM -0600, Jacques A. Vidrine wrote: statically. Unless we are talking about /bin/sh, they probably already have to go through special measures to get a statically linked binary. Something has been bothering me about the whole /bin/sh function, and today when fixing a box I realized what it was. To boot a machine into single user mode you need a kernel, init, and /bin/sh (minimally). It would seem to me that alone is a good argument for those three things to be static. Yes, I'm one of those people who rebuilds kernels without modules. The kernel problem doesn't bother me as much, as you can always boot without loading modules, so it's not really a part of the required kernel is dynamic problem. Outside of these issues I have no problem with a dynamic root and a /rescue. -- Leo Bicknell - [EMAIL PROTECTED] - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - [EMAIL PROTECTED], www.tmbg.org pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
On Wed, Nov 19, 2003 at 09:19:50AM -0500, Leo Bicknell wrote: To boot a machine into single user mode you need a kernel, init, and /bin/sh (minimally). Roughly the same thing was bothering me last night. You get a chance to specify the shell when init is in the last phase of getting you to single-user mode so you can say /rescue/sh at that point. init is another story and I asked someone about that, they said it either is or will shortly be a loader option so you can override that to be /rescue/init that way. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In a message written on Wed, Nov 19, 2003 at 09:25:35AM -0500, Ken Smith wrote: Roughly the same thing was bothering me last night. You get a chance to specify the shell when init is in the last phase of getting you to single-user mode so you can say /rescue/sh at that point. init is another story and I asked someone about that, they said it either is or will shortly be a loader option so you can override that to be /rescue/init that way. Perhaps /rescue/sh should be the default when booting into single user. The more I think about init the more I don't like dynamic linking for it. init needs to have as few failure modes as possible. I do still think it's fine for all the other /bin and /sbin things. -- Leo Bicknell - [EMAIL PROTECTED] - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - [EMAIL PROTECTED], www.tmbg.org pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
: Don't you think that people are able to change defaults if they think : thats appropriate? : : Prior to that Jordan had bumped the root partition size to 100MB : in 1.98.2.3 in March 2001. It was 50MB before then, which is too : small even for 4.x. : : Hm, then why do I have still room on my 50MB root partition? : :$ df :Filesystem 1K-blocksUsed Avail Capacity Mounted on :/dev/vinum/root49583 29040 1657764%/ : : Thats enough for installworld (if softupdates are turned off) : : Gunther Try running installkernel and see what happens when it tries to rename the old kernel and modules and install a new one. Or try installing a kernel.debug (which is 14MB+) instead of a kernel. The point here is that just because you can barely squeeze the system into 50MB doesn't mean it's a good idea to. It might work for a very narrow use pattern, but it will create terrible problems if you ever tried to expand your use and the system defaults have to cover more then just generic users reasonably. This is why the default is no longer 50MB. -Matt Matthew Dillon [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, Nov 19, 2003 at 09:25:35AM -0500, Ken Smith wrote: On Wed, Nov 19, 2003 at 09:19:50AM -0500, Leo Bicknell wrote: To boot a machine into single user mode you need a kernel, init, and /bin/sh (minimally). Roughly the same thing was bothering me last night. You get a chance to specify the shell when init is in the last phase of getting you to single-user mode so you can say /rescue/sh at that point. init is another story and I asked someone about that, they said it either is or will shortly be a loader option so you can override that to be /rescue/init that way. set init_path=/rescue/init It's rather non-intuitive. It works, but having a static /sbin/init avoids having to muck around in the loader in order to get to the rescue bits. If you need the rescue bits, you pretty much always need to use /rescue/init anyway. A dynamicly linked /sbin/init just makes it harder to get to the rescue bits, so it makes sense to link init(8) staticly. Especially since there's no advantage to dynamic linking init(8) that compensates for the inconvience. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
--On Wednesday, November 19, 2003 12:30 AM -0500 Garance A Drosihn [EMAIL PROTECTED] wrote: have a: chflags ldcache /bin/sh Shouldn't that be 'chmod +t /bin/sh' ??? --lyndon ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, 19 Nov 2003, Marcel Moolenaar wrote: set init_path=/rescue/init If dynamic root were ready to be turned on, then /rescue/init would be in the default init_path. A dynamicly linked /sbin/init just makes it harder to get to the rescue bits, so it makes sense to link init(8) staticly. Especially since there's no advantage to dynamic linking init(8) that compensates for the inconvience. It obviously uses NSS. How else could it be so bloated? : $ ls -l /sbin/init -r-x-- 1 root wheel 453348 Nov 18 10:30 /sbin/init (My version is linked statically of course.) The NSS parts of init might not be needed in normal operation, but its hard to tell. Bruce ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Thu, Nov 20, 2003 at 06:27:31AM +1100, Bruce Evans wrote: set init_path=/rescue/init If dynamic root were ready to be turned on, then /rescue/init would be in the default init_path. I had that explained to me too. :-) There is a loop in sys/kern/init_main.c that probes for an init to run. But it only does what you want for cases of the files not existing or otherwise just totally not executable. It won't handle the started but then dumped core case the way it would need to if /sbin/init were to fail because of shared library problems. So if just relying on this mechanism it would either not work right (/sbin/init in the path before /rescue/init) or it would always start /rescue/init (/rescue/init before /sbin/init in the path). -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Thu, 20 Nov 2003, Bruce Evans wrote: On Wed, 19 Nov 2003, Marcel Moolenaar wrote: set init_path=/rescue/init If dynamic root were ready to be turned on, then /rescue/init would be in the default init_path. The fallback path only works if the exec() fails cleanly without actually starting the userspace code. Peter and I have been talking about improving failure mode handling by having rtld return a special error code if (getpid() == 1 oops_shlib_problem) and having special kernel handling, but that's complicated by the fact that presumably you'd then need to pick up the whole init thing in exit() rather than continuing it in start_init(). In the meantime, Gordon has committed a change to always link init statically. A dynamicly linked /sbin/init just makes it harder to get to the rescue bits, so it makes sense to link init(8) staticly. Especially since there's no advantage to dynamic linking init(8) that compensates for the inconvience. It obviously uses NSS. How else could it be so bloated? : $ ls -l /sbin/init -r-x-- 1 root wheel 453348 Nov 18 10:30 /sbin/init (My version is linked statically of course.) The NSS parts of init might not be needed in normal operation, but its hard to tell. There appear to be at least two dependencies there, from my reading: (1) getpwnam(root) so that we can check the root password when the console isn't marked as secure. (2) Calls to setusercontext() to set up resources when running rc, also for uid 0, which results in libutil looking up the user to find the class. I think that while home directory completion is useful for sh, it's safe to have a working assumption of a local root user in the password file, and that when prompting for a single user mode password, access to directory services is probably going to fail anyway. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
SL Date: Tue, 18 Nov 2003 17:06:06 -0700 (MST) SL From: Scott Long SL 3. Binary security updates: there is a lot of interest in providing a SL binary update mechanism for doing security updates. Having a dynamic SL root means that vulnerable libraries can be updated without having to SL update all of the static binaries that might use them. Although this doesn't help the upgrade process, what if one symbol (such as function name + CVS tag) were exported per function? One could check for a vulnerability by strings | grep funcname | inspect CVS tag. A more elegant approach would be to store such versioning in another segment and have a tool that understands the data, a la debugger symbols. On a different note: + Some of us have had a few bad experiences with glibc (granted, it's glibc) upgrades when the shell, cp, ls, et cetera are dynamically linked. + I put the shell of choice and all of SSH's guts on the root partition... if /usr gets clobbered, I still want to be able to boot and log in remotely. If / gets clobbered, I have bigger problems. :-) Eddy -- Brotsman Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
GAD Date: Tue, 18 Nov 2003 21:54:53 -0500 GAD From: Garance A Drosihn GAD Many freebsd users (me for one) are still living on a modem, GAD where even one bump of 1.5 meg is a significant issue... GAD GAD Remember that the issue we're talking about is security GAD updates, not full system upgrades. Everyone would want GAD the security updates, even if they're on a slow link. When security updates change but a few bytes, it seems that some xdiff- or rsync-like algorithm would be an apropriate way to distribute patches. Eddy -- Brotsman Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
:GAD Many freebsd users (me for one) are still living on a modem, :GAD where even one bump of 1.5 meg is a significant issue... :GAD :GAD Remember that the issue we're talking about is security :GAD updates, not full system upgrades. Everyone would want :GAD the security updates, even if they're on a slow link. : :When security updates change but a few bytes, it seems that some :xdiff- or rsync-like algorithm would be an apropriate way to :distribute patches. : : :Eddy :-- :Brotsman Dreger, Inc. - EverQuick Internet Division Security updates are a red hearing regardless because they are few and far between compared to even a modem-user's bandwidth (especially those modem users who are likely to read the security lists aren't going to care if it takes an hour to download a non optimal binary patch if it only happens a few times a year). -Matt ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, 19 Nov 2003, Ken Smith wrote: On Thu, Nov 20, 2003 at 06:27:31AM +1100, Bruce Evans wrote: set init_path=/rescue/init If dynamic root were ready to be turned on, then /rescue/init would be in the default init_path. I had that explained to me too. :-) There is a loop in sys/kern/init_main.c that probes for an init to run. But it only does what you want for cases of the files not existing or otherwise just totally not executable. It won't handle the started but then dumped core case the way it would need to if /sbin/init were to fail because of shared library problems. So if just relying on this mechanism it would either not work right (/sbin/init in the path before /rescue/init) or it would always start /rescue/init (/rescue/init before /sbin/init in the path). Oops, better add ... and error handling for init_path would be fixed -). I should have remembered this since I got bitten by it recently. I was trying to boot RELENG_3 and had a backup init that worked but that didn't help because there was an execable init earlier in the path. Bruce ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Gordon Tetlow wrote: On Tue, Nov 18, 2003 at 08:03:23PM -0500, [EMAIL PROTECTED] wrote: However, PAM and NSS 'tricks' really seem to be exactly that, and certainly worthy of special builds. However, that isn't necessary, yet still not building everything with a shared libc. Things like nss_ldap (which is used *heavily* at my place of employment) are some reasons that FreeBSD doesn't make it into more places. It was the reason why FreeBSD isn't being used here. Calling them 'tricks' (and succumbing to the name calling you wanted to avoid) doesn't change the fact that every major contender (IRIX, Solaris, Linux to name a few) all support this feature set. The fact that you can't easily do centralized authentication (nss_ldap and pam_ldap) with FreeBSD is a major show stopper. At my former employer, we built several very large systems that required centralized authentication using ldap. We had to use Linux, since none of the BSD's supported this correctly at the time. I don't really care whether everything is statically or dynamically linked. With the fast machines and huge disks these days, bloat is not much of an issue. But nss and pam need to work correctly. If the folks that are against dynamic linking have an alternate method to make this work, I'm all for it. But it needs to be more than theory. We need code. To be honest, I've never understood the (seemingly irrational) resistance against this change. Solaris made this change 10 years ago. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In the last episode (Nov 19), Richard Coleman said: I don't really care whether everything is statically or dynamically linked. With the fast machines and huge disks these days, bloat is not much of an issue. But nss and pam need to work correctly. If the folks that are against dynamic linking have an alternate method to make this work, I'm all for it. But it needs to be more than theory. We need code. To be honest, I've never understood the (seemingly irrational) resistance against this change. Solaris made this change 10 years ago. Not completely: $ uname -a SunOS pd1 5.9 Generic_112233-08 sun4u sparc SUNW,Ultra-Enterprise $ file /bin/sh /sbin/sh: ELF 32-bit MSB executable SPARC Version 1, statically linked, stripped $ file /sbin/* | grep statically | cut -d: -f1 | fmt /sbin/autopush /sbin/fdisk /sbin/jsh /sbin/mount /sbin/sh /sbin/soconfig /sbin/sync /sbin/umount /sbin/uname -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Dan Nelson wrote: In the last episode (Nov 19), Richard Coleman said: I don't really care whether everything is statically or dynamically linked. With the fast machines and huge disks these days, bloat is not much of an issue. But nss and pam need to work correctly. If the folks that are against dynamic linking have an alternate method to make this work, I'm all for it. But it needs to be more than theory. We need code. To be honest, I've never understood the (seemingly irrational) resistance against this change. Solaris made this change 10 years ago. Not completely: $ uname -a SunOS pd1 5.9 Generic_112233-08 sun4u sparc SUNW,Ultra-Enterprise $ file /bin/sh /sbin/sh: ELF 32-bit MSB executable SPARC Version 1, statically linked, stripped $ file /sbin/* | grep statically | cut -d: -f1 | fmt /sbin/autopush /sbin/fdisk /sbin/jsh /sbin/mount /sbin/sh /sbin/soconfig /sbin/sync /sbin/umount /sbin/uname I have no problem with FreeBSD doing something similar and leaving a few binaries static. I think most of the resistance to that was due to the increased complexity of the build system. It seems /bin/sh is the real sticking point. But if the compromise is to statically link /bin/sh, that would be cool with me. Other than tilde expansion not working when using nss_ldap, I can't think of any other problems. I consider that a minor blemish I could easily live with. Normal users will not generally have /bin/sh has their shell anyways. And I could always compile a dynamically linked version into /usr/bin if necessary. To be honest, 98% of the time that someone notices brokeness due to nss_ldap, it comes when using /bin/ls. Richard Coleman [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, 19 Nov 2003, Dan Nelson wrote: In the last episode (Nov 19), Richard Coleman said: I don't really care whether everything is statically or dynamically linked. With the fast machines and huge disks these days, bloat is not much of an issue. But nss and pam need to work correctly. If the folks that are against dynamic linking have an alternate method to make this work, I'm all for it. But it needs to be more than theory. We need code. To be honest, I've never understood the (seemingly irrational) resistance against this change. Solaris made this change 10 years ago. Not completely: $ uname -a SunOS pd1 5.9 Generic_112233-08 sun4u sparc SUNW,Ultra-Enterprise $ file /bin/sh /sbin/sh: ELF 32-bit MSB executable SPARC Version 1, statically linked, stripped $ file /sbin/* | grep statically | cut -d: -f1 | fmt /sbin/autopush /sbin/fdisk /sbin/jsh /sbin/mount /sbin/sh /sbin/soconfig /sbin/sync /sbin/umount /sbin/uname Seems to depend a bit on the install, maybe... raven:~ uname -a SunOS raven 5.8 Generic_108528-06 sun4u sparc SUNW,Ultra-250 raven:~ file /bin/sh /sbin/sh /bin/sh:ELF 32-bit MSB executable SPARC Version 1, dynamically linked, stripped /sbin/sh: ELF 32-bit MSB executable SPARC Version 1, statically linked, stripped Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Richard Coleman wrote: It seems /bin/sh is the real sticking point. There is a problem here: Unix systems have historically used /bin/sh for two somewhat contradictory purposes: * the system script interpreter * as a user shell The user shell must be dynamically linked in order to support centralized administration. I personally see no way around that. Given that many users do rely on /bin/sh, it seems that /bin/sh must be dynamically linked. There are good reasons to want the system script interpreter statically linked. Maybe it's time to separate these two functions? I would be content to have a static /sbin/sh that is used as the system script interpreter for rc scripts, etc. Tim Kientzle ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Tim Kientzle said: Richard Coleman wrote: It seems /bin/sh is the real sticking point. There is a problem here: Unix systems have historically used /bin/sh for two somewhat contradictory purposes: * the system script interpreter * as a user shell The user shell must be dynamically linked in order to support centralized administration. I personally see no way around that. Given that many users do rely on /bin/sh, it seems that /bin/sh must be dynamically linked. It isn't necessary for the shell to be dynamically linked (efficiency issue WRT the sparse allocations and greater COW overheads/etc) for the shell to programmatically link in a module for optional feature sets. This can even be placed under a libc call (which then wouldn't encumber the shell unless the feature was active and increase the footprint of generally all libc routines.) John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Unfortunate dynamic linking for everything
Guys, Please revisit the dynamic linking for everything. The cost for using shared libs in cases like shells actually is higher than statically linking (both in memory and in time.) It appears that there is a loss of VM understanding over time. Don't confuse the advantages of using shared libs on X windows (and environments like that) with the same advantages being non-existant when using shared libs on normal layout programs (shells are perhaps the worst general case, but not necessarily the worst in all ways.) Another issue: if you continue to support the dynamic linking mechanisms for special shared libs, but otherwise link statically, much of the high overhead of shared libs for-everything will still be mitigated. Just because there might be a need for a special shared lib, that doesn't justify using shared libs for everything (and add the cost of sparse memory allocation and significantly higher fork/exec times than necessary.) For a 'fun time', take a look at the process map in the deprecated /proc filesystem. Compare the size (complexity) of a shared program with a static program... It doesn't show all of the internal differences, but is an externally accessable (and benchmark free) exemplar. The only real reason for building various programs dynamic in order to gain the advantage of specific shared object would be an all or nothing type argument (a typical fallacy in most religious discussions.) It really doesn't make sense to arbitrarily cut-off a discussion especially when a decision might be incorrect. Perhaps the all-dynamic scheme had been decided upon so as to give a competitive performance advantage to those who rebuild everything (appropriate) static? :-). If there hadn't been a noticed increase in cost by using all-shared-libs, then the measurements were done incorrectly. If the decision is made based upon allowing for 1.5X (at least) times increase in fork/exec times, and larger memory usage (due to sparse allocations), then it would be best to have decided that performance isn't as important as it used to be (which it just might not be anymore.) Last time that I heard, disk space is still much much less expensive than main memory :-). Remember: the cost for disk space is nil nowadays, space only made obvious by an insanely small default root filesystem size allocation. (An 'insanely' small root filesystem is okay when that root is a mini-root recovery system, but the cost of an extra 500MB is rather small on a non-specialty system is nil compared to other resources.) I do use an all-dynamic configuration for certain embedded applications (but also a case where there are no seperated filesystems, and the memory usage isn't quite as important because of the fact that AVAILABLE features is more important than lots of concurrently running programs.) For the best sharing and quickest system response (both due to memory and raw program/image invocation times), at least make the shells static. (Sorry for any misspellings or grammar errors -- it is early in the morning... I'll probably not participate in any further discussion on this matter either, but it would be good to generally avoid loosing 'ancient', but still 'accurate' technical history.) John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? Warner ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
In the last episode (Nov 18), M. Warner Losh said:
In message: [EMAIL PROTECTED]
[EMAIL PROTECTED] writes:
: It really doesn't make sense to arbitrarily cut-off a discussion
: especially when a decision might be incorrect.
I'd say that good technical discussion about why this is wrong would
be good. However, emotional ones should be left behind. Except for
John's message, most of the earlier messages have been more emotional
than technical.
John, do you have any good set of benchmarks that people can run to
illustrate your point?
Pretty much any benchmark that you can build statically and dynamically
should suffice. I've attached a simple one that fills an array with
random numbers then qsorts them. make compare will generate three
graphs at the end: time spent loading the executable, time spent within
the loops, and total time. Note that both load and loop timings are
higher for the dynamic binary. I ran it on a busy system, which is why
there are so many outliers. Make sure you have
src/tools/tools/ministat installed someplace in your path.
Also see
http://lists.freebsd.org/pipermail/freebsd-current/2003-April/001106.html ,
where I had posted proc/pid/maps for a static and dynamic ls.
--
Dan Nelson
[EMAIL PROTECTED]
all: compare
REPS=50
STATICLOGS=static.total.log static.loop.log static.load.log
DYNAMICLOGS=dynamic.total.log dynamic.loop.log dynamic.load.log
LOGS=${STATICLOGS} ${DYNAMICLOGS}
CFLAGS+=-Wall
static: svd.o
${CC} -static ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${.ALLSRC} ${LDADD}
dynamic: svd.o
${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${.ALLSRC} ${LDADD}
clean:
rm -f static dynamic ${LOGS} *.o
.PHONY: run
run ${LOGS}: static dynamic
@rm -f ${LOGS}
@reps=0; while [ $$reps -lt ${REPS} ] ; do \
time -p ./static 21 static.loop.log | sed -ne '/real/s/real //p'
static.total.log ; \
time -p ./dynamic 21 dynamic.loop.log | sed -ne '/real/s/real //p'
dynamic.total.log ; \
echo -n . ; \
reps=$$(($$reps+1)) ; \
done
@echo
@paste static.total.log static.loop.log | awk '{print $$1 - $$2}'
static.load.log
@paste dynamic.total.log dynamic.loop.log | awk '{print $$1 - $$2}'
dynamic.load.log
compare: ${LOGS}
ministat -s static.load.log dynamic.load.log
ministat -s static.loop.log dynamic.loop.log
ministat -s static.total.log dynamic.total.log
#include stdio.h
#include stdlib.h
#include string.h
#include sys/time.h
#define SIZE 1024000
int comp(const void *a, const void *b)
{
return memcmp(a, b, sizeof(int));
}
int main(int argc, char *argv[])
{
int numbers[SIZE];
int i;
struct timeval tv1, tv2;
gettimeofday(tv1, NULL);
srand(5);
for (i = 0; i SIZE; i++)
{
numbers[i] = rand();
}
qsort(numbers, SIZE, sizeof(*numbers), comp);
gettimeofday(tv2, NULL);
timersub(tv2, tv1, tv1);
printf(%ld.%06ld\n, tv1.tv_sec, tv1.tv_usec);
return 0;
}
ministat -s static.load.log dynamic.load.log
x static.load.log
+ dynamic.load.log
+--+
| xx + |
| xx ++ |
| xx ++ |
| ++ |
| ++ |
| ++ |
| ++ |
| +++|
| x+++|
| x* |
| x*+ + |
| x**+ + ++ |
| x**+ ++ ++ +|
| *+++**+ ++ |
| xxx**+****+*++ ++* x +++ +|
||___M_A| |
| |__M__A_| |
+--+
N Min MaxMedian AvgStddev
x 96 0.001561 0.027646 0.003969 0.0049661042 0.0040137086
+ 96 0.003765 0.05356 0.00824 0.010588323 0.0072195082
Difference at 95.0% confidence
0.0056 +/- 0.00165239
113.212% +/- 33.2733%
Re: Unfortunate dynamic linking for everything
On Tue, 18 Nov 2003, M. Warner Losh wrote: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. I used to use all dynamic linkage, but switched to all static linkage (except for ports) when I understood John's points many year ago. It shouldn't be necessary to repeat the argmuments. John, do you have any good set of benchmarks that people can run to illustrate your point? Almost any benchmark that does lots of forks or execs, or uses libraries a lot will do. IIRC, 5-10% of my speedups for makeworld was from building tools static. Makeworld is not such a good benchmark for this as it used to be since it always builds tools static so the non-staticness of standard binaries doesn't matter so much. Perhaps it still matters for /bin/sh. Bruce ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
M. Warner Losh said: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? I'll see if I can find some of my old benchmarks (from memory, not disk -- lots of stuff has rotted away.). I had hoped to avoid doing much in the way of proof. Pointing to the much more complex process map along with the implication for significantly higher overhead :-). (The VM code generally gets slower with more complex process maps and more memory used. For smallish programs -- including those with a few shared libs, changing the VM code won't help much, because the cost is built in to the complexity of the process image.) If set-up correctly (trying to find facts, rather than proving a foregone conclusion), even the fork/exec tests on LMBENCH can be informative. The LMBENCH fork/exec benchmarks don't really measure EVERYTHING, and there appears to be a additional overhead beyond the old-time a.out, but they will likely still show some of the additional overhead. (Actually, simple fork/exec tests of simple programs don't show all of the additional overhead, and also bias the overhead results, but are one data point.) There might be a certain 'coolness' WRT dynamically linking everything, but the overhead is certainly measurable. If the object is to maximally 'share', then for shells the FreeBSD VM shares maximally without using shared libs (perhaps there is a lost know-how about how aggressively the FreeBSD VM implements parent/child and exec based sharing?) I'll try to put together a few simple benchmarks -- mostly from my defective memory. I had recently refreshed myself on this issue (but lost again due to disk hardware disasters and being clobbered by vinum problems -- which I have given up on.) Gimme a day or so -- I have several other things in my queue. John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: M. Warner Losh said: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? I'll see if I can find some of my old benchmarks (from memory, not disk -- lots of stuff has rotted away.). I had hoped to avoid doing much in the way of proof. Pointing to the much more complex process map along with the implication for significantly higher overhead :-). (The VM code generally gets slower with more complex process maps and more memory used. For smallish programs -- including those with a few shared libs, changing the VM code won't help much, because the cost is built in to the complexity of the process image.) I'm glad that real arguments are finally starting to surface =-) Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. 2. NSS support out-of-the-box: Again, this is a user-experience issue in that forcing NSS users to recompile world was seen as a potential roadblock to them. 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. 4. I've probably forgotten the other factors... As for performance, we felt that the typical use of FreeBSD did not fall into the category of doing forkbomb performance tests. While there might certainly be users that do continuously create lots of short-lived processes, we felt that the above benefits outweighed this, but left the door open for tailoring to this need (recompiling world with NO_DYNAMICROOT). If people really are concerned with the VM microbenchmarks associated with complex process maps, then hopefully this is a challenge to look for optimizations there. Scott ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Scott Long said: On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: M. Warner Losh said: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? I'll see if I can find some of my old benchmarks (from memory, not disk -- lots of stuff has rotted away.). I had hoped to avoid doing much in the way of proof. Pointing to the much more complex process map along with the implication for significantly higher overhead :-). (The VM code generally gets slower with more complex process maps and more memory used. For smallish programs -- including those with a few shared libs, changing the VM code won't help much, because the cost is built in to the complexity of the process image.) I'm glad that real arguments are finally starting to surface =-) Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. /bin and /sbin don't need to contain everything :-). Adding a full featured /rescue should also help to counterbalance this (but maybe the upgrade would be selective?) There are reasons for /usr/bin and /usr/sbin (along with /usr/local, /usr/share.) Disk space is CHEAP and grandfathering 20MB hard drives (or the yr2000 equivalent of 4GB hard drives) is nice -- but repartitioning them with more reasonable root allocations seems like a good idea anyway :-). 50MB root was silly when 2-4GB disks were common. Continuing to pay for a poorly made partitioning decision in the past seems to be unworthy (in my opinion.) 2. NSS support out-of-the-box: Again, this is a user-experience issue in that forcing NSS users to recompile world was seen as a potential roadblock to them. The cool thing about properly implemented shared libs is that not everything has to be shared. Even the kernel supports runtime loading/addition of modules, and the NSS sounds like a good candidate for a library feature. Burdening everything with the sparse .data associated with libc seems to be a waste (but, perhaps the performance that was carefully crafted into the codebase is no longer important?) Really -- it is possible that performance isnt' important anymore, and fully accepting the ongoing loss of performance for features (without careful tradeoffs) might be an appropriate strategy. (In the case of the NSS stuff, it shouldn't require everything to be built dynamic.) 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. The additional hole of exploiting the system through the shared libs is a negative tradeoff. 4. I've probably forgotten the other factors... As for performance, we felt that the typical use of FreeBSD did not fall into the category of doing forkbomb performance tests. Nope - even shell execution times (e.g. long shell scripts) will see a difference. Prejudicial comments like 'fork-bombs' and 'microbenchmarks' aren't really fair. Proper benchmarking is time consuming and analysis is tricky. System loading conditions are difficult to control (for experiments.) Note that alot of the lost performance is hidden by the VM code (e.g. one of the purposes for prezeroing on SMP box -- before SMP was fine grained.) The cost of sparse data and needless inheritance of modified pages has performance hits that are incredibly difficult to accurately (and honestly) measure. Anyone can create 'preordained' benchmark results. This is one reason why I am loathe to get into the FreeBSD performance game again -- it is much more difficult than a few micro-benchmarks :-) that simply measure a fork exec time. John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
At 17:06 18/11/2003 -0700, Scott Long wrote: Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. Of course, making / dynamic results in added complication of removing old libraries from /usr/lib, now that some of them have moved to /lib... 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. As far as I'm concerned, this is a non-issue. Identifying which static binaries need to be replaced is now a solved problem, replacing them is easy, and if binary patches are used, there is effectively no impact on bandwidth usage either. On the issue of performance, however: I know people have benchmarked fork-bombs, but has anyone done benchmarks with moderate numbers of long-lived, library-intensive, processes? It seems to me that dynamic linking could have caching advantages. Colin Percival ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
[EMAIL PROTECTED] wrote: Scott Long said: On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: M. Warner Losh said: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? I'll see if I can find some of my old benchmarks (from memory, not disk -- lots of stuff has rotted away.). I had hoped to avoid doing much in the way of proof. Pointing to the much more complex process map along with the implication for significantly higher overhead :-). (The VM code generally gets slower with more complex process maps and more memory used. For smallish programs -- including those with a few shared libs, changing the VM code won't help much, because the cost is built in to the complexity of the process image.) I'm glad that real arguments are finally starting to surface =-) Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. [snip] 2. NSS support out-of-the-box: Again, this is a user-experience issue in that forcing NSS users to recompile world was seen as a potential roadblock to them. [snip] 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. [snip] 4. I've probably forgotten the other factors... As for performance, we felt that the typical use of FreeBSD did not fall into the category of doing forkbomb performance tests. One of ther things he might have forgot to mention is dynamic tricks releated to PAM, which sorta falls in the same league as NSS working out of the box. It was worth mentioning IMHO. Besides, I see nothing preventing anybody from building their system with static worlds, and there is nothing stopping anybody from putting /rescue in the PATH before /bin or /sbin. __ __ _ | \/ | __ _ ___| |_ __ _ | |\/| |/ _` / __| __/ _` | | | | | (_| \__ \ || (_| | |_| |_|\__,_|___/\__\__,_| unzip ; strip ; touch ; finger ; mount ; fsck ; more ; yes ; umount ; sleep [EMAIL PROTECTED] http://wifibsd.org ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
masta said: One of ther things he might have forgot to mention is dynamic tricks releated to PAM, which sorta falls in the same league as NSS working out of the box. It was worth mentioning IMHO. I guess that I have to remember that my own goals of 'performance' and handling 'highest workload' for efficient use of hardware isn't everyone's goal. However, PAM and NSS 'tricks' really seem to be exactly that, and certainly worthy of special builds. However, that isn't necessary, yet still not building everything with a shared libc. Note that none of this requires that libc be shared (libc and its ilk are the worst offenders.) Dynamic loading can certainly be used (if you want to load a wierd, special purpose, tricky module.) John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: Scott Long said: On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: M. Warner Losh said: In message: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: : It really doesn't make sense to arbitrarily cut-off a : discussion especially when a decision might be incorrect. I'd say that good technical discussion about why this is wrong would be good. However, emotional ones should be left behind. Except for John's message, most of the earlier messages have been more emotional than technical. John, do you have any good set of benchmarks that people can run to illustrate your point? I'll see if I can find some of my old benchmarks (from memory, not disk -- lots of stuff has rotted away.). I had hoped to avoid doing much in the way of proof. Pointing to the much more complex process map along with the implication for significantly higher overhead :-). (The VM code generally gets slower with more complex process maps and more memory used. For smallish programs -- including those with a few shared libs, changing the VM code won't help much, because the cost is built in to the complexity of the process image.) I'm glad that real arguments are finally starting to surface =-) Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. /bin and /sbin don't need to contain everything :-). Adding a full featured /rescue should also help to counterbalance this (but maybe the upgrade would be selective?) There are reasons for /usr/bin and /usr/sbin (along with /usr/local, /usr/share.) Disk space is CHEAP and grandfathering 20MB hard drives (or the yr2000 equivalent of 4GB hard drives) is nice -- but repartitioning them with more reasonable root allocations seems like a good idea anyway :-). 50MB root was silly when 2-4GB disks were common. Continuing to pay for a poorly made partitioning decision in the past seems to be unworthy (in my opinion.) /boot has grown quite large too and threatens to be unbounded in size as times go on. Shaving off the 30-40MB of size in /bin and /sbin can help alleviate this, even on system formatted in 5.x partition sized. This argument wasn't the most compelling one by itself, but it played a part in the decision so I included it here. 2. NSS support out-of-the-box: Again, this is a user-experience issue in that forcing NSS users to recompile world was seen as a potential roadblock to them. The cool thing about properly implemented shared libs is that not everything has to be shared. Even the kernel supports runtime loading/addition of modules, and the NSS sounds like a good candidate for a library feature. Burdening everything with the sparse .data associated with libc seems to be a waste (but, perhaps the performance that was carefully crafted into the codebase is no longer important?) Really -- it is possible that performance isnt' important anymore, and fully accepting the ongoing loss of performance for features (without careful tradeoffs) might be an appropriate strategy. (In the case of the NSS stuff, it shouldn't require everything to be built dynamic.) Unfortunately, our dynamic linker prevents dlopen() and friends from working in a static binary. There was some talk about having NSS lookups happen through a proxy process, but our implementation just plain wasn't done that way, and no one has stepped forward with alternate code. NSS modules can affect a surprisingly large number of utilities, including the shells. Selectively choosing which utilities to link static vs. shared would have resulted in quite a patchy result that probably would not have had a significant benefit. 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. The additional hole of exploiting the system through the shared libs is a negative tradeoff. Exploits in libraries happen though. The LD_LIBRARY_PATH attack is an old one that most Unixes are hopefully hardened against. 4. I've probably forgotten the other factors... As for performance, we felt that the typical use of FreeBSD did not fall into the category of doing forkbomb performance tests. Nope - even shell execution times (e.g. long shell scripts) will see a difference.
Re: Unfortunate dynamic linking for everything
At 8:07 AM -0500 11/18/03, [EMAIL PROTECTED] wrote: It really doesn't make sense to arbitrarily cut-off a discussion especially when a decision might be incorrect. All I wanted to cut off was the claim that this decision had not been discussed publicly before. It was also annoying that most the recent complaints (before your message) were issues that had been explicitly discussed and addressed before the decision had been reached. Many people seem to be complaining on what they think we did, as opposed to what we actually did. If there hadn't been a noticed increase in cost by using all-shared-libs, then the measurements were done incorrectly. If the decision is made based upon allowing for 1.5X (at least) times increase in fork/exec times, and larger memory usage (due to sparse allocations), ... I do remember some comments about benchmarks, and it was true that the all-dynamic bin/sbin does come out slower. I don't remember if the benchmarks were ours or from NetBSD's investigation. However, I think we measured increase in overall time for some set of commands, instead of increase in the fork() routine. Thus, the penalty observed was much less than 50%. I think it was under 2%, but I don't remember the exact number. When we're dealing with a 100% increase in the cost of compiling something with the newer gcc, the increase due to this change seemed pretty insignificant... It is probably true that there are several commands where we would see a worthwhile performance benefit by static linking, and which have no need for things like dynamic PAM modules. But commands which care about userids or group information would need to stay dynamic (IMO), and I think that means all shells. Also, when it comes to security fixes, it would be nice for binary upgrades if all we had to do was replace one library, instead of replacing every command which is statically-linked with the problem routine. The announcement of this change may have played up disk-savings more than it should have, because we weren't doing this to save disk space. It was just that the disk savings were a very nice side-effect. It's pretty rare that we can talk about the system getting smaller!:-) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
At 6:42 PM -0600 11/18/03, masta wrote: Besides, I see nothing preventing anybody from building their system with static worlds, true... and there is nothing stopping anybody from putting /rescue in the PATH before /bin or /sbin. Note that this will not have the same performance as the older all-static setup, because each binary in /rescue is a crunchgen'ed combination of *all* the commands there. They're all hardlinked together. Eg: (332) # ls -l /bin/cp /rescue/cp -r-xr-xr-x1 root wheel 113372 Nov 16 02:29 /bin/cp* -r-xr-xr-x 131 root wheel 3807836 Nov 16 02:30 /rescue/cp* I have no idea what kind of a difference that would make, I'm just saying that it is different... :-) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Wed, 19 Nov 2003, Colin Percival wrote: At 17:06 18/11/2003 -0700, Scott Long wrote: Our rationale for encouraging Gordon is as follows: 1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want to upgrade from 4-STABLE. Historically in 4.x, the / partition has been very modest in size. One just simply cannot cram the bloat that has grown in 5.x into a 4.x partition scheme. Of course there is the venerable 'dump - clean install - restore' scheme, but we were looking for something a little more user-friendly. Of course, making / dynamic results in added complication of removing old libraries from /usr/lib, now that some of them have moved to /lib... The magic for solving this exact problem already exists in the source upgrade path. Fixing this for the binary upgrade path (via sysinstall) is still under investigation. It is mitigated by the fact the systems that are binary-upgraded have their library search path set to look at /lib first. Leaving around stale libraries is of course bad, and we will hopefully come up with a solution soon. 3. Binary security updates: there is a lot of interest in providing a binary update mechanism for doing security updates. Having a dynamic root means that vulnerable libraries can be updated without having to update all of the static binaries that might use them. As far as I'm concerned, this is a non-issue. Identifying which static binaries need to be replaced is now a solved problem, replacing them is easy, and if binary patches are used, there is effectively no impact on bandwidth usage either. Bandwidth is still a concern for a lot of people, and this has the potential to save significant bandwidth in many situations. On the issue of performance, however: I know people have benchmarked fork-bombs, but has anyone done benchmarks with moderate numbers of long-lived, library-intensive, processes? It seems to me that dynamic linking could have caching advantages. Colin Percival I can't comment well here. To my untrained mind, I would think that more sharing of libc would help with memory pressure, but I'm just making simple assumptions. Scott ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Scott Long said: On Tue, 18 Nov 2003 [EMAIL PROTECTED] wrote: The cool thing about properly implemented shared libs is that not everything has to be shared. Even the kernel supports runtime loading/addition of modules, and the NSS sounds like a good candidate for a library feature. Burdening everything with the sparse .data associated with libc seems to be a waste (but, perhaps the performance that was carefully crafted into the codebase is no longer important?) Really -- it is possible that performance isnt' important anymore, and fully accepting the ongoing loss of performance for features (without careful tradeoffs) might be an appropriate strategy. (In the case of the NSS stuff, it shouldn't require everything to be built dynamic.) Unfortunately, our dynamic linker prevents dlopen() and friends from working in a static binary. There was some talk about having NSS lookups happen through a proxy process, but our implementation just plain wasn't done that way, and no one has stepped forward with alternate code. It seems like the defect is with the old limitation on the dlopen things then. (That limitation has been operative for years.) Making that decision 6months ago (or whenever the decision was made) seems to result in sidestepping the best solution (esp when 6months is alot of time to fix such limitations.) Again, I cannot take the responsibility, and mostly trying to keep 'improvements' from undoing previous work. I am worried (really) that there'll be a progressive loss of performance in order to gain features that could have been implemented with less impact. There was ALOT of work in trying to make FreeBSD very performant, and throwing that away with incrementalism seems to be wasteful. I am NOT meaning to negatively criticize, because that is quite seductive, and I am NOT suggesting that FreeBSD should have the Plan9 mimalist ethic. :-). Favorite 'tricky' features will come and go -- but basic system performance gets harder and hard to recover as time goes on!!! I guess that all OSes have a size/performance curve where they get bigger and fatter, SOMETIMES (not always) losing previous attributes for new features. Just don't let the performance loss go too far. IMO, I wouldn't have supported everything be built shared if there was a POTENTIAL mitigating solution of a proper dlopen implementation. DG would likely have had a 'fit.' Perhaps there should be a FreeBSD performance (and footprint) responsibility officer before it becomes too bloated to recover? :-). John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
On Tue, Nov 18, 2003 at 08:03:23PM -0500, [EMAIL PROTECTED] wrote: However, PAM and NSS 'tricks' really seem to be exactly that, and certainly worthy of special builds. However, that isn't necessary, yet still not building everything with a shared libc. Things like nss_ldap (which is used *heavily* at my place of employment) are some reasons that FreeBSD doesn't make it into more places. It was the reason why FreeBSD isn't being used here. Calling them 'tricks' (and succumbing to the name calling you wanted to avoid) doesn't change the fact that every major contender (IRIX, Solaris, Linux to name a few) all support this feature set. -gordon pgp0.pgp Description: PGP signature
Re: Unfortunate dynamic linking for everything
Gordon Tetlow said: On Tue, Nov 18, 2003 at 08:03:23PM -0500, [EMAIL PROTECTED] wrote: However, PAM and NSS 'tricks' really seem to be exactly that, and certainly worthy of special builds. However, that isn't necessary, yet still not building everything with a shared libc. Things like nss_ldap (which is used *heavily* at my place of employment) are some reasons that FreeBSD doesn't make it into more places. It was the reason why FreeBSD isn't being used here. Calling them 'tricks' Firstly -- I was answering back the 'tricks' comment made that you had elided :-). Please quote the message that set-up the context for the usage. (and succumbing to the name calling you wanted to avoid) doesn't change the fact that every major contender (IRIX, Solaris, Linux to name a few) all support this feature set. As discussed before, it DEFINITELY isn't necessary to dynamically link EVERYTHING to implement your favorite feature. Not everyone needs PAM/NSS, even though everyone needs memory management and scarce resource allocation. Why build in the overhead for everyone, and make it UNNCESSARILY worse (e.g. dynamically link libc when you want NSS or PAM?) Of course, there was a development resource limitation, but the decision (discussion) was made approx 6months ago? (Enough time to solve the problem without a GLOBAL performance hit.) John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
:Our rationale for encouraging Gordon is as follows: : :1. 4.x upgrade path: As we approach 5-STABLE, a lot of users might want :to upgrade from 4-STABLE. Historically in 4.x, the / partition has :been very modest in size. One just simply cannot cram the bloat that :has grown in 5.x into a 4.x partition scheme. Of course there is the :venerable 'dump - clean install - restore' scheme, but we were looking :for something a little more user-friendly. This argument would apply to very old 4.x users but not to anyone who installed it as of March 2001. I bumped the nominal size of the root partition to 128MB in 1.98.2.7 of sysinstall/label.c. Prior to that Jordan had bumped the root partition size to 100MB in 1.98.2.3 in March 2001. It was 50MB before then, which is too small even for 4.x. :2. NSS support out-of-the-box: Again, this is a user-experience issue :in that forcing NSS users to recompile world was seen as a potential :roadblock to them. Users have to recompile the world anyway, I don't think NSS is an issue. Or to put it another way: Nobody in their right mind should be contemplating upgrading a 4.x system to 5.x for production purposes. There will simply be too much 4.x cruft left over. Upgrading really requires a wipe and reinstall in this instance. I seem to recall that the main reason 5.x went to a dynamic /bin was to work around a terribly broken PAM design. The other issues were just eye candy compared to the PAM problems. Personally speaking, I think the solution is to fix PAM instead :-) :3. Binary security updates: there is a lot of interest in providing a :binary update mechanism for doing security updates. Having a dynamic :root means that vulnerable libraries can be updated without having to :update all of the static binaries that might use them. Non-issue if you have an automatic security update mechanism or script to do the work for the user, which we don't. Even so, /bin and /sbin are sufficiently self-contained in the source hierarchy that recompiling and reinstalling them is not a big deal. I think the security argument is a red-herring because, frankly, security problems are far more prevalient with ports and larger services (Apache, sendmail, sshd, etc... mostly in /usr/bin and /usr/local), and not as big an issue for /bin and /sbin. :As for performance, we felt that the typical use of FreeBSD did not fall :into the category of doing forkbomb performance tests. While there might :certainly be users that do continuously create lots of short-lived :processes, we felt that the above benefits outweighed this, but left the :door open for tailoring to this need (recompiling world with :NO_DYNAMICROOT). fork() is a non-issue (forking overhead for a dynamic executable imposes a slight time penalty but does not really impose any resource penalty). However, I personally believe that anything run by a shell script is an issue in regards to performance, and anything you exec multiple separate copies of is an issue in regards to memory overhead. A system running a large number of JAIL'd environments will wind up with a larger number of swap-managed dirty pages. But, all this aside, the reason I am decidedly against a dynamic /bin is simply one of system recovery and safety. I've blown up /usr so many times over the years that had /bin and /sbin been dynamic I would have been in real trouble on many occassions. If I recall correctly, some people have proposed and/or implemented some sort of emergency fall back or safety bin in 5.x to make up for this issue. I would humbly submit that this simply acknowledges that a dynamic /bin and /sbin is a bad idea in the first place. I do not intend to make /bin or /sbin dynamic in DragonFly by default. I don't mind adding support for those people who want it, but I am not going to make it the default. -Matt Matthew Dillon [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
: :As far as I'm concerned, this is a non-issue. Identifying which static : binaries need to be replaced is now a solved problem, replacing them is : easy, and if binary patches are used, there is effectively no impact on : bandwidth usage either. : :Bandwidth is still a concern for a lot of people, and this has the :potential to save significant bandwidth in many situations. :.. :Scott I would not consider this a viable argument since binary downloads are usually compressed. A compressed /bin stacks up as follows (from 4.x): -rw-r--r-- 1 root wheel 4034560 Nov 18 18:34 /tmp/x1.tar static -rw-r--r-- 1 root wheel 849920 Nov 18 18:34 /tmp/x2.tar dynamic -rw-r--r-- 1 root wheel 1860215 Nov 18 18:34 /tmp/x1.tgz static -rw-r--r-- 1 root wheel 354576 Nov 18 18:34 /tmp/x2.tgz dynamic So you are talking about 1.5 MBytes less bandwidth, which is nothing compared to the cost of doing a full install over the net. Yah, yah, /sbin too... but you get the idea. It certainly isn't enough of a difference to justify going to a dynamic /bin and /sbin. I'm not saying there aren't other arguments, just that this isn't one of them. -Matt Matthew Dillon [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
:/boot has grown quite large too and threatens to be unbounded in size as :times go on. Shaving off the 30-40MB of size in /bin and /sbin can :help alleviate this, even on system formatted in 5.x partition sized. :... :This argument wasn't the most compelling one by itself, but it played a :part in the decision so I included it here. You aren't saving that much. I don't have a 5.x box handy but on a 4.x box a static /bin is only 4MB and /sbin is only 12MB. The dynamic versions will save you around 12MB is my guess. -Matt ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
Gang, I suspect that my position has been expressed adequately. Further discussion might become divisive, but a decision that incurs the overhead of performance or a rebuild on the default user base seems wrong (JUST MY OPINION.) It took ALOT of WORK (person years) to make FreeBSD perform as well as it does. BOTH the add-on crew and the general user base can have the performance and feature set without rebuilding, but the decision was apparently made to impose the cost of performance or rebuild and binary maintenance on the default user base. It makes more sense to have appropriately upgraded the system (by the NSS project) to avoid the performance hit by others and also provide the feature set. Apparently (I haven't fully analysed this) implementing the dlopen stuff for non-dynamic programs would have helped to mitigate this issue. (It might have put more burden on the NSS/PAM/whatever addon projects, but those are indeed addons that shouldn't take ANYTHING away from the rest of the project.) I am suggesting that the NSS crew and those who are concerned about performance can BOTH have the results that they wish for. 'All or nothing' creates divisiveness, and in these discussions it is TOO EASY to fall into that trap. I am not suggesting the loss of the new NSS stuff, but also suggest that ANY loss of performance when it can be avoided, is unwise. My opinion is known, and hopefully the loss of hard earned performance with person-years of work won't happen as time goes on. A little loss isn't that bad, but how much loss is too much loss (esp when not necessary?) EOT John ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Unfortunate dynamic linking for everything
At 6:38 PM -0800 11/18/03, Matthew Dillon wrote: So you are talking about 1.5 MBytes less bandwidth, which is nothing compared to the cost of doing a full install over the net. Yah, yah, /sbin too... but you get the idea. Many freebsd users (me for one) are still living on a modem, where even one bump of 1.5 meg is a significant issue... Remember that the issue we're talking about is security updates, not full system upgrades. Everyone would want the security updates, even if they're on a slow link. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
