Casper.Dik at Sun.COM wrote: > >I solved that problem already via merging both Solaris and ksh93 libcmd, > >see > >http://polaris.blastwave.org/browser/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported/us > r/src/lib/libcmd/ > >- it seems to work perfectly and the risk of a namespace clash is IMO > >tiny since all the new symbols start with "b_" (and libcmd is a private > >library anyway so we (or better: Sun) control all consumers (and can fix > >them on demand) ... :-) ). > > Ok; what's the cost for linking with this additional stuff > (startup cost, extra memory used, particularly unshared data/bss?
1. libcmd (and all the libraries I added, e.g. libshell, libdll and libast) use "lazyload" which means they only load libraries and symbols on demand so I assume startup time impact is small unless they start to reference symbols. 2. libcmd's size is now larger (however I am not sure what's the exact reason, maybe it's due the debug flags I used and/or the fact that tons of help message strings are in ksh93's version of libcmd... I'll take a look at that later today) -- snip -- ls -la /home/test001/ksh93/on_build002_m1/test1/proto/root_i386/lib/libcmd.so.1 /lib/libcmd.so.1 -rwxr-xr-x 1 test001 users 186060 Apr 26 22:24 /home/test001/ksh93/on_build002_m1/test1/proto/root_i386/lib/libcmd.so.1 -rwxr-xr-x 1 root bin 20840 Jan 23 2005 /lib/libcmd.so.1 -- snip -- Startup time is AFAIK not really important in this case as libcmd is always loaded in a multiuser system since a bunch of deamon processes use it (for example /usr/sbin/in.telnetd). And /usr/bin/ksh is also loaded very early at startup during the boot process so both are loaded and fixed in memory by their consumers. The size of unshared data did not really grow much thanks to -xstrconst which puts the string literals into shared read-only sections and therefore no longer account for unshared data. 3. The following applications in /usr/bin/ and /usr/sbin/ use /lib/libcmd.so.1 (the list is incomplete as there are AFAIK both libraries and applications outside /usr/bin/, /usr/sbin/ and even outside OS/Net (such as dtlogin etc.)): -- snip -- % find /usr/bin /usr/sbin -type f | while read i ; do a="$(ldd "$i" 2>/dev/null | fgrep "/lib/libcmd.so.1")" ; [ ! -z "$a" ] && echo "## $i" ; done ## /usr/bin/i86/newtask ## /usr/bin/i86/prctl ## /usr/bin/i86/prstat ## /usr/bin/i86/ps ## /usr/bin/i86/savecore ## /usr/bin/at ## /usr/bin/atq ## /usr/bin/atrm ## /usr/bin/auths ## /usr/bin/crontab ## /usr/bin/csh ## /usr/bin/getent ## /usr/bin/id ## /usr/bin/kbd ## /usr/bin/ksh ## /usr/bin/login ## /usr/bin/mkpwdict ## /usr/bin/netstat ## /usr/bin/newgrp ## /usr/bin/passwd ## /usr/bin/pfexec ## /usr/bin/pgrep ## /usr/bin/profiles ## /usr/bin/projects ## /usr/bin/renice ## /usr/bin/roles ## /usr/bin/su ## /usr/bin/pfcsh ## /usr/bin/pfksh ## /usr/bin/pkill ## /usr/bin/rksh ## /usr/bin/kdestroy ## /usr/bin/kinit ## /usr/bin/klist ## /usr/bin/kpasswd ## /usr/bin/ktutil ## /usr/bin/ftp ## /usr/bin/telnet ## /usr/bin/rcp ## /usr/bin/rdist ## /usr/bin/rlogin ## /usr/bin/rsh ## /usr/bin/finger ## /usr/bin/prodreg ## /usr/bin/ssh ## /usr/bin/ipcs ## /usr/bin/cdrw ## /usr/bin/pppd ## /usr/bin/mailq ## /usr/sbin/allocate ## /usr/sbin/audit ## /usr/sbin/auditconfig ## /usr/sbin/auditd ## /usr/sbin/auditreduce ## /usr/sbin/auditstat ## /usr/sbin/clri ## /usr/sbin/cron ## /usr/sbin/devfsadm ## /usr/sbin/df ## /usr/sbin/dminfo ## /usr/sbin/ff ## /usr/sbin/fsck ## /usr/sbin/halt ## /usr/sbin/keyserv ## /usr/sbin/passmgmt ## /usr/sbin/praudit ## /usr/sbin/syslogd ## /usr/sbin/tar ## /usr/sbin/useradd ## /usr/sbin/userdel ## /usr/sbin/usermod ## /usr/sbin/volcopy ## /usr/sbin/audlinks ## /usr/sbin/deallocate ## /usr/sbin/devlinks ## /usr/sbin/disks ## /usr/sbin/drvconfig ## /usr/sbin/list_devices ## /usr/sbin/ports ## /usr/sbin/poweroff ## /usr/sbin/reboot ## /usr/sbin/roleadd ## /usr/sbin/roledel ## /usr/sbin/rolemod ## /usr/sbin/tapes ## /usr/sbin/gsscred ## /usr/sbin/in.rexecd ## /usr/sbin/in.rlogind ## /usr/sbin/in.rshd ## /usr/sbin/pmconfig ## /usr/sbin/smpatch ## /usr/sbin/in.telnetd ## /usr/sbin/nisbackup ## /usr/sbin/nisldapmaptest ## /usr/sbin/nislog ## /usr/sbin/nisrestore ## /usr/sbin/rpc.nisd ## /usr/sbin/rpc.rexd ## /usr/sbin/poolbind ## /usr/sbin/zoneadm ## /usr/sbin/kadmin ## /usr/sbin/kadmin.local ## /usr/sbin/kdb5_util ## /usr/sbin/kproplog ## /usr/sbin/in.uucpd ## /usr/sbin/in.ftpd -- snip -- Thanks to that list libcmd.so.1 is more or less guranteed to be loaded and locked in memory in a multiuser system > It's used by most if not all daemon processes (I count 32 processes > using it on my system alone). > > Have you created .spec files for the ksh libraries? For libcmd, libdll and libshell I did that already - only libast is still on my ToDo list (which little bit more work as all the symbol names need to be harvested from tons of headers). > What other tools use these libraries? In general the new libraries (libshell, libdll, libast) should be private (at least for the initial ksh93 integration putback) to the various shell versions, e.g. "ksh", "pfksh", "rksh", "tksh" and "dtksh" and tools which use (or may use) shell-like functionality (for example "dbx", "zfs" etc.). libast may be an interesting exception since it provides lots of usefull utility functions including sfio which is a drop-in stdio replacement with much better performance. > (If it's only ksh, I am of the opinion that they need to be > statically linked to ksh and not shipped as libraries; if they're > used bby many other tools then, of course, not) Startup time of a dynamically linked version of ksh93 (called "./ksh" below) is not that bad compared to Solaris /usr/bin/ksh: -- snip -- # 1000 iterations calling /usr/bin/ksh to do some stuff % sync ; sync ; timex bash -c '(i=0 ; while [ $i -lt 1000 ] ; do (LD_LIBRARY_PATH=/home/test001/ksh93/on_build002_m1/test1/proto/root_i386/lib /usr/bin/ksh -c "printf \"\" ; true ; a=9000 ; b=$(($a - 1)) ; export b") ; i=$(($i + 1)) ; done)' real 2:18.45 user 17.13 sys 1:17.93 # 1000 iterations calling dynamically linked ksh93 to do some stuff % sync ; sync ; timex bash -c '(i=0 ; while [ $i -lt 1000 ] ; do (LD_LIBRARY_PATH=/home/test001/ksh93/on_build002_m1/test1/proto/root_i386/lib ./ksh -c "printf \"\" ; true ; a=9000 ; b=$(($a - 1)) ; export b") ; i=$(($i + 1)) ; done)' real 1:39.84 user 21.48 sys 52.80 -- snip -- Note: This is no fair comparisation, ksh93+libshell+libdll+libast are non-optimizsed debug builds and /usr/bin/ksh is an optimised non-debug execuable, the whole test happened within a VMware VM and shell+performance experts may complain that this is no real benchmark after all...) - but the dynamically linked version of ksh93 performs IMO very well in this case. > We don't want to get into some "Mozilla" DLL craze. > (If it isn't shared, it shouldn't be a shared library; and if it is shared, > it shouldn't be 40 libraries) I agree, however we're talking about three new libraries plus some new functions in libcmd. It is really not that bad as Mozilla with all it's modules and ever-changing API, also the size is much smaller (42MB statically linked Mozilla Seamonkey vs. ~~1.8MB statically linked ksh93 (debug; the stripped dynamically linked version of ksh93 is just 9k(!!))). ----- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)
