Re: remaking bash, trying static, glibc refuses static...?
Dave Rutherford wrote: There are loadable builtins but I don't believe it is true that they can magically self-load as you describe. --- Magically self load? Right... Installed by the tooth fairy no doubt? Anything that's setup to load has to be configured by *someone*...whether it is 'you', or 'someone else' , whether or not it is with or without your knowledge, and how 'transparent' it is to some generic 'end' user. Example. how many generic end users of bash would know that some bash versions in some distro's like opensuse, have full session auditing and logging added in to their build? Now if they happen to store that in a special version of glibc, and update glibc -- I magically get the new keyboard logging features w/summaries sent to "whoever". But, if I have a static build, that I put together -- and I'm "reasonable" satisfied that it's not communicating my behaviors out on the net, I can feel someone safe that an update to one of the 100's of loadable libraries won't compromise the security on that single program. Of course there are lots of other ways -- kernel loadable device modules that are closed source. (The plugins I was talking about -- like ones samba can use talk in *text*). Did the rest of your message have anything to do with bash? You were trying to build a static version... you've been told how... was there more? Naw... builtin key-stroke loggers in ... well suse does get many of their patches from Redhat, upstream absolutely nothing to do with bash... you can go back to sleep now. :-)
Re: remaking bash, trying static, glibc refuses static...?
On Wed, Aug 19, 2015 at 5:55 PM, Linda Walsh wrote: [cut a lot about dynamic linking and the linux kernel] > Even bash can load many of it's builtin's dynamically -- but if they > aren't there, it could use the same named-programs -- and I don't > think it "falls-over" and dies if it can't dynamically load a builtin -- > it just looks for the same name in the path. There are loadable builtins but I don't believe it is true that they can magically self-load as you describe. The closest this comes to the truth is that many builtins have corresponding programs in /bin, for the use of programs that don't have them builtin or for entities that might naïvely search the PATH looking for things which are legal to execute. (Some unices even include a /bin/cd for that reason, as pointless as that would otherwise be.) Did the rest of your message have anything to do with bash? You were trying to build a static version... you've been told how... was there more?
Re: remaking bash, trying static, glibc refuses static...?
Mike Frysinger wrote: On 18 Aug 2015 21:41, Linda Walsh wrote: Mike Frysinger wrote: On 18 Aug 2015 13:34, Linda Walsh wrote: (2) it's using the nss system which lets people drop modules into the system at anytime and change the overall lookups to use that. statically linking a specific subset would block that ability. --- The linux kernel is a perfect example of a statically linked program that can dynamically load plugins to provide authorization data from external sources. Static doesn't mean you can't support 3rd party plugins/libs -- like LDAP. which is what the current glibc code already does and why you get a warning when you link things staticlly. i'm not sure how you can argue both sides of the fence on this. -mike --- The current glibc code doesn't allow static linking (apart from a special static glibc build that I'd have to build myself). If the kernel can be linked statically, and yet load 3rd party plugins dynamically, why can't programs be linked statically with glibc (this usually pops up as an Error -- though that could be because many or most build procedures treat warnings as Errors). Also, you don't need a dynamic linker at run time to load plugins -- but glibc does because it patches addresses in the code to make direct calls into a dynamically loaded lib transparent. With the kernel it performs callouts to 'helpers' -- usually giving 1 key to a helper with the helper returning the associated value(s). It doesn't require dynamic linking -- but a helper script. Examples cifs.idmap (8) - Userspace helper for mapping ids for Common Internet F... cifs.upcall (8) - Userspace upcall helper for Common Internet File Syste... cifscreds (1)- manage NTLM credentials in kernel keyring getcifsacl (1) - Userspace helper to display an ACL in a security descr... idmapwb (8) - winbind ID mapping plugin for cifs-utils pam_cifscreds (8)- PAM module to manage NTLM credentials in kernel keyring setcifsacl (1) - Userspace helper to alter an ACL in a security descrip... None of those are required to be present when the kernel runs and the kernel doesn't die if something that would use one of those helpers is used but it isn't there. There are two types of dynamic linking... one where all libs must be present when the program loads, the other -- linux makes far too little use of. I got spoiled at SGI, as they used runtime dynamic linking to support different feature sets with different products. If you have an admin tool to add users, it might try to dlopen various independently offered products -- if they are not there, the program didn't show them on the menu -- it wasn't an error to have a dl-failure as it usually is w/glibc. Right now, if you try to load 'vim/gvim' on opensuse, you also have to have perl "libified" and openable at program invocation. You can't edit w/o the perl lib installed, but gvim doesn't require perl any more than it requires ruby, tcl or python that it can script with if such libs are present. If the libs are not present -- the core feature(s) of vim are still there -- you can edit. You may not be able to use alternate scripting languages while you edit, but you can at least start the editor and use it's standard features. W/glibc, if you wanted to do the same, you'd have it link and programs runnable with whatever was statically built in -- just like gvim -- if I don't have ruby/tcl/whatever installed, it says the language isn't available. glibc could return a list of available modules for a specific function and return soft-errors for missing components, with things like falling back to /etc/passwd &/| shadow being hard coded alternatives. But even if the dlopen call *can* be done right, I've yet to see a linux prog do it right and dynamically configure itself -- try an ldap auth w/no ldap plugin script? -- get a permission denied. glibc could use scriptable plugins like the kernel, but uses the more arcane dlopen for runtime loading -- that most projects get wrong -- and fall back to load-time (non-lazy) loading of dynamic libs. If you wanted to convince me glibc could be distributed statically, you could have it interface (as an example or proof of concept) with the userspace helpers the kernel uses for alternate authentication and permission mechanisms. But with the current mechanisms, if I'm not using NIS or LDAP, but "winbind" for login, I still need to have the NIS and LDAP libs available to run login. Right now it's like most progs require static linking at *load time*, which is the least flexible linking (programs don't auto-config "down" services that aren't available -- but you have to have all the kerberos and SASL, and every other possible security lib installed -- and that also allows separation of binaries and libraries to happen as people use symlinks in /bin/ls -> /usr/bin/ls when /usr isn't mounted and won't work without it's load-time dynamic libraries. Right no
Re: remaking bash, trying static, glibc refuses static...?
I sent this in response about 12 hours ago but haven't seen it come though on the list, so thought I'd resend it. Original Message Subject: Re: remaking bash, trying static, glibc refuses static...? Date: Wed, 19 Aug 2015 14:55:24 -0700 From: Linda Walsh <> To: bug-bash@gnu.org References: <55d106b2.9000...@tlinx.org> <20150817044932.GB1584@vapier> <55d396ef.8030...@tlinx.org> <20150819031852.GU1584@vapier> <55d408e4.9000...@tlinx.org> <20150819145815.GF1584@vapier> Mike Frysinger wrote: On 18 Aug 2015 21:41, Linda Walsh wrote: Mike Frysinger wrote: On 18 Aug 2015 13:34, Linda Walsh wrote: (2) it's using the nss system which lets people drop modules into the system at anytime and change the overall lookups to use that. statically linking a specific subset would block that ability. --- The linux kernel is a perfect example of a statically linked program that can dynamically load plugins to provide authorization data from external sources. Static doesn't mean you can't support 3rd party plugins/libs -- like LDAP. which is what the current glibc code already does and why you get a warning when you link things staticlly. i'm not sure how you can argue both sides of the fence on this. -mike --- The current glibc code doesn't allow static linking (apart from a special static glibc build that I'd have to build myself). If the kernel can be linked statically, and yet load 3rd party plugins dynamically, why can't programs be linked statically with glibc (this usually pops up as an Error -- though that could be because many or most build procedures treat warnings as Errors). Also, you don't need a dynamic linker at run time to load plugins -- but glibc does because it patches addresses in the code to make direct calls into a dynamically loaded lib transparent. With the kernel it performs callouts to 'helpers' -- usually giving 1 key to a helper with the helper returning the associated value(s). It doesn't require dynamic linking -- but a helper script. Examples cifs.idmap (8) - Userspace helper for mapping ids for Common Internet F... cifs.upcall (8) - Userspace upcall helper for Common Internet File Syste... cifscreds (1)- manage NTLM credentials in kernel keyring getcifsacl (1) - Userspace helper to display an ACL in a security descr... idmapwb (8) - winbind ID mapping plugin for cifs-utils pam_cifscreds (8)- PAM module to manage NTLM credentials in kernel keyring setcifsacl (1) - Userspace helper to alter an ACL in a security descrip... None of those are required to be present when the kernel runs and the kernel doesn't die if something that would use one of those helpers is used but it isn't there. There are two types of dynamic linking... one where all libs must be present when the program loads, the other -- linux makes far too little use of. I got spoiled at SGI, as they used runtime dynamic linking to support different feature sets with different products. If you have an admin tool to add users, it might try to dlopen various independently offered products -- if they are not there, the program didn't show them on the menu -- it wasn't an error to have a dl-failure as it usually is w/glibc. Right now, if you try to load 'vim/gvim' on opensuse, you also have to have perl "libified" and openable at program invocation. You can't edit w/o the perl lib installed, but gvim doesn't require perl any more than it requires ruby, tcl or python that it can script with if such libs are present. If the libs are not present -- the core feature(s) of vim are still there -- you can edit. You may not be able to use alternate scripting languages while you edit, but you can at least start the editor and use it's standard features. W/glibc, if you wanted to do the same, you'd have it link and programs runnable with whatever was statically built in -- just like gvim -- if I don't have ruby/tcl/whatever installed, it says the language isn't available. glibc could return a list of available modules for a specific function and return soft-errors for missing components, with things like falling back to /etc/passwd &/| shadow being hard coded alternatives. But even if the dlopen call *can* be done right, I've yet to see a linux prog do it right and dynamically configure itself -- try an ldap auth w/no ldap plugin script? -- get a permission denied. glibc could use scriptable plugins like the kernel, but uses the more arcane dlopen for runtime loading -- that most projects get wrong -- and fall back to load-time (non-lazy) loading of dynamic libs. If you wanted to convince me glibc could be distributed statically, you could have it interface (as an example or proof of concept) with the userspace helpers the kernel uses for alterna
Re: remaking bash, trying static, glibc refuses static...?
On 18 Aug 2015 21:41, Linda Walsh wrote: > Mike Frysinger wrote: > > On 18 Aug 2015 13:34, Linda Walsh wrote: > > (2) it's using the nss system which lets people drop modules into the system > > at anytime and change the overall lookups to use that. statically linking a > > specific subset would block that ability. > --- > The linux kernel is a perfect example of a statically linked program that > can dynamically load plugins to provide authorization data from external > sources. Static doesn't mean you can't support 3rd party plugins/libs -- > like LDAP. which is what the current glibc code already does and why you get a warning when you link things staticlly. i'm not sure how you can argue both sides of the fence on this. -mike signature.asc Description: Digital signature
Re: remaking bash, trying static, glibc refuses static...?
Mike Frysinger wrote: On 18 Aug 2015 13:34, Linda Walsh wrote: Then can you give any technical reason why a static lib that uses no network services (i.e. running on a mini-root ) couldn't be made available for the various calls that currently claim "dynamic library support" is necessary. (1) http://www.akkadia.org/drepper/no_static_linking.html --- I've seen this -- much of it not applicable to a miniroot recovery situation. However one of his issues is he lists as a benefit is 'Security'. With a statically linked binary you have no hope of waking up to a non-working system because new shared libraries have changed all the behaviors on your system. Just ask how many MS users have this problem. He mentions security fixes -- If my system is in single user, it's not likely that I need them. I *like[d]* the old-school method of putting static binaries in /bin and and using /usr/[s]bin alternatives after boot -- and after /usr is mounted. But now, with the "benefits" of shared libraries for /bin/mount and /sbin/xfs_restore being in /usr/lib64, when the system boots it can't mount anything. Lovely -- shared binaries -- hate them all the more with making mount use shared libs only located in /usr. Brilliant. Oh, yes, I can copy them to /lib64 -- but if they wanted to do a "root" and /usr (bin+lib) merge, why not put them in /[s]bin & /lib[64] and put the "compatibility symlinks" in the /usr dirs pointing at their corresponding root dirs. But with dynamic linking, they are putting binaries and libraries in /usr/ while leaving symlinks in the /bin & /lib dirs. Yup... dynamic linking -- beautiful concept being used by in all the wrong ways. (2) it's using the nss system which lets people drop modules into the system at anytime and change the overall lookups to use that. statically linking a specific subset would block that ability. --- The linux kernel is a perfect example of a statically linked program that can dynamically load plugins to provide authorization data from external sources. Static doesn't mean you can't support 3rd party plugins/libs -- like LDAP. Tools like keyboard monitors, and background auditing would no longer work without LD_PRELOAD/PROFILE/AUDIT. Gee, now that's a shame. Most of my binaries I build shared -- I don't really care, but having a set of core binaries on a rescue partition makes sense. which means people using providers like ldap would be stuck with static binaries that don't work. Wrong -- the linux kernel is statically linked. It can use 3rd party security plugins for user auth & privs. https://www.gnu.org/software/libc/manual/html_node/Name-Service-Switch.html It's a description of how they did it to be opaque to users -- so developers, admins, hackers, crackers or law enforcement can easily put in new shivs in the next dynamic-lib update. Lovely. It Has happened with MS, you can't tell me it can't happen w/linux. it's already been provided. build glibc w/--enable-static-nss. --- Funny, my distro must have forgotten that option... I wonder if glibc is as easy to build as the kernel?
Re: remaking bash, trying static, glibc refuses static...?
On 18 Aug 2015 13:34, Linda Walsh wrote: > Then can you give any technical reason why a static > lib that uses no network services (i.e. running > on a mini-root ) couldn't be made available for > the various calls that currently claim "dynamic library > support" is necessary. (1) http://www.akkadia.org/drepper/no_static_linking.html (2) it's using the nss system which lets people drop modules into the system at anytime and change the overall lookups to use that. statically linking a specific subset would block that ability. which means people using providers like ldap would be stuck with static binaries that don't work. https://www.gnu.org/software/libc/manual/html_node/Name-Service-Switch.html i'm not going to debate the relevance of such a system nowadays as i don't care. purely pointing out that it's not a "political" issue (nor have you provided any references to back up your specious claim). > Seems simple enough to provide such a widely asked for > feature -- even if it has to be less functional/flexible > than the dynamic version (i.e. Gnu would have done the best > they could under the circumstances). it's already been provided. build glibc w/--enable-static-nss. -mike signature.asc Description: Digital signature
Re: remaking bash, trying static, glibc refuses static...?
Mike Frysinger wrote: it is not political, nor is it related to bash at all -mike Then can you give any technical reason why a static lib that uses no network services (i.e. running on a mini-root ) couldn't be made available for the various calls that currently claim "dynamic library support" is necessary. I know it is not just 'bash'. Googling for the subject shows it's a problem for many projects, so I find it very odd that such a static lib couldn't be provided. If an upstream DB provider (NSS, say), refuses to provide a static lib, then the static lib Gnu provided would exclude them, stating the reason why. Seems simple enough to provide such a widely asked for feature -- even if it has to be less functional/flexible than the dynamic version (i.e. Gnu would have done the best they could under the circumstances). But the bash option for static even lists the reason for such -- but with no way to actually use the option. *sigh*.
Re: remaking bash, trying static, glibc refuses static...?
it is not political, nor is it related to bash at all -mike signature.asc Description: Digital signature
remaking bash, trying static, glibc refuses static...?
bash-4.3/lib/readline/complete.c:2193: warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking Other syms: getpwnam, setpwent, endpwent. Is providing a static lib that reads the local machine's /etc/passwd file really beyond the capabilities of the glibc people? The configure comment says one would build bash static for 'miniroot operations by 'root' (formatting other partitions, restoring, etc...). I don't really see why dynamic libs are needed at a time that no other file systems may be mounted -- operating in like "run level 1 (single user). This has a political feel to it (only because I can't imagine any techncial reason to not provide it). Does anyone else ever try a static link on bash? My 'bash' is 'bigger than most distro bashes (1.2M), but everything but the libc stuff is builtin (all the builtins & help files -- given that's all in there, 1.2M doesn't seem that large these days ldd /bin/bash linux-vdso.so.1 (0x7ffd09d86000) libdl.so.2 => /lib64/libdl.so.2 (0x00300140) libc.so.6 => /lib64/libc.so.6 (0x003000c0) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00300400) /lib64/ld-linux-x86-64.so.2 (0x5612dbf11000)
