Re: Package requiring a customised version of libc6
On Fri, Aug 24, 2007 at 12:23:39PM +0100, David Given wrote: > (Incidentally, the more I look at fakechroot the more I'm coming to believe > that it's no use for anything whatsoever. The security aspects of it are... > erm... nil; it's trivial for the client app to break out of its jail. Is this > a potential problem?) No, because it's not meant to provide security, just like fakeroot isn't meant to provide real root privs. - Matt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Package requiring a customised version of libc6
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lucas Nussbaum wrote: [...] > Then what about using ptrace and overriding syscalls in the way > usermodelinux used to do it? Yes, indeed; that is currently looking like the best approach. Not only does it provide the low-level interface that upstream wants, but it also works on statically bound binaries and on anything else that makes syscalls directly. I'm a little worried about performance, but it can't be that bad or UML wouldn't use it. I'll suggest it to upstream. Thanks for the link. (Incidentally, the more I look at fakechroot the more I'm coming to believe that it's no use for anything whatsoever. The security aspects of it are... erm... nil; it's trivial for the client app to break out of its jail. Is this a potential problem?) - -- ┌── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─── http://www.cowlark.com ─── │ │ "There does not now, nor will there ever, exist a programming language in │ which it is the least bit hard to write bad programs." --- Flon's Axiom -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGzr+7f9E0noFvlzgRAnMFAKCp0NxkOWgEW4XMNFeHg0CaViWlqwCg0S45 unlRqCTamPtiz0Q8tjZ2spU= =X2Ph -END PGP SIGNATURE-
Re: Package requiring a customised version of libc6
On Thursday 23 August 2007 17:26, David Given wrote: > Don Armstrong wrote: > [...] > > > The people who have responded to you so far strongly suspect that it's > > not worth the effort, but without knowing why the glibc we already > > distribute can't be used, it's hard for us to give you a definitive > > answer. > > *nods* > > As far as I can tell --- I've contacted upstream to confirm this --- what > plasticfs wants to do is to override the underlying system calls (or at > least, the functions that call them) that access the file system. > Unfortunately, those system calls are not exposed by default, so plasticfs > wants a tweaked glibc in which they are exposed. By overriding the system > calls rather than the higher-level functions, it means that plasticfs > doesn't have to override the higher-level functions --- they work > automatically. > > fakeroot and fakechroot appear to operate by overriding *all* glibc > functions that might access the file system. I've had a look at the code... > it's unpleasant. There are a lot of functions that might do this, and not > all of them are easily overridable, and quite a lot of them are rather > obscure. (I'd never even heard of ftw() and nftw() before now.). This makes > it much harder to catch coverage issues. A function that isn't wrapped will > work on the real file system, rather than the virtual one. I notice that > fakechroot doesn't wrap getpwent(), for example --- which means it'll > always use the *real* /etc/passwd, rather than the emulated one. This could > be intentional, but as it's not mentioned in the docs I suspect it's a bug. > > I've given up on the replacing-glibc idea; it was pretty horrible anyway. > Unfortunately, the alternatives seem just as horrible, in different ways... Programmers *always* have idiosyncrasies to deal with, no matter what system they work on. Windows, Linux, BSD, any *nix, DOS (if you dare). They've all got problems. Every library in existence has it's own idea of how things should work. Try using 2 libraries at once -- if they conflict in their ideas of how things should work, it's a disaster waiting to happen. Programmers always have, and always will, work around these problems. glibc makes assumptions about it's underlying implementation -- specifically, in this case, the system calls. The higher-level functions are specifically based on these system calls. To modify this is to take the feet out from under the higher-level functions -- they can't depend on their own implementation any more. With all due respect, this may work. But it more probably will introduce a thousand bugs into the higher-level functions. It also completely blows away a programmer's idiom: implementation privacy. C++ goes as far as to build this into the language. C doesn't, but it's a convention that should be adhered to. glibc's developers can -- and probably will -- change the underlying implementation and the higher-level functions' implementations to suit it's needs later. The devs feel they have the right to do this, and, they do. Simple as that. You're swimming with sharks if you depend on the underlying implementation (even worse if you hack to expose it). I would venture to say that this package's upstream maintainers are going at this with the wrong approach (whatever that approach is). This may be due to laziness (as is often the case with me :-), or it may be due to lack of time, or it may be due to lack of information (such as another library or something that they don't know about yet). Whatever the cause, the source needs to be fixed to work with the environment given, not vice versa. That attitude may seem rigorous, but it's really pretty simple: If glibc has to change for this one package, it's changed for the other 15,000 packages too. (yes, i know you don't want to install it, but things happen, and having 2 side-by-side copies of glibc is asking for trouble). So, to make one package work, you broke 15,000 packages. No matter how you run those numbers, it doesn't make sense -- the one package should change to match the other 15,000. Disclaimer: This is relatively uninformed conjecture. I haven't looked at the package's source, any of this implementation, or anything about implementing filesystems in general. I know about programming in general, and these are some basic assumptions I make any time I write a program. I don't know the rationale for the package's development model. So, I really am not trying to judge anyone. But I'm looking at what I do know with a critical eye, and I hope you'll do the same. -- Sincerely, Jack [EMAIL PROTECTED] My GPG Public Key can be found at: https://www.theanythingbox.com/pgp.htm (top link is current) I appreciate signatures, but if you only know me online, please use the --lsign-key, not the --sign-key. I appreciate trust -- but too much makes it less valuable. pgplxOhYTv4IU.pgp Description: PGP signature
Re: Package requiring a customised version of libc6
On 24/08/07 at 01:26 +0100, David Given wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Don Armstrong wrote: > [...] > > The people who have responded to you so far strongly suspect that it's > > not worth the effort, but without knowing why the glibc we already > > distribute can't be used, it's hard for us to give you a definitive > > answer. > > *nods* > > As far as I can tell --- I've contacted upstream to confirm this --- what > plasticfs wants to do is to override the underlying system calls Then what about using ptrace and overriding syscalls in the way usermodelinux used to do it? You could point your upstream to this article: Rapid File System Development Using ptrace Richard P. Spillane, Charles P. Wright, Gopalan Sivathanu, and Erez Zadok Should still be available from http://www.cs.huji.ac.il/~feit/exp/prog.html -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Package requiring a customised version of libc6
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Don Armstrong wrote: [...] > The people who have responded to you so far strongly suspect that it's > not worth the effort, but without knowing why the glibc we already > distribute can't be used, it's hard for us to give you a definitive > answer. *nods* As far as I can tell --- I've contacted upstream to confirm this --- what plasticfs wants to do is to override the underlying system calls (or at least, the functions that call them) that access the file system. Unfortunately, those system calls are not exposed by default, so plasticfs wants a tweaked glibc in which they are exposed. By overriding the system calls rather than the higher-level functions, it means that plasticfs doesn't have to override the higher-level functions --- they work automatically. fakeroot and fakechroot appear to operate by overriding *all* glibc functions that might access the file system. I've had a look at the code... it's unpleasant. There are a lot of functions that might do this, and not all of them are easily overridable, and quite a lot of them are rather obscure. (I'd never even heard of ftw() and nftw() before now.). This makes it much harder to catch coverage issues. A function that isn't wrapped will work on the real file system, rather than the virtual one. I notice that fakechroot doesn't wrap getpwent(), for example --- which means it'll always use the *real* /etc/passwd, rather than the emulated one. This could be intentional, but as it's not mentioned in the docs I suspect it's a bug. I've given up on the replacing-glibc idea; it was pretty horrible anyway. Unfortunately, the alternatives seem just as horrible, in different ways... - -- ┌── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─── http://www.cowlark.com ─── │ │ "There does not now, nor will there ever, exist a programming language in │ which it is the least bit hard to write bad programs." --- Flon's Axiom -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGziWjf9E0noFvlzgRAmPdAJkBBRbIetG6x/T23fKqDZtetrir+wCeP7fY t5NJukVanIgk7i8iZSW2W9E= =pOkz -END PGP SIGNATURE-
Re: Package requiring a customised version of libc6
On Fri, 24 Aug 2007, David Given wrote: > Currently I am merely trying to figure out whether upstream's idea > of using a customised glibc is possible on Debian It's always possible to do so. However, actually doing so requires that you convince the security team, the maintainer(s), and the release managers that having a duplicate copy of the glibc code and duplicating the associated work for any bugs found in glibc is worth the extra effort. The people who have responded to you so far strongly suspect that it's not worth the effort, but without knowing why the glibc we already distribute can't be used, it's hard for us to give you a definitive answer. Don Armstrong -- If a nation values anything more than freedom, it will lose its freedom; and the irony of it is that if it is comfort or money it values more, it will lose that, too. -- W. Somerset Maugham http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Package requiring a customised version of libc6
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Neil Williams wrote: [...] > Do the work and come back to the list with > a detailed reasoning for what is a MAJOR packaging decision. This isn't > "yet another customised version of a package" it is a COPY of GLIBC! Don't shout at me, please. Yes, I am entirely aware of all the issues you raise. However, at this point I am still collecting information. I have no intention of doing *anything* until I know exactly what's going on. Currently I am merely trying to figure out whether upstream's idea of using a customised glibc is possible on Debian; I suspect it's not, but as I haven't actually received an answer to my question yet, it's still rather up in the air... - -- ┌── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─── http://www.cowlark.com ─── │ │ "There does not now, nor will there ever, exist a programming language in │ which it is the least bit hard to write bad programs." --- Flon's Axiom -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGzht7f9E0noFvlzgRAgufAJ45Bb0AsCfnWnkAsLEmozlEZrPsRgCfZO3z k6pVrsqtl3pSWbObn5drGWY= =4jv2 -END PGP SIGNATURE-
Re: Package requiring a customised version of libc6
On Thu, 23 Aug 2007 22:26:35 +0100 David Given <[EMAIL PROTECTED]> wrote: > > Please can you give the details of why this is necessary? > > It's an LD_PRELOAD hack. When glibc calls itself --- for example when fopen() > calls open() --- it does so using a hidden private interface, which means the > preloader library doesn't get a chance to override it. plasticfs wants a glibc > compiled with --disable-hidden-plt to expose this interface. > > Or so the plasticfs docs say --- I'm sure this can be worked around, since > fakeroot and fakechroot manage it, but I wouldn't know how to do that. Also, > I'm working on the assumption that upstream at least know something about what > they're talking about... As Debian maintainer, you need to know a whole lot more about why upstream decided to work this way before adding a duplicate glibc to the archive. 1. Investigate fakeroot and fakechroot. Find out how they work. If you cannot work this out, don't package plasticfs. You are to be the Debian maintainer for this package, you MUST understand enough about how it works to be able to justify both having the package in Debian in the first place and your own packaging decisions. You can't just transfer the blame to the mentors list and say "I have almost no idea how or why *my* package requires a duplicate glibc but this is how I was told to do it on mentors.debian.net". Do the work and come back to the list with a detailed reasoning for what is a MAJOR packaging decision. This isn't "yet another customised version of a package" it is a COPY of GLIBC! Such things are not to be done lightly - especially when you appear to have very little understanding of why this would even be desirable. 2. Find out precisely why your package needs this interface - maybe it is just that upstream are trying to get around problems that don't apply to the Debian package, maybe there were problems building the upstream code on Fedora or OSX or something. Find out if any upstream developers use Debian. 3. Detail *PRECISELY* why your package needs this interface - what it does with it, why it cannot use the standard interface, which calls are made and why (including the filenames, function names and line numbers in the source code). Which parts of the functionality of the package are affected. How other packages deal with the same issue and *detailed* explanations of why your package cannot use those methods. What are the alternatives, which parts of the codebase need patching to use the standard glibc? Do the proposed patches work? Why not? 4. Do not assume that upstream have any idea of what happens inside a distribution. Upstream cares about upstream generally - they care about the .tar.gz, not the .deb or .rpm. Find out if there is a .rpm or maybe a gentoo emerge build. Work out how they do things. This ties into (3) because if you know which distro(s) upstream developers are using, you know where to start looking at how the upstream code is packaged elsewhere. 5. You will need a watertight argument to persuade the glibc maintainers that Debian needs a slightly customised version that will *always* be out of sync and out of step with the Debian package. Generally, such versions are simply BAD IDEAS that have not been thought through. Do *not* approach the glibc maintainers for advice at this stage - find out a whole lot more if you want to be taken seriously. Describing your main reason as a "hack" is not going to win you any friends amongst the glibc maintainers and you DO need to get them on your side. If you cannot provide sufficient, detailed, arguments to put your case, you could find an RC bug appearing against your package before it even leaves NEW. There is zero point searching for a sponsor for a package that is likely to be rejected upon upload so do your homework beforehand and work out if, how and why you need to do things this way. Then double-check all your arguments and finally, throw out all your reasons to copy glibc and find a way to work with the standard glibc. :-) -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpTyLnKOky9G.pgp Description: PGP signature
