Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-11 Thread Xavier Bestel
On Fri, 2007-06-08 at 20:30 +0100, Alan Cox wrote: > > To avoid exactly the kind of problem we have now in future: programs > > relying on specific patterns. > > Which you seem to think is a bad thing, yet is actually a very good thing > because it means that crashes are repeatable and problems ar

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Davide Libenzi
On Sun, 10 Jun 2007, Paul Mackerras wrote: > > for (i = 0; i < NR_OPEN; i++) > > if (!fd_is_special_to_us(i)) > > close(i); > > > > Note that this is conceptually buggy, but occurs in several major C > > programming books, most of the major shells, and a lot of other > > softw

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Linus Torvalds
On Sun, 10 Jun 2007, Eric Dumazet wrote: > Linus Torvalds a écrit : > > > > close(0); > > close(1); > > close(2); > > .. generate filenames, whatever .. > > if (open(..) < 0 || open(..) < 0 || open(..) < 0) > > die("Couldn't redirect stdin/stdout/stderr"); > > >

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Linus Torvalds
On Sun, 10 Jun 2007, Al Viro wrote: > > On FreeBSD it will simply give you zero st_ino on almost all sockets; > AF_UNIX ones get st_ino invented (and stable). st_dev is NODEV in all > cases... So it will still work better than trying to do a "getsockname()" or something. If the file descripto

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Davide Libenzi
On Sun, 10 Jun 2007, Paul Mackerras wrote: > > for (i = 0; i < NR_OPEN; i++) > > if (!fd_is_special_to_us(i)) > > close(i); > > > > Note that this is conceptually buggy, but occurs in several major C > > programming books, most of the major shells, and a lot of other > > softw

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Davide Libenzi
On Sun, 10 Jun 2007, Paul Mackerras wrote: > What I am objecting to is this idea that many kernel developers seem > to have, that if there is some aspect of the kernel/user API that > becomes a bit inconvenient for the kernel to implement, then we can > put the blame on the applications that rely

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Alan Cox
> Real world example is nss_ldap / pam_ldap -- these needs open socket to > ldap server. That socket is cached. And because they can not trust that > application does not have closed file description of them, they check it with > getpeername + getsockname (at least it did when I looked code on

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Alan Cox
> > and there's absolutely nothing wrong with this kind of setup, even if you > > could obviously have done it other ways too (ie by using "dup2()" instead > > of "close + open"), > > > > This kind of setup was OK 25 years ago, before multithreading era. > You cannot reasonably expect it to wor

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread Eric Dumazet
Linus Torvalds a écrit : (And dammit, that _is_ a *real*issue*. No races necessary, no NR_OPEN iterations, no even *halfway* suspect code. It's perfectly fine to do close(0); close(1); close(2); .. generate filenames, whatever .. if (open(..) < 0 || open(

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-10 Thread William Lee Irwin III
On Sun, Jun 10, 2007 at 04:26:07PM +1000, Paul Mackerras wrote: > If you don't think we should be bound by POSIX, then you are perfectly > free to go off and write your own research kernel with whatever > interface you want, and no programs available to run on it. :) This isn't fair to research ke

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Paul Mackerras
Davide Libenzi writes: > > Why must everything that makes things a bit simpler and more > > predictable for application programmers be called a "mistake"? > > Because if you give guarantees on something, ppl start using such > guarantee in the wrong way. Kyle's email summarizes it. OK, my quest

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Paul Mackerras
Kyle Moffett writes: > 1) Linear FD allocation makes it IMPOSSIBLE for libraries to > reliably use persistent FDs behind an application's back. For That's not completely true; for example, openlog() opens a file descriptor for the library's own use, as does sethostent(). I agree that it cr

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Kari Hurtta
Linus Torvalds <[EMAIL PROTECTED]> writes in gmane.linux.kernel: <...> > But let's say that you want to do a library that does name resolution, and > you actually want to create the socket that binds to the DNS server just > once, and then re-use that socket across library calls. It's not that t

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 10:06:27PM -0700, Linus Torvalds wrote: > Why the hell doesn't that thing just do an "fstat()" on the thing, and > compare the inode number? Not that I would guarantee that it works either > for a socket, but it would seem to make more sense than what it apparently > does

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Linus Torvalds
On Sat, 9 Jun 2007, dean gaudet wrote: > > for an example of a library wanting to cache an open fd ... and failing > miserably at protecting itself from the application closing its fd read: Heh. Why the hell doesn't that thing just do an "fstat()" on the thing, and compare the inode number?

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread dean gaudet
On Sat, 9 Jun 2007, Linus Torvalds wrote: > IOW, the most common case for libraries is not that they get invoced to do > one thing, but that they get loaded and then used over and over and over > again, and the _reason_ for wanting to have a file descriptor open may > well be that the library w

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 09:03:23PM -0700, Linus Torvalds wrote: > > > On Sun, 10 Jun 2007, Al Viro wrote: > > > On Sat, Jun 09, 2007 at 08:48:39PM -0700, Linus Torvalds wrote: > > > Agreed. That was actually part of the reason why I thought clone() was > > > much better than the pthreads interfa

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Linus Torvalds
On Sun, 10 Jun 2007, Al Viro wrote: > On Sat, Jun 09, 2007 at 08:48:39PM -0700, Linus Torvalds wrote: > > Agreed. That was actually part of the reason why I thought clone() was > > much better than the pthreads interface. > > > > That said, the Linux !CLONE_FILES does have downsides: > > > >

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 08:48:39PM -0700, Linus Torvalds wrote: > Agreed. That was actually part of the reason why I thought clone() was > much better than the pthreads interface. > > That said, the Linux !CLONE_FILES does have downsides: > > - it is potentially much slower to do than sharing ev

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Davide Libenzi wrote: > On Sat, 9 Jun 2007, Linus Torvalds wrote: > > > On Sat, 9 Jun 2007, Al Viro wrote: > > > > > > Eww... Idea of pipe(2) taking flags as argument... > > > > Right. That was one of the patches, and it was one that I said was too > > damn ugly to live. >

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Linus Torvalds
On Sun, 10 Jun 2007, Al Viro wrote: > > > > And that means that libraries currently MUST NOT open their own file > > descriptors, exactly because they mess with the "application file > > descriptor namespace", namely the linear POSIX-defined fd allocation > > rules! > > Unless it does so in

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Linus Torvalds wrote: > On Sat, 9 Jun 2007, Al Viro wrote: > > > > Eww... Idea of pipe(2) taking flags as argument... > > Right. That was one of the patches, and it was one that I said was too > damn ugly to live. > > So I instead suggested the alternate approach of adding

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 01:21:24PM -0700, Linus Torvalds wrote: > (And dammit, that _is_ a *real*issue*. No races necessary, no NR_OPEN > iterations, no even *halfway* suspect code. It's perfectly fine to do > > close(0); > close(1); > close(2); > .. generate filenames,

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 03:27:43PM -0400, Kyle Moffett wrote: > SCENARIO 1: > > Program Thread: Library Thread: > fd = socket(AF_*, SOCK_*, 0); > fork(); > int x = FD_CLOEXEC; > fcntl(fd, F_SETFD, &x); BTW, regardless of anything else, in su

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Linus Torvalds
On Sat, 9 Jun 2007, Al Viro wrote: > > Eww... Idea of pipe(2) taking flags as argument... Right. That was one of the patches, and it was one that I said was too damn ugly to live. So I instead suggested the alternate approach of adding a single new system call that runs another system call

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Matt Mackall wrote: > On Sat, Jun 09, 2007 at 01:31:56PM -0700, Davide Libenzi wrote: > > On Sat, 9 Jun 2007, Linus Torvalds wrote: > > > > > So I think both the FD_CLOEXEC _and_ the "private fd space" are real > > > issues. I don't agree with the "random fd" approach. I'd mu

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Matt Mackall
On Sat, Jun 09, 2007 at 09:49:07PM +0100, Al Viro wrote: > On Sat, Jun 09, 2007 at 01:21:24PM -0700, Linus Torvalds wrote: > > Which is why you'd like to do the *initial* operation with a flag that > > says "please set the FD_CLOEXEC flag on the file descriptor", so that you > > *atomically* inst

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Matt Mackall
On Sat, Jun 09, 2007 at 01:31:56PM -0700, Davide Libenzi wrote: > On Sat, 9 Jun 2007, Linus Torvalds wrote: > > > So I think both the FD_CLOEXEC _and_ the "private fd space" are real > > issues. I don't agree with the "random fd" approach. I'd much rather have > > a non-random setup for the nonl

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 01:21:24PM -0700, Linus Torvalds wrote: > Which is why you'd like to do the *initial* operation with a flag that > says "please set the FD_CLOEXEC flag on the file descriptor", so that you > *atomically* install the file file descriptor and set the FD_CLOEXEC bit. > > It'

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Linus Torvalds wrote: > So I think both the FD_CLOEXEC _and_ the "private fd space" are real > issues. I don't agree with the "random fd" approach. I'd much rather have > a non-random setup for the nonlinear ones (it just shouldn't be linear). That is fine for me. So what ab

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Linus Torvalds
On Sat, 9 Jun 2007, Al Viro wrote: > > How the hell can it be racy wrt normal open()? F_DUPFD is not dup2(), > it's non-overriding. Al, you probably didn't read this thread from the beginning (not in this particular email thread - an earlier one on the whole feature). The problem is that a t

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 03:27:43PM -0400, Kyle Moffett wrote: > On Jun 09, 2007, at 13:24:29, Al Viro wrote: > >On Sat, Jun 09, 2007 at 10:08:59AM -0700, Ulrich Drepper wrote: > >>- - there are two interface to use: open + fcntl. This is racy. > >>And don't tell me this doesn't matter. > >Racy

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Kyle Moffett
On Jun 09, 2007, at 13:24:29, Al Viro wrote: On Sat, Jun 09, 2007 at 10:08:59AM -0700, Ulrich Drepper wrote: - - there are two interface to use: open + fcntl. This is racy. And don't tell me this doesn't matter. Racy with respect to what? Return-to-libc exploits from another thread? How

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 10:08:59AM -0700, Ulrich Drepper wrote: > - - there are two interface to use: open + fcntl. This is racy. And > don't tell me this doesn't matter. Racy with respect to what? Return-to-libc exploits from another thread? - To unsubscribe from this list: send the line "unsu

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Al Viro wrote: > That's simply not true. On the current kernel nothing stops you from e.g. > picking a random number and using F_DUPFD. Of course there are things stopping one from doing this (aside from the kernel not allowing this in the moment at

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Davide Libenzi wrote: > On Sat, 9 Jun 2007, Al Viro wrote: > > > That's simply not true. On the current kernel nothing stops you from e.g. > > picking a random number and using F_DUPFD. Voila - there's your randomized > > descriptor. Portable to earlier kernels. > > > > Mo

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Al Viro wrote: > That's simply not true. On the current kernel nothing stops you from e.g. > picking a random number and using F_DUPFD. Voila - there's your randomized > descriptor. Portable to earlier kernels. > > Moreover, nonsense^H^H^Hq_fd() can be implemented in userla

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Davide Libenzi
On Sat, 9 Jun 2007, Paul Mackerras wrote: > Davide Libenzi writes: > > > The only reason we use a floating base, is because Uli preferred to have > > non-exactly predictable fd allocations. There no reason of re-doing the > > same POSIX mistake all over again: > > Why must everything that make

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Sat, Jun 09, 2007 at 09:26:34AM -0700, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Al Viro wrote: > > So which code is supposed to do that open/write in your example? Library? > > Unmodified application? Application specifically modified to make *that* > > open

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Al Viro wrote: > So which code is supposed to do that open/write in your example? Library? > Unmodified application? Application specifically modified to make *that* > open() randomized? Why should that matter? All of the above. Any piece of code

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Al Viro
On Fri, Jun 08, 2007 at 07:10:03PM -0700, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Al Viro wrote: > > Any real-world examples of exploitable holes based on that? > > Return to libc exploit, calling dup2, where some privileged data is > redirected from the normal

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-09 Thread Kyle Moffett
On Jun 09, 2007, at 01:41:42, Paul Mackerras wrote: Davide Libenzi writes: The only reason we use a floating base, is because Uli preferred to have non-exactly predictable fd allocations. There no reason of re-doing the same POSIX mistake all over again: Why must everything that makes thin

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Paul Mackerras
Davide Libenzi writes: > The only reason we use a floating base, is because Uli preferred to have > non-exactly predictable fd allocations. There no reason of re-doing the > same POSIX mistake all over again: Why must everything that makes things a bit simpler and more predictable for applicati

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Al Viro wrote: > Any real-world examples of exploitable holes based on that? Return to libc exploit, calling dup2, where some privileged data is redirected from the normal file descriptor to one of the attackers choosing. The latter could be an outgo

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Al Viro
On Fri, Jun 08, 2007 at 06:19:28PM -0700, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Al Viro wrote: > > Exactly. Put it another way, randomizer is a stress-tester. > > ... and a security mechanism. And as such it is only useful if it is > used. Probably it shou

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Al Viro wrote: > Exactly. Put it another way, randomizer is a stress-tester. ... and a security mechanism. And as such it is only useful if it is used. Probably it should be policy-controlled whether you can turn it off. > Note that > #define NR_

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Al Viro
On Fri, Jun 08, 2007 at 05:03:57PM -0700, Linus Torvalds wrote: > > > On Fri, 8 Jun 2007, Davide Libenzi wrote: > > > > On Fri, 8 Jun 2007, Linus Torvalds wrote: > > > > > > You need things to be *repeatable* for debugging. No ifs, buts, or maybes > > > about it. > > > > It all depends on how

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Linus Torvalds wrote: > On Fri, 8 Jun 2007, Davide Libenzi wrote: > > > > On Fri, 8 Jun 2007, Linus Torvalds wrote: > > > > > > You need things to be *repeatable* for debugging. No ifs, buts, or maybes > > > about it. > > > > It all depends on how you use the file descriptor

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Linus Torvalds
On Fri, 8 Jun 2007, Davide Libenzi wrote: > > On Fri, 8 Jun 2007, Linus Torvalds wrote: > > > > You need things to be *repeatable* for debugging. No ifs, buts, or maybes > > about it. > > It all depends on how you use the file descriptor. Read what I wrote. "for debugging". If your code is b

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > > > > It does not work. What if the main application, library A and library B > > > > wants to implement their own allocation strategy? > > > > > > Its called "discipline". I would suggest that libc contains a default > > > allocator. You might also want to

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> > Unproven and dubious at best as a claim. > > I really don't mean to be rude and pointing you to read the archives, but > the proof and the reason why claims are valid is inside there. I've read the archive. I'm totally unconvinced by any of the fd allocation policy stuff. There are some good

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > > > #1: Throw the whole thing away and accept its not a good idea anyway > > > > Unfortunately (exactly because of the same guarantees you're asking for > > those handles), in order for userspace libraries to reliably internally > > use fds to interact with

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> > #1: Throw the whole thing away and accept its not a good idea anyway > > Unfortunately (exactly because of the same guarantees you're asking for > those handles), in order for userspace libraries to reliably internally > use fds to interact with the kernel, you need another kind of allocatio

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > > So, what do you plan to do? Those handle won't be zero-based. Your > > "working" system I immagine will do: > > > > bleeh[handle - BASE].duh = ...; > > > > How nice for a working system. If you *store* the handle returned by the > > OS, and you *use*

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> So, what do you plan to do? Those handle won't be zero-based. Your > "working" system I immagine will do: > > bleeh[handle - BASE].duh = ...; > > How nice for a working system. If you *store* the handle returned by the > OS, and you *use* the handle to call for OS services, you will be

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > On Fri, 08 Jun 2007 11:43:29 -0700 > Ulrich Drepper <[EMAIL PROTECTED]> wrote: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Alan Cox wrote: > > > Why does he want an unpredictable algorithm > > > > To avoid exactly the kind of problem we

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> We are talking about file descriptors here. If you're using file > descriptors as anything other than tokens you'll find out soon enough > that your code is broken. The new type of file descriptors cannot be > used as indeces and the randomization makes sure that no program by some > fluke happ

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
On Fri, 08 Jun 2007 11:43:29 -0700 Ulrich Drepper <[EMAIL PROTECTED]> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Alan Cox wrote: > > Why does he want an unpredictable algorithm > > To avoid exactly the kind of problem we have now in future: programs > relying on specific patter

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > > Errare humanum est, perseverare autem diabolicum > > Speaking latin didn't make the romans too wise either to look at their > history. If you really look at history, they were kinda wise. - Davide - To unsubscribe from this list: send the line "unsubsc

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Linus Torvalds wrote: > On Fri, 8 Jun 2007, Ulrich Drepper wrote: > > > > We are talking about file descriptors here. If you're using file > > descriptors as anything other than tokens you'll find out soon enough > > that your code is broken. The new type of file descriptors

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Linus Torvalds
On Fri, 8 Jun 2007, Ulrich Drepper wrote: > > We are talking about file descriptors here. If you're using file > descriptors as anything other than tokens you'll find out soon enough > that your code is broken. The new type of file descriptors cannot be > used as indeces and the randomization

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Al Viro wrote: > You are making debugging no end of fun... We are talking about file descriptors here. If you're using file descriptors as anything other than tokens you'll find out soon enough that your code is broken. The new type of file descript

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Al Viro
On Fri, Jun 08, 2007 at 11:43:29AM -0700, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Alan Cox wrote: > > Why does he want an unpredictable algorithm > > To avoid exactly the kind of problem we have now in future: programs > relying on specific patterns. You are m

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alan Cox wrote: > Why does he want an unpredictable algorithm To avoid exactly the kind of problem we have now in future: programs relying on specific patterns. - -- ➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖ -BEGIN PG

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Linus Torvalds
On Fri, 8 Jun 2007, Theodore Tso wrote: > > ... and it's not like sys_close() or sys_open() is a majorly critical > path, is it? open/close/stat/lstat are _the_ most important system calls, so yes, it's a majorly critical path. MUCH more so than opening a new TCP connection. You _may_ open a

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> The only reason we use a floating base, is because Uli preferred to have > non-exactly predictable fd allocations. There no reason of re-doing the > same POSIX mistake all over again: Why does he want an unpredictable algorithm - to make debugging hell because apps crash only for some patterns

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Alan Cox wrote: > > Um, how cheap do you need it? get_random_int() is actually pretty > > cheep, since it was designed to be usable by the networking stack for > > sequence numbers for TCP packets; and it's not like sys_close() or > > sys_open() is a majorly critical path, is

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Davide Libenzi
On Fri, 8 Jun 2007, Theodore Tso wrote: > On Thu, Jun 07, 2007 at 03:40:14PM -0700, Davide Libenzi wrote: > > Yes. Files with the CLOFORK and CLOEXEC flag do not count for fork and > > exec copies. > > I was also planning on doing it in __put_unused_fd(), every time > > fmap->count goes to zero.

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Jeff Dike
On Thu, Jun 07, 2007 at 01:10:23PM -0700, Linus Torvalds wrote: > HOWEVER. > > I think we could introduce a *single* new system call, which does > basically a "run the specified system call with the following flags". As long as we are considering indirecting system calls, how about reviving your

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Alan Cox
> Um, how cheap do you need it? get_random_int() is actually pretty > cheep, since it was designed to be usable by the networking stack for > sequence numbers for TCP packets; and it's not like sys_close() or > sys_open() is a majorly critical path, is it? If the concern is At this point wouldn'

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-08 Thread Theodore Tso
On Thu, Jun 07, 2007 at 03:40:14PM -0700, Davide Libenzi wrote: > Yes. Files with the CLOFORK and CLOEXEC flag do not count for fork and > exec copies. > I was also planning on doing it in __put_unused_fd(), every time > fmap->count goes to zero. But get_random_int() is not as cheap as I > thoug

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Fri, 8 Jun 2007, Eric Dumazet wrote: > Davide Libenzi a écrit : > > On Thu, 7 Jun 2007, Eric Dumazet wrote: > > > I am afraid randomization wont really work if /sbin/init or /bin/bash for > > > example uses one (or more) unseq fd : > > > The 'random base' will be propagated at fork()/exec() tim

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Andrew Morton wrote: > > Absolutely. Let's get them into their final form now, rather than letting > > some intermediate interface escape out into a major kernel release. > > Even if Linus' redirecti

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Eric Dumazet
Davide Libenzi a écrit : On Thu, 7 Jun 2007, Eric Dumazet wrote: I am afraid randomization wont really work if /sbin/init or /bin/bash for example uses one (or more) unseq fd : The 'random base' will be propagated at fork()/exec() time ? As I said to Uli, we can't move the base while fds are i

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew Morton wrote: > Absolutely. Let's get them into their final form now, rather than letting > some intermediate interface escape out into a major kernel release. Even if Linus' redirection is adopted, these interfaces should still be fixed up.

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Linus Torvalds
On Thu, 7 Jun 2007, Matt Mackall wrote: > > First, how does this work in-kernel? Does it set a flag in the thread > struct that magically gets used in the actual syscall? Or do we pass > flags down to the sys_foo() function in some manner? Set a flag in the thread-struct. In fact, that's how "

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Matt Mackall
On Thu, Jun 07, 2007 at 01:10:23PM -0700, Linus Torvalds wrote: > What do people think about that kind of approach? It has the advantage > that it does *not* involve multiple kernel entries (just a single entry to > a small wrapper that sets some process state temporarily), and that it > doesn't

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Linus Torvalds wrote: > > > On Thu, 7 Jun 2007, Davide Libenzi wrote: > > > > We'd still need sys_nonseqfd() though, to move/dup legacy fds into the > > non-sequential area. > > Umm. No we don't. Because it's no more than > > indirect_syscall(dup, FD_NONSEQ) > > i

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Linus Torvalds
On Thu, 7 Jun 2007, Davide Libenzi wrote: > > We'd still need sys_nonseqfd() though, to move/dup legacy fds into the > non-sequential area. Umm. No we don't. Because it's no more than indirect_syscall(dup, FD_NONSEQ) isn't it? Linus - To unsubscribe from this list:

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Davide Libenzi wrote: > > What we can sanily do, is re-random the base if no fds are in there (of > > course CLOFORK and CLOEXEC do not count). > > With the last comment you mean "count after CLOFORK

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Davide Libenzi wrote: > What we can sanily do, is re-random the base if no fds are in there (of > course CLOFORK and CLOEXEC do not count). With the last comment you mean "count after CLOFORK and CLOEXEC", right? So the re-basing would be done in tw

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Linus Torvalds wrote: > It has the disadvantage that it would need some per-architecture setup to > load the actual real arguments from memory: the system call would probably > look something like > > syscall_indirect(unsigned long flags, sigset_t *, >

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Eric Dumazet wrote: > Davide Libenzi a écrit : > > On Thu, 7 Jun 2007, Eric Dumazet wrote: > > > > > Davide, are you sure we want FIFO for non sequential allocations ? > > > > > > This tends to use all the fmap slots, and not very cache friendly > > > if an app does a lot of

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Eric Dumazet wrote: > > I am afraid randomization wont really work if /sbin/init or /bin/bash > > for example uses one (or more) unseq fd : > > The 'random base' will be propagated at fork()/exec() tim

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Linus Torvalds wrote: > On Thu, 7 Jun 2007, Eric Dumazet wrote: > > > > This is a nice idea, but 32/64 compat code is going to hate it :) > > It should be fairly simple for 32/64-bit compat code too. > > The compat code should just call the compat system call > > > syscall

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eric Dumazet wrote: > I am afraid randomization wont really work if /sbin/init or /bin/bash > for example uses one (or more) unseq fd : > The 'random base' will be propagated at fork()/exec() time ? The base certainly should be reset o fork. Yes, thi

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Linus Torvalds wrote: > syscall_indirect(unsigned long flags, sigset_t *, >int syscall, unsigned long args[6]); It's a kernel detail, so if this is how you want it, let it be. I can certainly live with this. The only c

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Linus Torvalds
On Thu, 7 Jun 2007, Eric Dumazet wrote: > > This is a nice idea, but 32/64 compat code is going to hate it :) It should be fairly simple for 32/64-bit compat code too. The compat code should just call the compat system call > syscall_indirect() would be writen in assembly for each arch, sinc

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Guillaume Chazarain
2007/6/7, Guillaume Chazarain <[EMAIL PROTECTED]>: Also, it sounds like a per-thread prctl, fun with syslets though ... ;-) Oops, even if the user takes care to reset the prctl after his code (the library problem), there are still signals that can run with an unexpected prctl. Sorry for the no

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Guillaume Chazarain
2007/6/7, Linus Torvalds <[EMAIL PROTECTED]>: syscall_indirect(unsigned long flags, sigset_t *, int syscall, unsigned long args[6]); Would that also be the user interface or all the wrappers would still be implemented by the glibc? Also, it sounds like a per-t

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Eric Dumazet
Linus Torvalds a écrit : On Wed, 6 Jun 2007, Alan Cox wrote: This still all seems really really ugly. I do agree that it's ugly. That many new system calls with new prototypes and new glibc support is just nasty. So I don't think this is viable. Is there anything wrong with throwing all t

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Linus Torvalds
On Wed, 6 Jun 2007, Alan Cox wrote: > > This still all seems really really ugly. I do agree that it's ugly. That many new system calls with new prototypes and new glibc support is just nasty. So I don't think this is viable. > Is there anything wrong with throwing all these extra cases out a

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Eric Dumazet
Davide Libenzi a écrit : On Thu, 7 Jun 2007, Eric Dumazet wrote: Davide, are you sure we want FIFO for non sequential allocations ? This tends to use all the fmap slots, and not very cache friendly if an app does a lot of [open(),...,close()] things. We already got a perf drop because of RCUi

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Andrew Morton
On Thu, 7 Jun 2007 11:39:39 -0700 (PDT) Davide Libenzi <[EMAIL PROTECTED]> wrote: > On Thu, 7 Jun 2007, Ulrich Drepper wrote: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Eric Dumazet wrote: > > > eventfd2(int count, int oflags); > > > > > > signalfd2(int ufd, sigset_t __user

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Davide Libenzi wrote: > Uli, would it be OK to rely only on base randomization and use a LIFO > instead? We have base randomization, plus LIFO does not mean strictly > sequential like legacy allocator, just more compatc and cache friendly. If FIFO i

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Eric Dumazet wrote: > Davide, are you sure we want FIFO for non sequential allocations ? > > This tends to use all the fmap slots, and not very cache friendly > if an app does a lot of [open(),...,close()] things. We already got a > perf drop because of RCUification of file f

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Davide Libenzi wrote: > >> And to add to your list: > >> > >> epoll_create(). Important if you think that's the interface people > >> should use. > > > > Can't we leave it as is, and use sys_nonseqfd

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Eric Dumazet
Davide Libenzi a écrit : On Thu, 7 Jun 2007, Eric Dumazet wrote: accept2(int fd, ...) I don't see any reason to not have it inherit the non-sequential characteristics of "fd". pipe2(int *fds, int oflags); I think pipe+sys_nonseqfd should be OK for those. yes, but O_CLOEXEC/O_CLOFORK

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Ulrich Drepper
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Davide Libenzi wrote: >> And to add to your list: >> >> epoll_create(). Important if you think that's the interface people >> should use. > > Can't we leave it as is, and use sys_nonseqfd() for those? So basically: The problem is O_CLOEXEC. They ar

Re: [patch 7/8] fdmap v2 - implement sys_socket2

2007-06-07 Thread Davide Libenzi
On Thu, 7 Jun 2007, Ulrich Drepper wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Eric Dumazet wrote: > > eventfd2(int count, int oflags); > > > > signalfd2(int ufd, sigset_t __user *user_mask, size_t sizemask, int oflags); > > > > timerfd2(int ufd, int clockid, int flags,const st

  1   2   >