Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-07 Thread Michael K. Edwards
An idiot using my keyboard wrote: - AIO requests that are serviced from cache ought to immediately invoke the callback, in the same thread context as the caller, fixing up the stack so that the callback returns to the instruction following the syscall. That way the "immediate completion" pat

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-07 Thread Michael K. Edwards
Man, I should have edited that down before sending it. Hopefully this is clearer: - The usual programming model for AIO completion in GUIs, media engines, and the like is an application callback. Data that is available immediately may be handled quite differently from data that arrives after

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Michael K. Edwards
On 2/6/07, Joel Becker <[EMAIL PROTECTED]> wrote: Not everything is in-cache. Databases will be doing O_DIRECT and will expect that 90% of their I/O calls will block. Why should they have to iterate this list every time? If this is the API, they *have* to. If there's an efficient way

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Joel Becker
On Tue, Feb 06, 2007 at 05:15:02PM -0800, Davide Libenzi wrote: > I think ATM the core kernel implementation should be the focus, because Yeah, I was thinking the same thing. I originally posted just to make the point :-) Joel -- Life's Little Instruction Book #99 "Think big

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Kent Overstreet
If that's what is wanted, then the async_submit() API can detect the syncronous completion soon, and drop a result inside the result-queue immediately. It means that an immediately following async_wait() will find some completions soon. Or: struct async_submit { void *cookie; int

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Kent Overstreet
On 2/6/07, Linus Torvalds <[EMAIL PROTECTED]> wrote: On Tue, 6 Feb 2007, Kent Overstreet wrote: > > The "struct aiocb" isn't something you have to or necessarily want to > keep around. Oh, don't get me wrong - the _only_ reason for "struct aiocb" would be backwards compatibility. The point is, w

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Davide Libenzi
On Tue, 6 Feb 2007, Joel Becker wrote: > > - Is it more expensive to forcibly have to wait and fetch a result even > > for in-cache syscalls, or it's faster to walk the submission array? > > Not everything is in-cache. Databases will be doing O_DIRECT > and will expect that 90% of their

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Joel Becker
On Tue, Feb 06, 2007 at 04:23:52PM -0800, Davide Libenzi wrote: > To how many "sessions" those 1000 *parallel* I/O operations refer to? > Because, if you batch them in an async fashion, they have to be parallel. They're independant. Of course they have to be parallel, that's what I/O wan

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Davide Libenzi
On Tue, 6 Feb 2007, Joel Becker wrote: > On Tue, Feb 06, 2007 at 03:56:14PM -0800, Davide Libenzi wrote: > > Async syscall submissions are a _one time_ things. It's not like a live fd > > that you can push inside epoll and avoid the multiple O(N) passes. > > First of all, the amount of syscalls t

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Joel Becker
On Tue, Feb 06, 2007 at 03:56:14PM -0800, Davide Libenzi wrote: > Async syscall submissions are a _one time_ things. It's not like a live fd > that you can push inside epoll and avoid the multiple O(N) passes. > First of all, the amount of syscalls that you'd submit in a vectored way > are limite

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Davide Libenzi
On Tue, 6 Feb 2007, Joel Becker wrote: > On Tue, Feb 06, 2007 at 03:23:47PM -0800, Davide Libenzi wrote: > > struct async_submit { > > void *cookie; > > int sysc_nbr; > > int nargs; > > long args[ASYNC_MAX_ARGS]; > > int async_result; > > }; > > > > int async_submit(struct asy

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Joel Becker
On Tue, Feb 06, 2007 at 03:23:47PM -0800, Davide Libenzi wrote: > struct async_submit { > void *cookie; > int sysc_nbr; > int nargs; > long args[ASYNC_MAX_ARGS]; > int async_result; > }; > > int async_submit(struct async_submit *a, int n); > > And async_submit() can

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Davide Libenzi
On Tue, 6 Feb 2007, Kent Overstreet wrote: > The trouble with differentiating between calls that block and calls > that don't is you completely loose the ability to batch syscalls > together; this is potentially a major win of an asynchronous > interface. It doesn't necessarly have to, once you e

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Linus Torvalds
On Tue, 6 Feb 2007, Kent Overstreet wrote: > > The "struct aiocb" isn't something you have to or necessarily want to > keep around. Oh, don't get me wrong - the _only_ reason for "struct aiocb" would be backwards compatibility. The point is, we'd need to keep that compatibility to be useful -

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Kent Overstreet
On 2/6/07, Linus Torvalds <[EMAIL PROTECTED]> wrote: On Mon, 5 Feb 2007, Kent Overstreet wrote: > > struct asys_ret { > int ret; > struct thread *p; > }; > > struct asys_ret r; > r.p = me; > > async_read(fd, buf, nbytes, &r); That's horrible. It means that "r" cannot have automatic linka

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Zach Brown
That's not how the patches work right now, but yes, I at least personally think that it's something we should aim for (ie the interface shouldn't _require_ us to always wait for things even if perhaps an early implementation might make everything be delayed at first) I agree that we shouldn'

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Linus Torvalds
On Tue, 6 Feb 2007, David Miller wrote: > > So the idea is to just run it to completion if it won't block and use > a fibril if it would? That's not how the patches work right now, but yes, I at least personally think that it's something we should aim for (ie the interface shouldn't _require_

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Eric Dumazet
David Miller a écrit : From: Linus Torvalds <[EMAIL PROTECTED]> Date: Tue, 6 Feb 2007 13:28:34 -0800 (PST) Yeah, in 1% of all cases it will block, and you'll want to wait for them. Maybe the kevent queue works then, but if it needs any more setup than the nonblocking case, that's a big no. S

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread David Miller
From: Linus Torvalds <[EMAIL PROTECTED]> Date: Tue, 6 Feb 2007 13:28:34 -0800 (PST) > Yeah, in 1% of all cases it will block, and you'll want to wait for them. > Maybe the kevent queue works then, but if it needs any more setup than the > nonblocking case, that's a big no. So the idea is to jus

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Linus Torvalds
On Tue, 6 Feb 2007, David Miller wrote: > > I really think that Evgeniy's kevent is a good event notification > mechanism for anything, including AIO. > > Events are events, applications want a centralized way to receive and > process them. Don't be silly. AIO isn't an event. AIO is an *action

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread David Miller
From: Linus Torvalds <[EMAIL PROTECTED]> Date: Tue, 6 Feb 2007 12:46:11 -0800 (PST) > And for user space, it means that we pass the _one_ thing around that we > need for both identifying the async operation to the kernel (the "cookie") > for wait or cancel, and the place where we expect the retu

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Linus Torvalds
On Mon, 5 Feb 2007, Kent Overstreet wrote: > > You don't need an explicit cookie if you're passing in a pointer to > the return code, it doesn't really save you anything to do so. Say > you've got a bunch of user threads (with or without stacks, it doesn't > matter). > > struct asys_ret { >

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Davide Libenzi
On Mon, 5 Feb 2007, Kent Overstreet wrote: > > > HOWEVER, they get returned differently. The cookie gets returned > > > immediately, the system call result gets returned in-memory only after the > > > async thing has actually completed. > > > > > > I would actually argue that it's not the kernel t

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-06 Thread Al Boldi
Linus Torvalds wrote: > On Mon, 5 Feb 2007, Zach Brown wrote: > > For syscalls, sure. > > > > The kevent work incorporates Uli's desire to have more data per event. > > Have you read his OLS stuff? It's been a while since I did so I've lost > > the details of why he cares to have more. > > You'd

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Joel Becker
On Mon, Feb 05, 2007 at 04:27:44PM -0800, Scot McKinley wrote: > Finally, it is agreed that neg-errno is a much better approach for the > return code. The threading/concurrency issues associated w/ the current > unix errno has always been buggy area for Oracle Networking code. As Scot kn

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread David Miller
From: Scot McKinley <[EMAIL PROTECTED]> Date: Mon, 05 Feb 2007 16:27:44 -0800 > As Joel mentioned earlier, from an Oracle perspective, one of the key > things we are looking for is a nice clean *common* wait point. How much investigation have the Oracle folks (besides Zach :-) done into Evgeniy'

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Davide Libenzi wrote: > On Mon, 5 Feb 2007, Linus Torvalds wrote: > > > Indeed. One word is *exactly* what a normal system call returns too. > > > > That said, normally we have a user-space library layer to turn that into > > the "errno + return value" thing, and in the case

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Scot McKinley
As Joel mentioned earlier, from an Oracle perspective, one of the key things we are looking for is a nice clean *common* wait point. We don't really care whether this common wait point is the old libaio:async-poll, epoll, or "wait_for_async". And if "wait_for_async" has the added benefit of s

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, David Miller wrote: > From: Davide Libenzi > Date: Mon, 5 Feb 2007 10:24:34 -0800 (PST) > > > Yes, no need for the above. We can just host a poll/epoll in an async() > > operation, and demultiplex once that gets ready. > > I can hear Evgeniy crying 8,000 miles away. > > I

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Linus Torvalds wrote: > On Mon, 5 Feb 2007, bert hubert wrote: > > > > From my end as an application developer, yes please. Either make it > > perfectly ok to have thousands of outstanding asynchronous system calls (I > > work with thousands of separate sockets), or allow me t

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
It has me excited in any case. Once anything even remotely testable appears (Zach tells me not to try the current code), I'll work it into MTasker (http://ds9a.nl/mtasker) and make it power a nameserver that does async i/o, for use with very very large zones that aren't preloaded. I'll be s

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread bert hubert
On Mon, Feb 05, 2007 at 01:57:15PM -0800, Linus Torvalds wrote: > I doubt very many people want to do that. It would tend to simply be nicer > to do > > async(poll); Yeah - I saw that technique being mentioned later on in the thread, and it would work, I think. To make up for the waste o

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Linus Torvalds
On Mon, 5 Feb 2007, bert hubert wrote: > > From my end as an application developer, yes please. Either make it > perfectly ok to have thousands of outstanding asynchronous system calls (I > work with thousands of separate sockets), or allow me to select/poll/epoll > on the "async fd". No can do

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread David Miller
From: Davide Libenzi Date: Mon, 5 Feb 2007 10:24:34 -0800 (PST) > Yes, no need for the above. We can just host a poll/epoll in an async() > operation, and demultiplex once that gets ready. I can hear Evgeniy crying 8,000 miles away. I strongly encourage a lot of folks commenting in this thread

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread bert hubert
On Fri, Feb 02, 2007 at 03:37:09PM -0800, Davide Libenzi wrote: > Since I still think that the many-thousands potential async operations > coming from network sockets are better handled with a classical event > machanism [1], and since smooth integration of new async syscall into the > standard

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Kent Overstreet
> HOWEVER, they get returned differently. The cookie gets returned > immediately, the system call result gets returned in-memory only after the > async thing has actually completed. > > I would actually argue that it's not the kernel that should generate any > cookie, but that user-space should *p

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
- we'd need to do it in the kernel (which is actually nasty, since different system calls have slightly different semantics - some don't return any error value at all, and negative numbers are real numbers) - we'd have to teach user space about the "negative errno" mechanism, in

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Linus Torvalds wrote: > Indeed. One word is *exactly* what a normal system call returns too. > > That said, normally we have a user-space library layer to turn that into > the "errno + return value" thing, and in the case of async() calls we > very basically wouldn't have th

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Linus Torvalds
On Mon, 5 Feb 2007, Zach Brown wrote: > > For syscalls, sure. > > The kevent work incorporates Uli's desire to have more data per event. Have > you read his OLS stuff? It's been a while since I did so I've lost the > details of why he cares to have more. You'd still do that as _arguments_ to

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Linus Torvalds
On Mon, 5 Feb 2007, Davide Libenzi wrote: > > No, that's *really* it ;) > > The cookie you pass, and the return code of the syscall. > If there other data transfered? Sure, but that data transfered during the > syscall processing, and handled by the syscall (filling up a sys_read > buffer just

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
No, that's *really* it ;) For syscalls, sure. The kevent work incorporates Uli's desire to have more data per event. Have you read his OLS stuff? It's been a while since I did so I've lost the details of why he cares to have more. Let me say it again, maybe a little louder this time: I

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Zach Brown wrote: > > The "result" of one async operation is basically a cookie and a result > > code. Eight or sixteen bytes at most. > > s/basically/minimally/ > > Well, yeah. The patches I sent had: > > struct asys_completion { >longreturn_code; >

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
The "result" of one async operation is basically a cookie and a result code. Eight or sixteen bytes at most. s/basically/minimally/ Well, yeah. The patches I sent had: struct asys_completion { longreturn_code; unsigned long cookie; }; That's as stupid as it gets

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Zach Brown wrote: > > Or we need some sort of enter_context()/leave_context() (adopt mm, files, > > ...) to have a per-CPU kthread to be able to execute the syscall from the > > async() caller context. > > I believe that's what Ingo is hoping for, yes. Ok, but then we should

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Zach Brown wrote: > > The normal and most optimal workflow should be a user-space ring-buffer > > of these constant-size struct async_syscall entries: > > > > struct async_syscall ringbuffer[1024]; > > > > LIST_HEAD(submitted); > > LIST_HEAD(pending); > > LIST_HEAD(comple

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
Or we need some sort of enter_context()/leave_context() (adopt mm, files, ...) to have a per-CPU kthread to be able to execute the syscall from the async() caller context. I believe that's what Ingo is hoping for, yes. - z - To unsubscribe from this list: send the line "unsubscribe linux-ke

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Zach Brown wrote: > > The 'pool' of kernel threads doesnt even have to be per-task, it can be > > a natural per-CPU thing > > Yeah, absolutely. Hmmm, so we issue an async sys_read(), what a get_file(fd) will return for a per-CPU kthread executing such syscall? Unless we teac

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Davide Libenzi
On Mon, 5 Feb 2007, Zach Brown wrote: > > Since I still think that the many-thousands potential async operations > > coming from network sockets are better handled with a classical event > > machanism [1], and since smooth integration of new async syscall into the > > standard POSIX infrastructure

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
+ current->per_call = next->per_call; Pointer instead of structure copy? Sure, there are lots of trade-offs there, but the story changes if we keep the 1:1 relationship between task_struct and thread_info. - z - To unsubscribe from this list: send the line "unsubscribe linux-kerne

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
But really, being a scheduler guy i was much more concerned about the duplication and problems caused by the fibril concept itself - which duplication and complexity makes up 80% of Zach's submitted patchset. For example this bit: [PATCH 3 of 4] Teach paths to wake a specific void * target wo

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
Since I still think that the many-thousands potential async operations coming from network sockets are better handled with a classical event machanism [1], and since smooth integration of new async syscall into the standard POSIX infrastructure is IMO a huge win, I think we need to have a "bri

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
ok, i think i noticed another misunderstanding. The kernel thread based scheme i'm suggesting would /not/ 'switch' to another kernel thread in the cached case, by default. It would just execute in the original context (as if it were a synchronous syscall), and the switch to a kernel thread from

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-05 Thread Zach Brown
Other questions really relate to the scheduling - Zach do you intend schedule_fibrils() to be a call code would make or just from schedule() ? I'd much rather keep the current sleeping API in as much as is possible. So, yeah, if we can get schedule() to notice and behave accordingly I'd p

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-03 Thread Davide Libenzi
On Tue, 30 Jan 2007, Zach Brown wrote: > + /* > + * XXX The idea is to copy all but the actual call stack. Obviously > + * this is wildly arch-specific and belongs abstracted out. > + */ > + *next->ti = *ti; > + *thread_info_pt_regs(next->ti) = *thread_info_pt_regs(ti)

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-03 Thread linux
First of all, may I say, this is a wonderful piece of work. It absolutely reeks of The Right Thing. Well done! However, while I need to study it in a lot more detail, I think Ingo's implementation ideas make a lot more immediate sense. It's the same idea that I thought up. Let me make it concre

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-03 Thread Ingo Molnar
* Matt Mackall <[EMAIL PROTECTED]> wrote: > On Sat, Feb 03, 2007 at 09:23:08AM +0100, Ingo Molnar wrote: > > The normal and most optimal workflow should be a user-space ring-buffer > > of these constant-size struct async_syscall entries: > > > > struct async_syscall ringbuffer[1024]; > > > >

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-03 Thread Matt Mackall
On Sat, Feb 03, 2007 at 09:23:08AM +0100, Ingo Molnar wrote: > The normal and most optimal workflow should be a user-space ring-buffer > of these constant-size struct async_syscall entries: > > struct async_syscall ringbuffer[1024]; > > LIST_HEAD(submitted); > LIST_HEAD(pending); > LIST_

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-03 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > On Sat, 3 Feb 2007, Ingo Molnar wrote: > > > > Well, in my picture, 'only if you block' is a pure thread > > utilization decision: bounce a piece of work to another thread if > > this thread cannot complete it. (if the kernel is lucky enough that

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Suparna Bhattacharya
On Fri, Feb 02, 2007 at 04:56:22PM -0800, Linus Torvalds wrote: > > On Sat, 3 Feb 2007, Ingo Molnar wrote: > > > > Well, in my picture, 'only if you block' is a pure thread utilization > > decision: bounce a piece of work to another thread if this thread cannot > > complete it. (if the kernel i

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Sat, 3 Feb 2007, Ingo Molnar wrote: > > Well, in my picture, 'only if you block' is a pure thread utilization > decision: bounce a piece of work to another thread if this thread cannot > complete it. (if the kernel is lucky enough that the user context told > it "it's fine to do that".) S

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread bert hubert
On Fri, Feb 02, 2007 at 03:17:57PM -0800, Linus Torvalds wrote: > threads. But you need to look at what it is we parallelize here, and ask > yourself why we're doing what we're doing, and why people aren't *already* > just using a separate thread for it. Partially this is for the bad reason tha

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > THAT'S THE POINT. That's what makes fibrils cooperative. The "only if > you block" is really what makes a fibril be something else than a > regular thread. Well, in my picture, 'only if you block' is a pure thread utilization decision: bounce a pi

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Davide Libenzi
On Fri, 2 Feb 2007, Davide Libenzi wrote: > On Fri, 2 Feb 2007, Ingo Molnar wrote: > > > in fact an app could also /trigger/ the execution of a syscall in a > > different context - to create parallelism artificially - without any > > blocking event. So we could do: > > > > cookie1 = sys_asyn

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Alan
> When parallelising "real work", I absolutely agree with you: we should use > threads. But you need to look at what it is we parallelize here, and ask > yourself why we're doing what we're doing, and why people aren't *already* > just using a separate thread for it. Because its a pain in the a

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Davide Libenzi
On Fri, 2 Feb 2007, Ingo Molnar wrote: > in fact an app could also /trigger/ the execution of a syscall in a > different context - to create parallelism artificially - without any > blocking event. So we could do: > > cookie1 = sys_async(sys_read, params); > cookie2 = sys_async(sys_write, p

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Linus Torvalds wrote: > On Fri, 2 Feb 2007, Ingo Molnar wrote: > > > > but if the application /has/ identified fundamental parallelism, we > > /must not/ shut that parallelism off by /designing/ this interface to > > use the fibril thing which is a limited cooperative, sin

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Ingo Molnar wrote: > > but if the application /has/ identified fundamental parallelism, we > /must not/ shut that parallelism off by /designing/ this interface to > use the fibril thing which is a limited cooperative, single-CPU entity. Right. We should for example encour

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Ingo Molnar wrote: > > Note: such a 'flip' would only occur when the original context blocks, > /not/ on every async syscall. Right. So can you take a look at Zach's fibril idea again? Because that's exactly what it does. It basically sets a flag, saying "flip to this whe

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > With cooperative scheduling (like the example Zach posted), there is > absolutely no "brown and sticky" wrt any CPU usage. Which is why > cooperative scheduling is a *good* thing. If you want to blow up your > 1024-node CPU cluster, you'd to it wit

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Alan
> > The brown and sticky will hit the rotating air impeller pretty hard if you > > are not very careful about how that ends up scheduled > > Why do you think that? > > With cooperative scheduling (like the example Zach posted), there is > absolutely no "brown and sticky" wrt any CPU usage. Which

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > On Fri, 2 Feb 2007, Ingo Molnar wrote: > > > > My only suggestion was to have a couple of transparent kernel threads > > (not fibrils) attached to a user context that does asynchronous > > syscalls! Those kernel threads would be 'switched in' if th

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Alan wrote: > > The brown and sticky will hit the rotating air impeller pretty hard if you > are not very careful about how that ends up scheduled Why do you think that? With cooperative scheduling (like the example Zach posted), there is absolutely no "brown and sticky" w

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Alan
> They are NOT the generic co-routine that some languages support natively. > So I think trying to call them coroutines would be even more misleading > than calling them fibrils. Its actually pretty damned close the Honeywell B co-routine package, with a kernel twist to be honest. > ends up loo

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Davide Libenzi wrote: > > Actually, coroutines are not too bad to program once you have a > total-coverage async scheduler to run them. No, no, I don't disagree at all. In fact, I agree emphatically. It's just that you need the scheduler to run them, in order to not "see"

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Davide Libenzi
On Fri, 2 Feb 2007, Linus Torvalds wrote: > > You get some other funny things from co-routines which are very powerful, > > very dangerous, or plain insane > > You forgot "very hard to think about". > > We DO NOT want coroutines in general. It's clever, but it's > (a) impossible to do without

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Alan wrote: > > This one got shelved while I sorted other things out as it warranted a > longer look. Some comments follow, but firstly can we please bury this > "fibril" name. The constructs Zach is using appear to be identical to > co-routines, and they've been called that i

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Alan
This one got shelved while I sorted other things out as it warranted a longer look. Some comments follow, but firstly can we please bury this "fibril" name. The constructs Zach is using appear to be identical to co-routines, and they've been called that in computer science literature for fifty year

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Linus Torvalds
On Fri, 2 Feb 2007, Ingo Molnar wrote: > > My only suggestion was to have a couple of transparent kernel threads > (not fibrils) attached to a user context that does asynchronous > syscalls! Those kernel threads would be 'switched in' if the current > user-space thread blocks - so instead of

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Andi Kleen
Christoph Hellwig <[EMAIL PROTECTED]> writes: > > I tend to agree. Note that there is one thing we should be doing one > one day (not only if we want to use it for aio) is to make kernel threads > more lightweight. Thereéis a lot of baggae we keep around in task_struct > and co that only makes s

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Andi Kleen
Ingo Molnar <[EMAIL PROTECTED]> writes: > and for one of the most important IO > disciplines, networking, that is reality already. Not 100% -- a few things in TCP/IP at least are blocking still. Mostly relatively obscure things though. Also the sockets model is currently incompatible with direc

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-02 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > So stop blathering about scheduling costs, RT kernels and interrupts. > Interrupts generally happen a few thousand times a second. This is > soemthing you want to do a *million* times a second, without any IO > happening at all except for when it h

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Zach Brown
Priorities cannot be shared, as they have to adapt to the per-request priority when we get down to the nitty gitty of POSIX AIO, as otherwise realtime issues like keepalive transmits will be handled incorrectly. Well, maybe not *blind* sharing. But something more than the disconnect thread

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Benjamin LaHaise
On Thu, Feb 01, 2007 at 01:52:13PM -0800, Zach Brown wrote: > >let me clarify this: i very much like your AIO patchset in general, in > >the sense that it 'completes' the AIO implementation: finally > >everything > >can be done via it, greatly increasing its utility and hopefully its > >penetrati

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Zach Brown
let me clarify this: i very much like your AIO patchset in general, in the sense that it 'completes' the AIO implementation: finally everything can be done via it, greatly increasing its utility and hopefully its penetration. This is the most important step, by far. We violently agree on this

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Linus Torvalds
On Thu, 1 Feb 2007, Ingo Molnar wrote: > > there's almost no scheduling cost from being able to arbitrarily > schedule a kernel thread - but there are /huge/ benefits in it. That's a singularly *stupid* argument. Of course scheduling is fast. That's the whole *point* of fibrils. They still s

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Ingo Molnar
* Mark Lord <[EMAIL PROTECTED]> wrote: > >also, we context-switch kernel threads in 350 nsecs on current > >hardware and the -rt kernel is certainly happy with that and runs all > >hardirqs > > Ingo, how relevant is that "350 nsecs on current hardware" claim? > > I don't mean that in a bad wa

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Mark Lord
Ingo Molnar wrote: also, we context-switch kernel threads in 350 nsecs on current hardware and the -rt kernel is certainly happy with that and runs all hardirqs Ingo, how relevant is that "350 nsecs on current hardware" claim? I don't mean that in a bad way, but my own experience suggests t

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Ingo Molnar
* Christoph Hellwig <[EMAIL PROTECTED]> wrote: > I tend to agree. Note that there is one thing we should be doing one > one day (not only if we want to use it for aio) is to make kernel > threads more lightweight. There a lot of baggae we keep around in > task_struct and co that only makes s

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Christoph Hellwig
On Thu, Feb 01, 2007 at 02:02:34PM +0100, Ingo Molnar wrote: > what i dont really like /the particular/ concept above - the > introduction of 'fibrils' as a hard distinction of kernel threads. They > are /almost/ kernel threads, but still by being different they create > alot of duplication and

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > * Zach Brown <[EMAIL PROTECTED]> wrote: > > > This patch introduces the notion of a 'fibril'. It's meant to be a > > lighter kernel thread. [...] > > as per my other email, i dont really like this concept. This is the > killer: let me clarify this:

Re: [PATCH 2 of 4] Introduce i386 fibril scheduling

2007-02-01 Thread Ingo Molnar
* Zach Brown <[EMAIL PROTECTED]> wrote: > This patch introduces the notion of a 'fibril'. It's meant to be a > lighter kernel thread. [...] as per my other email, i dont really like this concept. This is the killer: > [...] There can be multiple of them in the process of executing for a >