Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Evgeniy Polyakov
On Wed, Mar 07, 2007 at 03:21:19PM -0300, Kirk Kuchov ([EMAIL PROTECTED]) wrote: > On 3/7/07, Ingo Molnar <[EMAIL PROTECTED]> wrote: > > > >* Kirk Kuchov <[EMAIL PROTECTED]> wrote: > > > >> I don't believe I'm wasting my time explaining this. They don't exist > >> as /dev/null, they are just fuckin

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Kirk Kuchov
On 3/7/07, Ingo Molnar <[EMAIL PROTECTED]> wrote: * Kirk Kuchov <[EMAIL PROTECTED]> wrote: > I don't believe I'm wasting my time explaining this. They don't exist > as /dev/null, they are just fucking _LINKS_. [...] > > Either stop flaming kernel developers or become one. It is that > > simple.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Jens Axboe
On Wed, Mar 07 2007, Kirk Kuchov wrote: > On 3/7/07, Ingo Molnar <[EMAIL PROTECTED]> wrote: > > > >* Kirk Kuchov <[EMAIL PROTECTED]> wrote: > > > >> I don't believe I'm wasting my time explaining this. They don't exist > >> as /dev/null, they are just fucking _LINKS_. > >[...] > >> > Either stop fl

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Ingo Molnar
* Kirk Kuchov <[EMAIL PROTECTED]> wrote: > I don't believe I'm wasting my time explaining this. They don't exist > as /dev/null, they are just fucking _LINKS_. [...] > > Either stop flaming kernel developers or become one. It is that > > simple. > > If I were to become a kernel developer I wou

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Linus Torvalds
On Wed, 7 Mar 2007, Kirk Kuchov wrote: > > I don't believe I'm wasting my time explaining this. They don't exist > as /dev/null, they are just fucking _LINKS_. I could even "ln -s > /proc/self/fd/0 sucker". A real /dev/stdout can/could even exist, but > that's not the point! Actually, one large

Re: Trading Places (was: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3)

2007-03-07 Thread Kirk Kuchov
On 3/7/07, Al Boldi <[EMAIL PROTECTED]> wrote: Kirk Kuchov wrote: > > Either stop flaming kernel developers or become one. It is that > > simple. > > If I were to become a kernel developer I would stick with FreeBSD. At > least they have kqueue for about seven years now. I have been playing wit

Trading Places (was: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3)

2007-03-07 Thread Al Boldi
Kirk Kuchov wrote: > > Either stop flaming kernel developers or become one. It is that > > simple. > > If I were to become a kernel developer I would stick with FreeBSD. At > least they have kqueue for about seven years now. I have been playing with this thought for quite some time. The question

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-07 Thread Kirk Kuchov
On 3/6/07, Pavel Machek <[EMAIL PROTECTED]> wrote: > >As for why common abstractions like file are a good thing, think about why > >having "/dev/null" is cleaner that having a special plug DEVNULL_FD fd > >value to be plugged everywhere, > > This is a stupid comparaison. By your logic we should a

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-06 Thread Pavel Machek
> >As for why common abstractions like file are a good thing, think about why > >having "/dev/null" is cleaner that having a special plug DEVNULL_FD fd > >value to be plugged everywhere, > > This is a stupid comparaison. By your logic we should also have /dev/stdin, > /dev/stdout and /dev/stderr.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Michael K. Edwards
On 3/4/07, Kyle Moffett <[EMAIL PROTECTED]> wrote: Well, even this far into 2.6, Linus' patch from 2003 still (mostly) applies; the maintenance cost for this kind of code is virtually zilch. If it matters that much to you clean it up and make it apply; add an alarmfd() syscall (another 100 lines

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Magnus Naeslund(k)
Kirk Kuchov wrote: [snip] This is a stupid comparaison. By your logic we should also have /dev/stdin, /dev/stdout and /dev/stderr. Well, as a matter of fact (on my system): # ls -l /dev/std* lrwxrwxrwx 1 root root 4 Feb 1 2006 /dev/stderr -> fd/2 lrwxrwxrwx 1 root root 4 Feb 1 2006 /de

Discussing LKML community [OT from the Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3]

2007-03-04 Thread Oleg Verych
> From: "Michael K. Edwards" <[EMAIL PROTECTED]> > Newsgroups: gmane.linux.kernel > Subject: Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3 > Date: Wed, 28 Feb 2007 09:01:07 -0800 Michael, [] > In this instance, there didn't seem t

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Davide Libenzi
On Sun, 4 Mar 2007, Kirk Kuchov wrote: > I don't give a shit. Here's another good use of /dev/null: *PLONK* - Davide - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordo

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Kirk Kuchov
On 3/4/07, Davide Libenzi wrote: On Sun, 4 Mar 2007, Kirk Kuchov wrote: > On 3/3/07, Davide Libenzi wrote: > > > > > > Those *other* (tons?!?) interfaces can be created *when* the need comes > > (see Linus signalfd [1] example to show how urgent that was). *When* > > the need comes, they will

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Davide Libenzi
On Sun, 4 Mar 2007, Kirk Kuchov wrote: > On 3/3/07, Davide Libenzi wrote: > > > > > > Those *other* (tons?!?) interfaces can be created *when* the need comes > > (see Linus signalfd [1] example to show how urgent that was). *When* > > the need comes, they will work with existing POSIX interface

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Kyle Moffett
On Mar 04, 2007, at 11:23:37, Kirk Kuchov wrote: So here we are, 2007. epoll() works with files, pipes, sockets, inotify and anything pollable (file descriptors) but aio, timers, signals and user-defined event. Can we please get those working with epoll ? Something as simple as: [code snip

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Kirk Kuchov
On 3/3/07, Davide Libenzi wrote: Those *other* (tons?!?) interfaces can be created *when* the need comes (see Linus signalfd [1] example to show how urgent that was). *When* the need comes, they will work with existing POSIX interfaces, without requiring your own just-another event interface.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-04 Thread Michael K. Edwards
Please don't take this the wrong way, Ray, but I don't think _you_ understand the problem space that people are (or should be) trying to address here. Servers want to always, always block. Not on a socket, not on a stat, not on any _one_ thing, but in a condition where the optimum number of conc

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Ray Lee
Ihar `Philips` Filipau wrote: > On 3/3/07, Ray Lee <[EMAIL PROTECTED]> wrote: >> On 3/3/07, Ihar `Philips` Filipau <[EMAIL PROTECTED]> wrote: >> > What I'm trying to get to: keep things simple. The proposed >> > optimization by Ingo does nothing else but allowing AIO to probe file >> > cache - if d

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Ihar `Philips` Filipau
On 3/3/07, Ray Lee <[EMAIL PROTECTED]> wrote: On 3/3/07, Ihar `Philips` Filipau <[EMAIL PROTECTED]> wrote: > What I'm trying to get to: keep things simple. The proposed > optimization by Ingo does nothing else but allowing AIO to probe file > cache - if data there to go with fast path. So why not

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Davide Libenzi
On Sat, 3 Mar 2007, Davide Libenzi wrote: > Those *other* (tons?!?) interfaces can be created *when* the need comes > (see Linus signalfd [1] example to show how urgent that was). *When* > the need comes, they will work with existing POSIX interfaces, without > requiring your own just-another e

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Davide Libenzi
On Sat, 3 Mar 2007, Evgeniy Polyakov wrote: > > I was referring to dropping an event directly to a userspace buffer, from > > the poll callback. If pages are not there, you might sleep, and you can't > > since the wakeup function holds a spinlock on the waitqueue head while > > looping through

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Ray Lee
On 3/3/07, Ihar `Philips` Filipau <[EMAIL PROTECTED]> wrote: What I'm trying to get to: keep things simple. The proposed optimization by Ingo does nothing else but allowing AIO to probe file cache - if data there to go with fast path. So why not to implement what the people want - probing of cach

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Evgeniy Polyakov
On Sat, Mar 03, 2007 at 10:46:59AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Sat, 3 Mar 2007, Evgeniy Polyakov wrote: > > > > You've to excuse me if my memory is bad, but IIRC the whole discussion > > > and loong benchmark feast born with you throwing a benchmark at Ingo > > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Davide Libenzi
On Sat, 3 Mar 2007, Evgeniy Polyakov wrote: > > You've to excuse me if my memory is bad, but IIRC the whole discussion > > and loong benchmark feast born with you throwing a benchmark at Ingo > > (with kevent showing a 1.9x performance boost WRT epoll), not with you > > making any other point.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Ihar `Philips` Filipau
On 3/3/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > >Threadlets can work with any functionas a base - if it would be > >recv-like it will limit possible case for parallel programming, so you > >can code anything in threadlets - it is not only about IO. > > What I'm trying to get to: keep thi

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Evgeniy Polyakov
On Sat, Mar 03, 2007 at 11:58:17AM +0100, Ihar `Philips` Filipau ([EMAIL PROTECTED]) wrote: > On 3/3/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > >On Fri, Mar 02, 2007 at 08:20:26PM +0100, Ihar `Philips` Filipau > >([EMAIL PROTECTED]) wrote: > >> I'm not well versed in modern kernel developm

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Ihar `Philips` Filipau
On 3/3/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: On Fri, Mar 02, 2007 at 08:20:26PM +0100, Ihar `Philips` Filipau ([EMAIL PROTECTED]) wrote: > I'm not well versed in modern kernel development discussions, and > since you have put the thing into the networked context anyway, can > you pleas

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Evgeniy Polyakov
On Fri, Mar 02, 2007 at 09:28:10AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Fri, 2 Mar 2007, Evgeniy Polyakov wrote: > > > do we really want to have per process signalfs, timerfs and so on - each > > simple structure must be bound to a file, which becomes too cost. > > I may

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Evgeniy Polyakov
On Fri, Mar 02, 2007 at 09:13:40AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Fri, 2 Mar 2007, Evgeniy Polyakov wrote: > > > On Thu, Mar 01, 2007 at 11:31:14AM -0800, Davide Libenzi > > (davidel@xmailserver.org) wrote: > > > On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: > > > > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-03 Thread Davide Libenzi
On Sat, 3 Mar 2007, Ingo Molnar wrote: > * Davide Libenzi wrote: > > > [...] Status word and control bits should not be changed from > > underneath userspace AFAIK. [...] > > Note that the control bits do not just magically change during normal > FPU use. It's a bit like sys_setsid()/iopl/etc

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > Note that the control bits do not just magically change during normal > FPU use. It's a bit like sys_setsid()/iopl/etc., it makes little sense > to change those per-thread anyway. This is a non-issue anyway - what is > important is that the big bulk o

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Davide Libenzi wrote: > [...] Status word and control bits should not be changed from > underneath userspace AFAIK. [...] Note that the control bits do not just magically change during normal FPU use. It's a bit like sys_setsid()/iopl/etc., it makes little sense to change those per-thread

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Nicholas Miell wrote: > On Fri, 2007-03-02 at 16:52 -0800, Davide Libenzi wrote: > > On Fri, 2 Mar 2007, Nicholas Miell wrote: > > > > > The point Ingo was making is that the x86 ABI already requires the FPU > > > context to be saved before *all* function calls. > > > > I've

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Benjamin LaHaise
On Fri, Mar 02, 2007 at 05:36:01PM -0800, Nicholas Miell wrote: > > as an asynchronous context helps alot: the classic gcc convention for > > FPU use & function calls should apply: gcc does not call an external > > function with an in-use FPU stack/register, it always neatly unuses it, > > as no

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Nicholas Miell
On Fri, 2007-03-02 at 16:52 -0800, Davide Libenzi wrote: > On Fri, 2 Mar 2007, Nicholas Miell wrote: > > > The point Ingo was making is that the x86 ABI already requires the FPU > > context to be saved before *all* function calls. > > I've not seen that among Ingo's points, but yeah some status i

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Nicholas Miell wrote: > The point Ingo was making is that the x86 ABI already requires the FPU > context to be saved before *all* function calls. I've not seen that among Ingo's points, but yeah some status is caller saved. But, aren't things like status word and control bits

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Nicholas Miell
On Fri, 2007-03-02 at 12:53 -0800, Davide Libenzi wrote: > On Fri, 2 Mar 2007, Ingo Molnar wrote: > > > > > * Davide Libenzi wrote: > > > > > I think that the "dirty" FPU context must, at least, follow the new > > > head. That's what the userspace sees, and you don't want an async_exec > > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Michael K. Edwards
On 3/2/07, Davide Libenzi wrote: For threadlets, it might be. Now think about a task wanting to dispatch N parallel AIO requests as N independent syslets. Think about this task having USEDFPU set, so the FPU context is dirty. When it returns from async_exec, with one of the requests being become

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Ingo Molnar wrote: > > * Davide Libenzi wrote: > > > I think that the "dirty" FPU context must, at least, follow the new > > head. That's what the userspace sees, and you don't want an async_exec > > to re-emerge with a different FPU context. > > well. I think there's som

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Davide Libenzi wrote: > I think that the "dirty" FPU context must, at least, follow the new > head. That's what the userspace sees, and you don't want an async_exec > to re-emerge with a different FPU context. well. I think there's some confusion about terminology, so please let me describ

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Ingo Molnar wrote: > > * Davide Libenzi wrote: > > > [...] We're still missing proper FPU context switch in the > > move_user_context(). [...] > > yeah - i'm starting to be of the opinion that the FPU context should > stay with the threadlet, exclusively. I.e. when callin

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Davide Libenzi wrote: > [...] We're still missing proper FPU context switch in the > move_user_context(). [...] yeah - i'm starting to be of the opinion that the FPU context should stay with the threadlet, exclusively. I.e. when calling a threadlet, the 'outer loop' (the event loop) should

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Davide Libenzi wrote: > And if you really feel raw about the single O(nready) loop that epoll > currently does, a new epoll_wait2 (or whatever) API could be used to > deliver the event directly into a userspace buffer [1], directly from the > poll callback, w/out extra deliv

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Ingo Molnar wrote: > > After your changes epoll increased to 5k. > > Can we please stop this pointless episode of benchmarketing, where every > mail of yours shows different results and you even deny having said > something which you clearly said just a few days ago? At this

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Evgeniy Polyakov wrote: > do we really want to have per process signalfs, timerfs and so on - each > simple structure must be bound to a file, which becomes too cost. I may be old school, but if you ask me, and if you *really* want those events, yes. Reason? Unix's everythin

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Davide Libenzi
On Fri, 2 Mar 2007, Evgeniy Polyakov wrote: > On Thu, Mar 01, 2007 at 11:31:14AM -0800, Davide Libenzi > (davidel@xmailserver.org) wrote: > > On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: > > > > > Ingo, do you really think I will send mails with faked benchmarks? :)) > > > > I don't think he eve

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Evgeniy Polyakov
On Fri, Mar 02, 2007 at 11:57:13AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > > > > [...] The numbers are still highly suspect - and we are already > > > > > down from the prior claim of kevent being almost twice as fast > > > > > to a

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Evgeniy Polyakov
On Fri, Mar 02, 2007 at 11:56:18AM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > Even if kevent has the same speed, it still allows to handle _any_ > > kind of events without any major surgery - a very tiny structure of > > lock and list h

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > > [...] The numbers are still highly suspect - and we are already > > > > down from the prior claim of kevent being almost twice as fast > > > > to a 25% difference. > > > > > > Btw, there were never almost twice perfromance increase - epoll i

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > Even if kevent has the same speed, it still allows to handle _any_ > kind of events without any major surgery - a very tiny structure of > lock and list head and you can process your own kernel event in > userspace with timers, signals, io events

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Evgeniy Polyakov
On Fri, Mar 02, 2007 at 11:27:14AM +0100, Pavel Machek ([EMAIL PROTECTED]) wrote: > Maybe. It is not up to me to decide. But "it is faster" is _not_ the > only merge criterium. Of course not! Even if kevent has the same speed, it still allows to handle _any_ kind of events without any major surge

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Pavel Machek
Hi! > > > > If you can replace them with something simpler, and no worse than 10% > > > > slower in worst case, then go ahead. (We actually tried to do that at > > > > some point, only to realize that efence stresses vm subsystem in very > > > > unexpected/unfriendly way). > > > > > > Agh, only 1

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-02 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 11:31:14AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote: > On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: > > > Ingo, do you really think I will send mails with faked benchmarks? :)) > > I don't think he ever implied that. He was only suggesting that when you > pos

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Linus Torvalds
On Thu, 1 Mar 2007, Ingo Molnar wrote: > > wrt. one-shot syscalls, the user-space stack footprint would still > probably be there, because even async contexts that only do single-shot > processing need to drop out of kernel mode to handle signals. Why? The easiest thing to do with signals is

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Johann Borck
David Lang wrote: On Thu, 1 Mar 2007, Johann Borck wrote: I reported this a while ago and suggested to have the number of pending accepts reported with the event to save that last syscall. I created an ab replacement based on kevent, and at least with my machines, which are comparable to eac

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread David Lang
On Thu, 1 Mar 2007, Johann Borck wrote: I reported this a while ago and suggested to have the number of pending accepts reported with the event to save that last syscall. I created an ab replacement based on kevent, and at least with my machines, which are comparable to each other, the load o

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Davide Libenzi
Oh boy, wasn't this thread supposed to focus the syslets/threadlets ... :) On Thu, 1 Mar 2007, Eric Dumazet wrote: > On Thursday 01 March 2007 16:23, Evgeniy Polyakov wrote: > > They are there, since ab runs only 50k requests. > > If I change it to something noticebly more than 50/80k, ab cra

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Davide Libenzi
On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: > Ingo, do you really think I will send mails with faked benchmarks? :)) I don't think he ever implied that. He was only suggesting that when you post benchmarks, and even more when you make claims based on benchmarks, you need to be extra carefull ab

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Davide Libenzi
On Thu, 1 Mar 2007, Ingo Molnar wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I posted kevent/epoll benchmarks and related design issues too many > > times both with handmade applications (which might be broken as hell) > > and popular open-source servers to repeat them again.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Johann Borck
On Thu, Mar 01, 2007 at 04:41:27PM +0100, Eric Dumazet wrote: I had to loop on accept() : for (i=0; i On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: The same here - I would just enable a debug to find it. I reported this a while ago and suggested to have the number of pending accept

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread David Lang
On Thu, 1 Mar 2007, Evgeniy Polyakov wrote: On Thu, Mar 01, 2007 at 08:56:28AM -0800, David Lang ([EMAIL PROTECTED]) wrote: the ab numbers below do not seem that impressive to me, especially for such stripped down server processes. ... client and server are dual opteron 252 with 8G of ram, ru

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 08:56:28AM -0800, David Lang ([EMAIL PROTECTED]) wrote: > the ab numbers below do not seem that impressive to me, especially for such > stripped down server processes. ... > client and server are dual opteron 252 with 8G of ram, running debian in 64 > bit mode Decrease yo

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread David Lang
the ab numbers below do not seem that impressive to me, especially for such stripped down server processes. here are some numbers from a set of test boxes I've got in my lab. I've been useing them to test firewalls, and I've been getting throughput similar to what is listed below when going th

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 04:41:27PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > On Thursday 01 March 2007 16:32, Eric Dumazet wrote: > > On Thursday 01 March 2007 16:23, Evgeniy Polyakov wrote: > > > They are there, since ab runs only 50k requests. > > > If I change it to something noticebly m

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 04:32:37PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > On Thursday 01 March 2007 16:23, Evgeniy Polyakov wrote: > > They are there, since ab runs only 50k requests. > > If I change it to something noticebly more than 50/80k, ab crashes: > > # ab -c8000 -t 600 -n800

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 16:32, Eric Dumazet wrote: > On Thursday 01 March 2007 16:23, Evgeniy Polyakov wrote: > > They are there, since ab runs only 50k requests. > > If I change it to something noticebly more than 50/80k, ab crashes: > > # ab -c8000 -t 600 -n8 http://192.168.0.48/ > > Th

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 04:09:42PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > > > I can tell you that the problem (at least on my machine) comes from : > > > > > > > > gettimeofday(&tm, NULL); > > > > > > > > in evserver_epoll.c > > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 16:23, Evgeniy Polyakov wrote: > They are there, since ab runs only 50k requests. > If I change it to something noticebly more than 50/80k, ab crashes: > # ab -c8000 -t 600 -n8 http://192.168.0.48/ > This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apac

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 03:47:17PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > CPU: AMD64 processors, speed 2210.08 MHz (estimated) > > Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit > > mask of 0x00 (No unit m

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I can tell you that the problem (at least on my machine) comes from : > > > > > > gettimeofday(&tm, NULL); > > > > > > in evserver_epoll.c > > > > yeah, that's another difference - especially if it's something like > > an Athlon64 and gettim

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 05:43:50PM +0300, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: > On Thu, Mar 01, 2007 at 02:12:50PM +0100, Eric Dumazet ([EMAIL PROTECTED]) > wrote: > > On Thursday 01 March 2007 12:47, Evgeniy Polyakov wrote: > > > > > > Could you provide at least remote way to find it? >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 03:16:37PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Eric Dumazet <[EMAIL PROTECTED]> wrote: > > > I can tell you that the problem (at least on my machine) comes from : > > > > gettimeofday(&tm, NULL); > > > > in evserver_epoll.c > > yeah, that's another diffe

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > CPU: AMD64 processors, speed 2210.08 MHz (estimated) > Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit > mask of 0x00 (No unit mask) count 10 > samples %symbol name > 195750 67.3097 cpu_idle > 14111 4.

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 02:12:50PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > On Thursday 01 March 2007 12:47, Evgeniy Polyakov wrote: > > > > Could you provide at least remote way to find it? > > > > Sure :) > > > I only found the same problem at > > http://lkml.org/lkml/2006/10/27/3 > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Eric Dumazet <[EMAIL PROTECTED]> wrote: > On my machines (again ...), ab is the slow thing... not the 'server' Evgeniy said that both in the epoll and the kevent case the server side CPU was 98%-100% busy - so inefficiencies on the client side do not matter that much - the server is saturate

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 15:16, Ingo Molnar wrote: > * Eric Dumazet <[EMAIL PROTECTED]> wrote: > > I can tell you that the problem (at least on my machine) comes from : > > > > gettimeofday(&tm, NULL); > > > > in evserver_epoll.c > > yeah, that's another difference - especially if it's something l

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 02:32:42PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > [...] that is why number for kevent is higher - it uses edge-triggered > > handler (which you asked to remove from epoll), [...] > > no - i did not 'ask' it t

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Eric Dumazet <[EMAIL PROTECTED]> wrote: > I can tell you that the problem (at least on my machine) comes from : > > gettimeofday(&tm, NULL); > > in evserver_epoll.c yeah, that's another difference - especially if it's something like an Athlon64 and gettimeofday falls back to pm-timer, that

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 14:30, Evgeniy Polyakov wrote: > On Thu, Mar 01, 2007 at 02:11:18PM +0100, Ingo Molnar ([EMAIL PROTECTED]) > wrote: > > ok? > > I undesrtood you couple of mails ago. > No problem, I can put processing into the same function called from > different servers :) > > > Btw., a

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > [...] that is why number for kevent is higher - it uses edge-triggered > handler (which you asked to remove from epoll), [...] no - i did not 'ask' it to be removed from epoll, i only pointed out that with edge-triggered the results were highly u

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 02:11:18PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > ok? I undesrtood you couple of mails ago. No problem, I can put processing into the same function called from different servers :) > Btw., am i correct that in this particular 'ab' test, the 'immediately' > flag i

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 01:34:23PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > Document Length:3521 bytes > > > Concurrency Level: 8000 > > Time taken for tests: 16.686737 seconds > > Complete requests: 8 > > Faile

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > i dont care whether they are separate or not - but you have not > > replied to the request that there be a handle_web_request() function > > in /both/ files, which is precisely the same function. I didnt ask > > you to merge the two files - i o

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 12:47, Evgeniy Polyakov wrote: > > Could you provide at least remote way to find it? > Sure :) > I only found the same problem at > http://lkml.org/lkml/2006/10/27/3 > > but without any hits to solve the problem. > > I will try CVS oprofile, if it works I will provide de

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 01:43:36PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I separated epoll and kevent servers, since originally kevent server > > included additional kevent features, but then new ones were added and > > I moved slo

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > I separated epoll and kevent servers, since originally kevent server > included additional kevent features, but then new ones were added and > I moved slowly to the similar to epoll case. i dont care whether they are separate or not - but you hav

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > Document Length:3521 bytes > Concurrency Level: 8000 > Time taken for tests: 16.686737 seconds > Complete requests: 8 > Failed requests:0 > Write errors: 0 > Total transferred: 30976 bytes > HTML t

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:28:00PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > I used the CVS version of oprofile plus a patch you can find in the mailing > list archives. Dont remember exactly, since I hit this some months ago Ugh, I started - but CVS compilation requires about 40mb of add

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:47:35PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Ingo Molnar <[EMAIL PROTECTED]> wrote: > > > > I also changed client socket to nonblocking mode with the same result > > > in epoll server. If you will find it broken, please send me corrected > > > to test t

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:41:37PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I also changed client socket to nonblocking mode with the same result > > in epoll server. If you will find it broken, please send me corrected > > to test to

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Ingo Molnar <[EMAIL PROTECTED]> wrote: > > I also changed client socket to nonblocking mode with the same result > > in epoll server. If you will find it broken, please send me corrected > > to test too. > > this line in evserver_kevent.c looks a bit fishy: this one in evserver_kevent.c loo

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:28:00PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > On Thursday 01 March 2007 12:20, Evgeniy Polyakov wrote: > > On Thu, Mar 01, 2007 at 12:14:44PM +0100, Eric Dumazet ([EMAIL PROTECTED]) > wrote: > > > On Thursday 01 March 2007 11:59, Evgeniy Polyakov wrote: > > >

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > I also changed client socket to nonblocking mode with the same result > in epoll server. If you will find it broken, please send me corrected > to test too. this line in evserver_kevent.c looks a bit fishy: err = recv(s, buf, 100, 0); b

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:27:00PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I've uploaded them to: > > > > http://tservice.net.ru/~s0mbre/archive/kevent/evserver_epoll.c > > http://tservice.net.ru/~s0mbre/archive/kevent/evserver_kevent

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Ingo Molnar
* Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > I've uploaded them to: > > http://tservice.net.ru/~s0mbre/archive/kevent/evserver_epoll.c > http://tservice.net.ru/~s0mbre/archive/kevent/evserver_kevent.c thanks. > I also changed client socket to nonblocking mode with the same result > in epol

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Eric Dumazet
On Thursday 01 March 2007 12:20, Evgeniy Polyakov wrote: > On Thu, Mar 01, 2007 at 12:14:44PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > > On Thursday 01 March 2007 11:59, Evgeniy Polyakov wrote: > > > Yes, it is about 98-100% in both cases. > > > I've just re-run tests on my amd64 test mach

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:14:44PM +0100, Eric Dumazet ([EMAIL PROTECTED]) wrote: > On Thursday 01 March 2007 11:59, Evgeniy Polyakov wrote: > > > Yes, it is about 98-100% in both cases. > > I've just re-run tests on my amd64 test machine without debug options: > > > > epoll 4794.23

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 11:11:02AM +0100, Pavel Machek ([EMAIL PROTECTED]) wrote: > > > > > 10% gain in speed is NOT worth major complexity increase. > > > > > > > > Should I create a patch to remove rb-tree implementation? > > > > > > If you can replace them with something simpler, and no worse

Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

2007-03-01 Thread Evgeniy Polyakov
On Thu, Mar 01, 2007 at 12:00:22PM +0100, Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > I've just re-run tests on my amd64 test machine without debug options: > > > > epoll4794.23 > > kevent 6468.95 > > could you please post the two

  1   2   3   4   >