On Fri, Nov 13, 2009 at 14:01, Arturo 'Buanzo' Busleiman
wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Matthieu Estrade wrote:
>> What about the non http protocol like ftp, or smtp tested during summer
>> code ? The tentation to have a powerful core that we could adapt to any
>> p
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Matthieu Estrade wrote:
> What about the non http protocol like ftp, or smtp tested during summer
> code ? The tentation to have a powerful core that we could adapt to any
> protocol we want...
And Google just released SPDY ("Speedy"), a non-http pr
Woow =) Very nice and interesting thread =)
It's very hard to think how to design httpd 3.0 before knowing what is
the real aim of this new webserver. Many feedback here are from very
spoted problems.
I've started at the end of 1.3 and the beta release of 2.0, and i must
say that applicative archi
Jean-Marc Desperrier wrote:
> Last time I've heard about a large scale server thinking about switching
> from Apache to lighttpd, the one problem that site wanted to solve was a
> massive number slow clients simultaneously connected to the server, with
> the http server mostly just serving as a pi
Greg Stein wrote:
> we have to take into account that some of those httpd's, like lighttpd, are
> replacing Apache plain and simple. [...]
[...] I'm just trying to say those
aren't necessarily*better* than Apache, but that they are
*better-suited* to their admin's scenarios.[...]
Last time
On Thu, Nov 12, 2009 at 09:59, Arturo 'Buanzo' Busleiman
wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Greg Stein wrote:
>> Apache remains the broad solution, but for narrow requirements, people
>> will select something that is easier to handle for their particular
>> situation.
>
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Greg Stein wrote:
> Apache remains the broad solution, but for narrow requirements, people
> will select something that is easier to handle for their particular
> situation.
>
> I wouldn't say "wrong", but more along the lines of "not as well-suited
On Nov 11, 2009, at 2:14 PM, Akins, Brian wrote:
> On 11/10/09 6:20 PM, "Greg Stein" wrote:
>
>> I'd like to see a few "network" threads multiplexing all the writing
>> to clients.
>
> That's what I meant. I just didn't state it properly.
>
>
>> Then take all of *that*, and spread it across
On Wed, Nov 11, 2009 at 15:00, Arturo 'Buanzo' Busleiman
wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Greg Stein wrote:
>> Right. But they don't have the depth/breadth of modules like we do.
>
> ... yet. Keep going, but if there are great things like lighttpd and nginx
> (and ev
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Greg Stein wrote:
> Right. But they don't have the depth/breadth of modules like we do.
... yet. Keep going, but if there are great things like lighttpd and nginx (and
even more) http
daemons out there, then that means more than one thing is wrong
On Wed, Nov 11, 2009 at 14:14, Akins, Brian wrote:
> On 11/10/09 6:20 PM, "Greg Stein" wrote:
>
>> I'd like to see a few "network" threads multiplexing all the writing
>> to clients.
>
> That's what I meant. I just didn't state it properly.
>
>
>> Then take all of *that*, and spread it across sev
On 11/10/09 6:20 PM, "Greg Stein" wrote:
> I'd like to see a few "network" threads multiplexing all the writing
> to clients.
That's what I meant. I just didn't state it properly.
> Then take all of *that*, and spread it across several processes for
> solid uptime, with a master monitor proc
On Nov 11, 2009, at 6:09 AM, Graham Leggett wrote:
> William A. Rowe Jr. wrote:
>
>>> - Supporting prefork as httpd does now; and
>>
>> I'm very happy to see prefork die it's timely death.
>>
>> Let's go about working out where out-of-process magic happens.
>> Gated, single threaded handlers m
On Wed, 2009-11-11 at 13:09 +0200, Graham Leggett wrote:
> Apache httpd does lots of things right.
>
> We must resist the temptation to throw out what we do right, while we
> try move forward fixing what we do wrong.
And there is also a reason why Google's Chome is essentially (pre)fork.
This mod
William A. Rowe Jr. wrote:
>> - Supporting prefork as httpd does now; and
>
> I'm very happy to see prefork die it's timely death.
>
> Let's go about working out where out-of-process magic happens.
> Gated, single threaded handlers may be sensible in some cases.
> But for the core server it make
On Tue, Nov 10, 2009 at 05:30:34PM -0500, Akins, Brian wrote:
> On 11/10/09 1:56 PM, "Greg Stein" wrote:
>
>
> > But some buckets might be performing gzip or SSL encryption. That
> > consumes CPU within the network thread.
>
> You could just run x times CPU cores number of "network" threads. Y
Greg Stein wrote:
> On Mon, Nov 9, 2009 at 14:21, Paul Querna wrote:
>> ...
>> I agree in general, a serf-based core does give us a good start.
>>
>> But Serf Buckets and the event loop definitely do need some more work
>> -- simple things, like if the backend bucket is a socket, how do you
>> tel
Graham Leggett wrote:
> - Supporting prefork as httpd does now; and
I'm very happy to see prefork die it's timely death.
Let's go about working out where out-of-process magic happens.
Gated, single threaded handlers may be sensible in some cases.
But for the core server it makes async worthless,
On Tue, Nov 10, 2009 at 17:30, Akins, Brian wrote:
> On 11/10/09 1:56 PM, "Greg Stein" wrote:
>
>
>> But some buckets might be performing gzip or SSL encryption. That
>> consumes CPU within the network thread.
>
> You could just run x times CPU cores number of "network" threads. You can't
> use
On Tue, Nov 10, 2009 at 16:33, Lieven Govaerts wrote:
> On Tue, Nov 10, 2009 at 6:10 PM, Greg Stein wrote:
>...
>> You have 10k buckets representing the response for 10k clients. The
>> core loop reads the response from the bucket, and writes that to the
>> network.
>>
>> Now. A client socket wak
On 11/10/09 1:56 PM, "Greg Stein" wrote:
> But some buckets might be performing gzip or SSL encryption. That
> consumes CPU within the network thread.
You could just run x times CPU cores number of "network" threads. You can't
use more than 100% of a CPU anyway.
The model that some of us disc
On Tue, Nov 10, 2009 at 6:10 PM, Greg Stein wrote:
> On Tue, Nov 10, 2009 at 11:14, Akins, Brian wrote:
>> On 11/9/09 3:08 PM, "Greg Stein" wrote:
>>
>>> 2) If you have 10,000 client connections, and some number of sockets
>>> in the system ready for read/write... how do you quickly determine
>>
On Tue, Nov 10, 2009 at 12:54, Graham Leggett wrote:
> Greg Stein wrote:
>
>>> Who is "you"?
>>
>> Anybody who reads from a bucket. In this case, the core network loop
>> when a client connection is ready for writing.
>
> So would it be correct to say that in this theoretical httpd, the httpd
> co
Greg Stein wrote:
>> Who is "you"?
>
> Anybody who reads from a bucket. In this case, the core network loop
> when a client connection is ready for writing.
So would it be correct to say that in this theoretical httpd, the httpd
core, and nobody else, would read from the serf bucket?
>> Up till
On Tue, Nov 10, 2009 at 12:01, Jim Jagielski wrote:
> On Nov 9, 2009, at 2:19 PM, Akins, Brian wrote:
>> On 11/9/09 2:06 PM, "Greg Stein" wrote:
>>
>>> These issues are already solved by moving to a Serf core. It is fully
>>> asynchronous.
>>
>> Okay that's one convert, any others? ;)
Convert? B
On Tue, Nov 10, 2009 at 11:14, Akins, Brian wrote:
> On 11/9/09 3:08 PM, "Greg Stein" wrote:
>
>> 2) If you have 10,000 client connections, and some number of sockets
>> in the system ready for read/write... how do you quickly determine
>> *which* buckets to poll to get those sockets processed? Y
On Mon, Nov 9, 2009 at 18:47, Graham Leggett wrote:
>...
>> When you read from a serf bucket, it will return however much you ask
>> for, or as much as it has without blocking. When it gives you that
>> data, it can say "I have more", "I'm done", or "This is what I had
>> without blocking".
>
> Wh
On Nov 9, 2009, at 2:19 PM, Akins, Brian wrote:
> On 11/9/09 2:06 PM, "Greg Stein" wrote:
>
>> These issues are already solved by moving to a Serf core. It is fully
>> asynchronous.
>
> Okay that's one convert, any others? ;)
>
I said the same thing back on the 4th ;)
> That's what Paul and
Greg Stein wrote:
>> I am also concerned about the patterns of sendfile() in the current
>> serf bucket archittecture, and making a whole pipeline do sendfile
>> correctly seems quite difficult.
>
> Well... it generally *is* quite difficult in the presence of SSL,
> gzip, and chunking. Invariably
Paul Querna wrote:
> But Serf Buckets and the event loop definitely do need some more work
> -- simple things, like if the backend bucket is a socket, how do you
> tell the event loop, that a would block rvalue maps to a file
> descriptor talking to an origin server. You don't want to just keep
On 11/9/09 3:08 PM, "Greg Stein" wrote:
> 2) If you have 10,000 client connections, and some number of sockets
> in the system ready for read/write... how do you quickly determine
> *which* buckets to poll to get those sockets processed? You don't want
> to poll idle connections/buckets if o
On Mon, 9 Nov 2009, Graham Leggett wrote:
Akins, Brian wrote:
FWIW, nginx "buffers" backend stuff to a file, then sendfiles it out - I
think this is what perlbal does as well. Same can be done outside apache
using X-sendfile like methods. Seems like we could move this "inside"
apache fairly
Greg Stein wrote:
>> How is "pull" different from "push"[1]?
>
> The network loop pulls data from the content-generator.
>
> Apache 1.x and 2.x had a handler that pushed data at the network.
> There is no loop, of course, since each worker had direct control of
> the socket to push data into.
A
On Mon, Nov 9, 2009 at 16:19, Graham Leggett wrote:
> Greg Stein wrote:
>> These issues are already solved by moving to a Serf core. It is fully
>> asynchronous.
>>
>> Backend handlers will no longer "push" bits towards the network. The
>> core will "pull" them from a bucket. *Which* bucket is def
Greg Stein wrote:
> These issues are already solved by moving to a Serf core. It is fully
> asynchronous.
>
> Backend handlers will no longer "push" bits towards the network. The
> core will "pull" them from a bucket. *Which* bucket is defined by a
> {URL,Headers}->Bucket mapping system.
How is
Akins, Brian wrote:
What we discussed some on list some at Apachecon, was having a really good
and simple process manager. Mod_fcgid is too much work to configure for
mere mortals. If we just had something like:
AssociateExternal .php /path/to/my/php-cgi
Sounds interesting. Any notes from
On Mon, Nov 9, 2009 at 14:21, Paul Querna wrote:
>...
> I agree in general, a serf-based core does give us a good start.
>
> But Serf Buckets and the event loop definitely do need some more work
> -- simple things, like if the backend bucket is a socket, how do you
> tell the event loop, that a wo
On Mon, Nov 9, 2009 at 11:06 AM, Greg Stein wrote:
> On Mon, Nov 9, 2009 at 13:59, Graham Leggett wrote:
>> Akins, Brian wrote:
>>
> It works really well for proxy.
Aka "static data" :)
>>>
>>> Nah, we proxy to fastcgi php stuff, http java stuff, some horrid HTTP perl
>>> stuff, etc (Ful
On 11/9/09 2:06 PM, "Greg Stein" wrote:
> These issues are already solved by moving to a Serf core. It is fully
> asynchronous.
Okay that's one convert, any others? ;)
That's what Paul and I discussed a lot last week.
My ideal httpd 3.0 is:
Libev + serf + lua
--
Brian Akins
Akins, Brian wrote:
> FWIW, nginx "buffers" backend stuff to a file, then sendfiles it out - I
> think this is what perlbal does as well. Same can be done outside apache
> using X-sendfile like methods. Seems like we could move this "inside"
> apache fairly easy. May can do it with a filter.
On Mon, Nov 9, 2009 at 13:59, Graham Leggett wrote:
> Akins, Brian wrote:
>
It works really well for proxy.
>>> Aka "static data" :)
>>
>> Nah, we proxy to fastcgi php stuff, http java stuff, some horrid HTTP perl
>> stuff, etc (Full disclosure, I wrote the horrid perl stuff.)
>
> Doesn't mat
On 11/9/09 1:59 PM, "Graham Leggett" wrote:
> Doesn't matter, once httpd proxy gets hold of it, it's just shifting
> static bits.
True.
> Something I want to teach httpd to do is buffer up data for output, and
> then forget about the output to focus on releasing the backend resources
> ASAP, r
Akins, Brian wrote:
>>> It works really well for proxy.
>> Aka "static data" :)
>
> Nah, we proxy to fastcgi php stuff, http java stuff, some horrid HTTP perl
> stuff, etc (Full disclosure, I wrote the horrid perl stuff.)
Doesn't matter, once httpd proxy gets hold of it, it's just shifting
stati
On 11/9/09 1:40 PM, "Brian Akins" wrote:
> On 11/9/09 1:36 PM, "Graham Leggett" wrote:
>
>>> It works really well for proxy.
>>
>> Aka "static data" :)
>
> Nah, we proxy to fastcgi php stuff, http java stuff, some horrid HTTP perl
> stuff, etc (Full disclosure, I wrote the horrid perl stuff.)
On 11/9/09 1:36 PM, "Graham Leggett" wrote:
>> It works really well for proxy.
>
> Aka "static data" :)
Nah, we proxy to fastcgi php stuff, http java stuff, some horrid HTTP perl
stuff, etc (Full disclosure, I wrote the horrid perl stuff.)
--
Brian Akins
Akins, Brian wrote:
>> and we know
>> from the same period of experience from others that a pure event driven
>> model is useful for shipping static data and not much further.
>
> It works really well for proxy.
Aka "static data" :)
The key advantage to doing both prefork and event behaviour in
On 11/9/09 1:18 PM, "Graham Leggett" wrote:
> and we know
> from the same period of experience from others that a pure event driven
> model is useful for shipping static data and not much further.
It works really well for proxy.
--
Brian Akins
Akins, Brian wrote:
>> This gives us the option of prefork reliability, and event driven speed,
>> as required by the admin.
>
> I think if we try to do both, we will wind up with the worst of both worlds.
> (Or is it worse??) Blocking/buggy "modules" should be ran out of process
> (FactCGI/HTTP
On 11/9/09 12:52 PM, "Graham Leggett" wrote:
> This gives us the option of prefork reliability, and event driven speed,
> as required by the admin.
I think if we try to do both, we will wind up with the worst of both worlds.
(Or is it worse??) Blocking/buggy "modules" should be ran out of proce
Akins, Brian wrote:
> FWIW, nginx delivers on its performance promises, but is a horrible hairball
> of code (my opinion). We (httpd-dev type folks) could do much better - if
> we just would. (Easy for the guy with no time to say, I know...)
I think it is entirely reasonable for the httpd v3.0 c
On 11/9/09 12:32 AM, "Brian McCallister" wrote:
> A 3.0, a fundamental architectural shift, would be interesting to
> discuss, I am not sure there is a ton of value in it, though, to be
> honest.
So I should continue to investigate nginx? ;)
FWIW, nginx delivers on its performance promises, but
On Wed, Nov 4, 2009 at 10:26 AM, Akins, Brian wrote:
> So, after several conversations at Apachecon and on the list, we still have
> no real "vision" of how we want to move ahead with httpd "3.0." Or, if we
> do, it's not communicated very well.
>
> Some have suggested we just create a new server
On 06/11/09 20:07, Jim Jagielski wrote:
I'd like we remove the entire forwarding proxy stuff
for example.
So we have mod_forward_proxy and mod_reverse_proxy? Interesting
take. Would make some sense to make mod_proxy and top-level "framework"
and forward/reverse as submodules.
I'd like that
On a phone, so pls excuse my brevity...
I think a lot of your discussion can be easily passed off to Apache Thrift.
Let it handle all the message passing to external procceses, and its
provided multi-language support.
On Nov 5, 2009 4:31 PM, "Graham Dumpleton"
wrote:
2009/11/5 Graham Leggett :
On 11/5/09 4:30 PM, "Graham Dumpleton" wrote:
> Thoughts?
Still digesting, but generally +1 to the entire post.
--
Brian Akins
2009/11/5 Graham Leggett :
> Jim Jagielski wrote:
>
>> Let's get 2.4 out. And then let's rip it to shreds and drop
>> buckets/brigades and fold in serf.
>
> I think we should decide on exactly what problem we're trying to solve,
> before we start thinking about how it is to be solved.
>
> I'm keen
On 05/11/09 12:38, Graham Leggett wrote:
Jim Jagielski wrote:
Let's get 2.4 out. And then let's rip it to shreds and drop
buckets/brigades and fold in serf.
I think we should decide on exactly what problem we're trying to solve,
before we start thinking about how it is to be solved.
+1
I'
How about support of openmp?
Regards,
Jie
On Thu, 2009-11-05 at 13:38 +0200, Graham Leggett wrote:
> I'm keen to teach httpd v3.0 to work asynchronously throughout - still
> maintaining the prefork behaviour as a sensible default[1], but being
> asynchronous and non blocking throughout.
>
> [1] The fact that dodgy module code can leak, cr
I'm with Jim,
Head for 2.4 first.
IIRC there was some talk about moving to a 'd' project, since httpd
now does ftp (mod_ftp), echo, pop3,... and some other protocols.
I don't remember much from it though. I did like the idea back then
but thats about the only thing I remember from that.
Maybe we
Let's get 2.4 out. And then let's rip it to shreds and drop
buckets/brigades and fold in serf.
On Nov 4, 2009, at 1:26 PM, Akins, Brian wrote:
So, after several conversations at Apachecon and on the list, we
still have
no real "vision" of how we want to move ahead with httpd "3.0." Or,
if w
So, after several conversations at Apachecon and on the list, we still have
no real "vision" of how we want to move ahead with httpd "3.0." Or, if we
do, it's not communicated very well.
Some have suggested we just create a new server project. Others want to keep
hacking away at the current code
62 matches
Mail list logo
