Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes:
> Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes:
> > Theo Schlossnagle <[EMAIL PROTECTED]> writes:
> > > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes:
> > > > [...] session timeout is broken (commented out, actually) in
> > > > your patch, so broken backends and / or clients will bog you
> > > > down.
> > > Sure. When I said "works well," I meant "as well as on Linux.
> > Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does
> > *not* work as well on Solaris as on Linux, with or without your
> > patch.
> And Solaris supports portfs which is better than epoll.
This is not a valid parallel: event ports and epoll are two ways of
implementing the same functionality, but your patch completely
disables the idle session timeout instead of replacing it with
something that works in Solaris.
Implementing session timeouts portably (without SO_{RCV,SND}TIMEO)
would not be impossible, but it would certainly be hard. Removing
sendfile support entirely might help, by simplifying the code.
> I'd note that the performance from the umem stevedore implementation
> is pretty nice.
What umem stevedore implementation? I've never seen this mentioned
anywhere on or off the lists.
> And that works on FreeBSD and Linux now that libumem is ported
> there. Obviously, every implementation has its advantages and
> disadvantages, but umem stuff is an excellent alternative for the
> malloc based stevedore under similar usage.
The malloc based stevedore is not inteded to be used at all, except
for debugging - and it hasn't been used for even that in ages, so it's
really pretty much just a proof of concept, or a sample implementation
if you will.
DES
--
Dag-Erling Smørgrav
Senior Software Developer
Linpro AS - www.linpro.no
___
varnish-dev mailing list
varnish-dev@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
On Jan 8, 2008, at 3:45 PM, Dag-Erling Smørgrav wrote:
> Theo Schlossnagle <[EMAIL PROTECTED]> writes:
>> Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes:
>>> Theo Schlossnagle <[EMAIL PROTECTED]> writes:
We've been running this for a while on Solaris. Works really well.
>>> Only because you haven't noticed the bugs yet... for instance,
>>> session timeout is broken (commented out, actually) in your patch,
>>> so
>>> broken backends and / or clients will bog you down.
>> Sure. When I said "works well," I meant "as well as on Linux.
>
> Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does
> *not* work as well on Solaris as on Linux, with or without your patch.
And Solaris supports portfs which is better than epoll. It's not
really a competition. I'm kinda lost as to how this turned into an
argument. I have had a good experience on Solaris so far and I don't
know what is gained by rebutting my comments assuming I don't realize
there might be bugs. The patch is against /trunk/ and as such I would
assume many bugs, half baked features, prototype code, etc. I'd
assume both bugs in my path as well as to the /trunk/ to which it is
applied. Varnish as a whole only seems to work because of the bugs we
haven't noticed, right?
I just look forward to having solid support for the OS features that
are applicable.
I'd note that the performance from the umem stevedore implementation
is pretty nice. And that works on FreeBSD and Linux now that libumem
is ported there. Obviously, every implementation has its advantages
and disadvantages, but umem stuff is an excellent alternative for the
malloc based stevedore under similar usage.
>
sendfile and sendfilev on solaris
>>> Probably not a good idea unless sendfile() semantics are
>>> significantly
>>> better on Solaris than on FreeBSD and Linux.
>> It's sendfile, it has all the advantages of sendfile. To support
>> them, you have to conform to their APIs. I just added support so it
>> could say "oh, look, I know how to use that sendfile..." and then
>> actually use it (just as linux and freebsd now). And I think
>> sendfilev on Solaris is pretty slick.
>
> So you've missed the numerous threads on sendfile() bugs affecting
> Varnish, and the more recent threads on sendfile() in FreeBSD and
> Linux being broken by design so that Varnish cannot reliably use it,
> and Poul-Henning's commit disabling the sendfile() detection in
> configure.ac to stop the whining.
Well, so far so good. There are some bugs in Solaris' sendfile as
well, of course. I haven't been able to tickle them in my testing.
I didn't miss the discussion, but I did miss that commit.
>
using fcntl() when flock() is unavailable
>>> There are issues here as well; the semantics are subtly different
>>> from
>>> OS to OS. For instance, what happens if separate threads in the
>>> same
>>> process try to lock the same file? It's even less fun if you take
>>> into consideration systems that support both.
>> As I see it you only supported flock().
>
> You've got it exactly backwards - Varnish has used fcntl() locks
> exclusively for... what... five months now? ever since I determined
> that in addition to being more portable, fcntl() tends to be the least
> broken on platforms that support both (though not on FreeBSD, where
> flock() is slightly better, but I didn't consider it "better enough"
> to warrant an #ifdef). I even credited you in the commit log.
It looks like when updating to trunk that part was in conflict. I had
removed my code as yours did the trick. So, that patch was reverted
in my set a while ago and wasn't in the one I linked to.
--
Theo Schlossnagle
Esoteric Curio -- http://lethargy.org/
OmniTI Computer Consulting, Inc. -- http://omniti.com/
___
varnish-dev mailing list
varnish-dev@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes:
> Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes:
> > Theo Schlossnagle <[EMAIL PROTECTED]> writes:
> > > We've been running this for a while on Solaris. Works really well.
> > Only because you haven't noticed the bugs yet... for instance,
> > session timeout is broken (commented out, actually) in your patch, so
> > broken backends and / or clients will bog you down.
> Sure. When I said "works well," I meant "as well as on Linux.
Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does
*not* work as well on Solaris as on Linux, with or without your patch.
> > > sendfile and sendfilev on solaris
> > Probably not a good idea unless sendfile() semantics are significantly
> > better on Solaris than on FreeBSD and Linux.
> It's sendfile, it has all the advantages of sendfile. To support
> them, you have to conform to their APIs. I just added support so it
> could say "oh, look, I know how to use that sendfile..." and then
> actually use it (just as linux and freebsd now). And I think
> sendfilev on Solaris is pretty slick.
So you've missed the numerous threads on sendfile() bugs affecting
Varnish, and the more recent threads on sendfile() in FreeBSD and
Linux being broken by design so that Varnish cannot reliably use it,
and Poul-Henning's commit disabling the sendfile() detection in
configure.ac to stop the whining.
> > > using fcntl() when flock() is unavailable
> > There are issues here as well; the semantics are subtly different from
> > OS to OS. For instance, what happens if separate threads in the same
> > process try to lock the same file? It's even less fun if you take
> > into consideration systems that support both.
> As I see it you only supported flock().
You've got it exactly backwards - Varnish has used fcntl() locks
exclusively for... what... five months now? ever since I determined
that in addition to being more portable, fcntl() tends to be the least
broken on platforms that support both (though not on FreeBSD, where
flock() is slightly better, but I didn't consider it "better enough"
to warrant an #ifdef). I even credited you in the commit log.
DES
--
Dag-Erling Smørgrav
Senior Software Developer
Linpro AS - www.linpro.no
___
varnish-dev mailing list
varnish-dev@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
On Jan 8, 2008, at 2:18 PM, Dag-Erling Smørgrav wrote: > Theo Schlossnagle <[EMAIL PROTECTED]> writes: >> We've been running this for a while on Solaris. Works really well. > > Only because you haven't noticed the bugs yet... for instance, > session timeout is broken (commented out, actually) in your patch, so > broken backends and / or clients will bog you down. Sure. When I said "works well," I meant "as well as on Linux. > >> What we need is the function in cache_acceptor.c: >> [...] >> I see this as a cleaner mechanism regardless as there is no reason >> for >> the generic cache_acceptor to care about int vca_pipes[2]; -- it's an >> implementation detail. How's that sound? > > That sounds like a good idea. Even better if you can submit an > isolated patch :) Okay, I hadn't tackled that, I'll look at submitting a patch specific to that. > >> sendfile and sendfilev on solaris > > Probably not a good idea unless sendfile() semantics are significantly > better on Solaris than on FreeBSD and Linux. It's sendfile, it has all the advantages of sendfile. To support them, you have to conform to their APIs. I just added support so it could say "oh, look, I know how to use that sendfile..." and then actually use it (just as linux and freebsd now). And I think sendfilev on Solaris is pretty slick. > >> using fcntl() when flock() is unavailable > > There are issues here as well; the semantics are subtly different from > OS to OS. For instance, what happens if separate threads in the same > process try to lock the same file? It's even less fun if you take > into consideration systems that support both. As I see it you only supported flock(). If I don't have it, I use fcntl(). It will certainly not break any system that current works. It doesn't use fcntl() when available.. it uses fcntl() only when flock() isn't available. fcntl() on Linux acts really strange -- so it would be bad -- as you intimate above. Best regards, Theo -- Theo Schlossnagle Principal/CEO OmniTI Computer Consulting, Inc. W: http://omniti.com P: +1.443.325.1357 x201 F: +1.410.872.4911 ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes: > We've been running this for a while on Solaris. Works really well. Only because you haven't noticed the bugs yet... for instance, session timeout is broken (commented out, actually) in your patch, so broken backends and / or clients will bog you down. > What we need is the function in cache_acceptor.c: > [...] > I see this as a cleaner mechanism regardless as there is no reason for > the generic cache_acceptor to care about int vca_pipes[2]; -- it's an > implementation detail. How's that sound? That sounds like a good idea. Even better if you can submit an isolated patch :) >sendfile and sendfilev on solaris Probably not a good idea unless sendfile() semantics are significantly better on Solaris than on FreeBSD and Linux. >using fcntl() when flock() is unavailable There are issues here as well; the semantics are subtly different from OS to OS. For instance, what happens if separate threads in the same process try to lock the same file? It's even less fun if you take into consideration systems that support both. DES -- Dag-Erling Smørgrav Senior Software Developer Linpro AS - www.linpro.no ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Solaris support
Hello all,
We've been running this for a while on Solaris. Works really well.
We have some minor modifications to the source tree to make it work,
but there is one minor design change to make things work well with our
ports cache_acceptor.
What we need is the function in cache_acceptor.c:
void vca_return_session(struct sess *sp);
to be delegated to the acceptor implementations. Solaris allows for
very efficient user initiated notifications across its event system
and that does that by talking directly to the port and not using the
vca_pipes.
I'd like to have a return_session element added to the acceptor
structure that is void (*return_session)(struct sess *);
And the vca_return_session function be:
void
vca_return_session(struct sess *sp)
{
vca->return_session(sp);
}
I see this as a cleaner mechanism regardless as there is no reason for
the generic cache_acceptor to care about int vca_pipes[2]; -- it's an
implementation detail. How's that sound?
I'd like to see this change in trunk before I submit my patch as it
will pretty solidly affect how the return session stuff is handled in
our changeset.
My current patch supports:
sendfile and sendfilev on solaris
using fcntl() when flock() is unavailable
A cache_acceptor_ports.c that embraces Solaris' port eventing system
--
Theo Schlossnagle
Esoteric Curio -- http://lethargy.org/
OmniTI Computer Consulting, Inc. -- http://omniti.com/
___
varnish-dev mailing list
varnish-dev@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-dev
