Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes: > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > > Theo Schlossnagle <[EMAIL PROTECTED]> writes: > > > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > > > > [...] session timeout is broken (commented out, actually) in > > > > your patch, so broken backends and / or clients will bog you > > > > down. > > > Sure. When I said "works well," I meant "as well as on Linux. > > Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does > > *not* work as well on Solaris as on Linux, with or without your > > patch. > And Solaris supports portfs which is better than epoll. This is not a valid parallel: event ports and epoll are two ways of implementing the same functionality, but your patch completely disables the idle session timeout instead of replacing it with something that works in Solaris. Implementing session timeouts portably (without SO_{RCV,SND}TIMEO) would not be impossible, but it would certainly be hard. Removing sendfile support entirely might help, by simplifying the code. > I'd note that the performance from the umem stevedore implementation > is pretty nice. What umem stevedore implementation? I've never seen this mentioned anywhere on or off the lists. > And that works on FreeBSD and Linux now that libumem is ported > there. Obviously, every implementation has its advantages and > disadvantages, but umem stuff is an excellent alternative for the > malloc based stevedore under similar usage. The malloc based stevedore is not inteded to be used at all, except for debugging - and it hasn't been used for even that in ages, so it's really pretty much just a proof of concept, or a sample implementation if you will. DES -- Dag-Erling Smørgrav Senior Software Developer Linpro AS - www.linpro.no ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
On Jan 8, 2008, at 3:45 PM, Dag-Erling Smørgrav wrote: > Theo Schlossnagle <[EMAIL PROTECTED]> writes: >> Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: >>> Theo Schlossnagle <[EMAIL PROTECTED]> writes: We've been running this for a while on Solaris. Works really well. >>> Only because you haven't noticed the bugs yet... for instance, >>> session timeout is broken (commented out, actually) in your patch, >>> so >>> broken backends and / or clients will bog you down. >> Sure. When I said "works well," I meant "as well as on Linux. > > Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does > *not* work as well on Solaris as on Linux, with or without your patch. And Solaris supports portfs which is better than epoll. It's not really a competition. I'm kinda lost as to how this turned into an argument. I have had a good experience on Solaris so far and I don't know what is gained by rebutting my comments assuming I don't realize there might be bugs. The patch is against /trunk/ and as such I would assume many bugs, half baked features, prototype code, etc. I'd assume both bugs in my path as well as to the /trunk/ to which it is applied. Varnish as a whole only seems to work because of the bugs we haven't noticed, right? I just look forward to having solid support for the OS features that are applicable. I'd note that the performance from the umem stevedore implementation is pretty nice. And that works on FreeBSD and Linux now that libumem is ported there. Obviously, every implementation has its advantages and disadvantages, but umem stuff is an excellent alternative for the malloc based stevedore under similar usage. > sendfile and sendfilev on solaris >>> Probably not a good idea unless sendfile() semantics are >>> significantly >>> better on Solaris than on FreeBSD and Linux. >> It's sendfile, it has all the advantages of sendfile. To support >> them, you have to conform to their APIs. I just added support so it >> could say "oh, look, I know how to use that sendfile..." and then >> actually use it (just as linux and freebsd now). And I think >> sendfilev on Solaris is pretty slick. > > So you've missed the numerous threads on sendfile() bugs affecting > Varnish, and the more recent threads on sendfile() in FreeBSD and > Linux being broken by design so that Varnish cannot reliably use it, > and Poul-Henning's commit disabling the sendfile() detection in > configure.ac to stop the whining. Well, so far so good. There are some bugs in Solaris' sendfile as well, of course. I haven't been able to tickle them in my testing. I didn't miss the discussion, but I did miss that commit. > using fcntl() when flock() is unavailable >>> There are issues here as well; the semantics are subtly different >>> from >>> OS to OS. For instance, what happens if separate threads in the >>> same >>> process try to lock the same file? It's even less fun if you take >>> into consideration systems that support both. >> As I see it you only supported flock(). > > You've got it exactly backwards - Varnish has used fcntl() locks > exclusively for... what... five months now? ever since I determined > that in addition to being more portable, fcntl() tends to be the least > broken on platforms that support both (though not on FreeBSD, where > flock() is slightly better, but I didn't consider it "better enough" > to warrant an #ifdef). I even credited you in the commit log. It looks like when updating to trunk that part was in conflict. I had removed my code as yours did the trick. So, that patch was reverted in my set a while ago and wasn't in the one I linked to. -- Theo Schlossnagle Esoteric Curio -- http://lethargy.org/ OmniTI Computer Consulting, Inc. -- http://omniti.com/ ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes: > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > > Theo Schlossnagle <[EMAIL PROTECTED]> writes: > > > We've been running this for a while on Solaris. Works really well. > > Only because you haven't noticed the bugs yet... for instance, > > session timeout is broken (commented out, actually) in your patch, so > > broken backends and / or clients will bog you down. > Sure. When I said "works well," I meant "as well as on Linux. Uh, no, Linux actually supports SO_{RCV,SND}TIMEO, so Varnish does *not* work as well on Solaris as on Linux, with or without your patch. > > > sendfile and sendfilev on solaris > > Probably not a good idea unless sendfile() semantics are significantly > > better on Solaris than on FreeBSD and Linux. > It's sendfile, it has all the advantages of sendfile. To support > them, you have to conform to their APIs. I just added support so it > could say "oh, look, I know how to use that sendfile..." and then > actually use it (just as linux and freebsd now). And I think > sendfilev on Solaris is pretty slick. So you've missed the numerous threads on sendfile() bugs affecting Varnish, and the more recent threads on sendfile() in FreeBSD and Linux being broken by design so that Varnish cannot reliably use it, and Poul-Henning's commit disabling the sendfile() detection in configure.ac to stop the whining. > > > using fcntl() when flock() is unavailable > > There are issues here as well; the semantics are subtly different from > > OS to OS. For instance, what happens if separate threads in the same > > process try to lock the same file? It's even less fun if you take > > into consideration systems that support both. > As I see it you only supported flock(). You've got it exactly backwards - Varnish has used fcntl() locks exclusively for... what... five months now? ever since I determined that in addition to being more portable, fcntl() tends to be the least broken on platforms that support both (though not on FreeBSD, where flock() is slightly better, but I didn't consider it "better enough" to warrant an #ifdef). I even credited you in the commit log. DES -- Dag-Erling Smørgrav Senior Software Developer Linpro AS - www.linpro.no ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
On Jan 8, 2008, at 2:18 PM, Dag-Erling Smørgrav wrote: > Theo Schlossnagle <[EMAIL PROTECTED]> writes: >> We've been running this for a while on Solaris. Works really well. > > Only because you haven't noticed the bugs yet... for instance, > session timeout is broken (commented out, actually) in your patch, so > broken backends and / or clients will bog you down. Sure. When I said "works well," I meant "as well as on Linux. > >> What we need is the function in cache_acceptor.c: >> [...] >> I see this as a cleaner mechanism regardless as there is no reason >> for >> the generic cache_acceptor to care about int vca_pipes[2]; -- it's an >> implementation detail. How's that sound? > > That sounds like a good idea. Even better if you can submit an > isolated patch :) Okay, I hadn't tackled that, I'll look at submitting a patch specific to that. > >> sendfile and sendfilev on solaris > > Probably not a good idea unless sendfile() semantics are significantly > better on Solaris than on FreeBSD and Linux. It's sendfile, it has all the advantages of sendfile. To support them, you have to conform to their APIs. I just added support so it could say "oh, look, I know how to use that sendfile..." and then actually use it (just as linux and freebsd now). And I think sendfilev on Solaris is pretty slick. > >> using fcntl() when flock() is unavailable > > There are issues here as well; the semantics are subtly different from > OS to OS. For instance, what happens if separate threads in the same > process try to lock the same file? It's even less fun if you take > into consideration systems that support both. As I see it you only supported flock(). If I don't have it, I use fcntl(). It will certainly not break any system that current works. It doesn't use fcntl() when available.. it uses fcntl() only when flock() isn't available. fcntl() on Linux acts really strange -- so it would be bad -- as you intimate above. Best regards, Theo -- Theo Schlossnagle Principal/CEO OmniTI Computer Consulting, Inc. W: http://omniti.com P: +1.443.325.1357 x201 F: +1.410.872.4911 ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: Solaris support
Theo Schlossnagle <[EMAIL PROTECTED]> writes: > We've been running this for a while on Solaris. Works really well. Only because you haven't noticed the bugs yet... for instance, session timeout is broken (commented out, actually) in your patch, so broken backends and / or clients will bog you down. > What we need is the function in cache_acceptor.c: > [...] > I see this as a cleaner mechanism regardless as there is no reason for > the generic cache_acceptor to care about int vca_pipes[2]; -- it's an > implementation detail. How's that sound? That sounds like a good idea. Even better if you can submit an isolated patch :) >sendfile and sendfilev on solaris Probably not a good idea unless sendfile() semantics are significantly better on Solaris than on FreeBSD and Linux. >using fcntl() when flock() is unavailable There are issues here as well; the semantics are subtly different from OS to OS. For instance, what happens if separate threads in the same process try to lock the same file? It's even less fun if you take into consideration systems that support both. DES -- Dag-Erling Smørgrav Senior Software Developer Linpro AS - www.linpro.no ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: development efforts on the Solaris side.
On Jan 8, 2008, at 9:32 AM, Poul-Henning Kamp wrote: > In message <[EMAIL PROTECTED]>, Theo > Schlossnagle > writes: >> Hi guys, >> >> I'd really like to be able to contribute some of the improvements >> we've made to varnish back. Is there a way I can get access to >> commit. I'd be happy to stay in my own branch. My current patch set >> is unwieldy and I'm very tempted to just start my own repos... That, >> of course, seems silly. I've fixed up (removed) some of the gccism >> in >> favor or more portability (#include over -include). I've fixed a few >> bugs, made the VCC line a but smarter and more accepting of non gcc >> compiler, I've added a portfs acceptor and built a storage_umem >> allocator facility that rides on Solaris' excellent libumem (highly >> scalable allocator) which we also ported to run on Linux and FreeBSD >> (and Mac OS X): https://labs.omniti.com/trac/portableumem >> >> Next steps? > > Can you mail me a link to the patch ? http://lethargy.org/~jesus/misc/varnish-solaris-trunk-2328.diff Enjoy! -- Theo Schlossnagle Principal/CEO OmniTI Computer Consulting, Inc. W: http://omniti.com P: +1.443.325.1357 x201 F: +1.410.872.4911 ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
Re: development efforts on the Solaris side.
In message <[EMAIL PROTECTED]>, Theo Schlossnagle writes: >Hi guys, > >I'd really like to be able to contribute some of the improvements >we've made to varnish back. Is there a way I can get access to >commit. I'd be happy to stay in my own branch. My current patch set >is unwieldy and I'm very tempted to just start my own repos... That, >of course, seems silly. I've fixed up (removed) some of the gccism in >favor or more portability (#include over -include). I've fixed a few >bugs, made the VCC line a but smarter and more accepting of non gcc >compiler, I've added a portfs acceptor and built a storage_umem >allocator facility that rides on Solaris' excellent libumem (highly >scalable allocator) which we also ported to run on Linux and FreeBSD >(and Mac OS X): https://labs.omniti.com/trac/portableumem > >Next steps? Can you mail me a link to the patch ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev
development efforts on the Solaris side.
Hi guys, I'd really like to be able to contribute some of the improvements we've made to varnish back. Is there a way I can get access to commit. I'd be happy to stay in my own branch. My current patch set is unwieldy and I'm very tempted to just start my own repos... That, of course, seems silly. I've fixed up (removed) some of the gccism in favor or more portability (#include over -include). I've fixed a few bugs, made the VCC line a but smarter and more accepting of non gcc compiler, I've added a portfs acceptor and built a storage_umem allocator facility that rides on Solaris' excellent libumem (highly scalable allocator) which we also ported to run on Linux and FreeBSD (and Mac OS X): https://labs.omniti.com/trac/portableumem Next steps? Best regards, Theo -- Theo Schlossnagle Principal/CTO OmniTI Computer Consulting, Inc. W: http://omniti.com P: +1.443.325.1357 x201 F: +1.410.872.4911 ___ varnish-dev mailing list varnish-dev@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev