Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Federico Schwindt
Fix aside should we consider using sigaltstack to also catch these crashes? I imagine something like the attached but allocated on startup and shared among all workers. This might be specific to pcre at the moment but I can see its usefulness in the vmod world. On Tue, Sep 2, 2014 at 9:08 PM,

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Poul-Henning Kamp
In message <5406264f.3090...@schokola.de>, Nils Goroll writes: >> I don't think you're considering all the implications. > >let me pull this straight: No, I did not mean to say we should pull in pcre. Pulling in PCRE to move things onto the heap doesn't solve the problem. In fact it will

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
> I don't think you're considering all the implications. let me pull this straight: No, I did not mean to say we should pull in pcre. I only wanted to start a discussion about what is the best option, I have not thought through all implications. Again, I am not a pcre expert. I did mean to say th

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Federico Schwindt
I don't think you're considering all the implications. Bundling something is no small feat, specially when bugs are found. The libjemalloc problem is a good example. Leaving that aside the NO_RECURSIVE code in pcre is not the default. This code is likely much less exercised than the recursive one

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
On 02/09/14 21:50, Nils Goroll wrote: > On 02/09/14 21:45, Poul-Henning Kamp wrote: >> What we really need is for pcre to tell us an estimate of how much >> stack-space is needed during compilation > > Is this possible? My understanding is that the recursion depth depends on the > data and thus

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
> You tell me, I'm no PCRE expert... Neither am I :( ___ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Poul-Henning Kamp
In message <54061f88.2080...@schokola.de>, Nils Goroll writes: >On 02/09/14 21:45, Poul-Henning Kamp wrote: >> What we really need is for pcre to tell us an estimate of how much >> stack-space is needed during compilation > >Is this possible? My understanding is that the recursion depth de

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
On 02/09/14 21:45, Poul-Henning Kamp wrote: > What we really need is for pcre to tell us an estimate of how much > stack-space is needed during compilation Is this possible? My understanding is that the recursion depth depends on the data and thus all we can do is set a recursion limit for the sta

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Poul-Henning Kamp
I think moving pcre from stack to heap would hurt our performance more than increasing the thread stack size. What we really need is for pcre to tell us an estimate of how much stack-space is needed during compilation of the REs, so that we can refuse vcl.use if there isn't enough thread

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
On 02/09/14 21:24, Nils Goroll wrote: > Or, alternatively, realize that we need a larg-ish stack anyway For all those who have not yet read pcrestack(3): As a very rough rule of thumb, you should reckon on about 500 bytes per recursion. Thus, if you want to limit your stack usage

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
On 02/09/14 20:06, Federico Schwindt wrote: > What you mean not a disadvantage? Are you by any means suggesting to bundle > pcre > with Varnish? If it saved considerable amounts of per-thread memory then I'd consider the option. Or, alternatively, realize that we need a larg-ish stack anyway an

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Federico Schwindt
What you mean not a disadvantage? Are you by any means suggesting to bundle pcre with Varnish? There is also this somewhat worrisome comment in the docs: PCRE runs noticeably more slowly when built in this way. I personally don't consider this an option at all. On Tue, Sep 2, 2014 at 7:02 PM, N

Re: stack vs workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
On 02/09/14 19:56, Federico Schwindt wrote: > Only problem is that it's a build option. Yes. But I don't see this as a disadvantage. I think we need to make a decision. ___ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache

Re: stack ws workspace for pcre and others #1576

2014-09-02 Thread Federico Schwindt
Only problem is that it's a build option. Increasing the stack space will only work as long as the regex doesn't recurse more than the stack space allows. Not sure what's the best solution here but we should avoid crashing. On Tue, Sep 2, 2014 at 6:12 PM, Nils Goroll wrote: > Hi, > > on the

stack ws workspace for pcre and others #1576

2014-09-02 Thread Nils Goroll
Hi, on the weekend I have actually stumbled over the same cause that bug #1576 has (but unfortunately have not immediately realized the connection) and have changed our dcs_classifier vmod (which needs some reasonably large scratch memory in the order of ~64k) to use the client workspace rather th

Re: proposal part 1 - VCL: edit header tokens/entities

2014-09-02 Thread Leif Pedersen
On Tue, Sep 2, 2014 at 1:33 AM, Geoff Simmons wrote: > > I'm not a big fan of: > > > Accept: text/plain; q=0.5, text/x-dvi, */* > > > > set req.http.Accept."text/plain"."q" = 3.14; > > > > -> Accept: text/plain; q=3.14, text/x-dvi, */* > > I had to look at that several times to realize what's goi