Yes, connecting with SOCK_NONBLOCK shouldnt block. I don't believe this is mentioned previously. If your code blocks (e.g blocking connect or blocking send) then it would reduce nginx throughput substantially. Thats the point I was trying to make.
Have you investigated srcache ( https://github.com/openresty/srcache-nginx-module/)? srcache_store could be used without srcache_fetch to store only. Your store handler could be a proxy_pass call to pass a restful server or whatever you use for storing the HTML responses. Perhaps a better solution would be to use the nginx lua extension instead (less of a hack). Regards, Mathew On Mon, May 26, 2014 at 6:18 PM, Paulo Silva <pauloasi...@gmail.com> wrote: > On Mon, May 26, 2014 at 9:09 AM, SplitIce <mat...@gmail.com> wrote: > > As in blocking send and connect? I don't know the specifics of Unix > Sockets, > > but don't they block when the buffer fills (I know FIFO queues do)? > > > > Sorry, I don't fully understand your question. > I was expecting that with the SOCK_NONBLOCK it would not block. > > What would be your approach? > Do you know about any nginx internal mechanism to accomplish this goal > (get the upstream response body out of nginx)? > > > > > On Mon, May 26, 2014 at 9:22 AM, Paulo Silva <pauloasi...@gmail.com> > wrote: > >> > >> Hi, > >> I'm not sure whether I will face problems with other filters modifying > >> the response body after mine, but for know I'm comfortable as I can > >> rebuild the full response body just iterating buffers chains. > >> > >> As I said before I'm using nginx as reverse proxy and my main goal is > >> to pass the upstream (proxy_pass) response to another local process > >> (relative to nginx). > >> > >> I am benchmarking Unix sockets and Shared memory as IPC. > >> I did it already for Unix Sockets and with my prototype the nginx > >> "performance" dropped for half the number of requests per second. Of > >> course I'm doing something really bad. > >> > >> Is it OK to use socket/connect/send from inside an nginx module? > >> > >> I would be glad to hear from you. > >> Thanks, > >> > >> On Fri, May 23, 2014 at 2:50 PM, Paulo Silva <pauloasi...@gmail.com> > >> wrote: > >> > Because I don't have deep knowledge of nginx internal and I can not > >> > find a proper resource about it, the best I can do and with what I am > >> > comfortable is with body_filter. > >> > > >> > Do you think I can notice whether all other 3rd party module filters > >> > finish modifying the ngx_chain_t *in ? > >> > > >> > > >> > > >> > On Fri, May 23, 2014 at 2:41 PM, Maxim Dounin <mdou...@mdounin.ru> > >> > wrote: > >> >> Hello! > >> >> > >> >> On Fri, May 23, 2014 at 02:17:27PM +0100, Paulo Silva wrote: > >> >> > >> >>> Hi, > >> >>> > >> >>> On Fri, May 23, 2014 at 12:58 PM, Maxim Dounin <mdou...@mdounin.ru> > >> >>> wrote: > >> >>> > Hello! > >> >>> > > >> >>> > On Fri, May 23, 2014 at 11:57:20AM +0100, Paulo Silva wrote: > >> >>> > > >> >>> >> there is other option than modify the auto/modules file? > >> >>> >> > >> >>> >> According to my goal (capture the full request response body) I > >> >>> >> would > >> >>> >> say that my module must run right before the postpone. > >> >>> > > >> >>> > Before the postpone filter you'll get subrequest bodies in your > >> >>> > filter, which is probably not what you want (the postpone filter > >> >>> > is to glue subrequest together, correctly ordered). > >> >>> > > >> >>> >> Am I supposed to modify the auto/modules like follows? > >> >>> >> > >> >>> >> if [ $HTTP_POSTPONE = YES ]; then > >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES > >> >>> >> $HTTP_POSTPONE_FILTER_MODULE" > >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_POSTPONE_FILTER_SRCS" > >> >>> >> fi > >> >>> >> > >> >>> >> # insert my module here! > >> >>> >> > >> >>> >> if [ $HTTP_SSI = YES ]; then > >> >>> >> have=NGX_HTTP_SSI . auto/have > >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES > >> >>> >> $HTTP_SSI_FILTER_MODULE" > >> >>> >> HTTP_DEPS="$HTTP_DEPS $HTTP_SSI_DEPS" > >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_SSI_SRCS" > >> >>> >> fi > >> >>> >> > >> >>> >> > >> >>> >> I did check my modules config file and I did realize that it is > >> >>> >> "queued" as HTTP_AUX_FILTER_MODULES. There are different queues > for > >> >>> >> core modules and addons? > >> >>> > > >> >>> > The HTTP_AUX_FILTER_MODULES is a generic queue, and it's the > >> >>> > only one currently officially supported for 3rd party modules. > >> >>> > > >> >>> > If you want your filter to be called right before/after postpone > >> >>> > filter, it should be relatively safe to put it into the > >> >>> > HTTP_POSTPONE_FILTER_MODULE variable though (and may be with some > >> >>> > additional checks to make sure the postpone filter is enabled, or > >> >>> > just a code to enable it unconditionally). > >> >>> > > >> >>> > >> >>> And this is also valid when compiling nginx with the --add-module > >> >>> flag? > >> >>> How does config file look like? > >> >>> > >> >>> My knowledge is restricted to Emiller's Guide To Nginx Module > >> >>> Development (http://www.evanmiller.org/nginx-modules-guide.html) > and a > >> >>> few debugging hours. > >> >> > >> >> Uhm, looking again into auto/modules I think I was wrong, and > >> >> modifying the HTTP_POSTPONE_FILTER_MODULE variable won't work > >> >> (added module config scripts are executed later on), you should > >> >> modify HTTP_FILTER_MODULES variable instead, and put your module > >> >> into a proper position. > >> >> > >> >> Note that the "config" file of a module is just a shell script, > >> >> and you are free to do more or less anything there. > >> >> > >> >> -- > >> >> Maxim Dounin > >> >> http://nginx.org/ > >> >> > >> >> _______________________________________________ > >> >> nginx-devel mailing list > >> >> nginx-devel@nginx.org > >> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > >> > > >> > > >> > > >> > -- > >> > Paulo A. Silva > >> > http://tech.pauloasilva.com > >> > http://linkedin.com/in/devpauloasilva/ > >> > >> > >> > >> -- > >> Paulo A. Silva > >> http://tech.pauloasilva.com > >> http://linkedin.com/in/devpauloasilva/ > >> > >> _______________________________________________ > >> nginx-devel mailing list > >> nginx-devel@nginx.org > >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel@nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > -- > Paulo A. Silva > http://tech.pauloasilva.com > http://linkedin.com/in/devpauloasilva/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel@nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel >
_______________________________________________ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
