On Mon, May 26, 2014 at 9:28 AM, SplitIce <mat...@gmail.com> wrote: > Yes, connecting with SOCK_NONBLOCK shouldnt block. I don't believe this is > mentioned previously. If your code blocks (e.g blocking connect or blocking > send) then it would reduce nginx throughput substantially. Thats the point I > was trying to make. >
Yap, I didn't mention SOCK_NONBLOCK. Sorry about that. Although I'm working on a prototype to benchmark, my concern is about socket() and connect() calls on every nginx filter execution. How would you manage this? > Have you investigated srcache > (https://github.com/openresty/srcache-nginx-module/)? srcache_store could be > used without srcache_fetch to store only. Your store handler could be a > proxy_pass call to pass a restful server or whatever you use for storing the > HTML responses. > I didn't know about this module. Thanks, I will have a deep look on it. Do you think that I can pick the proxy_pass response and store it on memcache easily? > Perhaps a better solution would be to use the nginx lua extension instead > (less of a hack). > Do you think that lua extension has advantages regarding an nginx filter module. Sorry about so many questions but at the beginning, nginx is a all mystery, and you'll worried to do not break its performance. Thanks a bunch, > Regards, > Mathew > > > On Mon, May 26, 2014 at 6:18 PM, Paulo Silva <pauloasi...@gmail.com> wrote: >> >> On Mon, May 26, 2014 at 9:09 AM, SplitIce <mat...@gmail.com> wrote: >> > As in blocking send and connect? I don't know the specifics of Unix >> > Sockets, >> > but don't they block when the buffer fills (I know FIFO queues do)? >> > >> >> Sorry, I don't fully understand your question. >> I was expecting that with the SOCK_NONBLOCK it would not block. >> >> What would be your approach? >> Do you know about any nginx internal mechanism to accomplish this goal >> (get the upstream response body out of nginx)? >> >> > >> > On Mon, May 26, 2014 at 9:22 AM, Paulo Silva <pauloasi...@gmail.com> >> > wrote: >> >> >> >> Hi, >> >> I'm not sure whether I will face problems with other filters modifying >> >> the response body after mine, but for know I'm comfortable as I can >> >> rebuild the full response body just iterating buffers chains. >> >> >> >> As I said before I'm using nginx as reverse proxy and my main goal is >> >> to pass the upstream (proxy_pass) response to another local process >> >> (relative to nginx). >> >> >> >> I am benchmarking Unix sockets and Shared memory as IPC. >> >> I did it already for Unix Sockets and with my prototype the nginx >> >> "performance" dropped for half the number of requests per second. Of >> >> course I'm doing something really bad. >> >> >> >> Is it OK to use socket/connect/send from inside an nginx module? >> >> >> >> I would be glad to hear from you. >> >> Thanks, >> >> >> >> On Fri, May 23, 2014 at 2:50 PM, Paulo Silva <pauloasi...@gmail.com> >> >> wrote: >> >> > Because I don't have deep knowledge of nginx internal and I can not >> >> > find a proper resource about it, the best I can do and with what I am >> >> > comfortable is with body_filter. >> >> > >> >> > Do you think I can notice whether all other 3rd party module filters >> >> > finish modifying the ngx_chain_t *in ? >> >> > >> >> > >> >> > >> >> > On Fri, May 23, 2014 at 2:41 PM, Maxim Dounin <mdou...@mdounin.ru> >> >> > wrote: >> >> >> Hello! >> >> >> >> >> >> On Fri, May 23, 2014 at 02:17:27PM +0100, Paulo Silva wrote: >> >> >> >> >> >>> Hi, >> >> >>> >> >> >>> On Fri, May 23, 2014 at 12:58 PM, Maxim Dounin <mdou...@mdounin.ru> >> >> >>> wrote: >> >> >>> > Hello! >> >> >>> > >> >> >>> > On Fri, May 23, 2014 at 11:57:20AM +0100, Paulo Silva wrote: >> >> >>> > >> >> >>> >> there is other option than modify the auto/modules file? >> >> >>> >> >> >> >>> >> According to my goal (capture the full request response body) I >> >> >>> >> would >> >> >>> >> say that my module must run right before the postpone. >> >> >>> > >> >> >>> > Before the postpone filter you'll get subrequest bodies in your >> >> >>> > filter, which is probably not what you want (the postpone filter >> >> >>> > is to glue subrequest together, correctly ordered). >> >> >>> > >> >> >>> >> Am I supposed to modify the auto/modules like follows? >> >> >>> >> >> >> >>> >> if [ $HTTP_POSTPONE = YES ]; then >> >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES >> >> >>> >> $HTTP_POSTPONE_FILTER_MODULE" >> >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_POSTPONE_FILTER_SRCS" >> >> >>> >> fi >> >> >>> >> >> >> >>> >> # insert my module here! >> >> >>> >> >> >> >>> >> if [ $HTTP_SSI = YES ]; then >> >> >>> >> have=NGX_HTTP_SSI . auto/have >> >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES >> >> >>> >> $HTTP_SSI_FILTER_MODULE" >> >> >>> >> HTTP_DEPS="$HTTP_DEPS $HTTP_SSI_DEPS" >> >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_SSI_SRCS" >> >> >>> >> fi >> >> >>> >> >> >> >>> >> >> >> >>> >> I did check my modules config file and I did realize that it is >> >> >>> >> "queued" as HTTP_AUX_FILTER_MODULES. There are different queues >> >> >>> >> for >> >> >>> >> core modules and addons? >> >> >>> > >> >> >>> > The HTTP_AUX_FILTER_MODULES is a generic queue, and it's the >> >> >>> > only one currently officially supported for 3rd party modules. >> >> >>> > >> >> >>> > If you want your filter to be called right before/after postpone >> >> >>> > filter, it should be relatively safe to put it into the >> >> >>> > HTTP_POSTPONE_FILTER_MODULE variable though (and may be with some >> >> >>> > additional checks to make sure the postpone filter is enabled, or >> >> >>> > just a code to enable it unconditionally). >> >> >>> > >> >> >>> >> >> >>> And this is also valid when compiling nginx with the --add-module >> >> >>> flag? >> >> >>> How does config file look like? >> >> >>> >> >> >>> My knowledge is restricted to Emiller's Guide To Nginx Module >> >> >>> Development (http://www.evanmiller.org/nginx-modules-guide.html) >> >> >>> and a >> >> >>> few debugging hours. >> >> >> >> >> >> Uhm, looking again into auto/modules I think I was wrong, and >> >> >> modifying the HTTP_POSTPONE_FILTER_MODULE variable won't work >> >> >> (added module config scripts are executed later on), you should >> >> >> modify HTTP_FILTER_MODULES variable instead, and put your module >> >> >> into a proper position. >> >> >> >> >> >> Note that the "config" file of a module is just a shell script, >> >> >> and you are free to do more or less anything there. >> >> >> >> >> >> -- >> >> >> Maxim Dounin >> >> >> http://nginx.org/ >> >> >> >> >> >> _______________________________________________ >> >> >> nginx-devel mailing list >> >> >> nginx-devel@nginx.org >> >> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> > >> >> > >> >> > >> >> > -- >> >> > Paulo A. Silva >> >> > http://tech.pauloasilva.com >> >> > http://linkedin.com/in/devpauloasilva/ >> >> >> >> >> >> >> >> -- >> >> Paulo A. Silva >> >> http://tech.pauloasilva.com >> >> http://linkedin.com/in/devpauloasilva/ >> >> >> >> _______________________________________________ >> >> nginx-devel mailing list >> >> nginx-devel@nginx.org >> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > >> > >> > >> > _______________________________________________ >> > nginx-devel mailing list >> > nginx-devel@nginx.org >> > http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> >> >> -- >> Paulo A. Silva >> http://tech.pauloasilva.com >> http://linkedin.com/in/devpauloasilva/ >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel@nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel@nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Paulo A. Silva http://tech.pauloasilva.com http://linkedin.com/in/devpauloasilva/ _______________________________________________ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel