Hi,
Thanks everyone for your inputs. It was very useful. Have been busy the
past couple of weeks (even on weekends....it sucks) and only got to
replying this week. My solution to this problem was to simply make a curl
request from my backend to the other application.
Thanks Andi for your explanation. Yes, I do intend to use RabbitMQ
eventually once the load starts to increase, but for now, a curl is all I
need.
I was just over-complicating things.
Regards,
Mark Huang
On Sunday, 17 June 2012 17:36:57 UTC-5, Andi Balke wrote:
>
> but thinking again about that: why don't you just put some information to
> a queue, e.g. rabbitmq, from app A and app B consumes this? its much easier
> :)
>
> On 18.06.2012, at 00:23, Andi Balke wrote:
>
> hi mark,
>
> we used this setup in an authentication proxy, nginx + tornado as
> upstream. all non-public requests went to tornado first, this one did an
> ``x-accel-redirect`` response with information put in the responseheaders.
> nginx took the responsecode to distiguish between authorized and
> unauthorized. additional headers where used to apply information about a
> user to the real backend service.
>
> two things to note:
>
> - in our case this tornado was a simple http upstream pool, no uwsgi as in
> your case. i think that should be managable with a uwsgi location somehow
> too.
> - we had no need to pass any responsebody to the second request. and the
> more i think about it, it makes no sense to have a body passed on a
> redirect in general. so sorry for the confusion, if this hint was wrong
> (still i didn't try yet).
>
> you can do something more advanced - not to say crazy ;) - and use the
> nginx lua module. with that i'm sure it works. i built some request
> splitter for a friend which works well. looks somehow like that (you could
> do ``two sequences ``capture`` instead of the ``capture_multi``)::
>
> location /multi {
> default_type 'text/html';
>
> # echo does not work at all if ``content_by_lua`` is used
> # echo "start lua";
> content_by_lua '
> local res_one, res_two = ngx.location.capture_multi {
> { "/one" },
> { "/two" }
> }
>
> if res_one.status == 200
> and res_two.status == 200 then
> -- ``ngx.header`` must be set before print
> ngx.header["X-Status"] = "Lua"
> ngx.print(res_one.body.." ... "..res_two.body)
> -- ngx.exit(ngx.OK)
> -- return ngx.redirect("/cache");
> return
> end
>
> -- this one does not work: ``ngx.exit(ngx.ERROR)`
> -- whereas setting the status does...
> ngx.status = ngx.HTTP_INTERNAL_SERVER_ERROR
> ngx.print("error "..ngx.time())
> ';
> }
>
> location /one {
> default_type 'text/plain';
> echo "one";
> }
>
> location /two {
> default_type 'text/plain';
> echo "two";
> }
>
> according to http://wiki.nginx.org/HttpLuaModule this is still
> non-blocking: "Unlike Apache's
> mod_lua<http://httpd.apache.org/docs/2.3/mod/mod_lua.html>and Lighttpd's
> mod_magnet <http://redmine.lighttpd.net/wiki/1/Docs:ModMagnet>, Lua code
> executed using this module can be *100% non-blocking* on network traffic
> as long as the Nginx API for
> Lua<http://wiki.nginx.org/HttpLuaModule#Nginx_API_for_Lua>provided by this
> module is used to handle requests to upstream services..."
>
> here is also the relevant part from the buildout, i remember this was a
> bit bitchy to get it working. atm under lion it failed to compile, but its
> a start::
>
> [lua]
> recipe = hexagonit.recipe.cmmi
> url = http://www.lua.org/ftp/lua-5.1.4.tar.gz
> ; url = http://www.lua.org/ftp/lua-all.tar.gz
> strip-top-level-dir=true
> configure-command = /usr/bin/true
> make-options = INSTALL_TOP=${buildout:directory}/parts/lua ${os:make_opt}
>
> [lua_jit]
> recipe = hexagonit.recipe.cmmi
> url = http://luajit.org/download/LuaJIT-2.0.0-beta9.tar.gz
> strip-top-level-dir=true
> configure-command = /usr/bin/true
> make-options = PREFIX=${lua:location}
>
> [nginx_lua_module]
> recipe = hexagonit.recipe.download
> # url = https://github.com/chaoslawful/lua-nginx-module/tarball/v0.1.6rc4
> # url =
> https://download.github.com/saga-lua-nginx-module-v0.0.1rc8-0-g0ef14aa.zip
> url =
> https://download.github.com/chaoslawful-lua-nginx-module-v0.1.6rc10-0-g1ab26d0.zip
> strip-top-level-dir=true
>
> [nginx]
> recipe = hexagonit.recipe.cmmi
> url = http://sysoev.ru/nginx/nginx-0.9.4.tar.gz
> strip-top-level-dir=true
> configure-options = --with-debug
> --add-module=${upstream_fair:destination}
> --add-module=${headers_more_module:destination}
> --add-module=${echo_module:destination}
> --add-module=${eval_module:destination}
> --add-module=${nginx_devel_kit:destination}
> --add-module=${nginx_lua_module:destination}
> --with-cc-opt="-D NGX_HAVE_CASELESS_FILESYSTEM=0"
> --with-http_ssl_module
> --with-http_stub_status_module
>
> --http-proxy-temp-path=${buildout:directory}/var/nginx/cache/client_body_temp
> cache_size = 64
> dep-lua_jit = ${lua_jit:location}
> dep-lua-nginx = ${nginx_lua_module:url}
> environment =
> LUA_LIB=${lua:location}/lib
> LUA_INC=${lua:location}/include/luajit-2.0
>
> cheers, andi
>
>
> On 17.06.2012, at 19:49, Mark Huang wrote:
>
> Just wanted to confirm one thing before this is closed. How do I transfer
> the data that I processed via the response?
>
> Is it using the response.body to add the data in there?
>
>
>
> On Thursday, 14 June 2012 14:04:58 UTC-5, Mark Huang wrote:
>>
>> Thanks Michael for your response once again (here in google groups as
>> well as stackoverflow, I really appreciate the help).
>>
>> With your example would I be able to "send" some preprocessed data to the
>> real client? The real client requires some data that is processed on my
>> end to generate the PDF. Is that done using the Response object's body
>> attribute?
>>
>> Example.
>>
>> def my_view(request):
>> response = Response()
>> *response.body = {"title": "Orders", "table": [ ["Red", 1, 2, 1, 1,
>> 2], ['Blue', 3, 4, 3,1] ] }*
>> response.content_type = 'some-pdf-type'
>> response.headers['X-Accel'] = '/path/to/file'
>> return response
>>
>> Then over in the real client, I can read the data using the Request
>> object's body attribute like req.body?
>>
>> One last thing to clarify reagarding the path to a file that I set for
>> response.headers['X-Accel']:
>>
>> What if the path is not pointing to a file? Because I'm just passing
>> some data over to the real client to process and return me a PDF file. So
>> taking the example
>> here<http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-remote/>as
>> inspiration:
>>
>> response.headers['X-Accel-Redirect'] = '/api/pdf/generate_doc'
>>
>> And then in my nginx configuration, I add a location block like so:
>>
>> location ~* /api/(.*) {
>> set $download_uri $1;
>> set $download_url http://<the_real_client>.com/$download_uri;
>> proxy_set_header Host <the_real_client>.com
>> proxy_pass $download_url
>> }
>>
>> Then over in the real client I will receive the request and be able to
>> read the contents of the Response from A as req.body.
>>
>> Did I get this part right?
>>
>> I'm sorry for being such a noob.
>>
>>
>> On Friday, 15 June 2012 00:51:41 UTC+8, Michael Merickel wrote:
>>>
>>> On Thu, Jun 14, 2012 at 11:36 AM, Mark Huang <zhengha...@gmail.com>
>>> wrote:
>>> > Andi, could you illustrate what your setup was on Pyramid to do this
>>> X-Accel
>>> > thingy?
>>>
>>> The idea is to send a response that has the X-Accel header.. nginx
>>> sees that header in the response and sends to the real client a
>>> response without that header but containing the file content in the
>>> body.
>>>
>>> def my_view(request):
>>> response = Response()
>>> response.content_type = 'some-pdf-type'
>>> response.headers['X-Accel'] = '/path/to/file'
>>> return response
>>>
>>> You then need to setup nginx as per its docs to support watching for
>>> that header.
>>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "pylons-discuss" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/pylons-discuss/-/JziBchThlfoJ.
> To post to this group, send email to pylons-discuss@googlegroups.com.
> To unsubscribe from this group, send email to
> pylons-discuss+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/pylons-discuss?hl=en.
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/pylons-discuss/-/632q4CfB-kUJ.
To post to this group, send email to pylons-discuss@googlegroups.com.
To unsubscribe from this group, send email to
pylons-discuss+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en.
