Thanks Jonathan, your solution is what I was leaning to for proof-of-concept, good to know it's somewhat performant too. I don't know enough about fast response time situations to really know the pros and cons of short-polling, long-polling, and websockets.
Did you also look into erlang at all? I think all our complex domain login will stay on python, with some apps/services done in pyramid and some in Django, but we are looking into adding more messaging to it, and I wonder if for sending stuff back we might want to look at Erlang + Redis or Erlang + RabbitMQ. iain On Tue, Sep 29, 2015 at 2:41 PM, Jonathan Vanasco <jonat...@findmeon.com> wrote: > we have a similar situation. `pyramid` is the main web application. when > a user uploads a file, it is handed off to `celery` via `redis`, which > handles all the image resizing and uploading onto S3. > > the solution we went with: > > • short polling on a configurable delay. We default to 2500ms, and half or > double it (manually) depending on average server load. > > • we use a VERY STRIPPED DOWN pyramid view. It's part of the normal app, > but inherits from a different base class (we use class based views) and has > a very lean integration with pyramid. it basically has 20% the overhead of > our other views. this allows for the short polling to even be 100s of ms > if we want. > > • just to be safe and bust aggressive caching, we send a nonce with each > poll. (originally we put the nonce in the url, but that had a wee bit of > overhead in pyramid) > > Since the "answer" is going to be in redis, you'll have a blazing fast > lookup. > > • a streamlined view/route gets it done rather perfomant. > • if things slow down, you can pull it out and build a dedicated pyramid > service (it won't be serving other routes, so the memory footprint will be > smaller and you can run way more instances) > • if things still don't scale well, then you can think about nodejs. we > have some twisted services and would probably choose nodejs for this task > over twisted -- it's better suited and you'll be abandoning enough of your > codebase to work in Twisted, that it's worth starting such a small project > from scratch in nodejs. > > You could definitely do long polling instead. short polling is just super > simple to get up and running. > > -- > You received this message because you are subscribed to the Google Groups > "pylons-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pylons-discuss+unsubscr...@googlegroups.com. > To post to this group, send email to pylons-discuss@googlegroups.com. > Visit this group at http://groups.google.com/group/pylons-discuss. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. Visit this group at http://groups.google.com/group/pylons-discuss. For more options, visit https://groups.google.com/d/optout.
