Re: Auto-scaling

2011-01-17 Thread Daniel Huckstep
Almost the exact same code can be used to scale DJ stuff. If you check out 
the "Hooks" section of DJ, https://github.com/collectiveidea/delayed_job, 
you can see it supports the same kind of hooks resque does, specifically the 
ones needed for auto-scaling, `enqueue` and `after`.

You can take the code from my blog post about resque, and massage it into 
those methods in your DJ job class, and it should work pretty much the same 
way.

-- 
You received this message because you are subscribed to the Google Groups 
"Heroku" group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to 
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/heroku?hl=en.



Re: Auto-scaling

2011-01-16 Thread John Maxwell
I imagine that the concept would be similar. I don't think DJ supports the 
callbacks that Resque does though - so at the end of each Job you'd need to 
call a method manually to check on what the outstanding depth on the job 
queue is and adjust the workers.

The only problem with DJ is that because it requeues failed jobs for 
retrying then potentially you'd have to take that into account when setting 
the number of workers - query the jobs table to see how many are current 
rather than future, and go off that?

-- 
You received this message because you are subscribed to the Google Groups 
"Heroku" group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to 
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/heroku?hl=en.



Re: Auto-scaling

2011-01-16 Thread TiagoP
What about auto-scaling for delayed_jobs instead of resque?

On Jan 15, 10:24 pm, John Maxwell  wrote:
> The Heroku gem itself can do this for you - if you put in a before or
> after_filter a quick check on HTTP_X_HEROKU_QUEUE_DEPTH or 
> HTTP_X_HEROKU_QUEUE_WAIT_TIME
> depending on your criteria, then use the gem to add or remove a dyno as
> suits you.  Look 
> athttp://blog.darkhax.com/2010/07/30/auto-scale-your-resque-workers-on-...for
> inspiration on how to do it.

-- 
You received this message because you are subscribed to the Google Groups 
"Heroku" group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to 
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/heroku?hl=en.



Re: Auto-scaling

2011-01-15 Thread John Maxwell
The Heroku gem itself can do this for you - if you put in a before or 
after_filter a quick check on HTTP_X_HEROKU_QUEUE_DEPTH or 
HTTP_X_HEROKU_QUEUE_WAIT_TIME 
depending on your criteria, then use the gem to add or remove a dyno as 
suits you.  Look at 
http://blog.darkhax.com/2010/07/30/auto-scale-your-resque-workers-on-heroku for 
inspiration on how to do it.

-- 
You received this message because you are subscribed to the Google Groups 
"Heroku" group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to 
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/heroku?hl=en.