Re: Long running rake tasks freeze or crash
On Oct 7, 2010, at 6:52 AM, heroku+nore...@googlegroups.com wrote:
Anyone here have any experience with long running rake tasks on
Heroku?
...
When the rake task crashes, it'll usually die with the message:
! Internal server error
Other times it'll die with the message:
... snip ...
Yep. I see this all the time too when I try to run a rake task from the
command line for anything that takes a long time.
I have no answer for this other than to observe that the same code which causes
this error when run via rake never seems to have a problem when run via cron:
desc 'tasks run automatically on Heroku'
task :cron = :environment do
if Time.now.hour == 3
LongRunner.new.run
end
end
works fine where
desc 'long running task'
task :longrunner = :environment do
LongRunner.new.run
end
heroku rake longrunner
gives the error condition you describe.
The other workaround to consider is to use Delayed::Job. If your long runner
can break up the work into individual bits that can be processed independently,
put them on the job queue and run it that way.
class LongRunner
def run
MyModel.all.each{|m| m.send_later(:do_something)}
end
end
Of course, this costs money (as does cron if you want it to run more often than
once per day).
Mike
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to her...@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
Re: Long running rake tasks freeze or crash
I believe I have read several times that cron and rake both time out eventually, and that very long running stuff should always get processed by delayed job workers. This makes sense because you're paying directly for your worker, as opposed to getting a free lunch from Heroku for that 20 hour rake task. I ran into trouble with rake tasks that took about 20 minutes or so. I thought maybe the heroku gem was crashing, or I reached a memory limit, or maybe just a hard time limit. I first reduced the output to just a . for each model I processed instead of one line message. Some tasks that previously crashed were able to complete, but the biggest rake task (most models to process) still crashed. I replaced some of the active record model creation with straight SQL, and the rake task was able to complete. Next time I plan to go directly to delayed jobs and save all that headache. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Zencoder is looking for alpha testers
Hi everyone, We're wrapping up our Heroku add-on right now and we're looking for a few alpha testers. If you're not familiar with Zencoder check us out at zencoder.com. We provide an API to video and audio transcoding. If you'd like to help us with alpha testing, just let me know, email me at bran...@zencoder.com. Thanks! -- Brandon Arbini -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Koi (how variable is variable?) vs. Ronin (expensive!) vs. Amazon RDS (winner?)
I really want to pull the trigger with Heroku. I love so much of it. I'm looking to move over a ~500K page views/month site that is decently data intensive, and still weighing my options wrt database, which may make or break my decision to use Heroku vs. EY, etc, given the price differences. Koi seems like a great deal. 20 GB is plenty for my app. My benchmarks so far seem promising. But variable performance has me concerned a bit...does anyone have more concrete numbers on just what that means? Anyone running decently high traffic sites on just Koi? The jump to Ronin is obviously dramatic in terms of price. Is there any more info on just what a compute unit is? Like RAM and I/O specs? It seems to me, given EC2 prices (small ec2 is ~85/month variable, and large = ~910/year term $100/month), that there should be a dedicated option for less than $200/month. Any hints of a Koi- like price drop in the near future here? My 2 cents to Heroku's pricing team would be treat the data layer as a break-even loss leader, and make up the revenue on the dyno/worker side... Amazon RDS seems like quite possibly the way to go. 3 year term for a small instance (1.7 GB) is $350 2 months of the cost of a Ronin instance! Has anyone gone this route and had success? Should I be worried about latency between Heroku and RDS? (it's all EC2, right?) Which zone should I have a DB placed in? (Virginia vs. California?) m -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
