Having gone that path, we eventually gave up. Any high resolution image cannot be processed with ImageMagick without 2-4GB of RAM. We now run our workers on EC2, documented in http://artsy.github.io/blog/2012/01/31/beyond-heroku-satellite-delayed-job-workers-on-ec2/ .
On Thu, Jun 13, 2013 at 5:20 AM, Daniel Farina <dan...@fdr.io> wrote: > On Thu, Jun 13, 2013 at 2:03 AM, Josal <jos...@gmail.com> wrote: > > What I plan is to stop the process and flag it as "pending" in my > > backoffice, making the process manually later in my machine as an > exception. > > > > I've used 2X dynos, but all the memory available is used and R14 errors > also > > are thrown. Imagemagick works this way, with much memory. I've limited > > delegating to disk cache instead of memory cache > > (http://www.imagemagick.org/script/command-line-options.php#limit, > suggested > > in some places, example here: > > http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=23090), > but > > no changes. > > Tricky. Nominally this is what Linux cgroups was invented to help > with, but I think the amount of power delegated to non-root users (and > thus, on Heroku) is minimal. > > Here are two options that come to mind: > > 1) Somehow delegate these possibly expensive processes to their own 2X > dyno, taking the computation out-of-band. e.g., a queuing strategy, > multi-app delegation (one app posting to another), or the Heroku API > (be careful with that latter one, as it's basically automated "spend > money", and one can hit api limits, also, the control plane has worse > availability than standing-processes...but it has upsides, like burst > capacity and 0-cost when there's nothing to do) > > 2) Use /proc/[self|pid]/statm or /proc/[self|pid]/status (which has > similar information, but is more human-targeted) to inspect the RSS > size of a process over time to take control over over-bloated > processes. A process could watch itself grow in this way, which would > be useful if it can do it semi-frequently enough to write a "too big, > can't do" record and give up. > > If one uses /proc/pid/statm, be aware one will have to multiply by the > system page size, which has been 4096 bytes for years but *could* > change any time, so consider 'getconf PAGESIZE' to pick up the > multiplier. > > -- > -- > You received this message because you are subscribed to the Google > Groups "Heroku" group. > > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en_US?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "Heroku Community" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to heroku+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- dB. | Moscow - Geneva - Seattle - New York code.dblock.org - @dblockdotorg <http://twitter.com/#!/dblockdotorg> - artsy.net - github/dblock <https://github.com/dblock> -- -- You received this message because you are subscribed to the Google Groups "Heroku" group. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/heroku?hl=en_US?hl=en --- You received this message because you are subscribed to the Google Groups "Heroku Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to heroku+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
