Re: Resizing images on Amazon S3 from Heroku
Hi Mike I have ImageMagick running on Heroku for my website. Only admins can post images, and it can take over a dyno, so it's not a perfect solution. I use it with Mark Evans' excellent Rack-based Dragonfly gem that scales my images on the fly. Details for setting up Dragonfly on Heroku-based applications is here: http://markevans.github.com/dragonfly/file.Heroku.html I simply reference the Dragonfly gem in my Gemfile, and there's no mention of ImageMagic (or similar) in my Gemfile.lock file, so I guess in development it just uses the Homebrew-installed version on my MacBook and Heroku's own pre-installed ImageMagic in production. I do reference ImageMagic in my configuration settings for Dragonfly in my Initializers directory, instructions here: http://markevans.github.com/dragonfly/file.ImageMagick.html For alternatives, there are two image processing in the cloud add-ons for Heroku, Blitline and Cloudinary. And some other online services too. But I've been pretty happy with Dragonfly and how I can easily copy down my originally-formatted images from Amazon S3 to my laptop and still display them in development mode when I'm offline and demonstrating my website. I think Dragonfly, Blitline and Cloudinary all have documention for dealing with images that are already in S3. Stephen On 23 August 2012 23:32, m...@bolser.co.uk m...@bolser.co.uk wrote: Hi all - I have lots of JPEG images in an Amazon S3 bucket. - I have a Rails app running on Heroku, which knows the filenames of the images. - The rails app is not responsible for uploading the files to S3. In fact, it's the other way round: the file is POSTed to S3, which then informs the Rails app of the file name via a success_action_redirect as per http://aws.amazon.com/articles/1434/ *The Rails app requires greyscale, thumbnail copies of the images.* So I'm thinking about writing a rake task within the Rails app to produce the copies. My questions are: 1. Is Imagemagick available and supported on Heroku? 2. If so, is there any documentation on how to use Imagemagick on Heroku? 3. If so, do I have to pull down the file from S3 to a tmp directory, make the copy then upload back to S3? 4. If so, would I incur Amazon S3 data transfer fees? or... 5. Is there a better way? :) Thanks! -- You received this message because you are subscribed to the Google Groups Heroku group. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/heroku?hl=en_US?hl=en -- You received this message because you are subscribed to the Google Groups Heroku group. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/heroku?hl=en_US?hl=en
Re: Resque vs Appoxy SimpleWorker
I created another rake task that pushed jobs onto Resque, and indeed it did
push thousands of jobs onto the queue and let me ramp through them by
increasing the number of workers. Perhaps I was hit by the recent worker bug
outlined on Heroku's status page. If nobody else has direct experience with
Sendgrid on Resque vs Appoxy, I'll do my own experiment and report back what
kind of throughput I can get.
Stephen
On 28 August 2011 08:50, Gabriel ummo...@gmail.com wrote:
On a weekly basis I load 5 - 600 jobs into a DelayedJob queue in
seconds so I think something else is going on here. Maybe it's
ActiveRecord taking a while to load a bunch of objects?
On Aug 26, 6:10 pm, Stephen Cremin asianf...@gmail.com wrote:
I recently introuduced Resque to my application and I've started moving
background tasks into queues. In day-to-day use, I use an :after_save
hook
to index articles in the background, but I also have a rake task:
===
desc Indexing articles
task :index_articles = :environment do
begin
Article.all.each { |a| Resque.enqueue(IndexArticle, a.id,
a.main_text :
a.raw_main_text), a.class.name, a.publish_date.to_i) }
end
end
===
When I ran this, I expected to see a couple of thousand articles appear
almost instantly on the relevant queue on Resque, and be able to increase
the number of workers to speed through them. Instead, I never see more
than
5-10 at a time on the queue, and the rake task takes quite some time to
run.
I have 20,000 subscribers on a mailing list that currently takes 2-3
hours
to send via SendGrid in a rake task. I was hoping that queuing these up
on
Resque would allow me to send them in minutes rather than hours with
enough
workers, but I'm guessing now that's not the case.
Would Appoxy Simpleworker let me fire off 20,000 email jobs in a queue,
that
could be sent off in minutes rather than hours? Or would Resque handle
this
and I just haven't configured it properly. Or is it something inherent in
rake that is slowing batch jobs down in general.
Although, it's not essential that the email goes out in minutes, those
hours
waiting for the emails to go out are incredibly stressful since the
mailing
list will sometimes break.
Stephen
PS:I noticed that SendGrid can now manage newsletter delivery for you,
and I
may explore that option.
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
Resque vs Appoxy SimpleWorker
I recently introuduced Resque to my application and I've started moving
background tasks into queues. In day-to-day use, I use an :after_save hook
to index articles in the background, but I also have a rake task:
===
desc Indexing articles
task :index_articles = :environment do
begin
Article.all.each { |a| Resque.enqueue(IndexArticle, a.id, a.main_text :
a.raw_main_text), a.class.name, a.publish_date.to_i) }
end
end
===
When I ran this, I expected to see a couple of thousand articles appear
almost instantly on the relevant queue on Resque, and be able to increase
the number of workers to speed through them. Instead, I never see more than
5-10 at a time on the queue, and the rake task takes quite some time to run.
I have 20,000 subscribers on a mailing list that currently takes 2-3 hours
to send via SendGrid in a rake task. I was hoping that queuing these up on
Resque would allow me to send them in minutes rather than hours with enough
workers, but I'm guessing now that's not the case.
Would Appoxy Simpleworker let me fire off 20,000 email jobs in a queue, that
could be sent off in minutes rather than hours? Or would Resque handle this
and I just haven't configured it properly. Or is it something inherent in
rake that is slowing batch jobs down in general.
Although, it's not essential that the email goes out in minutes, those hours
waiting for the emails to go out are incredibly stressful since the mailing
list will sometimes break.
Stephen
PS:I noticed that SendGrid can now manage newsletter delivery for you, and I
may explore that option.
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
Re: dragonfly and/or rmagick/ImageMagick can't display uploaded images
What Heroku stack are you on? I remember choosing the Bamboo 1.8.7 stack
about a year ago because of RMagick problems at the time on the 1.9.2 stack.
I haven't since tested on 1.9.2.
I updated to Rails 3.0.x and DragonFly 0.8.x earlier this year, but not to
DragonFly 0.9.x yet.
In my Gemfile, I state:
gem 'dragonfly', '~ 0.8.1'
gem 'rmagick', '2.12.2', :require = 'RMagick'
Try the second line in your own Gemfile.
In my views, I use:
%= image_tag @news_item.main_image.thumb('500x1000').url(:suffix =
'.jpg'), :alt = @news_item.headline %
Please report back, as I plan to migrate to 1.9.2 myself at some point this
year.
Stephen
On 9 June 2011 16:43, Dokitura ingoditt...@googlemail.com wrote:
hi there,
first: i'm still more a rails 3/heroku newbie, working with it since
beginning of 2011 with it (before worked more with rails 2.3), so i'm
at least familiar with dragonflyimagemagick ;).
now to my problem:
since the last 2 days, i can't display images (jpg, png, gif) anymore,
when uploaded via form and saved to a dragonfly-image-field, e.g.
image_accessor :picture for field picutre_uid. that means: the
uploaded file is saved to database, whereat i don't know if the file-
format is still correct at the saving-point or not. i'm showing them
by calling the url-attribute on the image-field, e.g. picture.url or
with scaling picture.process(:resize, '190x150').url.
the strange thing still coming: i'm working with 3 different app-
versions: 1. locally, 2. test-server (heroku), 3. live-server
(heroku). all bundled with same Gemfile, of course, but it's only not
working in the 3rd one(!). the other thing is that i changed nothing
on Gemfile itself (since last week till today), so that new gem-
versions maybe could resolve some bugs ... no! (today only bundle
updated for fixing purposes).
i read in some forums, that rmagick/imagemagick needs to have it image-
libs installed on app-environment(...?), e.g. jpg - jpglib, gif -
tifflib (don't remember exact names, sry). so heroku needs to have
installed some of these, i guess.
gem versions: dragonfly = 0.9.3 and rmagick = 2.13.1
also, server log tells me these errors:
convert: no decode delegate for this image format
`/app/tmp/dragonfly20110609-1-d7u3jk-0' @ error/constitute.c/ReadImage/533.
convert: missing an image filename
`/app/tmp/dragonfly20110609-1-1cg3lvy-0' @
error/convert.c/ConvertImageCommand/2940.
the frustrating thing is, my testing possibilies are very rare, e.g.
testing locally, because all working fine there, as i said before. so,
hopefully some guys of you maybe got an idea to get me on the way
fixing this. i would appreciate it very much! thx :)
greetings,
dokitura
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
--
You received this message because you are subscribed to the Google Groups
Heroku group.
To post to this group, send email to heroku@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
Re: Not responding rails application
Could be any number of things. On Heroku, your application is running in production mode, not development mode, for greater speed, etc. And Heroku uses PostgreSQL, which you may not be using locally. Have you ran all database migrations on Heroku's side (heroku rake:db migrate)? Is your application coded to assume that there is data to display, which you haven't yet pushed up to the server (heroku db:push). Your first port of call is to look at the logs (heroku logs) to get a more precise cause of the error: http://devcenter.heroku.com/articles/logging Stephen On 8 March 2011 01:47, sanmugasundaram k k.sanmug...@gmail.com wrote: i am new bee for rails. i simply develop my rails application. It run perfect in local host. i deploy this application with heroku now i got error as We're sorry, but something went wrong We've been notified about this issue and we'll take a look at it shortly. here my application http://mmfas.heroku.com/ help me -- நன்றி ... சண்முகசுந்தரம்.க Kanchi Linux User Group Rocks ! http://kanchilug.wordpress.com My Experiments In Linux are here http://gowithfoss.wordpress.com/ -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
If WebSolr search stopped working for you in the past two hours...
About two hours ago I got a message through from Exceptional that a search failed with a RSolr::RequestError. Sure enough, search was down on my site. I checked the twitter feed of WebSolr: http://twitter.com/websolr It didn't seem a coincidence that about the time search stopped working on my site, they tweeted: A misconfiguration on one of our servers has a small number of indexes down. We're working on it. WebSolr has been rock solid for me in recent months and it was the first message on their Twitter feed since 22nd January. Their next Twitter message stated It's fixed now. But even though my account's status page indicated that my index looks ok, I was still getting errors. I was able to fix search on my site by going into the console and reindexing the articles on my site: heroku console Sunspot.index!(Article.all) Stephen PS: According to my accounts page, the number of documents in my index has somehow grown eight times after reindexing, but I'm not going to risk hitting the Destroy this index button and rebuilding it again right at this moment. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: If WebSolr search stopped working for you in the past two hours...
Thanks, Nick My search engine did go down again in the past hour and I had to re-index my records once more to fix that. I'm based in Asia where it's Monday morning and my site's busiest time so I'm not going to do anything drastic right now. I'll wipe the index and regenerate it overnight and contact your support if the problem hasn't resolved itself. I've been with you since May 2010, so it may be an unrelated problem. Stephen On 31 January 2011 10:59, Nick Zadrozny n...@onemorecloud.com wrote: Hi Stephen, Nick from Websolr here. Indeed, we did have a bit of a hiccup earlier today. The number of indexes affected was relatively small, as it occurred on only one server. Folks that were affected would have been those who added the Websolr add-on between mid-December and mid-January. Some more details, for those curious… It turns out that one of the other indexes on that server was improperly configured. Usually we can handle that case pretty gracefully, but this particular index exposed an edge case in Solr that we weren't previously aware of. Unfortunately, when we reloaded Solr this afternoon during the course of some normal maintenance, this particular configuration error prevented Solr from loading correctly. Hence the errors you experienced. We noticed the problem right away, and after a few minutes of troubleshooting were able to find and deploy a fix for the root cause. That fix has also since been deployed to all of our other servers to prevent this particular case from occurring again in the future. You mentioned some lingering issues… if anyone else experiences any persisting issues I encourage you to open a ticket at http://support.heroku.com/ or http://help.websolr.com/ with your WEBSOLR_URL so we can take a closer look. Also, the issue you mention with your number of documents being much larger than you expect sounds particularly puzzling… I'd definitely be curious to take a look at that, and can possibly help you migrate to a fresh index if that proves necessary. On Sun, Jan 30, 2011 at 4:57 PM, Stephen Cremin asianf...@gmail.comwrote: About two hours ago I got a message through from Exceptional that a search failed with a RSolr::RequestError. Sure enough, search was down on my site. I checked the twitter feed of WebSolr: http://twitter.com/websolr It didn't seem a coincidence that about the time search stopped working on my site, they tweeted: A misconfiguration on one of our servers has a small number of indexes down. We're working on it. WebSolr has been rock solid for me in recent months and it was the first message on their Twitter feed since 22nd January. Their next Twitter message stated It's fixed now. But even though my account's status page indicated that my index looks ok, I was still getting errors. I was able to fix search on my site by going into the console and reindexing the articles on my site: heroku console Sunspot.index!(Article.all) Stephen PS: According to my accounts page, the number of documents in my index has somehow grown eight times after reindexing, but I'm not going to risk hitting the Destroy this index button and rebuilding it again right at this moment. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.comheroku%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/heroku?hl=en. -- Nick Zadrozny http://websolr.com — hassle-free hosted search, powered by Apache Solr -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.comheroku%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: AWS Elastic Beanstalk
It arguably does one better than supporting Ruby by supporting JRuby. Charles Nutter has posted a blog entry on his initial experiences running JRuby on Rails on Elastic Beanstalk: http://blog.headius.com/2011/01/jruby-on-rails-on-amazon-elastic.html It's early days for Elastic Beanstalk, but I imagine I'll feel more comfortable with Heroku for a long time. Amazon allows you to fine-tune your application with all kinds of configuration options, but Heroku just works. One deploys on Beanstalk right now by uploading a .war file. When they offer more languages, perhaps they'll also offer git push deployment. I'd like to use a Java graph database (Neo4j) for my application, and Amazon's new offering permits me to build a domain-specific REST interface (perhaps as a Sinatra application) in (J)Ruby running on Beanstalk and have that communicate with low latency to my front-end Heroku application. I think that's the best of two worlds. Stephen On 20 January 2011 06:13, jgwmaxw...@gmail.com wrote: Has anyone looked at the AWS Elastic Beanstalk they've just put into Beta? Whilst it doesn't support Ruby as yet, does it offer something conceptually similar to Heroku? -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.comheroku%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to heroku@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Neo4j
This is something I'm also very interested in. Googling, there do seem to be examples of people who have set up their own Neo4j instances on Amazon's cloud, which would presumably have low-latency talking to Heroku. There are some very raw official instructions here: http://wiki.neo4j.org/content/Neo4j_in_the_Cloud The official REST interface is still at an early stage of development, and it might make more sense to implement your own REST interface using a Sinatra or Rails app in JRuby with the rapidly evolving Neo4j.rb gem: http://github.com/andreasronge/neo4j/wiki That would ideally be hosted on Amazon, with your Heroku application communicating with your data through the REST interface you create. The advantage of rolling your own, is that your REST interface can be domain specific. I'd LOVE to see a Neo4j add-on, but any REST interface offered is going to be extremely limited. And I'd still need to host an application running JRuby somewhere to use the descriptive power of the Neo4j.rb gem. If you make any progress, do post here. Stephen On 23 September 2010 16:18, Martin Solli mso...@gmail.com wrote: Is there anyone offering Neo4j instances in the cloud? If so, what are your experiences, and have you used it with a Heroku-hosted app? -martin -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.comheroku%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups Heroku group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
