Fork'd jobs also elicit the use of standard tools to figure out and recover from situations such as when your year old super inefficient job suddenly takes 20 hours. You can kill just that process, tell what job caused the problem, and attach gdb to it to get a backtrace. I think all of that is harder with a threaded model, although I don't know the tools you have available if you're running JRuby.
On Saturday, 8 September 2012 at 3:09 PM, Andy Newport wrote: > Sidekiq was originally built to wrangle around the fact that jruby is java so > forking is impossible (in principle) making resque a questionable choice in > the first place. To sell it on it's performance in that context is pretty > irrelevant for anyone not running a jruby app. > > There's also ruby's horrible memory model to consider, if you're forking off > before running a job your memory usage won't suddenly jump and never drop > (ruby's GC doesn't reclaim the heap very well due to it's lack of compaction) > when a job runs slightly beyond the current heap allocation. For anyone > running a decent sized deploy that's a Good Thing (tm) because that job you > wrote a year ago (that's super inefficient for that one edge case that will > "never happen") doesn't randomly take out your job system on your weekend > off... > > Finally, and this is a big one, debugging multi-threaded code is shite when > it's your own code you wrote yesterday so..... > > > On Fri, Sep 7, 2012 at 9:14 AM, Nahum Wild <[email protected] > (mailto:[email protected])> wrote: > > the decision to move to sidekiq is separate from using threadsafe! but its > > in a similar domain. I'll quote from the sidekiq readme > > (https://github.com/mperham/sidekiq) as to why I'm looking into sidekiq: > > > > "Sidekiq uses multithreading so it is much more memory efficient than > > Resque (which forks a new process for every job). You'll find that you > > might need 50 200MB resque processes to peg your CPU whereas one 300MB > > Sidekiq process will peg the same CPU and perform the same amount of work. > > Please see my blog post on Resque's memory efficiency > > (http://blog.carbonfive.com/2011/09/16/improving-resques-memory-efficiency/) > > and how I was able to shrink a Carbon Five client's resque processing farm > > from 9 machines to 1 machine." > > > > For further reading about threadsafe! I suggest this: > > http://tenderlovemaking.com/2012/06/18/removing-config-threadsafe.html > > > > Since my original post asking about this, I've now got my esdao staging > > site up and going with puma and sidekiq, still tweaking things to get zero > > downtime deploys working how I want (which is why I'm not using thin), but > > it's looking to be a winning combination. > > > > I should probably blog about it... > > > > --nahum > > > > > > On Thu, Sep 6, 2012 at 10:04 PM, Krasimir Angelov > > <[email protected] (mailto:[email protected])> wrote: > > > > Also, I've been having a very good experience with sidekiq instead of > > > > resque. > > > > > > Is this in context of enabling threadsafe! or in general? What do you > > > like in sidekiq that's not so good in resque? > > > > > > Cheers, > > > K. > > > > > > On Monday, September 3, 2012 5:29:34 AM UTC+2, nahum wrote: > > > > has anyone been doing this? > > > > > > > > I've got a backend intranet app and unreleased app that are running > > > > with this enabled in combination with "thin --threaded". Works a charm > > > > so for, although it's neither are very complicated. I'm looking at > > > > trying a more complitade Rails 3.2 app with it and puma, and wondering > > > > if anyone has had experience with that setup? > > > > > > > > Also, I've been having a very good experience with sidekiq instead of > > > > resque. > > > > > > > > cheers, > > > > nahum > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "WellRailed" group. > > > To view this discussion on the web visit > > > https://groups.google.com/d/msg/wellrailed/-/XFh4w0aNA1YJ. > > > To post to this group, send email to [email protected] > > > (mailto:[email protected]). > > > To unsubscribe from this group, send email to > > > [email protected] > > > (mailto:wellrailed%[email protected]). > > > For more options, visit this group at > > > http://groups.google.com/group/wellrailed?hl=en. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "WellRailed" group. > > To post to this group, send email to [email protected] > > (mailto:[email protected]). > > To unsubscribe from this group, send email to > > [email protected] > > (mailto:wellrailed%[email protected]). > > For more options, visit this group at > > http://groups.google.com/group/wellrailed?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "WellRailed" group. > To post to this group, send email to [email protected] > (mailto:[email protected]). > To unsubscribe from this group, send email to > [email protected] > (mailto:[email protected]). > For more options, visit this group at > http://groups.google.com/group/wellrailed?hl=en. -- You received this message because you are subscribed to the Google Groups "WellRailed" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/wellrailed?hl=en.
