Re: What are favored Redis-backed queues?
Also, I should mention that Ruby doesn't have very good built in parallelism support (no true threads when I was using, though this might have changed). As such, I've seen a fair bit of usage of Resque running on a single machine. This would be an insane overcomplication in Clojure given all it's concurrency/parallelism support. So unless you're actually planning to distribute tasks across a cluster, keep it simple and stick to things like the built in Clojure reference types and core.async. Chris -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What are favored Redis-backed queues?
I think you mean to be asking specifically about *job* queuing using Redis. You don't need anything other than Redis + Carmine to create queues. But obviously, the value of Resque (note spelling) is that it *uses* Redis in a number of non-trivial ways, such that all of the features above are available. There does appear to have been a Clojure port of Resqueue (https://github.com/jxa/resque-clojure) which garnered a bit of attention, but it hasn't been updated in a while, so I don't know what the state of it is. It's worth considering that if you don't *need* something very robust (and depending on your situation), you could just use Redis + Carmine directly to pass messages around (possibly representing jobs), and have workers pull from the message queue. There is nothing else you really need for this; it's quite straight forward. The rub is that you don't get any of the higher level features; if a job dies for instance, there's no way to know about that without building it yourself. It's also worth considering whether a job queue is really the right mode of distributed computing. This area is a real strength of Clojure's, and there are a lot of offerings here. I say this because the offerings are a lot slimmer in Ruby land (at least when I was working with it), and so if you're thinking queues based on prior experience with Ruby, I'd definitely dig a little more to make sure it's the right model for you. It might seem like already having Redis in your system makes it a good goto, but if you're bending the model it'll end up being more work. Some other things to consider: * storm * onyx * quasar (https://github.com/puniverse/quasar) * pulsar (http://docs.paralleluniverse.co/pulsar) Chris -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
What are favored Redis-backed queues?
I'm looking at this old post from Github, that lists the features they were looking for in a message queue: - Persistence - See what's pending - Modify pending jobs in-place - Tags - Priorities - Fast pushing and popping - See what workers are doing - See what workers have done - See failed jobs - Kill fat workers - Kill stale workers - Kill workers that are running too long - Keep Rails loaded / persistent workers - Distributed workers (run them on multiple machines) - Workers can watch multiple (or all) tags - Don't retry failed jobs - Don't release failed jobs https://github.com/blog/542-introducing-resque They ended up creating Rescue, and using Redis in the background. Lately I've been looking at Carmine, but I'm wondering, what are some of the queues that people are using with Clojure, in particular, those using Redis? (Since the subject is potentially immense, I figure I should limit conversation to Redis. I am already using Redis in production, so for me anything using Redis is easy to add in.) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
