This thread is a bit old, but it's seen two year awakenings before... and still shows up high in google searches. I wonder how you folks see it now. Onyx is a bit older by now, and not much seems new out there. What would you choose for reliable and resilient distributed computing with clojure, if you had to start over at this time? in particular, for a message driven architecture not a streaming one.
On Monday, July 20, 2015 at 3:52:44 AM UTC+3, Derek Troy-West wrote: > > We use Storm/Trident fairly extensively for distributed computation, it > has not been painless and the documentation is poor, however it does > perform well once you understand its peculiarities. I'm keeping an > interested eye on Onyx but my bandwidth is fairly limited at the moment. > > The only things I would add to your post are: > > * It's possible to write Storm and Trident topologies purely in Clojure, > in fact parts of Storm were originally written in Clojure. I'm not sure the > DSL were an afterthought, but I agree they are fairly impenetrable at > first. Storm has a large learning curve and lots of quirks, and the > documentation is pretty bad (in-fact, completely wrong in parts). > > * Storm is currently limited to Clojure 1.5.1, thought the latest beta > release has been updated to 1.6.0 > > * Recently the momentum of the project appears to have picked up, but > certainly for a while there (just after apache incubation) it appeared a > bit stagnant. I presume they were busy settling the project in. > > > On Monday, July 20, 2015 at 3:11:24 AM UTC+10, Christopher Small wrote: >> >> >> I'll also add that if you're interested in the Storm model (distributed >> stream processing), you may want to check out Onyx ( >> https://github.com/onyx-platform/onyx). It's newer, but I have a feeling >> that moving forward we're going to see it take a dominant position as far >> as that flavor of distributed computing goes. The reasons for this >> (briefly): >> >> * Data all the topologies: Because computational topologies are data >> structures, it's far simpler to have more dynamic computational workflows >> than with storms opaque macros. >> * It's more Clojure centric: Parts of Storm are Clojure, but the Clojure >> API seems almost an afterthought, reading the documentation. (However, >> support is on the way for other JVM languages). >> * Pace of development: Storm is now an Apache Incubator project, and >> development is slower than molasses. I was working on a project for over a >> year and many times encountered issues with stale dependencies that didn't >> play nicely with newer things. >> * Designed with batch processing in mind: Even though both are designed >> around streaming first, Onyx has a little more explicit support for batched >> computations. >> >> For a while, the biggest advantage of Storm was that it was a lot faster, >> but the gap there has closed considerably recently (perhaps entirely?). >> Storm is still more battle tested, but there are folks starting to use it >> in production. >> >> The biggest reason I see Onyx taking sway over Storm is the data centric >> approach. I see this resonating with the community's "data all the things" >> philosophy as a way of maximizing composability and productivity. >> Anecdotally, folks using Onyx are already saying this about it. >> >> My 2 c >> >> Chris >> >> >> >> On Sunday, July 19, 2015 at 9:38:09 AM UTC-7, Devin Walters (devn) wrote: >>> >>> http://docs.paralleluniverse.co/pulsar/ is out there. I can't say I've >>> used it in anger, but I did enjoy experimenting with it. >>> On Sun, Jul 19, 2015 at 11:24 AM Colin Yates <colin...@gmail.com> wrote: >>> >>>> I don’t have anything to add at that scale, but I wanted to echo >>>> Stuart’s comment about the serialisability of EDN. Moving EDN between the >>>> front and back tiers on our app has cut down a bunch of boilerplate. That >>>> principle can scale across machines as well. >>>> >>>> On 19 Jul 2015, at 16:54, Stuart Sierra <the.stua...@gmail.com> wrote: >>>> >>>> Absolutely would use again. But I'm biased towards Clojure already. :) >>>> –S >>>> >>>> On Sunday, July 19, 2015 09goral wrote: >>>>> >>>>> Thanks Stuart for your answer, it is very helpfull. Would you choose >>>>> Clojure again ? >>>>> >>>>> 2015-07-19 17:13 GMT+02:00 Stuart Sierra: >>>>> >>>>>> >>>>>> This is an old thread, but it showed up in my Google Groups so I >>>>>> figured I would give an answer. >>>>>> >>>>>> I have worked on fairly large (10-50 machines) distributed systems >>>>>> written entirely in Clojure.... >>>>>> >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> >>>> clojure+u...@googlegroups.com >>>> >>>> >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
