I'd have to know a little bit more about the specifics of your message driven architecture to say for sure, but in general I'd still highly recommend Onyx over Storm. In the last two years the Distributed Masonry crew have done a TON of fabulous work with Onyx, including closing the performance gap (maybe someone can chime in here about the current sate of that gap) by switching to Aeron for message passing (the old version used HornetMQ I believe). It's also significantly more battle tested now, and the team just recently got $500k for building out tooling and services around Onyx. There are also quite a few new features, like improved agregation support, and so on. They also implemented a runtime for onyx jobs in cljc, meaning you can test out jobs locally without all the overhead of firing up zookeeper! EVEN IN THE BROWSER! Imagine that; dynamically building and testing (with small data) a distributed system in the browser via some domain-specific UI, and then pushing it to the cluster when you're ready to let her rip... Nothing like this exists for Storm or any of the other competitors in this space. I've even heard some dashing and enterprizing group of fellows are considering building a re-frame-esque event-driven UI system using Onyx. Who knows? The future is bright.
Chris PS; Conflict of Interest Disclaimer: I have recieved no moneys or favors from Distributed Masonry for frequent and prefuse recommendation of their products. They're just that cool. On Fri, Jun 30, 2017 at 12:53 AM, Matan <dev.ma...@gmail.com> wrote: > This thread is a bit old, but it's seen two year awakenings before... and > still shows up high in google searches. > I wonder how you folks see it now. Onyx is a bit older by now, and not > much seems new out there. What would you choose for reliable and resilient > distributed computing with clojure, if you had to start over at this time? > in particular, for a message driven architecture not a streaming one. > > > On Monday, July 20, 2015 at 3:52:44 AM UTC+3, Derek Troy-West wrote: >> >> We use Storm/Trident fairly extensively for distributed computation, it >> has not been painless and the documentation is poor, however it does >> perform well once you understand its peculiarities. I'm keeping an >> interested eye on Onyx but my bandwidth is fairly limited at the moment. >> >> The only things I would add to your post are: >> >> * It's possible to write Storm and Trident topologies purely in Clojure, >> in fact parts of Storm were originally written in Clojure. I'm not sure the >> DSL were an afterthought, but I agree they are fairly impenetrable at >> first. Storm has a large learning curve and lots of quirks, and the >> documentation is pretty bad (in-fact, completely wrong in parts). >> >> * Storm is currently limited to Clojure 1.5.1, thought the latest beta >> release has been updated to 1.6.0 >> >> * Recently the momentum of the project appears to have picked up, but >> certainly for a while there (just after apache incubation) it appeared a >> bit stagnant. I presume they were busy settling the project in. >> >> >> On Monday, July 20, 2015 at 3:11:24 AM UTC+10, Christopher Small wrote: >>> >>> >>> I'll also add that if you're interested in the Storm model (distributed >>> stream processing), you may want to check out Onyx ( >>> https://github.com/onyx-platform/onyx). It's newer, but I have a >>> feeling that moving forward we're going to see it take a dominant position >>> as far as that flavor of distributed computing goes. The reasons for this >>> (briefly): >>> >>> * Data all the topologies: Because computational topologies are data >>> structures, it's far simpler to have more dynamic computational workflows >>> than with storms opaque macros. >>> * It's more Clojure centric: Parts of Storm are Clojure, but the Clojure >>> API seems almost an afterthought, reading the documentation. (However, >>> support is on the way for other JVM languages). >>> * Pace of development: Storm is now an Apache Incubator project, and >>> development is slower than molasses. I was working on a project for over a >>> year and many times encountered issues with stale dependencies that didn't >>> play nicely with newer things. >>> * Designed with batch processing in mind: Even though both are designed >>> around streaming first, Onyx has a little more explicit support for batched >>> computations. >>> >>> For a while, the biggest advantage of Storm was that it was a lot >>> faster, but the gap there has closed considerably recently (perhaps >>> entirely?). Storm is still more battle tested, but there are folks starting >>> to use it in production. >>> >>> The biggest reason I see Onyx taking sway over Storm is the data centric >>> approach. I see this resonating with the community's "data all the things" >>> philosophy as a way of maximizing composability and productivity. >>> Anecdotally, folks using Onyx are already saying this about it. >>> >>> My 2 c >>> >>> Chris >>> >>> >>> >>> On Sunday, July 19, 2015 at 9:38:09 AM UTC-7, Devin Walters (devn) wrote: >>>> >>>> http://docs.paralleluniverse.co/pulsar/ is out there. I can't say I've >>>> used it in anger, but I did enjoy experimenting with it. >>>> On Sun, Jul 19, 2015 at 11:24 AM Colin Yates <colin...@gmail.com> >>>> wrote: >>>> >>>>> I don’t have anything to add at that scale, but I wanted to echo >>>>> Stuart’s comment about the serialisability of EDN. Moving EDN between the >>>>> front and back tiers on our app has cut down a bunch of boilerplate. That >>>>> principle can scale across machines as well. >>>>> >>>>> On 19 Jul 2015, at 16:54, Stuart Sierra <the.stua...@gmail.com> wrote: >>>>> >>>>> Absolutely would use again. But I'm biased towards Clojure already. :) >>>>> –S >>>>> >>>>> On Sunday, July 19, 2015 09goral wrote: >>>>>> >>>>>> Thanks Stuart for your answer, it is very helpfull. Would you choose >>>>>> Clojure again ? >>>>>> >>>>>> 2015-07-19 17:13 GMT+02:00 Stuart Sierra: >>>>>> >>>>>>> >>>>>>> This is an old thread, but it showed up in my Google Groups so I >>>>>>> figured I would give an answer. >>>>>>> >>>>>>> I have worked on fairly large (10-50 machines) distributed systems >>>>>>> written entirely in Clojure.... >>>>>>> >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> >>>>> clojure+u...@googlegroups.com >>>>> >>>>> >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> clojure+u...@googlegroups.com >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/clojure/4GRJVxctlU0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
