Here's a good writeup that also mentions Kafka: http://highscalability.com/blog/2014/11/17/aeron-do-we-really-need-another-messaging-system.html
Comparing it to Kafka is an apples to oranges comparison from what I can tell. Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Wed, Nov 19, 2014 at 7:15 PM, Daniel Compton <[email protected]> wrote: > Hi Roger > > While Aeron does indeed prioritise latency, it also has exceptional > throughput performance as well. The early benchmark one user did put it at > 2x Kafka: > > https://groups.google.com/d/msg/mechanical-sympathy/GrEce0gP7RU/gDzl5JG2dfwJ > . > While benchmarking is fraught with perils and should be taken with FDA > approved amounts of salt, it looks like Aeron is at least in the same > ballpark for throughput. > > On 20 November 2014 06:37, Roger Hoover <[email protected]> wrote: > > > Hi Daniel, > > > > Thanks for sharing this. Looks like a great project. I probably don't > > know enough to give a great answer but will throw in my 2c anyway. > > > > I think Kafka prioritizes throughput and aeron prioritizes latency. As > you > > mentioned, maybe Aeron could replace the current Kafka TCP protocol. The > > issues I foresee are with things like batching and compression, supported > > by Kafka. These features trade latency for throughput to great effect > but > > may be something that Aeron protocol would never support because of it's > > focus on latency first. > > > > Cheers, > > > > Roger > > > > On Mon, Nov 17, 2014 at 11:12 AM, Daniel Compton <[email protected] > > > > wrote: > > > > > Recently there has been some interesting discussion on the Mechanical > > > Sympathy mailing list about Martin Thompson’s new extremely high > > > performance, extremely low latency, open source messaging system called > > > Aeron. For those of you that don’t know, Martin Thompson is an expert > on > > > wringing as much throughput and as little latency as possible out of > > > hardware for financial firms so his background made me very interested > in > > > this. > > > > > > The high level gist is that Aeron has publishers that write messages > to a > > > partitioned, replicated, immutable log, and subscribers that are > > delivered > > > messages from the immutable log using message offsets. Sound familiar? > > > There are a few differences ( > > > > > > https://groups.google.com/d/msg/mechanical-sympathy/fr7moLcsuiI/ZfUMk4QUbN8J > > ), > > > the key ones being that there are no brokers in this system, and that > > Aeron > > > is a pure OSI layer 4 implementation. > > > > > > At first it seems like Aeron is a competing product to Kafka, and > indeed > > > for some cases it could be competing. However because it is implemented > > as > > > a layer 4 transport, after thinking about it for a while, I think it > > could > > > be implemented into Kafka as an alternative to the TCP transport that > > Kafka > > > uses. The semantics would need to be thought through, but I think you > > could > > > implement the brokered model on top of this, as well as supporting > point > > to > > > point subscriptions bypassing the brokers. > > > > > > I wanted to bring this to the groups attention and see what others > > > thought. Aeron is still pretty new, and there’s a lot to digest in this > > so > > > I wouldn’t recommend trying to land it in 0.8.2 :), but I thought it’s > > > worth considering it for future development. > > > > > > The Youtube video where he talks about it is at > > > https://www.youtube.com/watch?v=tM4YskS94b0 and the GitHub repo is at > > > https://github.com/real-logic/Aeron. Discussion is spread across a few > > > threads but starts here: > > > > > > https://groups.google.com/d/msg/mechanical-sympathy/fr7moLcsuiI/04unxUfIJ8sJ > > > and > > > > https://groups.google.com/forum/#!topic/mechanical-sympathy/GrEce0gP7RU. > > > > > > Daniel. > > > > > > > > >
