Hi Roger While Aeron does indeed prioritise latency, it also has exceptional throughput performance as well. The early benchmark one user did put it at 2x Kafka: https://groups.google.com/d/msg/mechanical-sympathy/GrEce0gP7RU/gDzl5JG2dfwJ. While benchmarking is fraught with perils and should be taken with FDA approved amounts of salt, it looks like Aeron is at least in the same ballpark for throughput.
On 20 November 2014 06:37, Roger Hoover <[email protected]> wrote: > Hi Daniel, > > Thanks for sharing this. Looks like a great project. I probably don't > know enough to give a great answer but will throw in my 2c anyway. > > I think Kafka prioritizes throughput and aeron prioritizes latency. As you > mentioned, maybe Aeron could replace the current Kafka TCP protocol. The > issues I foresee are with things like batching and compression, supported > by Kafka. These features trade latency for throughput to great effect but > may be something that Aeron protocol would never support because of it's > focus on latency first. > > Cheers, > > Roger > > On Mon, Nov 17, 2014 at 11:12 AM, Daniel Compton <[email protected]> > wrote: > > > Recently there has been some interesting discussion on the Mechanical > > Sympathy mailing list about Martin Thompson’s new extremely high > > performance, extremely low latency, open source messaging system called > > Aeron. For those of you that don’t know, Martin Thompson is an expert on > > wringing as much throughput and as little latency as possible out of > > hardware for financial firms so his background made me very interested in > > this. > > > > The high level gist is that Aeron has publishers that write messages to a > > partitioned, replicated, immutable log, and subscribers that are > delivered > > messages from the immutable log using message offsets. Sound familiar? > > There are a few differences ( > > > https://groups.google.com/d/msg/mechanical-sympathy/fr7moLcsuiI/ZfUMk4QUbN8J > ), > > the key ones being that there are no brokers in this system, and that > Aeron > > is a pure OSI layer 4 implementation. > > > > At first it seems like Aeron is a competing product to Kafka, and indeed > > for some cases it could be competing. However because it is implemented > as > > a layer 4 transport, after thinking about it for a while, I think it > could > > be implemented into Kafka as an alternative to the TCP transport that > Kafka > > uses. The semantics would need to be thought through, but I think you > could > > implement the brokered model on top of this, as well as supporting point > to > > point subscriptions bypassing the brokers. > > > > I wanted to bring this to the groups attention and see what others > > thought. Aeron is still pretty new, and there’s a lot to digest in this > so > > I wouldn’t recommend trying to land it in 0.8.2 :), but I thought it’s > > worth considering it for future development. > > > > The Youtube video where he talks about it is at > > https://www.youtube.com/watch?v=tM4YskS94b0 and the GitHub repo is at > > https://github.com/real-logic/Aeron. Discussion is spread across a few > > threads but starts here: > > > https://groups.google.com/d/msg/mechanical-sympathy/fr7moLcsuiI/04unxUfIJ8sJ > > and > > https://groups.google.com/forum/#!topic/mechanical-sympathy/GrEce0gP7RU. > > > > Daniel. > > > > >
