Well, I kinda get the "level of abstraction" but what about a cluster node failure running my Akka streams stages? They are strictly local – how would you deal with an Iterator's iteration blowing up?
Having that said, we do actually work on "Acknowledged Sources / Sinks", the first of which is currently being worked on in: github.com/akka/reactive-kafka But that's the first step, the protocol and patterns that emerge from there will be general – to be applied in any kind of situation you need to ACK a source that you processed some data. How would you cope with it? See above. Now, Is my only option to use Kafka and some kind of partition and offset syncing? Of course it's not the only option to "use Kafka", you can build resiliency and at-most / at-least once processing any way you want, however that is not the core goal of Akka Streams, we're adding it as acknowlaged sink patterns etc. Don't confuse the level of abstraction and problems Akka Streams tackle with Spark for example (it's entire goal is assuring what you just asked for). Are you suggesting that event sourcing and reactive streams are not really to be "compatible"? What? Why? Totally the opposite is true actually – even sourcing is awesome for resiliency, elasticity and scalability. How do people solve it (resiliency and scalability) in real production systems? Could you share some recipes? Also see above, but you didn't state the exact problem so the answers will also be "fluffy" :-) If your goal fits spark, use spark; if it doesn't, here's another tool in the toolbox – with very different semantics and capabilities (note - all above comments about Akka Streams speak about any Reactive Streams implementation). -- Konrad -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
