Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Thanks for your kind words, Ashley. Glad you find it useful. On 27.08.14 16:25, Ashley Aitken wrote: Martin, Thank you very much for your most interesting and useful post. I think many of us benefit considerably from you sharing your practical experience with Akka Persistence and CQRS. It'

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Provability not probability (iPad helping) On Wednesday, August 27, 2014, Greg Young wrote: > I have used 1:n it's a fairly common pattern I just wanted to point out > it's not a panacea that works everywhere and had some rather large > downsides if applied in the wrong place. I agree in the dis

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Martin, Thank you very much for your most interesting and useful post. I think many of us benefit considerably from you sharing your practical experience with Akka Persistence and CQRS. It's probably too early for "best practices" but knowing what may work well (or not) in real-world applic

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

I have used 1:n it's a fairly common pattern I just wanted to point out it's not a panacea that works everywhere and had some rather large downsides if applied in the wrong place. I agree in the discussion being around when which is applicable. My original point was that you can't really do 1:1 wit

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

On 26.08.14 20:24, Andrzej Dębski wrote: You're right. If you want to keep all data in Kafka without ever deleting them, you'd need to add partitions dynamically (which is currently possible with APIs that back the CLI). On the other hand, using Kafka this way is the wrong appro

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Whether to go for a 1:1 approach or a 1:n approach (or a partitioned m:n approach where m << n) really depends on the concrete use case and non-functional requirements. Your example might be a good candidate for a 1:1 approach (see also further comments inline) but there are also examples for w

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

OK for bank accounts there is some amount of state needed to verify a transaction. Let's propose that for now its the branch you opened your account at, your current balance,your address and a risk classification as well as a customer profitability/loyalty score (these are all reasonable things to

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

> You're right. If you want to keep all data in Kafka without ever deleting > them, you'd need to add partitions dynamically (which is currently possible > with APIs that back the CLI). On the other hand, using Kafka this way is > the wrong approach IMO. If you really need to keep the full eve

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

On 26.08.14 20:12, Greg Young wrote: In particular I am interested in the associated state thats needed, I can see keeping it in a single actor but this does not turn out well at all for most production systems in particular as changes happen over time. I don't get your point. Please elabora

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

In particular I am interested in the associated state thats needed, I can see keeping it in a single actor but this does not turn out well at all for most production systems in particular as changes happen over time. On Tue, Aug 26, 2014 at 2:08 PM, Martin Krasser wrote: > See my eventsourced

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

See my eventsourced example(s), that I published 1-2 years ago, others are closed source On 26.08.14 20:06, Greg Young wrote: Love to see an example On Tuesday, August 26, 2014, Martin Krasser > wrote: On 26.08.14 19:56, Greg Young wrote: I'm curious

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Love to see an example On Tuesday, August 26, 2014, Martin Krasser wrote: > > On 26.08.14 19:56, Greg Young wrote: > > I'm curious how you would model say bank accounts with only a few hundred > actors can you go into a bit of detail > > > persistent-actor : bank-account = 1:n (instead of 1:1) >

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

On 26.08.14 19:56, Greg Young wrote: I'm curious how you would model say bank accounts with only a few hundred actors can you go into a bit of detail persistent-actor : bank-account = 1:n (instead of 1:1) On Tuesday, August 26, 2014, Martin Krasser > wrote:

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

I'm curious how you would model say bank accounts with only a few hundred actors can you go into a bit of detail On Tuesday, August 26, 2014, Martin Krasser wrote: > > On 26.08.14 16:44, Andrzej Dębski wrote: > > My mind must have filtered out the possibility of making snapshots using > Views -

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

On 26.08.14 16:44, Andrzej Dębski wrote: My mind must have filtered out the possibility of making snapshots using Views - thanks. About partitions: I suspected as much. The only thing that I am wondering now is: if it is possible to dynamically create partitions in Kafka? AFAIK the number of

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

This is definitely an issue as many people with event sourced systems have millions of topics. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://g

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

My mind must have filtered out the possibility of making snapshots using Views - thanks. About partitions: I suspected as much. The only thing that I am wondering now is: if it is possible to dynamically create partitions in Kafka? AFAIK the number of partitions is set during topic creation (be

Re: [akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Hi Andrzej, On 26.08.14 09:15, Andrzej Dębski wrote: Hello Lately I have been reading about a possibility of using Apache Kafka as journal/snapshot store for akka-persistence. I am aware of the plugin created by Martin Krasser: https://github.com/krasserm/akka-persistence-kafka/ and also I

[akka-user] Apache Kafka as journal - retention times/PersistentView and partitions

Hello Lately I have been reading about a possibility of using Apache Kafka as journal/snapshot store for akka-persistence. I am aware of the plugin created by Martin Krasser: https://github.com/krasserm/akka-persistence-kafka/ and also I read other topic about Kafka as journal https://group