Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-11 Thread Michael Frank
i don't think that maintaining a hazelcast map is 'un-akka' at all. it sounds like it works for you. what are you hoping to improve, specifically? what is it about hazelcast that causes you pain? what does "pure-non hazelcast approach" mean? you need to track some state (submitted jobs)

Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-10 Thread atomly
Not strictly akka, but the design seems relatively straightforward. Use some form of event sourcing or message queue-- Kafka seems popular for this type of thing-- to submit/coordinate processing. Give every ten minute iteration a unique id. When you process an item in that iteration, note that

Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-10 Thread kraythe
Ok, I am sorry, I can't be precise. NDAs and so on. Let me try to bullet point it. 1) Up to 10 million objects are in system. 2) At any time some of those 10 million can be ready to change state due to events in real life outside my control. 3) We check for eligible objects ever 10 minutes

Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-10 Thread Akka Team
I agree with Michael, I don't really understand what is actually needed here, it is too big wall of text :) But from what I understand, I second Michael's assesment is that you probably are after eventsourcing and akka-persistence. -Endre On Mon, May 9, 2016 at 7:28 PM, Michael Frank

Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-09 Thread Michael Frank
It would help if your problem statement was more concrete. however, in my vague understanding of the problem, it seems like event sourcing would be an appropriate way to model your business logic: http://martinfowler.com/eaaDev/EventSourcing.html if that is the case, then i think using akka

Re: [akka-user] Inspectable Inboxes? Resolving a large state processing problem.

2016-05-09 Thread kraythe
Thats the thing, if there were humans with inboxes I could have a staff call them on the phone and check. :) Reprocessing the messages is a pretty simple solution IF the messages were small in number. When you get to the point where there are literally millions of events the problem gets a bit