Re: [akka-user] Best practices for huge data sets and actor back-pressure

2014-10-02 Thread Michael Frank
cheers :) -Michael On 10/02/14 11:17, tigerfoot wrote: Very interesting! I'll be watching this project go forward. Like the back-flow Ack too! Simple and effective. Thank you! On Thursday, October 2, 2014 10:41:16 AM UTC-5, Michael Frank wrote: sounds to me like a use case for akka-str

Re: [akka-user] Best practices for huge data sets and actor back-pressure

2014-10-02 Thread tigerfoot
Very interesting! I'll be watching this project go forward. Like the back-flow Ack too! Simple and effective. Thank you! On Thursday, October 2, 2014 10:41:16 AM UTC-5, Michael Frank wrote: > > sounds to me like a use case for akka-streams: > > http://akka.io/news/2014/09/12/akka-streams-0.7-r

Re: [akka-user] Best practices for huge data sets and actor back-pressure

2014-10-02 Thread Michael Frank
sounds to me like a use case for akka-streams: http://akka.io/news/2014/09/12/akka-streams-0.7-released.html note that akka-streams is still in development preview. if you don't want to hitch your cart to it yet, why don't you modify the storage actor to send an Ack message back upstream to t

[akka-user] Best practices for huge data sets and actor back-pressure

2014-10-02 Thread tigerfoot
Hello, I have a system that does the following: read/parse from a huge (150+GB) file of data, run sanity/cleaning operations, and store the cleaned records in my DB. I've got this configured using actors such that each time I read an object from the input file I send a message to the cleaner a