Re: Review Request 48215: SAMZA-727: Manually rebased and updated to build. Waiting for refreshed patches from Chen Song

2016-06-06 Thread Jagadish Venkatraman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48215/#review136337 --- Ship it! lgtm. Thanks for rebasing! - Jagadish Venkatraman O

Re: Review Request 48182: SAMZA-958: Make store/cache thread safe

2016-06-06 Thread Chris Pettitt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48182/#review136335 --- Fix it, then Ship it! samza-test/src/test/scala/org/apache/sam

Re: Review Request 48213: SAMZA-960: Make system producer thread safe

2016-06-06 Thread Chris Pettitt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48213/#review136338 --- Fix it, then Ship it! samza-hdfs/src/main/scala/org/apache/sam

Re: Review Request 48182: SAMZA-958: Make store/cache thread safe

2016-06-06 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48182/#review135984 --- Fix it, then Ship it! some nits.. otherwise, +1 ! samza-kv-ro

Update all values in RocksDB

2016-06-06 Thread David Yu
We use Samza RocksDB to keep track of our user event sessions. The task periodically calls window() to update all sessions in the store and purge all closed sessions. We do all of this in the same iterator loop. Here's how we are doing it: public void window(MessageCollector collector, TaskCoor

Re: Update all values in RocksDB

2016-06-06 Thread Yi Pan
Hi, David, I would recommend to keep a separate table of closed sessions as a "queue", ordered by the time the session is closed. And in your window method, just create an iterator in the "queue" and only make progress toward the end of the "queue", and do a point deletion in the sessionStore, whi

Re: Update all values in RocksDB

2016-06-06 Thread David Yu
Hi, Yi, Yes, the sessions are keyed by the sessionId. In our case, iterating through all OPEN sessions is inevitable, since that is precisely where we evaluate (base on timestamp) and close sessions. In other words, the closed session queue you suggested cannot be constructed without going throug