----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34664/ -----------------------------------------------------------
(Updated May 27, 2015, 11:24 p.m.) Review request for samza, Yan Fang, Chinmay Soman, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram. Summary (updated) ----------------- SAMZA-650 window store implementation Bugs: SAMZA-650 https://issues.apache.org/jira/browse/SAMZA-650 Repository: samza Description ------- WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks Conflicts: samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoinSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOpSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowState.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/HashPrefixedMessageStore.java samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java WIP: updated operator API and use case in test tasks Conflicts: samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOp.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStore.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStoreSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowState.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowStore.java samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java SAMZA-552: Updated operator API w/o callbacks SAMZA-552: updated Operator API w/o callbacks SAMZA-552: use OperatorCallback to allow implementation of callbacks w/o inheriting and creating many sub-classes from operators SAMZA-552 update the operator API SAMZA-552: operator builder API update Squashed commit of the following: commit fad81106901e494d3950eeaafaeefef482ac0125 Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Mon May 25 23:40:00 2015 -0700 SAMZA-650 window message store and window store implementation commit 58c2eeebf4bb0975f70aeba733379e1104f3a7de Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Fri May 22 00:25:13 2015 -0700 WIP: window store implementation commit 917e1b599622c2d46ad9a6c63e52dcded893eb8e Merge: 174eeb0 1183e9f Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Fri May 22 00:18:35 2015 -0700 Merge branch 'samza-window-op' into samza-650-v2 Conflicts: samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Stream.java samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorSink.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoinSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java commit 1183e9fdce56eb2ffa7fbdf9bbe9df92b8229252 Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Thu May 7 17:44:28 2015 -0700 WIP: updated operator API and use case in test tasks commit 8ada0ccb5be708fe2f98d701387b910f88feb3ff Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Wed May 6 22:35:08 2015 -0700 WIP: update operator API to allow callbacks and allow a single API to trigger OperatorRouter execution w/ user callbacks commit ee00b610be39b380acb9cd09e41949df2d36b7c3 Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Fri May 1 14:50:40 2015 -0700 SAMZA-650: adding updateOutputs(), getResult(), and flush() in window operator. Adding example code as use case for window operator. commit 3c73d81e45c26e1f1a7d6cc43f5a61dd33d3f76d Author: Yi Pan (Data Infrastructure) <yi...@linkedin.com> Date: Sun Apr 26 12:54:36 2015 -0700 WIP: SAMZA-650 window store implementation Diffs ----- samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorSink.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/exception/OperatorException.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/OperatorTopology.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/SimpleOperatorImpl.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/SimpleRouter.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/TopologyBuilder.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoinSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowOp.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindow.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindowOp.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/RetentionPolicy.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOp.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOpSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowState.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/FilteredMessageIterator.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/HashPrefixedMessageStore.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStore.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStoreSpec.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OffsetKey.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/PrefixedKey.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/Range.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeAndOffsetKey.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeKey.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowOutputStream.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowState.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowStore.java PRE-CREATION samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION Diff: https://reviews.apache.org/r/34664/diff/ Testing ------- Thanks, Yi Pan (Data Infrastructure)