-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29754/
-----------------------------------------------------------
(Updated Jan. 31, 2015, 7:49 a.m.)
Review request for samza, Chris Riccomini, Jay Kreps, Milinda Pathirage, Navina
Ramesh, and Naveen Somasundaram.
Changes
-------
Updated RB description
Bugs: SAMZA-482
https://issues.apache.org/jira/browse/SAMZA-482
Repository: samza
Description (updated)
-------
Overview of the proposal:
- 4 different types of operators implemented via 2 interface classes:
* stream-to-relation operators implementing TupleOperator interface and
generate Relation output. E.g. Window operators
* stream-to-stream operators implementing TupleOperator interface and
generate Tuple output. E.g. Partition operators
* relation-to-relation operators implementing RelationOperator interface and
generate Relation output. E.g. All relational algebra operators, s.t. join,
where, group-by, select, etc.
* relation-to-stream operators implementing RelationOperator interface and
generate Tuple output. E.g. Istream or Dstream operators
Those operators are connected via the following two context interface classes:
- SqlMessageCollector which provides a MessageCollector interface for the
operators to send their output to
- OperatorRouter which provides the connection interface between the operators
In the example, we have enabled two execution models via the above two context
classes:
- OperatorMessageCollector that uses the routing information from a
OperatorRouter and directly invoking the next operator when the current
operator send its output via the OperatorMessageCollector
- StoreMessageCollector that provides a storage for each operator's outputs
that stored the output when the current operator send its output via
StoreMessageCollector.
Then, it is up to the programmer to query the StoreMessageCollector to get the
operator's output and proceed w/ the next steps
The first execution model allows the integration w/ future SQL parser and
planner to automatically run a task, while the second model allows a random
programmer to use the operators from the library in a random context.
Diffs
-----
build.gradle 7a40ad4ae916610186848c06c4577e7067de98ee
gradle/dependency-versions.gradle 44dd42603e93788562fd64c68312570cee71a2aa
samza-sql/README PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/data/EntityName.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/data/Relation.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/operators/Operator.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/relation/Join.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/operators/window/WindowState.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/sql/router/SimpleRouter.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java
PRE-CREATION
samza-sql/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java
PRE-CREATION
samza-sql/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java
PRE-CREATION
samza-sql/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java
PRE-CREATION
settings.gradle 3a01fd66359b8c79954ae8f34eeaf4b2e3fdc0b4
Diff: https://reviews.apache.org/r/29754/diff/
Testing
-------
run ./bin/check-all.sh passed
Thanks,
Yi Pan (Data Infrastructure)
