> On May 7, 2015, 2:22 p.m., Milinda Pathirage wrote: > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoiner.java, > > line 47 > > <https://reviews.apache.org/r/33749/diff/2/?file=948513#file948513line47> > > > > I think that stream-to-stream joining is not practical for possibly > > infinite streams. We may need to define some constraints or some other > > restrictions to make this practical.
Hi, Milinda, yes. Infinite stream-to-stream join should not be allowed. My thought is that the parser/planner should invalidate the unbounded join via inspecting the join conditions. Consider the each input stream is "ordered on field X in A" and "ordered on field Y in B" if join condition does not put a bounded range according to field X in A and field Y in B, the validation should fail. - Yi ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33749/#review82821 ----------------------------------------------------------- On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33749/ > ----------------------------------------------------------- > > (Updated May 4, 2015, 6:58 a.m.) > > > Review request for samza, Chris Riccomini, Guozhang Wang, Milinda Pathirage, > Navina Ramesh, and Naveen Somasundaram. > > > Bugs: SAMZA-650 > https://issues.apache.org/jira/browse/SAMZA-650 > > > Repository: samza > > > Description > ------- > > WIP: SAMZA-650 window store implementation > > First patch to implemement window store and message store. > > - Added window store initial implementation > - Added MessageStore initial implementation as well > - Completed skeleton APIs in window operators to illustrate the use cases of > window store and message store > - Added a StreamTask test case w/ a single window operator usage > - Unit tests are still WIP and will be updated later > > > Diffs > ----- > > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Stream.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/exception/OperatorException.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoinSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoiner.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindow.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowAutoOp.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowOp.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindowOp.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOp.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOpSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowState.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/FilteredMessageIterator.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/HashPrefixedMessageStore.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStore.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStoreSpec.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OffsetKey.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/PrefixedKey.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/Range.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeAndOffsetKey.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeKey.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowOutputStream.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowState.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowStore.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java > PRE-CREATION > samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java > PRE-CREATION > > samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java > PRE-CREATION > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java > PRE-CREATION > > samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java > PRE-CREATION > samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/33749/diff/ > > > Testing > ------- > > ./gradlew clean :samza-sql-core:build > > > Thanks, > > Yi Pan (Data Infrastructure) > >
