Yi Pan (Data Infrastructure) created SAMZA-483:
--------------------------------------------------
Summary: A common representation of relational algebra for
streaming SQL
Key: SAMZA-483
URL: https://issues.apache.org/jira/browse/SAMZA-483
Project: Samza
Issue Type: Sub-task
Reporter: Yi Pan (Data Infrastructure)
Priority: Minor
Per discussion with [~criccomini] and [~milinda], we agreed that it seems to be
a good idea to define a common representation of relational algebra on top of
the operators defined in the operator layer (see SAMZA-482), which can be the
common base that we can use to generate the description/configuration of a
Samza job.
This common layer can also be used by DSL-like language parser as a result of
parsing a DSL program.
Some additional requirements needed in addition to pure relational algebra:
1) the common representation should include window operators and stream
operators (i.e. IStream/DStream/RStream)
2) the common representation should include description on parallelism of the
jobs (i.e. how many partitions the resultant Samza job will use)
Some references:
http://web.cs.wpi.edu/~mukherab/i/DCAPE.pdf
https://cs.uwaterloo.ca/~david/cs848/stream-cql.pdf
http://davis.wpi.edu/dsrg/PROJECTS/CAPE/publications.htm
http://davis.wpi.edu/dsrg/PROJECTS/CAPE/slides.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
