[
https://issues.apache.org/jira/browse/BEAM-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889642#comment-16889642
]
Rui Wang edited comment on BEAM-7758 at 7/21/19 5:04 AM:
---------------------------------------------------------
1. I would suggest you start from
[here|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTableProvider.java#L54]
to trace down to understand current BeamSQL's table provider. In prod we don't
need a addTable implementation for table providers.
2. Windowing properties for the unbounded table is supposed to be solved by
https://s.apache.org/streaming-beam-sql. This is the proposal that we can use
pure SQL to specify windowing property. Therefore you don't need to worry about
windowing property now.
3. I actually think you can try to explore if you can do
"unbounded_PCollection.apply(SqlTransform.query().withSlowChangingTableProvider()".
However, I don't like such syntax. The more nature API should be
"pipeline.apply(Join(unbounded_PC, slow_changing_PC).ON(join_condition))."
However, it is out of scope now.
was (Author: amaliujia):
1. I would suggest you start from
[here|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTableProvider.java#L54]
to trace down to understand current BeamSQL's table provider. In prod we don't
need a addTable implementation for table providers.
2. Windowing properties for the unbounded table is supposed to be solved by
https://s.apache.org/streaming-beam-sql. This is the proposal that we can use
pure SQL to do windowing .
3. I actually think you can try to explore if you can do
"unbounded_PCollection.apply(SqlTransform.query().withSlowChangingTableProvider()".
However, I don't like such syntax. The more nature API should be
"pipeline.apply(Join(unbounded_PC, slow_changing_PC).ON(join_condition))."
However, it is out of scope now.
> Table returns PCollectionView in BeamSQL
> ----------------------------------------
>
> Key: BEAM-7758
> URL: https://issues.apache.org/jira/browse/BEAM-7758
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql
> Reporter: Rui Wang
> Assignee: Rahul Patwari
> Priority: Major
>
> We might be able to define a table with properties that says this table
> return a PCollectionView. By doing so we will have a trigger based
> PCollectionView available in SQL rel nodes.
> Relevant thread:
> https://lists.apache.org/thread.html/602121ee49886590ce4975f66aa0d270b4a5d64575337fca8bef1232@%3Cdev.beam.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
