[
https://issues.apache.org/jira/browse/BEAM-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16249892#comment-16249892
]
Kenneth Knowles commented on BEAM-3171:
---------------------------------------
I am not strongly opposed to this, actually. I think the current design for
side inputs in Beam's portable model already has the needed features designed,
but there is a lot of implementation to finish in all the runners. Part of the
concern is what the scope of this interface is. It really has little to do with
Beam if it is just a common interface across a few KV stores. If it is
experimental and part of the SQL package then I really think it is fine but
perhaps a more detailed design and scoping doc would help. SQL may or may not
have the same needs as other Beam uses of KV stores.
> convert a join into lookup
> --------------------------
>
> Key: BEAM-3171
> URL: https://issues.apache.org/jira/browse/BEAM-3171
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
> Labels: experimental
>
> We use BeamSQL to run streaming jobs mostly, and add a join_as_lookup
> improvement(internal branch) to cover the streaming-to-batch case(similar as
> [1]). I could submit a PR as experimental if people are interested.
> The rough solution is, if one source of join node implements
> {{BeamSeekableTable}} and the other is not, then the join node is converted
> to a fact-lookup operation.
> Ref:
> [1]
> https://docs.google.com/document/d/1B-XnUwXh64lbswRieckU0BxtygSV58hysqZbpZmk03A/edit?usp=sharing
>
> [~xumingming] [~takidau] for any comments
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
