[
https://issues.apache.org/jira/browse/BEAM-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770023#comment-16770023
]
Tony Moulton commented on BEAM-6654:
------------------------------------
Thanks [~iemejia]! The SDF blog post is where the original inspiration for
this work came from. Thank you for the reference to the HBaseIO.ReadAll
implementation, I'll review that code before putting together something for the
community to review and discuss.
> Support SDF within ElasticsearchIO
> ----------------------------------
>
> Key: BEAM-6654
> URL: https://issues.apache.org/jira/browse/BEAM-6654
> Project: Beam
> Issue Type: New Feature
> Components: io-java-elasticsearch
> Reporter: Tony Moulton
> Assignee: Tony Moulton
> Priority: Minor
>
> Similar to JdbcIO.ReadAll, it would be very useful to support a SDF based
> transform which can compose queries based off of input elements.
> Implementation would split the work similarly to how the current Source API
> implementation works (leveraging slices within Elasticsearch).
> This work can facilitate a transition of ElasticsearchIO to SDF when more
> runners have implemented support. Until then, I propose leaving the read()
> implementation alone and add a readAll method and associated transform to be
> implemented as an SDF. Once enough runner support is ready, the existing
> Read transform would be updated to use the same SDF as the ReadAll transform
> with a null input element.
> A prototype implementation currently exists (albeit developed in a bubble) at
> [https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall]
> however hasn't been synced with master for a few weeks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
