[
https://issues.apache.org/jira/browse/BEAM-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779019#comment-16779019
]
Tony Moulton commented on BEAM-6654:
------------------------------------
I took some time to put together a Google doc for the implementation proposal
and have also pushed a branch with an updated prototype implementation to
[https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall-2] for
reference and conversation. If it makes more sense to discuss these topics
over a PR, let me know and I'll create one.
> Support SDF within ElasticsearchIO
> ----------------------------------
>
> Key: BEAM-6654
> URL: https://issues.apache.org/jira/browse/BEAM-6654
> Project: Beam
> Issue Type: New Feature
> Components: io-java-elasticsearch
> Reporter: Tony Moulton
> Assignee: Tony Moulton
> Priority: Minor
>
> Similar to HbaseIO.ReadAll, it would be very useful to support a SDF based
> transform which can compose queries based off of input elements.
> Implementation would split the work similarly to how the current Source API
> implementation works (leveraging slices within Elasticsearch).
> This work can facilitate a transition of ElasticsearchIO to SDF when more
> runners have implemented support. Until then, I propose leaving the read()
> implementation alone and add a readAll method and associated transform to be
> implemented as an SDF. Once enough runner support is ready, the existing
> Read transform would be updated to use the same SDF as the ReadAll transform
> with a null input element.
> A prototype implementation currently exists (albeit developed in a bubble) at
> [https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
