[
https://issues.apache.org/jira/browse/BEAM-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766566#comment-16766566
]
Tony Moulton commented on BEAM-6654:
------------------------------------
If a design document is warranted I'm happy to put one together.
> Support SDF within ElasticsearchIO
> ----------------------------------
>
> Key: BEAM-6654
> URL: https://issues.apache.org/jira/browse/BEAM-6654
> Project: Beam
> Issue Type: New Feature
> Components: io-java-elasticsearch
> Reporter: Tony Moulton
> Priority: Minor
>
> Similar to JdbcIO.ReadAll, it would be very useful to support a SDF based
> transform which can compose queries based off of input elements.
> Implementation would split the work similarly to how the current Source API
> implementation works (leveraging slices within Elasticsearch).
> This work can facilitate a transition of ElasticsearchIO to SDF when more
> runners have implemented support. Until then, I propose leaving the read()
> implementation alone and add a readAll method and associated transform to be
> implemented as an SDF. Once enough runner support is ready, the existing
> Read transform would be updated to use the same SDF as the ReadAll transform
> with a null input element.
> A prototype implementation currently exists (albeit developed in a bubble) at
> [https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall]
> however hasn't been synced with master for a few weeks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
