[
https://issues.apache.org/jira/browse/BEAM-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033655#comment-17033655
]
Etienne Chauchot commented on BEAM-9270:
----------------------------------------
Hi [~krisnaru], thannks for raising this !
I can indeed reproduce this bug.
What happens is that:
Beam Read needs to evaluate the size of the data it needs to read to know how
to split for parallel read. To get the size of data the ElasticsearchIO calls
the stats API of ES. Right now the IO uses a low level ES REST client. So it
gets a json object from ES cluster listing all the indices available in the
cluster. The IO then searches for the index name provided by the user in the
json. In that case it does not find a match with the "*" character and
considers that the provided index does not exist and that the size is 0 and
returns no document.
To fix that it would require to update the json parsing by manually expanding
the wildcard. But there is an ongoing complete refactoring of ElasticsearchIO
(see the thread on the dev mailing list) that would allow to use high level ES
client and that will unlock many ES features. So either we do a fix to match
the index name in the stats using a regExpr or we wait for the new
ElasticsearchIO.
> Unable to use ElasticsearchIO for indexpatterns: index-*
> --------------------------------------------------------
>
> Key: BEAM-9270
> URL: https://issues.apache.org/jira/browse/BEAM-9270
> Project: Beam
> Issue Type: Bug
> Components: io-java-elasticsearch
> Affects Versions: 2.19.0
> Reporter: Krishnaiah Narukulla
> Priority: Major
>
> ElasticsearchIO input doesnot work with index patterns with wildcard. for
> example: index-*. I It works well with single index. It becomes problem if
> we need to query multiple indices using index pattern.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
