Re: Elasticsearch connector and number of shards

2016-08-25 Thread Flavio Pompermaier
I've just added a JIRA improvement ticket for this ( https://issues.apache.org/jira/browse/FLINK-4491). Best, Flavio On Wed, Jul 20, 2016 at 4:21 PM, Maximilian Michels wrote: > The connector doesn't cover this use case. Through the API you need to > use the IndicesAdminClient: > https://www.el

Re: Elasticsearch connector and number of shards

2016-07-20 Thread Maximilian Michels
The connector doesn't cover this use case. Through the API you need to use the IndicesAdminClient: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-admin-indices.html Otherwise Elasticsearch creates an index with shards automatically. We could add support for configuring

Re: Elasticsearch connector and number of shards

2016-07-18 Thread Flavio Pompermaier
Indeed, we've tried with the parameter *index.number_of_shards* but it didn't work so I fear that this parameter is not handled by the current implementation..am I wrong? On Mon, Jul 18, 2016 at 11:37 AM, Ufuk Celebi wrote: > I've never used the Elasticsearch sink, but the docs say: > > "Note ho

Re: Elasticsearch connector and number of shards

2016-07-18 Thread Ufuk Celebi
I've never used the Elasticsearch sink, but the docs say: "Note how a Map of Strings is used to configure the Sink. The configuration keys are documented in the Elasticsearch documentationhere. Especially important is the cluster.name parameter that must correspond to the name of your cluster." T

Elasticsearch connector and number of shards

2016-07-18 Thread Flavio Pompermaier
Hi to all, we tried to use the streaming ES connector of Flink 1.1-SNAPSHOT and we wanted to set the number of shards when creating a new index. Is that possible at the moment? Best, Flavio