Re: Subtask keeps on discovering new Kinesis shard when using Kinesalite

2016-11-18 Thread Philipp Bussche
-archive.2336050.n4.nabble.com/Subtask-keeps-on-discovering-new-Kinesis-shard-when-using-Kinesalite-tp10133p10213.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Subtask keeps on discovering new Kinesis shard when using Kinesalite

2016-11-16 Thread Tzu-Li (Gordon) Tai
resharding, and will start consuming the shard from sequence number EARLIEST_SEQUENCE_NUM with ShardConsumer 2 {"domainId":null,"title":null,"description":null,"venue":null,"message":"could not parse message"} -- View thi

Re: Subtask keeps on discovering new Kinesis shard when using Kinesalite

2016-11-16 Thread Philipp Bussche
e to resharding, and will start consuming the shard from sequence number EARLIEST_SEQUENCE_NUM with ShardConsumer 2 {"domainId":null,"title":null,"description":null,"venue":null,"message":"could not parse message"} -- View this

Re: Subtask keeps on discovering new Kinesis shard when using Kinesalite

2016-11-15 Thread Tzu-Li (Gordon) Tai
Hi Philipp, When used against Kinesalite, can you tell if the connector is already reading data from the test shard before any of the shard discovery messages? If you have any spare time to test this, you can set a larger value for the `ConsumerConfigConstants.SHARD_DISCOVERY_INTERVAL_MILLIS`

Subtask keeps on discovering new Kinesis shard when using Kinesalite

2016-11-15 Thread Philipp Bussche
if this is maybe a side effect ? Thanks Philipp -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Subtask-keeps-on-discovering-new-Kinesis-shard-when-using-Kinesalite-tp10133.html Sent from the Apache Flink User Mailing List archive. mailing