Re: Re-sharded kinesis stream starts generating warnings after kinesis shard numbers were doubled

2017-10-04 Thread Mikhailau, Alex
Filed SPARK-22200 From: "Mikhailau, Alex" <alex.mikhai...@mlb.com> Date: Wednesday, October 4, 2017 at 10:43 AM To: "user@spark.apache.org" <user@spark.apache.org> Subject: Re: Re-sharded kinesis stream starts generating warnings after kinesis shard numbers w

Re: Re-sharded kinesis stream starts generating warnings after kinesis shard numbers were doubled

2017-10-04 Thread Mikhailau, Alex
-4454 With 2.2.0 -Alex From: "Mikhailau, Alex" <alex.mikhai...@mlb.com> Date: Wednesday, September 13, 2017 at 4:16 PM To: "user@spark.apache.org" <user@spark.apache.org> Subject: Re-sharded kinesis stream starts generating warnings after kinesis shard numbe

Re-sharded kinesis stream starts generating warnings after kinesis shard numbers were doubled

2017-09-13 Thread Mikhailau, Alex
Has anyone seen the following warnings in the log after a kinesis stream has been re-sharded? com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask WARN Cannot get the shard for this ProcessTask, so duplicate KPL user records in the event of resharding will not be dropped during