Re: Kinesis connector SHARD_GETRECORDS_MAX default value

to the getRecords call, i.e. - SHARD_GETRECORDS_MAX - SHARD_GETRECORDS_INTERVAL_MILLIS - SHARD_GETRECORDS_BACKOFF_* Cheers, Gordon On March 23, 2017 at 2:12:32 AM, Steffen Hausmann (stef...@hausmann-family.de <mailto:stef...@hausmann-family.de>) wrote: Hi there, I recently ran into pr

Kinesis connector SHARD_GETRECORDS_MAX default value

Hi there, I recently ran into problems with a Flink job running on an EMR cluster consuming events from a Kinesis stream receiving roughly 15k event/second. Although the EMR cluster was substantially scaled and CPU utilization and system load were well below any alarming threshold, the

Re: Previously working job fails on Flink 1.2.0

>>> the hash of the tuple’s key should never change and must be >immutable. If >>> you can notice a change in hash code, that change is what breaks >your code. >>> I am pretty sure that Flink 1.1.x might just silently accept a >mutation of >>> the key, but ac

Re: Previously working job fails on Flink 1.2.0

eterministically, you could also try to set a breakpoint for the line of the exception and take a look if the key that is about to be inserted is somehow special. Best, Stefan Am 20.02.2017 um 14:32 schrieb Steffen Hausmann <stef...@hausmann-family.de>: Hi there, I’m having proble

Re: Kinesis Connector Dependency Problems

Hi Fabian, I can confirm that the behaviour is reproducible with both, Maven 3.3.9 and Maven 3.0.5. Cheers, Steffen Am 8. November 2016 11:11:19 MEZ, schrieb Fabian Hueske : >Hi, > >I encountered this issue before as well. > >Which Maven version are you using? >Maven 3.3.x

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

w if this helped. Aris -------- *From:* Steffen Hausmann <stef...@hausmann-family.de> *Sent:* Thursday, September 1, 2016 8:58 AM *To:* user@flink.apache.org *Subject:* NoClassDefFoundError with ElasticsearchSink on Yarn H

NoClassDefFoundError with ElasticsearchSink on Yarn

Hi there, I’m running a flink program that reads from a Kinesis stream and eventually writes to an Elasticsearch2 sink. When I’m running the program locally from the IDE, everything seems to work fine, but when I’m executing the same program on an EMR cluster with Yarn, a

Re: Submitting watermarks through a Kinesis stream

s. >As long as you match the Flink source parallelism to the number of >Kinesis >shared, that part is taken care of for you. > >You only need to publish watermarks to the shared that describe that >shard's particular event time. > >On Mon, Aug 29, 2016 at 3:30 PM, Steffen Hau

Submitting watermarks through a Kinesis stream

Hi there, I'm feeding a Flink stream with events from a Kinesis stream and I'm looking for some guidance on how to enable event time in the Flink stream. I've read through the documentation and it seems like I want to add events that carry watermark information to the Kinesis stream and