Kinesis Connector - NoClassDefFoundError

2018-11-19 Thread Steve Bistline
Hey all... upgrade from Flink 1.5.0 to 1.6.2 and for some reason cannot figure out what I missed in setting up the new environment. I am gettin this error: java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.kinesis.shaded.com.amazonaws.partitions.PartitionsLoader

Re: Savepoint failed with error "Checkpoint expired before completing"

2018-11-19 Thread Gagan Agrawal
Good to know that Steven. It will be useful feature to have separate time out configs for both. Gagan On Mon, Nov 5, 2018, 10:06 Steven Wu FYI, here is the jira to support timeout in savepoint REST api > https://issues.apache.org/jira/browse/FLINK-10360 > > On Fri, Nov 2, 2018 at 6:37 PM Gagan A

Job unable to stabilise after restart

2018-11-19 Thread Daniel Harper
Hi there, I’ve raised this issue https://issues.apache.org/jira/browse/FLINK-10928 I recognise it’s a bit vague and there is a lot of information on that ticket, but we’re having a lot of trouble getting to the root cause in our setup. Can anyone help/point us in the right direction? :)

Re: Deadlock happens when sink to mysql

2018-11-19 Thread miki haiat
can you share your entire code please On Mon, Nov 19, 2018 at 4:03 PM 徐涛 wrote: > Hi Experts, > I use the following sql, and sink to mysql, > select > > album_id, date > count(1) > from > coupon_5_discount_date_conv > group by > album_id, date; > > > when sink to mysql, the f

Deadlock happens when sink to mysql

2018-11-19 Thread 徐涛
Hi Experts, I use the following sql, and sink to mysql, select album_id, date count(1) from coupon_5_discount_date_conv group by album_id, date; when sink to mysql, the following SQL is executed: insert into xxx (c1,c2,c3) values (?,?,?) on duplicate k

Weird behavior in actorSystem shutdown in akka

2018-11-19 Thread Joshua Fan
Hi, Till and users, There is a weird behavior in actorSystem shutdown in akka of our flink platform. We use flink 1.4.2 on yarn as our flink deploy mode, and we use an ongoing agent to submit flink job to yarn which is based on YarnClient. User can connect to the agent to submit job and disconnect

Group by with null keys

2018-11-19 Thread Flavio Pompermaier
Hi to all, we wanted to do a group by on elements that can contains null values and we discovered that Table API support this while Dataset API does not. Is this documented somehwere on the Flink site? Best, Flavio --- PS: you can test this wit

Re: Enabling Flink’s checkpointing

2018-11-19 Thread Andrey Zagrebin
Hi Olga, Not sure where it is stated in the docs to make sure that source/sink implements the interface. From what I see in the source code, if a user function does not implement CheckpointedFunction or ListCheckpointed, its operator state will just not be snapshotted, but I do not see any pro

Re: Data loss when restoring from savepoint

2018-11-19 Thread Juho Autio
Hi Stefan, Bravo doesn't currently support reading a reducer state. I gave it a try but couldn't get to a working implementation yet. If anyone can provide some insight on how to make this work, please share at github: https://github.com/king/bravo/pull/11 Thanks. On Tue, Oct 23, 2018 at 3:32 PM

Re: Could not find previous entry with key.

2018-11-19 Thread Dawid Wysakowicz
Hi, Which version of flink do you use? I would recommend upgrading to 1.6.2 for which we reworked internal structures and such problem should not occur. Best, Dawid On 19/11/2018 08:30, Chesnay Schepler wrote: > @klou @dawid What is the most likely cause if this is not caused by an > improper eq