Re: error while querying state

Hi, Which Flink version are you using? Cheers, Gordon -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: ProducerFencedException when running 2 jobs with FlinkKafkaProducer

Hi, I think this is unexpected. The generated transactional ids should not be clashing. Looking at the FlinkKafkaProducer code, it seems like the generation is only a function of the subtask id of the FlinkKafkaProducer, which could be the same across 2 different Kafka sources. I'm not

Re: Flink 1.6 Yarn Session behavior

Hi, I'm forwarding this question to Gary (CC'ed), who most likely would have an answer for your question here. Cheers, Gordon On Wed, Feb 13, 2019 at 8:33 AM Jins George wrote: > Hello community, > > I am trying to upgrade a Flink Yarn session cluster running BEAM > pipelines from version

Re: How to register TypeInfoFactory for 'external' class

Hi Alexey, I don't think using the @TypeInfo annotation is doable at the moment. Is this class being used only for input / output types of functions / operators? Or are you using it as a state type? For the former, I think you can explicitly set the TypeInformation by calling setTypeInfo on

Re: In-Memory state serialization with kryo fails

Hi, I would suggest to avoid Kryo for state serialization, especially if this job is meant for production usage. It might get in the way in the future when you might decide to upgrade your value state schema. To do that, when declaring the descriptor for your MapState, provide a specific

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! Best, Kurt On Wed, Feb 13, 2019 at 10:02 AM Shaoxuan Wang wrote: > Congratulations, Thomas! > > On Tue, Feb 12, 2019 at 5:59 PM Fabian Hueske wrote: > >> Hi everyone, >> >> On behalf of the Flink PMC I am happy to announce Thomas Weise as a new >> member of the Apache Flink

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congratulations, Thomas! On Tue, Feb 12, 2019 at 5:59 PM Fabian Hueske wrote: > Hi everyone, > > On behalf of the Flink PMC I am happy to announce Thomas Weise as a new > member of the Apache Flink PMC. > > Thomas is a long time contributor and member of our community. > He is starting and

ProducerFencedException when running 2 jobs with FlinkKafkaProducer

Hey all, I am running into an issue where if I run 2 flink jobs (same jar, different configuration), that produce to different kafka topics on the same broker, using the 1.7 FlinkKafkaProducer set with EXACTLY_ONCE semantics, both jobs go into a checkpoint exception loop every 15 seconds or

Re: Limit in batch flink sql job

OK, thanks. It might be better to update the document which has the following example that confused me. SELECT * FROM Orders LIMIT 3 -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Flink 1.6 Yarn Session behavior

Hello community, I am trying to upgrade a Flink Yarn session cluster running BEAM pipelines from version 1.2.0 to 1.6.3. Here is my session start command: yarn-session.sh -d -n 4 -jm 1024 -tm 3072 -s 7 Because of the dynamic resource allocation, no taskmanager gets created initially.

How to register TypeInfoFactory for 'external' class

Hello, I've class, which I can't change (generated code or class from third party library), so I can't use @TypeInfo annotation, how can I register TypeInfoFactory for such class? It looks like TypeExtractor has map of registered factories - registeredTypeInfoFactories and even registration

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congratulations Thomas. Regards, Ananth From: zhijiang Reply-To: zhijiang Date: Wednesday, 13 February 2019 at 2:38 am To: Jark Wu , Kostas Kloudas Cc: Hequn Cheng , Stefan Richter , user Subject: Re: [ANNOUNCE] New Flink PMC member Thomas Weise Congrats Thomas! Best, Zhijiang

In-Memory state serialization with kryo fails

Hi mates ! I’ve implemented a job, that stores it’s progress using MapState[K, V], where K - is java.lang.String, and V - is a collection of some typed objects java.util.List[SomeClass[_]] When Flink is trying to serialize this state, it is using kryo serializer for value object and fails with

Re: HA HDFS

I had gotten everything setup with HDFS and Zookeeper. The problem I was having is that I was hard coding the online namenode. This would cause everything to fall apart in the event of a failover. I needed to configure Flink to use the hdfs config file in order to allow for correct failover. I

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! Best, Zhijiang -- From:Kostas Kloudas Send Time:2019年2月12日(星期二) 22:46 To:Jark Wu Cc:Hequn Cheng ; Stefan Richter ; user Subject:Re: [ANNOUNCE] New Flink PMC member Thomas Weise Congratulations Thomas! Best,

Re: Program of Flink Forward San Francisco 2019 announced

Something that I forgot to mention: We offer a 25% discount for every community member who is subscribed to a Flink mailing list. When you register and enter the discount code MailingList you will receive a 25% discount on all conference tickets, including tickets for training. Best, Fabian

Program of Flink Forward San Francisco 2019 announced

Hi everyone, We announced the program of Flink Forward San Francisco 2019. The conference takes place at the Hotel Nikko in San Francisco on April 1st and 2nd. On the first day we offer three training sessions [1]: * Introduction to Streaming with Apache Flink * Analyzing Streaming Data with

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congratulations Thomas! Best, Kostas On Tue, Feb 12, 2019 at 12:39 PM Jark Wu wrote: > Congrats Thomas! > > On Tue, 12 Feb 2019 at 18:58, Hequn Cheng wrote: > >> Congrats Thomas! >> >> Best, Hequn >> >> >> On Tue, Feb 12, 2019 at 6:53 PM Stefan Richter < >> s.rich...@data-artisans.com> wrote:

Re: stream of large objects

Thanks Konstantin. And when serialization of events does become an issue because of size (say 100s MBs or GBs) how does it manifest itself? Is it mostly latency or something else? Ajay From: Konstantin Knauf Date: Tuesday, February 12, 2019 at 3:41 AM To: "Aggarwal, Ajay" Cc: Chesnay

Re: Running single Flink job in a job cluster, problem starting JobManager

I have investigated this further: During normal operation, without Spring Boot, OptimizerPlanEnvironment.ProgramAbortException is thrown by Flink from StreamPlanEnvironment::execute():70. This is caught by PackagedProgram::callMainMethod():537, where it is re-thrown as an Error exception.

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! On Tue, 12 Feb 2019 at 18:58, Hequn Cheng wrote: > Congrats Thomas! > > Best, Hequn > > > On Tue, Feb 12, 2019 at 6:53 PM Stefan Richter < > s.rich...@data-artisans.com> wrote: > >> Congrats Thomas!, >> >> Best, >> Stefan >> >> Am 12.02.2019 um 11:20 schrieb Stephen Connolly <

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! Regards, Dian > 在 2019年2月12日，下午6:58，Hequn Cheng 写道： > > Congrats Thomas! > > Best, Hequn > > > On Tue, Feb 12, 2019 at 6:53 PM Stefan Richter > wrote: > Congrats Thomas!, > > Best, > Stefan > >> Am 12.02.2019 um 11:20 schrieb Stephen

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! Best, Hequn On Tue, Feb 12, 2019 at 6:53 PM Stefan Richter wrote: > Congrats Thomas!, > > Best, > Stefan > > Am 12.02.2019 um 11:20 schrieb Stephen Connolly < > stephen.alan.conno...@gmail.com>: > > Congratulations to Thomas. I see that this is not his first time in the > PMC

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas!, Best, Stefan > Am 12.02.2019 um 11:20 schrieb Stephen Connolly > : > > Congratulations to Thomas. I see that this is not his first time in the PMC > rodeo... also somebody needs to update LDAP as he's not on > https://people.apache.org/phonebook.html?pmc=flink >

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congratulations to Thomas. I see that this is not his first time in the PMC rodeo... also somebody needs to update LDAP as he's not on https://people.apache.org/phonebook.html?pmc=flink yet! -stephenc On Tue, 12 Feb 2019 at 09:59, Fabian Hueske wrote: > Hi everyone, > > On behalf of the Flink

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! I'm very happy to have you in our PMC as an experienced open source advocate :) On Tue, Feb 12, 2019 at 11:09 AM jincheng sun wrote: > Congrats Thomas ! > > Cheers, > Jincheng > > Fabian Hueske 于2019年2月12日周二 下午5:59写道： > >> Hi everyone, >> >> On behalf of the Flink PMC I am

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas ! Cheers, Jincheng Fabian Hueske 于2019年2月12日周二 下午5:59写道： > Hi everyone, > > On behalf of the Flink PMC I am happy to announce Thomas Weise as a new > member of the Apache Flink PMC. > > Thomas is a long time contributor and member of our community. > He is starting and

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas! Best, tison. Jeff Zhang 于2019年2月12日周二 下午6:01写道： > Congrats Thomas ! > > Fabian Hueske 于2019年2月12日周二 下午5:59写道： > >> Hi everyone, >> >> On behalf of the Flink PMC I am happy to announce Thomas Weise as a new >> member of the Apache Flink PMC. >> >> Thomas is a long time

Re: [ANNOUNCE] New Flink PMC member Thomas Weise

Congrats Thomas ! Fabian Hueske 于2019年2月12日周二 下午5:59写道： > Hi everyone, > > On behalf of the Flink PMC I am happy to announce Thomas Weise as a new > member of the Apache Flink PMC. > > Thomas is a long time contributor and member of our community. > He is starting and participating in lots of

[ANNOUNCE] New Flink PMC member Thomas Weise

Hi everyone, On behalf of the Flink PMC I am happy to announce Thomas Weise as a new member of the Apache Flink PMC. Thomas is a long time contributor and member of our community. He is starting and participating in lots of discussions on our mailing lists, working on topics that are of joint

error while querying state

Hi all , i am facing issue with QueryableStateClient i am getting this issue please help me Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.flink.util.FlinkRuntimeException: java.io.EOFException at

Re: Limit in batch flink sql job

Hi, It's as the error message says. LIMIT 10 without ORDER BY would pick 10 random rows and hence lead to non-deterministic results. That's why it is not supported yet. Best, Fabian Am Di., 12. Feb. 2019 um 07:02 Uhr schrieb yinhua.dai < yinhua.2...@outlook.com>: > Why flink said "Limiting the

Re: stream of large objects

Hi Ajay, when repartitioning the stream the events need to transferred between Taskmanagers (processes/nodes). Just passing a reference there won't work. If it is serialization you are worried about and you don't need access to the List of messages inside the job, you might as well store this