Re: Flink SQL: Execute DELETE queries

2019-05-28 Thread JingsongLee
Hi @Papadopoulos, Konstantinos I think you can try something like this: JDBCAppendTableSink sink = JDBCAppendTableSink.builder() .setDrivername("foo") .setDBUrl("bar") .setQuery("delete from %s where id = ?)") .setParameterTypes(FIELD_TYPES) .build(); Or you can build your own Sink

Building Flink distribution with Scala2.12

2019-05-28 Thread Boris Lublinsky
Hi, I am trying to build Flink distribution for Scala 2.12 using the following command: mvn clean package -pl flink-dist -am -Pscala-2.12 -DskipTests , but I am getting the following error: [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-runtime_2.12 --- [WARNING]

Re: Upgrading from 1.4 to 1.8, losing Kafka consumer state

2019-05-28 Thread Nikolas Davis
I checked the logs thanks to Paul's suggestion. I see a couple interesting things. Restoring into 1.8 from a 1.4 savepoint, some TMs receive partial state (e.g. only a partition/offset pair or two per TM -- we have 8 partitions on this topic). I'm not sure if this is normal (e.g. maybe TMs only

RE: Flink SQL: Execute DELETE queries

2019-05-28 Thread Papadopoulos, Konstantinos
The case I have in mind was to have an external JDBC table sink and try to delete a number of or all rows of the target DB table. Is it possible using Flink SQL? From: Vasyl Bervetskyi Sent: Tuesday, May 28, 2019 5:36 PM To: Papadopoulos, Konstantinos Cc: user@flink.apache.org Subject: RE:

RE: Flink SQL: Execute DELETE queries

2019-05-28 Thread Vasyl Bervetskyi
Hi Papadopoulos, Unfortunately no, there is no DELETE or MODIFY queries, you should create new table as a result of query which will filter records from existing one From: Papadopoulos, Konstantinos Sent: Tuesday, May 28, 2019 5:25 PM To: user@flink.apache.org Subject: Flink SQL: Execute

Re: [External] Re: How many task managers can a cluster reasonably handle?

2019-05-28 Thread Antonio Verardi
Thanks for the info, Xintong Song! Cheers, Antonio On Fri, May 24, 2019 at 3:38 AM Xintong Song wrote: > Hi Antonio, > > According to experience in our production, Flink totally can handle 150 > TaskManagers per cluster. Actually, we have encountered much larger jobs > with thousands that

Flink SQL: Execute DELETE queries

2019-05-28 Thread Papadopoulos, Konstantinos
Hi all, I experiment on Flink Table API & SQL and I have the following question; is there any way to execute DELETE queries using Flink SQL? Thanks in advance, Konstantinos

Flink CLI distributed cache fault

2019-05-28 Thread Vasyl Bervetskyi
Hi there, I faced with issue in adding file to distributed cache in Flink. My setup: - Java 1.8 - Flink 1.8 - OS: Windows, Linux Test scenario: 1. Create simple stream environment 2. Add to distributed cache local file 3. Add simple source

Re: Restore state class not found exception in 1.8

2019-05-28 Thread Lasse Nedergaard
Hi Gordon Thanks for the reply. No we haven’t moved it around namespaces. The only thing we have done is to add a new attribute to the object in another branch of our code and it could be we by mistake has used it but it should still not give a class not found exception. We have the save

[ANNOUNCE] Munich meetup: "Let's talk about "Stream Processing with Apache Flink"

2019-05-28 Thread Fabian Hueske
Hi folks, Next Tuesday (June, 4th), Vasia and I will be speaking at a meetup in Munich about Flink and how we wrote our book "Stream Processing with Apache Flink". We will also raffle a few copies of the book. Please RSVP if you'd like to attend: ->

Re: Distributed cache fault

2019-05-28 Thread Till Rohrmann
Hi Vasyl, please post these kind of question to Flink's user ML since the dev ML is used for development discussions. For the failure on Windows could you share the complete stack trace to see where exactly it fails? It looks as if on Windows the scheme part of the URI makes problems. Looking

Re: [SURVEY] Usage of flink-ml and [DISCUSS] Delete flink-ml

2019-05-28 Thread Till Rohrmann
+1 for removing it. I think it is effectively dead by now. Cheers, Till On Mon, May 27, 2019 at 4:00 PM Hequn Cheng wrote: > Hi Shaoxuan, > > Thanks a lot for driving this. +1 to remove the module. > > The git log of this module shows that it has been inactive for a long > time. I think it's

Flink 1.8: Job manager redirection not happening in High Availability mode

2019-05-28 Thread Kumar Bolar, Harshith
Hi all, Prior to upgrading to 1.8, there was one active job manager and when I try to access the inactive job manager's web UI, the page used to get redirected to the active job manager. But now there is no redirection happening from the inactive JM to active JM. Did something change to the

Re: Re: How can I add config file as classpath in taskmgr node when submitting a flink job?

2019-05-28 Thread wangl...@geekplus.com.cn
Thanks. Let me have a try wangl...@geekplus.com.cn From: Yang Wang Date: 2019-05-28 09:47 To: wangl...@geekplus.com.cn CC: user Subject: Re: How can I add config file as classpath in taskmgr node when submitting a flink job? Hi, wangleiYou could use the flink distributed cache to register

Re: Restore state class not found exception in 1.8

2019-05-28 Thread Tzu-Li (Gordon) Tai
Hi Lasse, Did you move the class to a different namespace / package or changed to be a nested class, across the Flink versions? That would be the only cause I could reason about at the moment. If possible, could you also have a very minimal snippet / instructions on how I can maybe reproduce

RE: Flink????Event Time????????????????????

2019-05-28 Thread Shi Quan
??Flink??CEP??notFollowedBy??within?? Session

Flink????Event Time????????????????????

2019-05-28 Thread 690963922
??Flink Committer Flink

Re: Blink在Hive表没有统计信息的情况下如何优化

2019-05-28 Thread Kurt Young
你先试试把HashJoin这个算子禁用看看,TableConfig里添加这个配置 sql.exec.disabled-operators: HashJoin Best, Kurt On Tue, May 28, 2019 at 3:23 PM bigdatayunzhongyan < bigdatayunzhong...@aliyun.com> wrote: > 感谢 @Kurt Young 大神的回复,报错信息在附件。谢谢! > > > 在2019年05月28日 14:10,Kurt Young 写道: > >

回复: Blink在Hive表没有统计信息的情况下如何优化

2019-05-28 Thread bigdatayunzhongyan
org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException: Fatal error at remote task manager '/xx:14941'. at org.apache.flink.runtime.io.network.netty.CreditBasedPartitionRequestClientHandler.decodeMsg(CreditBasedPartitionRequestClientHandler.java:276) at