Re: How to handle deletion of items using PyFlink SQL?

Hi John, If you are using Table API & SQL, the framework is handling the RowKind and it's transparent for you. So usually you don't need to handle RowKind in Table API & SQL. Regards, Dian On Thu, Jun 9, 2022 at 6:56 AM John Tipper wrote: > Hi Xuyang, > > Thank you very much, I’ll experiment t

Re: How to handle deletion of items using PyFlink SQL?

Hi Xuyang, Thank you very much, I’ll experiment tomorrow. Do you happen to know whether there is a Python example of udtf() with a RowKind being set (or whether it’s supported)? Many thanks, John Sent from my iPhone On 8 Jun 2022, at 16:41, Xuyang wrote:  Hi, John. What about use udtf [1]

Re: Flink Operator 1.0.0 not working

Seems like something is off with your CRD. You could try replacing it using: kubectl replace -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.0/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml After that you should be able to deploy the

Re: Apache Flink - Rest API for num of records in/out

Hi Andreas: Thanks for your pointers and reference. I will check it out. On Tuesday, June 7, 2022, 05:17:43 PM EDT, Hailu, Andreas wrote: Hi M,   We had a similar requirement – we were able to solve for this by: 1.  Supply the operators we’re interested in acquiring metrics

Recover watermark from savepoint

Hi, I want to understand if flink saves a watermark during savepoint and if not, how do we achieve this? We are seeing an issue where on recovery, the job processes some late events which should have been discarded if the job were to be running without any downtime. Thank you, Sweta

Re:How to handle deletion of items using PyFlink SQL?

Hi, John. What about use udtf [1]? In your UDTF, all resources are saved as a set or map as s1. When t=2 arrives, the new resources as s2 will be collected by crawl. I think what you want is the deletion data that means 's1' - 's2'. So just use loop to find out the deletion data and send RowData

Re: SourceFunction

Hi Alexey, There is a thread[1] discussing this issue right now. It would be great if you could share some thoughts about your experience. Thanks! Best regards, Jing [1]https://lists.apache.org/thread/d6cwqw9b3105wcpdkwq7rr4s7x4ywqr9 On Wed, Jun 8, 2022 at 4:42 PM Alexey Trenikhun wrote: > He

Flink Remote Stateful - Crontab Scheduler

Team, We are aware about Sending Delayed Messages which we can use to trigger delayed stateful functions. Example Use Case Invoke a Remote Stateful Function every day @ 11 AM We are

SourceFunction

Hello, Is there plan to deprecate SourceFunction in favor of Source API? We have custom SourceFunction based source, do we need to plan to rewrite it using new Source API ? Thanks, Alexey

Flink Operator 1.0.0 not working

Greetings all, I am trying to get the flink operator (https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.0/) working, however running into a number of issues. I have a fresh Kubernetes cluster running and have followed all the instructions for deploying the operator

Kafka Consumer commit error

Hi, we have some issues with a job using the flink-sql-connector-kafka (flink 1.15.0/standalone cluster). If one broker e.g. is restarted for maintainance (replication-factor=2), the taskmanagers executing the job are constantly logging errors on each checkpoint creation: Failed to commit cons

Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.delegation.ParserFactory' in the classpath.

Have you ever unzip your project jar and make sure the class HiveParserFactory exist? Best regards, Yuxia 发件人: "顾斌杰" 收件人: luoyu...@alumni.sjtu.edu.cn 抄送: "User" 发送时间: 星期三, 2022年 6 月 08日 下午 5:11:33 主题: Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.deleg

How to handle deletion of items using PyFlink SQL?

Hi all, I have some reference data that is periodically emitted by a crawler mechanism into an upstream Kinesis data stream, where those rows are used to populate a sink table (and where I am using Flink 1.13 PyFlink SQL within AWS Kinesis Data Analytics). What is the best pattern to handle de

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 1.0.0 released

Congrats! Thanks Yang for driving the release, and thanks to all contributors! Best, Lijie John Gerassimou 于2022年6月6日周一 22:38写道： > Thank you for all your efforts! > > Thanks > John > > On Sun, Jun 5, 2022 at 10:33 PM Aitozi wrote: > >> Thanks Yang and Nice to see it happen. >> >> Best, >> Aito

Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.delegation.ParserFactory' in the classpath.

can refer to this： https://stackoverflow.com/questions/52500048/flink-could-not-find-a-suitable-table-factory-for-org-apache-flink-table-facto On 6/8/2022 16:04，yuxia wrote： Have you ever put the flink-sql-connector-hive into you FLINK_HOME/lib? And make sure your JM/TM also contains th

Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.delegation.ParserFactory' in the classpath.

Have you ever put the flink-sql-connector-hive into you FLINK_HOME/lib? And make sure your JM/TM also contains the jar. Best regards, Yuxia 发件人: "顾斌杰" 收件人: "User" 发送时间: 星期三, 2022年 6 月 08日 下午 3:19:19 主题: Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.del

Re: Could not find a suitable table factory for 'org.apache.flink.table.planner.delegation.ParserFactory' in the classpath.

The following is part of the code ： String createKafkaSql = "create table if not exists x" + "(x\n" + ",update_time timestamp(3) comment '11'\n" + ",watermark for update_time as update_time - interval '20' second)\n" +

Could not find a suitable table factory for 'org.apache.flink.table.planner.delegation.ParserFactory' in the classpath.

Flink version: 1.13 When executed in the local environment (windows), there is no exception. When starting the project with flink web ui, I get the following error: Server Response: org.apache.flink.runtime.rest.handler.RestHandlerException: Could not execute application. at org.apache.fl