Execute PyFlink jobs: flink-python module seems to be missing or not working

Hi, When I execute a PyFlink-Job locally in a mini cluster, then everything works fine: (env) user@bla /cygdrive/c/flink1_13_3/examples/python/table/batch $ python word_count.py Results directory: C:\cygwin64\tmp/result But when I try to execute the PyFlink-Job in a remote cluster, the job exe

Re: Flink SQL does not honor "table.exec.source.idle-timeout"

Hi Caizhi, Thanks so much for responding. Our autowatermarkinterval is set to 2000 or 2 seconds. When I upgrade my project to Flink 1.13.2 the setting works but it does not work in Flink 1.12.1 or 1.12.0. I don't see anything in release notes that might have fixed this but we were able to verif

Re: OrcTableSource in flink 1.12

Hello, I am still looking into that same issue and I am not sure how to continue forward. We want to upgrade to the latest versions of flink (1.14) and I couldn't find any examples on how to properly do that. Does anybody help me on translating the example from the docs: https://ci.apache.org/pr

Re: Execute PyFlink jobs: flink-python module seems to be missing or not working

Hi Schmid, Just as you have seen that the jar package of flink-python is located in the directory $FLINK_HOME/opt. It seems that this jar doesn't exist in your environment. Could you double check that? Regards, Dian On Thu, Oct 28, 2021 at 3:35 PM Schmid Christian wrote: > Hi, > > > > When I e

NoClassDefFoundError for RMIServerImpl when running Flink with Scala 2.12 and Java 11

Hi Flink developers, I encountered a weird error like follows. It only happens when I build Flink with Scala 2.12 profile and run with Java 11. No such error for Scala 2.11 + Java 8/11. Just search for relative info, but don't find any useful. Do you have any idea about this? Thanks. java.lan

AW: Execute PyFlink jobs: flink-python module seems to be missing or not working

Hi Dian Thank you for your reply. I could solve this issue by setting an additional environment variable FLINK_OPT_DIR = "C:\flink1_13_3\opt" in Windows. Now it works fine. Regards, Christian Von: Dian Fu Gesendet: Donnerstag, 28. Oktober 2021 10:21 An: Schmid Christian Cc: user@flink.apache

Re: Why we need again kubernetes flink operator?

Thanks Yang for the clarification. Thanks for the great passion of this community, for the kind of features and responses over mails . That's the reason flink will evolve and will grow further in the future. Regards Bhaskar On Mon, Oct 25, 2021 at 4:42 PM Yang Wang wrote: > Hi Bhaskar, > > IIUC

Some question with flink rabbitmq connector?

Hi: recently,we want to use flink rabbitmq connector RMQSink,but we found that the RMQSink not use the message confirm mechanism,waitForConfirm() function or add Confirm listener,why?,I want to know the connector is there any garantee for message not loss.

Re: Some question with flink rabbitmq connector?

Hi, You are right the current RMQSink does not give any delivery guarantee. As far as I know there are also no ongoing efforts to improve the situation but we are always open for contributions if you would like to work on that. We are currently trying move all Sinks to the new unified Sink inte

Re: Execute PyFlink jobs: flink-python module seems to be missing or not working

Great! On Thu, Oct 28, 2021 at 5:38 PM Schmid Christian wrote: > Hi Dian > > > > Thank you for your reply. > > I could solve this issue by setting an additional environment variable > *FLINK_OPT_DIR* = "C:\flink1_13_3\opt" in Windows. > > Now it works fine. > > > > Regards, > > Christian > > > >

window join in flink sql

Hi, Flink users How to express multiple stream window join in flink sql? in datastream api, that's stream.join(otherStream) .where() .equalTo() .window() .apply() ( https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/operators/joining/ ) For example, in fl

Kafka Source Recovery Behavior

Hi all, I noticed that the KafkaSourceReader did not have a pointer to the KafkaSubscriber, so I was wondering if this could be a bug: 1. User has a flink job with topic set A and takes savepoint 2. User modifies flink job to read from topic set B; however, splits are still read from topic set A

Flink connector sink for DynamoDB (Java8)

Hi folks, Is there any package for writing from flink stream to DynamoDB? I checked: https://github.com/klarna-incubator/flink-connector-dynamodb but, it supports Java11 only. My Java runtime is Java8. Thanks, Jing

Re: Flink connector sink for DynamoDB (Java8)

Hi, I am not aware of any at the moment. There is an open Flink ticket for it [1] which I believe is targeted for Flink 1.15 Best regards, Martijn [1] https://issues.apache.org/jira/browse/FLINK-24229 Op do 28 okt. 2021 om 20:41 schreef Jing Lu > Hi folks, > > Is there any package for writin

Converting a Table to a DataStream[RowData] instead of DataStream[Row] with `toDataStream`

Flink 1.14 Scala 2.12.5 Hi, I want to be able to convert a Table into a DataStream[RowData]. I need to do this since I already have specific infrastructure in place that knows how to serialize RowData into a JSON format. Previously when using toAppendStream[RowData], this worked fine. However, no

Re: RE: Duplicate Calls to Cep Filter

Hi Yun Gao, Thank you for the immediate response. You are correct that any state in nfa will have 2 options.. either to TAKE(element) and undergo transition to another state Or it can IGNORE and remain in its current state. But if a state decides to ignore the element , then why will it evaluate i

S3 Source support in Flink

Hello, I see S3 supported as a Sink through StreamingFileSink but do not see a source equivalent StreamingFileSource *Questions:* 1. What is the current recommendation for using S3 as a continuous

Re: database as stream source issue

Hi, Qihua JDBC connector support Postgres dailect, but it is implemented as a bounded source which means it only captures the snapshot data(the existed records) and then finished its work, the new adding transaction log records (as known as MySql bin-log) won’t be captured. You should receive a

Re: window join in flink sql

Hi! Window join in Flink SQL is supported since Flink 1.14, see document [1]. [1] https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/queries/window-join/ Lu Niu 于2021年10月29日周五 上午2:12写道： > Hi, Flink users > > How to express multiple stream window join in flink sql? in

Re: window join in flink sql

Tips: The documentation in https://ci.apache.org/projects/flink is no longer updated any more, The new documentation site is https://nightlies.apache.org/flink/ , please use the new one. Best, Leonard > 在 2021年10月

Re: S3 Source support in Flink

Hi Abhishek, You can use `readFileStream` directly defined on DataStream. You will still have to pay the ListObjects for each iteration using that method. If you want a source that does not rely on listing, you can implement a custom SQS source (there is no official existing one currently) and use