Re: [E] RE: Recommendations on using multithreading in flink map functions in java

Thanks Ron and Thias, I understand flink parallelism works at task level. Distribute n subtasks of the operator across your cluster and parallel process the elements by distributing them across multiple instances of your operator. This gives me very high throughput. What I want to know is how to

[ANNOUNCE] Apache flink-connector-mongodb 1.0.2 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-mongodb 1.0.2. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is available for

[ANNOUNCE] Apache Flink Kubernetes Operator 1.6.0 released

The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.6.0. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Release highlights: - Improved rollback

[ANNOUNCE] Apache Flink Kubernetes Operator 1.6.0 released

The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.6.0. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Release highlights: - Improved rollback

Re: 404 Jar File Not Found w/ Web Submit Disabled

Hi, Patricia Maybe you should check the code carefully, a reply code of 404 typically indicates that an error occured in client side.Alternatively,could you share a snippet code where the error occured? This would greatly assist us in addressing your issue effectively. Best Jiadong Lu On

Re: [Issue] Repeatedly receiving same message from Kafka

Hi there It would be helpful if you could include the code for your pipeline. One suggestion, can you disable the "EXACTLY_ONCE" semantics on the producer. Using EXACTLY_ONCE will leverage Kafka transactions and thus add overhead. I would disable it to see if you still get the same situation.

[Issue] Repeatedly receiving same message from Kafka

Sorry, I've forgot putting title, so sending again. 2023년 8월 15일 (화) 오후 6:27, Dennis Jung 님이 작성: > (this is issue from Flink 1.14) > > Hello, > > I've set up following logic to consume messages from kafka, and produce > them to another kafka broker. For producer, I've configured >

[no subject]

(this is issue from Flink 1.14) Hello, I've set up following logic to consume messages from kafka, and produce them to another kafka broker. For producer, I've configured `Semantics.EXACTLY_ONCE` to send messages exactly once. (also setup 'StreamExecutionEnvironment::enableCheckpointing' as

Re: Question about serialization of java.util classes

Check this answer: https://stackoverflow.com/a/64721838/5793905 You could then use, for example, something like: new SetTypeInfo(Types.STRING) instead of Types.LIST(Types.STRING) Am Di., 15. Aug. 2023 um 10:40 Uhr schrieb : > Hello Alexis, > > Thank you for sharing the helper classes this but

Re: Question about serialization of java.util classes

Hello Alexis, Thank you for sharing the helper classes this but unfortunately I have no idea how to use these classes or how they might be able to help me. This is all very new to me and I honestly can't wrap my head around Flink's type information system. Best regards, Saleh. > On 14 Aug

RE: Recommendations on using multithreading in flink map functions in java

Hi Vignesh, In addition to what Ron has said, there are a number of options to consider, depending on the nature of your calculations: Given that your main focus seems to be latency: * As Ron has said, Flink manages parallelism in a coarse grained way that is optimized for spending as