Re: Flink Kubernetes Session sample from Documentation

her dns service is normal. > > Best, > Weihua > > > On Wed, Apr 5, 2023 at 12:43 PM Le Xu wrote: > >> Hello! >> >> I'm trying out the Kubernetes sample >> <https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/resource-providers/native_k

Flink Kubernetes Session sample from Documentation

Hello! I'm trying out the Kubernetes sample described in the official doc but I am not able to submit job with the following error:

Savepoint state keeps growing

Hello! I'm trying to do a savepoint and reconfigure my window aggregation jobs overtime and I noticed that the windowed results keep accumulating with time (I'm using heap backend). I modified the snapshot code and noticed that Flink does not drop my aggregated window states after the window has

Job hanging taking savepoint on legacy Flink

Hello! I would like to run a legacy flink project on top of old-version Flink (1.4.1) and I'm getting error when trying to cancel a job with savepoint. Specifically, it reports the following error on requestBuffer: My understanding would be that the save point operation probably requires all

Slot sharing behavior in Flink cluster

Hello! I have a quick question about slot-sharing behavior specified at this link ( https://nightlies.apache.org/flink/flink-docs-master/docs/concepts/flink-architecture/). My understanding would be each task slot in Flink cluster represents a sequentially running operator (and therefore can be

Customizing backpressure mechanism for RichParallelSourceFunction

Hello! I have a dataflow pipeline built using Flink's RichParallelSourceFunction as parallel sources. I'm wondering if there are any mechanisms that I could use to implement *ack-based* back-pressure mechanism by sending ACK messages from operators (within the runtime) to source functions to

[Statefun] Unable to locate the launcher jar

Hello! I was trying to run the python greeter example from the statefun example o my own flink cluster (where I use to deploy regular flink jobs) and I follow all setup steps listed here

Deploying python statefun program on standalone Flink cluster

Hello! I was wondering if there is an example to deploy Python statefun program directly on standalone cluster (or if this is supported at all). Right now I found an early write up here saying

Re: Converting Table API query to Datastream API

s-release-1.13/zh/docs/dev/table/sql/explain/ > [2] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/execution/execution_plans/ > > Best regards, > JING ZHANG > > Le Xu 于2021年7月1日周四 上午5:51写道： > >> Thanks -- Is there a way to quickly visualiz

Re: Converting Table API query to Datastream API

apache-flink> > is > a bit outdated. Since Flink 1.9 version, TableAPI & SQL is no longer > translated to DataStream API. TableAPI & SQL and DataStream are at the same > level, and both translated into StreamOperator DAG. > > Best regards, > JING ZHANG > > Le Xu

Converting Table API query to Datastream API

Hello! I have a basic question about the concept of using Flink Table API. Based on the link here it seems like if I implement stream query with Table API the program is translated to

Re: Support for sending generic class

who might be able to help you with your question. >> >> Looking at the interface Context, it looks that you cannot easily specify >> a TypeHint for the message you want to send. Hence, I guess that you >> explicitly need to register these types. >> >> Cheers, >> Til

Support for sending generic class

Hello! I'm trying to figure out whether Flink Statefun supports sending object with class that has generic parameter types (and potentially nested types). For example, I send a message that looks like this: context.send(SINK_EVENT, idString, new Tuple3<>(someLongObject, listOfLongObject, Long));

[Statefun] Exception occurs during function chaining / Async function

Hello! I'm getting an exception running a modified version of datastream/statefun example. (See exception details that follow.) The example was adapted from the original datastream example provided in statefun repo. I was trying to play with the example by chaining two functions (with the 1st

Re: Flink Stateful Function: The program's entry point class not found in the jar file

link-datastream-example/pom.xml#L161 > > You need to change it to your Example class, this should work. > > > On Tue, Dec 29, 2020 at 5:06 AM Le Xu wrote: > >> Hi Igal: >> >> Thanks for pointing that out. I was able to add the dependency in and >> su

Visualizing Flink Statefun's "Logical Co-location, Physical Separation" Properties

Hello! I'm trying to understand the internal mechanism used by Flink Statefun to dispatch functions to Flink cluster. In particular, I was trying to find a good example demonstrating Statefun's "Logical Co-location, Physical Separation" properties (as pointed out by [1]). My understanding based

Re: Flink Stateful Function: The program's entry point class not found in the jar file

l > [2] https://maven.apache.org/plugins/maven-shade-plugin/ > [3] > https://github.com/apache/flink-statefun/blob/master/statefun-flink/statefun-flink-distribution/pom.xml#L126,L199 > > On Sat, Dec 26, 2020 at 11:52 PM Le Xu wrote: > >> Thanks Igal! I might be missing so

Re: Compiling Error for Flink 1.11.3

Hi Chesnay: Thanks -- it works perfectly now. Le On Mon, Dec 28, 2020 at 9:00 AM Chesnay Schepler wrote: > Please delete the > flink-formats/flink-parquet/src/test/java/org/apache/flink/formats/parquet/generated > > directory, and then recompile the module. > > On 12/28/2

Compiling Error for Flink 1.11.3

Hello! I was trying to compile flink 1.11.3 from github (branch release-1.11.3) but I'm getting the following error saying that it cannot find symbol (adding full trace at the end of the email). Here is my output for mvn -v -- I'm using maven 3.2.5. Apache Maven 3.2.5

Re: Chaining DataStream API Functions results in collision in Flink Stateful Function

Hello! I seem to wire the wrong function by attaching the first function's output to the remote reply function of the second function. The process works great now. Thanks! Le On Sat, Dec 26, 2020 at 11:23 PM Le Xu wrote: > I apologize -- I meant to point out line 152, where context.s

Re: Chaining DataStream API Functions results in collision in Flink Stateful Function

I apologize -- I meant to point out line 152, where context.send was used. On Sat, Dec 26, 2020 at 11:21 PM Le Xu wrote: > Hi Igal: > > Thanks for the suggestion -- I changed the implementation based on your > suggestion by attaching the second function right after the fir

Re: Chaining DataStream API Functions results in collision in Flink Stateful Function

Embedding multiple copies of StateFun within a Datastream application is > currently not supported. > > Thanks, > Igal. > > > On Sat, Dec 26, 2020 at 6:22 AM Le Xu wrote: > >> Hello! >> >> I'm trying to modify the DataStream API example >> <https

Re: Flink Stateful Function: The program's entry point class not found in the jar file

cts/flink/flink-statefun-docs-release-2.2/deployment-and-operations/packaging.html#flink-jar > > > On Wed, Dec 23, 2020 at 3:07 AM Le Xu wrote: > >> Hello! >> I'm trying to deploy a Flink stateful function as a flink jar and I >> followed the instruction here <https://

Chaining DataStream API Functions results in collision in Flink Stateful Function

Hello! I'm trying to modify the DataStream API example provided by Flink Stateful

Flink Stateful Function: The program's entry point class not found in the jar file

Hello! I'm trying to deploy a Flink stateful function as a flink jar and I followed the instruction here . However I'm getting the error saying that the program entry point class was not found in the jar even after I added the dependency in my pom.

Re: Task slot sharing: force reallocation

eers, > Till > > On Wed, Mar 6, 2019 at 8:13 PM Le Xu wrote: > >> 1.3.2 -- should I update to the latest version? >> >> Thanks, >> >> Le >> >> On Wed, Mar 6, 2019 at 4:24 AM Till Rohrmann >> wrote: >> >>> Which version of Flin

Could not resolve ResourceManager address on Flink 1.7.1

Hello: I am trying to set up a standalone flink cluster (1.7.1) and I'm getting a very similar error as the user reported in this thread. However, I believe

Re: Task slot sharing: force reallocation

1.3.2 -- should I update to the latest version? Thanks, Le On Wed, Mar 6, 2019 at 4:24 AM Till Rohrmann wrote: > Which version of Flink are you using? > > On Tue, Mar 5, 2019 at 10:58 PM Le Xu wrote: > >> Hi Till: >> >> Thanks for the reply. The setup of

Re: Task slot sharing: force reallocation

task slots are registered before slot request, the code have a >> tendency to group requests together on the same machine because we >> are using a LinkedHashMap >> >> ? >> >> Piotrek >> >> On 4 Mar 2019, at 21:08, Le Xu wrote: >> >> Thanks

Re: Task slot sharing: force reallocation

> containers for Task Managers are created/requested randomly. > > Piotrek > > On 2 Mar 2019, at 23:53, Le Xu wrote: > > Hello! > > I'm trying to find out if there a way to force task slot sharing within a > job. The example on the website looks like the following (as

Re: Local combiner on each mapper in Flink

:) > > Best regards, > > Kien > > > On 10/26/2017 8:01 PM, Le Xu wrote: >> Hi Kien: >> >> Is there a similar API for DataStream as well? >> >> Thanks! >> >> Le >> >> >>> On Oct 26, 2017, at 7:58 AM, Kien Truo

Re: Local combiner on each mapper in Flink

@gmail.com > <mailto:ykt...@gmail.com>>: > Do you mean you want to keep the origin window as well as doing some combine > operations inside window in the same time? > What kind of data do you expect the following operator will receive? > > Best, > Kurt > > On

Re: Local combiner on each mapper in Flink

, 2017 at 4:14 AM, Kurt Young <ykt...@gmail.com> wrote: > I think you can use WindowedStream.aggreate > > Best, > Kurt > > On Tue, Oct 24, 2017 at 1:45 PM, Le Xu <sharonx...@gmail.com> wrote: > >> Thanks Kurt. Maybe I wasn't clear before, I was wonderi

Re: Local combiner on each mapper in Flink

t is pretty old, you can check the newest > version here: https://ci.apache.org/projects/flink/flink-docs- > release-1.3/dev/batch/dataset_transformations.html > > Regarding to your question, you can use combineGroup > > Best, > Kurt > > On Mon, Oct 23, 2017 at 5:22 AM, Le X

Local combiner on each mapper in Flink

Hello! I'm new to Flink and I'm wondering if there is a explicit local combiner to each mapper so I can use to perform a local reduce on each mapper? I looked up on https://ci.apache.org/projects/flink/flink-docs-release-0.8/dataset_transformations.html but couldn't find anything that matches.

Balancing operation across machines

Hello! Just wondering if there is a way to load-balancing all executors on one operator across the machine. I have a reduce operator that has less parallelism level than the total number of slots in the cluster. Currently flink tries to compact executors in an operator and schedule them to some

Flink SocketTextStream source scheduled to a single machine

Hello! I'm trying to figure out how it happens: I'm having a program reading from multiple socketTextStream and these text streams feed into different data flow (and these data streams never connect in my job). It looks something similar to below: for(int i =0; i< hosts.length; i++) {