Hi all,
I've been using Flink for quite some time now and for a university project
I'm planning to experiment with statefun. During the walkthrough I've run
into some issues, I hope you can help me with.
1) Is it correct that the Docker image of statefun is not yet published? I
couldn't find it a
Hi Wouter!
Glad to read that you are using Flink for quite some time, and also
exploring with StateFun!
1) yes it is correct and you can follow the Dockerhub contribution PR at [1]
2) I’m not sure I understand what do you mean by trigger from the browser.
If you mean, for testing / illustration
Hi Igal,
Thanks for your quick reply. Getting back to point 2, I was wondering if
you could trigger indeed a stateful function directly from Flask and also
get the reply there instead of using Kafka in between. We want to
experiment running stateful functions behind a front-end (which should be
ab
Hi,
Correct me if I'm wrong, but from the discussion so far it seems like what
Wouter is looking for is an HTTP-based ingress / egress.
We have been thinking about this in the past. The specifics of the
implementation is still to be discussed, but to be able to ensure
exactly-once processing sema
Hi Igal, all,
In the meantime we found a way to serve Flink stateful functions in a
frontend. We decided to add another (set of) Flask application(s) which
link to Kafka topics. These Kafka topics then serve as ingress and egress
for the statefun cluster. However, we're wondering how we can scale
Hi Wouter,
Triggering a stateful function from a frontend indeed requires an ingress
between them, so the way you've approached this is also the way we were
thinking of.
As Gordon mentioned a potential improvement might be an HTTP ingress, that
would allow triggering stateful functions directly fr
Dear Igal, all,
Thanks a lot. This is very helpful. I understand the architecture a bit
more now. We can just scale the stateful functions and put a load balancer
in front and Flink will contact them. The only part of the scaling I don't
understand yet is how to scale the 'Flink side'. So If I und
Hi,
I'm glad things are getting clearer, looking forward to seeing how statefun
is working out for you :-)
To change the parallelism you can simply set the "parallelism.default" [1]
key in flink-conf.yaml.
It is located in the statefun container at /opt/flink/conf/flink-conf.yaml.
To avoid rebuild
