factory), is that correct?
>
> Thank you again,
> Federico
>
> Il giorno gio 24 feb 2022 alle ore 15:11 Igal Shilman
> ha scritto:
>
>> Hello,
>>
>> For (1) I welcome you to visit our documentions, and many talks online to
>> understand more about the m
Hello Xin Li,
Indeed the built in ingress that ships with StateFun requires that the key
part will be a utf-8 string, This string then becomes the id part of the
target address.
StateFun is extensible via the StatefulFunctionModule[1] and customizing
the Kafka ingress is also possible, take a loo
ly thing I could think
> of is to not include com.google.protobuf in the
> classloader.parent-first-patterns.additional setting, and
> including protobuf-java 3.7.1 in the uber jar.
>
> I created a jira for this just now + a discuss thread on the dev group
> https://issues.apache
Hello,
For (1) I welcome you to visit our documentions, and many talks online to
understand more about the motivation and the value of StateFun. I can say
in a nutshell that StateFun provides few building blocks that makes
building distributed stateful applications easier.
For (2) checkout our pl
Hello,
The default kafka ingress for remote functions, does require a key
component. The key is being translated to the 'id' part of the
receiving function address.
If your functions are stateless, or the id doesn't have a meaning for you,
you can simply provide a random id.
I hope that helps,
Iga
Hello,
I've noticed that you've linked to a very old release of stateful function
(2.0) where statefun 3.2 which is the latest, added support for exactly
that.
You are no longer required to use Protobuf, and you can simply send strings
and even JSON.
Checkout the following repository for some examp
Hello,
You can take a look at the flink stateful functions playground[1] where you
can find a handful of examples, in all the supported
SDKs, in addition for each language you will find a walkthrough that shows
how to use the individual SDK features.
Furthermore take a look at the documentations [
Hello,
Make sure that you have added the state when creating the function spec
like in this example[1]
If that wasn't it, can you send your UserFn?
[1]
https://github.com/apache/flink-statefun-playground/blob/release-3.2/java/greeter/src/main/java/org/apache/flink/statefun/playground/java/greeter/
Hi Christopher,
Great to hear you've solved it, and thanks for sharing your findings with
the community!
Indeed RocksDB is a separate component that has to be added as a
dependency.
On Wed, Feb 9, 2022 at 3:55 PM Christopher Gustafson wrote:
> Solved it, and posting here in case anyone run into
Hello Christopher,
It seems to be like a version mismatch, which StateFun version are
you using, and what is the Flink version of the cluster that you are trying
to submit to?
StateFun 3.1.1 was built with Flink 1.13.5
StateFun 3.2.0 was built with Flink 1.14.3
The version needs to match, since
Hello Christopher,
The most common deployment of StateFun applications is via the community
provided Docker images[1] (and their derivations) . This image captures the
optimal deployment of Flink for StateFun.
In addition, there is also an example of how to deploy these images to
k8s[2].
If you ar
ndertow in our
> case) needs to authenticate and authorise statefun based on the latter's
> cert.
>
> Many thanks
> Fil
>
> On Mon, 24 Jan 2022 at 21:25, Igal Shilman wrote:
>
>> Hello Filip,
>>
>> As far as I know SslContextBuilder.forClient() should
Hello Filip,
As far as I know SslContextBuilder.forClient() should use the default trust
store, so if you will install your self signed certificate in the community
supported container image this should be picked up[1].
The following user has reported something similar, and it seems that
they've g
Hello Fil,
There is a PR opened today for upgrading the main branch to use the latest
Flink version, and after its merged we'll kick the 3.2 release.
Cheers,
Igal
On Mon 17. Jan 2022 at 13:53, Dawid Wysakowicz
wrote:
> I am pretty confident the goal is to be able to run on the newest Flink
> ve
Hi Deniz,
I assume that you are using the Python SDK,
Here is an example how to set the egress topic name[1]
I hope that helps,
Igal.
[1]
https://github.com/apache/flink-statefun-playground/blob/release-3.1/python/greeter/functions.py#L58
On Wed 19. Jan 2022 at 09:53, Deniz Koçak wrote:
> Hi,
Hello Galen,
StateFun is using some internal APIs so they might or might not stay
compatible between versions. You can try bump the version
If it compiles cleanly, most likely this would work.
We will be porting the main branch to Flink 1.14 this or next week.
Cheers,
Igal.
On Mon, Jan 10, 2022 a
The Apache Flink community is very happy to announce the release of Apache
Flink Stateful Functions (StateFun) 3.1.1.
This is a bugfix release that addresses the recent log4j vulnerabilities,
users are encouraged to upgrade.
StateFun is a cross-platform stack for building Stateful Serverless
appl
ment, we are planning to keep this custom build in
> our own repo, but it would be very handy to see that change in an
> official release.
>
> Thanks,
> Deniz
>
> On Thu, Dec 9, 2021 at 2:39 PM Igal Shilman wrote:
> >
> > Hello Deniz,
> >
> > Lookin
; > > ===
> > >
> > > Therefore, it's easier to use single jar in our pipelines instead of
> > > creating a different jar file for each env. (at least for development
> > > and production).
> > >
> > > For solution 2, you refer fli
Hello,
Glad to hear that you've successfully deployed a remote function with
StateFun :-)
It's not clear to me if the only way to deploy a custom egress is to use
> and embedded module because documentation states:
Indeed currently the only way to define a custom egress is by writing an
embedde
Hi Deniz,
My apologies for the late reply, I assume that by now you have figured this
out since I've seen your followup question :-)
StateFun uses the trust store configured in the JVM, so if you can install
your certificate there, StateFun should transparently pick it up.
Good luck,
Igal.
On Fr
Hi Deniz,
StateFun would be looking for module.yaml(s) in the classpath.
If you are submitting the job to an existing Flink cluster this really
means that it needs to be either:
1. packaged with the jar (like you are already doing)
2. be present at the classpath, this means that you can place your
Hello Mingmin,
Your described scenario and use case indeed seems like a good fit for
StateFun.
Also your analysis is correct, StateFun is executed as a very specific
Flink application (this is what you call "state engine"), and remote
functions are effectively a stateless service.
* when a host i
Hi Fil,
The default Kafka ingress that ships with StateFun indeed doesn't bundle
the headers with the incoming message, so there is no way of getting them
at the moment, without doing some work :(
I'd be also happy to kick off the discussion (I guess JIRA would be the
right place) about supporting
Hello Hady,
Glad to see that you are testing StateFun!
Regarding that exception, I think that this is not the root cause. The root
cause is as you wrote that the StateFun job failed because it wasn't able
to deliver a message to a remote function in the given time frame.
If you look at the logs yo
ay to achieve this
>> 'sequencial-per-key' behaviour with the use of embedded functions? Those
>> seem to be rock-solid. Maybe there are some internal classes that would at
>> least provide a template on how to do it? I have a naive implementation
>> ready (the one I
Hello Fil,
Indeed what you are describing is exactly what a remote function does.
I am curious to learn more about the current performance limitations that
you encounter with the remote functions.
One thing to try in combination with the async transport, is to increase
the total number of in fig
Forgot to include the user mailing list in my previous email.
On Fri, Oct 15, 2021 at 12:27 PM Igal Shilman wrote:
> Hello,
>
> Is there a specific reason you are using the 2.x branch? This is quite old
> and most importantly it is not compatible with the 3.x branch.
> If you are
Hi Mark,
For communicating with remote functions we use the default trust manager,
so I believe that if you add the self signed certificate to
the container the JVM will pick it up automatically.
I haven't done it myself, but I've found this blog post that explains how
to do it [1]
Let me know if
Hello Christian,
The challenge with generic back pressure and remote functions, is that
StateFun doesn't know if it targets a single process or a fleet of
processes behind a load balancer and an autoscaler.
Triggering back pressure too early might never kick in the autoscaling.
Indeed that parame
Hello Le,
Currently the only way to execute a Python function with StateFun is
through a remote function.
This means that you need to host the function separately. [1]
Good luck!
Igal
[1]
https://nightlies.apache.org/flink/flink-statefun-docs-master/docs/modules/http-endpoint/
On Thu, Oct 7, 20
t,
>
> Christian
>
> -Ursprüngliche Nachricht-
> Von: Roman Khachatryan
> Gesendet: Dienstag, 28. September 2021 00:31
> An: Christian Krudewig (Corporate Development) <
> christian.krude...@dpdhl.com>; Igal Shilman
> Cc: user@flink.apache.org
> Betreff
ter.
> I was only able to get the code running through IntelliJ by following the
> steps below. Once I reverted the hack and changed the config on our
> cluster, everything worked perfectly.
> Sorry for the confusion and thanks for all your help.
> Barry
>
> On 2021/09/23 10:57:36,
istribution dependency
> were shipped with the required value in it by default?
> Thanks a million,
> Barry
>
> On 2021/09/10 12:37:32, Igal Shilman wrote:
> > Hello Barry,
> >
> > I assume that by "we don't need another installation of Flink to manage
>
Hello Barry,
I assume that by "we don't need another installation of Flink to manage the
stateful functions." You mean that you already have a running Flink cluster
and you would like to submit an additional Flink Job that executes a
Stateful functions application?
Then perhaps just try to submit
Hi Barry,
I've forward your email to the user mailing list as it is more suitable
here :-)
Your question definitely makes sense, and let me try to provide you with
some pointers:
1. The architecture that you've outlined has many advantages and is
desirable if you can
afford that. Some of them are
The Apache Flink community is very happy to announce the release of Apache
Flink Stateful Functions (StateFun) 3.1.0.
StateFun is a cross-platform stack for building Stateful Serverless
applications, making it radically simpler to develop scalable, consistent,
and elastic distributed applications.
Hi Deniz,
Sorry for the late reply!
Let me first say that if your use case is a simple enrichment/side-lookup
then perhaps it is simpler to use Flink's Async operator that
calls into your HTTP service. StateFun comes with a programming model, an
SDK, and few limitations (for example only supports
Not yet unfortunately,
But I'd be very much happy to work with the community on a JS SDK.
On Tue, Jul 20, 2021 at 4:32 PM Omid Bakhshandeh
wrote:
> Igal,
>
> Thanks for the answers. Is there any JS SDK available?
>
> Best,
> --Omid
>
> On Tue, Jul 20, 2021 at
Hi Omid,
I'm glad to hear that you are evaluating StateFun in your company! let me
try to answer your questions:
1. In version 2.x, StateFun only supported messages of type
com.google.protobuf.Any, and we had a tiny optimization that
reads type hints and unpacked the real message out of the Any m
-docs-release-1.13/docs/dev/datastream/fault-tolerance/state/#cleanup-during-rocksdb-compaction
On Wed, Jul 14, 2021 at 1:59 PM Igal Shilman wrote:
> Hi Ammon,
>
> The duration is per item, and the cleanup happens transparently and
> incrementally via RocksDB (background compactions w
Hello Jessy,
Currently StateFun Kafka ingress interprets the keys of the record as the
destination address.
So you'd have to attach a key to use that specific ingress.
If this is not an option for you, you can consider @Tim's suggestion or
create a JIRA with a feature request,
which we will be ha
ing option is to attach a
>>> debugger [1]. I'd probably add a breakpoint to
>>> LegacySourceFunctionThread#run and see what happens. If the issue is in
>>> recovery, you should add a breakpoint to StreamTask#beforeInvoke.
>>>
>>> [1]
>>> http
Hi Tim,
Any additional logs from before are highly appreciated, this would help us
to trace this issue.
By the way, do you see something in the JobManager's UI?
On Fri, May 28, 2021 at 9:06 AM Tzu-Li (Gordon) Tai
wrote:
> Hi Timothy,
>
> It would indeed be hard to figure this out without any sta
ines to understand when the dependency is mandatory
> and how to solve conflicts as such in those cases?
>
> Best regards
>
> Dario
> On 5/25/21 12:04 PM, Igal Shilman wrote:
>
> Do you have the statefun-sdk-java as a dependency, if you are not using it
> can you remove it?
it should be fine.
Good luck,
Igal.
On Tue, May 25, 2021 at 12:04 PM Igal Shilman wrote:
> Do you have the statefun-sdk-java as a dependency, if you are not using it
> can you remove it?
>
> On Tue, May 25, 2021 at 11:28 AM Bonino Dario
> wrote:
>
>> Dear Igal,
>>
d this leads to the reported error.
>
> Any idea on how to overcome this issue?
>
> Cheers,
>
> Dario
> On 5/25/21 11:22 AM, Igal Shilman wrote:
>
> Hi Bonino,
>
> If you've also included the statefun-sdk-java into the class path then,
> make sure that yo
Hi Bonino,
If you've also included the statefun-sdk-java into the class path then,
make sure that you are not using accidentally the shaded version of a
TypedValue (check the package name)
Let me know if that helped, and we'll try to debug this together otherwise.
Cheers,
Igal.
On Mon, May 24,
timeouts [1]
[1]
https://ci.apache.org/projects/flink/flink-statefun-docs-release-3.0/docs/deployment/module/#timeouts
On Fri, May 21, 2021 at 10:53 AM Igal Shilman wrote:
> Hi Jan,
>
> I haven't stumbled upon this but I will try to reconstruct that scenario
> with a stress tes
Hi Jan,
I haven't stumbled upon this but I will try to reconstruct that scenario
with a stress test and report back.
Can you share a little bit about your environment. For example do you use
gunicorn, ngnix, aiohttp/or flask perhaps?
I'd suggest maybe checking for request size limits parameters
namesIngress)
> .withRequestReplyRemoteFunction(
> RequestReplyFunctionBuilder.requestReplyFunctionBuilder(
> REMOTE_GREET, URI.create(
> "http://localhost:5000/statefun"; <http://localhost:5000/statefun>))
> .wit
Hello!
Your analysis is correct, indeed what is passed is whatever is being handed
to withMessageBody(..).
Starting with StateFun 3.0, if you need to send a message to a remote
function the message needs to be a TypedValue.
You can create an instance of TypedValue manually, or you can add a
depen
Hello Cliff,
You are right, indeed defining custom metrics is not supported at the
moment.
I will file a JIRA issue so we can track this, and we will try to
prioritize this feature up.
Meanwhile, there are a lot of metrics that StateFun defines, like
invocations rates etc' perhaps you can find it
s://issues.apache.org/jira/browse/FLINK-22389
>> * Support for OpenTracing:
>> https://issues.apache.org/jira/browse/FLINK-22390
>> * Support for State Introspection:
>> https://issues.apache.org/jira/browse/FLINK-22391
>>
>> Please feel free to edit, comment on these iss
e:
> "base64bytes"}
> {"type": "DeleteState", "namespace": "foo", "type": "bar", "id": "1"}
>
> Also having opentracing integration where Statefun passes b3 headers with
> each request so we can tr
Hi Tim!
Indeed the StateFun SDK / StateFun runtime, has an internal concept of
batching, that kicks in the presence of a slow
/congested remote function. Keep in mind that under normal circumstances
batching does not happen (effectively a batch of size 1 will be sent). [1]
This batch is not curren
question not addressed:
>
> Can a stateful function be called by a process function?
>
> On Wed, Apr 7, 2021 at 8:19 AM Igal Shilman wrote:
>
>> Hello Marco!
>>
>> Your understanding is correct, but in addition
>> You can also use StateFun within
Hello Marco!
Your understanding is correct, but in addition
You can also use StateFun within a DataStream application [1]
[1]
https://ci.apache.org/projects/flink/flink-statefun-docs-master/docs/sdk/flink-datastream/
On Wed, Apr 7, 2021 at 2:49 AM Marco Villalobos
wrote:
> Upon reading about s
for this information, Piotr.
>
> The comment from Igal Shilman in FLINK-18894 issue says, "Obtaining a
> MAX_PRIO mailbox from StreamTask, solves this issue." I'm unclear what this
> means -- is this a workaround I can leverage?
>
> Dylan
> ---
Hi Miguel,
I think that there are a couple of ways to achieve this, and it really
depends on your specific use case, and the trade-offs
that you are willing to accept.
For example, one way to approach this:
- Suppose you have an external service somewhere that returns a
representation of the logi
Hello Stephan,
The values are loaded lazily, upon access. And this applies to both
PersistedTable and PersistedValue.
For a PersistedTable in particular, calling get/set/remove will only affect
the specific key, and calling entries() will fetch the entire map.
Cheers,
Igal.
On Sun, Feb 14, 202
Hello,
I believe that your assembly plugin configuration doesn't merge files under
META-INF/services.
Can you unzip your jar and examin manually the contents of:
META-INF/services/org.apache.flink.statefun.flink.io.spi.FlinkIoModule
It should include at least the following lines:
org.apache.flink
Hi Stephan,
I think that what you are trying to achieve is very interesting, and
possibly other users might find that useful as well
and we will definitely add that to our roadmap.
I think that Gordon's suggestion of using the state processor API to
examine a savepoint, makes a lot of sense in th
Hello Martijn,
Great to hear that you are exploring StateFun as part of your university
project!
Can you please clarify:
- how do you measure throughput?
- by co-located functions, do you mean a remote function on the same
machine?
- Can you share a little bit more about your functions, what are
Hi Jan,
StateFun enables object reuse automatically, and it can't be disabled with
a configuration.
There is a technical reason for that that has to do with how we translate
StateFun concepts to Flink concepts.
I've created an issue to remove this limitation [1].
I might come up with a workaround
Hi Frédérique!
Thank you for your kind words! let me try to answer your questions:
>From the email thread, it looks like there’s going to be support for
> dynamic function dispatch by name patterns which is pretty cool, but it
> sounds like you still need to redeploy if you add a new ingress or e
Adding user@flink (was accidentally omitted previously)
On Fri, Jan 29, 2021 at 5:18 PM Igal Shilman wrote:
> Hi Jan,
>
> Glad to hear that 3.71 and 3.3.0 works for you. You can still include
> protobuf in your project, but the version needs to be compatible with
> what you will
Hi Martijn,
I'm glad you like it! and we are always happy to learn about new use cases
:)
* How is the feedback stream exactly merged with the input stream from
> ingresses?
>
First, I'd like to refer you to this talk, that has a peek under the hood
part[1] (it starts at ~26min) where I outline
k-core/src/test/resources/module-v3_0/module.yaml#L29
On Tue, Jan 12, 2021 at 7:16 PM Ahmad Alkilani wrote:
> That's awesome, thank you! JIRA I can follow?
>
> On Tue, Jan 12, 2021 at 9:01 AM Igal Shilman wrote:
>
>> Hello!
>>
>> Is it possible to dynamically
Hello!
Is it possible to dynamically associate Remote Modules with Remote Function
> Endpoints to an existing/already running Flink stateful application?
>
Yes, in the upcoming StateFun release we are introducing exactly that :)
As of the upcoming version, we are adding a capability to dynamicall
orImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
> ... 8 more
>
>
> Thanks for the help!
>
> Le
>
>
> On Mon, Dec 28, 2020 at 6:24 AM Igal Shilman wrot
Hi Le,
Let me try to answer to your multiple questions, one by one:
> I'm trying to understand the internal mechanism used by Flink Statefun to
> dispatch functions to Flink cluster. In particular, I was trying to find a
> good example demonstrating Statefun's "Logical Co-location, Physical
> Sep
sendDownstream(FeedbackUnionOperator.java:186)
>>> at
>>> org.apache.flink.statefun.flink.core.feedback.FeedbackUnionOperator.processElement(FeedbackUnionOperator.java:86)
>>> at
>>> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emit
https://gist.github.com/flint-stone/059f00832d8b99af433a446771f4f740#file-pom-xml-L64
>
> Thanks!
>
> Le
>
> On Sat, Dec 26, 2020 at 2:23 PM Igal Shilman wrote:
>
>> Hello :-)
>>
>> It seems like in your attached pom you are not bundling the dependencies.
>> Check out the docs h
Hello :-)
It seems like in your attached pom you are not bundling the dependencies.
Check out the docs here [1].
[1]
https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.2/deployment-and-operations/packaging.html#flink-jar
On Wed, Dec 23, 2020 at 3:07 AM Le Xu wrote:
> Hello!
>
Hi Le,
You can attach many different functions in a single StateFun builder, and
let them message each other.
In your example, you can make the "Greet" function message Greet2 directly
(in addition to emitting a message as an egress).
Embedding multiple copies of StateFun within a Datastream appli
Hello Jan,
The next release candidate for stateful functions is expected in mid
February, and the release itself would follow shortly thereafter.
This release will be based on Flink 1.12.
Cheers,
Igal.
On Tue, Dec 15, 2020 at 4:19 PM Jan Brusch
wrote:
> Hi,
>
> just a quick question: Is there
Glad to hear that you were able to resolve the issue!
One comment tho: I would really encourage you to upgrade to statefun 2.2.1,
which was released recently and it fixes a checkpointing related issue.
Kind regards,
Igal.
On Tue, Nov 24, 2020 at 10:10 PM Lian Jiang wrote:
> Probolved solved. I
Hi George,
Specifically for StateFun, we have the following Helm charts [1] to help
you deploy Stateful Functions on k8s.
The greeter example's docker-compose file also includes Kafka (and hence
Zookeeper).
Indeed the Flink cluster is "included" in the master/worker stateful
functions docker image
Glad to hear that it worked out!
On Wed, Nov 11, 2020 at 9:07 AM Lian Jiang wrote:
> Just realized making autoservice class discoverable also solved "There are
> no routers defined" mentioned by Puneet. Yes, harness does test statefun
> module discovery. Thanks.
>
> On Tue, Nov 10, 2020 at 9:57
Hi Lian,
If you are using the statefun-sdk directly (an embedded mode) then, most
likely is that you are missing a
META-INF/services/org.apache.flink.statefun.sdk.spi.StatefulFunctionModule
file that would point to your modules class. We are using Java SPI [1] to
load all the stateful functions mo
>
> Could you give a quick opinion if these fears are even true and if so,
> what would be a recommended setup to satisfy these use cases?
>
>
> Best regards
>
> Jan
>
>
> On 05.11.20 17:02, Igal Shilman wrote:
>
> Hi Jan,
>
> The architecture outlined
Hi Jan,
The architecture outlined by you, sounds good and we've run successfully
mixed architectures like this.
Let me try to address your questions:
1)
To enable checkpointing you need to set the relevant values in your
flink-conf.yaml file.
execution.checkpointing.interval: (see [1])
state.che
Hi Lian,
Good to hear that you are learning about StateFun, and I'd be happy to
answer any of your questions while doing so :-)
Perhaps in the future it would be best if you start a new email thread, so
that it would be easier to spot your question.
The following is completely thread safe:
final
Hi Dan,
I'm assuming that you have different Kafka topics, and each topic contains
messages of a single protobuf type.
In that case, you have to specify the mapping between a topic name to it's
Protobuf message type.
To do that, assume that you have a Kafka topic *A* that contains protobuf
messag
Hi Mazen,
What are the differences between Flink stateful functions and Event driven
> microservices are they almost the same concept
>
You can think of Stateful Functions as an API and a runtime that helps
building event driven microservices. It addresses some of the hardest parts
of composing s
fig I
> can set in the "module.yaml" to have it just pack arbitrary bytes into the
> Any?
>
> Thanks,
>
> Tim
>
> On Wed, Sep 23, 2020 at 7:01 AM Igal Shilman wrote:
>
>> Hi Tim,
>>
>> You are correct, currently the argument to a remote funct
> infrastructure. With that said, gRPC is also part of our requirements, thus
> motivation for the question.
>
>
>
> I’d love to hear more about plans to implement support for gRPC and
> perhaps become an early adopter.
>
>
>
> I hope this helps with understanding o
Hi John,
Thank you for sharing your interesting use case!
Let me start from your second question:
> Are stateful functions available to all Flink jobs within a cluster?
Yes, the remote functions are some logic exposed behind an HTTP endpoint,
and Flink would forward any message addressed to th
Hi Tim,
You are correct, currently the argument to a remote function must be a
Protobuf Any, however StateFun doesn't interpret the contents of that Any,
and it would be passed as-is to the remote function.
As you mentioned in your email you can interpret the bytes as the bytes of
a JSON string.
Hi,
Your observation is correct, currently the only way to invoke a remote
function is trough an HTTP POST request to a service that exposes a
StateFun endpoint.
The endpoint must implement the client side of a the “RequestReply”
protocol as defined by StateFun (basically an invocation contains t
@Galen FYI: the upcoming StateFun release would use Scala2.12
On Thu, Sep 10, 2020 at 5:14 PM Seth Wiesman wrote:
> @glen
>
> Yes, we would absolutely migrate statefun. StateFun can be compiled with
> Scala 2.12 today, I'm not sure why it's not cross released.
>
> @aljoscha :)
>
> @mathieu Its
Hi Dan, let me try to answer your questions:
> I guess my question is if one can
> freely mix Flink core with SF's code with regards to performance,
> fault-tolerance, and checkpointing?
The main limitations at the moment is that, currently SF requires a
processing time watermark semantics only
nt = 1
> else:
> state.count += 1
> context.state('count').pack(state)
>
>
> envelope = Any()
> envelope.Pack(state)
> context.reply(envelope)
>
>
>
> module.yaml
&g
Hi Jan,
Stateful functions would look at the java class path for the module.yaml,
So one way would be including the module.yaml in your src/main/resources/
directory.
Good luck,
Igal.
On Mon, Jul 6, 2020 at 12:39 PM Jan Brusch
wrote:
> Hi,
>
> quick question about Deploying a Flink Stateful F
Hi Jan,
Judging by the exception message it seems like the function type
"demo/eventCounterPython" is not known to stateful functions.
This could happen if the module.yaml (provided in your email) was
accidentally excluded from the resulting artifact (Docker image or a
jar-with-dependencies)
Can yo
Hi Francesco,
It is absolutely possible to deploy some functions as embedded and some as
remote, and scale them independently, while technically being part of the
same
stateful function application instance (I think that what you meant by
"sharing the same master").
One possible way to do it in k
Hi,
Can you verify that your jar contains the following file
META-INF/services/org.apache.flink.statefun.sdk.spi.StatefulFunctionModule ?
Thanks,
Igal.
On Tue, May 26, 2020 at 11:49 AM C DINESH wrote:
> Hi Gordon,
>
> Thanks for your response.
>
> After adding this conf to flink-yml.
>
> `class
Hi Dinesh,
They are different projects that aim to solve different problems.
The streaming ledger goal is to bring serializable ACID Transactions to
streaming pipelines, while statefun's aim is to simplify building
distributed stateful applications in the serverless model.
To get more information
parallelism, but I
> couldn't really find that option yet. Is there some value I need to set in
> the module.yaml.
>
> Once again, thanks for the help so far. It has been useful.
>
> Regards,
> Wouter
>
> Op wo 13 mei 2020 om 00:03 schreef Igal Shilman :
>
&
1 - 100 of 118 matches
Mail list logo
