Re: Savepoint a failing job

gt;> [1] >> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#allowing-non-restored-state >> [2] >> https://nightlies.apache.org/flink/flink-docs-master/docs/libs/state_processor_api/ >> [3] >> https://nightlies.apache.org/flink/

Re: Savepoint a failing job

e/concepts/overview/#state-management > for more details on this topic. > > Best regards, > > Martijn > > On Fri, Dec 16, 2022 at 12:34 AM Timothy Bess wrote: > >> Hi there, >> >> We have a pyflink/SQL job that has a bug that we fixed and are trying to >

Savepoint a failing job

Hi there, We have a pyflink/SQL job that has a bug that we fixed and are trying to deploy. Here's the issue though. The job successfully restores from the checkpoint, but has no recent savepoints. We can't seem to get it to accept our new SQL unless we savepoint/restore, but we can't trigger a sav

Flink SQL JSON

Hi Flink Users, I use Flink SQL to ETL Kafka event data into tables we use for analysis, and I'm currently trying to use it to basically extract a few fields out of some JSON, but leave the original blob in a subdocument using the elasticsearch connector. I see that there is `ROW` and `MAP` but I

Re: Stateful Function Ingress issues

Hi Jessy, I had this issue as well, here's the resolution . I ended up forking the version of statefun I used and removing the null check to default to empty string, but I'm goin

Re: Statefun 2.2.2 Checkpoint restore NPE

-2.2/statefun-kafka-io/src/main/java/org/apache/flink/statefun/sdk/kafka/KafkaIngressBuilder.java > [4] https://github.com/igalshilman/custom-ingress > > > > > > On Fri, May 28, 2021 at 8:19 PM Timothy Bess wrote: > >> Ok so after digging into it a bit it seems that the exc

Re: Statefun 2.2.2 Checkpoint restore NPE

ng to keep from having to edit statefun and use my own jar, any thoughts? Thanks, Tim On Fri, May 28, 2021 at 10:33 AM Timothy Bess wrote: > Oh wow that Harness looks cool, I'll have to take a look at that. > > Unfortunately the JobManager UI seems to just show this: > [imag

Re: Statefun 2.2.2 Checkpoint restore NPE

y stack traces. >>> >>> Have you tried changing to debug level logs? Maybe you can also try >>> using the StateFun Harness to restore and run your job in the IDE - in that >>> case you should be able to see which code exactly is throwing this >>> exception.

Statefun 2.2.2 Checkpoint restore NPE

Hi, Just checking to see if anyone has experienced this error. Might just be a Flink thing that's irrelevant to statefun, but my job keeps failing over and over with this message: 2021-05-28 03:51:13,001 INFO org.apache.flink.streaming.connectors.kafka. FlinkKafkaProducer [] - Starting FlinkKafka

Re: Flink Statefun Python Batch

community can benefit >> a lot out of this, >> and definitely contributions are welcome! >> >> @Konstantin Knauf would you like to understand more >> in depth, Tim's use case with opentracing? >> >> Thanks, >> Igal. >> >> >> >

Re: Flink Statefun Python Batch

> @functions.bind_tim(..) > def ml(context, messages: typing.List[Message]): > ... > > > > Let me know what you think, > Igal. > > > > [1] > https://github.com/apache/flink-statefun/blob/master/statefun-sdk-protos/src/main/protobuf/sdk/request-reply.proto#L8

Flink Statefun Python Batch

Hi everyone, Is there a good way to access the batch of leads that Statefun sends to the Python SDK rather than processing events one by one? We're trying to run our data scientist's machine learning model through the SDK, but the code is very slow when we do single events and we don't get many of

Re: Flink Statefun TTL

briefing of this > change, as there is a change in the remote invocation protocol and will > require existing SDKs to be updated in order to work with StateFun 3.0. > > Cheers, > Gordon > > On Wed, Feb 24, 2021 at 11:00 PM Timothy Bess wrote: > >> He

Flink Statefun TTL

Hey, I noticed that the Flink Statefun 2.1.0 release notes had this snippet with regards to TTL: Note: The state expiration mode for remote functions is currently > restricted to AFTER_READ_AND_WRITE, and the actual TTL being set is the > longest duration across all registered state, not for each

Statefun delayed message

Hi everyone, I have a question about how delayed messages work, I tried to dig through some docs on it, but not sure it addresses exactly my question. Basically, if I send a delayed message with exactly-once mode on, does Flink need to wait until the delayed message sends to commit Kafka offsets?

Re: Flink Statefun Byte Ingress

can interpret the bytes as the bytes of > a JSON string. > > I hope it helps, > Igal. > > On Wed, Sep 23, 2020 at 5:06 AM Timothy Bess wrote: > >> Hi, >> >> So most of the examples of "module.yaml" files I've seen focus on >> protobuf ing

Flink Statefun Byte Ingress

Hi, So most of the examples of "module.yaml" files I've seen focus on protobuf ingress, but is there a way to just get bytes from Kafka? I want to integrate this with the rest of my codebase which uses JSON, but don't want to migrate to protobuf just yet. I'm not totally sure how it would work sin

Flink Stateful Functions API

The flink stateful function Python API looks cool, but is there a documented spec for how it communicates with Flink? I'd like to implement an SDK in Haskell if I can.