Difference between Session Mode and Session Job(Flink Opearator)

2022-07-06 Thread bat man
Hi, I want to understand the difference between session mode and the new deployment mode - Flink Session Job which I believe is newly introduced as part of the Flink Operator(1.15) release. What's the benefit of using this mode as opposed to session mode as both run sessions to which flink jobs ca

Re: Is Flink able to read a CSV file or just like in Blink this function does not work?

2022-07-06 Thread Alexander Fedulov
Hi Mike, I do not see any issues with your code. With a sample csv file like this a,1.0 b,2.0 c,3.0 it produces the expected result +--+ | some_value | +--+ |3 | +--+ 1 row in set Process finished with exi

Is Flink able to read a CSV file or just like in Blink this function does not work?

2022-07-06 Thread podunk
If I'm reading Flink manul correctly (and this is not simple - no examples), this code should read CSV file:     package flinkTest2; import org.apache.flink.table.api.EnvironmentSettings; import org.apache.flink.table.api.TableEnvironment;   public class flinkTest2 {     public static voi

Re: Restoring a job from a savepoint

2022-07-06 Thread Alexander Fedulov
Hi John, use *$ bin/flink run -s s3://my_bucket/path/to/savepoints/ *(no trailing slash, including schema). where should contain a valid _metadata file. You should see logs like this: *INFO o.a.f.r.c.CheckpointCoordinator [] - Starting job foobar from savepoint s3://my_bucket/path/to/savepoi

Re: Restoring a job from a savepoint

2022-07-06 Thread Yaroslav Tkachenko
Hi John, I've been using a path like this: s3:savepoint- (no trailing slash). I'm pretty sure you need to specify the full path. Yes, you can see savepoint restore in logs. It's also fairly easy to see it in the Flink UI, under the Checkpoints section (it shows the information about the lates

Restoring a job from a savepoint

2022-07-06 Thread John Tipper
Hi all, The docs on restoring a job from a savepoint (https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#resuming-from-savepoints) state that the syntax is: $ bin/flink run -s :savepointPath [:runArgs] and where "you may give a path to either the savepoint’s dire

Re: How to mock new DataSource/Sink

2022-07-06 Thread David Jost
> On 5. Jul 2022, at 01:48, Alexander Fedulov wrote: > > Hi David, > > I started working on FLIP-238 exactly with the concerns you've mentioned in > mind. It is currently in development, feel free to join the discussion [1]. > If you need something ASAP and are not interested in rate-limiting

RE: how to connect to the flink-state store and use it as cache to serve APIs.

2022-07-06 Thread Schwalbe Matthias
Hi Laxmi, Did you consider Apache Flink Table Store [1] which was introduced short time ago. Yours sounds like a case for early integration … Sincere greetings Thias [1] https://nightlies.apache.org/flink/flink-table-store-docs-release-0.1/ From: laxmi narayan Sent: Wednesday, July 6, 2022 6