Re:Flink 1.18 with Java 17 production version release

[Xuyang]

Hi, Meng.
I think you can follow this jira[1] and ping the creator about the latest 
progress.


[1] https://issues.apache.org/jira/browse/FLINK-34491



--

Best！
Xuyang




At 2024-03-13 04:02:09, "Meng, Ping via user"  wrote:

Hi,

The latest Flink 1.18.1 with Java 17 support is in beta mode, users can report 
issue, is there a planned release date for production version? Do you have a 
roadmap for production version?

 

Thank you,

Angela Meng

 

Re: Flink Batch Execution Mode

[irakli.keshel...@sony.com]

Hi Feng,

I'm using flink-connector-kafka 3.0.1-1.17. I see that 1.17 is affected, but 
the ticket is marked as fixed so I'm not sure if that is actually the issue.

Best,
Irakli

From: Feng Jin 
Sent: 12 March 2024 18:28
To: Keshelava, Irakli 
Cc: user@flink.apache.org 
Subject: Re: Flink Batch Execution Mode

Hi Irakli

What version of flink-connector-kafka are you using?
You may have encountered a bug [1] in the old version that prevents the source 
task from entering the finished state.


[1]. 
https://issues.apache.org/jira/browse/FLINK-31319

Best,
Feng


On Tue, Mar 12, 2024 at 7:21 PM 
irakli.keshel...@sony.com 
mailto:irakli.keshel...@sony.com>> wrote:
Hello,

I have a Flink job that is running in the Batch mode. The source for the job is 
a Kafka topic which has limited number of events. I can see that the job starts 
running fine and consumes the events, but never makes it past the first task 
and becomes idle. The Kafka source is defined to be bounded by following 
command: "KafkaSource.builder().setBounded(OffsetsInitializer.latest())".
I expect the job to consume all the events that are in the Kafka topic and then 
move to the next task, but I'm not sure if the "OffsetsInitializer.latest()" is 
the right OffsetInitializer. Can anyone help me out here? Thanks!

Cheers,
Irakli

FlinkSource to read iceberg table in Batch mode

[Chetas Joshi]

Hello,

I am using iceberg-flink-runtime lib (1.17-1.4.0) and running the following
code to read an iceberg table in BATCH mode.

var source = FlinkSource
.forRowData()
.streaming(false)
.env(execEnv)
.tableLoader(tableLoader)
.limit((long) operation.getLimit())
.filters(buildFilterExpression(operation))
.build();

var stream = source.map(rowDataMapper).name(operation.getName());

I am running into the following exception even though streaming = false.

Detected an UNBOUNDED source with the 'execution.runtime-mode' set to
'BATCH'. This combination is not allowed, please set the
'execution.runtime-mode' to STREAMING or AUTOMATIC

Would appreciate any pointers here.

Thank you
Chetas

Question around manually setting Flink jobId

[Allison Chang via user]

Hi,

I was wondering if there is any way to manually set the jobID for the jobGraph. 
I noticed that there is a configuration for PIPELINE_FIXED_JOB_ID, but there 
doesn't seem to be a way to set it via config with the 
StreamingJobGraphGenerator.java. Would appreciate any assistance if anyone has 
done something similar.

Best,

Allison Chang

Re: Question around manually setting Flink jobId

[Junrui Lee]

Hi Allison,

The PIPELINE_FIXED_JOB_ID configuration option is not intended for public
use. IIUC, the only way to manually specify the jobId is submitting a job
through the JAR RUN REST API, where you can provide the jobId in the
request body (
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jars-jarid-run
).

Best,
Junrui

Allison Chang via user  于2024年3月14日周四 08:16写道：

> Hi,
>
> I was wondering if there is any way to manually set the jobID for the
> jobGraph. I noticed that there is a configuration for
> PIPELINE_FIXED_JOB_ID, but there doesn't seem to be a way to set it via
> config with the StreamingJobGraphGenerator.java. Would appreciate any
> assistance if anyone has done something similar.
>
> Best,
>
> *Allison Chang*
>
>
>