Re: Force to commit kafka offset when stop a job.

[Zhanghao Chen]

Hi, you could stop the job with a final savepoint [1]. Flink which will trigger 
a final offset commit on the final savepoint.

[1] 
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/cli/#stopping-a-job-gracefully-creating-a-final-savepoint

Best,
Zhanghao Chen

From: Lei Wang 
Sent: Wednesday, June 5, 2024 22:47
To: user 
Subject: Force to commit kafka offset when stop a job.


When stopping a flink job that consuming kafka message, how to force it to 
commit kafka offset

Thanks,
Lei

Re: Force to commit kafka offset when stop a job.

[Hang Ruan]

Hi Lei.

I think you could try to use `stop with savepoint` to stop the job.
The offset will be committed when the checkpoint finished. So I think `stop
with savepoint` may be helpful.

Best,
Hang

Lei Wang  于2024年6月6日周四 01:16写道：

>
> When stopping a flink job that consuming kafka message, how to force it to
> commit kafka offset
>
> Thanks,
> Lei
>

Re: Flink job Deployement problem

[Hang Ruan]

Hi, Fokou Toukam.

This error occurs when the schema in the sink mismatches the schema you
provided from the upstream.
You may need to check whether the provided type of field `features` in sink
is the same as the type in the provided upstream.

Best,
Hang

Fokou Toukam, Thierry  于2024年6月6日周四
10:22写道：

> Hi, i'm trying to deploy flink job but i have this error. How to solve it
> please?
>
> *Thierry FOKOU *| * IT M.A.Sc  Student*
>
> Département de génie logiciel et TI
>
> École de technologie supérieure  |  Université du Québec
>
> 1100, rue Notre-Dame Ouest
>
> Montréal (Québec)  H3C 1K3
>
>
> [image: image001] 
>

Re: Flink job Deployement problem

[Xiqian YU]

Hi Fokou,

Seems `features` column was inferenced to be RAW type, which doesn’t carry any 
specific data information, and causes following type casting failed.

Sometimes it will happen when Flink can’t infer return type from a lambda 
expression but no explicit returning type information was provided[1], and 
could be solved by inserting an extraneous .returns() 
call.[2]

If that doesn’t solve your problem, could you please share more information 
like a code snippet or a minimum POC?

Regards,
yux

[1] 
https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/java_lambdas/
[2] 
https://stackoverflow.com/questions/70295970/flink-table-get-type-information


De : Fokou Toukam, Thierry 
Date : jeudi, 6 juin 2024 à 09:04
À : user 
Objet : Flink job Deployement problem
Hi, i'm trying to deploy flink job but i have this error. How to solve it 
please?
[cid:b13357ec-00b9-4a15-8d04-1c797a4eced3]

Thierry FOKOU |  IT M.A.Sc Student

Département de génie logiciel et TI

École de technologie supérieure  |  Université du Québec

1100, rue Notre-Dame Ouest

Montréal (Québec)  H3C 1K3

[image001]

Re:Flink job Deployement problem

[Xuyang]

Hi, 

Could you provide more details about it, such as a minimum reproducible sql? 




--

Best！
Xuyang




在 2024-06-06 09:03:16，"Fokou Toukam, Thierry" 
 写道：

Hi, i'm trying to deploy flink job but i have this error. How to solve it 
please?
|

Thierry FOKOU |  IT M.A.Sc Student

Département de génie logiciel et TI

École de technologie supérieure  |  Université du Québec

1100, rue Notre-Dame Ouest

Montréal (Québec)  H3C 1K3




|

|

Flink job Deployement problem

[Fokou Toukam, Thierry]

Hi, i'm trying to deploy flink job but i have this error. How to solve it 
please?
[cid:b13357ec-00b9-4a15-8d04-1c797a4eced3]

Thierry FOKOU |  IT M.A.Sc Student

Département de génie logiciel et TI

École de technologie supérieure  |  Université du Québec

1100, rue Notre-Dame Ouest

Montréal (Québec)  H3C 1K3


[image001]

Force to commit kafka offset when stop a job.

[Lei Wang]

When stopping a flink job that consuming kafka message, how to force it to
commit kafka offset

Thanks,
Lei