Hi Kishore,
Having followed this thread for a while it is still quite a bit of confusion of
concepts and in order to help resolve your original we would need to know,
* what makes your observation a problem to be solved?
* You write, you have no shuffling, does that mean you don’t use a
Hi Weijie,
In our case we do have 7 operators. All the 7 operators are getting executed as
one chain within a single StreamTask. As checkpoint barrier is passing through
all the operators, there are 7 checkpoints being stored. So our checkpoint size
is up by 7 times. We are investigating to see
Hi Talat,
In Flink, a checkpoint barrier will be injected from source, and then pass
through all operators in turn. Each stateful operator will do checkpoint in
this process, the state is managed at operator granularity, not operator
chain. So what is the significance of checkpoint based on the gr
Thank Martin for managing the release and all the people involved.
Best regards,
Weijie
Konstantin Knauf 于2023年2月2日周四 06:40写道：
> Great. Thanks, Martijn for managing the release.
>
> Am Mi., 1. Feb. 2023 um 20:26 Uhr schrieb Martijn Visser <
> martijnvis...@apache.org>:
>
> > The Apache Flink
Hey everyone,
I'm wondering if anyone has done any experiments trying to use non-temporal
watermarks? For example, a dataset may contain some kind of virtual
timestamp / version field that behaves just like a regular timestamp
(monotonically increasing, etc.), but has a different scale / range.
A
Great. Thanks, Martijn for managing the release.
Am Mi., 1. Feb. 2023 um 20:26 Uhr schrieb Martijn Visser <
martijnvis...@apache.org>:
> The Apache Flink community is very happy to announce the release of Apache
> Flink 1.16.1, which is the first bugfix release for the Apache Flink 1.16
> series.
The Apache Flink community is very happy to announce the release of Apache
Flink 1.16.1, which is the first bugfix release for the Apache Flink 1.16
series.
Apache Flink® is an open-source stream processing framework for
distributed, high-performing, always-available, and accurate data streaming
a
Hi Weijie,
Thanks for replying back.
Our job is a streaming job. The OperatorChain contains all operators that
are executed as one chain within a single StreamTask. But each
operator creates their own checkpoint at checkpointing time . Rather than
creating a checkpoint per operator in checkpoint
Hi Jan,
Thanks for the reply, I was able to submit the job to flink but it's
failing due to an OOM issue so I am moving to the GKE. I got the flink UI
there but submitted a job not appearing on flink UI. I am using the same
script which I shared with you.. Do I need to make some changes for Google
As I mentioned in the previous email, standalone mode is not on the
Autoscaler roadmap because the scheduling/resource model is different.
This applies to both standalone app and session clusters.
Thanks
Gyula
On Wed, Feb 1, 2023 at 4:48 PM Swathi Chandrashekar
wrote:
> Sure, thanks Gyula.
> Is
Sure, thanks Gyula.
Is there a roadmap to support standalone session clusters to scale based on the
jobs added/deleted and change in parallelism ?
Regards,
Swathi C
From: Gyula Fóra
Sent: Wednesday, February 1, 2023 8:54 PM
To: Swathi Chandrashekar
Cc: user@fli
The autoscaler currently only works with Native App clusters.
Native session clusters may be supported in the future but standalone is
not on our roadmap due to a different resource/scheduling model used.
Gyula
On Wed, Feb 1, 2023 at 4:22 PM Swathi Chandrashekar
wrote:
> Hi,
>
> I'm was testing
Hi,
I'm was testing OSS flink operator with flink 1.17 for autoscale feature. I was
able to scale the cluster based on load in application cluster in native mode,
but the same did not work in standalone mode as the operator gave the following
error as below [ both for app and session mode ].
I
Hi Marco,
I think you may need to do heap-dump for TM to check whether there is
memory leak.
Best regards,
Weijie
Hi Talat,
Can you elaborate on what it means to create one checkpoint object per
chain operator more than all operators? If you mean to do checkpoint
independently for each task, this is not supported.
Best regards,
Weijie
Talat Uyarer via user 于2023年2月1日周三 15:34写道：
> Hi,
>
> We have a job
15 matches
Mail list logo
