Hi team,Currently, I was facing the OutOfMemoryError: Java heap space. This was some how due to the fact that I was storing the state on FileSystem. With the FsStateBackend, the working state for each task manager is in memory (on the JVM heap), and state backups (checkpoints) go to a distributed
感谢感谢大佬指点
| |
应聘者昌呈呈
|
|
ccc0606fight...@163.com
|
回复的原邮件
| 发件人 | Xuyang |
| 发送日期 | 2022年9月6日 00:03 |
| 收件人 | |
| 主题 | Re:flink table API使用 |
Hi, 可以类似这样写 “.filter($("a").isGreater(10)) "。 更多的使用方法可以参考[1]
[1]
退订
Sent from my iPhone
Hi!
The operator supports both Flink native and standalone deployment modes and
in both cases the JM is deployed as k8s Deployment.
During upgrade Flink/operator deletes the deployment after savepoint and
waits for termination before it creates a new one with the updated spec.
Cheers,
Gyula
On
>
> I then noticed this message showing up twice and thought "this does not
> look right":
That's fine, this is how the sink works (see the comment here:
KafkaWriter.java#L294-L301
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#state-backend-rocksdb-localdir
Make sure to use a local SSD disk (not NFS/EBS).
Best,
Alexander Fedulov
On Mon, Sep 5, 2022 at 7:24 PM hjw <1010445...@qq.com> wrote:
> The EmbeddedRocksDBStateBackend holds
The EmbeddedRocksDBStateBackend holds in-flight data in
aRocksDBdatabase that is (per default) stored in the TaskManager
local data directories.
Which path does local data directories store RocksDB database in
TaskManagerpoint to in operating system?
If the job state is very large, I think I
Hi,Matthias
I have solved this problem as you say.The link to the PR [1] .thank you.
[1]https://github.com/apache/flink/pull/20671
Best,
Hjw
----
??:
Hi, 可以类似这样写 “.filter($("a").isGreater(10)) "。 更多的使用方法可以参考[1]
[1]
https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/java/org/apache/flink/table/examples/java/basics/GettingStartedExample.java
--
Best！
Xuyang
在 2022-09-05 20:53:03，"小昌同学"
Hi,
we were going to upgrade our application from Flink 1.14.4 to Flink 1.15.2,
when we noticed, that all our job tests, using a MiniClusterWithClientResource,
are multiple times slower in 1.15 than before in 1.14. I, unfortunately, have
not found mentions in that regard in the changelog or
Hey Yang,
The flink-conf.yaml submitted to the cluster does not contain
"kubernetes.config.file" at all.
In addition, I verified flink config maps under cluster's namespace do not
contain "kubernetes.config.file".
In addition, we also noticed the following exception (appears to happen
Hey Marco,
Unfortunately there is no built in k8s API that models an application mode
JM exactly but Deployments should be fine, in general. As Gyula notes,
where they can be difficult is during application upgrades as Deployments
never let their pods exit, even if successful, so there is no way
Table result = kafka_item.groupBy($("trans_number"))
.select($("trans_number"),$("sales_amount").sum().as("sum_amount"))
.filter($("sum_amount "));
各位大佬 请教一个问题 我这边想通过flink table API 达到这样一个效果：
根据trans_number进行分组 然后对另一个字段进行sum计算 然后我想最后进行过滤的时候 过滤出来这个sum值大于100的
我这个后续怎么使用API啊
Hi all,
i am quite new to flink and kafka, so i might mix something up here.
The situation is that we do have a flink application (1.14.5 with scala
2.12) running for a few hours to days and suddenly it stops working and
can't publish to kafka anymore.
I then noticed this message showing up twice
Could you please check whether the "kubernetes.config.file" is configured
to /opt/flink/.kube/config in the Flink configmap?
It should be removed before creating the Flink configmap.
Best,
Yang
Tamir Sagi 于2022年9月4日周日 18:08写道：
> Hey All,
>
> We recently updated to Flink 1.15.1. We deploy
I do not think we could add an additional port to the rest service since it
is created by Flink internally.
Actually, I do not suggest scrapping the metrics from rest service.
Instead, the port in the pod needs to be used.
Because the metrics might not work correctly if multiple JobManagers are
I think we have no concrete reason to always override the
"REST_SERVICE_EXPOSED_TYPE" to "ClusterIP".
It was introduced to fix the default value for releases before 1.15. And I
believe we need to respect the user configured values.
Best,
Yang
Vignesh Kumar Kathiresan 于2022年9月3日周六 05:07写道：
>
* September 7th
On 05/09/2022 11:27, Chesnay Schepler wrote:
On Wednesday, September 9th, the Flink 1.14.5/1.15.2 Docker images
will switch bases
FROM openjdk:8/11-jar (Debian-based)
TO eclipse-temurin:8/11-jre-jammy (Ubuntu-based)
due to the deprecation of the OpenJDK images.
Users that
You can use deployments of course , the operator and native k8s integration
does exactly that.
Even then job updates can be tricky so I believe you are much better off
with the operator.
Gyula
On Sun, 4 Sep 2022 at 11:11, marco andreas
wrote:
> Hello,
>
> Thanks for the response, I will take
On Wednesday, September 9th, the Flink 1.14.5/1.15.2 Docker images will
switch bases
FROM openjdk:8/11-jar (Debian-based)
TO eclipse-temurin:8/11-jre-jammy (Ubuntu-based)
due to the deprecation of the OpenJDK images.
Users that customized the images are advised to check for breaking changes.
Usually, it would be more helpful to provide a link to the PR to get a
better picture of the problem. I'm not 100% sure whether I grasp what's
wrong.
It looks like your branch is based on apache/flink:release-1.15 [1].
Therefore, you should fetch the most recent version from upstream and then
do
What I would need is to set
ports:
- name: metrics
port:
protocol: TCP
in the generated YAML fir the appname-rest service which properly
aggregates the metrics from the pods, but I can't not figure out either
from the job deployment file or modifying the operator
??
----
??:
"kcz"
23 matches
Mail list logo
