目前Flink社区版RocksDB尚不支持ARM架构机器。使用RocksDB的话，内存均是堆外管理，与JVM的堆上内存无关。
另外，有个题外话，你们是云上产品还是自建了ARM集群？有点好奇目前国内的ARM集群使用率情况。
祝好
唐云
________________________________
From: Wanghui (HiCampus) <wanghu...@huawei.com>
Sent: Thursday, July 15, 2021 11:33
To: user-zh@flink.apache.org <user-zh@flink.apache.org>
Subject: Re: flink大窗口性能问题
我在aarch64 + jre 8的环境下，使用rocksdb state backend时，碰到如下错误：
另外，使用rocksdb可以解决大窗口的oom问题吗，原理是什么？
Caused by: java.lang.Exception: Exception while creating
StreamOperatorStateContext.
at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:222)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:290)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$1(StreamTask.java:506)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:526)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkException: Could not restore keyed state
backend for KeyedProcessOperator_ae33e81d863e4093619373d1e1f77012_(1/1) from
any of the 1 provided restore options.
at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:335)
at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:148)
... 9 more
Caused by: java.io.IOException: Could not load the native RocksDB library
at
org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:948)
at
org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:489)
at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:319)
at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:142)
at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:121)
... 11 more
Caused by: java.lang.UnsatisfiedLinkError:
/tmp/rocksdb-lib-bd8659305e92a27fac27481baf57897b/librocksdbjni-linux64.so:
/tmp/rocksdb-lib-bd8659305e92a27fac27481baf57897b/librocksdbjni-linux64.so:
cannot open shared object file: No such file or directory (Possible cause:
can't load AMD 64-bit .so on a AARCH64-bit platform)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
at java.lang.Runtime.load0(Runtime.java:810)
at java.lang.System.load(System.java:1088)
at
org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78)
at
org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56)
at
org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:923)
... 15 more
On 2021/07/15 02:47:23, Jingsong Li <j...@gmail.com<mailto:j...@gmail.com>>
wrote:
> 没用rocksdb吗？>
>
>
>
> On Thu, Jul 15, 2021 at 10:46 AM Michael Ran
> <gr...@163.com<mailto:gr...@163.com>> wrote:>
>
>
>
> > 要么内存增大，或者并行增大，要么窗口改小，同时保留数据时间减少>
>
> > 在 2021-07-15 10:23:25，"Hui Wang"
> > <46...@qq.com.INVALID<mailto:46...@qq.com.INVALID>> 写道：>
>
> > >flink大窗口缓存数据量过大导致jvm频烦full gc，并且处理速度极低，最终OOM，该如何调优>
>
> >>
>
>
>
>
>
> -- >
>
> Best, Jingsong Lee>
>
>
