[
https://issues.apache.org/jira/browse/FLINK-22447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347616#comment-17347616
]
zlzhang0122 commented on FLINK-22447:
-------------------------------------
[~yunta] OK, If I have any progress on this, I will comment here.
> when job failed and restore from rocksdb, it produced a exception "Could not
> load the native RocksDB library"
> -------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-22447
> URL: https://issues.apache.org/jira/browse/FLINK-22447
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.11.1
> Reporter: zlzhang0122
> Priority: Major
>
> Have someone The full stack trace is:
> {code:java}
> // code placeholder
> java.lang.Exception: Exception while creating StreamOperatorStateContext. at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:204)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:274)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) at
> java.lang.Thread.run(Thread.java:748) Caused by:
> org.apache.flink.util.FlinkException: Could not restore keyed state backend
> for AdMetaJoinOperator_b6fdad9df33c017e430813a04b752f9a_(66/512) from any of
> the 1 provided restore options. at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:317)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:144)
> ... 9 more Caused by: java.io.IOException: Could not load the native RocksDB
> library at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:1007)
> at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:538)
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:301)
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:142)
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:121)
> ... 11 more Caused by: java.lang.UnsatisfiedLinkError:
> /home/work/hdd1/yarn/azmbcommonprc-hadoop/nodemanager/20800/usercache/s_global_ad/appcache/application_1617086162049_443089/rocksdb-lib-5280b3bf65c39fcc2c0c717a7bd6706f/librocksdbjni-linux64.so:
>
> /home/work/hdd1/yarn/azmbcommonprc-hadoop/nodemanager/20800/usercache/s_global_ad/appcache/application_1617086162049_443089/rocksdb-lib-5280b3bf65c39fcc2c0c717a7bd6706f/librocksdbjni-linux64.so:
> ELF load command past end of file at
> java.lang.ClassLoader$NativeLibrary.load(Native Method) at
> java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) at
> java.lang.Runtime.load0(Runtime.java:809) at
> java.lang.System.load(System.java:1086) at
> org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78)
> at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56)
> at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:982)
> ... 15 more{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
