[ https://issues.apache.org/jira/browse/FLINK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522624#comment-17522624 ]
Yun Tang commented on FLINK-24932: ---------------------------------- [~maver1ck] Thanks for your information. We actually always watch the progress. However, since Flink community leverage FRocksDB which is based on the original RocksDB, we need some time to bump the RocksDB version. > Frocksdb cannot run on Apple M1 > ------------------------------- > > Key: FLINK-24932 > URL: https://issues.apache.org/jira/browse/FLINK-24932 > Project: Flink > Issue Type: Sub-task > Components: Runtime / State Backends > Reporter: Yun Tang > Priority: Minor > > After we bump up RocksDB version to 6.20.3, we support to run RocksDB on > linux arm cluster. However, according to the feedback from Robert, Apple M1 > machines cannot run FRocksDB yet: > {code:java} > java.lang.Exception: Exception while creating StreamOperatorStateContext. > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:255) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:268) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:109) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) > ~[flink-runtime-1.14.0.jar:1.14.0] > at > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) > ~[flink-runtime-1.14.0.jar:1.14.0] > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) > ~[flink-runtime-1.14.0.jar:1.14.0] > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) > ~[flink-runtime-1.14.0.jar:1.14.0] > at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_312] > Caused by: org.apache.flink.util.FlinkException: Could not restore keyed > state backend for StreamFlatMap_c21234bcbf1e8eb4c61f1927190efebd_(1/1) from > any of the 1 provided restore options. > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > Caused by: java.io.IOException: Could not load the native RocksDB library > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:882) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:402) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:345) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:87) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:329) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > Caused by: java.lang.UnsatisfiedLinkError: > /private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib: > > dlopen(/private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib, > 0x0001): tried: > '/private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib' > (mach-o file, but is an incompatible architecture (have 'x86_64', need > 'arm64e')), '/usr/lib/librocksdbjni-osx.jnilib' (no such file) > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > ~[?:1.8.0_312] > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) > ~[?:1.8.0_312] > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) > ~[?:1.8.0_312] > at java.lang.Runtime.load0(Runtime.java:811) ~[?:1.8.0_312] > at java.lang.System.load(System.java:1088) ~[?:1.8.0_312] > at > org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:79) > ~[frocksdbjni-6.20.3-ververica-1.0.jar:?] > at > org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:57) > ~[frocksdbjni-6.20.3-ververica-1.0.jar:?] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:856) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:402) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:345) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:87) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:329) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > {code} > This issue is tracked by RocksDB community: > [rocksdb/issues/7720|https://github.com/facebook/rocksdb/issues/7720] -- This message was sent by Atlassian Jira (v8.20.1#820001)