[ https://issues.apache.org/jira/browse/FLINK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554575#comment-17554575 ]
Yun Tang commented on FLINK-24932: ---------------------------------- [~smattheis] What's the progress now? > Frocksdb cannot run on Apple M1 > ------------------------------- > > Key: FLINK-24932 > URL: https://issues.apache.org/jira/browse/FLINK-24932 > Project: Flink > Issue Type: Sub-task > Components: Runtime / State Backends > Reporter: Yun Tang > Assignee: Sebastian Mattheis > Priority: Minor > > After we bump up RocksDB version to 6.20.3, we support to run RocksDB on > linux arm cluster. However, according to the feedback from Robert, Apple M1 > machines cannot run FRocksDB yet: > {code:java} > java.lang.Exception: Exception while creating StreamOperatorStateContext. > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:255) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:268) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:109) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) > ~[flink-runtime-1.14.0.jar:1.14.0] > at > org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) > ~[flink-runtime-1.14.0.jar:1.14.0] > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) > ~[flink-runtime-1.14.0.jar:1.14.0] > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) > ~[flink-runtime-1.14.0.jar:1.14.0] > at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_312] > Caused by: org.apache.flink.util.FlinkException: Could not restore keyed > state backend for StreamFlatMap_c21234bcbf1e8eb4c61f1927190efebd_(1/1) from > any of the 1 provided restore options. > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > Caused by: java.io.IOException: Could not load the native RocksDB library > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:882) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:402) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:345) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:87) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:329) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > Caused by: java.lang.UnsatisfiedLinkError: > /private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib: > > dlopen(/private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib, > 0x0001): tried: > '/private/var/folders/js/yfk_y2450q7559kygttykwk00000gn/T/rocksdb-lib-5783c058ce68d31d371327abc9b51cac/librocksdbjni-osx.jnilib' > (mach-o file, but is an incompatible architecture (have 'x86_64', need > 'arm64e')), '/usr/lib/librocksdbjni-osx.jnilib' (no such file) > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > ~[?:1.8.0_312] > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) > ~[?:1.8.0_312] > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) > ~[?:1.8.0_312] > at java.lang.Runtime.load0(Runtime.java:811) ~[?:1.8.0_312] > at java.lang.System.load(System.java:1088) ~[?:1.8.0_312] > at > org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:79) > ~[frocksdbjni-6.20.3-ververica-1.0.jar:?] > at > org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:57) > ~[frocksdbjni-6.20.3-ververica-1.0.jar:?] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:856) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:402) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:345) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:87) > ~[flink-statebackend-rocksdb_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:329) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) > ~[flink-streaming-java_2.11-1.14.0.jar:1.14.0] > ... 11 more > {code} > This issue is tracked by RocksDB community: > [rocksdb/issues/7720|https://github.com/facebook/rocksdb/issues/7720] -- This message was sent by Atlassian Jira (v8.20.7#820007)