[jira] [Commented] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555255#comment-17555255 ] Dongjoon Hyun commented on SPARK-35782: --- We provide RocksDB alternative for this issue. > leveldbjni doesn't work in Apple Silicon on macOS > - > > Key: SPARK-35782 > URL: https://issues.apache.org/jira/browse/SPARK-35782 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.1.2 >Reporter: DB Tsai >Priority: Major > > leveldbjni doesn't contain the native library for Apple Silicon on macOS. We > will need to build native library for Apple Silicon on macOS, and cut a new > release so Spark can use it. > However, it is not maintained for a long time, and the last release was in > 2016. Per > [discussion|http://apache-spark-developers-list.1001551.n3.nabble.com/leveldbjni-dependency-td30146.html] > in spark dev mailing list, other platform also runs into the same support > issue. Perhaps, we should we consider racksdb as replacement. > Note, here is the rocksdb task to support Apple Silicon, > https://github.com/facebook/rocksdb/issues/7720 -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365014#comment-17365014 ] DB Tsai commented on SPARK-35782: - [~yikunkero] It will only work for Apple Silicon on Linux but not macOS. For macOS, we need to recompile for this specific OS. > leveldbjni doesn't work in Apple Silicon on macOS > - > > Key: SPARK-35782 > URL: https://issues.apache.org/jira/browse/SPARK-35782 > Project: Spark > Issue Type: Sub-task > Components: Spark Core >Affects Versions: 3.1.2 >Reporter: DB Tsai >Priority: Major > > leveldbjni doesn't contain the native library for Apple Silicon on macOS. We > will need to build native library for Apple Silicon on macOS, and cut a new > release so Spark can use it. > However, it is not maintained for a long time, and the last release was in > 2016. Per > [discussion|http://apache-spark-developers-list.1001551.n3.nabble.com/leveldbjni-dependency-td30146.html] > in spark dev mailing list, other platform also runs into the same support > issue. Perhaps, we should we consider racksdb as replacement. > Note, here is the rocksdb task to support Apple Silicon, > https://github.com/facebook/rocksdb/issues/7720 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364790#comment-17364790 ] Yikun Jiang commented on SPARK-35782: - I'm not sure the common arm64 native can meet Apple Silicon arch or not? See realted: [https://github.com/apache/spark/pull/26636] https://issues.apache.org/jira/browse/SPARK-27721 > leveldbjni doesn't work in Apple Silicon on macOS > - > > Key: SPARK-35782 > URL: https://issues.apache.org/jira/browse/SPARK-35782 > Project: Spark > Issue Type: Sub-task > Components: Spark Core >Affects Versions: 3.1.2 >Reporter: DB Tsai >Priority: Major > > leveldbjni doesn't contain the native library for Apple Silicon on macOS. We > will need to build native library for Apple Silicon on macOS, and cut a new > release so Spark can use it. > However, it is not maintained for a long time, and the last release was in > 2016. Per > [discussion|http://apache-spark-developers-list.1001551.n3.nabble.com/leveldbjni-dependency-td30146.html] > in spark dev mailing list, other platform also runs into the same support > issue. Perhaps, we should we consider racksdb as replacement. > Note, here is the rocksdb task to support Apple Silicon, > https://github.com/facebook/rocksdb/issues/7720 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
