dongjoon-hyun closed pull request #44173: [SPARK-46258][CORE] Add
`RocksDBPersistenceEngine`
URL: https://github.com/apache/spark/pull/44173
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840284937
Thank you so much both of you, @LuciferYang and @yaooqinn . Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
yaooqinn commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840281483
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe
yaooqinn commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415148979
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) und
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840278347
I updated the benchmark result with the final commit, too~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use t
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415147988
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415142557
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415142557
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415126869
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415108828
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
yaooqinn commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415095668
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) und
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840203815
Thank you, @LuciferYang .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
yaooqinn commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415066164
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
Spark
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840187474
Thank you for pointing out that.
> Maybe the documents also need to be updated in this pr?
Actually, I thought about making a new doc PR for the following accumulated
stuff
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415034531
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
LuciferYang commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840164897
Maybe the documents also need to be updated in this pr?
https://github.com/apache/spark/blob/1d80d80a56c418f841e282ad753fad6671c3baae/docs/spark-standalone.md?plain=1#L736
--
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414983766
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
LuciferYang commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415001683
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) u
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1415000391
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
LuciferYang commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414999619
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
Sp
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414996134
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414987963
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -186,6 +186,10 @@ private[deploy] class Master(
val fsFactory =
new
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414986088
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414985071
##
core/src/main/scala/org/apache/spark/deploy/master/RocksDBPersistenceEngine.scala:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
dongjoon-hyun commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414983766
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
LuciferYang commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414940118
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -186,6 +186,10 @@ private[deploy] class Master(
val fsFactory =
new Fi
LuciferYang commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414938380
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
Sp
LuciferYang commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414916176
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
Sp
yaooqinn commented on code in PR #44173:
URL: https://github.com/apache/spark/pull/44173#discussion_r1414926099
##
core/src/main/scala/org/apache/spark/deploy/master/RecoveryModeFactory.scala:
##
@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf:
Spark
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840044360
Could you review this PR, @yaooqinn ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
dongjoon-hyun commented on PR #44173:
URL: https://github.com/apache/spark/pull/44173#issuecomment-1840012871
Could you review this PR, @LuciferYang ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
31 matches
Mail list logo
