mridulm commented on code in PR #39202:
URL: https://github.com/apache/spark/pull/39202#discussion_r1056861664
##########
docs/monitoring.md:
##########
@@ -341,6 +341,16 @@ Security options for the Spark History Server are covered
more detail in the
</td>
<td>2.3.0</td>
</tr>
+ <tr>
+ <td>spark.history.store.serializer</td>
+ <td>JSON</td>
+ <td>
+ Serializer for writing/reading in-memory UI objects to/from disk-based
KV Store; JSON or PROTOBUF.
+ JSON serializer is the only choice before Spark 3.4.0, thus it is the
default value.
+ PROTOBUF serializer is fast and compact, and it is the default
serializer for disk-based KV store of live UI.
Review Comment:
Users cannot leverage/access the live db right ? (Unlike history db, which
is used across restarts).
Do we want to expose the impl details ? If we change it tomorrow to
something else, it should be completely transparent to users.
##########
core/src/main/scala/org/apache/spark/status/KVUtils.scala:
##########
@@ -111,7 +122,7 @@ private[spark] object KVUtils extends Logging {
// The default serializer is slow since it is using JSON+GZip encoding.
Some(new KVStoreProtobufSerializer())
} else {
- None
+ Some(serializerForHistoryServer(conf))
}
Review Comment:
QQ: If it is always non empty, change from Option ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
