HyukjinKwon commented on code in PR #58620:
URL: https://github.com/apache/spark/pull/58620#discussion_r3974115487
##########
core/src/main/scala/org/apache/spark/deploy/master/ZooKeeperPersistenceEngine.scala:
##########
@@ -39,6 +41,12 @@ private[master] class ZooKeeperPersistenceEngine(conf:
SparkConf, val serializer
private val workingDir = conf.get(ZOOKEEPER_DIRECTORY).getOrElse("/spark") +
"/master_status"
private val zk: CuratorFramework = SparkCuratorUtil.newClient(conf)
+ // Only instantiate well-known classes while reading persisted state back,
so corrupted
+ // or unexpected znode contents are dropped instead of being instantiated in
the newly
+ // elected master.
+ private val serializationFilter: ObjectInputFilter =
+
ObjectInputFilter.Config.createFilter(conf.get(RECOVERY_SERIALIZATION_FILTER))
Review Comment:
`spark.deploy.recoverySerializationFilter` is read only here.
`FileSystemPersistenceEngine` and `RocksDBPersistenceEngine` deserialize the
same `ApplicationInfo`/`DriverInfo`/`WorkerInfo` state through the same
unfiltered `JavaSerializer`, so the config's recovery-wide name and doc don't
actually cover them. Is limiting it to ZK intentional for now - the
file/RocksDB stores are local to the master host, a lower-exposure surface than
a shared ZooKeeper ensemble - with the other engines as follow-up? If so a
one-line note would help; if not, the same `JavaDeserializationStream` filter
applies directly in their deserialize paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
