sven-weber-db commented on code in PR #55611:
URL: https://github.com/apache/spark/pull/55611#discussion_r3195435849
##########
core/src/main/scala/org/apache/spark/SparkEnv.scala:
##########
@@ -120,6 +125,31 @@ class SparkEnv (
pythonExec: String, workerModule: String, daemonModule: String, envVars:
Map[String, String])
private val pythonWorkers = mutable.HashMap[PythonWorkersKey,
PythonWorkerFactory]()
+ /**
+ * :: Experimental ::
+ * Session factory to generate UDF worker sessions
+ * using the new UDF framework proposed in SPARK-55278
+ */
+ private val externalUDFSessionFactory: WorkerSessionFactory =
createUDFWorkerSessionFactory()
+
+ private def createUDFWorkerSessionFactory() : WorkerSessionFactory = {
Review Comment:
This method will probably only be used in `SparkEnv`. However, it is
intended that we will create different session/dispatcher factories, depending
on certain environment or Spark configurations. Therefore, I think a dedicated
method is more suitable here as it can easily be extended with creation
conditions in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
