Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/23166#discussion_r237875851 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -708,16 +709,36 @@ private[spark] class PythonBroadcast(@transient var path: String) extends Serial override def handleConnection(sock: Socket): Unit = { val env = SparkEnv.get val in = sock.getInputStream() - val dir = new File(Utils.getLocalDir(env.conf)) - val file = File.createTempFile("broadcast", "", dir) - path = file.getAbsolutePath - val out = env.serializerManager.wrapForEncryption(new FileOutputStream(path)) + val abspath = new File(path).getAbsolutePath + val out = env.serializerManager.wrapForEncryption(new FileOutputStream(abspath)) --- End diff -- ok I think we agree its good this way, (just to verify though I won't commit until you +1 it), but yes you are correct, now that we are using the decryption server which reads from the path in PythonBroadcast the path change isn't strictly necessary, but the value of self._path in broadcast.py doesn't match the path in PythonBroadcast so I think its better to have those match.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org