fhalde commented on PR #43735: URL: https://github.com/apache/spark/pull/43735#issuecomment-1822029844
Hi, we are super interested in having the isolated classloader per spark session ability for our usecase. i believe this today is only achievable if jobs are run from a connect client. we want to avoid using connect client but with this pr merged, it should be possible to have isolated classloaders per spark session right without any use of spark connect? our use-case involves starting a spark driver and dynamically loading & adding jars and running transformations present within the jars. without isolated classloaders per session, on the executor side we would risk classpath conflict @HyukjinKwon if your pr addresses my concern, i can back port it to 3.5.0 in my fork -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
