yadavay-amzn commented on PR #55869: URL: https://github.com/apache/spark/pull/55869#issuecomment-4536138430
@sarutak You're right that Thrift Server and most production apps launch via SparkSubmit, which handles classloader setup correctly. The gap we identified is specifically for programmatic SparkSession creation (without SparkSubmit) -- e.g., embedded applications or test frameworks that create SparkContext directly. In that path, spark.jars are never added to any driver classloader. That said, I agree this is a narrow use case. If you feel it's not worth the added complexity, I'm happy to close the PR. Would you prefer we file a separate JIRA specifically for the programmatic case, or is this not worth fixing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
