Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-57317439 I haven't looked at the code to fully understand what all changes would be required and why its so much more complex. So if it makes the implementation much simpler then I guess its a step in the right direction. I don't see any huge issues with it, other then you are grabbing the LD_LIBRARY_PATH of machines that you aren't necessarily going to run on. We grab it on the client for the AM and on the AM for the executors. If you have a heterogeneous cluster those could potentially be different, especially between the client and the AM as in many cases we have gateways that aren't managed by grid Se. Are you thinking we will have a followup to move to using LD_LIBRARY_PATH or thinking its not necessary?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org