Hi Steve and Sean, Didn't expect such a warm welcome from Sean and you! Since I'm with Spark on YARN these days, let me see what I can do to make it nicer. Thanks!
I'm going to change Spark to use buildPath first. And then propose another patch to use Environment.CLASS_PATH_SEPARATOR instead. And only then I could work on https://issues.apache.org/jira/browse/YARN-5247. Is this about changing the annotation(s) only? Thanks for your support! Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Tue, Jun 14, 2016 at 1:44 PM, Steve Loughran <ste...@hortonworks.com> wrote: > > if you want to be able to build up CPs on windows to run on a Linux cluster, > or vice-versa, you really need to be using the > Environment.CLASS_PATH_SEPARATOR field, "<CPS>". This is expanded in the > cluster, not in the client > > Although tagged as @Public, @Unstable, it's been in there sinceYARN-1824 & > Hadoop 2.4; things rely on it. If someone wants to fix that by submitting a > patch to YARN-5247; I'll review it. > >> On 13 Jun 2016, at 20:06, Sean Owen <so...@cloudera.com> wrote: >> >> Yeah it does the same thing anyway. It's fine to consistently use the >> method. I think there's an instance in ClientSuite that can use it. >> >> On Mon, Jun 13, 2016 at 6:50 PM, Jacek Laskowski <ja...@japila.pl> wrote: >>> Hi, >>> >>> Just noticed that yarn.Client#populateClasspath uses Path.SEPARATOR >>> [1] to build a CLASSPATH entry while another similar-looking line uses >>> buildPath method [2]. >>> >>> Could a pull request with a change to use buildPath at [1] be >>> accepted? I'm always confused how to fix such small changes. >>> >>> [1] >>> https://github.com/apache/spark/blob/master/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L1298 >>> [2] Path.SEPARATOR >>> >>> Pozdrawiam, >>> Jacek Laskowski >>> ---- >>> https://medium.com/@jaceklaskowski/ >>> Mastering Apache Spark http://bit.ly/mastering-apache-spark >>> Follow me at https://twitter.com/jaceklaskowski >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>> For additional commands, e-mail: dev-h...@spark.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
