Hi, I've been reviewing SparkContext and found preferredNodeLocationData that was made obsoleted by SPARK-8949 [1].
When you search where SparkContext.preferredNodeLocationData is used, you find 3 places - one constructor marked @deprecated, the other with logWarning telling us that "Passing in preferred locations has no effect at all, see SPARK-8949", and in org.apache.spark.deploy.yarn.ApplicationMaster.registerAM method. org.apache.spark.deploy.yarn.ApplicationMaster.registerAM method caught my eye and I found that it does the following in client.register: if (sc != null) sc.preferredNodeLocationData else Map() However, AFAIU client.register ignores the input parameter completely (!) It's not used in the body of the method and seems a leftover. The input parameter should be removed and so should the above line. What do you think? Should I report an issue and clean it up via a pull req? BTW, What do you think about removing SparkContext.preferredNodeLocationData as part of the cleanup? [1] https://issues.apache.org/jira/browse/SPARK-8949 Pozdrawiam, Jacek -- Jacek Laskowski | http://blog.japila.pl | http://blog.jaceklaskowski.pl Follow me at https://twitter.com/jaceklaskowski Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
