I have a list of cloudera jars which I need to provide in --jars clause, mainly for the HiveContext functionality I am using. However, many of these jars have version number as part of their names. This leads to an issue that the names might change when I do a Cloudera upgrade.
Just a note here, there are many jars which cloudera exposes as a symlink which is the link to the latest version of that jar(e.g /opt/cloudera/parcels/CDH/lib/parquet/parquet-hadoop-bundle.jar -> parquet-hadoop-bundle-1.5.0-cdh5.3.2.jar),in which case its good but there are many jars which aren't. Is there a flexible way to avoid this situation? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Generating-version-agnostic-jar-path-value-for-jars-clause-tp22734.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
