As we know, cabon.properties is the configuration file of carbondata. However ,for now, the driver and executors load this file from local disk which means we should distribute this file to all yarn nodes. At the same time ,we also should configure some items like follows:
--conf "spark.driver.extraJavaOptions=-Dcarbon.properties.filepath=/home/carbon/carbon.properties" --conf "spark.executor.extraJavaOptions=-Dcarbon.properties.filepath=/home/carbon/carbon.properties" I think the best way to handle this is using "--files". Eg. --files /home/carbon/carbon.properties Then carbondata should try to load from classpath first. Also,we hope we can using --conf to overwrite the configurations in carbon.properties. Eg. --conf 'carbon.properties.filepath=HDFSLock' It's easy to implements this function, we can just modify the the code like follows: <http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/file/n10687/WX20170411-145049%402x.png> -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/how-to-distribute-cabon-properties-file-tp10687.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
