Re: dependencies conflict in oozie spark action for spark 2

2018-03-07 Thread Lian Jiang
I found below inconsistency between oozie and spark2 jars: jackson-core-2.4.4.jar oozie jackson-core-2.6.5.jar spark2 jackson-databind-2.4.4.jar oozie jackson-databind-2.6.5.jar spark2 jackson-annotations-2.4.0.jar oozie jackson-annotations-2.6.5.jar spark2 I removed the lower version jars

dependencies conflict in oozie spark action for spark 2

2018-03-06 Thread Lian Jiang
I am using HDP 2.6.4 and have followed https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_spark-component-guide/content/ch_oozie-spark-action.html to make oozie use spark2. After this, I found there are still a bunch of issues: 1. oozie and spark tries to add the same jars multiple time