I found below inconsistency between oozie and spark2 jars:
jackson-core-2.4.4.jar oozie
jackson-core-2.6.5.jar spark2
jackson-databind-2.4.4.jar oozie
jackson-databind-2.6.5.jar spark2
jackson-annotations-2.4.0.jar oozie
jackson-annotations-2.6.5.jar spark2
I removed the lower version jars
I am using HDP 2.6.4 and have followed
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_spark-component-guide/content/ch_oozie-spark-action.html
to make oozie use spark2.
After this, I found there are still a bunch of issues:
1. oozie and spark tries to add the same jars multiple time