Hi Sean, I think that the main issue was users importing the package while starting SPARK just like the way we used to do in SPARK 1.6. After removing that option from --package while starting SPARK 2.0 the issue of conflicting libraries disappeared.
I have written about this in https://github.com/databricks/spark-csv/issues/367. But perhaps mentioning this in this email group as well helps. Regards, Gourav Sengupta