Do people have a reliable/repeatable method for solving dependency issues or tips?
The current world of spark-hadoop-hbase-parquet-... is very challenging given the huge footprint of dependent packages and we may be pushing against the limits of how many packages can be combined into one environment... The process of searching the web to pick at incompatibilities one at a time is at best tedious and at worst non-converging. It makes me wonder if there is (or ought to be) a page cataloging in one place the conflicts that Spark users have hit and what was done to solve it. Eugene Yokota wrote an interesting blog about current sbt dependency management in sbt v 0.13.7 that includes nice improvements for working with dependencies: https://typesafe.com/blog/improved-dependency-management-with-sbt-0137 After reading that, I refreshed on the sbt documentation and found "show update". It gives very extensive information. For reference, there was an extensive discussion thread about sbt and maven last year that touches on a lot of topics: http://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201402.mbox/%3ccabpqxsukhd4qsf5dg9ruhn7wvonxfm+y5b1k5d8g7h6s9bh...@mail.gmail.com%3E
