During development, I'd recommend giving Hadoop a version ending with -SNAPSHOT, and building spark with maven, as mvn knows to refresh the snapshot every day.
you can do this in hadoop with mvn versions:set 2.7.0.stevel-SNAPSHOT if you are working on hadoop branch-2 or trunk direct, they come with -SNAPSHOT anyway, but unless you build hadoop every morning, you may find maven pulls in the latest nightly builds from the apache snapshot repository, which will cause chaos and confusion. This is also why you must never have maven build which spans midnight in your time zone. On 9 Oct 2015, at 22:31, Matei Zaharia <matei.zaha...@gmail.com<mailto:matei.zaha...@gmail.com>> wrote: You can publish your version of Hadoop to your Maven cache with mvn publish (just give it a different version number, e.g. 2.7.0a) and then pass that as the Hadoop version to Spark's build (see http://spark.apache.org/docs/latest/building-spark.html). Matei On Oct 9, 2015, at 3:10 PM, Dogtail L <spark.ru...@gmail.com<mailto:spark.ru...@gmail.com>> wrote: Hi all, I have modified Hadoop source code, and I want to compile Spark with my modified Hadoop. Do you know how to do that? Great thanks!