Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 I saw: ``` Otherwise, we are not even able to start Spark shell. Currently, Hadoop 3 profile in Apache Spark doesn't work at all. It will face an error message such as: java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.0 ``` Other error messages are fixed by the current change. Yup, we could make a decision and change into Spark side. However, I would like to ask to do this separately ... I missed an answer to this question: > Do things work if you add hadoop-3.1 to sbt_maven_hadoop_profiles in run-tests.py and change the PR title accordingly? Don't remember what's the right incantation to trigger running with a different profile ("[test-hadoop-x.y]" maybe?). BTW, I plan to fix everything else about doc and Jenkins together when we are ready (which should be few line changes all together I think). I will try and fix these too when we are ready.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org