Re: Hadoop profile change to hadoop-2 and hadoop-3 since Spark 3.3

and @tgra...@apache.org too On Sat, 11 Dec 2021 at 21:38, Hyukjin Kwon wrote: > cc @Holden Karau @DB Tsai @Imran > Rashid @Mridul Muralidharan FYI > > On Thu, 9 Dec 2021 at 14:07, angers zhu wrote: > >> Hi all, >> >> Since Spark 3.2, we have supported Hadoop 3.3.1 now, but its profile name

Re: Hadoop profile change to hadoop-2 and hadoop-3 since Spark 3.3

cc @Holden Karau @DB Tsai @Imran Rashid @Mridul Muralidharan FYI On Thu, 9 Dec 2021 at 14:07, angers zhu wrote: > Hi all, > > Since Spark 3.2, we have supported Hadoop 3.3.1 now, but its profile name > is *hadoop-3.2* (and *hadoop-2.7*) that is not correct. > So we made a change in https://g

Hadoop profile change to hadoop-2 and hadoop-3 since Spark 3.3

Hi all, Since Spark 3.2, we have supported Hadoop 3.3.1 now, but its profile name is *hadoop-3.2* (and *hadoop-2.7*) that is not correct. So we made a change in https://github.com/apache/spark/pull/34715 Starting from Spark 3.3, we use hadoop profile *hadoop-2* and *hadoop-3 *, and default hadoop