Hi, All.
I made a PR to recover the PR Builder and the above Jenkins jobs since this
has been blocking us for a day.
https://github.com/apache/spark/pull/26375
There is a discussion how to proceed after recover this. We will recover
our `check-cran` test coverage as a follow-up.
Bests,
Hi, All.
CRAN instability seems to be a blocker for our dev process.
The following simple check causes consecutive failures in 4 of 9 Jenkins
jobs + PR builder.
- spark-branch-2.4-test-sbt-hadoop-2.6
- spark-branch-2.4-test-sbt-hadoop-2.7
- spark-master-test-sbt-hadoop-2.7
-
The changes for JDK 11 supports are not increasing the risk of Hadoop 3.2
profile.
Hive 1.2.1 execution JARs are much more stable than Hive 2.3.6 execution
JARs. The changes of thrift-servers are massive. We need more evidence to
prove the quality and stability before we switching the default to
yes i am not against hadoop 3 becoming the default. i was just questioning
the statement that we are close to dropping support for hadoop 2.
we build our own spark releases that we deploy on the clusters of our
clients. these clusters are hdp 2.x, cdh 5, emr, dataproc, etc.
i am aware that
Hi, Koert.
Could you be more specific to your Hadoop version requirement?
Although we will have Hadoop 2.7 profile, Hadoop 2.6 and older support is
officially already dropped in Apache Spark 3.0.0. We can not give you the
answer for Hadoop 2.6 and older version clusters because we are not
i dont see how we can be close to the point where we dont need to support
hadoop 2.x. this does not agree with the reality from my perspective, which
is that all our clients are on hadoop 2.x. not a single one is on hadoop
3.x currently. this includes deployments of cloudera distros, hortonworks
