Remove HiveContext from Apache Spark 4.0

[杨杰]

Hi all,

In SPARK-46171 (apache/spark#44077 [1]), I’m trying to remove the
deprecated HiveContext from Apache Spark 4.0 since HiveContext has been
marked as deprecated after Spark 2.0. This is a long-deprecated API, it
should be replaced with SparkSession with enableHiveSupport now, so I think
it's time to remove it.

Feel free to comment if you have any concerns.

[1] https://github.com/apache/spark/pull/44077

Thanks,
Jie Yang

Migrating the Junit framework used in Apache Spark 4.0 from 4.x to 5.x

[杨杰]

Hi all,

In SPARK-44170 (apache/spark#43074 [1]), I’m trying to migrate the Junit
test framework used in Spark 4.0 from Junit4 to Junit5.


Although this involves a fair amount of code modifications, given that
Junit 4 is still developed based on Java 6 source code and it hasn't
released a new version for over two years (the Junit 4.13.2 that Spark is
currently using was released on February 14, 2021.), I personally believe
it's worth it.

Feel free to comment if you have any concerns.

[1] https://github.com/apache/spark/pull/43074

Thanks,
Jie Yang