[
https://issues.apache.org/jira/browse/HBASE-30178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-30178 started by Wellington Chevreuil.
----------------------------------------------------
> Add support to spark 4 in hbase-connectors
> ------------------------------------------
>
> Key: HBASE-30178
> URL: https://issues.apache.org/jira/browse/HBASE-30178
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-connectors
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> With spark 4.0.0 GA for around a year now, we should start working on making
> hbase-connectors integrate with spark 4.
> Since we would still need to support spark 3 for a while and due to spark 4
> requiring minimum jdk17 and scala 2.13, it would be easier to define a
> specific spark 4 module under the hbase-connectors root, so that
> spark.version, scala.version, compiler flags, and optional spark-4 only APIs
> never collide with spark 3 in a single parent pom.
> We should keep feature parity as much as possible. Datasources, context, rdd
> and connection cache related logic can be duplicated under this new
> hbase-spark4 module, in order to compile and package with jdk17 and scala
> 2.13. This module build should be isolated on a specific maven profile, so it
> wouldn't be build by default.
> Pushdown filter related classes and filter protocol would remain on a common
> module (no code duplication needed).
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
