[
https://issues.apache.org/jira/browse/HBASE-30189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-30189 started by Wellington Chevreuil.
----------------------------------------------------
> Implement SparkSQL support in the spark4 module using DataSource V2 API
> -----------------------------------------------------------------------
>
> Key: HBASE-30189
> URL: https://issues.apache.org/jira/browse/HBASE-30189
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> The spark4 module currently has basic HBase RDD operations (put, get, delete,
> foreachPartition, mapPartitions). The next step is porting the SparkSQL
> datasource layer from the spark3 module, which enables reading/writing HBase
> tables via Spark SQL (spark.read.format("hbase")...). The spark3 module uses
> the deprecated DataSource V1 API (RelationProvider, BaseRelation,
> PrunedFilteredScan).
> For spark4, we'll implement the modern DataSource V2 API, which provides
> better performance (columnar reads, push-down at the connector level) and a
> cleaner contract.
> In order to have granular, incremental PRs for easy of reviews, I'll split
> this in smaller PRs to be merged into a feature branch first. Once this is
> complete, we can cherry-pick the commits into master.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
