[
https://issues.apache.org/jira/browse/PHOENIX-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498086#comment-17498086
]
Istvan Toth commented on PHOENIX-6632:
--------------------------------------
Assiging to me, as I'm working in intergating this pathc now.
Unfortunately, a lot of the scala tests fail.
While those tests use use the old deprecated Spark API, they are much more
thourough than the Java tests, and while a few of those failures are test
issues, quite a few of them highlight stuff that used to work witth Spark2, but
do not work with Spark 3.
You can test those on my WIP PR
[https://github.com/apache/phoenix-connectors/pull/71]
by running
{code:java}
mvn clean verify -am -pl phoenix5-spark3-it -Dscala-tests-enabled{code}
The tests log to the console, but you can find the results in
phoenix5-spark3-it/target/surefire-reports.
> Migrate connectors to Spark-3
> -----------------------------
>
> Key: PHOENIX-6632
> URL: https://issues.apache.org/jira/browse/PHOENIX-6632
> Project: Phoenix
> Issue Type: Improvement
> Components: spark-connector
> Affects Versions: connectors-6.0.0
> Reporter: Ashwin Balasubramani
> Assignee: Istvan Toth
> Priority: Major
>
> With Spark-3, the DatasourceV2 API has had major changes, where a new
> TableProvider Interface has been introduced. These new changes bring in more
> control to the data source developer and better integration with
> spark-optimizer.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
