gh-yzou commented on code in PR #3194:
URL: https://github.com/apache/polaris/pull/3194#discussion_r2587022183
##########
plugins/spark/v3.5/integration/build.gradle.kts:
##########
@@ -62,11 +62,51 @@ dependencies {
exclude("org.apache.logging.log4j", "log4j-core")
exclude("org.slf4j", "jul-to-slf4j")
}
+
+ // Add spark-hive for Hudi integration - provides HiveExternalCatalog that
Hudi needs
Review Comment:
@rahil-c could you also update the readme to include the support for hudi?
It would be great if we could also have a notebook in the get-started to
help people to onboard for hudi, we could do that in follow up, we should also
extend the regress test to include actual end to end test for hudi to avoid any
potential break of the feature
##########
plugins/spark/v3.5/spark/build.gradle.kts:
##########
@@ -46,6 +46,7 @@ dependencies {
// TODO: extract a polaris-rest module as a thin layer for
// client to depends on.
implementation(project(":polaris-core")) { isTransitive = false }
+
testImplementation("org.apache.hudi:hudi-spark3.5-bundle_${scalaVersion}:1.1.0")
Review Comment:
for the actual spark project, we don't really intend to introduce any table
format specific dependency, even for testing. i didn't see any change in the
actual spark project, is there a reason that we need this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
