github-actions[bot] closed pull request #44197: [SPARK-39800][SQL][WIP]
DataSourceV2: View Support
URL: https://github.com/apache/spark/pull/44197
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
github-actions[bot] commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-2050753239
We're closing this PR because it hasn't been updated in a while. This isn't
a judgement on the merit of the PR in any way. It's just a way of keeping the
PR queue manageable.
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1873702020
Done:
- Converted to v2 command framework
- Incorporated review comments
- Passed existing unit tests
TODO:
- ShowViewPropertiesExec: Redact properties and hide r
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1873688145
Only 2 unit tests failed but they succeeded on my laptop
- org.apache.spark.deploy.yarn.YarnClusterSuite
- org.apache.spark.sql.streaming.RocksDBStateStoreStreamingAggregationSuite
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1873076278
Failed unit tests
- org.apache.spark.sql.hive.execution.HiveSQLViewSuite
- org.apache.spark.sql.SQLQueryTestSuite
- org.apache.spark.sql.execution.PersistedViewTestSuite
- org.ap
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1872227012
Almost done converting to v2 command framework. Seeing great reduction in
changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on t
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1868064293
Looking to leverage the newly added `ViewInfo` more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jzhuge commented on code in PR #44197:
URL: https://github.com/apache/spark/pull/44197#discussion_r1435191176
##
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala:
##
@@ -514,6 +516,43 @@ class DataSourceV2Strategy(session: SparkSes
jzhuge commented on code in PR #44197:
URL: https://github.com/apache/spark/pull/44197#discussion_r1434824314
##
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/CreateViewExec.scala:
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (AS
nastra commented on code in PR #44197:
URL: https://github.com/apache/spark/pull/44197#discussion_r1424016706
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/VIewSubstitution.scala:
##
@@ -0,0 +1,104 @@
+/*
Review Comment:
nit: the file is named `VIewSu
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1842032036
Current status:
- Fix failed unit tests
- Incorporate all @MaxGekk comments (took care of some)
- Add unit tests
Follow up:
- Support user specified column names
jzhuge commented on PR #44197:
URL: https://github.com/apache/spark/pull/44197#issuecomment-1842031303
Had to create a new pull request since
https://github.com/apache/spark/pull/39796 was closed
--
This is an automated message from the Apache Git Service.
To respond to the message, pleas
jzhuge opened a new pull request, #44197:
URL: https://github.com/apache/spark/pull/44197
### What changes were proposed in this pull request?
This PR adds support to load, create, alter, and drop views in DataSource V2
catalogs.
- View substitution rule
- Create view DDL
holdenk commented on PR #39796:
URL: https://github.com/apache/spark/pull/39796#issuecomment-1841328847
It won't let me re-open, can you re-create the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
jzhuge commented on PR #39796:
URL: https://github.com/apache/spark/pull/39796#issuecomment-1839214419
Current status:
- Incorporate all @MaxGekk comments (took care of some)
- Add unit tests
Follow up:
- Support user specified column names
- Support viewSQLConfigs
- Con
15 matches
Mail list logo
