[spark] branch master updated (095f7b0 -> 5e9a155)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 095f7b0 Revert "[SPARK-29277][SQL] Add early DSv2 filter and projection pushdown" add 5e9a155 [SPARK-29520][SS] Fix checks of negative intervals No new revisions were added by this update. Summary of changes: .../catalyst/analysis/StreamingJoinHelper.scala| 2 +- .../sql/catalyst/expressions/TimeWindow.scala | 2 +- .../plans/logical/EventTimeWatermark.scala | 5 ++- .../spark/sql/catalyst/util/IntervalUtils.scala| 40 +- .../sql/catalyst/util/IntervalUtilsSuite.scala | 36 +++ .../main/scala/org/apache/spark/sql/Dataset.scala | 2 +- .../sql/execution/streaming/GroupStateImpl.scala | 7 ++-- .../spark/sql/execution/streaming/Triggers.scala | 2 +- .../streaming/FlatMapGroupsWithStateSuite.scala| 9 +++-- 9 files changed, 90 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (5e9a155 -> 4d302cb)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e9a155 [SPARK-29520][SS] Fix checks of negative intervals add 4d302cb [SPARK-11150][SQL][FOLLOW-UP] Dynamic partition pruning No new revisions were added by this update. Summary of changes: .../spark/sql/execution/datasources/PruneFileSourcePartitions.scala | 2 +- sql/core/src/main/scala/org/apache/spark/sql/execution/subquery.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4d302cb -> cd39cd4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4d302cb [SPARK-11150][SQL][FOLLOW-UP] Dynamic partition pruning add cd39cd4 [SPARK-28560][SQL][FOLLOWUP] support the build side to local shuffle reader as far as possible in BroadcastHashJoin No new revisions were added by this update. Summary of changes: .../execution/adaptive/AdaptiveSparkPlanExec.scala | 2 - .../adaptive/OptimizeLocalShuffleReader.scala | 92 ++- .../adaptive/ReduceNumShufflePartitions.scala | 3 +- .../ReduceNumShufflePartitionsSuite.scala | 2 +- .../adaptive/AdaptiveQueryExecSuite.scala | 101 - 5 files changed, 128 insertions(+), 72 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (cd39cd4 -> faf220a)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from cd39cd4 [SPARK-28560][SQL][FOLLOWUP] support the build side to local shuffle reader as far as possible in BroadcastHashJoin add faf220a [SPARK-29277][SQL][test-hadoop3.2] Add early DSv2 filter and projection pushdown No new revisions were added by this update. Summary of changes: .../spark/sql/catalyst/analysis/Analyzer.scala | 14 ++- .../sql/catalyst/analysis/CheckAnalysis.scala | 14 +++ .../spark/sql/catalyst/optimizer/Optimizer.scala | 14 ++- .../sql/catalyst/plans/logical/v2Commands.scala| 6 +- .../sql/connector/catalog/CatalogV2Util.scala | 7 +- .../datasources/v2/DataSourceV2Relation.scala | 52 +++-- .../main/scala/org/apache/spark/sql/Dataset.scala | 4 +- .../spark/sql/execution/SparkOptimizer.scala | 12 +- .../datasources/v2/DataSourceV2Strategy.scala | 123 +++-- .../execution/datasources/v2/PushDownUtils.scala | 103 + .../datasources/v2/V2ScanRelationPushDown.scala| 66 +++ .../spark/sql/FileBasedDataSourceSuite.scala | 10 +- .../spark/sql/connector/DataSourceV2Suite.scala| 8 +- .../sql/execution/datasources/orc/OrcTest.scala| 11 +- .../datasources/orc/OrcV2SchemaPruningSuite.scala | 2 +- .../datasources/parquet/ParquetFilterSuite.scala | 11 +- .../execution/datasources/orc/OrcFilterSuite.scala | 10 +- .../execution/datasources/orc/OrcFilterSuite.scala | 9 +- 18 files changed, 308 insertions(+), 168 deletions(-) create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (faf220a -> 121510c)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from faf220a [SPARK-29277][SQL][test-hadoop3.2] Add early DSv2 filter and projection pushdown add 121510c [SPARK-29604][SQL][FOLLOWUP][test-hadoop3.2] Let SparkSQLEnvSuite to be run in dedicated JVM No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (121510c -> 888cc46)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 121510c [SPARK-29604][SQL][FOLLOWUP][test-hadoop3.2] Let SparkSQLEnvSuite to be run in dedicated JVM add 888cc46 [SPARK-29675][SQL] Add exception when isolationLevel is Illegal No new revisions were added by this update. Summary of changes: .../spark/sql/execution/datasources/jdbc/JDBCOptions.scala | 4 .../test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala| 13 + 2 files changed, 17 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888cc46 -> 8a8ac00)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888cc46 [SPARK-29675][SQL] Add exception when isolationLevel is Illegal add 8a8ac00 [SPARK-29687][SQL] Fix JDBC metrics counter data type No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (888cc46 -> 8a8ac00)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 888cc46 [SPARK-29675][SQL] Add exception when isolationLevel is Illegal add 8a8ac00 [SPARK-29687][SQL] Fix JDBC metrics counter data type No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8a8ac00 -> 3175f4b)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8a8ac00 [SPARK-29687][SQL] Fix JDBC metrics counter data type add 3175f4b [SPARK-29664][PYTHON][SQL] Column.getItem behavior is not consistent with Scala No new revisions were added by this update. Summary of changes: docs/pyspark-migration-guide.md | 3 +++ python/pyspark/sql/column.py| 12 +--- python/pyspark/sql/tests/test_column.py | 14 +- 3 files changed, 21 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3175f4b -> ae7450d)
This is an automated email from the ASF dual-hosted git repository. viirya pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3175f4b [SPARK-29664][PYTHON][SQL] Column.getItem behavior is not consistent with Scala add ae7450d [SPARK-29676][SQL] ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands No new revisions were added by this update. Summary of changes: .../apache/spark/sql/catalyst/parser/SqlBase.g4| 2 +- .../spark/sql/catalyst/parser/AstBuilder.scala | 16 + .../sql/catalyst/plans/logical/statements.scala| 8 +++ .../spark/sql/catalyst/parser/DDLParserSuite.scala | 26 ++ .../catalyst/analysis/ResolveSessionCatalog.scala | 9 +++- .../spark/sql/execution/SparkSqlParser.scala | 16 - .../spark/sql/connector/DataSourceV2SQLSuite.scala | 17 +++--- .../sql/execution/command/DDLParserSuite.scala | 14 8 files changed, 73 insertions(+), 35 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (ae7450d -> 8a4378c)
This is an automated email from the ASF dual-hosted git repository. ruifengz pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from ae7450d [SPARK-29676][SQL] ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands add 8a4378c [SPARK-29686][ML] LinearSVC should persist instances if needed No new revisions were added by this update. Summary of changes: .../main/scala/org/apache/spark/ml/classification/LinearSVC.scala | 6 ++ 1 file changed, 6 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
