[spark] branch master updated (4559a82 -> eef5e6d)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4559a82 [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients add eef5e6d [SPARK-29113][DOC] Fix some annotation errors and remove meaningless annotations in project No new revisions were added by this update. Summary of changes: .../main/java/org/apache/spark/io/NioBufferedFileInputStream.java | 1 - core/src/main/java/org/apache/spark/memory/MemoryConsumer.java | 1 - .../java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java | 1 - .../spark/util/collection/unsafe/sort/UnsafeExternalSorter.java| 2 -- .../scala/org/apache/spark/deploy/history/ApplicationCache.scala | 7 +++ .../main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala | 1 - core/src/main/scala/org/apache/spark/storage/BlockManager.scala| 1 - .../apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala | 1 - .../main/scala/org/apache/spark/sql/execution/ExplainUtils.scala | 4 ++-- 9 files changed, 5 insertions(+), 14 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3ece8ee -> 4559a82)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3ece8ee [SPARK-29124][CORE] Use MurmurHash3 `bytesHash(data, seed)` instead of `bytesHash(data)` add 4559a82 [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/catalyst/catalog/interface.scala| 7 +-- .../scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (05988b2 -> 3ece8ee)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 05988b2 [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas UDFs add 3ece8ee [SPARK-29124][CORE] Use MurmurHash3 `bytesHash(data, seed)` instead of `bytesHash(data)` No new revisions were added by this update. Summary of changes: core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (197732e -> 05988b2)
This is an automated email from the ASF dual-hosted git repository.
cutlerb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 197732e [SPARK-29125][INFRA] Add Hadoop 2.7 combination to GitHub
Action
add 05988b2 [SPARK-27463][PYTHON] Support Dataframe Cogroup via Pandas
UDFs
No new revisions were added by this update.
Summary of changes:
.../org/apache/spark/api/python/PythonRunner.scala | 2 +
python/pyspark/rdd.py | 1 +
python/pyspark/serializers.py | 26 ++
python/pyspark/sql/cogroup.py | 98
python/pyspark/sql/functions.py| 5 +-
python/pyspark/sql/group.py| 12 +-
.../sql/tests/test_pandas_udf_cogrouped_map.py | 280 +
python/pyspark/sql/udf.py | 19 ++
python/pyspark/worker.py | 98 +++-
.../spark/sql/catalyst/analysis/Analyzer.scala | 6 +
.../plans/logical/pythonLogicalOperators.scala | 19 +-
.../spark/sql/RelationalGroupedDataset.scala | 46 +++-
.../spark/sql/execution/SparkStrategies.scala | 3 +
.../sql/execution/python/ArrowPythonRunner.scala | 76 +-
...honRunner.scala => BaseArrowPythonRunner.scala} | 92 +--
.../sql/execution/python/BasePandasGroupExec.scala | 137 ++
.../python/CogroupedArrowPythonRunner.scala| 113 +
.../python/FlatMapCoGroupsInPandasExec.scala | 97 +++
.../python/FlatMapGroupsInPandasExec.scala | 98 +---
19 files changed, 965 insertions(+), 263 deletions(-)
create mode 100644 python/pyspark/sql/cogroup.py
create mode 100644 python/pyspark/sql/tests/test_pandas_udf_cogrouped_map.py
copy
sql/core/src/main/scala/org/apache/spark/sql/execution/python/{ArrowPythonRunner.scala
=> BaseArrowPythonRunner.scala} (51%)
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/python/BasePandasGroupExec.scala
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/python/CogroupedArrowPythonRunner.scala
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapCoGroupsInPandasExec.scala
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (71e7516 -> 197732e)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 71e7516 [SPARK-29027][TESTS] KafkaDelegationTokenSuite fix when loopback canonical host name differs from localhost add 197732e [SPARK-29125][INFRA] Add Hadoop 2.7 combination to GitHub Action No new revisions were added by this update. Summary of changes: .github/workflows/master.yml | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (02db706 -> 71e7516)
This is an automated email from the ASF dual-hosted git repository. vanzin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 02db706 [SPARK-29115][SQL][TEST] Add benchmarks for make_date() and make_timestamp() add 71e7516 [SPARK-29027][TESTS] KafkaDelegationTokenSuite fix when loopback canonical host name differs from localhost No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/kafka010/KafkaTestUtils.scala | 17 ++--- 1 file changed, 10 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (dd32476 -> 02db706)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from dd32476 [SPARK-28792][SQL][DOC] Document CREATE DATABASE statement in SQL Reference add 02db706 [SPARK-29115][SQL][TEST] Add benchmarks for make_date() and make_timestamp() No new revisions were added by this update. Summary of changes: .../benchmarks/MakeDateTimeBenchmark-results.txt | 22 .../benchmark/MakeDateTimeBenchmark.scala | 120 + 2 files changed, 142 insertions(+) create mode 100644 sql/core/benchmarks/MakeDateTimeBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MakeDateTimeBenchmark.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3fc52b5 -> c6ca661)
This is an automated email from the ASF dual-hosted git repository. lixiao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3fc52b5 [SPARK-28950][SQL] Refine the code of DELETE add c6ca661 [SPARK-28814][SQL][DOC] Document SET/RESET in SQL Reference No new revisions were added by this update. Summary of changes: docs/sql-ref-syntax-aux-conf-mgmt-reset.md | 18 ++- docs/sql-ref-syntax-aux-conf-mgmt-set.md | 49 +- 2 files changed, 65 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (34915b2 -> 3fc52b5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 34915b2 [SPARK-29104][CORE][TESTS] Fix PipedRDDSuite to use `eventually` to check thread termination add 3fc52b5 [SPARK-28950][SQL] Refine the code of DELETE No new revisions were added by this update. Summary of changes: .../apache/spark/sql/catalyst/parser/SqlBase.g4| 2 +- .../spark/sql/catalyst/parser/AstBuilder.scala | 14 +-- .../plans/logical/basicLogicalOperators.scala | 2 +- .../plans/logical/sql/DeleteFromStatement.scala| 2 +- .../spark/sql/catalyst/parser/DDLParserSuite.scala | 29 -- .../datasources/v2/DataSourceV2Strategy.scala | 13 +- .../spark/sql/connector/DataSourceV2SQLSuite.scala | 27 +--- 7 files changed, 73 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (104b9b6 -> 34915b2)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 104b9b6 [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite add 34915b2 [SPARK-29104][CORE][TESTS] Fix PipedRDDSuite to use `eventually` to check thread termination No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/rdd/PipedRDDSuite.scala| 19 +-- 1 file changed, 13 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (79b10a1 -> 104b9b6)
This is an automated email from the ASF dual-hosted git repository. weichenxu123 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 79b10a1 [SPARK-28929][CORE] Spark Logging level should be INFO instead of DEBUG in Executor Plugin API add 104b9b6 [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite No new revisions were added by this update. Summary of changes: .../spark/scheduler/BarrierTaskContextSuite.scala | 72 +- 1 file changed, 30 insertions(+), 42 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (db996cc -> 79b10a1)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from db996cc [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` add 79b10a1 [SPARK-28929][CORE] Spark Logging level should be INFO instead of DEBUG in Executor Plugin API No new revisions were added by this update. Summary of changes: core/src/main/scala/org/apache/spark/executor/Executor.scala | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated (1c57da3 -> 68e29ba)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git. from 1c57da3 [SPARK-25277][YARN] YARN applicationMaster metrics should not register static metrics add 68e29ba [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping No new revisions were added by this update. Summary of changes: .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c862835 -> db996cc)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c862835 [SPARK-28996][SQL][TESTS] Add tests regarding username of HiveClient add db996cc [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` No new revisions were added by this update. Summary of changes: .../catalyst/expressions/datetimeExpressions.scala | 32 -- sql/core/benchmarks/DateTimeBenchmark-results.txt | 4 +-- 2 files changed, 26 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4d27a25 -> c862835)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 4d27a25 [SPARK-22797][ML][PYTHON] Bucketizer support multi-column
add c862835 [SPARK-28996][SQL][TESTS] Add tests regarding username of
HiveClient
No new revisions were added by this update.
Summary of changes:
.../apache/spark/sql/hive/client/HiveClient.scala | 2 +
.../spark/sql/hive/client/HiveClientImpl.scala | 2 +-
.../sql/hive/client/HiveClientUserNameSuite.scala | 63 ++
...Suites.scala => HiveClientUserNameSuites.scala} | 5 +-
...ite.scala => HivePartitionFilteringSuite.scala} | 3 +-
...es.scala => HivePartitionFilteringSuites.scala} | 4 +-
6 files changed, 71 insertions(+), 8 deletions(-)
create mode 100644
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientUserNameSuite.scala
copy
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/{HiveClientSuites.scala
=> HiveClientUserNameSuites.scala} (83%)
rename
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/{HiveClientSuite.scala
=> HivePartitionFilteringSuite.scala} (99%)
rename
sql/hive/src/test/scala/org/apache/spark/sql/hive/client/{HiveClientSuites.scala
=> HivePartitionFilteringSuites.scala} (87%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
