Author: pwendell
Date: Tue Aug 7 03:16:00 2018
New Revision: 28585
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_06_20_02-51bee7a docs
[This commit notification would consist of 1473 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 51bee7aca -> 4446a0b0d
[SPARK-23914][SQL][FOLLOW-UP] refactor ArrayUnion
## What changes were proposed in this pull request?
This PR refactors `ArrayUnion` based on [this
Repository: spark
Updated Branches:
refs/heads/master 18b6ec147 -> 51bee7aca
[SPARK-25018][INFRA] Use `Co-authored-by` and `Signed-off-by` git trailer in
`merge_spark_pr.py`
## What changes were proposed in this pull request?
In [Linux
Repository: spark
Updated Branches:
refs/heads/master 6afe6f32c -> 18b6ec147
[SPARK-24748][SS] Support for reporting custom metrics via StreamingQuery
Progress
## What changes were proposed in this pull request?
Currently the Structured Streaming sources and sinks does not have a way to
Repository: spark
Updated Branches:
refs/heads/master 1076e4f00 -> 6afe6f32c
[SPARK-24637][SS] Add metrics regarding state and watermark to dropwizard
metrics
## What changes were proposed in this pull request?
The patch adds metrics regarding state and watermark to dropwizard metrics, so
Repository: spark
Updated Branches:
refs/heads/master 0f3fa2f28 -> 1076e4f00
[MINOR][DOCS] Fix grammatical error in SortShuffleManager
## What changes were proposed in this pull request?
Fix a grammatical error in the comment of SortShuffleManager.
## How was this patch tested?
N/A
Closes
Repository: spark
Updated Branches:
refs/heads/master 408a3ff2c -> 0f3fa2f28
[SPARK-24996][SQL] Use DSL in DeclarativeAggregate
## What changes were proposed in this pull request?
The PR refactors the aggregate expressions which were not using DSL in order to
simplify them.
## How was this
Repository: spark
Updated Branches:
refs/heads/master 87ca7396c -> 408a3ff2c
[SPARK-25036][SQL] Should compare ExprValue.isNull with LiteralTrue/LiteralFalse
## What changes were proposed in this pull request?
This PR fixes a comparison of `ExprValue.isNull` with `String`.
Author: pwendell
Date: Mon Aug 6 23:16:08 2018
New Revision: 28584
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_06_16_02-87ca739 docs
[This commit notification would consist of 1470 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 3c96937c7 -> 87ca7396c
[SPARK-24161][SS] Enable debug package feature on structured streaming
## What changes were proposed in this pull request?
Currently, debug package has a implicit class "DebugQuery" which matches
Dataset to provide
Repository: spark
Updated Branches:
refs/heads/master 278984d5a -> 3c96937c7
[SPARK-24948][SHS] Delegate check access permissions to the file system
## What changes were proposed in this pull request?
In `SparkHadoopUtil. checkAccessPermission`, we consider only basic
permissions in order
Author: pwendell
Date: Mon Aug 6 19:16:08 2018
New Revision: 28578
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_06_12_01-278984d docs
[This commit notification would consist of 1470 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 51e2b38d9 -> 278984d5a
[SPARK-25019][BUILD] Fix orc dependency to use the same exclusion rules
## What changes were proposed in this pull request?
During upgrading Apache ORC to 1.5.2
Repository: spark
Updated Branches:
refs/heads/master 1a5e46076 -> 51e2b38d9
[SPARK-24992][CORE] spark should randomize yarn local dir selection
**Description:
[SPARK-24992](https://issues.apache.org/jira/browse/SPARK-24992)**
Utils.getLocalDir is used to get path of a temporary directory.
Author: pwendell
Date: Mon Aug 6 15:16:12 2018
New Revision: 28575
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_06_08_01-1a5e460 docs
[This commit notification would consist of 1470 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master 35700bb7f -> 1a5e46076
[SPARK-23913][SQL] Add array_intersect function
## What changes were proposed in this pull request?
The PR adds the SQL function `array_intersect`. The behavior of the function is
based on Presto's one.
This
Repository: spark
Updated Branches:
refs/heads/master c1760da5d -> 35700bb7f
[SPARK-24981][CORE] ShutdownHook timeout causes job to fail when succeeded when
SparkContext stop() not called by user program
**Description**
The issue is described in
Author: pwendell
Date: Mon Aug 6 13:59:00 2018
New Revision: 28572
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_08_06_06_39-c1760da docs
[This commit notification would consist of 1470 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
Repository: spark
Updated Branches:
refs/heads/master d063e3a47 -> c1760da5d
[SPARK-25025][SQL] Remove the default value of isAll in INTERSECT/EXCEPT
## What changes were proposed in this pull request?
Having the default value of isAll in the logical plan nodes INTERSECT/EXCEPT
could
Repository: spark
Updated Branches:
refs/heads/master 64ad7b841 -> d063e3a47
[SPARK-24940][SQL] Use IntegerLiteral in ResolveCoalesceHints
## What changes were proposed in this pull request?
Follow up to fix an unmerged review comment.
## How was this patch tested?
Unit test
Repository: spark
Updated Branches:
refs/heads/master ac527b520 -> 64ad7b841
[SPARK-23772][FOLLOW-UP][SQL] Provide an option to ignore column of all null
values or empty array during JSON schema inference
## What changes were proposed in this pull request?
The `dropFieldIfAllNull` parameter
Repository: spark
Updated Branches:
refs/heads/master 327bb3007 -> ac527b520
[SPARK-24991][SQL] use InternalRow in DataSourceWriter
## What changes were proposed in this pull request?
A follow up of #21118
Since we use `InternalRow` in the read API of data source v2, we should do the
same
22 matches
Mail list logo
