Author: pwendell
Date: Wed Jan 31 08:16:05 2018
New Revision: 24592
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_01_31_00_01-695f714 docs
[This commit notification would consist of 1443 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master 695f7146b -> 161a3f2ae
[SPARK-23112][DOC] Update ML migration guide with breaking and behavior changes.
Add breaking changes, as well as update behavior changes, to `2.3` ML migration
guide.
## How was this patch tested?
Doc only
Author
Repository: spark
Updated Branches:
refs/heads/branch-2.3 7ec8ad7ba -> c83246c9a
[SPARK-23112][DOC] Update ML migration guide with breaking and behavior changes.
Add breaking changes, as well as update behavior changes, to `2.3` ML migration
guide.
## How was this patch tested?
Doc only
Au
Author: pwendell
Date: Wed Jan 31 10:16:04 2018
New Revision: 24596
Log:
Apache Spark 2.3.1-SNAPSHOT-2018_01_31_02_01-c83246c docs
[This commit notification would consist of 1442 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master 161a3f2ae -> 3d0911bbe
[SPARK-23228][PYSPARK] Add Python Created jsparkSession to JVM's defaultSession
## What changes were proposed in this pull request?
In the current PySpark code, Python created `jsparkSession` doesn't add to
JVM's de
Author: pwendell
Date: Wed Jan 31 12:20:49 2018
New Revision: 24599
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_01_31_04_01-3d0911b docs
[This commit notification would consist of 1443 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master 3d0911bbe -> 48dd6a4c7
revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSet
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/19980 , we thought `anyNullsSet` can be
simply implemented by `nu
Repository: spark
Updated Branches:
refs/heads/branch-2.3 c83246c9a -> 33f17b28b
revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSet
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/19980 , we thought `anyNullsSet` can be
simply implemented by
Repository: spark
Updated Branches:
refs/heads/master 48dd6a4c7 -> 8c21170de
[SPARK-23249][SQL] Improved block merging logic for partitions
## What changes were proposed in this pull request?
Change DataSourceScanExec so that when grouping blocks together into
partitions, also checks the end
Repository: spark
Updated Branches:
refs/heads/branch-2.3 33f17b28b -> f5f21e8c4
[SPARK-23249][SQL] Improved block merging logic for partitions
## What changes were proposed in this pull request?
Change DataSourceScanExec so that when grouping blocks together into
partitions, also checks the
Repository: spark
Updated Branches:
refs/heads/master 8c21170de -> dd242bad3
[SPARK-21525][STREAMING] Check error code from supervisor RPC.
The code was ignoring the error code from the AddBlock RPC, which
means that a failure to write to the WAL was being ignored by the
receiver, and would le
Author: pwendell
Date: Wed Jan 31 20:16:23 2018
New Revision: 24605
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_01_31_12_01-dd242ba docs
[This commit notification would consist of 1443 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master dd242bad3 -> 9ff1d96f0
[SPARK-23281][SQL] Query produces results in incorrect order when a composite
order by clause refers to both original columns and aliases
## What changes were proposed in this pull request?
Here is the test snippet.
Repository: spark
Updated Branches:
refs/heads/branch-2.3 f5f21e8c4 -> 8ee3a71c9
[SPARK-23281][SQL] Query produces results in incorrect order when a composite
order by clause refers to both original columns and aliases
## What changes were proposed in this pull request?
Here is the test snipp
Repository: spark
Updated Branches:
refs/heads/branch-2.2 0e58fee9d -> 5273cc791
[SPARK-23281][SQL] Query produces results in incorrect order when a composite
order by clause refers to both original columns and aliases
## What changes were proposed in this pull request?
Here is the test snipp
Repository: spark
Updated Branches:
refs/heads/branch-2.2 5273cc791 -> cb73ecd2f
revert the removal of import in SPARK-23281
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cb73ecd2
Tree: http://git-wip-us.apache.org/repos
Author: pwendell
Date: Wed Jan 31 22:15:35 2018
New Revision: 24606
Log:
Apache Spark 2.3.1-SNAPSHOT-2018_01_31_14_01-8ee3a71 docs
[This commit notification would consist of 1442 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Author: pwendell
Date: Thu Feb 1 00:16:03 2018
New Revision: 24607
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_01_31_16_01-9ff1d96 docs
[This commit notification would consist of 1443 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master 9ff1d96f0 -> f470df2fc
[SPARK-23157][SQL][FOLLOW-UP] DataFrame -> SparkDataFrame in R comment
Author: Henry Robinson
Closes #20443 from henryr/SPARK-23157.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wi
Repository: spark
Updated Branches:
refs/heads/branch-2.3 8ee3a71c9 -> 7ccfc7530
[SPARK-23157][SQL][FOLLOW-UP] DataFrame -> SparkDataFrame in R comment
Author: Henry Robinson
Closes #20443 from henryr/SPARK-23157.
(cherry picked from commit f470df2fcf14e6234c577dc1bdfac27d49b441f5)
Signed-o
Repository: spark
Updated Branches:
refs/heads/master f470df2fc -> 52e00f706
[SPARK-23280][SQL] add map type support to ColumnVector
## What changes were proposed in this pull request?
Fill the last missing piece of `ColumnVector`: the map type support.
The idea is similar to the array type
Repository: spark
Updated Branches:
refs/heads/branch-2.3 7ccfc7530 -> 0d0f57936
[SPARK-23280][SQL] add map type support to ColumnVector
## What changes were proposed in this pull request?
Fill the last missing piece of `ColumnVector`: the map type support.
The idea is similar to the array t
Repository: spark
Updated Branches:
refs/heads/master 52e00f706 -> 2ac895be9
[SPARK-23247][SQL] combines Unsafe operations and statistics operations in Scan
Data Source
## What changes were proposed in this pull request?
Currently, we scan the execution plan of the data source, first the uns
Author: pwendell
Date: Thu Feb 1 04:15:49 2018
New Revision: 24611
Log:
Apache Spark 2.4.0-SNAPSHOT-2018_01_31_20_01-52e00f7 docs
[This commit notification would consist of 1444 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/branch-2.3 0d0f57936 -> 59e89a299
http://git-wip-us.apache.org/repos/asf/spark/blob/59e89a29/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/RateSourceProvider.scala
-
[SPARK-23268][SQL] Reorganize packages in data source V2
## What changes were proposed in this pull request?
1. create a new package for partitioning/distribution related classes.
As Spark will add new concrete implementations of `Distribution` in new
releases, it is good to
have a new pa
Repository: spark
Updated Branches:
refs/heads/master 2ac895be9 -> 56ae32657
http://git-wip-us.apache.org/repos/asf/spark/blob/56ae3265/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/RateSourceProvider.scala
-
[SPARK-23268][SQL] Reorganize packages in data source V2
## What changes were proposed in this pull request?
1. create a new package for partitioning/distribution related classes.
As Spark will add new concrete implementations of `Distribution` in new
releases, it is good to
have a new pa
Repository: spark
Updated Branches:
refs/heads/branch-2.3 59e89a299 -> 871fd48dc
[SPARK-21396][SQL] Fixes MatchError when UDTs are passed through Hive
Thriftserver
Signed-off-by: Atallah Hezbor
## What changes were proposed in this pull request?
This PR proposes modifying the match stateme
Repository: spark
Updated Branches:
refs/heads/master 56ae32657 -> b2e7677f4
[SPARK-21396][SQL] Fixes MatchError when UDTs are passed through Hive
Thriftserver
Signed-off-by: Atallah Hezbor
## What changes were proposed in this pull request?
This PR proposes modifying the match statement t
Repository: spark
Updated Branches:
refs/heads/master b2e7677f4 -> cc41245fa
[SPARK-23188][SQL] Make vectorized columar reader batch size configurable
## What changes were proposed in this pull request?
This PR include the following changes:
- Make the capacity of `VectorizedParquetRecordRead
Repository: spark
Updated Branches:
refs/heads/master cc41245fa -> b6b50efc8
[SQL][MINOR] Inline SpecifiedWindowFrame.defaultWindowFrame().
## What changes were proposed in this pull request?
SpecifiedWindowFrame.defaultWindowFrame(hasOrderSpecification,
acceptWindowFrame) was designed to ha
Repository: spark
Updated Branches:
refs/heads/master b6b50efc8 -> 4b7cd479a
Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore"
This reverts commit d1721816d26bedee3c72eeb75db49da500568376.
The patch is not fully tested and out-of-date. So revert it.
Project: http
Author: pwendell
Date: Thu Feb 1 06:16:10 2018
New Revision: 24614
Log:
Apache Spark 2.3.1-SNAPSHOT-2018_01_31_22_01-871fd48 docs
[This commit notification would consist of 1442 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary.]
---
Repository: spark
Updated Branches:
refs/heads/master 4b7cd479a -> 07cee3373
[SPARK-22274][PYTHON][SQL][FOLLOWUP] Use `assertRaisesRegexp` instead of
`assertRaisesRegex`.
## What changes were proposed in this pull request?
This is a follow-up pr of #19872 which uses `assertRaisesRegex` but i
35 matches
Mail list logo