spark git commit: [SPARK-23390][SQL] Flaky Test Suite: FileBasedDataSourceSuite in Spark 2.3/hadoop 2.7

2018-02-11 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 7e2a2b33c -> 79e8650cc [SPARK-23390][SQL] Flaky Test Suite: FileBasedDataSourceSuite in Spark 2.3/hadoop 2.7 ## What changes were proposed in this pull request? This test only fails with sbt on Hadoop 2.7, I can't reproduce it

spark git commit: [SPARK-23390][SQL] Flaky Test Suite: FileBasedDataSourceSuite in Spark 2.3/hadoop 2.7

2018-02-11 Thread sameerag
Repository: spark Updated Branches: refs/heads/master c0c902aed -> 6efd5d117 [SPARK-23390][SQL] Flaky Test Suite: FileBasedDataSourceSuite in Spark 2.3/hadoop 2.7 ## What changes were proposed in this pull request? This test only fails with sbt on Hadoop 2.7, I can't reproduce it locally,

svn commit: r24941 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_11_20_01-c0c902a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Mon Feb 12 04:15:01 2018 New Revision: 24941 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_11_20_01-c0c902a docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22119][FOLLOWUP][ML] Use spherical KMeans with cosine distance

2018-02-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4bbd7443e -> c0c902aed [SPARK-22119][FOLLOWUP][ML] Use spherical KMeans with cosine distance ## What changes were proposed in this pull request? In #19340 some comments considered needed to use spherical KMeans when cosine distance

svn commit: r24939 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_11_12_01-4bbd744-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Sun Feb 11 20:15:20 2018 New Revision: 24939 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_11_12_01-4bbd744 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24934 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_11_10_01-7e2a2b3-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Sun Feb 11 18:15:27 2018 New Revision: 24934 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_11_10_01-7e2a2b3 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail

2018-02-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 1b4c6ab44 -> 169483455 [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail This is a long-standing bug in `UnsafeKVExternalSorter` and was reported in the dev list multiple times. When creating

spark git commit: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail

2018-02-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 8875e47ce -> 7e2a2b33c [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail ## What changes were proposed in this pull request? This is a long-standing bug in `UnsafeKVExternalSorter` and was reported in

spark git commit: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail

2018-02-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/master eacb62fbb -> 4bbd7443e [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail ## What changes were proposed in this pull request? This is a long-standing bug in `UnsafeKVExternalSorter` and was reported in the

svn commit: r24929 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_11_06_01-8875e47-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Sun Feb 11 14:15:28 2018 New Revision: 24929 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_11_06_01-8875e47 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23387][SQL][PYTHON][TEST][BRANCH-2.3] Backport assertPandasEqual to branch-2.3.

2018-02-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 9fa7b0e10 -> 8875e47ce [SPARK-23387][SQL][PYTHON][TEST][BRANCH-2.3] Backport assertPandasEqual to branch-2.3. ## What changes were proposed in this pull request? When backporting a pr with tests using `assertPandasEqual` from master

svn commit: r24928 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_11_04_01-eacb62f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Sun Feb 11 12:21:36 2018 New Revision: 24928 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_11_04_01-eacb62f docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22624][PYSPARK] Expose range partitioning shuffle introduced by spark-22614

2018-02-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 8acb51f08 -> eacb62fbb [SPARK-22624][PYSPARK] Expose range partitioning shuffle introduced by spark-22614 ## What changes were proposed in this pull request? Expose range partitioning shuffle introduced by spark-22614 ## How was this

svn commit: r24927 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_11_02_01-9fa7b0e-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-02-11 Thread pwendell
Author: pwendell Date: Sun Feb 11 10:17:01 2018 New Revision: 24927 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_11_02_01-9fa7b0e docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23084][PYTHON] Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-02-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master a34fce19b -> 8acb51f08 [SPARK-23084][PYTHON] Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark ## What changes were proposed in this pull request? Added unboundedPreceding(), unboundedFollowing() and currentRow()

spark git commit: [SPARK-23314][PYTHON] Add ambiguous=False when localizing tz-naive timestamps in Arrow codepath to deal with dst

2018-02-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 b7571b9bf -> 9fa7b0e10 [SPARK-23314][PYTHON] Add ambiguous=False when localizing tz-naive timestamps in Arrow codepath to deal with dst ## What changes were proposed in this pull request? When tz_localize a tz-naive timetamp, pandas

spark git commit: [SPARK-23314][PYTHON] Add ambiguous=False when localizing tz-naive timestamps in Arrow codepath to deal with dst

2018-02-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 0783876c8 -> a34fce19b [SPARK-23314][PYTHON] Add ambiguous=False when localizing tz-naive timestamps in Arrow codepath to deal with dst ## What changes were proposed in this pull request? When tz_localize a tz-naive timetamp, pandas will