spark git commit: [SPARK-25591][PYSPARK][SQL] Avoid overwriting deserialized accumulator

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 219922422 -> cb90617f8 [SPARK-25591][PYSPARK][SQL] Avoid overwriting deserialized accumulator ## What changes were proposed in this pull request? If we use accumulators in more than one UDFs, it is possible to overwrite deserialized

spark git commit: [SPARK-25673][BUILD] Remove Travis CI which enables Java lint check

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.4 c8b94099a -> 4214ddd34 [SPARK-25673][BUILD] Remove Travis CI which enables Java lint check ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/12980 added Travis CI file mainly for linter because

spark git commit: [SPARK-25673][BUILD] Remove Travis CI which enables Java lint check

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master ebd899b8a -> 219922422 [SPARK-25673][BUILD] Remove Travis CI which enables Java lint check ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/12980 added Travis CI file mainly for linter because we

spark git commit: [SPARK-25591][PYSPARK][SQL] Avoid overwriting deserialized accumulator

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.4 4214ddd34 -> 692ddb3f9 [SPARK-25591][PYSPARK][SQL] Avoid overwriting deserialized accumulator ## What changes were proposed in this pull request? If we use accumulators in more than one UDFs, it is possible to overwrite deserialized

spark git commit: [SPARK-25677][DOC] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.4 692ddb3f9 -> 193ce77fc [SPARK-25677][DOC] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception ## What changes were proposed in this pull request? Documentation is updated

spark git commit: [SPARK-25677][DOC] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master cb90617f8 -> 1a6815cd9 [SPARK-25677][DOC] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception ## What changes were proposed in this pull request? Documentation is updated with

svn commit: r29935 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_08_02_02-193ce77-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Mon Oct 8 09:17:31 2018 New Revision: 29935 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_08_02_02-193ce77 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25666][PYTHON] Internally document type conversion between Python data and SQL types in normal UDFs

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 1a6815cd9 -> a853a8020 [SPARK-25666][PYTHON] Internally document type conversion between Python data and SQL types in normal UDFs ### What changes were proposed in this pull request? We are facing some problems about type conversions

svn commit: r29945 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_08_06_39-a853a80-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Mon Oct 8 13:54:02 2018 New Revision: 29945 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_08_06_39-a853a80 docs [This commit notification would consist of 1481 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25408] Move to more ideomatic Java8

2018-10-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master a853a8020 -> 1a2862535 [SPARK-25408] Move to more ideomatic Java8 While working on another PR, I noticed that there is quite some legacy Java in there that can be beautified. For example the use of features from Java8, such as: -

svn commit: r29946 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_08_08_02-1a28625-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Mon Oct 8 15:16:48 2018 New Revision: 29946 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_08_08_02-1a28625 docs [This commit notification would consist of 1481 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25641] Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

2018-10-08 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 1a2862535 -> 6353425af [SPARK-25641] Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100 ## What changes were proposed in this pull request? We want to change the default percentage to 100 for

spark git commit: [SPARK-25623][SPARK-25624][SPARK-25625][TEST] Reduce test time of LogisticRegressionSuite

2018-10-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master f3fed2823 -> a4b14a9cf [SPARK-25623][SPARK-25624][SPARK-25625][TEST] Reduce test time of LogisticRegressionSuite ...with intercept with L1 regularization ## What changes were proposed in this pull request? In the test, "multinomial

svn commit: r29960 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_08_18_02-4baa4d4-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Tue Oct 9 01:17:03 2018 New Revision: 29960 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_08_18_02-4baa4d4 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25659][PYTHON][TEST] Test type inference specification for createDataFrame in PySpark

2018-10-08 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master f9935a3f8 -> f3fed2823 [SPARK-25659][PYTHON][TEST] Test type inference specification for createDataFrame in PySpark ## What changes were proposed in this pull request? This PR proposes to specify type inference and simple e2e tests.

spark git commit: [SPARK-25639][DOCS] Added docs for foreachBatch, python foreach and multiple watermarks

2018-10-08 Thread tdas
Repository: spark Updated Branches: refs/heads/master 6a60fb0aa -> f9935a3f8 [SPARK-25639][DOCS] Added docs for foreachBatch, python foreach and multiple watermarks ## What changes were proposed in this pull request? Added - Python foreach - Scala, Java and Python foreachBatch - Multiple

svn commit: r29957 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_08_16_02-f9935a3-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Mon Oct 8 23:16:39 2018 New Revision: 29957 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_08_16_02-f9935a3 docs [This commit notification would consist of 1481 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25639][DOCS] Added docs for foreachBatch, python foreach and multiple watermarks

2018-10-08 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.4 193ce77fc -> 4baa4d42a [SPARK-25639][DOCS] Added docs for foreachBatch, python foreach and multiple watermarks ## What changes were proposed in this pull request? Added - Python foreach - Scala, Java and Python foreachBatch -

svn commit: r29961 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_08_20_04-a4b14a9-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Tue Oct 9 03:18:40 2018 New Revision: 29961 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_08_20_04-a4b14a9 docs [This commit notification would consist of 1481 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest

2018-10-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 6353425af -> 6a60fb0aa [SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest ## What changes were proposed in this pull request? There was 5 suites extends `HadoopFsRelationTest`, for testing "orc"/"parquet"/"text"/"json" data

svn commit: r29949 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_08_12_03-6353425-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-08 Thread pwendell
Author: pwendell Date: Mon Oct 8 19:17:26 2018 New Revision: 29949 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_08_12_03-6353425 docs [This commit notification would consist of 1481 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]