spark git commit: [SPARK-3685][CORE] Prints explicit warnings when configured local directories are set to URIs

2017-12-12 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master ecc179eca -> bc8933faf [SPARK-3685][CORE] Prints explicit warnings when configured local directories are set to URIs ## What changes were proposed in this pull request? This PR proposes to print warnings before creating local by `java.io.

svn commit: r23600 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_12_00_01-ecc179e-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-12 Thread pwendell
Author: pwendell Date: Tue Dec 12 08:14:38 2017 New Revision: 23600 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_12_00_01-ecc179e docs [This commit notification would consist of 1406 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r23618 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_12_04_01-bc8933f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-12 Thread pwendell
Author: pwendell Date: Tue Dec 12 12:16:10 2017 New Revision: 23618 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_12_04_01-bc8933f docs [This commit notification would consist of 1406 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-16986][WEB-UI] Converter Started, Completed and Last Updated to client time zone in history page

2017-12-12 Thread vanzin
Repository: spark Updated Branches: refs/heads/master bc8933faf -> d5007734b [SPARK-16986][WEB-UI] Converter Started, Completed and Last Updated to client time zone in history page ## What changes were proposed in this pull request? This PR is converted the ` Started`, `Completed` and `Last

spark git commit: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDialect

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master d5007734b -> e6dc5f280 [SPARK-22729][SQL] Add getTruncateQuery to JdbcDialect In order to enable truncate for PostgreSQL databases in Spark JDBC, a change is needed to the query used for truncating a PostgreSQL table. By default, PostgreS

spark git commit: [SPARK-22289][ML] Add JSON support for Matrix parameters (LR with coefficients bound)

2017-12-12 Thread yliang
Repository: spark Updated Branches: refs/heads/master e6dc5f280 -> 10c27a655 [SPARK-22289][ML] Add JSON support for Matrix parameters (LR with coefficients bound) ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-22289 add JSON encoding/dec

spark git commit: [SPARK-22289][ML] Add JSON support for Matrix parameters (LR with coefficients bound)

2017-12-12 Thread yliang
Repository: spark Updated Branches: refs/heads/branch-2.2 9e2d96d1d -> 00cdb38dc [SPARK-22289][ML] Add JSON support for Matrix parameters (LR with coefficients bound) ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-22289 add JSON encoding

spark git commit: [SPARK-22574][MESOS][SUBMIT] Check submission request parameters

2017-12-12 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 10c27a655 -> 7a51e7135 [SPARK-22574][MESOS][SUBMIT] Check submission request parameters ## What changes were proposed in this pull request? It solves the problem when submitting a wrong CreateSubmissionRequest to Spark Dispatcher was caus

spark git commit: [SPARK-22574][MESOS][SUBMIT] Check submission request parameters

2017-12-12 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 00cdb38dc -> 728a45e5a [SPARK-22574][MESOS][SUBMIT] Check submission request parameters ## What changes were proposed in this pull request? It solves the problem when submitting a wrong CreateSubmissionRequest to Spark Dispatcher was

svn commit: r23675 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_12_12_01-7a51e71-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-12 Thread pwendell
Author: pwendell Date: Tue Dec 12 20:15:02 2017 New Revision: 23675 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_12_12_01-7a51e71 docs [This commit notification would consist of 1406 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: Revert "[SPARK-22574][MESOS][SUBMIT] Check submission request parameters"

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 7a51e7135 -> 704af4bd6 Revert "[SPARK-22574][MESOS][SUBMIT] Check submission request parameters" This reverts commit 7a51e71355485bb176a1387d99ec430c5986cbec. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-w

spark git commit: Revert "[SPARK-22574][MESOS][SUBMIT] Check submission request parameters"

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 728a45e5a -> 0230515a2 Revert "[SPARK-22574][MESOS][SUBMIT] Check submission request parameters" This reverts commit 728a45e5a68a20bdd17227edc70e6a38d178af1c. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://g

spark git commit: [SPARK-19809][SQL][TEST] NullPointerException on zero-size ORC file

2017-12-12 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 704af4bd6 -> 17cdabb88 [SPARK-19809][SQL][TEST] NullPointerException on zero-size ORC file ## What changes were proposed in this pull request? Until 2.2.1, Spark raises `NullPointerException` on zero-size ORC files. Usually, these zero-si

svn commit: r23692 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_12_16_01-17cdabb-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-12 Thread pwendell
Author: pwendell Date: Wed Dec 13 00:14:38 2017 New Revision: 23692 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_12_16_01-17cdabb docs [This commit notification would consist of 1406 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-21087][ML][FOLLOWUP] Sync SharedParamsCodeGen and sharedParams.

2017-12-12 Thread yliang
Repository: spark Updated Branches: refs/heads/master 17cdabb88 -> b03af8b58 [SPARK-21087][ML][FOLLOWUP] Sync SharedParamsCodeGen and sharedParams. ## What changes were proposed in this pull request? #19208 modified ```sharedParams.scala```, but didn't generated by ```SharedParamsCodeGen.scal

spark git commit: [SPARK-22716][SQL] Avoid the creation of mutable states in addReferenceObj

2017-12-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master b03af8b58 -> 4117786a8 [SPARK-22716][SQL] Avoid the creation of mutable states in addReferenceObj ## What changes were proposed in this pull request? We have two methods to reference an object `addReferenceMinorObj` and `addReferenceObj `

spark git commit: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expressions under wholestage codegen

2017-12-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4117786a8 -> c7d014861 [SPARK-22600][SQL] Fix 64kb limit for deeply nested expressions under wholestage codegen ## What changes were proposed in this pull request? SPARK-22543 fixes the 64kb compile error for deeply nested expression for

svn commit: r23698 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_12_20_01-c7d0148-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-12 Thread pwendell
Author: pwendell Date: Wed Dec 13 04:14:36 2017 New Revision: 23698 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_12_20_01-c7d0148 docs [This commit notification would consist of 1407 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22644][ML][TEST] Make ML testsuite support StructuredStreaming test

2017-12-12 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master c7d014861 -> 0e36ba621 [SPARK-22644][ML][TEST] Make ML testsuite support StructuredStreaming test ## What changes were proposed in this pull request? We need to add some helper code to make testing ML transformers & models easier with str

spark git commit: [SPARK-19809][SQL][TEST][FOLLOWUP] Move the test case to HiveOrcQuerySuite

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0e36ba621 -> 6b80ce4fb [SPARK-19809][SQL][TEST][FOLLOWUP] Move the test case to HiveOrcQuerySuite ## What changes were proposed in this pull request? As a follow-up of #19948 , this PR moves the test case and adds comments. ## How was thi

spark git commit: [SPARK-22759][SQL] Filters can be combined iff both are deterministic

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 6b80ce4fb -> 13e489b67 [SPARK-22759][SQL] Filters can be combined iff both are deterministic ## What changes were proposed in this pull request? The query execution/optimization does not guarantee the expressions are evaluated in order. We

spark git commit: [SPARK-21322][SQL][FOLLOWUP] support histogram in filter cardinality estimation

2017-12-12 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 13e489b67 -> bdb5e55c2 [SPARK-21322][SQL][FOLLOWUP] support histogram in filter cardinality estimation ## What changes were proposed in this pull request? some code cleanup/refactor and naming improvement. ## How was this patch tested? e

spark git commit: [SPARK-22700][ML] Bucketizer.transform incorrectly drops row containing NaN

2017-12-12 Thread mlnick
Repository: spark Updated Branches: refs/heads/master bdb5e55c2 -> 874350905 [SPARK-22700][ML] Bucketizer.transform incorrectly drops row containing NaN ## What changes were proposed in this pull request? only drops the rows containing NaN in the input columns ## How was this patch tested? ex

spark git commit: [SPARK-22042][SQL] ReorderJoinPredicates can break when child's partitioning is not decided

2017-12-12 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 874350905 -> 682eb4f2e [SPARK-22042][SQL] ReorderJoinPredicates can break when child's partitioning is not decided ## What changes were proposed in this pull request? See jira description for the bug : https://issues.apache.org/jira/brow