spark git commit: [SPARK-21248][SS] The clean up codes in StreamExecution should not be interrupted

2017-07-05 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master c8d0aba19 -> ab866f117 [SPARK-21248][SS] The clean up codes in StreamExecution should not be interrupted ## What changes were proposed in this pull request? This PR uses `runUninterruptibly` to avoid that the clean up codes in

spark git commit: [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6

2017-07-05 Thread holden
Repository: spark Updated Branches: refs/heads/master c8e7f445b -> c8d0aba19 [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6 ## What changes were proposed in this pull request? This PR aims to bump Py4J in order to fix the following float/double bug. Py4J 0.10.5 fixes this

spark git commit: [SPARK-21307][SQL] Remove SQLConf parameters from the parser-related classes.

2017-07-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 742da0868 -> c8e7f445b [SPARK-21307][SQL] Remove SQLConf parameters from the parser-related classes. ### What changes were proposed in this pull request? This PR is to remove SQLConf parameters from the parser-related classes. ### How was

spark git commit: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFunction Should Support UDAFs

2017-07-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 960298ee6 -> 742da0868 [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFunction Should Support UDAFs ## What changes were proposed in this pull request? Support register Java UDAFs in PySpark so that user can use Java UDAF in PySpark.

spark git commit: [SPARK-20858][DOC][MINOR] Document ListenerBus event queue size

2017-07-05 Thread srowen
Repository: spark Updated Branches: refs/heads/master e3e2b5da3 -> 960298ee6 [SPARK-20858][DOC][MINOR] Document ListenerBus event queue size ## What changes were proposed in this pull request? This change adds a new configuration option `spark.scheduler.listenerbus.eventqueue.size` to the

spark git commit: [SPARK-21286][TEST] Modified StorageTabSuite unit test

2017-07-05 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 5787ace46 -> e3e2b5da3 [SPARK-21286][TEST] Modified StorageTabSuite unit test ## What changes were proposed in this pull request? The old unit test not effect ## How was this patch tested? unit test Author: he.qiao

spark git commit: [SPARK-20383][SQL] Supporting Create [temporary] Function with the keyword 'OR REPLACE' and 'IF NOT EXISTS'

2017-07-05 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 873f3ad2b -> 5787ace46 [SPARK-20383][SQL] Supporting Create [temporary] Function with the keyword 'OR REPLACE' and 'IF NOT EXISTS' ## What changes were proposed in this pull request? support to create [temporary] function with the

spark git commit: [SPARK-16167][SQL] RowEncoder should preserve array/map type nullability.

2017-07-05 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4852b7d44 -> 873f3ad2b [SPARK-16167][SQL] RowEncoder should preserve array/map type nullability. ## What changes were proposed in this pull request? Currently `RowEncoder` doesn't preserve nullability of `ArrayType` or `MapType`. It

spark git commit: [SPARK-21310][ML][PYSPARK] Expose offset in PySpark

2017-07-05 Thread yliang
Repository: spark Updated Branches: refs/heads/master a38643256 -> 4852b7d44 [SPARK-21310][ML][PYSPARK] Expose offset in PySpark ## What changes were proposed in this pull request? Add offset to PySpark in GLM as in #16699. ## How was this patch tested? Python test Author: actuaryzhang

spark git commit: [SPARK-21304][SQL] remove unnecessary isNull variable for collection related encoder expressions

2017-07-05 Thread wenchen
Repository: spark Updated Branches: refs/heads/master e9a93f814 -> f2c3b1dd6 [SPARK-21304][SQL] remove unnecessary isNull variable for collection related encoder expressions ## What changes were proposed in this pull request? For these collection-related encoder expressions, we don't need