spark git commit: [SPARK-16606][MINOR] Tiny follow-up to , to correct more instances of the same log message typo

2016-10-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 d3c78c4f3 -> a0c03c925 [SPARK-16606][MINOR] Tiny follow-up to , to correct more instances of the same log message typo ## What changes were proposed in this pull request? Tiny follow-up to SPARK-16606 /

spark git commit: [SPARK-16606][MINOR] Tiny follow-up to , to correct more instances of the same log message typo

2016-10-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 3fbf5a58c -> 7178c5643 [SPARK-16606][MINOR] Tiny follow-up to , to correct more instances of the same log message typo ## What changes were proposed in this pull request? Tiny follow-up to SPARK-16606 /

spark git commit: [STREAMING][KAFKA][DOC] clarify kafka settings needed for larger batches

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 3e9840f1d -> d3c78c4f3 [STREAMING][KAFKA][DOC] clarify kafka settings needed for larger batches ## What changes were proposed in this pull request? Minor doc change to mention kafka configuration for larger spark batches. ## How was

spark git commit: [STREAMING][KAFKA][DOC] clarify kafka settings needed for larger batches

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 268ccb9a4 -> c9720b219 [STREAMING][KAFKA][DOC] clarify kafka settings needed for larger batches ## What changes were proposed in this pull request? Minor doc change to mention kafka configuration for larger spark batches. ## How was this

spark git commit: [SPARK-17812][SQL][KAFKA] Assign and specific startingOffsets for structured stream

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 b113b5d9f -> 3e9840f1d [SPARK-17812][SQL][KAFKA] Assign and specific startingOffsets for structured stream ## What changes were proposed in this pull request? startingOffsets takes specific per-topicpartition offsets as a json

spark git commit: [SPARK-17812][SQL][KAFKA] Assign and specific startingOffsets for structured stream

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 140570252 -> 268ccb9a4 [SPARK-17812][SQL][KAFKA] Assign and specific startingOffsets for structured stream ## What changes were proposed in this pull request? startingOffsets takes specific per-topicpartition offsets as a json argument,

spark git commit: [SPARK-18044][STREAMING] FileStreamSource should not infer partitions in every batch

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master c1f344f1a -> 140570252 [SPARK-18044][STREAMING] FileStreamSource should not infer partitions in every batch ## What changes were proposed in this pull request? In `FileStreamSource.getBatch`, we will create a `DataSource` with specified

spark git commit: [SPARK-17929][CORE] Fix deadlock when CoarseGrainedSchedulerBackend reset

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 7a531e305 -> c1f344f1a [SPARK-17929][CORE] Fix deadlock when CoarseGrainedSchedulerBackend reset ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-17929 Now `CoarseGrainedSchedulerBackend`

spark git commit: [SPARK-17929][CORE] Fix deadlock when CoarseGrainedSchedulerBackend reset

2016-10-21 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 af2e6e0c9 -> b113b5d9f [SPARK-17929][CORE] Fix deadlock when CoarseGrainedSchedulerBackend reset ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-17929 Now

spark git commit: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-21 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 78458a7eb -> af2e6e0c9 [SPARK-17926][SQL][STREAMING] Added json for statuses ## What changes were proposed in this pull request? StreamingQueryStatus exposed through StreamingQueryListener often needs to be recorded (similar to

spark git commit: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-21 Thread yhuai
Repository: spark Updated Branches: refs/heads/master e371040a0 -> 7a531e305 [SPARK-17926][SQL][STREAMING] Added json for statuses ## What changes were proposed in this pull request? StreamingQueryStatus exposed through StreamingQueryListener often needs to be recorded (similar to

spark git commit: [SPARK-17811] SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.0 a65d40ab6 -> 78458a7eb [SPARK-17811] SparkR cannot parallelize data.frame with NA or NULL in Date columns ## What changes were proposed in this pull request? NA date values are serialized as "NA" and NA time values are serialized as

spark git commit: [SPARK-17811] SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master e21e1c946 -> e371040a0 [SPARK-17811] SparkR cannot parallelize data.frame with NA or NULL in Date columns ## What changes were proposed in this pull request? NA date values are serialized as "NA" and NA time values are serialized as NaN

spark git commit: [SPARK-18013][SPARKR] add crossJoin API

2016-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 4efdc764e -> e21e1c946 [SPARK-18013][SPARKR] add crossJoin API ## What changes were proposed in this pull request? Add crossJoin and do not default to cross join if joinExpr is left out ## How was this patch tested? unit test Author:

spark git commit: [SPARK-17674][SPARKR] check for warning in test output

2016-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b3b4b9542 -> 4efdc764e [SPARK-17674][SPARKR] check for warning in test output ## What changes were proposed in this pull request? testthat library we are using for testing R is redirecting warning (and disabling `options("warn" = 2)`),

spark git commit: [SPARK-18034] Upgrade to MiMa 0.1.11 to fix flakiness

2016-10-21 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-2.0 6cc6cb2a9 -> a65d40ab6 [SPARK-18034] Upgrade to MiMa 0.1.11 to fix flakiness We should upgrade to the latest release of MiMa (0.1.11) in order to include a fix for a bug which led to flakiness in the MiMa checks

spark git commit: [SPARK-18034] Upgrade to MiMa 0.1.11 to fix flakiness

2016-10-21 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 3a237512b -> b3b4b9542 [SPARK-18034] Upgrade to MiMa 0.1.11 to fix flakiness We should upgrade to the latest release of MiMa (0.1.11) in order to include a fix for a bug which led to flakiness in the MiMa checks

spark git commit: [SPARK-13275][WEB UI] Visually clarified executors start time in timeline

2016-10-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master a8ea4da8d -> 3a237512b [SPARK-13275][WEB UI] Visually clarified executors start time in timeline ## What changes were proposed in this pull request? Updated the Executors added/removed bubble in the time line so it's clearer where it

spark git commit: [SPARK-17960][PYSPARK][UPGRADE TO PY4J 0.10.4]

2016-10-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master 57e97fcbd -> 595893d33 [SPARK-17960][PYSPARK][UPGRADE TO PY4J 0.10.4] ## What changes were proposed in this pull request? 1) Upgrade the Py4J version on the Java side 2) Update the py4j src zip file we bundle with Spark ## How was this

spark git commit: [SPARK-17331][FOLLOWUP][ML][CORE] Avoid allocating 0-length arrays

2016-10-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master 595893d33 -> a8ea4da8d [SPARK-17331][FOLLOWUP][ML][CORE] Avoid allocating 0-length arrays ## What changes were proposed in this pull request? `Array[T]()` -> `Array.empty[T]` to avoid allocating 0-length arrays. Use regex `find . -name