spark git commit: [SPARK-20020][SPARKR] DataFrame checkpoint API

2017-03-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 0cdcf9114 -> c40597720 [SPARK-20020][SPARKR] DataFrame checkpoint API ## What changes were proposed in this pull request? Add checkpoint, setCheckpointDir API to R ## How was this patch tested? unit tests, manual tests Author: Felix

spark git commit: [SPARK-19849][SQL] Support ArrayType in to_json to produce JSON array

2017-03-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 990af630d -> 0cdcf9114 [SPARK-19849][SQL] Support ArrayType in to_json to produce JSON array ## What changes were proposed in this pull request? This PR proposes to support an array of struct type in `to_json` as below: ```scala import

[2/2] spark git commit: [SPARK-19067][SS] Processing-time-based timeout in MapGroupsWithState

2017-03-19 Thread tdas
[SPARK-19067][SS] Processing-time-based timeout in MapGroupsWithState ## What changes were proposed in this pull request? When a key does not get any new data in `mapGroupsWithState`, the mapping function is never called on it. So we need a timeout feature that calls the function again in such

[1/2] spark git commit: [SPARK-19067][SS] Processing-time-based timeout in MapGroupsWithState

2017-03-19 Thread tdas
Repository: spark Updated Branches: refs/heads/master 0ee9fbf51 -> 990af630d http://git-wip-us.apache.org/repos/asf/spark/blob/990af630/sql/core/src/main/scala/org/apache/spark/sql/streaming/KeyedState.scala -- diff --git

spark git commit: [SPARK-19990][TEST] Use the database after Hive's current Database is dropped

2017-03-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 422aa67d1 -> 0ee9fbf51 [SPARK-19990][TEST] Use the database after Hive's current Database is dropped ### What changes were proposed in this pull request? This PR is to fix the following test failure in maven and the PR

spark git commit: [SPARK-18817][SPARKR][SQL] change derby log output to temp dir

2017-03-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.1 780f6060c -> b60f69025 [SPARK-18817][SPARKR][SQL] change derby log output to temp dir ## What changes were proposed in this pull request? Passes R `tempdir()` (this is the R session temp dir, shared with other temp files/dirs) to

spark git commit: [SPARK-18817][SPARKR][SQL] change derby log output to temp dir

2017-03-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 60262bc95 -> 422aa67d1 [SPARK-18817][SPARKR][SQL] change derby log output to temp dir ## What changes were proposed in this pull request? Passes R `tempdir()` (this is the R session temp dir, shared with other temp files/dirs) to JVM,

spark git commit: [MINOR][R] Reorder `Collate` fields in DESCRIPTION file

2017-03-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 5c165596d -> 60262bc95 [MINOR][R] Reorder `Collate` fields in DESCRIPTION file ## What changes were proposed in this pull request? It seems cran check scripts corrects `R/pkg/DESCRIPTION` and follows the order in `Collate` fields. This