[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74511219 [Test build #27554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27554/consoleFull) for PR 4621 at commit [`b5f2816`](https://gith

[GitHub] spark pull request: [SPARK-3957]: show broadcast variable resource...

2015-02-16 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/2851#issuecomment-74510287 well I was leaning towards using a `ThreadLocal` to get the broadcast blocks into the task end event ... but I forgot that broadcast blocks are also created by the driver

[GitHub] spark pull request: [SPARK-5830][Core]Don't create unnecessary dir...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4620#issuecomment-74506157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5830][Core]Don't create unnecessary dir...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4620#issuecomment-74506146 **[Test build #27553 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27553/consoleFull)** for PR 4620 at commit [`26670d8`](https://git

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74502338 [Test build #27554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27554/consoleFull) for PR 4621 at commit [`b5f2816`](https://githu

[GitHub] spark pull request: [SPARK-5799][SQL] Compute aggregation function...

2015-02-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/4592#discussion_r24748159 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedData.scala --- @@ -39,13 +39,14 @@ class GroupedData protected[sql](df: DataFrameImpl, groupingEx

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74501886 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-5680][SQL] Sum function on all null val...

2015-02-16 Thread gvramana
Github user gvramana commented on the pull request: https://github.com/apache/spark/pull/4466#issuecomment-74500153 @marmbrus , Please review and merge the same. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74498269 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74498257 [Test build #27552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27552/consoleFull) for PR 4621 at commit [`b5f2816`](https://gith

[GitHub] spark pull request: [SPARK-5821] [SQL] JSON CTAS command should th...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4610#issuecomment-74494628 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5821] [SQL] JSON CTAS command should th...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4610#issuecomment-74494622 [Test build #27551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27551/consoleFull) for PR 4610 at commit [`5a42d83`](https://gith

[GitHub] spark pull request: [SPARK-5833] [SQL] Adds REFRESH TABLE command

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4624#issuecomment-74494621 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5833] [SQL] Adds REFRESH TABLE command

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4624#issuecomment-74494612 [Test build #27550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27550/consoleFull) for PR 4624 at commit [`8d1aa4c`](https://gith

[GitHub] spark pull request: [SPARK-5799][SQL] Compute aggregation function...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4592#issuecomment-74493864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5799][SQL] Compute aggregation function...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4592#issuecomment-74493853 [Test build #27549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27549/consoleFull) for PR 4592 at commit [`9446896`](https://gith

[GitHub] spark pull request: [SPARK-5830][Core]Don't create unnecessary dir...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4620#issuecomment-74492869 [Test build #27553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27553/consoleFull) for PR 4620 at commit [`26670d8`](https://githu

[GitHub] spark pull request: [SPARK-5830][Core]Don't create unnecessary dir...

2015-02-16 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4620#issuecomment-74492659 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-5830][Core]Don't create unnecessary dir...

2015-02-16 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4620#issuecomment-74492635 Since this appears to be exactly the same as https://issues.apache.org/jira/browse/SPARK-5801 I wonder if you can reassociate to that JIRA? and consider whether it solves

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74491100 [Test build #27552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27552/consoleFull) for PR 4621 at commit [`b5f2816`](https://githu

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74491029 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-5831][Streaming]When checkpoint file si...

2015-02-16 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4621#issuecomment-74490918 Eh, can you explain why this should be changed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-5296] [SQL] Add more filter types for d...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4623#issuecomment-74489327 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5296] [SQL] Add more filter types for d...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4623#issuecomment-74489320 [Test build #27547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27547/consoleFull) for PR 4623 at commit [`1b296f4`](https://gith

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74485193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74485187 [Test build #27548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27548/consoleFull) for PR 4472 at commit [`5a41e49`](https://gith

[GitHub] spark pull request: [SPARK-5832][Mllib] Add Affinity Propagation c...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4622#issuecomment-74484533 [Test build #27546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27546/consoleFull) for PR 4622 at commit [`d469d12`](https://gith

[GitHub] spark pull request: [SPARK-5832][Mllib] Add Affinity Propagation c...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4622#issuecomment-74484544 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5833] [SQL] Adds REFRESH TABLE command

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4624#issuecomment-74484386 [Test build #27550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27550/consoleFull) for PR 4624 at commit [`8d1aa4c`](https://githu

[GitHub] spark pull request: [SPARK-5821] [SQL] JSON external data source i...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4610#issuecomment-74484384 [Test build #27551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27551/consoleFull) for PR 4610 at commit [`5a42d83`](https://githu

[GitHub] spark pull request: [SPARK-5833] [SQL] Adds REFRESH TABLE command

2015-02-16 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/4624 [SPARK-5833] [SQL] Adds REFRESH TABLE command Lifts `HiveMetastoreCatalog.refreshTable` to `Catalog`. Adds `RefreshTable` command to refresh (possibly cached) metadata in external data sources tab

[GitHub] spark pull request: [SPARK-4553] [SPARK-5767] [SQL] Wires Parquet ...

2015-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4563 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4553] [SPARK-5767] [SQL] Wires Parquet ...

2015-02-16 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/4563#issuecomment-74481284 @yhuai @marmbrus Thanks for the review! I'm gonna merge this as it contains a bunch of fixes and enables full power of the new Parquet data source :) --- If your proje

[GitHub] spark pull request: [Minor] [SQL] Renames stringRddToDataFrame to ...

2015-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4613 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5799][SQL] Compute aggregation function...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4592#issuecomment-74480130 [Test build #27549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27549/consoleFull) for PR 4592 at commit [`9446896`](https://githu

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74479736 `aggregates.scala` is the only file in folder `expressions` which is executable, I think it was a mistake and we should make it un-executable. --- If your project is s

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74479214 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74479209 [Test build #27545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27545/consoleFull) for PR 4472 at commit [`cb37c94`](https://gith

[GitHub] spark pull request: [SPARK-5296] [SQL] Add more filter types for d...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4623#issuecomment-74478978 [Test build #27547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27547/consoleFull) for PR 4623 at commit [`1b296f4`](https://githu

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74478946 [Test build #27548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27548/consoleFull) for PR 4472 at commit [`5a41e49`](https://githu

[GitHub] spark pull request: [SPARK-5296] [SQL] Add more filter types for d...

2015-02-16 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/4623 [SPARK-5296] [SQL] Add more filter types for data sources API This PR adds the following filter types for data sources API: - `IsNull` - `IsNotNull` - `Not` - `And` - `Or`

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread kai-zeng
Github user kai-zeng commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74478451 @chenghao-intel you mean revert aggregates.scala to executable? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4553] [SPARK-5767] [SQL] Wires Parquet ...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4563#issuecomment-74476974 [Test build #27544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27544/consoleFull) for PR 4563 at commit [`fa98d27`](https://gith

[GitHub] spark pull request: [SPARK-4553] [SPARK-5767] [SQL] Wires Parquet ...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4563#issuecomment-74476986 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74476383 @kai-zeng can you also revert the file permission bit for `aggregates.scala`? --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-5832][Mllib] Add Affinity Propagation c...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4622#issuecomment-74475569 [Test build #27546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27546/consoleFull) for PR 4622 at commit [`d469d12`](https://githu

[GitHub] spark pull request: [SPARK-5832][Mllib] Add Affinity Propagation c...

2015-02-16 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/4622 [SPARK-5832][Mllib] Add Affinity Propagation clustering algorithm Affinity Propagation (AP), a graph clustering algorithm based on the concept of "message passing" between data points. Unlike cluster

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread kai-zeng
Github user kai-zeng commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74474525 @chenghao-intel Done rebasing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-3570] Include time to open files in shu...

2015-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4550#issuecomment-74474137 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-3570] Include time to open files in shu...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4550#issuecomment-74474130 [Test build #27543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27543/consoleFull) for PR 4550 at commit [`ea3a4ae`](https://gith

[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...

2015-02-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4472#issuecomment-74472313 [Test build #27545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27545/consoleFull) for PR 4472 at commit [`cb37c94`](https://githu

[GitHub] spark pull request: [Ml] SPARK-5804 Explicitly manage cache in Cro...

2015-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [Ml] SPARK-5804 Explicitly manage cache in Cro...

2015-02-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4595#issuecomment-74471911 Merged into master and branch-1.3. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4553] [SPARK-5767] [SQL] Wires Parquet ...

2015-02-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/4563#discussion_r24735725 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -139,15 +139,19 @@ private[hive] trait HiveStrategies {

<    1   2   3   4   5