[GitHub] spark pull request: [MINOR] Follow-up to #12625

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214623731
  
**[Test build #56974 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56974/consoleFull)**
 for PR 12686 at commit 
[`bb71988`](https://github.com/apache/spark/commit/bb71988c955ba875bcd82dc2d152492f267c71b2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14837][SQL][STREAMING] Added support in...

[ScrapCodes]

Github user ScrapCodes commented on the pull request:

https://github.com/apache/spark/pull/12616#issuecomment-214622436
  
This is also supported by hadoop. For example, we can pass the globs in 
sc.textFile("/dir/*/*"). I was wondering, if it will be implemented again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14912][SQL] Propagate data source optio...

[yhuai]

Github user yhuai commented on the pull request:

https://github.com/apache/spark/pull/12688#issuecomment-214622278
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12685#issuecomment-214620448
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14912][SQL] Propagate data source optio...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12688#issuecomment-214620464
  
cc @yhuai / @andrewor14 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14912][SQL] Propagate data source optio...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12688#issuecomment-214620503
  
**[Test build #56973 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56973/consoleFull)**
 for PR 12688 at commit 
[`3d53396`](https://github.com/apache/spark/commit/3d533968a0596a00cb53d8a8d1b228f543a15835).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12685#issuecomment-214620449
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56967/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14912][SQL] Propagate data source optio...

[rxin]

GitHub user rxin opened a pull request:

https://github.com/apache/spark/pull/12688

[SPARK-14912][SQL] Propagate data source options to Hadoop configuration

## What changes were proposed in this pull request?
We currently have no way for users to propagate options to the underlying 
library that rely in Hadoop configurations to work. For example, there are 
various options in parquet-mr that users might want to set, but the data source 
API does not expose a per-job way to set it. This patch propagates the 
user-specified options also into Hadoop Configuration.

## How was this patch tested?
Added a unit test to test both the read path and the write path.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rxin/spark SPARK-14912

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12688.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12688


commit 3d533968a0596a00cb53d8a8d1b228f543a15835
Author: Reynold Xin 
Date:   2016-04-26T05:30:33Z

[SPARK-14912][SQL] Propagate data source options to Hadoop configurations




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12685#issuecomment-214620374
  
**[Test build #56967 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56967/consoleFull)**
 for PR 12685 at commit 
[`520d6ac`](https://github.com/apache/spark/commit/520d6ac69c2ed7cc775d56ec39e8c011499e038a).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `  class AFTSurvivalRegressionWrapperWriter(instance: 
AFTSurvivalRegressionWrapper)`
  * `  class AFTSurvivalRegressionWrapperReader extends 
MLReader[AFTSurvivalRegressionWrapper] `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[mengxr]

Github user mengxr commented on the pull request:

https://github.com/apache/spark/pull/12685#issuecomment-214620302
  
LGTM pending Jenkins


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SQL] [DO NOT MERGE] Log native commands to fi...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12687#issuecomment-214619924
  
**[Test build #56972 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56972/consoleFull)**
 for PR 12687 at commit 
[`0fb0dc3`](https://github.com/apache/spark/commit/0fb0dc33a7ec4ca5947ad3bc0764c1620baa8837).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14756][Core] Use parseLong instead of v...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12520#issuecomment-214619179
  
**[Test build #2884 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2884/consoleFull)**
 for PR 12520 at commit 
[`12b5739`](https://github.com/apache/spark/commit/12b5739a37805cfded1b111b9224805fdf0bf7f4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

[kayousterhout]

Github user kayousterhout commented on the pull request:

https://github.com/apache/spark/pull/12655#issuecomment-214619200
  
I see -- now I understand the motivation for returning the shuffle 
dependencies topologically sorted, because it limits the depth of the recursion 
(it looks like the old code was trying to do that with the stack, but didn't 
quite get it right?).  Let me think about whether there's a simpler way to 
accomplish that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12629#issuecomment-214619175
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12629#issuecomment-214619176
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56954/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SQL] Log native commands to find the number o...

[yhuai]

GitHub user yhuai opened a pull request:

https://github.com/apache/spark/pull/12687

[SQL] Log native commands to find the number of tests that will be affected 
if we do not delegate any statements to Hive

For Spark 2.0, Spark will handle all queries and statements. This PR aims 
to get a list of test cases that will fail if we do not delegate any 
query/statement to hive right now.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yhuai/spark banNativeCommand

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12687.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12687


commit 0fb0dc33a7ec4ca5947ad3bc0764c1620baa8837
Author: Yin Huai 
Date:   2016-04-26T05:11:19Z

Log native commands.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12629#issuecomment-214619052
  
**[Test build #56954 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56954/consoleFull)**
 for PR 12629 at commit 
[`2fa4a12`](https://github.com/apache/spark/commit/2fa4a128fcb576914d4632ab4a71f135839ab287).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14756][Core] Use parseLong instead of v...

[javawithjiva]

Github user javawithjiva commented on the pull request:

https://github.com/apache/spark/pull/12520#issuecomment-214618915
  
Thank you for helping me get my first pull request out.  Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [WIP] Refactor MemoryManager internals to simp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12381#issuecomment-214618622
  
**[Test build #56971 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56971/consoleFull)**
 for PR 12381 at commit 
[`0a988e0`](https://github.com/apache/spark/commit/0a988e0c2b8631e3e78a5ec7ee6170b1249fe3a7).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214618462
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214618463
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56969/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214618451
  
**[Test build #56969 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56969/consoleFull)**
 for PR 12686 at commit 
[`961a7c1`](https://github.com/apache/spark/commit/961a7c1de11a4da8165e950e912f0939fb314cb1).
 * This patch **fails MiMa tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14911][Core] Fix a potential data race ...

[lw-lin]

Github user lw-lin commented on the pull request:

https://github.com/apache/spark/pull/12681#issuecomment-214617805
  
Some flaky tests not caused by this PR.

Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

[ueshin]

Github user ueshin commented on the pull request:

https://github.com/apache/spark/pull/12655#issuecomment-214617772
  
@kayousterhout 
I'll update the unit test and JIRA, PR descriptions to use the simpler 
example.

As for the `createAncestorShuffleMapStages` way, I think it will have a 
risk of `StackOverflowError` for a long job because the master branch version 
of `getAncestorShuffleDependencies` finds descendants first, which would have a 
lot of ancestors with long linage, and if we immediately create the shuffle 
stage there by the `newOrUsedShuffleStage` method, which builds all ancestor 
stages by recursive-call fashion, the `StackOverflowError` will be thrown.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214617649
  
**[Test build #56969 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56969/consoleFull)**
 for PR 12686 at commit 
[`961a7c1`](https://github.com/apache/spark/commit/961a7c1de11a4da8165e950e912f0939fb314cb1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14830][SQL] Add RemoveRepetitionFromGro...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12590#issuecomment-214617643
  
**[Test build #56970 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56970/consoleFull)**
 for PR 12590 at commit 
[`01ce1a4`](https://github.com/apache/spark/commit/01ce1a4a25c7d80a87b29efc7a83a9a424a827dc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14911][Core] Fix a potential data race ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12681#issuecomment-214617526
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56953/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14911][Core] Fix a potential data race ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12681#issuecomment-214617524
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14911][Core] Fix a potential data race ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12681#issuecomment-214617425
  
**[Test build #56953 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56953/consoleFull)**
 for PR 12681 at commit 
[`6b72b96`](https://github.com/apache/spark/commit/6b72b963d54855771dcabc1fca8ed963be28303c).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14664][SQL] Implement DecimalAggregates...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12421#issuecomment-214617100
  
**[Test build #56968 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56968/consoleFull)**
 for PR 12421 at commit 
[`14fe6b9`](https://github.com/apache/spark/commit/14fe6b978be9740feb4df4f4bd5c4dc1206af60b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14664][SQL] Implement DecimalAggregates...

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12421#issuecomment-214616662
  
Rebased.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214616125
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56966/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214616123
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214616120
  
**[Test build #56966 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56966/consoleFull)**
 for PR 12686 at commit 
[`599b180`](https://github.com/apache/spark/commit/599b180f80670417701065424fb650e89c424c30).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12686#issuecomment-214615986
  
**[Test build #56966 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56966/consoleFull)**
 for PR 12686 at commit 
[`599b180`](https://github.com/apache/spark/commit/599b180f80670417701065424fb650e89c424c30).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12685#issuecomment-214615981
  
**[Test build #56967 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56967/consoleFull)**
 for PR 12685 at commit 
[`520d6ac`](https://github.com/apache/spark/commit/520d6ac69c2ed7cc775d56ec39e8c011499e038a).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [MINOR] Follow-up to #12625

[andrewor14]

GitHub user andrewor14 opened a pull request:

https://github.com/apache/spark/pull/12686

[MINOR] Follow-up to #12625

## What changes were proposed in this pull request?

That patch mistakenly widened the visibility from `private[x]` to 
`protected[x]`. This patch reverts those changes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrewor14/spark visibility

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12686.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12686


commit 599b180f80670417701065424fb650e89c424c30
Author: Andrew Or 
Date:   2016-04-26T04:49:34Z

Tighten visibility again




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214615564
  
Thank you! :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214615647
  
It works now! Thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214615366
  
**[Test build #56965 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56965/consoleFull)**
 for PR 12268 at commit 
[`fe63ba2`](https://github.com/apache/spark/commit/fe63ba22d70c1427657b4967e769270d1956be38).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14313] [ML] [SparkR] AFTSurvivalRegress...

[yanboliang]

GitHub user yanboliang opened a pull request:

https://github.com/apache/spark/pull/12685

[SPARK-14313] [ML] [SparkR] AFTSurvivalRegression model persistence in 
SparkR

## What changes were proposed in this pull request?
```AFTSurvivalRegressionModel``` supports ```save/load``` in SparkR.

## How was this patch tested?
Unit tests.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yanboliang/spark spark-14313

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12685.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12685


commit 520d6ac69c2ed7cc775d56ec39e8c011499e038a
Author: Yanbo Liang 
Date:   2016-04-26T04:43:04Z

AFTSurvivalRegression model persistence in SparkR




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214614924
  
Fixed in 
https://github.com/apache/spark/commit/f8709218115f6c7aa4fb321865cdef8ceb443bd1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214614851
  
**[Test build #56964 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56964/consoleFull)**
 for PR 12257 at commit 
[`418772b`](https://github.com/apache/spark/commit/418772b090f537491891ab14707aad3a6029a134).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214614770
  
@rxin It looks this is still failing, 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56962
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56963



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214613784
  
**[Test build #56963 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56963/consoleFull)**
 for PR 12268 at commit 
[`f62755e`](https://github.com/apache/spark/commit/f62755e0875ae8f2947abf8a62505dd77b2ed9f5).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214613795
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56963/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214613897
  
OK pushed.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214613793
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214613611
  
Hah ok let me fix it for real.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12666#issuecomment-214613541
  
**[Test build #2883 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2883/consoleFull)**
 for PR 12666 at commit 
[`4658936`](https://github.com/apache/spark/commit/465893671c7514b1f38d4cbd568ceefa48737b1f).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214612978
  
It's a clean build after `git clean -fdx`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214612753
  
Ur, @rxin .
Could you check that again?
```
[error] 
/Users/dongjoon/spark/sql/hivecontext-compatibility/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala:50:
 super may not be used on value sparkSession
[error] new HiveContext(super.sparkSession.newSession(), isRootContext 
= false)
[error]   ^
[error] 
/Users/dongjoon/spark/sql/hivecontext-compatibility/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala:54:
 super may not be used on value sparkSession
[error] super.sparkSession.sessionState.asInstanceOf[HiveSessionState]
[error]   ^
[error] 
/Users/dongjoon/spark/sql/hivecontext-compatibility/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala:58:
 super may not be used on value sparkSession
[error] super.sparkSession.sharedState.asInstanceOf[HiveSharedState]
[error]   ^
[error] three errors found
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214612442
  
**[Test build #56962 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56962/consoleFull)**
 for PR 12257 at commit 
[`b8a0a7d`](https://github.com/apache/spark/commit/b8a0a7d07aec530f8b7eef55d4243ebf96a86882).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214612454
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214612460
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56962/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214611901
  
**[Test build #56963 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56963/consoleFull)**
 for PR 12268 at commit 
[`f62755e`](https://github.com/apache/spark/commit/f62755e0875ae8f2947abf8a62505dd77b2ed9f5).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214611834
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214611841
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56960/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214611632
  
**[Test build #56960 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56960/consoleFull)**
 for PR 12684 at commit 
[`1c8b246`](https://github.com/apache/spark/commit/1c8b2469f0d1ca4304ccd42b0586f52b3ea1e376).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12666#issuecomment-214611155
  
LGTM pending Jenkins.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12666#issuecomment-214611080
  
**[Test build #2883 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2883/consoleFull)**
 for PR 12666 at commit 
[`4658936`](https://github.com/apache/spark/commit/465893671c7514b1f38d4cbd568ceefa48737b1f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14445][SQL] Support native execution of...

[dilipbiswal]

Github user dilipbiswal commented on a diff in the pull request:

https://github.com/apache/spark/pull/1#discussion_r61027608
  
--- Diff: 
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
 ---
@@ -122,4 +134,105 @@ class HiveCommandSuite extends QueryTest with 
SQLTestUtils with TestHiveSingleto
   checkAnswer(sql("SHOW TBLPROPERTIES parquet_temp"), Nil)
 }
   }
+
+  test("show columns") {
+checkAnswer(
+  sql("SHOW COLUMNS IN parquet_tab3"),
+  Row("col1") :: Row("col 2") :: Nil)
+
+checkAnswer(
+  sql("SHOW COLUMNS IN default.parquet_tab3"),
+  Row("col1") :: Row("col 2") :: Nil)
+
+checkAnswer(
+  sql("SHOW COLUMNS IN parquet_tab3 FROM default"),
+  Row("col1") :: Row("col 2") :: Nil)
+
+checkAnswer(
+  sql("SHOW COLUMNS IN parquet_tab4 IN default"),
+  Row("price") :: Row("qty") :: Row("year") :: Row("month") :: Nil)
+
+val message = intercept[NoSuchTableException] {
+  sql("SHOW COLUMNS IN badtable FROM default")
+}.getMessage
+assert(message.contains("badtable not found in database"))
+  }
+
+  test("show partitions - show everything") {
+checkAnswer(
+  sql("show partitions parquet_tab4"),
+  Row("year=2015/month=1") ::
+Row("year=2015/month=2") ::
+Row("year=2016/month=2") ::
+Row("year=2016/month=3") :: Nil)
--- End diff --

@liancheng Thanks !! You are right !! Having >= 5 partition keys does 
expose the problem. Any advice on how to go about handling this ? Can we change 
TablePartitionSpec to be a LinkedHashMap instead ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14853] [SQL] Support LeftSemi/LeftAnti ...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12668#issuecomment-214610321
  
@hvanhovell want to review?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214609981
  
This was due to 
https://github.com/apache/spark/commit/d2614eaadb93a48fba27fe7de64aff942e345f8e


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214609281
  
**[Test build #56962 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56962/consoleFull)**
 for PR 12257 at commit 
[`b8a0a7d`](https://github.com/apache/spark/commit/b8a0a7d07aec530f8b7eef55d4243ebf96a86882).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214609252
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56961/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[lw-lin]

Github user lw-lin commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214609215
  
some build issues unrelated to this PR.

Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214609239
  
**[Test build #56961 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56961/consoleFull)**
 for PR 12268 at commit 
[`d59c7e9`](https://github.com/apache/spark/commit/d59c7e98f306fa9ff5dfe3b4caae14a2de746315).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `sealed abstract class LDAModel protected[ml] (`
  * `class LocalLDAModel protected[ml] (`
  * `class DistributedLDAModel protected[ml] (`
  * `class ContinuousQueryManager(sparkSession: SparkSession) `
  * `class DataFrameReader protected[sql](sparkSession: SparkSession) 
extends Logging `
  * `class Dataset[T] protected[sql](`
  * `class QueryExecution(val sparkSession: SparkSession, val logical: 
LogicalPlan) `
  * `class FileStreamSinkLog(sparkSession: SparkSession, path: String)`
  * `class HDFSMetadataLog[T: ClassTag](sparkSession: SparkSession, path: 
String)`
  * `class StreamFileCatalog(sparkSession: SparkSession, path: Path) 
extends FileCatalog with Logging `
  * `case class PlanSubqueries(sparkSession: SparkSession) extends 
Rule[SparkPlan] `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214609249
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14806] [SQL] Alias original Hive option...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12607#issuecomment-214609204
  
Hi @bomeng  - thanks for doing this. I thought about it more and actually 
think maybe it's not worth it to support these aliases, because it is going to 
be confusing when users do get all configs. Also it makes it sound like we can 
support all the Hive options, when in reality we don't. So unfortunately I 
think we should just close this one for now. What do you think?




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun closed the pull request at:

https://github.com/apache/spark/pull/12684


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214609078
  
Sure! Thank for quick fix. I found that when I rebase my PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214608932
  
Thanks - actually I pushed directly to master just now. Can you close this?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214608733
  
**[Test build #56960 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56960/consoleFull)**
 for PR 12684 at commit 
[`1c8b246`](https://github.com/apache/spark/commit/1c8b2469f0d1ca4304ccd42b0586f52b3ea1e376).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214608734
  
**[Test build #56961 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56961/consoleFull)**
 for PR 12268 at commit 
[`d59c7e9`](https://github.com/apache/spark/commit/d59c7e98f306fa9ff5dfe3b4caae14a2de746315).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214608678
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

Github user dongjoon-hyun commented on the pull request:

https://github.com/apache/spark/pull/12684#issuecomment-214608530
  
cc @rxin .



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX][SQL] sparkSession can't be private.

[dongjoon-hyun]

GitHub user dongjoon-hyun opened a pull request:

https://github.com/apache/spark/pull/12684

[HOTFIX][SQL] sparkSession can't be private.

## What changes were proposed in this pull request?

This fixes the following errors.
```
-@transient private val sparkSession: SparkSession,
+@transient override val sparkSession: SparkSession,
```

## How was this patch tested?

Pass the Jenkins build.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dongjoon-hyun/spark hotfix_sparksession

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12684.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12684


commit 1c8b2469f0d1ca4304ccd42b0586f52b3ea1e376
Author: Dongjoon Hyun 
Date:   2016-04-26T04:18:25Z

sparkSession can't be private.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214608176
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56959/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214608175
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214608169
  
**[Test build #56959 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56959/consoleFull)**
 for PR 12268 at commit 
[`d59c7e9`](https://github.com/apache/spark/commit/d59c7e98f306fa9ff5dfe3b4caae14a2de746315).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `sealed abstract class LDAModel protected[ml] (`
  * `class LocalLDAModel protected[ml] (`
  * `class DistributedLDAModel protected[ml] (`
  * `class ContinuousQueryManager(sparkSession: SparkSession) `
  * `class DataFrameReader protected[sql](sparkSession: SparkSession) 
extends Logging `
  * `class Dataset[T] protected[sql](`
  * `class QueryExecution(val sparkSession: SparkSession, val logical: 
LogicalPlan) `
  * `class FileStreamSinkLog(sparkSession: SparkSession, path: String)`
  * `class HDFSMetadataLog[T: ClassTag](sparkSession: SparkSession, path: 
String)`
  * `class StreamFileCatalog(sparkSession: SparkSession, path: Path) 
extends FileCatalog with Logging `
  * `case class PlanSubqueries(sparkSession: SparkSession) extends 
Rule[SparkPlan] `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [Spark-14314][SparkR] Add model persistence to...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/12680#discussion_r61026788
  
--- Diff: mllib/src/main/scala/org/apache/spark/ml/r/KMeansWrapper.scala ---
@@ -17,14 +17,21 @@
 
 package org.apache.spark.ml.r
 
+import org.apache.hadoop.fs.Path
+import org.json4s._
+import org.json4s.DefaultFormats
+import org.json4s.JsonDSL._
+import org.json4s.jackson.JsonMethods._
+
 import org.apache.spark.ml.{Pipeline, PipelineModel}
 import org.apache.spark.ml.attribute.AttributeGroup
 import org.apache.spark.ml.clustering.{KMeans, KMeansModel}
 import org.apache.spark.ml.feature.VectorAssembler
+import org.apache.spark.ml.util._
 import org.apache.spark.sql.{DataFrame, Dataset}
 
 private[r] class KMeansWrapper private (
-pipeline: PipelineModel) {
+val pipeline: PipelineModel) extends MLWritable {
--- End diff --

Is there a reason to add `val` for `pipeline`? I think they are same `val`s 
anyway.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14729][Scheduler] Refactored YARN sched...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12641#issuecomment-214607771
  
**[Test build #56956 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56956/consoleFull)**
 for PR 12641 at commit 
[`8117184`](https://github.com/apache/spark/commit/811718492e7b93af3b1c857829a687a7ba2cae39).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14729][Scheduler] Refactored YARN sched...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12641#issuecomment-214607778
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56956/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14729][Scheduler] Refactored YARN sched...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12641#issuecomment-21460
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214607350
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214607351
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56958/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214607346
  
**[Test build #56958 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56958/consoleFull)**
 for PR 12257 at commit 
[`256b812`](https://github.com/apache/spark/commit/256b81258f1c7d9006cf8e57b499653e6450986e).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214607395
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56957/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214607393
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14480][SQL] Simplify CSV parsing proces...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12268#issuecomment-214607369
  
**[Test build #56959 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56959/consoleFull)**
 for PR 12268 at commit 
[`d59c7e9`](https://github.com/apache/spark/commit/d59c7e98f306fa9ff5dfe3b4caae14a2de746315).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/12683#issuecomment-214607333
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214607377
  
**[Test build #56957 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56957/consoleFull)**
 for PR 12638 at commit 
[`14e6900`](https://github.com/apache/spark/commit/14e690037fbfc7840672e9a78b9abbccbc07c3ee).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14910] [SQL] Native DDL Command Support...

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/12679#issuecomment-214607013
  
@hvanhovell @yhuai Could you review the changes? Thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[lw-lin]

Github user lw-lin commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214606875
  
@marmbrus thanks for the patient reminder!

Since I've reverted the renaming, and I've checked there's no other 
completely unused class under `o.a.s.sql.execution.streaming` package, maybe 
this is ready to go (pending tests). So would mind take another look?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

[GayathriMurali]

GitHub user GayathriMurali opened a pull request:

https://github.com/apache/spark/pull/12683

[SPARK-14315][SparkR]Add model persistence to GLMs

## What changes were proposed in this pull request?

Add model persistence to GLMs in SparkR


Unit tests added



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/GayathriMurali/spark SPARK-14315

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12683.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12683


commit 31b2b679c041068fb60db4d36ecc28d149b04c75
Author: GayathriMurali 
Date:   2016-04-26T04:06:08Z

[SPARK-14315][SparkR]Add model persistence to GLMs




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14729][Scheduler] Refactored YARN sched...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12641#issuecomment-214606828
  
**[Test build #56956 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56956/consoleFull)**
 for PR 12641 at commit 
[`8117184`](https://github.com/apache/spark/commit/811718492e7b93af3b1c857829a687a7ba2cae39).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12638#issuecomment-214606829
  
**[Test build #56957 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56957/consoleFull)**
 for PR 12638 at commit 
[`14e6900`](https://github.com/apache/spark/commit/14e690037fbfc7840672e9a78b9abbccbc07c3ee).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/12257#issuecomment-214606825
  
**[Test build #56958 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56958/consoleFull)**
 for PR 12257 at commit 
[`256b812`](https://github.com/apache/spark/commit/256b81258f1c7d9006cf8e57b499653e6450986e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-14874][SQL][Streaming] Remove the obsol...

[lw-lin]

GitHub user lw-lin reopened a pull request:

https://github.com/apache/spark/pull/12638

[SPARK-14874][SQL][Streaming] Remove the obsolete Batch representation

## What changes were proposed in this pull request?

The `Batch` class, which had been used to indicate progress in a stream, 
was abandoned by [[SPARK-13985][SQL] Deterministic batches with 
ids](https://github.com/apache/spark/commit/caea15214571d9b12dcf1553e5c1cc8b83a8ba5b)
 and then became useless.

This patch:
- removes the `Batch` class
- renames `getBatch(...)` to `getData(...)` for `Source`: 
 - prior to 
[SPARK-13985](https://github.com/apache/spark/commit/caea15214571d9b12dcf1553e5c1cc8b83a8ba5b),
 it was: get**_NextBatch_**(start: Option[Offset]): **_Option[Batch]_**
 - after  
[SPARK-13985](https://github.com/apache/spark/commit/caea15214571d9b12dcf1553e5c1cc8b83a8ba5b),
 it became: get**_Batch_**(start: Option[Offset], end: Offset): **_DataFrame_**
 - proposed in this patch: get**_Data_**(start: Option[Offset], end: 
Offset): DataFrame
- renames `addBatch(...)` to `addData(...)` for `Sink`:
 - prior to 
[SPARK-13985](https://github.com/apache/spark/commit/caea15214571d9b12dcf1553e5c1cc8b83a8ba5b),
 it was: addBatch(**_batch: Batch_**)
 - after  
[SPARK-13985](https://github.com/apache/spark/commit/caea15214571d9b12dcf1553e5c1cc8b83a8ba5b),
 it became: addBatch(batchId: Long, **_data: DataFrame_**)
 - proposed in this patch: add**_Data_**(batchId: Long, data: DataFrame)

The renaming of public methods should be OK since they have not been in any 
release yet.

## How was this patch tested?

N/A

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lw-lin/spark remove-batch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/12638.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #12638


commit c79cba9059b7ac2d6398c81b57ceece50b6b7526
Author: Liwei Lin 
Date:   2016-04-23T10:15:51Z

remove the useless Batch class

commit 60aaf97c30f55f5791ec53f81f3ed90481586fed
Author: Liwei Lin 
Date:   2016-04-26T04:03:10Z

revert renaming

commit 14e690037fbfc7840672e9a78b9abbccbc07c3ee
Author: Liwei Lin 
Date:   2016-04-26T04:03:23Z

revert renaming




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org