[GitHub] spark pull request: SPARK-5888. [MLLIB]. Add OneHotEncoder as a Tr...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5500#issuecomment-92553404 [Test build #30206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30206/consoleFull) for PR 5500 at commit [`04590bc`](https://gith

[GitHub] spark pull request: [SQL] [Minor] Fix for SqlApp.scala

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5485#issuecomment-92553261 Thanks, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [Spark-4848] Allow different Worker configurat...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5140 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92552866 [Test build #30210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30210/consoleFull) for PR 5431 at commit [`26ae2d7`](https://gith

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92552876 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [Spark-4848] Allow different Worker configurat...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5140#issuecomment-92552856 Ok LGTM I'm merging this into master thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-92552765 [Test build #666 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/666/consoleFull) for PR 5173 at commit [`71535e9`](https://github

[GitHub] spark pull request: [SPARK-6877][SQL] Add code generation support ...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5487#issuecomment-92552525 Thanks, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-6877][SQL] Add code generation support ...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5487 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6303][SQL] Remove unnecessary Average i...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4996 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-13 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-92551078 Okay I made an observation on my cluster, the thrift server is started at 21:01:32 and it hadn't do anything from that. Its evnet log's modification time is 21:01

[GitHub] spark pull request: [SPARK-2973] [SQL] Avoid spark job for take on...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5247#issuecomment-92551194 You are correct, thanks for clarifying. Query planning was not the right phrase, but really my point was that ideally the logic in DataFrame would handle only ensuring

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-04-13 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-92550401 ping @JoshRosen . I think this is basically ready to go. The only open issue is figuring how to test the authentication for the new endpoints. (I'm not even sure how to

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-13 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-92550220 @vanzin It is not just theoretical. I tested using a ThriftServer instance, before this patch its event log is deleted by cleaner(when it expires). A

[GitHub] spark pull request: [SPARK-6871][SQL] WITH clause in CTE can not f...

2015-04-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5480#discussion_r28293883 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -414,6 +414,10 @@ class SQLQuerySuite extends QueryTest with BeforeAndAfte

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92549813 [Test build #30212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30212/consoleFull) for PR 5431 at commit [`d19236d`](https://githu

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-92549215 [Test build #30203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30203/consoleFull) for PR 5430 at commit [`0a49021`](https://gith

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-92549221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92549076 [Test build #665 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/665/consoleFull) for PR 5459 at commit [`2ead85b`](https://github

[GitHub] spark pull request: [SPARK-3468][WebUI] Timeline-View feature

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-92548877 [Test build #30211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30211/consoleFull) for PR 2342 at commit [`19815ae`](https://githu

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92548691 LGTM once tests pass --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6406] Launch Spark using assembly jar i...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5085#issuecomment-92548540 Looks like @davies filed a JIRA for this already: https://issues.apache.org/jira/browse/SPARK-6890 --- If your project is set up for it, you can reply to this email a

[GitHub] spark pull request: [SPARK-6703][Core][WIP] Provide a way to disco...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5501#issuecomment-92547808 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6703][Core][WIP] Provide a way to disco...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5501#issuecomment-92547798 [Test build #30207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30207/consoleFull) for PR 5501 at commit [`d7a06b8`](https://gith

[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-92547739 [Test build #30205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30205/consoleFull) for PR 5173 at commit [`71535e9`](https://gith

[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-92547751 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-04-13 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-92547494 @freeman-lab, do you know any good evaluations of a hierarchical clustering algorithm except Within Set Sumb of Squared Error(WSSSE)? For example, I know Silhouette Coeff

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92547091 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92547081 [Test build #30204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30204/consoleFull) for PR 5236 at commit [`4526c81`](https://gith

[GitHub] spark pull request: [SPARK-6406] Launch Spark using assembly jar i...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5085#issuecomment-92546252 @vanzin @nishkamravi2 When I try to run `bin/spark-shell --master local-cluster[2,1,512]` my executors keep failing complaining that the scala classes are not found. I

[GitHub] spark pull request: [MLLIB][WIP] SPARK-4638: Kernels feature for M...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5503#issuecomment-92544614 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-6734] [SQL] Add UDTF.close support in G...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/5383#issuecomment-92544533 Thank you @liancheng I've updated the code and it passed the unit test. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: [MLLIB][WIP] SPARK-4638: Kernels feature for M...

2015-04-13 Thread mandar2812
GitHub user mandar2812 opened a pull request: https://github.com/apache/spark/pull/5503 [MLLIB][WIP] SPARK-4638: Kernels feature for MLLIB 1) Class hierarchy for SVM Kernels, with unit tests. 2) Entropy based subset selection for low rank approximation of Large Kernel Matrices,

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92544002 [Test build #30210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30210/consoleFull) for PR 5431 at commit [`26ae2d7`](https://githu

[GitHub] spark pull request: [SPARK-6734] [SQL] Add UDTF.close support in G...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5383#issuecomment-92543221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6734] [SQL] Add UDTF.close support in G...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5383#issuecomment-92543213 [Test build #30202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30202/consoleFull) for PR 5383 at commit [`d719983`](https://gith

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28291065 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290995 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -55,58 +49,42 @@ class Param[T] ( */ def ->(value: T): ParamPair[

[GitHub] spark pull request: Changes to support checkpointing to BlockRDD a...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5502#issuecomment-92541715 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: Changes to support checkpointing to BlockRDD a...

2015-04-13 Thread mtimper
GitHub user mtimper opened a pull request: https://github.com/apache/spark/pull/5502 Changes to support checkpointing to BlockRDD as described in http://apac... ...he-spark-user-list.1001560.n3.nabble.com/java-lang-StackOverflowError-when-calling-count-td5649.html#a11970 You can me

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92541260 [Test build #664 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/664/consoleFull) for PR 5459 at commit [`2ead85b`](https://github

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290778 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -325,7 +379,7 @@ class ParamMap private[ml] (private val map: mutable.Map[Param[An

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290818 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92541203 I had triggered several runs in a row, waiting for the results. If they all passed, we could merge this. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92541148 [Test build #663 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/663/consoleFull) for PR 5459 at commit [`2ead85b`](https://github

[GitHub] spark pull request: [SPARK-6703][Core][WIP] Provide a way to disco...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5501#issuecomment-92541131 [Test build #30207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30207/consoleFull) for PR 5501 at commit [`d7a06b8`](https://githu

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92541125 [Test build #30208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30208/consoleFull) for PR 5459 at commit [`2ead85b`](https://githu

[GitHub] spark pull request: [SPARK-4705] Handle multiple app attempts even...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5432#issuecomment-92541143 [Test build #30209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30209/consoleFull) for PR 5432 at commit [`86de638`](https://githu

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290695 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable { /**

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92540777 Alright, one last time. If this passes, I'd suggest re-enabling the test, so that if it's still flaky we can look at logs and figure out why. (I worked with Sean and have

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5493 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92540562 For the first point, the conflict was in `httpclient` which was not resolved correctly by the parent, leading to this iteration of the PR. For the second point

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92540457 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable { /**

[GitHub] spark pull request: [SPARK-6703][Core][WIP] Provide a way to disco...

2015-04-13 Thread ilganeli
GitHub user ilganeli opened a pull request: https://github.com/apache/spark/pull/5501 [SPARK-6703][Core][WIP] Provide a way to discover existing SparkContext's I've added a static getOrCreate method to the static SparkContext object that allows one to either retrieve a previously cr

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5493#issuecomment-92539716 LGTM. Thanks @hlin09 - Merging this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-5888. [MLLIB]. Add OneHotEncoder as a Tr...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5500#issuecomment-92539907 [Test build #30206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30206/consoleFull) for PR 5500 at commit [`04590bc`](https://githu

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290414 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable { /**

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28290419 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable { /**

[GitHub] spark pull request: SPARK-5888. [MLLIB]. Add OneHotEncoder as a Tr...

2015-04-13 Thread sryza
GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/5500 SPARK-5888. [MLLIB]. Add OneHotEncoder as a Transformer This patch adds a one hot encoder for categorical features. Planning to add documentation and another test after getting feedback on the approa

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5493#issuecomment-92538479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5493#issuecomment-92538465 [Test build #30201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30201/consoleFull) for PR 5493 at commit [`e67fc40`](https://gith

[GitHub] spark pull request: [Minor][SparkR] Minor refactor and removes red...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5495#issuecomment-92537882 [Test build #30200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30200/consoleFull) for PR 5495 at commit [`74ec303`](https://gith

[GitHub] spark pull request: [Minor][SparkR] Minor refactor and removes red...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5495#issuecomment-92537908 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4688#issuecomment-92537887 Hi @harishreedharan - could you add some more documentation for this? The high level architecture here may be hard for users to see. Here are some places you might consi

[GitHub] spark pull request: [SPARK-6887][SQL] ColumnBuilder misses FloatTy...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5499#issuecomment-92537454 [Test build #30199 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30199/consoleFull) for PR 5499 at commit [`d169d33`](https://gith

[GitHub] spark pull request: [SPARK-6887][SQL] ColumnBuilder misses FloatTy...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5499#issuecomment-92537484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28289791 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -55,58 +49,42 @@ class Param[T] ( */ def ->(value: T): ParamPair[T]

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28289673 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -78,23 +81,42 @@ class ParamsSuite extends FunSuite { } t

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28289697 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala --- @@ -17,15 +17,16 @@ package org.apache.spark.ml.classificatio

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92533768 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92533725 [Test build #30198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30198/consoleFull) for PR 5459 at commit [`2ead85b`](https://gith

[GitHub] spark pull request: [Spark-4848] Allow different Worker configurat...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5140#discussion_r28289277 --- Diff: sbin/stop-slaves.sh --- @@ -29,10 +29,4 @@ if [ -e "$sbin"/../tachyon/bin/tachyon ]; then "$sbin/slaves.sh" cd "$SPARK_HOME" \; "$sbin"/../

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92532627 That ship may have sailed for better or worse. Yes we have to be careful about bringing things into core, so I'm glad to see the exclusions, but I think there are simpler

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5236 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92532459 Ok, LGTM I'm merging this into master. It's great to see things on your todo list getting ticked off by others in the community. Thanks @ilganeli @srowen @vanzin! --

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-92532203 @CodingCat We shouldn't have to worry about the case when the user asks for more resources per executors than are available on the cluster. If each machine only has 2 c

[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-92531906 [Test build #30205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30205/consoleFull) for PR 5173 at commit [`71535e9`](https://githu

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288632 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -582,32 +555,63 @@ private[master] class Master( pos = (pos

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288621 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -582,32 +555,63 @@ private[master] class Master( pos = (pos

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288516 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -582,32 +555,63 @@ private[master] class Master( pos = (pos

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92531201 In the original PR, the dependency management was left to maven but it caused dependency issues in tests which is why this new method is being proposed. For th

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288450 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -524,52 +524,25 @@ private[master] class Master( } /**

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92530750 [Test build #30204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30204/consoleFull) for PR 5236 at commit [`4526c81`](https://githu

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288173 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -524,52 +524,25 @@ private[master] class Master( } /**

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r28288122 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -524,52 +524,25 @@ private[master] class Master( } /**

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread ilganeli
Github user ilganeli commented on a diff in the pull request: https://github.com/apache/spark/pull/5236#discussion_r28288002 --- Diff: docs/running-on-yarn.md --- @@ -48,9 +48,9 @@ Most of the configs are the same for Spark on YARN as for other deployment modes s

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5236#discussion_r28287888 --- Diff: docs/running-on-yarn.md --- @@ -48,9 +48,9 @@ Most of the configs are the same for Spark on YARN as for other deployment modes

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5236#discussion_r28287786 --- Diff: docs/running-on-yarn.md --- @@ -48,9 +48,9 @@ Most of the configs are the same for Spark on YARN as for other deployment modes

[GitHub] spark pull request: [SPARK-2669] [yarn] Distribute client configur...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4142#issuecomment-92528269 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-5941] [SQL] Unit Test loads the table `...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4506 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2669] [yarn] Distribute client configur...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4142#issuecomment-92528245 [Test build #30197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30197/consoleFull) for PR 4142 at commit [`f5434b9`](https://gith

[GitHub] spark pull request: [SPARK-5941] [SQL] Unit Test loads the table `...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4506#issuecomment-92527942 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-92527921 [Test build #30203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30203/consoleFull) for PR 5430 at commit [`0a49021`](https://githu

[GitHub] spark pull request: [SPARK-6872] [SQL] add copy in external sort

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6872] [SQL] add copy in external sort

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5481#issuecomment-92527536 Thanks, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-92526609 I think this is getting pretty close! If you run into trouble implementing some of my suggestions on the `SparkPlan` interfaces for ordering please let me know and I ca

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r28287371 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoin.scala --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r28287227 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala --- @@ -41,6 +41,15 @@ case class Project(projectList: Seq[NamedExpress

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r28287180 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Exchange.scala --- @@ -33,7 +32,11 @@ import org.apache.spark.util.MutablePair * :: De

<    1   2   3   4   5   6   7   >