[GitHub] spark pull request: [SPARK-6673] spark-shell.cmd can't start in Wi...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5328#issuecomment-88779434 [Test build #29593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29593/consoleFull) for PR 5328 at commit [`aaefb19`](https://githu

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5319#issuecomment-88779113 @vanzin do you mind making a PR for 1.2 branch for this fix? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-6673] spark-shell.cmd can't start in Wi...

2015-04-01 Thread tsudukim
GitHub user tsudukim opened a pull request: https://github.com/apache/spark/pull/5328 [SPARK-6673] spark-shell.cmd can't start in Windows even when spark was built added equivalent script to load-spark-env.sh You can merge this pull request into a Git repository by running: $

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5324#issuecomment-88778174 [Test build #29592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29592/consoleFull) for PR 5324 at commit [`e5a51a2`](https://githu

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88776457 Since the behavior is undefined in Scala, I think it is ok to return anything, as I stated in comment. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27635442 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoin.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27635315 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -144,6 +145,12 @@ private[sql] class SQLConf extends Serializable { ge

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5286 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27635297 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -75,9 +76,9 @@ class JoinSuite extends QueryTest with BeforeAndAfterEach {

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88772266 @adrian-wang I leave some comments, but I need some more time in review the code of `SortMergeJoin`, will keep add more comments later. BTW, can you double check i

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-01 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-88771801 Is it not possible to create that simple patch that removes DISTINCT aggregation expressions? We only add `distinct` as a field value in `AggregateExpresion`, and then

[GitHub] spark pull request: [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5326#issuecomment-88771516 [Test build #29591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29591/consoleFull) for PR 5326 at commit [`b4cbabd`](https://gith

[GitHub] spark pull request: [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5326#issuecomment-88771528 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5303#issuecomment-88770278 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5303#issuecomment-88770218 [Test build #29586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29586/consoleFull) for PR 5303 at commit [`9dc32d1`](https://gith

[GitHub] spark pull request: Add status command to spark-daemon

2015-04-01 Thread pchanumolu
GitHub user pchanumolu opened a pull request: https://github.com/apache/spark/pull/5327 Add status command to spark-daemon SPARK-6671 Currently using the spark-daemon.sh script we can start and stop the spark demons. But we cannot get the status of the daemons. It will be nice t

[GitHub] spark pull request: Add status command to spark-daemon

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5327#issuecomment-88769600 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27635035 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoin.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed to the Apach

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27634876 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -75,9 +76,9 @@ class JoinSuite extends QueryTest with BeforeAndAfterEach

[GitHub] spark pull request: [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5326#issuecomment-88762001 [Test build #29591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29591/consoleFull) for PR 5326 at commit [`b4cbabd`](https://githu

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5208#discussion_r27634669 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -144,6 +145,12 @@ private[sql] class SQLConf extends Serializable {

[GitHub] spark pull request: [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin...

2015-04-01 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/5326 [SPARK-3862] [SQL] [WIP] MultiWayBroadcastJoin for LeftSemi & Inner JOIN Assume we have table `x`, `y`, `z`, and the `x` is the fact table with large mount of data, and `y`, `z` are dimension

[GitHub] spark pull request: [SPARK-6435] spark-shell --jars option does no...

2015-04-01 Thread tsudukim
Github user tsudukim commented on the pull request: https://github.com/apache/spark/pull/5227#issuecomment-88761331 Ah @vanzin, I didn't understand your suggestion. `CommandBuilderUtils` needs modified to escape comma. But I think we still need to modify `spark-class2.cmd` as well.

[GitHub] spark pull request: [SPARK-6435] spark-shell --jars option does no...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5227#issuecomment-88759471 [Test build #29590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29590/consoleFull) for PR 5227 at commit [`0d4dc41`](https://githu

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5324#issuecomment-88759230 [Test build #29589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29589/consoleFull) for PR 5324 at commit [`7977c2f`](https://githu

[GitHub] spark pull request: [SPARK-6663] [SQL] use Literal.create instread...

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5320 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6663] [SQL] use Literal.create instread...

2015-04-01 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5320#issuecomment-88758764 Thanks. Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-88756937 [Test build #29588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29588/consoleFull) for PR 5297 at commit [`f0bc54f`](https://githu

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88752957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88752937 [Test build #29585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29585/consoleFull) for PR 5286 at commit [`c71fbc7`](https://gith

[GitHub] spark pull request: [SPARK-4346][SPARK-3596][YARN] Commonize the m...

2015-04-01 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on the pull request: https://github.com/apache/spark/pull/5305#issuecomment-88752700 Jenkins, retest please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: spark-6106:Support user group mapping and grou...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5325#issuecomment-88750778 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-6663] [SQL] use Literal.create instread...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5320#issuecomment-88750863 [Test build #633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/633/consoleFull) for PR 5320 at commit [`1667604`](https://githu

[GitHub] spark pull request: spark-6106:Support user group mapping and grou...

2015-04-01 Thread colinmjj
GitHub user colinmjj opened a pull request: https://github.com/apache/spark/pull/5325 spark-6106:Support user group mapping and groups in view, modify and admin acls Add following configuration: spark.security.group.acls.enable: true|false to enable/disable group acl c

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/5324#issuecomment-88746568 After testing for a while, it seems that the retry does not work, but the timeout on client side can help: ``` 15/04/01 22:42:14 WARN PythonRDD: Timed out after 4 s

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5324#issuecomment-88739450 [Test build #29587 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29587/consoleFull) for PR 5324 at commit [`b838f35`](https://githu

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/5324 [SPARK-6667] [PySpark] retry after timeout to accept This is a small chance that the socket server will timed out after 3 seconds, but client had connected to it, then the Python process will hang.

[GitHub] spark pull request: [SPARK-6667] [PySpark] retry after timeout to ...

2015-04-01 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/5324#issuecomment-88738373 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88735424 cc @marmbrus @liancheng @yhuai @chenghao-intel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5303#issuecomment-88734217 [Test build #29586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29586/consoleFull) for PR 5303 at commit [`9dc32d1`](https://githu

[GitHub] spark pull request: [WIP][SPARK-3468] WebUI Timeline-View feature

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-88733710 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [WIP][SPARK-3468] WebUI Timeline-View feature

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-88733696 [Test build #29583 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29583/consoleFull) for PR 2342 at commit [`e85e9aa`](https://gith

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88732886 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88732869 [Test build #29584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29584/consoleFull) for PR 5208 at commit [`7a869c5`](https://gith

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5316#issuecomment-88732387 Thanks! I've merged this to master and branch-1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5316 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-01 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-88730386 @maropu Glad to know you're interested with the refactoring! Ideally, we'd better create a branch for a big code change, and then we can break down the task in

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-04-01 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-88730450 @jkbradley, thank you for your quick replying. I understand you are waiting for @freeman-lab 's review. I am making an example and writing the documentation in parallel n

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88729705 [Test build #29585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29585/consoleFull) for PR 5286 at commit [`c71fbc7`](https://githu

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-04-01 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/5267#discussion_r27632117 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,610 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-04-01 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/5267#discussion_r27632107 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,610 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88728689 /cc @brennonyork - this is giving me strange dependency messages even though the patch doesn't touch any pom files. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88728572 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-6663] [SQL] use Literal.create instread...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5320#issuecomment-88725778 [Test build #633 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/633/consoleFull) for PR 5320 at commit [`1667604`](https://github

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5316#issuecomment-88723687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5316#issuecomment-88723655 [Test build #29582 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29582/consoleFull) for PR 5316 at commit [`53b627a`](https://gith

[GitHub] spark pull request: [SPARK-6528][ML] Add IDF transformer

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5266#issuecomment-88721458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6528][ML] Add IDF transformer

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5266#issuecomment-88721453 [Test build #29581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29581/consoleFull) for PR 5266 at commit [`03fbecb`](https://gith

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-88717858 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-88717855 [Test build #29580 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29580/consoleFull) for PR 5297 at commit [`ba9b22b`](https://gith

[GitHub] spark pull request: [SPARK-2213] [SQL] sort merge join for spark s...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5208#issuecomment-88712992 [Test build #29584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29584/consoleFull) for PR 5208 at commit [`7a869c5`](https://githu

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2015-04-01 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-88711084 @chenghao-intel I'm also with your refactoring idea though, it's too big to merge into the master in bulk. ISTM this patch is better to split into some small ones.

[GitHub] spark pull request: [WIP][SPARK-3468] WebUI Timeline-View feature

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2342#issuecomment-88709309 [Test build #29583 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29583/consoleFull) for PR 2342 at commit [`e85e9aa`](https://githu

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5316#issuecomment-88708875 [Test build #29582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29582/consoleFull) for PR 5316 at commit [`53b627a`](https://githu

[GitHub] spark pull request: [SPARK-1502][YARN]Add config option to not inc...

2015-04-01 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on the pull request: https://github.com/apache/spark/pull/5294#issuecomment-88708773 @tgravescs @srowen @sryza As i have retest again, if we don't populate hadoop classpath, then in all case it dosen't work. This PR cann't solve this issue, i will c

[GitHub] spark pull request: [SPARK-1502][YARN]Add config option to not inc...

2015-04-01 Thread Sephiroth-Lin
Github user Sephiroth-Lin closed the pull request at: https://github.com/apache/spark/pull/5294 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-6658][SQL] Update DataFrame documentati...

2015-04-01 Thread chetmancini
Github user chetmancini commented on the pull request: https://github.com/apache/spark/pull/5316#issuecomment-88708808 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6639]Create a new script to start multi...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5323#issuecomment-88708786 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-6528][ML] Add IDF transformer

2015-04-01 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/5266#discussion_r27629781 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/IDF.scala --- @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: [SPARK-6528][ML] Add IDF transformer

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5266#issuecomment-88708429 [Test build #29581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29581/consoleFull) for PR 5266 at commit [`03fbecb`](https://githu

[GitHub] spark pull request: [SPARK-6639]Create a new script to start multi...

2015-04-01 Thread wangzhonnew
GitHub user wangzhonnew opened a pull request: https://github.com/apache/spark/pull/5323 [SPARK-6639]Create a new script to start multiple masters start-slaves.sh script is able to read from slaves file and start slaves node in multiple boxes. However in standalone mode if I wan

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88705534 **[Test build #632 timed out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/632/consoleFull)** for PR 5318 at commit [`0f66ec2`](https://gith

[GitHub] spark pull request: [SPARK-6663] [SQL] use Literal.create instread...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5320#issuecomment-88703996 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5886][ML] Add label indexer

2015-04-01 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4735#discussion_r27628773 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LabelIndexer.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3591][YARN]fire and forget for YARN clu...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5297#issuecomment-88700590 [Test build #29580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29580/consoleFull) for PR 5297 at commit [`ba9b22b`](https://githu

[GitHub] spark pull request: [SQL] Lock metastore client in analyzeTable

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5322#issuecomment-88700805 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SQL] Lock metastore client in analyzeTable

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5322#issuecomment-88700765 [Test build #29579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29579/consoleFull) for PR 5322 at commit [`351c9f3`](https://gith

[GitHub] spark pull request: [SPARK-6640][Core] Fix the race condition of c...

2015-04-01 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/5306#discussion_r27628404 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -71,12 +75,22 @@ private[spark] class HeartbeatReceiver(sc: SparkContext, schedu

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5319 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5319#issuecomment-88684054 Merging in 1.3 and master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88682995 Merged into master. The failed test is an unrelated yarn test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/5319#issuecomment-88682365 LGTM too --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5318 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5319#issuecomment-88680357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6578] Small rewrite to make the logic m...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5319#issuecomment-88680354 [Test build #29574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29574/consoleFull) for PR 5319 at commit [`7c62a64`](https://gith

[GitHub] spark pull request: [SQL] Throw UnsupportedOperationException inst...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5315#issuecomment-88679205 [Test build #29572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29572/consoleFull) for PR 5315 at commit [`c29e03b`](https://gith

[GitHub] spark pull request: [SQL] Lock metastore client in analyzeTable

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5322#issuecomment-88679219 [Test build #29579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29579/consoleFull) for PR 5322 at commit [`351c9f3`](https://githu

[GitHub] spark pull request: [SQL] Throw UnsupportedOperationException inst...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5315#issuecomment-88679212 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SQL] Lock metastore client in analyzeTable

2015-04-01 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/5322 [SQL] Lock metastore client in analyzeTable You can merge this pull request into a Git repository by running: $ git pull https://github.com/marmbrus/spark lockAnalyze Alternatively you can r

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88676039 [Test build #29575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29575/consoleFull) for PR 5318 at commit [`0f66ec2`](https://gith

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88676050 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88675476 [Test build #632 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/632/consoleFull) for PR 5318 at commit [`0f66ec2`](https://github

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88674477 [Test build #29573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29573/consoleFull) for PR 5318 at commit [`ad8c42f`](https://gith

[GitHub] spark pull request: [SPARK-6660][MLLIB] pythonToJava doesn't recog...

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5318#issuecomment-88674482 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88674434 [Test build #29576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29576/consoleFull) for PR 5286 at commit [`f36edd5`](https://gith

[GitHub] spark pull request: [SPARK-6627] Some clean-up in shuffle code.

2015-04-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5286#issuecomment-88674441 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29

[GitHub] spark pull request: [SPARK-5886][ML] Add label indexer

2015-04-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4735#discussion_r27625124 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LabelIndexer.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6553] [pyspark] Support functools.parti...

2015-04-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5206 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: use Literal.create instread of constructor

2015-04-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5320#issuecomment-88673311 [Test build #29578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29578/consoleFull) for PR 5320 at commit [`1667604`](https://githu

[GitHub] spark pull request: [SPARK-6553] [pyspark] Support functools.parti...

2015-04-01 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5206#issuecomment-88673324 LGTM, so I'm going to merge this into `master` (1.4.0) and `branch-1.3` (1.3.1). Thanks! --- If your project is set up for it, you can reply to this email and have yo

  1   2   3   4   >