[GitHub] spark pull request: [SQL] Don't shuffle code generated rows

2014-11-14 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/3263 [SQL] Don't shuffle code generated rows When sort based shuffle and code gen are on we were trying to ship the code generated rows during a shuffle. This doesn't work because the classes don't

[GitHub] spark pull request: [SPARK-4393] Fix memory leak in ConnectionMana...

2014-11-14 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3259#issuecomment-63022001 LGTM too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4327] [PySpark] Python API for RDD.rand...

2014-11-14 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3193#issuecomment-63021974 @mengxr I had updated it, the numpy is even slower when withReplacement is False. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346730 --- Diff: core/src/test/scala/org/apache/spark/ImplicitSuite.scala --- @@ -0,0 +1,73 @@ +package org.apache.spark --- End diff -- can u add the

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63022057 [Test build #23356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23356/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346733 --- Diff: core/src/test/scala/org/apache/spark/ImplicitSuite.scala --- @@ -0,0 +1,73 @@ +package org.apache.spark + +/** + * A test suite to make

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346738 --- Diff: core/src/test/scala/org/apache/spark/ImplicitSuite.scala --- @@ -0,0 +1,73 @@ +package org.apache.spark + +/** + * A test suite to make

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346755 --- Diff: core/src/test/scala/org/apache/spark/ImplicitSuite.scala --- @@ -0,0 +1,73 @@ +package org.apache.spark + +/** + * A test suite to make

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346838 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1427,46 +1427,56 @@ object SparkContext extends Logging { private[spark]

[GitHub] spark pull request: [SQL] Don't shuffle code generated rows

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3263#issuecomment-63022389 [Test build #23357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23357/consoleFull) for PR 3263 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3262#discussion_r20346905 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1427,46 +1427,56 @@ object SparkContext extends Logging {

[GitHub] spark pull request: [WIP] SPARK-4335, Mima false positives.

2014-11-14 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/3201#issuecomment-63022589 So basically our GenerateMimaIgnore adds all protected method to mima-ignores automatically. So we are not check binary compatibility on them anyway. The reason we

[GitHub] spark pull request: [WIP][SPARK-2816][SQL] Type-safe SQL Queries

2014-11-14 Thread marmbrus
Github user marmbrus closed the pull request at: https://github.com/apache/spark/pull/1759 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP][SPARK-2816][SQL] Type-safe SQL Queries

2014-11-14 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1759#issuecomment-63022692 Move to here: https://github.com/marmbrus/sql-typed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/3239#discussion_r20347028 --- Diff: docs/building-spark.md --- @@ -113,9 +113,9 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {%

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63023213 [Test build #23358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23358/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SQL] Minor cleanup of comments, errors and ov...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3257#issuecomment-63023343 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SQL] Minor cleanup of comments, errors and ov...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3257#issuecomment-63023338 [Test build #23353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23353/consoleFull) for PR 3257 at commit

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63023457 [Test build #23352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23352/consoleFull) for PR 2991 at commit

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63023461 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/3239#discussion_r20347403 --- Diff: docs/building-spark.md --- @@ -113,9 +113,9 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {% endhighlight

[GitHub] spark pull request: Added SparkGCE Script for Version 0.9.1

2014-11-14 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/681#issuecomment-63024468 @sigmoidanalytics Did you ever take a look at libcloud to see if we could abstract out some commonality with the spark_ec2 scripts? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-4327] [PySpark] Python API for RDD.rand...

2014-11-14 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3193#issuecomment-63024809 remove the test for withReplacement=True, because random.expovariate() is consistant between 2.6 and 2.7. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-3526 Add section about data locality to ...

2014-11-14 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/2519#issuecomment-63024861 In the absence of feedback about the above questions and in an effort to clarify this at least somewhat in the docs, I think we should merge this docs-only PR as-is for

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/3264 [SPARK-4398][PySpark] specialize sc.parallelize(xrange) `sc.parallelize(range(1 20), 1).count()` may take 15 seconds to finish and the rdd object stores the entire list, making task size very

[GitHub] spark pull request: [SPARK-4327] [PySpark] Python API for RDD.rand...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3193#issuecomment-63025255 [Test build #23359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23359/consoleFull) for PR 3193 at commit

[GitHub] spark pull request: SPARK-3663 Document SPARK_LOG_DIR and SPARK_PI...

2014-11-14 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/2518#issuecomment-63025274 @andrewor14 Updated, would you be able to merge in time for the Spark 1.2.0 release? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3264#issuecomment-63025682 [Test build #23360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23360/consoleFull) for PR 3264 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63025664 What's the distinction for intToIntWritable/writableConverters? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-3663 Document SPARK_LOG_DIR and SPARK_PI...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2518#issuecomment-63025707 [Test build #23361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23361/consoleFull) for PR 2518 at commit

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63025874 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63026569 [Test build #23362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23362/consoleFull) for PR 2991 at commit

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3264#discussion_r20348423 --- Diff: python/pyspark/context.py --- @@ -289,12 +289,31 @@ def stop(self): def parallelize(self, c, numSlices=None): -

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/3239#discussion_r20348481 --- Diff: docs/building-spark.md --- @@ -113,9 +113,9 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {%

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/3239#discussion_r20348576 --- Diff: docs/building-spark.md --- @@ -113,9 +113,9 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {% endhighlight

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3264#discussion_r20348996 --- Diff: python/pyspark/context.py --- @@ -289,12 +289,31 @@ def stop(self): def parallelize(self, c, numSlices=None): -

[GitHub] spark pull request: [SQL] Don't shuffle code generated rows

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3263#issuecomment-63028301 [Test build #23357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23357/consoleFull) for PR 3263 at commit

[GitHub] spark pull request: [SQL] Don't shuffle code generated rows

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3263#issuecomment-63028308 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3239#issuecomment-63029204 [Test build #23355 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23355/consoleFull) for PR 3239 at commit

[GitHub] spark pull request: SPARK-4375. no longer require -Pscala-2.10

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3239#issuecomment-63029216 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63030174 [Test build #23356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23356/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63030189 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63030582 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63030809 [Test build #23363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23363/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63031746 [Test build #23364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23364/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63032855 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63032841 [Test build #23358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23358/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3264#issuecomment-63037064 [Test build #23360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23360/consoleFull) for PR 3264 at commit

[GitHub] spark pull request: SPARK-3663 Document SPARK_LOG_DIR and SPARK_PI...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2518#issuecomment-63037099 [Test build #23361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23361/consoleFull) for PR 2518 at commit

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3264#issuecomment-63037074 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3663 Document SPARK_LOG_DIR and SPARK_PI...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2518#issuecomment-63037110 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4327] [PySpark] Python API for RDD.rand...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3193#issuecomment-63037231 [Test build #23359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23359/consoleFull) for PR 3193 at commit

[GitHub] spark pull request: [SPARK-4327] [PySpark] Python API for RDD.rand...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3193#issuecomment-63037243 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4398][PySpark] specialize sc.paralleliz...

2014-11-14 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3264#discussion_r20351945 --- Diff: python/pyspark/context.py --- @@ -289,12 +289,31 @@ def stop(self): def parallelize(self, c, numSlices=None): -

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63041639 [Test build #23365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23365/consoleFull) for PR 3248 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63042454 [Test build #23363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23363/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63042463 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63043183 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4062][Streaming]Add ReliableKafkaReceiv...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2991#issuecomment-63043173 **[Test build #23362 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23362/consoleFull)** for PR 2991 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63043256 [Test build #23364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23364/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63043268 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Added SparkGCE Script for Version 0.9.1

2014-11-14 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/681#issuecomment-63043646 Oh and also, all GitHub PRs should have a Jira SPARK-XYZ title in the title. I created one for you -- can you please update this PR name to be SPARK-4400 Added SparkGCE

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63044654 Only aliases around `GetField` should be trimmed. The reason of the last build failure is that some necessary aliases are trimmed by mistake. --- If your project is

[GitHub] spark pull request: Added SparkGCE Script for Version 0.9.1

2014-11-14 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/681#issuecomment-63045363 Correction, please use SPARK-1422 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63045964 [Test build #23366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23366/consoleFull) for PR 3248 at commit

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63051784 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63051780 [Test build #23365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23365/consoleFull) for PR 3248 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63053959 [Test build #23367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23367/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63055331 What's the distinction for intToIntWritable/writableConverters? `writableConverters` can work. Already done. Here is the code to test binary compatibility.

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63056607 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4322][SQL] Enables struct fields as sub...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3248#issuecomment-63056598 [Test build #23366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23366/consoleFull) for PR 3248 at commit

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread YanTangZhai
GitHub user YanTangZhai opened a pull request: https://github.com/apache/spark/pull/3265 [SPARK-4401] [SQL] RuleExecutor should log trace correct iteration num RuleExecutor should log trace correct iteration num You can merge this pull request into a Git repository by running:

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3265#issuecomment-63056938 Exact duplicate of https://github.com/apache/spark/pull/3180 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3265#issuecomment-63057086 [Test build #23368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23368/consoleFull) for PR 3265 at commit

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3265#issuecomment-63057192 [Test build #23368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23368/consoleFull) for PR 3265 at commit

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3265#issuecomment-63057194 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread YanTangZhai
Github user YanTangZhai closed the pull request at: https://github.com/apache/spark/pull/3265 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4401] [SQL] RuleExecutor should log tra...

2014-11-14 Thread YanTangZhai
Github user YanTangZhai commented on the pull request: https://github.com/apache/spark/pull/3265#issuecomment-63058643 @srowen Thanks. I close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Added SparkGCE Script for Version 0.9.1

2014-11-14 Thread sigmoidanalytics
Github user sigmoidanalytics commented on the pull request: https://github.com/apache/spark/pull/681#issuecomment-63060313 LibCloud looks decent for launching machine across datacenters providers, but to modify build scripts to either do installation from scratch (means longer

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63064757 [Test build #23367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23367/consoleFull) for PR 3262 at commit

[GitHub] spark pull request: [SPARK-4397][Core] Reorganize 'implicit's to i...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3262#issuecomment-63064764 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3722][Docs]minor improvement and fix in...

2014-11-14 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2579#issuecomment-63068715 looks good. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2014-11-14 Thread udnay
Github user udnay commented on a diff in the pull request: https://github.com/apache/spark/pull/3255#discussion_r20360054 --- Diff: python/pyspark/context.py --- @@ -191,7 +192,13 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,

[GitHub] spark pull request: [SPARK-4393] Fix memory leak in ConnectionMana...

2014-11-14 Thread cristianopris
Github user cristianopris commented on the pull request: https://github.com/apache/spark/pull/3259#issuecomment-63068893 Hi Josh, thanks for looking into this. I'm not sure this fully fixes the issue though. In my heap dump it looks like the memory leak is caused by the

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2014-11-14 Thread udnay
Github user udnay commented on a diff in the pull request: https://github.com/apache/spark/pull/3255#discussion_r20360107 --- Diff: python/pyspark/profiler.py --- @@ -0,0 +1,108 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request: [SPARK-3722][Docs]minor improvement and fix in...

2014-11-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2579 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-11-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-63069883 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-11-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-63069991 Good catch, LGTM, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-63070785 [Test build #23369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23369/consoleFull) for PR 3254 at commit

[GitHub] spark pull request: SPARK-4365: Remove unnecessary filter call on ...

2014-11-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3229#issuecomment-63071154 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-4365: Remove unnecessary filter call on ...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3229#issuecomment-63072072 [Test build #23370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23370/consoleFull) for PR 3229 at commit

[GitHub] spark pull request: SPARK-4338. Ditch yarn-alpha.

2014-11-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/3215#discussion_r20363809 --- Diff: docs/building-spark.md --- @@ -96,7 +75,7 @@ mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package mvn -Pyarn

[GitHub] spark pull request: SPARK-4338. Ditch yarn-alpha.

2014-11-14 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-63078085 was the yarn/stable directory removed with this? I couldn't tell from applying the patch and not sure how directory removals show up here. --- If your project is set

[GitHub] spark pull request: [SPARK-4344][DOCS] adding documentation on spa...

2014-11-14 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/3209#issuecomment-63078583 so @vanzin put up a pull request to implement it on yarn (https://github.com/apache/spark/pull/3233). Can we just close this one in favor of that? I'm fine

[GitHub] spark pull request: [SPARK-3090] Avoid not stopping SparkContext w...

2014-11-14 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2001#issuecomment-63078694 @sarutak can we close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-63081606 [Test build #23369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23369/consoleFull) for PR 3254 at commit

[GitHub] spark pull request: [SPARK-4386] Improve performance when writing ...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3254#issuecomment-63081617 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4365: Remove unnecessary filter call on ...

2014-11-14 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3229#issuecomment-63082026 [Test build #23370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23370/consoleFull) for PR 3229 at commit

[GitHub] spark pull request: SPARK-4365: Remove unnecessary filter call on ...

2014-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3229#issuecomment-63082039 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3530][MLLIB] pipeline and parameters wi...

2014-11-14 Thread huangjs
Github user huangjs commented on the pull request: https://github.com/apache/spark/pull/3099#issuecomment-63083266 Just a question. I'm comparing your example code to the scikit-learn examples in the design document. Looks like it's difficult for me to turn a configuration (data) to

  1   2   3   4   5   6   >