[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2473#issuecomment-57043606 Merging in master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18122178 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -43,9 +46,34 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2473 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3675][SQL] Allow starting a JDBC server...

2014-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2515 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57043652 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20906/consoleFull) for PR 2388 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57043654 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18122185 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18122188 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18122187 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1689#issuecomment-57043705 Actually I looked at it again. I don't think it would block the scheduler because we compute partitions outside the scheduler thread. This approach looks good to me! ---

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1689#discussion_r18122197 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -222,7 +228,8 @@ class RangePartitioner[K : Ordering : ClassTag, V]( }

[GitHub] spark pull request: SPARK-CORE [SPARK-3651] Group common CoarseGra...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2533#issuecomment-57043734 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-CORE [SPARK-3651] Group common CoarseGra...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2533#issuecomment-57043731 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20907/consoleFull) for PR 2533 at commit

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1689#discussion_r18122212 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -113,8 +113,12 @@ class RangePartitioner[K : Ordering : ClassTag, V]( private var

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1689#discussion_r18122214 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -113,8 +113,12 @@ class RangePartitioner[K : Ordering : ClassTag, V]( private var

[GitHub] spark pull request: Minor cleanup to tighten visibility and remove...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2555#issuecomment-57043812 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Minor cleanup to tighten visibility and remove...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2555#issuecomment-57043807 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20901/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1689#issuecomment-57043822 @erikerlandson i'm going to merge this first. Maybe we can do the cleanup later. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1689 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1689#issuecomment-57043862 BTW one thing that would be great to add is a test that makes sure we don't block the main dag scheduler thread. The reason I think we don't block is that we call

[GitHub] spark pull request: Minor cleanup to tighten visibility and remove...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2555#issuecomment-57043873 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/167/consoleFull) for PR 2555 at commit

[GitHub] spark pull request: [SPARK-1021] Defer the data-driven computation...

2014-09-27 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1689#issuecomment-57043930 Have either of you thought about how to coordinate this with Josh's work on SPARK-3626? https://github.com/apache/spark/pull/2482 --- If your project is set up for

[GitHub] spark pull request: [SPARK-3543] Clean up Java TaskContext impleme...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2557#issuecomment-57044386 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/166/consoleFull) for PR 2557 at commit

[GitHub] spark pull request: [SPARK-3543] Clean up Java TaskContext impleme...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2557#issuecomment-57044581 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/168/consoleFull) for PR 2557 at commit

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57044911 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20909/consoleFull) for PR 2538 at commit

[GitHub] spark pull request: Minor cleanup to tighten visibility and remove...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2555#issuecomment-57045015 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/167/consoleFull) for PR 2555 at commit

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57045108 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20910/consoleFull) for PR 2538 at commit

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57045191 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20911/consoleFull) for PR 2538 at commit

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57045280 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20912/consoleFull) for PR 2538 at commit

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-09-27 Thread ash211
Github user ash211 commented on a diff in the pull request: https://github.com/apache/spark/pull/2401#discussion_r18122496 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MemoryUtils.scala --- @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-09-27 Thread brndnmtthws
Github user brndnmtthws commented on a diff in the pull request: https://github.com/apache/spark/pull/2401#discussion_r18122508 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MemoryUtils.scala --- @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: use --total-executor-cores rather than --co...

2014-09-27 Thread CrazyJvm
Github user CrazyJvm commented on the pull request: https://github.com/apache/spark/pull/2540#issuecomment-57045529 @andrewor14 already modified title according to your suggestion. Thx --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: Minor cleanup to tighten visibility and remove...

2014-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2555 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57045714 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57045812 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18122598 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18122597 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57046286 @mengxr Repartition is very slow when caching at Python side. It takes 9 minutes to do the repartition where as caching in Java only takes 5s. --- If your project is

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57046312 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20915/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3543] Clean up Java TaskContext impleme...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2557#issuecomment-57046439 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/168/consoleFull) for PR 2557 at commit

[GitHub] spark pull request: [SPARK-3658][SQL]Take thrift server as a daemo...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2509#discussion_r18122666 --- Diff: sbin/stop-thriftserver.sh --- @@ -0,0 +1,25 @@ +#!/usr/bin/env bash --- End diff -- This file should be executable, please `chmod

[GitHub] spark pull request: [SPARK-3658][SQL]Take thrift server as a daemo...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2509#discussion_r18122679 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -320,6 +320,10 @@ object SparkSubmit { } catch { case e:

[GitHub] spark pull request: [SPARK-3658][SQL]Take thrift server as a daemo...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2509#discussion_r18122728 --- Diff: sbin/spark-daemon.sh --- @@ -142,8 +142,12 @@ case $startStop in spark_rotate_log $log echo starting $command, logging to

[GitHub] spark pull request: [SPARK-3658][SQL]Take thrift server as a daemo...

2014-09-27 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2509#issuecomment-57047019 Thanks for working on this! I tested this PR locally and it works fine. But there are still some minor issues pending to be resolved, please refer to the comments for

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047298 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047297 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20909/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047499 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20910/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047501 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047595 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20911/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047597 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047693 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20912/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [WIP] [SPARK-2377] Python API for Streaming

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57047695 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57047727 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20915/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57047728 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2542#discussion_r18123371 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -110,9 +110,18 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-57051215 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20916/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2542#discussion_r18123559 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -110,9 +110,18 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-09-27 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2542#discussion_r18123585 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -110,9 +110,18 @@ abstract class LogicalPlan

[GitHub] spark pull request: [SPARK-3658][SQL]Take thrift server as a daemo...

2014-09-27 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on a diff in the pull request: https://github.com/apache/spark/pull/2509#discussion_r18123634 --- Diff: sbin/spark-daemon.sh --- @@ -142,8 +142,12 @@ case $startStop in spark_rotate_log $log echo starting $command,

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-57052882 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-1720][SPARK-1719] Add the value of LD_L...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1031#issuecomment-57052879 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20916/consoleFull) for PR 1031 at commit

[GitHub] spark pull request: fix the Building Spark url

2014-09-27 Thread yangl
GitHub user yangl opened a pull request: https://github.com/apache/spark/pull/2558 fix the Building Spark url the Building Spark url now should http://spark.apache.org/docs/latest/building-with-maven.html You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: fix the Building Spark url

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2558#issuecomment-57053406 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: fix the Building Spark url

2014-09-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2558#issuecomment-57053829 No, `building-spark.html` is the new URL of the page. This should not be changed. The project site however does need to be rebuilt soon. --- If your project is set up

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-27 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/2559 [SPARK-3707] [SQL] Fix bug of type coercion in DIV Calling `BinaryArithmetic.dataType` will throws exception until it's resolved, but in type coercion rule `Division`, seems doesn't follow

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57055354 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20917/consoleFull) for PR 2559 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055491 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20918/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055518 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20918/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055519 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-09-27 Thread BigCrunsh
Github user BigCrunsh commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-57055562 @jkbradley: yes, sounds great! I will give feedback to the design sheet asap. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57055921 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20919/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056007 @ezhulenev I've pulled in your changes and fixed a small scalastyle error. I agree that we should avoid having too many methods for all the parameter

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056016 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20919/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57056019 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-09-27 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2542#issuecomment-57056092 @cloud-fan Sorry, made a mistake in the snippet I used, it should be: ```sql CREATE TABLE t1(x INT); CREATE TABLE t2(a STRUCTx: INT, k INT); SELECT

[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support

2014-09-27 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-57057116 @marmbrus , just in case it will be out of date soon... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124375 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124373 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -43,9 +46,34 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124383 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124388 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124397 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2455#discussion_r18124403 --- Diff: core/src/main/scala/org/apache/spark/util/random/RandomSampler.scala --- @@ -53,56 +81,237 @@ trait RandomSampler[T, U] extends Pseudorandom with

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread erikerlandson
Github user erikerlandson commented on the pull request: https://github.com/apache/spark/pull/2455#issuecomment-57057948 @srowen, regarding the testing for iterator types, inside of 'dd', that was the only way I found (so far) that scala would accept. The best solution (imo) would

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread erikerlandson
Github user erikerlandson commented on the pull request: https://github.com/apache/spark/pull/2455#issuecomment-57058031 Actually, the *other* best solution would be the patch for SI-8835, which would make the whole thing go away. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-3250] Implement Gap Sampling optimizati...

2014-09-27 Thread erikerlandson
Github user erikerlandson commented on the pull request: https://github.com/apache/spark/pull/2455#issuecomment-57058068 @srowen I'll look into using `Math.log1p(-f)` in my next push. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-3478] [PySpark] Profile the Python task...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2556#issuecomment-57058156 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/169/consoleFull) for PR 2556 at commit

[GitHub] spark pull request: Docs : use --total-executor-cores rather tha...

2014-09-27 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2540#issuecomment-57058190 Ok great, I'm merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Docs : use --total-executor-cores rather tha...

2014-09-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2540 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57058942 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57058940 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20917/consoleFull)** after a configured wait of `120m`. --- If your project

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread ezhulenev
Github user ezhulenev commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059126 @sjbrunst you need to rollback your changes in TwitterAlgebirdCMD TwitterAlgebirdHLL (remove Nil for locations), and after that project will compile and I should

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread sjbrunst
Github user sjbrunst commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059353 @ezhulenev I've rolled back those changes now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57059419 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20920/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-3478] [PySpark] Profile the Python task...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2556#issuecomment-57060226 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/169/consoleFull) for PR 2556 at commit

[GitHub] spark pull request: [SPARK-3325] Add a parameter to the method pri...

2014-09-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2216#issuecomment-57060715 Ah I see - I thought this was an abstract class instead of a trait being modified in this patch. This is not an error with the compatibility checker - it's a

[GitHub] spark pull request: [SPARK-3325] Add a parameter to the method pri...

2014-09-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2216#issuecomment-57060863 Yeah so I looked into it a bit more and since `JavaDStream` extends `JavaDStreamLike` this will break user code with custom DStream's. The issue is that under the hood

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57060929 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20920/consoleFull) for PR 1717 at commit

[GitHub] spark pull request: [SPARK-2788] [STREAMING] Add location filterin...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1717#issuecomment-57060931 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3707] [SQL] Fix bug of type coercion in...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2559#issuecomment-57062261 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/170/consoleFull) for PR 2559 at commit

[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support

2014-09-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2344#discussion_r18125250 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -220,20 +220,52 @@ trait HiveTypeCoercion {

  1   2   >