[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153501829 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153501486 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43812681 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +491,12 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43813114 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153506641 **[Test build #44963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44963/consoleFull)** for PR 9446 at commit

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153507504 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark-11478] [ML] ML StringIndexer return inc...

2015-11-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9440#issuecomment-153512622 cc @yanboliang --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11490][SQL] variance should alias var_s...

2015-11-03 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9449 [SPARK-11490][SQL] variance should alias var_samp instead of var_pop. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-11490

[GitHub] spark pull request: [SPARK-11477][SQL] support create Dataset from...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9434#discussion_r43820137 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -499,6 +499,15 @@ class SQLContext private[sql]( new Dataset[T](this,

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43820035 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -499,6 +499,10 @@ class SQLContext private[sql]( new Dataset[T](this,

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43820849 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/Encoder.scala --- @@ -37,3 +35,39 @@ trait Encoder[T] extends Serializable {

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43820879 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/Encoder.scala --- @@ -37,3 +35,39 @@ trait Encoder[T] extends Serializable {

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9358#issuecomment-153517822 It would be really great to also try and create a test suite that uses java 8 lambdas (though we may need to pull this into a separate PR as I'm not sure how many

[GitHub] spark pull request: [SPARK-11491] Update build to use Scala 2.10.5

2015-11-03 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9450 [SPARK-11491] Update build to use Scala 2.10.5 Spark should build against Scala 2.10.5, since that includes a fix for Scaladoc that will fix doc snapshot publishing:

[GitHub] spark pull request: [SPARK-11491] Update build to use Scala 2.10.5

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9450#issuecomment-153518606 **[Test build #44969 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44969/consoleFull)** for PR 9450 at commit

[GitHub] spark pull request: WIP: Optimize Inner joins with skewed null val...

2015-11-03 Thread vidma
GitHub user vidma opened a pull request: https://github.com/apache/spark/pull/9451 WIP: Optimize Inner joins with skewed null values Draft of first step in optimizing skew in joins (it is quite common to have skew in data, and lots of `nulls` on either side of join is quite common

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496072 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153496189 Going to merge this. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43811146 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43811779 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43811640 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153501196 **[Test build #44950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44950/consoleFull)** for PR 8531 at commit

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9446 [SPARK-11489][SQL] Only include common first order statistics in GroupedData We added a bunch of 2nd order statistics such as skewness and kurtosis to GroupedData. I don't think they are common enough

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9446#discussion_r43813063 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedData.scala --- @@ -316,86 +264,48 @@ class GroupedData protected[sql]( } /**

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153502185 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153502213 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9446#discussion_r43813023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedData.scala --- @@ -316,86 +264,48 @@ class GroupedData protected[sql]( } /**

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153503614 **[Test build #44962 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44962/consoleFull)** for PR 8984 at commit

[GitHub] spark pull request: [SPARK-11486][SQL] Fix TungstenAggregate's han...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9447#issuecomment-153506808 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11486][SQL] Fix TungstenAggregate's han...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9447#issuecomment-153506821 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11484][WebUi] Using proxyBase set by sp...

2015-11-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9448#issuecomment-153508301 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153508491 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153508426 **[Test build #44962 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44962/consoleFull)** for PR 8984 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153508492 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11484][WebUi] Using proxyBase set by sp...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9448#issuecomment-153509355 **[Test build #44966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44966/consoleFull)** for PR 9448 at commit

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153509437 **[Test build #44956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44956/consoleFull)** for PR 9403 at commit

[GitHub] spark pull request: [SPARK-11484][WebUi] Using proxyBase set by sp...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9448#issuecomment-153509047 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11433] [SQL] Cleanup the subquery name ...

2015-11-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/9385#discussion_r43817697 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1019,7 +1019,16 @@ class Analyzer( * scoping

[GitHub] spark pull request: SPARK-4921. TaskSetManager.dequeueTask returns...

2015-11-03 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3816#issuecomment-153513026 I think this is the right fix. However, more pairs of eyes on this change would be better. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43820335 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -499,6 +499,10 @@ class SQLContext private[sql]( new Dataset[T](this,

[GitHub] spark pull request: [SPARK-11491] Update build to use Scala 2.10.5

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9450#issuecomment-153518428 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43821203 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/Encoder.scala --- @@ -37,3 +35,39 @@ trait Encoder[T] extends Serializable {

[GitHub] spark pull request: [SPARK-11491] Update build to use Scala 2.10.5

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9450#issuecomment-153518050 /cc @srowen @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11477][SQL] support create Dataset from...

2015-11-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9434#issuecomment-153518772 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43810862 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43810744 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11485][SQL] Make DataFrameHolder and Da...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9445#issuecomment-153499217 **[Test build #44960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44960/consoleFull)** for PR 9445 at commit

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153500624 For stddev/variance, it's clear that using ImerativeAggregate will be faster. Then we don't have case that need the MutableProject to be atomic, should we revert this

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153500615 @davies, @yhuai: I tried taking two sample regression test queries that you listed here and putting them into AggregationQuerySuite, which should automatically

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153501959 So, you tried `TungstenAggregationQueryWithControlledFallbackSuite` and it did not fail? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153502212 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153502172 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153503203 I tried with master, it did failed with the sam exception: ``` [info] *** 1 TEST FAILED *** [error] Failed: Total 6, Failed 1, Errors 0, Passed 5

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43813962 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153504056 **[Test build #44961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44961/consoleFull)** for PR 9446 at commit

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43814406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -441,6 +537,17 @@ class Dataset[T] private( /** Collects the elements to an

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153506241 Yep, confirmed and reproduced. If we just wanted to address that bug, we could do something like #9447, but I think the broader set of changes here are a good idea.

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153507966 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11484][WebUi] Using proxyBase set by sp...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9448#issuecomment-153507926 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153507883 **[Test build #44961 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44961/consoleFull)** for PR 9446 at commit

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153507933 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153507942 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153507945 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11486][SQL] Fix TungstenAggregate's han...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9447#issuecomment-153508899 **[Test build #44964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44964/consoleFull)** for PR 9447 at commit

[GitHub] spark pull request: [SPARK-11484][WebUi] Using proxyBase set by sp...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9448#issuecomment-153509074 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153509965 **[Test build #44965 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44965/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9421#discussion_r43817473 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisCheckpointState.scala --- @@ -16,39 +16,77 @@ */ package

[GitHub] spark pull request: [SPARK-11378][STREAMING] make StreamingContext...

2015-11-03 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9336#issuecomment-153512190 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: SPARK-4921. TaskSetManager.dequeueTask returns...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3816#issuecomment-153513337 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11490][SQL] variance should alias var_s...

2015-11-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9449#issuecomment-153514954 Note that this also includes changes from #9446. Look at the last commit for diff. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496037 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11275][SQL] Reimplement Expand as a Gen...

2015-11-03 Thread aray
Github user aray commented on a diff in the pull request: https://github.com/apache/spark/pull/9429#discussion_r43811041 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -205,45 +205,30 @@ class Analyzer(

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43811813 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153501491 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153501309 cc @davies @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9446#discussion_r43812669 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedData.scala --- @@ -26,42 +26,14 @@ import org.apache.spark.sql.catalyst.expressions._ import

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r43814263 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153504632 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153504624 There was a problem in my test-selection regex... trying again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-11489][SQL] Only include common first o...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9446#issuecomment-153504587 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153505891 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153505715 **[Test build #44952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44952/consoleFull)** for PR 9313 at commit

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153505892 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11486][SQL] Fix TungstenAggregate's han...

2015-11-03 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9447 [SPARK-11486][SQL] Fix TungstenAggregate's handling of empty agg buffers during fallback This fixes an issue where TungstenAggregate would extract the wrong aggregation buffer columns when

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9206#issuecomment-153506214 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9206#issuecomment-153506072 **[Test build #44951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44951/consoleFull)** for PR 9206 at commit

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9206#issuecomment-153506212 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-4921. TaskSetManager.dequeueTask returns...

2015-11-03 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3816#issuecomment-153512508 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153513069 ah sorry just saw this message. Since the changed behavior is easier to reason about, let's keep it. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: SPARK-4921. TaskSetManager.dequeueTask returns...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3816#issuecomment-153513322 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11490][SQL] variance should alias var_s...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9449#issuecomment-153515606 **[Test build #44968 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44968/consoleFull)** for PR 9449 at commit

[GitHub] spark pull request: [SPARK-11269][SQL] Java API support & test cas...

2015-11-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9358#discussion_r43820397 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -441,6 +537,17 @@ class Dataset[T] private( /** Collects the elements to

[GitHub] spark pull request: [SPARK-11491] Update build to use Scala 2.10.5

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9450#issuecomment-153518453 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8984#discussion_r43797767 --- Diff: R/pkg/R/types.R --- @@ -0,0 +1,41 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] spark pull request: [SPARK-11260][SPARKR] with() function support

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9443#issuecomment-153468251 **[Test build #44948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44948/consoleFull)** for PR 9443 at commit

[GitHub] spark pull request: [SPARK-11265] [YARN] YarnClient can't get toke...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9438#issuecomment-153467262 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10359] Enumerate Spark's dependencies i...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8531#issuecomment-153471179 **[Test build #44950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44950/consoleFull)** for PR 8531 at commit

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153471074 Hi @zsxwing , could you take a look at this? There have been some superficial test failures, but the last one in CoarseMesosSchedulerBackendSuite looks like it

[GitHub] spark pull request: [SPARK-9104][SPARK-9105][SPARK-9106][SPARK-910...

2015-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/7753#discussion_r43801975 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Fix typo in WebUI

2015-11-03 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/9444 Fix typo in WebUI You can merge this pull request into a Git repository by running: $ git pull https://github.com/jaceklaskowski/spark TImely-fix Alternatively you can review and

<    1   2   3   4   5   6   7   8   9   10   >