[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9439#issuecomment-153439539 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153439292 Thanks! Merging! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9439#issuecomment-153439471 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9439#issuecomment-153444262 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153444018 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153444121 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153446897 **[Test build #44941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44941/consoleFull)** for PR 9403 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786437 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class

[GitHub] spark pull request: [SPARK-11483] [SQL] Cleanup from spark-11329 f...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9442#issuecomment-153457712 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-03 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43775309 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -58,14 +58,46 @@ class

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-03 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43775098 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -87,6 +89,12 @@ private[sql] object

[GitHub] spark pull request: [SPARK-10116] [core] XORShiftRandom.hashSeed i...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8314#issuecomment-153418248 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10116] [core] XORShiftRandom.hashSeed i...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8314#issuecomment-153420965 **[Test build #44938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44938/consoleFull)** for PR 8314 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153434983 > Currently, the old one is broken, I'd to remove that one. @davies, are you referring to the old Aggregate1 interface or the old implementation of sort

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8995#issuecomment-153439396 I've opened #9439 to take this over. @a-roberts, do you mind closing this one for now? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-11-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/8669#discussion_r43784748 --- Diff: sbin/start-slaves.sh --- @@ -52,11 +51,11 @@ if [ "$SPARK_MASTER_IP" = "" ]; then fi if [ "$START_TACHYON" == "true" ]; then -

[GitHub] spark pull request: [WIP] [SPARK-9656] [MLlib] [Python] Add missin...

2015-11-03 Thread dusenberrymw
Github user dusenberrymw commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-153454390 @holdenk Could you review this and provide any thoughts you may have? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [WIP] [SPARK-9656] [MLlib] [Python] Add missin...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-153457314 **[Test build #44943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44943/consoleFull)** for PR 9441 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153465470 **[Test build #44947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44947/consoleFull)** for PR 8984 at commit

[GitHub] spark pull request: [SPARK-10116] [core] XORShiftRandom.hashSeed i...

2015-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/8314#discussion_r43774397 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameSuite.java --- @@ -258,7 +258,9 @@ public void testSampleBy() { DataFrame df

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153416242 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153416340 **[Test build #44928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44928/consoleFull)** for PR 9399 at commit

[GitHub] spark pull request: [DOC] Missing link to R DataFrame API doc

2015-11-03 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/9394#issuecomment-153430287 Thanks @Lewuathe -- LGTM. This is good to go if you can revert the change to the `DESCRIPTION` file. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153434074 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153433990 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153440296 Let's also have some test cases that having a column that is used in handled filters as well as in unhandled/unconvertible filters. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153440458 **[Test build #44937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44937/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9399 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786160 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class

[GitHub] spark pull request: [Spark-11478] [ML] ML StringIndexer return inc...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9440#issuecomment-153449189 **[Test build #44942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44942/consoleFull)** for PR 9440 at commit

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

2015-11-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9432#issuecomment-153452389 This LGTM being basically a copy of the existing approach in `start-thriftserver.sh` et al. It does not appear to interfere with any existing usage of the scripts since

[GitHub] spark pull request: [Spark-11478] [ML] ML StringIndexer return inc...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9440#issuecomment-153458079 **[Test build #44942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44942/consoleFull)** for PR 9440 at commit

[GitHub] spark pull request: [Spark-11478] [ML] ML StringIndexer return inc...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9440#issuecomment-153458142 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153465459 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153465476 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153465464 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11265] [YARN] YarnClient can't get toke...

2015-11-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9438#issuecomment-153415945 @steveloughran you have to choose the right target branch when submitting the PR. Can you close this one and open a new one with the correct target branch? --- If your

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153416822 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153416739 @tdas @zsxwing This is ready for another pass --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8840#issuecomment-153419572 OK great. @viirya can you submit a followup pr to address @yhuai's feedback. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [DOC] Missing link to R DataFrame API doc

2015-11-03 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/9394#discussion_r43780243 --- Diff: R/pkg/DESCRIPTION --- @@ -35,3 +35,4 @@ Collate: 'sparkR.R' 'stats.R' 'utils.R' +RoxygenNote: 5.0.0 --- End diff

[GitHub] spark pull request: [SPARK-11403] Log something when killing execu...

2015-11-03 Thread darabos
Github user darabos commented on the pull request: https://github.com/apache/spark/pull/9355#issuecomment-153429008 I've done an artificial test with Spark 1.5.1 and got the `# -XX:OnOutOfMemoryError="kill %p"` message on stderr. Maybe I just missed this originally, or in fact

[GitHub] spark pull request: [SPARK-11403] Log something when killing execu...

2015-11-03 Thread darabos
Github user darabos closed the pull request at: https://github.com/apache/spark/pull/9355 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153431540 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153431536 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153431665 I will merge it once it passes jenkins. Let's have a test to make sure those handled filters will not show up in the Filter operator. --- If your project is set up for

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153434863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9439 [SPARK-10949] Update Snappy version to 1.1.2 This is an updated version of #8995 by @a-roberts. Original description follows: Snappy now supports concatenation of serialized streams,

[GitHub] spark pull request: Update branch-1.5 for 1.5.2 release.

2015-11-03 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/9435 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...

2015-11-03 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-153438669 @squito I guess the only issue with that is whether we will ever have need in the future to get the ancestor shuffle dependencies without registering them. I doubt

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43784097 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -386,6 +386,18 @@ public void

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153442775 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153447090 @JoshRosen I meant the old implementation of switching, it's broken in master (not 1.5 branch), had not figured out the root cause. --- If your project is set up for

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153448297 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11398][SQL] unnecessary def dialectClas...

2015-11-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9349#issuecomment-153448480 @wzhfy The first part change is good (remove dialectClassName). But other one may introduce regression, when you have `spark.sql.dialect sql` in conf/spark-default.conf,

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43787218 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMapSuite.scala --- @@ -291,28 +278,27 @@ class

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153448294 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153451108 @davies: in the stacktrace that you posted, it looks like it's failing when trying to update a string column when extracting a partial aggregate result from an input

[GitHub] spark pull request: [SPARK-11480][CORE][WEBUI] Wrong callsite is d...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9437#issuecomment-153451177 **[Test build #44935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44935/consoleFull)** for PR 9437 at commit

[GitHub] spark pull request: [SPARK-11480][CORE][WEBUI] Wrong callsite is d...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9437#issuecomment-153451405 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-11420 Updating Stddev support via Impera...

2015-11-03 Thread JihongMA
Github user JihongMA commented on the pull request: https://github.com/apache/spark/pull/9380#issuecomment-153453307 I propose to return null for all cases which currently Double.NaN is returned. and change getStatistics() to return Any instead of Double. --- If your project is set

[GitHub] spark pull request: [WIP] [SPARK-9656] [MLlib] [Python] Add missin...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-153455136 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] [SPARK-9656] [MLlib] [Python] Add missin...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-153455109 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11483] [SQL] Cleanup from spark-11329 f...

2015-11-03 Thread nongli
GitHub user nongli opened a pull request: https://github.com/apache/spark/pull/9442 [SPARK-11483] [SQL] Cleanup from spark-11329 fix. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nongli/spark spark-11483 Alternatively you

[GitHub] spark pull request: [SPARK-11483] [SQL] Cleanup from spark-11329 f...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9442#issuecomment-153460112 **[Test build #44945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44945/consoleFull)** for PR 9442 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153416277 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153417447 **[Test build #44937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44937/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-10116] [core] XORShiftRandom.hashSeed i...

2015-11-03 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/8314#issuecomment-153419725 I think I've fixed the easy stuff (though I was lazy on `DataFrameStatSuite`, I kept it to just checking the values that happen to come out, rather than using a bound,

[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...

2015-11-03 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-153423715 @suyanNone Yes, do check uniqueness, and if that looks fine, then there isn't really a reason to use a Stack instead of Set for `parents`. --- If your project is

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153423795 **[Test build #44927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44927/consoleFull)** for PR 9399 at commit

[GitHub] spark pull request: [SPARK-11403] Log something when killing execu...

2015-11-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9355#issuecomment-153429423 OK np thanks for checking that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153435577 **[Test build #44939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44939/consoleFull)** for PR 9317 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43785059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala --- @@ -762,15 +679,7 @@ class

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153442505 **[Test build #44934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44934/consoleFull)** for PR 9399 at commit

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153442781 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/9313#discussion_r43788466 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1611,8 +1611,14 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9206#discussion_r43793910 --- Diff: network/common/src/main/java/org/apache/spark/network/util/TransportFrameDecoder.java --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153465473 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10116] [core] XORShiftRandom.hashSeed i...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8314#issuecomment-153418277 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153431268 **[Test build #44932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44932/consoleFull)** for PR 9424 at commit

[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...

2015-11-03 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-153434627 I see, so this comes from a "diamond" dependency. I'm not seeing that in your "after" image though -- am I just missing it with too many criss-crossing lines? I'm

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153434574 **[Test build #44933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44933/consoleFull)** for PR 9399 at commit

[GitHub] spark pull request: SPARK-11420 Updating Stddev support via Impera...

2015-11-03 Thread JihongMA
Github user JihongMA commented on the pull request: https://github.com/apache/spark/pull/9380#issuecomment-153451606 so for skewness and kurtosis in case of count =1, we want to return null instead of 0. I can address it, but instead of returning Double.NaN, should we return null

[GitHub] spark pull request: [SPARK-11275][SQL][WIP] Rollup and Cube Genera...

2015-11-03 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9419#issuecomment-153451771 @chenghao-intel @hvanhovell Unit test cases are added. Will finish the code for resolving the comments by @holdenk @rick-ibm @rxin @marmbrus @liancheng

[GitHub] spark pull request: [SPARK-11483] [SQL] Cleanup from spark-11329 f...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9442#issuecomment-153457683 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9206#discussion_r43793226 --- Diff: network/common/src/main/java/org/apache/spark/network/util/TransportFrameDecoder.java --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153416824 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153440723 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-153440724 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

2015-11-03 Thread CharlesYeh
Github user CharlesYeh commented on the pull request: https://github.com/apache/spark/pull/9432#issuecomment-153446371 Changed it to ./sbin/start-worker.sh and ./sbin/start-master.sh --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-03 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/8760#discussion_r43786312 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -83,8 +74,6 @@ private[spark] class TaskSetManager( val

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9383#discussion_r43786236 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala --- @@ -762,15 +679,7 @@ class

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153458601 **[Test build #44944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44944/consoleFull)** for PR 9403 at commit

[GitHub] spark pull request: SPARK-11420 Updating Stddev support via Impera...

2015-11-03 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/9380#issuecomment-153462689 @JihongMA I'm not sure about that. I don't think we should return `null`, instead of `Double.NaN`. Why do we need to change the return type? --- If your project is set

[GitHub] spark pull request: [SPARK-11483] [SQL] Cleanup from spark-11329 f...

2015-11-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9442#issuecomment-153462638 Dumb question, but why wasn't this part of SPARK-11329? or attached to it? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9439#issuecomment-153496753 **[Test build #44958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44958/consoleFull)** for PR 9439 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153496764 **[Test build #44959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44959/consoleFull)** for PR 9383 at commit

[GitHub] spark pull request: [SPARK-11485][SQL] Make DataFrameHolder and Da...

2015-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9445#issuecomment-153496774 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11485][SQL] Make DataFrameHolder and Da...

2015-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9445#issuecomment-153496721 **[Test build #44957 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44957/consoleFull)** for PR 9445 at commit

[GitHub] spark pull request: [SPARK-11275][SQL] Reimplement Expand as a Gen...

2015-11-03 Thread aray
Github user aray commented on a diff in the pull request: https://github.com/apache/spark/pull/9429#discussion_r43810369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -205,45 +205,30 @@ class Analyzer(

<    1   2   3   4   5   6   7   8   9   10   >