[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11867: [SPARK-14049] [CORE] Add functionality in spark history ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11867 **[Test build #71690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71690/testReport)** for PR 11867 at commit

[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71689/ Test PASSed. ---

[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16647 **[Test build #71689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71689/testReport)** for PR 16647 at commit

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 we don't need accurate number. we can have a confident margin. the bad with broken rdd chain is re-processing the rows. anything else? I don't think it is worth changing core and

[GitHub] spark issue #16643: [SPARK-17724][Streaming][WebUI] Unevaluated new lines in...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16643 **[Test build #71695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71695/testReport)** for PR 16643 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-01-19 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/16578 @mallman Thanks for let me know. I'll try your patch and check #14957 take over or not. I also think we need getting feedback from @liancheng , from our last discussion, liancheng may do

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 For 1, my idea is not use the proposal in this PR, 1. how you determine `total rows in all partitions are (much) more than limit number.` and then go into this code path and how to decide the

[GitHub] spark issue #16581: [SPARK-18589] [SQL] Fix Python UDF accessing attributes ...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71681/ Test PASSed. ---

[GitHub] spark issue #16581: [SPARK-18589] [SQL] Fix Python UDF accessing attributes ...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16581 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16581: [SPARK-18589] [SQL] Fix Python UDF accessing attributes ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16581 **[Test build #71681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71681/testReport)** for PR 16581 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Reduce shuffled data size of Bl...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71691/ Test PASSed. ---

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Reduce shuffled data size of Bl...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16637: [SPARK-19225][SQL]round decimal return normal value but ...

2017-01-19 Thread discipleforteen
Github user discipleforteen commented on the issue: https://github.com/apache/spark/pull/16637 ok. i will try to update code gen and MathFunctionsSuite.scala --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Reduce shuffled data size of Bl...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #71691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71691/testReport)** for PR 15730 at commit

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #71694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71694/testReport)** for PR 16028 at commit

[GitHub] spark issue #16645: [SPARK-19290][SQL] add a new extending interface in Anal...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16645 **[Test build #71693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71693/testReport)** for PR 16645 at commit

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 Ok. I think it is clearer now. We have two cases needed to solve: 1. After local limit, total rows in all partitions are (much) more than limit number. 2. After local limit, total rows

[GitHub] spark issue #16592: [SPARK-19235] [SQL] [TESTS] Enable Test Cases in DDLSuit...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16592 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16645: [SPARK-19290][SQL] add a new extending interface ...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16645#discussion_r97003427 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -62,15 +62,17 @@ private[hive] class

[GitHub] spark pull request #16645: [SPARK-19290][SQL] add a new extending interface ...

2017-01-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16645#discussion_r97003206 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -62,15 +62,17 @@ private[hive] class

[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16647 LGTM waiting for tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16645: [SPARK-19290][SQL] add a new extending interface in Anal...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16645 I also understand the concern of @yhuai . But, when the number of rules in a single batch keeps growing, using a single condition `resolved` is a little bit hard to maintain the order of rules

[GitHub] spark issue #16645: [SPARK-19290][SQL] add a new extending interface in Anal...

2017-01-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16645 also ping @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16651: [SPARK-19298][Core] History server can't match Malformed...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16651 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16651: [SPARK-19298][Core] History server can't match Ma...

2017-01-19 Thread sharkdtu
GitHub user sharkdtu opened a pull request: https://github.com/apache/spark/pull/16651 [SPARK-19298][Core] History server can't match MalformedInputException and prompt the detail logs while repalying eventlog History server can't match MalformedInputException and prompt the detail

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-19 Thread sarutak
Github user sarutak commented on the issue: https://github.com/apache/spark/pull/16582 I understand. if there are no additional comments from anyone by tomorrow, I'll merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #71692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71692/testReport)** for PR 15219 at commit

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16605 okay, I'll update this pr in that way, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16582 > I think it is because ResourceManager's web proxy might not handle https properly. Yeah, that's a known issue with enabling SSL for the web UI on YARN with self-signed certificates. ---

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 all partitions after local limit are about/nearly 100,000,000 rows --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 Do you mean totally rows in all partitions after local limit are about/nearly 100,000,000 rows? Or each partition after local limit has about/nearly 100,000,000 rows? --- If your project is set up

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-19 Thread sarutak
Github user sarutak commented on the issue: https://github.com/apache/spark/pull/16582 @vanzin I'm looking into this change and it works well on standalone-mode but doesn't on yarn-mode. I think it is because ResourceManager's web proxy might not handle https properly. It

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 Again, to clean, I am against the performance regression in flowing case 0. limit num is 100,000,000 1. the original table rows is very big, much larger than 100,000,000 rows 2. after

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 That is why I propose to avoid shuffling to single partition. We can save shuffling and keep parallelism. So I don't know what you are against? --- If your project is set up for it, you can reply

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 I think shuffle is ok, but shuffle to one partition leads to the performance issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 @scwf So sounds like it is the problem of shuffling. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 Assume local limit output 100,000,000 rows, then in global limit it will be take in a single partition, so it is very slow and can not use other free cores to improve the parallelism. --- If your

[GitHub] spark issue #16313: [SPARK-18899][SPARK-18912][SPARK-18913][SQL] refactor th...

2017-01-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16313 Actually this PR was not backported to 2.1, now I've backported. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 @scwf I am not sure if you really think about this. Can you describe the single partition issue based on your understanding? --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/16633 @viirya my team member post the mail list, actually we mean the case i listed above, the main issue is the single partition issue in global limit, if in that case you fall back to old global limit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Reduce shuffled data size of Bl...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #71691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71691/testReport)** for PR 15730 at commit

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16566 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16650: [SPARK-16554][CORE] Automatically Kill Executors and Nod...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16650 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16566 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71683/ Test PASSed. ---

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16344 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71686/ Test PASSed. ---

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16566 **[Test build #71683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71683/testReport)** for PR 16566 at commit

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16344 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16344 **[Test build #71686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71686/testReport)** for PR 16344 at commit

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-19 Thread jsoltren
GitHub user jsoltren opened a pull request: https://github.com/apache/spark/pull/16650 [SPARK-16554][CORE] Automatically Kill Executors and Nodes when they are Blacklisted ## What changes were proposed in this pull request? In SPARK-8425, we introduced a mechanism for

[GitHub] spark issue #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of f...

2017-01-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16586 Build started: [TESTS] `org.apache.spark.scheduler.SparkListenerSuite`

[GitHub] spark issue #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of f...

2017-01-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16586 Current status of this PR: It seems these tests below constantly failing during 6 times build (please check the logs in https://ci.appveyor.com/project/spark-test/spark/history).

[GitHub] spark issue #11867: [SPARK-14049] [CORE] Add functionality in spark history ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11867 **[Test build #71690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71690/testReport)** for PR 11867 at commit

[GitHub] spark pull request #11867: [SPARK-14049] [CORE] Add functionality in spark h...

2017-01-19 Thread paragpc
Github user paragpc commented on a diff in the pull request: https://github.com/apache/spark/pull/11867#discussion_r96997288 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/ApplicationListResource.scala --- @@ -43,11 +45,24 @@ private[v1] class

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71688/ Test PASSed. ---

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #71688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71688/testReport)** for PR 16536 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71687/ Test PASSed. ---

[GitHub] spark issue #16645: [SPARK-19290][SQL] add a new extending interface in Anal...

2017-01-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16645 @yhuai yes we can use conditions and put them in `resolved` to control when the rules will fire, but another problem is checking and normalization, it's hard to detect if it's done and we will do

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #71687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71687/testReport)** for PR 16537 at commit

[GitHub] spark issue #16534: [SPARK-19161][PYTHON][SQL] Improving UDF Docstrings

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16534 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16534: [SPARK-19161][PYTHON][SQL] Improving UDF Docstrings

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71685/ Test PASSed. ---

[GitHub] spark issue #16534: [SPARK-19161][PYTHON][SQL] Improving UDF Docstrings

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16534 **[Test build #71685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71685/testReport)** for PR 16534 at commit

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71684/ Test PASSed. ---

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16535 **[Test build #71684 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71684/testReport)** for PR 16535 at commit

[GitHub] spark issue #16605: [SPARK-18884][SQL] Support Array[_] in ScalaUDF

2017-01-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16605 SGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL][WIP] Add udf decorator

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71682/ Test PASSed. ---

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL][WIP] Add udf decorator

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL][WIP] Add udf decorator

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16533 **[Test build #71682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71682/testReport)** for PR 16533 at commit

[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16647 **[Test build #71689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71689/testReport)** for PR 16647 at commit

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16344 @actuaryzhang This test failure is caused by Jenkins was not stable, you just need to retest if you encounter similar issue. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #71688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71688/testReport)** for PR 16536 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #71687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71687/testReport)** for PR 16537 at commit

[GitHub] spark pull request #16638: spark-19115

2017-01-19 Thread ouyangxiaochen
Github user ouyangxiaochen commented on a diff in the pull request: https://github.com/apache/spark/pull/16638#discussion_r96993195 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -58,6 +58,7 @@ import org.apache.spark.util.Utils

[GitHub] spark issue #16534: [SPARK-19161][PYTHON][SQL] Improving UDF Docstrings

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16534 **[Test build #71685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71685/testReport)** for PR 16534 at commit

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16535 **[Test build #71684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71684/testReport)** for PR 16535 at commit

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16344 **[Test build #71686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71686/testReport)** for PR 16344 at commit

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-19 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16344 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16566 **[Test build #71683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71683/testReport)** for PR 16566 at commit

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL][WIP] Add udf decorator

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16533 **[Test build #71682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71682/testReport)** for PR 16533 at commit

[GitHub] spark issue #16637: [SPARK-19225][SQL]round decimal return normal value but ...

2017-01-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16637 Also I think we need to update the code gen path as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16637: [SPARK-19225][SQL]round decimal return normal value but ...

2017-01-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16637 Can you add a test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16581: [SPARK-18589] [SQL] Fix Python UDF accessing attributes ...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16581 **[Test build #71681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71681/testReport)** for PR 16581 at commit

[GitHub] spark issue #16647: [SPARK-19292][SQL] filter with partition columns should ...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16647 LGTM, except two comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16647: [SPARK-19292][SQL] filter with partition columns ...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16647#discussion_r96990182 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2014,4 +2014,17 @@ class SQLQuerySuite extends

[GitHub] spark pull request #16647: [SPARK-19292][SQL] filter with partition columns ...

2017-01-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16647#discussion_r96989355 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2014,4 +2014,17 @@ class SQLQuerySuite extends

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71679/ Test PASSed. ---

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #71679 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71679/testReport)** for PR 16536 at commit

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71677/ Test PASSed. ---

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16535 **[Test build #71677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71677/testReport)** for PR 16535 at commit

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16633: [SPARK-19274][SQL] Make GlobalLimit without shuffling da...

2017-01-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16633 That case only happens when the all row counts in all partitions are less than or (nearly) equal to the limit number. So it needs to scan (almost) all partitions. One possible way to deal

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71676/ Test PASSed. ---

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL][WIP] UserDefinedFunction.__ca...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #71676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71676/testReport)** for PR 16537 at commit

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16566 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16566: [SPARK-18821][SparkR]: Bisecting k-means wrapper in Spar...

2017-01-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16566 **[Test build #71675 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71675/testReport)** for PR 16566 at commit

<    1   2   3   4   5   >