[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18899 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80523/ Test FAILed. ---

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80518/testReport)** for PR 18810 at commit

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18899 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #18736: [SPARK-21481][ML] Add indexOf method for ml.featu...

2017-08-10 Thread facaiy
Github user facaiy commented on a diff in the pull request: https://github.com/apache/spark/pull/18736#discussion_r132618802 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala --- @@ -80,20 +82,31 @@ class HashingTF @Since("1.4.0") (@Since("1.4.0")

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18899 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80517/ Test FAILed. ---

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18899 **[Test build #80517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80517/testReport)** for PR 18899 at commit

[GitHub] spark issue #18806: [SPARK-21600][docs] The description of "this requires sp...

2017-08-10 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18806 @srowen Help review the code,Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18253 I think `BusQueue` is a good abstraction, but we can still simplify other parts. My proposal: ``` // Do we need a better name like ListenesGroup? it's very similar to the previous

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80521/ Test FAILed. ---

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80521/testReport)** for PR 18907 at commit

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132616342 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18900 +1, similar to `CatalogTable.createTime`, we should have a `CatalogTablePartition.createTime` --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80522/testReport)** for PR 18810 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80521/testReport)** for PR 18907 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18907 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80520/testReport)** for PR 18907 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80520/ Test FAILed. ---

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18843 LGTM except one question, thanks for the fix! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132616033 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -149,4 +150,56 @@ class WholeStageCodegenSuite

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132615896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -844,24 +844,39 @@ object SQLConf { .stringConf

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132615780 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -844,24 +844,39 @@ object SQLConf { .stringConf

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132615759 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1139,9 +1154,14 @@ class SQLConf extends Serializable with

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80520/testReport)** for PR 18907 at commit

[GitHub] spark pull request #18907: [SPARK-18464][SQL][followup] support old table wh...

2017-08-10 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18907#discussion_r132615143 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -412,6 +412,11 @@ case class CatalogRelation(

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132615092 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -186,6 +186,18 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614716 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -659,13 +660,19 @@ case class StructsToJson(

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614534 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -590,4 +590,24 @@ class

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614524 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -590,4 +590,24 @@ class

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614326 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -186,6 +186,18 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614036 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3121,8 +3121,9 @@ object functions { to_json(e,

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3106,9 +3106,9 @@ object functions { } /** - *

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132614002 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3090,9 +3090,9 @@ object functions { } /** - *

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132613913 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -659,13 +660,19 @@ case class StructsToJson(

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-10 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/18902 I test the performance on a small data, the value in the following table is the average duration in seconds: |numColums| Old Mean | Old Median | New Mean | New Median |

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80515/ Test FAILed. ---

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80519/testReport)** for PR 18914 at commit

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18756 **[Test build #80515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80515/testReport)** for PR 18756 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18914 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18914 The change looks fine. @HyukjinKwon @gatorsmile Can you help trigger jenkins? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132613088 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -659,13 +660,19 @@ case class

[GitHub] spark pull request #18907: [SPARK-18464][SQL][followup] support old table wh...

2017-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18907#discussion_r132611451 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -412,6 +412,11 @@ case class CatalogRelation(

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18810 LGTM except for few comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132611417 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -149,4 +150,56 @@ class WholeStageCodegenSuite extends

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132611346 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132611289 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -149,4 +150,56 @@ class WholeStageCodegenSuite extends

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132611153 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,14 @@ case class

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132610861 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18899 Hi @sethah , the unit test is added. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80518/testReport)** for PR 18810 at commit

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-10 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132610543 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -149,4 +149,75 @@ class WholeStageCodegenSuite

[GitHub] spark issue #18913: [SPARK-21563][CORE] Fix race condition when serializing ...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80512/ Test PASSed. ---

[GitHub] spark issue #18913: [SPARK-21563][CORE] Fix race condition when serializing ...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18913 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18913: [SPARK-21563][CORE] Fix race condition when serializing ...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18913 **[Test build #80512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80512/testReport)** for PR 18913 at commit

[GitHub] spark pull request #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/18899#discussion_r132610165 --- Diff: project/MimaExcludes.scala --- @@ -1012,6 +1012,10 @@ object MimaExcludes {

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18899 **[Test build #80517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80517/testReport)** for PR 18899 at commit

[GitHub] spark pull request #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-10 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/18899#discussion_r132610049 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Vectors.scala --- @@ -635,8 +642,9 @@ class SparseVector @Since("2.0.0") ( nnz

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r132609949 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -659,13 +660,19 @@ case class StructsToJson(

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17583 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80516/testReport)** for PR 18810 at commit

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80516/ Test FAILed. ---

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17583 **[Test build #80514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80514/testReport)** for PR 17583 at commit

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80514/ Test PASSed. ---

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80516/testReport)** for PR 18810 at commit

[GitHub] spark issue #18897: [SPARK-21655][YARN] Support Kill CLI for Yarn mode

2017-08-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18897 I think this RPC layer could also serve for

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18877 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18877 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80511/ Test PASSed. ---

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18877 **[Test build #80511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80511/testReport)** for PR 18877 at commit

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18756 **[Test build #80515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80515/testReport)** for PR 18756 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18519 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80509/ Test PASSed. ---

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18519 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18519 **[Test build #80509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80509/testReport)** for PR 18519 at commit

[GitHub] spark pull request #18914: [MINOR][SQL][TEST]no uncache table in joinsuite t...

2017-08-10 Thread heary-cao
GitHub user heary-cao opened a pull request: https://github.com/apache/spark/pull/18914 [MINOR][SQL][TEST]no uncache table in joinsuite test ## What changes were proposed in this pull request? this may be a small mistake, let's fix it. thanks. ## How was this

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-10 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/18902 @hhbyyh Yes, I will test the performance. Btw, the median computation by call `stat.approxQuantile` will also transform df into rdd before aggregation. see

[GitHub] spark pull request #18912: [SPARK-21699][SQL] Remove unused getTableOption i...

2017-08-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18912 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18912: [SPARK-21699][SQL] Remove unused getTableOption in Exter...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18912 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80510/ Test PASSed. ---

[GitHub] spark issue #18912: [SPARK-21699][SQL] Remove unused getTableOption in Exter...

2017-08-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18912 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18912: [SPARK-21699][SQL] Remove unused getTableOption in Exter...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18912 **[Test build #80510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80510/testReport)** for PR 18912 at commit

[GitHub] spark issue #18912: [SPARK-21699][SQL] Remove unused getTableOption in Exter...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18912 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17583 **[Test build #80514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80514/testReport)** for PR 17583 at commit

[GitHub] spark pull request #17583: [SPARK-20271]Add FuncTransformer to simplify cust...

2017-08-10 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17583#discussion_r132605415 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/FuncTransformer.scala --- @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-08-10 Thread KevinZwx
Github user KevinZwx commented on the issue: https://github.com/apache/spark/pull/18903 @zsxwing @brkyvz Can you review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18901 LGTM generally, CC @vanzin to review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18901: [SPARK-21689][YARN] Download user jar from remote...

2017-08-10 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18901#discussion_r132604202 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -219,8 +219,10 @@ object SparkSubmit extends CommandLineUtils { *

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18724 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80507/ Test PASSed. ---

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18724 **[Test build #80507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80507/testReport)** for PR 18724 at commit

[GitHub] spark pull request #18724: [SPARK-21519][SQL] Add an option to the JDBC data...

2017-08-10 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18724#discussion_r132600534 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -1007,4 +1007,23 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80513/ Test PASSed. ---

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #80513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80513/testReport)** for PR 1 at commit

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #80513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80513/testReport)** for PR 1 at commit

[GitHub] spark pull request #18837: [Spark-20812][Mesos] Add secrets support to the d...

2017-08-10 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/18837#discussion_r132597784 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -529,18 +570,120 @@

  1   2   3   4   5   >