[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81379/ Test PASSed. ---

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #81379 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81379/testReport)** for PR 17980 at commit

[GitHub] spark issue #19119: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-09-04 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19119 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-09-04 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/18902 Sure. I will create JIRA after this perf gap is confirmed. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2017-09-04 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r136776281 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,146 @@ trait CodegenSupport extends

[GitHub] spark issue #19121: [SPARK-21906][YARN][Spark Core]Don't runAsSparkUser to s...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19121 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #19121: [SPARK-21906][YARN][Spark Core]Don't runAsSparkUs...

2017-09-04 Thread yaooqinn
GitHub user yaooqinn opened a pull request: https://github.com/apache/spark/pull/19121 [SPARK-21906][YARN][Spark Core]Don't runAsSparkUser to switch UserGroupInformation in YARN mode ## What changes were proposed in this pull request? 1、The Yarn application‘s ugi is

[GitHub] spark issue #18983: [SPARK-21771][SQL]remove useless hive client in SparkSQL...

2017-09-04 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/18983 cc @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81380/testReport)** for PR 18875 at commit

[GitHub] spark pull request #19068: [SPARK-21428][SQL][FOLLOWUP] Reused state should ...

2017-09-04 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19068#discussion_r136766581 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -81,11 +81,7 @@ private[hive] object

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-09-04 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18748 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18628: [SPARK-18061][ThriftServer] Add spnego auth support for ...

2017-09-04 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18628 @cloud-fan @jiangxb1987, what do you think about this PR, I think it mostly copies from HS2, and it is quite isolated unless we enabled spnego, so it should be safe to merge. --- If your

[GitHub] spark issue #18624: [SPARK-21389][ML][MLLIB] Optimize ALS recommendForAll by...

2017-09-04 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18624 Thanks @MLnick , I think the ML ALS suite is ok, just MLLIB ALS suite is too simple. One possible enhancement is to add the same test cases as ML ALS suite. How do you think about it? --- If your

[GitHub] spark issue #18624: [SPARK-21389][ML][MLLIB] Optimize ALS recommendForAll by...

2017-09-04 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18624 @mpjlu also feel free to look into enhancing the tests. One issue however is we don't want to add to the run time too much as the ALS suite already is very heavy on time. --- If your project is

[GitHub] spark issue #18624: [SPARK-21389][ML][MLLIB] Optimize ALS recommendForAll by...

2017-09-04 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18624 @mpjlu sorry for delay on this. I think the idea of the change is good but I still need to review in detail. One concern I have is it now looks quite convoluted, so I want to see if we can simplify

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-09-04 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18748 Any further comments on this? @srowen @mpjlu @jkbradley? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-09-04 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18902 Seems fine to me to use the DF version even though it's slower. But we should open a JIRA issue to track where the gap is on the SQL side of things and try to improve the performance. --- If your

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #81379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81379/testReport)** for PR 17980 at commit

[GitHub] spark issue #19116: [SPARK-21903][BUILD] Upgrade scalastyle to 1.0.0.

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19116 **[Test build #81378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81378/testReport)** for PR 19116 at commit

[GitHub] spark pull request #19120: Please ignore this

2017-09-04 Thread ghandzhipeng
Github user ghandzhipeng closed the pull request at: https://github.com/apache/spark/pull/19120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #19116: [SPARK-21903][BUILD] Upgrade scalastyle to 1.0.0.

2017-09-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19116 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #19120: --

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19120 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-09-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18865 Just to summarise my opinion: I think I am unclear if we should call, selecting `_corrupt_record` alone, a bug to disallow. If I understood correctly, I think we are basically saying:

[GitHub] spark pull request #19120: Ghandzhipeng

2017-09-04 Thread ghandzhipeng
GitHub user ghandzhipeng opened a pull request: https://github.com/apache/spark/pull/19120 Ghandzhipeng ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81374/ Test FAILed. ---

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17014 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19116: [SPARK-21903][BUILD] Upgrade scalastyle to 1.0.0.

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19116 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81371/ Test FAILed. ---

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17014 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81376/ Test FAILed. ---

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81375/ Test FAILed. ---

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81374/testReport)** for PR 19086 at commit

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19117 **[Test build #81375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81375/testReport)** for PR 19117 at commit

[GitHub] spark issue #19116: [SPARK-21903][BUILD] Upgrade scalastyle to 1.0.0.

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19116 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19119: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81377/ Test FAILed. ---

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19117 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19086 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19116: [SPARK-21903][BUILD] Upgrade scalastyle to 1.0.0.

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19116 **[Test build #81371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81371/testReport)** for PR 19116 at commit

[GitHub] spark issue #19119: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19119 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19119: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19119 **[Test build #81377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81377/testReport)** for PR 19119 at commit

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17014 **[Test build #81376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81376/testReport)** for PR 17014 at commit

[GitHub] spark issue #19119: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19119 **[Test build #81377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81377/testReport)** for PR 19119 at commit

[GitHub] spark pull request #19119: [SPARK-21845] [SQL] Make codegen fallback of expr...

2017-09-04 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19119 [SPARK-21845] [SQL] Make codegen fallback of expressions configurable ## What changes were proposed in this pull request? We should make codegen fallback of expressions configurable. So far,

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136752724 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -200,7 +202,7 @@ class SourceProgress protected[sql]( */

[GitHub] spark issue #19118: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19118 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18975: [SPARK-4131] Support "Writing data into the filesystem f...

2017-09-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18975 @tejasapatil So far, we disallow overwriting a path that is also being read from. Atomicity is not guaranteed even if we use the staging directory. Atomicity still cannot be achieved if we use

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17014 **[Test build #81376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81376/testReport)** for PR 17014 at commit

[GitHub] spark pull request #19118: [SPARK-21882][CORE] OutputMetrics doesn't count w...

2017-09-04 Thread awarrior
GitHub user awarrior opened a pull request: https://github.com/apache/spark/pull/19118 [SPARK-21882][CORE] OutputMetrics doesn't count written bytes correctly in the saveAsHadoopDataset function spark-21882 ## What changes were proposed in this pull request?

[GitHub] spark pull request #19112: [SPARK-21901][SS] Define toString for StateOperat...

2017-09-04 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/19112#discussion_r136750244 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -200,7 +202,7 @@ class SourceProgress protected[sql](

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19117 **[Test build #81375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81375/testReport)** for PR 19117 at commit

[GitHub] spark issue #19117: [SPARK-21904] [SQL] Rename tempTables to tempViews in Se...

2017-09-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19117 @cloud-fan @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #19117: [SPARK-21904] [SQL] Rename tempTables to tempView...

2017-09-04 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19117 [SPARK-21904] [SQL] Rename tempTables to tempViews in SessionCatalog ### What changes were proposed in this pull request? `tempTables` is not right. To be consistent, we need to rename the

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17014 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17014 **[Test build #81373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81373/testReport)** for PR 17014 at commit

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17014 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81373/ Test FAILed. ---

[GitHub] spark issue #19115: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-09-04 Thread awarrior
Github user awarrior commented on the issue: https://github.com/apache/spark/pull/19115 ok, thx --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #19115: [SPARK-21882][CORE] OutputMetrics doesn't count w...

2017-09-04 Thread awarrior
Github user awarrior closed the pull request at: https://github.com/apache/spark/pull/19115 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #19086: [SPARK-21874][SQL] Support changing database when rename...

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19086 **[Test build #81374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81374/testReport)** for PR 19086 at commit

[GitHub] spark issue #17014: [SPARK-18608][ML] Fix double-caching in ML algorithms

2017-09-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17014 **[Test build #81373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81373/testReport)** for PR 17014 at commit

<    1   2