[GitHub] spark pull request: [SPARK-13702][CORE][SQL][MLLIB] Use diamond op...

2016-03-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11541#issuecomment-194010914 Thank you, @mengxr ! I will remove `pom.xml` and checkstyle part from this PR soon. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-13400] Stop using deprecated Octal esca...

2016-03-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11584#issuecomment-194004205 At last, it passed and there is no `Octal` warning. Thank you all. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-13702][CORE][SQL][MLLIB] Use diamond op...

2016-03-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11541#issuecomment-194014737 Hi, @mengxr . I removed the change on build related files. Now this PR contains pure `Java` and `md` files. Also, the description of PR/JIRA issue

[GitHub] spark pull request: [SPARK-13400] Stop using deprecated Octal esca...

2016-03-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11584#issuecomment-193987270 Thank you for confirming that, @holdenk ! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [MINOR][DOCS] Replace `DataFrame` with `Datase...

2016-03-12 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/11675 [MINOR][DOCS] Replace `DataFrame` with `Dataset` in Javadoc. ## What changes were proposed in this pull request? SPARK-13817 (PR #11656) replaces `DataFrame` with `Dataset` from from

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and MiMa plugi...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11669#issuecomment-195708035 I've took a look quickly for plugins in `plugins.sbt` except libraries. The following are the result. **A. Up-to-date** - sbt-mima-plugin 0.1.9 (ok

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and MiMa plugi...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11669#issuecomment-195699718 Hi, @srowen . Thank you for review. Yes, I got inspiration for this PR from your previous JIRA issue. I tried to upgrade `sbt` first in #11661

[GitHub] spark pull request: [MINOR][DOCS] Replace `DataFrame` with `Datase...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11675#issuecomment-195698835 Hi, @liancheng . Could you review this PR, please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and MiMa plugi...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11669#issuecomment-195710498 Thank you for giving further directions, @srowen . Then, could you merge this PR? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SQL] Removes Dataset.collectRows()/takeRows()

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11678#issuecomment-195779128 Hi, @liancheng . Sorry for leaving a message here. Could you review #11675 if you have some time? It's a supplement update for your replacing `DataFrame

[GitHub] spark pull request: [WIP] Update sbt-dependency-graph/sbteclipse-p...

2016-03-12 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/11680 [WIP] Update sbt-dependency-graph/sbteclipse-plugin plugin based on SPARK-13834 ## What changes were proposed in this pull request? This PR bumps the following sbt plugins based

[GitHub] spark pull request: [MINOR][DOCS] Replace `DataFrame` with `Datase...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11675#issuecomment-195871389 Oh, thank you, @liancheng ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [WIP] Update sbt-dependency-graph/sbteclipse-p...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/11680 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and sbt plugin...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11669#issuecomment-195887027 Hi, @srowen . I updated the description of this PR and JIRA issue. In short, I could update most plugins except `sbt-assembly`. It show many

[GitHub] spark pull request: [WIP] Update sbt-dependency-graph/sbteclipse-p...

2016-03-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11680#issuecomment-195887050 Merged into PR #11669 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13986][CORE][MLLIB] Make `@DeveloperApi...

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11797#issuecomment-198142439 Just rebased. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [MINOR][SQL][BUILD] Remove duplicated lines

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11773#discussion_r56431059 --- Diff: project/MimaExcludes.scala --- @@ -299,13 +299,11 @@ object MimaExcludes { // [SPARK-13244][SQL] Migrates DataFrame to Dataset

[GitHub] spark pull request: [SPARK-13942][CORE][DOCS] Remove Shark-related...

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11770#issuecomment-197591298 Thank you, @rxin ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13942][CORE][DOCS] Remove Shark-related...

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11770#issuecomment-197568243 Sure! No problem. Thank you for discuss about this. Thank you, @andrewor14 and @rxin . --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [MINOR][SQL][BUILD] Remove duplicated lines

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11773#issuecomment-197954539 Thank you, @rxin ! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13986][CORE][MLLIB] Make `@DeveloperApi...

2016-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11797#issuecomment-198138169 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13920][BUILD] MIMA checks should apply ...

2016-03-15 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/11751 [SPARK-13920][BUILD] MIMA checks should apply to @Experimental and @DeveloperAPI APIs ## What changes were proposed in this pull request? We are able to change `@Experimental

[GitHub] spark pull request: [SPARK-13920][BUILD] MIMA checks should apply ...

2016-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11751#discussion_r56283381 --- Diff: tools/src/main/scala/org/apache/spark/tools/GenerateMIMAIgnore.scala --- @@ -39,12 +39,14 @@ import org.clapper.classutil.ClassFinder

[GitHub] spark pull request: [SPARK-13920][BUILD] MIMA checks should apply ...

2016-03-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11751#issuecomment-197166425 Thank you. MiMa check parts also passed in both tests. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12653][SQL] Re-enable test "SPARK-8489:...

2016-03-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11744#discussion_r56286847 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala --- @@ -87,13 +88,16 @@ class HiveSparkSubmitSuite

[GitHub] spark pull request: [SPARK-13920][BUILD] MIMA checks should apply ...

2016-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11751#issuecomment-197162819 @JoshRosen , I removed the legacy code and related comments now. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-13920][BUILD] MIMA checks should apply ...

2016-03-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11751#issuecomment-197170829 Thank you, @rxin ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12653][SQL] Re-enable test "SPARK-8489:...

2016-03-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11744#issuecomment-197199216 I updated the code to use exact matching. Now, it becomes much robust as you said. Thank you again, @rxin. --- If your project is set up for it, you can

[GitHub] spark pull request: [MINOR][DOCS] Update build descriptions and co...

2016-03-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11838#issuecomment-198593293 I added some minor things, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [MINOR][DOCS] Update build descriptions and co...

2016-03-18 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/11838 [MINOR][DOCS] Update build descriptions and commands ## What changes were proposed in this pull request? This PR updates Scala and Hadoop versions in the build description

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and sbt plugin...

2016-03-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11669#discussion_r55934983 --- Diff: project/SparkBuild.scala --- @@ -144,7 +143,7 @@ object SparkBuild extends PomBuild { "org.spark-project" %% "ge

[GitHub] spark pull request: [SPARK-13834][BUILD] Update sbt and sbt plugin...

2016-03-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11669#discussion_r55935019 --- Diff: project/SparkBuild.scala --- @@ -241,7 +240,7 @@ object SparkBuild extends PomBuild { /* Enable shared settings on all projects

[GitHub] spark pull request: [SPARK-13686][MLLIB][STREAMING] Add a construc...

2016-03-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11527#issuecomment-195935665 Right, as you said, we can use optimizer object directly. I thought that's the main reason why this PR doesn't get any response from ML part committers

[GitHub] spark pull request: [SPARK-13660][SQL][Tests] ContinuousQuerySuite...

2016-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11703#issuecomment-196550005 Hi, @keypointt . FYI, the MiMa failures seem to irrelevant to your PR. It occurs on #11700 , too. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-13660][SQL][Tests] ContinuousQuerySuite...

2016-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11703#issuecomment-196553476 Sure, you're right. If the claims are correct, you did the right things. I meant the MiMa status is strange now. :) --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-14164][MLLIB] Improve input layer valid...

2016-03-29 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11964#issuecomment-202972522 Thank you, @MLnick . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14282][SQL] CodeFormatter should handle...

2016-03-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12072#discussion_r57966347 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala --- @@ -74,7 +74,8 @@ private class

[GitHub] spark pull request: [SPARK-14282][SQL] CodeFormatter should handle...

2016-03-30 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12072 [SPARK-14282][SQL] CodeFormatter should handle oneline comment with /* */ properly ## What changes were proposed in this pull request? This PR improves `CodeFormatter` to fix

[GitHub] spark pull request: [SPARK-14219][GraphX] Fix `pickRandomVertex` n...

2016-03-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12018#issuecomment-202642216 Hi, @rxin . Could you review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-14219][GraphX] Fix `pickRandomVertex` n...

2016-03-28 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12018 [SPARK-14219][GraphX] Fix `pickRandomVertex` not to fall into infinite loops for graphs with one vertex ## What changes were proposed in this pull request? Currently

[GitHub] spark pull request: [SPARK-14219][GRAPHX] Fix `pickRandomVertex` n...

2016-03-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12021#issuecomment-202656295 Hi, @rxin . This is [SPARK-14219] for **branch-1.6**. Thank you again. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-14219][GraphX] Fix `pickRandomVertex` n...

2016-03-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12018#issuecomment-202646784 Sure. I will create another PR for branch-1.6, too. Thank you, @rxin . --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-14219][GRAPHX] Fix `pickRandomVertex` n...

2016-03-28 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12021 [SPARK-14219][GRAPHX] Fix `pickRandomVertex` not to fall into infinit… ## What changes were proposed in this pull request? Currently, `GraphOps.pickRandomVertex()` falls

[GitHub] spark pull request: [SPARK-14164][MLLIB] Improve input layer valid...

2016-03-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11964#issuecomment-203551212 Sure! Thank you so much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14282][SQL] CodeFormatter should handle...

2016-03-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12072#issuecomment-203679256 Thank you, @rxin and @sameeragarwal . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12192#issuecomment-207107119 Sure. I agree for merging. The underlying problem will be investigated and resolved someday. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207123707 The following is the failure message. It seems o be irrelevant. I'll rebase this to trigger again. ``` [error] Test

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12192#issuecomment-207025582 Hi, All. The original code is made by me, so I feel guilty. :) I think Jenkins build does not show additional information for this PR since our

[GitHub] spark pull request: [HOTFIX][SPARK-14402] Fix ExpressionDescriptio...

2016-04-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12192#issuecomment-207031919 Just for a record, here is the current error message from **doc-jar**. (The version of doc-jar is the latest.) ``` [INFO] <<< scala-maven-plugin:

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12242 [SPARK-14465][BUILD] Checkstyle should check all Java files ## What changes were proposed in this pull request? Currently, `checkstyle` is configured to check the files under `src

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207085112 Yes. It's strange for me, too. We may change the paths, but there is one understandable example: `package-info.java`. (It's not related to this PR, but I found

[GitHub] spark pull request: [WIP][SPARK-14402][CORE] initcap UDF doesn't m...

2016-04-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12175#issuecomment-205902962 Hi, @srowen . I minimized the change on master. * Undo the changes on `common` module. * Implement `initCap` by the following changes

[GitHub] spark pull request: [SPARK-14402][SQL] initcap UDF doesn't match H...

2016-04-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12175#issuecomment-205913234 Thank you, @srowen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14402][SQL] initcap UDF doesn't match H...

2016-04-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12175#issuecomment-205928323 Thank you, @marmbrus . I will update the scala docand add description annotation for InitCap. And the for last comment, if you agree, I hope to implement

[GitHub] spark pull request: [SPARK-14444] Add a new scalastyle `NoScalaDoc...

2016-04-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12221#issuecomment-206602929 Thank you, @andrewor14 ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14444] Add a new scalastyle `NoScalaDoc...

2016-04-06 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12221 [SPARK-1] Add a new scalastyle `NoScalaDoc` to prevent ScalaDoc-style multiline comments ## What changes were proposed in this pull request? According to the Spark Code Style

[GitHub] spark pull request: [SPARK-14444][BUILD] Add a new scalastyle `NoS...

2016-04-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12221#issuecomment-206611644 Thank you, @andrewor14 ! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [MINOR][DOCS] Use multi-line JavaDoc comments ...

2016-04-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12130#issuecomment-206611258 Hi, @rxin . I made a new PR for ScalaStyle rule enforcement ( #12221 ). --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-14402][SQL] initcap UDF doesn't match H...

2016-04-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12175#issuecomment-205977980 Finally, I passed the Jenkins with the updated code. Please let me know if there is something I missed. Thank you, @marmbrus and @srowen . --- If your

[GitHub] spark pull request: [SPARK-14577][SQL] Add spark.sql.codegen.maxCa...

2016-04-13 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12353 [SPARK-14577][SQL] Add spark.sql.codegen.maxCaseBranches config option ## What changes were proposed in this pull request? We currently disable codegen for `CaseWhen` if the number

[GitHub] spark pull request: [SPARK-14577][SQL] Add spark.sql.codegen.maxCa...

2016-04-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12353#discussion_r59509786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -52,6 +52,8 @@ case class ExprCode

[GitHub] spark pull request: [MINOR][SQL] Re-enable `explode()` and `json_t...

2016-04-12 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12329 [MINOR][SQL] Re-enable `explode()` and `json_tuple()` testcases in ExpressionToSQLSuite ## What changes were proposed in this pull request? Since [SPARK-12719: SQL Generation

[GitHub] spark pull request: [SPARK-14545][SQL] Improve `LikeSimplification...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12312#discussion_r59412120 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/LikeSimplificationSuite.scala --- @@ -61,6 +61,20 @@ class

[GitHub] spark pull request: [SPARK-14545][SQL] Improve `LikeSimplification...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12312#issuecomment-209011439 Hi, @rxin . Sorry for late response. I added comments and changed the testcase name according to your comments. Thank you for review! --- If your project

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12280#issuecomment-209004510 Thank you, @rxin and @srowen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14545][SQL] Improve `LikeSimplification...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12312#discussion_r59412030 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -529,6 +530,9 @@ object LikeSimplification

[GitHub] spark pull request: [SPARK-14580][SQL] Hive IfCoercion should pres...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12340#issuecomment-209132312 Amazing. There must be some problem in Jenkins server. It's JVM error. ``` Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m

[GitHub] spark pull request: [SPARK-14580][SQL] Hive IfCoercion should pres...

2016-04-12 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12340 [SPARK-14580][SQL] Hive IfCoercion should preserve predicate. ## What changes were proposed in this pull request? Currently, `HiveTypeCoercion.IfCoercion` removes all predicates

[GitHub] spark pull request: [SPARK-14580][SQL] Hive IfCoercion should pres...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12340#issuecomment-209128788 Hi, @rxin . Could you review this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14580][SQL] Hive IfCoercion should pres...

2016-04-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12340#issuecomment-209133359 Rebased to trigger Jenkins again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Non-Nu...

2016-04-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12267#issuecomment-207665847 Hi, @rxin . Could you review this PR please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Non-Nu...

2016-04-08 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12267 [SPARK-14502][SQL] Add optimization for Non-Nullable Binary Comparison Simplification ## What changes were proposed in this pull request? For non-nullable binary comparisons, we can

[GitHub] spark pull request: [SPARK-14545][SQL] Improve `LikeSimplification...

2016-04-11 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12312 [SPARK-14545][SQL] Improve `LikeSimplification` by adding `a%b` rule ## What changes were proposed in this pull request? Current `LikeSimplification` handles the following four rules

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12280#discussion_r59127601 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -72,11 +72,11 @@ class ExecutorClassLoader( override def

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12280#discussion_r59127580 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -148,29 +147,25 @@ private

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207867755 Oh, @yhuai . Thank you for deep and detail review! I'll fix all of them and improve as possible as I can. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207879147 Thank you. I agree. I will apply them all related descriptions. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207874641 Sure. You mean 'Returns the XXX of a set of numbers' -> 'Returns the XXX calculated from values of a group' Right? --- If your project is set

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12185#discussion_r59123665 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -23,7 +23,10 @@ import

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207887639 Thank you for merging, @rxin . Jenkins seems to have some trouble during merging. :) --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-14508] Add a new ScalaStyle Rule `OmitB...

2016-04-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12280 [SPARK-14508] Add a new ScalaStyle Rule `OmitBracesInCase` ## What changes were proposed in this pull request? According to the [Spark Code Style Guide](https://cwiki.apache.org

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207875330 Since I just borrowed from the Hive, it seems to be weird. I think we can improve them in this PR as much as possible. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12280#issuecomment-207923837 Thank you, @srowen . I updated this PR according to your comments. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-208045636 Thank you, @yhuai and @rxin . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12280#issuecomment-208045946 Hi, @srowen . Could you merge this when you have some time? :) --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Binary...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12267#discussion_r59173880 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BinaryComparisonSimplificationSuite.scala --- @@ -0,0 +1,101

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Binary...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12267#discussion_r59173780 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BinaryComparisonSimplificationSuite.scala --- @@ -0,0 +1,101

[GitHub] spark pull request: [MINOR][DOCS] Fix wrong data types in JSON Dat...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12290#issuecomment-208113694 Sure, I verified on master branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [MINOR][DOCS] Fix wrong data types in JSON Dat...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12290#issuecomment-208117591 Other examples in `http://spark.apache.org/docs/latest/sql-programming-guide.html` also shows `long`. By the way, as you requested, I cleanly synced

[GitHub] spark pull request: [MINOR][DOCS] Fix wrong data types in JSON Dat...

2016-04-10 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12290 [MINOR][DOCS] Fix wrong data types in JSON Datasets example. ## What changes were proposed in this pull request? This PR fixes the `age` data types from `integer` to `long` in `SQL

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207847958 Thank you so much! @rxin :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207848160 @srowen . Could you merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14508][BUILD] Add a new ScalaStyle Rule...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12280#issuecomment-208219802 Sure, no problem. Then, I will keep this PR up-to-date if there occurs new code to be changed. Thank you! --- If your project is set up for it, you can

[GitHub] spark pull request: [MINOR][DOCS] Fix wrong data types in JSON Dat...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12290#issuecomment-208219964 Thank you for merging, @srowen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Binary...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12267#issuecomment-208222674 The failure is irrelevant to this PR. So, I rebased to the master in order to trigger Jenkins again. ``` [info] - event ordering *** FAILED *** (1 minute

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Non-Nu...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12267#discussion_r59154120 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -787,6 +788,28 @@ object BooleanSimplification

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Non-Nu...

2016-04-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12267#discussion_r59155788 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -787,6 +788,28 @@ object BooleanSimplification

[GitHub] spark pull request: [SPARK-14502][SQL] Add optimization for Non-Nu...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12267#discussion_r59158077 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -787,6 +788,28 @@ object BooleanSimplification

[GitHub] spark pull request: [SPARK-14051][SQL] Implement `Double.NaN==Floa...

2016-04-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/11868#discussion_r59162191 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala --- @@ -402,11 +402,13 @@ trait Row extends Serializable

[GitHub] spark pull request: [HOTFIX] Fix `optional` to `createOptional`.

2016-04-05 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12187 [HOTFIX] Fix `optional` to `createOptional`. ## What changes were proposed in this pull request? This PR fixes the following line. ``` private[spark] val STAGING_DIR

<    1   2   3   4   5   6   7   8   9   10   >