[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-02-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10757#discussion_r53006549 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -499,6 +489,8 @@ abstract class BinaryOperator

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-02-16 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-184599206 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-02-15 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-184522878 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-02-15 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-184140279 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12503][SPARK-12505] Limit pushdown in U...

2016-02-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11121#issuecomment-184019891 @cloud-fan I agree. For SQL generation, currently we can only focus on resolved plans parsed from HiveQL. @gatorsmile I think after finishing that part, we may gain

[GitHub] spark pull request: [SPARK-13314][SQL] Fixes malformed WholeStageC...

2016-02-14 Thread liancheng
Github user liancheng closed the pull request at: https://github.com/apache/spark/pull/11200 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-13314][SQL] Fixes malformed WholeStageC...

2016-02-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11200#issuecomment-184019253 Oh, I didn't realize that those extra vertical lines are used to indicate scopes of codegen'd stages, then the original tree string makes sense. I thought they were

[GitHub] spark pull request: [SPARK-13314][SQL] Fixes malformed WholeStageC...

2016-02-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11200#issuecomment-184013932 At least there should be a vertical tree line connecting two `WholeStageCodegen` nodes to indicate that they are both children of `Union`? The `:` in plan tree

[GitHub] spark pull request: [SPARK-13314][SQL] Fixes malformed WholeStageC...

2016-02-14 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/11200 [SPARK-13314][SQL] Fixes malformed WholeStageCodegen tree string When the query plan contains binary operator(s), `WholeStageCodegen` tree string can be malformed. This PR fixes this issue

[GitHub] spark pull request: [SPARK-13314][SQL] Fixes malformed WholeStageC...

2016-02-14 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11200#issuecomment-183882394 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13282][SQL] LogicalPlan toSql should ju...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11171#discussion_r52714801 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/SQLBuilderTest.scala --- @@ -50,10 +50,8 @@ abstract class SQLBuilderTest extends QueryTest

[GitHub] spark pull request: [SPARK-13282][SQL] LogicalPlan toSql should ju...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11171#discussion_r52714511 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -37,157 +39,137 @@ import

[GitHub] spark pull request: [SPARK-13282][SQL] LogicalPlan toSql should ju...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11171#discussion_r52714530 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -37,157 +39,137 @@ import

[GitHub] spark pull request: [SPARK-13282][SQL] LogicalPlan toSql should ju...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11171#discussion_r52714504 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -37,157 +39,137 @@ import

[GitHub] spark pull request: [SPARK-13282][SQL] LogicalPlan toSql should ju...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11171#discussion_r52714696 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/SQLBuilderTest.scala --- @@ -50,10 +50,8 @@ abstract class SQLBuilderTest extends QueryTest

[GitHub] spark pull request: [SPARK-13263] [SQL] SQL Generation Support for...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11148#discussion_r52728653 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -119,6 +120,18 @@ class SQLBuilder(logicalPlan: LogicalPlan

[GitHub] spark pull request: [SPARK-13263] [SQL] SQL Generation Support for...

2016-02-12 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11148#discussion_r52729051 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala --- @@ -145,6 +145,16 @@ class LogicalPlanToSQLSuite extends

[GitHub] spark pull request: [SPARK-13205] [SQL] SQL Generation Support for...

2016-02-10 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11084#issuecomment-182680640 LGTM, merged to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-10 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11050#issuecomment-182673514 LGTM, merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13244] [WIP] Prototyping DataFrame to D...

2016-02-09 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/11137 [SPARK-13244] [WIP] Prototyping DataFrame to Dataset migration This PR prototypes the `DataFrame`-to-`Dataset` migration. It demonstrates main migration strategies but hasn't migrate all

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-07 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11050#issuecomment-181175638 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13205] [SQL] SQL Generation Support for...

2016-02-04 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11084#discussion_r51976383 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/SQLBuilder.scala --- @@ -142,7 +142,15 @@ class SQLBuilder(logicalPlan: LogicalPlan

[GitHub] spark pull request: [SPARK-13205] [SQL] SQL Generation Support for...

2016-02-04 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11084#issuecomment-180195917 One minor issue, overall looks good. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-04 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11050#discussion_r51977520 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -114,16 +117,21 @@ abstract class Attribute

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-04 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11050#discussion_r51977612 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -114,16 +117,21 @@ abstract class Attribute

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-04 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11050#discussion_r51977660 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -197,7 +209,8 @@ case class

[GitHub] spark pull request: [SPARK-12725] [SQL] Resolving Name Conflicts i...

2016-02-04 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11050#discussion_r51977477 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -413,9 +413,11 @@ class Analyzer

[GitHub] spark pull request: [SPARK-13101][SQL][branch-1.6] nullability of ...

2016-02-03 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11042#issuecomment-179314804 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13157][SQL] Support any kind of input f...

2016-02-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11052#discussion_r51758504 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ASTNodeSuite.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-13157][SQL] Support any kind of input f...

2016-02-03 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11052#issuecomment-179373144 This PR probably also fixes SPARK-13150, which was about flaky tests caused by the @ removal bug. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-13101][SQL] nullability of array type e...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11035#issuecomment-178940673 I'm investigating the failed flaky tests, and @davies is marking them as ignored temporarily to bring back the PR builder. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/11040 [SPARK-13150] [SQL] [WIP] Adds extensive logs to investigate SPARK-13150 **DON'T MERGE** This PR adds extensive logs to investigate two flaky test cases tracked by SPARK-13150. You can

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11040#discussion_r5166 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -786,6 +787,7 @@ abstract

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179028394 Found 3 interesting log lines: ``` 16/02/02 18:38:00 INFO ParseDriver: Parsing command: ADD JAR file:///home/jenkins/workspace/SparkPullRequestBuilder@2

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179030322 Actually the `ADD JAR` command doesn't fail, but the jar wasn't successfully added, and the following `CREATE TEMPORARY FUNCTION` command failed. Verified

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179030901 My guess is that when a PR Jenkins build is triggered by the Spark PR dashboard and the build is scheduled to any builder other than the 1st builder, which doesn't

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179031474 Oh, maybe the `@` character is treated as the separator between user/password and host/port in a URL. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-13150] [SQL] Fixes flaky Thrift server ...

2016-02-02 Thread liancheng
Github user liancheng closed the pull request at: https://github.com/apache/spark/pull/11040 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-13150] [SQL] Fixes flaky Thrift server ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179074117 Seems that the `@` removal bug is causing more serious build failures like [this one][1]. My theory is that some test case added a resource whose path containing

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179038743 @davies The `@` character is removed after being parsed by HiveQL parser (`ParserDriver`). --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179039604 Complete log line snippet: ``` 16/02/02 18:38:00 INFO ParseDriver: Parsing command: ADD JAR file:///home/jenkins/workspace/SparkPullRequestBuilder@2/sql

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179039783 I'm going to fix this issue using the workaround mentioned above and file a separate JIRA ticket for the `@` character issue. --- If your project is set up

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179044891 The Thrift server and the `ADD JAR` command are both pretty tricky to get right, I think it would be better to enable these tests first? --- If your project is set

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179045198 Actually I've already pushed the fix to this PR branch... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-13150] [SQL] Fixes flaky Thrift server ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179048502 Filed SPARK-13157 to track the `@` character issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-13101][SQL][branch-1.6] nullability of ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11042#issuecomment-179053489 Memo: `CodegenContext.references` can only hold `Expression` objects in branch-1.6. That's why the error message can't be moved to codegen references like what

[GitHub] spark pull request: [SPARK-13150] [SQL] Fixes flaky Thrift server ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-179055300 Ah, our Jenkins PR builder overrides `java.io.tmpdir` property and the created temporary directory is still under a parent directory like `SparkPullRequestBuilder@2

[GitHub] spark pull request: [SPARK-13101][SQL] nullability of array type e...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11035#issuecomment-178901752 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11040#discussion_r51668973 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -493,12 +493,12 @@ class

[GitHub] spark pull request: [SPARK-13150] [SQL] [WIP] Adds extensive logs ...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11040#issuecomment-178957690 Just scheduled a build wave, trying to reproduce the test failure. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-13101][SQL] nullability of array type e...

2016-02-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11035#issuecomment-178961608 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2016-02-01 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-178237590 Oh, just found that it's covered by PR #10650 and has already been backported. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-8813][SQL]Combine splits by size

2016-02-01 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/9097#discussion_r51503553 --- Diff: sql/hive/src/main/java/org/apache/spark/sql/hive/mapred/CombineSplit.java --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-8813][SQL]Combine splits by size

2016-02-01 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/9097#discussion_r51503559 --- Diff: sql/hive/src/main/java/org/apache/spark/sql/hive/mapred/CombineSplitRecordReader.java --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-8813][SQL]Combine splits by size

2016-02-01 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/9097#discussion_r51503571 --- Diff: sql/hive/src/main/java/org/apache/spark/sql/hive/mapred/CombineSplitRecordReader.java --- @@ -0,0 +1,128 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-8813][SQL]Combine splits by size

2016-02-01 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/9097#discussion_r51503595 --- Diff: sql/hive/src/main/java/org/apache/spark/sql/hive/mapred/CombineSplitInputFormat.java --- @@ -0,0 +1,110 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2016-02-01 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-178118573 @marmbrus Should we backport this to branch-1.6? I missed @yhuai's comment above and haven't done it yet. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2016-01-31 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10388#issuecomment-177674881 Cherry-picked to branch-1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-30 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10979#issuecomment-177407724 LGTM, thanks for helping fixing this! Will get it merged once I get access to my laptop. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-176894068 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-177044623 @rxin Shall we merge this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12818] Polishes spark-sketch module

2016-01-29 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/10985 [SPARK-12818] Polishes spark-sketch module You can merge this pull request into a Git repository by running: $ git pull https://github.com/liancheng/spark sketch-polishing Alternatively

[GitHub] spark pull request: [SPARK-12818] Polishes spark-sketch module

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10985#issuecomment-176930921 Also went through generated Javadoc to fix all visibility and style issues. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-12818] Polishes spark-sketch module

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10985#issuecomment-176943645 Thanks for the review, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12818] Polishes spark-sketch module

2016-01-29 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10985#discussion_r51308254 --- Diff: common/sketch/src/main/java/org/apache/spark/util/sketch/BloomFilter.java --- @@ -68,14 +65,14 @@ int getVersionNumber

[GitHub] spark pull request: [SPARK-12818] Polishes spark-sketch module

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10985#issuecomment-176933324 cc @cloud-fan @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-176955903 @rxin Thanks for the review and the summary. Will finish them in a follow-up PR. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10757#discussion_r51230274 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -39,11 +39,11 @@ case class ScalaUDF

[GitHub] spark pull request: [SPARK-13050] [Build] Scalatest tags fail buil...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10954#issuecomment-176620214 I'm merging this. Thanks for fixing this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11955][SQL] Mark optional fields in mer...

2016-01-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/9940#discussion_r51232182 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala --- @@ -359,6 +361,18 @@ object StructType extends AbstractDataType

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-176616783 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-28 Thread liancheng
Github user liancheng closed the pull request at: https://github.com/apache/spark/pull/10972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10972#issuecomment-176622130 I'm closing this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10972#issuecomment-176618727 @viirya It would be great if you can help since you are pretty familiar with this part of code :) --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10972#discussion_r51232373 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRelation.scala --- @@ -799,9 +799,26 @@ private[sql] object

[GitHub] spark pull request: [SPARK-13050] [Build] Scalatest tags fail buil...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10954#issuecomment-176619493 @vanzin I see, thanks for the explanation! Then this LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-12818][SQL] Specialized integral and st...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10968#issuecomment-176306162 Just reverted all accidental formatting changes made by IntelliJ. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12818][SQL] Specialized integral and st...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10968#issuecomment-176304119 @rxin Updated, thanks for the review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10757#issuecomment-176466748 It took me sometime to fix those R test failures. Hopefully the last commit fixes everything (finger crossed). --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12401][SQL] Add integration tests for p...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10596#issuecomment-176477402 Thank, I'm merging this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13050] [Build] Scalatest tags fail buil...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10954#issuecomment-176477123 Hm, but I do find that spark-test-tags is listed as default compile dependency in other modules. Is that necessary? @JoshRosen --- If your project is set up

[GitHub] spark pull request: [SPARK-12799] Simplify various string output f...

2016-01-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10757#discussion_r51201446 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala --- @@ -131,17 +131,17 @@ abstract class OrcSuite extends QueryTest

[GitHub] spark pull request: [SPARK-13050] [Build] Scalatest tags fail buil...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10954#issuecomment-176475976 One of the requirements of the spark-sketch module is to have zero external runtime dependencies. Is it possible to make it a test dependency? It seems quite

[GitHub] spark pull request: [SPARK-11955][SQL] Mark optional fields in mer...

2016-01-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/9940#issuecomment-176495087 Thanks! I'm going to merging this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13070][SQL] Better error message when P...

2016-01-28 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/10972 [SPARK-13070][SQL] Better error message when Parquet schema merging fails Now we also report path and schema of the file in trouble. You can merge this pull request into a Git repository

[GitHub] spark pull request: [SPARK-12818][SQL] Specialized integral and st...

2016-01-27 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/10968 [SPARK-12818][SQL] Specialized integral and string types for Count-min Sketch This PR is a follow-up of #10911. It adds specialized update methods for `CountMinSketch` so that we can avoid

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50877569 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -309,4 +311,84 @@ final class DataFrameStatFunctions private

[GitHub] spark pull request: [SQL] Minor Scaladoc format fix

2016-01-26 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/10926 [SQL] Minor Scaladoc format fix Otherwise the `^` character is always marked as error in IntelliJ since it represents an unclosed superscript markup tag. You can merge this pull request

[GitHub] spark pull request: [SPARK-12728][SQL] Integrates SQL generation w...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10733#discussion_r50903539 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateViewAsSelect.scala --- @@ -44,55 +47,83 @@ private[hive] case class

[GitHub] spark pull request: [SPARK-12728][SQL] Integrates SQL generation w...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10733#discussion_r50903806 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala --- @@ -155,6 +155,17 @@ private[sql] trait SQLTestUtils

[GitHub] spark pull request: [SPARK-12728][SQL] Integrates SQL generation w...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10733#discussion_r50904655 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1358,67 +1358,112 @@ class SQLQuerySuite extends

[GitHub] spark pull request: [SPARK-12937][SQL] bloom filter serialization

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10920#discussion_r50905576 --- Diff: common/sketch/src/main/java/org/apache/spark/util/sketch/BloomFilter.java --- @@ -39,6 +43,28 @@ * The implementation is largely based

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50906324 --- Diff: common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java --- @@ -348,4 +374,30 @@ public static CountMinSketchImpl

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50906771 --- Diff: common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java --- @@ -17,17 +17,37 @@ package

[GitHub] spark pull request: [SQL] Minor Scaladoc format fix

2016-01-26 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10926#issuecomment-175267990 I'm merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10911#issuecomment-175275779 Josh is looking into the PySpark test failure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50914282 --- Diff: common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java --- @@ -348,4 +374,30 @@ public static CountMinSketchImpl

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50919715 --- Diff: sql/core/pom.xml --- @@ -44,6 +44,11 @@ org.apache.spark + spark-sketch_2.10 --- End diff

[GitHub] spark pull request: [SPARK-12935][SQL] DataFrame API for Count-Min...

2016-01-26 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10911#discussion_r50919784 --- Diff: sql/core/pom.xml --- @@ -44,6 +44,11 @@ org.apache.spark + spark-sketch_2.10 --- End diff

[GitHub] spark pull request: [SPARK-12728][SQL] Integrates SQL generation w...

2016-01-26 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10733#issuecomment-175306181 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

<    9   10   11   12   13   14   15   16   17   18   >