[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-31 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451914#comment-15451914 ] Cheng Hao commented on SPARK-17299: --- Or come after SPARK-14878 ? > TRIM/LTRIM/RTRIM strips characters

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-31 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451810#comment-15451810 ] Cheng Hao commented on SPARK-17299: --- Yes, that's my bad, I thought it should be the same behavior of

[jira] [Created] (SPARK-15859) Optimize the Partition Pruning with Disjunction

2016-06-09 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-15859: - Summary: Optimize the Partition Pruning with Disjunction Key: SPARK-15859 URL: https://issues.apache.org/jira/browse/SPARK-15859 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-07 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318654#comment-15318654 ] Cheng Hao commented on SPARK-15730: --- [~jameszhouyi], can you please verify this fixing? > [Spark SQL]

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300072#comment-15300072 ] Cheng Hao commented on SPARK-15034: --- [~yhuai], but it probably not respect the `hive-site.xml`, and

[jira] [Commented] (SPARK-13894) SQLContext.range should return Dataset[Long]

2016-03-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195274#comment-15195274 ] Cheng Hao commented on SPARK-13894: --- The existing functions "SQLContext.range()" returns the underlying

[jira] [Commented] (SPARK-13326) Dataset in spark 2.0.0-SNAPSHOT missing columns

2016-03-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195022#comment-15195022 ] Cheng Hao commented on SPARK-13326: --- Can not reproduce it anymore, can you try it again? > Dataset in

[jira] [Commented] (HADOOP-12756) Incorporate Aliyun OSS file system implementation

2016-02-03 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131540#comment-15131540 ] Cheng Hao commented on HADOOP-12756: Thank you so much [~ste...@apache.org], [~cnauroth] for the

[jira] [Commented] (HADOOP-12756) Incorporate Aliyun OSS file system implementation

2016-02-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127645#comment-15127645 ] Cheng Hao commented on HADOOP-12756: +1 This is critical for AliYun users when integrated with

[jira] [Updated] (SPARK-12610) Add Anti Join Operators

2016-01-03 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-12610: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-4226 > Add Anti Join Operators >

[jira] [Created] (SPARK-12610) Add Anti Join Operators

2016-01-03 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-12610: - Summary: Add Anti Join Operators Key: SPARK-12610 URL: https://issues.apache.org/jira/browse/SPARK-12610 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-12196) Store blocks in different speed storage devices by hierarchy way

2015-12-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072634#comment-15072634 ] Cheng Hao commented on SPARK-12196: --- Thank you wei wu to support this feature! However, we're trying

[jira] [Updated] (SPARK-8360) Streaming DataFrames

2015-12-02 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-8360: - Attachment: StreamingDataFrameProposal.pdf This is a proposal for streaming dataframes that we were

[jira] [Comment Edited] (SPARK-8360) Streaming DataFrames

2015-12-02 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao edited comment on SPARK-8360 at 12/2/15 12:14 PM: Remove the google

[jira] [Comment Edited] (SPARK-8360) Streaming DataFrames

2015-12-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao edited comment on SPARK-8360 at 12/2/15 6:19 AM: --- Add some thoughts on

[jira] [Commented] (SPARK-8360) Streaming DataFrames

2015-12-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao commented on SPARK-8360: -- Add some thoughts on StreamingSQL.

[jira] [Created] (SPARK-12064) Make the SqlParser as trait for better integrated with extensions

2015-11-30 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-12064: - Summary: Make the SqlParser as trait for better integrated with extensions Key: SPARK-12064 URL: https://issues.apache.org/jira/browse/SPARK-12064 Project: Spark

[jira] [Resolved] (SPARK-12064) Make the SqlParser as trait for better integrated with extensions

2015-11-30 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao resolved SPARK-12064. --- Resolution: Won't Fix DBX has plan to remove the SqlParser in 2.0. > Make the SqlParser as trait

[jira] [Commented] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-11-11 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001423#comment-15001423 ] Cheng Hao commented on SPARK-10865: --- 1.5.2 is released, I am not sure whether part of it now or not. >

[jira] [Commented] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-11-11 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001422#comment-15001422 ] Cheng Hao commented on SPARK-10865: --- We actually follow the criteria of Hive, and actually I tested it

[jira] [Created] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11512: - Summary: Bucket Join Key: SPARK-11512 URL: https://issues.apache.org/jira/browse/SPARK-11512 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990867#comment-14990867 ] Cheng Hao commented on SPARK-11512: --- Oh, yes, but SPARK-5292 is only about to support the Hive bucket,

[jira] [Commented] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990868#comment-14990868 ] Cheng Hao commented on SPARK-11512: --- We need to support the "bucket" for DataSource API. > Bucket Join

[jira] [Commented] (SPARK-10371) Optimize sequential projections

2015-10-29 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981650#comment-14981650 ] Cheng Hao commented on SPARK-10371: --- Eliminating the common sub expression within the projection? >

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979646#comment-14979646 ] Cheng Hao commented on SPARK-11330: --- [~saif.a.ellafi] I've checked that with 1.5.0 and it's confirmed

[jira] [Comment Edited] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979699#comment-14979699 ] Cheng Hao edited comment on SPARK-11330 at 10/29/15 2:48 AM: - OK, seems it's

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979699#comment-14979699 ] Cheng Hao commented on SPARK-11330: --- OK, seems it's solved in

[jira] [Comment Edited] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977600#comment-14977600 ] Cheng Hao edited comment on SPARK-11330 at 10/28/15 2:28 AM: - Hi,

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977600#comment-14977600 ] Cheng Hao commented on SPARK-11330: --- Hi, [~saif.a.ellafi], I've tried the code like below: {code} case

[jira] [Updated] (SPARK-9735) Auto infer partition schema of HadoopFsRelation should should respected the user specified one

2015-10-20 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9735: - Description: This code is copied from the hadoopFsRelationSuite.scala {code} partitionedTestDF = (for {

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2015-10-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958524#comment-14958524 ] Cheng Hao commented on SPARK-4226: -- [~nadenf] Actually I am working on it right now, and the first PR is

[jira] [Created] (SPARK-11076) Decimal Support for Ceil/Floor

2015-10-12 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11076: - Summary: Decimal Support for Ceil/Floor Key: SPARK-11076 URL: https://issues.apache.org/jira/browse/SPARK-11076 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-11041) Add (NOT) IN / EXISTS support for predicates

2015-10-09 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao closed SPARK-11041. - Resolution: Duplicate > Add (NOT) IN / EXISTS support for predicates >

[jira] [Created] (SPARK-11041) Add (NOT) IN / EXISTS support for predicates

2015-10-09 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11041: - Summary: Add (NOT) IN / EXISTS support for predicates Key: SPARK-11041 URL: https://issues.apache.org/jira/browse/SPARK-11041 Project: Spark Issue Type:

[jira] [Updated] (SPARK-10992) Partial Aggregation Support for Hive UDAF

2015-10-07 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10992: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-4366 > Partial Aggregation Support for

[jira] [Created] (SPARK-10831) Spark SQL Configuration missing in the doc

2015-09-25 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10831: - Summary: Spark SQL Configuration missing in the doc Key: SPARK-10831 URL: https://issues.apache.org/jira/browse/SPARK-10831 Project: Spark Issue Type:

[jira] [Created] (SPARK-10829) Scan DataSource with predicate expression combine partition key and attributes doesn't work

2015-09-24 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10829: - Summary: Scan DataSource with predicate expression combine partition key and attributes doesn't work Key: SPARK-10829 URL: https://issues.apache.org/jira/browse/SPARK-10829

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14904778#comment-14904778 ] Cheng Hao commented on SPARK-10733: --- [~jameszhouyi] Can you please patch the

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-17 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802912#comment-14802912 ] Cheng Hao commented on SPARK-10474: --- The root reason for this failure, is because of the

[jira] [Comment Edited] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-17 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802912#comment-14802912 ] Cheng Hao edited comment on SPARK-10474 at 9/17/15 1:48 PM: The root reason

[jira] [Commented] (SPARK-10606) Cube/Rollup/GrpSet doesn't create the correct plan when group by is on something other than an AttributeReference

2015-09-16 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791499#comment-14791499 ] Cheng Hao commented on SPARK-10606: --- [~rhbutani] Which version are you using, actually I've fixed the

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746642#comment-14746642 ] Cheng Hao commented on SPARK-4226: -- Thank you [~brooks], you're right! I meant it will makes more

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744969#comment-14744969 ] Cheng Hao commented on SPARK-10474: --- The root causes for the exception is the executor don't have

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744966#comment-14744966 ] Cheng Hao commented on SPARK-10466: --- [~naliazheli] It's an irrelevant issue, you'd better to subscribe

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745008#comment-14745008 ] Cheng Hao commented on SPARK-10474: --- But from the current implementation, we'd better not to throw

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744967#comment-14744967 ] Cheng Hao commented on SPARK-10466: --- [~naliazheli] It's an irrelevant issue, you'd better to subscribe

[jira] [Issue Comment Deleted] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10466: -- Comment: was deleted (was: [~naliazheli] It's an irrelevant issue, you'd better to subscribe the

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745467#comment-14745467 ] Cheng Hao commented on SPARK-4226: -- [~marmbrus] [~yhuai] After investigating a little bit, I think using

[jira] [Commented] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734395#comment-14734395 ] Cheng Hao commented on SPARK-10484: --- In cartesian produce implementation, there is 2 level nested

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736016#comment-14736016 ] Cheng Hao commented on SPARK-10466: --- Sorry, [~davies], I found the spark conf doens't take effect when

[jira] [Created] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-06 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10466: - Summary: UnsafeRow exception in Sort-Based Shuffle with data spill Key: SPARK-10466 URL: https://issues.apache.org/jira/browse/SPARK-10466 Project: Spark Issue

[jira] [Created] (SPARK-10327) Cache Table is not working while subquery has alias in its project list

2015-08-27 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10327: - Summary: Cache Table is not working while subquery has alias in its project list Key: SPARK-10327 URL: https://issues.apache.org/jira/browse/SPARK-10327 Project: Spark

[jira] [Created] (SPARK-10270) Add/Replace some Java friendly DataFrame API

2015-08-25 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10270: - Summary: Add/Replace some Java friendly DataFrame API Key: SPARK-10270 URL: https://issues.apache.org/jira/browse/SPARK-10270 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10215) Div of Decimal returns null

2015-08-25 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710719#comment-14710719 ] Cheng Hao commented on SPARK-10215: --- Yes, that's a blocker issue for our customer, I

[jira] [Created] (SPARK-10215) Div of Decimal returns null

2015-08-24 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10215: - Summary: Div of Decimal returns null Key: SPARK-10215 URL: https://issues.apache.org/jira/browse/SPARK-10215 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10134: -- Priority: Minor (was: Major) Improve the performance of Binary Comparison

[jira] [Updated] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10134: -- Fix Version/s: (was: 1.6.0) Improve the performance of Binary Comparison

[jira] [Commented] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708766#comment-14708766 ] Cheng Hao commented on SPARK-10134: --- We can improve that by enable the comparison every

[jira] [Commented] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704513#comment-14704513 ] Cheng Hao commented on SPARK-10130: --- Can you change the fix version to 1.5? Lots of

[jira] [Created] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-20 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10134: - Summary: Improve the performance of Binary Comparison Key: SPARK-10134 URL: https://issues.apache.org/jira/browse/SPARK-10134 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9357) Remove JoinedRow

2015-08-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704311#comment-14704311 ] Cheng Hao commented on SPARK-9357: -- JoinedRow does increase the overhead by adding layer

[jira] [Comment Edited] (SPARK-9357) Remove JoinedRow

2015-08-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704311#comment-14704311 ] Cheng Hao edited comment on SPARK-9357 at 8/20/15 5:28 AM: ---

[jira] [Comment Edited] (SPARK-9357) Remove JoinedRow

2015-08-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704311#comment-14704311 ] Cheng Hao edited comment on SPARK-9357 at 8/20/15 5:29 AM: ---

[jira] [Commented] (SPARK-9357) Remove JoinedRow

2015-08-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702603#comment-14702603 ] Cheng Hao commented on SPARK-9357: -- JoinedRow is probably in high efficiency for case

[jira] [Commented] (SPARK-7218) Create a real iterator with open/close for Spark SQL

2015-08-19 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702584#comment-14702584 ] Cheng Hao commented on SPARK-7218: -- Can you give some BKM for this task? Create a real

[jira] [Created] (SPARK-10044) AnalysisException in resolving reference for sorting with aggregation

2015-08-16 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10044: - Summary: AnalysisException in resolving reference for sorting with aggregation Key: SPARK-10044 URL: https://issues.apache.org/jira/browse/SPARK-10044 Project: Spark

[jira] [Commented] (SPARK-8240) string function: concat

2015-08-13 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696471#comment-14696471 ] Cheng Hao commented on SPARK-8240: -- It's probably very difficult to define the function

[jira] [Commented] (SPARK-8240) string function: concat

2015-08-13 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696469#comment-14696469 ] Cheng Hao commented on SPARK-8240: -- It works for me like: {code} sql(select concat('a',

[jira] [Commented] (SPARK-9879) OOM in LIMIT clause with large number

2015-08-13 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696338#comment-14696338 ] Cheng Hao commented on SPARK-9879: -- I create a new physical operator called LargeLimit,

[jira] [Created] (SPARK-9879) OOM in CTAS with LIMIT

2015-08-12 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-9879: Summary: OOM in CTAS with LIMIT Key: SPARK-9879 URL: https://issues.apache.org/jira/browse/SPARK-9879 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-9879) OOM in LIMIT clause with large number

2015-08-12 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9879: - Summary: OOM in LIMIT clause with large number (was: OOM in CTAS with LIMIT) OOM in LIMIT clause with

[jira] [Updated] (SPARK-9879) OOM in CTAS with LIMIT

2015-08-12 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9879: - Description: {code} create table spark.tablsetest as select * from dpa_ord_bill_tf order by member_id

[jira] [Updated] (SPARK-9879) OOM in CTAS with LIMIT

2015-08-12 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9879: - Description: {code} create table spark.tablsetest as select * from dpa_ord_bill_tf order by member_id

[jira] [Created] (SPARK-9735) Auto infer partition schema of HadoopFsRelation should should respected the user specified one

2015-08-07 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-9735: Summary: Auto infer partition schema of HadoopFsRelation should should respected the user specified one Key: SPARK-9735 URL: https://issues.apache.org/jira/browse/SPARK-9735

[jira] [Commented] (SPARK-9689) Cache doesn't refresh for HadoopFsRelation based table

2015-08-06 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661359#comment-14661359 ] Cheng Hao commented on SPARK-9689: -- After investigation, the root cause for the failure,

[jira] [Updated] (SPARK-9689) Cache doesn't refresh for HadoopFsRelation based table

2015-08-06 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9689: - Description: {code:title=example|borderStyle=solid} // create a HadoopFsRelation based table

[jira] [Created] (SPARK-9689) Cache doesn't refresh for HadoopFsRelation based table

2015-08-06 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-9689: Summary: Cache doesn't refresh for HadoopFsRelation based table Key: SPARK-9689 URL: https://issues.apache.org/jira/browse/SPARK-9689 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-08-03 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652892#comment-14652892 ] Cheng Hao commented on SPARK-7119: -- [~marmbrus] This is actually a bug fixing, and it

[jira] [Created] (SPARK-9381) Migrate JSON data source to the new partitioning data source

2015-07-27 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-9381: Summary: Migrate JSON data source to the new partitioning data source Key: SPARK-9381 URL: https://issues.apache.org/jira/browse/SPARK-9381 Project: Spark Issue

[jira] [Commented] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642706#comment-14642706 ] Cheng Hao commented on SPARK-9374: -- [~cloud_fan]] Can you also take look at this failure?

[jira] [Commented] (SPARK-9239) HiveUDAF support for AggregateFunction2

2015-07-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638276#comment-14638276 ] Cheng Hao commented on SPARK-9239: -- [~yhuai] are you working on this now? Or I can take

[jira] [Commented] (SPARK-8230) complex function: size

2015-07-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629107#comment-14629107 ] Cheng Hao commented on SPARK-8230: -- [~pedrorodriguez], actually [~TarekAuel] set a good

[jira] [Commented] (SPARK-8956) Rollup produces incorrect result when group by contains expressions

2015-07-12 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624121#comment-14624121 ] Cheng Hao commented on SPARK-8956: -- Sorry, I didn't notice this jira issue when I created

[jira] [Updated] (SPARK-8972) Incorrect result for rollup

2015-07-12 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-8972: - Description: {code:java} import sqlContext.implicits._ case class KeyValue(key: Int, value: String) val

[jira] [Created] (SPARK-8972) Wrong result for rollup

2015-07-09 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8972: Summary: Wrong result for rollup Key: SPARK-8972 URL: https://issues.apache.org/jira/browse/SPARK-8972 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-8972) Incorrect result for rollup

2015-07-09 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-8972: - Summary: Incorrect result for rollup (was: Wrong result for rollup) Incorrect result for rollup

[jira] [Issue Comment Deleted] (SPARK-8864) Date/time function and data type design

2015-07-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-8864: - Comment: was deleted (was: Thanks for explanation. The design looks good to me now.) Date/time function

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618200#comment-14618200 ] Cheng Hao commented on SPARK-8864: -- Thanks for explanation. The design looks good to me

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618201#comment-14618201 ] Cheng Hao commented on SPARK-8864: -- Thanks for explanation. The design looks good to me

[jira] [Updated] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-07-08 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-7119: - Priority: Blocker (was: Major) ScriptTransform doesn't consider the output data type

[jira] [Created] (SPARK-8867) Show the UDF usage for user.

2015-07-07 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8867: Summary: Show the UDF usage for user. Key: SPARK-8867 URL: https://issues.apache.org/jira/browse/SPARK-8867 Project: Spark Issue Type: Task Components:

[jira] [Created] (SPARK-8883) Remove the class OverrideFunctionRegistry

2015-07-07 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8883: Summary: Remove the class OverrideFunctionRegistry Key: SPARK-8883 URL: https://issues.apache.org/jira/browse/SPARK-8883 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617846#comment-14617846 ] Cheng Hao commented on SPARK-8864: -- Long = 2 ^ 63 = 9.2E18, the timestamp is in us, the

[jira] [Created] (SPARK-8791) Make a better hashcode for InternalRow

2015-07-02 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8791: Summary: Make a better hashcode for InternalRow Key: SPARK-8791 URL: https://issues.apache.org/jira/browse/SPARK-8791 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-8159) Improve SQL/DataFrame expression coverage

2015-07-02 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612728#comment-14612728 ] Cheng Hao commented on SPARK-8159: -- Will that possible to add all of the expressions

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-07-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609627#comment-14609627 ] Cheng Hao commented on SPARK-8653: -- Yes, I agree that we cannot make a clear cut, as

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-07-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609609#comment-14609609 ] Cheng Hao commented on SPARK-8653: -- For most of the Mathematical expressions, we can get

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-07-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609629#comment-14609629 ] Cheng Hao commented on SPARK-8653: -- What do you think [~rxin]? Add constraint for

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-29 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14607195#comment-14607195 ] Cheng Hao commented on SPARK-8653: -- [~rxin] I'll agree that we need to rename the trait.

[jira] [Created] (SPARK-8653) Add constraint for Children expression for data type

2015-06-26 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-8653: Summary: Add constraint for Children expression for data type Key: SPARK-8653 URL: https://issues.apache.org/jira/browse/SPARK-8653 Project: Spark Issue Type:

  1   2   3   4   5   >