[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Attachment: (was: stop-after-physical-plan.pdf) > Filter operator should have “stop if

[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Attachment: stop-after-physical-plan.pdf > Filter operator should have “stop if false”

[jira] [Closed] (SPARK-18002) Prune unnecessary IsNotNull predicates from Filter

2016-10-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-18002. --- Resolution: Won't Fix > Prune unnecessary IsNotNull predicates from Filter >

[jira] [Created] (SPARK-18002) Prune unnecessary IsNotNull predicates from Filter

2016-10-18 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-18002: --- Summary: Prune unnecessary IsNotNull predicates from Filter Key: SPARK-18002 URL: https://issues.apache.org/jira/browse/SPARK-18002 Project: Spark

[jira] [Closed] (SPARK-17956) ProjectExec has incorrect outputOrdering property

2016-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-17956. --- Resolution: Won't Fix > ProjectExec has incorrect outputOrdering property >

[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Attachment: (was: stop-after-physical-plan.pdf) > Filter operator should have “stop if

[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Attachment: stop-after-physical-plan.pdf > Filter operator should have “stop if false”

[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Priority: Major (was: Minor) > Filter operator should have “stop if false” semantics for

[jira] [Created] (SPARK-17956) ProjectExec has incorrect outputOrdering property

2016-10-15 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17956: --- Summary: ProjectExec has incorrect outputOrdering property Key: SPARK-17956 URL: https://issues.apache.org/jira/browse/SPARK-17956 Project: Spark

[jira] [Updated] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-10-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17254: Attachment: stop-after-physical-plan.pdf > Filter operator should have “stop if false”

[jira] [Created] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2016-10-10 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17867: --- Summary: Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name Key: SPARK-17867 URL: https://issues.apache.org/jira/browse/SPARK-17867

[jira] [Created] (SPARK-17866) Dataset.dropDuplicates (i.e., distinct) should not change the output of child plan

2016-10-10 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17866: --- Summary: Dataset.dropDuplicates (i.e., distinct) should not change the output of child plan Key: SPARK-17866 URL: https://issues.apache.org/jira/browse/SPARK-17866

[jira] [Created] (SPARK-17821) Expression Canonicalization should support Add and Or

2016-10-06 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17821: --- Summary: Expression Canonicalization should support Add and Or Key: SPARK-17821 URL: https://issues.apache.org/jira/browse/SPARK-17821 Project: Spark

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537544#comment-15537544 ] Liang-Chi Hsieh commented on SPARK-17556: - Update the design document again to address some

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534774#comment-15534774 ] Liang-Chi Hsieh commented on SPARK-17556: - Update the design document to add more description for

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: (was: executor-side-broadcast.pdf) > Executor side broadcast for broadcast

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-28 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529439#comment-15529439 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/28/16 12:42 PM: ---

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-28 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529476#comment-15529476 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/28/16 12:41 PM: --- For

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-28 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529476#comment-15529476 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/28/16 12:40 PM: --- For

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-28 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529476#comment-15529476 ] Liang-Chi Hsieh commented on SPARK-17556: - For example, assume we have 3 executors and the

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-28 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529439#comment-15529439 ] Liang-Chi Hsieh commented on SPARK-17556: - Actually I am hesitant to add the feature to handle

[jira] [Comment Edited] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

2016-09-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522051#comment-15522051 ] Liang-Chi Hsieh edited comment on SPARK-17527 at 9/26/16 4:58 AM: -- Do

[jira] [Commented] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

2016-09-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522051#comment-15522051 ] Liang-Chi Hsieh commented on SPARK-17527: - Do you have a small spark code snippet which can

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-24 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518790#comment-15518790 ] Liang-Chi Hsieh commented on SPARK-17556: - For 1). It is true only if your driver is outside of

[jira] [Updated] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-24 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17556: Attachment: executor-side-broadcast.pdf > Executor side broadcast for broadcast joins >

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516668#comment-15516668 ] Liang-Chi Hsieh commented on SPARK-17556: - No. It doesn't. I think the point is not only the

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516577#comment-15516577 ] Liang-Chi Hsieh commented on SPARK-17556: - In other words, from the jira description we say "the

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516568#comment-15516568 ] Liang-Chi Hsieh commented on SPARK-17556: - OK. You create the broadcast object on one executor.

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516490#comment-15516490 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/23/16 1:49 PM: --

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516490#comment-15516490 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/23/16 1:48 PM: --

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516478#comment-15516478 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/23/16 1:45 PM: --

[jira] [Comment Edited] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516490#comment-15516490 ] Liang-Chi Hsieh edited comment on SPARK-17556 at 9/23/16 1:45 PM: --

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516490#comment-15516490 ] Liang-Chi Hsieh commented on SPARK-17556: - [~Fei Wang] I quickly go through your design doc.

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2016-09-23 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516478#comment-15516478 ] Liang-Chi Hsieh commented on SPARK-17556: - [~scwf]I already submitted a PR for this. Can you also

[jira] [Commented] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

2016-09-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503595#comment-15503595 ] Liang-Chi Hsieh commented on SPARK-17527: - Of course. > mergeSchema with `_OPTIONAL_` metadata

[jira] [Updated] (SPARK-17590) Analyze CTE definitions at once and allow CTE subquery to define CTE

2016-09-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17590: Summary: Analyze CTE definitions at once and allow CTE subquery to define CTE (was:

[jira] [Updated] (SPARK-17590) Analyze CTE definitions at once and allow CTE subquery to define CTE

2016-09-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17590: Description: We substitute logical plan with CTE definitions in the analyzer rule

[jira] [Created] (SPARK-17590) Analyze CTE definitions at once

2016-09-18 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17590: --- Summary: Analyze CTE definitions at once Key: SPARK-17590 URL: https://issues.apache.org/jira/browse/SPARK-17590 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17527) mergeSchema with `_OPTIONAL_` metadata fails

2016-09-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15498980#comment-15498980 ] Liang-Chi Hsieh commented on SPARK-17527: - Can you provide more hints about how to reproduce

[jira] [Closed] (SPARK-17574) Cache ShuffleExchange RDD when the exchange is reused

2016-09-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-17574. --- Resolution: Won't Fix > Cache ShuffleExchange RDD when the exchange is reused >

[jira] [Updated] (SPARK-17574) Cache ShuffleExchange RDD when the exchange is reused

2016-09-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17574: Component/s: (was: SQL) Spark Core > Cache ShuffleExchange RDD when

[jira] [Created] (SPARK-17574) Cache ShuffleExchange RDD when the exchange is reused

2016-09-17 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17574: --- Summary: Cache ShuffleExchange RDD when the exchange is reused Key: SPARK-17574 URL: https://issues.apache.org/jira/browse/SPARK-17574 Project: Spark

[jira] [Updated] (SPARK-17357) Simplified predicates can't be pushed down through operators because of the rule order in Optimizer

2016-09-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17357: Summary: Simplified predicates can't be pushed down through operators because of the rule

[jira] [Updated] (SPARK-17357) Simplified predicates should be able to pushdown through operators because of the rule order in Optimizer

2016-09-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17357: Summary: Simplified predicates should be able to pushdown through operators because of the

[jira] [Created] (SPARK-17357) Simplified predicates should be able to pushdown through operators

2016-09-01 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17357: --- Summary: Simplified predicates should be able to pushdown through operators Key: SPARK-17357 URL: https://issues.apache.org/jira/browse/SPARK-17357 Project:

[jira] [Commented] (SPARK-17278) better error message for NPE during ScalaUDF execution

2016-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445201#comment-15445201 ] Liang-Chi Hsieh commented on SPARK-17278: - Duplicate to SPARK-17279? > better error message for

[jira] [Commented] (SPARK-17285) ZeroOutPaddingBytes Causing Fatal JVM Error

2016-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444970#comment-15444970 ] Liang-Chi Hsieh commented on SPARK-17285: - It should be helpful if you can provide the way to

[jira] [Created] (SPARK-17206) Support ANALYZE TABLE on analyzable temoprary table/view

2016-08-23 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17206: --- Summary: Support ANALYZE TABLE on analyzable temoprary table/view Key: SPARK-17206 URL: https://issues.apache.org/jira/browse/SPARK-17206 Project: Spark

[jira] [Updated] (SPARK-17107) Remove redundant pushdown rule for Union

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17107: Summary: Remove redundant pushdown rule for Union (was: Remove redundant pushdown rule

[jira] [Created] (SPARK-17107) Remove redundant pushdown rule for set

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17107: --- Summary: Remove redundant pushdown rule for set Key: SPARK-17107 URL: https://issues.apache.org/jira/browse/SPARK-17107 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17104) LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17104: Description: Currently LogicalRelation.newInstance() simply creates another

[jira] [Updated] (SPARK-17104) LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17104: Description: Currently LogicalRelation.newInstance() simply creates another

[jira] [Updated] (SPARK-17104) LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17104: Summary: LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

[jira] [Updated] (SPARK-17104) Self-joining of a Hive table converted to datasource parquet table will fail

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-17104: Description: Converted Hive table will > Self-joining of a Hive table converted to

[jira] [Created] (SPARK-17104) Self-joining of a Hive table converted to datasource parquet table will fail

2016-08-17 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17104: --- Summary: Self-joining of a Hive table converted to datasource parquet table will fail Key: SPARK-17104 URL: https://issues.apache.org/jira/browse/SPARK-17104

[jira] [Created] (SPARK-17056) Fix a wrong assert in MemoryStore

2016-08-14 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17056: --- Summary: Fix a wrong assert in MemoryStore Key: SPARK-17056 URL: https://issues.apache.org/jira/browse/SPARK-17056 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-16849) Improve subquery execution by deduplicating the subqueries with the same results

2016-08-01 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16849: --- Summary: Improve subquery execution by deduplicating the subqueries with the same results Key: SPARK-16849 URL: https://issues.apache.org/jira/browse/SPARK-16849

[jira] [Created] (SPARK-16767) existsRecursively method in UserDefinedType is not correct

2016-07-28 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16767: --- Summary: existsRecursively method in UserDefinedType is not correct Key: SPARK-16767 URL: https://issues.apache.org/jira/browse/SPARK-16767 Project: Spark

[jira] [Commented] (SPARK-16628) OrcConversions should not convert an ORC table represented by MetastoreRelation to HadoopFsRelation if metastore schema does not match schema stored in ORC files

2016-07-26 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393436#comment-15393436 ] Liang-Chi Hsieh commented on SPARK-16628: - I submitted another PR to implement the option 2

[jira] [Commented] (SPARK-16628) OrcConversions should not convert an ORC table represented by MetastoreRelation to HadoopFsRelation if metastore schema does not match schema stored in ORC files

2016-07-22 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389173#comment-15389173 ] Liang-Chi Hsieh commented on SPARK-16628: - I think it depends whether Hive also writes wrong

[jira] [Commented] (SPARK-16628) OrcConversions should not convert an ORC table represented by MetastoreRelation to HadoopFsRelation if metastore schema does not match schema stored in ORC files

2016-07-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385586#comment-15385586 ] Liang-Chi Hsieh commented on SPARK-16628: - I've tried to address this issue by the PR with the

[jira] [Created] (SPARK-16640) Add codegen for Elt function

2016-07-19 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16640: --- Summary: Add codegen for Elt function Key: SPARK-16640 URL: https://issues.apache.org/jira/browse/SPARK-16640 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16622) Fix NullPointerException when the returned value of the called method in Invoke is null

2016-07-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-16622: Summary: Fix NullPointerException when the returned value of the called method in Invoke

[jira] [Created] (SPARK-16622) Throws NullPointerException when the returned value of the called method in Invoke is null

2016-07-19 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16622: --- Summary: Throws NullPointerException when the returned value of the called method in Invoke is null Key: SPARK-16622 URL: https://issues.apache.org/jira/browse/SPARK-16622

[jira] [Created] (SPARK-16362) Suport ArrayType and StructType in vectorization Parquet reader

2016-07-04 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16362: --- Summary: Suport ArrayType and StructType in vectorization Parquet reader Key: SPARK-16362 URL: https://issues.apache.org/jira/browse/SPARK-16362 Project: Spark

[jira] [Commented] (SPARK-16062) PySpark SQL python-only UDTs don't work well

2016-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339070#comment-15339070 ] Liang-Chi Hsieh commented on SPARK-16062: - Related to but not exactly the same issue. > PySpark

[jira] [Comment Edited] (SPARK-16062) PySpark SQL python-only UDTs don't work well

2016-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339070#comment-15339070 ] Liang-Chi Hsieh edited comment on SPARK-16062 at 6/20/16 6:31 AM: --

[jira] [Updated] (SPARK-16062) PySpark SQL python-only UDTs don't work well

2016-06-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-16062: Description: Python-only UDTs can't work well. One example is: {code} import

[jira] [Created] (SPARK-16062) PySpark SQL python-only UDTs don't work well

2016-06-20 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16062: --- Summary: PySpark SQL python-only UDTs don't work well Key: SPARK-16062 URL: https://issues.apache.org/jira/browse/SPARK-16062 Project: Spark Issue

[jira] [Created] (SPARK-16060) Vectorized Orc reader

2016-06-19 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-16060: --- Summary: Vectorized Orc reader Key: SPARK-16060 URL: https://issues.apache.org/jira/browse/SPARK-16060 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-15911) Remove additional Project to be consistent with SQL when insert into table

2016-06-12 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15911: --- Summary: Remove additional Project to be consistent with SQL when insert into table Key: SPARK-15911 URL: https://issues.apache.org/jira/browse/SPARK-15911

[jira] [Closed] (SPARK-15701) Constant ColumnVector only needs to prepare one capacity

2016-06-08 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-15701. --- Resolution: Not A Problem > Constant ColumnVector only needs to prepare one capacity >

[jira] [Commented] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-06-05 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315797#comment-15315797 ] Liang-Chi Hsieh commented on SPARK-15433: - [~davies] Can you set the assignee to me? Thanks. >

[jira] [Created] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

2016-06-03 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15753: --- Summary: Move some Analyzer stuff to Analyzer from DataFrameWriter Key: SPARK-15753 URL: https://issues.apache.org/jira/browse/SPARK-15753 Project: Spark

[jira] [Created] (SPARK-15701) Constant ColumnVector only needs to prepare one capacity

2016-06-01 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15701: --- Summary: Constant ColumnVector only needs to prepare one capacity Key: SPARK-15701 URL: https://issues.apache.org/jira/browse/SPARK-15701 Project: Spark

[jira] [Created] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-05-27 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15639: --- Summary: Try to push down filter at RowGroups level for parquet reader Key: SPARK-15639 URL: https://issues.apache.org/jira/browse/SPARK-15639 Project: Spark

[jira] [Updated] (SPARK-15444) Default value mismatch of param linkPredictionCol for GeneralizedLinearRegression

2016-05-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-15444: Description: There is a default value mismatch of param linkPredictionCol for

[jira] [Created] (SPARK-15444) Default value mismatch of param linkPredictionCol for GeneralizedLinearRegression

2016-05-20 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15444: --- Summary: Default value mismatch of param linkPredictionCol for GeneralizedLinearRegression Key: SPARK-15444 URL: https://issues.apache.org/jira/browse/SPARK-15444

[jira] [Updated] (SPARK-15444) Default value mismatch of param linkPredictionCol for GeneralizedLinearRegression

2016-05-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-15444: Priority: Blocker (was: Major) > Default value mismatch of param linkPredictionCol for

[jira] [Created] (SPARK-15433) PySpark core test should not use SerDe from PythonMLLibAPI

2016-05-19 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15433: --- Summary: PySpark core test should not use SerDe from PythonMLLibAPI Key: SPARK-15433 URL: https://issues.apache.org/jira/browse/SPARK-15433 Project: Spark

[jira] [Created] (SPARK-15430) Access ListAccumulator's value could possibly cause java.util.ConcurrentModificationException

2016-05-19 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15430: --- Summary: Access ListAccumulator's value could possibly cause java.util.ConcurrentModificationException Key: SPARK-15430 URL:

[jira] [Created] (SPARK-15342) PySpark test for non ascii column name does not actually test with unicode column name

2016-05-16 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15342: --- Summary: PySpark test for non ascii column name does not actually test with unicode column name Key: SPARK-15342 URL: https://issues.apache.org/jira/browse/SPARK-15342

[jira] [Created] (SPARK-15268) Make JavaTypeInference work with UDTRegistration

2016-05-11 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15268: --- Summary: Make JavaTypeInference work with UDTRegistration Key: SPARK-15268 URL: https://issues.apache.org/jira/browse/SPARK-15268 Project: Spark Issue

[jira] [Updated] (SPARK-15240) Use buffer variables to improve buffer serialization/deserialization in TungstenAggregate

2016-05-09 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-15240: Summary: Use buffer variables to improve buffer serialization/deserialization in

[jira] [Created] (SPARK-15240) Use buffer variables for update/merge expressions instead duplicate serialization/deserialization

2016-05-09 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15240: --- Summary: Use buffer variables for update/merge expressions instead duplicate serialization/deserialization Key: SPARK-15240 URL:

[jira] [Created] (SPARK-15225) Replace SQLContext with SparkSession in Encoder documentation

2016-05-09 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15225: --- Summary: Replace SQLContext with SparkSession in Encoder documentation Key: SPARK-15225 URL: https://issues.apache.org/jira/browse/SPARK-15225 Project: Spark

[jira] [Created] (SPARK-15211) Select features column from LibSVMRelation causes failure

2016-05-08 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15211: --- Summary: Select features column from LibSVMRelation causes failure Key: SPARK-15211 URL: https://issues.apache.org/jira/browse/SPARK-15211 Project: Spark

[jira] [Created] (SPARK-15180) Support subexpression elimination in Fliter

2016-05-06 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-15180: --- Summary: Support subexpression elimination in Fliter Key: SPARK-15180 URL: https://issues.apache.org/jira/browse/SPARK-15180 Project: Spark Issue

[jira] [Commented] (SPARK-14906) Move VectorUDT and MatrixUDT in PySpark to new ML package

2016-05-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267704#comment-15267704 ] Liang-Chi Hsieh commented on SPARK-14906: - ok. I will do it soon. > Move VectorUDT and MatrixUDT

<    5   6   7   8   9   10   11   12   13   14   >