[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

2018-07-19 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549504#comment-16549504 ] Qifan Pu commented on SPARK-24838: -- Thanks for the PR [~maurits]! Should we also fix it for Aggregate

[jira] [Created] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

2018-07-17 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-24838: Summary: Support uncorrelated IN/EXISTS subqueries for more operators Key: SPARK-24838 URL: https://issues.apache.org/jira/browse/SPARK-24838 Project: Spark Issue

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-07 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472037#comment-15472037 ] Qifan Pu commented on SPARK-17405: -- [~joshrosen] Yes, running local[32] will reproduce the exception.

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-07 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472009#comment-15472009 ] Qifan Pu commented on SPARK-17405: -- One quick fix is to set memory capacity in configuration to make

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-07 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15471747#comment-15471747 ] Qifan Pu commented on SPARK-17405: -- [~joshrosen] Yes likely. The new hashmap asks for 64MB per task, and

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-07 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15471640#comment-15471640 ] Qifan Pu commented on SPARK-17405: -- [~joshrosen][~jlaskowski]Thanks for the comments and suggestions. I

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-06 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466676#comment-15466676 ] Qifan Pu commented on SPARK-17405: -- [~joshrosen] Thanks for reporting. I haven't been able to reproduce

[jira] [Commented] (SPARK-17053) DROP statement should not require IF EXISTS

2016-08-14 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420443#comment-15420443 ] Qifan Pu commented on SPARK-17053: -- [~dongjoon]sorry, it was a accident click. > DROP statement should

[jira] [Resolved] (SPARK-17053) DROP statement should not require IF EXISTS

2016-08-14 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qifan Pu resolved SPARK-17053. -- Resolution: Won't Fix > DROP statement should not require IF EXISTS >

[jira] [Created] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16928: Summary: Recursive call of ColumnVector::getInt() breaks JIT inlining Key: SPARK-16928 URL: https://issues.apache.org/jira/browse/SPARK-16928 Project: Spark Issue

[jira] [Updated] (SPARK-16713) Limit codegen method size to 8KB

2016-07-25 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qifan Pu updated SPARK-16713: - Description: Ideally, we would wish codegen methods to be less than 8KB for bytecode size. Beyond 8K JIT

[jira] [Created] (SPARK-16713) Limit codegen method size to 8KB

2016-07-25 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16713: Summary: Limit codegen method size to 8KB Key: SPARK-16713 URL: https://issues.apache.org/jira/browse/SPARK-16713 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-07-25 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392521#comment-15392521 ] Qifan Pu commented on SPARK-15258: -- Just want to leave a note that we might want to limit method size to

[jira] [Updated] (SPARK-16699) Fix performance bug in hash aggregate on long string keys

2016-07-24 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qifan Pu updated SPARK-16699: - Description: In the following code in `VectorizedHashMapGenerator.scala`: ``` def hashBytes(b:

[jira] [Created] (SPARK-16699) Fix performance bug in hash aggregate on long string keys

2016-07-24 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16699: Summary: Fix performance bug in hash aggregate on long string keys Key: SPARK-16699 URL: https://issues.apache.org/jira/browse/SPARK-16699 Project: Spark Issue

[jira] [Created] (SPARK-16526) Benchmarking Performance for Fast HashMap Implementations and Set Knobs

2016-07-13 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16526: Summary: Benchmarking Performance for Fast HashMap Implementations and Set Knobs Key: SPARK-16526 URL: https://issues.apache.org/jira/browse/SPARK-16526 Project: Spark

[jira] [Created] (SPARK-16525) Enable Row Based HashMap in HashAggregateExec

2016-07-13 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16525: Summary: Enable Row Based HashMap in HashAggregateExec Key: SPARK-16525 URL: https://issues.apache.org/jira/browse/SPARK-16525 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16524) Add RowBatch and RowBasedHashMapGenerator

2016-07-13 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16524: Summary: Add RowBatch and RowBasedHashMapGenerator Key: SPARK-16524 URL: https://issues.apache.org/jira/browse/SPARK-16524 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16523) Support Row Based Aggregation HashMap

2016-07-13 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16523: Summary: Support Row Based Aggregation HashMap Key: SPARK-16523 URL: https://issues.apache.org/jira/browse/SPARK-16523 Project: Spark Issue Type: Story

[jira] [Updated] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qifan Pu updated SPARK-16269: - External issue URL: https://github.com/apache/spark/pull/13960 > Support null handling for vectorized

[jira] [Created] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16269: Summary: Support null handling for vectorized hashmap during hash aggregate Key: SPARK-16269 URL: https://issues.apache.org/jira/browse/SPARK-16269 Project: Spark