[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-13.patch Namit, Thanks for correction. Upload a new patch. I guess the diffs came from the order of file output operator's column names. And the new patch sorted field schemas according to their names. Hope this can pass the tests. Again it passed in my local. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-2-195mapjoin.patch, > hive-460-2009-06-04-testcli-passed.patch, hive-460-2009-06-06-2.patch, > hive-460-2009-06-10-2.patch, hive-460-2009-06-12.patch, > hive-460-2009-06-13.patch, HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-12.patch Added the new testcase, but join32 does not get diff in my local. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-2-195mapjoin.patch, > hive-460-2009-06-04-testcli-passed.patch, hive-460-2009-06-06-2.patch, > hive-460-2009-06-10-2.patch, hive-460-2009-06-12.patch, HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-10-2.patch A new patch against trunk code(hive-460-2009-06-10-2.patch) and it passed all TestCliDriver and TestParse in my local. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-2-195mapjoin.patch, > hive-460-2009-06-04-testcli-passed.patch, hive-460-2009-06-06-2.patch, > hive-460-2009-06-10-2.patch, HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-06-2.patch A new patch(hive-460-2009-06-06-2.patch) according to Namit's feedbacks. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-2-195mapjoin.patch, > hive-460-2009-06-04-testcli-passed.patch, hive-460-2009-06-06-2.patch, > HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-04-2-195mapjoin.patch Sorry for delay. hive-460-2009-06-04-2-195mapjoin.patch is a new one against truck and intergrates with Zheng's suggestions. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-2-195mapjoin.patch, > hive-460-2009-06-04-testcli-passed.patch, HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Yongqiang updated HIVE-460: -- Attachment: hive-460-2009-06-04-testcli-passed.patch hive-460-2009-06-04-testcli-passed.patch now works. The modifications made: 1) added an array of output column names to Operators like select, groupby, join etc 2) removed plan regeneration code in ColumnPruner 3) instead of using integer as column internal names, now use _col1 _col2 etc > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: He Yongqiang > Attachments: hive-460-2009-05-29-noaliasallowed.patch, > hive-460-2009-06-04-testcli-passed.patch, HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[
https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-460:
--
Attachment: hive-460-2009-05-29-noaliasallowed.patch
hive-460-2009-05-29-noaliasallowed.patch is not a finished version and it still
has errors. The major issue is that it does not allow alias.
For example:
{noformat}
SELECT kv1.key, sum(substr(kv1.value,5)) GROUP BY kv1.key LIMIT 5
{noformat}
works well. But
{noformat}
SELECT kv1.key as key_alias, sum(substr(kv1.value,5)) GROUP BY key_alias LIMIT 5
{noformat}
even can not pass the parser.
Another error example is
{noformat}
SELECT src1.key as k1, src1.value as v1, src2.key as k2, src2.value as v2 FROM
(SELECT * FROM kv1 WHERE kv1.key < 10) src1 "
JOIN (SELECT * FROM kv1 WHERE kv1.key < 10) src2;
{noformat}
works well, but the one with a sort clause
{noformat}
SELECT src1.key as k1, src1.value as v1, src2.key as k2, src2.value as v2 FROM
(SELECT * FROM kv1 WHERE kv1.key < 10) src1 "
JOIN (SELECT * FROM kv1 WHERE kv1.key < 10) src2 sort by
k1,v1,k2,v2;
{noformat}
either can not pass the parser.
I am still trying to figure out why these errors are caused and how to fix
them. And really appreciate some input.
BTW, with this patch, explain ... now will give 'meaningful names', but i
don't know whether they are ok.
> Improve ColumnPruner to prune more aggressively and keep column information
> for input tables
>
>
> Key: HIVE-460
> URL: https://issues.apache.org/jira/browse/HIVE-460
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 0.4.0
>Reporter: Zheng Shao
>Assignee: He Yongqiang
> Attachments: hive-460-2009-05-29-noaliasallowed.patch,
> HIVE-460.1.patch
>
>
> This is required for column-based table format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-460: Status: Open (was: Patch Available) Moving the status back since the patch is not ready for commit. Yongqiang is working on this issue right now. I will reassign. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: Zheng Shao > Attachments: HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Thusoo updated HIVE-460: --- Affects Version/s: 0.4.0 Status: Patch Available (was: Open) submitting patch. > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.4.0 >Reporter: Zheng Shao >Assignee: Zheng Shao > Attachments: HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
[ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao updated HIVE-460: Attachment: HIVE-460.1.patch This patch illustrates my idea. However it does not pass all the tests, and an elegant fix will depend on HIVE-153 > Improve ColumnPruner to prune more aggressively and keep column information > for input tables > > > Key: HIVE-460 > URL: https://issues.apache.org/jira/browse/HIVE-460 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Zheng Shao > Attachments: HIVE-460.1.patch > > > This is required for column-based table format. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
