[jira] Updated: (HIVE-1497) support COMMENT clause on CREATE INDEX, and add new command for SHOW INDEXES
[ https://issues.apache.org/jira/browse/HIVE-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sichi updated HIVE-1497: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed. Thanks Russell! > support COMMENT clause on CREATE INDEX, and add new command for SHOW INDEXES > > > Key: HIVE-1497 > URL: https://issues.apache.org/jira/browse/HIVE-1497 > Project: Hive > Issue Type: Improvement > Components: Indexing >Affects Versions: 0.7.0 >Reporter: John Sichi >Assignee: Russell Melick > Fix For: 0.7.0 > > Attachments: HIVE-1497.4.patch, HIVE-1497.5.patch, HIVE-1497.6.patch, > HIVE-1497.7.patch, HIVE-1497.8.patch, hive-1497.p1.patch, hive-1497.p2.patch, > hive-1497.p3.patch > > > We need to work out the syntax for SHOW/DESCRIBE, taking partitioning into > account. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1764) Add command for DESCRIBE indexes
[ https://issues.apache.org/jira/browse/HIVE-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931628#action_12931628 ] John Sichi commented on HIVE-1764: -- Also, one other followup from HIVE-1497: * Use MetaStoreUtils.getColumnNamesFromFieldSchema in getAllColumnsInformation(Index) > Add command for DESCRIBE indexes > > > Key: HIVE-1764 > URL: https://issues.apache.org/jira/browse/HIVE-1764 > Project: Hive > Issue Type: Improvement > Components: Indexing >Affects Versions: 0.7.0 >Reporter: Russell Melick >Assignee: Jeffrey Lym > Fix For: 0.7.0 > > > Followup for HIVE-1497 > * Work out the syntax for DESCRIBE, taking partitioning into account. > * Allow regular expressions for selecting tables (like SHOW TABLES ) > * Enable showing indexes which are only on specific partitions -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931620#action_12931620 ] Namit Jain commented on HIVE-1642: -- great work Liyin - I will take a look > Convert join queries to map-join based on size of table/row > --- > > Key: HIVE-1642 > URL: https://issues.apache.org/jira/browse/HIVE-1642 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Liyin Tang > Fix For: 0.7.0 > > Attachments: hive_1642_1.patch > > > Based on the number of rows and size of each table, Hive should automatically > be able to convert a join into map-join. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1790) Patch to support HAVING clause in Hive
[ https://issues.apache.org/jira/browse/HIVE-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Aggarwal updated HIVE-1790: --- Attachment: HIVE-1790.patch > Patch to support HAVING clause in Hive > -- > > Key: HIVE-1790 > URL: https://issues.apache.org/jira/browse/HIVE-1790 > Project: Hive > Issue Type: Improvement >Reporter: Vaibhav Aggarwal >Assignee: Vaibhav Aggarwal > Attachments: HIVE-1790.patch > > > Currently Hive users have to do nested queries in order to apply filter on > group by expressions. > This patch allows users to directly apply filter on group by expressions by > using HAVING clause. > This patch also helps us integrate Hive with other data analysis tools which > rely on HAVING expression. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1790) Patch to support HAVING clause in Hive
Patch to support HAVING clause in Hive -- Key: HIVE-1790 URL: https://issues.apache.org/jira/browse/HIVE-1790 Project: Hive Issue Type: Improvement Reporter: Vaibhav Aggarwal Assignee: Vaibhav Aggarwal Currently Hive users have to do nested queries in order to apply filter on group by expressions. This patch allows users to directly apply filter on group by expressions by using HAVING clause. This patch also helps us integrate Hive with other data analysis tools which rely on HAVING expression. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1642) Convert join queries to map-join based on size of table/row
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HIVE-1642: - Attachment: hive_1642_1.patch > Convert join queries to map-join based on size of table/row > --- > > Key: HIVE-1642 > URL: https://issues.apache.org/jira/browse/HIVE-1642 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Liyin Tang > Fix For: 0.7.0 > > Attachments: hive_1642_1.patch > > > Based on the number of rows and size of each table, Hive should automatically > be able to convert a join into map-join. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row
[ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931599#action_12931599 ] Liyin Tang commented on HIVE-1642: -- I just finished converting common join into map join based on the file size. There are 2 flags to control this optimization. 1) set hive.auto.convert.join = true; It means this optimization is enabled. By default right now, this flag is disabled in order not to break any existing test cases. Also I put 25 additional test cases, auto_join0.q - auto_join25.q, which covers this optimization code. 2) Set hive.hashtable.max.memory.usage = 0.9; It means if the memory usage of local task is more than 90% of its heap size, then the local task will abort by itself. The Driver will know the local work fails and it won't submit the MapJoinTask (a Map Only MapRedTask) to Hadoop, but instead, it will submit the originally CommonJoinTask to Hadoop to run. 3) Set hive.smalltable.filesize = 2500L; It means if the summary of the small table file size is less than 25M, then it will run the map join task. If not, just run the originally common join task. The following is the basic flow how it works. For each common join, create a conditional task. 1) For each join table, generate a mapjoin task by assuming this table is big table. a. The left side of right outer join must be small table. b. The right side of left outer join must be small table. c. No full outer join can be optimized. d. Eg. A left outer join B right outer join C. Only C can be big table table. e. Eg. A right outer join B left outer join C. Only B can be big table table. f. Eg. A left outer join B left outer join C. Only A can be big table table. g. Eg. A right outer join B right outer join C. Both B and C can be big table table. 2) Put all these generated map join tasks into conditional task and set the mapping between big table's alias with the corresponding map join task. 3) During the execution time, the resolver will read the input file size. If the input file size of small table is less than a threshold, than run the converted map join task. 4) Set each map join task with a backup task. The backup task is the originally common join task. This mapping relationship is set during execution time. 5) If the map join task return abnormally, launch the backup task. > Convert join queries to map-join based on size of table/row > --- > > Key: HIVE-1642 > URL: https://issues.apache.org/jira/browse/HIVE-1642 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Liyin Tang > Fix For: 0.7.0 > > > Based on the number of rows and size of each table, Hive should automatically > be able to convert a join into map-join. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1789) Extend metastore schema to store column properties
Extend metastore schema to store column properties -- Key: HIVE-1789 URL: https://issues.apache.org/jira/browse/HIVE-1789 Project: Hive Issue Type: Bug Components: Metastore Reporter: Ashish Thusoo Assignee: Ashish Thusoo A common use case in data warehousing deployments is to set column level policies, e.g. certain columns in the table need to be retained for a longer period than others. Without column level properties, you would have to encode this information into table properties and maintain those properties as columns change. A better solution is to have properties associated with columns. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1788) Add more calls to the metastore thrift interface
[ https://issues.apache.org/jira/browse/HIVE-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931533#action_12931533 ] Ashish Thusoo commented on HIVE-1788: - 3 can be addressed by the current thrift interface so only an iterator over tables owned by particular users is needed. > Add more calls to the metastore thrift interface > > > Key: HIVE-1788 > URL: https://issues.apache.org/jira/browse/HIVE-1788 > Project: Hive > Issue Type: New Feature >Reporter: Ashish Thusoo >Assignee: Ashish Thusoo > > For administrative purposes the following calls to the metastore thrift > interface would be very useful: > 1. Get the table metadata for all the tables owned by a particular users > 2. Ability to iterate over this set of tables > 3. Ability to change a particular key value property of the table -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931518#action_12931518 ] Namit Jain commented on HIVE-78: Also, can you refresh and re-apply the patch ? It does not apply cleanly and is therefore not possible to actually compile/test and understand. > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1788) Add more calls to the metastore thrift interface
Add more calls to the metastore thrift interface Key: HIVE-1788 URL: https://issues.apache.org/jira/browse/HIVE-1788 Project: Hive Issue Type: New Feature Reporter: Ashish Thusoo Assignee: Ashish Thusoo For administrative purposes the following calls to the metastore thrift interface would be very useful: 1. Get the table metadata for all the tables owned by a particular users 2. Ability to iterate over this set of tables 3. Ability to change a particular key value property of the table -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1497) support COMMENT clause on CREATE INDEX, and add new command for SHOW INDEXES
[ https://issues.apache.org/jira/browse/HIVE-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931512#action_12931512 ] John Sichi commented on HIVE-1497: -- +1. Will commit when tests pass. You could have used MetaStoreUtils.getColumnNamesFromFieldSchema, but I'll put a cleanup note for that on HIVE-1764 so we can get this one committed as is (assuming tests pass). > support COMMENT clause on CREATE INDEX, and add new command for SHOW INDEXES > > > Key: HIVE-1497 > URL: https://issues.apache.org/jira/browse/HIVE-1497 > Project: Hive > Issue Type: Improvement > Components: Indexing >Affects Versions: 0.7.0 >Reporter: John Sichi >Assignee: Russell Melick > Fix For: 0.7.0 > > Attachments: HIVE-1497.4.patch, HIVE-1497.5.patch, HIVE-1497.6.patch, > HIVE-1497.7.patch, HIVE-1497.8.patch, hive-1497.p1.patch, hive-1497.p2.patch, > hive-1497.p3.patch > > > We need to work out the syntax for SHOW/DESCRIBE, taking partitioning into > account. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-78) Authorization infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931504#action_12931504 ] Namit Jain commented on HIVE-78: Can you add the tests in the non-thrift patch ? It becomes easier to review > Authorization infrastructure for Hive > - > > Key: HIVE-78 > URL: https://issues.apache.org/jira/browse/HIVE-78 > Project: Hive > Issue Type: New Feature > Components: Metastore, Query Processor, Server Infrastructure >Reporter: Ashish Thusoo >Assignee: He Yongqiang > Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, > hive-78-syntax-v1.patch, HIVE-78.1.nothrift.patch, HIVE-78.1.thrift.patch, > HIVE-78.2.nothrift.patch, HIVE-78.2.thrift.patch, hive-78.diff > > > Allow hive to integrate with existing user repositories for authentication > and authorization infromation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1786) better documentation for str_to_map
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-1786: - Resolution: Fixed Fix Version/s: 0.7.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed. Thanks Siying > better documentation for str_to_map > --- > > Key: HIVE-1786 > URL: https://issues.apache.org/jira/browse/HIVE-1786 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Siying Dong > Fix For: 0.7.0 > > Attachments: HIVE.1786.1.patch > > > Currently, desc and desc extended return the same info. > There is no mention of defaults anywhere -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1786) better documentation for str_to_map
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931448#action_12931448 ] Namit Jain commented on HIVE-1786: -- +1 > better documentation for str_to_map > --- > > Key: HIVE-1786 > URL: https://issues.apache.org/jira/browse/HIVE-1786 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Siying Dong > Attachments: HIVE.1786.1.patch > > > Currently, desc and desc extended return the same info. > There is no mention of defaults anywhere -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1538) FilterOperator is applied twice with ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931447#action_12931447 ] Namit Jain commented on HIVE-1538: -- Talked to Amareshwari offline - the proposal looks good. As a follow-up, we can merge consecutive filters in a follow-up task > FilterOperator is applied twice with ppd on. > > > Key: HIVE-1538 > URL: https://issues.apache.org/jira/browse/HIVE-1538 > Project: Hive > Issue Type: Bug > Components: Query Processor >Reporter: Amareshwari Sriramadasu >Assignee: Amareshwari Sriramadasu > > With hive.optimize.ppd set to true, FilterOperator is applied twice. And it > seems second operator is always filtering zero rows. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1786) better documentation for str_to_map
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1786: -- Status: Patch Available (was: Open) > better documentation for str_to_map > --- > > Key: HIVE-1786 > URL: https://issues.apache.org/jira/browse/HIVE-1786 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Siying Dong > Attachments: HIVE.1786.1.patch > > > Currently, desc and desc extended return the same info. > There is no mention of defaults anywhere -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1786) better documentation for str_to_map
[ https://issues.apache.org/jira/browse/HIVE-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siying Dong updated HIVE-1786: -- Attachment: HIVE.1786.1.patch Added "describe function extend" to function str_to_map(), mentioning the default values. > better documentation for str_to_map > --- > > Key: HIVE-1786 > URL: https://issues.apache.org/jira/browse/HIVE-1786 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Siying Dong > Attachments: HIVE.1786.1.patch > > > Currently, desc and desc extended return the same info. > There is no mention of defaults anywhere -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
