[jira] [Updated] (HIVE-16746) Reduce number of index lookups for same table in IndexWhereTaskDispatcher

2017-05-23 Thread Rajesh Balamohan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan updated HIVE-16746:

Status: Patch Available  (was: Open)

> Reduce number of index lookups for same table in IndexWhereTaskDispatcher
> -
>
> Key: HIVE-16746
> URL: https://issues.apache.org/jira/browse/HIVE-16746
> Project: Hive
>  Issue Type: Bug
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-16746.1.patch
>
>
> {{IndexWhereTaskDispatcher}} is used when 
> {{hive.optimize.index.filter=true}}. It lists all indices for the table and 
> depending on the query complexity, this ends up being in the hotpath. For 
> e.g, Q14 explain plan takes 180-200 seconds and this index querying multiple 
> times for same tables take up 30-40 seconds. This function was invoked around 
> 24000 times for same set of tables.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16746) Reduce number of index lookups for same table in IndexWhereTaskDispatcher

2017-05-23 Thread Rajesh Balamohan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan updated HIVE-16746:

Attachment: HIVE-16746.1.patch

With patch, response time came down from 185 seconds to 140-145 seconds. This 
benefits irrespective of store implementation. Currently CachedStore does not 
yet implement index caching.

> Reduce number of index lookups for same table in IndexWhereTaskDispatcher
> -
>
> Key: HIVE-16746
> URL: https://issues.apache.org/jira/browse/HIVE-16746
> Project: Hive
>  Issue Type: Bug
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-16746.1.patch
>
>
> {{IndexWhereTaskDispatcher}} is used when 
> {{hive.optimize.index.filter=true}}. It lists all indices for the table and 
> depending on the query complexity, this ends up being in the hotpath. For 
> e.g, Q14 explain plan takes 180-200 seconds and this index querying multiple 
> times for same tables take up 30-40 seconds. This function was invoked around 
> 24000 times for same set of tables.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16746) Reduce number of index lookups for same table in IndexWhereTaskDispatcher

2017-05-24 Thread Rajesh Balamohan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan updated HIVE-16746:

   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

Thanks for the review [~ashutoshc]. Committed to master.

> Reduce number of index lookups for same table in IndexWhereTaskDispatcher
> -
>
> Key: HIVE-16746
> URL: https://issues.apache.org/jira/browse/HIVE-16746
> Project: Hive
>  Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-16746.1.patch
>
>
> {{IndexWhereTaskDispatcher}} is used when 
> {{hive.optimize.index.filter=true}}. It lists all indices for the table and 
> depending on the query complexity, this ends up being in the hotpath. For 
> e.g, Q14 explain plan takes 180-200 seconds and this index querying multiple 
> times for same tables take up 30-40 seconds. This function was invoked around 
> 24000 times for same set of tables.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)