Re: LIKE filter pushdown for tables and partitions

2013-08-27 Thread Stephen Sprague
Thanks Sergey for that. Good stuff. I can't speak for everybody here obviously but Hive partition elimination is critical - its gotta happen somehow. However, if JDOQL method isn't robust around the edges i'm fine with finding something better. So if I get you right you're saying by removing th

Re: LIKE filter pushdown for tables and partitions

2013-08-27 Thread Sergey Shelukhin
This method is used to prune partitions for the job (separately from actually processing data). There are a few ways to get partitions from Hive for a query (to avoid reading all partitions when filtering involves partition columns) - get-by-filter that I want to modify is one of them. Hive itself

Re: LIKE filter pushdown for tables and partitions

2013-08-27 Thread Stephen Sprague
sorry to be dumb-ass but what does that translate into in the HSQL dialect? Judging from the name you use, getPartitionsByFilter, you're saying you want to remove the use case of using like clause on a partition column? if so, um, yeah, i would think that's surely used. On Mon, Aug 26, 2013 at

Re: LIKE filter pushdown for tables and partitions

2013-08-26 Thread Sergey Shelukhin
Adding user list. Any objections to removing LIKE support from getPartitionsByFilter? On Mon, Aug 26, 2013 at 2:54 PM, Ashutosh Chauhan wrote: > Couple of questions: > > 1. What about LIKE operator for Hive itself? Will that continue to work > (presumably because there is an alternative path for

Re: LIKE filter pushdown for tables and partitions

2013-08-26 Thread Ashutosh Chauhan
Couple of questions: 1. What about LIKE operator for Hive itself? Will that continue to work (presumably because there is an alternative path for that). 2. This will nonetheless break other direct consumers of metastore client api (like HCatalog). I see your point that we have a buggy implementat

Re: LIKE filter pushdown for tables and partitions

2013-08-26 Thread Sergey Shelukhin
Since there's no response I am assuming nobody cares about this code... Jira is HIVE-5134, I will attach a patch with removal this week. On Wed, Aug 21, 2013 at 2:28 PM, Sergey Shelukhin wrote: > Hi. > > I think there are issues with the way hive can currently do LIKE > operator JDO pushdown and

LIKE filter pushdown for tables and partitions

2013-08-21 Thread Sergey Shelukhin
Hi. I think there are issues with the way hive can currently do LIKE operator JDO pushdown and it the code should be removed for partitions and tables. Are there objections to removing LIKE from Filter.g and related areas? If no I will file a JIRA and do it. Details: There's code in metastore tha