Subject: Re: Partition Columns
Mungeol,
I did check the # of mappers and that did not change between the two queries
but when I ran a count(*) query the total execution time reduced significantly
for Query1 vs Query2. Also, the amount data the query reads does change when
the where clause
Mungeol,
I did check the # of mappers and that did not change between the two
queries but when I ran a count(*) query the total execution time reduced
significantly for Query1 vs Query2. Also, the amount data the query reads
does change when the where clause changes. I still can't explain why one
Hi, Appan.
you can just simply check the amount of data your query reads from the
table. or the number of the mapper for running that query.
then, you can know whether it filtering or scanning all table.
Of course, it is a lazy approach. but, you can give a try.
I think query 1 should work fine. b
I agree with you Viral. I see the same behavior as well. We are on Hive
0.13 for the cluster where I'm testing this.
On Thu, May 14, 2015 at 2:16 PM, Viral Bajaria
wrote:
> Hi Appan,
>
> In my experience I have seen that Query 2 does not use partition pruning
> because it's not a straight up fil
Hi Appan,
In my experience I have seen that Query 2 does not use partition pruning
because it's not a straight up filtering and involves using functions (aka
UDFs).
What version of Hive are you using ?
Thanks,
Viral
On Thu, May 14, 2015 at 1:48 PM, Appan Thirumaligai
wrote:
> Hi,
>
> I have
