Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-14 Thread Steve Singer
On 13-04-13 04:54 PM, Jeff Janes wrote: On Sat, Apr 13, 2013 at 9:14 AM, Steve Singer ssin...@ca.afilias.info mailto:ssin...@ca.afilias.info wrote: indexTotalCost += index-pages * spc_random_page_cost / 10.0; Is driving my high costs on the inner loop. The index has 2-5

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-13 Thread Steve Singer
On 13-04-12 09:20 PM, Jeff Janes wrote: On Thursday, April 11, 2013, Steve Singer wrote: I think the reason why it is picking the hash join based plans is because of Index Scan using table_b_1_ptid_orgid_ym_unq on table_b_1 b (cost=0.00..503.86 rows=1 width=10) (actual

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-13 Thread Jeff Janes
On Sat, Apr 13, 2013 at 9:14 AM, Steve Singer ssin...@ca.afilias.infowrote: indexTotalCost += index-pages * spc_random_page_cost / 10.0; Is driving my high costs on the inner loop. The index has 2-5 million pages depending on the partition . If I run this against 9.2.2 with /

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-12 Thread Jeff Janes
On Thursday, April 11, 2013, Steve Singer wrote: I think the reason why it is picking the hash join based plans is because of Index Scan using table_b_1_ptid_orgid_ym_unq on table_b_1 b (cost=0.00..503.86 rows=1 width=10) (actual time=0.016..0.017 rows=1 loops=414249)

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-11 Thread Steve Singer
On 13-04-10 07:54 PM, Steve Singer wrote: On 13-04-10 02:06 PM, Jeff Janes wrote: On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer ssin...@ca.afilias.info mailto:ssin...@ca.afilias.info wrote: I think the index recheck means your bitmap is overflowing (i.e. needing more space than work_mem)

[PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Steve Singer
I'm encountering an issue where PG 9.2.4 (we also see this with 9.2.3) is picking a plan involving a bitmap heap scan that turns out to be much slower than a nested-loop plan using indexes. The planner picks the hashjoin plan by default (see attached files) Bitmap Heap Scan on

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread k...@rice.edu
On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: I'm encountering an issue where PG 9.2.4 (we also see this with 9.2.3) is picking a plan involving a bitmap heap scan that turns out to be much slower than a nested-loop plan using indexes. The planner picks the hashjoin plan by

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Steve Singer
On 13-04-10 09:56 AM, k...@rice.edu wrote: On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: Hi Steve, The one thing that stands out to me is that you are working with 200GB of data on a machine with 4-8GB of ram and you have the random_page_cost set to 2.0. That is almost

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread k...@rice.edu
On Wed, Apr 10, 2013 at 11:56:32AM -0400, Steve Singer wrote: On 13-04-10 09:56 AM, k...@rice.edu wrote: On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: Hi Steve, The one thing that stands out to me is that you are working with 200GB of data on a machine with 4-8GB of ram

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Jeff Janes
On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer ssin...@ca.afilias.infowrote: I'm encountering an issue where PG 9.2.4 (we also see this with 9.2.3) is picking a plan involving a bitmap heap scan that turns out to be much slower than a nested-loop plan using indexes. The planner picks the

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Jeff Janes
On Wed, Apr 10, 2013 at 8:56 AM, Steve Singer ssin...@ca.afilias.infowrote: On 13-04-10 09:56 AM, k...@rice.edu wrote: On Wed, Apr 10, 2013 at 09:49:55AM -0400, Steve Singer wrote: Hi Steve, The one thing that stands out to me is that you are working with 200GB of data on a machine with

Re: [PERFORM] slow bitmap heap scans on pg 9.2

2013-04-10 Thread Steve Singer
On 13-04-10 02:06 PM, Jeff Janes wrote: On Wed, Apr 10, 2013 at 6:49 AM, Steve Singer ssin...@ca.afilias.info mailto:ssin...@ca.afilias.info wrote: I think the index recheck means your bitmap is overflowing (i.e. needing more space than work_mem) and so keeping only the pages which have at