Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

[Jinyu Zhang]

Update the patch_brin_optimze_mem according to your comment. 






At 2015-10-16 10:13:35, "Alvaro Herrera"  wrote:
>zhangjinyu wrote:
>
>> However I wonder if it would be simpler to have the dtup structure have
>> the pointers, so that you can pass it as NULL in the first call and then
>> followup calls reuse the one allocated in the first call.
>> Jinyu:  the memory is allocated from perRangeCxt  and perRangeCxt will be
>> reset in loop,
>> so this way don't work. 
>
>You're right.  I think we can do better: have brin_deform_tuple accept
>another argument of type BrinMemTuple *, which can be NULL.  If NULL,
>the function calls brin_new_memtuple to allocate a new one (just like
>current code); if not NULL, that one is used.  Have brin_new_memtuple
>also allocate the values/allnulls/hasnulls arrays, which are now part of
>the BrinMemTuple struct.  Then, bringetbitmap calls brin_new_memtuple
>once before changing to perRangeCxt, then reuse the returned inside the
>loop (we probably need brin_memtuple_initialize to clear out the struct
>at the end of each loop iteration).  Other callers of brin_deform_tuple
>just pass NULL to get the current behavior.
>
>-- 
>Álvaro Herrerahttp://www.2ndQuadrant.com/
>PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>
>
>-- 
>Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-hackers


patch_brin_optimize_mem
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

[Jinyu Zhang]

BRIN Scan: Optimize memory allocation in function 'bringetbitmap'.
We can allocate memory for some pointer before do long loop instead of 
allocating
memory in long loop.

Before optimizing code (warm run)
postgres=# select count(*) from lineitem where l_orderkey=1;
 count
---
 6
(1 row)

Time: 456.219 ms

After optimizing code (warm run)
postgres=# select count(*) from lineitem where l_orderkey=1;
 count
---
 6
(1 row)

Time: 349.219 ms

The following shows the DDL of this test case.
CREATE TABLE LINEITEM ( L_ORDERKEYINTEGER NOT NULL,
 L_PARTKEY INTEGER NOT NULL,
 L_SUPPKEY INTEGER NOT NULL,
 L_LINENUMBER  INTEGER NOT NULL,
 L_QUANTITYDECIMAL(15,2) NOT NULL,
 L_EXTENDEDPRICE  DECIMAL(15,2) NOT NULL,
 L_DISCOUNTDECIMAL(15,2) NOT NULL,
 L_TAX DECIMAL(15,2) NOT NULL,
 L_RETURNFLAG  CHAR(1) NOT NULL,
 L_LINESTATUS  CHAR(1) NOT NULL,
 L_SHIPDATEDATE NOT NULL,
 L_COMMITDATE  DATE NOT NULL,
 L_RECEIPTDATE DATE NOT NULL,
 L_SHIPINSTRUCT CHAR(25) NOT NULL,
 L_SHIPMODE CHAR(10) NOT NULL,
 L_COMMENT  VARCHAR(44) NOT NULL);

copy lineitem from '/home/jinyu/mywork/dbgen/lineitem.tbl' delimiter '|';
create index brinLineitem on lineitem using brin(L_ORDERKEY) 
with(pages_per_range = 1);

Jinyu Zhang

[HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

[Jinyu Zhang]

BRIN Scan: Optimize memory allocation in function 'bringetbitmap'.
We can allocate memory for some pointer before do long loop instead of 
allocating
memory in long loop.

Before optimizing code (warm run)
postgres=# select count(*) from lineitem where l_orderkey=1;
 count
---
 6
(1 row)

Time: 456.219 ms

After optimizing code (warm run)
postgres=# select count(*) from lineitem where l_orderkey=1;
 count
---
 6
(1 row)

Time: 349.219 ms

The following shows the DDL of this test case.
CREATE TABLE LINEITEM ( L_ORDERKEYINTEGER NOT NULL,
 L_PARTKEY INTEGER NOT NULL,
 L_SUPPKEY INTEGER NOT NULL,
 L_LINENUMBER  INTEGER NOT NULL,
 L_QUANTITYDECIMAL(15,2) NOT NULL,
 L_EXTENDEDPRICE  DECIMAL(15,2) NOT NULL,
 L_DISCOUNTDECIMAL(15,2) NOT NULL,
 L_TAX DECIMAL(15,2) NOT NULL,
 L_RETURNFLAG  CHAR(1) NOT NULL,
 L_LINESTATUS  CHAR(1) NOT NULL,
 L_SHIPDATEDATE NOT NULL,
 L_COMMITDATE  DATE NOT NULL,
 L_RECEIPTDATE DATE NOT NULL,
 L_SHIPINSTRUCT CHAR(25) NOT NULL,
 L_SHIPMODE CHAR(10) NOT NULL,
 L_COMMENT  VARCHAR(44) NOT NULL);

copy lineitem from '/home/jinyu/mywork/dbgen/lineitem.tbl' delimiter '|';
create index brinLineitem on lineitem using brin(L_ORDERKEY) 
with(pages_per_range = 1);

Jinyu Zhang





网易考拉iPhone6s玫瑰金5288元，现货不加价

patch_optimize_mem
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Did we forget to unpin buf in function "revmap_physical_extend" ?

[Jinyu Zhang]

In function "revmap_physical_extend",  should we add "ReleaseBuffer(buf);" 
between line 438 and 439 ?
422 else
 423 {
 424 if (needLock)
 425 LockRelationForExtension(irel, ExclusiveLock);
 426 
 427 buf = ReadBuffer(irel, P_NEW);
 428 if (BufferGetBlockNumber(buf) != mapBlk)
 429 {
 430 /*
 431  * Very rare corner case: somebody extended the relation
 432  * concurrently after we read its length.  If this happens, 
give
 433  * up and have caller start over.  We will have to evacuate 
that
 434  * page from under whoever is using it.
 435  */
 436 if (needLock)
 437 UnlockRelationForExtension(irel, ExclusiveLock);
 438 LockBuffer(revmap->rm_metaBuf, BUFFER_LOCK_UNLOCK);
 439 return;
 440 }
 441 LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
 442 page = BufferGetPage(buf);
 443 
 444 if (needLock)
 445 UnlockRelationForExtension(irel, ExclusiveLock);
 446 }


Jinyu,
regards