Re: [HACKERS] about fsync in CLOG buffer write

2015-10-06 Thread Robert Haas
On Sun, Oct 4, 2015 at 3:25 PM, Andres Freund wrote: > I don't think it's worth investing time and complexity to bypass SLRU in > certain cases. We should rather rewrite the thing completely. +1. That code is considerably past its sell-by date. -- Robert Haas EnterpriseDB: http://www.enterpris

Re: [HACKERS] about fsync in CLOG buffer write

2015-10-04 Thread Andres Freund
On 2015-10-04 12:14:05 -0700, Jeff Janes wrote: > My (naive) expectation is that no additional locking is needed. > > Once we decide to consult the clog, we already know the transaction is no > longer in progress, so it can't be in-flight to change that clog entry we > care about because it was re

Re: [HACKERS] about fsync in CLOG buffer write

2015-10-04 Thread Jeff Janes
On Sat, Sep 12, 2015 at 5:21 PM, Andres Freund wrote: > On September 12, 2015 5:18:28 PM PDT, Jeff Janes > wrote: > >On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund > >wrote: > > > >> On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > >> > We found there is a fsync call when CLOG buffer > >> > is wri

Re: [HACKERS] about fsync in CLOG buffer write

2015-09-12 Thread Andres Freund
On September 12, 2015 5:18:28 PM PDT, Jeff Janes wrote: >On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund >wrote: > >> On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> > We found there is a fsync call when CLOG buffer >> > is written out in SlruPhysicalWritePage(). It is often called when >a >> backe

Re: [HACKERS] about fsync in CLOG buffer write

2015-09-12 Thread Jeff Janes
On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund wrote: > On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > > We found there is a fsync call when CLOG buffer > > is written out in SlruPhysicalWritePage(). It is often called when a > backend > > needs to check transaction status with SimpleLruReadPage()

[HACKERS] 答复:[HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-09 Thread 周正中(德歌)
 the selected page is clean, we're set.    */   if (!shared->page_dirty[bestvalidslot])    return bestvalidslot; --发件人:Andres Freund 发送时间:2015年9月9日(星期三) 02:28收件人:周正中(德歌) 抄 送:Amit Kapila ,张广舟(明虚) ,pgsql-hackers ,范孝剑(康贤) ,曾文旌(义从) ,窦贤明(执白) ,萧少聪(铁庵) ,陈新坚(惧留孙) 主 题:Re:

Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-09 Thread Andres Freund
On 2015-09-09 10:46:47 -0400, Robert Haas wrote: > Well, if you're filling ~1 clog page per second, you're doing ~1 fsync > per second too. Or if you are not, then you are thrashing the > progressively smaller and smaller number of clean slots ever-harder > until no clean pages remain and you're f

[HACKERS] Re: [HACKERS] Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-09 Thread Robert Haas
On Wed, Sep 9, 2015 at 10:35 AM, Tom Lane wrote: > Robert Haas writes: >> ... How often such a workload actually has to replace a *dirty* clog >> buffer obviously depends on how often you checkpoint, but if you're >> getting ~28k TPS you can completely fill 32 clog buffers (1 million >> transacti

Re: [HACKERS] Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-09 Thread Tom Lane
Robert Haas writes: > ... How often such a workload actually has to replace a *dirty* clog > buffer obviously depends on how often you checkpoint, but if you're > getting ~28k TPS you can completely fill 32 clog buffers (1 million > transactions) in less than 40 seconds, and you're probably not >

[HACKERS] Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-09 Thread Robert Haas
On Tue, Sep 8, 2015 at 2:28 PM, Andres Freund wrote: > On 2015-09-08 15:58:26 +0800, 周正中(德歌) wrote: >> postgres@digoal-> cat 7.sql >> select txid_current(); >> >> postgres@digoal-> pgbench -M prepared -n -r -P 1 -f ./7.sql -c 1 -j 1 -T >> 10 >> About 32K tps. >> progress: 240.0 s, 31164.4 tps

Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-08 Thread Andres Freund
Hi, On 2015-09-08 15:58:26 +0800, 周正中(德歌) wrote: > postgres@digoal-> cat 7.sql > select txid_current(); > > postgres@digoal-> pgbench -M prepared -n -r -P 1 -f ./7.sql -c 1 -j 1 -T  > 10 > About 32K tps. > progress: 240.0 s, 31164.4 tps, lat 0.031 ms stddev 0.183 > progress: 241.0 s, 33243.3 

[HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-08 Thread 周正中(德歌)
- time:1,  pp:process("/opt/pgsql9.4.4/bin/postgres").function("pg_fsync@/opt/soft_bak/postgresql-9.4.4/src/backend/storage/file/fd.c:315").return,  par:0 2089265 postgres(2924): <- time:1,  pp:process("/opt/pgsql9.4.4/bin/postgres").function("SlruPhysicalWriteP

[HACKERS] Re: [HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-08 Thread Amit Kapila
On Sun, Sep 6, 2015 at 7:34 AM, 周正中(德歌) wrote: > > >On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > >> We found there is a fsync call when CLOG buffer > > >> is written out in SlruPhysicalWritePage(). It is often called when a > >> backend > >> needs to check transaction status with SimpleLruRead

[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write

2015-09-07 Thread 周正中(德歌)
>On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> We found there is a fsync call when CLOG buffer >> is written out in SlruPhysicalWritePage(). It is often called when a backend >> needs to check transaction status with SimpleLruReadPage(). >That's when there's not enough buffers available some ot

Re: [HACKERS] about fsync in CLOG buffer write

2015-09-07 Thread 张广舟(明虚)
Andres, Thanks for the reply! I will try the ForwardFsyncRequest-like approach. 在 15-9-2 下午8:32, "Andres Freund" 写入: >On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> We found there is a fsync call when CLOG buffer >> is written out in SlruPhysicalWritePage(). It is often called when a >>backen

[HACKERS] about fsync in CLOG buffer write

2015-09-02 Thread 张广舟(明虚)
Hi Hackers, We host many PG instances on each of our machines, and we want to minimize the calls to fsync within each instance, in order to minimize possible impact on other instances. We found there is a fsync call when CLOG buffer is written out in SlruPhysicalWritePage(). It is often called whe

Re: [HACKERS] about fsync in CLOG buffer write

2015-09-02 Thread Andres Freund
On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > We found there is a fsync call when CLOG buffer > is written out in SlruPhysicalWritePage(). It is often called when a backend > needs to check transaction status with SimpleLruReadPage(). That's when there's not enough buffers available some other, a

[HACKERS] about fsync in CLOG buffer write

2015-09-02 Thread 张广舟(明虚)
Hi Hackers, We host many PG instances on each of our machines, and we want to minimize the calls to fsync within each instance, in order to minimize possible impact on other instances. We found there is a fsync call when CLOG buffer is written out in SlruPhysicalWritePage(). It is often called wh