Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-07 Thread Amul Sul
On Fri, Nov 3, 2023 at 10:59 AM Dilip Kumar wrote: > On Mon, Oct 30, 2023 at 11:50 AM Dilip Kumar > wrote: > > [...] > [1] 0001-Make-all-SLRU-buffer-sizes-configurable: This is the same > patch as the previous patch set > [2] 0002-Add-a-buffer-mapping-table-for-SLRUs: Patch to introduce > buffer

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-07 Thread Amul Sul
On Mon, Nov 6, 2023 at 4:44 PM Andrey M. Borodin wrote: > > > > On 6 Nov 2023, at 14:31, Alvaro Herrera wrote: > > > > dynahash is notoriously slow, which is why we have simplehash.h since > > commit b30d3ea824c5. Maybe we could use that instead. > > Dynahash has lock partitioning. Simplehash h

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-06 Thread Dilip Kumar
On Mon, Nov 6, 2023 at 4:44 PM Andrey M. Borodin wrote: > > On 6 Nov 2023, at 14:31, Alvaro Herrera wrote: > > > > dynahash is notoriously slow, which is why we have simplehash.h since > > commit b30d3ea824c5. Maybe we could use that instead. > > Dynahash has lock partitioning. Simplehash has no

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-06 Thread Andrey M. Borodin
> On 6 Nov 2023, at 14:31, Alvaro Herrera wrote: > > dynahash is notoriously slow, which is why we have simplehash.h since > commit b30d3ea824c5. Maybe we could use that instead. Dynahash has lock partitioning. Simplehash has not, AFAIK. The thing is we do not really need a hash function - p

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-06 Thread Alvaro Herrera
On 2023-Nov-06, Dilip Kumar wrote: > Yeah so we can see with a small bank size <=16 slots we are seeing > that the fetching page with hash is 30% slower than the sequential > search, but beyond 32 slots sequential search is become slower as you > grow the number of slots whereas with hash it stays

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-06 Thread Dilip Kumar
On Mon, Nov 6, 2023 at 1:05 PM Andrey M. Borodin wrote: > > On 6 Nov 2023, at 09:09, Dilip Kumar wrote: > > > > > >> Having hashtable to find SLRU page in the buffer IMV is too slow. Some > >> comments on this approach can be found here [0]. > >> I'm OK with having HTAB for that if we are sure

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-05 Thread Andrey M. Borodin
> On 6 Nov 2023, at 09:09, Dilip Kumar wrote: > > >> Having hashtable to find SLRU page in the buffer IMV is too slow. Some >> comments on this approach can be found here [0]. >> I'm OK with having HTAB for that if we are sure performance does not degrade >> significantly, but I really doubt

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-05 Thread Dilip Kumar
On Sun, Nov 5, 2023 at 1:37 AM Andrey M. Borodin wrote: > On 30 Oct 2023, at 09:20, Dilip Kumar wrote: > > changed the logic of SlruAdjustNSlots() in 0002, such that now it > starts with the next power of 2 value of the configured slots and > keeps doubling the number of banks until we reach the

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-11-04 Thread Andrey M. Borodin
> On 30 Oct 2023, at 09:20, Dilip Kumar wrote: > > changed the logic of SlruAdjustNSlots() in 0002, such that now it > starts with the next power of 2 value of the configured slots and > keeps doubling the number of banks until we reach the number of banks > to the max SLRU_MAX_BANKS(128) and b

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-25 Thread Dilip Kumar
On Wed, Oct 25, 2023 at 5:58 PM Amit Kapila wrote: > > On Fri, Oct 20, 2023 at 9:40 AM Dilip Kumar wrote: > > > > On Sat, Oct 14, 2023 at 9:43 AM Amit Kapila wrote: > > > > > > This and other results shared by you look promising. Will there be any > > > improvement in workloads related to clog b

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-25 Thread Amit Kapila
On Fri, Oct 20, 2023 at 9:40 AM Dilip Kumar wrote: > > On Sat, Oct 14, 2023 at 9:43 AM Amit Kapila wrote: > > > > This and other results shared by you look promising. Will there be any > > improvement in workloads related to clog buffer usage? > > I did not understand this question can you explai

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Dilip Kumar
On Tue, Oct 24, 2023 at 9:34 PM Alvaro Herrera wrote: > > On 2023-Oct-11, Dilip Kumar wrote: > > > In my last email, I forgot to give the link from where I have taken > > the base path for dividing the buffer pool in banks so giving the same > > here[1]. And looking at this again it seems that th

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Nathan Bossart
On Tue, Oct 24, 2023 at 06:04:13PM +0200, Alvaro Herrera wrote: > Like everybody else, I like having less GUCs to configure, but going > this far to avoid them looks rather disastrous to me. IMO we should > just use Munro's older patches that gave one GUC per SLRU, and users > only need to increas

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-24 Thread Alvaro Herrera
On 2023-Oct-11, Dilip Kumar wrote: > In my last email, I forgot to give the link from where I have taken > the base path for dividing the buffer pool in banks so giving the same > here[1]. And looking at this again it seems that the idea of that > patch was from Andrey M. Borodin and the idea of

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-19 Thread Dilip Kumar
On Sat, Oct 14, 2023 at 9:43 AM Amit Kapila wrote: > > This and other results shared by you look promising. Will there be any > improvement in workloads related to clog buffer usage? I did not understand this question can you explain this a bit? In short, if it is regarding the performance then

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-13 Thread Amit Kapila
On Wed, Oct 11, 2023 at 4:35 PM Dilip Kumar wrote: > > The small size of the SLRU buffer pools can sometimes become a > performance problem because it’s not difficult to have a workload > where the number of buffers actively in use is larger than the > fixed-size buffer pool. However, just increas

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-12 Thread Dilip Kumar
On Wed, Oct 11, 2023 at 5:57 PM Dilip Kumar wrote: > > On Wed, Oct 11, 2023 at 4:34 PM Dilip Kumar wrote: > In my last email, I forgot to give the link from where I have taken > the base path for dividing the buffer pool in banks so giving the same > here[1]. And looking at this again it seems

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-11 Thread Dilip Kumar
On Wed, Oct 11, 2023 at 4:34 PM Dilip Kumar wrote: > > The small size of the SLRU buffer pools can sometimes become a > performance problem because it’s not difficult to have a workload > where the number of buffers actively in use is larger than the > fixed-size buffer pool. However, just increas

SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-11 Thread Dilip Kumar
The small size of the SLRU buffer pools can sometimes become a performance problem because it’s not difficult to have a workload where the number of buffers actively in use is larger than the fixed-size buffer pool. However, just increasing the size of the buffer pool doesn’t necessarily help, beca

<    1   2