Re: [HACKERS] Optimization for lazy_scan_heap

2016-10-17 Thread Masahiko Sawada
On Mon, Oct 3, 2016 at 10:59 AM, Michael Paquier wrote: > On Mon, Sep 26, 2016 at 5:26 AM, Rahila Syed wrote: >> Some initial comments on optimize_lazy_scan_heap_v2.patch. > > Seems worth pursuing. Marking as returned with feedback because of >

Re: [HACKERS] Optimization for lazy_scan_heap

2016-10-02 Thread Michael Paquier
On Mon, Sep 26, 2016 at 5:26 AM, Rahila Syed wrote: > Some initial comments on optimize_lazy_scan_heap_v2.patch. Seems worth pursuing. Marking as returned with feedback because of lack of activity and some basic reviews sent. -- Michael -- Sent via pgsql-hackers

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-25 Thread Rahila Syed
Hello, Some initial comments on optimize_lazy_scan_heap_v2.patch. >- while (next_unskippable_block < nblocks) >+ while (next_unskippable_block < nblocks && >+ !FORCE_CHECK_PAGE(next_unskippable_block)) Dont we need similar check of

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Masahiko Sawada
On Thu, Sep 8, 2016 at 11:27 PM, Jim Nasby wrote: > On 9/8/16 3:03 AM, Masahiko Sawada wrote: >> >> Current manual vacuum doesn't output how may all_frozen pages we >> skipped according to visibility map. >> That's why I attached 0001 patch which makes the manual vacuum

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Robert Haas
On Thu, Sep 8, 2016 at 4:03 AM, Masahiko Sawada wrote: > On Wed, Sep 7, 2016 at 4:11 PM, Simon Riggs wrote: >> On 7 September 2016 at 04:13, Masahiko Sawada wrote: >> >>> Since current HEAD could scan visibility map twice, the

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Jim Nasby
On 9/8/16 3:03 AM, Masahiko Sawada wrote: Current manual vacuum doesn't output how may all_frozen pages we skipped according to visibility map. That's why I attached 0001 patch which makes the manual vacuum emit such information. I think we should add that, and info about all-frozen skips,

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Masahiko Sawada
On Wed, Sep 7, 2016 at 4:11 PM, Simon Riggs wrote: > On 7 September 2016 at 04:13, Masahiko Sawada wrote: > >> Since current HEAD could scan visibility map twice, the execution time >> of Patched is approximately half of HEAD. > > Sounds good. > > To

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-07 Thread Robert Haas
On Tue, Sep 6, 2016 at 11:13 PM, Masahiko Sawada wrote: > I understood, thank you. > > I've measured the performance benefit of this patch by following steps. > 1. Create very large table and set all-visible flag to all blocks. > 2. Measure the execution time of vacuum that

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-07 Thread Simon Riggs
On 7 September 2016 at 04:13, Masahiko Sawada wrote: > Since current HEAD could scan visibility map twice, the execution time > of Patched is approximately half of HEAD. Sounds good. To ensure we are doing exactly same amount of work as before, did you see the output of

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-06 Thread Masahiko Sawada
On Wed, Sep 7, 2016 at 1:47 AM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >>> What performance difference does this make, in a realistic use case? >> >> I have yet to measure performance effect but it would be effect

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-06 Thread Robert Haas
On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >> What performance difference does this make, in a realistic use case? > > I have yet to measure performance effect but it would be effect for > very large frozen table. I think if you are proposing this patch because

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Masahiko Sawada
On Mon, Sep 5, 2016 at 6:27 PM, Simon Riggs wrote: > On 4 August 2016 at 05:57, Masahiko Sawada wrote: > >> While reviewing freeze map code, Andres pointed out[1] that >> lazy_scan_heap could accesses visibility map twice and its logic is >> seems a

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Simon Riggs
On 4 August 2016 at 05:57, Masahiko Sawada wrote: > While reviewing freeze map code, Andres pointed out[1] that > lazy_scan_heap could accesses visibility map twice and its logic is > seems a bit tricky. > As discussed before, it's not nice especially when large relation

Re: [HACKERS] Optimization for lazy_scan_heap

2016-08-25 Thread Masahiko Sawada
On Thu, Aug 25, 2016 at 1:41 AM, Anastasia Lubennikova wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: not tested > Spec compliant: not tested >

Re: [HACKERS] Optimization for lazy_scan_heap

2016-08-24 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, I haven't tested the performance yet, but the patch itself looks

[HACKERS] Optimization for lazy_scan_heap

2016-08-03 Thread Masahiko Sawada
Hi all, While reviewing freeze map code, Andres pointed out[1] that lazy_scan_heap could accesses visibility map twice and its logic is seems a bit tricky. As discussed before, it's not nice especially when large relation is entirely frozen. I posted the patch for that before but since this is