Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-10-02 Thread Michael Paquier
On Wed, Oct 03, 2018 at 10:54:14AM +0500, Andrey Borodin wrote: > Let's move this to CF 2018-11? Obviously, it is WiP, but it seems that > patch is being discussed, author cares about it. If you are still working on it, which is not something obvious based on the thread activity, then moving it t

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-10-02 Thread Andrey Borodin
Hi! > 2 окт. 2018 г., в 11:55, Michail Nikolaev > написал(а): > > > Okay, it has been more than a couple of days and the patch has not been > > updated, so I am marking as returned with feedback. > > Yes, it is more than couple of days passed, but also there is almost no > feedback since 20

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-10-01 Thread Michail Nikolaev
Hello. > Okay, it has been more than a couple of days and the patch has not been > updated, so I am marking as returned with feedback. Yes, it is more than couple of days passed, but also there is almost no feedback since 20 Mar after patch design was changed :) But seriously - I still working on

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-10-01 Thread Michael Paquier
On Mon, Jul 16, 2018 at 02:11:57PM +0300, Michail Nikolaev wrote: > Thanks a lot for your feedback. I'll try to update patch in few days > (currently stuck at small performance regression in unknown place). Okay, it has been more than a couple of days and the patch has not been updated, so I am ma

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-07-16 Thread Michail Nikolaev
Hello. Thanks a lot for your feedback. I'll try to update patch in few days (currently stuck at small performance regression in unknown place). Regarding issue with delete: yes, it is valid point, but record removing should clear visibility buffer - and tuple will be fetched from heap to test its

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-07-13 Thread Heikki Linnakangas
On 21/05/18 18:43, Michail Nikolaev wrote: Hello everyone. This letter related to “Extended support for index-only-scan” from my previous message in the thread. WIP version of the patch is ready for a while now and I think it is time to resume the work on the feature. BTW, I found a small articl

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-05-22 Thread Michail Nikolaev
Hello. > 1. Charts are measured in percents of pgbench TPS, right? Yes, correct. Actual values are calculated as TPS_of_patched / TPS_of_vanilla. TPS was measured using single postgres process (one core) (I was also did tests with multiple processes, but they shows pretty same results). > 2. For

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-05-22 Thread Andrey Borodin
Hi, Michail! > 21 мая 2018 г., в 20:43, Michail Nikolaev > написал(а): > This letter related to “Extended support for index-only-scan” from my > previous message in the thread. This is great that you continue your work in this direction! The concept of scan that you propose looks interesting.

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-04-09 Thread Andrey Borodin
Hi! The work on the patch goes on, where was some discussion of this patch off-list with author. Advise-request is still actual. I think that we should move this patch to next CF. So I'm marking patch as needs review. Best regards, Andrey Borodin.

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-20 Thread Michail Nikolaev
Hello everyone. I need an advice. I was reworking the patch: added support for the planner, added support for queries with projection, addded support for predicates which could be executed over index data. And.. I realized that my IndexScan is even more index-only than the original IndexOnlyScan.

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-13 Thread Michail Nikolaev
Hello. Tom, thanks a lot for your thorough review. > What you've done to > IndexNext() is a complete disaster from a modularity standpoint: it now > knows all about the interactions between index_getnext, index_getnext_tid, > and index_fetch_heap. I was looking into the current IndexOnlyNext imp

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-12 Thread Andrey Borodin
Tom, thanks for inspecting the patch. There's so many problems that slipped from my attention... But one thing that bothers me most is the problem with predicate locking > 13 марта 2018 г., в 0:55, Tom Lane написал(а): > > The PredicateLockPage call also troubles me quite a bit, not only from >

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-12 Thread Tom Lane
Michail Nikolaev writes: > [ offset_index_only_v4.patch ] I started to go through this patch with the intention of committing it, but the more I looked at it the less happy I got. What you've done to IndexNext() is a complete disaster from a modularity standpoint: it now knows all about the inte

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-12 Thread Andrey Borodin
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed I've tested this patch with different types of order by, incl

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-12 Thread Michail Nikolaev
Hello. > Sorry, seems like I've incorrectly expressed what I wanted to say. > I mean this Assert() can be checked before loop, not on every loop cycle. Yes, I understood it. Condition should be checked every cycle - at least it is done this way for index only scan: https://github.com/postgres/pos

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-11 Thread Andrey Borodin
> 10 марта 2018 г., в 19:20, Michail Nikolaev > написал(а): > > > Also, I think that this check could be removed from loop. We do not expect > > that it's state will change during execution, do we? > > Removed. Sorry, seems like I've incorrectly expressed what I wanted to say. I mean this

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-10 Thread Michail Nikolaev
Hello. Andrey, Tels - thanks for review. > It could be named "SkipTuples" (e.g. this is the number of tuples we need > to skip, not the number we have skipped), and the other one then > "iss_SkipTuplesRemaining" so they are consistent with each other. Agreed, done. > Also, I think that this che

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-06 Thread Andrey Borodin
> 7 марта 2018 г., в 3:25, Tels написал(а): > > It could be named "SkipTuples" (e.g. this is the number of tuples we need > to skip, not the number we have skipped), and the other one then > "iss_SkipTuplesRemaining" so they are consistent with each other. I agree that name sounds strange (eve

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-06 Thread Tels
Hello Michail, On Tue, March 6, 2018 4:03 pm, Michail Nikolaev wrote: > Hello, Andrey. > > Thanks for review. > > I have updated comments according your review also renamed some fields for > consistency. > Additional some notes added to documentation. > > Updated patch in attach, github updated to

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-06 Thread Michail Nikolaev
Hello, Andrey. Thanks for review. I have updated comments according your review also renamed some fields for consistency. Additional some notes added to documentation. Updated patch in attach, github updated too. offset_index_only_v3.patch Description: Binary data

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-01 Thread Andrey Borodin
Hi, Michail! Here are points that we need to address before advancing the patch. > 20 февр. 2018 г., в 11:45, Andrey Borodin написал(а): > > Minor spots: > There are some trailing whitespaces at line ends >> Offset cannot be optimized because parallel execution > I'd replace with >> Offset can

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-02-19 Thread Andrey Borodin
Hi! I've played with patch. I observe that in some expected scenarios it reduces read buffers significantly. > 14 февр. 2018 г., в 0:04, Michail Nikolaev > написал(а): > Patch updated + rebased on master. check-world is passing. Minor spots: There are some trailing whitespaces at line ends >

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-02-13 Thread Tom Lane
Michail Nikolaev writes: > Still not sure about comment formatting. Have you seen any style guid about > it except "strict ANSI C comment formatting"? Anyway still need to work on > comments. The short answer is "make the new code look like the code around it". But there is actually documentation

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-02-13 Thread Michail Nikolaev
Hello. Thanks a lot for review. Patch updated + rebased on master. check-world is passing. Still not sure about comment formatting. Have you seen any style guid about it except "strict ANSI C comment formatting"? Anyway still need to work on comments. Also, non-MVCC snaphots are now supported.

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-02-05 Thread Andrey Borodin
Hi, Michail! Thanks for the patch! > 1 февр. 2018 г., в 1:17, Michail Nikolaev > написал(а): > > Hello. > > WIP-Patch for optimisation of OFFSET + IndexScan using visibility map. While the patch seems to me useful improvement, I see few problems with code: 1. Both branches of "if (node->iss_

[WIP PATCH] Index scan offset optimisation using visibility map

2018-01-31 Thread Michail Nikolaev
Hello. WIP-Patch for optimisation of OFFSET + IndexScan using visibility map. Patch based on idea of Maxim Boguk [1] with some inspiration from Douglas Doole [2]. - Everyone knows - using OFFSET (especially big) is not an good practice. But in reality they widely used mostly for paging (be