Re: Amcheck: do rightlink verification with lock coupling

2020-08-08 Thread Andrey M. Borodin
> 8 авг. 2020 г., в 23:14, Peter Geoghegan написал(а): > > I pushed a cleaned up version of this patch just now. I added some > commentary about this canonical example in header comments for the new > function. Thanks for working on this! Best regards, Andrey Borodin.

Re: Amcheck: do rightlink verification with lock coupling

2020-08-08 Thread Peter Geoghegan
On Thu, Aug 6, 2020 at 10:59 PM Andrey M. Borodin wrote: > But having complete solution with no false positives seems much better. Agreed. I know that you didn't pursue this for no reason -- having the check available makes bt_check_index() a lot more valuable in practice. It detects what is actu

Re: Amcheck: do rightlink verification with lock coupling

2020-08-06 Thread Andrey M. Borodin
> 6 авг. 2020 г., в 21:38, Peter Geoghegan написал(а): > > On Wed, Aug 5, 2020 at 9:50 PM Andrey M. Borodin wrote: >> Sounds great! Thanks! > > I'm afraid that there is another problem, this time with > btree_xlog_split(). It's possible to get false positives when running > the new test cont

Re: Amcheck: do rightlink verification with lock coupling

2020-08-06 Thread Peter Geoghegan
On Wed, Aug 5, 2020 at 9:50 PM Andrey M. Borodin wrote: > Sounds great! Thanks! I'm afraid that there is another problem, this time with btree_xlog_split(). It's possible to get false positives when running the new test continually on a standby. You can see this by running verification on a stand

Re: Amcheck: do rightlink verification with lock coupling

2020-08-05 Thread Andrey M. Borodin
> 6 авг. 2020 г., в 04:25, Peter Geoghegan написал(а): > > * Added _bt_checkpage() calls for buffers, as is standard practice in nbtree. > > * Added protection against locking the same page a second time in the > event of a faulty sibling link -- we should avoid a self-deadlock in > the event

Re: Amcheck: do rightlink verification with lock coupling

2020-08-05 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 9:33 AM Andrey M. Borodin wrote: > BTW, reviewing this patch again I cannot understand why we verify link > coherence only on leaf level but not for internal pages? > I do not see any actual problems here. Well, I thought that it might be a good idea to limit it to the lea

Re: Amcheck: do rightlink verification with lock coupling

2020-08-04 Thread Andrey M. Borodin
> 4 авг. 2020 г., в 03:44, Peter Geoghegan написал(а): > > On Mon, Jul 20, 2020 at 11:46 AM Andrey M. Borodin > wrote: >> In this thread [0] we decided that lock coupling is necessary for >> btree_xlog_unlink_page(). >> So, maybe let's revive this patch? > > Yes, let's do that. Can you resu

Re: Amcheck: do rightlink verification with lock coupling

2020-08-03 Thread Peter Geoghegan
On Mon, Jul 20, 2020 at 11:46 AM Andrey M. Borodin wrote: > In this thread [0] we decided that lock coupling is necessary for > btree_xlog_unlink_page(). > So, maybe let's revive this patch? Yes, let's do that. Can you resubmit it, please? Peter Geoghegan

Re: Amcheck: do rightlink verification with lock coupling

2020-07-20 Thread Andrey M. Borodin
Hi! > 23 янв. 2020 г., в 00:59, Peter Geoghegan написал(а): > > On Fri, Jan 17, 2020 at 5:43 PM Peter Geoghegan wrote: >> I tried to come up with a specific example of how this could be >> unsafe, but my explanation was all over the place (this could have had >> something to do with it being Fr

Re: Amcheck: do rightlink verification with lock coupling

2020-01-22 Thread Peter Geoghegan
On Fri, Jan 17, 2020 at 5:43 PM Peter Geoghegan wrote: > I tried to come up with a specific example of how this could be > unsafe, but my explanation was all over the place (this could have had > something to do with it being Friday evening). Even still, it's up to > the patch to justify why it's

Re: Amcheck: do rightlink verification with lock coupling

2020-01-17 Thread Peter Geoghegan
On Thu, Jan 16, 2020 at 5:11 PM Peter Geoghegan wrote: > I find this argument convincing. I'll try to get this committed soon. > > While you could have used bt_index_parent_check() or heapallindexed to > detect the issue, those two options are a lot more expensive (plus the > former option won't w

Re: Amcheck: do rightlink verification with lock coupling

2020-01-16 Thread Peter Geoghegan
On Mon, Jan 13, 2020 at 8:47 PM Andrey Borodin wrote: > > 11 янв. 2020 г., в 7:49, Peter Geoghegan написал(а): > > I'm curious why Andrey's corruption problems were not detected by the > > cross-page amcheck test, though. We compare the first non-pivot tuple > > on the right sibling leaf page wit

Re: Amcheck: do rightlink verification with lock coupling

2020-01-16 Thread Andrey Borodin
> 14 янв. 2020 г., в 9:47, Andrey Borodin написал(а): > > Page updates may be lost due to bug in backup software with incremental > backups, bug in storage layer of Aurora-style system, bug in page cache, > incorrect > fsync error handling, bug in ssd firmware etc. And our data checksums do

Re: Amcheck: do rightlink verification with lock coupling

2020-01-13 Thread Andrey Borodin
Hi Peter! Sorry for answering so long. > 11 янв. 2020 г., в 7:49, Peter Geoghegan написал(а): > > I'm curious why Andrey's corruption problems were not detected by the > cross-page amcheck test, though. We compare the first non-pivot tuple > on the right sibling leaf page with the last one on th

Re: Amcheck: do rightlink verification with lock coupling

2020-01-13 Thread Tomas Vondra
On Mon, Jan 13, 2020 at 03:49:40PM -0800, Peter Geoghegan wrote: On Sat, Jan 11, 2020 at 4:25 AM Tomas Vondra wrote: Understood. Is that a reason to not commit of this patch now, though? It could use some polishing. Are you interested in committing it? Not really - as a CFM I was trying to

Re: Amcheck: do rightlink verification with lock coupling

2020-01-13 Thread Peter Geoghegan
On Sat, Jan 11, 2020 at 4:25 AM Tomas Vondra wrote: > Understood. Is that a reason to not commit of this patch now, though? It could use some polishing. Are you interested in committing it? -- Peter Geoghegan

Re: Amcheck: do rightlink verification with lock coupling

2020-01-11 Thread Tomas Vondra
On Fri, Jan 10, 2020 at 06:49:33PM -0800, Peter Geoghegan wrote: On Fri, Jan 10, 2020 at 5:45 PM Tomas Vondra wrote: Peter, any opinion on this proposed amcheck patch? In the other thread [1] you seemed to agree this is worth checking, and Alvaro's proposal to make this check optional seems lik

Re: Amcheck: do rightlink verification with lock coupling

2020-01-10 Thread Peter Geoghegan
On Fri, Jan 10, 2020 at 5:45 PM Tomas Vondra wrote: > Peter, any opinion on this proposed amcheck patch? In the other thread > [1] you seemed to agree this is worth checking, and Alvaro's proposal to > make this check optional seems like a reasonable compromise with respect > to the locking. It's

Re: Amcheck: do rightlink verification with lock coupling

2020-01-10 Thread Tomas Vondra
On Thu, Sep 12, 2019 at 10:16:12AM -0300, Alvaro Herrera wrote: On 2019-Sep-12, Andrey Borodin wrote: This patch violates one of amcheck design principles: current code does not ever take more than one page lock. I do not know: should we hold this rule or should we use more deep check? The ch

Re: Amcheck: do rightlink verification with lock coupling

2019-09-12 Thread Alvaro Herrera
On 2019-Sep-12, Andrey Borodin wrote: > This patch violates one of amcheck design principles: current code > does not ever take more than one page lock. I do not know: should we > hold this rule or should we use more deep check? The check does seem worthwhile to me. As far as I know, in btree yo

Amcheck: do rightlink verification with lock coupling

2019-09-12 Thread Andrey Borodin
Hi! This is a thread to discuss amcheck feature started in other thread[0]. Currently amcheck is scanning every B-tree level. If verification is done with ShareLock - amcheck will test that each page leftlink is pointing to page with rightlink backwards. This is important invariant, in our expe