Re: Assert in pageinspect with NULL pages

2022-04-13 Thread Michael Paquier
On Wed, Mar 16, 2022 at 05:43:48PM +0900, Michael Paquier wrote: > So, I have looked at this second part of the thread, and concluded > that we should not fail for empty pages. First, we fetch pages from > the buffer pool in normal mode, where empty pages are valid. There is > also a second point

Re: Assert in pageinspect with NULL pages

2022-03-29 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 08:29:29PM +0300, Maxim Orlov wrote: > I've suddenly found that the test in this patch is based on a fact that > heap pages don't have PageSpecial or it is of different size with btree > pages Special area: > > CREATE TABLE test1 (a int8, b int4range); > > SELECT bt_pag

Re: Assert in pageinspect with NULL pages

2022-03-28 Thread Maxim Orlov
I've suddenly found that the test in this patch is based on a fact that heap pages don't have PageSpecial or it is of different size with btree pages Special area: CREATE TABLE test1 (a int8, b int4range); SELECT bt_page_items(get_raw_page('test1', 0)); In the current state is is so, but it is

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Michael Paquier
On Sun, Mar 27, 2022 at 02:36:54PM -0700, Peter Geoghegan wrote: > I think that it might actually be fundamentally impossible to > guarantee that that'll be safe, because we have no idea what the > output function might be doing. It's arbitrary user-defined code that > could easily be implemented i

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Peter Geoghegan
On Sun, Mar 27, 2022 at 2:02 PM Robert Haas wrote: > On Sun, Mar 27, 2022 at 4:26 PM Peter Geoghegan wrote: > > We're not dealing > > with adversarial page images here. > > I think it's bad that we have to make that assumption, considering > that there's nothing whatever to keep users from supply

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Robert Haas
On Sun, Mar 27, 2022 at 4:26 PM Peter Geoghegan wrote: > We're not dealing > with adversarial page images here. I think it's bad that we have to make that assumption, considering that there's nothing whatever to keep users from supplying arbitrary page images to pageinspect. But I also agree that

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Peter Geoghegan
On Sun, Mar 27, 2022 at 7:24 AM Robert Haas wrote: > It's my impression that there are many ways of crashing the system > using pageinspect right now. We aren't very careful about making sure > that our code that reads from disk doesn't crash if the data on disk > is corrupted, and all of that sys

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Matthias van de Meent
On Sun, 27 Mar 2022 at 16:24, Robert Haas wrote: > > On Fri, Mar 25, 2022 at 12:57 AM Michael Paquier wrote: > > On Thu, Mar 24, 2022 at 08:54:14PM -0700, Peter Geoghegan wrote: > > > amcheck's palloc_btree_page() function validates that an 8KiB page is > > > in fact an nbtree page, in a maximall

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Robert Haas
On Fri, Mar 25, 2022 at 12:57 AM Michael Paquier wrote: > On Thu, Mar 24, 2022 at 08:54:14PM -0700, Peter Geoghegan wrote: > > amcheck's palloc_btree_page() function validates that an 8KiB page is > > in fact an nbtree page, in a maximally paranoid way. Might be an > > example worth following here

Re: Assert in pageinspect with NULL pages

2022-03-27 Thread Michael Paquier
On Fri, Mar 25, 2022 at 12:27:24PM +0900, Michael Paquier wrote: > Makes sense. Fine by me to stick an extra "btree" here. I have been able to look at that again today (earlier than expected), and except for one incorrect thing in the GIN tests where we were not testing the correct pattern, the r

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Thu, Mar 24, 2022 at 08:54:14PM -0700, Peter Geoghegan wrote: > amcheck's palloc_btree_page() function validates that an 8KiB page is > in fact an nbtree page, in a maximally paranoid way. Might be an > example worth following here. Sure, we could do that. However, I don't think that going dow

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Peter Geoghegan
On Wed, Mar 23, 2022 at 1:16 AM Michael Paquier wrote: > As far as I can see, this is > also possible in bt_page_items_bytea(), gist_page_opaque_info(), > gin_metapage_info() and gin_page_opaque_info(). All those code paths > should be protected with some checks on PageGetSpecialSize(), I > guess

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:03:47AM +0800, Julien Rouhaud wrote: > I'm happy with all the changes, except: > > + if (P_ISLEAF(opaque) && opaque->btpo_level != 0) > + ereport(ERROR, > + > (errcode(ERRCODE_INVALID_PARAMETER_VALUE),

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Julien Rouhaud
On Fri, Mar 25, 2022 at 11:44:26AM +0900, Michael Paquier wrote: > I have reviewed what you have sent, bumping on a couple of issues: Thanks! I'm happy with all the changes, except: + if (P_ISLEAF(opaque) && opaque->btpo_level != 0) + ereport(ERROR, +

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:27:40PM +0800, Julien Rouhaud wrote: > I worked on a patch to fix the problem. The functions you mention were indeed > missing some check, but after digging a bit more I found that other problem > existed. For instance, feeding a btree page to a gist_page_items_bytea()

Re: Assert in pageinspect with NULL pages

2022-03-24 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 05:16:41PM +0900, Michael Paquier wrote: > On Fri, Mar 18, 2022 at 06:43:39AM +0300, Alexander Lakhin wrote: > > Hello Michael, > > No, just x86_64, Ubuntu 20.04, gcc 11, valgrind 3.15. I just put that query > > in page.sql and see the server abort. > > Bah, of course,

Re: Assert in pageinspect with NULL pages

2022-03-23 Thread Michael Paquier
On Fri, Mar 18, 2022 at 06:43:39AM +0300, Alexander Lakhin wrote: > Hello Michael, > No, just x86_64, Ubuntu 20.04, gcc 11, valgrind 3.15. I just put that query > in page.sql and see the server abort. Bah, of course, I have missed the valgrind part of the problem. The problem is that we attempt t

Re: Assert in pageinspect with NULL pages

2022-03-17 Thread Alexander Lakhin
Hello Michael, 18.03.2022 05:06, Michael Paquier wrote: Okay. I'll try to look more closely at that, then. It feels like we are just missing a MAXALIGN() in those code paths. Are you using any specific CFLAGS or something environment-sensitive (macos, 32b, etc.)? No, just x86_64, Ubuntu 20.04,

Re: Assert in pageinspect with NULL pages

2022-03-17 Thread Michael Paquier
On Wed, Mar 16, 2022 at 08:35:59PM +0300, Alexander Lakhin wrote: > Yes, I've reproduced that issue on master (46d9bfb0). Okay. I'll try to look more closely at that, then. It feels like we are just missing a MAXALIGN() in those code paths. Are you using any specific CFLAGS or something environ

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Alexander Lakhin
Hello Michael, 16.03.2022 10:39, Michael Paquier wrote: On Mon, Feb 21, 2022 at 10:00:00AM +0300, Alexander Lakhin wrote: Could you please confirm before committing the patchset that it fixes the bug #16527 [1]? Or maybe I could check it? (Original patch proposed by Daria doesn't cover that case

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Michael Paquier
On Wed, Feb 23, 2022 at 12:09:02PM +0500, Daria Lepikhova wrote: > And one more addition. In the previous version of the patch, I forgot to add > tests for the gist index, but the described problem is also relevant for it. So, I have looked at this second part of the thread, and concluded that we

Re: Assert in pageinspect with NULL pages

2022-03-16 Thread Michael Paquier
On Mon, Feb 21, 2022 at 10:00:00AM +0300, Alexander Lakhin wrote: > Could you please confirm before committing the patchset that it fixes > the bug #16527 [1]? Or maybe I could check it? > (Original patch proposed by Daria doesn't cover that case, but if the > patch going to be improved, probably i

Re: Assert in pageinspect with NULL pages

2022-03-15 Thread Michael Paquier
On Tue, Mar 15, 2022 at 06:56:46AM -0500, Justin Pryzby wrote: > On Tue, Mar 15, 2022 at 06:32:44PM +0900, Michael Paquier wrote: >> +-- Suppress the DETAIL message, to allow the tests to work across various >> +-- default page sizes. > > I think you mean "various non-default page sizes" or "vario

Re: Assert in pageinspect with NULL pages

2022-03-15 Thread Justin Pryzby
On Tue, Mar 15, 2022 at 06:32:44PM +0900, Michael Paquier wrote: > + if (!IS_BRIN(indexRel)) > + ereport(ERROR, > + (errcode(ERRCODE_WRONG_OBJECT_TYPE), > + errmsg("\"%s\" is not a %s index", > +

Re: Assert in pageinspect with NULL pages

2022-03-15 Thread Michael Paquier
On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: > BRIN can also crash if passed a non-brin index. > > I've been sitting on this one for awhile. Feel free to include it in your > patchset. So, I have begun a close lookup of this thread, and the problem you are reporting here is wor

Re: Assert in pageinspect with NULL pages

2022-02-22 Thread Daria Lepikhova
18.02.2022 08:00, Justin Pryzby пишет: BRIN can also crash if passed a non-brin index. I've been sitting on this one for awhile. Feel free to include it in your patchset. commit 08010a6037fc4e24a9ba05e5386e766f4310d35e Author: Justin Pryzby Date: Tue Jan 19 00:25:15 2021 -0600 pagei

Re: Assert in pageinspect with NULL pages

2022-02-22 Thread Daria Lepikhova
First of all, thanks for the review and remarks! 18.02.2022 08:02, Michael Paquier пишет: On Thu, Feb 17, 2022 at 05:40:41PM +0800, Julien Rouhaud wrote: About the patch, it's incorrectly using a hardcoded 8192 block-size rather than using the computed :block_size (or computing one when it's no

Re: Assert in pageinspect with NULL pages

2022-02-21 Thread Michael Paquier
On Mon, Feb 21, 2022 at 10:00:00AM +0300, Alexander Lakhin wrote: > Could you please confirm before committing the patchset that it fixes > the bug #16527 [1]? Or maybe I could check it? > (Original patch proposed by Daria doesn't cover that case, but if the > patch going to be improved, probably i

Re: Assert in pageinspect with NULL pages

2022-02-20 Thread Alexander Lakhin
Hello Michael, 18.02.2022 06:07, Michael Paquier wrpte: > On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: >> BRIN can also crash if passed a non-brin index. >> >> I've been sitting on this one for awhile. Feel free to include it in your >> patchset. > Ugh. Thanks! I am keeping a

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: > BRIN can also crash if passed a non-brin index. > > I've been sitting on this one for awhile. Feel free to include it in your > patchset. Ugh. Thanks! I am keeping a note about this one. -- Michael signature.asc Description: PG

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 05:40:41PM +0800, Julien Rouhaud wrote: > About the patch, it's incorrectly using a hardcoded 8192 block-size rather > than > using the computed :block_size (or computing one when it's not already the > case). Well, the tests of pageinspect fail would already fail when usi

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Justin Pryzby
BRIN can also crash if passed a non-brin index. I've been sitting on this one for awhile. Feel free to include it in your patchset. commit 08010a6037fc4e24a9ba05e5386e766f4310d35e Author: Justin Pryzby Date: Tue Jan 19 00:25:15 2021 -0600 pageinspect: brin_page_items(): check that given

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Julien Rouhaud
On Thu, Feb 17, 2022 at 05:57:49PM +0900, Michael Paquier wrote: > On Thu, Feb 17, 2022 at 01:46:40PM +0500, Daria Lepikhova wrote: > > INSERT INTO test1(y) SELECT 0 FROM generate_series(1,1E6) AS x; > > SELECT page_checksum(repeat(E'\\000', 8192)::bytea, 1); > > Indeed. Good catch, and that seem

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 01:46:40PM +0500, Daria Lepikhova wrote: > INSERT INTO test1(y) SELECT 0 FROM generate_series(1,1E6) AS x; > SELECT page_checksum(repeat(E'\\000', 8192)::bytea, 1); Indeed. Good catch, and that seems pretty old at quick glance for the checksum part. I'll try to look at al

Assert in pageinspect with NULL pages

2022-02-17 Thread Daria Lepikhova
Hi, hackers! If we trying to call pageinspect functions for pages which are filled with nulls, we will get core dump. It happens with null pages for all indexes in pageinspect and for page_checksum. This problem was founded firstly by Anastasia Lubennikova, and now I continue this task. For