Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-27 Thread Michael Paquier
On Tue, Oct 26, 2021 at 02:43:26PM +0900, Michael Paquier wrote: > Indeed. There was also a problem in the regex itself, where '|' was > not escaped so the regex was not strict enough. While on it, I have > added a policy in the set copied to the new database. Testing the > case where the set

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-26 Thread Daniel Gustafsson
> On 23 Oct 2021, at 00:47, Michael Paquier wrote: > > On Fri, Oct 22, 2021 at 10:49:38PM +0200, Daniel Gustafsson wrote: >> On 22 Oct 2021, at 20:34, Tom Lane wrote: >>> Ah, you're right, InsertPgAttributeTuples is the only other spot in >>> that patch that's actually touching slots. I'd

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-25 Thread Michael Paquier
On Mon, Oct 25, 2021 at 11:59:52AM -0400, Tom Lane wrote: > I agree that we're not testing that area well enough. Proposed > patch seems basically OK, but I think the test needs to be stricter > about what the expected output looks like --- for instance, it > wouldn't complain if tab_foobar were

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-25 Thread Tom Lane
Michael Paquier writes: > I was thinking on this one over the last couple of days, and doing a > copy of shared deps from a template within createdb still feels > natural, as of this patch: > https://www.postgresql.org/message-id/yxdtl+pfsnqmb...@paquier.xyz > Would somebody object to the

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-25 Thread Michael Paquier
On Thu, Oct 21, 2021 at 11:42:31AM +0900, Michael Paquier wrote: > It is easy enough to get an error on the new database with > pg_describe_object(). Your part about adding a shared dependency with > a table on a given role is simple enough, as well. While looking for > a place where to put such

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Michael Paquier
On Fri, Oct 22, 2021 at 10:49:38PM +0200, Daniel Gustafsson wrote: > On 22 Oct 2021, at 20:34, Tom Lane wrote: >> Ah, you're right, InsertPgAttributeTuples is the only other spot in >> that patch that's actually touching slots. I'd skimmed it a little >> too quickly. > > Thanks for confirming,

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Daniel Gustafsson
> On 22 Oct 2021, at 20:34, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 22 Oct 2021, at 15:38, Tom Lane wrote: >>> (By "here" I mean all of e3931d0, because it made the same omission >>> in several places.) > >> The attached fixes the the two ones I spotted, are there any I missed? >

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Tom Lane
Daniel Gustafsson writes: >> On 22 Oct 2021, at 15:38, Tom Lane wrote: >> (By "here" I mean all of e3931d0, because it made the same omission >> in several places.) > The attached fixes the the two ones I spotted, are there any I missed? Ah, you're right, InsertPgAttributeTuples is the only

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Daniel Gustafsson
> On 22 Oct 2021, at 15:38, Tom Lane wrote: > > Michael Paquier writes: >> On Fri, Oct 22, 2021 at 10:48:57AM +0300, Aleksander Alekseev wrote: >>> ... where the slot is allocated with palloc0. The assumption that >>> MakeSingleTupleTableSlot() returns valid TupleTableSlot* with zeroed >>>

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Aleksander Alekseev
Hi Michael, > Do you have something in mind here? Yep. This is not a priority though, thus I created a separate CF entry: https://commitfest.postgresql.org/35/3371/ -- Best regards, Aleksander Alekseev

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Tom Lane
Michael Paquier writes: > On Fri, Oct 22, 2021 at 10:48:57AM +0300, Aleksander Alekseev wrote: >> ... where the slot is allocated with palloc0. The assumption that >> MakeSingleTupleTableSlot() returns valid TupleTableSlot* with zeroed >> tts_isnull[] seems reasonable, no? > Yes, I don't see any

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Michael Paquier
On Fri, Oct 22, 2021 at 10:48:57AM +0300, Aleksander Alekseev wrote: > ... where the slot is allocated with palloc0. The assumption that > MakeSingleTupleTableSlot() returns valid TupleTableSlot* with zeroed > tts_isnull[] seems reasonable, no? Yes, I don't see any need to do something more here.

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-22 Thread Aleksander Alekseev
Hi Tom, > BTW, I think there is an additional bug in copyTemplateDependencies: > I do not see it initializing slot->tts_isnull[] anywhere. It > probably accidentally works (at least in devel builds) because we zero > that memory somewhere else, but surely this code shouldn't assume that?

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-21 Thread David G. Johnston
On Thu, Oct 21, 2021 at 8:52 AM Tom Lane wrote: > We're fortunate > that cloning a nonempty template database is rare already. > > That, and a major use case for doing so is to quickly stage up testing data in a new database (i.e., not a production use case). Though I could see tenant-based

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-21 Thread Tom Lane
Alvaro Herrera writes: > I suppose pg_describe_object can be used on the contents of pg_shdepend > to detect it. I'm less sure what to do to correct it -- delete the > bogus entries and regenerate them with some bulk query? Seems that what copyTemplateDependencies wants to do can easily be

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-21 Thread Alvaro Herrera
On 2021-Oct-21, Michael Paquier wrote: > On Wed, Oct 20, 2021 at 09:19:51AM -0300, Alvaro Herrera wrote: > > Ouch ... this means that pg_shdepends contents are broken for databases > > created with 14.0? hmm ... yes. > > Yes, it means so :( For the upcoming release notes in 14.1 I think we'd

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Peter Geoghegan
On Wed, Oct 20, 2021 at 8:27 PM Michael Paquier wrote: > Perhaps. This means the creation of a new database with shared deps > in contrib/amcheck/t/. But is amcheck really a correct target here? > The fields involved here are an int, some OIDs and a char with a given > subset of values making

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Michael Paquier
On Wed, Oct 20, 2021 at 07:59:50PM -0700, Peter Geoghegan wrote: > I think that EDB's pg_catcheck tool can detect problems like this one. Yes, pg_catcheck is able to catch that. > Perhaps it can be converted into an amcheck/pg_amcheck patch, and > submitted. That would give us very broad

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Peter Geoghegan
On Wed, Oct 20, 2021 at 5:20 AM Alvaro Herrera wrote: > Ouch ... this means that pg_shdepends contents are broken for databases > created with 14.0? hmm ... yes. I think that EDB's pg_catcheck tool can detect problems like this one. Perhaps it can be converted into an amcheck/pg_amcheck patch,

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Michael Paquier
On Wed, Oct 20, 2021 at 09:19:51AM -0300, Alvaro Herrera wrote: > Ouch ... this means that pg_shdepends contents are broken for databases > created with 14.0? hmm ... yes. Yes, it means so :( I have fixed the issue for now, and monitored the rest of the tree. Another issue is that we have zero

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Alvaro Herrera
On 2021-Oct-20, Michael Paquier wrote: > On Wed, Oct 20, 2021 at 01:01:31PM +0300, Aleksander Alekseev wrote: > > I checked the rest of the PostgreSQL code and apparently, it should > > have been tts_values[Anum_pg_shdepend_FOO - 1]. > > > > The patch is attached. The problem was first reported

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Daniel Gustafsson
> On 20 Oct 2021, at 12:55, Michael Paquier wrote: > > On Wed, Oct 20, 2021 at 01:01:31PM +0300, Aleksander Alekseev wrote: >> I checked the rest of the PostgreSQL code and apparently, it should >> have been tts_values[Anum_pg_shdepend_FOO - 1]. >> >> The patch is attached. The problem was

Re: [PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Michael Paquier
On Wed, Oct 20, 2021 at 01:01:31PM +0300, Aleksander Alekseev wrote: > I checked the rest of the PostgreSQL code and apparently, it should > have been tts_values[Anum_pg_shdepend_FOO - 1]. > > The patch is attached. The problem was first reported offlist by Sven > Klemm. Investigated and fixed by

[PATCH] Fix memory corruption in pg_shdepend.c

2021-10-20 Thread Aleksander Alekseev
Hi hackers, One of our test runs under the memory sanitizer cathed [1] the following stacktrace: ``` heaptuple.c:1044:13: runtime error: load of value 111, which is not a valid value for type '_Bool' #0 0x55fbb5e0857b in heap_form_tuple