> On May 6, 2018, at 12:08 PM, John Naylor wrote:
>
> On 5/7/18, Mark Dilger wrote:
>> Hackers,
>>
>> Have you already considered and rejected the idea of having
>> genbki.pl/Catalog.pm define constants that can be used in
>> the catalog .dat
On 5/7/18, Mark Dilger wrote:
> Hackers,
>
> Have you already considered and rejected the idea of having
> genbki.pl/Catalog.pm define constants that can be used in
> the catalog .dat files? I'm mostly curious if people think
> the resulting .dat files are better or
Mark Dilger writes:
> Have you already considered and rejected the idea of having
> genbki.pl/Catalog.pm define constants that can be used in
> the catalog .dat files? I'm mostly curious if people think
> the resulting .dat files are better or worse using constants
> of
John Naylor writes:
> On 4/26/18, Tom Lane wrote:
>> (The prosrc-from-proname case, in isolation, could be handled about
>> as well by adding a hardwired rule like we have for pronargs.)
> If we think we might go in the direction of more special-case
>
On 4/26/18, Tom Lane wrote:
> (The prosrc-from-proname case, in isolation, could be handled about
> as well by adding a hardwired rule like we have for pronargs.)
If we think we might go in the direction of more special-case
behavior, the attached patch more fully documents
On Wed, Apr 25, 2018 at 04:39:42PM -0400, Tom Lane wrote:
> Mark Dilger writes:
> >> On Apr 25, 2018, at 1:00 PM, Robert Haas wrote:
> >> -1 for trying to automate this. It varies between fooin and foo_in,
> >> and it'll be annoying to have to
On Thu, Apr 26, 2018 at 2:11 AM, Mark Dilger wrote:
>
>> On Apr 25, 2018, at 1:31 PM, Tom Lane wrote:
>>
>> Robert Haas writes:
>>> On Wed, Apr 25, 2018 at 3:44 PM, Mark Dilger
>>> wrote:
There
>>
+ textprosrc BKI_DEFAULT("${proname}")
BKI_FORCE_NOT_NULL;
>>
>>> That one I kinda like.
>>
>> Agreed, this seems more compelling. However, I think we need more than
>> one compelling example to justify the additional infrastructure.
>
> I'll look for more.
> On Apr 25, 2018, at 1:31 PM, Tom Lane wrote:
>
> Robert Haas writes:
>> On Wed, Apr 25, 2018 at 3:44 PM, Mark Dilger wrote:
>>> There still seems to be a lot of boilerplate in the .dat files
>>> that could be eliminated.
Mark Dilger writes:
>> On Apr 25, 2018, at 1:00 PM, Robert Haas wrote:
>> -1 for trying to automate this. It varies between fooin and foo_in,
>> and it'll be annoying to have to remember which one happens
>> automatically and which one needs an
Robert Haas writes:
> On Wed, Apr 25, 2018 at 3:44 PM, Mark Dilger wrote:
>> There still seems to be a lot of boilerplate in the .dat files
>> that could be eliminated. ...
>> {... typname => 'X', ... typinput => 'Xin', typoutput => 'Xout',
>>
> On Apr 25, 2018, at 1:00 PM, Robert Haas wrote:
>
> On Wed, Apr 25, 2018 at 3:44 PM, Mark Dilger wrote:
>> There still seems to be a lot of boilerplate in the .dat files
>> that could be eliminated. Tom mentioned upthread that he did
>> not
On Wed, Apr 25, 2018 at 3:44 PM, Mark Dilger wrote:
> There still seems to be a lot of boilerplate in the .dat files
> that could be eliminated. Tom mentioned upthread that he did
> not want too much magic in genbki.pl or Catalog.pm, but I think
> I can propose putting
John Naylor writes:
> On 4/6/18, Tom Lane wrote:
>> Some of the CATALOG lines spill well past 80 characters with this,
>> although many of the affected ones already were overlength, eg ...
> Thinking about this some more, a way occurred to me to shorten
On 4/6/18, Tom Lane wrote:
> I experimented with converting all frontend code to include just the
> catalog/pg_foo_d.h files instead of catalog/pg_foo.h, as per the
> proposed new policy. I soon found that we'd overlooked one thing:
> some clients expect to see the relation
On April 8, 2018 10:19:59 AM PDT, Tom Lane wrote:
>John Naylor writes:
>> There were a couple more catalog changes that broke patch context, so
>> attached is version 16.
>
>Pushed with a few more adjustments (mostly, another round of
>copy-editing
>on
On 4/9/18, Tom Lane wrote:
> Congratulations, and many thanks, to John for seeing this through!
> I know it's been a huge amount of work, but we've needed this for years.
Many thanks for your review, advice, and additional hacking. Thanks
also to Álvaro for review and initial
John Naylor writes:
> There were a couple more catalog changes that broke patch context, so
> attached is version 16.
Pushed with a few more adjustments (mostly, another round of copy-editing
on bki.sgml). Now we wait to see what the buildfarm thinks, particularly
about the
I wrote:
> I'll check back in 24 hours to see if everything still applies.
There were a couple more catalog changes that broke patch context, so
attached is version 16.
-John Naylor
v16-bootstrap-data-conversion.tar.gz
Description: GNU Zip compressed data
Tom Lane wrote:
> Alvaro Herrera writes:
> > John Naylor wrote:
> >> Commit 9fdb675fc added a symbol to pg_opfamily.h
> >> where there were none before, so I went ahead and wrapped it with an
> >> EXPOSE_TO_CLIENT_CODE macro.
>
> > Actually, after pushing that, I was
Alvaro Herrera writes:
> John Naylor wrote:
>> Commit 9fdb675fc added a symbol to pg_opfamily.h
>> where there were none before, so I went ahead and wrapped it with an
>> EXPOSE_TO_CLIENT_CODE macro.
> Actually, after pushing that, I was thinking maybe it's better to
John Naylor wrote:
> On 4/6/18, Tom Lane wrote:
>
> > I don't think there's any great need to incorporate this into your patch
> > set. As far as I'm concerned, v14 is ready as-is, and I'll just apply
> > this over the top of it. (Note that I'll probably smash the whole
On 4/6/18, Tom Lane wrote:
> I don't think there's any great need to incorporate this into your patch
> set. As far as I'm concerned, v14 is ready as-is, and I'll just apply
> this over the top of it. (Note that I'll probably smash the whole thing
> to one commit when the
Oh, one more thing: looking again at the contents of pg_proc.dat,
I find myself annoyed at the need to specify pronargs. That's
entirely derivable from proargtypes, and if we did so, we'd get
down to this for the first few pg_proc entries:
{ oid => '1242', descr => 'I/O',
proname => 'boolin',
Attached is a round of minor review fixes. Much of this is cleaning
up existing mistakes or out-of-date comments, rather than anything
introduced by this patchset, but I noticed it while going through the
patch.
The additional EXPOSE_TO_CLIENT_CODE bits actually are necessary,
in some cases, as
I wrote:
> John Naylor writes:
>> On 4/6/18, Tom Lane wrote:
>>> BTW, I experimented with adding blank lines between the hash items in the
>>> .dat files, and that seemed to make a nice improvement in readability,
> Anyway, as I said, I'm not set on this
John Naylor writes:
> On 4/6/18, Tom Lane wrote:
>> BTW, I experimented with adding blank lines between the hash items in the
>> .dat files, and that seemed to make a nice improvement in readability,
>> converting masses of rather gray text into visibly
For version 14, diffed against f1464c53804:
-Use majority values for proisstrict, provolatile, proparallel (patch 0006)
-Use valid C string for multi-char defaults containing a backslash (patch 0006)
-Apply Tom's patch for additional lookups, slightly modified by me
(convert_oid2name.pl, patch
On 4/6/18, Tom Lane wrote:
> BTW, I experimented with adding blank lines between the hash items in the
> .dat files, and that seemed to make a nice improvement in readability,
> converting masses of rather gray text into visibly distinct stanzas.
> I'm not dead set on that,
On 4/6/18, Tom Lane wrote:
> Just had another thought about this business: if practical, we should
> remove the distinction between "descr" and "shdescr" and just use the
> former name in .dat files. genbki.pl knows which catalogs are shared,
> so it ought to be able to
Just had another thought about this business: if practical, we should
remove the distinction between "descr" and "shdescr" and just use the
former name in .dat files. genbki.pl knows which catalogs are shared,
so it ought to be able to figure out where to route the descriptions.
John Naylor writes:
> Looking at convert_oid2name.patch again, I see this:
> + elsif ($catname eq 'pg_am')
> + {
> + $values{aggfnoid} =
> lookup_procname($values{aggfnoid});
> + }
>
John Naylor writes:
> It seems most of the time the FooRelationId labels are predictable,
> although not as pristine as the Anum_* constants. One possibility that
> came to mind is to treat these like pg_type OID #defines -- have a
> simple rule that can be overridden for
On 4/5/18, Tom Lane wrote:
> I've generalized the BKI_LOOKUP(pg_proc) code so that
> you can use either regproc-like or regprocedure-like notation, and then
> applied that to relevant columns.
> [...]
> bootstrap-v13-delta.patch is a diff atop your patch series for the
>
On 4/6/18, Tom Lane wrote:
> I experimented with converting all frontend code to include just the
> catalog/pg_foo_d.h files instead of catalog/pg_foo.h, as per the
> proposed new policy. I soon found that we'd overlooked one thing:
> some clients expect to see the relation
BTW, I experimented with adding blank lines between the hash items in the
.dat files, and that seemed to make a nice improvement in readability,
converting masses of rather gray text into visibly distinct stanzas.
I'm not dead set on that, but try it and see what you think.
A small example from
I experimented with converting all frontend code to include just the
catalog/pg_foo_d.h files instead of catalog/pg_foo.h, as per the
proposed new policy. I soon found that we'd overlooked one thing:
some clients expect to see the relation OID macros, eg
LargeObjectRelationId. Attached is a
John Naylor writes:
> On 4/5/18, Tom Lane wrote:
>> I did not like the hard-wired handling of proargtypes and proallargtypes
>> in genbki.pl; it hardly seems impossible that we'll want similar
>> conversions for other array-of-OID columns in future. After
On 4/5/18, Tom Lane wrote:
> Here are the results of an evening's desultory hacking on v13.
>
> [numeric function oids with overloaded name]
Thank you for the detailed review and for improving the function
references (not to mention the type references I somehow left on the
Here are the results of an evening's desultory hacking on v13.
I was dissatisfied with the fact that we still had several
function-referencing columns that had numeric instead of symbolic
contents, for instance pg_aggregate.aggfnoid. Of course, the main reason
is that those are declared regproc
Hi,
On 2018-04-04 18:29:31 -0400, Tom Lane wrote:
> I'm starting to look through v13 seriously, and one thing struck
> me that could use some general discussion: what is our policy
> going to be for choosing the default values for catalog columns?
>
> [...]
>
> In short, I'm tempted to say that
I'm starting to look through v13 seriously, and one thing struck
me that could use some general discussion: what is our policy
going to be for choosing the default values for catalog columns?
In particular, I noticed that you have for pg_proc
boolproisstrict BKI_DEFAULT(f);
John Naylor writes:
> And in the department of second thoughts, it occurred to me that the
> only reason that the .dat files are in include/catalog is because
> that's where the DATA() statements were. Since they are separate now,
> one could make the case that they actually
Attached is v13, rebased against b0c90c85fc.
Patch 0001:
-The data files are formatted to at most 80 columns wide.
-Rename rewrite_dat.pl to reformat_dat_file.pl.
-Refactor Catalog.pm and reformat_dat_file.pl to have better
separation of concerns.
-Add src/include/catalog/Makefile with
Tom Lane wrote:
>> -Maybe document examples of how to do bulk-editing of data files?
>
> +1. In the end, that's the reason we're doing all this work, so showing
> people how to benefit seems like a good thing.
I'll hold off on posting a new patchset until I add this to the
documentation, but I
> On Mar 26, 2018, at 10:44 PM, Tom Lane wrote
> Layout of .dat files seems generally reasonable, but I don't understand
> the proposed make rule:
>
> +reformat-dat-files:
> +$(PERL) -I $(catalogdir) $< catalog/rewrite_dat.pl -o catalog
> catalog/pg_*.dat
>
> This
John Naylor writes:
> With the attachments this time.
Layout of .dat files seems generally reasonable, but I don't understand
the proposed make rule:
+reformat-dat-files:
+ $(PERL) -I $(catalogdir) $< catalog/rewrite_dat.pl -o catalog
catalog/pg_*.dat
This rule has
On 3/25/18, Tom Lane wrote:
> Well, as I said, I hadn't really reviewed the .dat files, but if that's
> what you're doing I'm going to request a change. Project style is to
> fit in 80 columns as much as possible. I do not see a reason to exempt
> the .dat files from that,
John Naylor writes:
> -I shortened the data example in the README so it would comfortably
> fit on two lines. Spreading it out over three lines doesn't match
> what's in the data files. It's valid syntax, but real data is
> formatted to at most two lines (See rewrite_dat.pl.
On 3/22/18, Tom Lane wrote:
> I don't really think it's legal C; I'd rather write BKI_DEFAULT('\054').
Okay, I'll do that.
>> Were you thinking of this comment in
>> pg_attribute.h? We use the double-quoted empty string for postgres.bki
>> and change it to '\0' for
On 3/22/18, Alvaro Herrera wrote:
> how about letting the line go long, with the comment at the right of
> each definition, with one blank line between struct members, as in the
> sample below? You normally don't care that these lines are too long
> since you seldom edit
John Naylor writes:
> On 3/22/18, Tom Lane wrote:
>> While I'm thinking of it --- I noticed one or two places where you
>> had "BKI_DEFAULT(\0)".
> Hmm, I only see this octal in pg_type.h
> chartypdelim BKI_DEFAULT(\054);
Sorry, I was going by
John Naylor wrote:
> I've attached an earlier version of pg_proc.h with both formats as I
> understand them. I turned a couple comments into multi-line comments
> to demonstrate. I think without spaces it's just as hard to read as
> with multiple annotations. I'd vote for spaces, but then again
On 3/22/18, Tom Lane wrote:
> Looking at this again, I think a big chunk of the readability problem here
> is just from the fact that we have long, similar-looking lines tightly
> packed. If it were reformatted to have comment lines and whitespace
> between, it might not look
I wrote:
> On 3/21/18, Tom Lane wrote:
>> I've got mixed feelings about the whitespace lines between fields. They
>> seem like they are mostly bulking up the code and we could do without
>> 'em.
>> On the other hand, pgindent will insist on putting one before any
>>
John Naylor writes:
> On 3/21/18, Tom Lane wrote:
>> regproc aggmtransfn BKI_DEFAULT(-) BKI_LOOKUP(pg_proc);
>> regproc aggminvtransfn BKI_DEFAULT(-) BKI_LOOKUP(pg_proc);
>> regproc aggmfinalfn BKI_DEFAULT(-) BKI_LOOKUP(pg_proc);
On 3/21/18, Tom Lane wrote:
> John Naylor writes:
>> [ v11-bootstrap-data-conversion.tar.gz ]
>
> I've done a round of review work on this, focusing on the Makefile
> infrastructure. I found a bunch of problems with parallel builds and
> VPATH builds,
John Naylor writes:
> [ v11-bootstrap-data-conversion.tar.gz ]
I've done a round of review work on this, focusing on the Makefile
infrastructure. I found a bunch of problems with parallel builds and
VPATH builds, which are addressed in the attached incremental patch.
The
On 3/15/18, Tom Lane wrote:
> John Naylor writes:
>> It didn't take that long to rebase the remaining parts of the
>> patchset, so despite what I said above I went ahead and put them in
>> version 10 (attached), this time via scripted bulk editing rather
John Naylor writes:
> It didn't take that long to rebase the remaining parts of the
> patchset, so despite what I said above I went ahead and put them in
> version 10 (attached), this time via scripted bulk editing rather than
> as large patches.
Starting to look into this
John Naylor writes:
> On 3/3/18, Tom Lane wrote:
>> * In 0006, I'm not very pleased with the introduction of
>> "Makefile.headers".
> I wasn't happy with it either, but I couldn't get it to build
> otherwise. The sticking point was the symlinks in
>
John Naylor writes:
> I've attached a patch that takes care of these cleanups so they don't
> clutter the patch set.
Pushed. I made a couple of cosmetic changes in genbki.pl.
regards, tom lane
John Naylor writes:
> On 3/3/18, Tom Lane wrote:
>> * I'm a little disturbed by the fact that 0002 has to "re-doublequote
>> values that are macros expanded by initdb.c". I see that there are only
>> a small number of affected places, so maybe it's not
I wrote:
> I'll submit a
> preliminary patch soon to get some of those items out of the way.
I've attached a patch that takes care of these cleanups so they don't
clutter the patch set.
-John Naylor
From d97a8b2e5fa4977e656d1aca7ee7bf1289ecbd40 Mon Sep 17 00:00:00 2001
From: John Naylor
Thanks for taking a look.
On 3/3/18, Tom Lane wrote:
> John Naylor writes:
>> Version 8, rebased against 76b6aa41f41d.
>
> I took a preliminary look through this, without yet attempting to execute
> the script against HEAD. I have a few thoughts:
>
> *
John Naylor writes:
> Version 8, rebased against 76b6aa41f41d.
I took a preliminary look through this, without yet attempting to execute
the script against HEAD. I have a few thoughts:
* I'm inclined not to commit the conversion scripts to the repo. I doubt
there are third
John Naylor writes:
> On 1/14/18, Tom Lane wrote:
>> So, for each catalog header pg_foo.h, there would be a
>> generated file, say pg_foo_d.h, containing:
>> * Natts_ and Anum_ macros for pg_foo
>> * Any EXPOSE_TO_CLIENT_CODE sections copied from pg_foo.h
Tom Lane wrote:
> I had a thought about how to do that. It's clearly desirable that that
> sort of material remain in the manually-maintained pg_*.h files, because
> that's basically where you look to find out C-level details of what's
> in a particular catalog. However, that doesn't mean that
On 1/14/18, Tom Lane wrote:
> I'm not sure about the decision to move all the OID macros into
> one file; that seems like namespace pollution.
> The design I'd kind of imagined was one generated file of #define's
> per pg_*.h file, not just one giant one.
First, thanks for
I wrote:
> Another thing that I'd sort of hoped might happen from this patchset
> is to cure the problem of keeping some catalog headers safe for
> client-side inclusion, because some clients want the OID value macros
> and/or macros for column values (eg PROVOLATILE_IMMUTABLE), so they
>
Greg Stark writes:
> I'm 1000% on board with replacing oid constants with symbolic names
> that get substituted programmatically.
Yeah, that's almost an independent feature --- we could do that without
any of this other stuff, if we wanted.
> However I wonder why we're bothering
I'm 1000% on board with replacing oid constants with symbolic names
that get substituted programmatically.
However I wonder why we're bothering inventing a new syntax that
doesn't actually do much more than present static tabular data. If
things like magic proname->prosrc behaviour are not
John Naylor writes:
> On 1/12/18, Alvaro Herrera wrote:
>> Pushed 0001 with some changes of my own. I'm afraid I created a few
>> conflicts for the later patches in your series; please rebase.
> Attached, rebased over 255f14183ac.
I decided that I
On 1/13/18, Tom Lane wrote:
> According to my understanding, part of what's going on here is that
> we're going to teach genbki.pl to parse these object references and
> convert them to hard-coded OIDs in the emitted BKI file. That seems
> good to me, but one thing we're
On 2018-01-12 18:36:14 -0300, Alvaro Herrera wrote:
> As above -- do we really like our current format so much that we're
> satisfied with minor tweaks?
A definite no from here. I think especially pg_proc desperately needs
something key=value like to be understandable, and that very clearly
Peter Eisentraut wrote:
> On 1/12/18 12:24, Alvaro Herrera wrote:
> > Here's a small sample pg_proc entry:
> >
> > { oid => '2147', descr => 'number of input rows for which the input
> > expression is not null',
> > n => 'count', proisagg => 't', v => 'i', p => 's', rt => 'int8', at =>
> >
On Fri, Jan 12, 2018 at 04:22:26PM -0500, Peter Eisentraut wrote:
> On 1/12/18 12:24, Alvaro Herrera wrote:
> > Here's a small sample pg_proc entry:
> >
> > { oid => '2147', descr => 'number of input rows for which the input
> > expression is not null',
> > n => 'count', proisagg => 't', v =>
On 1/12/18 12:24, Alvaro Herrera wrote:
> Here's a small sample pg_proc entry:
>
> { oid => '2147', descr => 'number of input rows for which the input
> expression is not null',
> n => 'count', proisagg => 't', v => 'i', p => 's', rt => 'int8', at =>
> 'any', s => 'aggregate_dummy' },
>
>
Tom, everyone,
It's getting late in my timezone, but I wanted to give a few quick
answers. I'll follow up tomorrow. Thanks Alvaro for committing my
refactoring of pg_attribute data creation. I think your modifications
are sensible and I'll rebase soon.
On 1/13/18, Tom Lane
Alvaro Herrera writes:
> Tom Lane wrote:
>> So could we have an explanation of what it is we're agreeing to?
> Here's a small sample pg_proc entry:
> { oid => '2147', descr => 'number of input rows for which the input
> expression is not null',
> n => 'count',
Tom Lane wrote:
> Alvaro Herrera writes:
> > Others: Now is the time to raise concerns related to the proposed file
> > formats and tooling, so please do have a look when you have a moment.
> > At this stage, the proposed data format seems a good choice to me.
>
> It's
On Fri, Jan 12, 2018 at 11:38:54AM -0500, Tom Lane wrote:
> Alvaro Herrera writes:
> > Others: Now is the time to raise concerns related to the proposed
> > file formats and tooling, so please do have a look when you have a
> > moment. At this stage, the proposed data
Alvaro Herrera writes:
> Others: Now is the time to raise concerns related to the proposed file
> formats and tooling, so please do have a look when you have a moment.
> At this stage, the proposed data format seems a good choice to me.
It's not very clear to me what the
Pushed 0001 with some changes of my own. I'm afraid I created a few
conflicts for the later patches in your series; please rebase.
I don't think we introduced anything that would fail on old Perls, but
let's see what buildfarm has to say.
Others: Now is the time to raise concerns related to the
Robert Haas wrote:
> On Thu, Dec 21, 2017 at 5:32 PM, Alvaro Herrera
> wrote:
> > Hmm, patch 0008 removes data lines from the .h but leaves the dependent
> > OID define lines around:
>
> Just a question here -- do we actually have consensus on doing the
> stuff that
On 12/22/17, Alvaro Herrera wrote:
> Hmm, patch 0008 removes data lines from the .h but leaves the dependent
> OID define lines around:
>
> #define BTREE_AM_OID 403
>
> This is not good, because then the define depends on data that appears
> in a distant file.
I see what
On Thu, Dec 21, 2017 at 5:32 PM, Alvaro Herrera wrote:
> Hmm, patch 0008 removes data lines from the .h but leaves the dependent
> OID define lines around:
Just a question here -- do we actually have consensus on doing the
stuff that these patches do? Because I'm not
Hmm, patch 0008 removes data lines from the .h but leaves the dependent
OID define lines around:
#define BTREE_AM_OID 403
This is not good, because then the define depends on data that appears
in a distant file. Another consideration is that the current system has
the property that these OIDs
Pushed 0001 and 0002.
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Dec 14, 2017 at 05:59:12PM +0700, John Naylor wrote:
> On 12/13/17, Peter Eisentraut wrote:
> > On 12/13/17 04:06, John Naylor wrote:
> >> There doesn't seem to be any interest in bootstrap data at the moment,
> >> but rather than give up just yet, I've
On 12/13/17 04:06, John Naylor wrote:
> There doesn't seem to be any interest in bootstrap data at the moment,
> but rather than give up just yet, I've added a couple features to make
> a data migration more compelling:
I took a brief look at your patches, and there appear to be a number of
good
91 matches
Mail list logo