Re: Proposal to use JSON for Postgres Parser format

2023-12-04 Thread Matthias van de Meent
On Mon, 31 Oct 2022 at 15:56, Michel Pelletier wrote: > On Mon, Oct 31, 2022 at 6:15 AM Matthias van de Meent > wrote: >> On Mon, 31 Oct 2022 at 13:46, Alexander Korotkov >> wrote: >>> On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan wrote: On 2022-10-27 Th 19:38, Andres Freund wrote:

Re: Proposal to use JSON for Postgres Parser format

2023-10-09 Thread Thomas Munro
On Tue, Sep 20, 2022 at 4:16 PM Thomas Munro wrote: > To explain my earlier guess: reader code for #S(STRUCTNAME ...) can > bee seen here, though it's being lexed as "PLAN_SYM" so perhaps the > author of that C already didn't know that was a general syntax for > Lisp structs. (Example: at a Lisp

Re: Proposal to use JSON for Postgres Parser format

2022-10-31 Thread Michel Pelletier
On Mon, Oct 31, 2022 at 6:15 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Mon, 31 Oct 2022 at 13:46, Alexander Korotkov > wrote: > > On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan > wrote: > >> On 2022-10-27 Th 19:38, Andres Freund wrote: > >> > Hi, > >> > > >> > On

Re: Proposal to use JSON for Postgres Parser format

2022-10-31 Thread Matthias van de Meent
On Mon, 31 Oct 2022 at 13:46, Alexander Korotkov wrote: > On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan wrote: >> On 2022-10-27 Th 19:38, Andres Freund wrote: >> > Hi, >> > >> > On 2022-09-19 22:29:15 -0400, Tom Lane wrote: >> >> Maybe a compromise could be found whereby we provide a conversion

Re: Proposal to use JSON for Postgres Parser format

2022-10-31 Thread Alexander Korotkov
On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan wrote: > On 2022-10-27 Th 19:38, Andres Freund wrote: > > Hi, > > > > On 2022-09-19 22:29:15 -0400, Tom Lane wrote: > >> Maybe a compromise could be found whereby we provide a conversion function > >> that converts whatever the catalog storage format

Re: Proposal to use JSON for Postgres Parser format

2022-10-28 Thread Andrew Dunstan
On 2022-10-27 Th 19:38, Andres Freund wrote: > Hi, > > On 2022-09-19 22:29:15 -0400, Tom Lane wrote: >> Maybe a compromise could be found whereby we provide a conversion function >> that converts whatever the catalog storage format is to some JSON >> equivalent. That would address the needs of

Re: Proposal to use JSON for Postgres Parser format

2022-10-27 Thread Andres Freund
Hi, On 2022-09-19 22:29:15 -0400, Tom Lane wrote: > There are certainly reasons to think about changing the node tree > storage format; but if we change it, I'd like to see it go to something > more compact not more verbose. Very much seconded - the various pg_node_trees are a quite significant

Re: Proposal to use JSON for Postgres Parser format

2022-10-27 Thread Michel Pelletier
On Wed, Sep 21, 2022 at 11:04 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Tue, 20 Sept 2022 at 17:29, Alvaro Herrera > wrote: > > > > On 2022-Sep-20, Matthias van de Meent wrote: > > > > > Allow me to add: compressability > > > > > > In the thread surrounding [0] there

Re: Proposal to use JSON for Postgres Parser format

2022-09-21 Thread Matthias van de Meent
On Tue, 20 Sept 2022 at 17:29, Alvaro Herrera wrote: > > On 2022-Sep-20, Matthias van de Meent wrote: > > > Allow me to add: compressability > > > > In the thread surrounding [0] there were complaints about the size of > > catalogs, and specifically the template database. Significant parts of > >

Re: Proposal to use JSON for Postgres Parser format

2022-09-20 Thread Alvaro Herrera
On 2022-Sep-20, Matthias van de Meent wrote: > Allow me to add: compressability > > In the thread surrounding [0] there were complaints about the size of > catalogs, and specifically the template database. Significant parts of > that (688kB of 8080kB a fresh PG14 database) are in pg_rewrite,

Re: Proposal to use JSON for Postgres Parser format

2022-09-20 Thread Matthias van de Meent
On Tue, 20 Sept 2022 at 12:00, Alexander Korotkov wrote: > On Tue, Sep 20, 2022 at 7:48 AM Tom Lane wrote: > > Peter Geoghegan writes: > > > On Mon, Sep 19, 2022 at 8:39 PM Tom Lane wrote: > > >> Our existing format is certainly not great on those metrics, but > > >> I do not see how "let's

Re: Proposal to use JSON for Postgres Parser format

2022-09-20 Thread Alexander Korotkov
On Tue, Sep 20, 2022 at 1:00 PM Alexander Korotkov wrote: > On Tue, Sep 20, 2022 at 7:48 AM Tom Lane wrote: > > Peter Geoghegan writes: > > > On Mon, Sep 19, 2022 at 8:39 PM Tom Lane wrote: > > >> Our existing format is certainly not great on those metrics, but > > >> I do not see how "let's

Re: Proposal to use JSON for Postgres Parser format

2022-09-20 Thread Alexander Korotkov
On Tue, Sep 20, 2022 at 7:48 AM Tom Lane wrote: > Peter Geoghegan writes: > > On Mon, Sep 19, 2022 at 8:39 PM Tom Lane wrote: > >> Our existing format is certainly not great on those metrics, but > >> I do not see how "let's use JSON!" is a route to improvement. > > > The existing format was

Re: Proposal to use JSON for Postgres Parser format

2022-09-20 Thread Thomas Munro
On Tue, Sep 20, 2022 at 4:58 PM Peter Geoghegan wrote: > On Mon, Sep 19, 2022 at 9:48 PM Tom Lane wrote: > > As Munro adduces nearby, it'd be a stretch to conclude that the current > > format was designed with any Postgres-related goals in mind at all. > > I think he's right that it's a variant

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Peter Geoghegan
On Mon, Sep 19, 2022 at 9:48 PM Tom Lane wrote: > As Munro adduces nearby, it'd be a stretch to conclude that the current > format was designed with any Postgres-related goals in mind at all. > I think he's right that it's a variant of some Lisp-y dump format that's > probably far hoarier than

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Sep 19, 2022 at 8:39 PM Tom Lane wrote: >> Our existing format is certainly not great on those metrics, but >> I do not see how "let's use JSON!" is a route to improvement. > The existing format was designed with developer convenience as a goal, > though --

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Thomas Munro
On Tue, Sep 20, 2022 at 4:03 PM Thomas Munro wrote: > On Tue, Sep 20, 2022 at 3:58 PM Tom Lane wrote: > > Thomas Munro writes: > > > FWIW, it derives from Lisp s-expressions, but deviates from Lisp's > > > default reader/printer behaviour in small ways, including being case > > > sensitive and

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Thomas Munro
On Tue, Sep 20, 2022 at 3:58 PM Tom Lane wrote: > Thomas Munro writes: > > FWIW, it derives from Lisp s-expressions, but deviates from Lisp's > > default reader/printer behaviour in small ways, including being case > > sensitive and using {NAME :x 1 ...} instead of #S(NAME :x 1 ...) for > >

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Peter Geoghegan
On Mon, Sep 19, 2022 at 8:58 PM Tom Lane wrote: > Wow, where did you find a commit history for Berkeley's code? > There's evidence in the tarballs I have that they were using > RCS, but I never heard that the repo was made public. It's on Github: https://github.com/kelvich/postgres_pre95 --

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Peter Geoghegan
On Mon, Sep 19, 2022 at 8:39 PM Tom Lane wrote: > There are certainly use-cases for something like that, but let's > be clear about it: that's a niche case of interest to developers > and pretty much nobody else. For ordinary users, what matters about > the node tree storage format is

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Tom Lane
Thomas Munro writes: > FWIW, it derives from Lisp s-expressions, but deviates from Lisp's > default reader/printer behaviour in small ways, including being case > sensitive and using {NAME :x 1 ...} instead of #S(NAME :x 1 ...) for > structs for reasons that are lost AFAIK (there's a dark age

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Amit Kapila
On Tue, Sep 20, 2022 at 7:59 AM Tom Lane wrote: > > Michel Pelletier writes: > > I would like to propose a discussion that in a future major release > > Postgres switch from this custom format to JSON. > > There are certainly reasons to think about changing the node tree > storage format; but if

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Tom Lane
Peter Geoghegan writes: > Writing declarative @> containment queries against (say) a JSON > variant of node tree format seems like it could be a huge quality of > life improvement. There are certainly use-cases for something like that, but let's be clear about it: that's a niche case of interest

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Thomas Munro
On Tue, Sep 20, 2022 at 12:16 PM Michel Pelletier wrote: > This non-standard format FWIW, it derives from Lisp s-expressions, but deviates from Lisp's default reader/printer behaviour in small ways, including being case sensitive and using {NAME :x 1 ...} instead of #S(NAME :x 1 ...) for structs

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Peter Geoghegan
On Mon, Sep 19, 2022 at 7:29 PM Tom Lane wrote: > Maybe a compromise could be found whereby we provide a conversion > function that converts whatever the catalog storage format is to > some JSON equivalent. That would address the needs of external > code that doesn't want to write a custom

Re: Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Tom Lane
Michel Pelletier writes: > I would like to propose a discussion that in a future major release > Postgres switch from this custom format to JSON. There are certainly reasons to think about changing the node tree storage format; but if we change it, I'd like to see it go to something more compact

Proposal to use JSON for Postgres Parser format

2022-09-19 Thread Michel Pelletier
Hello hackers, As noted in the source: https://github.com/postgres/postgres/blob/master/src/include/nodes/pg_list.h#L6-L11 * Once upon a time, parts of Postgres were written in Lisp and used real * cons-cell lists for major data structures. When that code was rewritten * in C, we initially