Re: Good News Everyone! + feature proposal
On 10/5/23 8:53 AM, Tom Lane wrote: > Laurenz Albe writes: >> On Thu, 2023-10-05 at 02:22 +0000, Jon Erdman wrote: >>> For the proposal (this one is a bit Apple specific): because my team >>> offers managed postgres to our Apple-internal customers, many of whom >>> are not database experts, or at least not postgres experts, we'd like to >>> be able to toggle the availability of UNLOGGED tables in >>> postgresql.conf, so our less clueful users have fewer footguns. > > I'm doubtful that this is a problem that needs a solution. > If anything, the right answer is to fix whatever part of the > documentation isn't warning of the hazards strongly enough. > > Even more to the point: if we accept this, how many other > footgun-preventing GUCs will have the same or stronger claim to > existence? > >> It certainly sounds harmless, but there are two things that make me >> unhappy about this: > >> - Yet another GUC. It's not like we don't have enough of them. >>(This is a small quibble.) > >> - This setting would influence the way SQL is processed. >>We have had bad experiences with those; an often-quoted example is >>the "autocommit" parameter that got removed in 7.4. >>This certainly is less harmfuls, but still another pitfall that >>can confuse users. > > Same objections here. Also note that the semantics we've defined > for GUCs (when they can be set and where) don't always line up > nicely with requirements of this sort. It's far from clear to me > whether such a GUC should be SUSET (making it a hard prohibition > for ordinary users) or USERSET (making it just a training wheel). Someone on linked-in suggested an event trigger, so now I'm thinking of a custom extension that would do nothing but create said event trigger, and maybe could be toggled with a customized setting (but that might allow users to turn it off themselves...which is maybe ok). If the extension were installed by the DBA user, the customer wouldn't be able to drop it, and if we decided to give them an exception, we just drop or disable the extension. As a second more general question: could my original idea (i.e. sans event trigger) be implemented in an extension somehow, or is that not technically possible (I suspect not)? -- Jon Erdman (aka StuckMojo on IRC) PostgreSQL Zealot
Re: [HACKERS] Logical replication and multimaster
Oops, Looking at my old message from 2015 on this subject, it was Magnus who pish-poshed it, and Page who liked it. Don't want to throw shaded at the wrong person ;) -- Jon Erdman (aka StuckMojo) PostgreSQL Zealot On 10/5/23 12:14 AM, Jon Erdman wrote: > > Well, > > Given my earlier message about Apple wanting to pay me for writing > patches now, maybe I can revisit this idea. > > For background: I brought up the idea of an FDW that could read custom > dump files and expose them as tables so that you could grab just a > single record (or of course more) from a custom dump file without having > to load the whole thing up, if you were stuck reaching into a backup to > get at accidentally deleted tables, rows, etc.. The stopper, which was > communicated to me by Tom at the following pgcon was that the code for > parsing custom dumps is duplicated in pg_dump only, and would need to be > duplicated into the server for the FDW, or broken out into a library. > > And for posterity, Dave Page said that was a stupid idea, while Magnus > said that it sounded useful. And IIRC Bruce and Robert H said it was > doable, just a good deal of work on the refactor needed. > > This convo went down at the Amsterdam conf where I spoke about using > writeable LVM snapshots to expose each developer a copy of prod to > noodle on, without having to actually make a full copy for each dev. > > Added trivia: I gave the talk with a can of Heineken in my hand at the > podium, and my lightning talk had the work F( Cool in the title ;) > > That was also when I bought my plush Slony which was supposedly the very > last one. (turns out they made more) > -- > Jon Erdman (aka StuckMojo) > PostgreSQL Zealot > > On 12/15/15 9:48 PM, Jim Nasby wrote: >> On 12/13/15 7:37 AM, David Fetter wrote: >>> As I understand it, pushing these into a library has been proposed but >>> not rejected. That it hasn't happened yet is mostly about the lack of >>> tuits (the round ones) to rewrite the functionality as libraries and >>> refactor pg_dump/pg_restore to use only calls to same. As usual, it's >>> less about writing the code and more about the enormous amount of >>> testing any such a refactor would entail. >> >> My understanding as well. IIRC Jon Erdman brought this question up a >> couple years ago and the response was "It'd probably be accepted, it's >> just that no one has done the work." >> >>> I believe that refactoring much of pg_dump's functionality for the >>> current version of the server into SQL-accessible functions and making >>> pg_dump use only those functions is achievable with available >>> resources. >>> >>> Such a refactor need not be all-or-nothing. For example, the >>> dependency resolution stuff is a first step that appears to be worth >>> doing by itself even if the effort then pauses, possibly for some >>> time. >> >> If someone wanted to spend time on this, I suspect it'd be worth >> looking at how bad some of the backward compatibility issues would be >> if done in the server. Maybe they wouldn't be that bad. I suspect the >> audience for this code would be much larger if it was in the server as >> opposed to a C library.
Re: [HACKERS] Logical replication and multimaster
Well, Given my earlier message about Apple wanting to pay me for writing patches now, maybe I can revisit this idea. For background: I brought up the idea of an FDW that could read custom dump files and expose them as tables so that you could grab just a single record (or of course more) from a custom dump file without having to load the whole thing up, if you were stuck reaching into a backup to get at accidentally deleted tables, rows, etc.. The stopper, which was communicated to me by Tom at the following pgcon was that the code for parsing custom dumps is duplicated in pg_dump only, and would need to be duplicated into the server for the FDW, or broken out into a library. And for posterity, Dave Page said that was a stupid idea, while Magnus said that it sounded useful. And IIRC Bruce and Robert H said it was doable, just a good deal of work on the refactor needed. This convo went down at the Amsterdam conf where I spoke about using writeable LVM snapshots to expose each developer a copy of prod to noodle on, without having to actually make a full copy for each dev. Added trivia: I gave the talk with a can of Heineken in my hand at the podium, and my lightning talk had the work F( Cool in the title ;) That was also when I bought my plush Slony which was supposedly the very last one. (turns out they made more) -- Jon Erdman (aka StuckMojo) PostgreSQL Zealot On 12/15/15 9:48 PM, Jim Nasby wrote: > On 12/13/15 7:37 AM, David Fetter wrote: >> As I understand it, pushing these into a library has been proposed but >> not rejected. That it hasn't happened yet is mostly about the lack of >> tuits (the round ones) to rewrite the functionality as libraries and >> refactor pg_dump/pg_restore to use only calls to same. As usual, it's >> less about writing the code and more about the enormous amount of >> testing any such a refactor would entail. > > My understanding as well. IIRC Jon Erdman brought this question up a > couple years ago and the response was "It'd probably be accepted, it's > just that no one has done the work." > >> I believe that refactoring much of pg_dump's functionality for the >> current version of the server into SQL-accessible functions and making >> pg_dump use only those functions is achievable with available >> resources. >> >> Such a refactor need not be all-or-nothing. For example, the >> dependency resolution stuff is a first step that appears to be worth >> doing by itself even if the effort then pauses, possibly for some >> time. > > If someone wanted to spend time on this, I suspect it'd be worth looking > at how bad some of the backward compatibility issues would be if done in > the server. Maybe they wouldn't be that bad. I suspect the audience for > this code would be much larger if it was in the server as opposed to a C > library.
Good News Everyone! + feature proposal
Hiya Hackers! So I have some good news! At long last I've found a company/manager that wants to actually factually pay me to do some work on PG!! Had my performance review today, and Apple wants me to get a patch accepted this quarter, with the promise of more to come after that. Luckily, this first patch can be anything (doesn't have to be of use to Apple, more to prove that I can get a patch accepted), so I'm open to suggestions of smaller stuff that is in demand at the moment. For the proposal (this one is a bit Apple specific): because my team offers managed postgres to our Apple-internal customers, many of whom are not database experts, or at least not postgres experts, we'd like to be able to toggle the availability of UNLOGGED tables in postgresql.conf, so our less clueful users have fewer footguns. So, my proposal is for a GUC to implement that, which would *OF COURSE* undefault to allowing UNLOGGED. The reasoning here is we have autofailover set up for our standard cluster offering that we give to customers, using sync-rep to guarantee no data loss if we flop to the HA node. Any users not up to speed on what UNLOGGED really means could inadvertently lose data, so we'd like to be able to force it to be off, and turn it on upon request after educating the customer in question it's it's a valid use case. So to begin with: I'm sure some folks will hate this idea, but maybe a good many wont, and remember, this would default to UNLOGGED enabled, so no change to current behaviour. and no encouragement to disallow it, but just the ability to do so, which i think is useful in hosted/managed/multi-tenant environment where most things are taken care of for the customer. So I'd like to get a general idea how likely this would be to getting accepted if it did it, and did it right? Let the flame war begin! PS: I'm SO happy that this phase of my postgres journey has finally started -- Jon Erdman (aka StuckMojo on IRC) PostgreSQL Zealot
