Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Greg Sabino Mullane
On Fri, May 17, 2024 at 7:11 AM Tomas Vondra wrote: > It does seem to me a part of the solution needs to be helping to get > those patches reviewed. I don't know how to do that, but perhaps there's > a way to encourage people to review more stuff, or review stuff from a > wider range of

Re: Logging which interface was connected to in log_line_prefix

2024-05-01 Thread Greg Sabino Mullane
Thank you for taking the time to review this. I've attached a new rebased version, which has no significant changes. > There is a comment in the patch that states: > > /* We do not need clean_ipv6_addr here: just report verbatim */ > > I am not quite sure what it means, but I am guessing it

Re: Tarball builds in the new world order

2024-04-24 Thread Greg Sabino Mullane
On Tue, Apr 23, 2024 at 6:06 PM Tom Lane wrote: > This change seems like a good thing anyway for anyone who's tempted > to use "make dist" manually, since they wouldn't necessarily want > to package HEAD either. Now, if we just do it exactly like that > then trying to "make dist" without

Re: psql: Greatly speed up "\d tablename" when not using regexes

2024-04-10 Thread Greg Sabino Mullane
Patch looks good to me. Great idea overall, that forced regex has always bugged me. + char *regexChars = "|*+?()[]{}.^$\\"; One super minor optimization is that we technically do not need to scan for ')' and ']'. If they appear without their partner, the query will fail anyway. :)

Re: Security lessons from liblzma

2024-04-04 Thread Greg Sabino Mullane
> > It would be better if we created the required test files as part of the > test run. (Why not? Too slow?) Alternatively, I have been thinking > that maybe we could make the output more reproducible by messing with > whatever random seed OpenSSL uses. Or maybe use a Python library to >

Re: Reports on obsolete Postgres versions

2024-04-04 Thread Greg Sabino Mullane
On Thu, Apr 4, 2024 at 2:23 PM Bruce Momjian wrote: > -end-of-life (EOL) and no longer supported. > +after its initial release. After this, a final minor version will be > released > +and the software will then be unsupported (end-of-life). Would be a shame to lose the EOL acronym. +Such

Re: On disable_cost

2024-04-03 Thread Greg Sabino Mullane
On Wed, Apr 3, 2024 at 3:21 PM Robert Haas wrote: > It's also pretty clear to me that the fact that enable_indexscan > and enable_indexonlyscan work completely differently from each other > is surprising at best, wrong at worst, but here again, what this patch > does about that is not above

Re: On disable_cost

2024-04-02 Thread Greg Sabino Mullane
On Mon, Apr 1, 2024 at 7:54 PM Robert Haas wrote: > What I think we're mostly doing in the regression tests is shutting > off every relevant type of plan except one. I theorize that what we > actually want to do is tell the planner what we do want to happen, > rather than what we don't want to

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Greg Sabino Mullane
> > The purpose of the setting is to prevent accidental > modifications via ALTER SYSTEM in environments where The emphasis on 'accidental' seems a bit heavy here, and odd. Surely, just "to prevent modifications via ALTER SYSTEM in environments where..." is enough? Cheers, Greg

Re: Adding comments to help understand psql hidden queries

2024-03-22 Thread Greg Sabino Mullane
On Fri, Mar 22, 2024 at 11:39 AM David Christensen wrote: > I think it's easier to keep the widths balanced than constant (patch > version included here) Yeah, I'm fine with that, especially because nobody is translating it, nor are they likely to, to be honest. Cheers, Greg

Re: Adding comments to help understand psql hidden queries

2024-03-22 Thread Greg Sabino Mullane
On Thu, Mar 21, 2024 at 6:20 PM Peter Eisentraut wrote: > lines are supposed to align vertically. With your patch, the first line > would have variable length depending on the command. > Yes, that is a good point. Aligning those would be quite tricky, what if we just kept a standard width for

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-20 Thread Greg Sabino Mullane
My mistake. Attached please find version 3, which should hopefully make cfbot happy again. pgbench.dash.d.or.not.dash.d.v3.patch Description: Binary data

Re: Possibility to disable `ALTER SYSTEM`

2024-03-20 Thread Greg Sabino Mullane
> > As a bonus, if that GUC is set, we could even check at server startup that > all the configuration files are not writable by the postgres user, > and print a warning or refuse to start up if they are. > Ugh, please let's not do this. This was bouncing around in my head last night, and this is

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-19 Thread Greg Sabino Mullane
Rebased version attached (v2), with another sentence in the sgml to explain the optional use of -d Cheers, Greg pgbench.dash.d.or.not.dash.d.v2.patch Description: Binary data

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Greg Sabino Mullane
On Tue, Mar 19, 2024 at 12:05 PM Tom Lane wrote: > If you aren't willing to build a solution that blocks off mods > using COPY TO FILE/PROGRAM and other readily-available-to-superusers > tools (plpythonu for instance), I think you shouldn't bother asking > for a feature at all. Just trust your

Re: Possibility to disable `ALTER SYSTEM`

2024-03-18 Thread Greg Sabino Mullane
Going to agree with Robert Treat here about an extension being a great solution. I resisted posting earlier as I wanted to see how this all pans out, but I wrote a quick little POC extension some months ago that does the disabling and works well (and cannot be easily worked around). On Mon, Mar

Re: Reports on obsolete Postgres versions

2024-03-11 Thread Greg Sabino Mullane
On Mon, Mar 11, 2024 at 4:38 PM Bruce Momjian wrote: > https://www.postgresql.org/support/versioning/ > > This web page should correct the idea that "upgrades are more risky than > staying with existing versions". Is there more we can do? Should we have > a more consistent response for

Logging which interface was connected to in log_line_prefix

2024-03-06 Thread Greg Sabino Mullane
Someone on -general was asking about this, as they are listening on multiple IPs and would like to know which exact one clients were hitting. I took a quick look and we already have that information, so I grabbed some stuff from inet_server_addr and added it as part of a "%L" (for 'local

Re: Reducing the log spam

2024-03-06 Thread Greg Sabino Mullane
On Tue, Mar 5, 2024 at 7:55 AM Laurenz Albe wrote: > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "xy" Forcibly hiding those at the Postgres level seems a heavy hammer for what

Avoiding inadvertent debugging mode for pgbench

2024-02-29 Thread Greg Sabino Mullane
Attached please find a patch to adjust the behavior of the pgbench program and make it behave like the other programs that connect to a database (namely, psql and pg_dump). Specifically, add support for using -d and --dbname to specify the name of the database. This means that -d can no longer be

Re: An improved README experience for PostgreSQL

2024-02-28 Thread Greg Sabino Mullane
+1 on the general idea. Maybe make that COPYRIGHT link go to an absolute URI, like all the other links, in case this file gets copied somewhere? Perhaps point it to https://www.postgresql.org/about/licence/ Cheers, Greg

Re: What about Perl autodie?

2024-02-08 Thread Greg Sabino Mullane
> > 2. Don't wait, migrate them all now. This would mean requiring > Perl 5.10.1 or later to run the TAP tests, even in back branches. > #2 please. For context, meson did not even exist in 2009. Cheers, Greg

Re: What about Perl autodie?

2024-02-07 Thread Greg Sabino Mullane
On Wed, Feb 7, 2024 at 9:05 AM Peter Eisentraut wrote: > I came across the Perl autodie pragma > (https://perldoc.perl.org/autodie). This seems pretty useful; is this > something we can use? Any drawbacks? Any minimum Perl version? Big +1 No drawbacks. I've been using it heavily for many,

Adding comments to help understand psql hidden queries

2023-12-11 Thread Greg Sabino Mullane
The use of the --echo-hidden flag in psql is used to show people the way psql performs its magic for its backslash commands. None of them has more magic than "\d relation", but it suffers from needing a lot of separate queries to gather all of the information it needs. Unfortunately, those queries

Adding ordering to list of available extensions

2023-10-30 Thread Greg Sabino Mullane
Please find attached a patch to provide some basic ordering to the system views pg_available_extensions and pg_available_extension_versions. It is sorely tempting to add ORDER BYs to many of the other views in that file, but I understand that would be contentious as there are reasons for not

Re: [PATCH] Add inline comments to the pg_hba_file_rules view

2023-09-26 Thread Greg Sabino Mullane
Also a reluctant -1, as the comment-at-EOL style is very rare in my experience over the years of seeing many a pg_hba file.

Re: Make --help output fit within 80 columns per line

2023-09-18 Thread Greg Sabino Mullane
On Fri, Sep 15, 2023 at 11:11 AM torikoshia wrote: > I do not intend to adhere to this rule(my terminals are usually bigger > than 80 chars per line), but wouldn't it be a not bad direction to use > 80 characters for all commands? > Well, that's the question du jour, isn't it? The 80 character

Re: Make --help output fit within 80 columns per line

2023-09-13 Thread Greg Sabino Mullane
On Tue, Jul 4, 2023 at 9:47 PM torikoshia wrote: > Since it seems preferable to have consistent line break policy and some > people use 80-column terminal, wouldn't it be better to make all commands > in 80 > columns per line? > All this seems an awful lot of work to support this mythical

Re: Possibility to disable `ALTER SYSTEM`

2023-09-13 Thread Greg Sabino Mullane
Seems to be some resistance to getting this in core, so why not just use an extension? I was able to create a quick POC to do just that. Hook into PG and look for AlterSystemStmt, throw a "Sorry, ALTER SYSTEM is not currently allowed" error. Put into shared_preload_libraries and you're done. As a

Re: Prevent psql \watch from running queries that return no rows

2023-08-22 Thread Greg Sabino Mullane
Thank you for the feedback, everyone. Attached is version 4 of the patch, featuring a few tests and minor rewordings. Cheers, Greg psql_watch_exit_on_zero_rows_v4.patch Description: Binary data

Re: Improve pg_stat_statements by making jumble handle savepoint names better

2023-07-25 Thread Greg Sabino Mullane
On Mon, Jul 24, 2023 at 6:46 PM Michael Paquier wrote: > Shouldn't this new field be marked as query_jumble_location > Yes, it should. I had some trouble getting it to work that way in the first place, but now I realize it was just my unfamiliarity with this part of the code. So thanks for the

Improve pg_stat_statements by making jumble handle savepoint names better

2023-07-24 Thread Greg Sabino Mullane
Please find attached a patch to jumble savepoint name, to prevent certain transactional commands from filling up pg_stat_statements. This has been a problem with some busy systems that use django, which likes to wrap everything in uniquely named savepoints. Soon, over 50% of your

Re: Forgive trailing semicolons inside of config files

2023-07-11 Thread Greg Sabino Mullane
On Tue, Jul 11, 2023 at 11:04 AM Isaac Morland wrote: > Please, no! > > There is no end to accepting sloppy syntax. What next, allow "SET > random_page_cost = 2.5;" (with or without semicolon) in config files? > Well yes, there is an end. A single, trailing semicolon. Full stop. It's not a

Forgive trailing semicolons inside of config files

2023-07-11 Thread Greg Sabino Mullane
This has been a long-standing annoyance of mine. Who hasn't done something like this?: psql> SET random_page_cost = 2.5; (do some stuff, realize that rpc was too high) Let's put that inside of postgresql.conf: #-- #

Re: Prevent psql \watch from running queries that return no rows

2023-07-05 Thread Greg Sabino Mullane
Thanks for the feedback! On Wed, Jul 5, 2023 at 5:51 AM Daniel Gustafsson wrote: > > The comment on ExecQueryAndProcessResults() needs to be updated with an > explanation of what this parameter is. > I added a comment in the place where min_rows is used, but not sure what you mean by adding it

Re: Bytea PL/Perl transform

2023-06-22 Thread Greg Sabino Mullane
> > So I decided to propose a simple transform extension to pass bytea as > native Perl octet strings. Quick review, mostly housekeeping things: * Needs a rebase, minor failure on Mkvcbuild.pm * Code needs standardized formatting, esp. bytea_plperl.c * Needs to be meson-i-fied (i.e. add a

Re: Bypassing shared_buffers

2023-06-17 Thread Greg Sabino Mullane
On Thu, Jun 15, 2023 at 4:16 AM Vladimir Churyukin wrote: > We're trying to see what is the worst performance in terms of I/O, i.e. >> when the database just started up or the data/indexes being queried are not >> cached at all. > > You could create new tables that are copies of the existing

Re: Let's make PostgreSQL multi-threaded

2023-06-08 Thread Greg Sabino Mullane
On Thu, Jun 8, 2023 at 8:44 AM Hannu Krosing wrote: > Do we have any statistics for the distribution of our user base ? > > My gut feeling says that for performance-critical use the non-Linux is > in low single digits at best. > Stats are probably not possible, but based on years of consulting,

Re: Prevent psql \watch from running queries that return no rows

2023-06-04 Thread Greg Sabino Mullane
On Sat, Jun 3, 2023 at 5:58 PM Michael Paquier wrote: > > Wouldn't something like a target_rows be more flexible? You could use > this parameter with a target number of rows to expect, zero being one > choice in that. > Thank you! That does feel better to me. Please see attached a new v2 patch

Prevent psql \watch from running queries that return no rows

2023-06-02 Thread Greg Sabino Mullane
Attached is a patch to allow a new behavior for the \watch command in psql. When enabled, this instructs \watch to stop running once the query returns zero rows. The use case is the scenario in which you are watching the output of something long-running such as pg_stat_progress_create_index, but

Re: PostgreSQL 14 press release draft

2021-09-22 Thread Greg Sabino Mullane
Some super quick nitpicks; feel free to ignore/apply/laugh off. ... administrators to deploy their data-backed applications. PostgreSQL continues to add innovations on complex data types, including more conveniences for accessing JSON and support for noncontiguous ranges of data. This latest

Re: Add option --drop-cascade for pg_dump/restore

2021-08-12 Thread Greg Sabino Mullane
On Wed, Aug 11, 2021 at 10:53 PM Wu Haotian wrote: > Maybe we can add checks like "option --clean requires plain text format"? > If so, should I start a new mail thread for this? > Shrug. To me, that seems related enough it could go into the existing patch/thread. Cheers, Greg

Re: make MaxBackends available in _PG_init

2021-08-12 Thread Greg Sabino Mullane
On Wed, Aug 11, 2021 at 10:08 AM Tom Lane wrote: > You must not have enabled EXEC_BACKEND properly. It's a compile-time > #define that affects multiple modules, so it's easy to get wrong. > The way I usually turn it on is > Thank you. I was able to get it all working, and withdraw any

Re: make MaxBackends available in _PG_init

2021-08-11 Thread Greg Sabino Mullane
On Mon, Aug 9, 2021 at 8:22 PM Bossart, Nathan wrote: > > Is this going to get tripped by a call from restore_backend_variables? > > I ran 'make check-world' with EXEC_BACKEND with no problems, so I > don't think so. > v3 looks good, but I'm still not sure how to test the bit mentioned above.

Re: Add option --drop-cascade for pg_dump/restore

2021-08-10 Thread Greg Sabino Mullane
On Fri, Jul 16, 2021 at 9:40 AM Tom Lane wrote: > That would require pg_restore to try to edit the DROP commands during > restore, which sounds horribly fragile. I'm inclined to think that > supporting this option only during initial dump is safer. > Safer, but not nearly as useful. Maybe see

Re: Avoid stuck of pbgench due to skipped transactions

2021-08-10 Thread Greg Sabino Mullane
Apologies, just saw this. I found no problems, those "failures" were just me missing checkboxes on the commitfest interface. +1 on the patch. Cheers, Greg

Re: make MaxBackends available in _PG_init

2021-08-09 Thread Greg Sabino Mullane
On Sat, Aug 7, 2021 at 2:01 PM Bossart, Nathan wrote: > Here is a rebased version of the patch. > Giving this a review. Patch applies cleanly and `make check` works as of e12694523e7e4482a052236f12d3d8b58be9a22c Overall looks very nice and tucks MaxBackends safely away. I have a few

Mention --enable-tap-tests in the TAP section page

2021-07-01 Thread Greg Sabino Mullane
Quick patch to add mention of the need for compiling with --enable-tap-tests on the TAP section page https://www.postgresql.org/docs/current/regress-tap.html Searching about the TAP tests often leads to this page, but there is no easy link or mention of the fact that the sample invocations will

Re: Speed up pg_checksums in cases where checksum already set

2021-06-29 Thread Greg Sabino Mullane
On Tue, Jun 29, 2021 at 2:59 AM Michael Paquier wrote: > Does that look fine to you? > Looks great, I appreciate the renaming. Cheers, Greg

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-06-22 Thread Greg Sabino Mullane
Those code comments look good.

Re: Avoid stuck of pbgench due to skipped transactions

2021-06-22 Thread Greg Sabino Mullane
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested Looks fine to me, as a way of catching this edge case.

Re: Speed up pg_checksums in cases where checksum already set

2021-06-18 Thread Greg Sabino Mullane
On Fri, Jun 18, 2021 at 1:57 AM Michael Paquier wrote: > This doc addition is a bit confusing, as it could mean that each file > has just one single checksum. We could be more precise, say: > "When enabling checksums, each relation file block with a changed > checksum is rewritten in place." >

Re: Update maintenance_work_mem/autovacuum_work_mem to reflect the 1GB limitation with VACUUM

2021-06-17 Thread Greg Sabino Mullane
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Latest patch looks fine to me, to be clear. The new status of this patch

Re: Update maintenance_work_mem/autovacuum_work_mem to reflect the 1GB limitation with VACUUM

2021-06-04 Thread Greg Sabino Mullane
Should we say "currently has"?

Re: Speed up pg_checksums in cases where checksum already set

2021-06-02 Thread Greg Sabino Mullane
Newer version attach that adds a small documentation tweak as well. Cheers, Greg pg_checksums.optimize.writes.and.always.sync.patch Description: Binary data

Re: Speed up pg_checksums in cases where checksum already set

2021-06-02 Thread Greg Sabino Mullane
Fair enough; thanks for the feedback. Attached is a new version that does an unconditional sync (well, unless do_sync is false, a flag I am not particularly fond of). Cheers, Greg pg_checksums.optimize.writes.always.sync.patch Description: Binary data

Re: Update maintenance_work_mem/autovacuum_work_mem to reflect the 1GB limitation with VACUUM

2021-06-01 Thread Greg Sabino Mullane
s/Node/Note/ Other than that, +1 to the patch and +1 to backpatching. The new status of this patch is: Waiting on Author

Re: Add option --drop-cascade for pg_dump/restore

2021-05-28 Thread Greg Sabino Mullane
Overall the patch looks good, but I did notice a few small things: 1. In pg_dumpall.c, the section /* Add long options to the pg_dump argument list */, we are now passing along the --drop-cascade option. However, --clean is not passed in, so any call to pg_dumpall using --drop-cascade fails a

Re: Reduce lock level for ALTER TABLE ... ADD CHECK .. NOT VALID

2021-05-28 Thread Greg Sabino Mullane
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Looks fine to me The new status of this patch is: Ready for

Re: Speed up pg_checksums in cases where checksum already set

2021-05-27 Thread Greg Sabino Mullane
Thanks for the quick replies, everyone. On Wed, May 26, 2021 at 10:17 PM Michael Paquier wrote: > > -if (do_sync) > +if (do_sync && total_files_modified) > Here, I am on the edge. It could be an advantage to force a flush of > the data folder anyway, no? I was originally on the fence

Speed up pg_checksums in cases where checksum already set

2021-05-26 Thread Greg Sabino Mullane
The attached patch makes an optimization to pg_checksums which prevents rewriting the block if the checksum is already what we expect. This can lead to much faster runs in cases where it is already set (e.g. enabled -> disabled -> enable, external helper process, interrupted runs, future parallel

Re: psql \df choose functions by their arguments

2021-04-07 Thread Greg Sabino Mullane
I like the wildcard aspect, but I have a few issues with the patch: * It doesn't respect some common abbreviations that work elsewhere (e.g. CREATE FUNCTION). So while "int4" works, "int" does not. Nor does "float", which thus requires the mandatory-double-quoted "double precision" * Adding

Re: psql \df choose functions by their arguments

2021-01-19 Thread Greg Sabino Mullane
Ha ha ha, my bad, I am not sure why I left those out. Here is a new patch with int2, int4, and int8. Thanks for the email. Cheers, Greg v6-psql-df-pick-function-by-type.patch Description: Binary data

Re: psql \df choose functions by their arguments

2021-01-14 Thread Greg Sabino Mullane
Thanks for the feedback: new version v5 (attached) has int8, plus the suggested code formatting. Cheers, Greg v5-psql-df-pick-function-by-type.patch Description: Binary data

Re: psql \df choose functions by their arguments

2021-01-06 Thread Greg Sabino Mullane
On Sat, Jan 2, 2021 at 1:56 AM Thomas Munro wrote: > ... > It looks like there is a collation dependency here that causes the > test to fail on some systems: > Thanks for pointing that out. I tweaked the function definitions to hopefully sidestep the ordering issue - attached is v4. Cheers,

Re: psql \df choose functions by their arguments

2020-12-30 Thread Greg Sabino Mullane
Attached is the latest patch against HEAD - basically fixes a few typos. Cheers, Greg v3-psql-df-pick-function-by-type.patch Description: Binary data

Re: Prefer TG_TABLE_NAME over TG_RELNAME in tests

2020-11-11 Thread Greg Sabino Mullane
Thanks so much: the Bucardo PR has been merged in.

Re: psql \df choose functions by their arguments

2020-11-03 Thread Greg Sabino Mullane
On Sun, Nov 1, 2020 at 12:05 PM David Christensen wrote: > > I can’t speak for the specific patch, but tab completion of proc args for > \df, \ef and friends has long been a desired feature of mine, particularly > when you are dealing with functions with huge numbers of arguments and the > same

Re: psql \df choose functions by their arguments

2020-11-03 Thread Greg Sabino Mullane
Thanks for looking this over! > some Abbreviations of common types are not added to the > type_abbreviations[] Such as: > > Int8=> bigint > I wasn't aiming to provide a canonical list, as I personally have never seen anyone use int8 instead of bigint when (for example) creating

Re: psql \df choose functions by their arguments

2020-11-01 Thread Greg Sabino Mullane
Thanks for the feedback, attached is version two of the patch. Major changes: * Use booleans not generic "int x" * Build a quick list of abbreviations at the top of the function * Add array mapping for all types * Removed the tab-complete bit, it was too fragile and unhelpful Cheers, Greg diff

Re: psql \df choose functions by their arguments

2020-10-29 Thread Greg Sabino Mullane
Thank you for looking this over. > This isn't working for arrays: > ... > postgres=# \df aa aa int[] > Arrays should work as expected, I think you have one too many "aa" in there? > I think it should use the same syntax as \sf and \ef, which require > parenthesis > and commas, not spaces. >

psql \df choose functions by their arguments

2020-10-15 Thread Greg Sabino Mullane
nargs to the current number of valid tokens, thereby forcing a match to one (or zero) functions. 6ab7a45d541f2c31c5631b811f14081bf7b22271 v1-psql-df-pick-function-by-type.patch - -- Greg Sabino Mullane PGP Key: 0x14964AC8 202010151316 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964A

WIP psql \df choose functions by their arguments

2020-10-14 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Improve psql \df to choose functions by their arguments == OVERVIEW Having to scroll through same-named functions with different argument types when you know exactly which one you want is annoying at best, error causing at worst. This patch