Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 12:23 PM, shveta malik wrote: On Wed, Dec 6, 2023 at 4:28 PM shveta malik wrote: PFA v43, changes are: Thanks! v43-001: 1) Support of 'failover' dump in pg_dump. It was missing earlier. v43-002: 1) Slot-sync worker now checks validity of primary_slot_name by connecting

Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 11:58 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 3:00 PM Drouvot, Bertrand wrote: Hi, On 12/6/23 7:18 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: I feel that is indirectly relying on the fact that the primary won't advance logical slots

Re: Remove MSVC scripts from the tree

On 06.12.23 21:52, Andrew Dunstan wrote: On 2023-12-06 We 12:24, Peter Eisentraut wrote: On 06.12.23 17:27, Andrew Dunstan wrote: But the VS2019 compiler implements enough of C99 to pass our meson test, unlike VS2017. Maybe the test is too strict. After all, we know we can in fact build with

Proposal to add page headers to SLRU pages

Hi all, PostgreSQL currently maintains several data structures in the SLRU cache. The current SLRU pages do not have any header, so it is impossible to checksum a page and verify its integrity. It is very difficult to debug issues caused by corrupted SLRU pages. Also, without a page header,

Re: catalog access with reset GUCs during parallel worker startup

On Thu, 10 Feb 2022 at 13:33, Andres Freund wrote: > Postmaster's GUC state will already be loaded via read_nondefault_variables(), > much earlier in startup. Well before bgworkers get control and before > transaction environment is up. > > Setting a watchpoint on enableFsync, in a parallel

Re: Improve rowcount estimate for UNNEST(column)

Hello, On 11/26/23 12:22, Tom Lane wrote: > Yes, this regression test is entirely unacceptable; the numbers will > not be stable enough. Even aside from the different-settings issue, > you can't rely on ANALYZE deriving exactly the same stats every time. > Usually what we try to do is devise a

Re: catalog access with reset GUCs during parallel worker startup

On Wed, 23 Feb 2022 at 15:51, Andres Freund wrote: > The tests fail: > +ERROR: invalid value for parameter "session_authorization": "andres" > +CONTEXT: while setting parameter "session_authorization" to "andres" > +parallel worker > +while scanning relation "public.pvactst" > > but that's

Re: [PoC] pg_upgrade: allow to upgrade publisher node

On Wed, Dec 6, 2023 at 10:02 AM Amit Kapila wrote: > > On Wed, Dec 6, 2023 at 9:40 AM vignesh C wrote: > > > > On Tue, 5 Dec 2023 at 11:11, Hayato Kuroda (Fujitsu) > > wrote: > > > > > > Dear Sawada-san, hackers, > > > > > > Based on comments I made a fix. PSA the patch. > > > > > > > Thanks

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

Hi, > I mean to commit the open transaction at the below place in > wait_for_relation_state_change() > > wait_for_relation_state_change() > { > ... > -- commit the xact > WaitLatch(); > ... > } > > Then start after the wait is over. This is just to test whether it > improves the difference in

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

On Wed, Dec 06, 2023 at 05:40:44PM -0800, Andres Freund wrote: > This looks good to me! Cool. I've applied this one, then. -- Michael signature.asc Description: PGP signature

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 7:09 PM Tomas Vondra wrote: > > Yes, if something like this happens, that'd be a problem: > > 1) decoding starts, with > >SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT > > 2) transaction that creates a new refilenode gets decoded, but we skip >it because

Re: Make COPY format extendable: Extract COPY TO format implementations

Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 6 Dec 2023 22:07:51 +0800, Junwang Zhao wrote: > Should we extract both *copy to* and *copy from* for the first step, in that > case we can add the pg_copy_handler catalog smoothly later. I don't

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 7:17 PM Tomas Vondra wrote: > > On 12/6/23 12:05, Dilip Kumar wrote: > > On Wed, Dec 6, 2023 at 3:36 PM Amit Kapila wrote: > >> > >>> Why can't we use the same concept of > >>> SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the > >>> non-transactional

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 7:17 PM Tomas Vondra wrote: > > On 12/6/23 12:05, Dilip Kumar wrote: > > On Wed, Dec 6, 2023 at 3:36 PM Amit Kapila wrote: > >> > >>> Why can't we use the same concept of > >>> SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the > >>> non-transactional

RE: pg_upgrade and logical replication

On Thursday, December 7, 2023 10:23 AM Amit Kapila wrote: > > On Thu, Dec 7, 2023 at 7:26 AM Masahiko Sawada > wrote: > > > > On Tue, Dec 5, 2023 at 6:37 PM Amit Kapila > wrote: > > > > > > On Tue, Dec 5, 2023 at 10:56 AM Michael Paquier > wrote: > > > > > > > > On Mon, Dec 04, 2023 at

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 7:20 PM Tomas Vondra wrote: > > On 12/6/23 11:19, Amit Kapila wrote: > > On Sun, Dec 3, 2023 at 11:56 PM Tomas Vondra > > wrote: > >> > >> On 12/3/23 18:52, Tomas Vondra wrote: > >>> ... > >>> > >> > >> Another idea is that maybe we could somehow inform ReorderBuffer

Re: pg16 && GSSAPI && Heimdal/Macos

kov...@omniscient.com writes: > Earlier this year, there was a thread about GSSAPI for delegated > credentials and various operating systems ultimately that Heimdal had > atrophied enough that you were comfortable not supporting it anymore as > a GSSAPI library. Yup. > As you may have surmised,

Re: [PoC] Improve dead tuple storage for lazy vacuum

On Mon, Nov 27, 2023 at 1:45 PM Masahiko Sawada wrote: > > On Sat, Oct 28, 2023 at 5:56 PM John Naylor wrote: > bool > RT_SET(RT_RADIX_TREE *tree, uint64 key, RT_VALUE_TYPE *value_p); > or for variable-length value support, > RT_SET(RT_RADIX_TREE *tree, uint64 key, RT_VALUE_TYPE *value_p,

Re: Transaction timeout

On Wed, 06 Dec 2023 at 21:05, Andrey M. Borodin wrote: >> On 30 Nov 2023, at 20:06, Andrey M. Borodin wrote: >> >> >> Tomorrow I plan to fix raising of the timeout when the transaction is idle. >> Renaming transaction_timeout to something else (to avoid confusion with >> prepared xacts) also

Re: remaining sql/json patches

On Wed, Dec 6, 2023 at 10:02 PM Amit Langote wrote: > > Finally, I also fixed a couple of silly mistakes in 0003 around > transformJsonBehavior() and some further assorted tightening in the ON > ERROR / EMPTY expression coercion handling code. > typo: + * If a soft-error occurs, it will be

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 03:20:46PM -0500, Tom Lane wrote: > If Nathan's perf results hold up elsewhere, it seems like some > micro-optimization around the text-pushing (appendStringInfoString) > might be more useful than caching. The 7% spent in cache lookups > could be worth going after later,

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023, at 3:59 PM, Daniel Verite wrote: > The first Copy data message with contents "5b0a" does not qualify > as a row of data with 3 columns as advertised in the CopyOut > message. Isn't that a problem? > > At least the json non-ARRAY case ("json lines") doesn't have > this issue,

Re: Is WAL_DEBUG related code still relevant today?

On Wed, Dec 06, 2023 at 11:32:19PM -0300, Euler Taveira wrote: > IIUC trace_recovery_messages was a debugging aid in the 9.0 era when the HS > was > introduced. I'm also wondering if anyone used it in the past years. FWIW, I'd be +1 for getting rid of entirely, with its conditional block in

Re: Is WAL_DEBUG related code still relevant today?

On Wed, Dec 6, 2023, at 9:51 PM, Michael Paquier wrote: > PerformWalRecovery() with its log for RM_XACT_ID is something that > stresses me a bit though because this is in the main redo loop which > is never free. The same can be said about GenericXLogFinish() because > the extra computation

Re: pg_upgrade and logical replication

On Thu, Dec 7, 2023 at 7:26 AM Masahiko Sawada wrote: > > On Tue, Dec 5, 2023 at 6:37 PM Amit Kapila wrote: > > > > On Tue, Dec 5, 2023 at 10:56 AM Michael Paquier wrote: > > > > > > On Mon, Dec 04, 2023 at 04:30:49PM +0530, Amit Kapila wrote: > > > > I have made minor changes in the comments

[PATCH]: Not to invaldiate CatalogSnapshot for local invalidation messages

Hi hackers, For local invalidation messages, there is no need to call `InvalidateCatalogSnapshot` to set the CatalogSnapshot to NULL and rebuild it later. Instead, just update the CatalogSnapshot's `curcid` in `SnapshotSetCommandId`, this way can make the CatalogSnapshot work well too. This

Re: pg_upgrade and logical replication

On Tue, Dec 5, 2023 at 6:37 PM Amit Kapila wrote: > > On Tue, Dec 5, 2023 at 10:56 AM Michael Paquier wrote: > > > > On Mon, Dec 04, 2023 at 04:30:49PM +0530, Amit Kapila wrote: > > > I have made minor changes in the comments and code at various places. > > > See and let me know if you are not

Re: pg_upgrade and logical replication

On Mon, Dec 4, 2023 at 8:01 PM Amit Kapila wrote: > > On Fri, Dec 1, 2023 at 11:24 PM vignesh C wrote: > > > > The attached v22 version patch has the changes for the same. > > > > I have made minor changes in the comments and code at various places. > See and let me know if you are not happy

Re: gai_strerror() is not thread-safe on Windows

At Thu, 7 Dec 2023 09:43:37 +1300, Thomas Munro wrote in > On Tue, Dec 5, 2023 at 3:43 PM Kyotaro Horiguchi > wrote: > > Windows' gai_strerror outputs messages that correspond to the language > > environment. Similarly, I think that the messages that the messages > > returned by our version

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

Hi, This looks good to me! Greetings, Andres Freund

Re: Change GUC hashtable to use simplehash?

On Wed, Dec 6, 2023 at 11:48 PM Jeff Davis wrote: > > On Wed, 2023-12-06 at 07:39 +0700, John Naylor wrote: > > "git grep cstring_hash" found nothing, so not sure what you're > > asking. > > Sorry, I meant string_hash(). Your v5-0002 changes the way hashing > works for cstrings, and that means

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

On Wed, Dec 06, 2023 at 04:33:33PM -0800, Noah Misch wrote: > On Wed, Dec 06, 2023 at 03:17:12PM +0900, Michael Paquier wrote: >>> The "cannot" part of the message is also inaccurate, and it's not clear to >>> me >>> why we have this specific restriction at all. REINDEX INDEX CONCURRENTLY >>>

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 6:14 PM Joe Conway wrote: > > > But the point that we should introduce a 2 still stands. The new code > > would mean: use text output functions but that there is no inherent > > tabular structure in the underlying contents. Instead the copy format > > was JSON and the

Re: Emitting JSON to file using COPY TO

On 12/6/23 20:09, David G. Johnston wrote: On Wed, Dec 6, 2023 at 5:57 PM Joe Conway > wrote: On 12/6/23 19:39, David G. Johnston wrote: > On Wed, Dec 6, 2023 at 4:45 PM Joe Conway mailto:m...@joeconway.com> >

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:09, Joe Conway wrote: On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 5:57 PM Joe Conway wrote: > On 12/6/23 19:39, David G. Johnston wrote: > > On Wed, Dec 6, 2023 at 4:45 PM Joe Conway > > wrote: > > > But I still cannot shake the belief that using a format code of 1 - > > which really could be interpreted as

Re: Emitting JSON to file using COPY TO

On 12/6/23 19:39, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:45 PM Joe Conway > wrote: " The backend sends a CopyOutResponse message to the frontend, followed     by zero or more CopyData messages (always one per row), followed by     CopyDone"

Re: Is WAL_DEBUG related code still relevant today?

On Wed, Dec 06, 2023 at 09:46:09AM -0300, Euler Taveira wrote: > On Wed, Dec 6, 2023, at 8:27 AM, Peter Eisentraut wrote: >> This kind of thing could be mostly avoided if we didn't hide all the >> WAL_DEBUG behind #ifdefs. > > AFAICS LOCK_DEBUG also hides its GUCs behind #ifdefs. The fact that

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:45 PM Joe Conway wrote: > > " The backend sends a CopyOutResponse message to the frontend, followed > by zero or more CopyData messages (always one per row), followed by > CopyDone" > > probably "always one per row" would be changed to note that json array >

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 06, 2023 at 10:07:51PM +0800, Junwang Zhao wrote: > I read the thread[1] you posted and I think Andres's suggestion sounds great. > > Should we extract both *copy to* and *copy from* for the first step, in that > case we can add the pg_copy_handler catalog smoothly later. > >

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

On Wed, Dec 06, 2023 at 03:17:12PM +0900, Michael Paquier wrote: > On Sat, Nov 18, 2023 at 04:32:36PM -0800, Noah Misch wrote: > > On Sat, Nov 18, 2023 at 03:09:58PM -0800, Andres Freund wrote: > >> Unfortunately, there is a case of such an sqlstate that's not at all > >> indicating > >>

RE: [CAUTION!! freemail] Re: Partial aggregates pushdown

Hi Mr.Haas. > -Original Message- > From: Robert Haas > Sent: Wednesday, December 6, 2023 10:25 PM > On Wed, Dec 6, 2023 at 3:41 AM fujii.y...@df.mitsubishielectric.co.jp > wrote: > > Are you concerned about the hassle and potential human errors of > > manually adding new partial

pg16 && GSSAPI && Heimdal/Macos

Earlier this year, there was a thread about GSSAPI for delegated credentials and various operating systems ultimately that Heimdal had atrophied enough that you were comfortable not supporting it anymore as a GSSAPI library. Thread:

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:38, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:28 PM David G. Johnston mailto:david.g.johns...@gmail.com>> wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway mailto:m...@joeconway.com>> wrote: On 12/6/23 14:47, Joe Conway wrote: > On 12/6/23 13:59,

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:28, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway > wrote: On 12/6/23 14:47, Joe Conway wrote: > On 12/6/23 13:59, Daniel Verite wrote: >>      Andrew Dunstan wrote: >> >>> IMNSHO, we should produce either a

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:28 PM David G. Johnston wrote: > On Wed, Dec 6, 2023 at 4:09 PM Joe Conway wrote: > >> On 12/6/23 14:47, Joe Conway wrote: >> > On 12/6/23 13:59, Daniel Verite wrote: >> >> Andrew Dunstan wrote: >> >> >> >>> IMNSHO, we should produce either a single JSON >> >>>

Re: RFC: Logging plan of the running query

Hello hackers, Last Saturday I submitted a patch to the pgsql-hackers list with the title "Proposal: In-flight explain logging" with a patch proposing a feature very similar to the one being worked on in this thread. I should have done a better search in the commitfest before implementing

Re: PATCH: Add REINDEX tag to event triggers

On Wed, Dec 06, 2023 at 10:00:01AM +0300, Alexander Lakhin wrote: > I agree with it. I had worried a bit about ReindexRelationConcurrently() > becoming twofold for callers (it can leave the snapshot or pop it), but I > couldn't find a way to hide this twofoldness inside without adding more >

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:09 PM Joe Conway wrote: > On 12/6/23 14:47, Joe Conway wrote: > > On 12/6/23 13:59, Daniel Verite wrote: > >> Andrew Dunstan wrote: > >> > >>> IMNSHO, we should produce either a single JSON > >>> document (the ARRAY case) or a series of JSON documents, one per row >

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 11:30:02AM -0600, Nathan Bossart wrote: > On Wed, Dec 06, 2023 at 06:27:04PM +0100, Peter Eisentraut wrote: >> Makes sense. Can you commit that? > > Yes, I will do so shortly. Committed. Apologies for the delay. -- Nathan Bossart Amazon Web Services:

Re: Emitting JSON to file using COPY TO

On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operations" in the doc says: " The backend

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 3:38 PM Joe Conway wrote: > So the questions are: > 1. Do those two formats work for the initial implementation? > Yes. We provide a stream-oriented format and one atomic-import format. 2. Is the default correct or should it be switched > e.g. rather than specifying

Re: Emitting JSON to file using COPY TO

On 12/6/23 16:42, Sehrope Sarkuni wrote: On Wed, Dec 6, 2023 at 4:29 PM Joe Conway > wrote: > 1. Outputting a top level JSON object without the additional column > keys. IIUC, the top level keys are always the column names. A common use > case would

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:29 PM Joe Conway wrote: > > 1. Outputting a top level JSON object without the additional column > > keys. IIUC, the top level keys are always the column names. A common use > > case would be a single json/jsonb column that is already formatted > > exactly as the user

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 03:20:46PM -0500, Tom Lane wrote: > If Nathan's perf results hold up elsewhere, it seems like some > micro-optimization around the text-pushing (appendStringInfoString) > might be more useful than caching. The 7% spent in cache lookups > could be worth going after later,

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:03 PM Andrew Dunstan wrote: > > The output size difference does say that maybe we should pay some > > attention to the nearby request to not always label every field. > > Perhaps there should be an option for each row to transform to > > a JSON array rather than an

Re: Emitting JSON to file using COPY TO

On 12/6/23 11:28, Sehrope Sarkuni wrote: Big +1 to this overall feature. cool! Regarding the defaults for the output, I think JSON lines (rather than a JSON array of objects) would be preferred. It's more natural to combine them and generate that type of data on the fly rather than forcing

Re: automating RangeTblEntry node support

On Wed, 6 Dec 2023 at 21:02, Peter Eisentraut wrote: > > I have been looking into what it would take to get rid of the > custom_read_write and custom_query_jumble for the RangeTblEntry node > type. This is one of the larger and more complex exceptions left. > [...] > Now one could probably

Reducing output size of nodeToString

Hi, PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its size signficantly. The basic techniques

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 15:20, Tom Lane wrote: Joe Conway writes: I'll see if I can add some caching to composite_to_json(), but based on the relative data size it does not sound like there is much performance left on the table to go after, no? If Nathan's perf results hold up elsewhere, it seems

Re: Remove MSVC scripts from the tree

On 2023-12-06 We 12:24, Peter Eisentraut wrote: On 06.12.23 17:27, Andrew Dunstan wrote: But the VS2019 compiler implements enough of C99 to pass our meson test, unlike VS2017. Maybe the test is too strict. After all, we know we can in fact build with the earlier versions. I just realized

Re: gai_strerror() is not thread-safe on Windows

On Tue, Dec 5, 2023 at 3:43 PM Kyotaro Horiguchi wrote: > At Tue, 5 Dec 2023 08:26:54 +1300, Thomas Munro > wrote in > > On second thoughts, I guess it would make more sense to use the exact > > messages Windows' own implementation would return instead of whatever > > we had in the past

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > I'll see if I can add some caching to composite_to_json(), but based on > the relative data size it does not sound like there is much performance > left on the table to go after, no? If Nathan's perf results hold up elsewhere, it seems like some micro-optimization around

automating RangeTblEntry node support

I have been looking into what it would take to get rid of the custom_read_write and custom_query_jumble for the RangeTblEntry node type. This is one of the larger and more complex exceptions left. (Similar considerations would also apply to the Constraint node type.) Allegedly, only certain

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 11:14 AM Matthias van de Meent wrote: > I was thinking more along the lines of page splits+deletions while > we're doing _bt_stepright(), but forgot to consider that we first lock > the right sibling, and only then release the left sibling for splits, > so we should be fine

Re: Emitting JSON to file using COPY TO

On 12/6/23 11:44, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:33:49AM -0600, Nathan Bossart wrote: (format csv) Time: 12295.480 ms (00:12.295) Time: 12311.059 ms (00:12.311) Time: 12305.469 ms (00:12.305) (format json) Time: 24568.621 ms

Re: Emitting JSON to file using COPY TO

On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operations" in the doc says: " The backend sends a CopyOutResponse message to the

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, 6 Dec 2023 at 19:55, Peter Geoghegan wrote: > > On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent > wrote: > > 1. When scanning an index in ascending order using scankey a > 1 (so, > > one that defines a start point of the scan), we don't need to check > > items for consistency with

Re: Emitting JSON to file using COPY TO

Andrew Dunstan wrote: > IMNSHO, we should produce either a single JSON > document (the ARRAY case) or a series of JSON documents, one per row > (the LINES case). "COPY Operations" in the doc says: " The backend sends a CopyOutResponse message to the frontend, followed by zero or

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent wrote: > 1. When scanning an index in ascending order using scankey a > 1 (so, > one that defines a start point of the scan), we don't need to check > items for consistency with that scankey once we've found the first > value that is consistent

Re: generic plans and "initial" pruning

Reviewing 0001: Perhaps ExecEndCteScan needs an adjustment. What if node->leader was never set? Other than that, I think this is in good shape. Maybe there are other things we'd want to adjust here, or maybe there aren't, but there doesn't seem to be any good reason to bundle more changes into

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

Hi, On 2023-12-01 23:06:59 +, Sacha Hottinger wrote: > // I used the patch command to patch the src/backend/port/Makefile with your > attached file and tried again with the Sun Studio compiler. There is now a > different error at this stage: > … > /opt/developerstudio12.6/bin/cc -m64

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Tue, Dec 5, 2023 at 8:20 PM Peter Geoghegan wrote: > On Tue, Dec 5, 2023 at 8:06 PM Alexander Korotkov > wrote: > > Thank you for raising this issue. Preprocessing of btree scan keys is > > normally removing the redundant scan keys. However, redundant scan > > keys aren't removed when they

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 06:27:04PM +0100, Peter Eisentraut wrote: > Makes sense. Can you commit that? Yes, I will do so shortly. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Clean up some signal usage mainly related to Windows

On 06.12.23 17:18, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of ‘write’,

Re: Remove MSVC scripts from the tree

On 06.12.23 17:27, Andrew Dunstan wrote: But the VS2019 compiler implements enough of C99 to pass our meson test, unlike VS2017. Maybe the test is too strict. After all, we know we can in fact build with the earlier versions. I just realized that the C99 test is actually our own, not provided

Re: Change GUC hashtable to use simplehash?

On Wed, 2023-12-06 at 07:39 +0700, John Naylor wrote: > "git grep cstring_hash" found nothing, so not sure what you're > asking. Sorry, I meant string_hash(). Your v5-0002 changes the way hashing works for cstrings, and that means it's no longer equivalent to hash_bytes with strlen. That's

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent wrote: > On Wed, 6 Dec 2023 at 14:11, Robert Haas wrote: > > It isn't very clear from the commit message that this commit is doing > > two different things, and in fact I'm still unclear on what exactly > > the other optimization is. > > I

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 10:33:49AM -0600, Nathan Bossart wrote: > (format csv) > Time: 12295.480 ms (00:12.295) > Time: 12311.059 ms (00:12.311) > Time: 12305.469 ms (00:12.305) > > (format json) > Time: 24568.621 ms (00:24.569) > Time: 23756.234 ms

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 10:28:49AM -0600, Tristan Partin wrote: > According to my setup, I am hitting the /* Ignore */ variant of __wur. I am > guessing that Fedora doesn't add fortification to the default CFLAGS. What > distro are you using? But yes, something like what you proposed sounds good >

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 11:28:59AM -0500, Tom Lane wrote: > It might be acceptable to plan on improving the performance later, > depending on just how bad it is now. On 10M rows with 11 integers each, I'm seeing the following: (format text) Time: 10056.311 ms (00:10.056)

Re: Emitting JSON to file using COPY TO

Andrew Dunstan writes: > On 2023-12-06 We 10:44, Tom Lane wrote: >> In particular, has anyone done any performance testing? >> I'm concerned about that because composite_to_json() has >> zero capability to cache any metadata across calls, meaning >> there is going to be a large amount of

Re: Clean up some signal usage mainly related to Windows

On Wed Dec 6, 2023 at 10:18 AM CST, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: > Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of

Re: Emitting JSON to file using COPY TO

Big +1 to this overall feature. This is something I've wanted for a long time as well. While it's possible to use a COPY with text output for a trivial case, the double escaping falls apart quickly for arbitrary data. It's really only usable when you know exactly what you are querying and know it

Re: Remove MSVC scripts from the tree

On 2023-12-06 We 01:18, Peter Eisentraut wrote: On 04.12.23 21:11, Andrew Dunstan wrote: I just had a look at shifting bowerbird to use meson, and it got stymied at the c99 test, which apparently doesn't compile with anything less than VS2019. If that is the case, then wouldn't that

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > On 12/6/23 10:44, Tom Lane wrote: >> In particular, has anyone done any performance testing? > I will devise some kind of test and report back. I suppose something > with many rows and many narrow columns comparing time to COPY > text/csv/json modes would do the trick?

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 10:44, Tom Lane wrote: Joe Conway writes: I believe this is ready to commit unless there are further comments or objections. I thought we were still mostly at proof-of-concept stage? In particular, has anyone done any performance testing? I'm concerned about that because

Re: Emitting JSON to file using COPY TO

On 12/6/23 10:44, Tom Lane wrote: Joe Conway writes: I believe this is ready to commit unless there are further comments or objections. I thought we were still mostly at proof-of-concept stage? The concept is narrowly scoped enough that I think we are homing in on the final patch. In

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: > Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of ‘write’, declared with attribute warn_unused_result

Re: Emitting JSON to file using COPY TO

On 12/6/23 10:32, Andrew Dunstan wrote: On 2023-12-06 We 08:49, Joe Conway wrote: On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > I believe this is ready to commit unless there are further comments or > objections. I thought we were still mostly at proof-of-concept stage? In particular, has anyone done any performance testing? I'm concerned about that because composite_to_json() has zero capability

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 08:49, Joe Conway wrote: On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and

Re: remaining sql/json patches

On 2023-Dec-06, Amit Langote wrote: > I think I'm inclined toward adapting the LA-token fix (attached 0005), > because we've done that before with SQL/JSON constructors patch. > Also, if I understand the concerns that Tom mentioned at [1] > correctly, maybe we'd be better off not assigning

Re: [RFC] Clang plugin for catching suspicious typedef casting

> On Sun, Dec 03, 2023 at 07:02:55PM -0800, Peter Geoghegan wrote: > On Sun, Dec 3, 2023 at 6:31 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > Only advantage I see to using libclang is that you can run programs built > > > with libclang regardless of what your C compiler is. I typically

Re: Is WAL_DEBUG related code still relevant today?

Peter Eisentraut writes: > This kind of thing could be mostly avoided if we didn't hide all the > WAL_DEBUG behind #ifdefs. For example, in the attached patch, I instead > changed it so that > if (XLOG_DEBUG) > resolves to > if (false) > in the normal case. That way, we don't need

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 6, 2023 at 8:32 PM Daniel Verite wrote: > > Sutou Kouhei wrote: > > > * 2022-04: Apache Arrow [2] > > * 2018-02: Apache Avro, Apache Parquet and Apache ORC [3] > > > > (FYI: I want to add support for Apache Arrow.) > > > > There were discussions how to add support for more

Re: logical decoding and replication of sequences, take 2

On 12/6/23 09:56, Amit Kapila wrote: > On Tue, Dec 5, 2023 at 10:23 PM Tomas Vondra > wrote: >> >> On 12/5/23 13:17, Amit Kapila wrote: >> >>> (b) for transactional >>> cases, we see overhead due to traversing all the top-level txns and >>> check the hash table for each one to find whether change

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 6, 2023 at 3:28 PM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Wed, 6 Dec 2023 15:11:34 +0800, > Junwang Zhao wrote: > > > For the extra curly braces, I mean the following code block in > >

Re: logical decoding and replication of sequences, take 2

On 12/6/23 11:19, Amit Kapila wrote: > On Sun, Dec 3, 2023 at 11:56 PM Tomas Vondra > wrote: >> >> On 12/3/23 18:52, Tomas Vondra wrote: >>> ... >>> >> >> Another idea is that maybe we could somehow inform ReorderBuffer whether >> the output plugin even is interested in sequences. That'd help

Re: Emitting JSON to file using COPY TO

On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why

  1   2   >