Re: meson: Specify -Wformat as a common warning flag for extensions

2024-05-29 Thread Sutou Kouhei
Hi, In <4707d4ed-f268-43c0-b4dd-cdbc7520f...@eisentraut.org> "Re: meson: Specify -Wformat as a common warning flag for extensions" on Tue, 28 May 2024 23:31:05 -0700, Peter Eisentraut wrote: > On 07.04.24 18:01, Sutou Kouhei wrote: >> +# We don't

Re: Is it acceptable making COPY format extendable?

2024-05-06 Thread Sutou Kouhei
Hi, In <7a978dd2-371b-4d10-b58e-cb32ef728...@eisentraut.org> "Re: Is it acceptable making COPY format extendable?" on Thu, 25 Apr 2024 09:59:18 +0200, Peter Eisentraut wrote: > PostgreSQL development is currently in feature freeze for PostgreSQL > 17 (since April 8). So most participants

Re: 2024-05-09 release announcement draft,2024-05-09 release announcement draft

2024-05-06 Thread Sutou Kouhei
Hi, In <779790c7-45d7-4010-9305-c3f9e6a60...@postgresql.org>,<779790c7-45d7-4010-9305-c3f9e6a60...@postgresql.org> "2024-05-09 release announcement draft,2024-05-09 release announcement draft" on Mon, 6 May 2024 13:44:05 -0400, "Jonathan S. Katz" wrote: > * Added optimization for certain

Re: Is it acceptable making COPY format extendable?

2024-04-24 Thread Sutou Kouhei
Hi, Thanks for replying this. In <02cccd36-3083-4a50-aae4-f957e96fb...@eisentraut.org> "Re: Is it acceptable making COPY format extendable?" on Wed, 24 Apr 2024 09:57:38 +0200, Peter Eisentraut wrote: >> I'm proposing a patch that making COPY format extendable: >>

Is it acceptable making COPY format extendable?

2024-04-22 Thread Sutou Kouhei
Hi, I'm proposing a patch that making COPY format extendable: https://www.postgresql.org/message-id/20231204.153548.2126325458835528809.kou%40clear-code.com https://commitfest.postgresql.org/48/4681/ It's based on the discussion at:

Re: pg_stat_statements and "IN" conditions

2024-04-15 Thread Sutou Kouhei
Hi, In <20240404143514.a26f7ttxrbdfc73a@erthalion.local> "Re: pg_stat_statements and "IN" conditions" on Thu, 4 Apr 2024 16:35:14 +0200, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Here is the rebased version. Thanks. I'm not familiar with this code base but I've reviewed these patches

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-04-10 Thread Sutou Kouhei
In <20240320.232732.488684985873786799@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 20 Mar 2024 23:27:32 +0900 (JST), Sutou Kouhei wrote: > Hi, > > Could someone review the v17 patch to proceed this? > > The v17 patch: > https://

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Sutou Kouhei
way. OK. I've removed "/W*" flags and warning_level==3/everything cases. How about the attached v5 patch? Thanks, -- kou >From 205ef88c66cf1050eedfc1e72d951de93a02e53a Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Fri, 15 Mar 2024 18:27:30 +0900 Subject: [PATCH v5] meson: Restore implic

Re: Commitfest Manager for March

2024-04-02 Thread Sutou Kouhei
Hi, In "Re: Commitfest Manager for March" on Mon, 1 Apr 2024 11:57:27 +0300, Heikki Linnakangas wrote: > On 01/04/2024 09:05, Andrey M. Borodin wrote: >> I think it makes sense to close this commitfest only after Feature >> Freeze on April 8, 2024 at 0:00 AoE. >> What do you think? > >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-20 Thread Sutou Kouhei
%40paquier.xyz#eccfd1a0131af93c48026d691cc247f4 Thanks, -- kou In <20240308.092254.359611633589181574@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 08 Mar 2024 09:22:54 +0900 (JST), Sutou Kouhei wrote: > Hi, > &g

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-03-15 Thread Sutou Kouhei
gt; For someone who wants to write a fix for this, the relevant variable > is var_cflags in our meson scripts. And var_cxxflags as well. How about the attached v4 patch? Thanks, -- kou >From a515720338dc49e764f598021200316c6d01ddf9 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Fri, 15 Mar 2024 18:

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-15 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 13 Mar 2024 16:00:46 +0800, jian he wrote: >> More use cases will help us which callbacks are needed. We >> will be able to collect more use cases by providing basic >> callbacks. > Let's say the

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-03-13 Thread Sutou Kouhei
the wording! I used it for the v3 patch. Thanks, -- kou >From 0ba2e6dd55b00ee8a57313a629a1e5fa8c9e40cc Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Wed, 13 Mar 2024 16:10:34 +0900 Subject: [PATCH v3] Add -Wformat to common warning flags We specify -Wformat-security as a common warning f

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-03-12 Thread Sutou Kouhei
Hi, In "Re: meson: Specify -Wformat as a common warning flag for extensions" on Fri, 08 Mar 2024 10:05:27 -0600, "Tristan Partin" wrote: > Ok, I figured this out. -Wall implies -Wformat=1. We set warning_level > to 1 in the Meson project() call, which implies -Wall, and set -Wall > in

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-10 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 11 Mar 2024 08:00:00 +0800, jian he wrote: > Hi, here are my cents: > Currently in v17, we have 3 extra functions within DoCopyTo > CopyToStart, one time, start, doing some preliminary work. >

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-03-08 Thread Sutou Kouhei
Hi, In "Re: meson: Specify -Wformat as a common warning flag for extensions" on Fri, 8 Mar 2024 15:32:22 +0900, Michael Paquier wrote: > Are there version and/or > environment requirements to be aware of? I'm using Debian GNU/Linux sid and I

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-07 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 7 Mar 2024 15:32:01 +0900, Michael Paquier wrote: > While on it, here are some profiles based on HEAD and v17 with the > previous tests (COPY TO /dev/null, COPY FROM data sent to the void). > ... >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-05 Thread Sutou Kouhei
* Export CopyReadBinaryData() [1] https://www.postgresql.org/message-id/flat/20240124.144936.67229716500876806.kou%40clear-code.com#f1ad092fc5e81fe38d3c376559efd52c Thanks, -- kou >From a78b8ee88575e2c2873afc3acf3c8c4e535becf0 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Mon,

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-03-03 Thread Sutou Kouhei
Hi, In <20240301.154443.618034282613922707@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 01 Mar 2024 15:44:43 +0900 (JST), Sutou Kouhei wrote: >> I guess so. It does not make much of a differen

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 1 Mar 2024 14:31:38 +0900, Michael Paquier wrote: > I guess so. It does not make much of a difference, though. The thing > is that the dispatch caused by the custom callbacks called for each > row

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-29 Thread Sutou Kouhei
Hi, In <20240222.183948.518018047578925034@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 22 Feb 2024 18:39:48 +0900 (JST), Sutou Kouhei wrote: > How about adding "is_csv" to CopyReadline() and >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-22 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 22 Feb 2024 15:44:16 +0900, Michael Paquier wrote: > I was comparing what you have here, and what's been attached by Andres > at [1] and the top of the changes on my development branch at [2] >

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-02-21 Thread Sutou Kouhei
as a common warning flag for extensions" on Mon, 22 Jan 2024 14:11:39 +0900 (JST), Sutou Kouhei wrote: > Hi, > > I'm an extension developer. If I use PostgreSQL built with > Meson, I get the following warning: > > cc1: warning: '-Wformat-security' ignored withou

Re: Why is pq_begintypsend so slow?

2024-02-21 Thread Sutou Kouhei
Hi, In <20240219195351.5vy7cdl3wxia6...@awork3.anarazel.de> "Re: Why is pq_begintypsend so slow?" on Mon, 19 Feb 2024 11:53:51 -0800, Andres Freund wrote: >> I don't have a strong opinion how to implement this >> optimization as I said above. It seems that you like your >> approach. So I

Re: Why is pq_begintypsend so slow?

2024-02-18 Thread Sutou Kouhei
Hi, In <20240218200906.zvihkrs46yl2j...@awork3.anarazel.de> "Re: Why is pq_begintypsend so slow?" on Sun, 18 Feb 2024 12:09:06 -0800, Andres Freund wrote: >> [1] >>

Re: Why is pq_begintypsend so slow?

2024-02-18 Thread Sutou Kouhei
Hi, In <20240218015955.rmw5mcmobt5hb...@awork3.anarazel.de> "Re: Why is pq_begintypsend so slow?" on Sat, 17 Feb 2024 17:59:55 -0800, Andres Freund wrote: > v3-0008-wip-make-in-out-send-recv-calls-in-copy.c-cheaper.patch It seems that this is an alternative approach of [1]. [1]

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-15 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 15 Feb 2024 17:09:20 +0800, jian he wrote: > My environment is slow (around 10x) but consistent. > I see around 2-3 percent increase consistently. > (with patch 7369.068 ms, without patch 7574.802 ms)

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-14 Thread Sutou Kouhei
Hi, In <20240213.173340.1518143507526518973@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 13 Feb 2024 17:33:40 +0900 (JST), Sutou Kouhei wrote: > I'll reply other comments later... I've read other comments and

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-14 Thread Sutou Kouhei
chmark. So they may not be valid numbers. Thanks, -- kou >From b677732f46f735a5601b889f79671e91be41 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Thu, 15 Feb 2024 15:01:08 +0900 Subject: [PATCH v2] Reuse fcinfo used in COPY FROM Each NextCopyFrom() calls input functions or binary-in

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-13 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 14 Feb 2024 12:28:38 +0900, Michael Paquier wrote: >> How about the attached patch approach? If it's a desired >> approach, I can also write a separated patch for COPY TO. > > Hmm, I have not studied

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-13 Thread Sutou Kouhei
ttached patch? Thanks, -- kou >From 6eb9669f97c54f8b85fac63db40ad80664692d12 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Wed, 14 Feb 2024 11:44:13 +0900 Subject: [PATCH v2] Add a test for invalid encoding for COPY FROM The test data use an UTF-8 character (U+3042 HIRAGANA LETTER A) but the test specifies

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-13 Thread Sutou Kouhei
Hi, In <20240209192705.5qdilvviq3py2...@awork3.anarazel.de> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 9 Feb 2024 11:27:05 -0800, Andres Freund wrote: >> +static void >> +CopyFromTextInFunc(CopyFromState cstate, Oid atttypid, >> +

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-08 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 9 Feb 2024 13:21:34 +0900, Michael Paquier wrote: >> - Revisit what we have here, looking at more profiles to see how HEAD >> an v13 compare. It looks like we are on a good path, but let's tackle >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-08 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 7 Feb 2024 13:33:18 +0900, Michael Paquier wrote: > Hmm. That explains why I was not seeing any differences with this > callback then. It seems to me that the order of actions to take is > clear,

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-08 Thread Sutou Kouhei
Hi, In <20240208.172501.2177371292839763981@clear-code.com> "Re: confusing / inefficient "need_transcoding" handling in copy" on Thu, 08 Feb 2024 17:25:01 +0900 (JST), Sutou Kouhei wrote: > How about the following to avoid needless transcoding? Oh, sorry.

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-08 Thread Sutou Kouhei
rate_series(1, 100::int4)) TO '/dev/null' \watch c=5 Thanks, -- kou From 387f11bde4eb928af23f6a66101aa9d63b3dcfdf Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Thu, 8 Feb 2024 17:17:25 +0900 Subject: [PATCH v1] Add a test for invalid encoding for COPY FROM The test data uses UTF-8 "hel

Re: meson: catalog/syscache_ids.h isn't installed

2024-02-07 Thread Sutou Kouhei
Hi, In <4b60e9bd-426c-4d4b-bbbd-1abd06fa0...@eisentraut.org> "Re: meson: catalog/syscache_ids.h isn't installed" on Mon, 5 Feb 2024 17:53:52 +0100, Peter Eisentraut wrote: > On 05.02.24 02:29, Sutou Kouhei wrote: >> catalog/syscache_ids.h is referred by uti

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-05 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 5 Feb 2024 16:14:08 +0900, Michael Paquier wrote: > So, I've looked at all that today, and finished by applying two > patches as of 2889fd23be56 and 95fb5b49024a to get some of the > weirdness with

meson: catalog/syscache_ids.h isn't installed

2024-02-04 Thread Sutou Kouhei
Hi, catalog/syscache_ids.h is referred by utils/syscache.h but it's not installed with Meson. FYI: * 9b1a6f50b91dca6610932650c8c81a3c924259f9 It uses catalog/syscache_ids.h in utils/syscache.h but catalog/syscache_ids.h isn't installed. * 6eb6086faa3842c2a38a1ee2f97bf9a42ce27610 It changes

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-02 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 2 Feb 2024 17:04:28 +0900, Michael Paquier wrote: > One idea I was considering is whether we should use a special value in > the "format" DefElem, say "custom:$my_custom_format" where it would be >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-01 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 2 Feb 2024 15:27:15 +0800, Junwang Zhao wrote: > I agree CopyToRoutine should be placed into CopyToStateData, but > why set it after ProcessCopyOptions, the implementation of > CopyToGetRoutine

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-01 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 2 Feb 2024 15:21:31 +0900, Michael Paquier wrote: > I have done a review of v10, see v11 attached which is still WIP, with > the patches for COPY TO and COPY FROM merged together. Note that I'm >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-01 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 2 Feb 2024 06:51:02 +0900, Michael Paquier wrote: > On Fri, Feb 02, 2024 at 12:19:51AM +0900, Sutou Kouhei wrote: >> Here are some numbers on my local machine (Note that my >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-01 Thread Sutou Kouhei
Hi, Thanks for preparing benchmark. In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 1 Feb 2024 12:49:59 +0900, Michael Paquier wrote: > On Thu, Feb 01, 2024 at 10:57:58AM +0900, Michael Paquier wrote: >> And here are the results I get for text and

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-30 Thread Sutou Kouhei
/* Called when COPY TO is ended. This will send a trailer. */ void(*CopyToEnd) (CopyToState cstate); } CopyToRoutine; Thanks, -- kou >From f827f1f1632dc330ef5d78141b85df8ca1bce63b Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Wed, 31 Jan

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-30 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 30 Jan 2024 16:20:54 +0900, Michael Paquier wrote: >>> +if (!format_specified) >>> +/* Set the default format. */ >>> +ProcessCopyOptionFormatTo(pstate,

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 30 Jan 2024 11:11:59 +0900, Masahiko Sawada wrote: > --- > +if (!format_specified) > +/* Set the default format. */ > +ProcessCopyOptionFormatTo(pstate,

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-29 Thread Sutou Kouhei
al() and pass the read attributes callback explicitly to eliminate more cstate->opts.csv_mode branches Thanks, -- kou >From c136833f4a385574474b246a381014abeb631377 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Fri, 26 Jan 2024 16:46:51 +0900 Subject: [PATCH v9 1/2] Extract COPY TO format implement

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-28 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Sat, 27 Jan 2024 14:15:02 +0800, Junwang Zhao wrote: > I have been working on a *COPY TO JSON* extension since yesterday, > which is based on your V6 patch set, I'd like to give you more input > so you can

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-26 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 26 Jan 2024 16:41:50 +0800, Junwang Zhao wrote: > CopyToProcessOption()/CopyFromProcessOption() can only handle > single option, and store the options in the opaque field, but it can not > check the

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-26 Thread Sutou Kouhei
(cstate->opts.csv_mode)" branches such as one in NextCopyFromRawFields(). Should we do it in this feature improvement (make COPY format extendable)? Can we defer this as a separated improvement? * Remove CopyFrom*_function typedefs Thanks, -- kou >From 3e75129c2e9d9d3

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-26 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 26 Jan 2024 16:18:14 +0800, Junwang Zhao wrote: > In the current implementation, there is no way that one can check > incompatibility > options in ProcessCopyOptions, we can postpone the check in

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-25 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 25 Jan 2024 13:36:03 +0900, Masahiko Sawada wrote: > I've experimented with a similar optimization for csv > and text format; have different callbacks for text and csv format and >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-25 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 25 Jan 2024 12:17:55 +0900, Michael Paquier wrote: > +typedef bool (*CopyToProcessOption_function) (CopyToState cstate, DefElem > *defel); > +typedef int16 (*CopyToGetFormat_function) (CopyToState

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-25 Thread Sutou Kouhei
Hi, Thanks for trying these patches! In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 25 Jan 2024 10:53:58 +0800, jian he wrote: > COPY data TO '/dev/null' WITH (FORMAT csv) \watch count=5 Wow! I didn't know the "\watch count="! I'll use it. > Time:

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-24 Thread Sutou Kouhei
Hi, In <20240124.144936.67229716500876806@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 24 Jan 2024 14:49:36 +0900 (JST), Sutou Kouhei wrote: > I've implemented custom COPY format feature based on the > current

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-24 Thread Sutou Kouhei
24, 2024 at 02:49:36PM +0900, Sutou Kouhei wrote: >>> For COPY TO: >>> >>> 0001: This adds CopyToRoutine and use it for text/csv/binary >>> formats. No implementation change. This just move codes. >> 10M without this change: >> >> format,el

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-23 Thread Sutou Kouhei
. CopyReadBinaryData() is renamed to CopyFromStateRead(). Thanks, -- kou In <20240115.152702.2011620917962812379@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 15 Jan 2024 15:27:02 +0900 (JST), Sutou Kouhei wrote: >

meson: Specify -Wformat as a common warning flag for extensions

2024-01-21 Thread Sutou Kouhei
y -Wformat as a common warning flag too? If we do it, "pg_config --cflags" includes both of -Wformat-security and -Wformat. So I don't get the warning. Thanks, -- kou >From 0913033512c9b75ee3d2941c89ff8696f3c5f53b Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Mon, 22 Jan 2024

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-15 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 15 Jan 2024 16:03:41 +0900, Masahiko Sawada wrote: >> Defining one more static const struct instead of providing a >> convenient (but a bit tricky) macro may be straightforward: >> >> static const

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-14 Thread Sutou Kouhei
0] I'll export Copy{To,From}State. Thanks, -- kou In <20240112.144615.157925223373344229@clear-code.com> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 12 Jan 2024 14:46:15 +0900 (JST), Sutou Kouhei wrote: > Hi, > > Here is the c

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 12 Jan 2024 14:40:41 +0800, Junwang Zhao wrote: >> Could you clarify what should we discuss? We should require >> that COPY TO/FROM handlers should use PostgreSQL's memory >> context for all internal

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-11 Thread Sutou Kouhei
Hi, Here is the current summary for a this discussion to make COPY format extendable. It's for reaching consensus and starting implementing the feature. (I'll start implementing the feature once we reach consensus.) If you have any opinion, please share it. Confirmed: 1.1 Making COPY format

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-10 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 10 Jan 2024 16:53:48 +0900, Masahiko Sawada wrote: >> Interesting. But I feel that it introduces another (a bit) >> tricky mechanism... > > Right. On the other hand, I don't think the idea 3 is good

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 10 Jan 2024 15:33:22 +0900, Masahiko Sawada wrote: >> We can use the satic const struct approach by choosing one >> of the followings: >> >> ... >> >> 4. ... other idea? > > It's a just idea but the

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 22 Dec 2023 10:58:05 +0800, Junwang Zhao wrote: >> 1. Add an opaque space for custom COPY TO handler >>* Add CopyToState{Get,Set}Opaque() >> >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 22 Dec 2023 10:48:18 +0900, Masahiko Sawada wrote: >> I needed to extend the patch: >> >> 1. Add an opaque space for custom COPY TO handler >>* Add CopyToState{Get,Set}Opaque() >> >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 22 Dec 2023 10:00:24 +0900, Michael Paquier wrote: >> 3. Export CopySend*() >> >>* If we like minimum API, we just need to export >> CopySendData() and CopySendEndOfRow(). But >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-21 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 11 Dec 2023 23:31:29 +0900, Masahiko Sawada wrote: > I've sketched the above idea including a test module in > src/test/module/test_copy_format, based on v2 patch. It's not splitted > and is dirty so

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 15 Dec 2023 11:27:30 +0800, Junwang Zhao wrote: >> > Adding a prefix or suffix would be one option but to give extensions >> > more flexibility, another option would be to support format = 'custom' >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-14 Thread Sutou Kouhei
Hi, In "RE: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 12 Dec 2023 02:31:53 +, "Hayato Kuroda (Fujitsu)" wrote: >> Can we discuss how to proceed this improvement? >> >> There are 2 approaches for it: >> >> 1. Do the followings concurrently: >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 15 Dec 2023 05:19:43 +0900, Masahiko Sawada wrote: > To avoid collisions, extensions can be created in a > different schema than public. Thanks. I didn't notice it. > And note that built-in format

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 11 Dec 2023 10:57:15 +0900, Masahiko Sawada wrote: > IIUC we cannot create two same name functions with the same arguments > but a different return value type in the first place. It seems to me > to

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Sat, 9 Dec 2023 12:38:46 +0100, Hannu Krosing wrote: > Please also see my presentation slides from last years PostgreSQL > Conference in Berlin (attached) Thanks for sharing your idea here. > The main

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-09 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 8 Dec 2023 15:42:06 +0900, Masahiko Sawada wrote: > So a custom COPY extension would not be able to define SQL functions > just like arrow(internal) for example. We might need to define a rule > like

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-09 Thread Sutou Kouhei
Hi, Thanks for reviewing our latest patch! In "RE: Make COPY format extendable: Extract COPY TO format implementations" on Sat, 9 Dec 2023 02:43:49 +, "Hayato Kuroda (Fujitsu)" wrote: > (I remember that this theme was talked at Japan PostgreSQL conference) Yes. I should have

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-06 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Wed, 6 Dec 2023 22:07:51 +0800, Junwang Zhao wrote: > Should we extract both *copy to* and *copy from* for the first step, in that > case we can add the pg_copy_handler catalog smoothly later. I don't

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
, -- kou >From 9fe0087d9a6a79a7d1a7d0af63eb16abadbf0d4a Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Mon, 4 Dec 2023 12:32:54 +0900 Subject: [PATCH v3] Extract COPY TO format implementations This is a part of making COPY format extendable. See also these past discussions: * New Copy For

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
anks for the review! I attach the v2 patch that removes extra curly braces for "if (isnull)". Thanks, -- kou >From 2cd0d344d68667db71b621a8c94f376ddf1707c3 Mon Sep 17 00:00:00 2001 From: Sutou Kouhei Date: Mon, 4 Dec 2023 12:32:54 +0900 Subject: [PATCH v2] Extract COPY TO format

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-05 Thread Sutou Kouhei
Hi, Thanks for replying to this proposal! In <20231205182458.GC2757816@nathanxps13> "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 5 Dec 2023 12:24:58 -0600, Nathan Bossart wrote: > I think it makes sense to do this part independently, but we should be >

Make COPY format extendable: Extract COPY TO format implementations

2023-12-03 Thread Sutou Kouhei
fba%40gmail.com [4]: https://www.postgresql.org/message-id/flat/3741749.1655952719%40sss.pgh.pa.us#2bb7af4a3d2c7669f9a49808d777a20d [5]: https://www.postgresql.org/message-id/20180211211235.5x3jywe5z3lkgcsr%40alap3.anarazel.de [6]: https://www.postgresql.org/message-id/3741749.1655952719%40sss.pgh.pa.us Thanks, -- kou >F