Re: Test of a partition with an incomplete detach has a timing issue

2021-05-27 Thread Noah Misch
On Tue, May 25, 2021 at 11:32:38AM -0400, Alvaro Herrera wrote: > On 2021-May-24, Noah Misch wrote: > > What if we had a standard that the step after the cancel shall send a query > > to > > the backend that just received the cancel? Something like: > > Hmm ... I don't understand why this fixes

Re: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread Bharath Rupireddy
On Fri, May 28, 2021 at 6:24 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Bharath Rupireddy > > I'm still not sure why the execution time with 0 workers (or serial > > execution or > > no parallelism involved) on my testing system is 112 sec compared to 58 sec > > on > > Hou-San's system

Re: Two patches to speed up pg_rewind.

2021-05-27 Thread Paul Guo
> On 2021/2/19, 10:33 AM, "Paul Guo" wrote: > Refactored the code a bit along with fixes. Manually tested them on centos > & Ubuntu (the later has copy_file_range()) > For the first patch, actually I have some concerns. My assumption is that > the target pg_data directory should be fsync-ed

[BUG]Update Toast data failure in logical replication

2021-05-27 Thread tanghy.f...@fujitsu.com
Hi I think I just found a bug in logical replication. Data couldn't be synchronized while updating toast data. Could anyone take a look at it? Here is the steps to proceduce the BUG: --publisher-- CREATE TABLE toasted_key ( id serial, toasted_key text PRIMARY KEY,

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Paul Guo
On Thu, May 27, 2021 at 10:22 PM Julien Rouhaud wrote: > > On Thu, May 27, 2021 at 10:05 PM Paul Guo wrote: > > > > Also note that ForwardSyncRequest() does wake up the checkpointer if > > it thinks the requests in shared memory are "too full", but does not > > wake up when the request is

Re: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread Amit Kapila
On Thu, May 27, 2021 at 7:37 PM Bharath Rupireddy wrote: > > On Thu, May 27, 2021 at 9:53 AM Bharath Rupireddy > wrote: > > > One idea to find this out could be that we have three counters for > > > each worker which counts the number of times each worker extended the > > > relation in bulk, the

Re: Race condition in recovery?

2021-05-27 Thread Tatsuro Yamada
Hi Horiguchi-san, In a project I helped with, I encountered an issue where the archive command kept failing. I thought this issue was related to the problem in this thread, so I'm sharing it here. If I should create a new thread, please let me know. * Problem - The archive_command is failed

Re: Consider parallel for lateral subqueries with limit

2021-05-27 Thread Greg Nancarrow
On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > While I haven't actually tracked down to guarantee this is handled > elsewhere, a thought experiment -- I think -- shows it must be so. > Here's why: suppose we don't have a limit here, but the query return > order is different in different

RE: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread houzj.f...@fujitsu.com
From: Tsunakawa, Takayuki/綱川 貴之 Sent: Friday, May 28, 2021 8:55 AM > To: 'Bharath Rupireddy' ; Hou, > Zhijie/侯 志杰 > Cc: Amit Kapila ; Tang, Haiying/唐 海英 > ; PostgreSQL-development > ; Zhihong Yu ; Luc > Vlaming ; Dilip Kumar ; > vignesh C > Subject: RE: Parallel Inserts in CREATE TABLE AS > >

RE: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread tsunakawa.ta...@fujitsu.com
From: Bharath Rupireddy > I'm still not sure why the execution time with 0 workers (or serial execution > or > no parallelism involved) on my testing system is 112 sec compared to 58 sec on > Hou-San's system for the same use case. Maybe the testing system I'm using > is not of the latest

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-27 Thread Peter Geoghegan
On Sun, May 23, 2021 at 11:34 PM Masahiko Sawada wrote: > I think the possible side effect of this hard-coded > BYPASS_THRESHOLD_PAGES would be that by default, bulkdelete is not > called for a long term and the index becomes bloat. What do you think of the approach taken in the attached POC

Re: [HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

2021-05-27 Thread Alvaro Herrera
On 2019-Mar-15, Etsuro Fujita wrote: > (2019/03/04 12:10), Etsuro Fujita wrote: > > (2019/03/02 3:57), Andres Freund wrote: > > > FWIW, I pushed the EPQ patch, doing this conversion blindly. It'd be > > > awesome if you'd check that it actually works... > > > > I'll start the work later this

Re: Replacing pg_depend PIN entries with a fixed range check

2021-05-27 Thread Tom Lane
I wrote: > Robert Haas writes: >> The first hunk of the patch seems to back away from the idea that the >> cutoff is 13000, but the second half of the patch says 13000 still >> matters. Not sure I understand what's going on there exactly. > Not sure exactly what you're looking at, but IIRC there

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Peter Geoghegan
On Thu, May 27, 2021 at 1:29 PM Tom Lane wrote: > Yeah. My belief here is that users might bother to change > default_toast_compression, or that we might do it for them in a few > years, but the gains from doing so are going to be only incremental. > That being the case, most DBAs will be

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 16:55:29 -0400, Robert Haas wrote: > No. You're confusing what I was saying here, in the contents of your > comments about the limitations of AES-GCM-SIV, with the discussion > with Bruce about nonce generation. Ah. I think the focus on LSNs confused me a bit. FWIW: Nist

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Justin Pryzby
On Tue, May 25, 2021 at 08:33:47PM -0500, Justin Pryzby wrote: > On Sun, May 23, 2021 at 12:25:10PM -0400, Tom Lane wrote: > > However, the more I looked at that code the less I liked it. > > I think the way that compression selection is handled for indexes, > > ie consult

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, May 27, 2021 at 04:09:13PM -0400, Stephen Frost wrote: > > The above article, at least, suggested encrypting the sector number > > using the second key and then multiplying that times 2^(block number), > > where those blocks were

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 04:09:13PM -0400, Stephen Frost wrote: > The above article, at least, suggested encrypting the sector number > using the second key and then multiplying that times 2^(block number), > where those blocks were actually AES 128bit blocks. The article further > claims that

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 4:04 PM Andres Freund wrote: > On 2021-05-27 15:48:09 -0400, Robert Haas wrote: > > That said, I want to mention a point which I think may be relevant > > here. As far as I know, in the case of a permanent table page, we > > never write X then X' then X again. > > Well,

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-05-27 16:09:13 -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2021-05-27 15:22:21 -0400, Stephen Frost wrote: > > > > I'm also not sure how much of the effort would really be duplicated. > > > >

Re: Support for NSS as a libpq TLS backend

2021-05-27 Thread Daniel Gustafsson
> On 25 Mar 2021, at 00:56, Jacob Champion wrote: > Databases that are opened *after* the first one are given their own separate > slots. Any certificates that are part of those databases seemingly can't be > referenced directly by nickname. They have to be prefixed by their token name > -- a

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Tom Lane
Peter Geoghegan writes: > On Thu, May 27, 2021 at 7:25 AM Robert Haas wrote: >> To say that nobody cares about that is to deem the >> feature useless. Maybe that's what Tom thinks, but it's not what I >> think. > I don't think that follows at all. Yeah. My belief here is that users might

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 16:13:44 -0400, Alvaro Herrera wrote: > Maybe that problem could be solved by having PageRepairFragmentation, > compactify_tuples et al always fill the hole with zeroes, in encrypted > databases. If that were the only issue, maybe. But there's plenty other places were similar

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 16:09:13 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2021-05-27 15:22:21 -0400, Stephen Frost wrote: > > > I'm also not sure how much of the effort would really be duplicated. > > > > > > Were we to start with XTS, that's almost drop-in with

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Tom Lane
Alvaro Herrera writes: > Now about the former. If we do think that recompressing causes an > unacceptable 10% slowdown for every single VACUUM FULLs, then yeah we > should discuss changing that behavior -- maybe remove promises of > recompression and wait for pg15 to add "VACUUM (RECOMPRESS)" or

Re: storing an explicit nonce

2021-05-27 Thread Alvaro Herrera
On 2021-May-27, Andres Freund wrote: > On 2021-05-27 15:48:09 -0400, Robert Haas wrote: > > Another case where this sort of thing might happen is a standby doing > > whatever the master did. I suppose that could be avoided if the > > standby always has its own encryption keys, but that forces a

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Tom Lane
Alvaro Herrera writes: > Sorry, I'm unclear on exactly what are you proposing. Are you proposing > to rip out the fact that VACUUM FULL promises to recompress everything, > or are you proposing to rip out the whole attcompression feature? Just the former. regards, tom

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-05-27 15:22:21 -0400, Stephen Frost wrote: > > I'm also not sure how much of the effort would really be duplicated. > > > > Were we to start with XTS, that's almost drop-in with what Bruce has > > (actually, it should simplify some

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 15:48:09 -0400, Robert Haas wrote: > That said, I want to mention a point which I think may be relevant > here. As far as I know, in the case of a permanent table page, we > never write X then X' then X again. Well, there's crash recovery / restarts. And as previously explained

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Peter Geoghegan
On Thu, May 27, 2021 at 7:25 AM Robert Haas wrote: > TBH, I'm more concerned about the other direction. Surely someone who > upgrades from an existing release to v14 and sets their compression > method to lz4 is going to want a way of actually converting their data > to using lz4. Your argument

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 3:22 PM Stephen Frost wrote: > Trying to break it down, the end-goal states look like: > > GCM-only: no binary upgrade path due to having to store the tag > XTS-only: no data integrity option > GCM+XTS: binary upgrade path for XTS, data integrity with GCM > > If we want

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 15:22:21 -0400, Stephen Frost wrote: > I'm also not sure how much of the effort would really be duplicated. > > Were we to start with XTS, that's almost drop-in with what Bruce has > (actually, it should simplify some parts since we no longer need to deal > with making sure we

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 1:07 PM Andres Freund wrote: > But won't we then end up with a different set of requirements around > nonce assignment durability when introducing GCM support? That's not > actually entirely trivial to do correctly on a standby. I guess we can > use AES-GCM-SSIV and be ok

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Alvaro Herrera
On 2021-May-27, Tom Lane wrote: > I say it's time to stop the bleeding and rip it out. When and if > there are actual field requests to have a way to do this, we can > discuss what's the best way to respond to those requests. Hacking > VACUUM probably isn't the best answer, anyway. But right

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-05-27 13:26:11 -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2021-05-27 12:49:15 -0400, Stephen Frost wrote: > > > > That's not really a reason to rule it out though and Bruce's point about > >

Re: Race condition in recovery?

2021-05-27 Thread Robert Haas
On Wed, May 26, 2021 at 8:49 PM Kyotaro Horiguchi wrote: > So in the mail [1] and [2] I tried to describe what's going on around > the two issues. Although I haven't have a response to [2], can I > think that we clarified the intention of ee994272ca? And may I think > that we decided that we

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 13:26:11 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2021-05-27 12:49:15 -0400, Stephen Frost wrote: > > > That's not really a reason to rule it out though and Bruce's point about > > > having a way to get to an encrypted cluster from an

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 10:39 AM Tom Lane wrote: > What I'm hearing is a whole lot of hypothesizing and zero evidence of > actual field requirements. On the other side of the coin, we've already > wasted significant person-hours on fixing this feature's memory leakage, > and now people are

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 12:49 PM Stephen Frost wrote: > Right, if we simply can't solve the nonce-reuse concern then that would > be better. Given the issues that Andres raised about standbys and the treatment of the "hole," I see using the LSN for the nonce as a dead-end. I think it's pretty

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 12:00:03 -0400, Bruce Momjian wrote: > On Wed, May 26, 2021 at 05:02:01PM -0400, Bruce Momjian wrote: > > Rather than surprise anyone, I might as well just come out and say some > > things. First, I have always admitted this feature has limited > > usefulness. > > > > I

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 12:15 PM Bruce Momjian wrote: > > Well, in the design where the nonce is stored in the page, there is no > > need for every hint-type change to appear in the WAL at all. Once per > > checkpoint cycle, you need to write a full page image, as we do for > > checksums or

Re: Reducing the range of OIDs consumed by genbki.pl

2021-05-27 Thread Tom Lane
John Naylor writes: > The patch looks good to me. Thanks for reviewing! regards, tom lane

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-05-27 12:49:15 -0400, Stephen Frost wrote: > > That's not really a reason to rule it out though and Bruce's point about > > having a way to get to an encrypted cluster from an unencrypted one is > > certainly worth consideration.

Re: Reducing the range of OIDs consumed by genbki.pl

2021-05-27 Thread John Naylor
On Wed, May 26, 2021 at 5:43 PM Tom Lane wrote: > > The attached patch stems from the conversation at [1]; > I'm starting a new thread to avoid confusing the cfbot. The patch looks good to me. -- John Naylor EDB: http://www.enterprisedb.com

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 12:49:15 -0400, Stephen Frost wrote: > That's not really a reason to rule it out though and Bruce's point about > having a way to get to an encrypted cluster from an unencrypted one is > certainly worth consideration. Naturally, we'd need to document > everything appropriately

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-05-27 Thread Bharath Rupireddy
On Thu, May 27, 2021 at 8:36 PM vignesh C wrote: > > On Wed, May 26, 2021 at 7:44 PM Bharath Rupireddy > wrote: > > > > On Wed, May 26, 2021 at 7:18 PM vignesh C wrote: > > > +1 for fixing this issue, we have handled this error in other places. > > > The patch does not apply on head, could you

Re: Incorrect snapshots while promoting hot standby node when 2PC is used

2021-05-27 Thread Andres Freund
Hi, On 2021-05-26 16:57:31 +0900, Michael Paquier wrote: > Yes, there should not be any as far as I recall. 2PC is kind of > special with its fake ProcArray entries. It's really quite an awful design :( > > I think to fix the issue we'd have to move > >

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-05-27 Thread Bharath Rupireddy
On Thu, May 27, 2021 at 9:02 PM vignesh C wrote: > > Do you say that we replace table_open in publication_add_relation with > > relation_open and have the "\"%s\" is an index" or "\"%s\" is a > > composite type" checks in check_publication_add_relation? If that is > > so, I don't think it's a

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-05-27 12:28:39 -0400, Robert Haas wrote: > > All that having been said, I am pretty sure I don't fully understand > > what any of these modes involve. I gather that XTS requires two keys, > > but it seems like it doesn't require a

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 12:31 PM Andres Freund wrote: > What does that protect against that I was concerned about? That still > allows hint bits to be leaked, via > > 1) replay WAL record with FPI > 2) hint bit change during read > 3) incremental page change > > vs 1) 3). Even if we declare that

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 12:28:39PM -0400, Robert Haas wrote: > On Thu, May 27, 2021 at 12:01 PM Andres Freund wrote: > > What prevents us from using something like XTS? I'm not saying that that > > is the right approach, due to the fact that it leaks information about a > > block being the same

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 12:28:39 -0400, Robert Haas wrote: > All that having been said, I am pretty sure I don't fully understand > what any of these modes involve. I gather that XTS requires two keys, > but it seems like it doesn't require a nonce. It needs a second secret, but that second secret can

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 10:47:13 -0400, Robert Haas wrote: > Now, another option is to do what you suggest here. We could say that > if a dirty page is evicted, but the page is only dirty because of > hint-type changes, we don't actually write it out. That does avoid > using the same nonce for multiple

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 12:01 PM Andres Freund wrote: > What prevents us from using something like XTS? I'm not saying that that > is the right approach, due to the fact that it leaks information about a > block being the same as an earlier version of the same block. But right > now we are

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 10:57:24 -0400, Bruce Momjian wrote: > On Wed, May 26, 2021 at 04:26:01PM -0700, Andres Freund wrote: > > I suspect that if we try to not disclose data if an attacker has write > > access, this still leaves us with issues around nonce reuse, unless we > > also employ integrity

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 08:34:51AM -0700, Andres Freund wrote: > Hi, > > On Thu, May 27, 2021, at 08:10, Bruce Momjian wrote: > > On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2021-05-25 17:12:05 -0400, Bruce Momjian wrote: > > > > If we used a block

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Wed, May 26, 2021 at 05:02:01PM -0400, Bruce Momjian wrote: > Rather than surprise anyone, I might as well just come out and say some > things. First, I have always admitted this feature has limited > usefulness. > > I think a non-LSN nonce adds a lot of code complexity, which adds a code >

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 11:10:00 -0400, Bruce Momjian wrote: > On Wed, May 26, 2021 at 04:46:29PM -0700, Andres Freund wrote: > > On 2021-05-25 22:23:46 -0400, Stephen Frost wrote: > > > Andres mentioned other possible cases where the LSN doesn’t change even > > > though we change the page and, as he’s

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 12:03:00PM -0400, Robert Haas wrote: > On Thu, May 27, 2021 at 11:19 AM Bruce Momjian wrote: > > I was asking how decoupling the nonce from the LSN allows for us to > > avoid full page writes for hint bit changes. I am guessing you are > > saying that on recovery, if we

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 12:01:16 -0400, Bruce Momjian wrote: > On Thu, May 27, 2021 at 08:34:51AM -0700, Andres Freund wrote: > > On Thu, May 27, 2021, at 08:10, Bruce Momjian wrote: > > > On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > > > > On 2021-05-25 17:12:05 -0400, Bruce Momjian

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 11:19 AM Bruce Momjian wrote: > On Thu, May 27, 2021 at 10:47:13AM -0400, Robert Haas wrote: > > On Wed, May 26, 2021 at 4:40 PM Bruce Momjian wrote: > > > You are saying that by using a non-LSN nonce, you can write out the page > > > with a new nonce, but the same LSN,

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On 2021-05-27 11:49:33 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On Thu, May 27, 2021, at 08:10, Bruce Momjian wrote: > > > On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > > > > On 2021-05-25 17:12:05 -0400, Bruce Momjian wrote: > > > > >

Re: storing an explicit nonce

2021-05-27 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On Thu, May 27, 2021, at 08:10, Bruce Momjian wrote: > > On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > > > On 2021-05-25 17:12:05 -0400, Bruce Momjian wrote: > > > > If we used a block cipher instead of a streaming one

Re: storing an explicit nonce

2021-05-27 Thread Andres Freund
Hi, On Thu, May 27, 2021, at 08:10, Bruce Momjian wrote: > On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > > Hi, > > > > On 2021-05-25 17:12:05 -0400, Bruce Momjian wrote: > > > If we used a block cipher instead of a streaming one (CTR), this might > > > not work because the

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-05-27 Thread vignesh C
On Wed, May 26, 2021 at 7:55 PM Bharath Rupireddy wrote: > > On Wed, May 26, 2021 at 7:38 PM vignesh C wrote: > > > Attaching v5 patch, please have a look. > > > > We get the following error while adding an index: > > create publication mypub for table idx_t1; > > ERROR: "idx_t1" is an index >

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 10:47:13AM -0400, Robert Haas wrote: > On Wed, May 26, 2021 at 4:40 PM Bruce Momjian wrote: > > You are saying that by using a non-LSN nonce, you can write out the page > > with a new nonce, but the same LSN, and also discard the page during > > crash recovery and use the

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Thu, May 27, 2021 at 05:45:21PM +0800, Neil Chen wrote: > Greetings, > > On Thu, May 27, 2021 at 4:52 PM Bruce Momjian wrote: > > > > > I am confused why checksums, which are widely used, acceptably require > > wal_log_hints, but there is concern that file encryption, which is >

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Wed, May 26, 2021 at 05:11:24PM -0700, Andres Freund wrote: > Hi, > > On 2021-05-25 17:12:05 -0400, Bruce Momjian wrote: > > If we used a block cipher instead of a streaming one (CTR), this might > > not work because the earlier blocks can be based in the output of > > later blocks. > > What

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Wed, May 26, 2021 at 04:46:29PM -0700, Andres Freund wrote: > Hi, > > On 2021-05-25 22:23:46 -0400, Stephen Frost wrote: > > Andres mentioned other possible cases where the LSN doesn’t change even > > though we change the page and, as he’s probably right, we would have to > > figure out a

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-05-27 Thread vignesh C
On Wed, May 26, 2021 at 7:44 PM Bharath Rupireddy wrote: > > On Wed, May 26, 2021 at 7:18 PM vignesh C wrote: > > +1 for fixing this issue, we have handled this error in other places. > > The patch does not apply on head, could you rebase the patch on head > > and post a new patch. > > Thanks. I

Re: storing an explicit nonce

2021-05-27 Thread Bruce Momjian
On Wed, May 26, 2021 at 04:26:01PM -0700, Andres Freund wrote: > Hi, > > On 2021-05-26 07:14:47 +0200, Antonin Houska wrote: > > Bruce Momjian wrote: > > > On Tue, May 25, 2021 at 04:48:21PM -0700, Andres Freund wrote: > > > My point was about whether we need to change the nonce, and hence > > >

Re: storing an explicit nonce

2021-05-27 Thread Robert Haas
On Wed, May 26, 2021 at 4:40 PM Bruce Momjian wrote: > You are saying that by using a non-LSN nonce, you can write out the page > with a new nonce, but the same LSN, and also discard the page during > crash recovery and use the WAL copy? I don't know what "discard the page during crash recovery

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Tom Lane
Robert Haas writes: > On Thu, May 27, 2021 at 10:18 AM Dilip Kumar wrote: >>> [ shrug... ] I think the history of the SET STORAGE option teaches us >>> that there is no such requirement, and you're inventing a scenario that >>> doesn't exist in the real world. >> But can we compare SET STORAGE

Re: Reducing the range of OIDs consumed by genbki.pl

2021-05-27 Thread Robert Haas
On Wed, May 26, 2021 at 5:43 PM Tom Lane wrote: > So I propose shoehorning this into v14, to avoid shipping a > release where FirstBootstrapObjectId has been bumped up. Just to repeat on this thread what I said on the other one, I am +1 on this as a concept. I have not reviewed the patch.

Re: Speed up pg_checksums in cases where checksum already set

2021-05-27 Thread Greg Sabino Mullane
Thanks for the quick replies, everyone. On Wed, May 26, 2021 at 10:17 PM Michael Paquier wrote: > > -if (do_sync) > +if (do_sync && total_files_modified) > Here, I am on the edge. It could be an advantage to force a flush of > the data folder anyway, no? I was originally on the fence

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 10:18 AM Dilip Kumar wrote: > > [ shrug... ] I think the history of the SET STORAGE option teaches us > > that there is no such requirement, and you're inventing a scenario that > > doesn't exist in the real world. > > But can we compare SET STORAGE with SET compression?

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Julien Rouhaud
On Thu, May 27, 2021 at 10:05 PM Paul Guo wrote: > > Also note that ForwardSyncRequest() does wake up the checkpointer if > it thinks the requests in shared memory are "too full", but does not > wake up when the request is actually full. This does not seem to be > reasonable. > See below code in

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Paul Guo
> You said "trigger a checkpoint", which sounded more like forcing a > checkpointer rather than waking up the checkpointer so that it can > absorb the pending requests, so it seems worth to mention what it > would really do. Yes it is not accurate. Thanks for the clarification. -- Paul Guo

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Julien Rouhaud
On Thu, May 27, 2021 at 9:59 PM Paul Guo wrote: > > > It seems like a really unlikely scenario, but maybe possible if you > > use a lot of unlogged tables maybe (as you could eventually > > dirty/evict more than NBuffers buffers without triggering enough WALs > > activity to trigger a checkpoint

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Dilip Kumar
On Thu, May 27, 2021 at 7:04 PM Tom Lane wrote: > > Robert Haas writes: > > On Thu, May 27, 2021 at 12:11 AM Tom Lane wrote: > >> AFAIR, there are zero promises about how effective, or when effective, > >> changes in SET STORAGE will be. And the number of complaints about > >> that has also

Re: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread Bharath Rupireddy
On Thu, May 27, 2021 at 9:53 AM Bharath Rupireddy wrote: > > One idea to find this out could be that we have three counters for > > each worker which counts the number of times each worker extended the > > relation in bulk, the number of times each worker extended the > > relation by one block,

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Paul Guo
On Thu, May 27, 2021 at 9:59 PM Paul Guo wrote: > > On Thu, May 27, 2021 at 7:11 PM Julien Rouhaud wrote: > > > > On Tue, May 25, 2021 at 4:39 PM Paul Guo wrote: > > > > > > Hi hackers, > > > > > > I found this when reading the related code. Here is the scenario: > > > > > > bool > > >

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Paul Guo
On Thu, May 27, 2021 at 7:11 PM Julien Rouhaud wrote: > > On Tue, May 25, 2021 at 4:39 PM Paul Guo wrote: > > > > Hi hackers, > > > > I found this when reading the related code. Here is the scenario: > > > > bool > > RegisterSyncRequest(const FileTag *ftag, SyncRequestType type, > >

Re: pg_rewind fails if there is a read only file.

2021-05-27 Thread Paul Guo
On Wed, May 26, 2021 at 10:32 PM Andrew Dunstan wrote: > > > On 5/26/21 2:43 AM, Laurenz Albe wrote: > > On Wed, 2021-05-26 at 08:57 +0900, Michael Paquier wrote: > >> On Tue, May 25, 2021 at 03:17:37PM -0400, Andrew Dunstan wrote: > >>> If we do decide to do something the question arises what

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Tom Lane
Robert Haas writes: > On Thu, May 27, 2021 at 12:11 AM Tom Lane wrote: >> AFAIR, there are zero promises about how effective, or when effective, >> changes in SET STORAGE will be. And the number of complaints about >> that has also been zero. So I'm not sure why we need to do more for >> SET

Re: Bracket, brace, parenthesis

2021-05-27 Thread Michael Paquier
On Thu, May 27, 2021 at 03:20:10PM +0900, Kyotaro Horiguchi wrote: > At Fri, 14 May 2021 10:04:57 -0400, Tom Lane wrote in >> +1. I tend to write "square bracket" or "curly brace" when I want to >> be extra clear, but I think the bare terms are widely understood to >> have those meanings. > >

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Robert Haas
On Thu, May 27, 2021 at 12:11 AM Tom Lane wrote: > AFAIR, there are zero promises about how effective, or when effective, > changes in SET STORAGE will be. And the number of complaints about > that has also been zero. So I'm not sure why we need to do more for > SET COMPRESSION. Especially

Re: Add PortalDrop in exec_execute_message

2021-05-27 Thread Yura Sokolov
Tom Lane wrote 2021-05-27 00:19: Alvaro Herrera writes: (I didn't add a Close Portal message to PQsendQueryInternal in pipeline mode precisely because there is no such message in PQsendQueryGuts. I think it would be wrong to unconditionally add a Close Portal message to any of those

Re: Add PortalDrop in exec_execute_message

2021-05-27 Thread Yura Sokolov
Alvaro Herrera писал 2021-05-26 23:59: On 2021-May-25, Yura Sokolov wrote: Tom Lane писал 2021-05-21 21:23: > Yura Sokolov writes: > > I propose to add PortalDrop at the 'if (completed)' branch of > > exec_execute_message. > > This violates our wire protocol specification, which >

security_definer_search_path GUC

2021-05-27 Thread Marko Tiikkaja
Hi, Since writing SECURITY DEFINER functions securely requires annoying incantations[1], wouldn't it be nice if we provided a way for the superuser to override the default search path via a GUC in postgresql.conf? That way you can set search_path if you want to override the default, but if you

RE: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread houzj.f...@fujitsu.com
From: Bharath Rupireddy Sent: Thursday, May 27, 2021 3:41 PM > > On Thu, May 27, 2021 at 1:03 PM tsunakawa.ta...@fujitsu.com > wrote: > > > > From: houzj.f...@fujitsu.com > > > Although, the 4 workers case still has performance degradation > > > compared to serial case. > > > > > > SERIAL:

Re: sync request forward function ForwardSyncRequest() might hang for some time in a corner case?

2021-05-27 Thread Julien Rouhaud
On Tue, May 25, 2021 at 4:39 PM Paul Guo wrote: > > Hi hackers, > > I found this when reading the related code. Here is the scenario: > > bool > RegisterSyncRequest(const FileTag *ftag, SyncRequestType type, > bool retryOnError) > > For the case retryOnError is true, the

Re: logical replication empty transactions

2021-05-27 Thread vignesh C
On Tue, May 25, 2021 at 6:36 PM Ajin Cherian wrote: > > On Tue, Apr 27, 2021 at 1:49 PM Ajin Cherian wrote: > > Rebased the patch as it was no longer applying. Thanks for the updated patch, few comments: 1) I'm not sure if we could add some tests for skip empty transactions, if possible add a

Re: Skipping logical replication transactions on subscriber side

2021-05-27 Thread Amit Kapila
On Thu, May 27, 2021 at 12:01 PM Masahiko Sawada wrote: > > On Wed, May 26, 2021 at 6:11 PM Amit Kapila wrote: > > > > I agree with you that specifying XID could be easier and > > understandable for users. I was thinking and studying a bit about what > > other systems do in this regard. Why

Re: Skipping logical replication transactions on subscriber side

2021-05-27 Thread Amit Kapila
On Thu, May 27, 2021 at 1:46 PM Masahiko Sawada wrote: > > On Thu, May 27, 2021 at 2:48 PM Amit Kapila wrote: > > > > Okay, that makes sense but still not sure how will you identify if we > > need to reset XID in case of failure doing that in the previous > > attempt. > > It's a just idea but we

Re: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread Bharath Rupireddy
On Thu, May 27, 2021 at 2:26 PM Amit Kapila wrote: > > I think some other cause of contention on relation extension locks are > > 1. CTAS is using a buffer strategy and due to that, it might need to > > evict out the buffer frequently for getting the new block in. Maybe > > we can identify by

Re: storing an explicit nonce

2021-05-27 Thread Neil Chen
Greetings, On Thu, May 27, 2021 at 4:52 PM Bruce Momjian wrote: > > > > I am confused why checksums, which are widely used, acceptably require > > wal_log_hints, but there is concern that file encryption, which is > > heavier, cannot acceptably require wal_log_hints. I must be missing > >

Re: Parallel Inserts in CREATE TABLE AS

2021-05-27 Thread Amit Kapila
On Thu, May 27, 2021 at 10:27 AM Dilip Kumar wrote: > > On Thu, May 27, 2021 at 10:16 AM Bharath Rupireddy > wrote: > > > > On Thu, May 27, 2021 at 7:12 AM houzj.f...@fujitsu.com > > wrote: > > > I am afraid that the using the FSM seems not get a stable performance > > > gain(at least on my

Re: Skip partition tuple routing with constant partition key

2021-05-27 Thread Amit Langote
Hou-san, On Thu, May 27, 2021 at 3:56 PM houzj.f...@fujitsu.com wrote: > From: Amit Langote > Sent: Thursday, May 27, 2021 1:54 PM > > On Thu, May 27, 2021 at 11:47 AM houzj.f...@fujitsu.com > > wrote: > > > About teaching relcache about caching the target partition. > > > > > > David-san

Fix RADIUS error reporting in hba file parsing

2021-05-27 Thread Peter Eisentraut
The RADIUS-related checks in parse_hba_line() did not respect elevel and did not fill in *err_msg. Also, verify_option_list_length() pasted together error messages in an untranslatable way. To fix the latter, remove the function and do the error checking inline. It's a bit more verbose but

  1   2   >