Re: thoughts on "prevent wraparound" vacuum

2019-07-20 Thread Michail Nikolaev
Hello. >- Which version of postgres is this? Newer versions avoid scanning > unchanged parts of the heap even for freezing (9.6+, with additional > smaller improvements in 11). Oh, totally forgot about version and settings... server_version 10.9 (Ubuntu 10.9-103) So, "don't vacuum all-frozen

Re: Performance issue in foreign-key-aware join estimation

2019-07-20 Thread David Rowley
On Thu, 18 Jul 2019 at 19:24, David Rowley wrote: > Unless there's some objection, I'll be looking into pushing both 0001 > and 0002 in a single commit in the next few days. I've pushed this after doing a bit of final tweaking. After a re-read, I didn't really like all the code that rechecked

Fix typos and inconsistencies for HEAD (take 7)

2019-07-20 Thread Alexander Lakhin
Hello hackers, Please consider fixing the next pack of typos and inconsistencies in the tree: 7.1. h04m05s06 -> h04mm05s06 (in fact it's broken since 6af04882, but h04mm05s06.789 still accepted) 7.2. hasbucketcleanup -> hashbucketcleanup 7.3. _hashm_spare -> hashm_spares 7.4. hashtbl -> hash

Re: contrib make check-world fail if data have been modified and there's vpath

2019-07-20 Thread Tom Lane
didier writes: > because there's destination data > src/makefiles/pgxs.mk line > ln -s $< $@ > fails and make clean doesn't remove these links. > ln -sf > is an option but it's not tested in configure > or rm -f Can you be more specific about what the problem case is?

Re: Catching missing Datum conversions

2019-07-20 Thread Corey Huinker
> > I should probably split this into "actionable" (categories 3 and 4) > and "noise and scaffolding" patches. > Breaking down the noise-and-scaffolding into some subgroups might make the rather long patches more palatable/exceedingly-obvious: * (Datum) 0 ---> NullDatum * 0 > NullDatum * The

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread James Coleman
On Sat, Jul 20, 2019 at 1:00 PM Tomas Vondra wrote: > > On Sat, Jul 20, 2019 at 12:21:01PM -0400, James Coleman wrote: > >On Sat, Jul 20, 2019 at 12:12 PM James Coleman wrote: > >> > >> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra > >> wrote: > >> ... > >> > >My current line of investigation

Re: [PATCH v2] Introduce spgist quadtree @<(point,circle) operator

2019-07-20 Thread Alexander Korotkov
Hi Matwey, On Tue, May 21, 2019 at 10:23 AM Matwey V. Kornilov wrote: > вт, 21 мая 2019 г. в 08:43, Michael Paquier : > > > > On Mon, May 20, 2019 at 02:32:39PM +0300, Matwey V. Kornilov wrote: > > > This patch series is to add support for spgist quadtree @<(point,circle) > > > operator. The

Re: benchmarking Flex practices

2019-07-20 Thread Tom Lane
John Naylor writes: > The pre-existing ecpg var "state_before" was a bit confusing when > combined with the new var "state_before_quote_stop", and the former is > also used with C-comments, so I decided to go with > "state_before_lit_start" and "state_before_lit_stop". Even though > comments

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Andres Freund
Hi, On 2019-07-20 12:58:55 -0700, Noah Misch wrote: > On Sat, Jul 20, 2019 at 12:56:34PM -0700, Andres Freund wrote: > > Fair enough. Are you planning to commit your changes? > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8a0cbb88524e8b6121597285b811640ee793b3e8 Oh,

Re: thoughts on "prevent wraparound" vacuum

2019-07-20 Thread Andres Freund
Hi, On 2019-07-20 15:35:57 +0300, Michail Nikolaev wrote: > Currently I am working a lot with cluster consist a few of big tables. > About 2-3 TB. These tables are heavily updated, some rows are removed, new > rows are inserted... Kind of typical OLTP workload. > > Physical table size keeps

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Noah Misch
On Sat, Jul 20, 2019 at 12:56:34PM -0700, Andres Freund wrote: > On 2019-07-20 11:21:52 -0700, Noah Misch wrote: > > On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > > > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > > > +# Test concurrent OID generation via pg_enum_oid_index.

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Andres Freund
Hi, On 2019-07-20 11:21:52 -0700, Noah Misch wrote: > On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > > +# Test concurrent OID generation via pg_enum_oid_index. This indirectly > > > +# exercises LWLock and spinlock

Re: SQL/JSON path issues/questions

2019-07-20 Thread Steven Pousty
Thanks so much, hope to get to it over this weekend. On Sat, Jul 20, 2019, 11:48 AM Alexander Korotkov wrote: > Hi Steven, > > On Fri, Jul 19, 2019 at 9:53 PM Steven Pousty > wrote: > > I would like to help review this documentation. Can you please point me > in the right direction? > > Thank

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-20 Thread Sehrope Sarkuni
On Sat, Jul 20, 2019 at 1:30 PM Tomas Vondra wrote: > Forbid checksums? I don't see how that could be acceptable. We either have > to accept the limitations of the current design (having to decrypt > everything before checking the checksums) or change the design. > > I personally think we should

Re: SQL/JSON path issues/questions

2019-07-20 Thread Alexander Korotkov
Hi Steven, On Fri, Jul 19, 2019 at 9:53 PM Steven Pousty wrote: > I would like to help review this documentation. Can you please point me in > the right direction? Thank you for your interest. You're welcome to do review. Please take a look at instruction for reviewing a patch [1] and

Re: Index Skip Scan

2019-07-20 Thread Dmitry Dolgov
> On Thu, Jul 11, 2019 at 2:40 AM Floris Van Nee > wrote: > I verified that the backwards index scan is indeed functioning now. However, > I'm afraid it's not that simple, as I think the cursor case is broken now. I > think having just the 'scan direction' in the btree code is not enough to get

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Noah Misch
On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > +# Test concurrent OID generation via pg_enum_oid_index. This indirectly > > +# exercises LWLock and spinlock concurrency. > > +my $labels = join ',', map { "'l$_'" } 1 .. 1000; >

Re: block-level incremental backup

2019-07-20 Thread vignesh C
Hi Jeevan, The idea is very nice. When Insert/update/delete and truncate/drop happens at various combinations, How the incremental backup handles the copying of the blocks? On Wed, Jul 17, 2019 at 8:12 PM Jeevan Chalke wrote: > > > > On Wed, Jul 17, 2019 at 7:38 PM Ibrar Ahmed wrote: >> >> >>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-20 Thread Tomas Vondra
On Fri, Jul 19, 2019 at 04:02:19PM +0200, Antonin Houska wrote: Tomas Vondra wrote: On Fri, Jul 19, 2019 at 12:04:36PM +0200, Antonin Houska wrote: >Tomas Vondra wrote: > >> On Mon, Jul 15, 2019 at 03:42:39PM -0400, Bruce Momjian wrote: >> >On Sat, Jul 13, 2019 at 11:58:02PM +0200, Tomas

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread Tomas Vondra
On Sat, Jul 20, 2019 at 12:21:01PM -0400, James Coleman wrote: On Sat, Jul 20, 2019 at 12:12 PM James Coleman wrote: On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra wrote: ... > >My current line of investigation is whether we need to do anything in > >the parallel portion of

Re: [HACKERS] Block level parallel vacuum

2019-07-20 Thread Sergei Kornilov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Hello I reviewed v25 patches and have just a few notes. missed

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread James Coleman
On Sat, Jul 20, 2019 at 12:12 PM James Coleman wrote: > > On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra > wrote: > ... > > >My current line of investigation is whether we need to do anything in > > >the parallel portion of create_ordered_paths(). I noticed that the > > >first-pass patch adding

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread James Coleman
On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra wrote: ... > >My current line of investigation is whether we need to do anything in > >the parallel portion of create_ordered_paths(). I noticed that the > >first-pass patch adding generate_useful_gather_paths() modified that > >section but wasn't

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread Tomas Vondra
On Sat, Jul 20, 2019 at 10:33:02AM -0400, James Coleman wrote: On Sat, Jul 20, 2019 at 9:22 AM Tomas Vondra wrote: On Fri, Jul 19, 2019 at 04:59:21PM -0400, James Coleman wrote: >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra > wrote: >> Now, consider this example: >> >> create table t (a int,

Re: [sqlsmith] Crash in mcv_get_match_bitmap

2019-07-20 Thread Tomas Vondra
On Fri, Jul 19, 2019 at 02:37:19PM -0400, Tom Lane wrote: Tomas Vondra writes: [ mcv fixes ] These patches look OK to me. OK, thanks. Pushed. -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-07-20 Thread Tomas Vondra
On Fri, Jul 19, 2019 at 04:59:21PM -0400, James Coleman wrote: On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra wrote: Now, consider this example: create table t (a int, b int, c int); insert into t select mod(i,100),mod(i,100),i from generate_series(1,1000) s(i); create index on t (a);

thoughts on "prevent wraparound" vacuum

2019-07-20 Thread Michail Nikolaev
Hello. Currently I am working a lot with cluster consist a few of big tables. About 2-3 TB. These tables are heavily updated, some rows are removed, new rows are inserted... Kind of typical OLTP workload. Physical table size keeps mostly stable while regular VACUUM is working. It is fast enough

Re: Compiler warnings with MinGW

2019-07-20 Thread Michael Paquier
On Fri, Jul 19, 2019 at 08:41:28AM -0400, Andrew Dunstan wrote: > On 7/19/19 1:08 AM, Michael Paquier wrote: >> The second one is rather obvious to fix, because we don't care about >> the file mode on Windows, so the attached should do the work. I am >> actually surprised that the Visual Studio

Re: pgsql: Sync our copy of the timezone library with IANA release tzcode20

2019-07-20 Thread Michael Paquier
On Fri, Jul 19, 2019 at 02:56:34PM -0400, Tom Lane wrote: > I'm not quite convinced whether that will silence the warning, but > at least it's a bit less unreadable. Thanks for working with upstream on this. From what I can see, woodlouse & friends do not complain anymore. -- Michael

Re: POC: Cleaning up orphaned files using undo logs

2019-07-20 Thread Dilip Kumar
On Sat, Jul 20, 2019 at 12:40 PM Amit Kapila wrote: > > On Fri, Jul 19, 2019 at 6:35 PM Robert Haas wrote: > > > > On Fri, Jul 19, 2019 at 12:10 AM Amit Kapila > > wrote: > > > We are doing exactly what you have written in the last line of the > > > next paragraph "stop the transaction from

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2019-07-20 Thread Dent John
Hi Nikolay, Thanks for the revised patch. It applies now no problem, and seems to work fine. For me, I still find the relopts area quite odd. I wonder if your patch doesn’t go far enough? For example, take log_autovacuum_min_duration. It’s described intRelOpts, which implicitly defines its

Re: POC: Cleaning up orphaned files using undo logs

2019-07-20 Thread Amit Kapila
On Fri, Jul 19, 2019 at 6:35 PM Robert Haas wrote: > > On Fri, Jul 19, 2019 at 12:10 AM Amit Kapila wrote: > > We are doing exactly what you have written in the last line of the > > next paragraph "stop the transaction from writing undo when the hash > > table is already too full.". So we will