Re: [HACKERS] pgbench - allow to store select results into variables

2018-04-07 Thread Fabien COELHO
Hello Stephen, Might be interesting to support mutli-row (or no row?) returns in the future, but not something this patch needs to do, so this looks fine to me. It could match PL/pgSQL's INTO, that is first row or NULL if none. Yeah, but that's not really something that needs to go into

Fix slot's xmin advancement and subxact's lost snapshots in decoding.

2018-04-07 Thread Arseny Sher
Hello, I've discovered a couple of bugs in logical decoding code, both leading to incorrect decoding results in somewhat rare cases. First, xmin of slots is advanced too early. This affects the results only when interlocking allows to perform DDL concurrently with looking at the schema. In fact,

Re: Checkpoint not retrying failed fsync?

2018-04-07 Thread Amit Kapila
On Fri, Apr 6, 2018 at 6:26 AM, Thomas Munro wrote: > On Fri, Apr 6, 2018 at 11:36 AM, Thomas Munro > wrote: >> On Fri, Apr 6, 2018 at 11:34 AM, Andrew Gierth >> wrote: >>> Right. >>> >>> But I don't

Re: lazy detoasting

2018-04-07 Thread Amit Kapila
On Thu, Apr 5, 2018 at 8:04 AM, Chapman Flack wrote: > On 04/01/18 01:19, Tom Lane wrote: >> Chapman Flack writes: >>> If I copy an out-of-line, on-disk TOAST pointer into a memory context >>> with transaction lifetime, with an eye to detoasting it

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Christophe Pettus
> On Apr 7, 2018, at 20:27, Craig Ringer wrote: > > Right now I think we're at option (4): If you see anything that smells like a > write error in your kernel logs, hard-kill postgres with -m immediate (do NOT > let it do a shutdown checkpoint). If it did a checkpoint

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Peter Geoghegan
On Sat, Apr 7, 2018 at 8:27 PM, Craig Ringer wrote: > More below, but here's an idea #5: decide InnoDB has the right idea, and go > to using a single massive blob file, or a few giant blobs. > > We have a storage abstraction that makes this way, way less painful than it >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Craig Ringer
On 8 April 2018 at 10:16, Thomas Munro wrote: > So, what can we actually do about this new Linux behaviour? > Yeah, I've been cooking over that myself. More below, but here's an idea #5: decide InnoDB has the right idea, and go to using a single massive blob

Re: pgsql: Support partition pruning at execution time

2018-04-07 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Support partition pruning at execution time > > Buildfarm member lapwing doesn't like this. I can reproduce the > failures here by setting force_parallel_mode = regress. Kind > of looks like instrumentation counts aren't

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Christophe Pettus
> On Apr 7, 2018, at 19:33, Bruce Momjian wrote: > Idea 4 would be for people to assume their database is corrupt if their > server logs report any I/O error on the file systems Postgres uses. Pragmatically, that's where we are right now. The best answer in this bad

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Bruce Momjian
On Sun, Apr 8, 2018 at 02:16:07PM +1200, Thomas Munro wrote: > So, what can we actually do about this new Linux behaviour? > > Idea 1: > > * whenever you open a file, either tell the checkpointer so it can > open it too (and wait for it to tell you that it has done so, because > it's not safe

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-07 Thread Thomas Munro
So, what can we actually do about this new Linux behaviour? Idea 1: * whenever you open a file, either tell the checkpointer so it can open it too (and wait for it to tell you that it has done so, because it's not safe to write() until then), or send it a copy of the file descriptor via IPC

Re: PATCH: psql tab completion for SELECT

2018-04-07 Thread Edmund Horner
On 6 April 2018 at 13:29, Peter Eisentraut wrote: > The selection of which functions to show can be further refined. > > I don't think the contents of pg_amproc and pg_cast should be excluded. > Some of those functions are useful. Similarly for

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 14:23:38 -0400, Tom Lane wrote: > I think I'd just drop those asserts altogether. The hardware is in charge > of complaining about misaligned pointers. I came around to that view. The problem with "natural" and atomic alignment differing is only relevant for 64bit integers (4

Re: [HACKERS] pgbench - allow to store select results into variables

2018-04-07 Thread Stephen Frost
Fabien, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > >> > >>+ > >>+ > >>+ \cset [prefix] or > >>+ \gset [prefix] > >>+ > > > >Seems like this should really be moved down below the section for > >'\set'. > > Dunno. > > I put them there because it is in alphabetical order

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-04-07 Thread Peter Eisentraut
This part of the patch didn't end up being necessary, since the updated implementation of logical decoding of TRUNCATE could do without it. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-07 Thread Peter Eisentraut
On 4/5/18 13:07, Alvaro Herrera wrote: > Note that you start the loop having the Relation; yet you go extra > length to grab the relnamespace and relname from syscache instead of > just relations[i]->rd_rel->relname etc. fixed > Maybe not a big deal, but for future pg_waldump users I'm sure

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > (If not, this bodes ill for the Windows results.) Ah, there go the Windows builds... Missing a #else clause where we should have one for those systems, will fix. Thanks! Stephen signature.asc Description: PGP signature

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2018-04-07%2021%3A50%3A02 > > > Yes, I'm taking a look at it. > > Since other animals

Searching for: Fast windows buildfarm animal

2018-04-07 Thread Andres Freund
Hi, It's common that half the buildfarm has reported back before a single windows buildfarm animal reports. And if they report a failure one often has to wait for hours for the next run. It'd be awesome if somebody could set up a windows animal that runs frequently (i.e. checks for build needed

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2018-04-07%2021%3A50%3A02 > Yes, I'm taking a look at it. Since other animals are coming back successfully, my first suspicion lights on

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Time to watch the buildfarm, > > That didn't take long. > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2018-04-07%2021%3A50%3A02 Yes, I'm taking a look at it. Thanks! Stephen

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Tom Lane
Stephen Frost writes: > Time to watch the buildfarm, That didn't take long. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2018-04-07%2021%3A50%3A02 (I'm really starting to get a bit upset at the amount of stuff that's being pushed in on the very last

Re: PATCH: Configurable file mode mask

2018-04-07 Thread David Steele
On 4/7/18 5:49 PM, Stephen Frost wrote: > * David Steele (da...@pgmasters.net) wrote: >> >> OK, one last review. I did't make any code changes, but I improved some >> comments, added documentation and fixed a test. > > Thanks! I took those and then added a bit more commentary around the >

Re: PATCH: Configurable file mode mask

2018-04-07 Thread Stephen Frost
David, * David Steele (da...@pgmasters.net) wrote: > On 4/6/18 10:22 PM, Stephen Frost wrote: > > * David Steele (da...@pgmasters.net) wrote: > >> On 4/6/18 6:04 PM, David Steele wrote: > >>> On 4/6/18 3:02 PM, Stephen Frost wrote: > > - Further discussion in the commit messages > >>> >

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread David Rowley
On 8 April 2018 at 09:13, Alvaro Herrera wrote: > I pushed this patch -- 0001, 0002 and 0003 only. I did not include > anything from 0004 and 0005; I didn't even get to the point of reading > them, so that I could focus on the first part. Oh great! Thank you for working

Re: Removing useless DISTINCT clauses

2018-04-07 Thread David Rowley
On 8 April 2018 at 08:55, Tom Lane wrote: > regression=# create table t1 (a int,b int, c int, d int, primary key(a,b)); > CREATE TABLE > regression=# explain verbose select distinct * from t1 order by a,c,b; > server closed the connection unexpectedly > This probably

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Alvaro Herrera
I pushed this patch -- 0001, 0002 and 0003 only. I did not include anything from 0004 and 0005; I didn't even get to the point of reading them, so that I could focus on the first part. I did not find anything to complain about. I made a few adjustments and asked David to supply a paragraph for

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Andrew Gierth
> "Teodor" == Teodor Sigaev writes: >> Support for testing amcaninclude via >> pg_indexam_has_property(oid,'can_include') seems to be missing? >> >> Also the return values of pg_index_column_has_property for included >> columns seem a bit dubious... should probably be

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Teodor Sigaev
Support for testing amcaninclude via pg_indexam_has_property(oid,'can_include') seems to be missing? Also the return values of pg_index_column_has_property for included columns seem a bit dubious... should probably be returning NULL for most properties except 'returnable'. Damn, you right, it's

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Tomas Vondra
On 04/07/2018 06:23 PM, Tom Lane wrote: > Teodor Sigaev writes: >>> I dunno, how would you estimate whether this is actually a win or not? >>> I don't think our model of sort costs is anywhere near refined enough >>> or accurate enough to reliably predict whether this is better

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Peter Geoghegan
On Sat, Apr 7, 2018 at 1:52 PM, Andrew Gierth wrote: > Support for testing amcaninclude via > pg_indexam_has_property(oid,'can_include') seems to be missing? > > Also the return values of pg_index_column_has_property for included > columns seem a bit dubious... should

Re: Removing useless DISTINCT clauses

2018-04-07 Thread Tom Lane
I wrote: > My gripe about > this as it stands is mainly that it seems like it's going to do > a lot of catalog-lookup work twice over, at least in cases that > have both DISTINCT and GROUP BY --- or is that too small a set to > worry about? I convinced myself that that was a silly objection, and

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Andrew Gierth
> "Teodor" == Teodor Sigaev writes: >> I'll keep an eye on the buildfarm, since it's late in Russia. Teodor> Thank you very much! Now 23:10 MSK and I'll be able to follow Teodor> during approximately hour. Support for testing amcaninclude via

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Teodor Sigaev
Thank you, I looked to buildfarm and completely forget about commitfest site Andres Freund wrote: On 2018-04-07 23:02:08 +0300, Teodor Sigaev wrote: Thanks to everyone, pushed. Marked CF entry as committed. Greetings, Andres Freund -- Teodor Sigaev E-mail:

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-04-07 Thread Tomas Vondra
On 04/07/2018 07:52 PM, Dean Rasheed wrote: > On 7 April 2018 at 15:12, Bruce Momjian wrote: >> Uh, where are we on this patch? It isn't going to make it into PG 11? >> Feature development for this has been going on for years. > > Unfortunately, I think there's no way that

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-07 Thread Amit Langote
On Sun, Apr 8, 2018 at 5:37 AM, Andres Freund wrote: > On 2018-04-06 19:25:20 -0400, Robert Haas wrote: >> On Thu, Apr 5, 2018 at 6:21 AM, Etsuro Fujita >> wrote: >> > Attached is an updated version of the patch set plus the patch in [1]. >> >

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-07 Thread Andres Freund
On 2018-04-06 19:25:20 -0400, Robert Haas wrote: > On Thu, Apr 5, 2018 at 6:21 AM, Etsuro Fujita > wrote: > > Attached is an updated version of the patch set plus the patch in [1]. Patch > > 0003_foreign-routing-fdwapi-6.patch can be applied on top of patch > >

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Andres Freund
On 2018-04-07 23:02:08 +0300, Teodor Sigaev wrote: > Thanks to everyone, pushed. Marked CF entry as committed. Greetings, Andres Freund

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-07 Thread Andres Freund
On 2018-04-07 20:13:36 +0530, amul sul wrote: > Attached is the patch does the renaming of this tests -- need to apply > to the top of v10 patch[1]. These indeed are a bit too long, so I went with the numbers. I've pushed the patch now. Two changes: - I've added one more error patch to

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Teodor Sigaev
I'll keep an eye on the buildfarm, since it's late in Russia. Thank you very much! Now 23:10 MSK and I'll be able to follow during approximately hour. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Peter Geoghegan
On Sat, Apr 7, 2018 at 1:02 PM, Teodor Sigaev wrote: > Thanks to everyone, pushed. I'll keep an eye on the buildfarm, since it's late in Russia. -- Peter Geoghegan

Sv: Re: WIP: Covering + unique indexes.

2018-04-07 Thread Andreas Joseph Krogh
På lørdag 07. april 2018 kl. 22:02:08, skrev Teodor Sigaev >: Thanks to everyone, pushed.   Rock!   -- Andreas Joseph Krogh  

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Teodor Sigaev
Thanks to everyone, pushed. Peter Geoghegan wrote: On Sat, Apr 7, 2018 at 5:48 AM, Teodor Sigaev wrote: On close look, bts_btentry.ip_posid is not used anymore, I change bts_btentry type to BlockNumber. As result, BTEntrySame() is removed. That seems like a good idea.

Re: [HACKERS] path toward faster partition pruning

2018-04-07 Thread Alvaro Herrera
Andres Freund wrote: > On 2018-04-07 08:13:23 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > I've also attempted to fix rhinoceros's failure I remarked upon a couple > > > hours ago in > > > https://postgr.es/m/20180406210330.wmqw42wqgiick...@alap3.anarazel.de > > > > And this, too.

Re: [sqlsmith] Segfault in expand_tuple

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 21:28:39 +0200, Andreas Seltenreich wrote: > the following query triggers a segfault for me when run against the > regression database. Testing was done with master at 039eb6e92f. > Backtrace below. Andrew, that looks like it's in your area? > Core was generated by `postgres:

[sqlsmith] Segfault in expand_tuple

2018-04-07 Thread Andreas Seltenreich
Hi, the following query triggers a segfault for me when run against the regression database. Testing was done with master at 039eb6e92f. Backtrace below. regards, Andreas select case when pg_catalog.lastval() < pg_catalog.pg_stat_get_bgwriter_maxwritten_clean() then case when

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-07 Thread Andreas Seltenreich
Hi, sqlsmith triggered an assertion with the following MERGE statement against the regression database. Testing was done with master at 039eb6e92f. Backtrace below. regards, Andreas MERGE INTO public.pagg_tab_ml_p3 as target_0 USING public.hash_i4_heap as ref_0 ON target_0.b = ref_0.seqno

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-07 Thread Andreas Seltenreich
Tom Lane writes: > Alvaro Herrera writes: >> Andreas Seltenreich wrote: >>> as of 039eb6e92f: >>> TRAP: FailedAssertion("!(subpath->parallel_safe)", File: "pathnode.c", >>> Line: 1813) > >> Uh, that's pretty strange -- that patch did not touch the planner at >> all, and

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-07 Thread Tom Lane
Alvaro Herrera writes: > Andreas Seltenreich wrote: >> sqlsmith found a query that triggers the following assertion in master >> as of 039eb6e92f: >> TRAP: FailedAssertion("!(subpath->parallel_safe)", File: "pathnode.c", Line: >> 1813) > Uh, that's pretty strange --

Re: [sqlsmith] Failed assertion in create_gather_path

2018-04-07 Thread Alvaro Herrera
Andreas Seltenreich wrote: > Hi, > > sqlsmith found a query that triggers the following assertion in master > as of 039eb6e92f: > > TRAP: FailedAssertion("!(subpath->parallel_safe)", File: "pathnode.c", Line: > 1813) > > Backtrace and recipe against the regression database below. Uh, that's

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-04-07 Thread Bruce Momjian
On Sat, Apr 7, 2018 at 06:52:42PM +0100, Dean Rasheed wrote: > On 7 April 2018 at 15:12, Bruce Momjian wrote: > > Uh, where are we on this patch? It isn't going to make it into PG 11? > > Feature development for this has been going on for years. > > Unfortunately, I think

[sqlsmith] Failed assertion on pfree() via perform_pruning_combine_step

2018-04-07 Thread Andreas Seltenreich
Hi, testing with master at 039eb6e92f yielded another query triggering an assertion. Backtrace and query against the regression database below. regards, Andreas #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x7f25474cf42a in __GI_abort () at abort.c:89 #2

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 14:23:38 -0400, Tom Lane wrote: > I think I'd just drop those asserts altogether. The hardware is in charge > of complaining about misaligned pointers. Well, the problem is that some atomics operations on some platforms do not fail for unaligned pointers, they just loose their

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Tom Lane
Andres Freund writes: > On 2018-04-07 14:07:05 -0400, Tom Lane wrote: >> TRAP: UnalignedPointer("(((uintptr_t) ((uintptr_t)(ptr)) + ((sizeof(*ptr)) - >> 1)) & ~((uintptr_t) ((sizeof(*ptr)) - 1))) != (uintptr_t)(ptr)", File: >> "../../../src/include/port/atomics.h", Line:

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Andres Freund
On 2018-04-07 14:07:05 -0400, Tom Lane wrote: > Andres Freund writes: > > As Daniel pointed out in: > > https://postgr.es/m/fb948276-7b32-4b77-83e6-d00167f8e...@yesql.se the > > pg_atomic_flag fallback implementation is broken. That has gone > > unnoticed because the fallback

[sqlsmith] Failed assertion in create_gather_path

2018-04-07 Thread Andreas Seltenreich
Hi, sqlsmith found a query that triggers the following assertion in master as of 039eb6e92f: TRAP: FailedAssertion("!(subpath->parallel_safe)", File: "pathnode.c", Line: 1813) Backtrace and recipe against the regression database below. regards, Andreas #0 __GI_raise (sig=sig@entry=6) at

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Tom Lane
Andres Freund writes: > As Daniel pointed out in: > https://postgr.es/m/fb948276-7b32-4b77-83e6-d00167f8e...@yesql.se the > pg_atomic_flag fallback implementation is broken. That has gone > unnoticed because the fallback implementation wasn't testable until now: > ... > The

Re: json(b)_to_tsvector with numeric values

2018-04-07 Thread Teodor Sigaev
Thank you, pushed with some editorization Dmitry Dolgov wrote: On 7 April 2018 at 17:09, Teodor Sigaev wrote: See workable sketch for parsing jsonb flags and new worker variant. Yep, thanks for the sketch. Here is the new version of patch, does it look close to what you

Re: pgsql: New files for MERGE

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 13:33:53 -0400, Bruce Momjian wrote: > To summarize how I see this patch, we have this workflow at the top of > the TODO list (which I think Simon helped with or suggested): > > Desirability -> Design -> Implement -> Test -> Review -> Commit > > I think the MERGE patch

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-04-07 Thread Dean Rasheed
On 7 April 2018 at 15:12, Bruce Momjian wrote: > Uh, where are we on this patch? It isn't going to make it into PG 11? > Feature development for this has been going on for years. Unfortunately, I think there's no way that this will be ready for PG11. So far, I have only read

Re: pgsql: New files for MERGE

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 13:45:21 -0400, Tom Lane wrote: > Simon Riggs writes: > > On 6 April 2018 at 17:22, Bruce Momjian wrote: > >> My point was that people didn't ask you to work harder on fixing the > >> patch, but in reverting it. You can work harder on

Re: pgsql: New files for MERGE

2018-04-07 Thread Tom Lane
Simon Riggs writes: > On 6 April 2018 at 17:22, Bruce Momjian wrote: >> My point was that people didn't ask you to work harder on fixing the >> patch, but in reverting it. You can work harder on fixing things in the >> hope they change their minds, but

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Peter Geoghegan
On Sat, Apr 7, 2018 at 5:48 AM, Teodor Sigaev wrote: > On close look, bts_btentry.ip_posid is not used anymore, I change > bts_btentry type to BlockNumber. As result, BTEntrySame() is removed. That seems like a good idea. > I'm not very happy with massive usage of >

Re: pgsql: New files for MERGE

2018-04-07 Thread Bruce Momjian
On Sat, Apr 7, 2018 at 06:19:19PM +0100, Simon Riggs wrote: > Now I can see some people are annoyed, so I'm happy to apologize if > I've done things that weren't understood or caused annoyance. Time is > short at end of CF and tempers fray for us all. > > If Tom or Andres still feel that their

Re: pgsql: New files for MERGE

2018-04-07 Thread Simon Riggs
On 6 April 2018 at 17:22, Bruce Momjian wrote: > On Fri, Apr 6, 2018 at 09:21:54AM +0100, Simon Riggs wrote: >> On 5 April 2018 at 21:02, Bruce Momjian wrote: >> > Simon, you have three committers in this thread suggesting this patch be >> > reverted. Are

Re: PATCH: Configurable file mode mask

2018-04-07 Thread David Steele
Hi Stephen, On 4/6/18 10:22 PM, Stephen Frost wrote: > > * David Steele (da...@pgmasters.net) wrote: >> On 4/6/18 6:04 PM, David Steele wrote: >>> On 4/6/18 3:02 PM, Stephen Frost wrote: - Further discussion in the commit messages >>> >>> Agreed, these need some more work.  I'm happy

Re: Removing useless DISTINCT clauses

2018-04-07 Thread Tom Lane
David Rowley writes: > It's certainly possible to do more here. I'm uncertain what needs to > be done in regards to cached plan invalidation, but we'd certainly > need to ensure cached plans are invalidated whenever the attnotnull is > changed. They already are;

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Tom Lane
Teodor Sigaev writes: >> I dunno, how would you estimate whether this is actually a win or not? >> I don't think our model of sort costs is anywhere near refined enough >> or accurate enough to reliably predict whether this is better than >> just doing it in one step. Even if

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Alexander Korotkov
On Sat, Apr 7, 2018 at 5:37 PM, Teodor Sigaev wrote: > by this patch. Revised version is attached. >> > > Fine, patch got several rounds of review in all its parts. Is any places > which should be improved before commit? Also I found that after planner changes of Alexander

Re: Online enabling of checksums

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 08:57:03 +0200, Magnus Hagander wrote: > Note however that I'm sans-laptop until Sunday, so I will revert it then or > possibly Monday. I'll deactive the isolationtester tests until then. They've been intermittently broken for days now and prevent other tests from being

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Teodor Sigaev
I dunno, how would you estimate whether this is actually a win or not? I don't think our model of sort costs is anywhere near refined enough or accurate enough to reliably predict whether this is better than just doing it in one step. Even if the cost model is good, it's not going to be better

Re: Bring atomic flag fallback up to snuff

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 12:57:53 +, Magnus Hagander wrote: > This is the same one you already committed right? Not a further one on top > of that? Yes, I pushed a few hours later. > That said,+1 for not bothering to back patch it. I did ;). I was more wondering about whether to backpatch the

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Andres Freund
On 2018-04-07 12:06:52 -0400, Tom Lane wrote: > Alexander Korotkov writes: > > On Wed, Mar 28, 2018 at 6:38 PM, Alvaro Herrera > > wrote: > >> Can we have a recap on what the patch *does*? > > > Ggeneral idea hasn't been changed much since

Re: json(b)_to_tsvector with numeric values

2018-04-07 Thread Dmitry Dolgov
> On 7 April 2018 at 17:09, Teodor Sigaev wrote: >>> See workable sketch for parsing jsonb flags and new worker variant. >> >> >> Yep, thanks for the sketch. Here is the new version of patch, does it look >> close to what you have in mind? > > > Patch looks good except error

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Tom Lane
Alexander Korotkov writes: > On Wed, Mar 28, 2018 at 6:38 PM, Alvaro Herrera > wrote: >> Can we have a recap on what the patch *does*? > Ggeneral idea hasn't been changed much since first email. > Incremental sort gives benefit when you need

Re: [HACKERS] pgbench - allow to store select results into variables

2018-04-07 Thread Fabien COELHO
Hello Stephen, Here's that review. Thanks for the review. + + + \cset [prefix] or + \gset [prefix] + Seems like this should really be moved down below the section for '\set'. Dunno. I put them there because it is in alphabetical order (for cset at least) and

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Alexander Korotkov
On Wed, Mar 28, 2018 at 6:38 PM, Alvaro Herrera wrote: > Teodor Sigaev wrote: > > > BTW, patch had conflicts with master. Please, find rebased version > attached. > > > > Despite by patch conflist patch looks commitable, has anybody objections > to > > commit it? > > >

Re: [HACKERS] path toward faster partition pruning

2018-04-07 Thread Andres Freund
On 2018-04-07 08:13:23 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > I've also attempted to fix rhinoceros's failure I remarked upon a couple > > hours ago in > > https://postgr.es/m/20180406210330.wmqw42wqgiick...@alap3.anarazel.de > > And this, too. I was unsure if this was because

Re: Online enabling of checksums

2018-04-07 Thread Andres Freund
Hi, On 2018-04-07 10:14:49 +0200, Michael Banck wrote: > Can the pg_verify_checksums command be kept at least, please? > > AFAICT this one is not contentious, the code is isolated, it's really > useful, orthogonal to online checksum activation and argueably could've > been committed as a

Re: Boolean partitions syntax

2018-04-07 Thread Jonathan S. Katz
> On Mar 21, 2018, at 10:59 PM, Amit Langote > wrote: > > Hi David. > > On 2018/03/21 23:31, David Steele wrote: >> Hi Amit, >> >> On 3/6/18 9:44 AM, David Steele wrote: >>> On 3/2/18 2:27 AM, Amit Langote wrote: On 2018/03/02 15:58, Andres Freund wrote:

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-07 Thread Alvaro Herrera
amul sul wrote: > On Sat, Apr 7, 2018 at 9:08 AM, Andres Freund wrote: > >> +test: partition-key-update-1 > >> +test: partition-key-update-2 > >> +test: partition-key-update-3 > > > > Can you give these more descriptive names please (or further combine them)? > > As I

Re: json(b)_to_tsvector with numeric values

2018-04-07 Thread Teodor Sigaev
See workable sketch for parsing jsonb flags and new worker variant. Yep, thanks for the sketch. Here is the new version of patch, does it look close to what you have in mind? Patch looks good except error messaging, you took it directly from sketch where I didn't spend time for it. Please,

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Tomas Vondra
On 04/07/2018 04:37 PM, Teodor Sigaev wrote: >> by this patch.  Revised version is attached. > > Fine, patch got several rounds of review in all its parts. Is any > places which should be improved before commit? > I personally feel rather uneasy about committing it, TBH. While I don't see any

Re: json(b)_to_tsvector with numeric values

2018-04-07 Thread Dmitry Dolgov
> On 6 April 2018 at 18:55, Teodor Sigaev wrote: > > > Dmitry Dolgov wrote: >>> >>> On 6 April 2018 at 16:25, Teodor Sigaev wrote: >>> 1) I don't like jsonb_all_to_tsvector too.. What if we will accept new >>> variant to index? Let me suggest: >>> >>> tsvector

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-07 Thread amul sul
On Sat, Apr 7, 2018 at 9:08 AM, Andres Freund wrote: > Hi Tom, All, > > On 2018-04-06 14:19:02 +0530, amul sul wrote: >> Thanks for the reminder -- fixed in the attached version. > > Tom, this seems to be the best approach for fixing the visibility issues > around this. I've

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Teodor Sigaev
by this patch.  Revised version is attached. Fine, patch got several rounds of review in all its parts. Is any places which should be improved before commit? -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/

Re: [HACKERS] Parallel Append implementation

2018-04-07 Thread Adrien Nayrat
Hello, I notice Parallel append is not listed on Parallel Plans documentation : https://www.postgresql.org/docs/devel/static/parallel-plans.html If you agree I can add it to Open Items. Thanks, -- Adrien NAYRAT signature.asc Description: OpenPGP digital signature

Re: WIP: a way forward on bootstrap data

2018-04-07 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > John Naylor wrote: > >> Commit 9fdb675fc added a symbol to pg_opfamily.h > >> where there were none before, so I went ahead and wrapped it with an > >> EXPOSE_TO_CLIENT_CODE macro. > > > Actually, after pushing that, I was

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-04-07 Thread Teodor Sigaev
Thanks to everyone, pushed Alexander Korotkov wrote: On Fri, Mar 2, 2018 at 6:57 AM, Thomas Munro > wrote: My thought experiments about pseudo-pages and avoiding the split stuff were not intended to get the patch

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-04-07 Thread Bruce Momjian
On Sun, Apr 1, 2018 at 06:07:55PM +0200, Tomas Vondra wrote: > Hi, > > The attached patch version modifies how the non-MCV selectivity is > computed, along the lines explained in the previous message. > > The comments in statext_clauselist_selectivity() explain it in far more > detail, but we

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Alexander Korotkov
On Sat, Apr 7, 2018 at 4:56 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov < > a.kuzmen...@postgrespro.ru> wrote: > >> On 06.04.2018 20:26, Tomas Vondra wrote: >> >>> I personally am OK with reducing the scope of the patch like

Re: [HACKERS] [PATCH] Incremental sort

2018-04-07 Thread Alexander Korotkov
On Fri, Apr 6, 2018 at 11:40 PM, Alexander Kuzmenkov < a.kuzmen...@postgrespro.ru> wrote: > On 06.04.2018 20:26, Tomas Vondra wrote: > >> I personally am OK with reducing the scope of the patch like this. It's >> still beneficial for the common ORDER BY + LIMIT case, which is good. I >> don't

Re: WIP: a way forward on bootstrap data

2018-04-07 Thread Tom Lane
Alvaro Herrera writes: > John Naylor wrote: >> Commit 9fdb675fc added a symbol to pg_opfamily.h >> where there were none before, so I went ahead and wrapped it with an >> EXPOSE_TO_CLIENT_CODE macro. > Actually, after pushing that, I was thinking maybe it's better to

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Jesper Pedersen
Hi, On 04/07/2018 04:45 AM, David Rowley wrote: Ok, so I've gone and done this. PartitionPruning has become PartitionPruneState PartitionRelPruning has become PartitionPruningData I've changed pointers to PartitionPruneStates to be named prunestate, sometimes having the node prefix; as_, ma_,

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Daniel Verite wrote: > The output might still differ compared to COPY in that line endings > depend on the client-side OS. There's also the minor issue > of a single \ by itself on a line, which gets quoted by COPY > and not by psql. I meant \. or backslash followed by period. This

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Alvaro Herrera
David Rowley wrote: > On 8 April 2018 at 01:18, Alvaro Herrera wrote: > > Amit had it as "indexes" also in his original. I wanted to avoid using > > the "indexes" word alone, whose meaning is so overloaded. > > hmm, good point. > > > How about this? > > "... which are

Re: WIP: Covering + unique indexes.

2018-04-07 Thread Alvaro Herrera
I didn't like rel.h being included in itup.h. Do you really need a Relation as argument to index_truncate_tuple? It looks to me like you could pass the tupledesc instead; indnatts could be passed as a separate argument instead of IndexRelationGetNumberOfAttributes. -- Álvaro Herrera

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Isaac Morland wrote: > OK, mostly trying to avoid commenting because I doubt I have much to add. > But. If I ask for CSV and don't specify any overrides, I expect to get > "C"omma separated values, not some other character. More specifically, if I > say --csv I expect to get files that

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread David Rowley
On 8 April 2018 at 01:18, Alvaro Herrera wrote: > Amit had it as "indexes" also in his original. I wanted to avoid using > the "indexes" word alone, whose meaning is so overloaded. hmm, good point. > How about this? > "... which are then executed to produce a set of

Re: [HACKERS] Runtime Partition Pruning

2018-04-07 Thread Alvaro Herrera
David Rowley wrote: > It's not exactly wrong but: > > + * are turned into a set of "pruning steps", which are then executed to > + * produce a set of RTIs of partitions whose bounds satisfy the constraints > in > + * the step. Partitions not in the set are said to have been pruned. > > It's

  1   2   >