Re: [HACKERS] Logical tape pause/resume

2016-10-09 Thread Heikki Linnakangas
On 10/09/2016 03:27 AM, Peter Geoghegan wrote: You shouldn't really waste 8% of the budget with low work_mem settings with my cap patch applied, because the new cap never limits the number of tapes. IIRC, the cap of 500 tapes doesn't start to matter until you have about 1GB of work_mem. So, if th

Re: [HACKERS] pg_stat_statements and non default search_path

2016-10-09 Thread Julien Rouhaud
On 10/10/2016 05:00, Jim Nasby wrote: > On 10/7/16 4:39 AM, Julien Rouhaud wrote: >> I see two ways of fixing this. First one would be to store normalized >> queries while fully qualifiying the relation's names. After a quick look >> this can't be done without storing at least a token location in >

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Pavel Stehule
2016-10-10 5:26 GMT+02:00 Jim Nasby : > On 10/9/16 9:54 PM, Bruce Momjian wrote: > >> I understand, but I am not sure how difficult implementation it is. This >>> part >>> > (COPY input) doesn't support parametrization - and parametrization can >>> have a >>> > negative performance impact. >>> >>

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Corey Huinker
On Sun, Oct 9, 2016 at 11:26 PM, Jim Nasby wrote: > On 10/9/16 9:54 PM, Bruce Momjian wrote: > >> I understand, but I am not sure how difficult implementation it is. This >>> part >>> > (COPY input) doesn't support parametrization - and parametrization can >>> have a >>> > negative performance im

Re: [HACKERS] Fast AT ADD COLUMN with DEFAULTs

2016-10-09 Thread Corey Huinker
> > > There's actually another use case here that's potentially extremely > valuable for warehousing and other "big data": compact representation of a > default value. > > I too would benefit from tables having either a default value in the event of a NOT-NULL column being flagged null, or a flat-o

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Jim Nasby
On 10/9/16 9:54 PM, Bruce Momjian wrote: I understand, but I am not sure how difficult implementation it is. This part > (COPY input) doesn't support parametrization - and parametrization can have a > negative performance impact. And it would need to be \:file_ref in COPY so real data doesn't tr

Re: [HACKERS] Fast AT ADD COLUMN with DEFAULTs

2016-10-09 Thread Jim Nasby
On 10/6/16 11:01 AM, Tom Lane wrote: Something based on missing_value/absent_value could work for me too. If we name it something involving "default", that definitely increases the possibility for confusion with the regular user-settable default. Also worth thinking about here is that the regula

Re: [HACKERS] pg_stat_statements and non default search_path

2016-10-09 Thread Jim Nasby
On 10/7/16 4:39 AM, Julien Rouhaud wrote: I see two ways of fixing this. First one would be to store normalized queries while fully qualifiying the relation's names. After a quick look this can't be done without storing at least a token location in RangeTblEntry nodes, which sounds like a bad ide

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Bruce Momjian
On Sun, Oct 9, 2016 at 06:12:16PM +0200, Pavel Stehule wrote: > > > 2016-10-09 17:27 GMT+02:00 Corey Huinker : > > here is new update - some mentioned issues are fixed + regress > tests and docs > > > > >   > This might tie into some work I'm doing. Is

Re: [HACKERS] Question / requests.

2016-10-09 Thread Jim Nasby
On 10/5/16 9:58 AM, Francisco Olarte wrote: Is the system catalog a bottleneck for people who has real use for paralell vacuum? I mean, to me someone who does this must have a very big db on a big iron. If that does not consist of thousands and thousands of smallish relations, it will normally be

Re: [HACKERS] Is it time to kill support for very old servers?

2016-10-09 Thread Jim Nasby
On 10/7/16 1:08 PM, Steve Crawford wrote: This is effectively a 5-year upgrade "grace period" *after* the EOL date of a given version which seems plenty generous. IMHO we need to be careful here. It's not at all unusual to see servers running versions that are *far* older than that. It's certa

Re: [HACKERS] pg_dump getBlobs query broken for 7.3 servers

2016-10-09 Thread Jim Nasby
On 10/7/16 12:48 PM, Tom Lane wrote: Branches before about 7.3 or 7.4 don't build cleanly on modern tools. In fact, they don't even build cleanly on my old HPUX 10.20 box ... I just tried, and they have problems with the bison and flex I have installed there now. As a data point, that bison exec

Re: [HACKERS] regular 10devel pdf build

2016-10-09 Thread Jim Nasby
On 10/8/16 9:51 AM, Erik Rijkers wrote: Would it be possible to regularly build and provide a .pdf of the development version? This is probably best asked on -www. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trou

Re: [HACKERS] Relids in upper relations

2016-10-09 Thread Robert Haas
On Sun, Oct 9, 2016 at 6:33 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Oct 5, 2016 at 9:42 AM, Tom Lane wrote: >>> I think this is fundamentally wrongheaded. If we go that route, >>> the only valid relids for any upper path would be the union of all >>> baserel RTIs, ... > >> Hmm, but

Re: [HACKERS] pg_upgrade documentation improvement patch

2016-10-09 Thread Bruce Momjian
On Mon, Oct 3, 2016 at 10:36:30AM +0900, Michael Paquier wrote: > On Tue, Apr 19, 2016 at 3:42 AM, Yuri Niyazov wrote: > > Should I update the documentation patch to instruct the use of > > pg_controldata exclusively? > > I guess so. Marked as returned with feedback because the thread has died.

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-09 Thread Andres Freund
Hi, Attached is an updated version of the patchset. The main changes are - address most of Tomas' feedback - address regression test output changes by adding explicit ORDER BYs, in a separate commit. - fix issues around hash tables sized up to PG_UINT32_MAX - fix a bug in iteration with "concurr

Re: [HACKERS] Relids in upper relations

2016-10-09 Thread David Rowley
On 10 October 2016 at 11:33, Tom Lane wrote: > Robert Haas writes: >> SELECT order_line.order_id, order.customer_id, SUM(order_line.amount) >> FROM order_line, order WHERE order_line.order_id = order.order_id >> GROUP BY 1,2; > >> Doing the aggregation step first is likely to be much faster than

Re: [HACKERS] Relids in upper relations

2016-10-09 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 5, 2016 at 9:42 AM, Tom Lane wrote: >> I think this is fundamentally wrongheaded. If we go that route, >> the only valid relids for any upper path would be the union of all >> baserel RTIs, ... > Hmm, but this is only true if the upper steps are always done las

Re: [HACKERS] autonomous transactions

2016-10-09 Thread Andres Freund
On 2016-10-06 10:56:34 +0100, Simon Riggs wrote: > On 7 September 2016 at 20:46, Robert Haas wrote: > > On Sat, Sep 3, 2016 at 7:09 AM, Simon Riggs wrote: > > True. I believe this issue has been discussed before, and I think > > it's a solvable issue. I believe that an autonomous transaction co

Re: [HACKERS] autonomous transactions

2016-10-09 Thread Robert Haas
On Thu, Oct 6, 2016 at 4:53 PM, Simon Riggs wrote: > For myself, I don't care what you call it. > > I just want to be able to invoke it by saying PRAGMA AUTONOMOUS_TRANSACTION; > and I know many others do also. To me, those statements are rather contradictory, and I don't support the latter propo

Re: [HACKERS] [COMMITTERS] pgsql: Remove -Wl,-undefined,dynamic_lookup in macOS build.

2016-10-09 Thread Robert Haas
On Fri, Oct 7, 2016 at 7:58 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Oct 7, 2016 at 7:14 PM, Tom Lane wrote: >>> BTW, OS X hasn't got libintl AFAICT: >>> What are you using to get past that? > >> MacPorts. > > OK, I'll take a look. Thanks. Everything seems OK again now. -- Robert

Re: [HACKERS] Relids in upper relations

2016-10-09 Thread Robert Haas
On Wed, Oct 5, 2016 at 9:42 AM, Tom Lane wrote: >> We could prevent the crash by passing input_rel->relids to >> fetch_upper_rel() in create_grouping_path() as seen in the attached >> patch. > > I think this is fundamentally wrongheaded. If we go that route, > the only valid relids for any upper

Re: [HACKERS] pg_upgrade 9.5 -> 9.6 fails when pg_largeobject is in separate tablespace

2016-10-09 Thread Robert Haas
On Sat, Oct 8, 2016 at 9:02 AM, Andreas Joseph Krogh wrote: > (I've set allow_system_table_mods=on in postgresql.conf) > Any configuration that includes this step is considered unsupported by the PostgreSQL community. It might be a good idea if we supported storing large objects in an alternate

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-09 Thread Tomas Vondra
On 10/08/2016 07:47 AM, Amit Kapila wrote: On Fri, Oct 7, 2016 at 3:02 PM, Tomas Vondra wrote: > > ... > In total, I plan to test combinations of: (a) Dilip's workload and pgbench (regular and -N) (b) logged and unlogged tables (c) scale 300 and scale 3000 (both fits into RAM) (d) sync_commit

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Pavel Stehule
2016-10-09 19:27 GMT+02:00 Corey Huinker : > look to code - some parts allows psql session variables, some not - I can >> use variables in statements - not in data block. >> >> More, there is ambiguity - :xxx should not be part of SQL statement (and >> then psql variables are possible), but :

Re: [HACKERS] vacuumdb -f and -j options (was Question / requests.)

2016-10-09 Thread Francisco Olarte
On Sat, Oct 8, 2016 at 2:22 PM, Michael Paquier wrote: > On Sat, Oct 8, 2016 at 9:12 PM, Amit Kapila wrote: >> On Fri, Oct 7, 2016 at 10:16 PM, Alvaro Herrera >> wrote: >>> Robert Haas wrote: On Wed, Oct 5, 2016 at 10:58 AM, Francisco Olarte I don't know, but it seems like the document

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Corey Huinker
> > look to code - some parts allows psql session variables, some not - I can > use variables in statements - not in data block. > > More, there is ambiguity - :xxx should not be part of SQL statement (and > then psql variables are possible), but : should be part of data. > > Regards > > Pavel

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Pavel Stehule
2016-10-09 19:14 GMT+02:00 Corey Huinker : > On Sun, Oct 9, 2016 at 12:12 PM, Pavel Stehule > wrote: > >> >> >> 2016-10-09 17:27 GMT+02:00 Corey Huinker : >> >>> here is new update - some mentioned issues are fixed + regress tests and > docs > >>> This might tie into some w

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Corey Huinker
On Sun, Oct 9, 2016 at 12:12 PM, Pavel Stehule wrote: > > > 2016-10-09 17:27 GMT+02:00 Corey Huinker : > >> here is new update - some mentioned issues are fixed + regress tests and docs >>> >>> >>> >> This might tie into some work I'm doing. Is there any way these filerefs >> could be u

[HACKERS] cygwin64 assertion failure

2016-10-09 Thread Andrew Dunstan
lorikeet seems to be stuck running the parallel tests, after having failed an assertion. Here's an excerpt from the log: 2016-10-09 11:52:35.751 EDT [57fa67c3.1148:11] LOG: statement: alter table tenk1 set (parallel_workers = 4); 2016-10-09 11:52:35.753 EDT [57fa67c3.1148:12] LOG: statement

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Pavel Stehule
2016-10-09 17:27 GMT+02:00 Corey Huinker : > here is new update - some mentioned issues are fixed + regress tests and >>> docs >>> >> >> >> > This might tie into some work I'm doing. Is there any way these filerefs > could be used as the inline portion of a COPY command? > i.e. like this: > > COPY

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Corey Huinker
> > here is new update - some mentioned issues are fixed + regress tests and >> docs >> > > > This might tie into some work I'm doing. Is there any way these filerefs could be used as the inline portion of a COPY command? i.e. like this: COPY my_table FROM STDIN :file_ref \.

[HACKERS] B-tree index row size limit

2016-10-09 Thread Florian Weimer
The index row size limit reared its ugly head again. My current use of PostgreSQL is to load structured data into it but from sources I don't have control over, to support a wide range of queries whose precise nature is not yet known to me. (Is this called a data warehouse?) Anyway, what happens

Re: [HACKERS] proposal: psql \setfileref

2016-10-09 Thread Pavel Stehule
hi 2016-10-04 9:18 GMT+02:00 Gilles Darold : > Le 03/10/2016 à 23:23, Gilles Darold a écrit : > > Le 03/10/2016 à 23:03, Robert Haas a écrit : > >> On Mon, Oct 3, 2016 at 3:54 PM, Gilles Darold > wrote: > >>> 4) An other problem is that like this this patch will allow anyone to > upload into a >

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

2016-10-09 Thread Christoph Berg
Re: Tom Lane 2016-10-08 <29244.1475959...@sss.pgh.pa.us> > So I'm still of the opinion that there's not likely to be any meaningful > performance difference in practice, at least not on reasonably recent > Linux machines. But this does indicate that if there is any difference, > it will probably f