Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-04-07 Thread Alvaro Herrera
Tomas Vondra wrote: > 1) bringetbitmap(PG_FUNCTION_ARGS) > > + /* > + * Estimate the approximate size of btup and allocate memory for btup. > + */ > + btupInitSize = bdesc->bd_tupdesc->natts * 16; > + btup = palloc(btupInitSize); > > What is the reasoning behind this estimate? I mean, this

Re: [HACKERS] BRIN desummarization writes junk WAL records

2017-04-07 Thread Alvaro Herrera
Tom Lane wrote: > The proximate cause of the exception seems to be that > brinSetHeapBlockItemptr is being passed pagesPerRange = 0, > which is problematic since HEAPBLK_TO_REVMAP_INDEX tries to > divide by that. Looking one level down, the bogus value > seems to be coming out of an

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > My offer to work with you on amcheck verification of WARM invariants > remains open. If nothing else, structuring things so that verification > is possible may clarify your design. Formalizing the preconditions, > postconditions, and legal states for on-disk structures

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. Given the number of votes against putting this on pg10, I am going to back off from this

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Unique Joins > - Tom's discussing things with David, not sure. This one was already included-and-removed from 9.6, Tom had said he'd give it priority during the current cycle as I recall. It seems unfair that it's still waiting for review on the last day of pg10's last

Re: [HACKERS] BRIN cost estimate

2017-04-06 Thread Alvaro Herrera
Tom Lane wrote: > TBH, I think that code is in the noise. It doesn't involve any disk > access, or catalog access, or user-defined function calls. I wouldn't > bother trying to account for it. I removed it in the pushed version. > What you should be accounting for is the ensuing heap page

Re: [HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread Alvaro Herrera
mark wrote: > m=# create table mytable (myid serial, mytext text); > CREATE TABLE > m=# \d mytable > ERROR: relation "pg_catalog.pg_statistic_ext" does not exist > LINE 8: FROM pg_catalog.pg_statistic_ext stat WHERE starelid = '163... > ^ Ah, what happens is you're using a new

Re: [HACKERS] Re: extended stats not friendly towards ANALYZE with subset of columns

2017-04-05 Thread Alvaro Herrera
Noah Misch wrote: > The above-described topic is currently a PostgreSQL 10 open item. Álvaro, > since you committed the patch believed to have created it, you own this open > item. I'll commit a fix for this problem no later than Friday 14th. -- Álvaro Herrera

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

2017-04-03 Thread Alvaro Herrera
Daniel Gustafsson wrote: > Testing DefElem options is done with both strcmp() and pg_strcasecmp() a bit > mixed. Since the option defnames are all lowercased, either via IDENT, > keyword > rules or “by hand” with makeString(), using strcmp() is safe (and assumed to > be > so in quite a lot of

Re: [HACKERS] Can't compile with profiling after BRIN autosummarization

2017-04-03 Thread Alvaro Herrera
Jeff Janes wrote: > A newer gcc gave a better error message which lead me to add the line: > > #include "storage/block.h" /* for typedef BlockNumber */ > > into autovacuum.h, which fixes the problem. Ah, Peter already fixed it. -- Álvaro Herrera

Re: [HACKERS] Can't compile with profiling after BRIN autosummarization

2017-04-03 Thread Alvaro Herrera
Jeff Janes wrote: > This git bisects down to: > > commit 7526e10224f0792201e99631567bbe44492bbde4 > Author: Alvaro Herrera <alvhe...@alvh.no-ip.org> > Date: Sat Apr 1 14:00:53 2017 -0300 > > BRIN auto-summarization > > The lines are: >

Re: [HACKERS] BRIN de-summarize ranges

2017-04-01 Thread Alvaro Herrera
Alvaro Herrera wrote: > > However, I found that when calling brin_desummarize_range > > successively, an assertion is failed. It seems to me that it occurs > > when desummarizing a revmap page that is already desummarized. > > You're right, it's broken for that cas

Re: [HACKERS] BRIN de-summarize ranges

2017-04-01 Thread Alvaro Herrera
- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From d584e88386dd8c775a7b147b3dba328c26585858 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera <alvhe...@alvh.no-ip.org> Date: Tue, 28 Feb 2017 01:45:21 -0300 Su

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-04-01 Thread Alvaro Herrera
Alvaro Herrera wrote: > I also removed the behavior that on index creation the final partial > block range is always summarized. It's pointless. I just pushed this, without this change, because it breaks src/test/modules/brin. I still think it's pointless, but it'd require more than on

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-04-01 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Mar 21, 2017 at 4:10 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > Well, the number of work items is currently fixed; but if you have many > > BRIN indexes, then you'd overflow (lose requests). By using DSA I am > > making it

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-31 Thread Alvaro Herrera
Here's a version of this patch which I consider final. Thanks for your tips on using DSA. No crashes now. I am confused about not needing dsa_attach the second time around. If I do that, the dsa handle has been 0x7f'd, which I don't understand since it is supposed to be allocated in

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-03-31 Thread Alvaro Herrera
Ashutosh Bapat wrote: > Please add this to the next commitfest. If this cannot be reproduced in 9.6, then it must be added to the Open Items wiki page instead. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-30 Thread Alvaro Herrera
Robert Haas wrote: > - Alvaro proposes allowing more time next time, but not to change the > dates for this time. FWIW I didn't realize that the NY conference was ongoing, so count me for postponing the end of the current CF. -- Álvaro Herrerahttps://www.2ndQuadrant.com/

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-30 Thread Alvaro Herrera
Stephen Frost wrote: > I'd be fine with removing --verbose globally, as your patch does, but > there was some argument that we then would have long 'quiet' periods. > I haven't had a chance to go test if that's really the case yet though. Michael said that a running counter was displayed, which

Re: [HACKERS] error handling in RegisterBackgroundWorker

2017-03-29 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 29, 2017 at 2:10 PM, Peter Eisentraut > wrote: > > How specifically would we do that? And what user would choose the > > behavior "start this background worker but don't worry if it doesn't work"? > > Well, if the background

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-29 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 29, 2017 at 3:04 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > I was rather surprised to see the March commitfest declared to exactly > > one month and feature freeze immediately thereafter. > > That's true, but at >

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-29 Thread Alvaro Herrera
Tom Lane wrote: > My own thought is that there's room for at least a few days' slop in > the end date of the final commitfest, depending on what patches remain > open and what the prospects are for getting them done. (In the past > we've sometimes let the final fest stretch on indefinitely,

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-29 Thread Alvaro Herrera
Stephen Frost wrote: > * Dagfinn Ilmari Mannsåker (ilm...@ilmari.org) wrote: > > Non-verbose prove still lists each test script, it just doesn't list > > each individual test within the script. > > I agree that it'd be better to just show the per-script results rather > than every little test

Re: [HACKERS] logical replication access control patches

2017-03-29 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 3/15/17 21:54, Peter Eisentraut wrote: > > 0004 Add subscription apply worker privilege checks > > 0005 Add CREATE SUBSCRIPTION privilege on databases > > It would be nice to reach a conclusion on these (the second one > particularly), because otherwise we'll be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-29 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Wed, Mar 29, 2017 at 3:42 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > > I pushed 0002 after some makeup, since it's just cosmetic and not > > controversial. > > Thanks. I think your patch of tracking interestin

Re: [HACKERS] dsm.c API tweak

2017-03-28 Thread Alvaro Herrera
Thomas Munro wrote: > + if (CurrentResourceOwner) > + { > + seg->resowner = CurrentResourceOwner; > + ResourceOwnerRememberDSM(CurrentResourceOwner, seg); > + } > > You need to assign seg->resowner = CurrentResourceOwner > unconditionally here. Otherwise seg->resowner is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Alvaro Herrera
a different strategy, if we want to enable WARM on GiST/GIN indexes.) -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From f6ba238dd46416eb29ac43dadac0c69a75f106fe Mon Sep 17 00:00:00 2001 From: Alvaro Herrera

Re: [HACKERS] [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-28 Thread Alvaro Herrera
Fujii Masao wrote: > This is the evidence that no one cares about the details of VACUUM VERBOSE > output example. So I'm tempted to simplify the example (please see the > attached patch) instead of keeping updating the example. Agreed. > > If this patch is applied, it should back-patch to all

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-28 Thread Alvaro Herrera
Nikolay Shaplov wrote: > В письме от 17 марта 2017 14:21:26 пользователь Alvaro Herrera написал: > > > Please make sure to mark functions as static (e.g. bringetreloptcatalog). > I am a bit confused here: > > For brin and nbtree this is a good idea: brin.c and nbtree.c has

Re: [HACKERS] Valgrind failures caused by multivariate stats patch

2017-03-28 Thread Alvaro Herrera
Tomas Vondra wrote: > OK, this did the trick. And just as I suspected, it seems to be due to doing > memcpy+offsetof when serializing the statistic into a bytea. The attached > patch fixes that for me. Can you test it on your build? Buildfarm member skink confirms that this is fixed. Thanks!

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-27 Thread Alvaro Herrera
Is the WARM tap test suite supposed to work when applied without all the other patches? I just tried applied that one and running "make check -C src/test/modules", and it seems to hang after giving "ok 5" for t/002_warm_stress.pl. (I had to add a Makefile too, attached.) -- Álvaro Herrera

Re: [HACKERS] pg_get_statisticsextdef() is not quite the full shilling

2017-03-26 Thread Alvaro Herrera
David Rowley wrote: > Seems pg_get_statisticsextdef() has a couple of things wrong: > > 1. HeapTupleIsValid() called on the wrong tuple. > 2. Did not schema qualify names. Actually we can solve both 1 and the first half of 2 by just using generate_relation_name, which is less code and gives

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-26 Thread Alvaro Herrera
Nikolay Shaplov wrote: > If I would think about it now: we always know how many options we will have. > So we can just pass this number to palloc and assert if somebody adds more > options then expected... What do yo think about it. I think we need to preserve the ability to add custom

Re: [HACKERS] dsm.c API tweak

2017-03-25 Thread Alvaro Herrera
Thomas Munro wrote: > I'd word this slightly differently: > > + * If there is a CurrentResourceOwner, the new segment is born unpinned and > the > + * resource owner is in charge of destroying it (and will be blamed if it > + * doesn't). If there's no current resource owner, then the segment

Re: [HACKERS] dsm.c API tweak

2017-03-24 Thread Alvaro Herrera
Alvaro Herrera wrote: > Per > https://postgr.es/m/CAEepm=11ma_Z1HoPxPcSCANnh5ykHORa=hca1u1v1+5s_jw...@mail.gmail.com > it seems that the dsm.c API is a bit inconvenient right now. I proposed > in the first patch in that thread to change the API so that a segment is > ma

[HACKERS] dsm.c API tweak

2017-03-24 Thread Alvaro Herrera
Per https://postgr.es/m/CAEepm=11ma_Z1HoPxPcSCANnh5ykHORa=hca1u1v1+5s_jw...@mail.gmail.com it seems that the dsm.c API is a bit inconvenient right now. I proposed in the first patch in that thread to change the API so that a segment is marked as "pinned" if created with no ResourceOwner set as

Re: [HACKERS] Re: [COMMITTERS] pgsql: Implement multivariate n-distinct coefficients

2017-03-24 Thread Alvaro Herrera
Tom Lane wrote: > Why not use COSTS OFF? Or I'll put that even more strongly: all the > existing regression tests use COSTS OFF, exactly to avoid this sort of > machine-dependent output. There had better be a really damn good > reason not to use it here. If we use COSTS OFF, the test is

[HACKERS] Re: [COMMITTERS] pgsql: Implement multivariate n-distinct coefficients

2017-03-24 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Mar 24, 2017 at 1:16 PM, Alvaro Herrera <alvhe...@alvh.no-ip.org> > wrote: > > Implement multivariate n-distinct coefficients > > dromedary and arapaima have failures like this, which seems likely > related to this commit: > >

Re: [HACKERS] multivariate statistics (v25)

2017-03-24 Thread Alvaro Herrera
Alvaro Herrera wrote: > Here's a rebased series on top of today's a3eac988c267. I call this > v28. > > I put David's pg_dump and COMMENT patches as second in line, just after > the initial infrastructure patch. I suppose those three have to be > committed together, while the

Re: [HACKERS] extended statistics: n-distinct

2017-03-24 Thread Alvaro Herrera
Pushed this after some more tweaking. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-23 Thread Alvaro Herrera
Copying Fabrízio Mello here, who spent some time trying to work on reloptions too. He may have something to say about the new functionality that this patch provides. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] parallel "return query" is no good

2017-03-23 Thread Alvaro Herrera
Robert Haas wrote: > I guess the downside of back-patching this is that it could cause a > plan change for somebody which ends up being worse. On the whole, > serial execution of queries intended to be run in parallel isn't > likely to work out well, but it's always possible somebody has a cases

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix and simplify check for whether we're running as Windows serv

2017-03-22 Thread Alvaro Herrera
Heikki Linnakangas wrote: > I did some archeology, and found CheckTokenMembership() in MinGW's w32api > packages version 3.14 > (https://sourceforge.net/projects/mingw/files/MinGW/Base/w32api/w32api-3.14/, > in include/winbase.h). According to the timestamps on that download page, > that was

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-21 Thread Alvaro Herrera
Thomas Munro wrote: > What is your motivation for using DSA? It seems you are creating an > area and then using it to make exactly one allocation of a constant > size known up front to hold your fixed size workitems array. You > don't do any dynamic allocation at runtime, apart from the detail

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-21 Thread Alvaro Herrera
Thomas Munro wrote: > Another thought about this design: Why autovacuum? One reason is that autovacuum is already there, so it's convenient to give it the responsibility for this kind of task. Another reason is that autovacuum is already doing this, via vacuum. I don't see the need to have a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > I don't think it makes sense to try and save bits and add complexity > > > when we have no idea if we will ever use them, > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Bruce Momjian wrote: > I don't think it makes sense to try and save bits and add complexity > when we have no idea if we will ever use them, If we find ourselves in dire need of additional bits, there is a known mechanism to get back 2 bits from old-style VACUUM FULL. I assume that the reason

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Mar 21, 2017 at 10:01 AM, Amit Kapila wrote: > > Sure, we can try that. I think we need to try it with > > synchronous_commit = off, otherwise, WAL writes completely overshadows > > everything. > > synchronous_commit = off is a much more

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Amit Kapila wrote: > I think it is because heap_getattr() is not that cheap. We have > noticed the similar problem during development of scan key push down > work [1]. One possibility to reduce the cost of that is to use whole tuple deform instead of repeated individual heap_getattr() calls.

Re: [HACKERS] extended statistics: n-distinct

2017-03-20 Thread Alvaro Herrera
Alvaro Herrera wrote: > * I'm not terribly happy with the header organization. I think > VacAttrStats should be in its own (new) src/include/statistics/analyze.h > for example (which cleans up a bunch of existing stuff a bit) I tried this and it doesn't actually do any good. Patch

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > I didn't like this comment very much. But it's not necessary: you have > > already given relcache responsibility for setting rd_supportswarm. The > > onl

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Alvaro Herrera
Stephen Frost wrote: > Is there any hope of getting a "quiet" mode, where all the "ok" lines > aren't printed when things work..? Well, we currently have --verbose in PROVE_FLAGS. Maybe you can take it out, or even add --quiet or --QUIET (see the prove(1) manpage). -- Álvaro Herrera

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Alvaro Herrera
Stephen Frost wrote: > Andrew, > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > > ISTM that the test setup and breakdown code, both in individual tests > > and in PostgresNode.pm should be liberally sprinkled with diag() calls > > to make it easier to narrow down errors.. > > While

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-19 Thread Alvaro Herrera
Tom Lane wrote: > I'm not entirely convinced that function-per-command is an improvement > though. Seems like it would only help to the extent that you could do a > simple "return" to implement early exit, and it looks to me like that > doesn't work in a lot of places because you still have to

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-19 Thread Alvaro Herrera
Fabien COELHO wrote: > > Hello Corey, > > > v24 highlights: > > The v24 patch is twice larger that the previous submission. Sigh. The reason this is so large is that there is an entangled refactoring patch, splitting the exec_command() function from one giant switch() into one routine for each

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-17 Thread Alvaro Herrera
By the way, it would be very good if you could review some patches, too. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

[HACKERS] Re: [COMMITTERS] pgsql: Remove objname/objargs split for referring to objects

2017-03-17 Thread Alvaro Herrera
Peter Eisentraut wrote: > cc -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute > -Wformat-security -fno-strict-aliasing -fwrapv > -fexcess-precision=standard -g -O2 -I../../../src/include > -D_FORTIFY_SOURCE=2 -DLINUX_OOM_ADJ

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-17 Thread Alvaro Herrera
I gave this patch a quick skim. At first I was confused by the term "catalog"; I thought it meant we stored options in a system table. But that's not what is meant at all; instead, what we do is build these "catalogs" in memory. Maybe a different term could have been used, but I'm not sure it's

[HACKERS] Re: [COMMITTERS] pgsql: Remove objname/objargs split for referring to objects

2017-03-17 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 3/16/17 11:56, Alvaro Herrera wrote: > > Michael Paquier wrote: > > > >> What are you using as CFLAGS? As both typenames should be normally > >> set, what about initializing those fields with NULL and add an > >> assertio

[HACKERS] Re: [COMMITTERS] pgsql: Remove objname/objargs split for referring to objects

2017-03-16 Thread Alvaro Herrera
Michael Paquier wrote: > What are you using as CFLAGS? As both typenames should be normally > set, what about initializing those fields with NULL and add an > assertion like the attached? Actually, my compiler was right -- this was an ancient bug I introduced in 9.5 (commit a61fd533), and this

Re: [HACKERS] multivariate statistics (v25)

2017-03-15 Thread Alvaro Herrera
David Rowley wrote: > + k = -1; > + while ((k = bms_next_member(attnums, k)) >= 0) > + { > + bool attr_found = false; > + for (i = 0; i < info->stakeys->dim1; i++) > + { > + if (info->stakeys->values[i] == k) > + { > + attr_found = true; > + break; > + } > + } > + > + /* found attribute not

Re: [HACKERS] 2017-03 Commitfest Midterm

2017-03-15 Thread Alvaro Herrera
David Steele wrote: > We are now halfway through the 2017-03 CF. Here's the breakdown: > > Needs review: 83 (-45) > Waiting on Author: 36 (+10) > Ready for Committer: 19 (-6) > Total: 208 (+28) > > It's interesting that there are 28 more patches than there were on the > 1st. Either I had a

Re: [HACKERS] multivariate statistics (v25)

2017-03-14 Thread Alvaro Herrera
Alvaro Herrera wrote: > I tried patch 0002 today and again there are conflicts, so I rebased and > fixed the merge problems. ... and attached the patch. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] multivariate statistics (v25)

2017-03-14 Thread Alvaro Herrera
I tried patch 0002 today and again there are conflicts, so I rebased and fixed the merge problems. I also changed a number of minor things, all AFAICS cosmetic in nature: * moved src/backend/statistics/common.h to src/include/statistics/common.h, as previously commented. I also took out

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
Pavan Deolasee wrote: > BTW I wanted to share some more numbers from a recent performance test. I > thought it's important because the latest patch has fully functional chain > conversion code as well as all WAL-logging related pieces are in place > too. I ran these tests on a box borrowed from

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > I have already commented about the executor involvement in btrecheck(); > > that doesn't seem good. I previously suggested to pass the EState down >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
After looking at how index_fetch_heap and heap_hot_search_buffer interact, I can't say I'm in love with the idea. I started thinking that we should not have index_fetch_heap release the buffer lock only to re-acquire it five lines later, so it should keep the buffer lock, do the recheck and only

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-03-14 Thread Alvaro Herrera
David Steele wrote: > On 3/4/17 12:43 AM, Robert Haas wrote: > > I think we should regard this resubmission as untimely, unless there > > is an argument that it amounts to a bug fix. > > I agree and I'm also confused about which author this is waiting on. Is > it Jinyu or Álvaro? I don't think

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-14 Thread Alvaro Herrera
Ashutosh Sharma wrote: > Yes. But, as i said earlier I am getting negative checksum value for > page_header as well. Isn't that wrong. For eg. When I debug the > following query, i could pd_checksum value as '40074' in gdb where > page_header shows it as '-25462'. Yes; the point is that this is

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Alvaro Herrera
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 3d6a3801c0..d205101b89 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -47,7 +47,14 @@ #include "utils/rel.h" -static bool get_last_attnums(Node *node, ProjectionInfo

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Alvaro Herrera
Andres Freund wrote: > EEO_SWITCH(op->opcode) > { > EEO_CASE(EEO_DONE): > goto out; Oh my. > which is a bit annoying. (the EEO_CASE is either a jump label or a case > statement, depending on computed goto availability). > > It seems we could either: > 1) live with the damage >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-13 Thread Alvaro Herrera
> @@ -234,6 +236,21 @@ index_beginscan(Relation heapRelation, > scan->heapRelation = heapRelation; > scan->xs_snapshot = snapshot; > > + /* > + * If the index supports recheck, make sure that index tuple is saved > + * during index scans. > + * > + * XXX

Re: [HACKERS] delta relations in AFTER triggers

2017-03-13 Thread Alvaro Herrera
Kevin Grittner wrote: > >> What is necessary to indicate an additional SQL feature covered? > > > > I assume you're talking about information_schema.sql_features > > I had forgotten we had that in a table. I was thinking more of the docs: > >

Re: [HACKERS] [COMMITTERS] pgsql: Improve postmaster's logging of listen socket creation.

2017-03-13 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > So now on every startup I get this: > > > 2017-03-13 10:08:49.399 EDT [90059] LOG: listening on IPv6 address "::1" > > 2017-03-13 10:08:49.399 EDT [90059] LOG: listening on IPv4 address > > "127.0.0.1" > > 2017-03-13

Re: [HACKERS] make check-world output

2017-03-11 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <alvhe...@2ndquadrant.com> writes: > > Jeff Janes wrote: > >> There was some recent discussion about making "make check-world" faster. > >> I'm all for that, but how about making it quieter? On both machines I've &

Re: [HACKERS] make check-world output

2017-03-10 Thread Alvaro Herrera
Jeff Janes wrote: > There was some recent discussion about making "make check-world" faster. > I'm all for that, but how about making it quieter? On both machines I've > run it on (CentOS6.8 and Ubuntu 16.04.2), it dumps some gibberish to > stderr, example attached. I think you're complaining

Re: [HACKERS] Should we eliminate or reduce HUP from docs?

2017-03-10 Thread Alvaro Herrera
David G. Johnston wrote: > On Fri, Mar 10, 2017 at 3:28 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > > I think we could split 19.1.2 in two parts, where the first one is the > > current content minus the paragraph "The configuration file is

Re: [HACKERS] Should we eliminate or reduce HUP from docs?

2017-03-10 Thread Alvaro Herrera
David G. Johnston wrote: > On Fri, Mar 10, 2017 at 1:02 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > There are several ways to cause a config file reload (pg_ctl reload, > > pg_reload_conf, direct SIGHUP). We could have a section in docs listing > > th

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-03-10 Thread Alvaro Herrera
Robert Haas wrote: > The smoking gun was in 009_twophase_slave.log: > > TRAP: FailedAssertion("!(nsubxids == MyPgXact->nxids)", File: > "clog.c", Line: 288) > > ...and then the node shuts down, which is why this hangs forever. > (Also... what's up with it hanging forever instead of timing out

Re: [HACKERS] Upgrading postmaster's log messages about bind/listen errors

2017-03-10 Thread Alvaro Herrera
Tom Lane wrote: > Euler Taveira writes: > > 2017-03-10 1:43 GMT-03:00 Tom Lane : > >> 2017-03-09 23:40:12.334 EST [19335] LOG: MultiXact member wraparound > >> protections are now enabled > > > It should be DEBUG1 as soon as 9.3 is deprecated. > > Uh,

Re: [HACKERS] make check-world output

2017-03-10 Thread Alvaro Herrera
Jeff Janes wrote: > Also, it runs in about 8 minutes, not the 20 minutes reported by others. > My system is virtualized, and not particularly fast. I wonder if it is > failing early somewhere without running to completion? How would/should I > know? Maybe you don't have --enable-tap-tests? --

Re: [HACKERS] Should we eliminate or reduce HUP from docs?

2017-03-10 Thread Alvaro Herrera
Joshua D. Drake wrote: > I am a bad speaker, I am writing a talk three weeks before the conference > (as opposed to on the plane). Hah. > I noticed in the docs we still reference the > passing of SIGHUP for reloading conf file but we now have pg_reload_conf(); > > It seems the use of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-10 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 8, 2017 at 2:30 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > > Not really -- it's a bit slower actually in a synthetic case measuring > > exactly the slowed-down case. See > > https://www.postgresql.org/m

Re: [HACKERS] tzdata2017a breaks timestamptz regression test

2017-03-10 Thread Alvaro Herrera
Tom Lane wrote: > For the cases involving the America/Santiago zone, I'm a bit inclined > to just switch that to America/New_York, which seems much less likely > to get fooled with by IANA. But I'm wondering if Alvaro had a specific > reason for using the Santiago zone in those test cases. No,

Re: [HACKERS] Bizarre choice of case for RELKIND_PARTITIONED_TABLE

2017-03-09 Thread Alvaro Herrera
Tom Lane wrote: > (And no, I don't especially > approve of RELKIND_SEQUENCE being 'S' either, but it's far too late to > change that.) FWIW the reason SEQUENCE uses S instead of 's' is that the latter was taken for "special" relations, which we removed a few releases ago (commit 3a694bb0a1). --

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-08 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 8, 2017 at 12:14 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: > > Alvaro Herrera wrote: > >> Here's a rebased set of patches. This is the same Pavan posted; I only > >> fixed some whitespace and a trivial conflict

Re: [HACKERS] patch: function xmltable

2017-03-08 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-03-08 17:32 GMT+01:00 Alvaro Herrera <alvhe...@2ndquadrant.com>: > > > For daily work the default schema support is much more interesting. > > > > Let's see that one, then. It was part of the original submission so > > depending

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-08 Thread Alvaro Herrera
Here's a rebased set of patches. This is the same Pavan posted; I only fixed some whitespace and a trivial conflict in indexam.c, per 9b88f27cb42f. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via

Re: [HACKERS] new gcc 7.0.1 warnings

2017-03-08 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-02-18 18:35 GMT+01:00 Tom Lane : > > > Pavel Stehule writes: > > Do the warnings go away if you add some explicit guard to the precision > > variable, say like this: > > > > { > > int

Re: [HACKERS] patch: function xmltable

2017-03-08 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-03-08 17:01 GMT+01:00 Alvaro Herrera <alvhe...@2ndquadrant.com>: > > I didn't add the change you proposed here to keep COLUMNS optional; > > instead, I just made COLUMNS mandatory. I think what you propose here > > is not entirely out of the

Re: [HACKERS] patch: function xmltable

2017-03-08 Thread Alvaro Herrera
Pavel Stehule wrote: > Hi > > I used your idea about special columns when COLUMNS are not explicitly > defined. > > All lines that you are dislike removed. I just pushed XMLTABLE, after some additional changes. Please test it thoroughly and report any problems. I didn't add the change you

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-07 Thread Alvaro Herrera
FWIW, +1 on improving matters here. Andres Freund wrote: > The best I can come up so far is a toplevel target that creates the temp > install, starts a cluster and then runs the 'installcheck-or-check' > target on all the subdirectories via recursion. Individual makefiles can > either use the

Re: [HACKERS] patch: function xmltable

2017-03-03 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-03-03 19:15 GMT+01:00 Alvaro Herrera <alvhe...@2ndquadrant.com>: > > 2. As I've complained many times, I find the way we manage an empty > > COLUMNS clause pretty bad. The standard doesn't require that syntax > > (COLUMNS is

Re: [HACKERS] patch: function xmltable

2017-03-03 Thread Alvaro Herrera
Pavel Stehule wrote: > attached update with fixed tests Heh, I noticed that you removed the libxml "context" lines that differentiate xml.out from xml_2.out when doing this. My implementation emits those lines, so it was failing for me. I restored them. I also changed a few things to avoid

Re: [HACKERS] patch: function xmltable

2017-03-02 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-03-02 19:32 GMT+01:00 Alvaro Herrera <alvhe...@2ndquadrant.com>: > > > So in the old (non-executor-node) implementation, you could attach WITH > > ORDINALITY to the xmltable expression and it would count the output > > rows, regardless

Re: [HACKERS] patch: function xmltable

2017-03-02 Thread Alvaro Herrera
So in the old (non-executor-node) implementation, you could attach WITH ORDINALITY to the xmltable expression and it would count the output rows, regardless of which XML document it comes from. With the new implementation, the grammar no longer accepts it. To count output rows, you still need to

Re: [HACKERS] patch: function xmltable

2017-03-02 Thread Alvaro Herrera
Pavel Stehule wrote: > It is documented already > > "If the PATH matches an empty tag the result is an empty string" Hmm, okay. But what we have here is not an empty tag, but a tag that is completely missing. I don't think those two cases should be treated in the same way ... > Attached new

Re: [HACKERS] patch: function xmltable

2017-03-01 Thread Alvaro Herrera
I've been giving this a look. I started by tweaking the docs once again, and while verifying that the example works as expected, I replayed what I have in sgml: ... begin SGML paste ... For example, given the following XML document: the following query produces the result

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-03-01 Thread Alvaro Herrera
Alvaro Herrera wrote: > Jinyu Zhang wrote: > > > Update the patch_brin_optimze_mem according to your comment. > > I have added this patch to the commitfest, which I've been intending to > get in for a long time. I'll be submitting an updated patch soon. Here it is. I ad

Re: [HACKERS] Possible spelling fixes

2017-03-01 Thread Alvaro Herrera
Josh Soref wrote: > One thing that would be helpful is if someone could comment on: > https://github.com/jsoref/postgres/commit/9050882d601134ea1ba26f77ce5f1aaed75418de > -#undef SH_ITERTOR > +#undef SH_ITERATOR > > It's unclear to me what that line is/was doing. It's possible that it > could be

<    1   2   3   4   5   6   7   8   9   10   >