Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-06-16 Thread Kevin Grittner
e first try. For examples, to help get a feel of why that is, see: https://wiki.postgresql.org/wiki/SSI -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.pos

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-06-15 Thread Kevin Grittner
On Thu, Jun 15, 2017 at 4:18 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: > As far as I understand her proposal, it is exactly the opposite -- if a > transaction fails, it is discarded. And this P.S. note is asking > whether this is a good idea, or would we prefer

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-06-15 Thread Kevin Grittner
transactions. Essentially, the transaction should only count toward the TPS rate when it eventually completes without a serialization failure. Marina, did I understand you correctly? -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] GSoC 2017 weekly progress reports (week 2)

2017-06-13 Thread Kevin Grittner
ndex Only Scan, but also on Bitmap Index Scan? And may be > KNN version of scan too? > I couldn't find such tests for B-tree, do we have them? Off-hand, I don't know. It would be interesting to run regression tests with code coverage and look at the index AMs. https://www.postgre

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Kevin Grittner
atement -- well, it won't be a problem for matview maintenance. The biggest hurt there would be to *my* sense of aesthetics. ;-) -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Kevin Grittner
if the algorithm treats all > tuples in the old table as deletes (even though in this case they > result from updates) and all tuples in the new table as inserts (even > though in this case *some* of them result from updates) anyway then I > don't think it matters. I don't th

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Kevin Grittner
7;ll try to keep to small enough issues that the resources required to support what I do is not beyond what I can deliver. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Kevin Grittner
table for any rows affected by an update or delete. If we have a single statement that combines INSERT and UPDATE behaviors, it might make sense to have an "old" table for updates and a single "new" table for both. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-07 Thread Kevin Grittner
functions stand out: 10.86% PredicateLockTuple 3.51% CheckForSerializableConflictIn In both cases, most of that seems to go to lightweight locking. Since you said this is a CPU graph, that again suggests spinlock contention issues. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/

Re: [HACKERS] GSoC 2017 : Proposal for predicate locking in gist index

2017-06-06 Thread Kevin Grittner
ion, but still. Should we skip > finishing split or should we add it to collision check too? When a page is split, I think you need to call PredicateLockPageSplit() before it gets to the point that an insert to get to it. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Se

[HACKERS] Re: GSoC 2017 weekly progress reports ("Explicitly support predicate locks in index access methods besides b-tree")

2017-06-06 Thread Kevin Grittner
zableConflictIn() ...etc) When you have a patch for any AM, please post it to the list. Each AM will be reviewed and considered for commit separately. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-06 Thread Kevin Grittner
On Tue, Jun 6, 2017 at 3:41 PM, Marko Tiikkaja wrote: > On Tue, Jun 6, 2017 at 10:25 PM, Kevin Grittner wrote: >> It took years to get an in-depth review, then I was asked >> not to commit it because others were working on patches that would >> conflict. That just doesn&#

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-06 Thread Kevin Grittner
27;ll give it a few days for objections before reverting. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-03 Thread Kevin Grittner
id further complicating things with CTE/transition table usage until we sort out the basics of triggers firing from CTE DML in the first place. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-03 Thread Kevin Grittner
(or some portion of it larger than 10%) instead, I think we could adjust the scope based on those results. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-02 Thread Kevin Grittner
f its runtime waiting on the corresponding kernel mutexes." If you cannot duplicate their results, you might want to contact the authors for more details on their testing methodology. Note that the locking around access to the list is likely to be a bigger problem than the actual walking a

Re: [HACKERS] GSoC 2017 : Proposal for predicate locking in gist index

2017-05-31 Thread Kevin Grittner
ccessful split > operation. For that, we can insert a call for PredicateLockPageSplit() in > gistplacetopage(). That all sounds good. When you code a patch, don't forget to add tests to src/test/isolation. Do you need any help getting a development/build/test environment set up? -- Kevin Gr

[HACKERS] Re: transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-31 Thread Kevin Grittner
On Wed, May 31, 2017 at 1:44 AM, Noah Misch wrote: > IMMEDIATE ATTENTION REQUIRED. I should be able to complete review and testing by Friday. If there are problems I might not take action until Monday; otherwise I should be able to do so on Friday. -- Kevin Grittner VMware vCenter Ser

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-18 Thread Kevin Grittner
e reflected in the transition table. This needs to be fixed. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-16 Thread Kevin Grittner
On Tue, May 16, 2017 at 4:50 PM, Thomas Munro wrote: > On Wed, May 17, 2017 at 9:20 AM, Kevin Grittner wrote: >> On Wed, May 10, 2017 at 7:02 AM, Thomas Munro >> wrote: >>> On Wed, May 10, 2017 at 11:10 PM, Thomas Munro >>> wrote: >>>> 2. If y

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-16 Thread Kevin Grittner
combination of the function and the target relation, rather than having one cache entry regardless of the target table. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-15 Thread Kevin Grittner
. If your statement is ONLY on the a given level in the hierarchy, the row triggers for only that level would fire. If you don't use ONLY, a row trigger at that level would fire for operations at that level or any child level, but with a record format matching the level of the trigger. No

Re: [HACKERS] How huge does mvtest_huge need to be?

2017-05-03 Thread Kevin Grittner
On Wed, May 3, 2017 at 4:37 PM, Tom Lane wrote: > At this point I'm inclined to just delete the whole test. ok -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] How huge does mvtest_huge need to be?

2017-05-03 Thread Kevin Grittner
ject names then seems odd, of course. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Transition tables for triggers on foreign tables and views

2017-05-02 Thread Kevin Grittner
lar odd behavior under inheritance. I hope to post something Friday. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-01 Thread Kevin Grittner
On Mon, May 1, 2017 at 11:53 AM, Robert Haas wrote: > On Mon, May 1, 2017 at 12:10 PM, Kevin Grittner wrote: >> On Mon, May 1, 2017 at 10:01 AM, Robert Haas wrote: >>> It seems pretty clear to me that this is busted. >> >> I don't think you actually teste

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-01 Thread Kevin Grittner
h. I think best case is Friday. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Transition tables for triggers on foreign tables and views

2017-04-28 Thread Kevin Grittner
On Fri, Apr 28, 2017 at 3:54 PM, Kevin Grittner wrote: > Not firing a table's DML because it was fired off a view would be crazy, should read: Not firing a table's trigger because the trigger is on DML run from a view's INSTEAD OF trigger would be crazy, -- Kevin Gritt

Re: [HACKERS] Transition tables for triggers on foreign tables and views

2017-04-28 Thread Kevin Grittner
On Fri, Apr 28, 2017 at 2:42 PM, Tom Lane wrote: > Kevin Grittner writes: >> On Tue, Apr 25, 2017 at 6:17 PM, Thomas Munro >> wrote: >>> For views, aside from the question of transition tables, I noticed >>> that statement triggers don't seem to fire at

Re: [HACKERS] Transition tables for triggers on foreign tables and views

2017-04-28 Thread Kevin Grittner
attempt to do otherwise, or is there something to salvage that has worked at some point? If we do get these working, don't they deserve at least one regression test? Will post again after I have a chance to review the FDW issue. -- Kevin Grittner VMware vCenter Server https://www.vmwar

Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...

2017-04-17 Thread Kevin Grittner
rting with the list Craig included) as section headers, and put links to useful reading below each? -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] recent deadlock regression test failures

2017-04-10 Thread Kevin Grittner
On Mon, Apr 10, 2017 at 1:17 PM, Tom Lane wrote: > Kevin Grittner writes: >> On Mon, Apr 10, 2017 at 9:28 AM, Tom Lane wrote: >>> I notice that the safe-snapshot code path is not paying attention to >>> parallel-query cases, unlike the lock code path. I'm not

Re: [HACKERS] recent deadlock regression test failures

2017-04-10 Thread Kevin Grittner
ove rules are followed, nor can I see anything special it *could* do. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] recent deadlock regression test failures

2017-04-08 Thread Kevin Grittner
its array > argument directly; it could probably do that significantly faster than the > general-purpose array && code. So we'd have to expend a bit of backend C > code, but we'd have something that would be quite speedy and we could > customize in future without fea

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
t link to be following? If you're starting from the blocked (read-only) transaction (which you are), inLink is the one to follow. Note: It would be better form to use the SxactIsDeferrableWaiting() macro than repeat the bit-testing code directly in your function. -- Kevin Grittner -- Sen

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-04-07 Thread Kevin Grittner
On Mon, Feb 27, 2017 at 7:35 AM, Dagfinn Ilmari Mannsåker wrote: > Kevin Grittner writes: > >> It occurred to me that it would make sense to allow these settings >> to be attached to a database or table (though *not* a role). I'll >> look at that when I get back

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 3:47 PM, Thomas Munro wrote: > On Sat, Apr 8, 2017 at 6:35 AM, Kevin Grittner wrote: >> On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: >> >>> I'd rather fix the issue, than remove the tests entirely. Seems quite >>> possible t

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Kevin Grittner
enough I was planning on pushing this today. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: > I'd rather fix the issue, than remove the tests entirely. Seems quite > possible to handle blocking on Safesnapshot in a similar manner as > pg_blocking_pids? I'll see what I can figure out. -- Kevin Grittner

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
code, but it's probably not worth adding 4 hours to any tests, even if it only shows up with CLOBBER_CACHE_ONLY. I assume the consensus is that I should revert it? -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Kevin Grittner
Sorry, I didn't notice that this was going to a public list. That URL is only available to people who signed up as mentors for PostgreSQL GSoC participation this year. Does the link to the draft work for you? -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Kevin Grittner
proposal is here: https://summerofcode.withgoogle.com/serve/5874530240167936/ Also, I just entered a comment about an important question that I think needs to be answered right up front. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] delta relations in AFTER triggers

2017-04-04 Thread Kevin Grittner
On Mon, Apr 3, 2017 at 7:16 PM, Thomas Munro wrote: > On Tue, Apr 4, 2017 at 3:41 AM, Kevin Grittner wrote: >> On Mon, Apr 3, 2017 at 8:59 AM, Tom Lane wrote: >>> Thomas Munro writes: >>>> Or perhaps the code to inject trigger data transition tables into SPI >

Re: [HACKERS] SERIALIZABLE with parallel query

2017-04-03 Thread Kevin Grittner
ered by this testing, which at high concurrency on a 16 core machine might hit a particular problem less often than once per day. I also remember that both Anssi Kääriäinen and Yamamoto Takahashi did a lot of valuable testing of the patch and found problems that we had missed. Perhaps they can

Re: [HACKERS] GSoC 2017 Proposal for "Explicitly support predicate locks in index access methods besides btree"

2017-04-03 Thread Kevin Grittner
message-id/CAD=a8sbzwgd4u-tasjp0owtpileffhoz5o5ev0um6digqbv...@mail.gmail.com Unless you can produce convincing proof to the contrary, your proposal will be disqualified because of plagiarism. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] delta relations in AFTER triggers

2017-04-03 Thread Kevin Grittner
ing these 30-some lines of code to every PL. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2017 Proposal for "Explicitly support predicate locks in index access methods besides btree"

2017-04-01 Thread Kevin Grittner
g > exists in the btree AM, where it required the addition of 17 function calls > to implement, but remains unimplemented in the gin, gist, spgist, brin, and > hash index AMs. So we nned to insert function calls at other places also. Exactly. > Also tell me can we evaluate the perfor

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
es, where that seemed sensible. I really do *not* like >> "Enr" as a global name. > > Yeah, I had the same thought about capitalization but wasn't sure if > it was worth suggesting. But since Tom did, +1 from me. Will do. -- Kevin Grittner -- Sent via pgsql-hackers

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
On Thu, Mar 30, 2017 at 11:51 AM, Kevin Grittner wrote: > New version attached. It needs some of these problem cases added to > the testing, and a mention in the docs that only C and plpgsql > triggers can use the feature so far. I'll add those tomorrow. Done and attached. No

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
change and balked when I saw things like EphemeralNamedRelationMetadataData and a function named EphemeralNamedRelationMetadataGetTupDesc() in place of EnrmdGetTupDesc(). A 40 character function name make for a lot of line-wrapping to stay within pgindent limits. Any suggestions? -- Kev

Re: [HACKERS] delta relations in AFTER triggers

2017-03-30 Thread Kevin Grittner
f both. :-) Thanks for the reviews and the fixes! New version attached. It needs some of these problem cases added to the testing, and a mention in the docs that only C and plpgsql triggers can use the feature so far. I'll add those tomorrow. -- Kevin Grittner transition-v13.diff.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [pgsql-students] GSoC 2017 Proposal for "Explicitly support predicate locks in index access methods besides btree"

2017-03-30 Thread Kevin Grittner
A description of what they will do, and how - Milestones with dates - What they consider to be the test that they have successfully completed the project -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://

[HACKERS] Re: [pgsql-students] GSoC 2017 Proposal for "Explicitly support predicate locks in index access methods besides btree"

2017-03-30 Thread Kevin Grittner
hot); src/backend/access/nbtree/nbtsearch.c:1874: PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot); src/backend/access/nbtree/nbtpage.c:1410: PredicateLockPageCombine(rel, leafblkno, leafrightsib); -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-30 Thread Kevin Grittner
without generalization, but you would still have most of the same shared memory issues, since the structure *must* live in shared memory. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-27 Thread Kevin Grittner
new issues came to mind reading it again. (Nothing serious; just some points that could benefit from a little elaboration.) -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-24 Thread Kevin Grittner
r you or you need clarification. Overall, if the comments are addressed, I think it is great. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-23 Thread Kevin Grittner
/docs.google.com/document/d/17TAs3EJIokwPU7UTUmnlVY3ElB-VHViyX1zkQJmrD1A/edit?usp=sharing > > I was wondering if you have time to review the proposal and give me some > comments? Will take a look and give you an initial review in a day or two. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Candidate for local inline function?

2017-03-17 Thread Kevin Grittner
On Fri, Mar 17, 2017 at 3:23 PM, Andres Freund wrote: > On 2017-03-17 15:17:33 -0500, Kevin Grittner wrote: >> Why do we warn of a hazard here instead of eliminating said hazard >> with a static inline function declaration in executor.h? > > Presumably because it was w

[HACKERS] Candidate for local inline function?

2017-03-17 Thread Kevin Grittner
effects, modern versions of gcc have generated the same code for a macro versus a static inline function, at least in the cases I checked. -- Kevin Grittner

[HACKERS] Guidelines for GSoC student proposals

2017-03-17 Thread Kevin Grittner
I've found various sources that give hints about what a student proposal should look like, but nothing I could just give as a link, so I pulled together what I could find, tempered by my own ideas and opinions. I suggest that we send the below, or something like it to each student who expresses in

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Kevin Grittner
the memory, with heavy contention on shared memory, the situation is very different. Shared resource access time is going to be non-uniform no matter what you do. The difference is that the OS can still make *process local* allocations from nearby memory segments, while BIOS can

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Kevin Grittner
On Tue, Mar 14, 2017 at 3:45 PM, Kevin Grittner wrote: >> On 3/14/17 17:34, Mengxing Liu wrote: >>> There are several alternative benchmarks. Tony suggested that we >>> should use TPC-E and TPC-DS. > > More benchmarks is better, all other things being equal. Keep in

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-14 Thread Kevin Grittner
h --enable-cassert, you get the best stack traces and you can more easily break down execution profiles. That especially true if you disable optimization and don't omit frame pointers. But all of those things distort the benchmarks -- adding a lot of time, and not necessarily in proportion to where the time goes with an optimized build. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] delta relations in AFTER triggers

2017-03-13 Thread Kevin Grittner
e how that works in FOR EACH ROW triggers. We should match that, I think. Keep in mind that not just anyone can put a trigger on a table. > * In future we could consider teaching it about functional > dependencies as required by the spec; if you can SELECT id, name FROM > GROUP BY id, I

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-13 Thread Kevin Grittner
the project should be to develop a set of benchmarks we want to compare to at the end. That would need to include a stress test that clearly shows the problem we're trying to solve, along with some cases involving 1 or two connections -- just to make sure we don't harm performance for l

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-11 Thread Kevin Grittner
r more sockets with a large number of active connections. I don't know whether you have access to a machine capable of putting this kind of stress on it (perhaps at your university?), but if not, the community has access to various resources we should be able to schedule time on. It may pay for

Re: [HACKERS] GSOC Introduction / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-10 Thread Kevin Grittner
voting by the PostgreSQL mentors. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSOC Introduction / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-10 Thread Kevin Grittner
iewing patches is a good way to become involved and familiar with the community processes. https://wiki.postgresql.org/wiki/CommitFest -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] SQL Standard Feature T211

2017-03-09 Thread Kevin Grittner
ity" on our "unsupported" list until those two points are addressed? Does anyone know of anything else that would need to be done? -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] delta relations in AFTER triggers

2017-03-09 Thread Kevin Grittner
On Tue, Mar 7, 2017 at 6:28 PM, Kevin Grittner wrote: > New patch attached. And bit-rotted less than 24 hours later by fcec6caa. New patch attached just to fix bit-rot. That conflicting patch might be a candidate to merge into the new Ephemeral Named Relation provided by my patch, for m

Re: [HACKERS] delta relations in AFTER triggers

2017-03-07 Thread Kevin Grittner
ter_relation here? Essentially, because plpgsql does some things through SPI and some things not. Both cases are covered. > On Mon, Dec 19, 2016 at 4:35 AM, Thomas Munro > wrote: >> On Sun, Dec 18, 2016 at 3:15 PM, Kevin Grittner wrote: >>> There were a few changes Thom

[HACKERS] WARNING: relcache reference leak: relation "p1" not closed

2017-03-06 Thread Kevin Grittner
: relcache reference leak: relation "p1" not closed VACUUM -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WARNING: relcache reference leak: relation "p1" not closed

2017-03-06 Thread Kevin Grittner
[original message held up for review -- should be along eventualy...] On Mon, Mar 6, 2017 at 3:11 PM, Kevin Grittner wrote: > With e434ad39ae7316bcf35fd578dd34ad7e1ff3c25f I did a `make world`, > `make install-world`, a fresh default initdb, a start with default > config, `make ins

Re: [HACKERS] delta relations in AFTER triggers

2017-03-02 Thread Kevin Grittner
gt; parserSetupArg=0x7fa58185c2a0, queryEnv=0x7fa581857798) + 128 >> at postgres.c:706 [opt] > > Do you know when you will have a new patch ready? > > This looks like an interesting and important feature but I think it's > going to have to come together quickly i

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-09 Thread Kevin Grittner
re code is concerned, 3b and 3c are the same thing. I think so, too, if we're talking about an extension in core. My vote is for 3b. If someone wants to write the alias extension and make it available outside of core, fine -- though they don't need anyone's vote to do so. -- Kevin

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-06 Thread Kevin Grittner
id S, Michael P (with > aliases for functions, I could live without at this point...) I vote for 3 as well, with 1 as the only sane alternative. -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-03 Thread Kevin Grittner
inishing the job is better. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-01-23 Thread Kevin Grittner
; max_predicate_locks_per_transaction / -max_predicate_locks_per_relation. > Making the default value -2 keeps backwards compatibility. > > Alternatively we could have a separate setting for the fraction (which > could then be a floating-point number, for finer control), and use the > ab

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-01-23 Thread Kevin Grittner
On Mon, Jan 23, 2017 at 9:50 AM, Dagfinn Ilmari Mannsåker wrote: > Kevin Grittner writes: > >> (1) The new GUCs are named max_pred_locks_per_*, but the >> implementation passes them unmodified from a function specifying at >> what count the lock should be promoted.

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-01-20 Thread Kevin Grittner
like effective_cache_size. Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-01-20 Thread Kevin Grittner
t week, so I would not do so before then; in fact I may not have a chance to respond to any comments before then. If the author wishes to make these changes, great; otherwise I can do it before commit. I will mark this Ready for Committer. -- Kevin Grittner EDB: http://www.enterprisedb.com The En

Re: [HACKERS] delta relations in AFTER triggers

2017-01-20 Thread Kevin Grittner
On Fri, Jan 20, 2017 at 2:08 PM, Nico Williams wrote: > On Fri, Jan 20, 2017 at 01:37:33PM -0600, Kevin Grittner wrote: >> There is currently plenty of room for pseudo-MV implementations, >> and may be for a while. It's a good indication of the need for the >

Re: [HACKERS] delta relations in AFTER triggers

2017-01-20 Thread Kevin Grittner
me "higher order" SQL > features that make this sort of thing easier. In particular, here, row > values and record types, and being able to refer to a table as a column > of the table's record type. Yeah, I found that quite handy in developing the REFRESH feature, and expect t

Re: [HACKERS] delta relations in AFTER triggers

2017-01-20 Thread Kevin Grittner
Attached is v10 which fixes bitrot from v9 caused by ea15e186. Still needs review. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company transition-v10.diff.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] delta relations in AFTER triggers

2017-01-19 Thread Kevin Grittner
Attached is v9 which fixes bitrot from v8. No other changes. Still needs review. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company transition-v9.diff.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Packages: Again

2017-01-13 Thread Kevin Grittner
documents. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-13 Thread Kevin Grittner
not present a similar hazard. So while I agree that there is no harm in including current_logfiles in base backups, I object to the comparisons to the more dangerous files. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing l

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-12 Thread Kevin Grittner
f pg_proc.h will not be any harder than removing > a contrib directory would be. How about just leaving it to someone who cares about the aliases to post such an extension at pgxn.org (or anywhere else they like). It can be around as long as someone cares enough to maintain it

Re: [DOCS] [HACKERS] Questionable tag usage

2017-01-10 Thread Kevin Grittner
ight question whether the benefit was worth the effort of adding it. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] RustgreSQL

2017-01-10 Thread Kevin Grittner
rmance boost which would not be available if they were instead converted to another language. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Odd behavior with PG_TRY

2017-01-06 Thread Kevin Grittner
s immune to the problem? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

2017-01-06 Thread Kevin Grittner
vement in performance. Is there compression and/or encryption being performed by the network layers? My experience with both is that they run faster on bigger chunks of data, and that might happen before the data is broken into packets. -- Kevin Grittner EDB: http://www.enterprisedb.com The E

Re: [HACKERS] delta relations in AFTER triggers

2016-12-17 Thread Kevin Grittner
ggested change (e.g., the inclusion of the query environment in the parameter list of a few more functions). Changed to "Needs review" status. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company transition-v8.diff.gz Description: GNU Zip compressed data

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-16 Thread Kevin Grittner
licit read-only transactions. It is still worthwhile to explicitly set serializable transactions to read-only whenever possible, for performance reasons. The idea that a serialization failure is not possible on the first (or only) statement o a read-only transaction was in error, and should be disrega

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-16 Thread Kevin Grittner
On Fri, Dec 16, 2016 at 8:24 AM, Robert Haas wrote: > On Thu, Dec 15, 2016 at 9:01 AM, Kevin Grittner wrote: >> I also realized some other properties of read-only transactions >> that might interest you (and that I should probably document). >> Since the only way for a rea

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-15 Thread Kevin Grittner
only transaction had multiple statements; but the fact that such long saturation runs with SLRU summarization showed no errors on read-only transactions supports the idea that such summarization doesn't compromise that guarantee. Unfortunately, it falls short of proof. :-( -- Kevin Grittn

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-15 Thread Kevin Grittner
odified to correct serious bugs or security vulnerabilities, in order to make it as safe as possible for people to apply minor releases without fear of breaking something that works, I think we could consider an argument for back-patching a doc change that clarifies or fills omissions that make it diff

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-14 Thread Kevin Grittner
On Wed, Dec 14, 2016 at 11:12 AM, Ian Jackson wrote: > Kevin Grittner writes ("Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: > Retry on constraint violation [and 2 more messages] [and 1 more messages]"): >> On Wed, Dec 14, 2016 at 10:20 AM, Ian Jackson >> w

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation [and 2 more messages] [and 1 more messages]

2016-12-14 Thread Kevin Grittner
terminate the transaction; however, (to step into very convoluted territory) if that other exception is caught and suppressed, the serialization failure error would occur. Once a serialization failure occurs the transaction is flagged as "doomed" and will not, under any circumstances be allow

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2016-12-14 Thread Kevin Grittner
On Wed, Dec 14, 2016 at 8:16 AM, Dagfinn Ilmari Mannsåker wrote: > Attached is a patch Please add this to the open CommitFest to ensure that it is reviewed. http://commitfest.postgresql.org/action/commitfest_view/open -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterpr

  1   2   3   4   5   6   7   8   9   10   >