Re: [HACKERS] Freeze avoidance of very large table.

2015-07-01 Thread Simon Riggs
. They are essential for reviewers, not a cosmetic thing to be added later. To gain wide agreement we need wide understanding. (I recommend a development approach where you write the comments first, then add code later.) -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 09:00, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jun 30, 2015 at 12:32 PM, Simon Riggs si...@2ndquadrant.com wrote: ClogControlLock contention is high at commit time. This appears to be due to the fact that ClogControlLock is acquired in Exclusive mode prior

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-01 Thread Simon Riggs
, but measuring progress of VACUUM wouldn't be it, IMHO. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 08:22, Simon Riggs si...@2ndquadrant.com wrote: This contention is masked by contention elsewhere, e.g. ProcArrayLock, so the need for testing here should come once other patches ahead of this are in. Let me explain more clearly. Andres' patch to cache snapshots

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-06-30 Thread Simon Riggs
first, then we can begin adding other operations as we work out how (for that operation). -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 08:13, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Jun 30, 2015 at 4:02 PM, Simon Riggs si...@2ndquadrant.com wrote: ClogControlLock contention is high at commit time. This appears to be due to the fact that ClogControlLock is acquired in Exclusive mode

[HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
in clog, which would require a barrier. Two concurrent writers might access the same word concurrently, so we protect against that with a new CommitLock. We could partition that by pageno also, if needed. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 03:43, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 29, 2015 at 1:22 PM, Simon Riggs si...@2ndquadrant.com wrote: Yes, I know. And we all had a long conversation about how to do it without waking up the other procs. Forming a list, like we use for sync rep

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 07:34, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jun 30, 2015 at 11:00 AM, Simon Riggs si...@2ndquadrant.com wrote: On 30 June 2015 at 05:02, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs si...@2ndquadrant.com wrote

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-30 Thread Simon Riggs
been sitting on a patch for a month or so and will post now I'm done travelling. These ideas have been around some time and are even listed on the PostgreSQL TODO: http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php -- Simon Riggshttp://www.2ndQuadrant.com/ http://www

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-30 Thread Simon Riggs
. +1 -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 28 June 2015 at 17:17, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 27 June 2015 at 15:10, Tom Lane t...@sss.pgh.pa.us wrote: I don't like this too much because it will fail badly if the caller is wrong about the maximum possible page number

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
is implemented exactly the way I said to implement it publicly at PgCon. Was nobody recording the discussion at the unconference?? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 29 June 2015 at 18:11, Andres Freund and...@anarazel.de wrote: On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs si...@2ndquadrant.com wrote: On 29 June 2015 at 16:27, Amit Kapila amit.kapil...@gmail.com wrote: Thanks to Robert Haas for having discussion (offlist) about the idea

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 05:02, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On 28 June 2015 at 17:17, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure what you consider dire, but missing a dirty buffer belonging

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-28 Thread Simon Riggs
destroyed by the user. So ISTM that we should be able to use this technique. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] UPSERT on partition

2015-06-26 Thread Simon Riggs
? Or we should just treat it as a limitation of UPSERT and add that document? +1 There are many problems that cannot be resolved for 9.5. UPSERT works fine with tables with BRIN indexes. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-06-25 Thread Simon Riggs
it does not. You would find the old grammar with: 1[AAA,BBB,CCC] Let's start with a complex, fully described use case then work out how to specify what we want. I'm nervous of it would be good ifs because we do a ton of work only to find a design flaw. -- Simon Riggshttp://www

Re: [HACKERS] Multixid hindsight design

2015-06-24 Thread Simon Riggs
should specify the on-disk tuple representation as a state machine and work out how to recheck the new on-disk state matches the state transition that we performed. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] Multixid hindsight design

2015-06-24 Thread Simon Riggs
On 24 June 2015 at 16:30, Simon Riggs si...@2ndquadrant.com wrote: Though TED sounds nice, the way to avoid another round of on-disk bugs is by using a new kind of testing, not simply by moving the bits around. It might be argued that we are increasing the diagnostic/forensic capabilities

Re: [HACKERS] Trustly PostgreSQL Data Corruption Bug Bounty Program

2015-06-24 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Extension support for postgres_fdw

2015-06-20 Thread Simon Riggs
like that, but currently we handle things in terms of Schemas. It would be strange to have differing ways of specifying groups of objects. Maybe that's not a problem, but we'd probably need to analyse that to make sure it didn't make things more complex. -- Simon Riggshttp://www

Re: [HACKERS] Is it possible to have a fast-write Index?

2015-06-19 Thread Simon Riggs
- which is now covered by BRIN. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-12 Thread Simon Riggs
On Thu, Jun 11, 2015 at 6:17 PM, Simon Riggs si...@2ndquadrant.com wrote: On 11 June 2015 at 16:56, Shay Rojansky r...@roji.org wrote: Npgsql (currently) sends Parse for the second command before sending Execute for the first one. Look no further than that. -- Simon Riggshttp

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-11 Thread Simon Riggs
see a test case? Most commonly in such cases the first request failed and error messages weren't checked before running second message. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Simon Riggs
proposal. The balance of coordinating release activities is mechanical, and -packagers seems adequate for it. Packagers should be about HOW do we make the next release, which is separate from the above. Ultimately, How effects When, but When is it needed? is an earlier thought. -- Simon Riggs

Re: [HACKERS] Is it possible to have a fast-write Index?

2015-06-11 Thread Simon Riggs
because the index could get out of sync, so it can omit results and this is unacceptable. But i think maybe that could be fixed in several ways and we can have a fast and reliable index (but maybe not so fast on selects). This is exactly the use case and mechanism for BRIN indexes. -- Simon

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-11 Thread Simon Riggs
On 11 June 2015 at 16:56, Shay Rojansky r...@roji.org wrote: Npgsql (currently) sends Parse for the second command before sending Execute for the first one. Look no further than that. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Simon Riggs
the xmax for something useful in a larger insert-mostly database rather than just leaving it at zero. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-06 Thread Simon Riggs
On 5 June 2015 at 17:20, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs wrote: On 5 June 2015 at 15:00, Robert Haas robertmh...@gmail.com wrote: Stamping it a beta implies that we think it's something fairly stable that we'd be pretty happy to release if things go well

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
, how did it work before 9.3? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
so, 2ndQuadrant has zero income tied to the release of 9.5 or the commit of any feature, so as far as that company is concerned, the release could wait for 10 years. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] Restore-reliability mode

2015-06-05 Thread Simon Riggs
/stop-the-line/ So lets do our normal things, not do a total stop for an indefinite period. If someone has specific things that in their opinion need to be addressed, list them and we can talk about doing them, together. I thought that was what the Open Items list was for. Let's use it. -- Simon

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
. That way if we need to make Offsets SLRU persistent it won't bloat. We then leave the Members SLRU as non-persistent, just as it was 9.3 -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 11:02, Andres Freund and...@anarazel.de wrote: On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Simon Riggs
for. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
features that fix real world problems, just as much as bug fixes do, hence why I don't wish to divert from the normal process and schedule. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 15:00, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Agreed. Cleanup can occur while we release code for public testing. The code is available for public testing right now. People test when they get

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [CORE] [HACKERS] postpone next week's release

2015-06-04 Thread Simon Riggs
and it works we release it. If doesn't, we pull it. Not releasing our software yet making a list of our fears doesn't work towards a solution. Our fears will make us shout at each other too, so I for one would rather skip that part and do some practical actions. -- Simon Riggshttp://www

Re: [HACKERS] Need Force flag for pg_drop_replication_slot()

2015-05-29 Thread Simon Riggs
On 29 May 2015 at 18:15, Josh Berkus j...@agliodbs.com wrote: While I'm just doing this during testing That part is good. I'm sure you will find something in need of improvement. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development

Re: [HACKERS] Need Force flag for pg_drop_replication_slot()

2015-05-29 Thread Simon Riggs
want it you can write a function to do that for private use. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] pg_upgrade resets timeline to 1

2015-05-28 Thread Simon Riggs
like the change to TimeLine 1 is just a kludge anyway. The rule that TimeLine 1 doesn't need a history file is itself a hack. What we should be saying is that the last timeline doesn't need a history file. Then no change is needed here. -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.

2015-05-27 Thread Simon Riggs
is that the command tag should vary according to whether an INSERT or an UPDATE was performed, so we get a visible difference without any new tags. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.

2015-05-27 Thread Simon Riggs
On 27 May 2015 at 15:06, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs wrote: What I think should happen is that the command tag should vary according to whether an INSERT or an UPDATE was performed, so we get a visible difference without any new tags. The problem

Re: [HACKERS] Order of columns in query is important?!

2015-05-26 Thread Simon Riggs
specified columns *exactly* matches the tuple descriptor). ...and it matters a lot in those cases because we are sorting all of the data scanned, not just 25 rows. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
to be defined on the table - is handled gracefully. What has CREATE INDEX CONCURRENTLY got to do with this? If you don't specify the conflict-target at all, it wouldn't matter what the indexes are. If you have two indexes the same then it clearly wouldn't matter which one was checked first. -- Simon

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
will happen in later releases when they ask. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
On 19 May 2015 at 19:59, Peter Geoghegan p...@heroku.com wrote: On Tue, May 19, 2015 at 2:28 PM, Simon Riggs si...@2ndquadrant.com wrote: On 19 May 2015 at 17:10, Peter Geoghegan p...@heroku.com wrote: On Tue, May 19, 2015 at 1:57 PM, Simon Riggs si...@2ndquadrant.com wrote: We

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
On 20 May 2015 at 05:49, Geoff Winkless pgsqlad...@geoff.dj wrote: On 19 May 2015 at 21:57, Simon Riggs si...@2ndquadrant.com wrote: It's not clear to me how a single INSERT could cause two or more UPDATEs. ​ CREATE TABLE mytable ( c1 int NOT NULL, c2 int NOT NULL, PRIMARY KEY (c1

Re: [HACKERS] a few thoughts on the schedule

2015-05-20 Thread Simon Riggs
in our thinking how we nurture and grow reviewers, contributors and committers. I am more likely to treat a low-value patch seriously if it is an early contribution from someone, for example. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
you apply the change of unique constraint. 2) Compatibility with MySQL -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
On 19 May 2015 at 17:10, Peter Geoghegan p...@heroku.com wrote: On Tue, May 19, 2015 at 1:57 PM, Simon Riggs si...@2ndquadrant.com wrote: We should allow DO UPDATE to exclude a constraint and apply a deterministic order to the constraints. 1. PK if it exists. 2. Replica Identity, when

Re: [HACKERS] RFC: Non-user-resettable SET SESSION AUTHORISATION

2015-05-19 Thread Simon Riggs
On 19 May 2015 at 16:49, Robert Haas robertmh...@gmail.com wrote: On Tue, May 19, 2015 at 3:00 PM, Simon Riggs si...@2ndquadrant.com wrote: As long as the cookie is randomly generated for each use, then I don't see a practical problem with that approach. If the client sets the cookie via

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
if multiple constraints end up triggering for the same INSERT, it would require UPDATEs of multiple rows. Is that the issue? I'm sure we'll be asked these questions many times. Can you comment on whether the docs are sufficiently detailed to explain this answer? -- Simon Riggshttp

Re: [HACKERS] a few thoughts on the schedule

2015-05-19 Thread Simon Riggs
. It's quicker than normal, but if we've lost a month or two we should just skip the usual open items chase, which can be done in parallel with users finding and reporting real bugs. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] ERROR: cannot GetMultiXactIdMembers() during recovery

2015-05-18 Thread Simon Riggs
with waiting. That's a very low risk fix. It's more like a should-have-been-a-basic-check. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Simon Riggs
number of MVs in play that search would not be a problem in practice. I'm also aware that LIMIT is still very badly optimized, so I'm hoping it helps there also. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] feature freeze and beta schedule

2015-05-15 Thread Simon Riggs
On 1 May 2015 at 18:05, Simon Riggs si...@2ndquadrant.com wrote: * TABLESAMPLE clause Doesn't seem very far from being done. Some questions about including (or not) DDL and contrib modules seem to remain. Will commit this soon OK, completely happy with this now and will commit today

Re: [HACKERS] ERROR: cannot GetMultiXactIdMembers() during recovery

2015-05-15 Thread Simon Riggs
change for 9.3 so I don't see what the objection is. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Sequence Access Method WIP

2015-05-13 Thread Simon Riggs
thought. There is one clear use case for this and it is of benefit to many distributed architectures. I don't see what calamity will occur if we commit this. If you don't want a sequence AM, don't ever use this. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Simon Riggs
and have them enabled by default with no parameter to disable them. In the early years of PostgreSQL everything had an off switch, e.g. stats, bgwriter and even autovacuum defaulted to off for many years. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL

Re: [HACKERS] subxcnt defined as signed integer in SnapshotData and SerializeSnapshotData

2015-05-08 Thread Simon Riggs
not set to -1 at overflow anymore? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] subxcnt defined as signed integer in SnapshotData and SerializeSnapshotData

2015-05-08 Thread Simon Riggs
On 8 May 2015 at 13:02, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, May 8, 2015 at 3:55 PM, Simon Riggs si...@2ndquadrant.com wrote: On 7 May 2015 at 21:40, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, Coverity is complaining about the following assertion

Re: [HACKERS] feature freeze and beta schedule

2015-05-01 Thread Simon Riggs
said he'll have a look. Maybe me, but running out of time * TABLESAMPLE clause Doesn't seem very far from being done. Some questions about including (or not) DDL and contrib modules seem to remain. Will commit this soon -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Simon Riggs
-HOT update, causing the index to bloat. It seems better to wait for a short period to see if we can get the cleanup lock. The short period is currently 0, so lets start there and vary the duration of wait upwards proportionally as the index gets more bloated. -- Simon Riggshttp://www

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-04-29 Thread Simon Riggs
the standard says we will eventually support, so we should be acting with an eye to that future. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Feedback on getting rid of VACUUM FULL

2015-04-25 Thread Simon Riggs
be set manually to enforce bloat minimization. We can also design a utility to actively use TARGBLOCK_NEW and FSM_SHRINK to reduce table size without blocking writes. But this is all stuff for 9.6... -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Simon Riggs
1000. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Replication identifiers, take 4

2015-04-21 Thread Simon Riggs
of replication identifier - replication origin to the patch. Or even transaction origin. Sounds good to me. +1 -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Logical Decoding follows timelines

2015-04-21 Thread Simon Riggs
approach to be coded for next release. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Simon Riggs
and then and does it thing. Why should the one massive SELECT have horrible performance just because it was run right before autovacuum would have kicked in instead of right after if finished? +1 -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Simon Riggs
bits or dirtied the block in other ways. Since we have many votes in favour of change in this area I'll post a new version and look for an early review/commit for next release. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Replication identifiers, take 4

2015-04-17 Thread Simon Riggs
On 17 April 2015 at 09:54, Andres Freund and...@anarazel.de wrote: Hrmpf. Says the person that used a lot of padding, without much discussion, for the WAL level infrastructure making pg_rewind more maintainable. Sounds bad. What padding are we talking about? -- Simon Riggs

Re: [HACKERS] Replication identifiers, take 4

2015-04-17 Thread Simon Riggs
resilience, it is a sensible use of bytes. +1 to Andres' very reasonable suggestion. Lets commit this and go home. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Simon Riggs
On 16 April 2015 at 15:21, Andres Freund and...@anarazel.de wrote: On 2015-04-16 10:20:20 -0300, Alvaro Herrera wrote: I think you're failing to consider that in the patch there is a distinction between read-only page accesses and page updates. During a page update, HOT cleanup is always

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 08:04, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 15, 2015 at 3:37 AM, Simon Riggs si...@2ndquadrant.com wrote: On 14 April 2015 at 21:53, Robert Haas robertmh...@gmail.com wrote: Peter commented previously that README.HOT should get an update. The relevant

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 09:10, Andres Freund and...@anarazel.de wrote: On 2015-04-15 08:42:33 -0400, Simon Riggs wrote: Because it makes it subsequent accesses to the page cheaper. Cheaper for whom? Everyone. I think what you mean is Everyone else. It is demonstrably quicker and more

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 12:39, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 15, 2015 at 8:42 AM, Simon Riggs si...@2ndquadrant.com wrote: I won't take responsibility for paying my neighbor's tax bill, but I might take responsibility for picking up his mail while he's on holiday

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 16:01, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs wrote: On 15 April 2015 at 12:39, Robert Haas robertmh...@gmail.com wrote: On 04/15/2015 05:44 PM, Alvaro Herrera wrote: Robert's proposal is when reading a page, if dirty HOT-clean it; if not dirty, also

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
be a useful heuristic to still prune pages if those pages are already dirty. Useful for who? This is about responsibility. Why should someone performing a large SELECT take the responsibility for cleaning pages? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] [COMMITTERS] pgsql: Use Intel SSE 4.2 CRC instructions where available.

2015-04-14 Thread Simon Riggs
good builds later if desired. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-14 Thread Simon Riggs
to add/remove a parameter to control this. Currently there isn't one. I'd like to commit this. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services hot_disable.v9.patch Description: Binary data -- Sent via pgsql-hackers

Re: [HACKERS] TABLESAMPLE patch

2015-04-12 Thread Simon Riggs
be technically possible with what we have here, but I think a lot more thought required yet. These have all come out of detailed discussions with two different groups of data mining researchers. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA

Re: [HACKERS] TABLESAMPLE patch

2015-04-09 Thread Simon Riggs
On 9 April 2015 at 04:52, Simon Riggs si...@2ndquadrant.com wrote: TABLESAMPLE BERNOULLI could work in this case, or any other non-block based sampling mechanism. Whether it does work yet is another matter. This query should be part of the test suite and should generate a useful message

Re: [HACKERS] TABLESAMPLE patch

2015-04-09 Thread Simon Riggs
work in this case, or any other non-block based sampling mechanism. Whether it does work yet is another matter. This query should be part of the test suite and should generate a useful message or work correctly. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
were generated, but not which ones they were. IMHO that makes the resulting audit trail unusable for auditing purposes. I would like to see that functionality put back before it gets committed, if that occurs. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
preference also. My concern was raised when it was *removed* without confirming others agreed. Typical questions: Who has written to table X? Who has read data from table Y yesterday between time1 and time2? Has anyone accessed a table directly, rather than through a security view? -- Simon Riggs

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Simon Riggs
be one of list. Are we ready for a final detailed review and commit? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-04-05 Thread Simon Riggs
On 5 April 2015 at 19:19, Michael Paquier michael.paqu...@gmail.com wrote: Cool! Thanks for showing up. Visibility Activity. How is REINDEX CONCURRENTLY doing? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-04-05 Thread Simon Riggs
the same banner as AT_AddConstraint. Thoughts? A new version of the patch is attached which treats them as the same for locking. I think it is correct and improves readability to do so. Committed. We move forwards, slowly but surely. Thanks for the patch. -- Simon Riggs

Re: [HACKERS] Combining Aggregates

2015-03-30 Thread Simon Riggs
On 30 March 2015 at 01:08, David Rowley dgrowle...@gmail.com wrote: On 18 December 2014 at 02:48, Simon Riggs si...@2ndquadrant.com wrote: David, if you can update your patch with some docs to explain the behaviour, it looks complete enough to think about committing it in early January

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-18 Thread Simon Riggs
a large chunk of the use cases. If it wasn't clear, I had already said that my idea was for next release, before Robert said that. I used to have this quote (or something close to it) on my whiteboard... I think it's appropriate here ;) The perfect is the enemy of the good. -Simon Riggs

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-16 Thread Simon Riggs
On 16 March 2015 at 12:48, Kevin Grittner kgri...@ymail.com wrote: Simon Riggs si...@2ndquadrant.com wrote: On 13 March 2015 at 15:41, Kevin Grittner kgri...@ymail.com wrote: The feedback was generally fairly positive except for the fact that snapshot age (for purposes of being too old

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-16 Thread Simon Riggs
actually get it. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Removing INNER JOINs

2015-03-16 Thread Simon Riggs
. If I had kept the AlternativePlan node around then the EXPLAIN output would have 2 plans, both sitting under the AlternativePlan node. I guess that is at least compatible with how we currently handle other join elimination, so that is acceptable to me. -- Simon Riggs http://www

Re: [HACKERS] recovery_target_action = pause hot_standby = off

2015-03-15 Thread Simon Riggs
. We know that the *only* possible other setting is 'shutdown', so it seems more user friendly to do the thing we *know* they want (1), rather than pretend that we don't. (1) is completely predictable and not at all surprising. Add a LOG message if you wish, but don't throw an error. -- Simon

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-14 Thread Simon Riggs
On 13 March 2015 at 13:17, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 13, 2015 at 8:52 AM, Simon Riggs si...@2ndquadrant.com wrote: On 15 February 2015 at 00:19, Kevin Grittner kgri...@ymail.com wrote: What they wanted was what happened in the other database product -- if a snapshot

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-03-14 Thread Simon Riggs
as there is agreement to improve this in the future -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-14 Thread Simon Riggs
the size of the bloat we care about, not the time. So we should be thinking in terms of limits that we actually care about. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training Services -- Sent via pgsql-hackers mailing list

<    5   6   7   8   9   10   11   12   13   14   >