Re: [HACKERS] Test code is worth the space

2015-08-12 Thread Simon Riggs
. (That is, if the function belongs in contrib for other reasons, put tests calling it in the contrib module itself.) Place non-user-visible test support functions in regress.c, or use one of the options Robert described. That helps, thanks. -- Simon Riggshttp://www

Re: [HACKERS] optimizing vacuum truncation scans

2015-08-12 Thread Simon Riggs
. Kept simple, its a patch with a clear win in a restricted use case and I would be happy to commit that sometime in the future. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2015-08-12 Thread Simon Riggs
returning this with feedback. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-11 Thread Simon Riggs
many blocks we need to scan. I think just storing (total num blocks, scanned blocks) is sufficiently accurate to be worth holding, rather than make it even more complex. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] max_connections and standby server

2015-08-11 Thread Simon Riggs
that could have happened, but that time has been spent on the more useful logical replication which is slowly making its way into core. More coming in 9.6. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-11 Thread Simon Riggs
On 11 August 2015 at 10:55, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Jul 1, 2015 at 3:49 PM, Simon Riggs si...@2ndquadrant.com wrote: On 1 July 2015 at 11:14, Andres Freund and...@anarazel.de wrote: On 2015-07-01 09:08:11 +0100, Simon Riggs wrote: On 1 July 2015 at 09:00

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-11 Thread Simon Riggs
of CommitLock? That prevents read only access, not just commits, so that isn't a better suggestion. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-11 Thread Simon Riggs
On 11 August 2015 at 11:39, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Aug 11, 2015 at 3:44 PM, Simon Riggs si...@2ndquadrant.com wrote: On 11 August 2015 at 10:55, Amit Kapila amit.kapil...@gmail.com wrote: What tricks are being used?? Please explain why taking 2 locks is bad

Re: [HACKERS] 64-bit XIDs again

2015-08-11 Thread Simon Riggs
first we just increment the Epoch. By doing this we can reuse the page-level epoch for both XID and CSN. Now hinting a tuple is just replacing a 32-bit XID with a 32-bit CSN. We would probably need an extra flag bit for the case where the CSN is one epoch later than the XID. -- Simon Riggs

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-08-10 Thread Simon Riggs
with multixact logs again? Just thought I'd open a papercut and pour some lemon juice on it. I doubt we have seen the last of that pain, but its not my fingers on the chopping board, so squeeze all you want. ;-) -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-10 Thread Simon Riggs
not wonderful, but la vida es loca. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Test code is worth the space

2015-08-10 Thread Simon Riggs
, advice needed. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-08-10 Thread Simon Riggs
On 10 August 2015 at 07:14, Peter Geoghegan p...@heroku.com wrote: On Sun, Aug 9, 2015 at 11:03 PM, Simon Riggs si...@2ndquadrant.com wrote: If 5) fails to bring a workable solution by the Jan 2016 CF then we commit 2) instead. Is there actually a conflict there? I didn't think so. I

[HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-08-10 Thread Simon Riggs
then we commit 2) instead. If Heikki wishes to work on (5), that's good. Otherwise, I think its something I can understand and deliver by 1 Jan, though likely for 1 Nov CF. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] Test code is worth the space

2015-08-10 Thread Simon Riggs
On 10 August 2015 at 13:55, Robert Haas robertmh...@gmail.com wrote: On Mon, Aug 10, 2015 at 2:02 AM, Simon Riggs si...@2ndquadrant.com wrote: On another review I suggested we add a function to core to allow it to be used in regression tests. A long debate ensued, deciding that we must

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-06 Thread Simon Riggs
give some kind of performance problem, which isn't the case here. On a table 100x larger, the sample is 1000 +/- 40, which seems more acceptable. The variability of Bernoulli is much less, but then it is roughly 100x slower. On a 1TB table, SYSTEM is going to be your only choice. -- Simon Riggs

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-06 Thread Simon Riggs
On 6 August 2015 at 21:14, Josh Berkus j...@agliodbs.com wrote: On 08/06/2015 01:10 PM, Simon Riggs wrote: Given, user-stated probability of accessing a block of P and N total blocks, there are a few ways to implement block sampling. 1. Test P for each block individually. This gives

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-06 Thread Simon Riggs
possible to return zero rows from a populated table. Please bear in mind you have requested a very small random sample of blocks. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-08-06 Thread Simon Riggs
sure we are ready with diagnostic tools to allow run-time checks and confirm everything is good. If people feel that means I've asked for something in the wrong place, I am happy to skip that request and place it wherever requested. -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Simon Riggs
On 4 August 2015 at 14:55, Heikki Linnakangas hlinn...@iki.fi wrote: On 08/04/2015 04:35 PM, Simon Riggs wrote: This and the OP seem like 9.5 open items to me. Why? This is nothing new in 9.5. gin_pending_list_limit is new in 9.5 We're in Alpha, so if something has been added and isn't

Re: [HACKERS] tablecmds.c and lock hierarchy

2015-08-04 Thread Simon Riggs
now that can't be solved by saying Assert(locklevel==ShareUpdateExclusiveLock || locklevelShareRowExclusiveLock); -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Simon Riggs
On 4 August 2015 at 09:39, Simon Riggs si...@2ndquadrant.com wrote: On 4 August 2015 at 06:03, Jeff Janes jeff.ja...@gmail.com wrote: The attached proof of concept patch greatly improves the bloat for both the insert and the update cases. You need to turn on both features: adding the pages

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Simon Riggs
On 4 August 2015 at 15:18, Andres Freund and...@anarazel.de wrote: On 2015-08-04 14:59:11 +0100, Simon Riggs wrote: On 4 August 2015 at 14:55, Heikki Linnakangas hlinn...@iki.fi wrote: On 08/04/2015 04:35 PM, Simon Riggs wrote: This and the OP seem like 9.5 open items to me

Re: [HACKERS] FSM versus GIN pending list bloat

2015-08-04 Thread Simon Riggs
this. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Minimum tuple threshold to decide last pass of VACUUM

2015-08-03 Thread Simon Riggs
-wraparound VACUUMs we shouldn't scan indexes at all, since they aren't critical path activities at that point * For normal VACUUMs we should scan indexes only if (num_dead_tuples * 20) (blocks to be scanned in any one index), which allows some index bloat but not much -- Simon Riggs

Re: [HACKERS] 64-bit XIDs again

2015-08-01 Thread Simon Riggs
traffic or I/O. In practice, larger clog would only happen with higher transaction rate, which means more system resources, so I don't think its too much of a problem overall. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] brin index vacuum versus transaction snapshots

2015-07-31 Thread Simon Riggs
that the default_transaction_isolation is read committed and so the test fails at other levels. If anything, it is the test that is at fault. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] brin index vacuum versus transaction snapshots

2015-07-31 Thread Simon Riggs
-committed or repeatable read makes no difference. Does brin_summarize_new_values() need to track what it reads for serializable transactions? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Doubt about AccessExclusiveLock in ALTER TABLE .. SET ( .. );

2015-07-31 Thread Simon Riggs
just added. That's too much. Why do we need more isolation tests? There isn't anything critical here, its just different lock levels for ALTER TABLE. A few normal regression tests are fine for this. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] 64-bit XIDs again

2015-07-31 Thread Simon Riggs
* changes to allow xmin to be reused when freeze bit set Very little additional path length in the common case. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

[HACKERS] Re: Using quicksort and a merge step to significantly improve on tuplesort's single run external sort

2015-07-30 Thread Simon Riggs
it because Heikki's comment that we could use this in the general case would not be true for larger numbers of runs. Number of runs decreases quickly with more memory anyway, so the technique is mostly good for larger memory but certainly not for small memory allocations. -- Simon Riggs

Re: [HACKERS] 64-bit XIDs again

2015-07-30 Thread Simon Riggs
investigate 64-bit xids. I don't see this as mere debate, I see this as something that we can make a patch for and scientifically analyze the pros and cons through measurement. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] 64-bit XIDs again

2015-07-30 Thread Simon Riggs
in time. Having a nice, simple change to remove old bugs and new would help us be more robust. But let's measure the overhead before we try to optimize it away. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] Using quicksort and a merge step to significantly improve on tuplesort's single run external sort

2015-07-30 Thread Simon Riggs
multiple runs on tape, then they will need to be merged and you need memory to do that efficiently. If there are tuples in the last batch still in memory then it can work, but it depends upon how full memory is from the last batch and how many batches there are. -- Simon Riggshttp

Re: [HACKERS] pg_basebackup and replication slots

2015-07-29 Thread Simon Riggs
On 29 July 2015 at 09:09, Andres Freund and...@anarazel.de wrote: On 2015-07-29 08:57:38 +0100, Simon Riggs wrote: On 22 July 2015 at 05:43, Michael Paquier michael.paqu...@gmail.com wrote: Now, do we plan to do something about the creation of a slot. I imagine that it would

Re: [HACKERS] Don'st start streaming after creating a slot in pg_receivexlog

2015-07-29 Thread Simon Riggs
with psql? I'd suggest just removing --create-slot and --drop-slot altogether -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Don'st start streaming after creating a slot in pg_receivexlog

2015-07-29 Thread Simon Riggs
On 29 July 2015 at 09:01, Andres Freund and...@anarazel.de wrote: On 2015-07-29 08:54:40 +0100, Simon Riggs wrote: --drop-slot seems pointless, since you can just do that with psql If we make --create-slot do nothing but add the slot, then that seems pointless also Would we need

Re: [HACKERS] pg_basebackup and replication slots

2015-07-29 Thread Simon Riggs
On 29 July 2015 at 11:43, Andres Freund and...@anarazel.de wrote: On 2015-07-29 09:17:04 +0100, Simon Riggs wrote: On 29 July 2015 at 09:09, Andres Freund and...@anarazel.de wrote: The point of using a temporary slot is to not have a leftover slot afterwards, reserving resources

Re: [HACKERS] MultiXact member wraparound protections are now enabled

2015-07-29 Thread Simon Riggs
On 28 July 2015 at 14:20, Robert Haas robertmh...@gmail.com wrote: On Sat, Jul 25, 2015 at 4:11 AM, Simon Riggs si...@2ndquadrant.com wrote: On 22 July 2015 at 21:45, Robert Haas robertmh...@gmail.com wrote: But it seemed to me that this could be rather confusing. I thought it would

Re: [HACKERS] Don'st start streaming after creating a slot in pg_receivexlog

2015-07-29 Thread Simon Riggs
code that needs maintaining. - Remove the --start switch in pg_recvlogical, and let the utility start streaming by default. By doing so both utilities will behave similarly with the same set of options. +1 That way all the utilities have just one thing they do. -- Simon Riggs

Re: [HACKERS] pg_basebackup and replication slots

2015-07-29 Thread Simon Riggs
be needed when creating a standalone backup that would not be persistently connected to the master, yet we want to bring it up as a live/writable server in a single command, and we want to make it easy to script in case our script is killed? -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] Remaining 'needs review' patchs in July commitfest

2015-07-28 Thread Simon Riggs
, or as a new psql backslash option. I think there is something salvageable there. I've added my name as committer to a few things, but won't be able to work on them until at least next week when I've finished 9.5 stuff. Happy to step back if anyone else wants to claim those. -- Simon Riggs

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-27 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] MultiXact member wraparound protections are now enabled

2015-07-27 Thread Simon Riggs
backup (or since initdb), one can rule out some causes. Would it be better to replace it with a less specific and more generally useful message? For example, Server started with release X.y.z from which we could infer various useful things. -- Simon Riggshttp://www

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-25 Thread Simon Riggs
On 22 July 2015 at 14:59, Robert Haas robertmh...@gmail.com wrote: Urgh. So if we do this, that forever precludes having HOT pruning set the all-visible bit. What is the reason why we don't do that already? Surely its a one liner? -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] MultiXact member wraparound protections are now enabled

2015-07-25 Thread Simon Riggs
; that is what the release notes are for. If something is disabled, we should say so, but otherwise silence means safety and success. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Proposal for CSN based snapshots

2015-07-25 Thread Simon Riggs
On 24 July 2015 at 19:21, Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 24, 2015 at 1:00 PM, Simon Riggs si...@2ndquadrant.com wrote: It depends on the exact design we use to get that. Certainly we do not want them if they cause a significant performance regression. Yeah. I think

Re: [HACKERS] BRIN index and aborted transaction

2015-07-24 Thread Simon Riggs
. I'll do that. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Proposal for CSN based snapshots

2015-07-24 Thread Simon Riggs
their patch, we can pick up this work from last community version of this patch. Sounds good. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-23 Thread Simon Riggs
get too excited about a half-fix that leaves you with data corruption in some scenarios. On further consideration, it seems obvious that Andres' suggestion would not work for UPDATE or DELETE, so I now agree. It does seem a big thing to backpatch; alternative suggestions? -- Simon Riggs

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-22 Thread Simon Riggs
On 21 July 2015 at 21:24, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 30, 2015 at 4:32 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, I suggest just a single column on pg_stat_activity called pct_complete trace_completion_interval = 5s (default) Every interval, we report

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-22 Thread Simon Riggs
in other code paths, AFAICS. Patches attached. Martijn, please verify. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services fix_wal_logging_copy_truncate.v1.patch Description: Binary data

Re: [HACKERS] Alpha2/Beta1

2015-07-22 Thread Simon Riggs
On 21 July 2015 at 22:01, Josh Berkus j...@agliodbs.com wrote: My question for Hackers is: should this be Alpha2 or Beta 1? Alpha2 -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-22 Thread Simon Riggs
On 22 July 2015 at 13:00, Robert Haas robertmh...@gmail.com wrote: On Wed, Jul 22, 2015 at 3:02 AM, Simon Riggs si...@2ndquadrant.com wrote: For me, the user workflow looks like these Worried: Task X is taking ages? When is it expected to finish? Ops: 13:50 sometime later, about

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-20 Thread Simon Riggs
On 20 July 2015 at 08:18, Beena Emerson memissemer...@gmail.com wrote: Simon Riggs wrote: synchronous_standby_name= is already 25 characters, so that leaves 115 characters - are they always single byte chars? I am sorry, I did not get why there is a 140 byte limit. Can you please explain

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-19 Thread Simon Riggs
altogether and we should just be storing cluster metadata in database tables, which is what was discussed and agreed at the BDR meeting at PGCon. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-19 Thread Simon Riggs
On 19 July 2015 at 21:56, Tom Lane t...@sss.pgh.pa.us wrote: Since I'm not observing any movement Apologies if nothing visible was occurring. Petr and I had arranged to meet and discuss Mon/Tues. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-19 Thread Simon Riggs
/if there is a benefit? Jeff, can you add detailed comments to explain the theory of operation of these patches? The patches add the code, but don't say why. We will forget, so lets put the detail in there now please. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-17 Thread Simon Riggs
addition). Options already exist on CREATE FOREIGN DATA WRAPPER, so it should be easy to support that. I'd rather add it once on the wrapper than be forced to list all the options on every foreign server, unless required to do so. -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-17 Thread Simon Riggs
server as -postgis, for example. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-16 Thread Simon Riggs
to programmatically manipulate in a range of languages -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
. Inventing a new one doesn't make sense. Most important for me is the ability to programmatically manipulate/edit the config string, which would be harder with a new custom format. ... Group labels are essential. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
the config after a failure is a Failure of Best Practice in an HA system. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
strange to specifically avoid checking before you proceed. Bacon not Aristotle. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
, if we care. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
to manipulate JSON in SQL that can be solved outside of the scope of this patch, because its just JSON. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 12:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs wrote: JSON seems the most sensible format for the string. Inventing a new one doesn't make sense. Most important for me is the ability to programmatically manipulate/edit the config string, which would

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 13:26, Andrew Dunstan and...@dunslane.net wrote: On 07/15/2015 07:58 AM, Simon Riggs wrote: For me the design summary is this * CREATE GLOBAL TEMP TABLE creates catalog entries like a normal table but with different relkind * When we see a request to INSERT, DEL, UPD

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 16:44, Andres Freund and...@anarazel.de wrote: On 2015-07-15 16:36:12 +0100, Simon Riggs wrote: On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: I think that's generally a fair point. But here we're discussing to add a fair amount of wrinkles

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
to overload the main feature request with additional requirements. Doing that is just scope creep that prevents us getting features out. Nice, simple patches from newer developers. Later tuning and tweaking from more expert community members. -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: On 2015-07-15 16:24:52 +0100, Simon Riggs wrote: It may be possible to do this, though I'm sure there's a wrinkle somewhere. But there doesn't seem to be a need to overload the main feature request with additional

Re: [HACKERS] TABLESAMPLE doesn't actually satisfy the SQL spec, does it?

2015-07-14 Thread Simon Riggs
me, I haven't yet understood your proposal about sampling rule above. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-14 Thread Simon Riggs
: there is nothing riding on this from me. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-14 Thread Simon Riggs
On 14 July 2015 at 15:32, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 13 July 2015 at 14:39, Tom Lane t...@sss.pgh.pa.us wrote: TBH, I think the right thing to do at this point is to revert the entire patch and send it back for ground-up rework. I think

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-14 Thread Simon Riggs
this again before CF2. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-14 Thread Simon Riggs
that is; at this time I presume not that hard, but I will wait for Petr's opinion also when he returns on Friday. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-13 Thread Simon Riggs
record freeze map information now it is acceptable to begin after upgrade with all freeze info set to zero. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-13 Thread Simon Riggs
On 11 July 2015 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote: What are we going to do about this? I will address the points you raise, one by one. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-13 Thread Simon Riggs
available and works well. This patch is not required for some evil plan, it simply allows PostgreSQL users to gain the benefit of some cool and useful features. The fact that both of us can cite details of other implementations means we're overdue for this and I'm happy its in 9.5 -- Simon Riggs

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-10 Thread Simon Riggs
state acts as a diagnostic check for us to ensure the bitmap is not itself corrupt. -1 for using it for another purpose. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-08 Thread Simon Riggs
On 7 July 2015 at 18:45, Sawada Masahiko sawada.m...@gmail.com wrote: On Wed, Jul 8, 2015 at 12:37 AM, Andres Freund and...@anarazel.de wrote: On 2015-07-07 16:25:13 +0100, Simon Riggs wrote: I don't think pg_freespacemap is the right place. I agree that pg_freespacemap sounds like

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-07 Thread Simon Riggs
into core, so we can write formal tests. I would not personally commit this feature without rigorous and easily repeatable verification. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-07 Thread Simon Riggs
On 6 July 2015 at 17:28, Simon Riggs si...@2ndquadrant.com wrote: I think we need something for pg_upgrade to rewrite existing VMs. Otherwise a large read only database would suddenly require a massive revacuum after upgrade, which seems bad. That can wait for now until we all agree this patch

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-06 Thread Simon Riggs
On 3 July 2015 at 09:25, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, Jul 3, 2015 at 1:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On 2 July 2015 at 16:30, Sawada Masahiko sawada.m...@gmail.com wrote: Also, the flags of each heap page header might be set PD_ALL_FROZEN

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-07-03 Thread Simon Riggs
peace with yourself that probably 50% of development time is wasted. But we try to keep the best half. Thank you for your time spent contributing. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] Memory Accounting v11

2015-07-02 Thread Simon Riggs
? Not tracking memory could be worse than tracking it. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-07-02 Thread Simon Riggs
, but that is not a reason not to commit this, as is. Commit, please. This has been in place for a while and still remains: TODO: reduce impact of full page writes -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 09:44, Beena Emerson memissemer...@gmail.com wrote: I am not sure how combining both will work out. Use cases needed. Catalog Method: Is it safe to assume we may not going ahead with the Catalog approach? Yes -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:38, Andres Freund and...@anarazel.de wrote: On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: This was pushed back from last CF and I haven't worked on it at all, nor will I. Pushing back again. Let's return with feedback, not move, it then.. Moving a entries

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:31, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-02 Thread Simon Riggs
doubt that it can be managed efficiently while retaining optimal memory management. If it can its going to be very low priority (or should be). This approach works, so lets do it, now. If someone comes up with a better way later, great. -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] raw output from copy

2015-07-02 Thread Simon Riggs
from Tom, so there is no history of preference there. Can we have both please, gentlemen? If we implemented Andrew's solution, how would we request it in a COPY statement? Seems like we would want the RAW format keyword anyway. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www

Re: [HACKERS] raw output from copy

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 15:07, Pavel Stehule pavel.steh...@gmail.com wrote: It can be used from psql without any problems. It can, but your patch does not yet do that, while Andrew's does. We want a solution that works from psql and other clients. Hopefully the same-ish solution. -- Simon Riggs

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-02 Thread Simon Riggs
other. Are they? Or not? Do we test for an impossible state? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Doubt about AccessExclusiveLock in ALTER TABLE .. SET ( .. );

2015-07-02 Thread Simon Riggs
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Looks functionally complete Need a test to show that ALTER TABLE works on

Re: [HACKERS] WAL-related tools and .paritial WAL file

2015-07-01 Thread Simon Riggs
all only a debug tool. This would also make fuzzy_open_file more complicated by having to detect two times more potential grammars... That's a bit crazy for a very narrow use-case. +1 to all -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 11:14, Andres Freund and...@anarazel.de wrote: On 2015-07-01 09:08:11 +0100, Simon Riggs wrote: On 1 July 2015 at 09:00, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jun 30, 2015 at 12:32 PM, Simon Riggs si...@2ndquadrant.com a. the semantics of new LWLock

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 11:11, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Jul 1, 2015 at 1:38 PM, Simon Riggs si...@2ndquadrant.com wrote: On 1 July 2015 at 09:00, Amit Kapila amit.kapil...@gmail.com wrote: I think it will be better to partition it or use it in some other way

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-01 Thread Simon Riggs
? I think we would need some way to trigger a scan when the amount of deferred dropped data files hits a certain size. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

<    4   5   6   7   8   9   10   11   12   13   >