Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-16 Thread Simon Riggs
On 15 November 2015 at 14:50, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Nov 15, 2015 at 5:41 AM, Simon Riggs <si...@2ndquadrant.com> > wrote: > > Hmm, if that's where we're at, I'll summarize my thoughts. > > > > All of this discussion presuppose

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-16 Thread Simon Riggs
On 16 November 2015 at 11:01, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 16 November 2015 at 18:44, Simon Riggs <si...@2ndquadrant.com> wrote: > >> >> The pooler knows which statements are reads and writes >> > > I think that's an iffy

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-15 Thread Simon Riggs
further things like "GUC causal_reads_standby_names" should be implemented by Node Registry as a named group of nodes. We can have as many arbitrary groups of nodes as we want. If that sounds strange look back at exactly why GUCs are called GUCs. -- Simon Riggshttp://www.2nd

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-12 Thread Simon Riggs
o look at whether this does actually give us anything, or whether we are missing the underlying Heisenberg reality. More later. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-11 Thread Simon Riggs
it should be readers that wait. Let's have that debate up front before we start reviewing the patch. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] can we add SKIP LOCKED to UPDATE?

2015-11-09 Thread Simon Riggs
ave unpredictable results from the update then. > True, but given the already restricted use case of SKIP LOCKED, the request makes sense for the following UPDATE ... SKIP LOCKED RETURNING xxx would be better than BEGIN SELECT xxx FOR UPDATE SKIP LOCKED UPDATE COMMIT -- Simo

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-09 Thread Simon Riggs
needing to > solve the same problems. The whole purpose of that XTM API is to allow different solutions for that to be created. Konstantin has made a very good case for such an API to exist, based around 3 markedly different approaches. Whether we allow the API into core to be accessible via exten

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-09 Thread Simon Riggs
. In this case, the role of the GTM is to record the commit. Other nodes consult the GTM, not local state to see whether the transaction has committed, acting as the transaction manager in an XA sense. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Erroneous cost estimation for nested loop join

2015-11-09 Thread Simon Riggs
rkload is random access. > We don't have a clustered index in Postgres. We do store correlation stats for the index, which is used in some places to reduce cost. Could you look some more at this and see if a model change that uses the correlation can improve this? -- Simon Riggs

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-08 Thread Simon Riggs
ion. > Assume bank debit-credit example. If some distributed transaction > transfers money from the account at one node to the account and another > node, > then committing transaction just at one node cause incorrect total balance. > The main goal of DTM is to preserve ACID semantic f

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-08 Thread Simon Riggs
was used within Oracle Parallel server for a buffer lock manager, which is not what is being discussed). -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
- Index Scan using f_x_y_idx on f (cost=0.42..26075.71 rows=209 width=37) Index Cond: (x = 5) Filter: (y ~~ '%abc%'::text) So it should be possible to do the Filter condition on the BitmapIndexScan. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
On 4 November 2015 at 16:14, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > On 4 November 2015 at 15:54, Tom Lane <t...@sss.pgh.pa.us> wrote: > > We generate this plan > > Index Scan using f_x_y_idx on f

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
On 4 November 2015 at 16:59, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > On 4 November 2015 at 16:14, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> You're missing my point: that is possible in an indexscan, but *not* i

Re: [HACKERS] NOTIFY in Background Worker

2015-11-03 Thread Simon Riggs
and notify commands inside a background worker? > > Well, parallel workers are something different from general background > workers. I don't see why it'd make sense to allow listen/notify there, > given the rest of the restrictions? > What are the restrictions and/or where ar

Re: [HACKERS] Patent warning about the Greenplum source code

2015-11-03 Thread Simon Riggs
there is nothing Greenplum-specific about any of > this. Good. Now we have addressed the issue of balance, the fundamental issue raised in your original post is still important and does need to be addressed, against any and all companies/patents. Your vigilance on patent issues is useful. Thank you

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-11-03 Thread Simon Riggs
On 3 November 2015 at 15:23, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Oct 23, 2015 at 6:29 AM, Simon Riggs <si...@2ndquadrant.com> > wrote: > >> On 21 October 2015 at 13:31, Jeff Janes <jeff.ja...@gmail.com> wrote: >> >> Index-only s

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-03 Thread Simon Riggs
t depends where we want to put the patch; discussion here means in-core, so for it to go in here, we do need subxacts etc. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Patent warning about the Greenplum source code

2015-11-02 Thread Simon Riggs
n informed observer only. Please say no more until you have a full set of information; I suggest you discuss that privately with each person/company first, to give them time to explain. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-02 Thread Simon Riggs
t present, I can't tell whether you need subtrans and multixact calls in the API as well. I would imagine we probably do, though we might imagine an implementation that didn't support those concepts. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] onlyvalue aggregate (was: First Aggregate Funtion?)

2015-11-02 Thread Simon Riggs
in the names since those already have other meanings in Postgres. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] pg_dump LOCK TABLE ONLY question

2015-10-31 Thread Simon Riggs
mp of "tab1". No data is included from tables that inherit tab1 because COPY refers only to the target table. Why should that action cause a lock to be taken on another table that inherits from tab1? It seems clear that the user is requesting an action ONLY on tab1, so we

Re: [HACKERS] Personal note: changing employers

2015-10-29 Thread Simon Riggs
o spending more-or-less full time on community work. > Excellent news. Glad to have you back; we all missed you. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Proposal: Trigonometric functions in degrees

2015-10-26 Thread Simon Riggs
laughing matter. How would we even know how to capitalise "Fortnight"? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Proposal: Trigonometric functions in degrees

2015-10-26 Thread Simon Riggs
t, thinking he meant "radians". Definitely -1 to gradians in PostgreSQL. Also -1 to furlongs, fortnights, pecks and hundredweight, amongst others. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] make Gather node projection-capable

2015-10-25 Thread Simon Riggs
little reason to force a Result node atop a Gather node when we don't > do that for most other node types. > Presumably this is a performance issue then? If we are calculating something *after* a join which increases rows then the query will be slower than need be. I agree the rule shou

Re: [HACKERS] Proposal: Trigonometric functions in degrees

2015-10-24 Thread Simon Riggs
; Possibly the earthdistance module would benefit from these functions too. > > Thoughts? > +1, yes, please. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-10-22 Thread Simon Riggs
le. > > I think we really want to create _vm file as well as set PD_ALL_VISIBLE, > but I don't know the best way to do that. Set a flag somewhere and then > create it in bulk at the end of the transaction? Set it bit by bit as the > pages are extended and initialized? > Easy enough to d

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-10-22 Thread Simon Riggs
Either way, my bug, my bad, thanks for the report. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Multi-column distinctness.

2015-10-20 Thread Simon Riggs
Koyotaro's changes to force COLUMN to be required won't get through. ISTM that we could use COLLECT STATISTICS instead of ADD STATISTICS, and use REMOVE STATISTICS instead of DROP STATISTICS. That way we can use ALTER TABLE rather than inventing a new command. 5 minute change... Unless there is some

Re: [HACKERS] Multi-column distinctness.

2015-10-20 Thread Simon Riggs
T STATISTICS. > > Why is this an improvement over using already-existing keywords? > The earlier patch changed the grammar for the DROP (column) subcommand, which I am saying is not acceptable. So by using an alternate keyword we are able to keep the existing syntax untouched. I

Re: [HACKERS] Multi-column distinctness.

2015-10-20 Thread Simon Riggs
f statistic, rather than specific names for the statistic entry? Since this command doesn't actually ADD the statistics, it just creates a specification used by the next ANALYZE, it would seem better to use a different keyword than ADD, perhaps DECLARE STATISTICS ON... and DROP STATISTICS ON -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-20 Thread Simon Riggs
ust noting the need for change there. I'll help as well, but if you could start with enough basics to allow me to ask questions that will help. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-20 Thread Simon Riggs
ly raise one? Bruce would have me believe that we added months onto 9.5 to improve robustness. So lets actually do that. Starting at the first opportunity. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Multi-column distinctness.

2015-10-20 Thread Simon Riggs
nd anything based on ALTER TABLE just cannot do > that reasonably. > +1 Good argument, so now I understand and agree with Tomas' original suggestion So consider > > CREATE STATISTICS name ON table(columnlist) [, table(columnlist) ...] > [ WITH options ] > Seems good -- S

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Simon Riggs
f comments high if we are to encourage wider participation in the project. So reviewing patch 13 isn't possible without prior knowledge. Hoping we'll be able to find some time on this at PGConf.eu; thanks for coming over. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/

Re: [HACKERS] Allow ssl_renegotiation_limit in PG 9.5

2015-10-17 Thread Simon Riggs
h for the extensible, backpatchable, non-invasive proposal attached. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services startup_option_driver.v1.patch Description: Binary data -

Re: [HACKERS] remaining open items

2015-10-17 Thread Simon Riggs
do we need to fix it now? Surely this will break things in later releases, but not in 9.5, since we aren't ever going to add DDL to 9.5 again. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Allow ssl_renegotiation_limit in PG 9.5

2015-10-17 Thread Simon Riggs
On 17 October 2015 at 13:27, Andres Freund <and...@anarazel.de> wrote: > On 2015-10-17 12:49:17 +0100, Simon Riggs wrote: > > Agreed, but I don't like the idea of hardcoding something so horribly > > specific into the server. > > What's that specific about accept

Re: [HACKERS] Allow ssl_renegotiation_limit in PG 9.5

2015-10-17 Thread Simon Riggs
ed to change the protocol. We can backpatch a simple fix to all supported versions. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel Aggregate

2015-10-13 Thread Simon Riggs
gate in parallel, that's exactly what you don't > want. > > Also, I think the path for parallel aggregation should probably be > something like FinalizeAgg -> Gather -> PartialAgg -> some partial > path here. I'm not clear whether that is what you are thinking or > not. > Yes, but not sure of names. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-08 Thread Simon Riggs
don't care if we change the name, but I do care if we can't tell the difference between a failed upgrade, a normal upgrade and a server that has been upgraded multiple times. Alternate suggestions welcome. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-05 Thread Simon Riggs
On 10 September 2015 at 01:58, Andres Freund <and...@anarazel.de> wrote: > On 2015-09-04 23:35:42 +0100, Simon Riggs wrote: > > This looks OK. You saw that I was proposing to solve this problem a > > different way ("Summary of plans to avoid the annoyance of Freezing

Re: [HACKERS] Re: BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-30 Thread Simon Riggs
a patch on a separate thread. Thanks. Let's close this thread, so we have just one. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
since it has no value for immediate action or post-facto diagnosis, sorry to say - making it happen for every backend just makes it even less useful. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 20:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs <si...@2ndquadrant.com> > wrote: > >> On 29 September 2015 at 12:52, Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-29 Thread Simon Riggs
general for sorts, leading to an optimization of merge joins using BRIN indexes. All this was foreseen in the original design; if it really was trivial it would be in 9.5 already... -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
be another backend, or it allows the query process to perform self-observation based upon a timeout (e.g. >1 hour) or a row limit (e.g. when an optimizer estimate is seen to be badly wrong). -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQ

Re: [HACKERS] BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-28 Thread Simon Riggs
e optimizing code (warm run) > postgres=# select count(*) from lineitem where l_orderkey=1; > count > --- > 6 > (1 row) > > Time: 456.219 ms > > After optimizing code (warm run) > postgres=# select count(*) from lineitem where l_orderkey=1; > count > --- >

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-25 Thread Simon Riggs
it has for CF app. I have frequently been the agent of change in matters of process, but I see no useful change here, just lots of wasted time. But then why are we even talking about change? What thing is broken that needs to be fixed? Why is adopting a new package a fix for that? -- Simon Rigg

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-25 Thread Simon Riggs
> > I would argue that there is some benefit for us in terms of advocacy. There are also some negatives in terms of advocacy. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-25 Thread Simon Riggs
On 25 September 2015 at 11:32, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > I have frequently been the agent of change in matters of process, but I > see > > no useful change here, just lots of wasted time. But then why

Re: [HACKERS] a funnel by any other name

2015-09-23 Thread Simon Riggs
On 22 September 2015 at 20:34, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Sep 22, 2015 at 10:34 AM, Simon Riggs <si...@2ndquadrant.com> > wrote: > > Robert, thanks for asking. We'll be stuck with these words for some time, > > user visible via EXPLAIN so

Re: [HACKERS] a funnel by any other name

2015-09-23 Thread Simon Riggs
On 22 September 2015 at 21:14, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Robert Haas wrote: > > On Tue, Sep 22, 2015 at 10:34 AM, Simon Riggs <si...@2ndquadrant.com> > wrote: > > > > For 1, Gather makes most sense. > > > > Yeah, I'm lea

Re: [HACKERS] a funnel by any other name

2015-09-22 Thread Simon Riggs
nodes is "Distribution" (or DISTRIBUTED BY) For 3 we should definitely use Redistribute, since this is what Teradata has been calling it for 30 years, which is where Greenplum got it from. For 1, Gather makes most sense. For 2, it could be either Scatter or Distribute. The f

Re: [HACKERS] numbering plan nodes

2015-09-17 Thread Simon Riggs
that scheme. It also assumes that all actors have access to a single memory structure that describes everything. Hopefully we are working on a parallel query system that will work intranode as well as across nodes, so access to memory should not be assumed. -- Simon Riggs

Re: [HACKERS] synchronous_commit = apply

2015-09-16 Thread Simon Riggs
andby? Perhaps those sorts of practical problems are the reason no one > has done or wanted this. > It means we need quorum sync rep as well, to make this useful in practice without sacrificing HA. Bringing my patch and Beena's patch together will solve this for us in 9.6 So yes, 1) we have

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-09-16 Thread Simon Riggs
ully things can be made faster with less significant digits. I figure this is important enough to trigger a maint release, but since we already agreed when the next one is, I don't see we need to do it any quicker, do we? Well done Dean for excellent research. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Spurious standby query cancellations

2015-09-16 Thread Simon Riggs
M has checked and it knows those blocks do not contain tuples visible to the scan. Which sounds a little hairy. I think Andres is working on putting an end of data watermark in shared memory rather than using the end of physical file, which sounds like a better plan. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] hot_standby_feedback default and docs

2015-09-16 Thread Simon Riggs
_standby So the docs look correct. IMHO the default is the best one at the current time. See recovery_min_apply_delay. I've got some changes that I'd like to make once Kevin's snapshot too old patch has gone thru that might change that. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Reducing the size of BufferTag & remodeling forks

2015-09-13 Thread Simon Riggs
On 12 September 2015 at 21:19, Andres Freund <and...@anarazel.de> wrote: > On 2015-09-12 13:12:26 +0100, Simon Riggs wrote: > > Why do we have to do buffer lookups using the full buffer tag? > > We don't necessarily. > > > Why not just use (relNode, blockNum) and r

Re: [HACKERS] Reducing the size of BufferTag & remodeling forks

2015-09-12 Thread Simon Riggs
Why do we have to do buffer lookups using the full buffer tag? Why not just use (relNode, blockNum) and resolve hash collisions, if any? Your suggestion to avoid hashing the whole buffer tag was a good one. Having a permanent table to produce a smaller tag is a fairly pessimistic solution; why not

Re: [HACKERS] Partitioned checkpointing

2015-09-11 Thread Simon Riggs
to one of N batches so we can make N roughly equal sized mini-checkpoints 3. Make N passes through shared buffers, writing out files assigned to each batch as we go -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Multi-column distinctness.

2015-09-06 Thread Simon Riggs
is is our most common cause of optimizer failures. > Glad to see you agree. Why have you said this on *this* patch, when Tomas' patch has been very obviously addressing this problem since January, with very little traction. Please add your backing to Tomas' patch. -- Simon Riggs

Re: [HACKERS] Multi-column distinctness.

2015-09-06 Thread Simon Riggs
et to avoid confusion and to avoid encouraging people to take this approach. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

[HACKERS] Separating Buffer LWlocks

2015-09-06 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services separate_buffer_lwlocks.v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-05 Thread Simon Riggs
.49 1.01 0.89 > What are these? Times? in ms? > However that was the idea of the benchmark, and I got no difference. > Please explain what this means and your conclusion, so its clear. That way we can either reject the patch or commit it. Thanks -- Simon Riggs

Re: [HACKERS] Declarative partitioning

2015-09-04 Thread Simon Riggs
ave any catalog entries > other than the pg_class tuple? Everything should start from the requirements of the optimization approach. Once we have that clear, we can confirm other requirements. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Testing WAL replay by comparing before and after images again

2015-09-04 Thread Simon Riggs
assumption that we are replaying all changes in single threaded mode is not appropriate for use with logical replication. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Waits monitoring

2015-09-04 Thread Simon Riggs
h according to it. > I see the need for both current wait information and for cumulative historical detail. I'm willing to wait before reviewing this, but not for more than 1 more CF. Andres, please decide whether we should punt to next CF now, based upon other developments. Thanks -- Simon R

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-09-04 Thread Simon Riggs
On 10 August 2015 at 19:21, Josh Berkus <j...@agliodbs.com> wrote: > Question: does regular vacuum update the visibility map in the same way > vacuum freeze does? Yes -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Develop

Re: [HACKERS] Spurious standby query cancellations

2015-09-04 Thread Simon Riggs
that to 9.3 if needed, where they were introduced. There's an example of how to use lock waits further down on ResolveRecoveryConflictWithBufferPin(). Could you have a look at doing it that way? We probably also need to resurrect my earlier patch to avoid logging AccessExclusiveLocks on temp tables. -- Simon

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-04 Thread Simon Riggs
avoid the annoyance of Freezing"), suggesting that we wait for a few CFs to see if a patch emerges for that - then fall back to this patch if it doesn't? So I am moving this patch to next CF. I apologise for the personal annoyance caused by this; I hope whatever solution we find we can work togethe

Re: [HACKERS] Horizontal scalability/sharding

2015-08-30 Thread Simon Riggs
is a huge flashing red warning light for me at present. If that requires a meeting of all Developers, why are the meetings for this specifically not happening at the agreed Developer meetings? -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development

Re: [HACKERS] 9.5 feature count

2015-08-28 Thread Simon Riggs
in default settings} has been distilled too far to draw any meaningful conclusions. They simply reflect the editing style in use at that time, which has changed over time. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-26 Thread Simon Riggs
attached. Doesn't seem worth a separate thread. It's not clear to me if anyone wanted to do further review/testing of this patch, or if I should go ahead and push it (after fixing whatever comments need to be fixed). Push, please. -- Simon Riggshttp://www.2ndQuadrant.com/ http

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-26 Thread Simon Riggs
On 26 August 2015 at 11:40, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Aug 25, 2015 at 2:21 PM, Simon Riggs si...@2ndquadrant.com wrote: On 22 August 2015 at 15:14, Andres Freund and...@anarazel.de wrote: TransactionIdSetPageStatus() calls TransactionIdSetStatusBit(), which

Re: [HACKERS] Declarative partitioning

2015-08-26 Thread Simon Riggs
Support routines are usually pretty straightforward too, so that can be left for now. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Declarative partitioning

2015-08-26 Thread Simon Riggs
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Needs planner work and tests of that. ALTER TABLE etc can wait. The new

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-25 Thread Simon Riggs
On 12 August 2015 at 04:49, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Aug 11, 2015 at 7:27 PM, Simon Riggs si...@2ndquadrant.com wrote: On 11 August 2015 at 14:53, Amit Kapila amit.kapil...@gmail.com wrote: One more point here why do we need CommitLock before calling

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-25 Thread Simon Riggs
On 22 August 2015 at 15:14, Andres Freund and...@anarazel.de wrote: On 2015-06-30 08:02:25 +0100, Simon Riggs wrote: Proposal for improving this is to acquire the ClogControlLock in Shared mode, if possible. I find that rather scary. That requires for all read and write paths in clog.c

Re: [HACKERS] Planned release for PostgreSQL 9.5

2015-08-25 Thread Simon Riggs
then include any later changes into 2.2.1, if any, or simply just announce 2.2.0 as supporting 9.5 if nothing changed. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Declarative partitioning

2015-08-24 Thread Simon Riggs
; when a sequence of partitions has been defined -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Using quicksort for every external sort run

2015-08-21 Thread Simon Riggs
On 20 August 2015 at 18:41, Peter Geoghegan p...@heroku.com wrote: On Thu, Aug 20, 2015 at 8:15 AM, Simon Riggs si...@2ndquadrant.com wrote: On 20 August 2015 at 03:24, Peter Geoghegan p...@heroku.com wrote: The patch is ~3.25x faster than master I've tried to read this post twice

Re: [HACKERS] Declarative partitioning

2015-08-20 Thread Simon Riggs
On 20 August 2015 at 10:10, Amit Langote langote_amit...@lab.ntt.co.jp wrote: On 2015-08-20 AM 05:10, Josh Berkus wrote: On 08/19/2015 04:59 AM, Simon Riggs wrote: I like the idea of a regular partitioning step because it is how you design such tables - lets use monthly partitions

Re: [HACKERS] Extension upgrade and GUCs

2015-08-20 Thread Simon Riggs
DefineCustomStringVariable() allows, something that can overwrite things instead of just erroring out. Are you trying to preserve the in-memory state across upgrade as well? It sounds unlikely we can support that directly in the general case. Sounds like we need RedefineCustomStringVariable() -- Simon Riggs

Re: [HACKERS] Declarative partitioning

2015-08-20 Thread Simon Riggs
the way. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Extension upgrade and GUCs

2015-08-20 Thread Simon Riggs
On 20 August 2015 at 13:21, Paul Ramsey pram...@cleverelephant.ca wrote: On August 20, 2015 at 2:17:31 AM, Simon Riggs (si...@2ndquadrant.com (mailto:si...@2ndquadrant.com)) wrote: On 18 August 2015 at 21:03, Paul Ramsey wrote: So I need a way to either (a) notice when I already have

Re: [HACKERS] Using quicksort for every external sort run

2015-08-20 Thread Simon Riggs
On 20 August 2015 at 03:24, Peter Geoghegan p...@heroku.com wrote: The patch is ~3.25x faster than master I've tried to read this post twice and both times my work_mem overflowed. ;-) Can you summarize what this patch does? I understand clearly what it doesn't do... -- Simon Riggs

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-19 Thread Simon Riggs
On 19 August 2015 at 12:55, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2015-08-19 20:12 GMT+09:00 Simon Riggs si...@2ndquadrant.com: On 12 June 2015 at 00:29, Tomas Vondra tomas.von...@2ndquadrant.com wrote: I see two ways to fix this: (1) enforce the 1GB limit (probably better

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-19 Thread Simon Riggs
. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] allowing wal_level change at run time

2015-08-19 Thread Simon Riggs
, and everything. Therefore I propose something slightly different: We change the default of max_wal_senders, max_replication_slots, to some reasonably high setting Agreed, I suggest 8 as the default for each. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com

Re: [HACKERS] Declarative partitioning

2015-08-19 Thread Simon Riggs
unique indexes will be allowed on partitions. So if the partition key is unique, we can say the whole partitioned table is unique and call that a Primary Key. I would want individual partitions to be placed on separate tablespaces, but not by default. -- Simon Riggshttp://www

Re: [HACKERS] Declarative partitioning

2015-08-19 Thread Simon Riggs
the feature, so its all about the details now. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-19 Thread Simon Riggs
and it will stop there. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-19 Thread Simon Riggs
, but its cheap and a straightforward patch. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-19 Thread Simon Riggs
On 19 August 2015 at 14:53, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 19 August 2015 at 12:55, Kohei KaiGai kai...@kaigai.gr.jp wrote: Please don't be rush. :-) Please explain what rush you see? Yours. You appear to be in a hurry to apply patches

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-19 Thread Simon Riggs
believing the answer from the snapshot? Under what circumstances could TransactionIdIsInProgress() return true where XidInMVCCSnapshot() had not? I'm thinking maybe TransactionIdIsInProgress is only needed for non-MVCC snapshot types. +1 -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] Management of simple_eval_estate for plpgsql DO blocks

2015-08-16 Thread Simon Riggs
solution. Not everyone uses large numbers of DO blocks, but casts apply everywhere, right? If the choice is efficiency or memory bloat, can't we choose a point where we switch from one to the other? i.e. use private structures after N uses of the shared structures. -- Simon Riggs

Re: [HACKERS] Test code is worth the space

2015-08-14 Thread Simon Riggs
in separate files accessed only by a different schedule. make check == fast-parallel_schedule make check-full == parallel_schedule Probably easier to make one schedule call the other, so we don't duplicate anything. Tom gets his fast schedule, others get their full schedule. -- Simon Riggs

<    3   4   5   6   7   8   9   10   11   12   >