Re: [HACKERS] Hm, table constraints aren't so unique as all that

2013-01-30 Thread Jim Nasby
On 1/28/13 6:25 PM, Tom Lane wrote: I think we need to tighten this down by having index-constraint creation check for conflicts with other constraint types. It also seems like it might be a good idea to put in a unique index to enforce the intended lack of conflicts --- note that the existing i

Re: [HACKERS] autovacuum not prioritising for-wraparound tables

2013-01-30 Thread Jim Nasby
On 1/25/13 11:56 AM, Christopher Browne wrote: With a little bit of noodling around, here's a thought for a joint function that I*think* has reasonably common scales: f(deadtuples, relpages, age) = deadtuples/relpages + e ^ (age*ln(relpages)/2^32) Be careful with dead/relpages, because de

Re: [HACKERS] Hm, table constraints aren't so unique as all that

2013-01-30 Thread Jim Nasby
On 1/29/13 6:40 PM, Tom Lane wrote: I wrote: >Over in the thread about enhanced error fields, I claimed that >"constraints are uniquely named among those associated with a table, >or with a domain". But it turns out that that ain't necessarily so, >because the code path for index constraints do

Re: [HACKERS] autovacuum not prioritising for-wraparound tables

2013-01-30 Thread Jim Nasby
On 1/30/13 3:28 PM, Kevin Grittner wrote: Jim Nasby wrote: then the *ideal* time to start a freeze vacuum on a table is so that the vacuum would end *exactly* as we were about to hit XID wrap. For a tuple which you know is going to survive long enough to be frozen, the *ideal* time to

Re: [HACKERS] autovacuum not prioritising for-wraparound tables

2013-01-30 Thread Jim Nasby
On 1/30/13 4:37 PM, Christopher Browne wrote: From a more practical standpoint, I think it would be extremely useful to >have a metric that showed how quickly a table churned. Something like dead >tuples per time period. Comparing that to the non-bloated table size should >give a very strong ind

Re: [HACKERS] auto_explain WAS: RFC: Timing Events

2013-02-24 Thread Jim Nasby
Sorry for the late reply, but I think I can add some ideas here... On 11/21/12 5:33 PM, Gavin Flower wrote: On 22/11/12 12:15, Greg Smith wrote: On 11/8/12 2:16 PM, Josh Berkus wrote: Also, logging only the long-running queries is less useful than people on this list seem to think. When I'm

Re: [HACKERS] auto_explain WAS: RFC: Timing Events

2013-02-26 Thread Jim Nasby
On 2/26/13 11:19 AM, Robert Haas wrote: On Mon, Feb 25, 2013 at 10:22 PM, Greg Stark wrote: On Mon, Feb 25, 2013 at 8:26 PM, Robert Haas wrote: On Sun, Feb 24, 2013 at 7:27 PM, Jim Nasby wrote: We actually do that in our application and have discovered that random sampling can end up

Re: [HACKERS] autovacuum not prioritising for-wraparound tables

2013-02-28 Thread Jim Nasby
On 1/31/13 2:18 PM, Alvaro Herrera wrote: My intention was to apply a Nasby correction to Browne Strength and call the resulting function Browne' (Browne prime). Does that sound better? I suggest painting that bikeshed "Browneby". :P -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Jim Nasby
On 3/4/13 10:00 AM, Jeff Davis wrote: On Mon, 2013-03-04 at 10:36 +0200, Heikki Linnakangas wrote: >On 04.03.2013 09:11, Simon Riggs wrote: > >Are there objectors? > >FWIW, I still think that checksumming belongs in the filesystem, not >PostgreSQL. Doing checksums in the filesystem has some d

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Jim Nasby
On 3/4/13 2:48 PM, Jeff Davis wrote: On Mon, 2013-03-04 at 13:58 -0500, Greg Smith wrote: >On 3/4/13 2:11 AM, Simon Riggs wrote: > >It's crunch time. Do you and Jeff believe this patch should be > >committed to Postgres core? > >I want to see a GUC to allow turning this off, to avoid the probl

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Jim Nasby
On 3/4/13 3:00 PM, Heikki Linnakangas wrote: On 04.03.2013 22:51, Jim Nasby wrote: The time to object to the concept of a checksuming feature was a long time ago, before a ton of development effort went into this... :( I did. Development went ahead anyway. Right, because the community felt

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Jim Nasby
On 3/4/13 5:20 PM, Craig Ringer wrote: On 03/05/2013 04:48 AM, Jeff Davis wrote: We would still calculate the checksum and print the warning; and then pass it through the rest of the header checks. If the header checks pass, then it proceeds. If the header checks fail, and if zero_damaged_pages

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Jim Nasby
On 3/4/13 6:22 PM, Craig Ringer wrote: On 03/05/2013 08:15 AM, Jim Nasby wrote: Would it be better to do checksum_logging_level = ? That way someone could set the notification to anything from DEBUG up to PANIC. ISTM the default should be ERROR. That seems nice at first brush, but I don&#

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Jim Nasby
On 3/4/13 7:04 PM, Daniel Farina wrote: Corruption has easily occupied more than one person-month of time last year for us. Just FYI for anyone that's experienced corruption... we've looked into doing row-level checksums at work. The only challenge we ran into was how to check them when readi

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Jim Nasby
On 3/6/13 1:14 PM, Josh Berkus wrote: There may be good reasons to reject this patch. Or there may not. But I completely disagree with the idea that asking them to solve the problem at the filesystem level is sensible. Yes, can we get back to the main issues with the patch? 1) argument over

Re: [HACKERS] Using indexes for partial index builds

2013-03-06 Thread Jim Nasby
On 2/2/13 4:05 AM, Paul Norman wrote: Hello, After a discussion on IRC in #postgresql, I had a feature suggestion and it was suggested I write it up here. I have a large (200GB, 1.7b rows) table with a number of columns, but the two of interest here are a hstore column, tags and a postgis geomet

Re: [HACKERS] proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-03-06 Thread Jim Nasby
On 2/2/13 3:23 AM, Pavel Stehule wrote: Hello I propose enhancing GET DIAGNOSTICS statement about new field PG_CONTEXT. It is similar to GET STACKED DIAGNOSTICS' PG_EXCEPTION_CONTEXT. Motivation for this proposal is possibility to get call stack for debugging without raising exception. This c

Re: [HACKERS] Enabling Checksums

2013-03-13 Thread Jim Nasby
On 3/7/13 9:31 PM, Bruce Momjian wrote: 1 storage 2 storage controller 3 file system 4 RAM 5 CPU I would add 2.5 in there: storage interconnect. iSCSI, FC, what-have-you. Obviously not everyone has that. My guess is that storage checksums only cover la

Re: [HACKERS] Using indexes for partial index builds

2013-03-13 Thread Jim Nasby
On 3/12/13 9:10 AM, Ants Aasma wrote: I have a feeling this is an increasingly widespread pattern with a proliferation of mobile devices that need syncing. If you're doing that with timestamps you're asking for a slew of problems, not all of which can be solved by just adding some random amoun

Re: [HACKERS] Temporal features in PostgreSQL

2013-03-13 Thread Jim Nasby
On 2/13/13 10:06 PM, Vlad Arkhipov wrote: - I don't need to deal with update conflicts, because I use clock_timestamp() instead of current_timestamp. You can still come across a conflict even with clock_timestamp(). What if clocks go back during the time synchronization? Even if you have

Re: [HACKERS] Enabling Checksums

2013-03-22 Thread Jim Nasby
On 3/18/13 2:25 PM, Simon Riggs wrote: On 18 March 2013 19:02, Jeff Davis wrote: On Sun, 2013-03-17 at 22:26 -0700, Daniel Farina wrote: as long as I am able to turn them off easily To be clear: you don't get the performance back by doing "ignore_checksum_failure = on". You only get around t

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-03-22 Thread Jim Nasby
On 3/22/13 7:27 PM, Ants Aasma wrote: On Fri, Mar 22, 2013 at 10:22 PM, Merlin Moncure wrote: well if you do a non-locking test first you could at least avoid some cases (and, if you get the answer wrong, so what?) by jumping to the next buffer immediately. if the non locking test comes good,

Re: [HACKERS] Let's invent a function to report lock-wait-blocking PIDs

2013-03-22 Thread Jim Nasby
On 3/20/13 10:36 PM, Tom Lane wrote: Simon Riggs writes: On 20 March 2013 18:02, Tom Lane wrote: The API that comes to mind is (name subject to bikeshedding) pg_blocking_pids(pid int) returns int[] Useful. Can we also have an SRF rather than an array? I thought about that, but at least

Re: [HACKERS] Enabling Checksums

2013-03-22 Thread Jim Nasby
I realize Simone relented on this, but FWIW... On 3/16/13 4:02 PM, Simon Riggs wrote: Most other data we store doesn't consist of large runs of 0x00 or 0xFF as data. Most data is more complex than that, so any runs of 0s or 1s written to the block will be detected. ... It's not that uncommon f

Re: [HACKERS] Enabling Checksums

2013-03-22 Thread Jim Nasby
On 3/20/13 8:41 AM, Bruce Momjian wrote: On Mon, Mar 18, 2013 at 01:52:58PM -0400, Bruce Momjian wrote: I assume a user would wait until they suspected corruption to turn it on, and because it is only initdb-enabled, they would have to dump/reload their cluster. The open question is whether thi

Re: [HACKERS] Enabling Checksums

2013-03-29 Thread Jim Nasby
On 3/25/13 8:25 AM, Bruce Momjian wrote: On Fri, Mar 22, 2013 at 11:35:35PM -0500, Jim Nasby wrote: >On 3/20/13 8:41 AM, Bruce Momjian wrote: > >Also, if a users uses checksums in 9.3, could they initdb without > >checksums in 9.4 and use pg_upgrade? As coded, the pg_controld

Re: [PATCH] Exorcise "zero-dimensional" arrays (Was: Re: [HACKERS] Should array_length() Return NULL)

2013-04-01 Thread Jim Nasby
On 4/1/13 8:58 AM, Merlin Moncure wrote: On Mon, Apr 1, 2013 at 6:43 AM, Robert Haas wrote: On Tue, Mar 26, 2013 at 4:44 PM, Tom Lane wrote: Robert Haas writes: Well, you could easily change array_ndims() to error out if ARR_NDIM() is negative or more than MAXDIM and return NULL only if it'

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-01 Thread Jim Nasby
On 3/23/13 7:41 AM, Ants Aasma wrote: On Sat, Mar 23, 2013 at 6:04 AM, Jim Nasby wrote: Partitioned clock sweep strikes me as a bad idea... you could certainly get unlucky and end up with a lot of hot stuff in one partition. Surely that is not worse than having everything in a single

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-01 Thread Jim Nasby
On 4/1/13 4:55 PM, Merlin Moncure wrote: On Mon, Apr 1, 2013 at 4:09 PM, Andres Freund wrote: >On 2013-04-01 08:28:13 -0500, Merlin Moncure wrote: >>On Sun, Mar 31, 2013 at 1:27 PM, Jeff Janes wrote: >> >On Friday, March 22, 2013, Ants Aasma wrote: >> >> >> >>On Fri, Mar 22, 2013 at 10:22 P

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-01 Thread Jim Nasby
On 3/23/13 4:43 AM, Amit Kapila wrote: I have tried one of the idea's : Adding the buffers background writer finds reusable to freelist. http://www.postgresql.org/message-id/6C0B27F7206C9E4CA54AE035729E9C382852FF97@szxeml509-mbs This can reduce the clock swipe as it can find buffers from freelist

Re: [HACKERS] Page replacement algorithm in buffer cache

2013-04-01 Thread Jim Nasby
On 3/24/13 8:11 AM, Greg Smith wrote: On 3/22/13 8:45 AM, Ants Aasma wrote: However, I think the main issue isn't finding new algorithms that are better in some specific circumstances. The hard part is figuring out whether their performance is better in general. Right. The current page replac

Re: [PATCH] Exorcise "zero-dimensional" arrays (Was: Re: [HACKERS] Should array_length() Return NULL)

2013-04-03 Thread Jim Nasby
On 4/3/13 10:34 AM, Gavin Flower wrote: Maybe we should adopt the famous compromise of '0.5'? +0.5. ;P -- Jim C. Nasby, Data Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] Interesting post-mortem on a near disaster with git

2013-04-03 Thread Jim Nasby
On 3/26/13 6:42 AM, Cédric Villemain wrote: Le lundi 25 mars 2013 19:35:12, Daniel Farina a écrit : > On Mon, Mar 25, 2013 at 11:07 AM, Stefan Kaltenbrunner > > wrote: > >> Back when we used CVS for quite a few years I kept 7 day rolling > >> snapshots of the CVS repo, against just suc

Re: [HACKERS] [PATCH] add --throttle option to pgbench

2013-04-29 Thread Jim Nasby
On 4/29/13 1:08 PM, Fabien COELHO wrote: While I don't understand the part about his laptop battery, I think that there is a good use case for this. If you are looking at latency distributions or spikes, you probably want to see what they are like with a load which is like the one you expect h

Re: [HACKERS] Graph datatype addition

2013-04-29 Thread Jim Nasby
On 4/29/13 2:20 PM, Florian Pflug wrote: On Apr29, 2013, at 21:00 , Atri Sharma wrote: I think we find work arounds or make shifts at the moment if we need to use graphs in our database in postgres. If we have a datatype itself, with support for commonly used operations built inside the type it

Re: [HACKERS] Analyzing bug 8049

2013-04-29 Thread Jim Nasby
On 4/28/13 7:00 PM, Tom Lane wrote: Thoughts? Anybody know of a counterexample to the idea that no plug-ins call query_planner()? I would assume that anyone writing anything that calls such a low-level function reads -hackers regularly and would easily be able to handle whatever changes to t

Re: [HACKERS] Proposal to add --single-row to psql

2013-04-29 Thread Jim Nasby
On 4/28/13 7:50 AM, Craig Ringer wrote: I find it frustrating that I've never seen an @paraccel email address here and that few of the other vendors of highly customised Pg offshoots are contributing back. It's almost enough to make me like the GPL. FWIW, I think there's a pretty large barrie

Re: [HACKERS] Graph datatype addition

2013-05-08 Thread Jim Nasby
On 5/8/13 1:40 PM, Atri Sharma wrote: On Thu, May 2, 2013 at 7:58 AM, Atri Sharma wrote: Sent from my iPad On 02-May-2013, at 4:33, Misa Simic wrote: On Wednesday, May 1, 2013, Atri Sharma wrote: Hi all, Please find a probable prototype for the same: struct GraphNode { Oid NodeO

Re: [HACKERS] RETURNING syntax for COPY

2013-05-08 Thread Jim Nasby
On 5/8/13 12:54 PM, Jonathan S. Katz wrote: On May 8, 2013, at 1:16 PM, Tom Lane wrote: Heikki Linnakangas writes: On 08.05.2013 19:44, Tom Lane wrote: No there isn't; what you suggest would require FE/BE protocol extensions, making it several orders of magnitude more work than the other thi

Re: [HACKERS] RETURNING syntax for COPY

2013-05-08 Thread Jim Nasby
On 5/8/13 12:33 PM, Dimitri Fontaine wrote: Karol Trzcionka writes: as a continuation of my proposal expanding RETURNING syntax by What about implementing support for OLD/NEW in per-statement triggers? I guess you would expose the data via a SRF. Per statement NEW/OLD is an interesting case

Re: [HACKERS] about index inheritance

2013-05-08 Thread Jim Nasby
On 5/8/13 2:17 PM, Martijn van Oosterhout wrote: On Wed, May 08, 2013 at 10:19:08AM +0200, Vincenzo Melandri wrote: On Tue, May 7, 2013 at 11:55 PM, Robert Haas wrote: This is a really hard problem. If you pick this as your first project hacking on PostgreSQL, you will almost certainly fail.

Re: [HACKERS] GSOC Student Project Idea

2013-05-08 Thread Jim Nasby
On 5/8/13 3:54 AM, Heikki Linnakangas wrote: On 24.04.2013 14:31, Florian Pflug wrote: On Apr23, 2013, at 23:25 , Alexander Korotkov wrote: I've taken a brief look on the paper and implementation. As I can see iDistance implements some global building strategy. I mean, for example, it selects s

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-08 Thread Jim Nasby
On 4/5/13 6:39 PM, Jeff Davis wrote: On Fri, 2013-04-05 at 10:34 +0200, Florian Pflug wrote: Maybe we could scan forward to check whether a corrupted WAL record is followed by one or more valid ones with sensible LSNs. If it is, chances are high that we haven't actually hit the end of the WAL. I

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-08 Thread Jim Nasby
On 5/1/13 7:36 PM, Robert Haas wrote: On Mon, Apr 29, 2013 at 4:33 PM, Jim Nasby wrote: >On 4/28/13 7:50 AM, Craig Ringer wrote: >> >>I find it frustrating that I've never seen an @paraccel email address here >>and that few of the other vendors of highly c

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jim Nasby
On 5/8/13 7:34 PM, Jeff Davis wrote: On Wed, 2013-05-08 at 17:56 -0500, Jim Nasby wrote: Apologies if this is a stupid question, but is this mostly an issue due to torn pages? IOW, if we had a way to ensure we never see torn pages, would that mean an invalid CRC on a WAL page indicated there

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-12 Thread Jim Nasby
On 5/9/13 5:18 PM, Jeff Davis wrote: On Thu, 2013-05-09 at 14:28 -0500, Jim Nasby wrote: What about moving some critical data from the beginning of the WAL record to the end? That would make it easier to detect that we don't have a complete record. It wouldn't necessarily replace the

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-12 Thread Jim Nasby
On 5/10/13 1:06 PM, Jeff Janes wrote: Of course the paranoid DBA could turn off restart_after_crash and do a manual investigation on every crash, but in that case the database would refuse to restart even in the case where it perfectly clear that all the following WAL belongs to the recycled f

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-12 Thread Jim Nasby
On 5/11/13 11:27 AM, Tom Lane wrote: David Fetter writes: On Sat, May 11, 2013 at 11:17:03AM -0400, Robert Haas wrote: Some kind of extendable parser would be awesome. It would need to tie into the rewriter also. No, I don't have a clue what the design looks like. That's a direction sever

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-08-09 Thread Jim Nasby
On 8/9/12 10:56 AM, Jameison Martin wrote: [separate topic: pluggable heap manager] I'm quite interested in pursuing more aggressive compression strategies, and I'd like to do so in the context of the heap manager. I'm exploring having a pluggable heap manager implementation and would be intere

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-08-28 Thread Jim Nasby
On 8/27/12 5:19 PM, Greg Sabino Mullane wrote: Tom Lane replied: >>>Come on, really? Note that the above example works without casts if >>>you use int*or* bigint*or* numeric, but not smallint. That could be >>>fixed by causing sufficiently-small integers to lex as smallints, >>Is there any

Re: [HACKERS] Audit Logs WAS: temporal support patch

2012-08-28 Thread Jim Nasby
On 8/22/12 3:03 AM, Pavel Stehule wrote: SELECT coverage_amt >FROM policy FOR SYSTEM_TIME AS OF '2010-12-01' >WHERE id = ; > >SELECT count(*) >FROM policy FOR SYSTEM_TIME FROM '2011-11-30' TO '-12-30' >WHERE vin = 'A'; I like this design - it is simple without other objects The thi

Re: [HACKERS] temporal support patch

2012-08-28 Thread Jim Nasby
On 8/27/12 12:40 PM, Robert Haas wrote: On Sat, Aug 25, 2012 at 1:30 PM, David Johnston wrote: >My internals knowledge is basically zero but it would seem that If you >simply wanted the end-of-transaction result you could just record nothing >during the transaction and then copy whatever values

Re: [HACKERS] Audit Logs WAS: temporal support patch

2012-08-28 Thread Jim Nasby
On 8/28/12 2:51 PM, Pavel Stehule wrote: >The thing I don't like about this is it assumes that time is the best way to >refer to when things changed in a system. Not only is that a bad assumption, >it also means that relating things to history becomes messy. On second hand I don't have a problem

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Jim Nasby
On 9/2/12 7:23 PM, Tom Lane wrote: 4. As coded, the backend assumes the incoming pipe is on its FD 0 and the outgoing pipe is on its FD 1. This made the command line simple but I'm having second thoughts about it: if anything inside the backend tries to read stdin or write stdout, unpleasant thi

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2012-10-08 Thread Jim Nasby
On 10/5/12 9:57 PM, Michael Paquier wrote: In the current version of the patch, at the beginning of process a new index is created. It is a twin of the index it has to replace, meaning that it copies the dependencies of old index and creates twin entries of the old index even in pg_depend and

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2012-10-08 Thread Jim Nasby
On 10/8/12 6:12 PM, Tom Lane wrote: Jim Nasby writes: Yeah, what's the risk to renaming an index during concurrent access? SnapshotNow searches for the pg_class row could get broken by *any* transactional update of that row, whether it's for a change of relname or some other field.

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2012-10-08 Thread Jim Nasby
On 10/8/12 5:08 PM, Andres Freund wrote: On Monday, October 08, 2012 11:57:46 PM Jim Nasby wrote: >On 10/5/12 9:57 PM, Michael Paquier wrote: > >In the current version of the patch, at the beginning of process a new > >index is created. It is a twin of the index it has to r

Re: [HACKERS] PQping command line tool

2012-10-08 Thread Jim Nasby
On 10/4/12 11:34 AM, Greg Sabino Mullane wrote: I was wondering recently if there was any command line tool that >utilized PQping() or PQpingParams(). I searched the code and couldn't >find anything and was wondering if there was any interest to have >something like this included? I wrote somethi

Re: [HACKERS] Deparsing DDL command strings

2012-10-08 Thread Jim Nasby
On 10/5/12 11:15 AM, Tom Lane wrote: Also, we need to normalize that command string. Tools needing to look at >it won't want to depend on random white spacing and other oddities. Instead, they'll get to depend on the oddities of parse transformations (ie, what's done in the raw grammar vs. what'

[HACKERS] Optimizer regression

2012-10-13 Thread Jim Nasby
Just upgraded to 8.4 (I know, I know…) and ran across this. Unfortunately I have no way to test this on 9.x, so I don't know if it's been fixed or not. I'm hoping that someone *cough*Tom*cough* would quickly recognize whether this push into subquery issue has been fixed or not, so I haven't incl

Re: [HACKERS] Optimizer regression

2012-10-13 Thread Jim Nasby
On 10/13/12 2:45 PM, Tom Lane wrote: BTW, your workaround looks wrong --- you need to constrain the outside of the left join not the inside, no? Ugh, yes, you're correct. :( -- Jim C. Nasby, Database Architect j...@nasby.net 512.569.9461 (cell) http://j

Re: [HACKERS] Optimizer regression

2012-10-13 Thread Jim Nasby
On 10/13/12 2:45 PM, Tom Lane wrote: Jim Nasby writes: Just upgraded to 8.4 (I know, I know…) and ran across this. Unfortunately I have no way to test this on 9.x, so I don't know if it's been fixed or not. I'm hoping that someone *cough*Tom*cough* would quickly recognize wh

Re: [HACKERS] Optimizer regression

2012-10-13 Thread Jim Nasby
On 10/13/12 3:15 PM, Jim Nasby wrote: FWIW, it's definitely an issue of not being able to push down past the GROUP BY: I take that back... GROUP BY doesn't matter. It's an issue of having the EXISTS in the inner query. I realize the examples have gotten a bit silly, but this

Re: [HACKERS] [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

2012-10-22 Thread Jim Nasby
On 10/19/12 1:26 PM, Josh Berkus wrote: What I'm saying is, we'll get nowhere promoting an application queue which is permanently inferior to existing, popular open source software. My advice: Forget about the application queue aspects of this. Focus on making it work for replication and matvi

Re: [HACKERS] Deparsing DDL command strings

2012-10-29 Thread Jim Nasby
On 10/9/12 2:57 AM, Dimitri Fontaine wrote: Jim Nasby writes: I definitely want to be able to parse DDL commands to be able to either enforce things or to drive other parts of the system based on what's changing. Without the ability to capture (and parse) DDL commands I'm stuc

Re: [HACKERS] WIP checksums patch

2012-10-29 Thread Jim Nasby
On 10/1/12 12:22 PM, Josh Berkus wrote: >Perhaps we don't allow this to be turned per page, but rather per >cluster, and per-cluster would require the entire cluster to be >rewritten. We dicussed this last year, and options which require a total rewrite of the database in order to turn on the op

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Jim Nasby
On 10/28/12 10:22 AM, David Lee wrote: Thanks. Is this something viable as a feature request? Possibly, but it's not terribly high on the list. In the meantime, we've built a user-space index daemon at $WORK that generally solves this issue. We intend to release it at some point, but if you n

Re: [HACKERS] Deparsing DDL command strings

2012-10-29 Thread Jim Nasby
On 10/29/12 4:30 PM, Dimitri Fontaine wrote: In some cases we may need to divert or reject DDL, but that's a >secondary concern. Reject is already in, just RAISE ERROR from the trigger code. Divert is another sell entirely, we currently miss that capability. I'm interested into it for some DDLs.

Re: [HACKERS] Odd cruft in .psql_history in HEAD

2010-02-10 Thread Jim Nasby
On Jan 13, 2010, at 9:32 PM, Tom Lane wrote: > Jim Nasby writes: >> I noticed odd stuff showing up when I fired up an 8.3 psql after using psql >> in HEAD. It shows up in .psql_history as well: > > Platform? readline version? This is on snow leopard. FWIW it's still

Re: [HACKERS] shared_buffers documentation

2010-04-20 Thread Jim Nasby
On Apr 16, 2010, at 4:56 PM, Robert Haas wrote: > From reading this and other threads, I think I generally understand > that the perils of setting shared_buffers too high: memory is needed > for other things, like work_mem, a problem which is exacerbated by the > fact that there is some double buff

Re: [HACKERS] global temporary tables

2010-04-24 Thread Jim Nasby
On Apr 24, 2010, at 12:31 PM, Tom Lane wrote: > Robert Haas writes: >> At least AIUI, the use case for this feature is that you want to avoid >> creating "the same" temporary table over and over again. > > The context that I've seen it come up in is that people don't want to > clutter their funct

Re: [HACKERS] global temporary tables

2010-04-24 Thread Jim Nasby
On Apr 24, 2010, at 8:14 PM, Robert Haas wrote: >> One possibility: rename the existing pg_stats to pg_stats_permanent. Create >> a global temporary table called pg_stats_temporary. pg_stats becomes a union >> of the two. I know the backend wouldn't be able to use the view, but >> hopefully acce

Re: [HACKERS] global temporary tables

2010-04-25 Thread Jim Nasby
On Apr 24, 2010, at 10:02 PM, Tom Lane wrote: > Robert Haas writes: >> Pushing it into the RelFileNode has some advantages in terms of being >> able to get at the information from everywhere, but one thing that >> makes me think that's probably not a good decision is that we somtimes >> WAL-log re

Re: [HACKERS] inlining SQL functions

2010-04-25 Thread Jim Nasby
On Apr 2, 2010, at 12:12 PM, Tom Lane wrote: > Alexey Klyukin writes: >> Is there a reason why only a table free SQL functions are allowed to >> be inlined ? I wonder why a simple SQL function containing only a >> SELECT * FROM table can't be expanded inline ? > > If you're thinking of just repl

Re: [HACKERS] proposal - structured funcid and lineno as new fields in error message

2010-04-25 Thread Jim Nasby
On Mar 29, 2010, at 11:47 AM, Pavel Stehule wrote: > 2010/3/29 Tom Lane : >> Pavel Stehule writes: >>> can we add well structured information about function id and lineno to >>> ErrorData? >> >> The idea that I was toying with was to report the function OID and line >> number, which might as well

Re: [HACKERS] inlining SQL functions

2010-04-25 Thread Jim Nasby
On Apr 25, 2010, at 2:13 PM, Tom Lane wrote: > Jim Nasby writes: >> On Apr 2, 2010, at 12:12 PM, Tom Lane wrote: >>> If you're thinking of just replacing the call with a sub-SELECT >>> construct, that's no good in general because it would change the >>

Re: [HACKERS] Keepalive for max_standby_delay

2010-05-17 Thread Jim Nasby
On May 15, 2010, at 12:05 PM, Heikki Linnakangas wrote: > What exactly is the user trying to monitor? If it's "how far behind is > the standby", the difference between pg_current_xlog_insert_location() > in the master and pg_last_xlog_replay_location() in the standby seems > more robust and well-de

Re: [HACKERS] Partitioning/inherited tables vs FKs

2010-05-17 Thread Jim Nasby
On May 6, 2010, at 4:31 AM, Florian Pflug wrote: >> The use case for this was there were different news items, >> and there were another table for summaries, that could point >> to any of the news items table. Another use case could be >> a large partitioned table with an FK to the main table where

Re: [HACKERS] including PID or backend ID in relpath of temp rels

2010-05-17 Thread Jim Nasby
On May 6, 2010, at 10:24 PM, Robert Haas wrote: > On Tue, May 4, 2010 at 3:03 PM, Alvaro Herrera > wrote: > [smgr.c,inval.c] Do we need to call CacheInvalidSmgr for temporary > relations? I think the only backend that can have an smgr reference > to a temprel other than the owning bac

Re: [HACKERS] GUCs that need restart

2010-05-17 Thread Jim Nasby
On May 4, 2010, at 3:48 PM, Gurjeet Singh wrote: > There are quite a few GUC parameters that need restart. Is there a way we can > avoid some of them needing restart? I am specifically looking at archive_mode > and the new wal_level. For archive_mode you should check the archives; where was dis

Re: [HACKERS] Hot Standby tuning for btree_xlog_vacuum()

2010-05-17 Thread Jim Nasby
On Apr 29, 2010, at 3:20 PM, Tom Lane wrote: > Simon Riggs writes: >> Objections to commit? > > This is not the time to be hacking stuff like this. You haven't even > demonstrated that there's a significant performance issue here. I tend to agree that this point of the cycle isn't a good one to

Re: [HACKERS] SELECT * in a CREATE VIEW statement doesn't update column set automatically

2010-05-17 Thread Jim Nasby
On May 6, 2010, at 4:29 PM, Merlin Moncure wrote: > On Thu, May 6, 2010 at 3:23 PM, Andrew Dunstan wrote: >> And many places regard "select *" in anything other than throw-away queries >> as bad practice anyway. I have seen people get bitten by it over and over >> again, and I have worked at compa

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Jim Nasby
On 4/10/12 5:07 PM, Greg Smith wrote: I'd prefer to see at least usec resolution and 8 bytes of "dynamic range" for query related statistics. Any of these would be fine from a UI perspective to me: -float8 seconds -float8 msec -float8 usec -int8 usec I don't think int8 msec will be enough res

Re: [HACKERS] Dump EXTENSION sequences too

2012-04-19 Thread Jim Nasby
On 4/19/12 5:42 AM, Gianni Ciolli wrote: currently an EXTENSION can mark some of its tables as "configuration tables" using pg_catalog.pg_extension_config_dump(), so that pg_dump "does the right thing". I think it would be useful to mark sequences too, but unfortunately it is not possible; hence

Re: [HACKERS] Plan stability versus near-exact ties in cost estimates

2012-04-19 Thread Jim Nasby
On 4/19/12 5:39 PM, Tom Lane wrote: Now, neither of these fixes is perfect: what they would do is remove platform-specific instability from where the costs are basically equal and add some more in the range where the costs differ by almost exactly the fuzz factor. But the behavior near that poin

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Jim Nasby
On 4/26/12 7:41 AM, Simon Riggs wrote: 5. WRITE-SCALEABLE - the ability to partition data across nodes in a way that allows the solution to improve beyond the write rate of a single node. It would be valuable to look at READ-SCALEABLE as well; specifically a second form of "synchronous" replic

Re: [HACKERS] Memory usage during sorting

2012-05-01 Thread Jim Nasby
On 4/17/12 7:19 AM, Greg Stark wrote: On Mon, Apr 16, 2012 at 10:42 PM, Peter Geoghegan wrote: > All but 4 regression tests pass, but they don't really count > as failures, since they're down to an assumption in the tests that the > order certain tuples appear should be the same as our curre

Re: [HACKERS] Temporary tables under hot standby

2012-05-01 Thread Jim Nasby
On 4/25/12 6:16 AM, Simon Riggs wrote: The way standard-like temporary tables work is exactly why I assume > Noah proposes to implement them: because they work nicely with HS. Well, following a standard that no other major DBMS has followed is not great, especially if it leads to a non-useful

Re: [HACKERS] Temporary tables under hot standby

2012-05-01 Thread Jim Nasby
On 4/25/12 6:15 PM, Jaime Casanova wrote: On Wed, Apr 25, 2012 at 5:46 PM, Merlin Moncure wrote: > > I don't know how GTT play inside the Oracle stack such that they > aren't super popular, but if they work in the standby they will > quickly become a killer feature. IMNSHO it's annoying but

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-02 Thread Jim Nasby
On 4/29/12 9:27 AM, Kevin Grittner wrote: Maybe I can help with that by describing what the Wisconsin court system does for circuit court data. Thanks for the write-up, it was insightful. One thing I wanted to mention is that non-binary replication has an added advantage over binary from a DR

Re: [HACKERS] Future In-Core Replication

2012-05-02 Thread Jim Nasby
On 4/29/12 6:03 AM, Simon Riggs wrote: The DML-WITH-LIMIT-1 is required to do single logical updates on tables > with non-unique rows. > And as for any logical updates we will have huge performance problem > when doing UPDATE or DELETE on large table with no indexes, but > fortunately this pr

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-04 Thread Jim Nasby
On 5/3/12 2:54 PM, Josh Berkus wrote: (2) If logical transactions had been implemented as additions to > the WAL stream, and Slony was using that, do you think they would > still have been usable for this recovery? Quite possibly not. The key advantage that I see in londiste/slony replicati

Re: [HACKERS] Strange issues with 9.2 pg_basebackup & replication

2012-05-14 Thread Jim Nasby
On May 13, 2012, at 3:08 PM, Josh Berkus wrote: > More issues: promoting intermediate standby breaks replication. > > To be a bit blunt here, has anyone tested cascading replication *at all* > before this? Josh, do you have scripts that you're using to do this testing? If so can you post them so

Re: [HACKERS] Strange issues with 9.2 pg_basebackup & replication

2012-05-16 Thread Jim Nasby
Well, that is a form of testing. :) My point was that we need some kind of regression tests around all the new replication stuff, and if you had some scripts that would be a useful starting point. But it sounds like you haven't gotten that far with it, so... On 5/15/12 10:12 AM, Joshua Berkus

Re: [HACKERS] Strange issues with 9.2 pg_basebackup & replication

2012-05-16 Thread Jim Nasby
On 5/16/12 10:53 AM, Fujii Masao wrote: On Wed, May 16, 2012 at 3:43 AM, Joshua Berkus wrote: Before restarting it, you need to do pg_basebackup and make a base backup onto the standby again. Since you started the standby without recovery.conf, a series of WAL in the standby has gotten incons

Re: [HACKERS] heap metapages

2012-05-25 Thread Jim Nasby
On 5/22/12 12:09 PM, Simon Riggs wrote: On 22 May 2012 13:52, Robert Haas wrote: It seems pretty clear to me that making pg_upgrade responsible for emptying block zero is a non-starter. But I don't think that's a reason to throw out the design; I think it's a problem we can work around. I l

Re: [HACKERS] Fake async rep target

2012-05-29 Thread Jim Nasby
On 5/29/12 2:46 PM, james wrote: How easy would it be to implement a fake async rep target? Perhaps even as something that a server could allow a connection to request? (ie a suitably permissioned connection could convert itself to receive n async replication stream, rather than being statical

Re: [HACKERS] temporal support patch

2012-05-29 Thread Jim Nasby
On 5/18/12 2:06 AM, Miroslav Šimulčík wrote: - no data redundancy - in my extension current versions of entries are stored only once in original table (in table_log - entries are inserted to both original and log table) That's not necessarily a benefit... it makes querying for both history *a

Re: [HACKERS] VIP: new format for psql - shell - simple using psql in shell

2012-06-04 Thread Jim Nasby
On 5/26/12 10:16 PM, Pavel Stehule wrote: My idea is secure to separator - because separator is just space and new line and these symbols are escaped. ISTM it'd be a really good idea to support something other than space, since presumably that'd be trivial. I'm not a fan of supporting the arr

Re: [HACKERS] No, pg_size_pretty(numeric) was not such a hot idea

2012-06-04 Thread Jim Nasby
On 5/27/12 2:54 PM, Euler Taveira wrote: On 27-05-2012 10:45, Fujii Masao wrote: OK, let me propose another approach: add pg_size_pretty(int). If we do this, all usability and performance problems will be solved. I wouldn't like to add another function but if it solves both problems... +1. F

<    6   7   8   9   10   11   12   13   14   15   >