Re: [HACKERS] Experimental evaluation of PostgreSQL's query optimizer

2015-12-23 Thread Jim Nasby
estimate as well as the estimated upper and lower error margin for the estimate. An estimate of 827 +0 -400 could have very different meaning than an estimate of [427,827]. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] Experimental evaluation of PostgreSQL's query optimizer

2015-12-23 Thread Jim Nasby
d) be automated though, which is computing these statistics for all foreign keys. We can have a way to disable that for specific keys if necessary, but I'd bet it's extremely rare to have a FK that you never join on. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Exp

Re: [HACKERS] Possible marginally-incompatible change to array subscripting

2015-12-22 Thread Jim Nasby
. which I suspect would make any such "type" all but unusable. The other problem would be having it deal with any other data type, but at least there's ways you can work around that for the most part. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics

Re: [HACKERS] psql - -dry-run option

2015-12-20 Thread Jim Nasby
happens to contain a COMMIT, you're hosed. I can see some use for a "must rollback" mode of BEGIN. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent

Re: [HACKERS] Optimizing away second VACUUM heap scan when only BRIN indexes on table

2015-12-20 Thread Jim Nasby
ot; of BRIN index(es) occurs, avoiding a call to lazy_vacuum_heap(), just as when there are no indexes on the table whatsoever? ISTM the big question here is how vacuum would know it can skip this since we wouldn't want to hard-code this for BRIN. -- Jim Nasby, Data Architect, Blue Tre

Re: [HACKERS] Weighted Stats

2015-12-20 Thread Jim Nasby
On 11/2/15 5:46 PM, David Fetter wrote: I'd like to add weighted statistics to PostgreSQL Anything happen with this? If community isn't interested, ISTM it'd be good to put this in PGXN. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Da

Re: [HACKERS] plpgsql - DECLARE - cannot to use %TYPE or %ROWTYPE for composite types

2015-12-20 Thread Jim Nasby
it into a commitfest? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Remove array_nulls?

2015-12-18 Thread Jim Nasby
re than just one GUC, and I agree with that, and I'm willing to investigate when the current compat GUCs went in and create a patch to remove the really old ones. My inclination would be to just do this as part of 10.0. (And I agree with Robert's comments about parallel being the most

Re: [HACKERS] Disabling an index temporarily

2015-12-16 Thread Jim Nasby
g. (Doing a single bulk insert to the index at the end of an INSERT should be safe though because none of those tuples are visible yet, though I'd have to make sure your backend didn't try to use the index for anything while the command was running... like as part of a trigger

Re: [HACKERS] Proposal: custom compression methods

2015-12-16 Thread Jim Nasby
was a version that would expand 4 byte varlena to 8 byte as needed. And we're not painting ourselves in the corner - if we decide to increase the varlena header size in the future, this patch does not make it any more complicated. True. -- Jim Nasby, Data Architect, Blue Treble Con

Re: [HACKERS] Remove array_nulls?

2015-12-16 Thread Jim Nasby
On 12/16/15 6:01 PM, Robert Haas wrote: On Tue, Dec 15, 2015 at 1:26 AM, Michael Paquier wrote: On Tue, Dec 15, 2015 at 2:57 AM, Jim Nasby wrote: On 12/11/15 2:57 PM, Tom Lane wrote: Jim Nasby writes: Perhaps, but I'd like to have a less ad-hoc process about it. What's our

Re: [HACKERS] Disabling an index temporarily

2015-12-15 Thread Jim Nasby
me set of queries you can test against and call it good. FWIW, I also don't see the use case for disabling maintenance on an index. Just drop it and if you know you'll want to recreate it squirrel away pg_get_indexdef() before you do. -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] Logical replication and multimaster

2015-12-15 Thread Jim Nasby
y issues would be if done in the server. Maybe they wouldn't be that bad. I suspect the audience for this code would be much larger if it was in the server as opposed to a C library. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and Postg

Re: [HACKERS] 9.5RC1 wraps *today*

2015-12-15 Thread Jim Nasby
y, I think this is actually a good luck sign. ;P -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

[HACKERS] Clarify vacuum verbose message

2015-12-15 Thread Jim Nasby
been a large amount of deletes; I'll check with them tomorrow. IMHO we need to change the messages so they are explicit about line pointers vs actual tuples. Trying to obfuscate that just leads to confusion. heap_page_prune needs to report only non-rootlp tuples that were pruned. (None

Re: [HACKERS] Proposal: custom compression methods

2015-12-14 Thread Jim Nasby
TOAST then maybe that's enough. The other thing this might buy us are a few bits that could be used to support Datum versioning for other purposes, such as when the binary format of something changes. I would think that at some point we'll need that for pg_upgrade. -- Jim Nasby, Data Ar

Re: [HACKERS] Remove array_nulls?

2015-12-14 Thread Jim Nasby
On 12/11/15 2:57 PM, Tom Lane wrote: Jim Nasby writes: A quick doc search indicates this config was created in 9.0, though the docs state it's for a change that happened in 8.2[1]. Don't know what you're looking at, but the GUC is definitely there (and documented) in 8.2.

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/11/15 6:25 PM, Jim Nasby wrote: On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config. I thought we'd more or less rej

Re: [HACKERS] Add IS (NOT) DISTINCT to subquery_Op

2015-12-11 Thread Jim Nasby
On 12/10/15 7:03 PM, Tom Lane wrote: Jim Nasby writes: Is there any reason we couldn't/shouldn't support IS DISTINCT in subquery_Op? (Or really, just add support to ANY()/ALL()/(SELECT ...)?) It's not an operator (in the sense of something with a pg_operator OID), which mean

[HACKERS] Remove array_nulls?

2015-12-11 Thread Jim Nasby
onfig-compatible.html -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Add IS (NOT) DISTINCT to subquery_Op

2015-12-10 Thread Jim Nasby
Is there any reason we couldn't/shouldn't support IS DISTINCT in subquery_Op? (Or really, just add support to ANY()/ALL()/(SELECT ...)?) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Tr

[HACKERS] array_remove(anyarray, anyarray)

2015-12-10 Thread Jim Nasby
search and repalace arrays instead of Datums? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

[HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-10 Thread Jim Nasby
ninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config. Forwarded Message Subject: [GENERAL] pgxs/config/missing is... missing Date: Wed, 28 Oct 2015 12:54:54 -0500 From: Jim Nasby To: pgsql-general CC:

Re: [HACKERS] [PATCH] Equivalence Class Filters

2015-12-08 Thread Jim Nasby
that ever gets built it might be useful for what you propose as well. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] [PATCH] Equivalence Class Filters

2015-12-08 Thread Jim Nasby
that we're willing to add the brawn, but we're not willing to add the brain. If this is the case then it's a shame, as I think we can have both. So I very much agree on the fact that we must find a way to maintain support and high performance of small OLTP databases too. +1 -- Jim

Re: Fwd: [HACKERS] Another little thing about psql wrapped expanded output

2015-12-08 Thread Jim Nasby
On 12/8/15 1:36 PM, Robert Haas wrote: Your point is also valid, so I don't mean to detract from that. But the status quo is definitely annoying. +1, and I even use -S. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL

Re: [HACKERS] [PATCH] Equivalence Class Filters

2015-12-07 Thread Jim Nasby
practical that would be. Maybe a better starting point would be a planner timeout. I definitely agree we need some method to limit planning time when necessary (ie: OLTP). Without that we'll never be able to start testing more complex optimizations. -- Jim Nasby, Data Architect, Blue Tr

Re: [HACKERS] [PATCH] Equivalence Class Filters

2015-12-07 Thread Jim Nasby
On 12/7/15 9:54 AM, Tom Lane wrote: Jim Nasby writes: >On 12/6/15 10:38 AM, Tom Lane wrote: >>I said "in most cases". You can find example cases to support almost any >>weird planner optimization no matter how expensive and single-purpose; >>but that is the wro

Re: [HACKERS] [PATCH] Equivalence Class Filters

2015-12-07 Thread Jim Nasby
if there was a fast, easy way to figure out whether a query would be expensive enough to go the whole 9 yards on planning it but at this point I suspect a simple GUC would be a big improvement. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Archi

Re: [HACKERS] Attach comments to functions' parameters and return value

2015-12-06 Thread Jim Nasby
ce to fix that, but that's mostly a separate matter. Though, it would probably be nice if all of this stuff (along with the regprocedure input function) could be factored into a single piece of code... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Dat

Re: [HACKERS] Double linking MemoryContext children

2015-12-06 Thread Jim Nasby
them. If we do this, I think we need to change both places that are affected, so ResourceOwnerCreate() in resowner.c would need a line or two added. ResourceOwnerCreate() sets ResourceOwnerData.nextchild, not MemoryContextData.nextchild. Anything ever happen with this? -- Jim Nasby, Data Arch

Re: [HACKERS] Size of Path nodes

2015-12-04 Thread Jim Nasby
On 12/4/15 5:14 PM, Peter Geoghegan wrote: On Fri, Dec 4, 2015 at 2:44 PM, Jim Nasby wrote: >I suspect Cachegrind[1] would answer a lot of these questions (though I've >never actually used it). I can't get postgres to run under valgrind on my >laptop, but maybe someone tha

Re: [HACKERS] Size of Path nodes

2015-12-04 Thread Jim Nasby
sed it). I can't get postgres to run under valgrind on my laptop, but maybe someone that's been successful at valgrind can try cachegrind (It's just another mode of valgrind). [1] http://valgrind.org/docs/manual/cg-manual.html -- Jim Nasby, Data Architect, Blue Treble Consult

Re: [HACKERS] proposal: function parse_ident

2015-12-02 Thread Jim Nasby
;s a PITA to cut and paste the whole argument list into multiple REVOKE/GRANT/COMMENT on statements. Even worse, not all the options of CREATE FUNCTION are supported in those other commands, so often you can't even just cut and paste. -- Jim Nasby, Data Architect, Blue Treble Consulting, Au

Re: [HACKERS] psql: add \pset true/false

2015-12-02 Thread Jim Nasby
On 11/15/15 7:37 PM, Peter Eisentraut wrote: On 11/15/15 3:20 PM, Jim Nasby wrote: As to the argument about displaying a check or an X, why should that capability only exist for boolean types? For example, why not allow psql to convert a numeric value into a bar of varying sizes? I&#x

Re: [HACKERS] custom function for converting human readable sizes to bytes

2015-11-24 Thread Jim Nasby
On 11/24/15 10:57 PM, Pavel Stehule wrote: +errmsg("parameter \"%s\" isn't valid size value", Should read " isn't a valid size value" -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Arc

Re: [HACKERS] Revisiting pg_stat_statements and IN() (Was: Re: pg_stat_statements fingerprinting logic and ArrayExpr)

2015-11-24 Thread Jim Nasby
On 11/24/15 7:46 PM, Peter Geoghegan wrote: On Tue, Nov 24, 2015 at 5:39 PM, Jim Nasby wrote: Another not-uncommon case is IN ( '1', '2', ... , '2342' ); in other words, treating an integer as text. A lot of frameworks like to do that and just push the problem o

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-24 Thread Jim Nasby
compares to reltuples/relpages from pg_class for the heap... but I suspect that's pretty serious overkill. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via

Re: [HACKERS] Revisiting pg_stat_statements and IN() (Was: Re: pg_stat_statements fingerprinting logic and ArrayExpr)

2015-11-24 Thread Jim Nasby
se.. Since there's a few different things people might want, maybe a good first step is to allow extending/changing the jumbling decision at the C level. That would make it easy for a knowledgeable enough person to come up with an alternative as a plugin that regular users could use. -- Jim N

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Jim Nasby
nly scan of 10% of data instead of scanning and sorting the full table. There are other cases where you'd want to build an index off an existing index as well. It's not that uncommon to have small, specialized indexes that are fully or partially a subset of another index. -- Jim Nasby, D

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Jim Nasby
reading an index could keep a number of children busy retrieving heap tuples and processing them. It might be nice if an index scan node just fired up it's own workers and talked to them directly. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Arch

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-24 Thread Jim Nasby
it might be worth it to allow index AMs to provide their own vacuuming feedback, but I think that's way out of scope for this patch. :) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! ht

Re: [HACKERS] custom function for converting human readable sizes to bytes

2015-11-23 Thread Jim Nasby
' too (or prettysize?). No reason it has to be tied to bytes (in particular this would work for bits too). If we're going to add this, I suppose it should support the 'i prefixes' too (GiB, MiB, etc). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts i

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2015-11-23 Thread Jim Nasby
relation, because if you can't actually shrink the heap you're not going to make any progress. But since none of this will help at all in the default case where WAL is on the same filesystem as the data, I don't know that it's worth it. -- Jim Nasby, Data Architect, Blue Treb

Re: [HACKERS] [PROPOSAL] Inputs on forcing VACUUM VERBOSE to write timestamp

2015-11-23 Thread Jim Nasby
manual vacuum, unless there was a LOGTIME option added to vacuum as well. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] custom function for converting human readable sizes to bytes

2015-11-22 Thread Jim Nasby
On 11/22/15 2:11 PM, Pavel Stehule wrote: What about pg_size(text), pg_size(value bigint, unit text) ? I like, though I'd make it numeric or float. pg_size(3.5, 'GB') certainly seems like a reasonable use case... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austi

Re: [HACKERS] count_nulls(VARIADIC "any")

2015-11-22 Thread Jim Nasby
On 11/22/15 11:34 AM, Marko Tiikkaja wrote: On 2015-11-22 18:29, Jim Nasby wrote: Only if you know how many columns there already are. Or does this not work if you hand it a row? It "works" in the sense that it tells you whether the row is NULL or not. I.e. the answer will always

Re: [HACKERS] custom function for converting human readable sizes to bytes

2015-11-22 Thread Jim Nasby
g_size_pretty(text)? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] count_nulls(VARIADIC "any")

2015-11-22 Thread Jim Nasby
On 11/20/15 11:55 PM, Marko Tiikkaja wrote: On 2015-11-21 06:52, Jim Nasby wrote: On 11/20/15 11:12 PM, Marko Tiikkaja wrote: On 2015-11-21 06:02, I wrote: Here's a patch implementing this under the name num_nulls(). For January's CF, of course. I forgot to update the some ref

Re: [HACKERS] GIN pending list clean up exposure to SQL

2015-11-20 Thread Jim Nasby
On 11/19/15 10:47 AM, Jaime Casanova wrote: - only superusers? I would think the owner of the table (index?) should also be able to run this. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in

Re: [HACKERS] count_nulls(VARIADIC "any")

2015-11-20 Thread Jim Nasby
o be a not-null equivalent as well? I've definitely wanted both variations in the past. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mail

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-20 Thread Jim Nasby
Presumably that's going to be a lot more expensive than an increment operation. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing li

Re: [HACKERS] [PROPOSAL] TAP test example

2015-11-19 Thread Jim Nasby
re we don't have to worry about random output changes, like what line number on a script caused an error? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent v

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-19 Thread Jim Nasby
could be reported for the same. Note that, it would have to be reported from lazy_vacuum_rel(). ISTM this is similar to the problem of reporting index status, namely that a progress reporting method needs to accept reports from multiple places in the code. -- Jim Nasby, Data Architect, Blue Treb

Re: [HACKERS] [PROPOSAL] TAP test example

2015-11-19 Thread Jim Nasby
g benefits to tap is not dealing with raw psql output... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-17 Thread Jim Nasby
e the title originally sought? Basically, it follows the same pattern that all-visible bits do, except instead of indicating a page is all-visible, the bit shows that all tuples on the page are frozen. That allows a scan_all vacuum to skip those pages. -- Jim Nasby, Data Architect, Blue T

Re: [HACKERS] Extracting fields from 'infinity'::TIMESTAMP[TZ]

2015-11-17 Thread Jim Nasby
y isn't a normal number. My specific fear is that now people will have to do a bunch of IF timestamp IS NOT NULL THEN ... to get the behavior they need. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Tro

Re: [HACKERS] [PATCH] SQL function to report log message

2015-11-16 Thread Jim Nasby
On 11/15/15 10:56 PM, dinesh kumar wrote: So, shall we make this pg_report_log TO pg_write_log OR pg_ereport OR from you. Why not pg_raise to mirror plpgsql? (The function does have the same semantics, right? It's not doing something like only sending to the log and not the client?) -

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2015-11-16 Thread Jim Nasby
hat you've just written, which means the data's still in shared buffers. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] psql: add \pset true/false

2015-11-15 Thread Jim Nasby
CT repeat( '*', blah * 30 / max_of_blah ). I'm sure there's other examples people could think of. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com

Re: [HACKERS] Documentation tweak for row-valued expressions and null

2015-11-10 Thread Jim Nasby
his is a common point of confusion. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Some questions about the array.

2015-11-06 Thread Jim Nasby
l for ditching variable start, full stop. Since the start-pos is recorded in the array, I wonder if it's worth supporting negative indexing for arrays with the default 1-indexed element numbering, and just ERRORing for others. Does anyone really use anything else? I'd prefer that over us

Re: [HACKERS] Request: pg_cancel_backend variant that handles 'idle in transaction' sessions

2015-11-06 Thread Jim Nasby
's a start. Barring that, at least Even then it would be very easy to mess this up. warnings tend to stand out in the database log. That depends greatly on how much other stuff is in the log. Something else I wish we had was the ability to send different log output to different pla

Re: [HACKERS] RFC/WIP: adding new configuration options to TOAST

2015-11-03 Thread Jim Nasby
ch like to drive that column out of main and into toast. I think target_tuple_size would let me do that. +1 on having a way to induce that behavior, as I've faced the same thing in the past. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture

Re: [HACKERS] Request: pg_cancel_backend variant that handles 'idle in transaction' sessions

2015-11-02 Thread Jim Nasby
they're using transactions, but short of that this is the next best thing. Actually, one other thing that would help is to have the ability to turn this into an ERROR: begin; WARNING: there is already a transaction in progress -- Jim Nasby, Data Architect, Blue Treble Consulting, Aust

Re: [HACKERS] pglogical_output - a general purpose logical decoding output plugin

2015-11-02 Thread Jim Nasby
raw asciidoctor. ;) Great work on this! -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] September 2015 Commitfest

2015-11-01 Thread Jim Nasby
e of pg_regress itself. Would that mostly suffice for what you're looking for? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2015-11-01 Thread Jim Nasby
test to pgbench or may be use some test which adheres to TPC-C specification. Infact, I remember [1] people posting test results with such a workload showing ProcArrayLock as contention. [1] - http://www.postgresql.org/message-id/e8870a2f6a4b1045b1c292b77eab207c77069...@szxema501-mbx.china.huawe

[HACKERS] Patch to install config/missing

2015-10-30 Thread Jim Nasby
Currently, config/missing isn't being installed. This can lead to confusing error messages, such as if Perl isn't found and something needs it [1]. Attached patch adds it to install and uninstall recipes. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analy

Re: [HACKERS] Personal note: changing employers

2015-10-28 Thread Jim Nasby
Elephant that ended up on a Cloud[1] is suddenly turning into a Hippo. :P [1] https://secure2.sfdcstatic.com/common/assets/images/home/sfdc-home-footer-logo.png -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get

Re: [HACKERS] Proposal: Trigonometric functions in degrees

2015-10-26 Thread Jim Nasby
(Ok, I guess we just sprayed the planet with debris instead of poking an eye out...) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] questions about PG update performance

2015-10-26 Thread Jim Nasby
t of work. To me, the real tradeoff between Postgres and Oracle (or any other commercial database) is whether you'd rather spend money on expert employees or software contracts. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and

Re: [HACKERS] Freezing without cleanup lock

2015-10-25 Thread Jim Nasby
On 10/22/15 6:39 PM, Alvaro Herrera wrote: Jim Nasby wrote: That would be the minimal-impact version, yes. But I suspect if we went through the trouble to do that, it would be just as easy to attempt the freeze regardless of what scan_all is set to. You mean if !scan_all we conditional-get

Re: [HACKERS] [patch] extensions_path GUC

2015-10-23 Thread Jim Nasby
it a lot easier for someone to customize how testing works under PGXS. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Freezing without cleanup lock

2015-10-22 Thread Jim Nasby
On 10/21/15 3:14 PM, Alvaro Herrera wrote: Jim Nasby wrote: While warning a client that just did a Slony-based version upgrade to make sure to freeze the new database, it occurred to me that it should be safe to freeze without the cleanup lock. This is interesting because it would allow a

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:53 PM, Alvaro Herrera wrote: Jim Nasby wrote: But yes, this is all very hand-wavy without any actual data on what percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work for WAL... does anyone have a script/tool that breaks out what percentage of a WAL fi

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
To me this sounds like a recipe for disaster (i.e. complex bugs). WAL (and thus CRC checksums) differing between nodes. Ugh. The WAL would *not* differ. This would only affect streaming replication, and only the stream itself. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Jim Nasby
On 10/22/15 5:07 PM, Robert Haas wrote: On Thu, Oct 22, 2015 at 5:51 PM, Jim Nasby wrote: >It's also a permanent ID when the relation is first created. No it isn't. Is there no case where it can be a permanent XID for a table or toast table? The other issue is relminmxid, wh

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
avy without any actual data on what percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work for WAL... does anyone have a script/tool that breaks out what percentage of a WAL file is FPIs? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics,

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:03 PM, Andres Freund wrote: On 2015-10-22 16:34:38 -0500, Jim Nasby wrote: ISTM it should be possible to avoid sending full page writes to a streaming replica once the replica has reached a consistent state. I assume that the replica would still need to write full pages to it&#

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
ivalent to RAISE, to avoid the make-work of writing that function. That's why I disagree with your statement that there's no point to this function even if it acts the same as RAISE. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture a

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
ver still need full page images in case of a crash? (Assuming full_page_writes is enabled...) The other issue is chained replicas, where one of the children may need full page writes (during initial copy). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, D

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
can use RAISE statement. It prevents everyone from reinventing the 'create a function wrapper around RAISE' wheel that several people on this list alone have admitted to. I think there's plenty of value in that. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austi

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Jim Nasby
On 10/22/15 4:18 PM, Robert Haas wrote: On Wed, Oct 21, 2015 at 1:33 PM, Jim Nasby wrote: Currently, xid_age() returns INT_MAX for a permanent xid. The comment in the function that 'Permanent XIDs are always infinitely old' may be technically correct, but returning INT_MAX is

[HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
when to do so, instead of sending the full page image. Presumably this would be a big win for replication over a WAN. Am I missing something? I see that pglesslog is no longer supported but couldn't find any particular reason for that... -- Jim Nasby, Data Architect, Blue Treble Consulting,

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
ame as RAISE does. In other words, this function and raise should operate exactly the same unless there's a really strong reason not to. Otherwise it's just going to create confusion. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architect

[HACKERS] Freezing without cleanup lock

2015-10-21 Thread Jim Nasby
eanup lock. Does anyone have a feel for whether scan_all vacuums blocking on the cleanup lock is an actual problem? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com --

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-21 Thread Jim Nasby
ISTM VISIBILITY_MAP_FROZEN_BIT_CAT_VER shold be defined in catversion.h instead of pg_upgrade.h though, to ensure it's correctly updated when this gets committed though. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble?

[HACKERS] Change behavior of (m)xid_age

2015-10-21 Thread Jim Nasby
danger. I think we should change it to return either 0, -1, or INT_MIN. To me, 0 makes the most sense for monitoring relfrozenxid. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://Blue

Re: [HACKERS] Why no CONSTANT for row variables in plpgsql?

2015-10-20 Thread Jim Nasby
On 10/19/15 7:12 PM, Tom Lane wrote: Jim Nasby writes: What did seem odd is that while processing the DECLARE section there were plpgsql datums for tt.a and tt.b. I would have expected the assignment to produce a row datum of type tt. Yeah, that's the thing that's weird about plp

Re: [HACKERS] [PROPOSAL] Improvements of Hunspell dictionaries support

2015-10-20 Thread Jim Nasby
seems pretty excessive, if that's the case... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] [PATCH] Typos in comments

2015-10-20 Thread Jim Nasby
On 10/20/15 11:08 AM, CharSyam wrote: I fixed some typos in posgres. They are all in comments. :) These all look good to me. RFC. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-20 Thread Jim Nasby
ugh on 32-bit you probably can't get to 2GB anyway ...). FWIW, I've verified on $CLIENT's system that this works as Tom described. The truncation happened somewhere a bit north of 3GB, which seems odd as this is a 64 bit system. But at least there were no OOM errors. -- Jim Nasby,

Re: [HACKERS] ROWS FROM(): A Foolish (In)Consistency?

2015-10-19 Thread Jim Nasby
ALUES (...), ..., (...)), (SELECT ... ), (INSERT ... RETURNING ... ), my_srf() ) AS t(...) would actually work. There's been a few places where I would have found that handy. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Archit

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-19 Thread Jim Nasby
ase I think it's inline with what the C code is already doing by testing for \0. I suppose if we get the function it's not that bad since at least we get the functionality, so I'll stop arguing it. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analyti

Re: [HACKERS] Why no CONSTANT for row variables in plpgsql?

2015-10-19 Thread Jim Nasby
On 10/19/15 5:16 PM, Jim Nasby wrote: Yeah, was hoping someone knew offhand why this was a problem. Guess I'll rip the checks out and see what explodes. ... and what blows up is exec_eval_datum(): case PLPGSQL_DTYP

Re: [HACKERS] Why no CONSTANT for row variables in plpgsql?

2015-10-19 Thread Jim Nasby
On 10/18/15 10:16 PM, Tom Lane wrote: Jim Nasby writes: Is there a particular reason why row and record variables can't be CONSTANT in plpgsql? Well, you can't usefully do anything with such a variable unless it can be initialized, which isn't currently supported either:

Re: [HACKERS] SuperUser check in pg_stat_statements

2015-10-19 Thread Jim Nasby
all users. You can set that up today by defining a view on top of pg_stat_statements (or maybe it needs a SECDEF SRF... been a while since I've done it). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? G

[HACKERS] Why no CONSTANT for row variables in plpgsql?

2015-10-18 Thread Jim Nasby
Is there a particular reason why row and record variables can't be CONSTANT in plpgsql? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-ha

<    3   4   5   6   7   8   9   10   11   12   >