Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-10 Thread Jim Nasby
On 1/9/14, 10:58 PM, Tom Lane wrote: Jim Nasby writes: ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of it, can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to dea

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Jim Nasby
On 1/10/14, 12:59 PM, Joshua D. Drake wrote: I know I am the one that instigated all of this so I want to be very clear on what I and what I am confident that my customers would expect. If a synchronous slave goes down, the master continues to operate. That is all. I don't care if it is config

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-10 Thread Jim Nasby
On 1/10/14, 4:14 PM, Merlin Moncure wrote: On Fri, Jan 10, 2014 at 4:10 PM, Jeff Janes wrote: On Fri, Jan 10, 2014 at 1:26 PM, Jim Nasby wrote: On 1/9/14, 10:58 PM, Tom Lane wrote: Jim Nasby writes: ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I&#x

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-10 Thread Jim Nasby
On 1/10/14, 4:40 PM, Peter Geoghegan wrote: My problem is that in general I'm not sold on the actual utility of making this kind of row locking work with exclusion constraints. I'm sincerely having a hard time thinking of a practical use-case (although, as I've said, I want to make it work with I

Re: [HACKERS] Add CREATE support to event triggers

2014-01-10 Thread Jim Nasby
On 1/10/14, 3:40 PM, Simon Riggs wrote: Given that CREATE SCHEMA with multiple objects is less well used, its a reasonable restriction to accept for one release, if the alternative is to implement nothing at all of value. Especially since we are now in the third year of development of this set of

Re: [HACKERS] Add CREATE support to event triggers

2014-01-10 Thread Jim Nasby
On 1/10/14, 5:22 PM, Alvaro Herrera wrote: Here's one idea: create a contrib module that (somehow, via APIs to be >invented) runs every DDL command that gets executed through the >deparsing code, and then parses the result and executes*that* instead >of the original command. Then, add a build t

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-10 Thread Jim Nasby
On 1/10/14, 1:07 PM, Tom Lane wrote: Florian Pflug writes: >On Jan10, 2014, at 19:08 , Tom Lane wrote: >>Although, having said that ... maybe "build your own aggregate" would >>be a reasonable suggestion for people who need this? I grant that >>it's going to be a minority requirement, maybe

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Jim Nasby
On 1/10/14, 6:19 PM, Adrian Klaver wrote: 1) Async. Runs at the speed of the master as it does not have to wait on the standby to signal a successful commit. There is some degree of offset between master and standby(s) due to latency. 2) Sync. Runs at the speed of the standby + latency between

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-10 Thread Jim Nasby
On 1/10/14, 6:51 PM, Peter Geoghegan wrote: On Fri, Jan 10, 2014 at 4:09 PM, Jim Nasby wrote: >Well, the usual example for exclusion constraints is resource scheduling >(ie: scheduling what room a class will be held in). In that context is it >hard to believe that you might want to ME

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:19 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: On a related note, there's also the problem of double-buffering. When we read a page into shared_buffers, we leave a copy behind in the OS buffers, and similarly on write-out. It's very unclear what

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:27 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: On 1/13/14, 2:19 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: On a related note, there's also the problem of double-buffering. When we read a page

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:37 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:32 PM, Jim Nasby wrote: That's my point. In terms of kernel-postgres interaction, it's fairly simple. What's not so simple, is figuring out what policy to use. Remember, you cannot tell the kernel to put s

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Jim Nasby
On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres is

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Jim Nasby
On 1/13/14, 10:40 AM, Merlin Moncure wrote: On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer wrote: >Implicit casts to text, anybody? This backward compatibility break orphaned the company I work for on 8.1 until last year and very nearly caused postgres to be summarily extirpated (only rescued a

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 1:44 AM, Pavel Stehule wrote: 2014/1/12 Florian Pflug mailto:f...@phlo.org>> On Jan12, 2014, at 22:37 , Pavel Stehule mailto:pavel.steh...@gmail.com>> wrote: > There is GUC for variable_conflict already too. In this case I would to > enable this functionality everyw

Re: [HACKERS] Syntax of INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Jim Nasby
On 1/12/14, 9:35 PM, Andreas Karlsson wrote: On 01/12/2014 11:20 PM, Peter Geoghegan wrote: On Sun, Jan 12, 2014 at 8:12 AM, Andreas Karlsson wrote: On 01/11/2014 11:42 PM, Peter Geoghegan wrote: I recently suggested that rather than RETURNING REJECTS, we could have a REJECTING clause, which

Re: [HACKERS] Syntax of INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Jim Nasby
On 1/13/14, 5:05 PM, Peter Geoghegan wrote: On Mon, Jan 13, 2014 at 2:20 PM, Jim Nasby wrote: Well, a common case for INSERT RETURNING is to get your set of surrogate keys back; so I think users would want the ability to RETURN what finally made it into the table. Your update can also have a

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 5:57 PM, Josh Berkus wrote: On 01/13/2014 03:41 PM, Florian Pflug wrote: It therefor isn't an oversight that SELECT ... INTO allows multiple result rows but INSERT/UPDATE/DELETE forbids them, it's been done that way on purpose and for a reason. We shouldn't be second-guessing ourselv

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 6:16 PM, Florian Pflug wrote: On Jan14, 2014, at 00:52 , Marko Tiikkaja wrote: When I've worked with PL/PgSQL, this has been a source of a few bugs that would have been noticed during testing if the behaviour of INTO wasn't as dangerous as it is right now. The question is, how man

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Jim Nasby
On 1/13/14, 5:33 PM, Craig Ringer wrote: So I guess the question is: Is it worth all that hassle to remove a misfeature you have to go out of your way to use? Is support for non-1 lower bounds stopping us from doing something useful and important? Or is it just an irritation that it exists? It'

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 4:44 PM, Andres Freund wrote: > > One major usecase is transplanting a page comming from postgres' > >buffers into the kernel's buffercache because the latter has a much > >better chance of properly allocating system resources across independent > >applications running. > >If you wa

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 3:04 PM, Jeff Janes wrote: I think the above is pretty simple for both interaction (allow us to inject a clean page into the file page cache) and policy (forget it after you hand it to us, then remember it again when we hand it back to you clean). And I think it would pretty like

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 4:47 PM, Jan Kara wrote: Note to postgres guys: I think you should have a look at the proposed 'vrange' system call. The latest posting is here: http://www.spinics.net/lists/linux-mm/msg67328.html. It contains a rather detailed description of the feature. And if the feature looks good

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 6:36 PM, Florian Pflug wrote: On Jan14, 2014, at 01:20 , Jim Nasby wrote: >On 1/13/14, 5:57 PM, Josh Berkus wrote: >>On 01/13/2014 03:41 PM, Florian Pflug wrote: >>>It therefor isn't an oversight that SELECT ... INTO allows multiple result rows >

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 7:06 PM, Josh Berkus wrote: On 01/13/2014 04:20 PM, Jim Nasby wrote: On 1/13/14, 5:57 PM, Josh Berkus wrote: I *really* don't want to go through all my old code to find places where I used SELECT ... INTO just to pop off the first row, and ignored the rest. I doubt anyone

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Jim Nasby
On 1/13/14, 7:10 PM, Kevin Grittner wrote: Tom Lane wrote: >I think the argument really is that some people don't want to >make their application code work with such cases (which is fine) >so they'd like an inside-the-database guarantee that the app code >won't ever see such cases. Which is l

Re: [HACKERS] Increase pltcl test coverage

2017-01-09 Thread Jim Nasby
that's very possibly the only animals running 8.4... I'm compiling 8.4 now, will see if I can duplicate. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TR

Re: [HACKERS] Increase pltcl test coverage

2017-01-09 Thread Jim Nasby
On 1/9/17 3:12 PM, Jim Nasby wrote: I'm compiling 8.4 now, will see if I can duplicate. Got a stack trace. The abort happens in TclObjLookupVar: if (nsPtr->varResProc != NULL || iPtr->resolverPtr != NULL) { nsPtr itself is NULL. * thread #1: tid = 0x, 0x0

Re: [HACKERS] Increase pltcl test coverage

2017-01-09 Thread Jim Nasby
On 1/9/17 4:23 PM, Tom Lane wrote: Jim Nasby writes: Got a stack trace. The abort happens in TclObjLookupVar: Yeah, I found the problem: pltcl_returnnext calls pltcl_build_tuple_result which may throw elog(ERROR), leaving the Tcl interpreter's state all screwed up, so that later functio

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-09 Thread Jim Nasby
Solves the most common use case and is backwards compatible. That won't allow you to use a variable in multiple places though... is there a reason we couldn't support something like IS DEFINED and UNSET? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics,

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-09 Thread Jim Nasby
stuck supporting the old version for a LONG time. A big part of why standard_conforming_strings was so ugly is users didn't have enough time to adjust. If we'd had that enabled by default for 4-5 releases it wouldn't have been nearly as much of an issue. -- Jim Nasby, Data Ar

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-09 Thread Jim Nasby
ave 2 explicit namespaces: the top one being auto variables and the one below that being function arguments. The namespace below that would be the top-most *user* block. Both of the pre-defined namespaces need the ability to change their name; I don't see any issue with using PRAGMA for that. --

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-09 Thread Jim Nasby
(though arguably you might not need to be able to un-assign...). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-

Re: [HACKERS] Increase pltcl test coverage

2017-01-09 Thread Jim Nasby
On 1/9/17 5:38 PM, Tom Lane wrote: Jim Nasby writes: Hmm... I suspect there's more places where this could be a problem. For example, pltcl_quote calls palloc, which could ereport as well. Yeah. I looked at that but couldn't get terribly excited about it, because AFAICS, Tcl in

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-09 Thread Jim Nasby
today. Being able to check the existence of a variable is a very common idiom in other languages, so I'm don't see why plpgsql shouldn't have it. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] RustgreSQL

2017-01-09 Thread Jim Nasby
nced pltcl is one possible solution for that problem. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing l

Re: [HACKERS] RustgreSQL

2017-01-09 Thread Jim Nasby
anguage other than C, and could reasonably be ported. Especially if that could be done in such a way that the net result is still C code so we're not adding dependencies to non developers (similar to bison). Extensions are a step in that direction, but they're ultimately not core Pos

Re: [HACKERS] pg_background contrib module proposal

2017-01-09 Thread Jim Nasby
On 1/9/17 7:22 AM, amul sul wrote: On Sun, Jan 8, 2017 at 8:50 AM, Jim Nasby wrote: [skipped...] Oh, hmm. So I guess if you do that when the background process is idle it's the same as a close? I think we need some way to safeguard against accidental forkbombs for cases where users a

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-12 Thread Jim Nasby
n extension up there. But that's certainly not the case. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers ma

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-12 Thread Jim Nasby
compatibility with plsql, as well as pulling PSM into core. The former would be to help migrating from Oracle; the latter would be to provide everyone a cleaner built-in PL. (IMHO a PLSQL equivalent could certainly be an external extension). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX

Re: [HACKERS] WARM and indirect indexes

2017-01-12 Thread Jim Nasby
ID changes. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] remove floats from bootstrap scanner/parser

2017-01-12 Thread Jim Nasby
would just rip it out, per the attached. Verified this works for make check. Looks sane to me. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)

Re: [HACKERS] Off-by-one oddity in minval for decreasing sequences

2017-01-12 Thread Jim Nasby
losing the hole" that you can end up with now. I agree it makes sense to sen the minimum value correctly. Not sure if this necessitates changes in pg_upgrade... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Troub

Re: [HACKERS] GSoC 2017

2017-01-12 Thread Jim Nasby
On 1/10/17 1:53 AM, Alexander Korotkov wrote: 1. What project ideas we have? Perhaps allowing SQL-only extensions without requiring filesystem files would be a good project. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL

Re: [HACKERS] plpgsql - additional extra checks

2017-01-12 Thread Jim Nasby
. Shouldn't this also apply to var := blah FROM some_table WHERE ...; ? AIUI that's one of the beefs the plpgsql2 project has. FWIW, I'd also be in favor of a NOMULTI option to INTO, but I don't see any way to do something like that with var := blah FROM. -- Jim Nasby,

[HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-01-15 Thread Jim Nasby
com/git-in-2016-fad96ae22a15?imm_mid=0ec3e0&cmp=em-prog-na-na-newsltr_20170114#.shgj609ad -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sen

Re: [HACKERS] Packages: Again

2017-01-16 Thread Jim Nasby
to mark those namespaces (and possibly other objects) as private. - A way to reference extensions from other extensions and deal with extensions being moved to a different schema (or namespace). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architect

Re: [HACKERS] PSQL commands: \quit_if, \quit_unless

2017-01-16 Thread Jim Nasby
l abuse it... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] GSoC 2017

2017-01-16 Thread Jim Nasby
On 1/13/17 4:08 PM, Alvaro Herrera wrote: Jim Nasby wrote: On 1/10/17 1:53 AM, Alexander Korotkov wrote: 1. What project ideas we have? Perhaps allowing SQL-only extensions without requiring filesystem files would be a good project. Don't we already have that in patch form? Di

Re: [HACKERS] GSoC 2017

2017-01-16 Thread Jim Nasby
t before Postgres threw in the towel and said a cast was impossible. 1: https://github.com/BlueTreble/variant/ -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE

Re: Updating MATERIALIZED VIEWs (Re: [HACKERS] delta relations in AFTER triggers)

2017-01-21 Thread Jim Nasby
for the extension to track arbitrary database objects that were created. 1: https://github.com/decibel/pg_classy/blob/master/doc/pg_classy.asc -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Jim Nasby
4 pgbench -rT 300 No checksumschecksums 818 tps 758 tps 821 tps 877 tps 879 tps 799 tps 739 tps 808 tps 867 tps 845 tps 854 tps 831 tps Looking at per-statement latency, the variation is always in the update to branches. I'll try to get some sequent

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-21 Thread Jim Nasby
ecessitate another GUC, but it seems a lot simpler than most of the other ideas. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) --

Re: [HACKERS] Too many autovacuum workers spawned during forced auto-vacuum

2017-01-22 Thread Jim Nasby
r simply is honor adl_next_worker in the logic that looks for freeze, something like the attached. On another note, does anyone else find the database selection logic rather difficult to trace through? The logic is kinda spread throughout several functions. The naming of rebuild_database_list()

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due to user activity? It varies depending on the particular syscache, but in at

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/22/17 4:41 PM, Jim Nasby wrote: On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due to user activity? It varies depending

Re: [HACKERS] Online enabling of page level checksums

2017-01-22 Thread Jim Nasby
that'd be fine. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-01-22 Thread Jim Nasby
's not THAT hard to do that by hand, but it'd be great if there was a more automated method. Such a method might also be very useful for transforming expressions like date_part('quarter', ...) into something that could use existing indexes. -- Jim Nasby, Data Architect, Bl

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-23 Thread Jim Nasby
ally help could reduce some of those concerns. Or perhaps the original complaint about STATRELATTINH could be solved by just disabling negative entries on that cache. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in T

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-01-23 Thread Jim Nasby
expr it should probably be removed as an option to exec_*. finit_ would be better named free_. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-23 Thread Jim Nasby
On 1/21/17 6:42 PM, Jim Nasby wrote: On 12/26/16 2:31 AM, Kyotaro HORIGUCHI wrote: The points of discussion are the following, I think. 1. The first patch seems working well. It costs the time to scan the whole of a catcache that have negative entries for other reloids. However, such

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-23 Thread Jim Nasby
I took a look at this again, and it doesn't appear to be working for me. The library is being loaded during startup, but I don't see any further activity in the log, and I don't see an autoprewarm file in $PGDATA. There needs to be some kind of documentation change as part of this patch. I'm no

Re: [HACKERS] Contrib: alternative MATERIALIZED VIEWs

2017-01-23 Thread Jim Nasby
n has been working on. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] GSoC 2017

2017-01-23 Thread Jim Nasby
able that stored the contents of TZDATA any time it changed, as well as a fast way to find the surrogate key for the current TZDATA. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http:

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
gnificantly increase the rate of buffers being written out. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
might provide some data, though the horse is already well out of the barn by then. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent

Re: [HACKERS] [PATCH] Generic type subscription

2017-01-23 Thread Jim Nasby
ing and haven't gotten around to replacing. Since there's a static variable that gets set to the relevant OID it's not that bad performance-wise from what I can tell, but I suspect that's not something we want to be recommending to others... -- Jim Nasby, Data Architect, Blue Tr

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Jim Nasby
right now I think it's OK to force users that are enabling this to manually connect to datallowcon=false and run vacuum. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://Blu

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
On 1/23/17 6:55 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: As others have mentioned, right now practically no one enables this, so we've got zero data on how useful it might actually be. Uhm, Peter G just said that Heroku enables this on all their databases and

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
blems. In any case, how can we go about collecting data that checksums help? We certainly know people suffer data corruption. We can only guess at how many of those incidents would be caught by checksums. I don't see how we can get data on that unless we get a lot more users running

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
e exposed it presumably wouldn't be difficult to do that in an extension, as a bgworker. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
On 1/23/17 8:24 PM, Tom Lane wrote: Jim Nasby writes: On 1/23/17 7:47 PM, Stephen Frost wrote: It might be interesting to consider checking them in 'clean' pages in shared_buffers in a background process, as that, presumably, *would* detect shared buffers corruption. Hmm... tha

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Jim Nasby
I think it's entirely reasonable *from a technical standpoint* to enable by default in 10, with only the ability to dynamically disable. Given the concerns that keep popping up about dynamically enabling, I'm not at all sure that we could get that into 10. -- Jim Nasby, Data Architect, B

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-01-23 Thread Jim Nasby
On 1/5/17 9:50 PM, Jim Nasby wrote: The * on that is there's something odd going on where plpython starts out really fast at this, then gets 100% slower. I've reached out to some python folks about that. Even so, the overall results from a quick test on my laptop are (IMHO)

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-01-24 Thread Jim Nasby
use of this for similar reasons. I'll post a plpython patch that doesn't add the output format control. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TRE

Re: [HACKERS] Checksums by default?

2017-01-24 Thread Jim Nasby
x27;m not completely grokking your second paragraph, but I would think that an average user would love got get a heads-up that their hardware is failing. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-24 Thread Jim Nasby
find terminology other than "buffer dump", because that makes it sound like we're dumping the contents of the buffers themselves. Maybe block_map? Buffer_map? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Dat

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-24 Thread Jim Nasby
25.012 IST [91755] LOG: Buffer Dump: saved metadata of 59 blocks. Yeah, I wasn't getting that at all, though I did see the shared library being loaded. If I get a chance I'll try it again. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Archite

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-25 Thread Jim Nasby
On 1/24/17 11:13 PM, Beena Emerson wrote: On Wed, Jan 25, 2017 at 10:36 AM, Jim Nasby mailto:jim.na...@bluetreble.com>> wrote: On 1/24/17 2:26 AM, Mithun Cy wrote: Thanks for looking into this patch, I just downloaded the patch and applied same to the latest code,

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-25 Thread Jim Nasby
On 1/25/17 1:46 PM, Jim Nasby wrote: Based on that and other feedback I'm going to mark this as returned with feedback, though if you're able to get a revised patch in the next few days please do. Actually, based on the message that popped up when I went to do that I guess it'

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-01-25 Thread Jim Nasby
at I'd like is the ability to set a GUC in a plpgsql block *and have the setting revert on block exit*. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBL

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-27 Thread Jim Nasby
t focusing on the issues that have already been identified before trying to add more features. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532)

Re: [HACKERS] GSoC 2017

2017-01-27 Thread Jim Nasby
having a timestamp type that stores the original timezone. BTW, time itself fits in the multi-unit pattern, since months don't have a fixed conversion to days (and technically seconds don't have a fixed conversion to anything thanks to leap seconds). -- Jim Nasby, Data Architect, B

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-29 Thread Jim Nasby
8 ms Execution time: 48.917 ms (5 rows) In any case, +1 for not promoting count(*) <> 0; that's a really, really bad way to test for existence. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get

Re: [HACKERS] Superowners

2017-01-29 Thread Jim Nasby
they wanted to grant specific roles the ability to read everything in the database (or maybe cluster; I don't think the conversation got into that level of detail). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-01-31 Thread Jim Nasby
the limit to INT_MAX / 2, which is the same as shared_buffers. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-25

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-02-01 Thread Jim Nasby
On 1/27/17 4:14 AM, Greg Stark wrote: On 25 January 2017 at 20:06, Jim Nasby wrote: GUCs support SET LOCAL, but that's not the same as local scoping because the setting stays in effect unless the substrans aborts. What I'd like is the ability to set a GUC in a plpgsql block *an

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-01 Thread Jim Nasby
27;s similar to what happens when you miss a quote or a semicolon. We handle those cases with %R, and I think %R needs to support if as well. Perhaps there's value to providing more info (active branch, etc), but ISTM trying to do that will just confuse the original (%R) case. -- Jim N

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 10:27 AM, Robert Haas wrote: On Tue, Jan 31, 2017 at 5:07 PM, Jim Nasby wrote: On 11/29/16 9:58 AM, Jeff Janes wrote: Considering a single SSD can do 70% of that limit, I would say yes. Next question becomes... should there even be an upper limit? Where the

Re: [HACKERS] Cast jsonb to numeric, int, float, bool

2017-02-01 Thread Jim Nasby
rator I'm not sure what your intent here is, but if the idea is that a json array would magically cast to a bool or a number data type I think that's a bad idea. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 3:36 PM, Michael Paquier wrote: On Thu, Feb 2, 2017 at 7:01 AM, Jim Nasby wrote: On 2/1/17 10:27 AM, Robert Haas wrote: This looks fine to me. This could go without the comments, they are likely going to be forgotten if any updates happen in the future. I'm confused... I pu

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 4:27 PM, Andres Freund wrote: On 2017-02-02 09:22:46 +0900, Michael Paquier wrote: On Thu, Feb 2, 2017 at 9:17 AM, Jim Nasby wrote: Speaking of which... I have a meeting in 15 minutes to discuss moving to a server with 4TB of memory. With current limits shared buffers maxes at 16TB

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-02 Thread Jim Nasby
On 2/1/17 4:28 PM, Andres Freund wrote: On 2016-11-28 11:40:53 -0800, Jim Nasby wrote: With current limits, the most bgwriter can do (with 8k pages) is 1000 pages * 100 times/sec = 780MB/s. It's not hard to exceed that with modern hardware. Should we increase the limit on bgwriter_lru_max

Re: [HACKERS] Checksums by default?

2017-02-03 Thread Jim Nasby
't it be fairly trivial to write an extension that did that though? foreach r in pg_class where relkind in (...) for (b = 0; b < r.relpages; b++) ReadBufferExtended(..., BAS_BULKREAD); -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-03 Thread Jim Nasby
currently ~4x larger than the cluster, yet there's a non-trivial amount of buffers being written by backends. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com

[HACKERS] PinBuffer() no longer makes use of strategy

2017-02-03 Thread Jim Nasby
Commit 48354581a49c30f5757c203415aa8412d85b0f70 (Allow Pin/UnpinBuffer to operate in a lockfree manner) removed the code in PinBuffer that conditionally incremented usage_count when a ring buffer was in use. Was that intentional? ISTM the old behavior should have been retained. -- Jim Nasby

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-03 Thread Jim Nasby
On 2/2/17 4:39 PM, Corey Huinker wrote: On Wed, Feb 1, 2017 at 4:58 PM, Jim Nasby mailto:jim.na...@bluetreble.com>> wrote: I think the issue here is that the original case for this is a user accidentally getting into an \if and then having no clue what's going on. That&#

Re: [HACKERS] libpq Alternate Row Processor

2017-02-03 Thread Jim Nasby
n for a formal patch submission. I just did essentially the same thing for SPI (use a callback to allow the caller to handle the tuple instead of shoving it into a tuplestore). A simple test in plpython showed a 460% improvement. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Expert

Re: [HACKERS] Checksums by default?

2017-02-03 Thread Jim Nasby
owledge of how many forks you might have. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-02-03 Thread Jim Nasby
On 2/3/17 6:39 PM, Andres Freund wrote: Hi, On 2017-02-03 18:32:03 -0600, Jim Nasby wrote: Commit 48354581a49c30f5757c203415aa8412d85b0f70 (Allow Pin/UnpinBuffer to operate in a lockfree manner) removed the code in PinBuffer that conditionally incremented usage_count when a ring buffer was in

<    8   9   10   11   12   13   14   15   16   17   >