Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config. I thought we'd more or less rej

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/11/15 6:25 PM, Jim Nasby wrote: On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config

Re: [HACKERS] Remove array_nulls?

2015-12-14 Thread Jim Nasby
On 12/11/15 2:57 PM, Tom Lane wrote: Jim Nasby writes: A quick doc search indicates this config was created in 9.0, though the docs state it's for a change that happened in 8.2[1]. Don't know what you're looking at, but the GUC is definitely there (and documented) in 8.2.

Re: [HACKERS] Proposal: custom compression methods

2015-12-14 Thread Jim Nasby
TOAST then maybe that's enough. The other thing this might buy us are a few bits that could be used to support Datum versioning for other purposes, such as when the binary format of something changes. I would think that at some point we'll need that for pg_upgrade. -- Jim Nasby, Data Ar

[HACKERS] Clarify vacuum verbose message

2015-12-15 Thread Jim Nasby
been a large amount of deletes; I'll check with them tomorrow. IMHO we need to change the messages so they are explicit about line pointers vs actual tuples. Trying to obfuscate that just leads to confusion. heap_page_prune needs to report only non-rootlp tuples that were pruned. (None

Re: [HACKERS] 9.5RC1 wraps *today*

2015-12-15 Thread Jim Nasby
y, I think this is actually a good luck sign. ;P -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Logical replication and multimaster

2015-12-15 Thread Jim Nasby
y issues would be if done in the server. Maybe they wouldn't be that bad. I suspect the audience for this code would be much larger if it was in the server as opposed to a C library. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and Postg

Re: [HACKERS] Disabling an index temporarily

2015-12-15 Thread Jim Nasby
me set of queries you can test against and call it good. FWIW, I also don't see the use case for disabling maintenance on an index. Just drop it and if you know you'll want to recreate it squirrel away pg_get_indexdef() before you do. -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] Remove array_nulls?

2015-12-16 Thread Jim Nasby
On 12/16/15 6:01 PM, Robert Haas wrote: On Tue, Dec 15, 2015 at 1:26 AM, Michael Paquier wrote: On Tue, Dec 15, 2015 at 2:57 AM, Jim Nasby wrote: On 12/11/15 2:57 PM, Tom Lane wrote: Jim Nasby writes: Perhaps, but I'd like to have a less ad-hoc process about it. What's our

Re: [HACKERS] Proposal: custom compression methods

2015-12-16 Thread Jim Nasby
was a version that would expand 4 byte varlena to 8 byte as needed. And we're not painting ourselves in the corner - if we decide to increase the varlena header size in the future, this patch does not make it any more complicated. True. -- Jim Nasby, Data Architect, Blue Treble Con

Re: [HACKERS] Disabling an index temporarily

2015-12-16 Thread Jim Nasby
g. (Doing a single bulk insert to the index at the end of an INSERT should be safe though because none of those tuples are visible yet, though I'd have to make sure your backend didn't try to use the index for anything while the command was running... like as part of a trigger

Re: [HACKERS] Remove array_nulls?

2015-12-18 Thread Jim Nasby
re than just one GUC, and I agree with that, and I'm willing to investigate when the current compat GUCs went in and create a patch to remove the really old ones. My inclination would be to just do this as part of 10.0. (And I agree with Robert's comments about parallel being the most

Re: [HACKERS] plpgsql - DECLARE - cannot to use %TYPE or %ROWTYPE for composite types

2015-12-20 Thread Jim Nasby
it into a commitfest? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Weighted Stats

2015-12-20 Thread Jim Nasby
On 11/2/15 5:46 PM, David Fetter wrote: I'd like to add weighted statistics to PostgreSQL Anything happen with this? If community isn't interested, ISTM it'd be good to put this in PGXN. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Da

Re: [HACKERS] Optimizing away second VACUUM heap scan when only BRIN indexes on table

2015-12-20 Thread Jim Nasby
ot; of BRIN index(es) occurs, avoiding a call to lazy_vacuum_heap(), just as when there are no indexes on the table whatsoever? ISTM the big question here is how vacuum would know it can skip this since we wouldn't want to hard-code this for BRIN. -- Jim Nasby, Data Architect, Blue Tre

Re: [HACKERS] psql - -dry-run option

2015-12-20 Thread Jim Nasby
happens to contain a COMMIT, you're hosed. I can see some use for a "must rollback" mode of BEGIN. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent

Re: [HACKERS] Possible marginally-incompatible change to array subscripting

2015-12-22 Thread Jim Nasby
. which I suspect would make any such "type" all but unusable. The other problem would be having it deal with any other data type, but at least there's ways you can work around that for the most part. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics

Re: [HACKERS] Experimental evaluation of PostgreSQL's query optimizer

2015-12-23 Thread Jim Nasby
d) be automated though, which is computing these statistics for all foreign keys. We can have a way to disable that for specific keys if necessary, but I'd bet it's extremely rare to have a FK that you never join on. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Exp

Re: [HACKERS] Experimental evaluation of PostgreSQL's query optimizer

2015-12-23 Thread Jim Nasby
estimate as well as the estimated upper and lower error margin for the estimate. An estimate of 827 +0 -400 could have very different meaning than an estimate of [427,827]. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] tracking owner of extension-managed objects

2015-12-23 Thread Jim Nasby
reg* casts last I looked. I was planning on just making a best possible attempt and solving this in an extension via a combination of event triggers, reg* and other voodoo, but being able to insert things directly into pg_(sh)depend or equivalent tables would be a lot more robust. -- Jim N

Re: [HACKERS] [POC] FETCH limited by bytes.

2015-12-23 Thread Jim Nasby
ssing something; can you elaborate? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] On columnar storage (2)

2015-12-29 Thread Jim Nasby
iration, it might be useful to prototype a columnstore using Series (or maybe ndarrays). [1] http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html [2] http://pandas.pydata.org/pandas-docs/stable/api.html#series [3] http://docs.scipy.org/doc/numpy-1.10.0/reference/int

Re: [HACKERS] tracking owner of extension-managed objects

2015-12-31 Thread Jim Nasby
On 12/31/15 3:49 PM, Chapman Flack wrote: On 12/23/15 15:02, Jim Nasby wrote: >BTW, I've been pondering a very similar problem to this. I'm working on a >metacoding framework, and it's inevitable that at some point it will want to >know what objects it's created.

[HACKERS] Improved error reporting in format()

2016-01-02 Thread Jim Nasby
of the message: select format( '% moo'); ERROR: unrecognized format() type specifier " " HINT: For a single "%" use "%%" I also made the use of "format()" consistent in all the other error messages. -- Jim Nasby, Data Architect, Blue Treble Consult

Re: [HACKERS] Improved error reporting in format()

2016-01-02 Thread Jim Nasby
On 1/2/16 5:57 PM, Jim Nasby wrote: Attached patch clarifies that %-related error messages with hints as well as (IMHO) improving the clarity of the message: Sorry, forgot to update regression tests. New patch attached. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in

Re: [HACKERS] count_nulls(VARIADIC "any")

2016-01-03 Thread Jim Nasby
ignificantly reduce the size of the test. Not a huge deal though... Also, I don't think anything is testing multiples of whatever value... how 'bout change the generate_series CASE statement to >40 instead of <>40? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX

[HACKERS] Very confusing installcheck behavior with PGXS

2016-01-03 Thread Jim Nasby
/i/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --psqldir=/Users/decibel/pgsql/9.4/i/bin REGRESS_OPTS = --inputdir=test --load-language=plpgsql --dbname=contrib_regression REGRESS = all build -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Ana

[HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
e*: select 'with spaces'::regclass; ERROR: invalid name syntax LINE 1: select 'with spaces'::regclass; select '"with spaces"'::regclass; regclass --- "with spaces" (1 row) I think this needs to be fixed before 9.5 releases. :(

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
On 1/3/16 9:23 PM, Tom Lane wrote: Jim Nasby writes: regrole and regnamespace don't run their output through quote_ident(). That's contrary to all the other reg* operators. Worse, they also don't *allow* quoted input. Not only is that different from reg*, it's the *opposit

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
On 1/3/16 9:43 PM, Tom Lane wrote: Jim Nasby writes: On 1/3/16 9:23 PM, Tom Lane wrote: Another potential problem for regnamespace is that it doesn't allow an entry for the catalog. I'm not sure what the spec says about that, but every other function allows dbname.schema.bl

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
On 1/3/16 10:20 PM, Jim Nasby wrote: What I went with. Now to figure out why this is happening... Nevermind, see my stupidity now. Should have full patch soon. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
On 1/3/16 9:23 PM, Tom Lane wrote: (Though at least in HEAD we ought to fix them to take type text as input. Using cstring for ordinary functions is just sloppy.) BTW, *all* the reg*in() functions do that... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
copied the same syntax used elsewhere... whoever commits feel free to editorialize... A couple of tests in regproc.sql would be a good addition as well. Added. I'm gonna call this good for now. Note this is just against HEAD since I don't have 9.5 setup yet. Presumably the pat

Re: [HACKERS] count_nulls(VARIADIC "any")

2016-01-03 Thread Jim Nasby
On 1/3/16 10:23 PM, Pavel Stehule wrote: Hi 2016-01-03 22:49 GMT+01:00 Jim Nasby mailto:jim.na...@bluetreble.com>>: On 1/3/16 2:37 PM, Pavel Stehule wrote: + /* num_nulls(VARIADIC NULL) is defined as NULL */ + if (PG_ARGIS

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
On 1/3/16 10:46 PM, Jim Nasby wrote: Added. I'm gonna call this good for now. Note this is just against HEAD since I don't have 9.5 setup yet. Presumably the patch should still apply... BTW, in case it's helpful... https://github.com/decibel/postgres/tree/regquote --

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
tch. BTW, in case you miss it... I was inconsistent with the list length_names checks... one is if (list_length(names) > 1) the other is if (list_length(names) != 1) (stringToQualifiedNameList() can't actually return a 0 length list and IIRC there was another place doing a > che

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-03 Thread Jim Nasby
looks good to me. FWIW, RhodiumToad and macdice looked at my patch as well and didn't see any problems you didn't mention. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://Blue

Re: [HACKERS] Accessing non catalog table in backend

2016-01-04 Thread Jim Nasby
but I think unless you have a really good reason to go that route you're much better off just using SPI. If it's good enough for plpgsql... :) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it

Re: [HACKERS] Accessing non catalog table in backend

2016-01-04 Thread Jim Nasby
On 1/4/16 12:53 PM, Atri Sharma wrote: Please don't top-post. On 5 Jan 2016 12:20 am, "Jim Nasby" mailto:jim.na...@bluetreble.com>> wrote: On 1/4/16 12:07 PM, Atri Sharma wrote: Hi All, I wanted to check if it is possible to query a non catalog tab

Re: [HACKERS] 9.5 BLOCKER: regrole and regnamespace and quotes

2016-01-04 Thread Jim Nasby
item for a new hacker to handle, so I'm going to figure out what we're doing with those things now-a-days and put it there. If no one picks it up I'll get it into the last commitfest. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Archi

[HACKERS] Beginner hacker item: Fix to_reg*() input type

2016-01-04 Thread Jim Nasby
tions that need to change. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-05 Thread Jim Nasby
message-id/568ada20.7090...@bluetreble.com [2] http://www.postgresql.org/message-id/568c6e6d.1040...@bluetreble.com -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-05 Thread Jim Nasby
On 1/5/16 8:19 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: Which doesn't help anyone, because neither of those provide a list of "hey, here's stuff you could do to contribute". The closest we come to that is the TODO, which isn't well known an

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-01-05 Thread Jim Nasby
fo is someone sitting at psql or pgAdmin. Maybe schema info could be presented in HINT or DETAIL messages as well? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sen

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-01-05 Thread Jim Nasby
On 1/5/16 8:41 PM, Tom Lane wrote: Jim Nasby writes: >does psql do anything with those fields? ISTM the biggest use for this >info is someone sitting at psql or pgAdmin. Sure, if you turn up the error verbosity. FWIW, I suspect very few people know about the verbosity setting (I

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-01-05 Thread Jim Nasby
On 1/5/16 9:16 PM, Tom Lane wrote: Jim Nasby writes: FWIW, I suspect very few people know about the verbosity setting (I didn't until a few months ago...) Maybe psql should hint about it the first time an error is reported in a session. Actually, what'd be really handy IMO is so

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Jim Nasby
ey wouldn't have to do the gruntwork. If, say, the Ops teams at 2nd Quadrant, CMD, and EDB wanted to work together on improving infrastructure, that's pretty much community at that point, and not a dependence on a single external entity. -- Jim Nasby, Data Architect, Blue Trebl

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Jim Nasby
l activity to see where things are at. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Jim Nasby
On 1/6/16 6:18 PM, Greg Stark wrote: On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby wrote: Right. Personally, I feel the TODO has pretty much outlived it's usefulness. An issue tracker would make maintaining items like this a lot more reasonable, but it certainly wouldn't be free.

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Jim Nasby
On 1/6/16 9:22 PM, Michael Paquier wrote: On Thu, Jan 7, 2016 at 9:28 AM, Tom Lane wrote: Jim Nasby writes: Somewhat related to that, I don't believe there's any reason why commit fest managers need to be committers; it seems like the job is really just about reading through emai

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
On 1/6/16 11:54 AM, Tom Lane wrote: Robert Haas writes: On Sun, Jan 3, 2016 at 5:22 PM, Jim Nasby wrote: The rule that gets executed if you do `make installcheck` with something using PGXS is pgxs.mk:$(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) where

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
On 1/7/16 8:47 AM, Tom Lane wrote: Jim Nasby writes: However, if I do this: mv test/sql/acl_type.sql test/sql/acl.sql mv test/expected/acl_type.out test/expected/acl.out And change acl_type to acl in that pg_regress command: /Users/decibel/pgsql/HEAD/i/lib/pgxs/src/makefiles/../../src/test

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
On 1/7/16 9:12 AM, Tom Lane wrote: Jim Nasby writes: On 1/7/16 8:47 AM, Tom Lane wrote: That's pretty hard to believe. There's nothing in pg_regress that looks in places other than the given --inputdir. Actually, I think it does... from pg_regress_main.c: /*

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
On 1/7/16 9:56 AM, Tom Lane wrote: Jim Nasby writes: On 1/7/16 9:12 AM, Tom Lane wrote: (I'm also wondering how convert_sourcefiles() works at all in a vpath build, considering that I don't see it doing anything like this ...) It's only looking at outputdir, which I s

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
using this pattern (or something similar). The other common pattern I see is to just pile everything into the top level extension directory. That's OK at first (and for a really, really simple extension might be all you ever want), but if you start having a few tests, a doc file, and sever

Re: [HACKERS] Very confusing installcheck behavior with PGXS

2016-01-07 Thread Jim Nasby
On 1/7/16 1:04 PM, Alvaro Herrera wrote: Jim Nasby wrote: Also worth noting: the only reason I'm using pg_regress is it's the easiest way to get a test cluster. If not for that, I'd just use pg_prove since I'm already using pgTap. In 9.5 you might want to "use Postg

[HACKERS] Interesting read on SCM upending software and hardware architecture

2016-01-07 Thread Jim Nasby
these numbers in context, acquiring a single uncontested lock on today's systems takes approximately 20ns, while a non-blocking cache invalidation can cost up to 100ns, only 25x less than an I/O operation." -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Anal

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-07 Thread Jim Nasby
en if not auto-closed at least it'd be easy to find old items. Bonus points if we attract some volunteer project managers that will keep tabs of all those kinds of things... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQ

[HACKERS] BEGINNER HACKERS: array_remove(anyarray, anyarray)

2016-01-07 Thread Jim Nasby
On 12/10/15 6:25 PM, Jim Nasby wrote: Recently I had need of removing occurrences of a number of values from an array. Obviously I could have nested array_remove() call or wrapped the whole thing in a SELECT unnest(), but that seems rather silly and inefficient. Any one have objections to

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-08 Thread Jim Nasby
will be very helpful, but without a strong consensus in the community that it's desirable any efforts will probably fail. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://Blue

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-01-11 Thread Jim Nasby
On 1/11/16 12:33 PM, Pavel Stehule wrote: 1. break compatibility and SPIError replace by Error At this point I've lost track... what's the incompatibility between the two? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

2016-01-11 Thread Jim Nasby
;d be able to re-enter the process during recursive/nested calls. Obviously this is a lot more work than what you're proposing though. :( -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! h

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-01-11 Thread Jim Nasby
On 1/11/16 12:46 PM, Pavel Stehule wrote: 2016-01-11 19:41 GMT+01:00 Jim Nasby mailto:jim.na...@bluetreble.com>>: On 1/11/16 12:33 PM, Pavel Stehule wrote: 1. break compatibility and SPIError replace by Error At this point I've lost track... what's the incompa

Re: [HACKERS] Making plpython 2 and 3 coexist a bit better

2016-01-11 Thread Jim Nasby
e a strong preference either way. If they both get loaded is there risk of bad data happening? Personally, I'll take a traceable FATAL (or even PANIC) over data corruption every time. But I'm guessing that if you tried to use both you'd pretty immediately end up crashing the back

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 2:59 PM, Robert Haas wrote: > 2. Hint bits are necessary because an old XID can't be viewed as > guaranteed committed. Hmm... I thought hint bits were necessary because it's too expensive to query CLOG for every tuple. If my understanding is correct then if we fix the CLOG per

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 11:09 AM, Florian Pflug wrote: > An UPDATE on such a SHARE locked row would be allowed despite the lock if it > only changed columns not mentioned by any unique index. On a side-note, by "changed columns" do you mean the column appeared in the UPDATE statement, or the data act

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 8:07 AM, Yeb Havinga wrote: > FK's cannot refer to rows in inheritance childs. We have partially solved this issue at work. In our scenario, we're not using inheritance for partitioning, we're using it for, well, inheriting. As part of that, we have a field in the parent table

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 8:59 AM, Andrew Dunstan wrote: > On 12/01/2010 09:41 AM, Tom Lane wrote: >> ghatpa...@vsnl.net writes: >>> Create domain is only useful for abstracting common constraints on fields >>> into single location for maintenance. It may not be useful to link tables. >> It's still uncle

Re: [HACKERS] Re: Rethinking hint bits WAS: Protecting against unexpected zero-pages: proposal

2010-12-06 Thread Jim Nasby
On Nov 14, 2010, at 3:40 PM, Greg Stark wrote:On Sun, Nov 14, 2010 at 8:52 PM, Josh Berkus wrote:For example, imagine if the hint bits were moved to a separate per-tablebitmap outside the table instead of being stored with each row, as thecurrent FSM is.How many times do we have to keep going arou

Re: [HACKERS] BufFreelistLock

2010-12-09 Thread Jim Nasby
On Dec 8, 2010, at 11:44 PM, Jeff Janes wrote: >>> For the clock sweep algorithm, I think you could access >>> nextVictimBuffer without any type of locking. >> >> This is wrong, mainly because you wouldn't have any security against two >> processes decrementing the usage count of the same buffer b

Re: [HACKERS] would hw acceleration help postgres (databases in general) ?

2010-12-11 Thread Jim Nasby
On Dec 10, 2010, at 6:18 PM, Jeff Janes wrote: > On Fri, Dec 10, 2010 at 3:09 PM, Hamza Bin Sohail wrote: >> >> Hello hackers, >> >> I think i'm at the right place to ask this question. >> >> Based on your experience and the fact that you have written the Postgres >> code, >> can you tell what

Re: [HACKERS] proposal: auxiliary functions for record type

2010-12-11 Thread Jim Nasby
On Dec 11, 2010, at 9:24 AM, Florian Pflug wrote: > begin > for v_field in select * from fieldinfos(myrec) loop > case > when v_field.fieldtype = 'type1'::regtype then > v_value_type1 := fieldvalue(myrec, NULL::type1, > false)

Re: [HACKERS] BufFreelistLock

2010-12-12 Thread Jim Nasby
On Dec 10, 2010, at 10:49 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Jeff Janes's message of vie dic 10 12:24:34 -0300 2010: >>> As far as I can tell, bgwriter never adds things to the freelist. >>> That is only done at start up, and when a relation or a database is >>> dropped.

Re: [HACKERS] BufFreelistLock

2010-12-13 Thread Jim Nasby
On Dec 12, 2010, at 8:48 PM, Jim Nasby wrote: >> There might be some advantage in having it move buffers >> to a freelist that's just protected by a simple spinlock (or at least, >> a lock different from the one that protects the clock sweep). The >> idea would be th

Re: [HACKERS] BufFreelistLock

2010-12-14 Thread Jim Nasby
On Dec 14, 2010, at 11:08 AM, Jeff Janes wrote: > On Sun, Dec 12, 2010 at 6:48 PM, Jim Nasby wrote: >> >> BTW, when we moved from 96G to 192G servers I tried increasing shared >> buffers from 8G to 28G and performance went down enough to be noticeable (we >> don

Re: [HACKERS] Default mode for shutdown

2010-12-15 Thread Jim Nasby
On Dec 15, 2010, at 9:11 AM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of miƩ dic 15 12:03:06 -0300 2010: > >> Certainly, if you have an environment where people are mostly logging >> into the database directly (not through a connection pooler) and they >> do a few important quer

Re: [HACKERS] BufFreelistLock

2010-12-15 Thread Jim Nasby
On Dec 15, 2010, at 2:40 PM, Jeff Janes wrote: > On Tue, Dec 14, 2010 at 1:42 PM, Jim Nasby wrote: >> >> On Dec 14, 2010, at 11:08 AM, Jeff Janes wrote: >>> I wouldn't expect an increase in shared_buffers to make contention on >>> BufFreelistLock worse. If

Re: [HACKERS] Can postgres create a file with physically continuous blocks.

2010-12-19 Thread Jim Nasby
On Dec 19, 2010, at 1:10 AM, flyusa2010 fly wrote: > Does postgres make an effort to create a file with physically continuous > blocks? AFAIK all files are expanded as needed. I don't think there's any flags you can pass to the filesystem to tell it "this file will eventually be 1GB in size".

Re: [HACKERS] keeping a timestamp of the last stats reset (for a db, table and function)

2010-12-19 Thread Jim Nasby
On Dec 18, 2010, at 8:51 PM, Tom Lane wrote: > Tomas Vondra writes: >> I've done several small changes to the patch, namely > >> - added docs for the functions (in SGML) >> - added the same thing for background writer > >> So I think now it's 'complete' and I'll add it to the commit fest in a >>

Re: [HACKERS] Avoiding rewrite in ALTER TABLE ALTER TYPE

2010-12-30 Thread Jim Nasby
On Dec 29, 2010, at 10:14 PM, Robert Haas wrote: > +1 for trying to optimize these cases (but maybe after we optimize the > varchar -> text and varchar(less) -> varchar(more) cases to skip the > scan altogether). +1 on getting the obvious cases of varchar and numeric done first; we run into those

Re: [HACKERS] Sync Rep Design

2010-12-30 Thread Jim Nasby
On Dec 30, 2010, at 3:27 PM, Robert Haas wrote: >> synchronous_replication (boolean) >>Specifies whether transaction commit will wait for WAL records >>to be replicated before the command returns a "success" >>indication to the client. > > The word "replicated" here could b

Re: [HACKERS] estimating # of distinct values

2010-12-31 Thread Jim Nasby
On Dec 31, 2010, at 7:34 AM, Alvaro Herrera wrote: > Excerpts from Tom Lane's message of jue dic 30 23:02:04 -0300 2010: >> Alvaro Herrera writes: >>> I was thinking that we could have two different ANALYZE modes, one >>> "full" and one "incremental"; autovacuum could be modified to use one or >>>

Re: [HACKERS] contrib/snapshot

2011-01-02 Thread Jim Nasby
On Dec 31, 2010, at 1:35 PM, Joel Jacobson wrote: > 2010/12/31 Simon Riggs > Please call it something other than "snapshot". There's already about 3 > tools called something similar and a couple of different meanings of the > term in the world of Postgres. > > > Thanks, good point. > Renamed to

Re: [HACKERS] contrib/snapshot

2011-01-03 Thread Jim Nasby
On Jan 2, 2011, at 6:50 PM, Joel Jacobson wrote: > 2011/1/3 Joel Jacobson > 2011/1/2 Jim Nasby > Is it actually limited to functions? ISTM this concept would be valuable for > anything that's not in pg_class (in other words, anything that doesn't have > user dat

Re: [HACKERS] page compression

2011-01-03 Thread Jim Nasby
On Jan 2, 2011, at 5:36 PM, Simon Riggs wrote: > On Tue, 2010-12-28 at 09:10 -0600, Andy Colson wrote: > >> I know its been discussed before, and one big problem is license and >> patent problems. > > Would like to see a design for that. There's a few different ways we > might want to do that, a

Re: [HACKERS] crash-safe visibility map, take three

2011-01-07 Thread Jim Nasby
On Jan 5, 2011, at 8:10 PM, Robert Haas wrote: > On Wed, Jan 5, 2011 at 3:22 PM, Jesper Krogh wrote: >> Given a crash-safe visibility map, what purpuse does the PD_ALL_VISIBLE bit >> serve? > > If we modify a page on which PD_ALL_VISIBLE isn't set, we don't > attempt to update the visibility map.

Re: [HACKERS] estimating # of distinct values

2011-01-07 Thread Jim Nasby
On Jan 7, 2011, at 5:32 AM, t...@fuzzy.cz wrote: > Another thing I'm not sure about is where to store those intermediate > stats (used to get the current estimate, updated incrementally). I was > thinking about pg_stats but I'm not sure it's the right place - depending > on the algorithm, this may

Re: [HACKERS] obj_unique_identifier(oid)

2011-01-07 Thread Jim Nasby
On Jan 7, 2011, at 1:46 PM, Tom Lane wrote: > Joel Jacobson writes: >> The function obj_unique_identifier(oid) will return a unique name for _any_ >> oid. > > Surely this is broken by design? You can *not* assume that the same OID > isn't in use for different things in different system catalogs

Re: [HACKERS] estimating # of distinct values

2011-01-09 Thread Jim Nasby
> A resource fork? Not sure what you mean, could you describe it in more > detail? Ooops, resource forks are a filesystem thing; we call them relation forks. >From src/backend/storage/smgr/README: Relation Forks == Since 8.4, a single smgr relation can be comprised of multiple physi

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-17 Thread Jim Nasby
On Jan 17, 2011, at 9:22 AM, Noah Misch wrote: > On Mon, Jan 17, 2011 at 07:35:52AM +0100, Magnus Hagander wrote: >> On Mon, Jan 17, 2011 at 06:51, Itagaki Takahiro >> wrote: >>> On Mon, Jan 17, 2011 at 04:05, Andy Colson wrote: This is a review of: https://commitfest.postgresql.org/act

Re: [HACKERS] Spread checkpoint sync

2011-01-17 Thread Jim Nasby
On Jan 15, 2011, at 8:15 AM, Robert Haas wrote: > Well, the point of this is not to save time in the bgwriter - I'm not > surprised to hear that wasn't noticeable. The point is that when the > fsync request queue fills up, backends start performing an fsync *for > every block they write*, and that

Re: [HACKERS] estimating # of distinct values

2011-01-17 Thread Jim Nasby
On Jan 17, 2011, at 6:36 PM, Tomas Vondra wrote: > 1) Forks are 'per relation' but the distinct estimators are 'per > column' (or 'per group of columns') so I'm not sure whether the file > should contain all the estimators for the table, or if there should > be one fork for each estimator. Th

Re: [HACKERS] limiting hint bit I/O

2011-01-17 Thread Jim Nasby
On Jan 14, 2011, at 7:24 PM, Josh Berkus wrote: > On 1/14/11 11:51 AM, Tom Lane wrote: >> The people whose tables are mostly insert-only complain about it, but >> that's not the majority of our userbase IMO. We just happen to have a >> couple of particularly vocal ones, like Berkus. > > It might

[HACKERS] Confusing comment in TransactionIdIsInProgress

2011-01-17 Thread Jim Nasby
Shouldn't the comment read "If first time through"? /* * If not first time through, get workspace to remember main XIDs in. We * malloc it permanently to avoid repeated palloc/pfree overhead. */ if (xids == NULL) { ... xids = (Tran

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> a quick-and-dirty attempt to limit the amount of I/O caused by hint >> bits. I'm still very interested in knowing what people think about >> that. > > I found the elimination of the response-time spike promising. I > d

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote: > a few weeks back I hacked an experimental patch that removed the hint > bit action completely. the results were very premature and/or > incorrect, but my initial findings suggested that hint bits might not > be worth the cost from performance st

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: > On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >> - Forks are very possibly a more efficient way to deal with TOAST than >> having separate tables. There's a fair amount of overhead we pay for the >> current setu

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:24 AM, Robert Haas wrote: > On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >>> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>>> - Forks are very possibly a more efficient way to dea

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:32 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >>> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>>> - Forks are very possib

Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED

2011-01-18 Thread Jim Nasby
On Jan 14, 2011, at 5:15 AM, Simon Riggs wrote: > On Mon, 2010-12-13 at 17:15 +, Peter Geoghegan wrote: >> On 13 December 2010 16:08, Robert Haas wrote: >>> On Mon, Dec 13, 2010 at 10:49 AM, Simon Riggs wrote: 2. pg_validate_foreign_key('constraint name'); Returns immediately if FK

<    5   6   7   8   9   10   11   12   13   14   >