Re: pg_ctl failover Re: [HACKERS] Latches, signals, and waiting

2011-01-18 Thread Fujii Masao
On Thu, Jan 13, 2011 at 9:08 PM, Fujii Masao wrote: > I did s/failover/promote. Here is the updated patch. I rebased the patch to current git master. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center pg_ctl_promote_v3.patch Description: Binary

Re: [HACKERS] Add support for logging the current role

2011-01-18 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Point being, until we get Andrew's jagged-csv-import magic committed to > > core, we won't be able to import a log file that a user has changed the > > field list for mid-stream (following the add-a-new-column use-case we've > > be

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Fujii Masao
On Wed, Jan 19, 2011 at 4:12 AM, Magnus Hagander wrote: > Ah, ok. I've added the errorcode now, PFA. I also fixed an error in > the change for result codes I broke in the last patch. github branch > updated as usual. Great. Thanks for the quick update! Here are another comments: + * IDENTIFICAT

Re: [HACKERS] Snapshot synchronization, again...

2011-01-18 Thread Joachim Wieland
Noah, thank you for this excellent review. I have taken over most (allmost all) of your suggestions (except for the documentation which is still missing). Please check the updated & attached patch for details. On Fri, Jan 14, 2011 at 10:13 PM, Noah Misch wrote: > "" is a valid

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Fujii Masao
On Tue, Jan 18, 2011 at 8:40 PM, Magnus Hagander wrote: >> When I untar the tar file taken by pg_basebackup, I got the following >> messages: >> >>    $ tar xf base.tar >>    tar: Skipping to next header >>    tar: Archive contains obsolescent base-64 headers >>    tar: Error exit delayed from pre

Re: [HACKERS] pl/python refactoring

2011-01-18 Thread Alvaro Herrera
Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011: > #16: This is probably pointless because pgindent will reformat this. pgindent used to remove useless braces around single-statement blocks, but this behavior was removed years ago because it'd break formatting around PG

Re: [HACKERS] auto-sizing wal_buffers

2011-01-18 Thread Fujii Masao
On Tue, Jan 18, 2011 at 8:50 PM, Greg Smith wrote: >> Why is the minimum value 64kB only when wal_buffers is set to >> -1? This seems confusing for users. >> > > That's because the current default on older versions is 64kB.  Since the > automatic selection is going to be the new default, I hope, I

Re: [HACKERS] REVIEW: PL/Python validator function

2011-01-18 Thread Hitoshi Harada
2011/1/18 Jan Urbański : > On 17/01/11 09:26, Jan Urbański wrote: >> On 17/01/11 01:02, Hitoshi Harada wrote: >>> This is a review for the patch sent as >>> https://commitfest.postgresql.org/action/patch_view?id=456 >>> It includes adequate amount of test. I found regression test failure >>> in plp

Re: [HACKERS] log_hostname and pg_stat_activity

2011-01-18 Thread Alvaro Herrera
Excerpts from Steve Singer's message of mar ene 18 21:24:25 -0300 2011: > Coding Review > - > > As Alvaro pointed out BackendStatusShmemSize should be updated. > > To answer his question about why clientaddr works: clientaddr is a > SockAddr which is a structure not a pointer so th

Re: [HACKERS] pl/python refactoring

2011-01-18 Thread Hitoshi Harada
2011/1/19 Peter Eisentraut : > On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote: >> On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote: >> > Here they are. There are 16 patches in total. They amount to what's >> > currently in my refactor branch (and almost to what I've sent as the >> >

Re: [HACKERS] psql: Add \dL to show languages

2011-01-18 Thread Josh Kupershmidt
On Tue, Jan 18, 2011 at 1:35 PM, Andreas Karlsson wrote: > Hi Josh, > > Nope, I do not have any better ideas than "DO Blocks?". > > Everything looks good with the exception one bug now. > > \dL foo > * QUERY ** > SELECT l.lanname AS "Name", >       pg_catalog.pg_get_userbyid(l.lano

Re: [HACKERS] Extending opfamilies for GIN indexes

2011-01-18 Thread Tom Lane
I wrote: > So here's what I'm thinking: we could redefine a GIN opclass, per se, as > needing only compare() and extractValue() procs to be bound into it. > The other three procs, as well as the query operators, could be "loose" > in the containing opfamily. The index AM would choose which set of

Re: [HACKERS] log_hostname and pg_stat_activity

2011-01-18 Thread Steve Singer
Here is my review for this patch Submission Review -Patch applies cleanly -Patch does not include documentation changes. At a minimum: update the table that lists what pg_stat_activity and pg_stat_replication includes in monitoring.sgml but I propose more below. -No tests

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
-Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Tuesday, January 18, 2011 2:53 PM To: Simone Aiken Cc: Alvaro Herrera; pgsql-hackers Subject: Re: [HACKERS] ToDo List Item - System Table Index Clustering >Sure - my point is just that we usually have as a criteri

[HACKERS] Extending opfamilies for GIN indexes

2011-01-18 Thread Tom Lane
I just got annoyed by the fact that contrib/intarray has support for queries on GIN indexes on integer[] columns, but they only work if you use the intarray-provided opclass, not the core-provided GIN opclass for integer[] columns. In general, of course, two different GIN opclasses aren't compatib

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 3:46 PM, Tom Lane wrote: > The above is "using" the index, but only as a guide to where the rows > satisfying the partial-index predicate are --- note the lack of any > index condition in the indexscan node. That's because the query_int > query is not in fact compatible with t

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
I wrote: > No, I see no reason to think that has much to do with it. I'm wondering > if your table is itself a bit bloated ... Actually ... I notice you did not show EXPLAIN ANALYZE output for your tests. Now I'm wondering whether you tested the right thing at all. I got burnt that way too. Obs

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 3:08 PM, Tom Lane wrote: >> Shall I send you data with the other two columns?: > > No, I see no reason to think that has much to do with it. I'm wondering > if your table is itself a bit bloated ... Nope. Just ran the bloat query from check_postgres.pl. Bloat is 0. Not surp

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
A.M. wrote: > > On Jan 18, 2011, at 5:41 PM, Bruce Momjian wrote: > > > A.M. wrote: > Because the fastest option may not be syncing to disk. For example, > the only option that makes sense on OS X is fsync_writethrough- it > would be helpful if the tool pointed that out (on OS X on

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 5:41 PM, Bruce Momjian wrote: > A.M. wrote: Because the fastest option may not be syncing to disk. For example, the only option that makes sense on OS X is fsync_writethrough- it would be helpful if the tool pointed that out (on OS X only, obviously). >>> >>> Y

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Bruce Momjian writes: > >>> I have modified test_fsync to use test labels that match wal_sync_method > >>> values, and and added more tests for open_sync with different sizes. > > >> Given that it was unclear whether the first suc

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > On Jan 18, 2011, at 1:58 PM, Tom Lane wrote: >> I'm noticing also that I get different rowcounts than you do, although >> possibly that has something to do with the partial-index conditions, >> which I'm not trying to duplicate here (all rows in my table pass those >>

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Bruce Momjian writes: >>> I have modified test_fsync to use test labels that match wal_sync_method >>> values, and and added more tests for open_sync with different sizes. >> Given that it was unclear whether the first such test was of any value, >> wh

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 1:58 PM, Tom Lane wrote: > I'm noticing also that I get different rowcounts than you do, although > possibly that has something to do with the partial-index conditions, > which I'm not trying to duplicate here (all rows in my table pass those > two tests). Shall I send you dat

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > I have modified test_fsync to use test labels that match wal_sync_method > > values, and and added more tests for open_sync with different sizes. > > Given that it was unclear whether the first such test was of any value, > why are you slowing down the

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Tom Lane
Bruce Momjian writes: > I have modified test_fsync to use test labels that match wal_sync_method > values, and and added more tests for open_sync with different sizes. Given that it was unclear whether the first such test was of any value, why are you slowing down the program by adding more?

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
A.M. wrote: > >> Because the fastest option may not be syncing to disk. For example, > >> the only option that makes sense on OS X is fsync_writethrough- it > >> would be helpful if the tool pointed that out (on OS X only, obviously). > > > > Yes, that would be a serious problem. :-( > > > > I a

Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED

2011-01-18 Thread Simon Riggs
On Tue, 2011-01-18 at 14:26 -0600, Jim Nasby wrote: > > > > 2 sub-command changes: > > > > ALTER TABLE foo ADD FOREIGN KEY fkoo ... NOT VALID; > > > > ALTER TABLE foo VALIDATE CONSTRAINT fkoo; > > Sorry for the late reply; I just saw this... > > Is there any way to be able to get the bad recor

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 5:21 PM, Bruce Momjian wrote: > A.M. wrote: >> >> On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: >> >>> A.M. wrote: On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > I have modified test_fsync to use test labels that match wal_sync_method > valu

Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED

2011-01-18 Thread Jim Nasby
On Jan 14, 2011, at 5:15 AM, Simon Riggs wrote: > On Mon, 2010-12-13 at 17:15 +, Peter Geoghegan wrote: >> On 13 December 2010 16:08, Robert Haas wrote: >>> On Mon, Dec 13, 2010 at 10:49 AM, Simon Riggs wrote: 2. pg_validate_foreign_key('constraint name'); Returns immediately if FK

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane : > Pavel Stehule writes: >> 2011/1/18 Tom Lane : >>> I looked at this patch and found it fairly awkward.  What is the point >>> of adding an additional flag to every variable, as opposed to just >>> forcibly detoasting during assignment? > >> But detoasting on assignment isn't

Re: [HACKERS] pl/python refactoring

2011-01-18 Thread Peter Eisentraut
On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote: > On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote: > > Here they are. There are 16 patches in total. They amount to what's > > currently in my refactor branch (and almost to what I've sent as the > > complete refactor patch, while doi

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
A.M. wrote: > > On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: > > > A.M. wrote: > >> > >> On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > >> > >>> I have modified test_fsync to use test labels that match wal_sync_method > >>> values, and and added more tests for open_sync with different

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: > A.M. wrote: >> >> On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: >> >>> I have modified test_fsync to use test labels that match wal_sync_method >>> values, and and added more tests for open_sync with different sizes. >>> This should make

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Tomas Vondra
Dne 18.1.2011 18:12, Robert Haas napsal(a): > On Tue, Jan 18, 2011 at 4:53 AM, wrote: >> So the most important question is how to intercept the new/updated rows, >> and where to store them. I think each backend should maintain it's own >> private list of new records and forward them only in case

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
A.M. wrote: > > On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > > > I have modified test_fsync to use test labels that match wal_sync_method > > values, and and added more tests for open_sync with different sizes. > > This should make the program easier for novices to understand. Here is >

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > On Jan 18, 2011, at 1:39 PM, Tom Lane wrote: >> At the moment my opinion is that gist__int_ops is too broken to be >> usable, and it's also too uncommented to be fixable by anyone other >> than the original author. > That seems to jibe with your comments from last yea

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 1:39 PM, Tom Lane wrote: > At the moment my opinion is that gist__int_ops is too broken to be > usable, and it's also too uncommented to be fixable by anyone other > than the original author. That seems to jibe with your comments from last year: http://archives.postgresql.o

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Pavel Stehule writes: > 2011/1/18 Tom Lane : >> I looked at this patch and found it fairly awkward.  What is the point >> of adding an additional flag to every variable, as opposed to just >> forcibly detoasting during assignment? > But detoasting on assignment isn't enought: > for i in array_l

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > These numbers are a bit crazy-making, but the upshot is that Gist is > slow out of the gate, but with data cached, it's pretty speedy. With > indexscan and bitmapscan disabled, these queries all took 300-400 > ms. So GIN was never better performing than a table scan.

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > I have modified test_fsync to use test labels that match wal_sync_method > values, and and added more tests for open_sync with different sizes. > This should make the program easier for novices to understand. Here is > a test run for Ubuntu 11

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:16 PM, Simone Aiken wrote: > When I'm learning a new system I like to first learn how to use it, > second learn its data model, third start seriously looking at the code. > So that Todo is ideal for my learning method. Sure - my point is just that we usually have as a c

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Mark Kirkwood
On 19/01/11 05:51, Robert Haas wrote: I'm not sure why they'd care, but it certainly doesn't seem worth spending the amount of time arguing about it that we are. David and Mark are, of course, free to spend their time petitioning Red Hat for relicensing if they are so inclined, but they aren't

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane : > Noah Misch writes: >> On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >>> I am sending a updated version with little bit more comments. But I am >>> sure, so somebody with good English have to edit my comments. >>> Minimally I hope, so your questions will be a

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > One of the reasons our client wants GIN for the integer[] column so bad is > because recreating the GiST integer[] index is quite painful. Before I duped > the table, I was just dropping and recreating the index on the original > table. It was great to create the GI

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera > wrote: > > Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: > >> > >> Hello Postgres Hackers, > >> > >> In reference to this todo item about clustering system table indexes, > >> ( http://archives.postgre

[HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
I have modified test_fsync to use test labels that match wal_sync_method values, and and added more tests for open_sync with different sizes. This should make the program easier for novices to understand. Here is a test run for Ubuntu 11.04: $ ./test_fsync 2000 operations per tes

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
"Kevin Grittner" wrote: > If my back-of-the-envelope math is right, a carefully constructed > pessimal load could need up to (max_connections / 2)^2 -- so 100 > connections could conceivably require 2500 structures, although > such a scenario would be hard to create. Current "picked from > thin

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:18 PM, Kevin Grittner wrote: > That's all of them. Our existing code has plenty of TODOs in it already, so I see no problem with continuing to comment places where future enhancements are possible, as long as they don't reflect deficiencies that are crippling in the pres

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:00 PM, Heikki Linnakangas wrote: > On 18.01.2011 21:16, Robert Haas wrote: >> >> On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane  wrote: >>> >>> While I was trying to performance-test the texteq patch, it occurred to >>> me that this proposed hint-bit change has got a serious d

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
Heikki Linnakangas wrote: > There's a few remaining TODO comments in the code, which obviously > need to be resolved one way or another Not all of these are "must haves" for 9.1. Here's how they break down: The two in predicate_internals.h mark places which would need to be touched if we fu

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Heikki Linnakangas
On 18.01.2011 21:16, Robert Haas wrote: On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote: While I was trying to performance-test the texteq patch, it occurred to me that this proposed hint-bit change has got a serious drawback. To wit, that it will totally destroy reproducibility of any perform

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Noah Misch writes: > On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >> I am sending a updated version with little bit more comments. But I am >> sure, so somebody with good English have to edit my comments. >> Minimally I hope, so your questions will be answered. > Thanks. I edit

Re: [HACKERS] Replication logging

2011-01-18 Thread Josh Berkus
All, Just speaking as someone who does a lot of log-crunching for performance review, I don't find having a separate log_connections option for replication terribly useful. It's easy enough for me to just log all connections and filter for the type of connections I want. Even in cases where ther

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote: > Robert Haas writes: I think you may be confused about what the patch does - currently, pages with hint bit changes are considered dirty, period. Therefore, they are written whenever any other dirty page would be written: by th

Re: [HACKERS] texteq/byteaeq: avoid detoast

2011-01-18 Thread Tom Lane
Noah Misch writes: > texteq, textne, byteaeq and byteane detoast their arguments, then check for > equality of length. Unequal lengths imply the answer trivially; given equal > lengths, the functions proceed to compare the actual bytes. We can skip > detoasting entirely when the lengths are uneq

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Josh Berkus
> To be frank, I really don't care about fixing this behavior on ext3, > especially in the context of that sort of hack. That filesystem is not > the future, it's not possible to ever really make it work right, and > every minute spent on pandering to its limitations would be better spent > elsew

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:33, Robert Haas wrote: > On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas > wrote: >> I also find it weird that incoming replication connections are logged by >> default. In the standby, we also log "streaming replication successfully >> connected to primary", which

[HACKERS] PgEast: 2011, 2nd call for papers

2011-01-18 Thread Joshua D. Drake
Hey folks, PgEast is being held in NYC this year from 03/22-03-25. Get your papers in, the deadline is soon! http://www.postgresqlconference.org/ Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support,

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Is there a value to this test_fsync test? > > > > Compare open_sync with different sizes: > > (This is designed to compare the cost of one large > > sync'ed write and two smaller sync'ed writes.) > > open_sync 16k write

[HACKERS] Performance bug in DO blocks

2011-01-18 Thread Tom Lane
I just noticed that if you execute the same DO command over and over within a session, it gets slower and slower. And if you keep it up you'll notice the backend's RAM consumption bloating too. The cause appears to be that we leak the cached plans created for any SQL statements or expressions wit

Re: [HACKERS] [PERFORM] pgbench to the MAXINT

2011-01-18 Thread Greg Smith
Euler Taveira de Oliveira wrote: (i) If we want to support and scale factor greater than 21474 we have to convert some columns to bigint; it will change the test. From the portability point it is a pity but as we have never supported it I'm not too worried about it. Why? Because it will use big

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Simon Riggs
On Tue, 2011-01-18 at 18:03 +0100, Magnus Hagander wrote: > So it'd be pg_receive_wal and pg_receive_base_backup then? OK for me. Maybe even pg_receive_wal_stream Don't see any reason why command names can't be long. We have many function names already that long. -- Simon Riggs htt

Re: [HACKERS] psql: Add \dL to show languages

2011-01-18 Thread Andreas Karlsson
Hi Josh, Nope, I do not have any better ideas than "DO Blocks?". Everything looks good with the exception one bug now. \dL foo * QUERY ** SELECT l.lanname AS "Name", pg_catalog.pg_get_userbyid(l.lanowner) as "Owner", l.lanpltrusted AS "Trusted" FROM pg_catalog.pg_la

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Tom Lane
Robert Haas writes: >>> I think you may be confused about what the patch does - currently, >>> pages with hint bit changes are considered dirty, period. >>> Therefore, they are written whenever any other dirty page would be >>> written: by the background writer cleaning scan, at checkpoints, >>> a

Re: [HACKERS] Re: new patch of MERGE (merge_204) & a question about duplicated ctid

2011-01-18 Thread Greg Smith
David Fetter wrote: I think I haven't communicated clearly what I'm suggesting, which is that we ship with both an UPSERT and a MERGE, the former being ugly, crude and simple, and the latter festooned with dire warnings about isolation levels and locking. I don't know that I completely agree

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 11:53:55 -0300 2011: > On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera > wrote: > > Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > > > >> Ok, thanks for clarifying. I've updated to use strerror(). Guess it's > >> tim

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera wrote: >> Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: >>> >> >Hello Postgres Hackers, >>> >> >In reference to this todo item about clustering system table indexes, >>> ( http://archives.postgresql.org/pgsql-hackers/2004

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Jan 18, 2011, at 6:35 AM, Alvaro Herrera wrote: >> > > Wow, this is really old stuff. I don't know if this is really of any > benefit, given that these catalogs are loaded into syscaches anyway. The benefit is educational primarily. I was looking for a todo list item that would expose me

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:03 PM, Magnus Hagander wrote: > So it'd be pg_receive_wal and pg_receive_base_backup then? Votes from > others? (it's easy to rename so far, so I'll keep plugging away under > the name pg_basebackup based on Fujii-sans comments until such a time > as we have a reasonable

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:32 PM, Jim Nasby wrote: >> How's that different from what vacuum does on a TOAST table now? > > TOAST vacuum is currently an entirely separate vacuum. It might run at the > same time as the main table vacuum, but it still has all the work that would > be associated wit

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: > a few weeks back I hacked an experimental patch that removed the hint > bit action completely.  the results were very premature and/or > incorrect, but my initial findings suggested that hint bits might not > be worth the cost from performan

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby wrote: > On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: >> Robert Haas  wrote: >> >>> a quick-and-dirty attempt to limit the amount of I/O caused by hint >>> bits. I'm still very interested in knowing what people think about >>> that. >> >> I found th

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:32 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >>> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: - Forks are very possibly a more efficient way to deal with TOAS

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:24 AM, Robert Haas wrote: > On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >>> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: - Forks are very possibly a more efficient way to deal with TOAST than having s

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: > On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>> - Forks are very possibly a more efficient way to deal with TOAST than >>> having separate tables. There's a fair a

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: > On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>> - Forks are very possibly a more efficient way to deal with TOAST than >>> having separate tables. There's a fair amount of overhead we pa

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: > On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >> - Forks are very possibly a more efficient way to deal with TOAST than >> having separate tables. There's a fair amount of overhead we pay for the >> current setup. > > That seems like an inte

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote: > a few weeks back I hacked an experimental patch that removed the hint > bit action completely. the results were very premature and/or > incorrect, but my initial findings suggested that hint bits might not > be worth the cost from performance st

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Cédric Villemain
2011/1/18 Robert Haas : > On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane wrote: >> David Fetter writes: >>> I'm guessing there's a PolicyŽ at Red Hat that software made on its >>> dime be GPL (v2, I'd guess), and that getting an exception would >>> involve convening its board or similarly drastic act

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 4:53 AM, wrote: > So the most important question is how to intercept the new/updated rows, > and where to store them. I think each backend should maintain it's own > private list of new records and forward them only in case of commit. Does > that sound reasonable? At the

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Cédric Villemain
2011/1/18 Magnus Hagander : > On Tue, Jan 18, 2011 at 17:31, Tom Lane wrote: >> Magnus Hagander writes: > Actually, after some IM chats, I think pg_streamrecv should be > renamed, probably to pg_walstream (or pg_logstream, but pg_walstream > is a lot more specific than that) >> p

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:31, Tom Lane wrote: > Magnus Hagander writes: Actually, after some IM chats, I think pg_streamrecv should be renamed, probably to pg_walstream (or pg_logstream, but pg_walstream is a lot more specific than that) > >>> pg_stream_log >>> pg_stream_backup >

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane wrote: >> Oh, I misread Itagaki-san's comment to imply that that *was* in the >> patch.  Maybe I should go read it. > Perhaps. :-) > While you're at it you might commit it. :-) Yeah, as penance I'll take this one.

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane wrote: > David Fetter writes: >> I'm guessing there's a PolicyŽ at Red Hat that software made on its >> dime be GPL (v2, I'd guess), and that getting an exception would >> involve convening its board or similarly drastic action. > > It's company policy,

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >>> No, I don't think so.  Has any evidence been submitted that that part of >>> the patch is of benefit? > >> I think you might be mixing up what's actually in the patch

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Is there a value to this test_fsync test? > > > > Compare open_sync with different sizes: > > (This is designed to compare the cost of one large > > sync'ed write and two smaller sync'ed writes.) > > open_sync 16k write

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >> No, I don't think so.  Has any evidence been submitted that that part of >> the patch is of benefit? > I think you might be mixing up what's actually in the patch with > another idea that was proposed but isn't actually i

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera wrote: > Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: >> >> Hello Postgres Hackers, >> >> In reference to this todo item about clustering system table indexes, >> ( http://archives.postgresql.org/pgsql-hackers/2004-05/msg00

Re: [HACKERS] Replication logging

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas wrote: > I also find it weird that incoming replication connections are logged by > default. In the standby, we also log "streaming replication successfully > connected to primary", which serves much of the same debugging purpose. Oh, good point

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >> It's a very light-weight alternative of memcmp the byte data, >> but there is still the same issue -- we might have different >> compressed results if we use different algorithm for TOASTing. > > Which makes it a lightweight waste of cycles. > >

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Tom Lane
Magnus Hagander writes: >>> Actually, after some IM chats, I think pg_streamrecv should be >>> renamed, probably to pg_walstream (or pg_logstream, but pg_walstream >>> is a lot more specific than that) >> pg_stream_log >> pg_stream_backup > Those seem better. > Tom, would those solve your conce

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Itagaki Takahiro writes: > On Tue, Jan 18, 2011 at 05:39, Tom Lane wrote: >> I haven't looked at this patch, but it seems to me that it would be >> reasonable to conclude A != B if the va_extsize values in the toast >> pointers don't agree. > It's a very light-weight alternative of memcmp the by

Re: [HACKERS] Replication logging

2011-01-18 Thread Tom Lane
Magnus Hagander writes: > Is there *any* usecase for setting them differently though? I can't believe we're still engaged in painting this bikeshed. Let's just control it off log_connections and have done. BTW, what about log_disconnections --- does a walsender emit a message according to that?

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Tom Lane
David Fetter writes: > I'm guessing there's a Policy® at Red Hat that software made on its > dime be GPL (v2, I'd guess), and that getting an exception would > involve convening its board or similarly drastic action. It's company policy, and while it *might* be possible to get an exception, the e

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 10:56, Fujii Masao wrote: > On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander wrote: >>> We should treat log_disconnections the same? >> >> We could keep it a boolean, but then only log disconnections for the >> cases that are mentioned in log_connections? >> >> It doesn't

Re: [HACKERS] SQL/MED - file_fdw

2011-01-18 Thread Shigeru HANADA
n call even if the planning is not for EXPLAIN. I'll try to defer generating explainInfo until EXPLAIN VERBOSE really uses it. It might need new hook point in expalain.c, though. Regards, -- Shigeru Hanada 20110118-file_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mail

Re: [HACKERS] review: FDW API

2011-01-18 Thread Shigeru HANADA
I've (hopefully) fixed issues above. Please find attached patches. == patch list == 1) 20110118-no_fdw_perm_check.patch - This patch is not included in last post. This had been proposed on 2011-01-05 first, but maybe has not been reviewd yet. I re-propose this patch for SQL standard conforman

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera wrote: > Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > >> Ok, thanks for clarifying. I've updated to use strerror(). Guess it's >> time for another patch, PFA :-) > > Thanks ...  Message nitpick: > +   if (compresslevel >

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > Ok, thanks for clarifying. I've updated to use strerror(). Guess it's > time for another patch, PFA :-) Thanks ... Message nitpick: + if (compresslevel > 0) + { + fprintf(stderr, + _("%s: this bu

  1   2   >