Re: [HACKERS] Page Checksums

2012-01-10 Thread Heikki Linnakangas
On 10.01.2012 02:12, Jim Nasby wrote: Filesystem CRCs very likely will not happen to data that's in the cache. For some users, that's a huge amount of data to leave un-protected. You can repeat that argument ad infinitum. Even if the CRC covers all the pages in the OS buffer cache, it still

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: Joachim Wieland j...@mcknight.de writes: [ send NOTIFYs to slaves by means of: ] Good idea. I wonder whether it'd be practical to not involve WAL per se in this at all, but to transmit NOTIFY messages by having walsender

Re: [HACKERS] Page Checksums

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 8:04 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 10.01.2012 02:12, Jim Nasby wrote: Filesystem CRCs very likely will not happen to data that's in the cache. For some users, that's a huge amount of data to leave un-protected. You can repeat

Re: [HACKERS] Generate call graphs in run-time

2012-01-10 Thread Joel Jacobson
Is this only to print out the stack upon errors? Looks like the stack is in the variable error_context_stack. Is it always available containing all the parent functions, even when there is no error? Can I reach it from within pgstat.c? 2012/1/10 Jim Nasby j...@nasby.net On Jan 9, 2012, at 2:08

Re: [HACKERS] LWLOCK_STATS

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 12:24 AM, Jim Nasby j...@nasby.net wrote: IIRC, pg_bench is *extremely* write-heavy. There's probably not that many systems that operate that way. I suspect that most OLTP systems read more than they write, and some probably have as much as a 10-1 ratio. IMHO the

Re: [HACKERS] Page Checksums

2012-01-10 Thread Benedikt Grundmann
On 10/01/12 09:07, Simon Riggs wrote: You can repeat that argument ad infinitum. Even if the CRC covers all the pages in the OS buffer cache, it still doesn't cover the pages in the shared_buffers, CPU caches, in-transit from one memory bank to another etc. You have to draw the line

Re: [HACKERS] [PATCH] Allow breaking out of hung connection attempts

2012-01-10 Thread Heikki Linnakangas
On 09.01.2012 15:49, Ryan Kelly wrote: On Mon, Jan 09, 2012 at 10:35:50AM +0200, Heikki Linnakangas wrote: That assumes that it's safe to longjmp out of PQconnectdbParams at any instant. It's not. I'm guessing because it could result in a resource leak? Yes, and other unfinished business,

[HACKERS] [v9.2] Add GUC sepgsql.client_label

2012-01-10 Thread Kohei KaiGai
This patch adds a new GUC sepgsql.client_label that allows client process to switch its privileges into another one, as long as the system security policy admits this transition. Because of this feature, I ported two permissions from process class of SELinux; setcurrent and dyntransition. The

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Joachim Wieland
On Tue, Jan 10, 2012 at 12:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: So this design is non-optimal both for existing uses and for the proposed new uses, which means nobody will like it.  You could ameliorate #1 by adding a GUC that determines whether NOTIFY actually writes WAL, but that's

Re: [HACKERS] pgsphere

2012-01-10 Thread Dave Cramer
Hi Oleg, On Sun, Jan 8, 2012 at 1:19 PM, Oleg Bartunov o...@sai.msu.su wrote: Dave, The situation with pgshpere is so, that I think we need new developer, since Janko keep silence :)  I wrote him several time, since I wanted pgsphere now could benefit very much from our KNNGiST feature.

Re: [HACKERS] LWLOCK_STATS

2012-01-10 Thread Merlin Moncure
On Tue, Jan 10, 2012 at 3:16 AM, Simon Riggs si...@2ndquadrant.com wrote: So benchmarking write-heavy workloads and separately benchmarking read-only workloads is more representative. Absolutely. High write activity applications are much more difficult to optimize with simple tricks like

Re: [HACKERS] pgsphere

2012-01-10 Thread Andrew Dunstan
On 01/10/2012 09:04 AM, Dave Cramer wrote: So where do we go from here ? First, please note that -hackers is not the right place for this discussion. pgsphere is a pgfoundry project, which is not the province of -hackers. As I suggested, the best solution is for Teodor to add you as a

Re: [HACKERS] pgsphere

2012-01-10 Thread Oleg Bartunov
I think Dave you fork project, I suggest different name pgsphere-2 to avoid confusion. github would be ok. Teodor and I are really busy guys, so I don't believe we could participate much, except discussion and testing. We implemented KNNGiST, you add neighbourhood search support to pgsphere :)

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 12:56 PM, Joachim Wieland j...@mcknight.de wrote: I chose to do it this way because it seemed like the most natural way to do it (which of course doesn't mean it's the best)  :-). If its any consolation its exactly how I would have done it also up until about 2 months

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: It might be a bit tricky to get walreceivers to inject the data into the slave-side ring buffer at the right time, ie, not until after the commit a given message describes has been

Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved

2012-01-10 Thread Peter Eisentraut
On tis, 2011-12-13 at 07:55 -0500, Alex Goncharov wrote: char *PQcmdStatus(PGresult *res); char *PQcmdTuples(PGresult *res); Unreasonable: a. What, these two can modify 'res' I pass in?.. b. Oh, yes, because they return 'char *' pointing to 'res-cmdStatus+n', so, a libpq

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Andrew Dunstan
On 01/03/2012 09:11 PM, Andrew Dunstan wrote: On 01/03/2012 08:40 PM, Robert Haas wrote: On Tue, Jan 3, 2012 at 9:33 AM, Pavel Stehulepavel.steh...@gmail.com wrote: I'd suppose it interesting to add a table to pg_catalog containing this data. - it is useless overhead I tend to agree.

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Pavel Stehule
2012/1/10 Andrew Dunstan and...@dunslane.net: On 01/03/2012 09:11 PM, Andrew Dunstan wrote: On 01/03/2012 08:40 PM, Robert Haas wrote: On Tue, Jan 3, 2012 at 9:33 AM, Pavel Stehulepavel.steh...@gmail.com  wrote: I'd suppose it interesting to add a table to pg_catalog containing this

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 4:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: It might be a bit tricky to get walreceivers to inject the data into the slave-side ring buffer at the right time,

Re: [HACKERS] pgsphere

2012-01-10 Thread Jan-Benedict Glaw
On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov o...@sai.msu.su wrote: pgsphere now could benefit very much from our KNNGiST feature. This is number one development from my point of view. I and Teodor have no time to work on pgsphere, sorry. But, there are some astronomers I'm working with,

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Andrew Dunstan
On 01/10/2012 12:34 PM, Pavel Stehule wrote: Actually, now I look closer I see that PLPerl passes back a stringified status from SPI_execute(), so there is no great need for setting up these constants. It's probably water under the bridge now, but maybe PLPython should have done this too.

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Pavel Stehule
2012/1/10 Andrew Dunstan and...@dunslane.net: On 01/10/2012 12:34 PM, Pavel Stehule wrote: Actually, now I look closer I see that PLPerl passes back a stringified status from SPI_execute(), so there is no great need for setting up these constants. It's probably water under the bridge now,

Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved

2012-01-10 Thread Alex Goncharov
,--- You/Peter (Tue, 10 Jan 2012 19:13:42 +0200) * | On tis, 2011-12-13 at 07:55 -0500, Alex Goncharov wrote: |char *PQcmdStatus(PGresult *res); |char *PQcmdTuples(PGresult *res); | | Unreasonable: | |a. What, these two can modify 'res' I pass in?.. | |b. Oh, yes,

Re: [HACKERS] random_page_cost vs seq_page_cost

2012-01-10 Thread Greg Smith
On 1/5/12 5:04 AM, Benedikt Grundmann wrote: I have a question of how to benchmark hardware to determine the appropriate ratio of seq_page_cost vs random_page_cost. Emails in this mailing lists archive seem to indicate that 1.0 vs 3.0 - 4.0 are appropriate values on modern hardware. Which

Re: [HACKERS] pgsphere

2012-01-10 Thread Oleg Bartunov
On Tue, 10 Jan 2012, Jan-Benedict Glaw wrote: On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov o...@sai.msu.su wrote: pgsphere now could benefit very much from our KNNGiST feature. This is number one development from my point of view. I and Teodor have no time to work on pgsphere, sorry. But,

Re: [HACKERS] Command Triggers

2012-01-10 Thread Dimitri Fontaine
Hi, Robert Haas robertmh...@gmail.com writes: Maybe we should try to split the baby here and defer the question of whether to expose any of the parse tree internals, and if so how much, to a future release. It seems to me that we could design a fairly useful set of functionality around

Re: [HACKERS] pgsphere

2012-01-10 Thread Greg Smith
On 1/10/12 9:54 AM, Andrew Dunstan wrote: Speaking with my pgfoundry admin hat on, I am extremely reluctant to take control of a project away from its owners. If you don't get any action in a week or so, you can approach the pgfoundry admins for help. In the meantime you might want to fork the

Re: [HACKERS] pgsphere

2012-01-10 Thread Jan-Benedict Glaw
On Wed, 2012-01-11 00:43:25 +0400, Oleg Bartunov o...@sai.msu.su wrote: On Tue, 10 Jan 2012, Jan-Benedict Glaw wrote: On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov o...@sai.msu.su wrote: pgsphere now could benefit very much from our KNNGiST feature. This is number one development from

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Joachim Wieland
On Tue, Jan 10, 2012 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: [ Tom sketches a design ] Seems a bit overcomplicated.  I was just thinking of having walreceiver note the WAL endpoint at the instant of receipt of a notify message, and not release

Re: [HACKERS] log messages for archive recovery progress

2012-01-10 Thread Euler Taveira de Oliveira
On 08-01-2012 11:59, Satoshi Nagayasu / Uptime Technologies, LLC. wrote: [2011-12-08 15:14:36 JST] 16758: LOG: restored log file 00080046 from archive [2011-12-08 15:14:36 JST] 16758: LOG: recoverying 00080046 [2011-12-08 15:14:36 JST] 16758: LOG: restored

Re: [HACKERS] JSON for PG 9.2

2012-01-10 Thread Robert Haas
On Tue, Dec 20, 2011 at 9:06 PM, David E. Wheeler da...@kineticode.com wrote: On Dec 20, 2011, at 10:39 AM, Claes Jakobsson wrote: Are people explicitly asking for a) *JSON* datatype or b) a type that lets you store arbitrary complex semi-untyped data structures? Yes. if b) then this might

[HACKERS] checkpoint writeback via sync_file_range

2012-01-10 Thread Robert Haas
Greg Smith muttered a while ago about wanting to do something with sync_file_range to improve checkpoint behavior on Linux. I thought he was talking about trying to sync only the range of blocks known to be dirty, which didn't seem like a very exciting idea, but after looking at the man page for

Re: [HACKERS] Real-life range datasets

2012-01-10 Thread Stefan Keller
Hi I'm proposing OpenStreetMap which is of variable size up to 250 GB XML Data for whole world. It's downloadable from CloudMade.com or Geofabrik.de and can be imported into PostgreSQL using osm2pgsql. It's a key/value schema literally of the real world. I'm using hstore option of osm2pgsql and

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
Joachim Wieland j...@mcknight.de writes: On Tue, Jan 10, 2012 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Suggest we add something to initial handshake from standby to say please send me notify traffic, +1 on that. From what you said I imagined

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
BTW ... it occurs to me to ask whether we really have a solid use-case for having listeners attached to slave servers. I have personally never seen an application for LISTEN/NOTIFY in which the listeners were entirely read-only. Even if there are one or two cases out there, it's not clear to me

Re: [HACKERS] checkpoint writeback via sync_file_range

2012-01-10 Thread Greg Smith
On 1/10/12 9:14 PM, Robert Haas wrote: Based on that, I whipped up the attached patch, which, if sync_file_range is available, simply iterates through everything that will eventually be fsync'd before beginning the write phase and tells the Linux kernel to put them all under write-out. I

Re: [HACKERS] JSON for PG 9.2

2012-01-10 Thread Pavel Stehule
2012/1/11 Robert Haas robertmh...@gmail.com: On Tue, Dec 20, 2011 at 9:06 PM, David E. Wheeler da...@kineticode.com wrote: On Dec 20, 2011, at 10:39 AM, Claes Jakobsson wrote: Are people explicitly asking for a) *JSON* datatype or b) a type that lets you store arbitrary complex semi-untyped