[HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Heikki Linnakangas
On 24.03.2012 22:12, Joshua Berkus wrote: Qi, Yeah, I can see that. That's a sign that you had a good idea for a project, actually: your idea is interesting enough that people want to debate it. Make a proposal on Monday and our potential mentors will help you refine the idea. Yep. The

Re: [HACKERS] Why can't I use pgxs to build a plpgsql plugin?

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 07:56, Pavel Stehule wrote: 2012/4/16 Heikki Linnakangasheikki.linnakan...@enterprisedb.com: Ok, committed. I fixed the .PHONY line as Tom pointed out, and changed MSVC install.pm to also copy the header file. Hello, it doesn't work for modules from contrib directory pavel

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The first one is for 9.2dev, and the second is 9.1.3 backported version. === By the way, I took a close look around there, I agree with it

Re: [HACKERS] Why can't I use pgxs to build a plpgsql plugin?

2012-04-17 Thread Pavel Stehule
2012/4/17 Heikki Linnakangas heikki.linnakan...@enterprisedb.com: On 17.04.2012 07:56, Pavel Stehule wrote: 2012/4/16 Heikki Linnakangasheikki.linnakan...@enterprisedb.com: Ok, committed. I fixed the .PHONY line as Tom pointed out, and changed MSVC install.pm to also copy the header file.

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 09:50, Kyotaro HORIGUCHI wrote: This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows

Re: [HACKERS] Clobbered parameter names via DECLARE in PL/PgSQL

2012-04-17 Thread Pavel Stehule
Hello there is VIP patch of plpgsql_check_function that supports this warning Regards Pavel 2012/4/15 Pavel Stehule pavel.steh...@gmail.com: 2012/4/15 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: We can raise warning from CREATE OR REPLACE FUNCTION - but I

Re: [HACKERS] Last gasp

2012-04-17 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 If the feature set is desirable, though, I wonder if Postgres is big/high profile enough for them to figure out some sort of better arrangement. They *love* it when big open-source projects use GitHub as their public repo - they'll email

Re: [HACKERS] Slow temporary tables when using sync rep

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 02:54, Michael Nolan wrote: On Mon, Apr 16, 2012 at 6:27 PM, Thom Brownt...@linux.com wrote: Hi, I've noticed that when using synchronous replication (on 9.2devel at least), temporary tables become really slow: Since temporary tables are only present until the session ends

Re: [HACKERS] Slow temporary tables when using sync rep

2012-04-17 Thread Thom Brown
On 17 April 2012 11:30, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: What happens is that we write the commit record if the transaction accesses a temporary table, but we don't flush it. However, we still wait until it's replicated to the standby. The obvious fix is to not wait

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows down the recovery unnecessarily if you're restoring from a backup and you replay, The variable StandbyMode is false on archive recovery, so no checkpoint triggerred during then. xlog.c:10026

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Qi Huang
Hi, Heikki Thanks for your advice.I will change my plan accordingly. But I have a few questions. 1. We probably don't want the SQL syntax to be added to the grammar. This should be written as an extension, using custom functions as the API, instead of extra SQL syntax. 1. This

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Qi Huang
Besides, I saw the Gsoc site editing has been closed. Should I just submit through this mailing list with attachment? Best Regards and ThanksHuang Qi VictorComputer Science of National University of Singapore Date: Tue, 17 Apr 2012 09:16:29 +0300 From: heikki.linnakan...@enterprisedb.com

Re: [HACKERS] Memory usage during sorting

2012-04-17 Thread Greg Stark
On Mon, Apr 16, 2012 at 10:42 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: All but 4 regression tests pass, but they don't really count as failures, since they're down to an assumption in the tests that the order certain tuples appear should be the same as our current quicksort

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Sorry, I've wrote something wrong. The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows down the recovery unnecessarily if you're restoring from a backup and you replay, The variable StandbyMode is false on archive recovery, so no checkpoint

Re: [HACKERS] [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

2012-04-17 Thread Thom Brown
On 16 April 2012 17:21, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown t...@linux.com wrote: No, that's not what I was referring to.  If you don't have a standby (i.e. a single, isolated database cluster with no replication), and its synchronous_commit is

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-17 Thread Atri Sharma
On Wed, Apr 11, 2012 at 8:17 PM, Atri Sharma atri.j...@gmail.com wrote: On Tue, Apr 10, 2012 at 10:41 PM, Atri Sharma atri.j...@gmail.com wrote: Well. maybe I spoke too soon...JNI is probably the best route.  Since SPI is off the table, all we're really pulling in from pl/java is the

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Magnus Hagander wrote: On Mon, Apr 16, 2012 at 23:48, Jay Levittjay.lev...@gmail.com wrote: - Familiarity: Many developers already have a GitHub account and use it Most of the more senior developers don't use github. Other than possibly as a place to store a plain git repository. So that's

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Alex Shulgin
Jay Levitt jay.lev...@gmail.com writes: (A quick Google shows redmine and especially Trac having spam issues of their own.) Ugh, redmine (or trac for that matters) has nothing to with handling spam. I believe a typical bug tracker doesn't handle spam itself, it lets the mailing system do

Re: [HACKERS] Slow temporary tables when using sync rep

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 14:10, Thom Brown wrote: On 17 April 2012 11:30, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: What happens is that we write the commit record if the transaction accesses a temporary table, but we don't flush it. However, we still wait until it's replicated to the

Re: [HACKERS] Slow temporary tables when using sync rep

2012-04-17 Thread Thom Brown
On 17 April 2012 14:35, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 17.04.2012 14:10, Thom Brown wrote: On 17 April 2012 11:30, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: What happens is that we write the commit record if the transaction accesses a

[HACKERS] libpq URI and regression testing

2012-04-17 Thread Alvaro Herrera
Hi, When I committed Alex Shulgin's patch to add URI support to libpq, I included the test harness as well. However, due to it being in a separate subdirectory that did not previously had tests, it's not being run by buildfarm. It's not considered in make installcheck-world either. What's

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 14:55, Qi Huang wrote: Hi, Heikki Thanks for your advice.I will change my plan accordingly. But I have a few questions. 1. We probably don't want the SQL syntax to be added to the grammar. This should be written as an extension, using custom functions as the API, instead of

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Stephen Frost
* Heikki Linnakangas (heikki.linnakan...@enterprisedb.com) wrote: 1. We probably don't want the SQL syntax to be added to the grammar. This should be written as an extension, using custom functions as the API, instead of extra SQL syntax. Err, I missed that, and don't particularly agree with

Re: [HACKERS] Last gasp

2012-04-17 Thread Alex Shulgin
Jay Levitt jay.lev...@gmail.com writes: No meaningful search, eh? Works for me. Redmine searches return partial-word matches, and there's no way to disable that. Searching for test finds latest. To me, that's broken. Well, I believe one can plug in a different search engine, like lucene

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-17 Thread Andrew Dunstan
On 04/17/2012 09:12 AM, Atri Sharma wrote: I just had a small doubt I wanted to clarify.I initially said in my proposal that I would be using SPI for getting the FDW API to call Pl/Java functions,but now,after discussion with the community,I have changed the approach and I will be using JNI

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-17 Thread Atri Sharma
On Tue, Apr 17, 2012 at 7:37 PM, Andrew Dunstan and...@dunslane.net wrote: On 04/17/2012 09:12 AM, Atri Sharma wrote: I just had a small doubt I wanted to clarify.I initially said in my proposal that I would be using SPI for getting the FDW API to call Pl/Java functions,but now,after

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Heikki Linnakangas (heikki.linnakan...@enterprisedb.com) wrote: Another idea that Robert Haas suggested was to add support doing a TID scan for a query like WHERE ctid '(501,1)'. That's not enough work for GSoC project on its own, but could certainly

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, this message is attached with the patch which did not tested. That is for show the way. On Tue, Apr 17, 2012 at 9:38 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: But I think referring checkpoint_segment on such case should be inhibited, and I suppose it is possible using

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Alex Shulgin wrote: Jay Levittjay.lev...@gmail.com writes: (A quick Google shows redmine and especially Trac having spam issues of their own.) Ugh, redmine (or trac for that matters) has nothing to with handling spam. I believe a typical bug tracker doesn't handle spam itself, it lets the

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-17 Thread Greg Stark
On Mon, Apr 16, 2012 at 10:20 PM, Simon Riggs si...@2ndquadrant.com wrote: The thing I like most about temp indexes is that they needn't be temporary. I'd like to see something along the lines of demand-created optional indexes, that we reclaim space/maintenance overhead on according to some

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Greg Stark
On Tue, Apr 17, 2012 at 2:49 PM, Stephen Frost sfr...@snowman.net wrote: * Heikki Linnakangas (heikki.linnakan...@enterprisedb.com) wrote: 1. We probably don't want the SQL syntax to be added to the grammar. This should be written as an extension, using custom functions as the API, instead of

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Qi Huang
2. It's not very useful if it's just a dummy replacement for WHERE random() ?. It has to be more advanced than that. Quality of the sample is important, as is performance. There was also an interesting idea of on implementing monetary unit sampling. In reviewing this, I got the

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Stephen Frost
Qi, * Qi Huang (huangq...@hotmail.com) wrote: Doing it 'right' certainly isn't going to be simply taking what Neil did and updating it, and I understand Tom's concerns about having this be more than a hack on seqscan, so I'm a bit nervous that this would turn into something bigger than a

[HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Jameison Martin
The following patch truncates trailing null attributes from heap rows to reduce the size of the row bitmap.  Applications often have wide rows in which many of the trailing column values are null. On an insert/update, all of the trailing null columns are tracked in the row bitmap. This can

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Christopher Browne
On Tue, Apr 17, 2012 at 11:27 AM, Stephen Frost sfr...@snowman.net wrote: Qi, * Qi Huang (huangq...@hotmail.com) wrote: Doing it 'right' certainly isn't going to be simply taking what Neil did and updating it, and I understand Tom's concerns about having this be more than a hack on

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Tom Lane
Jameison Martin jameis...@yahoo.com writes: The following patch truncates trailing null attributes from heap rows to reduce the size of the row bitmap. This has been discussed before, but it always seemed that the cost-benefit ratio was exceedingly questionable. You don't get any savings

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Greg Stark
On Tue, Apr 17, 2012 at 5:33 PM, Christopher Browne cbbro...@gmail.com wrote: I get the feeling that this is a somewhat-magical feature (in that users haven't much hope of understanding in what ways the results are deterministic) that is sufficiently magical that anyone serious about their

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
On 04/17/2012 09:20 AM, Jay Levitt wrote: Antispam is (in the large) a technically unsolvable problem; even in the '90s, we'd see hackers start poking at our newest countermeasures within the hour. GitHub is a giant target, and PG probably benefits here from NOT being one. Everyone who deals

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-17 Thread Robert Haas
On Mon, Apr 16, 2012 at 1:43 PM, Jeff Davis pg...@j-davis.com wrote: On Mon, 2012-04-16 at 02:52 -0400, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes:  1. Order the ranges on both sides by the lower bound, then upper bound. Empty ranges can be excluded entirely.  2. Left := first

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-17 Thread Robert Haas
On Mon, Apr 16, 2012 at 7:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm. This sounds like something that Tom's recent work on parameterized plans ought to have fixed, or if not, it seems closely related. Not really.  It's still going to be a nestloop, and as such not terribly well suited for

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Josh Berkus
Qi, Hackers: FWIW, the PostGIS folks would *really* love to have a TABLESAMPLE which worked with geographic indexes. This would be tremendously useful for constructing low-resolution zoom out tiles on maps and similar. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] libpq URI and regression testing

2012-04-17 Thread Peter Eisentraut
On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote: What's the preferred way to make it automatically tested as much as possible? I know the buildfarm does not run installcheck-world, so if we want it there, it'd need a bit more code on the client side. I think it would be wise to have

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 12:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: I've been hacking away on a patch to do this, and attached is something that I think is pretty close to committable.  It needs another going-over and some new regression test cases, but it seems to work, and it fixes a number

Re: [HACKERS] libpq URI and regression testing

2012-04-17 Thread Alvaro Herrera
Excerpts from Peter Eisentraut's message of mar abr 17 15:41:04 -0300 2012: On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote: What's the preferred way to make it automatically tested as much as possible? I know the buildfarm does not run installcheck-world, so if we want it there,

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-04-17 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar abr 17 15:46:23 -0300 2012: On Tue, Apr 17, 2012 at 12:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: The core of the patch is in the new functions get_baserel_parampathinfo and get_joinrel_parampathinfo, which look up or construct ParamPathInfos, and

Re: [HACKERS] libpq URI and regression testing

2012-04-17 Thread Andrew Dunstan
On 04/17/2012 02:47 PM, Alvaro Herrera wrote: Excerpts from Peter Eisentraut's message of mar abr 17 15:41:04 -0300 2012: On tis, 2012-04-17 at 10:47 -0300, Alvaro Herrera wrote: What's the preferred way to make it automatically tested as much as possible? I know the buildfarm does not run

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-04-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 17, 2012 at 12:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, after writing the code for it I decided to remove creation of parameterized MergeAppendPaths from allpaths.c, though there is still some support for them elsewhere. On

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 3:50 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: These seems quite reasonable. These conditions make following conditional expression.     restorePtr = replayPtr = receivePtr But XLByteLT(recievePtr, replayPtr) this should not return true under the

Re: [HACKERS] Last gasp

2012-04-17 Thread Robert Haas
On Sun, Apr 15, 2012 at 8:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, Apr 7, 2012 at 9:51 PM, Robert Haas robertmh...@gmail.com wrote: I think this basically just boils down to too many patches and not enough people.  I was interested in Command Triggers from the beginning of this

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 11:50 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hmm. StandbyMode is a local variable which cannot be accessed in checkpointer. But WalRcvInProgress() which shows if wal receiver is running seems to be usable to ENABLE governing progress by

[HACKERS] extension allocating shared memory

2012-04-17 Thread Kevin Grittner
What is the best way for an extension to allocate shared memory and to access it from every backend? Or, if there is no support existing for that, what advice do people have if I want to make that happen? I don't need a lot (probably 1KB would do). If this just can't be done I guess I could

Re: [HACKERS] [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 9:52 PM, Thom Brown t...@linux.com wrote: On 16 April 2012 17:21, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown t...@linux.com wrote: No, that's not what I was referring to.  If you don't have a standby (i.e. a single, isolated

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, we already made a policy decision that we weren't going to try very hard to support merge joins inside parameterized subtrees, because the potential growth in planning time looked nasty.  My thought was that we might

Re: [HACKERS] extension allocating shared memory

2012-04-17 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of mar abr 17 16:27:21 -0300 2012: What is the best way for an extension to allocate shared memory and to access it from every backend? Or, if there is no support existing for that, what advice do people have if I want to make that happen? I don't need

Re: [HACKERS] libpq URI and regression testing

2012-04-17 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of mar abr 17 16:03:50 -0300 2012: That's one reason for that, but there are probably others in the way of making this fully portable and automatable. This test setup also appears to labor under the illusion that we live in a Unix-only world. And

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Jay Levitt
Greg Smith wrote: On 04/17/2012 09:20 AM, Jay Levitt wrote: Antispam is (in the large) a technically unsolvable problem; even in the '90s, we'd see hackers start poking at our newest countermeasures within the hour. GitHub is a giant target, and PG probably benefits here from NOT being one.

[HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Greg Stark
On Tue, Apr 17, 2012 at 5:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: This has been discussed before, but it always seemed that the cost-benefit ratio was exceedingly questionable.  You don't get any savings whatsoever unless you reduce the size of the null bitmap across a MAXALIGN boundary,

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: FWIW, the PostGIS folks would *really* love to have a TABLESAMPLE which worked with geographic indexes. This would be tremendously useful for constructing low-resolution zoom out tiles on maps and similar. I'm familiar with the concept of 'zoom

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-04-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 17, 2012 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Personally, I find required_outer more clear. YMMV. Perhaps. What's bothering me is the potential for confusion with outer joins; the parameter-supplying rels are *not* necessarily

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Tom Lane
Jay Levitt jay.lev...@gmail.com writes: Greg Smith wrote: Tracking when and how a bug is backported to older versions is one hard part of the problem here. That's a great point. Both GitHub and git itself have no real concept of releases, and can't tell you when a commit made it in. We do

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Tue, Apr 17, 2012 at 5:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: This has been discussed before, but it always seemed that the cost-benefit ratio was exceedingly questionable.  You don't get any savings whatsoever unless you reduce the size of the null

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Jameison Martin
Thanks for the response. The use-case I'm targeting is a schema that has multiple tables with ~800 columns, most of which have only the first 50 or so values set. 800 columns would require 800 bits in a bitmap which equates to 100 bytes. With 8-byte alignment the row bitmap would take up 104

Re: [HACKERS] extension allocating shared memory

2012-04-17 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Kevin Grittner's message: What is the best way for an extension to allocate shared memory and to access it from every backend? RequestAddinShmemSpace Perfect! That's exactly what I wanted. I see that the pg_stat_statements

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Andrew Dunstan
On 04/17/2012 04:38 PM, Tom Lane wrote: Jay Levittjay.lev...@gmail.com writes: Greg Smith wrote: Tracking when and how a bug is backported to older versions is one hard part of the problem here. That's a great point. Both GitHub and git itself have no real concept of releases, and can't

[HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 6:39 PM, Andrew Dunstan and...@dunslane.net wrote: Don't override arguments set via options with positional arguments. A number of utility programs were rather careless about paremeters that can be set via both an option argument and a positional argument. This leads

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Christopher Browne
On Tue, Apr 17, 2012 at 4:15 PM, Jay Levitt jay.lev...@gmail.com wrote: That's a great point. Both GitHub and git itself have no real concept of releases, and can't tell you when a commit made it in. Those factors likely play together in this. Git is a tool, not a workflow, and intentionally

[HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen

2012-04-17 Thread Andrew Dunstan
On 04/17/2012 07:08 PM, Robert Haas wrote: On Tue, Apr 17, 2012 at 6:39 PM, Andrew Dunstanand...@dunslane.net wrote: Don't override arguments set via options with positional arguments. A number of utility programs were rather careless about paremeters that can be set via both an option

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen

2012-04-17 Thread Andrew Dunstan
On 04/17/2012 07:19 PM, Andrew Dunstan wrote: On 04/17/2012 07:08 PM, Robert Haas wrote: On Tue, Apr 17, 2012 at 6:39 PM, Andrew Dunstanand...@dunslane.net wrote: Don't override arguments set via options with positional arguments. A number of utility programs were rather careless about

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Ants Aasma
On Tue, Apr 17, 2012 at 7:33 PM, Christopher Browne cbbro...@gmail.com wrote: Well, there may be cases where the quality of the sample isn't terribly important, it just needs to be reasonable. I browsed an article on the SYSTEM/BERNOULLI representations; they both amount to simple picks of

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 1:47 AM, Magnus Hagander mag...@hagander.net wrote: That's probably one reason people aren't jumping on this. Because there is no tracker out there that people actually *like*... I think this is a point worth serious thought. The bug trackers I've used have been mostly

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hmm. StandbyMode is a local variable which cannot be accessed in checkpointer. But WalRcvInProgress() which shows if wal receiver is running seems to be usable to ENABLE governing progress by checkpoint_segments. Even when walreceiver is not running and WAL files are read from the

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 17, 2012 at 1:47 AM, Magnus Hagander mag...@hagander.net wrote: That's probably one reason people aren't jumping on this. Because there is no tracker out there that people actually *like*... I think this is a point worth serious thought.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen

2012-04-17 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: You know, I could have sworn it was discussed, but when I look back I see it wasn't. I must have been remembering the recent logging protocol bug. I'll revert it if people want, although I still think it's a bug. I think we discussed it to the

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 But for any given ABC there are also people who will tell you that it's got significant problems. We don't need to change anything to get a system that's got significant problems; we already have one. Let's not let perfect be the enemy of

Re: [HACKERS] Gsoc2012 idea, tablesample

2012-04-17 Thread Qi Huang
Date: Wed, 18 Apr 2012 02:45:09 +0300 Subject: Re: [HACKERS] Gsoc2012 idea, tablesample From: a...@cybertec.at To: cbbro...@gmail.com CC: sfr...@snowman.net; pgsql-hackers@postgresql.org On Tue, Apr 17, 2012 at 7:33 PM, Christopher Browne cbbro...@gmail.com wrote: Well, there may

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 11:07 PM, Greg Sabino Mullane g...@turnstep.com wrote: Let's not let perfect be the enemy of good. In this case, *anything* that actually tracks bugs (and they are all quite good at that, if nothing else) is an improvement over what we have now, and thus, quite good. :)

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't override arguments set via options with positional argumen

2012-04-17 Thread Robert Haas
On Tue, Apr 17, 2012 at 10:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: I can see both sides of this.  I agree that the old behavior is buggy, but what I imagine Robert is worried about is scripts that accidentally work okay today and would stop working once the PG programs are fixed to complain

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
On 04/17/2012 10:30 PM, Tom Lane wrote: Indeed. The only one I've got extensive experience with is Bugzilla (because Red Hat uses it) and I do cordially hate it. At least some of that is due to bureaucratic practices RH has evolved, like cloning bugs N times for N affected releases, but I

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: Rather than talk about adopting one of the available torture devices, I'd happily consider the simplest thing possible that would be useful here instead. Here's my proposed tiny tracker: Wasn't Jay just muttering about writing your own bug tracker

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
On 04/17/2012 11:44 PM, Tom Lane wrote: At the same time, I think we'd likely be a lot better off squirting this data into bugzilla or another standard tracker, instead of building our own infrastructure. Perhaps. It just struck me that a lot of the custom bits needed here regardless could

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Brendan Jurd
On 18 April 2012 13:44, Tom Lane t...@sss.pgh.pa.us wrote: ... I think you'll find a lot of that data could be mined out of our historical commit logs already.  I know I make a practice of mentioning bug # whenever there is a relevant bug number, and I think other committers do too.  It

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Tom Lane
Jameison Martin jameis...@yahoo.com writes: The use-case I'm targeting is a schema that has multiple tables with ~800 columns, most of which have only the first 50 or so values set. 800 columns would require 800 bits in a bitmap which equates to 100 bytes. With 8-byte alignment the row

Re: [HACKERS] Improving our clauseless-join heuristics

2012-04-17 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: I might still be misunderstanding, but I think what you are suggesting is that in the loop in make_rels_by_clause_joins, if we find that the old_rel doesn't have a join clause/restriction with the current other_rel, we check to see whether other_rel

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2012-04-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Apr 16, 2012 at 1:43 PM, Jeff Davis pg...@j-davis.com wrote: ... Only one side really needs the mark and restore logic, but it was easier to write the pseudocode in a symmetrical way (except step 7). I'm actually not sure these are equivalent

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Magnus Hagander
On Wed, Apr 18, 2012 at 04:30, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Apr 17, 2012 at 1:47 AM, Magnus Hagander mag...@hagander.net wrote: That's probably one reason people aren't jumping on this. Because there is no tracker out there that people

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Magnus Hagander
On Tue, Apr 17, 2012 at 19:59, Greg Smith g...@2ndquadrant.com wrote: On 04/17/2012 09:20 AM, Jay Levitt wrote: Let's pick a real example from the last week of my life, where having a bug tracker would have helped me out.  This appears in a log: ERROR: missing chunk number 0 for toast value

Re: [HACKERS] Improving our clauseless-join heuristics

2012-04-17 Thread Amit Kapila
I'm afraid I'm still not following you very well. Perhaps you could submit a proposed patch? Before that can you please explain in little more detail (if possible with small example) about the idea you have told in original mail : is there any join clause that both these relations participate

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Magnus Hagander
On Wed, Apr 18, 2012 at 05:44, Tom Lane t...@sss.pgh.pa.us wrote: Greg Smith g...@2ndquadrant.com writes: Rather than talk about adopting one of the available torture devices, I'd happily consider the simplest thing possible that would be useful here instead.  Here's my proposed tiny tracker:

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-17 Thread Jameison Martin
Regarding the schema: I'm afraid the schema cannot be changed at this point, though I appreciate the suggestions.  Regarding an INSERT performance test, what kind of table shape would you like me to exercise?  The patch as submitted may actually shave some cycles off of the insertion of rows

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Apr 18, 2012 at 04:30, Tom Lane t...@sss.pgh.pa.us wrote: So when I read Andrew's recent suggestion that we use Bugzilla, my immediate reaction was egad, can't we do better?. Maybe we can't :-(. Personally, I'd say we *already* do better

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Magnus Hagander
On Wed, Apr 18, 2012 at 07:52, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Apr 18, 2012 at 04:30, Tom Lane t...@sss.pgh.pa.us wrote: So when I read Andrew's recent suggestion that we use Bugzilla, my immediate reaction was egad, can't we do better?.