Re: [HACKERS] new correlation metric

2008-11-02 Thread Brendan Jurd
Hi Jeff, I've been assigned to do an initial review of your "new correlation metric" patch. If I'm grokking the thread, it looks like Tom suggested a substantial change in the approach (targetting per-index correlation rather than per-column) [1], and although you agreed with the spirit of his su

Re: [HACKERS] Hot standby v5 patch assertion failure

2008-11-02 Thread Simon Riggs
On Mon, 2008-11-03 at 12:16 +1300, Mark Kirkwood wrote: > Trying out a few different scenarios I ran across this: > > CONTEXT: xlog redo update: rel 1663/16384/16397; tid 9614/62; new 158828/59 > DEBUG: start recovery xid = 7002 lsn = 0/6F012EE4 > CONTEXT: xlog redo update: rel 1663/16384/163

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-02 Thread Greg Smith
On Sun, 2 Nov 2008, Josh Berkus wrote: I'd start with command-line switches, e.g. config --memory=32GB --type=DW --size=500GB --connections=20 That seems reasonable, I think I'll push a fancier UI on the backburner then and just spec out an options interface like this one. I think in initia

[HACKERS] How to parse Datum

2008-11-02 Thread Zhe He
I met a problem with parsing Datum, is there any good way to parse this kind of value? Because I want to get the value in an attribute of a table. In TupleTableSlot, there is a tts_values which is Datum type. Does anyone know how to parse it? Best Regards, Zhe HE TEL: (001) 646-789-3008 Address:96

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > On Sun, 2 Nov 2008, Tom Lane wrote: >> The directions on the "move" tab imply that this is taken care of >> automatically? > What you get when you move something is redirect page that translates the > old name into the new one. But if you were already tar

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Greg Smith
On Sun, 2 Nov 2008, Tom Lane wrote: "Brendan Jurd" <[EMAIL PROTECTED]> writes: Note that if you rename a page you also need to update any redirects which point to it. The directions on the "move" tab imply that this is taken care of automatically? What you get when you move something is red

Re: [HACKERS] [WIP] In-place upgrade

2008-11-02 Thread Robert Haas
I tried to apply this patch to CVS HEAD and it blew up all over the place. It doesn't seem to be intended to apply against CVS HEAD; for example, I don't have backend/access/heap/htup.c at all, so can't apply changes to that file. I was able to clone the GIT repository with the following command.

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
"Brendan Jurd" <[EMAIL PROTECTED]> writes: > Note that if you rename a page you also need to update any redirects > which point to it. The directions on the "move" tab imply that this is taken care of automatically? But anyway it'll just be a minor amount of extra work when the time comes.

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Note that I'm going to propose replacing the wiki with real software > come 8.5. Where is this "real software" going to come from? I can't imagine that software with more functionality than the wiki is going to spring into existence without a huge invest

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Brendan Jurd
On Mon, Nov 3, 2008 at 2:31 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > The problem I'm having with this is that the current scheme for naming > commitfest pages assumes you know the date for each one. It seems way > premature to bet on when the first 8.5 fest will start. > > Wikimedia does seem to

Re: [HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Josh Berkus
Tom, Thoughts, better ideas? Nope, that's why I ignored CommitFestOpen. I couldn't figure out where to point it. Note that I'm going to propose replacing the wiki with real software come 8.5. The wiki method confuses new reviewers, and makes me spend about 500% as much time managing the

[HACKERS] Where to point CommitFestOpen?

2008-11-02 Thread Tom Lane
http://wiki.postgresql.org/wiki/CommitFest_2008-11 currently says that it's closed to new submissions, but this isn't particularly helpful when the link it gives for where to submit them points right back to itself. So, where should we make CommitFestOpen redirect to? The problem I'm having with

Re: [HACKERS] array_agg and array_accum (patch)

2008-11-02 Thread Jeff Davis
Here's an updated patch for just array_accum() with some simple docs. If I should document this in more places, let me know. I decided not to include array_agg() in this patch because it doesn't support the standard's ORDER BY clause. My reasoning is that, if someone is using the standard array_a

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Lawrence, Ramon
> From: Tom Lane [mailto:[EMAIL PROTECTED] > What alternatives are there for people who do not run Windows? > > regards, tom lane The TPC-H generator is a standard code base provided at http://www.tpc.org/tpch/. We have been able to compile this code on Linux. However, we

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Tom Lane
"Lawrence, Ramon" <[EMAIL PROTECTED]> writes: > The easiest way to test would be to generate your own TPC-H data and > load it into a database for testing. I have posted the TPC-H generator > at: > http://people.ok.ubc.ca/rlawrenc/TPCHSkew.zip > The generator can produce skewed data sets. It was

Re: [HACKERS] reducing statistics write overhead

2008-11-02 Thread Tom Lane
Martin Pihlak <[EMAIL PROTECTED]> writes: > Attached is a patch that implements the described signalling. Additionally > following non-related changes have been made: > 1. fopen/fclose replaced with AllocateFile/FreeFile > 2. pgstat_report_stat() now also checks if there are functions to report > b

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-02 Thread Josh Berkus
Greg Smith wrote: While I've got a pretty clear vision for what I'm doing with this next and will kick off a pgfoundry project real soon, I wanted to throw this out as a WIP for feedback at this point. I was ultimately hoping to one day have something like this shipped as a contrib/ module to

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Joshua Tolley
On Sun, Nov 2, 2008 at 4:48 PM, Lawrence, Ramon <[EMAIL PROTECTED]> wrote: > Joshua, > > Thank you for offering to review the patch. > > The easiest way to test would be to generate your own TPC-H data and > load it into a database for testing. I have posted the TPC-H generator > at: > > http://pe

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-02 Thread Lawrence, Ramon
Joshua, Thank you for offering to review the patch. The easiest way to test would be to generate your own TPC-H data and load it into a database for testing. I have posted the TPC-H generator at: http://people.ok.ubc.ca/rlawrenc/TPCHSkew.zip The generator can produce skewed data sets. It was

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Greg Stark
I think the k hash functions are actually normally just different slices of bits taken from one actual hash function anyways so it sounds like you've done the right thing. This sounds most interesting for multibatch hash joins if you could build a bloom filter for the future batches to avoi

[HACKERS] Hot standby v5 patch assertion failure

2008-11-02 Thread Mark Kirkwood
Trying out a few different scenarios I ran across this: 1/ Setup master and replica with replica using pg_standby 2/ Create a new database ("bench" in my case) 3/ Initialize pgbench schema size 100 4/ Run with 2 clients and 1 transactions 5/ Replica gets assertion failure This is Postgres he

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Jonah H. Harris
On Sun, Nov 2, 2008 at 5:36 PM, Hannes Eder <[EMAIL PROTECTED]> wrote: > On Sun, Nov 2, 2008 at 10:49 PM, Jonah H. Harris <[EMAIL PROTECTED]> wrote: >> Similarly, I >> created a GUC to enable pruning, named bloom_pruning. > > I guess calls to bloom_filter_XXX should be surrounded by "if > (bloom_pr

Re: [HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Hannes Eder
On Sun, Nov 2, 2008 at 10:49 PM, Jonah H. Harris <[EMAIL PROTECTED]> wrote: > Similarly, I > created a GUC to enable pruning, named bloom_pruning. I guess calls to bloom_filter_XXX should be surrounded by "if (bloom_pruning) ..." or a similar construct, i.e. make use of the GUC variable bloom_prun

[HACKERS] WIP: Hash Join-Filter Pruning using Bloom Filters

2008-11-02 Thread Jonah H. Harris
All, Attached is an initial patch I've been playing with which uses Bloom filters to reduce unnecessary processing of outer tuples in hash joins. In short, this works by creating a Bloom filter, adding all relevant tuples for the inner relation, and querying the filter (for existence) when retrie

Re: [HACKERS] don't use MAKE_PTR/OFFSET for shmem pointers

2008-11-02 Thread Tom Lane
Kris Jurka <[EMAIL PROTECTED]> writes: > Since we require every process to map the shared memory region to the same > address, we don't need the MAKE_PTR/OFFSET code that was needed when that > was not the case. This patch makes shared memory pointers just like > regular pointers. Applied with

[HACKERS] Re: [COMMITTERS] pgsql: Unite ReadBufferWithFork, ReadBufferWithStrategy, and

2008-11-02 Thread Heikki Linnakangas
Simon Riggs wrote: On Sat, 2008-11-01 at 15:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Fri, 2008-10-31 at 15:05 +, Heikki Linnakangas wrote: Log Message: --- Unite ReadBufferWithFork, ReadBufferWithStrategy, and ZeroOrReadBuffer functions into one ReadBufferExtended

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-11-02 Thread Zdenek Kotala
Tom Lane napsal(a): Heikki Linnakangas <[EMAIL PROTECTED]> writes: Hmm, you're right. I think it can be made to work by storing the *end* offset of each chunk. To find the chunk containing offset X, search for the first chunk with end_offset > X. Yeah, that seems like it would work, and it wo

Re: [HACKERS] Windowing Function Patch Review -> Performance Comparison.

2008-11-02 Thread David Rowley
Hitoshi Harada Wrote: > Thanks for your test. Didn't post publicly, I've also tested real > problems and performed better than I thought. If you can afford it, > could you add selfjoin cases? It's like: Ok, did self joins with some. I don't know if it's possible with all. Test Sub query Self jo

Re: [HACKERS] WIP: Column-level Privileges

2008-11-02 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > ... A case I just realized might be an issue is > > doing a 'select 1 from x;' where you have *no* rights on x, or any > > columns in it, would still get you the rowcount. > > Well, if you have table-level select

Re: [HACKERS] WIP: Column-level Privileges

2008-11-02 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > ... A case I just realized might be an issue is > > doing a 'select 1 from x;' where you have *no* rights on x, or any > > columns in it, would still get you the rowcount. > > Well, if you have table-level select

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Martijn van Oosterhout
On Sat, Nov 01, 2008 at 08:07:13PM -0400, Andrew Sullivan wrote: > On Sat, Nov 01, 2008 at 04:21:30PM -0400, Robert Haas wrote: > > > It looks to me like there are at least half a dozen patches submitted > > in the last week that are pretty half-baked and fall into the category > > of "Let's submi

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Dave Page
Sent from my iPhone On 2 Nov 2008, at 06:57, "Joshua D. Drake" <[EMAIL PROTECTED]> wrote: Josh Berkus wrote: Tom, Robert, Simon, What, are people just on edge because of the US election? It looks to me like the commitfest system is going really well. Of course, we'll see how long it takes

Re: [HACKERS] Well done, Hackers

2008-11-02 Thread Joshua D. Drake
Josh Berkus wrote: Tom, Robert, Simon, What, are people just on edge because of the US election? It looks to me like the commitfest system is going really well. Of course, we'll see how long it takes to close out 8.4. But I think we're in much better shape than we were for 8.3. We're even