[HACKERS] Postgres optimizer

2007-04-05 Thread Suresh
Hi, I want to get the coding details regarding postgres optimizer. Essentially, which files correspond to which functions, how the algo is implemented, the flow etc.. Where can I find this material ? Also, how to I start changing Postgres Code using eclipse platform (with tracing, debug fu

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Devrim Gündüz
Hi, On Fri, 2007-04-06 at 01:23 -0400, Matthew T. O'Connor wrote: > > The other thing to consider is that CentOS 5 has Xen built right in, > so you should be able run VMs without VMWare on it. ... if the kernel of the OS has Xen support, there will be no performance penalty (only 2%-3%) (Para-v

Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-05 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > In my understanding from the discussion, we'd better to take "cycle ID" > approach instead of "making a copy of pendingOpsTable", because duplicated > table is hard to debug and requires us to pay attention not to leak memories. > I'll adopt the cycle

Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-05 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > The patches look good to me. Please commit whatever you think is reasonable. > BTW, is anybody working on enabling the fill factor to the tables used > by pgbench? 8.3 will introduce HOT, and I think adding the feature > will make it easier to test HOT.

Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-05 Thread ITAGAKI Takahiro
Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > Itagaki, would you like to take a stab at this? Yes, I'll try to fix the mdsync problem. I'll separate this fix from LDC patch. If we need to backport the fix to the back branches, a stand-alone patch would be better. In my understanding from the di

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Tom Lane
Markus Schiltknecht <[EMAIL PROTECTED]> writes: > Uh.. can you elaborate on that? AFAICS, you would simply have to query > multiple btree indexes and make sure non of them is violated. That only works for the partition-key indexes, ie, ones where you can be sure a-priori that there cannot be dupl

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Matthew T. O'Connor
Larry Rosenman wrote: I might use that as the base then, since the hardware finishes getting here tomorrow. The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ---(end of broadcast)--

[HACKERS] Hey, there's a free infomask bit

2007-04-05 Thread Tom Lane
Along the way in applying Greg Stark's varvarlena patch, I removed HEAP_HASCOMPRESSED, which was not used and AFAIR never had been used for anything. Don't all propose a use for that bit at once... regards, tom lane ---(end of broadcast)---

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-05 Thread Koichi Suzuki
Hi, I agree to put the patch to core and the others (pg_compresslog and pg_decompresslog) to contrib/lesslog. I will make separate materials to go to core and contrib. As for patches, we have tested against pgbench, DBT-2 and our propriatery benchmarks and it looked to run correctly. Regar

Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-05 Thread Tatsuo Ishii
> > Tatsuo, would you please comment on this patch? > > Sure. I will come up with a comment by the end of this week. The patches look good to me. BTW, is anybody working on enabling the fill factor to the tables used by pgbench? 8.3 will introduce HOT, and I think adding the feature will make it

Re: [HACKERS] [PATCHES] pgbench transaction timestamps

2007-04-05 Thread Tatsuo Ishii
> > Tatsuo, would you please comment on this patch too? > > No problem. I will come up with a comment by the end of this week. Here are comments. 1) latency log file format extention looks usefull (-x option). 2) it seems the "cleanup feature" (-X option) was withdrawed by the author, but th

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Chris Browne
ler@lerctr.org ("Larry Rosenman") writes: > I might use that as the base then, since the hardware finishes getting here > tomorrow. > > My question still stands on what OS's we need coverage for. I've got Debian testing/unstable covered. I'm not sure we have Novell/SuSE covered... -- output = ("

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-05 Thread Mark Kirkwood
I'm seeing a segfault on a size TPC-H size 10 database. The patch and code are: - bitmap patch from 12 Mar - 8.3 dev from 27 Mar SELECT count(distinct(o_orderkey)) FROM orders orders_alias WHERE o_orderpriority IN ('1-URGENT', '3-MEDIUM') AND o_orderstatus='P'; (gdb) bt #0 0x in ?? ()

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Larry Rosenman
I might use that as the base then, since the hardware finishes getting here tomorrow. My question still stands on what OS's we need coverage for. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loo

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Joshua D. Drake
It will have CentOS 4.4 X86_64 as the base os with VMWare Server running on it. I am willing to run any X86 or X64 OS’s in VM’s as buildfarm clients. What OS’s do we need coverage for? CentOS5 hits ina couple days. J LER -- Larry Rosenman htt

[HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Larry Rosenman
I'm in the process of building a new box that will have Dual Xeon 5120's (Dual Core), and 4G of ram and 2.4T of disk (6x400G SATA). It will have CentOS 4.4 X86_64 as the base os with VMWare Server running on it. I am willing to run any X86 or X64 OS's in VM's as buildfarm clients. Wh

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Martijn van Oosterhout
On Thu, Apr 05, 2007 at 10:00:37PM +0200, Markus Schiltknecht wrote: > >If someone manages to crack uniqueness for GiST indexes, we'll have our > >answer, since it has exactly the same problem but on a different scale. > >(Or vice-versa, if some gets uniqueness for multiple indexes, we can do > >it

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Markus Schiltknecht
Hi, Martijn van Oosterhout wrote: The executor would have to be clever enough to not do a single index scan, but possibly scan through multiple indexes when asking for uniqueness, depending on the partitioning rule set. But it's not the executor that checks uniqueness, it's built into the btr

[HACKERS] buildfarm minimum script versions

2007-04-05 Thread Andrew Dunstan
Following a suggestion by Tom Lane, I have written some code that will reject reports from members running very old script versions. In a week I intend to turn it on and require script version 1.45 as a minimum. That version is 18 months old. 1 month later I propose to set that to 1.68, whic

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Markus Schiltknecht
Hi, Zeugswetter Andreas ADI SD wrote: CREATE INDEX x ON test(a, b, c); isn't the same as CRETAE INDEX x ON test(c, b, a); That is only a problem if you also want to avoid a sort (e.g. for an order by), ..or if you want to use that index for 'WHERE a = 5'. The first one is probably helpi

Re: [HACKERS] "Garbled" postgres logs

2007-04-05 Thread Tom Lane
[EMAIL PROTECTED] (Tim Goodaire) writes: > I've compared the libc and kernel versions between a misbehaving machine and a > machine that is logging properly and they're the same: [ scratches head... ] So what's different? Anyway, if you are interested in trying to cut libc out of the equation, t

Re: [HACKERS] "Garbled" postgres logs

2007-04-05 Thread Tim Goodaire
On Tue, Apr 03, 2007 at 04:51:33PM -0400, Tom Lane wrote: > No, syslogger is single-threaded so it can't be at fault. The > interleaving must be happening when the data is inserted into the pipe > that leads to syslogger. We've got multiple backends concurrently > writing that pipe, remember. >

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Zeugswetter Andreas ADI SD
> > That lets you enforce unique constraints as long as the partition key > > is part of the unique constraint. > > Is that already sufficient? yes > That would alter the ordering of > the columns in the index, no? I mean: It produces ordered blocks of append nodes for range queries that spa

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Robert Treat
On Wednesday 04 April 2007 21:17, Gregory Stark wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > >> If we don't have multi-table indexes how do we enforce a primary key > >> against a partitioned set? What about non primary keys that are just > >> UNIQUE? What about check constraints that aren'

Re: [HACKERS] Bug in UTF8-Validation Code?

2007-04-05 Thread Tom Lane
Martijn van Oosterhout writes: > I think the problem is that most encodings do not have the concept of a > code point anyway, so implementing it for them is fairly useless. Yeah. I'm beginning to think that the right thing to do is (a) make chr/ascii do the same thing as Oracle (ie, as in the d

Re: [HACKERS] Checkpoint gets stuck in mdsync

2007-04-05 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > My proposed fix is to make a copy of pendingOpsTable before entering the > loop. AbsorbFsyncRequest will put new requests to a fresh new > pendingOpsTable, while the mdsync loop will drain the copy. I'll write a > patch along those lines if there'

Re: [HACKERS] Interaction of PITR backups andBulkoperationsavoiding WAL

2007-04-05 Thread Bruce Momjian
Simon Riggs wrote: > On Wed, 2007-04-04 at 22:05 -0400, Bruce Momjian wrote: > > > I _still_ have no patch for this. > > Bruce, > > As I've mentioned, there is another thread where the discussion > continued, which you should refer to. > > The subject of this thread is a potential bug that has

Re: [HACKERS] pg_index updates and SI invalidation

2007-04-05 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. --- Pa

Re: [HACKERS] Last minute mini-proposal (I know, I know) for PQexecf()

2007-04-05 Thread Brian Hurt
My apologies for the late reply... Tom Lane wrote: <[EMAIL PROTECTED]> writes: I'd like to see a new variant on PQexec(): PGresult * PQexecf(PGconn *conn, const char *fmt, ...); Way too late for 8.3 --- if we were going to do something like this, we should think first and program

Re: [HACKERS] Buildfarm failures en masse

2007-04-05 Thread Bruce Momjian
Bruce Momjian wrote: > Alvaro Herrera wrote: > > Magnus Hagander wrote: > > > Seems the change of the XML messages now causes more or less every > > > buildfarm member to fail the XML test. Meaning all the ones that don't > > > have > > > libxml, but that seems to be the vast majority... > > > >

Re: [HACKERS] Buildfarm failures en masse

2007-04-05 Thread Bruce Momjian
Alvaro Herrera wrote: > Magnus Hagander wrote: > > Seems the change of the XML messages now causes more or less every > > buildfarm member to fail the XML test. Meaning all the ones that don't have > > libxml, but that seems to be the vast majority... > > > > Looks like a missing update to regress

Re: [HACKERS] Buildfarm failures en masse

2007-04-05 Thread Alvaro Herrera
Magnus Hagander wrote: > Seems the change of the XML messages now causes more or less every > buildfarm member to fail the XML test. Meaning all the ones that don't have > libxml, but that seems to be the vast majority... > > Looks like a missing update to regression tests? I don't think the new

Re: [HACKERS] Bug in UTF8-Validation Code?

2007-04-05 Thread Martijn van Oosterhout
On Thu, Apr 05, 2007 at 11:52:14AM +0200, Albe Laurenz wrote: > But isn't a simple fix for chr() and ascii(), which does not > require a redesign, a Good Thing for 8.3 if possible? Something > that maintains as much upward and/or Oracle compatibility as > possible while doing away with ascii('EUR')

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread NikhilS
So we are unable to load any of the tables using COPY. Aww, guess should have stuck to triggers as a first choice. Mea culpa, since I should have investigated some more before deciding on rules, or should have prodded you more earlier:) Regards, Nikhils -- EnterpriseDB http://

[HACKERS] Buildfarm failures en masse

2007-04-05 Thread Magnus Hagander
Seems the change of the XML messages now causes more or less every buildfarm member to fail the XML test. Meaning all the ones that don't have libxml, but that seems to be the vast majority... Looks like a missing update to regression tests? //Magnus ---(end of broadcast

Re: [HACKERS] Bug in UTF8-Validation Code?

2007-04-05 Thread Martijn van Oosterhout
On Thu, Apr 05, 2007 at 09:34:25AM +0900, Tatsuo Ishii wrote: > I'm not sure what kind of use case for unicode_char() you are thinking > about. Anyway if you want a "code point" from a character, we could > easily add such functions to all backend encodings currently we > support. Probably it would

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread NikhilS
Hi, > I had raised this issue about rules/triggers back then and the > responses seemed to be evenly split as to which ones to use. Presumably your implementation already uses Triggers for INSERTs though, so why not use triggers for everything? No I am using rules for all the 3 cases. I am

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Simon Riggs
On Thu, 2007-04-05 at 16:35 +0530, NikhilS wrote: > Hi, > > > I had raised this issue about rules/triggers back then and > the > > responses seemed to be evenly split as to which ones to > use. > > Presumably your implementation already us

Re: [HACKERS] Bug in UTF8-Validation Code?

2007-04-05 Thread Albe Laurenz
Tatsuo Ishii wrote: > I think we need to continute design discussion, probably > targetting for 8.4, not 8.3. But isn't a simple fix for chr() and ascii(), which does not require a redesign, a Good Thing for 8.3 if possible? Something that maintains as much upward and/or Oracle compatibility as po

Re: [HACKERS] Checkpoint gets stuck in mdsync

2007-04-05 Thread Heikki Linnakangas
ITAGAKI Takahiro wrote: Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Now that the CheckpointStartLock starvation has been taken care of, I'm seeing another problem with checkpoints in my test run: mdsync never finishes. My proposed fix is to make a copy of pendingOpsTable before entering th

[HACKERS] Checkpoint gets stuck in mdsync

2007-04-05 Thread Heikki Linnakangas
Now that the CheckpointStartLock starvation has been taken care of, I'm seeing another problem with checkpoints in my test run: mdsync never finishes. Here's what's happening: 1. checkpoint calls mdsync 2. mdsync start processing pending fsyncs from pendingOpsTable (at this point, normal backen

Re: [HACKERS] Checkpoint gets stuck in mdsync

2007-04-05 Thread ITAGAKI Takahiro
Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > Now that the CheckpointStartLock starvation has been taken care of, I'm > seeing another problem with checkpoints in my test run: mdsync never > finishes. > > My proposed fix is to make a copy of pendingOpsTable before entering the > loop. Absor

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-05 Thread Simon Riggs
On Tue, 2007-04-03 at 19:45 +0900, Koichi Suzuki wrote: > Bruce Momjian wrote: > > Your patch has been added to the PostgreSQL unapplied patches list at: > > > > http://momjian.postgresql.org/cgi-bin/pgpatches > > Thank you very much for including. Attached is an update of the patch > acco

Re: [HACKERS] Checkpoint gets stuck in mdsync

2007-04-05 Thread Heikki Linnakangas
ITAGAKI Takahiro wrote: Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Now that the CheckpointStartLock starvation has been taken care of, I'm seeing another problem with checkpoints in my test run: mdsync never finishes. My proposed fix is to make a copy of pendingOpsTable before entering th

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Simon Riggs
On Thu, 2007-04-05 at 13:59 +0530, NikhilS wrote: > Hi, > > The only problem I have with this is that the shops I know > with big > > partitioned tables favor triggers over rules for both > performance reason and > > a cleaner implementation. Even with auto

Re: [HACKERS] Interaction of PITR backups andBulkoperationsavoiding WAL

2007-04-05 Thread Simon Riggs
On Wed, 2007-04-04 at 22:05 -0400, Bruce Momjian wrote: > I _still_ have no patch for this. Bruce, As I've mentioned, there is another thread where the discussion continued, which you should refer to. The subject of this thread is a potential bug that has existed since 8.1 and that I recently p

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread NikhilS
Hi, > The only problem I have with this is that the shops I know with big > partitioned tables favor triggers over rules for both performance reason and > a cleaner implementation. Even with automated rule creation this isnt going > to change afaics... not to mention we already create our rules

Re: [HACKERS] Auto Partitioning

2007-04-05 Thread Martijn van Oosterhout
On Wed, Apr 04, 2007 at 09:34:03PM +0200, Markus Schiltknecht wrote: > Joshua D. Drake wrote: > >If we don't have multi-table indexes how do we enforce a primary key > >against a partitioned set? > > The executor would have to be clever enough to not do a single index > scan, but possibly scan t