Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I'd like to see a clear explanation of what assumptions are being made >> and why they represent a useful case. > Absolutely agreed there. Just to be clear: I believe our current assumptions can be stated as "Postgres will not los

Re: [HACKERS] XML export

2007-02-10 Thread David Fetter
On Sat, Feb 10, 2007 at 11:35:08AM -0800, Joshua D. Drake wrote: > Peter Eisentraut wrote: > > The issue of XML export has been discussed a few times throughout > > history. Right now you've got the HTML output in psql. A few > > people have proposed "real" XML output formats in psql or elsewhere

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
Tom Lane wrote: > Gene <[EMAIL PROTECTED]> writes: >> ... just my two cents. on a side note, would putting the wal on a >> tmpfs partition give you something similar? > > Indeed, I'm wondering why one needs to hack the Postgres core to throw > away data integrity guarantees; there are plenty of wa

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
Christopher Browne wrote: > [EMAIL PROTECTED] (Hideyuki Kawashima) wrote: >> Joshua, >> >> I appreciate your quick & informative reply. And, I also really >> appreciate your kind comments. Since I have joined this ML 3 hours >> ago, I tried to be polite and slightly nervous. But I was relieved >> b

Re: [HACKERS] [COMMITTERS] pgsql: StrNCpy -> strlcpy (not complete)

2007-02-10 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Woh. Peter, you realize one of the reasons we use StrNCpy as a macro is > for performance. I don't see strlcpy as a macro. Huh? StrNCpy is a wrapper around strncpy(). Do you have reason to think that strncpy() is especially tightly implemented on mos

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Tom Lane
Gene <[EMAIL PROTECTED]> writes: > ... just my two cents. on a side note, would putting the wal on a > tmpfs partition give you something similar? Indeed, I'm wondering why one needs to hack the Postgres core to throw away data integrity guarantees; there are plenty of ways to do that already :-(.

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread J. Andrew Rogers
On Feb 10, 2007, at 9:33 PM, Christopher Browne wrote: The fundamental trouble with this mechanism is that a power outage can instantly turn a database into crud. One may try to mitigate that problem by supporting the memory device with multiple power supplies *and* multiple UPSes. Ask me ab

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Hideyuki Kawashima
Gene, Thanks for your comments ! > On our project, when the power is out, we aren't receiving data anyways... just my two cents. I am sorry, but I do not know how I can appropriately answer since I do not understand what you would like to ask me in this sentence. I understand that you have

Re: [HACKERS] [COMMITTERS] pgsql: StrNCpy -> strlcpy (not complete)

2007-02-10 Thread mark
On Sat, Feb 10, 2007 at 09:21:04PM -0500, Bruce Momjian wrote: > Woh. Peter, you realize one of the reasons we use StrNCpy as a macro is > for performance. I don't see strlcpy as a macro. Are you going to > change all call locations to strlcpy? If so, have you measured the > performance impact?

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Christopher Browne
A long time ago, in a galaxy far, far away, [EMAIL PROTECTED] (Hideyuki Kawashima) wrote: > I appreciate your great suggestion! > It is great honor for me if Sigres will be merged to PostgreSQL. > Since the changes of Sigres from PostgreSQL-8.2.1 are not many, > and moreover, all of changes are su

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Christopher Browne
[EMAIL PROTECTED] (Hideyuki Kawashima) wrote: > Joshua, > > I appreciate your quick & informative reply. And, I also really > appreciate your kind comments. Since I have joined this ML 3 hours > ago, I tried to be polite and slightly nervous. But I was relieved > by your message. Your idea sounds

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
Hideyuki Kawashima wrote: > Joshua, > > I appreciate your great suggestion! > It is great honor for me if Sigres will be merged to PostgreSQL. > Since the changes of Sigres from PostgreSQL-8.2.1 are not many, > and moreover, all of changes are surrounded with #ifdef SIGRES --- #endif, > incorporat

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Hideyuki Kawashima
Joshua, I appreciate your great suggestion! It is great honor for me if Sigres will be merged to PostgreSQL. Since the changes of Sigres from PostgreSQL-8.2.1 are not many, and moreover, all of changes are surrounded with #ifdef SIGRES --- #endif, incorporating Sigres into PostgreSQL would be easy

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Gene
I think it would be great to have this kind of functionality built into postgres (optional and disabled by default of course) I use postgres mainly for its querying and concurrency features (a 10x increase in insert/update speed would be phenomenal) I know most people need 100% data integrity but

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
Hideyuki Kawashima wrote: > Joshua, :) > The reason why I made the Sigres is, the advances of recent non volatile > memories. Just now we do not usually use non volatile memories. But in > the near future, situation would change. I think if a non volatile > memories can be considered as a persist

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Hideyuki Kawashima
Joshua, I appreciate your quick & informative reply. And, I also really appreciate your kind comments. Since I have joined this ML 3 hours ago, I tried to be polite and slightly nervous. But I was relieved by your message. Major topic. The full_page_writes option is already ignored. In Sigres, bg

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
> BTW, Joshua, could you please let me know or give me any pointers for > the reason why fsync=off option exists on PostgreSQL although PostgreSQL A couple of reasons that I can think of. One would be data loads or restoring from backup. Another would be on data that you can afford to throw away.

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Hideyuki Kawashima
Joshua, Thanks for your comments ! Right. As you pointed out, Sigres cannot recover if UPS fails. Therefore, I think currently Sigres cannot be used for mission critical applications. Sigres keeps data integrity *only when* UPS works. On the other hand, some users such as researchers of sensor n

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Joshua D. Drake
Hideyuki Kawashima wrote: > Hello PostgreSQL Hackers, > > I have made a modification of PostgreSQL which accelerates INSERT/UPDATE > using UPS. The name of the software is "Sigres", and the philosophy is > considering a battery supplied memory as a persistent device instead of a > disk. You can

Re: [HACKERS] [COMMITTERS] pgsql: StrNCpy -> strlcpy (not complete)

2007-02-10 Thread Bruce Momjian
Woh. Peter, you realize one of the reasons we use StrNCpy as a macro is for performance. I don't see strlcpy as a macro. Are you going to change all call locations to strlcpy? If so, have you measured the performance impact?

[HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-10 Thread Hideyuki Kawashima
Hello PostgreSQL Hackers, I have made a modification of PostgreSQL which accelerates INSERT/UPDATE using UPS. The name of the software is "Sigres", and the philosophy is considering a battery supplied memory as a persistent device instead of a disk. You can download Sigres from http://sourcefor

Re: [HACKERS] 10 weeks to feature freeze (Pending Work)

2007-02-10 Thread Rick Gigger
Jim Nasby wrote: On Feb 5, 2007, at 12:53 PM, Andrew Hammond wrote: On Jan 26, 2:38 pm, [EMAIL PROTECTED] (Tom Lane) wrote: Rick Gigger <[EMAIL PROTECTED]> writes: I thought that the following todo item just barely missed 8.2: "Allow a warm standby system to also allow read-only statements [pi

Re: [HACKERS] XML export

2007-02-10 Thread Peter Eisentraut
Andrew Dunstan wrote: > How do you treat columns whose names are not legal XML names? There are escape mechanisms in place. You can verify yourself how they work using select xmlelement(name "something unusual"); > I'm glad to see you treat NULL as an attribute - that's definitely > the right

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: >> key constraints loaded from dumps from pre-7.3 databases will no longer >> work. Those dumps just contain "CREATE CONSTRAINT TRIGGER" commands >> which will not provide enough information. We can make the triggers >> throw errors suggesting that th

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Joshua D. Drake
> After digging around in the code for awhile I realized that there's a > potentially bigger backwards-compatibility issue here: if we make the > RI triggers dependent on finding a pg_constraint entry, then foreign > key constraints loaded from dumps from pre-7.3 databases will no longer > work. T

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Tom Lane
Stephan Szabo <[EMAIL PROTECTED]> writes: > On Fri, 9 Feb 2007, Tom Lane wrote: >> I am pretty strongly tempted to stop storing anything in tgargs for RI >> triggers --- it's ugly, and updating the info during RENAME commands >> is a pain in the rear. > I'd say we probably want to keep the tgargs

Re: [HACKERS] XML export

2007-02-10 Thread Andrew Dunstan
Peter Eisentraut wrote: Below are examples of what it can do. I'm thinking about hosting this on PgFoundry, but if the crowd thinks this should be somewhere else, short of the moon, let me know. How do you treat columns whose names are not legal XML names? I'm glad to see you treat N

Re: [HACKERS] XML export

2007-02-10 Thread Stefan Kaltenbrunner
Peter Eisentraut wrote: > The issue of XML export has been discussed a few times throughout > history. Right now you've got the HTML output in psql. A few > people have proposed "real" XML output formats in psql or elsewhere. > > I dug out some old code today that implements what SQL/XML has to

Re: [HACKERS] XML export

2007-02-10 Thread Dave Page
Joshua D. Drake wrote: > Peter Eisentraut wrote: >> The issue of XML export has been discussed a few times throughout >> history. Right now you've got the HTML output in psql. A few >> people have proposed "real" XML output formats in psql or elsewhere. >> >> I dug out some old code today that im

Re: [HACKERS] XML export

2007-02-10 Thread Theo Schlossnagle
On Feb 10, 2007, at 2:35 PM, Joshua D. Drake wrote: Peter Eisentraut wrote: The issue of XML export has been discussed a few times throughout history. Right now you've got the HTML output in psql. A few people have proposed "real" XML output formats in psql or elsewhere. I dug out some old

Re: [HACKERS] XML export

2007-02-10 Thread Joshua D. Drake
Peter Eisentraut wrote: > The issue of XML export has been discussed a few times throughout > history. Right now you've got the HTML output in psql. A few > people have proposed "real" XML output formats in psql or elsewhere. > > I dug out some old code today that implements what SQL/XML has to

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Tom Lane
Stephan Szabo <[EMAIL PROTECTED]> writes: > The reason I was wondering is that it uses tgnargs == 4 as the check, and > if we change the meanings of tgnargs, we'd need to change the check. Sure, it'd be looking for a zero-length conkeys array instead. regards, tom lane --

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Stephan Szabo
On Sat, 10 Feb 2007, Tom Lane wrote: > Stephan Szabo <[EMAIL PROTECTED]> writes: > > One side question is what should we do about the > > places in the current system where it checks for the key sets being empty? > > I don't see that this affects that either way. I can't quite imagine > what the

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Tom Lane
Stephan Szabo <[EMAIL PROTECTED]> writes: > I assume you're speaking of the version where we just change the > constraints to use statements with the OPERATOR() syntax and potential > casts rather than the discussion at the end about changing the pk checks > to avoid planning entirely? Yeah, we mi

[HACKERS] XML export

2007-02-10 Thread Peter Eisentraut
The issue of XML export has been discussed a few times throughout history. Right now you've got the HTML output in psql. A few people have proposed "real" XML output formats in psql or elsewhere. I dug out some old code today that implements what SQL/XML has to say on the matter and fitted the c

Re: [HACKERS] Foreign keys for non-default datatypes, redux

2007-02-10 Thread Stephan Szabo
On Fri, 9 Feb 2007, Tom Lane wrote: > Almost a year ago, we talked about the problem that referential > integrity should be selecting comparison operators on the basis > of b-tree index opclasses, instead of assuming that the appropriate > operator is always named "=": > http://archives.postgresq

Re: [HACKERS] Ooops ... seems we need a re-release pronto

2007-02-10 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Shouldn't we at least add the one or two exemplary statements that > failed so we have some sort of coverage of the problem? We could, but I'm unexcited about it. The known failures are an extremely narrow case: we're trying to evaluate expressions

Re: [HACKERS] Ooops ... seems we need a re-release pronto

2007-02-10 Thread Peter Eisentraut
Tom Lane wrote: > Jim Nasby <[EMAIL PROTECTED]> writes: > > On Feb 6, 2007, at 12:40 PM, Tom Lane wrote: > >> ... massive expansion of the tests doesn't seem justified > > > > What about the idea that's been floated in the past about a -- > > extensive mode for regression testing that would (genera

Re: [HACKERS] [PATCHES] How can I use 2GB of shared buffers on Windows?

2007-02-10 Thread Magnus Hagander
Takayuki Tsunakawa wrote: > From: "Magnus Hagander" <[EMAIL PROTECTED]> >> Right. Which is why you're likely to see better performance if you > keep >> shared buffers smaller. There is something in dealing with it that's >> slow on win32, per reports from the field. It needs to be > investigated >>