Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread jtv
> [EMAIL PROTECTED] writes:

> pg_locks certainly seems like a better solution.  Perhaps it didn't
> exist when you went with pg_stat_activity?  Can't recall offhand.

Neither do I...  But I do need something that will work with at least any
recent backend version--say, 7.2 or since.  The more the better, really. 
Any idea how old pg_locks is?


> Note that you still want to look for your old backend's PID; it seems
> impractically expensive to keep track of the current transaction's XID.
> (At a minimum that would cost another query per xact...)

Yes, I see now--I thought I had the transaction ID handy already anyway,
but I didn't.  Thanks.


Jeroen



---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread Peter Eisentraut
John Hansen wrote:
> currently, upper/lower does not work with 2+ byte unicode characters,
> on any OS under the C locale.

Sure it does.  It's just that the defined behavior of the C locale is 
often useless in practice.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] UTF8 or Unicode

2005-02-24 Thread Peter Eisentraut
Bruce Momjian wrote:
> We are not consistent in favoring the
> official names vs. the common names.

The problem is rather that there are too many standards and conventions 
to choose from.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread Peter Eisentraut
Bruce Momjian wrote:
> Oh, sorry.  So there is no ordering in Unicode?

That statement is meaningless.  Unicode is a character set, not a 
collation order.

> No wonder some 
> languages can't use Unicode effectively.

That has nothing to do with it.

> o Disallow encodings like UTF8 which PostgreSQL supports
>   but the operating system does not (already disallowed by
>   pginstaller)

I think the warning that initdb shouts out is already enough for this.  
I don't think we want to disallow this for people who know what they 
are doing.

> I assume C just compares the bytes, meaning equality comparisons are
> fine, but greater/less than is consistent but meaningless.

That statement is independent of whether you use Unicode or something 
else.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] 8.0.X and the ARC patent

2005-02-24 Thread Marc G. Fournier
On Fri, 25 Feb 2005, Bruce Momjian wrote:
Simon Riggs wrote:
On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote:
For development, this means we will _not_ have a shortened, non-initdb
8.1 release but a regular release cycle with the typical big batch of
features.
Might we set a rough date for Beta freeze for 8.1 then?
September 30th 2005 ?
I see only benefit from publishing a not-before date now. It's up to
Core if it slips, but it'll really help with gaining funding if people
can accurately determine whether or not features can be added for
inclusion in the next release. There are lots of potential donors
waiting, so lets give them some certainty about which release their
payback will occur in
Yea, probably September, but you can't dump a huge feature on us in
August either without having talked about it first, so knowing the date
might not be that helpful.
I thought we were looking at a 12-18 month cycle for 8.1?  Which would put 
beta around January '06, no?


Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email: [EMAIL PROTECTED]   Yahoo!: yscrappy  ICQ: 7615664
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faq


Re: [HACKERS] Where are we on stored procedures?

2005-02-24 Thread Neil Conway
Tom Lane wrote:
Essentially I'm thinking about the JDBC solution, but automated a bit
better.
So would your proposal invent a new "stored procedure" construct, or 
just add some sugar to the existing function stuff? i.e. will you be 
able to issue a CREATE FUNCTION that specifies OUT parameters?

This doesn't address the question of SETOF results, of course.  I'm
leaning towards returning those as cursors.
This is part of the reason I liked the approach of introduced SQL-level 
variables. Besides being a feature that has some use in itself, it could 
be extended reasonably cleanly to allow (effectively) SETOF variables 
and rowtype variables.

Well, I think that when people ask us for "stored procedures", most of
them mean that they want transaction control.
Yes, that is certainly what Gavin and I spent most of our time banging 
our heads against the wall on :(

But if you can pass over what you have, I'd like to see about
pressing forward.
Sure, I've attached a very WIP patch with the utility command 
definitions; unfortunately I don't think it will be of much use, as much 
of it is CREATE PROCEDURE-related boilerplate. Gavin will update the 
matching-arguments-by-name code to HEAD at some point in the future; I 
believe that works fine for functions (since we just error out in case 
of ambiguity), so we can include it in 8.1 independently on any other 
work on SPs.

-Neil


spdevel-2.patch.gz
Description: application/gzip

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread Tom Lane
[EMAIL PROTECTED] writes:
> Tom, I believe you said at the time that I should check pg_stat_activity. 
> My current code polls it for the old backend pid.  But if that is neither
> 100% reliable nor unconditionally available, wouldn't it be better if I
> just queried pg_locks for the transaction's ID?  Would that work for all
> backend versions I can expect to see?

pg_locks certainly seems like a better solution.  Perhaps it didn't
exist when you went with pg_stat_activity?  Can't recall offhand.

Note that you still want to look for your old backend's PID; it seems
impractically expensive to keep track of the current transaction's XID.
(At a minimum that would cost another query per xact...)

regards, tom lane

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread jtv
Bruce Momjian wrote:

>> > You can get that from pg_stat_activity, if you have the relevant stats
>> > turned on.
>>
>> pg_stat_activity will tell you about the oldest active query, but not
>> about oldest open transaction.
>
> And pg_stat_activity can lose information when the network is under
> heavy load too.

On a side note, a similar issue came up with libpqxx, in the part that
deals with connections being lost while committing a transaction.  The
library tries to reconnect and figure out whether the commit completed or
not, but it was pointed out that the commit might actually still be in
progress by that time.

Tom, I believe you said at the time that I should check pg_stat_activity. 
My current code polls it for the old backend pid.  But if that is neither
100% reliable nor unconditionally available, wouldn't it be better if I
just queried pg_locks for the transaction's ID?  Would that work for all
backend versions I can expect to see?


Jeroen



---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])


Re: [HACKERS] 8.0.X and the ARC patent

2005-02-24 Thread Bruce Momjian
Simon Riggs wrote:
> On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote:
> > For development, this means we will _not_ have a shortened, non-initdb
> > 8.1 release but a regular release cycle with the typical big batch of
> > features.
> 
> Might we set a rough date for Beta freeze for 8.1 then?
> 
> September 30th 2005 ?
> 
> I see only benefit from publishing a not-before date now. It's up to
> Core if it slips, but it'll really help with gaining funding if people
> can accurately determine whether or not features can be added for
> inclusion in the next release. There are lots of potential donors
> waiting, so lets give them some certainty about which release their
> payback will occur in

Yea, probably September, but you can't dump a huge feature on us in
August either without having talked about it first, so knowing the date
might not be that helpful.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Greg Stark

"Magnus Hagander" <[EMAIL PROTECTED]> writes:

> > I'm a bit surprised that the write-cache lead to a corrupt database, and
> > not merely lost transactions. I had the impression that drives still
> > handled the writes in the order received.
> 
> In this case, it was lost transactions, not data corruption. 
> ...
> A couple of the latest transactions were gone, but the database came up
> in a consistent state, if a bit old.

That's interesting. It would be very interesting to know how reliably this is
true. It could potentially vary depending on the drive firmware.

I can't see any painless way to package up this kind of test for people to run
though. Powercycling machines repeatedly really isn't fun and takes a long
time. And testing this on vmware doesn't buy us anything.

-- 
greg


---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Can we remove SnapshotSelf?

2005-02-24 Thread Bruce Momjian
Tom Lane wrote:
> As of CVS tip, there is no code in the system that uses SnapshotSelf.
> I am wondering if we can get rid of it and thereby save one test in
> the heavily used HeapTupleSatisfiesVisibility() macro.
> 
> There is one place in the foreign-key triggers that uses the underlying
> HeapTupleSatisfiesItself() test directly.  So it seems possible that we
> might someday need SnapshotSelf again as the representation of the
> appropriate snapshot test for a foreign-key check.  But I suspect that
> any future changes in the FK stuff will go in the direction of using
> dirty reads instead.  In any case we could always put back SnapshotSelf
> if we needed it.
> 
> Comments?

I think we should either remove it or mark it as NOT_USED.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] UTF8 or Unicode

2005-02-24 Thread Bruce Momjian
Tatsuo Ishii wrote:
> I do not object the changing UNICODE->UTF-8, but all these discussions
> sound a little bit funny to me.
> 
> If you want to blame UNICODE, you should blame LATIN1 etc. as
> well. LATIN1(ISO-8859-1) is actually a character set name, not an
> encoding name. ISO-8859-1 can be encoded in 8-bit single byte
> stream. But it can be encoded in 7-bit too. So when we refer to
> LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.

Wow, Tatsuo has a point here.  Looking at encnames.c, I see:

"UNICODE", PG_UTF8

but also:

"WIN", PG_WIN1251
"LATIN1", PG_LATIN1

and I see conversions for those:

"iso88591", PG_LATIN1
"win", PG_WIN1251

so I see what he is saying.  We are not consistent in favoring the
official names vs. the common names.

I will work on a patch that people can review and test.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


[HACKERS] Interesting NetBSD annual report

2005-02-24 Thread Bruce Momjian
I read the following report from the NetBSD group:

http://kerneltrap.org/node/4680

It has some interesting points.  First, they analyze how the fit with
other open source database offerings.  Their position is somewhat
similar to ours.

Their development style is also similar to ours.

They complain about release delays and the challenge of organizing
volunteers, just like us.

Second, they have a much more rigid hierarchy structure.  There are good
and bad things about that and you can see it in the report.

Third, they admit they don't have the get*ent_r() functions we need for 
thread-safety on that platform.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c

2005-02-24 Thread Bruce Momjian
Tom Lane wrote:
> Bruce Momjian  writes:
> > Your patch has been added to the PostgreSQL unapplied patches list at:
> 
> Didn't we do that already?

This patch is for thread safety:

> Thanks a lot. The patch attached solves the tread
> safety problem. Please review it before applying,
> I am not sure I am doing the right thing

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c

2005-02-24 Thread Tom Lane
Bruce Momjian  writes:
> Your patch has been added to the PostgreSQL unapplied patches list at:

Didn't we do that already?

regards, tom lane

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c

2005-02-24 Thread Bruce Momjian

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---


Nicolai Tufar wrote:
> > On Mon, Feb 21, 2005 at 10:53:08PM -0500, Bruce Momjian wrote:
> >
> > Applied.
> 
> Thanks a lot. The patch attached solves the tread
> safety problem. Please review it before applying, 
> I am not sure I am doing the right thing
> 
> 
> On Tue, 22 Feb 2005 19:57:15 +0100, Kurt Roeckx <[EMAIL PROTECTED]> wrote:
> > The configure test is a little broken.  It needs to quote the
> > $'s.
> > 
> > I've rewritten the test a little.
> 
> This one needs applying too. $'s do get scrambled.
> 
> Best regards, 
> Nicolai.

[ Attachment, skipping... ]

> 
> ---(end of broadcast)---
> TIP 7: don't forget to increase your free space map settings

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Where are we on stored procedures?

2005-02-24 Thread Tom Lane
Gavin Sherry <[EMAIL PROTECTED]> writes:
> On Thu, 24 Feb 2005, Tom Lane wrote:
>> For instance, a procedure foo(x IN int, y OUT text, z OUT float)
>> could perhaps be called via
>> SELECT y, z FROM foo(42);
>> where foo(x) is seen as returning the rowtype (y text, z float).

> The composite type stuff is precisely what the JDBC driver does at the
> moment. I guess the feeling is that for those used to SPs in other
> databases, having to use composite types is a bit of a hack.

Yeah, but only because you have to do it explicitly.  I was wondering
whether we couldn't bury that mechanism under the hood.  (In particular,
given the improved support in 8.0 for anonymous record types, we could
in theory have the backend invent a record type on-the-fly to match
whatever list of OUT parameters a particular function has.)

>> (Gavin and Neil's first proposal also involved inventing a concept of
>> variables at the SQL level, primarily so that there would be something
>> to receive the results of OUT parameters.  I found that distasteful and
>> would prefer to avoid it.

> I'd like to hear what you had in mind for OUT parameters.

Essentially I'm thinking about the JDBC solution, but automated a bit
better.  You do
SELECT * FROM function(a,b,c);
where only the IN (including INOUT) parameters are listed in the call,
and the OUT (including INOUT) parameters make up a result record type
that doesn't need to be explicitly declared.  I don't know yet what's
the cleanest way to handle this in terms of what appears in pg_proc.

If you insist we can allow "SELECT * FROM" to be spelled "CALL" or
some such, but I think I'd prefer to keep that notation in reserve
for "real" stored procedures.

This doesn't address the question of SETOF results, of course.  I'm
leaning towards returning those as cursors.

> This is one of the real tough issues Neil and I were trying to work out. I
> seem to remember you noting that without transaction control, SPs were
> irrelevant :-).

Well, I think that when people ask us for "stored procedures", most of
them mean that they want transaction control.  But the JDBC issues that
my Red Hat compatriots are currently worried about just have to do with
OUT parameters in a CallableStatement, so I'd like to make sure we solve
that in 8.1 regardless of whether anyone makes any progress on
outside-of-transactions stored procedures.

> As you've seen internally at Red Hat, the OUT parameter stuff is a real
> show stopper for the JDBC guys. It would be good to see this in 8.1 but
> I'm not sure when either Neil or I will find some time to look at it.

Rats, I was hoping you'd answer you had it about done ;-).  But if you
can pass over what you have, I'd like to see about pressing forward.

regards, tom lane

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread Bruce Momjian
Tom Lane wrote:
> Christopher Kings-Lynne <[EMAIL PROTECTED]> writes:
> >> It sure would be nice to be able to have a way to query the start time
> >> of the eldest transaction on the system.  If that could be done at a
> >> not-too-high cost, it would be eminently helpful for various sorts of
> >> maintenance processes so that you could assortedly:
> 
> > You can get that from pg_stat_activity, if you have the relevant stats 
> > turned on.
> 
> pg_stat_activity will tell you about the oldest active query, but not
> about oldest open transaction.

And pg_stat_activity can lose information when the network is under
heavy load too.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread John Hansen
> currently, upper/lower does not work with 2+ byte unicode 
> characters, on any OS under the C locale.

Btw,...

There are only 15 cases in the utf8 repertoire that depends on locale, these 
are the only cases where pg should report:

ERROR:  invalid multibyte character for locale
HINT:  The server's LC_CTYPE locale is probably incompatible with the database 
encoding.

When doing a select upper/lower (col)
All others should work just fine.

The error should probably also be changed to a warning, and just return the 
offending character unmodified.


... John

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread Tom Lane
"John Hansen" <[EMAIL PROTECTED]> writes:
> Right,. So if that's fixed, then UTF8 will work only on windows?

No.

> (currently, upper/lower does not work with 2+ byte unicode characters, on any 
> OS)

This information is obsolete.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread John Hansen
K, let me rephrase:

currently, upper/lower does not work with 2+ byte unicode characters, on any OS 
under the C locale.

... John

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Bruce Momjian
Neil Conway wrote:
> Bruce Momjian wrote:
> > And what would the TODO item be?  "Improve psql's handling of multi-line
> > queries" is too vague.
> 
> If you can include a link to the archives or the text of the relevant 
> mails, it seems fine to me. I'm not sure specifically _how_ we want to 
> improve the handling of multi-line queries, and it seems silly to make 
> that decision now just for the sake of a specific TODO item description.

OK, I will just add the wording.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Neil Conway
Bruce Momjian wrote:
And what would the TODO item be?  "Improve psql's handling of multi-line
queries" is too vague.
If you can include a link to the archives or the text of the relevant 
mails, it seems fine to me. I'm not sure specifically _how_ we want to 
improve the handling of multi-line queries, and it seems silly to make 
that decision now just for the sake of a specific TODO item description.

-Neil
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread John Hansen
>   To fix UTF8, the data needs to be converted to 
> UTF16 and then
>   the Win32 wcscoll() can be used, and perhaps other functions
>   like towupper().  However, UTF8 already works with normal
>   locales but provides no ordering.

Right,. So if that's fixed, then UTF8 will work only on windows?
(currently, upper/lower does not work with 2+ byte unicode characters, on any 
OS)

... John


---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Bruce Momjian
Neil Conway wrote:
> Bruce Momjian wrote:
> > Is there a TODO here?
> 
> Probably -- I think there is definitely room for improving psql's 
> handling of multi-line queries. However, \e works well enough for me, 
> and I don't think I'll get around to looking at this for 8.1. So feel 
> free to add a TODO item.

And what would the TODO item be?  "Improve psql's handling of multi-line
queries" is too vague.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32

2005-02-24 Thread Bruce Momjian
Magnus Hagander wrote:
> The installer does not permit it, but initdb lets you do anything yuo
> want - I think that's where we are. If you know what you're doing, you
> can use it by manually initdbing.
> 
> There is no such thing as "unicode locale". Unicode (UTF8) is an
> encoding, that has to be paired with a locale. I assume you mean C
> locale. 

Oh, sorry.  So there is no ordering in Unicode?  No wonder some
languages can't use Unicode effectively.  I can see why ordering is
meaningless for creating a document that is just displayed but important
for a database.

I have added the last sentence to the TODO list:

o Disallow encodings like UTF8 which PostgreSQL supports
  but the operating system does not (already disallowed by
  pginstaller)

  To fix UTF8, the data needs to be converted to UTF16 and then
  the Win32 wcscoll() can be used, and perhaps other functions
  like towupper().  However, UTF8 already works with normal
  locales but provides no ordering.

> 
> While UPPER/LOWER does not matter, sort order does - for indexes if
> nothing else. I'm unsure if this works - I think I read reports about
> itn ot working, but I haven't tried it out myself.

I assume C just compares the bytes, meaning equality comparisons are
fine, but greater/less than is consistent but meaningless.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Where are we on stored procedures?

2005-02-24 Thread Gavin Sherry
Hi Tom,

On Thu, 24 Feb 2005, Tom Lane wrote:

> Gavin and Neil made some noise in late September about implementing
> stored procedures for PG 8.1, but I haven't heard anything more about
> it since that thread died off.  I've been getting some pressure inside
> Red Hat to see us support more of the JDBC CallableProcedure spec, so
> I'd like to reopen the discussion.
>
> In the previous discussion starting here:
> http://archives.postgresql.org/pgsql-hackers/2004-09/msg00702.php
> it seemed that we were bandying around several different issues.
> People wanted "procedures" to differ from our current implementation
> of "functions" in such ways as:
>
> 1. Executing outside the database engine, and therefore being able to
> start/commit transactions.  (This is *fundamentally* different from our
> current concept of functions, and I think that any implementation that
> tries to gloss over the difference will be doomed to failure.)

We have some ideas on this but haven't made in progress in terms of code.
The main reason that Josh and others were arguing strongly for transaction
level control within SPs was that they wanted to be able to VACUUM,
CLUSTER, etc from within an SP.

>
> 2. Having OUT parameters, and perhaps also INOUT parameters.  At least
> in the JDBC spec, these are seen as scalar values, and so the feature
> is really just syntactic sugar for returning a composite type or row
> value.  For instance, a procedure foo(x IN int, y OUT text, z OUT float)
> could perhaps be called via
>   SELECT y, z FROM foo(42);
> where foo(x) is seen as returning the rowtype (y text, z float).

Again, ideas and design but not too much code. We got the parameter modes
into the grammar, stored them in a new system catalog, and some other
minor stuff.

The composite type stuff is precisely what the JDBC driver does at the
moment. I guess the feeling is that for those used to SPs in other
databases, having to use composite types is a bit of a hack.

>
> 3. Being able to return multiple result sets, ie, more than one SETOF
> something.  In our previous discussion we tied this to OUT parameters
> but they're not necessarily the same thing --- the JDBC spec sees result
> sets as totally different objects.
>
> 4. Not having a distinguished function result, a/k/a returning void.
> While a function result is unnecessary given OUT parameters, this feels
> like an accidental thing rather than an essential aspect.
>
> 5. Matching parameters by name instead of by position.
>

This is kind of independent of SPs, I have some code which implements it
but I need to bring it up to HEAD.

> 6. Allowing parameters to be omitted from a call, with default values
> used instead.

Some code on this too.

>
> #5 and #6 would also be interesting for regular functions, but it's
> unclear how well we can support them without disallowing overloading
> of procedure/function names --- which of course is a nonstarter for
> the existing function facility.

We can support 5 for functions but not 6.

>
> (Gavin and Neil's first proposal also involved inventing a concept of
> variables at the SQL level, primarily so that there would be something
> to receive the results of OUT parameters.  I found that distasteful and
> would prefer to avoid it.  Another thing that came up was allowing a
> procedure -- in one or more of these senses -- to be used as a trigger,
> but I think that's a red herring.  None of the above attributes are
> particularly relevant to a trigger.)

I'd like to hear what you had in mind for OUT parameters. Oracle and IBM
have host variables which is more or less what we were looking at but SQL
server just returns a result set as if it were a function. Strangely
enough, it can return multiple result sets with different descriptors (ie,
different row types).

I think there is some merit in supporting procedures with triggers but I'm
not sure its necessary for a first attempt.

>
> On looking at this list, it seems to me that #1 (transactionality) is
> largely orthogonal to the other points, which all have to do with
> passing and returning values.  The main reason why we might consider
> all these things together is that no one is very excited about the idea
> of having even more than two kinds of objects, so there is a lot of
> temptation to press for having all these features in "procedures"
> rather than recognize that they are largely separable needs.
>
> The more I think about it, the more I think that #1 belongs outside the
> database entirely.  The database engine more or less has to operate
> entirely within transactions --- heck, we cannot even look up a stored
> procedure's definition in a system catalog without starting a
> transaction.  This is not to say that the facility can't exist
> physically within the backend, but that it would work a lot better if
> it weren't logically inside the database.  What about inventing a
> protocol facility that lets clients send a chunk of, say, Perl or
> Python code to execute in 

Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Neil Conway
Bruce Momjian wrote:
Is there a TODO here?
Probably -- I think there is definitely room for improving psql's 
handling of multi-line queries. However, \e works well enough for me, 
and I don't think I'll get around to looking at this for 8.1. So feel 
free to add a TODO item.

-Neil
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] [NOVICE] Question on TRUNCATE privleges

2005-02-24 Thread Tom Lane
"Keith Worthington" <[EMAIL PROTECTED]> writes:
> On Thu, 24 Feb 2005 17:15:42 -0500, Tom Lane wrote
>> Yeah.  I've dropped the idea personally -- the suggestion that the table
>> owner can provide a SECURITY DEFINER procedure to do the TRUNCATE if 
>> he wants to allow others to do it seems to me to cover the problem.

> Could someone point me in the direction of documentation on this SECURITY
> DEFINER feature?

See CREATE FUNCTION.  Something like (untested)

create function truncate_my_table() returns void as
$$ truncate my_table $$ language sql security definer;

You'd probably then revoke the default public EXECUTE rights on this
function, and grant EXECUTE only to selected users.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Some download statistics

2005-02-24 Thread Marc G. Fournier
On Thu, 24 Feb 2005, Tom Lane wrote:
"Marc G. Fournier" <[EMAIL PROTECTED]> writes:
On Thu, 24 Feb 2005, Tom Lane wrote:
Also: I notice that the README file that's supposed to tell people about
the split-tarball scheme is not present in any of the recent-version
subdirectories, so it's no wonder that they are confused.

Actually, its never  been ... just  checked, its in the root directory ...
but, with that in mind, I've added putting a copy in there to the release
script ...
It is present in the subdirectories for some older releases, eg v7.3.
'k, now I see it ... I was looking for a README file .. fixed ...

Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email: [EMAIL PROTECTED]   Yahoo!: yscrappy  ICQ: 7615664
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faq


Re: [HACKERS] [JDBC] Where are we on stored procedures?

2005-02-24 Thread Tom Lane
"Francisco Figueiredo Jr." <[EMAIL PROTECTED]> writes:
> Could I add another item?

> Could we have the row count of statements executed inside a
> procedure/function returned to client?

IMHO that request is completely bogus; if the procedure wants to tell
the client that, it's the procedure's responsibility to return the
number as a result.  Doing what you ask (a) would arguably be a security
violation, and (b) the info would be impossible for the client to
interpret in any but the most trivial cases anyway.  Consider for
instance a procedure that contains different queries in the THEN and
the ELSE arms of an IF, or contains loops executed a variable number
of times.  Without access to the results of the control flow tests, the
client could not know which number means what.

regards, tom lane

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Some download statistics

2005-02-24 Thread Marc G. Fournier
On Thu, 24 Feb 2005, Magnus Hagander wrote:
Frankly, I'd suggest dropping the splits. Thoughts?

I also found the split sources + a non-split sources version to be
confusing. As you, I think that splitting should be dropped.
Perhaps the confusion issue could be addressed by keeping the split
sources in a separate subdirectory:
pub/source/v.8.0.1/
postgresql-8.0.1.tar.bz2
postgresql-8.0.1.tar.bz2.md5
postgresql-8.0.1.tar.gz
postgresql-8.0.1.tar.gz.md5
split-tarballs/
postgresql-base-8.0.1.tar.bz2
postgresql-base-8.0.1.tar.bz2.md5
etc...
That sounds like a reasonable compromise ... I could do that
right away,
if nobody is in disagreement ... ?
Seems reasonable to me.

I suspect the demand for the split tarballs is mighty low nowaways,
but it's probably not zero yet.
Considering that every new FreeBSD install uses them
exclusively, I don't
expect it to drop to zero in the near future ..
Do they pull fromt eh pg mirrors, or from their own? I realise they
still need it, of course, but it'd be interesting to know.
MASTER_SITES=   ${MASTER_SITE_PGSQL}
which expands to:
MASTER_SITE_PGSQL+= \
ftp://ftp3.us.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp8.us.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp9.us.postgresql.org/pub/mirrors/postgresql/%SUBDIR%/ \
ftp://ftp10.us.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp13.us.postgresql.org/mirror/postgresql/%SUBDIR%/ \
${MASTER_SITE_RINGSERVER:S,%SUBDIR%,misc/db/postgresql/&,} \
ftp://ftp.au.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.at.postgresql.org/db/www.postgresql.org/pub/%SUBDIR%/ \
ftp://ftp.be.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp.ba.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.br.postgresql.org/pub/PostgreSQL/%SUBDIR%/ \
ftp://ftp3.ca.postgresql.org/pub/%SUBDIR%/ \
ftp://ftp2.cr.postgresql.org/pub/Unix/postgres/%SUBDIR%/ \
ftp://ftp.cz.postgresql.org/pub/ftp.postgresql.org/%SUBDIR%/ \
ftp://ftp2.cz.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.ee.postgresql.org/mirrors/postgresql/%SUBDIR%/ \
ftp://ftp.fr.postgresql.org/%SUBDIR%/ \
ftp://ftp2.fr.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp.de.postgresql.org/mirror/postgresql/%SUBDIR%/ \

ftp://ftp2.de.postgresql.org/pub/comp/os/unix/database/postgresql/%SUBDIR%/ \
ftp://ftp3.de.postgresql.org/pub/Mirrors/ftp.postgresql.org/%SUBDIR%/ \
ftp://ftp4.de.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.gr.postgresql.org/pub/databases/postgresql/%SUBDIR%/ \
ftp://ftp.hk.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp2.is.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.ie.postgresql.org/mirrors/ftp.postgresql.org/pub/%SUBDIR%/ \
ftp://ftp2.it.postgresql.org/mirrors/postgres/%SUBDIR%/ \
ftp://ftp.kr.postgresql.org/mirror/database/postgresql/%SUBDIR%/ \
ftp://ftp.lv.postgresql.org/pub/software/postgresql/%SUBDIR%/ \
ftp://ftp.eu.postgresql.org/pub/unix/db/postgresql/%SUBDIR%/ \
ftp://ftp2.nl.postgresql.org/mirror/postgresql/%SUBDIR%/ \
ftp://ftp4.nl.postgresql.org/postgresql.zeelandnet.nl/%SUBDIR%/ \
ftp://ftp.no.postgresql.org/pub/databases/postgresql/%SUBDIR%/ \
ftp://ftp6.pl.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp7.pl.postgresql.org/pub/mirror/ftp.postgresql.org/%SUBDIR%/ \
ftp://ftp.pt.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp6.ro.postgresql.org/pub/mirrors/ftp.postgresql.org/%SUBDIR%/ \
ftp://ftp.ru.postgresql.org/pub/unix/database/pgsql/%SUBDIR%/ \
ftp://ftp2.ru.postgresql.org/pub/databases/postgresql/%SUBDIR%/ \
ftp://ftp3.ru.postgresql.org/pub/mirror/postgresql/pub/%SUBDIR%/ \
ftp://ftp5.es.postgresql.org/mirror/postgresql/%SUBDIR%/ \

ftp://ftp.se.postgresql.org/pub/databases/relational/postgresql/%SUBDIR%/ \
ftp://ftp2.ch.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.tw.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp3.tw.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.tr.postgresql.org/pub/SQL/postgresql/%SUBDIR%/ \
ftp://ftp.postgresql.org/pub/%SUBDIR%/

Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email: [EMAIL PROTECTED]   Yahoo!: yscrappy  ICQ: 7615664
---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] [PATCHES] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c

2005-02-24 Thread Bruce Momjian
Tom Lane wrote:
> Kurt Roeckx <[EMAIL PROTECTED]> writes:
> > The configure test is a little broken.  It needs to quote the
> > $'s.
> 
> > I've rewritten the test a little.
> 
> Applied, thanks.

Oops, Tom got to it first.  (Darn!)  :-)

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c

2005-02-24 Thread Bruce Momjian

Oh, thanks.  That is a great fix.  Applied.  Glad you could test it on a
machine that supports positional parameters.

---

Kurt Roeckx wrote:
> On Mon, Feb 21, 2005 at 10:53:08PM -0500, Bruce Momjian wrote:
> > 
> > Applied.
> 
> The configure test is a little broken.  It needs to quote the
> $'s.
> 
> I've rewritten the test a little.
> 
> 
> Kurt
> 

[ Attachment, skipping... ]

> 
> ---(end of broadcast)---
> TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] [NOVICE] Question on TRUNCATE privleges

2005-02-24 Thread Bruce Momjian
Tom Lane wrote:
> Bruce Momjian  writes:
> > Uh, that seems like it adds extra complexity just for this single case.
> 
> Yeah.  I've dropped the idea personally -- the suggestion that the table
> owner can provide a SECURITY DEFINER procedure to do the TRUNCATE if he
> wants to allow others to do it seems to me to cover the problem.
> 
> > Why don't we allow TRUNCATE by non-owners only if no triggers are
> > defined, and if they are defined, we throw an error and mention it is
> > because triggers/contraints exist?
> 
> I don't think we should put weird special cases in the rights checking
> to allow this -- that's usually a recipe for introducing unintended
> security holes.

Yea, good point.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Some download statistics

2005-02-24 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes:
> On Thu, 24 Feb 2005, Tom Lane wrote:
>> Also: I notice that the README file that's supposed to tell people about
>> the split-tarball scheme is not present in any of the recent-version
>> subdirectories, so it's no wonder that they are confused.

> Actually, its never  been ... just  checked, its in the root directory ... 
> but, with that in mind, I've added putting a copy in there to the release 
> script ...

It is present in the subdirectories for some older releases, eg v7.3.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [NOVICE] Question on TRUNCATE privleges

2005-02-24 Thread Tom Lane
Bruce Momjian  writes:
> Uh, that seems like it adds extra complexity just for this single case.

Yeah.  I've dropped the idea personally -- the suggestion that the table
owner can provide a SECURITY DEFINER procedure to do the TRUNCATE if he
wants to allow others to do it seems to me to cover the problem.

> Why don't we allow TRUNCATE by non-owners only if no triggers are
> defined, and if they are defined, we throw an error and mention it is
> because triggers/contraints exist?

I don't think we should put weird special cases in the rights checking
to allow this -- that's usually a recipe for introducing unintended
security holes.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] [NOVICE] Question on TRUNCATE privleges

2005-02-24 Thread Bruce Momjian
Thomas Hallgren wrote:
> > It looks to me like the asymmetry between CREATE TRIGGER and DROP
> > TRIGGER is actually required by SQL99, though, so changing it would
> > be a hard sell (unless SQL2003 fixes it?).
> > 
> > Comments anyone?
> > 
> Why not say that TRUNCATE requires the same privilige as a DELETE and 
> add a trigger type that fires (once) on a TRUNCATE? That would give an 
> owner a chance to prevent it. Such a trigger would probably be useful 
> for other things too.

Uh, that seems like it adds extra complexity just for this single case.

Why don't we allow TRUNCATE by non-owners only if no triggers are
defined, and if they are defined, we throw an error and mention it is
because triggers/contraints exist?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] [JDBC] Where are we on stored procedures?

2005-02-24 Thread Francisco Figueiredo Jr.
 --- Tom Lane <[EMAIL PROTECTED]> escreveu: 
> Gavin and Neil made some noise in late September about implementing
> stored procedures for PG 8.1, but I haven't heard anything more about
> it since that thread died off.  I've been getting some pressure inside
> Red Hat to see us support more of the JDBC CallableProcedure spec, so
> I'd like to reopen the discussion.
> 


I don't know if I can talk about it here, but here I go. Please let me know if
it is not appropriate.

> In the previous discussion starting here:
> http://archives.postgresql.org/pgsql-hackers/2004-09/msg00702.php
> it seemed that we were bandying around several different issues.
> People wanted "procedures" to differ from our current implementation
> of "functions" in such ways as:
> 
> 
> 2. Having OUT parameters, and perhaps also INOUT parameters.  At least
> in the JDBC spec, these are seen as scalar values, and so the feature
> is really just syntactic sugar for returning a composite type or row
> value.  For instance, a procedure foo(x IN int, y OUT text, z OUT float)
> could perhaps be called via
>   SELECT y, z FROM foo(42);
> where foo(x) is seen as returning the rowtype (y text, z float).
> 

That would be very good.


Could I add another item?

Could we have the row count of statements executed inside a procedure/function
returned to client?

I know Tom Lane, that you already talked about that this could be unnecessary,
but we from Npgsql sometimes get reports about the support of needing to know
the number of rows affected by a function/procedure call.

You told me about the get diagnostics ROWCOUNT and it worked like a charm, but
I was thinking about something without the need to change the procedure, as
even this modification needs some aware of client to be able to get the result.

Please, jdbc guys, if you know some easy way of doing this, please let me know
so I can implement in Npgsql.



Thanks in advance.

Regards,

Francisco Figueiredo Jr.


__
Converse com seus amigos em tempo real com o Yahoo! Messenger 
http://br.download.yahoo.com/messenger/ 

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Some download statistics

2005-02-24 Thread Dave Page
 

> -Original Message-
> From: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] On Behalf Of John DeSoi
> Sent: 24 February 2005 19:20
> To: Magnus Hagander
> Cc: pgsql-hackers@postgresql.org; [EMAIL PROTECTED]
> Subject: Re: [HACKERS] Some download statistics
> 
> Hi Magnus,
> 
> On Feb 24, 2005, at 11:35 AM, Magnus Hagander wrote:
> 
> > I did some simple pivoting in Excel and split it into 
> categories win32,
> > source, sig (MD5 or PGP signatures), RPMs, split (the split 
> tarballs),
> > pgadmin and ODBC. Other stuff was so little that I cut it.
> 
> 
> Assuming this is from a HTTP log, 

It's not. When you click on a mirror flag after selecting a file under
http://www.postgresql.org/ftp/ the click is logged and the client
redirected to the requested file on the selected mirror.

/D

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Magnus Hagander
>> * Linux, with fsync (default), write-cache enabled: usually no data
>> corruption, but two runs which had
>
>Are you verifying that all the data that was committed was 
>actually stored? Or
>just verifying that the database works properly after rebooting?

I verified the data.


>I'm a bit surprised that the write-cache lead to a corrupt 
>database, and not
>merely lost transactions. I had the impression that drives 
>still handled the
>writes in the order received.

In this case, it was lost transactions, not data corruption. Should be
more careful. I had copy/pasted the "no data corruption", should specify
what was lost.

A couple of the latest transactions were gone, but the database came up
in a consistent state, if a bit old.

Since Linux wasn't the stuff I actually was testing, I didn't run very
many tests on it though.

//Magnus

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Magnus Hagander
>> You may find that if you check this case again that the 
>"usually no data
>> corruption" is actually "usually lost transactions but no 
>corruption".
>
>That's a good point, but it seems difficult to be sure of the last
>reportedly-committed transaction in a powerfail situation.  Maybe if
>you drive the test from a client on another machine?

FYI, that's what I did. Test client ran across the network to the
server, so it could output on the console which transaction was last
reported commityted.

In a couple of cases, the server came up with a transaction the client
had *not* reported as committed. But I think that can be explained by
the commit message not reaching the client over the network before power
went out.

//Magnus

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Some download statistics

2005-02-24 Thread Magnus Hagander
 Frankly, I'd suggest dropping the splits. Thoughts?
>>
>>> I also found the split sources + a non-split sources version to be
>>> confusing. As you, I think that splitting should be dropped.
>>
>> Perhaps the confusion issue could be addressed by keeping the split
>> sources in a separate subdirectory:
>>
>>  pub/source/v.8.0.1/
>>  postgresql-8.0.1.tar.bz2
>>  postgresql-8.0.1.tar.bz2.md5
>>  postgresql-8.0.1.tar.gz
>>  postgresql-8.0.1.tar.gz.md5
>>  split-tarballs/
>>  postgresql-base-8.0.1.tar.bz2
>>  postgresql-base-8.0.1.tar.bz2.md5
>>  etc...
>
>That sounds like a reasonable compromise ... I could do that 
>right away, 
>if nobody is in disagreement ... ?

Seems reasonable to me.


>> I suspect the demand for the split tarballs is mighty low nowaways,
>> but it's probably not zero yet.
>
>Considering that every new FreeBSD install uses them 
>exclusively, I don't 
>expect it to drop to zero in the near future ..

Do they pull fromt eh pg mirrors, or from their own? I realise they
still need it, of course, but it'd be interesting to know.


//Magnus

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Some download statistics

2005-02-24 Thread Ernst Herzberg
On Thursday 24 February 2005 21:43, Marc G. Fournier wrote:
[...]
> >
> > pub/source/v.8.0.1/
> > postgresql-8.0.1.tar.bz2
> > postgresql-8.0.1.tar.bz2.md5
> > postgresql-8.0.1.tar.gz
> > postgresql-8.0.1.tar.gz.md5
> > split-tarballs/
> > postgresql-base-8.0.1.tar.bz2
> > postgresql-base-8.0.1.tar.bz2.md5
> > etc...
>
> That sounds like a reasonable compromise ... I could do that right away,
> if nobody is in disagreement ... ?

Hm, there is a distribution that uses the split tarballs: gentoo.

snipped from /usr/portage/dev-db/postgresql/postgresql-8.0.1-r1.ebuild :

SRC_URI="mirror://postgresql/source/v${PV}/${PN}-base-${MY_PV}.tar.bz2
mirror://postgresql/source/v${PV}/${PN}-opt-${MY_PV}.tar.bz2
doc? ( mirror://postgresql/source/v${PV}/${PN}-docs-${MY_PV}.tar.bz2 )"

Better is to drop a 'bug' to bugs.gentoo.org so that the ebuild will be 
fixed;-)




---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] psql: recall previous command?

2005-02-24 Thread Thomas F.O'Connell
One interesting artifact of using \e to edit a multi-line command is 
that the same command is then treated as a single-line command in 
subsequent up-arrow or Ctrl-P attempts.

I use this frequently to achieve a similar effect to what you're after. 
The one downside is that if you leave the psql session, it's returned 
to multi-line format in the psql history.

-tfo
--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC
http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005d
On Feb 21, 2005, at 10:26 PM, Neil Conway wrote:
Is there a way to recall the previous command in psql? Obviously, "up 
arrow" or Ctrl-P using readline and the default readline bindings is 
close, but it recalls the previous _line_ of input. That is not at all 
the same thing in the case of a multiple line command, for example.

If there is no way at present, I think there should be. Using "up 
arrow"
is quite annoying when dealing with multiple line SQL statements.

Two issues:
- how to handle slash commands? Slash commands and SQL statements can 
be intermixed:

neilc=# select 1 \timing
Timing is off.
neilc-# ;
 ?column?
--
1
(1 row)
So I'm not quite sure what the right behavior here is. We could always 
just ignore slash commands (the command would "recall the previous SQL 
statement") -- since few slash commands are multi-line, I don't think 
that would be too bad.

- when a multiple-line command is recalled, how should it be presented 
in the psql input buffer? Perhaps we could strip newlines from 
recalled command text, so that the recalled command would fit on a 
single line. That would mean the recalled command would look somewhat 
different than how the user typed it, although of course the semantics 
of the query should be the same. Any better ideas?

-Neil
---(end of 
broadcast)---
TIP 8: explain analyze is your friend

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Some download statistics

2005-02-24 Thread Marc G. Fournier
On Thu, 24 Feb 2005, Tom Lane wrote:
Troels Arvin <[EMAIL PROTECTED]> writes:
On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote:
Frankly, I'd suggest dropping the splits. Thoughts?

I also found the split sources + a non-split sources version to be
confusing. As you, I think that splitting should be dropped.
Perhaps the confusion issue could be addressed by keeping the split
sources in a separate subdirectory:
pub/source/v.8.0.1/
postgresql-8.0.1.tar.bz2
postgresql-8.0.1.tar.bz2.md5
postgresql-8.0.1.tar.gz
postgresql-8.0.1.tar.gz.md5
split-tarballs/
postgresql-base-8.0.1.tar.bz2
postgresql-base-8.0.1.tar.bz2.md5
etc...
That sounds like a reasonable compromise ... I could do that right away, 
if nobody is in disagreement ... ?

I suspect the demand for the split tarballs is mighty low nowaways,
but it's probably not zero yet.
Considering that every new FreeBSD install uses them exclusively, I don't 
expect it to drop to zero in the near future ..

Also: I notice that the README file that's supposed to tell people about
the split-tarball scheme is not present in any of the recent-version
subdirectories, so it's no wonder that they are confused.
Actually, its never  been ... just  checked, its in the root directory ... 
but, with that in mind, I've added putting a copy in there to the release 
script ...


Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email: [EMAIL PROTECTED]   Yahoo!: yscrappy  ICQ: 7615664
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [HACKERS] Some download statistics

2005-02-24 Thread Tom Lane
Troels Arvin <[EMAIL PROTECTED]> writes:
> On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote:
>> Frankly, I'd suggest dropping the splits. Thoughts?

> I also found the split sources + a non-split sources version to be
> confusing. As you, I think that splitting should be dropped.

Perhaps the confusion issue could be addressed by keeping the split
sources in a separate subdirectory:

pub/source/v.8.0.1/
postgresql-8.0.1.tar.bz2
postgresql-8.0.1.tar.bz2.md5
postgresql-8.0.1.tar.gz
postgresql-8.0.1.tar.gz.md5
split-tarballs/
postgresql-base-8.0.1.tar.bz2
postgresql-base-8.0.1.tar.bz2.md5
etc...

I suspect the demand for the split tarballs is mighty low nowaways,
but it's probably not zero yet.

Also: I notice that the README file that's supposed to tell people about
the split-tarball scheme is not present in any of the recent-version
subdirectories, so it's no wonder that they are confused.

regards, tom lane

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Some download statistics

2005-02-24 Thread Troels Arvin
On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote:

> I know several people who downloaded source *plus* the split ones,
> because "hey, I need postgresql. And I certainly need base too. And I
> need docs.". They don't realise it's included in the main tarball.
> Frankly, I'd suggest dropping the splits. Thoughts?

I also found the split sources + a non-split sources version to be
confusing. As you, I think that splitting should be dropped.

-- 
Greetings from Troels Arvin, Copenhagen, Denmark



---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


[HACKERS] Where are we on stored procedures?

2005-02-24 Thread Tom Lane
Gavin and Neil made some noise in late September about implementing
stored procedures for PG 8.1, but I haven't heard anything more about
it since that thread died off.  I've been getting some pressure inside
Red Hat to see us support more of the JDBC CallableProcedure spec, so
I'd like to reopen the discussion.

In the previous discussion starting here:
http://archives.postgresql.org/pgsql-hackers/2004-09/msg00702.php
it seemed that we were bandying around several different issues.
People wanted "procedures" to differ from our current implementation
of "functions" in such ways as:

1. Executing outside the database engine, and therefore being able to
start/commit transactions.  (This is *fundamentally* different from our
current concept of functions, and I think that any implementation that
tries to gloss over the difference will be doomed to failure.)

2. Having OUT parameters, and perhaps also INOUT parameters.  At least
in the JDBC spec, these are seen as scalar values, and so the feature
is really just syntactic sugar for returning a composite type or row
value.  For instance, a procedure foo(x IN int, y OUT text, z OUT float)
could perhaps be called via
SELECT y, z FROM foo(42);
where foo(x) is seen as returning the rowtype (y text, z float).

3. Being able to return multiple result sets, ie, more than one SETOF
something.  In our previous discussion we tied this to OUT parameters
but they're not necessarily the same thing --- the JDBC spec sees result
sets as totally different objects.

4. Not having a distinguished function result, a/k/a returning void.
While a function result is unnecessary given OUT parameters, this feels
like an accidental thing rather than an essential aspect.

5. Matching parameters by name instead of by position.

6. Allowing parameters to be omitted from a call, with default values
used instead.

#5 and #6 would also be interesting for regular functions, but it's
unclear how well we can support them without disallowing overloading
of procedure/function names --- which of course is a nonstarter for
the existing function facility.

(Gavin and Neil's first proposal also involved inventing a concept of
variables at the SQL level, primarily so that there would be something
to receive the results of OUT parameters.  I found that distasteful and
would prefer to avoid it.  Another thing that came up was allowing a
procedure -- in one or more of these senses -- to be used as a trigger,
but I think that's a red herring.  None of the above attributes are
particularly relevant to a trigger.)

On looking at this list, it seems to me that #1 (transactionality) is
largely orthogonal to the other points, which all have to do with
passing and returning values.  The main reason why we might consider
all these things together is that no one is very excited about the idea
of having even more than two kinds of objects, so there is a lot of
temptation to press for having all these features in "procedures"
rather than recognize that they are largely separable needs.

The more I think about it, the more I think that #1 belongs outside the
database entirely.  The database engine more or less has to operate
entirely within transactions --- heck, we cannot even look up a stored
procedure's definition in a system catalog without starting a
transaction.  This is not to say that the facility can't exist
physically within the backend, but that it would work a lot better if
it weren't logically inside the database.  What about inventing a
protocol facility that lets clients send a chunk of, say, Perl or
Python code to execute in an interpreter that can in turn send commands
to the DB engine proper?  The point here is that that interpreter is
wrapped around the DB engine, not vice versa as occurs when executing a
plperl or plpython function.  (The only real difference between this
idea and just executing the same code on the client side is avoiding
network round trips.)

BTW, using plpgsql in this fashion is a nonstarter, at least with
anything resembling its current implementation.  Because plpgsql relies
on the database engine to do even simple expression evaluation, it's
just hopeless to think of it doing anything useful outside a
transaction.  But we have plenty of external programming languages
available that are perfectly capable of doing their own arithmetic and
logic, and so could meaningfully drive the database engine through a
series of transactions.

Having said all that, I don't have any personal interest in pursuing #1
immediately.  (Though anyone who does is welcome to.)  What I would
like to see is some forward movement on the other points, particularly
#2 which is blocking my Red Hat coworkers from making progress.  So the
real bottom line to this overly long email is that I'd like to create
a consensus that it's OK to work on #2 and perhaps #3 in the context of
our existing function facility, without tackling #1.  This'd involve
work in both the server and the JDBC driver.

Comments

Re: [HACKERS] Some download statistics

2005-02-24 Thread John DeSoi
Hi Magnus,
On Feb 24, 2005, at 11:35 AM, Magnus Hagander wrote:
I did some simple pivoting in Excel and split it into categories win32,
source, sig (MD5 or PGP signatures), RPMs, split (the split tarballs),
pgadmin and ODBC. Other stuff was so little that I cut it.

Assuming this is from a HTTP log, I'm just curious if you took into 
account that many Windows users have "download managers" that make 
repeated requests to get a single file. You see this in the log with a 
206 (partial content) HTTP response code. So you might have anywhere 
from 10 to 20 hits in the log to download a single file. This makes it 
a pain to get accurate download statistics.

John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faq


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread Vsevolod Lobko
On Thu, Feb 24, 2005 at 11:14:07AM -0500, Tom Lane wrote:
> Christopher Kings-Lynne <[EMAIL PROTECTED]> writes:
> >> It sure would be nice to be able to have a way to query the start time
> >> of the eldest transaction on the system.  If that could be done at a
> >> not-too-high cost, it would be eminently helpful for various sorts of
> >> maintenance processes so that you could assortedly:
> 
> > You can get that from pg_stat_activity, if you have the relevant stats 
> > turned on.
> 
> pg_stat_activity will tell you about the oldest active query, but not
> about oldest open transaction.

You can get list of currently runing transactions from pg_locks table, 
but no start time...
But if you can remember oldest transaction_id from the last vacuum then 
you got what you need: if oldest transaction still here - you not 
need to vacuum

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Greg Stark

Tom Lane <[EMAIL PROTECTED]> writes:

> Greg Stark <[EMAIL PROTECTED]> writes:
> > I'm a bit surprised that the write-cache lead to a corrupt database, and not
> > merely lost transactions. I had the impression that drives still handled the
> > writes in the order received.
> 
> There'd be little point in having a cache if they did, I should think.
> I thought the point of the cache was to allow the disk to schedule I/O
> in an order that minimizes seek time (ie, such a disk has got its own
> elevator queue or similar).

If that were the case then SCSI drives that ship with write caching disabled
and using tagged command queuing instead would perform poorly.

I think the main motivation for write caching on IDE drives is that the IDE
protocol forces commands to be issued synchronously. So you can't send a
second command until the first command has completed. Without write caching
that limits the write bandwidth tremendously. Write caching is being used here
as a poor man's tcq.

-- 
greg


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes:
> I'm a bit surprised that the write-cache lead to a corrupt database, and not
> merely lost transactions. I had the impression that drives still handled the
> writes in the order received.

There'd be little point in having a cache if they did, I should think.
I thought the point of the cache was to allow the disk to schedule I/O
in an order that minimizes seek time (ie, such a disk has got its own
elevator queue or similar).

> You may find that if you check this case again that the "usually no data
> corruption" is actually "usually lost transactions but no corruption".

That's a good point, but it seems difficult to be sure of the last
reportedly-committed transaction in a powerfail situation.  Maybe if
you drive the test from a client on another machine?

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] [ADMIN] invalid multibyte character for locale

2005-02-24 Thread Tom Lane
Bjoern Metzdorf <[EMAIL PROTECTED]> writes:
> I assume I could just remove
> #define USE_WIDE_UPPER_LOWER
> from oracle_compat.c to emulate the old behaviour. But a cleaner fix 
> would be to check if we are using UNICODE and locale is C or POSIX and 
> only then skip USE_WIDE_UPPER_LOWER.

Perhaps it would be reasonable to do something like this:

#ifdef USE_WIDE_UPPER_LOWER
/*
 * use wide char code only when max encoding length > one
 * and we aren't in C locale
 */
if (pg_database_encoding_max_length() > 1 &&
!lc_ctype_is_c())
{

where lc_ctype_is_c() is the obvious clone of the existing 
lc_collate_is_c() routine.  We can reasonably assume that mbstowcs
is going to be unable to offer any useful behavior in C locale.

Comments?

regards, tom lane

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Greg Stark

"Magnus Hagander" <[EMAIL PROTECTED]> writes:

> * Linux, with fsync (default), write-cache enabled: usually no data
> corruption, but two runs which had

Are you verifying that all the data that was committed was actually stored? Or
just verifying that the database works properly after rebooting?

I'm a bit surprised that the write-cache lead to a corrupt database, and not
merely lost transactions. I had the impression that drives still handled the
writes in the order received.

You may find that if you check this case again that the "usually no data
corruption" is actually "usually lost transactions but no corruption".

-- 
greg


---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Some download statistics

2005-02-24 Thread Marc G. Fournier
On Thu, 24 Feb 2005, Magnus Hagander wrote:
3) There doesnt' seem to be much point to the distribution splits. A
total of less than 5% the *number* of downloads. And most people
probably get more than one file, so in reality that number shuold
proably be divided by 4 or 5.
I know several people who downloaded source *plus* the split ones,
because "hey, I need postgresql. And I certainly need base too. And I
need docs.". They don't realise it's included in the main tarball.
Frankly, I'd suggest dropping the splits. Thoughts?
this tracks only those going in through the web ... this doesn't track 
those using an ftp client going in and downloading files ... specifically, 
I know that all of the FreeBSD ports are based on the split distributions 
... in the case of the postgresql80-server port, it only downloads base 
and opt and ignores the rest ...


Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email: [EMAIL PROTECTED]   Yahoo!: yscrappy  ICQ: 7615664
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faq


[HACKERS] Some download statistics

2005-02-24 Thread Magnus Hagander
(crossposting this to hackers, I'm sure there are interested people
there as well)

Since Dave set the site up for tracking clickthroughs, I hit the db with
a couple of queries to count our downloads. This is what I came up with:

I did some simple pivoting in Excel and split it into categories win32,
source, sig (MD5 or PGP signatures), RPMs, split (the split tarballs),
pgadmin and ODBC. Other stuff was so little that I cut it.

The numbers for just-past-release may be off a bit because of the move
of wwwmaster. Not sure what happened to the db.

Just before and about 24 hours after the release of 8.0, we had the
following:
win32   16480
source  5223
sig 2016
rpm 1908
split   1689
pgadmin 370
odbc261
Grand Total 27947


Total tally since the tracking started (2005-01-11):
win32   106878  56,37%
source  39058   20,60%
rpm 11703   6,17%
split   91354,82%
sig 89314,71%
pgadmin 72813,84%
odbc66013,48%
Total   189587


And the versions being downloaded (total):
8.0.1   85511
8.0.0   82517
notpg   14526   (pgadmin, odbc etc)
7.4.6   4297
7.4.7   2736
Grand Total 189587


This does *not* include bittorrent downloads.

Some conclusions that I draw from this:
1) The win32 distribution is very popular. Some of this is no doubt
because it's new. Also, about 3500 of those downloads are RC5 downloads.
RC5 downloads of the source were during this time so few that they
didn't show up after my cutoff.

2) Most of the people who use RPMs on their linux systems wait for their
distribution to ship the RPMs and don't get it. Or they are alraedy set
up to download directly from their mirror and not from the website (the
tracker only counts downloads that were initiated from the website)

3) There doesnt' seem to be much point to the distribution splits. A
total of less than 5% the *number* of downloads. And most people
probably get more than one file, so in reality that number shuold
proably be divided by 4 or 5. 
I know several people who downloaded source *plus* the split ones,
because "hey, I need postgresql. And I certainly need base too. And I
need docs.". They don't realise it's included in the main tarball.
Frankly, I'd suggest dropping the splits. Thoughts?

4) People upgraded to 8.0.1. Not a lot of people have upgraded their 7.4
systems.


As always, it's statistics. You can say anything you want with it...
Someone who chews numbers for a living might be able to get more out of
it. If someone is interested in the excel pivot tables with this stuff
in it, let me know. But it's nothing exciting there :)


//Magnus

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread Tom Lane
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes:
>> It sure would be nice to be able to have a way to query the start time
>> of the eldest transaction on the system.  If that could be done at a
>> not-too-high cost, it would be eminently helpful for various sorts of
>> maintenance processes so that you could assortedly:

> You can get that from pg_stat_activity, if you have the relevant stats 
> turned on.

pg_stat_activity will tell you about the oldest active query, but not
about oldest open transaction.

regards, tom lane

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync

2005-02-24 Thread pgsql
> "Magnus Hagander" <[EMAIL PROTECTED]> writes:
>> My results are:
>> Fisrt, baseline:
>> * Linux, with fsync (default), write-cache disabled: no data corruption
>> * Linux, with fsync (default), write-cache enabled: usually no data
>> corruption, but two runs which had
>
> That makes sense.
>
>> * Win32, with fsync, write-cache disabled: no data corruption
>> * Win32, with fsync, write-cache enabled: no data corruption
>> * Win32, with osync, write cache disabled: no data corruption
>> * Win32, with osync, write cache enabled: no data corruption. Once I
>> got:
>> 2005-02-24 12:19:54 LOG:  could not open file "C:/Program
>> Files/PostgreSQL/8.0/data/pg_xlog/00010010" (log file 0,
>> segment 16): No such file or directory
>>   but the data in the database was consistent.
>
> It disturbs me that you couldn't produce data corruption in the cases
> where it theoretically should occur.  Seems like this is an indication
> that your test was insufficiently severe, or that there is something
> going on we don't understand.
>
I was thinking about that. A few years back, Microsoft had some serious
issues with write caching drives. They were taken to task for losing data
if Windows shut down too fast, especially on drives with a large cache.

MS is big enough and bad enough to get all the info they need from the
various drive makers to know how to handle write cache flushing. Even the
stuff that isn't documented.

If anyone has a very good debugger and/or emulator or even a logic
analyzer, it would be interesting to see if MS sends commands to the
drives after a disk write or a set of disk writes.

Also, I would like to see this test performed on NTFS and FAT32, and see
if you are more likely to lose data on FAT32.

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Magnus Hagander
> > * Win32, with fsync, write-cache disabled: no data corruption
> > * Win32, with fsync, write-cache enabled: no data corruption
> > * Win32, with osync, write cache disabled: no data corruption
> > * Win32, with osync, write cache enabled: no data corruption. Once I
> > got:
> > 2005-02-24 12:19:54 LOG:  could not open file "C:/Program 
> > Files/PostgreSQL/8.0/data/pg_xlog/00010010" 
> (log file 
> > 0, segment 16): No such file or directory
> >   but the data in the database was consistent.
> 
> It disturbs me that you couldn't produce data corruption in 
> the cases where it theoretically should occur.  Seems like 
> this is an indication that your test was insufficiently 
> severe, or that there is something going on we don't understand.

The Windows driver knows abotu the write cache, and at least fsync()
pushes through the write cache even if it's there. This seems to
indicate taht O_SYNC at least partiallyi does this as well. This is why
there is no performance difference at all on fsync() with write cache on
or off.

I don't know if this is true for all IDE disks. COuld be that my disk is
particularly well-behaved.

//Magnus

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes:
> My results are:
> Fisrt, baseline:
> * Linux, with fsync (default), write-cache disabled: no data corruption
> * Linux, with fsync (default), write-cache enabled: usually no data
> corruption, but two runs which had

That makes sense.

> * Win32, with fsync, write-cache disabled: no data corruption
> * Win32, with fsync, write-cache enabled: no data corruption
> * Win32, with osync, write cache disabled: no data corruption
> * Win32, with osync, write cache enabled: no data corruption. Once I
> got:
> 2005-02-24 12:19:54 LOG:  could not open file "C:/Program
> Files/PostgreSQL/8.0/data/pg_xlog/00010010" (log file 0,
> segment 16): No such file or directory
>   but the data in the database was consistent.

It disturbs me that you couldn't produce data corruption in the cases
where it theoretically should occur.  Seems like this is an indication
that your test was insufficiently severe, or that there is something
going on we don't understand.

regards, tom lane

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Christopher Kings-Lynne
My results are:
Fisrt, baseline:
* Linux, with fsync (default), write-cache disabled: no data corruption
* Linux, with fsync (default), write-cache enabled: usually no data
corruption, but two runs which had
* Win32, with fsync, write-cache disabled: no data corruption
* Win32, with fsync, write-cache enabled: no data corruption
* Win32, with osync, write cache disabled: no data corruption
* Win32, with osync, write cache enabled: no data corruption. Once I
got:
2005-02-24 12:19:54 LOG:  could not open file "C:/Program
Files/PostgreSQL/8.0/data/pg_xlog/00010010" (log file 0,
segment 16): No such file or directory
In case anyone is wondering, you can turn off write caching on FreeBSD, 
for a terrible perfomance loss...

http://freebsd.active-venture.com/handbook/configtuning-disk.html#AEN8015
Chris
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
 joining column's datatypes do not match


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Christopher Kings-Lynne
In the final test, the BIOS decided the disk was giving up and
reassigned it as 0Mb.. Required two extra cold boots, then it was back
up to 20Gb. Still no data loss.
I think it would be fun to re-run these tests with MySQL...
Chris
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [pgsql-hackers-win32] [HACKERS] win32 performance - fsync question

2005-02-24 Thread Magnus Hagander
> > Magnus prepared a trivial patch which added the O_SYNC flag for 
> > windows and mapped it to FILE_FLAG_WRITE_THROUGH in win32_open.c.
> 
> Attached is this trivial patch. As Merlin says, it needs some 
> more reliability testing. But the numbers are at least reasonable - it
> *seems* like it's doing the right thing (as long as you turn 
> off write cache). And it's certainly a significant 
> performance increase - it brings the speed almost up to the 
> same as linux.

I have now run a bunch of pull-the-plug testing on this patch (literally
pulling the plug, yes. to the point of some of my co-workers thinking
I'm crazy)

My results are:
Fisrt, baseline:
* Linux, with fsync (default), write-cache disabled: no data corruption
* Linux, with fsync (default), write-cache enabled: usually no data
corruption, but two runs which had
* Win32, with fsync, write-cache disabled: no data corruption
* Win32, with fsync, write-cache enabled: no data corruption
* Win32, with osync, write cache disabled: no data corruption
* Win32, with osync, write cache enabled: no data corruption. Once I
got:
2005-02-24 12:19:54 LOG:  could not open file "C:/Program
Files/PostgreSQL/8.0/data/pg_xlog/00010010" (log file 0,
segment 16): No such file or directory

  but the data in the database was consistent.

Almost all runs showed a line along the line:
2005-02-24 11:22:41 LOG:  record with zero length at 0/A450548


In the final test, the BIOS decided the disk was giving up and
reassigned it as 0Mb.. Required two extra cold boots, then it was back
up to 20Gb. Still no data loss.


My tests was three clients doing lots of inserts and updates, some in
transactions some bare. In some tests, I kicked in a manual vacuum while
at it. Then I yanked the powercord, rebooted, manually started pg, and
verified taht the data in the db came up with the same values the cliens
reported as last committed. I also ran vacuum verbose on all tables
after it was back up to see if there were any warnings.

Test machine is a 1GHz Celeron, 256Mb RAM and a Maxtor IDE disk.

It'd of course be good if others could also test, but I'm getting the
feeling that this patch at least doesn't make things worse than before
:-) ANd it's *a lot* faster.

//Magnus

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] SQL99 Hierarchical queries

2005-02-24 Thread Evgen Potemkin
> > I have done initial implementation of SQL99 WITH clause (attached).
> > It's now only for v7.3.4 and haven't a lot of checks and restrictions.
> 
> What kind of restrictions are on it?
Main restriction is that the query inside WITH alias can refer only to
back and to itself.
For example 
WITH a as (...),b as (...)select ...;
a can refer only to itself, b - to a and b.

Its needed to restrict WITH aliased query not to use WITH aliasing
itself, i.e. avoid situation of WITH inside WITH. And make also some
other checks. But this isn't done yet.

Regards, Evgen.

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] SQL99 Hierarchical queries

2005-02-24 Thread Christopher Kings-Lynne
I have done initial implementation of SQL99 WITH clause (attached).
It's now only for v7.3.4 and haven't a lot of checks and restrictions.
What kind of restrictions are on it?
Chris
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
   (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])


Re: [HACKERS] Finding if old transactions are running...

2005-02-24 Thread Christopher Kings-Lynne
It sure would be nice to be able to have a way to query the start time
of the eldest transaction on the system.  If that could be done at a
not-too-high cost, it would be eminently helpful for various sorts of
maintenance processes so that you could assortedly:
You can get that from pg_stat_activity, if you have the relevant stats 
turned on.

Chris
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly