Re: [HACKERS] SSI non-serializable UPDATE performance

2011-04-24 Thread Dan Ports
On Sat, Apr 23, 2011 at 08:54:31AM -0500, Kevin Grittner wrote: > Even though this didn't show any difference in Dan's performance > tests, it seems like reasonable insurance against creating a new > bottleneck in very high concurrency situations. > > Dan, do you have a patch for this, or should

Re: [HACKERS] fsync reliability

2011-04-24 Thread Daniel Farina
On Thu, Apr 21, 2011 at 8:51 PM, Greg Smith wrote: > There's still the "fsync'd a data block but not the directory entry yet" > issue as fall-out from this too.  Why doesn't PostgreSQL run into this > problem?  Because the exact code sequence used is this one: > > open > write > fsync > close > >

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Greg Stark
On Sun, Apr 24, 2011 at 11:15 PM, Simon Riggs wrote: > IMHO deleting all the data is a surprising option that will > cause many people to curse us. I don't see preserving some of the data > as being worse. What possible damage to you want to recover from? Without WAL logging after a software cra

Re: [HACKERS] fsync reliability

2011-04-24 Thread Daniel Farina
On Thu, Apr 21, 2011 at 1:26 AM, Simon Riggs wrote: > Daniel Farina points out to me that the Linux man page for fsync() says > "Calling fsync() does not necessarily ensure that the entry in the directory >       containing the file has also reached disk.  For that an > explicit fsync() on a >    

Re: [HACKERS] Some TODO items for collations

2011-04-24 Thread Peter Eisentraut
On lör, 2011-04-23 at 13:17 -0400, Tom Lane wrote: > Should we have a TODO item to find a way of providing > platform-independent collation names? This is a multifold problem. One issue is, if I'm looking for a locale for, say, "English, Canada", I will find it under "en_CA", if it exists at all.

Re: [HACKERS] Collation patch's handling of wcstombs/mbstowcs is sheerest fantasy

2011-04-24 Thread Peter Eisentraut
On lör, 2011-04-23 at 11:37 -0400, Tom Lane wrote: > I wrote: > > * Where they're not, install the locale_t with uselocale(), do > > mbstowcs or wcstombs, and revert to the former locale_t setting. > > This is ugly as sin, and not thread-safe, but of course lots of > > the backend is not thread-saf

Re: [HACKERS] Collation patch's handling of wcstombs/mbstowcs is sheerest fantasy

2011-04-24 Thread Peter Eisentraut
On fre, 2011-04-22 at 16:32 -0400, Tom Lane wrote: > It's possible that things are not too broken in practice, because it's > likely that the transformations done by these functions only depend on > the encoding indicated by LC_CTYPE, and we (try to) enforce that all > locales used in a given datab

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Peter Eisentraut
On sön, 2011-04-24 at 12:25 -0400, Tom Lane wrote: > This file is in fundamental violation of the first commandment of > Postgres #includes, which is "thou shalt have no other gods before c.h". > We need to put postgres.h *before* the Python.h include. I don't know > what issues led to the current

[HACKERS] make check in contrib

2011-04-24 Thread Peter Eisentraut
I noticed again that make check in contrib doesn't work, so here is a patch to fix it. Perhaps someone wants to fill in the Windows support for it. Naturally, this works only for contrib itself, not for external packages that use pgxs. A secondary issue that actually led to this: I was preparin

Re: [HACKERS] Proposed fix for NOTIFY performance degradation

2011-04-24 Thread Simon Riggs
On Sat, Apr 23, 2011 at 8:44 PM, Tom Lane wrote: > Gianni Ciolli writes: >> [ proposes lobotomization of duplicate-elimination behavior in NOTIFY ] > > I think this change is likely to be penny-wise and pound-foolish. > The reason the duplicate check is in there is that things like triggers > may

Re: [HACKERS] Proposed fix for NOTIFY performance degradation

2011-04-24 Thread Simon Riggs
On Sat, Apr 23, 2011 at 2:57 PM, Gianni Ciolli wrote: > * "avg_usec" denotes the average time in microseconds required by each >  NOTIFY statement. > > h2. Test 1 - PostgreSQL HEAD > >   n   |   m   | iter | avg_usec > ---+---+--+-- >    10 |     1 |   10 |   43.730 >   100 |

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Simon Riggs
On Sun, Apr 24, 2011 at 7:41 PM, Greg Stark wrote: > On Sun, Apr 24, 2011 at 6:22 PM, Simon Riggs wrote: >> My implementation path for that would be to add a crash_number onto >> pg_control and pg_index. Any index marked as "unlogged, persistent" >> would only be usable if it's crash number is th

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Christopher Browne
On Sun, Apr 24, 2011 at 6:15 PM, Simon Riggs wrote: > For large sets of low value data, it makes sense. Deleting all data, > just simply because some of it might be damaged, is not the only > option. IMHO deleting all the data is a surprising option that will > cause many people to curse us. I don

Re: [HACKERS] Extension Packaging

2011-04-24 Thread David E. Wheeler
On Apr 24, 2011, at 3:03 PM, Tom Lane wrote: > Yeah. It seems like a bad idea if the distribution "name" doesn't > include sufficient information to tell which version it contains. > I had in mind a convention like "distribution version x.y.z always > contains extension version x.y". Seems like

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Simon Riggs
On Sun, Apr 24, 2011 at 10:27 PM, Tom Lane wrote: > Simon Riggs writes: >> If you don't log changes to tables you have two choices if we crash >> 1) truncate the table and any indexes >> 2) rebuild any indexes damaged by the crash > > No, you have only one choice, and that's (1), because there's

Re: [HACKERS] Extension Packaging

2011-04-24 Thread Tom Lane
"David E. Wheeler" writes: > On Apr 24, 2011, at 2:55 PM, Tom Lane wrote: >> Hmm ... it's sufficient, but I think people are going to be confused as >> to proper usage if you call two different things the "version". In RPM >> terminology there's a clear difference between "version" and "release";

Re: [HACKERS] Extension Packaging

2011-04-24 Thread Daniele Varrazzo
On Sun, Apr 24, 2011 at 10:46 PM, Tom Lane wrote: > Daniele Varrazzo writes: >> On Thu, Apr 21, 2011 at 4:16 AM, Tom Lane wrote: >>> If you did not actually change the contents of the install script, you >>> should not change its version number either. > >> Sorry, I'm not entirely convinced. If

Re: [HACKERS] Extension Packaging

2011-04-24 Thread David E. Wheeler
On Apr 24, 2011, at 2:55 PM, Tom Lane wrote: > Hmm ... it's sufficient, but I think people are going to be confused as > to proper usage if you call two different things the "version". In RPM > terminology there's a clear difference between "version" and "release"; > maybe some similar wording sh

Re: [HACKERS] Extension Packaging

2011-04-24 Thread Tom Lane
"David E. Wheeler" writes: > On Apr 24, 2011, at 2:46 PM, Tom Lane wrote: >> IMO it'd be better if the bug fix level was tracked outside the >> database, for instance via an RPM package version/release number. >> I'm not sure whether PGXN has anything for that at the moment. > Distributions may h

Re: [HACKERS] Extension Packaging

2011-04-24 Thread David E. Wheeler
On Apr 24, 2011, at 2:46 PM, Tom Lane wrote: > IMO it'd be better if the bug fix level was tracked outside the > database, for instance via an RPM package version/release number. > I'm not sure whether PGXN has anything for that at the moment. Distributions may have their own versions independent

Re: [HACKERS] Extension Packaging

2011-04-24 Thread Tom Lane
Daniele Varrazzo writes: > On Thu, Apr 21, 2011 at 4:16 AM, Tom Lane wrote: >> If you did not actually change the contents of the install script, you >> should not change its version number either. > Sorry, I'm not entirely convinced. If I release an extension 1.0, then > find a bug in the C cod

Re: [HACKERS] operator classes for index?

2011-04-24 Thread Tom Lane
=?ISO-8859-15?Q?Yves_Wei=DFig?= writes: > again index access methods, can somebody shed some light into operator > classes for indexes? The documentation is an entry point, but after > reading I still don't have a clue how exactly they are used and created? > Perhaps somebody with great knowledge

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Tom Lane
Simon Riggs writes: > If you don't log changes to tables you have two choices if we crash > 1) truncate the table and any indexes > 2) rebuild any indexes damaged by the crash No, you have only one choice, and that's (1), because there's no guarantee that what's in the table file is meaningful.

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Tom Lane
Andrew Dunstan writes: > On 04/24/2011 09:11 AM, Andrew Chernow wrote: >> The macro is overriding the prototype declared at line 627, which has >> a void argument list (assuming YY_SKIP_YYWRAP is !defined). Since all >> code references to this do not provide an argument, I'd say the macro >> i

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Dunstan
On 04/24/2011 09:11 AM, Andrew Chernow wrote: On 4/24/2011 1:29 AM, Andrew Dunstan wrote: The attached patch is intended to clean up a bunch of compiler warnings seen on Windows due to mismatches of signedness or constness, unused variables, redefined macros and a missing prototype. It doe

Re: [HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Greg Stark
On Sun, Apr 24, 2011 at 6:22 PM, Simon Riggs wrote: > My implementation path for that would be to add a crash_number onto > pg_control and pg_index. Any index marked as "unlogged, persistent" > would only be usable if it's crash number is the same as current > system crash number. > > REINDEX woul

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Dunstan
On 04/24/2011 12:14 PM, Tom Lane wrote: Peter Eisentraut writes: The hunk below looks a bit evil. At least a comment would be good to explain why this is necessary. Yeah, having to cast away const seems uglier than the original problem. Can't we avoid that? I'm not sure how, since the sec

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Dunstan
On 04/24/2011 12:25 PM, Tom Lane wrote: Andrew Dunstan writes: The attached patch is intended to clean up a bunch of compiler warnings seen on Windows due to mismatches of signedness or constness, unused variables, redefined macros and a missing prototype. BTW, this hunk: [snip] is ind

[HACKERS] Unlogged tables, persistent kind

2011-04-24 Thread Simon Riggs
Unlogged tables are a good new feature. I noticed Bruce had mentioned they were the equivalent of NoSQL, which I don't really accept. I guess it depends upon whether you mean NoSQL for caches (e.g. memcached) or NoSQL for useful datastores (e.g. Mongo). It seems worth discussin now before we get t

[HACKERS] operator classes for index?

2011-04-24 Thread Yves Weißig
Hi, again index access methods, can somebody shed some light into operator classes for indexes? The documentation is an entry point, but after reading I still don't have a clue how exactly they are used and created? Perhaps somebody with great knowledge can supply an 101 on opeartor classes? Becau

[HACKERS] offline consistency check and info on attributes

2011-04-24 Thread Tomas Vondra
Hi, a recent discussion about possible data corruption (in the general list) suggests that a tool for offline integrity check might be useful. Something that might be run when the database is shut down, to check that the data files are OK. The idea is to read all heap blocks and check for various

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Tom Lane
Andrew Dunstan writes: > The attached patch is intended to clean up a bunch of compiler warnings > seen on Windows due to mismatches of signedness or constness, unused > variables, redefined macros and a missing prototype. BTW, this hunk: > *** a/src/pl/plpython/plpython.c > --- b/src/pl/plpyt

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Tom Lane
Peter Eisentraut writes: > The hunk below looks a bit evil. > At least a comment would be good to explain why this is necessary. Yeah, having to cast away const seems uglier than the original problem. Can't we avoid that? BTW, all of my machines as well as the Single Unix Spec are agreed that th

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Dunstan
On 04/24/2011 10:53 AM, Peter Eisentraut wrote: On sön, 2011-04-24 at 01:29 -0400, Andrew Dunstan wrote: One thing I'm a bit confused about is this type of warning: src\backend\utils\misc\guc-file.c(977): warning C4003: not enough actual parameters for macro 'GUC_yywrap' If someone

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Dunstan
On 04/24/2011 09:11 AM, Andrew Chernow wrote: On 4/24/2011 1:29 AM, Andrew Dunstan wrote: The attached patch is intended to clean up a bunch of compiler warnings seen on Windows due to mismatches of signedness or constness, unused variables, redefined macros and a missing prototype. It doe

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Peter Eisentraut
The hunk below looks a bit evil. At least a comment would be good to explain why this is necessary. On sön, 2011-04-24 at 01:29 -0400, Andrew Dunstan wrote: > *** a/src/backend/port/win32/socket.c > --- b/src/backend/port/win32/socket.c > *** > *** 370,376 pgwin32_recv(SOCKET s,

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Peter Eisentraut
On sön, 2011-04-24 at 01:29 -0400, Andrew Dunstan wrote: > One thing I'm a bit confused about is this type of warning: > >src\backend\utils\misc\guc-file.c(977): warning C4003: not enough > actual parameters for macro 'GUC_yywrap' > > > If someone can suggest a good fix That would be ni

Re: [HACKERS] "stored procedures"

2011-04-24 Thread Susanne Ebrecht
On 21.04.2011 17:24, Peter Eisentraut wrote: I would like to collect some specs on this feature. So does anyone have links to documentation of existing implementations, or their own spec writeup? A lot of people appear to have a very clear idea of this concept in their own head, so let's start

Re: [HACKERS] windows consolidated cleanup

2011-04-24 Thread Andrew Chernow
On 4/24/2011 1:29 AM, Andrew Dunstan wrote: The attached patch is intended to clean up a bunch of compiler warnings seen on Windows due to mismatches of signedness or constness, unused variables, redefined macros and a missing prototype. It doesn't clean up all the warnings by any means, but it