Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Albe Laurenz
Tom Lane wrote: Would it be an option to have a checksum somewhere in each data block that is verified upon read? That's been proposed before and rejected before. See the archives ... I searched for checksum and couldn't find it. Could someone give me a pointer? I'm not talking about WAL

Re: [HACKERS] Segmentation fault using digest from pg_crypto

2007-08-27 Thread Marko Kreen
On 8/24/07, Manuel Sugawara [EMAIL PROTECTED] wrote: Tom Lane [EMAIL PROTECTED] writes: Manuel Sugawara [EMAIL PROTECTED] writes: Marko Kreen [EMAIL PROTECTED] writes: In 8.0 the pgcrypto functions were non-strict and checked for NULLs. In 8.1 they were made STRICT. In 8.2 the NULL

Re: [HACKERS] Buildfarm failures MSVC

2007-08-27 Thread Magnus Hagander
On Fri, Aug 24, 2007 at 09:08:18PM -0400, Tom Lane wrote: I wrote: Magnus Hagander [EMAIL PROTECTED] writes: Just so people know, I'm fixing those (tsearch-created make check failures on msvc). It's been a bad week this week at work, but I hope to get around to fix these beginning of next

[HACKERS] initdb failed on Windows 2000

2007-08-27 Thread Yoshiyuki Asaba
Hi, I have compiled PostgreSQL 8.2.4 with MinGW on Windows 2000. Then I have executed initdb as Administrator. However initdb failed with the following message. The program postgres is needed by initdb but was not found in the same directory as C:\msys\1.0\local\pgsql\bin/initdb. Check your

[HACKERS] anoncvs update schedule

2007-08-27 Thread Magnus Hagander
Hello! A point of information to those scheduling jobs based on the anoncvs server: Since the move of the cvs server a couple of weeks back, we've replaced the hourly sync job from anoncvs to a push replication system that runs automatically on every commit. This means that there will now

[HACKERS] Problem with locks

2007-08-27 Thread Gregory Stark
Well the problem with spurious waits for deadlocks turned out to be quite mundane. The deadlocks were not so spurious after all... there were real deadlocks which were being hidden by a plpgsql function that caught the exception and didn't report the error. I think this is still a minor problem

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: I wrote: Florian G. Pflug [EMAIL PROTECTED] writes: Yeah - I do not really like that dual-locking thing either. But it makes prepared transaction handling much easier - if we were to only lock the RID, we'd have to store the rid-xid mapping for prepared transactions Hmmm

Re: [HACKERS] [GENERAL] LDAP service lookup

2007-08-27 Thread Albe Laurenz
Dawid Kuroczko wrote: Then again, apart from libpq I don't see it mentioned anywhere. [...] Looking at the 8.3devel documentation... I think it should be mentioned in 18. Server Configuration. probably somewhere in 18.3 Connections and Authentication, that there is a possibility of using

[HACKERS] MSVC build system

2007-08-27 Thread Andrew Dunstan
I want to overhaul the MSVC build system somewhat and want to discuss my plans. The first thing I want to do is get rid of at least most of the .bat files in it altogether and make a single coherent perl-based system. The reasons are: . the XP_EXIT_FIX hack we built in to cater for the XP

[HACKERS] Problem with recent permission changes commits

2007-08-27 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I just saw this in the weekly news: Restrict pg_relation_size to relation owner, pg_database_size to DB owner, and pg_tablespace_size to superusers. Perhaps we could weaken the first case to just require SELECT privilege, but that doesn't

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Tom Lane
Albe Laurenz [EMAIL PROTECTED] writes: Tom Lane wrote: Would it be an option to have a checksum somewhere in each data block that is verified upon read? That's been proposed before and rejected before. See the archives ... I searched for checksum and couldn't find it. Could someone give

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Jonah H. Harris
On 8/27/07, Tom Lane [EMAIL PROTECTED] wrote: that and the lack of evidence that they'd actually gain anything I find it somewhat ironic that PostgreSQL strives to be fairly non-corruptable, yet has no way to detect a corrupted page. The only reason for not having CRCs is because it will slow

Re: [HACKERS] Problem with recent permission changes commits

2007-08-27 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Joshua D. Drake wrote: Hello, I just saw this in the weekly news: Restrict pg_relation_size to relation owner, pg_database_size to DB owner, and pg_tablespace_size to superusers. Perhaps we could weaken the first case to just require SELECT

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Trevor Talbot
On 8/27/07, Jonah H. Harris [EMAIL PROTECTED] wrote: On 8/27/07, Tom Lane [EMAIL PROTECTED] wrote: that and the lack of evidence that they'd actually gain anything I find it somewhat ironic that PostgreSQL strives to be fairly non-corruptable, yet has no way to detect a corrupted page. The

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Sounds good, if we decide to go with the transient XID idea. So below for an alternative that I just came up with. This proposal appears to require taking and releasing a brand-new lock type every time a snapshot is made or destroyed. That is

Re: [HACKERS] Undetected corruption of table files

2007-08-27 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Albe Laurenz [EMAIL PROTECTED] writes: Tom Lane wrote: Would it be an option to have a checksum somewhere in each data block that is verified upon read? That's been proposed before and rejected before. See the archives ... I searched for checksum and

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Alban Hertroys
Jonah H. Harris wrote: On 8/27/07, Tom Lane [EMAIL PROTECTED] wrote: that and the lack of evidence that they'd actually gain anything I find it somewhat ironic that PostgreSQL strives to be fairly non-corruptable, yet has no way to detect a corrupted page. The only reason for not having

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Tom Lane
Trevor Talbot [EMAIL PROTECTED] writes: On 8/27/07, Jonah H. Harris [EMAIL PROTECTED] wrote: I find it somewhat ironic that PostgreSQL strives to be fairly non-corruptable, yet has no way to detect a corrupted page. But how does detecting a corrupted data page gain you any durability? All it

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Jonah H. Harris
On 8/27/07, Tom Lane [EMAIL PROTECTED] wrote: Indeed. In fact, the most likely implementation of this (refuse to do anything with a page with a bad CRC) would be a net loss from that standpoint, because you couldn't get *any* data out of a page, even if only part of it had been zapped. At

Re: [HACKERS] Final background writer cleanup for 8.3

2007-08-27 Thread Kevin Grittner
On Sun, Aug 26, 2007 at 7:35 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Sun, Aug 26, 2007 at 4:16 PM, in message [EMAIL PROTECTED], Greg Smith [EMAIL PROTECTED] wrote: I'll try to get some more meaningful numbers tomorrow. Well, I ran the query against

Re: [HACKERS] MSVC build system

2007-08-27 Thread Neil Conway
On Mon, 2007-08-27 at 11:00 -0400, Andrew Dunstan wrote: In the longer run I want to make the whole system more data driven, so that it's comparatively easy for someone to add stuff. I don't mean to hijack your thread, but I wonder if maintaining two separate build systems is the best approach

Re: [HACKERS] [COMMITTERS] pgsql: Fix brain fade in DefineIndex(): it was continuing to access the

2007-08-27 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: [redirected to -hackers] Tom Lane wrote: Found by trying to run the regression tests with CLOBBER_CACHE_ALWAYS enabled. Maybe we should try to do that on a regular basis --- it's awfully slow, but perhaps some fast buildfarm machine could do it once

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: Sounds good, if we decide to go with the transient XID idea. So below for an alternative that I just came up with. This proposal appears to require taking and releasing a brand-new lock type every time a snapshot is made or destroyed.

Re: [HACKERS] [COMMITTERS] pgsql: Fix brain fade in DefineIndex(): it was continuing to access the

2007-08-27 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: [redirected to -hackers] Tom Lane wrote: Found by trying to run the regression tests with CLOBBER_CACHE_ALWAYS enabled. Maybe we should try to do that on a regular basis --- it's awfully slow, but perhaps some fast

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: What do you think about solving the requirements of the *first* waiting phase (Where we wait for current ShareLock holders) inside the lock manager? The only real downside I can see is that I feel uneasy about messing with that code... It seems to be

Re: [HACKERS] [COMMITTERS] pgsql: Fix brain fade in DefineIndex(): it was continuing to access the

2007-08-27 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: This particular issue could be implemented just by adding -DCLOBBER_CACHE_ALWAYS to CFLAGS (or CPPFLAGS if you want to be anal about it). I suppose that no new buildfarm mechanism is required --- someone just needs to set up an animal

Re: [HACKERS] MSVC build system

2007-08-27 Thread Andrew Dunstan
Neil Conway wrote: On Mon, 2007-08-27 at 11:00 -0400, Andrew Dunstan wrote: In the longer run I want to make the whole system more data driven, so that it's comparatively easy for someone to add stuff. I don't mean to hijack your thread, but I wonder if maintaining two separate

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: What do you think about solving the requirements of the *first* waiting phase (Where we wait for current ShareLock holders) inside the lock manager? The only real downside I can see is that I feel uneasy about messing with that code...

Re: [HACKERS] MSVC build system

2007-08-27 Thread Alvaro Herrera
Andrew Dunstan wrote: Neil Conway wrote: I don't mean to hijack your thread, but I wonder if maintaining two separate build systems is the best approach in the long term. I think CMake[1] is an interesting alternative: it would allow us to generate both makefiles and MSVC .proj's from a

Re: [HACKERS] Problem with recent permission changes commits

2007-08-27 Thread Decibel!
On Mon, Aug 27, 2007 at 08:08:34AM -0700, Joshua D. Drake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I just saw this in the weekly news: Restrict pg_relation_size to relation owner, pg_database_size to DB owner, and pg_tablespace_size to superusers. Perhaps we could

Re: [HACKERS] MSVC build system

2007-08-27 Thread David Boreham
Neil Conway wrote: On Mon, 2007-08-27 at 11:00 -0400, Andrew Dunstan wrote: In the longer run I want to make the whole system more data driven, so that it's comparatively easy for someone to add stuff. I don't mean to hijack your thread, but I wonder if maintaining two separate build

Re: [HACKERS] [GENERAL] Undetected corruption of table files

2007-08-27 Thread Decibel!
On Mon, Aug 27, 2007 at 12:08:17PM -0400, Jonah H. Harris wrote: On 8/27/07, Tom Lane [EMAIL PROTECTED] wrote: Indeed. In fact, the most likely implementation of this (refuse to do anything with a page with a bad CRC) would be a net loss from that standpoint, because you couldn't get *any*

Re: [HACKERS] Problem with recent permission changes commits

2007-08-27 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Decibel! wrote: On Mon, Aug 27, 2007 at 08:08:34AM -0700, Joshua D. Drake wrote: This is a problem. Our analytics software purposefully does not use a super user, you are going to force the use of superusers with admin and monitoring tools.

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Josh Berkus
Tom, dblink_get_pkey() will tell you what its primary key is. This is not a big deal seeing that you can get the info by looking into the system catalogs, but someone who was trying to lock down the system catalogs wouldn't be happy. I think this should require SELECT privilege on the

Re: [HACKERS] Problem with recent permission changes commits

2007-08-27 Thread Decibel!
On Mon, Aug 27, 2007 at 11:59:05AM -0700, Joshua D. Drake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Decibel! wrote: On Mon, Aug 27, 2007 at 08:08:34AM -0700, Joshua D. Drake wrote: This is a problem. Our analytics software purposefully does not use a super user, you are

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: pgrowlocks tells you about row lock states, which maybe is not that interesting for security, but still it's information that one wouldn't expect to be exposed to someone who isn't allowed to read the table. I suppose knowing the number of live tuples

Re: [HACKERS] MSVC build system

2007-08-27 Thread Magnus Hagander
Alvaro Herrera wrote: Andrew Dunstan wrote: Neil Conway wrote: I don't mean to hijack your thread, but I wonder if maintaining two separate build systems is the best approach in the long term. I think CMake[1] is an interesting alternative: it would allow us to generate both makefiles

Re: [HACKERS] MSVC build system

2007-08-27 Thread Magnus Hagander
David Boreham wrote: Neil Conway wrote: On Mon, 2007-08-27 at 11:00 -0400, Andrew Dunstan wrote: In the longer run I want to make the whole system more data driven, so that it's comparatively easy for someone to add stuff. I don't mean to hijack your thread, but I wonder if

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Josh Berkus
Tom, That's OK for individual tables, but we have no equivalent concept for whole databases or tablespaces. What do you propose for them? CONNECT, of course. There are no permissions for tablespaces except CREATE, so presumably if the user can connect to the db, they ought to be able to

Re: [HACKERS] MSVC build system

2007-08-27 Thread Andrew Dunstan
Magnus Hagander wrote: IIRC, we previously looked into cmake and concluded it supported a lot fewer platforms than pgsql. However, if we can go by Alvaros suggestion and keep the makefiles as derived files, that could certainly work... But everyone would still need to learn it. Maybe

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Tom, That's OK for individual tables, but we have no equivalent concept for whole databases or tablespaces. What do you propose for them? CONNECT, of course. There are no permissions for tablespaces except CREATE, so presumably if the user can connect

Re: [HACKERS] MSVC build system

2007-08-27 Thread Magnus Hagander
Andrew Dunstan wrote: Magnus Hagander wrote: IIRC, we previously looked into cmake and concluded it supported a lot fewer platforms than pgsql. However, if we can go by Alvaros suggestion and keep the makefiles as derived files, that could certainly work... But everyone would

Re: [HACKERS] [GENERAL] LDAP service lookup

2007-08-27 Thread Dawid Kuroczko
On 8/27/07, Albe Laurenz [EMAIL PROTECTED] wrote: Dawid Kuroczko wrote: Then again, apart from libpq I don't see it mentioned anywhere. [...] Looking at the 8.3devel documentation... I think it should be mentioned in 18. Server Configuration. probably somewhere in 18.3 Connections and

Re: [HACKERS] MSVC build system

2007-08-27 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Alvaro Herrera wrote: That, or we create the makefiles in a fixed system and keep the Makefiles in CVS (though would be derived files). IIRC, we previously looked into cmake and concluded it supported a lot fewer platforms than pgsql. However, if

Re: [HACKERS] Problem with locks

2007-08-27 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: I think we should go ahead and print a LOG message in the case where a deadlock is detected. That seems reasonable, but I don't find any of these other changes to be an improvement... regards, tom lane

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Josh Berkus
Tom, Another idea is to not hardwire any restriction into the C code, but instead have initdb revoke the default public execute access on the tablespace size function. It would still work for superusers, and a particular DBA could choose to grant execute to trustworthy people. The problem

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Yeah, the big issue here is that Tablespaces do not have any kind of read permission, so there's nothing for us to go by. There's a good reason for them not to, since they're orthagonal to schema and databases, but it leaves us without a handle for

Re: [HACKERS] Problem with locks

2007-08-27 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: I think we should go ahead and print a LOG message in the case where a deadlock is detected. After chewing on it some more, I liked this patch better ... but I found a couple other issues, particularly with the soft-deadlock-recovery messaging sequence.

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Hmmm ... execept we're not requiring even permission on *one* DB in the tablespace are we? The status-quo-ante was that any user could get the number for any database and/or any tablespace. I'm prepared to admit that what I committed is too strong, but no

Re: [HACKERS] Insufficient attention to security in contrib (mostly)

2007-08-27 Thread Josh Berkus
Tom, Now you can argue that approximate database size information simply isn't that useful to an attacker, and maybe that's true. But are we prepared to make a policy decision that we aren't going to try to protect such information at all? But it's not making *no* attempt. This is a