[HACKERS] notify with payload (pgkill, notify)
Is the intent to replace most uses of (pg)kill with a general purpose messaging system between the processes, or (just) to address notify per se? (Presumably with 'fire-and-forget' and also rpc semantics? And pub-sub? And some sort of write to an fd protected by an atomic flag to elide multiple writes when the process hasn't woken and acknowledged the ATTN yet?) If pgkill is not used for signalling, could this reduce the reliance on signals (except for trying to kill off processes) to the point where ot becomes much less scary to link to libraries that use signals themselves and/or threaded runtimes? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] bug in float8in()
I run the following sql statements in linux and get the results: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); INSERT 0 1 postgres=# select * from test_double; col1 -- Infinity (1 row) but in windows: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); ERROR: 1797693134862315900 00 is out of range for type double precision postgres=# select * from test_double; col1 -- (0 rows) This is a bug, the behavior in windows and linux is not the smae I check the code and find that float8in exists a bug: the strtod() function in linux dosen't set errno when dealing with 1.7976931348623159E308 but in windows it does How should we improve it? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] Problem identifying constraints which should not be inherited
Hi Alvaro On Fri, Mar 28, 2008 at 6:05 PM, Alvaro Herrera [EMAIL PROTECTED] wrote: NikhilS escribió: I will take a look at the pg_dump related changes if you want. We will need changes in flagInhAttrs() and in getTableAttrs() to query the backend for these 2 attributes for post 80300 versions. Oh, BTW, I have not added this patch to any Commitfest page on the wiki, since it has obvious things that need more work. If you do work on them, please post the improved patch later and add it to the corresponding Commitfest, as appropriate. I submitted the combined latest patch to the patches list yesterday. How can I add it to the commitfest (presumably to the May one?)? Please let me know. Regards, Nikhils -- EnterpriseDB http://www.enterprisedb.com
Re: [HACKERS] bug in float8in()
If you look into documentation, the behavior of float/double is platform depend. If you want to same result on any platform, use numeric instead. Zdenek Richard Wang napsal(a): I run the following sql statements in linux and get the results: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); INSERT 0 1 postgres=# select * from test_double; col1 -- Infinity (1 row) but in windows: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); ERROR: 1797693134862315900 00 is out of range for type double precision postgres=# select * from test_double; col1 -- (0 rows) This is a bug, the behavior in windows and linux is not the smae I check the code and find that float8in exists a bug: the strtod() function in linux dosen't set errno when dealing with 1.7976931348623159E308 but in windows it does How should we improve it? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New boxes available for QA
On Wed, Apr 2, 2008 at 1:53 AM, Greg Smith [EMAIL PROTECTED] wrote: As far as the other more powerful machines you mentioned go, would need to know a bit more about the disks and disk controller in there to comment about whether those are worth the trouble to integrate. The big missing piece of community hardware that remains elusive would be a system with Here we go: - a couple of 1750 servers: dual Xeon 2.8 boxes with PERC 4/DI, 2 internal disks, from 2 to 3 GB of RAM, we can probably get one of them up to 4 GB if needed - a PV 220 S disk array with: 4 x 36 GB + 5 x 73 GB. I think I can get 8 identical disks in the box by switching the 73 GB disks with the 36 GB ones from the other boxes but I'm not sure we can make only one RAID array from the 2 parts of the PV 220 S. - one of the above boxes also has a PERC 4/DC and is connected to the disk array. - a 6650 box: quad Xeon MP 2.2 with 4 GB: it has 2 internal disks and an external attachment to the disk array. All the disks are 10k rpm. What I was thinking about is that it can be useful to have several boxes connected to validate features too, not only performances (who says read access to a warm standby?). Note that if we don't find any good usage for them, it won't be a problem to affect them to our internal test platform. If everything goes well, we plan to buy a big box for internal PostgreSQL benchmarking and testing. It's obvious we won't use it night and day so I may be able to provide windows of time when the community can use it. This one is hypothetical though, the other ones are real and dedicated to community usage (yeah, it wasn't an April's fool). -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] GiST opclass and varlena
Le mardi 25 mars 2008, Dimitri Fontaine a écrit :
postgres=# create index idx_prefix on ranges using gist(prefix
gist_prefix_range_ops);
NOTICE: gpr_picksplit(): entryvec-n= 234 maxoff= 232 l= 176 r= 56 l+r=
232 unionL='01[0-7]' unionR='01[4-7]'
NOTICE: gpr_picksplit(): v-spl_ldatum='01[0-7]' v-spl_rdatum='01[4-7]'
ERROR: invalid memory alloc request size 3049878020
My previous tests were only done with REL8_2_STABLE cvs branch, I just redone
some tests with REL8_3_STABLE and got no error. The index is still buggy, in
the sense some requests returns different results with or without it
(enable_seqscan).
I've received some help on testing it too, and it seems the behavior is also
dependent on the architecture used. I'm using 32 bits linux arch, tests in 64
bit arch showed no error.
The code is available at pgfoundry here:
http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/prefix/prefix/
It still is :)
The easy way to test it is:
create table prefixes (
prefixtext primary key,
name text not null,
shortname text,
state char default 'S',
check( state in ('S', 'R') )
);
comment on column prefixes.state is 'S: - R: reserved';
\copy prefixes from 'prefixes.fr.csv' with delimiter ; csv quote ''
create table ranges as
select prefix::prefix_range, name, shortname, state from prefixes ;
create index idx_prefix on ranges using gist(prefix gist_prefix_range_ops);
Then enable_seqscan to on or off, and queries such as
select * from ranges where prefix @ '0100101234';
select * from ranges where prefix @ '0146640123';
On my 8.3 testing, the former query gives the same result with or without
using the GiST index, the latter doesn't.
Regards,
--
dim
signature.asc
Description: This is a digitally signed message part.
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, and are thus encouraging people to use md5 everywhere. At the very least, I think we need to add sha1. Adding sha* would be better, and adding other hashes would be better still (and make PG a better product, in my opinion: having things builtin vs. contrib is a huge distinction). I'd also like to emphasize that this is not a pgcrypto issue: while it provides the same functionality that this proposal does, so does creating a Pl/Perl function, which is the route I usually go, as it is much easier and portable. So I see this as adding missing features to core. We will obviously never replace pgcrypto entirely, due to the silly state of encryption legislation. - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key: 0x14964AC8 200804020906 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkfzhMQACgkQvJuQZxSWSsiTCwCgvauRh/hqOK0BEMEPoKXhzCf5 +GkAn0ma+cZy3fas1Y/uQ0Zf8WAqfLo8 =tTlK -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Bruce Momjian escribió: Because of this: variadic function, named params exist only as WIP and I see it for next commit fest. I'll send new version in next months. This has been saved for the next commit-fest: http://momjian.postgresql.org/cgi-bin/pgpatches_hold Yes, it was already listed here: http://wiki.postgresql.org/wiki/CommitFest:May Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Alvaro Herrera wrote: Bruce Momjian escribi??: Because of this: variadic function, named params exist only as WIP and I see it for next commit fest. I'll send new version in next months. This has been saved for the next commit-fest: http://momjian.postgresql.org/cgi-bin/pgpatches_hold Yes, it was already listed here: http://wiki.postgresql.org/wiki/CommitFest:May Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. Can you give me an example of a link that changed? They shouldn't. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Bruce Momjian escribió: Alvaro Herrera wrote: Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. Can you give me an example of a link that changed? They shouldn't. They used to be http://momjian.us/mhonarc/patches/[EMAIL PROTECTED] but now are http://momjian.us/mhonarc/message-id/[EMAIL PROTECTED] -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Alvaro Herrera wrote: Bruce Momjian escribi??: Alvaro Herrera wrote: Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. Can you give me an example of a link that changed? They shouldn't. They used to be http://momjian.us/mhonarc/patches/[EMAIL PROTECTED] but now are http://momjian.us/mhonarc/message-id/[EMAIL PROTECTED] Oh, OK, the first one was my _old_ permanent version that is permanent while messages are added/removed, but obviously not permanent for mailbox movement. The new permanent ones are permanent against mailbox movement, and in fact the comments and thread merging also travels with the email. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Alvaro Herrera [EMAIL PROTECTED] writes: Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. I don't think the wiki pages should be relying on Bruce's queue at all --- they should just link into the mail archives. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] POSIX shared memory support
The original patch author: Chris Marcellino [EMAIL PROTECTED] was not CC'ed as part of this email thread. That was a mistake. Chris, the email thread discussing your patch is here: http://archives.postgresql.org/pgsql-hackers/2008-03/msg01262.php Please read the discussion --- the bottom line is that there isn't much support for the patch. Magnus was able to do the POSIX usage without relying on shared memory, but I just talked to him via IM and he said it used a Win32-specific feature that isn't portable to Unix. I am holding this patch for the next commit fest in hopes you can adjust it, but if not the patch will be rejected at that time. --- Stephen Frost wrote: -- Start of PGP signed section. Chris, et al, (commit-fest consensus discussion) * Chris Marcellino wrote: In case you haven't had enough, here is another version of the code to make Postgres use POSIX shared memory. Along with the issues that have already been addressed, this version ensures that orphaned backends are not in the database when restarting Postgres by using a single 1 byte SysV segment to see who is attached to the segment using shmctl/IPC_STAT/nattach. This really feels like a deal-breaker to me. My first reaction to this patch, honestly, is that it's being justified for all the wrong reasons. Changing to POSIX shm seems like a reasonable goal in general, provided it can do what we need, but doing it to work around silly defaults doesn't really work for me. If the real issue you have is with the SysV limits then I'd suggest you bring that up with the kernel/distribution folks to get them to use something more sane. Looking around a bit, it looks like it's already being addressed in some places, for example Solaris 10 apparently uses 1/4th of memory, while Centos 5 uses 4GB. Suse also uses a larger default, from what I understand. Supporting this effort to get it raised on various platforms and distributions seems like a much better approach. Additionally, it strikes me that there *is* a limit on POSIX shared memory too, generally half of ram on the systems I've looked at, but there's no guarentee that'll always be the default or that half of ram will always be enough for us. So, even with this change, the problem isn't completely 'solved'. Finding a way for POSIX shm to do what we need, including Tom's concerns, without depending on SvsV shm as a crutch work around, would make this change much more reasonable and could be justified as moving to a well defined POSIX standard, and means we may be able to support platforms which either are new and don't implement SysV but just POSIX, or cases where SysV is being actively depreceated. Neither of which is possible if we're stuck with using it in some cases. Thanks, Stephen -- End of PGP section, PGP failed! -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Upon verifying this I noticed that you broke all the permanent links the other day, thus rendering both commitfest wiki pages useless -- just fixed them. It would be nice that if you promise things to be permanent, they are really permanent. Otherwise they are no better than the other urls with message numbers. I don't think the wiki pages should be relying on Bruce's queue at all --- they should just link into the mail archives. Except the comments are on my queue, and they are updated to join threads by adding comments to patches posted months ago. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] varadic patch
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I don't think the wiki pages should be relying on Bruce's queue at all --- they should just link into the mail archives. Except the comments are on my queue, and they are updated to join threads by adding comments to patches posted months ago. Well, as I told you yesterday, I don't see why you're expending large amounts of effort to create facilities that already exist trivially if the patch queue was just a bunch of URLs and comment text on a wiki page. I feel that your current approach to the patch queue is dead after this commit fest, so there's little point in putting so much work into it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: Agreed ... but, stupid question here ... if our tags are wrong in CVS, are the 7.1.x releases themselves wrong too? When I do a release tarball, I run: cvs -q export -rREL7_1_1 pgsql so, if the tags are wrong, then all of those releases are wrong too, since they are based on the tag ... I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. I sporadically tested the correspondence of tarball and tag for some other older releases, and they seem OK. At least to the extent that corresponding tags exist at all ... ;-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 01:07:01PM -, Greg Sabino Mullane wrote: There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, and are thus encouraging people to use md5 everywhere. At the very least, I think we need to add sha1. Adding sha* would be better, and adding other hashes would be better still (and make PG a better product, in my opinion: having things builtin vs. contrib is a huge distinction). I'd also like to emphasize that this is not a pgcrypto issue: while it provides the same functionality that this proposal does, so does creating a Pl/Perl function, which is the route I usually go, as it is much easier and portable. So I see this as adding missing features to core. We will obviously never replace pgcrypto entirely, due to the silly state of encryption legislation. Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 12:27:15PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. Which places, and what laws? Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
Peter Eisentraut wrote: Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: Agreed ... but, stupid question here ... if our tags are wrong in CVS, are the 7.1.x releases themselves wrong too? When I do a release tarball, I run: cvs -q export -rREL7_1_1 pgsql I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. Hmm, if we use a CVS export, why do we have, on make distdir, the business to remove CVS files? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Hi Tom, md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? Sanjay Sharma To: [EMAIL PROTECTED] CC: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3 Date: Wed, 2 Apr 2008 11:38:31 -0400 From: [EMAIL PROTECTED] Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers _ Tried the new MSN Messenger? It’s cool! Download now. http://messenger.msn.com/Download/Default.aspx?mkt=en-in
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter wrote: On Wed, Apr 02, 2008 at 12:27:15PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. Which places, and what laws? http://rechten.uvt.nl/koops/cryptolaw/cls-sum.htm (Info only - I have not spent time considering the issue at hand thus, counter to net tradition, offer no opinion.) Cheers, Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. We aren't actually doing any such thing. We are just making sure that our distribution is split up in such a way that if someone *else* needs to comply with such a law, it's not impractical to do so. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
Alvaro Herrera [EMAIL PROTECTED] writes: Peter Eisentraut wrote: I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. Hmm, if we use a CVS export, why do we have, on make distdir, the business to remove CVS files? Leftover from the old way, no doubt. I have a vague recollection that the rest of core pressed Marc to start using cvs export precisely because we were worried about, or actually had seen, inconsistencies between the tagging and the tarballs. It might be that what Peter has noticed was exactly what prompted that, and we never attempted to clean it up. But it was a long time ago and I'm not sure about it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 12:49:38PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. We aren't actually doing any such thing. We are just making sure that our distribution is split up in such a way that if someone *else* needs to comply with such a law, it's not impractical to do so. That cat is already out of the bag with every Linux distribution. There's just no point in our going three sides around the square in order to have someone, somewhere claim that our tarball isn't violating their law. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: That cat is already out of the bag with every Linux distribution. There's just no point in our going three sides around the square in order to have someone, somewhere claim that our tarball isn't violating their law. [shrug...] It's not exactly terribly complicated to keep this stuff in a contrib module. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
sanjay sharma [EMAIL PROTECTED] writes: md5 is not being recommended anywhere because it contains hash collision. For the purposes we are using it for, that's just about 100% irrelevant. Would replacing md5 with SHA1 in core involve much work? Yes, it would be a tremendous problem, because the use of md5 is part of our password protocol. We'd have to change client-side code in sync with the servers to do that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] first time hacker ;) messing with prepared statements
The MAJOR benefit of Microsoft's approach is that it works on existing application, Yes, that is a nice benefit ! Is there a way to turn it on/off ? Or is it smart enough to only cache plans for cases where it is relevant ? For instance, I absolutely want some queries to be planned according to real parameters (makes huge difference on some search queries, as expected), whereas most simple queries like the proverbial select by ID etc could be cached without problems... and, most importantly makes NO assumptions on the volatile server state. A few cases where the Microsoft solution works, while yours will fail is: * Server restart and assorted like failover (you need to redo a global prepare). * Cleanup and instantiation of a prepared statement. Hehe, actually, mine does work after restart since the statements are stored in a database-specific system catalog which is persistent. Actually, what I store is not the result of PREPARE (a plan) but the text of the SQL query PREPARE foo., that is I just cut the GLOBAL from GLOBAL PREPARE and store the rest. The actual PREPARE is realized by each connection when it encounters an EXECUTE request and doesn't find the cached plan. It is actually extremely simple ;) did you expect a fancy shared memory cache (ahem...) ? No, no, it's very basic. This way, if a table was dropped and recreated, or whatever other stuff that can invalidate a plan since the GLOBAL PREPARE was issued, no problem, since there was no global stored plan anyway, just some SQL text. Also if a needed table was dropped, the user will get the same error message as he would have got issuing a PREPARE for the associated SQL query string. The overhead of each connection doing its own PREPARE is negligible, since, if you use that feature, you intend to issue this query many, many times during the life of the persistent connection. What you are doing for a global query cache is already in consideration and having plan invalidation mechanism on schema changes or, maybe, statistic updates was a step into that direction. You code mostly contributed the other parts already. As I said it is much simpler than that : I store no plans ;) Of course this means it only works with persistent connections. Another considerations is whether most task are getting CPU bound or IO bound. A better, per query, plan might reduce IO load due to better use of statistics on that single case, while for CPU bound it is very nice to reduce the planning overhead significantly. Well, if it is IO bound, then this thing is useless. However, since the purpose is to optimize often-used, simple queries, the likes of which abound in web applications, then it is relevant... because, if this kind of simple selects become IO bound, and you have a few on each page, you're really in trouble... Another possible implementation would be to use a connection pooler which, when opening a new connection, can be configured to send a SQL script containing all the PREPARE statements. This is, IMHO, an application side feature that might be a good addition to PHP and other languages that provide the persistent connection feature. On second thought, if it is not in Postgres, I'd rather put this in the connection pooler, because this way it can be used by different applications. But then you have to use a connection pooler. Also, by putting it in Postgres, statements are only prepared as needed, whereas the pooler would have to issue a lot of PREPAREs at connection startup, making new connection startup slower. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Am Mittwoch, 2. April 2008 schrieb David Fetter: That cat is already out of the bag with every Linux distribution. There are other operating systems besides Linux. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
sanjay sharma wrote: Hi Tom, md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? sanjay - please do not top-answer, especially when others have put their answers below. MD5 is not broken for purposes that would require a pre-image attack, AIUI. That means there is a whole series of uses for which it is still quite OK, including password hashing. That said, there might well be a reason for including a collision-resistant hash function in core without including the whole of pg_crypto. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 07:16:53PM +0200, Peter Eisentraut wrote: Am Mittwoch, 2. April 2008 schrieb David Fetter: That cat is already out of the bag with every Linux distribution. There are other operating systems besides Linux. And the cat's out of the bag there, too. Have you tried to get a system that doesn't include strong crypto running any time recently? Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Am Mittwoch, 2. April 2008 schrieb sanjay sharma: md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? The vulnerabilities that exist for MD5 and SHA1 involve finding two random input strings that create the same hash values. This is possible for MD5 *and* SHA1 now, so asking for SHA1 to replace MD5 is completely pointless. What is not possible with either MD5 or SHA1 is finding an input string that creates the same hash value as a given input string (except by googling, but that affects all algorithms). So using MD5 for encrypting passwords or digesting known data values or tarballs can be considered secure at the moment. If you are dealing with certificate infrastructures, where the hash collision vulnerability described above might be relevant, you are certainly going to use some library such as openssl, and those have already moved away from using MD5 and SHA1 anyway. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] US VISA CISP PCI comp. needs SHA1
Not sure if I posted in correct spot pg_8.2.6 Centos5 Windows based app. encryped pwd = yes SSL = yes, hostssl with explicit IP w/md5. (no pg_crypto) We are in process of VISA CISP PCI compliance for our application. (online cc auth - no stored cc data) [next phase will include stored cc data] We just heard back today that they would like to use SHA1 for pwd auth. does anyone have any doco that will support md5 vs. SHA1? We also have global customers so we understand the us v non-US export stuff. Any direction is appreciated. Thanks in advance. /matthew wetmore -- Matthew Wetmore Secom International, Inc 9610 Bellanca, Ave. Los Angeles, CA 90045 310-641-1290 This e-mail is intended for the addressee shown. It contains information that is confidential and protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons or unauthorized employees of the intended organisations is strictly prohibited. The contents of this email do not necessarily represent the views or policies of Secom International Inc., or its employees. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] US VISA CISP PCI comp. needs SHA1
Matthew Wetmore wrote: We just heard back today that they would like to use SHA1 for pwd auth. Why would anyone want to do something so pointless? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Visa CISP PCI compliance needs SHA1?
Not sure if I posted in correct spot But seems to be topic of today...funny on same day I hear from Visa. pg_8.2.6 Centos5 Windows based app. encryped pwd = yes SSL = yes, hostssl, with explicit IP w/md5,. (no pg_crypto) This is just with client / server pwd auth We are in process of VISA CISP PCI compliance for our application. (online cc auth - no stored cc data) [next phase will include stored cc data] We just heard back today that they would like to use SHA1 NOT md5 for pwd auth. does anyone have any doco that will support md5 vs. SHA1? is PG_crypto in the db (meaning crypt the md5 hash )still the same as md5 auth We also have global customers so we understand the US v non-US export stuff. Any direction is appreciated. Thanks in advance. /matthew wetmore -- Matthew Wetmore Secom International, Inc 9610 Bellanca, Ave. Los Angeles, CA 90045 310-641-1290 This e-mail is intended for the addressee shown. It contains information that is confidential and protected from disclosure. Any review, dissemination or use of this transmission or its contents by persons or unauthorized employees of the intended organisations is strictly prohibited. The contents of this email do not necessarily represent the views or policies of Secom International Inc., or its employees. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Patch queue - wiki (was varadic patch)
On Wed, 2 Apr 2008, Bruce Momjian wrote: The new permanent ones are permanent against mailbox movement, and in fact the comments and thread merging also travels with the email. The someone replied to your comment links in e-messages I've been getting the last few days have all been working, which is a first. The configuration you're running right now I'd consider the first candidate to be a stable version, so thumbs up from me for reaching that point. It's clear to me only now that you can think of the patch queue as being a list with this structure: 1) Patch name (defaults to the subject of the first message) 2) List of messages related to that patch 3) List of comments 4) Status 5) Assigned reviewers Bruce's toolchain converts an mbox of messages to generate the first two, then has a web interface to allow adding the third. Right now the message list is internally consistant but not useful in the long term (doesn't have links to the archives, just this temporary page). Until the search for message ID feature is added to the archives I don't know that this situation can be improved. Those hacking on tools to convert Bruce's currently preferred working form (that revolves around mbox files) into something else that's web oriented are stuck with considering how all the above information is going to be handled before everybody will be satisfied. I can see how a script that converts the current pages into wiki markup, with placeholders where someone can manually update the comments to summarize those on the page, would be helpful. That basically creates an easier to read Queue summary like Stephan was doing for 8.3--that included items 1,4,5 from the above. But that's a one-way operation that doesn't really help with the commenting situation, and it's inevitably going to lag behind the mailbox-centered queue unless it's made fully automatic. I can't think of anything better that doesn't require building some sort of database that holds all this information and drives page generation. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] US VISA CISP PCI comp. needs SHA1
Matthew Wetmore wrote: Not sure if I posted in correct spot pg_8.2.6 Centos5 Windows based app. encryped pwd = yes SSL = yes, hostssl with explicit IP w/md5. (no pg_crypto) We are in process of VISA CISP PCI compliance for our application. (online cc auth - no stored cc data) [next phase will include stored cc data] We just heard back today that they would like to use SHA1 for pwd auth. does anyone have any doco that will support md5 vs. SHA1? We also have global customers so we understand the us v non-US export stuff. Any direction is appreciated. You could use pg_crypto plus application level passwords. As has been pointed out elsewhere, there is no security virtue in swapping MD5 password hashing in Postgres for SHA1. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter wrote: And the cat's out of the bag there, too. Have you tried to get a system that doesn't include strong crypto running any time recently? Well yes, I have recently been involved in a large deployment of such systems. The details are a bit too involved for this thread, but the notion expressed elsewhere in this thread to make it easy to separate or rip out the different parts are exactly what facilitated that project. Now I am not saying that we must keep pgcrypto away from the core code forever for the benefit of these specific and well-funded external requirements. But these requirements do exist, and I haven't seen a good reason _for_ moving pgcrypto in this thread. Note that the original post actually wanted an sha1 function for MySQL compatibility. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch queue - wiki (was varadic patch)
Greg Smith [EMAIL PROTECTED] writes: Those hacking on tools to convert Bruce's currently preferred working form (that revolves around mbox files) into something else that's web oriented are stuck with considering how all the above information is going to be handled before everybody will be satisfied. I can see how a script that converts the current pages into wiki markup, with placeholders where someone can manually update the comments to summarize those on the page, would be helpful. That basically creates an easier to read Queue summary like Stephan was doing for 8.3--that included items 1,4,5 from the above. But that's a one-way operation that doesn't really help with the commenting situation, and it's inevitably going to lag behind the mailbox-centered queue unless it's made fully automatic. I can't think of anything better that doesn't require building some sort of database that holds all this information and drives page generation. This seems to be ignoring the possibility of those involved with the patch queue simply manually editing the wiki page. For the past couple of weeks I've been dealing with both Bruce's queue and the one at http://wiki.postgresql.org/wiki/CommitFest:March and frankly I find the latter a *whole* lot more satisfactory, despite the fact that it's got exactly zero custom tooling or infrastructure behind it. It doesn't have artificial constraints on page organization; I can update it as soon as I've done something rather than waiting around for Bruce to do so; and there's an automatically maintained history of changes. Bruce has put a whole lot of man-hours into getting his page to do a few of the things we could do for free with the wiki page, but it's still got a long way to go. Now it would certainly be nice if there were some tools that would assist with dumping URLs for newly-arrived messages into the wiki page. Perhaps some of the pgsql-www crowd can think about how to do that. But even if we had to do that entirely by hand, I'd rather go with the wiki page. At some point it might be worth building the sort of heavy-duty infrastructure for the patch queue that you have in mind here. I don't think we need it yet, though, and I definitely don't think we understand our requirements well enough yet to start designing it. One of the reasons I like the wiki approach is exactly that there's such a low barrier to getting started with it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] notify with payload (pgkill, notify)
James Mansion wrote: Is the intent to replace most uses of (pg)kill with a general purpose messaging system between the processes, or (just) to address notify per se? (Presumably with 'fire-and-forget' and also rpc semantics? And pub-sub? And some sort of write to an fd protected by an atomic flag to elide multiple writes when the process hasn't woken and acknowledged the ATTN yet?) If pgkill is not used for signalling, could this reduce the reliance on signals (except for trying to kill off processes) to the point where ot becomes much less scary to link to libraries that use signals themselves and/or threaded runtimes? My intention is to revamp the listen/notify system, pure and simple. If you have an alternative suggestion them you need to make it now. We are not intending to use FDs for message passing. They will be stored in shared memory. See previous discussions for details: http://groups.google.com/group/pgsql.hackers/browse_frm/thread/e63a5ac43e2508ce/e0892fb3316cc327?hl=enlnk=gstq=notify+payload#e0892fb3316cc327 http://groups.google.com/group/pgsql.hackers/browse_frm/thread/6a59675a3e11bedc/87e0ce6dd6cce6a6?hl=enlnk=gstq=notification+payload#87e0ce6dd6cce6a6 cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch queue - wiki (was varadic patch)
Tom Lane wrote: For the past couple of weeks I've been dealing with both Bruce's queue and the one at http://wiki.postgresql.org/wiki/CommitFest:March and frankly I find the latter a *whole* lot more satisfactory, despite the fact that it's got exactly zero custom tooling or infrastructure behind it. It doesn't have artificial constraints on page organization; I can update it as soon as I've done something rather than waiting around for Bruce to do so; and there's an automatically maintained history of changes. Bruce has put a whole lot of man-hours into getting his page to do a few of the things we could do for free with the wiki page, but it's still got a long way to go. Now it would certainly be nice if there were some tools that would assist with dumping URLs for newly-arrived messages into the wiki page. Perhaps some of the pgsql-www crowd can think about how to do that. But even if we had to do that entirely by hand, I'd rather go with the wiki page. At some point it might be worth building the sort of heavy-duty infrastructure for the patch queue that you have in mind here. I don't think we need it yet, though, and I definitely don't think we understand our requirements well enough yet to start designing it. One of the reasons I like the wiki approach is exactly that there's such a low barrier to getting started with it. + MAXINT. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in float8in()
Richard Wang wrote: I run the following sql statements in linux and get the results: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); INSERT 0 1 postgres=# select * from test_double; col1 -- Infinity (1 row) but in windows: postgres=# create table test_double(col1 float8); CREATE TABLE postgres=# insert into test_double values(1.7976931348623159E308); ERROR: 1797693134862315900 00 is out of range for type double precision postgres=# select * from test_double; col1 -- (0 rows) This is a bug, the behavior in windows and linux is not the smae float relies on platform behavior, AFAIK. You're better off using numeric if you need to process these values without that. Float is an approximate datatype. In this case, linux makes a fairly wild approximation (that it's the same as infinity) whereas windows says it just can't approximate it. You could argue for either one being correct, but in the end you just need to use a different datatype if you need consistent behavior. the strtod() function in linux dosen't set errno when dealing with 1.7976931348623159E308 but in windows it does How should we improve it? Not sure we should, really... //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Tom Lane wrote: Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. I think that claim is completely incorrect. A lot of people use the md5() function in PostgreSQL today to hash the passwords for the users of whatever webbapp they are running. It only uses one account to connect to PostgreSQL and handles the rest of the auth elsewhere in the app. These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
Marc G. Fournier wrote: - --On Tuesday, April 01, 2008 14:06:09 -0400 Tom Lane [EMAIL PROTECTED] wrote: Peter Eisentraut [EMAIL PROTECTED] writes: In the meantime, does anyone have more information about how this came about? Marc's always done both the tagging and the tarball-making, so you'd have to ask him about that. I believe he's made it more scripted over the years, so this might reflect a manual foulup that (hopefully) is no longer possible. Ya, I'll go with that (considering 7.1 was back in 2001 ... ) ... but, from the way Peter describes it (taging partially checked out code), I'm not 100% how its possible to 'foul up' ... a tag operation is: cvs -q update -APd . cvs -q tag REL7_1 . unless its a sub-tagging, which would have: cvs -q update -rREL7_1_STABLE -Pd . cvs -q tag REL7_1_1 . And since I don't do the update until things are quiet (generally when Tom has finished his last commit before release), I'm not sure how I could have gotten a 'partial checkout' ... Could it be that a commit was done while the tag operation was running? Given that neither is an atomic operation in cvs, and it used to be that large repo operations could take quite a long time? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Was that really the conclusion? My memory of this thread showed that most people who actually deal with hashes and cryptography *wanted* a SHA based hash in core (because our users ask for it!) and the only disagreement was in *what* should be included. //Magnus Bruce Momjian wrote: There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. --- Bruce Momjian wrote: I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? --- Marko Kreen wrote: On 1/21/08, Tom Lane [EMAIL PROTECTED] wrote: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. Note: this was bad idea. The function that should have been added to core would be pg_password_hash(username, password). Adding md5() lessens incentive to install pgcrypto or push/accept digest() into core and gives impression there will be sha1(), etc in the future. Now users who want to store passwords in database (the most popular usage) will probably go with md5() without bothering with pgcrypto. They probably see Postgres itself uses MD5 too, without realizing their situation is totally different from pg_authid one. It's like we have solution that is ACID-compliant 99% of the time in core, so why bother with 100% one. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong
Pavan Deolasee [EMAIL PROTECTED] writes: Please see the attached patch. One change I made is to hold the SHARE lock on the page while ANALYZE is reading tuples from it. I thought it would be a right thing to do instead of repeatedly acquiring/releasing the lock. Bruce pointed out to me off-list that this patch is closely related to the patch I proposed awhile back for ANALYZE *overcounting* dead tuples. That one is on the current commit-fest list because we held it over after this discussion: http://archives.postgresql.org/pgsql-hackers/2007-11/msg00771.php On reflection it seems to me that we allowed ourselves to get distracted by schemes for reducing the error attributable to the uncertain state of in-doubt tuples. That's still something interesting to think about, but we forgot the fact that there's a serious problem in 8.3 and the patches we have would clearly make it better. What I propose therefore is combining this patch with my older one so that ANALYZE counts according to the following rules: REDIRECT line pointer: ignore DEAD line pointer: count as dead HEAPTUPLE_LIVE tuple: count as live, include in statistics pool HEAPTUPLE_DEAD: count as dead HEAPTUPLE_RECENTLY_DEAD:count as dead HEAPTUPLE_INSERT_IN_PROGRESS: ignore HEAPTUPLE_DELETE_IN_PROGRESS: count as live We might want to adjust these rules later after more thought, but in any case ANALYZE has to be fixed to be able to distinguish these cases in the first place. This is better than what we have and is reasonable to back-patch. Trying to reduce the race conditions for in-doubt tuples should go on the TODO list. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander wrote: I think that claim is completely incorrect. A lot of people use the md5() function in PostgreSQL today to hash the passwords for the users of whatever webbapp they are running. It only uses one account to connect to PostgreSQL and handles the rest of the auth elsewhere in the app. These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. Hi Magnus: I don't think this is a compelling argument, and I mostly agree with Tom. PHP, Perl and Java are just three languages at the tip of my tongue that have built in support for MD5 and SHA1, and in all cases I can think of in a few seconds (I might be missing something?), it's far more desirable to do the MD5 / SHA1 in the language. If the document being encoded is large, doing it in the client is more efficient from a network transport perspective, as well as allowing ensuring that performance cost is on the web side, not the database side. If the text to be encoded requires security, then transmitting the password in clear text to the server only to be MD5 / SHA1 summed is not a great solution, as it involves transmission of the password. In both cases, I would do it client side, inside the web app. So, I believe your argument that web apps need it is faulty. I think a legitimate use would involve around using such a function in pl/pgsql. I can't think of a case where I've ever needed to do that. Cheers, mark -- Mark Mielke [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in float8in()
Magnus Hagander [EMAIL PROTECTED] writes: Float is an approximate datatype. In this case, linux makes a fairly wild approximation (that it's the same as infinity) whereas windows says it just can't approximate it. Note that the behavior isn't consistent across Unixen either --- my HPUX box acts like Windows here. Basically Postgres just tries to reflect the platform's floating-point support. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 02, 2008 10:10:59 +0200 Magnus Hagander [EMAIL PROTECTED] wrote: Could it be that a commit was done while the tag operation was running? Given that neither is an atomic operation in cvs, and it used to be that large repo operations could take quite a long time? I doubt it, back in 7.1 days, we had something like 4 committers, I think, and all core members ... very little chance of overlap ... the weird thing is Peter is reporting its *all* on the 7.1 branch ... none of hte others affected ... - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/It4QvfyHIvDvMRAqnKAJ4h/+4elPLe3XEfW9lRU6IfAAiSWgCcCRCd pWs76n1JzLgiOExj6kGytBA= =XVkS -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander [EMAIL PROTECTED] writes: ... These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. That line of argument could be used to justify putting anything and everything in core. I think that our extensible architecture is an important feature and one we should not hesitate to use to the fullest. There is clearly work that could/should be done to improve it, such as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type work to encourage people such as hosting companies to allow access to non-core packages. But allowing arguments such as the above to drive our design is not the way to go. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Tom Lane wrote: There is clearly work that could/should be done to improve it, such as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type work to encourage people such as hosting companies to allow access to non-core packages. Funny you should mention that. I was just thinking the other day that I needed to re-raise the question of getting rid of contrib, which has long been a horrible misnomer, and calling its contents The PostgreSQL Standard Modules. Maybe we could rename the directory modules. IIRC that was the consensus name last time this topic was discussed. We've already vastly improved their usefulness by adding documentation. Let's take the next jump and start advertising them as exemplars of our extensibility. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 05:09:14PM -0400, Andrew Dunstan wrote: Standard Modules. Maybe we could rename the directory modules. IIRC This seems like an easy and practical answer. A -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 02, 2008 12:33:24 -0400 Alvaro Herrera [EMAIL PROTECTED] wrote: Peter Eisentraut wrote: Am Mittwoch, 2. April 2008 schrieb Marc G. Fournier: Agreed ... but, stupid question here ... if our tags are wrong in CVS, are the 7.1.x releases themselves wrong too? When I do a release tarball, I run: cvs -q export -rREL7_1_1 pgsql I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. Hmm, if we use a CVS export, why do we have, on make distdir, the business to remove CVS files? Using export was a relatively recent change ... wasn't a command I knew about at the start, someone (possibly Peter) pointed it out to me and we changed the script ... never thought to change distdir though ... - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/vv4QvfyHIvDvMRAu2NAKDjiCnjJfJ3pXtUX9PFH8vZfSnr5ACfcWdg EiNTuC6bd2PFxlZ1tt8oA68= =VbGW -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 02, 2008 18:08:05 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. 'k, you need to explain this one to me ... how does export affect what files are tag'd? export isn't used as part of the tagging process, only to create the tar balls themselves ... - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH8/w44QvfyHIvDvMRAmXeAKDc7V43uQmP8SdxuLBaIyvjntKlzgCfYF5N Tsn/rXOk2AH9RgjlOO9duKQ= =29dq -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] notify with payload (pgkill, notify)
Andrew Dunstan wrote: If you have an alternative suggestion them you need to make it now. well, I'm not sure about the non-collapsing business but no matter. We are not intending to use FDs for message passing. They will be stored in shared memory. See previous discussions for details: I'm more interested in replacing the pgkill wakeup mechanism with something that is less sucky on Win32. If you just want to send a wakeup then making a pipe RPC is crazy since it introduces a lot of scheduling. A unix domain datagram socket to which processes can send one byte is enough to toggle the readability if the socket without doing much transfer, and using an atomic flag in shm means that subsequent wakeups can be elided until the process acknowledges the signal, which it should do before scanning the shm queue. I don't see any reason to pass the data through the kernel if you have the shm handling routines, particularly if you have ones that handle spilling. If you're happy with using signals on UNIX (and Tom suggested that SIGUSR usage and signal handling might be one of the concerns about allowing threaded addins) then at least on Win32 it should be worth adding a variant of kill which has a void return and does not try to return information about the validity of the target process id. Personally I try to avoid signals and would prefer an fd level-based signal. But YMMV. I was just wondering if what you're doing could be generalised as a 'post to backend' system that can be used for other duties in future, so you'd need a message type and payload blob rather than assuming that the type is 'notify' and the payload is the payload length. If you had a pgsql extension that was 'wait for notificiation' or 'wait for row in message table' then the wakeup could essentially be directed at code in the target backend, rather than using it as a conduit to target code in the client. James -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
Marc G. Fournier [EMAIL PROTECTED] writes: - --On Wednesday, April 02, 2008 18:08:05 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: I believe we moved to using cvs export many years after 7.1. Before that, the releases were made straight out of a cvs checkout. With cvs export it is of course nearly impossible to create such a mess. 'k, you need to explain this one to me ... how does export affect what files are tag'd? It doesn't, of course. What it does do is guarantee that the tarball matches the tag that has already been laid down in CVS. But there must have been more to it than that. Peter is reporting that the tag is on mutually inconsistent versions of some files; which means that the problem was with the tagging operation more than with the tarball-making. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TRUNCATE TABLE with IDENTITY
On Mar 25, 2008, at 11:40 AM, Zoltan Boszormenyi wrote: All of them? PostgreSQL allow multiple SERIALs to be present, the standard allows only one IDENTITY column in a table. And what about this case below? CREATE TABLE t1 (id1 serial, ...); ALTER SEQUENCE seq_t1_id1 RESTART WITH 5432 CYCLE; or the equivalent CREATE SEQUENCE seq_t1_id1 START WITH 5432 CYCLE; CREATE TABLE t1 (id1 serial, ...); ALTER SEQUENCE seq_t1_id1 OWNED BY t1.id1; PostgreSQL doesn't keep the START WITH information. But it should to perform a restart on the sequence, using the minval in this case wouldn't be correct. I think you misunderstand what ALTER SEQUENCE RESTART does; it only changes the current value of the sequence. -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] Proposal: new ereport option errdetail_log
On Mar 24, 2008, at 6:21 PM, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Gregory Stark wrote: The axis on which I still see real room for improvement here is on the description of the locks. It's awfully hard for a user to tell from the deadlock message exactly what operation of the query was acquiring what lock when it deadlocked. Are the involved queries not enough? Why? What would you like to have? Greg's certainly got a point. Consider for example tuple-level locks taken as a result of an FK check --- which one, and which rows are involved? Or the case where the logged query is just SELECT some_huge_user_defined_function() and you have no idea what part of the function is triggering it. (The CONTEXT traceback will help here if the backend running the function is the one that errors out, but not when it's some other backend.) I don't have any immediate ideas for improvement either, but we certainly shouldn't consider this a totally solved problem. Something I always find myself wanting when debugging locking issues is what's in pg_locks. Could we save that information somewhere when we detect a deadlock? In a real table would be nice, but I'd settle for just dumping it to a file somewhere. Or maybe into the logs. -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander wrote: Was that really the conclusion? My memory of this thread showed that most people who actually deal with hashes and cryptography *wanted* a SHA based hash in core (because our users ask for it!) and the only disagreement was in *what* should be included. Sorry if it looked like I was jumping to conclusions on this. I was reporting the summary of the comments I had from the patch queue. Based on the subsequent discussion, it seems it was accurate none-the-less. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong
I wrote: ... What I propose therefore is combining this patch with my older one so that ANALYZE counts according to the following rules: REDIRECT line pointer:ignore DEAD line pointer:count as dead HEAPTUPLE_LIVE tuple: count as live, include in statistics pool HEAPTUPLE_DEAD: count as dead HEAPTUPLE_RECENTLY_DEAD: count as dead HEAPTUPLE_INSERT_IN_PROGRESS: ignore HEAPTUPLE_DELETE_IN_PROGRESS: count as live While working on this I realized that there's a special case ANALYZE has to face that is not faced by VACUUM: it might see tuples inserted or deleted by its own transaction. For example consider begin; ... load lots of data into mytable ... analyze mytable; ... issue complex queries against mytable ... commit; This is not an uncommon scenario, particularly with respect to temporary tables. ANALYZE's historical behavior of sampling everything that's good according to SnapshotNow does the right thing here, but ignoring INSERT_IN_PROGRESS tuples would not. The right way seems to be to treat our own insertions as live during ANALYZE, but then subtract off our own pending insertions from the live-tuples count sent to the stats collector. pgstat_report_analyze() can handle the latter part by groveling through the backend's pending statistics data. Comments? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Several tags around PostgreSQL 7.1 broken
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, April 02, 2008 17:49:49 -0400 Tom Lane [EMAIL PROTECTED] wrote: It doesn't, of course. What it does do is guarantee that the tarball matches the tag that has already been laid down in CVS. 'k, that was my thought, so using export vs update to create the tarbal is irrelevant to this discussion ... But there must have been more to it than that. Peter is reporting that the tag is on mutually inconsistent versions of some files; which means that the problem was with the tagging operation more than with the tarball-making. Has anyone actually checked the tarballs themselves? If the tag's are wrong, then doesn't it follow that the tarballs themselves are all wrong too? - -- Marc G. FournierHub.Org Hosting Solutions S.A. (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFH9BQr4QvfyHIvDvMRAjC9AJ9qBRom7aU7LWmZGnhfOFtbwv7zRQCgxPqx qv5B4ffClv4RRXc2FVg6LpI= =zjTy -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] writing a MIN(RECORD) aggregate
On Mar 25, 2008, at 11:33 AM, Sam Mason wrote: On Mon, Mar 24, 2008 at 05:27:04PM -0500, Decibel! wrote: On Mar 20, 2008, at 2:23 PM, Sam Mason wrote: SELECT i, (MIN((j,k))).k FROM tbl GROUP BY i; How is that any better than SELECT i, min(k) FROM tbl GROUP BY i ? Because I want the value of k associated with the minimum value of j. Ahh, makes sense. FWIW... SELECT i, (SELECT k FROM ... WHERE i = i.i ORDER BY j LIMIT 1) FROM (SELECT DISTINCT i FROM ...) i ; If you needed more than just k, I think there's a way to do that in the FROM clause, too. -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 smime.p7s Description: S/MIME cryptographic signature
Re: [HACKERS] Proposal: new ereport option errdetail_log
Decibel! wrote: I don't have any immediate ideas for improvement either, but we certainly shouldn't consider this a totally solved problem. Something I always find myself wanting when debugging locking issues is what's in pg_locks. Could we save that information somewhere when we detect a deadlock? In a real table would be nice, but I'd settle for just dumping it to a file somewhere. Or maybe into the logs. That option will be in 8.4, logging deadlock information. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch queue - wiki (was varadic patch)
Tom Lane wrote: For the past couple of weeks I've been dealing with both Bruce's queue and the one at http://wiki.postgresql.org/wiki/CommitFest:March and frankly I find the latter a *whole* lot more satisfactory, despite the fact that it's got exactly zero custom tooling or infrastructure behind it. It doesn't have artificial constraints on page organization; I can update it as soon as I've done something rather than waiting around for Bruce to do so; and there's an automatically maintained history of changes. Bruce has put a whole lot of man-hours into getting his page to do a few of the things we could do for free with the wiki page, but it's still got a long way to go. Now it would certainly be nice if there were some tools that would assist with dumping URLs for newly-arrived messages into the wiki page. Perhaps some of the pgsql-www crowd can think about how to do that. But even if we had to do that entirely by hand, I'd rather go with the wiki page. At some point it might be worth building the sort of heavy-duty infrastructure for the patch queue that you have in mind here. I don't think we need it yet, though, and I definitely don't think we understand our requirements well enough yet to start designing it. One of the reasons I like the wiki approach is exactly that there's such a low barrier to getting started with it. It is not clear to me how a wiki can be easily created for 2k emails and then maintained in a reasonable way, or how emails can be added to it easily. There are several steps: o getting those 2k emails to start the commit fest o getting them into a wiki in a way that is fast/efficient o updating the wiki for changes efficiently Keep in mind the patch emails are pretty dynamic. As you get closer to the end of the commit fest, the wiki is easier because the list of open items becomes more stable. I am able to give others the ability to add, move, and delete emails in my patch queue, if desired. If people want to use the wiki, go ahead --- this would be one less job for me to do. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] modules
Andrew Dunstan wrote: Tom Lane wrote: as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type ...re-raise the question of getting rid of contrib... The PostgreSQL Standard Modules. While renaming, could we go one step further and come up with a clear definition of what it takes for something to qualify as a module? In particular I think standardizing the installation would go a long way to letting packagers automate the installation of modules from pgfoundry. I think it'd be especially cool if one could one-day have a command pg_install_module [modulename] -d [databasename] and it would magically get (or verify that it had) the latest version from pgfoundry; compile it (if needed) and install it in the specified database. The closest analogy to what I'm thinking is the perl CPAN or ruby gems. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] modules
Ron Mayer wrote: Andrew Dunstan wrote: Tom Lane wrote: as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type ...re-raise the question of getting rid of contrib... The PostgreSQL Standard Modules. While renaming, could we go one step further and come up with a clear definition of what it takes for something to qualify as a module? In particular I think standardizing the installation would go a long way to letting packagers automate the installation of modules from pgfoundry. I think it'd be especially cool if one could one-day have a command pg_install_module [modulename] -d [databasename] and it would magically get (or verify that it had) the latest version from pgfoundry; compile it (if needed) and install it in the specified database. The closest analogy to what I'm thinking is the perl CPAN or ruby gems. Yes, and the CPAN analogy that has been in several minds, but it only goes so far. Perl and Ruby are languages - Postgres is a very different animal. We do in fact have some support for building / installing some modules in a standard way. It's called pgxs and it is used by quite a number of existing modules. One thing that might be worth looking at is an install command at the SQL level, so the INSTALL foo would run the install script for the foo module in the current database, assuming it's in the standard location. We don't have a central repository of non-standard modules, like CPAN, and so of course no facility for fetching / building / installing them. Not all modules fit a single pattern, either. There are addon languages, types, and function libraries, as we all as utilities that are not installed in the database at all. Finally, setting up modules so they can be built for Windows, especially using MSVC, will probably be quite a challenge. So if someone wants to make a start on any of this I'm sure we would all listen up. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] column level privileges
It would be great help to me, and I am sure for many other people too who are working with security solutions, if this feature is released as patch before 8.4 release. Sanjay Sharma Date: Tue, 1 Apr 2008 22:02:30 -0400 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] CC: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] column level privilegesThe earliest will be 8.4, which is many many months away. It should be possible to produce a patch for 8.3 if you're interested. cheers andrew sanjay sharma wrote: Hello Andrew, When do you expect this patch to go in production and available for public use? I would keep an eye for its release.Sanjay SharmaDate: Tue, 1 Apr 2008 18:40:24 -0400 From: [EMAIL PROTECTED] To: pgsql-hackers@postgresql.org Subject: [HACKERS] column level privileges Apologies if this gets duplicated - original seems to have been dropped due to patch size - this time I am sending it gzipped. cheers andrew Original Message Subject: column level privileges Date: Tue, 01 Apr 2008 08:32:25 -0400 From: Andrew Dunstan [EMAIL PROTECTED] To: Patches (PostgreSQL) [EMAIL PROTECTED] This patch by Golden Lui was his work for the last Google SoC. I was his mentor for the project. I have just realised that he didn't send his final patch to the list. I guess it's too late for the current commit-fest, but it really needs to go on a patch queue (my memory on this was jogged by Tom's recent mention of $Subject). I'm going to see how much bitrot there is and see what changes are necessary to get it to apply. cheers andrew - Here is a README for the whole patch. According to the SQL92 standard, there are four levels in the privilege hierarchy, i.e. database, tablespace, table, and column. Most commercial DBMSs support all the levels, but column-level privilege is hitherto unaddressed in the PostgreSQL, and this patch try to implement it. What this patch have done: 1. The execution of GRANT/REVOKE for column privileges. Now only INSERT/UPDATE/REFERENCES privileges are supported, as SQL92 specified. SELECT privilege is now not supported. This part includes: 1.1 Add a column named 'attrel' in pg_attribute catalog to store column privileges. Now all column privileges are stored, no matter whether they could be implied from table-level privilege. 1.2 Parser for the new kind of GRANT/REVOKE commands. 1.3 Execution of GRANT/REVOKE for column privileges. Corresponding column privileges will be added/removed automatically if no column is specified, as SQL standard specified. 2. Column-level privilege check. Now for UPDATE/INSERT/REFERENCES privilege, privilege check will be done ONLY on column level. Table-level privilege check was done in the function InitPlan. Now in this patch, these three kind of privilege are checked during the parse phase. 2.1 For UPDATE/INSERT commands. Privilege check is done in the function transformUpdateStmt/transformInsertStmt. 2.2 For REFERENCES, privilege check is done in the function ATAddForeignKeyConstraint. This function will be called whenever a foreign key constraint is added, like create table, alter table, etc. 2.3 For COPY command, INSERT privilege is check in the function DoCopy. SELECT command is checked in DoCopy too. 3. While adding a new column to a table using ALTER TABLE command, set appropriate privilege for the new column according to privilege already granted on the table. 4. Allow pg_dump and pg_dumpall to dump in/out column privileges. 5. Add a column named objsubid in pg_shdepend catalog to record ACL dependencies between column and roles. 6. modify the grammar of ECPG to support column level privileges. 7. change psql's \z (\dp) command to support listing column privileges for tables and views. If \z(\dp) is run with a pattern, column privileges are listed after table level privileges. 8. Regression test for column-level privileges. I changed both privileges.sql and expected/privileges.out, so regression check is now all passed. Best wishes Dong -- Guodong Liu Database Lab, School of EECS, Peking University Room 314, Building 42, Peking University, Beijing, 100871, China Exclusive Marriage Proposals! Find UR life partner at Shaadi.com Try it! http://ss1.richmedia.in/recurl.asp?pid=430 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers _ Tried the new MSN Messenger? It’s cool! Download now. http://messenger.msn.com/Download/Default.aspx?mkt=en-in
Re: [HACKERS] [PATCHES] psql slash# command
Based on recent patch feedback from Tom, this has been saved for the next commit-fest: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Tom Lane wrote: Sibte Abbas [EMAIL PROTECTED] writes: On 9/9/07, Sibte Abbas [EMAIL PROTECTED] wrote: Attached is the patch for the TODO item mentioned at http://archives.postgresql.org/pgsql-hackers/2007-09/msg00352.php I looked this over and realized that it has little to do with the functionality that was so painfully hashed out in the original discussion thread here: http://archives.postgresql.org/pgsql-hackers/2006-12/msg00207.php As I understood it, the consensus was: 1. Invent a switch (probably a variable instead of a dedicated \-command) that determines whether \s includes command numbers in its output. 2. Add \# n to re-execute command number n. You've twisted this around into \#: displays the command history. Like \s but prefixes the lines with line numbers \# line_no: executes the command(if any) executed at the line specified by line_no This is a serious regression in functionality from what was agreed to, because there is no possibility of shoehorning the equivalent of \s file into it --- you've already decided that any argument is a line number. It also seems to me to be pretty unintuitive and even dangerous that the same \-command would do *fundamentally* different things depending on whether it has an argument or not. Especially if one of those things involves executing an arbitrary SQL-command. The attached patch adds the following new functionality: \#e lineno: Will open the command at the given lineno in an editor. \#e with no lineno will behave exactly like \e. None of that was anywhere in the original discussion; and what pray tell is the use of the second variant? I wonder whether it wouldn't be safer and more convenient if we defined '\# n' as pulling command n into the edit buffer, rather than immediately executing it. Actual execution is only a return away, but this definition would allow you to edit the command a bit more before you execute it --- including \e to use an editor. It also closes the loop in terms of providing some confidence that you typed the number you should have typed. BTW, not related to the original discussion, but I fail to understand how anyone finds \s useful interactively, when it doesn't paginate its output. Shouldn't we fix that? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] column level privileges
Postgres does not backport features, so you would need to retrofit the
patch to 8.3 yourself, or pay / persuade somebody else to do that for
you. That should not be too hard, as it was in fact developed late in
the 8.3 cycle.
Before you jump on it as suiting your needs, read carefully. In
particular, take note of the fact that it is SQL92 privileges, which
specifically do NOT include SELECT restrictions.
cheers
andrew
sanjay sharma wrote:
It would be great help to me, and I am sure for many other people too
who are working with security solutions, if this feature is released
as patch before 8.4 release.
Sanjay Sharma
Date: Tue, 1 Apr 2008 22:02:30 -0400
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
CC: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] column level privileges
The earliest will be 8.4, which is many many months away.
It should be possible to produce a patch for 8.3 if you're interested.
cheers
andrew
sanjay sharma wrote:
Hello Andrew,
When do you expect this patch to go in production and available for
public use? I would keep an eye for its release.
Sanjay Sharma
Date: Tue, 1 Apr 2008 18:40:24 -0400
From: [EMAIL PROTECTED]
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] column level privileges
Apologies if this gets duplicated - original seems to have been
dropped
due to patch size - this time I am sending it gzipped.
cheers
andrew
Original Message
Subject: column level privileges
Date: Tue, 01 Apr 2008 08:32:25 -0400
From: Andrew Dunstan [EMAIL PROTECTED]
To: Patches (PostgreSQL) [EMAIL PROTECTED]
This patch by Golden Lui was his work for the last Google SoC. I
was
his
mentor for the project. I have just realised that he didn't send his
final patch to the list.
I guess it's too late for the current commit-fest, but it really
needs
to go on a patch queue (my memory on this was jogged by Tom's recent
mention of $Subject).
I'm going to see how much bitrot there is and see what changes are
necessary to get it to apply.
cheers
andrew
-
Here is a README for the whole patch.
According to the SQL92 standard, there are four levels in the
privilege
hierarchy, i.e. database, tablespace, table, and column. Most
commercial
DBMSs support all the levels, but column-level privilege is hitherto
unaddressed in the PostgreSQL, and this patch try to implement it.
What this patch have done:
1. The execution of GRANT/REVOKE for column privileges. Now only
INSERT/UPDATE/REFERENCES privileges are supported, as SQL92
specified.
SELECT privilege is now not supported. This part includes:
1.1 Add a column named 'attrel' in pg_attribute catalog to store
column privileges. Now all column privileges are stored, no matter
whether they could be implied from table-level privilege.
1.2 Parser for the new kind of GRANT/REVOKE commands.
1.3 Execution of GRANT/REVOKE for column privileges. Corresponding
column privileges will be added/removed automatically if no
column is
specified, as SQL standard specified.
2. Column-level privilege check.
Now for UPDATE/INSERT/REFERENCES privilege, privilege check will be
done ONLY on column level. Table-level privilege check was done
in the
function InitPlan. Now in this patch, these three kind of
privilege are
checked during the parse phase.
2.1 For UPDATE/INSERT commands. Privilege check is done in the
function transformUpdateStmt/transformInsertStmt.
2.2 For REFERENCES, privilege check is done in the function
ATAddForeignKeyConstraint. This function will be called whenever a
foreign key constraint is added, like create table, alter table,
etc.
2.3 For COPY command, INSERT privilege is check in the function
DoCopy. SELECT command is checked in DoCopy too.
3. While adding a new column to a table using ALTER TABLE
command, set
appropriate privilege for the new column according to privilege
already
granted on the table.
4. Allow pg_dump and pg_dumpall to dump in/out column privileges.
5. Add a column named objsubid in pg_shdepend catalog to record ACL
dependencies between column and roles.
6. modify the grammar of ECPG to support column level privileges.
7. change psql's \z (\dp) command to support listing column
privileges
for tables and views. If \z(\dp) is run with a pattern, column
privileges are listed after table level privileges.
8. Regression test for column-level privileges. I changed both
privileges.sql and expected/privileges.out, so regression check
is now
all passed.
Best wishes
Dong
--
Guodong Liu
Database Lab, School of EECS, Peking University
Room 314, Building 42, Peking University, Beijing, 100871, China
Exclusive
Re: [HACKERS] printTable API (was: Show INHERIT in \du)
The author has been given feedback so this has been saved for the next commit-fest: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Brendan Jurd wrote: On 31/03/2008, Tom Lane [EMAIL PROTECTED] wrote: There isn't any functional difference there. I am not sure, but I think the reason print.c has its own malloc wrappers instead of depending on common.c's is that we use print.c in some bin/scripts/ programs that do not want common.c too. Okay, thanks (to Heikki as well) for the clarification. It's good to know they are functionally equivalent. I'll do some snooping in /scripts to get a better view of the situation. 2. describe only does an mbvalidate for WIN32, but print does it in all cases. I don't know why describe only does that for WIN32; it looks inconsistent to me too. Possibly some trolling in the CVS history would give a clue about this. Alright, I'll be spending some quality time with 'annotate' then =) If you're not actively working on this patch right now, I am going to go ahead and commit the other open patches for describe.c. If you do have a patch in progress, I'm willing to hold off to avoid any merge conflicts. Let me know. I didn't get much beyond sketching out my struct. Now that I have answers to the questions I raised above, I can push forward with the patch, but I wouldn't expect to have anything to submit for another couple of days at least. Short answer: I have zero objections to you committing those patches. Thanks for your time, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] modules
Andrew Dunstan wrote: I think it'd be especially cool if one could one-day have a command pg_install_module [modulename] -d [databasename] Yes, and the CPAN analogy that has been in several minds, but it only goes so far. Perl and Ruby are languages - Postgres is a very different animal. Sure - but the benefits of standardizing installers for optional components seems to apply the same for both. We do in fact have some support for building / installing some modules in a standard way. It's called pgxs and it is used by quite a number of existing modules. Cool. Seems to handle at least quite a bit of the building part of standardized modules. One thing that might be worth looking at is an install command at the SQL level, so the INSTALL foo would run the install script for the foo module in the current database, assuming it's in the standard location. I'm guessing that this would be harder to add various options (install/ignore dependancies ; specify a different source web site) that a standard installer would like to have. We don't have a central repository of non-standard modules, like CPAN, and so of course no facility for fetching / building / installing them. Seems that could easily be improved in a number of ways. * The installer could specify the source. For example pg_install_module postgis -source http://www.refractions.net in exactly the same way ruby uses gem install rails –source http://gems.rubyonrails.org * pgfoundry could provide a repository of installable modules for projects hosted there. * perhaps pgfoundry could even have a section where it points to installers on third party sites? Not all modules fit a single pattern, either. There are addon languages, types, and function libraries, as we all as utilities that are not installed in the database at all. Agreed. Such a mechanism would only really apply for things that are installed in the database. But from an end user's point of view, installing functions, index types, languages, data types, etc all see to fit the pg_install postgis -d mydb, pg_install pl_ruby -d mydb, etc. pattern pretty well. Finally, setting up modules so they can be built for Windows, especially using MSVC, will probably be quite a challenge. Indeed. Seems ruby gems give you the option of installing a ruby version or a windows version that I'm guessing has pre-compiled object files. So if someone wants to make a start on any of this I'm sure we would all listen up. I'm happy to try, though might need pointing in the right directions. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] modules
On Wed, 02 Apr 2008 20:15:49 -0400 Andrew Dunstan [EMAIL PROTECTED] wrote: I think it'd be especially cool if one could one-day have a command pg_install_module [modulename] -d [databasename] and it would magically get (or verify that it had) the latest version from pgfoundry; compile it (if needed) and install it in the specified database. The closest analogy to what I'm thinking is the perl CPAN or ruby gems. Check out NetBSD pkgsrc as a model. It is very flexible. One nice thing would be the ability to specify where the packages are rather than always insisting that they be on pgfoundry. Yes, and the CPAN analogy that has been in several minds, but it only goes so far. Perl and Ruby are languages - Postgres is a very different animal. So the underlying struture needs to keep that in mind. Overall though I don't think that what is being installed to changes much. The basics remain the same - define the package with latest version, download if necessary,check that the source package is the correct, tested one, build, install, register. There are some special considerations for PostgreSQL but I think that the fact that there are unsolved problems shouldn't stop us from solving them. -- D'Arcy J.M. Cain [EMAIL PROTECTED] | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for pg_dump (function dumps)
The author has received feedback so this has been saved for the next commit-fest: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Stephen Frost wrote: -- Start of PGP signed section. * Dany DeBontridder ([EMAIL PROTECTED]) wrote: I often need in command line to get the code of function, so I make a patch for pg_dump, thanks this patch pg_dump is able to dump only one functions or all the functions. First, a couple of general comments about the patch: #1: You need to read the developer FAQ located here: http://www.postgresql.org/docs/faqs.FAQ_DEV.html Particularly question 1.5, which discusses how a patch should be submitted. #2: The patch should be as readable as possible. This includes not making gratuitous whitespace changes (which are in a number of places and just confuse things), comments like this: /* Now we can get the object ?? */ also don't make for very easy reading. #3: The patch should be in contextual diff format, not unified diff. #4: Re-use existing structure and minimize code duplication While I can understand some desire to restructure pg_dump code to handle things as generalized objects, this patch doesn't actually go all the way through and do that. Instead it starts that work, only adds support for functions, and then leaves the old methods and whatnot the same. Instead it should either be a large overhaul (probably not necessary for the specific functionality being looked for here) which is complete and well tested (and removes the old, no longer used code), or it should be integrated into the existing structure (which is what I would recommend here). Given that both the new approach and the old were left after this patch, there's some code duplication and really process duplication happening. #5: Given the above, I would suggest making '-B' explicitly for functions and drop the 'function:' heading requirement. #6: Passing an sql snippet to getFuncs to do the filtering strikes me as a really terrible approach. Instead, the approach used for schemas and tables is much cleaner and using it would make it be consistant with those other types. #7: Again, following with the existing approach, the schemas and tables use global variables to pass around what to include/exclude. Unless you're going to rewrite the whole thing to not do that, you should follow that example when adding support for functions. eg, getFuncs really doesn't/shouldn't need to have its function definition changed. #8: Functions *can* be mixed-case, I'm pretty sure, and pg_dump should definitely support that. These kinds of issues would have been handled for you if you had used processSQLNamePattern as the other functions do. This would also remove the need for the pg_strcat, pg_free functions you've added, I believe. #9: The vast majority of the code doesn't use 'pg_malloc' and so I would hesitate to add more things which use it, and to add more pg_X functions which then *also* are rarely used. If it makes sense to have pg_malloc/pg_free (and I'm not sold on that idea at all), then it should be done consistantly, and probably seperately, from this. This is probably enough. My general feeling about this patch is that it needs a rewrite and to be done using the existing structures and following the same general processes we use for tables. The resulting code should be consistant and at least look like it was all written towards a specific, defined structure. That makes the code much more maintainable and easier to pick up since you only have to understand one structure which can be well documented rather than multiple not fully thought out or documented structures. As such, I would recommend rejecting this patch for this round and waiting for a rewrite of it which can be reviewed during the next commit-fest. Thanks, Stephen -- End of PGP section, PGP failed! -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TRUNCATE TABLE with IDENTITY
Decibel! írta: On Mar 25, 2008, at 11:40 AM, Zoltan Boszormenyi wrote: All of them? PostgreSQL allow multiple SERIALs to be present, the standard allows only one IDENTITY column in a table. And what about this case below? CREATE TABLE t1 (id1 serial, ...); ALTER SEQUENCE seq_t1_id1 RESTART WITH 5432 CYCLE; or the equivalent CREATE SEQUENCE seq_t1_id1 START WITH 5432 CYCLE; CREATE TABLE t1 (id1 serial, ...); ALTER SEQUENCE seq_t1_id1 OWNED BY t1.id1; PostgreSQL doesn't keep the START WITH information. But it should to perform a restart on the sequence, using the minval in this case wouldn't be correct. I think you misunderstand what ALTER SEQUENCE RESTART does; it only changes the current value of the sequence. I didn't misunderstood, I know that. I quoted both because (currently) CREATE SEQUENCE ... START WITH does the same. zozo= create sequence seq1 start with 327; CREATE SEQUENCE zozo= select * from seq1; sequence_name | last_value | increment_by | max_value | min_value | cache_value | log_cnt | is_cycled | is_called ---++--+-+---+-+-+---+--- seq1 |327 |1 | 9223372036854775807 | 1 | 1 | 1 | f | f (1 row) Note the difference between min_value and last_value. Using the standard syntax of CREATE TABLE ( id integer IDENTITY GENERATED ALWAYS AS (START WITH 327), ... ); and assuming you use the existing sequence infrastructure there's a problem with TRUNCATE ... RESTART IDENTITY; Where is the info in the sequence to provide restarting with the _original_ start value? -- -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
