Re: [PATCHES] review: table function support
On 7/10/08, Pavel Stehule [EMAIL PROTECTED] wrote:
I am sending actualized patch
Regards
Pavel Stehule
2008/7/9 Pavel Stehule [EMAIL PROTECTED]:
2008/7/9 Marko Kreen [EMAIL PROTECTED]:
Generally, the patch looks fine. There are few issues still:
- plpgsql: the result columns _do_ create local variables.
AIUI, they should not?
it was my mistake - it doesn't do local variables - fixed
- pg_dump: is the psql_assert() introduction necessary, considering it
is used only in one place?
removed - argmode variables is checked before
- There should be regression test for plpgsql too, that test if
the behaviour is correct.
addeded
- The documentation should mention behaviour difference from OUT
parameters.
I will do it.
Wishlist (probably out of scope for this patch):
this is in my wishlist too, but postgresql doesn't support types like
result of functions.
- plpgsql: a way to create record variable for result row. Something
like:
CREATE FUNCTION foo(..) RETURNS TABLE (..) AS $$
DECLARE
retval foo%ROWTYPE;
Currently the OUT parameters are quite painful to use due to bad
name resolving logic. Such feature would be perfect replacement.
--
marko
I'll send patch early, thank you much
Ok, last items:
- Attached is a patch that fixes couple C comments.
- I think plpgsql 38.1.2 chapter of Supported Argument and Result Data
Types should also have a mention of TABLE functions.
Then I'm content with the patch.
--
marko
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3f90c93..6ae4521 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -6847,7 +6847,7 @@ dumpFunc(Archive *fout, FuncInfo *finfo)
appendPQExpBuffer(q, CREATE FUNCTION %s , funcsig);
- /* swich between RETURNS SETOF RECORD and RETURNS TABLE functions */
+ /* switch between RETURNS SETOF RECORD and RETURNS TABLE functions */
if (!is_returns_table_function(nallargs, argmodes))
{
rettypename = getFormattedTypeName(finfo-prorettype, zeroAsOpaque);
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 612a84f..c683aea 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -402,7 +402,7 @@ do_compile(FunctionCallInfo fcinfo,
PLpgSQL_variable *argvariable;
int argitemtype;
-/* skip PROARGMODE_TABLE params - these params hasn't local variables */
+/* PROARGMODE_TABLE params should not create local variables */
if (argmode == PROARGMODE_TABLE)
continue;
--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches
Re: [PATCHES] pg_dump lock timeout
On 5/11/08, daveg [EMAIL PROTECTED] wrote: Attached is a patch to add a commandline option to pg_dump to limit how long pg_dump will wait for locks during startup. My quick review: - It does not seem important enough to waste a short option on. Having only long option should be enough. - It would be more polite to do SET LOCAL instead SET. (Eg. it makes safer to use pg_dump through pooler.) - The statement_timeout is set back with statement_timeout = default Maybe it would be better to do = 0 here? Although such decision would go outside the scope of the patch, I see no sense having any other statement_timeout for actual dumping. -- marko -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [PATCHES] updated hash functions for postgresql v1
On 4/6/08, Tom Lane [EMAIL PROTECTED] wrote: So adopting the mixing changes would make it faster yet. What we need to be certain of is that this doesn't expose us to poorer hashing. We know that it is critical that all bits of the input affect all bits of the hash fairly uniformly --- otherwise we are subject to very serious performance hits at higher levels in hash join, for instance. The comments in the new code led me to worry that Jenkins had compromised on that property in search of speed. I looked at his website but couldn't find any real discussion of the design principles for the new mixing code ... Scroll at the end of doobs.html, there is the longer discussion. My understanding is following: His design is based on 2 properties of mixing function: - reversible - that means for each input tuple of (a,b,c) corresponds exactly one output tuple of (a,b,c). Such property guarantees that after repeatedly applying mixing function, no bits get lost. - avalanche - that means any single bit change in input (a,b,c) half of the output bits are affected. His insight (as he called it) when creating lookup3 was that the bulk mixing that is applied repeatedly does not need to have avalanche, it only needs to be reversible, meaning all the bits that went in, are still there after mixing repeatedly. And only final mixing needs to have avalanche as it produces final result, but it does not need to be reversible as it wont be applied repeatedly and most of the result is dropped anyway. IMHO his choices are reasonable. -- marko -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
[PATCHES] minor pgcrypto doc update
- s/2440/4880/ - mention all built-in hashes - s/pgcrypto/PostgreSQL/ in one place - no need to mention 2440bis anymore Explanation - pgp code was written from the start to conform with rfc2440bis (now rfc4880), so any mention of 2440 is obsolete. -- marko pgcrypto.doc.cleanup.diff Description: Binary data ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython crash on exception
On 11/23/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: On 11/23/07, Tom Lane [EMAIL PROTECTED] wrote: Why? I can't imagine any real use for it. If you're thinking that it could provide a guide as to what to resize the buffer to, think again. If the output was truncated due to this limit then the return value is the number of characters (not including the trailing '\0') which would have been written to the final string if enough space had been available. What problem do you see? The problem is that you are quoting from some particular system's manual, and not any kind of standard ... much less any standard that every platform we support follows. The real-world situation is that we are lucky to be able to tell vsnprintf success from failure at all :-( Ah, ok. I just saw the result used inside the function, so I thought it is standard enough. Actually, the meaning could be changed to *needmore and compensated inside function: *needmore = (nprinted buf-maxlen) ? buf-maxlen : nprinted + 1; Then it would not matter if libc is conforming or not. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] Cleaner API for appendStringInfoVA
On 11/23/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: FWIW, SUS says that vsnprintf should act like snprintf and snprintf: I dunno where you're reading that, but it's certainly nowhere to be found in the version that I read: http://www.opengroup.org/onlinepubs/007908799/xsh/vfprintf.html http://www.opengroup.org/onlinepubs/009695399/functions/vsnprintf.html Seems its clarified in SUSv3. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] Cleaner API for appendStringInfoVA
On 11/23/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Attached patch moves decision how much more room to allocate from callers of appendStringInfoVA to inside the function, where more info is available. This is by no stretch of the imagination cleaner. Cleaner as in avoiding unnecessary work and avoiding unnecessary guesswork for callers when in can be done in function. Also conforming to current coding standards, see below. On systems with broken vsnprintf() it falls back to doubleing the buffer. The problem with this is that you are defining one particular vsnprintf behavior as non broken, without any evidence for that opinion. (Indeed, one could argue that that behavior is contradictory to what the Single Unix Spec says, although the SUS is a bit vague about it.) FWIW, SUS says that vsnprintf should act like snprintf and snprintf: The snprintf() function shall be equivalent to sprintf(), with the addition of the n argument which states the size of the buffer referred to by s. If n is zero, nothing shall be written and s may be a null pointer. Otherwise, output bytes beyond the n-1st shall be discarded instead of being written to the array, and a null byte is written at the end of the bytes actually written into the array. RETURN VALUE: Upon successful completion, the snprintf() function shall return the number of bytes that would be written to s had n been sufficiently large excluding the terminating null byte. Our own vsnprintf doesn't follow that behavior, for instance, so we couldn't even get there by forcing it to be used always. It's one of those broken implementations then. I'd want to see some significant evidence of a performance issue before considering hacking this up like this. *shrug* It's a minor cleanup. I think it's worthwhile to remove historical warts from code but if you are not interested, no problem. Are you interested in fixing src/port/snprintf.c behaviour? I can prepare a patch, it does not seem to be very hard. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[PATCHES] Cleaner API for appendStringInfoVA
Attached patch moves decision how much more room to allocate from callers of appendStringInfoVA to inside the function, where more info is available. On systems with broken vsnprintf() it falls back to doubleing the buffer. Fixme: the +1 could be something larger? Aligned? -- marko cleaner_append_va.diff Description: Binary data ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[PATCHES] plpython crash on exception
Following function crashes plpython on x86-64 / gcc 4.1.2 / debian 4.0: CREATE FUNCTION crashme(str_len integer) RETURNS text AS $$ raise Exception(X * str_len) $$ LANGUAGE plpythonu; SELECT crashme(1000); Problem turns out to be va_list handling in PLy_vprintf() which uses same va_list repeatedly. Fix is to va_copy to temp variable. Additionally the atteched patch fixes 2 more problems in that function: - its nonsensical to check existing buffer length for 0, instead the function result should be checked. (which for vsnprintf() should always be 0, but maybe there are non-standard systems out there?) - the * sizeof(char) in malloc() is pointless - even if we want to support systems where sizeof(char) != 1, current code is wrong as from by reading of manpage, vsnprintf() takes buffer length in bytes but returns chars, so the 'blen' must be bytes anyway and the sizeof(char) must be in line: blen = bchar + 1; The function seems to be essentially same since 7.2 so the patch should apply to all branches. If you prefer you can apply cleanups to HEAD only. -- marko plpy.vprintf.diff Description: Binary data ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython crash on exception
On 11/23/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Just a note - appendStringInfoVA should take *nprinted argument. Why? I can't imagine any real use for it. If you're thinking that it could provide a guide as to what to resize the buffer to, think again. It seems perfectly appropriate: If the output was truncated due to this limit then the return value is the number of characters (not including the trailing '\0') which would have been written to the final string if enough space had been available. What problem do you see? Note that I don't want to fix unnecessary memory usage with that but several reallocs in case some args are large. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython crash on exception
On 11/22/07, Alvaro Herrera [EMAIL PROTECTED] wrote: One problem here is that 7.3 does not have appendStringInfoVA. From 7.4 onwards there is no problem. Should I backport the code from 7.4? (On a first try to do it I introduced a nasty bug, so it's not as easy as a cut'n paste). Just a note - appendStringInfoVA should take *nprinted argument. Can this be added in 8.4? Should make usage saner. -- marko ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] plpython crash on exception
On 11/23/07, Alvaro Herrera [EMAIL PROTECTED] wrote: Marko Kreen escribió: Problem turns out to be va_list handling in PLy_vprintf() which uses same va_list repeatedly. Fix is to va_copy to temp variable. Marko, I just applied what I posted earlier today from 7.4 to 8.2 and HEAD. I assume you had a more complex test case; please have a try with the code in CVS and let me know. Thanks, I'll test it. (If you are really interested in a fix for 7.3, let me know too ... I couldn't even get plpython to run a trivial function due to RExec issues. I didn't try 7.2). RExec seems to hint that 7.3 can work only with python 2.2 and below. What version did you try it on? But no, I don't personally care about 7.3... -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] hashlittle(), hashbig(), hashword() and endianness
On 11/16/07, Alex Vinokur [EMAIL PROTECTED] wrote: I would like to use both hashlittle() and hashword() (or hashword_little) on little-endian and big-endian machine and to get identical hashValues. Whats wrong with hashlittle()? It does use the same optimized reading on LE platform that hashword() does. Or you could wrap the read values with some int2le() macro that is NOP on LE cpu. Although I suspect the performance wont be better than using hashlittle() directly. -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] txid cleanup
Per Tom comments, attached patch does following: - Binary I/O support - Use ActiveSnapshot instead of SerializableSnapshot - Clarify function purposes - remove dev targets from Makefile There should not be any code depending on SerializableSnapshot, if there is thats a bug that needs fixing. Fix is easy - use SERIALIZABLE transaction isolation level. Also - either Tom or Jan has changed the code to allow xmin == xmax. Is such case really valid? In _recv code I followed the style, but I was just wondering. -- marko txid.cleanup.diff Description: Binary data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [RFC] extended txid docs
On 10/17/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Thanks. Here is a version with your changes applied, plus minor code cleanup and example output. I can't really see the reasoning for putting this into the PG documentation. It's tremendously complicated and doesn't seem like something very many people would want to read about. In any case it seems rather out of place where it is --- we don't have large code examples elsewhere in func.sgml. It almost looks like something that should be turned into a pgfoundry or contrib module. The whole point of the functions it to allow doing snapshot-based queries. It is indeed tricky, but that increases the need for documentaton, no? I think the last more realistic code section can be dropped, it shows more user-friendly function but adds nothing new, and the code is rather unreadeable. -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [RFC] extended txid docs
On 10/16/07, Chris Browne [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Marko Kreen) writes: Even the realistic code may be too much for general docs, but considering this is not a functionality covered by general SQL textbooks, I think it is worth having. I also put rendered pages up here: http://skytools.projects.postgresql.org/txid/datatype-txid-snapshot.html http://skytools.projects.postgresql.org/txid/functions-txid.html I believe that those suggested texts describe what you intended, and they should represent better English text for this. Thanks. Here is a version with your changes applied, plus minor code cleanup and example output. I uploaded full docs to above urls, should be easier to browse. -- marko txid.docs.v2.diff.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[PATCHES] [RFC] extended txid docs
Although the new txid functions are very clean 1:1 interface to the internal MVCC info and they don't need much docs in that respect, their killer usage comes from the possibility to query txids committed between 2 snapshots. But how to do that (efficiently) is far from obvious when just looking at the API. So with attached docs patch I try to fill the gap. Here I also show 2 variants for the common query helper function. But I'm pretty bad at SGML, english and writing docs, so please review it. In addition to english/typos/sgml the suspicious aspects are: - code style - writing style - used mostly PgQ terminology (ticks), could there be something better? - giving two variants of helper function may be too much Even the realistic code may be too much for general docs, but considering this is not a functionality covered by general SQL textbooks, I think it is worth having. I also put rendered pages up here: http://skytools.projects.postgresql.org/txid/datatype-txid-snapshot.html http://skytools.projects.postgresql.org/txid/functions-txid.html -- marko txid-docs.diff.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[PATCHES] txid cleanup
Per Tom comments, attached patch does following: - Binary I/O support - Use ActiveSnapshot instead of SerializableSnapshot - Clarify function purposes - remove dev targets from Makefile There should not be any code depending on SerializableSnapshot, if there is thats a bug that needs fixing. Fix is easy - use SERIALIZABLE transaction isolation level. Also - either Tom or Jan has changed the code to allow xmin == xmax. Is such case really valid? In _recv code I followed the style, but I was just wondering. -- marko txid.cleanup.diff Description: Binary data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] Patch: txid in core
Just in case there is initdb required in beta2, here is patch that adds txid into core. Otherwise please register this as submission to 8.4. I'd like to avoid any process related discussions in the future... It is syned with the latest patch I sent to -patches. The docs are minimal, but I think first the /contrib doc should be fleshed out more. -- marko txid-core-v1.diff.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] txid cleanup
On 10/11/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Also - either Tom or Jan has changed the code to allow xmin == xmax. Is such case really valid? Yes, at least in CVS HEAD. Aha, I had missed that. Thanks for confirmation. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[PATCHES] txid strtoull fix
I noticed strtoull() causes problems on some buildfarm machines. As a fix I propose small str2num function inside txid.c itself. Main reason is that the txid actually does not need fully-featured strtoull (radixes, whitespace skipping). We could include strtoull() under port/ in the future, I think that would be good, but just for txid it is not necessary. -- marko txid.strtoull.diff Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] txid strtoull fix
On 10/8/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: I noticed strtoull() causes problems on some buildfarm machines. As a fix I propose small str2num function inside txid.c itself. Main reason is that the txid actually does not need fully-featured strtoull (radixes, whitespace skipping). Seems like a sane solution to me --- applied. (Note that your overflow test didn't actually work; AFAIK there really isn't any decent solution other than dividing at each step.) Hmm. It did seem to work here. Just in case I peeked into FreeBSD strtoull() and this patch imitates their method. This also seems to work, but please review, I'm don't feel very sharp anymore... -- marko txid.overflow.diff Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] OpenSSL Applink
On 9/28/07, Tom Lane [EMAIL PROTECTED] wrote: Dave Page [EMAIL PROTECTED] writes: Andrew Dunstan wrote: Then I think I'd rather disable use of client certs for the offending openssl versions in libpq, or let the apps die and refer the customers to the openssl people to lobby them for a sane solution. If this were 8.0 I'd agree, but thats not a nice solution for those already using client certs (such as the pgAdmin user who brought this to my attention). Doesn't really matter. Even if we were willing to hack our own client apps like that (which I'm not), we can *not* transfer such a requirement onto every libpq-using application. It's just not acceptable. Is it possible to use GNUTLS on Windows? -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] pgcrypto: fix for broken solaris openssl, v03
solaris openssl refuses to handle keys longer than 128bits. * aes will crash on longer keys * blowfish will silently cut the key which can result data corruption to fix it: - test errors from AES functions - bf errors cannot be tested, do test encryption - change aes compat macros to static function so they can return values (Original patch by Zdenek Kotala) I really dislike the patch, as it too specific for the crippled openssl in solaris. But still something should be done because of the possible silent corruption. More general appriaches that also fix the problems are: - test all ciphers on first use and test fails then disable completely. This is nice as it could detect much braded range of errors. Problem with this approach is that its too big overhead for small gain, as it cannot still 100% guarantee that everything is working correctly. - Use EVP functions for encryption as they have better error handling. So crippled openssl can report via regular means that something is not supported. Problem with this is that it can be done only from 0.9.7 onwards. Which pushes the solution to 8.4. So something like the current patch should be still applied as a near-term fix. But I'm starting to think that the blowfish check should be #ifdef __solaris__ only. Has anyone good reasons why it should apply to everyone? -- marko solaris-fix-v03.diff Description: Binary data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] Lazy xid assignment V3
On 9/3/07, Alvaro Herrera [EMAIL PROTECTED] wrote: Florian G. Pflug wrote: Since we didn't really reach an agreement on how xid_age should behave, I've reverted it back to the original version. So with this patch, xid_age will just force assignment of a xid. Is this really a good idea? I'm repeating myself, but a query like select age(xmin) from bigtable could accelerate Xid wraparound. If the server is running close to the limit it could cause a shutdown to prevent the actual wraparound. Such query would take only one xid, which should not be a problem? -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Lazy xid assignment V3
On 9/3/07, Alvaro Herrera [EMAIL PROTECTED] wrote: Marko Kreen escribió: On 9/3/07, Alvaro Herrera [EMAIL PROTECTED] wrote: Florian G. Pflug wrote: Since we didn't really reach an agreement on how xid_age should behave, I've reverted it back to the original version. So with this patch, xid_age will just force assignment of a xid. Is this really a good idea? I'm repeating myself, but a query like select age(xmin) from bigtable could accelerate Xid wraparound. If the server is running close to the limit it could cause a shutdown to prevent the actual wraparound. Such query would take only one xid, which should not be a problem? My guess is that it would execute age(xid) once per tuple? Even if all the tuples had the same xmin, there's no cache therefore it would consume as many Xids as there are tuples. Am I missing something? First age() assigns CurrentXid, rest reuse it. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [BUGS] BUG #3571: call to decrypt causes segfault
On 8/23/07, Tom Lane [EMAIL PROTECTED] wrote:
Ken Colson [EMAIL PROTECTED] writes:
this statement:
select decrypt(''::bytea,'password','bf')
causes the postgresql backend to crash:
This seems to be a 64bit problem.
Reproduced here in HEAD. The problem is here:
293 pad = res[*rlen - 1];
The problem clearly is that combo_decrypt()'s depadding code fails to
consider the possibility of a zero-length input, but I'm not entirely
sure how far up the food chain we ought to fix it --- perhaps
pg_decrypt() should not have bothered to light up the decryptor at all?
The fix should be in combo_decrypt() because other code
should not need to guess whether zero-length input is
allowed or not.
Patch attached.
Also, what other pgcrypto routines might have similar bugs?
Well, PGP code accesses anything thru wrappers, so should be OK.
Rest of the code does not try to parse user data, just passes
it thru.
Except armor()/dearmor(), which does lot of pointer-juggling.
I can do a review of that, just in case.
--
marko
decrypt.fix.diff
Description: Binary data
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [PATCHES] pgcrypto
On 7/27/07, Zdenek Kotala [EMAIL PROTECTED] wrote: I attach pgcrypto patch which fix two problems on system without strong crypto support (e.g. default Solaris 10 installation): 1) postgres crashes when AES cipher uses long key 2) Blowfish silently cut longer keys. It could bring problem when crypted data are transfered from one server to another with strong keys support. Couple of style nitpicks: * please use hex arrays, instead octal-quoted strings. easier on the eye. * use memcmp() instead of for() loop. * 16 byte bufs for 8 bytes is confusing. This patch was discussed there: http://archives.postgresql.org/pgsql-hackers/2007-07/msg00762.php This patch is applicable also on 8.2, 8.1 (and maybe older) version of postgresql. OpenSSL autoconfiguration was added in 8.1, so patching older versions is not that critical. -- marko ps. I looked into use of EVP, and I'm not that optimistic anymore. EVP has 3 differenct name for AES different keys, but only one for CAST5, which also supports different key lengths. Plus Blowfish is tagged as VARIABLE_LENGTH. So it seems per-algo support code cannot be avoided, which makes whole EVP usage rather pointless. The situation could be simplified by dropping encrypt()/decrypt() functions, which allow users to specify final keys. But that would be a nasty event, 8.4 is too early for that... ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] SPI-header-files safe for C++-compiler
On 6/29/07, Jacob Rief [EMAIL PROTECTED] wrote:
On Thu, 2007-28-06 at 01:15 -0400, Tom Lane wrote:
Sure, but we don't break them just on a whim. The bottom line here is
whether we are going to make a real commitment to making C++ usable as
a backend extension language --- and for the reasons I mentioned, that
would entail a lot more than renaming a few identifiers. It was already
pointed out upthread that wrapping the inclusions in extern C {...}
would fix the identifier part of the problem from the user side, so I do
No, wrong. Adding extern C does not fix the C++-keywords as identifiers
problem. Adding extern C only tells the compiler to switch off
name-mangling. A C++-compiler does not allow any kind of plain-old C in
such blocks. With some drawbacks, it is even perfectly legal to use some
C++ features inside an extern C block.
I think you are better off wrpapping your C++ code to small
pure-C module and export only that to Postgres-interfacing
C code. Thus you keep away making C++ seeing Postgres code.
It is really a mistake to think C as subset of C++. They are
really different languages. C-as-subset-of-C++ works only
with trivial C code. If you have big, old, real-life codebase
like Postgres, it really does not pay off to force it to comply
with C++ rules. Just to avoid the work of writing intermediate
wrapper code.
--
marko
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [PATCHES] RESET SESSION v3
On 4/12/07, Neil Conway [EMAIL PROTECTED] wrote: On Sun, 2007-04-08 at 11:08 +0300, Marko Kreen wrote: I think implicit ABORT would annoy various tools that partially parse user sql and expect to know what transaction state currently is. For them a new tranaction control statement would be nuisance. That's not the only alternative: we could also either disallow all of the ALL variants in a transaction block, or allow RESET SESSION inside a transaction block. I've committed the patch basically as-is: thanks for the patch. I don't feel strongly about the above, but if there's a consensus, we can change the behavior later. Thanks for reviewing it. One argument for top-level ALL commands is also that poolers and other tools in the middle of connection can track them. But it could also argued that they should have similar rules than ordinary CLOSE/DEALLOCATE statements. Also it seems that disallowing them inside functions for no good reason is couterproductive. But I also dont feel strongly either way. -- marko ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] RESET SESSION v3
Changes in v3: * DEALLOCATE PREPARE ALL * RESET PLANS wont check for ACL_CREATE_TEMP anymore. * Add prepare.h and portal.h to guc.c. On 4/7/07, Neil Conway [EMAIL PROTECTED] wrote: * RESET SESSION does not ABORT anymore, instead fails if in transaction. I think it's quite bizarre to have RESET SESSION fail if used in a transaction, but to allow an equivalent sequence of commands to be executed by hand inside a transaction. I think implicit ABORT would annoy various tools that partially parse user sql and expect to know what transaction state currently is. For them a new tranaction control statement would be nuisance. guc.c is missing some #includes. For some reason gcc4.0 did not show any warnings by default. * DEALLOCATE PREPARE ALL gives bison conflicts. Is that even needed? Seems best to have it, for the sake of consistency. The shift/reduce conflict is easy to workaround, provided you're content to duplicate the body of the DEALLOCATE ALL rule -- e.g. see the attached incremental diff. Thanks, included. * Are the CommandComplete changes needed? Seems warranted to me. BTW, why is CLOSE's command tag CLOSE CURSOR, not CLOSE? That seems needlessly verbose, and inconsistent with other commands (e.g. DEALLOCATE). Because the regular tag is CLOSE CURSOR. I did not want to break any expectations. But yes, the inconsistency is weird. * ResetPlanCache() is implemented as PlanCacheCallback((Datum)0, InvalidOid); That seems to leave plans for utility commands untouched. Is it problem? Yes, I'd think you'd also want to cleanup plans for utility commands. Tom thought otherwise, so I kept the old way. -- marko reset_session_v3.diff Description: Binary data ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[PATCHES] RESET SESSION v2
New commands: CLOSE ALL -- close all cursors DEALLOCATE ALL -- close all prepared stmts RESET PLANS-- drop all plans RESET TEMP | TEMPORARY -- drop all temp tables RESET SESSION -- drop/close/free everything So in the end RESET SESSION is eqivalent to following commands: SET SESSION AUTHORIZATION DEFAULT; RESET ALL; DEALLOCATE ALL; CLOSE ALL; UNLISTEN *; RESET PLANS; RESET TEMP; Changes in v2: * RESET TEMPS - RESET TEMP | TEMPORARY * RESET SESSION does not ABORT anymore, instead fails if in transaction. * DEALLOCATE ALL, CLOSE ALL and RESET SESSION change CommandComplete string to DEALLOCATE ALL, CLOSE CURSOR ALL and RESET SESSION respectively. * Regression tests. * Some docs. * The ParamStatuses for changed options are already sent by ResetAllOptions(), so this already works. Questions: * DEALLOCATE PREPARE ALL gives bison conflicts. Is that even needed? * Are the CommandComplete changes needed? As there is possible to hide DEALLOCATE ALL inside function? OTOH, I like the idea of more descriptive CommandComplete string. I'd like it to include even actual item name for ordinary DECLARE/CLOSE, PREPARE/DEALLOCATE and SET/RESET in the future. * ResetPlanCache() is implemented as PlanCacheCallback((Datum)0, InvalidOid); That seems to leave plans for utility commands untouched. Is it problem? Should it walk plan list itself? -- marko reset_session_v2.diff Description: Binary data ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] DEALLOCATE ALL
On 3/30/07, Alvaro Herrera [EMAIL PROTECTED] wrote: Neil Conway escribió: As to the implementation, calling hash_remove() in a loop seems a pretty unfortunate way to clear a hash table -- adding a hash_reset() function to the dynahash API would be cleaner and faster. I wonder why hash_drop cannot be used? hash_destroy()? Each element need separate destruction. -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[PATCHES] RESET SESSION
This is draft version, for discussion. New commands: CLOSE ALL DEALLOCATE ALL RESET PLANS RESET TEMPS-- please suggest better name RESET SESSION So in the end RESET SESSION basically executes following commands: ABORT; DEALLOCATE ALL; CLOSE ALL; RESET ALL; SET SESSION AUTHORIZATION DEFAULT; UNLISTEN *; RESET TEMPS; RESET PLANS; I think CLOSE ALL and DEALLOCATE ALL are useful in their own, without considering poolers. They just make user lives easier. The ALL just fits there. RESET PLANS and RESET TEMPS are for the principle that components for RESET SESSION should be available for users for special-case situations and fine-tuning poolers. Also they add very little complexity. (RESET PLANS could be useful its awn also). Todo: * Docs * Disallow some commands inside TX? * DEALLOCATE PREPARE ALL gives bison conflicts. is it needed? * Send more details to client. Variants: - CommandComplete: RESET XXX, CLOSE ALL, DEALLOCATE ALL (CLOSE name and DEALLOCATE name would be good in normal case too, that help anyone who wants to track what exists) - RESET SESSION should send ParamStatus of all parms -- marko session.reset.diff Description: Binary data ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] DEALLOCATE ALL
On 3/30/07, Alvaro Herrera [EMAIL PROTECTED] wrote: If by destruction you mean something different than pfree, then maybe hash_remove in a loop is the best solution, the other idea being passing a function pointer to hash_destroy_deep to call on each element, which is probably too messy an API. Yes, callback function is needed, either in HASHCTL or as argument to deep_free(). In any case it's not likely that there are going to be thousands of prepared statements, so is this really an issue? I think the issue is here that its very common thing to do, so open-coding it everywhere is waste, there should be some utility function for that. void hash_foreach(HTAB, void (*cb_func)(void *)); -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[PATCHES] Unbroke srcdir!=builddir compilation
Let's be decent. -- marko pg13.diff Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] DEALLOCATE ALL
When pooling connections where prepared statements are in use, it is hard to give new client totally clean connection as there may be allocated statements that give errors when new client starts preparing statements again. Currently PgPool solves the situation by parsing all queries and keeping list of prepares statements. This may not be a big problem for it as it parses the queries anyway, but for simple pooler like PgBouncer (see pgfoundry) that does not look inside libpq packets it is huge problem. Attached is a patch that allows keyword ALL to be used with DEALLOCATE and then frees all prepared queryes. I think it is useful for most pooling situations, not only PgBouncer. Also its similar in the spirit to RESET ALL. I did it slightly hacky way - if DeallocateStmt-name is NULL is signifies DEALLOCATE ALL command. All the code that looks into DeallocateStmt seems to survive the situation so it should be problem. If still a new node is needed or additional field in the node I can rework the patch. -- marko dealloc_all.diff Description: Binary data ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] DEALLOCATE ALL
On 3/27/07, Tom Lane [EMAIL PROTECTED] wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Marko Kreen wrote: When pooling connections where prepared statements are in use, it is hard to give new client totally clean connection as there may be allocated statements that give errors when new client starts preparing statements again. Huh, didn't we have a RESET SESSION command to do just that? What about cursors, for example? We don't actually *have* one, but I believe it was agreed that that is the right API to provide. If a pooler has to remember to clear prepared statements, GUCs, cursors, and who knows what else, it'll be perpetually broken because there'll be something it omits. Well. Please apply following patch then: http://archives.postgresql.org/pgsql-patches/2004-12/msg00228.php Even if it is incomplete, the missing parts can be added later. I see no reason to keep it from users. There might be a use-case for DEALLOCATE ALL, but needs of poolers aren't it. I'd be inclined to vote against this unless someone can point to a better use-case. Ok, a non-pooler argument: prepared statements are supposed to be garbage-collected by the user. Thats it. There should be friendly way to get a clean state without the need for user to specifically keep track of whats allocated, or to do messy exception-handling around PREPARE/DEALLOCATE. (PREPARE OR REPLACE and DEALLOCATE IF EXISTS would also lessen the pain.) Then a pooler argument: there is one pooler where RandomJoe executes queries and another for specific app where the subset of SQL it uses is known. I want to RESET only specific things in app case. So it would be good if the RESET-s for specific areas would be available. Also the objections to the Hans' patch give impression that different pooling solutions want different RESET EVERYTHING, so again, it would be good if RESET-s for different areas are available and the all-encomassing RESET EVERYTHING just ties all the specific RESETs together. -- marko ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [pgsql-patches] [PATCHES] vcbuild optional packages
On 1/8/07, Magnus Hagander [EMAIL PROTECTED] wrote: This patch fixes vcbuild so you can build without OpenSSL and libz should you want to. This disables the sslinfo and pgcrypto modules because they require the libraries to build at all. Both openssl and zlib are optional for pgcrypto. Please look at the real Makefile. (Put -DDISABLE_ZLIB into CFLAGS.) -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-patches] [PATCHES] vcbuild optional packages
On 1/14/07, Magnus Hagander [EMAIL PROTECTED] wrote: Is there any reason why the check is backwards in pgcrypto compared to the rest of the system? All the other places check for HAVE_ZLIB from what I can tell. Eh, just an oversight from my part. I was so focused on Makefile hacking so I forgot to check if there is already a define somewhere. And its reverse to have less clutter in command-line in default case. Attached patch replaces unnecessary DISABLE_ZLIB define in pgcrypto with HAVE_LIBZ from core. Result is bit simpler Makefile which is only good. -- marko pgcrypto-simpler-zlib.diff Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] Gen_fmgrtab.sh fails with LANG=et_EE
I recently changed locale in my machine and PostgreSQL build broke. The reason is that Gen_fmgrtab.sh uses [^A-Z] expression to sed and the GNU people in their infinite wisdom made that locale-dependant. As the script uses the complicated pipeline only for fmgroids.h - FMGROIDS_H conversion, it seems simpler to just make it explicitly set, instead to try to work around GNU sed. I grepped around source and did not find other instances of this. The A-Z experssion was only in perl scripts or in configure and configure should be fine as it explicitly resets locale. -- marko Index: src/backend/utils/Gen_fmgrtab.sh === RCS file: /opt/cvs/pgsql/src/backend/utils/Gen_fmgrtab.sh,v retrieving revision 1.32 diff -u -c -r1.32 Gen_fmgrtab.sh *** src/backend/utils/Gen_fmgrtab.sh 5 Mar 2006 15:58:40 - 1.32 --- src/backend/utils/Gen_fmgrtab.sh 5 Sep 2006 08:45:55 - *** *** 62,69 fi SORTEDFILE=$$-fmgr.data - OIDSFILE=fmgroids.h TABLEFILE=fmgrtab.c trap 'echo Caught signal. ; cleanup ; exit 1' 1 2 15 --- 62,70 fi SORTEDFILE=$$-fmgr.data TABLEFILE=fmgrtab.c + OIDSFILE=fmgroids.h + OIDSFILE_DEFINE=FMGROIDS_H trap 'echo Caught signal. ; cleanup ; exit 1' 1 2 15 *** *** 89,95 fi - cpp_define=`echo $OIDSFILE | tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ | sed -e 's/[^A-Z]/_/g'` # # Generate fmgroids.h --- 90,95 *** *** 116,123 * *- */ ! #ifndef $cpp_define ! #define $cpp_define /* * Constant macros for the OIDs of entries in pg_proc. --- 116,123 * *- */ ! #ifndef $OIDSFILE_DEFINE ! #define $OIDSFILE_DEFINE /* * Constant macros for the OIDs of entries in pg_proc. *** *** 147,153 cat $$-$OIDSFILE FuNkYfMgRsTuFf ! #endif /* $cpp_define */ FuNkYfMgRsTuFf # --- 147,153 cat $$-$OIDSFILE FuNkYfMgRsTuFf ! #endif /* $OIDSFILE_DEFINE */ FuNkYfMgRsTuFf # ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [PATCHES] Gen_fmgrtab.sh fails with LANG=et_EE
On 9/5/06, Tom Lane [EMAIL PROTECTED] wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Well, the line of code is cpp_define=`echo $OIDSFILE | tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ | sed -e 's/[^A-Z]/_/g'` so it ought to be pretty obvious what the correct solution for the problem character ranges are locale-dependent is. Doh. Patched that way. Curiously, I couldn't replicate the failure on Fedora 5 --- Marko's platform must have different locale behavior for et_EE. Did you add it to locale-gen config and ran it? Btw, I removed all the pipeline in my patch, because I felt such messy pipeline for such a tiny thing is ugly. Especially as the filename wont change that much. Thus I though it would be cleaner to just put the symbol together with filename definition. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] [PATCH] Provide 8-byte transaction IDs to
On 8/21/06, Tom Lane [EMAIL PROTECTED] wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Tom Lane wrote:
(I wouldn't do it like this though --- TransactionIdAdvance itself is
the place to bump the secondary counter.)
Agreed.
I reconsidered after trying to do it that way --- although fixing
TransactionIdAdvance itself to maintain a 2-word counter isn't hard,
there are a whole lot of other places that can advance nextXid,
mostly bits like this in WAL recovery:
/* Make sure nextXid is beyond any XID mentioned in the record */
max_xid = xid;
for (i = 0; i xlrec-nsubxacts; i++)
{
if (TransactionIdPrecedes(max_xid, sub_xids[i]))
max_xid = sub_xids[i];
}
if (TransactionIdFollowsOrEquals(max_xid,
ShmemVariableCache-nextXid))
{
ShmemVariableCache-nextXid = max_xid;
TransactionIdAdvance(ShmemVariableCache-nextXid);
}
We could hack all these places to know about maintaining an XID-epoch
value, but it's not looking like a simple single-place-to-touch fix :-(
As I was asked to rework the patch, I planned to use
TransactionIdAdvance(ShmemVariableCache), although that would
be conceptually ugly. Right Thing for this approach would be
to have special struct, but that would touch half the codebase.
That was also the reason I did not want to go that path.
There's still a lot more cruft in the submitted patch than I think
belongs in core, but I'll work on extracting something we can apply.
The only cruft I see is the snapshot on-disk compression and maybe
the pg_sync_txid() funtionality. Dropping the compression would not
matter much, snapshots would waste space, but at least for our
usage it would not be a problem. The reast of the functions are all
required for efficient handling.
Dropping the pg_sync_txid() would be loss, because that means that
user cannot just dump and restore the data and just continue where
it left off. Maybe its not a problem for replication but for generic
queueing it would need delicate juggling when restoring backup.
Although I must admit the pg_sync_txid() is indeed ugly part
of the patch, and it creates new mode for failure - wrapping
epoch. So I can kind of agree for removing it.
I hope you don't mean that none of the user-level functions belong
to core. It's not like there is several ways to expose the info.
And it not like there are much more interesting ways for using
the long xid in C level. Having long xid available in SQL level
means that efficient async replication can be done without any
use of C.
Now that I am back from vacation I can do some coding myself,
if you give hints what needs rework.
--
marko
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
[PATCHES] pgcrypto merge cleanup
- Few README fixes - Keep imath Id string, put $PostgreSQL$ separately. -- marko Index: contrib/pgcrypto/README.pgcrypto === RCS file: /opt/cvs/pgsql/contrib/pgcrypto/README.pgcrypto,v retrieving revision 1.15 diff -u -c -r1.15 README.pgcrypto *** contrib/pgcrypto/README.pgcrypto 13 Jul 2006 04:15:24 - 1.15 --- contrib/pgcrypto/README.pgcrypto 18 Jul 2006 13:02:27 - *** *** 36,48 `configure` script. The options that affect it are `--with-zlib` and `--with-openssl`. ! Without zlib, the PGP functions will not support compressed data inside ! PGP encrypted packets. ! Without OpenSSL, public-key encryption does not work, as pgcrypto does ! not yet contain math functions for large integers. ! There are some other differences with and without OpenSSL: ``-` Functionalitybuilt-in OpenSSL --- 36,49 `configure` script. The options that affect it are `--with-zlib` and `--with-openssl`. ! When compiled with zlib, PGP encryption functions are able to ! compress data before encrypting. ! When compiled with OpenSSL there will be more algorithms available. ! Also public-key encryption functions will be faster as OpenSSL ! has more optimized BIGNUM functions. ! Summary of functionality with and without OpenSSL: ``-` Functionalitybuilt-in OpenSSL *** *** 421,427 Options are named to be similar to GnuPG. Values should be given after an equal sign; separate options from each other with commas. Example: ! pgp_sym_encrypt(data, psw, 'compress-also=1, cipher-algo=aes256') All of the options except `convert-crlf` apply only to encrypt functions. Decrypt functions get the parameters from PGP data. --- 422,428 Options are named to be similar to GnuPG. Values should be given after an equal sign; separate options from each other with commas. Example: ! pgp_sym_encrypt(data, psw, 'compress-algo=1, cipher-algo=aes256') All of the options except `convert-crlf` apply only to encrypt functions. Decrypt functions get the parameters from PGP data. *** *** 571,577 --- Those functions only run a cipher over data, they don't have any advanced ! features of PGP encryption. In addition, they have some major problems: 1. They use user key directly as cipher key. 2. They don't provide any integrity checking, to see --- 572,578 --- Those functions only run a cipher over data, they don't have any advanced ! features of PGP encryption. Therefore they have some major problems: 1. They use user key directly as cipher key. 2. They don't provide any integrity checking, to see Index: contrib/pgcrypto/imath.c === RCS file: /opt/cvs/pgsql/contrib/pgcrypto/imath.c,v retrieving revision 1.3 diff -u -c -r1.3 imath.c *** contrib/pgcrypto/imath.c 16 Jul 2006 02:44:00 - 1.3 --- contrib/pgcrypto/imath.c 16 Jul 2006 10:58:02 - *** *** 3,9 Name: imath.c Purpose: Arbitrary precision integer arithmetic routines. Author: M. J. Fromberger http://www.dartmouth.edu/~sting/ ! Info: $PostgreSQL$ Copyright (C) 2002 Michael J. Fromberger, All Rights Reserved. --- 3,9 Name: imath.c Purpose: Arbitrary precision integer arithmetic routines. Author: M. J. Fromberger http://www.dartmouth.edu/~sting/ ! Info: Id: imath.c 21 2006-04-02 18:58:36Z sting Copyright (C) 2002 Michael J. Fromberger, All Rights Reserved. *** *** 27,32 --- 27,33 CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ + /* $PostgreSQL$ */ #include postgres.h #include px.h Index: contrib/pgcrypto/imath.h === RCS file: /opt/cvs/pgsql/contrib/pgcrypto/imath.h,v retrieving revision 1.3 diff -u -c -r1.3 imath.h *** contrib/pgcrypto/imath.h 16 Jul 2006 02:44:01 - 1.3 --- contrib/pgcrypto/imath.h 16 Jul 2006 11:00:17 - *** *** 2,8 Name: imath.h Purpose: Arbitrary precision integer arithmetic routines. Author: M. J. Fromberger http://www.dartmouth.edu/~sting/ ! Info: $PostgreSQL$ Copyright (C) 2002 Michael J. Fromberger, All Rights Reserved. --- 2,8 Name: imath.h Purpose: Arbitrary precision integer arithmetic routines. Author: M. J. Fromberger http://www.dartmouth.edu/~sting/ ! Info: Id: imath.h 21 2006-04-02 18:58:36Z sting Copyright (C) 2002 Michael J. Fromberger, All Rights Reserved. *** *** 26,31 --- 26,32 CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ + /*
Re: [PATCHES] [patch 0/9] annual pgcrypto update
On 7/13/06, Neil Conway [EMAIL PROTECTED] wrote: Marko, can you take a look at what is causing this regression test failure? The failing machine is kudu: Seems you have skipped the CAST5 patch. Could you recheck? -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] [patch 7/9] remove file misc.c
misc.c annoys me and creates unnecesary inter-dependencies just
for sharing a couple of bytes. This patch splits it out
to crypt-md5.c and crypt-des.c
Please do 'cvs remove misc.c' after applying this patch.
Index: pgsql/contrib/pgcrypto/Makefile
===
*** pgsql.orig/contrib/pgcrypto/Makefile
--- pgsql/contrib/pgcrypto/Makefile
*** CF_PGP_TESTS = $(if $(subst no,,$(with_z
*** 23,29
PG_CPPFLAGS = $(CF_CFLAGS)
! SRCS = pgcrypto.c px.c px-hmac.c px-crypt.c misc.c \
crypt-gensalt.c crypt-blowfish.c crypt-des.c \
crypt-md5.c $(CF_SRCS) \
mbuf.c pgp.c pgp-armor.c pgp-cfb.c pgp-compress.c \
--- 23,29
PG_CPPFLAGS = $(CF_CFLAGS)
! SRCS = pgcrypto.c px.c px-hmac.c px-crypt.c \
crypt-gensalt.c crypt-blowfish.c crypt-des.c \
crypt-md5.c $(CF_SRCS) \
mbuf.c pgp.c pgp-armor.c pgp-cfb.c pgp-compress.c \
Index: pgsql/contrib/pgcrypto/crypt-des.c
===
*** pgsql.orig/contrib/pgcrypto/crypt-des.c
--- pgsql/contrib/pgcrypto/crypt-des.c
***
*** 71,76
--- 71,79
#define _PASSWORD_EFMT1 '_'
+ static const char _crypt_a64[] =
+ ./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz;
+
static uint8 IP[64] = {
58, 50, 42, 34, 26, 18, 10, 2, 60, 52, 44, 36, 28, 20, 12, 4,
62, 54, 46, 38, 30, 22, 14, 6, 64, 56, 48, 40, 32, 24, 16, 8,
Index: pgsql/contrib/pgcrypto/crypt-md5.c
===
*** pgsql.orig/contrib/pgcrypto/crypt-md5.c
--- pgsql/contrib/pgcrypto/crypt-md5.c
***
*** 17,22
--- 17,36
#include px-crypt.h
#define MD5_SIZE 16
+
+ static const char _crypt_a64[] =
+ ./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz;
+
+ static void
+ _crypt_to64(char *s, unsigned long v, int n)
+ {
+ while (--n = 0)
+ {
+ *s++ = _crypt_a64[v 0x3f];
+ v = 6;
+ }
+ }
+
/*
* UNIX password
*/
Index: pgsql/contrib/pgcrypto/px-crypt.h
===
*** pgsql.orig/contrib/pgcrypto/px-crypt.h
--- pgsql/contrib/pgcrypto/px-crypt.h
*** int px_gen_salt(const char *salt_type,
*** 55,68
* internal functions
*/
- /* misc.c */
- extern void px_crypt_to64(char *s, unsigned long v, int n);
- extern char px_crypt_a64[];
-
- /* avoid conflicts with system libs */
- #define _crypt_to64 px_crypt_to64
- #define _crypt_a64 px_crypt_a64
-
/* crypt-gensalt.c */
char *_crypt_gensalt_traditional_rn(unsigned long count,
const char *input, int size, char *output, int
output_size);
--- 55,60
Index: pgsql/contrib/pgcrypto/crypt-gensalt.c
===
*** pgsql.orig/contrib/pgcrypto/crypt-gensalt.c
--- pgsql/contrib/pgcrypto/crypt-gensalt.c
***
*** 19,25
typedef unsigned int BF_word;
! unsigned char _crypt_itoa64[64 + 1] =
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz;
char *
--- 19,25
typedef unsigned int BF_word;
! static unsigned char _crypt_itoa64[64 + 1] =
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz;
char *
Index: pgsql/contrib/pgcrypto/misc.c
===
*** pgsql.orig/contrib/pgcrypto/misc.c
--- /dev/null
***
*** 1,51
- /*
- * Copyright (c) 1999
- *University of California. All rights reserved.
- *
- * $PostgreSQL: pgsql/contrib/pgcrypto/misc.c,v 1.3 2006/03/11 04:38:30
momjian Exp $
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- * 3. Neither the name of the author nor the names of any co-contributors
- * may be used to endorse or promote products derived from this software
- * without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY CONTRIBUTORS ``AS IS'' AND ANY EXPRESS
- * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL CONTRIBUTORS BE LIABLE FOR ANY
- * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT
[PATCHES] [patch 2/9] Fortuna tweaks
* Use Fortuna for pseudo-random bytes too, as those need to
be high-quality also. OpenSSL internally acts same way.
* Clarify add_entropy logic - before first reseed get_rand_pool()
will return only 0, so make it explicit.
* On reseeding on first request, don't check if the pool#0 is
filled, use whatever is available.
* The counter value is already randomized, to avoid known state.
Do the same with pools. Thus no publicly known values in state.
Index: pgsql/contrib/pgcrypto/internal.c
===
*** pgsql.orig/contrib/pgcrypto/internal.c
--- pgsql/contrib/pgcrypto/internal.c
*** px_find_cipher(const char *name, PX_Ciph
*** 821,839
*/
/*
! * Use libc for all 'public' bytes.
! *
! * That way we don't expose bytes from Fortuna
! * to the public, in case it has some bugs.
*/
int
px_get_pseudo_random_bytes(uint8 *dst, unsigned count)
{
! int i;
!
! for (i = 0; i count; i++)
! *dst++ = random();
! return i;
}
static time_t seed_time = 0;
--- 821,832
*/
/*
! * Use always strong randomness.
*/
int
px_get_pseudo_random_bytes(uint8 *dst, unsigned count)
{
! return px_get_random_bytes(dst, count);
}
static time_t seed_time = 0;
Index: pgsql/contrib/pgcrypto/fortuna.c
===
*** pgsql.orig/contrib/pgcrypto/fortuna.c
--- pgsql/contrib/pgcrypto/fortuna.c
*** struct fortuna_state
*** 125,131
struct timeval last_reseed_time;
unsignedpool0_bytes;
unsignedrnd_pos;
! int counter_init;
};
typedef struct fortuna_state FState;
--- 125,131
struct timeval last_reseed_time;
unsignedpool0_bytes;
unsignedrnd_pos;
! int tricks_done;
};
typedef struct fortuna_state FState;
*** add_entropy(FState * st, const uint8 *da
*** 332,338
/*
* Make sure the pool 0 is initialized, then update randomly.
*/
! if (st-reseed_count == 0 st-pool0_bytes POOL0_FILL)
pos = 0;
else
pos = get_rand_pool(st);
--- 332,338
/*
* Make sure the pool 0 is initialized, then update randomly.
*/
! if (st-reseed_count == 0)
pos = 0;
else
pos = get_rand_pool(st);
*** rekey(FState * st)
*** 357,377
}
/*
! * Fortuna relies on AES standing known-plaintext attack.
! * In case it does not, slow down the attacker by initialising
! * the couter to random value.
*/
static void
! init_counter(FState * st)
{
/* Use next block as counter. */
encrypt_counter(st, st-counter);
/* Hide the key. */
rekey(st);
! /* The counter can be shuffled only once. */
! st-counter_init = 1;
}
static void
--- 357,390
}
/*
! * Hide public constants. (counter, pools 0)
! *
! * This can also be viewed as spreading the startup
! * entropy over all of the components.
*/
static void
! startup_tricks(FState * st)
{
+ int i;
+ uint8 buf[BLOCK];
+
/* Use next block as counter. */
encrypt_counter(st, st-counter);
+ /* Now shuffle pools, excluding #0 */
+ for (i = 1; i NUM_POOLS; i++)
+ {
+ encrypt_counter(st, buf);
+ encrypt_counter(st, buf + CIPH_BLOCK);
+ md_update(st-pool[i], buf, BLOCK);
+ }
+ memset(buf, 0, BLOCK);
+
/* Hide the key. */
rekey(st);
! /* This can be done only once. */
! st-tricks_done = 1;
}
static void
*** extract_data(FState * st, unsigned count
*** 380,392
unsignedn;
unsignedblock_nr = 0;
! /* Can we reseed? */
! if (st-pool0_bytes = POOL0_FILL enough_time_passed(st))
! reseed(st);
!
! /* Is counter initialized? */
! if (!st-counter_init)
! init_counter(st);
while (count 0)
{
--- 393,406
unsignedn;
unsignedblock_nr = 0;
! /* Should we reseed? */
! if (st-pool0_bytes = POOL0_FILL || st-reseed_count == 0)
! if (enough_time_passed(st))
! reseed(st);
!
! /* Do some randomization on first call */
! if (!st-tricks_done)
! startup_tricks(st);
while (count 0)
{
--
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
[PATCHES] [patch 8/9] Provide SHA2 for older OpenSSL
I'd like to promote SHA2 algortihms (SHA224/256/384/512)
to always available status. Problem is that OpenSSL
only provides them from version 0.9.8. Considering
that 0.9.7 has not yet replaced 0.9.6, the 0.9.8 version
won't be generally available for long time.
Following patch splits support for SHA2 out from internal.c
and then uses same trick as for AES to provide it for older
OpenSSL versions.
Index: pgsql/contrib/pgcrypto/internal-sha2.c
===
*** /dev/null
--- pgsql/contrib/pgcrypto/internal-sha2.c
***
*** 0
--- 1,317
+ /*
+ * internal.c
+ *Wrapper for builtin functions
+ *
+ * Copyright (c) 2001 Marko Kreen
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $PostgreSQL: pgsql/contrib/pgcrypto/internal.c,v 1.22 2005/07/18 17:12:54
tgl Exp $
+ */
+
+ #include postgres.h
+
+ #include time.h
+
+ #include px.h
+ #include sha2.h
+
+ void init_sha224(PX_MD * h);
+ void init_sha256(PX_MD * h);
+ void init_sha384(PX_MD * h);
+ void init_sha512(PX_MD * h);
+
+ /* SHA224 */
+
+ static unsigned
+ int_sha224_len(PX_MD * h)
+ {
+ return SHA224_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha224_block_len(PX_MD * h)
+ {
+ return SHA224_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha224_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA224_CTX *ctx = (SHA224_CTX *) h-p.ptr;
+
+ SHA224_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha224_reset(PX_MD * h)
+ {
+ SHA224_CTX *ctx = (SHA224_CTX *) h-p.ptr;
+
+ SHA224_Init(ctx);
+ }
+
+ static void
+ int_sha224_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA224_CTX *ctx = (SHA224_CTX *) h-p.ptr;
+
+ SHA224_Final(dst, ctx);
+ }
+
+ static void
+ int_sha224_free(PX_MD * h)
+ {
+ SHA224_CTX *ctx = (SHA224_CTX *) h-p.ptr;
+
+ memset(ctx, 0, sizeof(*ctx));
+ px_free(ctx);
+ px_free(h);
+ }
+
+ /* SHA256 */
+
+ static unsigned
+ int_sha256_len(PX_MD * h)
+ {
+ return SHA256_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha256_block_len(PX_MD * h)
+ {
+ return SHA256_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha256_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha256_reset(PX_MD * h)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Init(ctx);
+ }
+
+ static void
+ int_sha256_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Final(dst, ctx);
+ }
+
+ static void
+ int_sha256_free(PX_MD * h)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ memset(ctx, 0, sizeof(*ctx));
+ px_free(ctx);
+ px_free(h);
+ }
+
+ /* SHA384 */
+
+ static unsigned
+ int_sha384_len(PX_MD * h)
+ {
+ return SHA384_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha384_block_len(PX_MD * h)
+ {
+ return SHA384_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha384_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha384_reset(PX_MD * h)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Init(ctx);
+ }
+
+ static void
+ int_sha384_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Final(dst, ctx);
+ }
+
+ static void
+ int_sha384_free(PX_MD * h)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ memset(ctx, 0, sizeof(*ctx));
+ px_free(ctx);
+ px_free(h
[PATCHES] [patch 0/9] annual pgcrypto update
Few cleanups and couple of new things: - add SHA2 algorithm to older OpenSSL - add BIGNUM math to have public-key cryptography work on non-OpenSSL build. - gen_random_bytes() function The status of SHA2 algoritms and public-key encryption can now be changed to 'always available.' That makes pgcrypto functionally complete and unless there will be new editions of AES, SHA2 or OpenPGP standards, there is no major changes planned. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[PATCHES] [patch 1/9] Silence compiler warnings in openssl.c
Function DES_ecb3_encrypt has unstable signature in OpenSSL. Different versions of OpenSSL have different argument types and it is not possible to pick right types by OpenSSL version. Following patch silents compiler by forcing argument to (void *). Index: pgsql/contrib/pgcrypto/openssl.c === *** pgsql.orig/contrib/pgcrypto/openssl.c --- pgsql/contrib/pgcrypto/openssl.c *** ossl_des3_ecb_encrypt(PX_Cipher * c, con *** 526,532 ossldata *od = c-ptr; for (i = 0; i dlen / bs; i++) ! DES_ecb3_encrypt(data + i * bs, res + i * bs, od-u.des3.k1, od-u.des3.k2, od-u.des3.k3, 1); return 0; } --- 526,532 ossldata *od = c-ptr; for (i = 0; i dlen / bs; i++) ! DES_ecb3_encrypt((void *)(data + i * bs), (void *)(res + i * bs), od-u.des3.k1, od-u.des3.k2, od-u.des3.k3, 1); return 0; } *** ossl_des3_ecb_decrypt(PX_Cipher * c, con *** 540,546 ossldata *od = c-ptr; for (i = 0; i dlen / bs; i++) ! DES_ecb3_encrypt(data + i * bs, res + i * bs, od-u.des3.k1, od-u.des3.k2, od-u.des3.k3, 0); return 0; } --- 540,546 ossldata *od = c-ptr; for (i = 0; i dlen / bs; i++) ! DES_ecb3_encrypt((void *)(data + i * bs), (void *)(res + i * bs), od-u.des3.k1, od-u.des3.k2, od-u.des3.k3, 0); return 0; } -- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[PATCHES] [patch 6/9] Provide function to generate random keys
Following patch provides function gen_random_bytes() to SQL-level,
that uses internal strong random number generator to generate
a bytea value.
I've looked at server-managed-encryption-keys that seems new trend
in commercial databases. I don't think such functionality has a
place in pgcrypto, but I think it would be good to have low-level
tools to build such scheme on top of pgcrypto. That was the only
missing piece that
Index: pgsql/contrib/pgcrypto/pgcrypto.c
===
*** pgsql.orig/contrib/pgcrypto/pgcrypto.c
--- pgsql/contrib/pgcrypto/pgcrypto.c
*** pg_decrypt_iv(PG_FUNCTION_ARGS)
*** 537,542
--- 537,570
PG_RETURN_BYTEA_P(res);
}
+ /* SQL function: pg_random_bytes(int4) returns bytea */
+ PG_FUNCTION_INFO_V1(pg_random_bytes);
+
+ Datum
+ pg_random_bytes(PG_FUNCTION_ARGS)
+ {
+ int err;
+ int len = PG_GETARG_INT32(0);
+ bytea *res;
+
+ if (len 1 || len 1024)
+ ereport(ERROR,
+
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
+errmsg(Length not in range)));
+
+ res = palloc(VARHDRSZ + len);
+ VARATT_SIZEP(res) = VARHDRSZ + len;
+
+ /* generate result */
+ err = px_get_random_bytes((uint8*)VARDATA(res), len);
+ if (err 0)
+ ereport(ERROR,
+
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
+errmsg(Random generator error: %s,
px_strerror(err;
+
+ PG_RETURN_BYTEA_P(res);
+ }
+
/* SQL function: pg_cipher_exists(text) returns bool */
PG_FUNCTION_INFO_V1(pg_cipher_exists);
Index: pgsql/contrib/pgcrypto/pgcrypto.h
===
*** pgsql.orig/contrib/pgcrypto/pgcrypto.h
--- pgsql/contrib/pgcrypto/pgcrypto.h
*** Datum pg_decrypt(PG_FUNCTION_ARGS);
*** 47,51
--- 47,52
Datum pg_encrypt_iv(PG_FUNCTION_ARGS);
Datum pg_decrypt_iv(PG_FUNCTION_ARGS);
Datum pg_cipher_exists(PG_FUNCTION_ARGS);
+ Datum pg_random_bytes(PG_FUNCTION_ARGS);
#endif
Index: pgsql/contrib/pgcrypto/pgcrypto.sql.in
===
*** pgsql.orig/contrib/pgcrypto/pgcrypto.sql.in
--- pgsql/contrib/pgcrypto/pgcrypto.sql.in
*** RETURNS bool
*** 71,76
--- 71,81
AS 'MODULE_PATHNAME', 'pg_cipher_exists'
LANGUAGE C IMMUTABLE STRICT;
+ CREATE OR REPLACE FUNCTION gen_random_bytes(int4)
+ RETURNS bytea
+ AS 'MODULE_PATHNAME', 'pg_random_bytes'
+ LANGUAGE 'C' VOLATILE STRICT;
+
--
-- pgp_sym_encrypt(data, key)
--
Index: pgsql/contrib/pgcrypto/uninstall_pgcrypto.sql
===
*** pgsql.orig/contrib/pgcrypto/uninstall_pgcrypto.sql
--- pgsql/contrib/pgcrypto/uninstall_pgcrypto.sql
*** DROP FUNCTION decrypt(bytea, bytea, text
*** 18,23
--- 18,24
DROP FUNCTION encrypt_iv(bytea, bytea, bytea, text);
DROP FUNCTION decrypt_iv(bytea, bytea, bytea, text);
DROP FUNCTION cipher_exists(text);
+ DROP FUNCTION gen_random_bytes(int4);
DROP FUNCTION pgp_sym_encrypt(text, text);
DROP FUNCTION pgp_sym_encrypt_bytea(bytea, text);
Index: pgsql/contrib/pgcrypto/README.pgcrypto
===
*** pgsql.orig/contrib/pgcrypto/README.pgcrypto
--- pgsql/contrib/pgcrypto/README.pgcrypto
*** is equal to
*** 621,627
encrypt(data, 'fooz', 'bf-cbc/pad:pkcs')
! 7. Credits
I have used code from following sources:
--- 621,637
encrypt(data, 'fooz', 'bf-cbc/pad:pkcs')
! 7. Random bytes
! -
!
! gen_random_bytes(count integer)
!
! Returns `count` cryptographically strong random bytes as bytea value.
! There can be maximally 1024 bytes extracted at a time. This is to avoid
! draining the randomness generator pool.
!
!
! 8. Credits
I have used code from following sources:
*** I have used code from following sources:
*** 639,656
--
! 8. Legalese
-
* I owe a beer to Poul-Henning.
* This product includes software developed by Niels Provos.
! 9. References/Links
! -
! 9.1. Useful reading
! ~
http://www.gnupg.org/gph/en/manual.html[]::
The GNU Privacy Handbook
--- 649,666
--
! 9. Legalese
-
* I owe a beer to Poul-Henning.
* This product includes software developed by Niels Provos.
! 10. References/Links
! --
! 10.1. Useful reading
! ~~
[PATCHES] [patch 4/9] Fix use of CAST5 in regtests.
In PGP public key tests, the password-encrypted secret key happened to be encrypted with CAST5 instead of AES. As OpenSSL has CAST5 always available I did not notice it before. Re-encrypt the key with AES. Index: pgsql/contrib/pgcrypto/expected/pgp-pubkey-decrypt.out === *** pgsql.orig/contrib/pgcrypto/expected/pgp-pubkey-decrypt.out --- pgsql/contrib/pgcrypto/expected/pgp-pubkey-decrypt.out *** ig3hGY1Rb4NEk1gAn1u9IuQB+BgDP40YHHz6bKWS *** 301,307 -BEGIN PGP PRIVATE KEY BLOCK- Version: GnuPG v1.4.1 (GNU/Linux) ! lQHhBELIIUgRBACp401L6jXrLB28c3YA4sM3OJKnxM1GT9YTkWyE3Vyte65H8WU9 tGPBX7OMuaX5eGZ84LFUGvaP0k7anfmXcDkCO3P9GgL+ro/dS2Ps/vChQPZqHaxE xpKDUt47B7DGdRJrC8DRnIR4wbSyQA6ma3S1yFqC5pJhSs+mqf9eExOjiwCgntth klRxIYw352ZX9Ov9oht/p/ED/1Xi4PS+tkXVvyIw5aZfa61bT6XvDkoPI0Aj3GE5 --- 301,307 -BEGIN PGP PRIVATE KEY BLOCK- Version: GnuPG v1.4.1 (GNU/Linux) ! lQHpBELIIUgRBACp401L6jXrLB28c3YA4sM3OJKnxM1GT9YTkWyE3Vyte65H8WU9 tGPBX7OMuaX5eGZ84LFUGvaP0k7anfmXcDkCO3P9GgL+ro/dS2Ps/vChQPZqHaxE xpKDUt47B7DGdRJrC8DRnIR4wbSyQA6ma3S1yFqC5pJhSs+mqf9eExOjiwCgntth klRxIYw352ZX9Ov9oht/p/ED/1Xi4PS+tkXVvyIw5aZfa61bT6XvDkoPI0Aj3GE5 *** YmCHJlKA/IhEr8QJOLV++5VEv4l6KQ1/DFoJzoNd *** 309,329 PHQme5oAWoHa6bVQZOwvbJh3mOXDq/Tk/KF22go8maM44vMn4bvv+SBbslviYLiL jZJ1A/9JXF1esNq+X9HehJyqHHU7LEEf/ck6zC7o2erM3/LZlZuLNPD2cv3oL3Nv saEgcTSZl+8XmO8pLmzjKIb+hi70qVx3t2IhMqbb4B/dMY1Ck62gPBKa81/Wwi7v ! IsEBQLEtyBmGmI64YpzoRNFeaaF9JY+sAKqROqe6dLjJ7vebQP4DAwL3TCgrYdj6 ! +GAnoSqGa87twi8a6QRRYIlEx3ddUCDCjzkJmRfF+LFtvX3OtWWK0+Syi3kj2IK9 ! YT7pF7QfRWxnYW1hbCAxMDI0IDx0ZXN0QGV4YW1wbGUub3JnPoheBBMRAgAeBQJC ! yCFIAhsDBgsJCAcDAgMVAgMDFgIBAh4BAheAAAoJEBwpvA0YF3NkOtsAn1ynoCyM ! 6GIvHDOewwmF4Z/jGQfzAJ9Q+MwIubi0ASfJifaEM23sIHwHop0BVwRCyCFKEAQA ! h5SNbbJMAsJ+sQbcWEzdku8AdYB5zY7Qyf9EOvn0g39bzANhxmmb6gbRlQN0ioym ! lDwraTKUAfuCZgNcg/0PsxFGb9nDcvjIV8qdVpnq1PuzMFuBbmGI6weg7Pj01dlP ! iO0wt1lLX+SubktqbYxI+h31c3RDZqxj+KAgxR8YNGMAAwYD+wQs2He1Z5+p4OSg ! MERiNzF0acZUYmc0e+/96gfL0ft3IP+SSFo6hEBrkKVhZKoPSSRr5KpNaEobhdxs ! nKjUaw/qyoaFcNMzb4sFk8wq5UlCkR+h72u6hv8FuleCV8SJUT1U2JjtlXJR2Pey ! 9ifh8rZfu57UbdwdHa0viWc4Dilh/gMDAvdMKCth2Pr4YCCPsELdgJuzhGfDNRSg ! nKMRWBWHSJRk6JmCjM1iJQNHc4mMhR8gvi2TeqYLOhYjcF7nr/LA+JvLV+adj/mI ! SQQYEQIACQUCQsghSgIbDAAKCRAcKbwNGBdzZO2vAJ4hRaLcNcdl/qK8rt0N5zbZ ! saCh6QCfR1O48O8nYN93SPSfIFZK5rEmdv8= ! =Y6Qv -END PGP PRIVATE KEY BLOCK- '); insert into keytbl (id, name, pubkey, seckey) --- 309,329 PHQme5oAWoHa6bVQZOwvbJh3mOXDq/Tk/KF22go8maM44vMn4bvv+SBbslviYLiL jZJ1A/9JXF1esNq+X9HehJyqHHU7LEEf/ck6zC7o2erM3/LZlZuLNPD2cv3oL3Nv saEgcTSZl+8XmO8pLmzjKIb+hi70qVx3t2IhMqbb4B/dMY1Ck62gPBKa81/Wwi7v ! IsEBQLEtyBmGmI64YpzoRNFeaaF9JY+sAKqROqe6dLjJ7vebQP4HAwImKZ5q2QwT ! D2DDAY/IQBjes7WgqZeacfLPDoB8ecD/KLoSCH6Z3etvbPHSOKiazxoJ962Ix74H ! ZAE6ZbMTtl5dZW1ptB9FbGdhbWFsIDEwMjQgPHRlc3RAZXhhbXBsZS5vcmc+iF4E ! ExECAB4FAkLIIUgCGwMGCwkIBwMCAxUCAwMWAgECHgECF4AACgkQHCm8DRgXc2Q6 ! 2wCfXKegLIzoYi8cM57DCYXhn+MZB/MAn1D4zAi5uLQBJ8mJ9oQzbewgfAeinQFf ! BELIIUoQBACHlI1tskwCwn6xBtxYTN2S7wB1gHnNjtDJ/0Q6+fSDf1vMA2HGaZvq ! BtGVA3SKjKaUPCtpMpQB+4JmA1yD/Q+zEUZv2cNy+MhXyp1WmerU+7MwW4FuYYjr ! B6Ds+PTV2U+I7TC3WUtf5K5uS2ptjEj6HfVzdENmrGP4oCDFHxg0YwADBgP7BCzY ! d7Vnn6ng5KAwRGI3MXRpxlRiZzR77/3qB8vR+3cg/5JIWjqEQGuQpWFkqg9JJGvk ! qk1oShuF3GycqNRrD+rKhoVw0zNviwWTzCrlSUKRH6Hva7qG/wW6V4JXxIlRPVTY ! mO2VclHY97L2J+Hytl+7ntRt3B0drS+JZzgOKWH+BwMCJimeatkMEw9gRkFjt4Xa ! 9rX8awMBE5+vVcGKv/DNiCvJnlYvSdCj8VfuHsYFliiJo6u17NJon+K43e3yvDNk ! f631VOVanGEz7TyqOkWQiEkEGBECAAkFAkLIIUoCGwwACgkQHCm8DRgXc2TtrwCe ! IUWi3DXHZf6ivK7dDec22bGgoekAn0dTuPDvJ2Dfd0j0nyBWSuaxJnb/ ! =SNvr -END PGP PRIVATE KEY BLOCK- '); insert into keytbl (id, name, pubkey, seckey) Index: pgsql/contrib/pgcrypto/sql/pgp-pubkey-decrypt.sql === *** pgsql.orig/contrib/pgcrypto/sql/pgp-pubkey-decrypt.sql --- pgsql/contrib/pgcrypto/sql/pgp-pubkey-decrypt.sql *** ig3hGY1Rb4NEk1gAn1u9IuQB+BgDP40YHHz6bKWS *** 308,314 -BEGIN PGP PRIVATE KEY BLOCK- Version: GnuPG v1.4.1 (GNU/Linux) ! lQHhBELIIUgRBACp401L6jXrLB28c3YA4sM3OJKnxM1GT9YTkWyE3Vyte65H8WU9 tGPBX7OMuaX5eGZ84LFUGvaP0k7anfmXcDkCO3P9GgL+ro/dS2Ps/vChQPZqHaxE xpKDUt47B7DGdRJrC8DRnIR4wbSyQA6ma3S1yFqC5pJhSs+mqf9eExOjiwCgntth klRxIYw352ZX9Ov9oht/p/ED/1Xi4PS+tkXVvyIw5aZfa61bT6XvDkoPI0Aj3GE5 --- 308,314 -BEGIN PGP PRIVATE KEY BLOCK- Version: GnuPG v1.4.1 (GNU/Linux) ! lQHpBELIIUgRBACp401L6jXrLB28c3YA4sM3OJKnxM1GT9YTkWyE3Vyte65H8WU9 tGPBX7OMuaX5eGZ84LFUGvaP0k7anfmXcDkCO3P9GgL+ro/dS2Ps/vChQPZqHaxE xpKDUt47B7DGdRJrC8DRnIR4wbSyQA6ma3S1yFqC5pJhSs+mqf9eExOjiwCgntth klRxIYw352ZX9Ov9oht/p/ED/1Xi4PS+tkXVvyIw5aZfa61bT6XvDkoPI0Aj3GE5 *** YmCHJlKA/IhEr8QJOLV++5VEv4l6KQ1/DFoJzoNd *** 316,336 PHQme5oAWoHa6bVQZOwvbJh3mOXDq/Tk/KF22go8maM44vMn4bvv+SBbslviYLiL
Re: [PATCHES] [PATCH] Magic block for modules
On 5/8/06, Martijn van Oosterhout kleptog@svana.org wrote: This implements a proposal made last november: http://archives.postgresql.org/pgsql-hackers/2005-11/msg00578.php All it requires is to include the header pgmagic.h and to put somewhere in their source: PG_MODULE_MAGIC Could you serve this as special docstring instead? Eg: PG_MODULE(foomodule) is mandatory, there you can to your magic, and optional: PG_MODULE_DESC(Do foo) PG_MODULE_AUTHOR(FooMan [EMAIL PROTECTED]) This provides more motivation for module authors and also creates (visually) smooth path to provide automatic install, uninstall and registration: PG_MODULE_INSTALL(inst_sql) PG_MODULE_UNINSTALL(uninst_sql) create module foo from '$libdir/foo'; drop module foo; This seems like worthwhile direction to move, especially as it requires pretty small amount of changes. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [PATCH] Magic block for modules
On 5/31/06, Martijn van Oosterhout kleptog@svana.org wrote: On Wed, May 31, 2006 at 01:08:41PM +0300, Marko Kreen wrote: On 5/8/06, Martijn van Oosterhout kleptog@svana.org wrote: All it requires is to include the header pgmagic.h and to put somewhere in their source: PG_MODULE_MAGIC Could you serve this as special docstring instead? Eg: PG_MODULE(foomodule) is mandatory, there you can to your magic, and optional: snip I like it, but I'm not sure there's enough consensus for that. I've suggested before including install info inside the modules themselves but there doesn't seem to be much interest in that. I am not suggesting to try to go all the way, just to make sure that your current patch fits into that direction. Apart from that there's issues with implementation. The Linux kernel can do it easily because it knows it will be using ELF, thus can use sections to store this info. Postgresql has to support many more types, making things like this tricky (but not impossible). PostgreSQL already requires symbol loading functionality for V1 function signatures, so per-module symbols won't be much burden. Personally I'd like postgres to move to a system where external modules can easily be installed, uninstalled and upgraded. However, I've not seen the demand yet. Demand happens only when users get used to such niceties on some other databases. Considering that PostgreSQL is extensibility-wise most advanced database and anything we offer is worlds best, there won't be any demand in years to come. I rather think we should create that demand. Tasks like - see what modules are installed in database. - install module - remove module are rather clunky in current setup. Making them easier would be good thing. Ofcourse, its easy to tell others to do things. I'll try to hack on that area myself also. If not earlier then maybe on Summit Code Sprint at least. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [PATCH] Magic block for modules
On 5/31/06, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Could you serve this as special docstring instead? Eg: PG_MODULE(foomodule) I have no objection to that, and see no real implementation problem with it: we just add a const char * field to the magic block. The other stuff seems too blue-sky, and I'm not even sure that it's the right direction to proceed in. It was not blue-sky, it was handwaving :) Marko seems to be envisioning a future where an extension module is this binary blob with install/deinstall/etc code all hardwired into it. I don't like that a bit. I think the current scheme with separate SQL scripts is a *good* thing, because it makes it a lot easier for users to tweak the SQL definitions, eg, install the functions into a non-default schema. Also, I don't have a problem imagining extension modules that contain no C code, just PL functions --- so the SQL script needs to be considered the primary piece of the module, not the shared library. I'll later post a list of ideas that we can hopefully agree on and discuss them further. Is it worth adding a module name to the magic block, or should we just leave well enough alone? It's certainly not something foreseen as part of the purpose of that block. In the absence of some fairly concrete ideas what to do with it, I'm probably going to vote keep-it-simple. Yes, if we want to keep separate SQL for modules then putting stuff into .so is pointless. -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Have configure complain about unknown options
On 5/9/06, Peter Eisentraut [EMAIL PROTECTED] wrote: Am Freitag, 5. Mai 2006 20:07 schrieb Martijn van Oosterhout: 1. Provide an escape option they can add 2. Package systems can usually apply patches prior to compiling, they can always remove the offending line if they like. 3. Try and get feedback from them now rather than wait My feedback is this: You are going to enter a world of pain. Seems that way. Especially if the non-PGAC options won't be handled automatically. Some projects have solved the problem different way - by printing out the summary of most important options at the end of configure: PostgreSQL version X.X.X OpenSSL: no Integer datatime: yes Python: yes Perl: yes So at the end of configure the user can visually confirm his expectations without needing to parse the noise from full configure output. Maybe this would be better solution. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [PATCH] Have configure complain about unknown options
On 5/4/06, Tom Lane [EMAIL PROTECTED] wrote: Martijn van Oosterhout kleptog@svana.org writes: Currently, configure ignores unknown --enable/disable/with/without options. The autoconf people consider that a feature, not a bug. I'm disinclined to second-guess the designers of the tool, especially with a patch like this that mucks with the internals to the extent that it'll probably break in every future autoconf revision. Feel free to try to convince them to change it though ... AFAIK that 'feature' is there to support configuring a 'tree' of projects (like gcc), where subprojects have their own configure scripts with different options. That way you can give all options to top-level configure script which passes them to other scripts and each picks only whats needed. In such setting all scripts need to ignore unknown options. As PostgreSQL tree is not set up that way, I think for clarity sake it would be better to give explicit errors for unknown options. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] pgcrypto: fix memory leak in openssl.c
On 2/18/06, Marko Kreen [EMAIL PROTECTED] wrote: pgcrypto crypt()/md5 and hmac() leak memory when compiled against OpenSSL as openssl.c digest -reset will do two DigestInit calls against a context. This happened to work with OpenSSL 0.9.6 but not with 0.9.7+. Ugh, seems I read the old code slightly wrong. The leak happens also with regular digest(), although it will leak only 1 context instance, not the 1000+ as the crypt-md5 does. And on 8.1 there is pgp_sym_encrypt that also does lots of resets on one context, like crypt-md5. In addition it does regular digest() in several places. So if compiled against OpenSSL, its leaking everywhere. The positive side is that only 8.1 has openssl autoconfiguration, older versions default to builtin algorithms that can be changed only by editing Makefile, thus most packages are hopefully safe. -- marko ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] pgcrypto: fix memory leak in openssl.c
On 2/20/06, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: On 2/18/06, Marko Kreen [EMAIL PROTECTED] wrote: pgcrypto crypt()/md5 and hmac() leak memory when compiled against OpenSSL as openssl.c digest -reset will do two DigestInit calls against a context. This happened to work with OpenSSL 0.9.6 but not with 0.9.7+. Ugh, seems I read the old code slightly wrong. The leak happens also with regular digest(), although it will leak only 1 context instance, not the 1000+ as the crypt-md5 does. I'm confused --- does this mean that the patch you sent recently needs further work? No, it's fine. As I did not 'fix' old code but replaced it. It's just that I gave wrong answer to the question 'who is affected?' -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[PATCHES] pgcrypto: fix memory leak in openssl.c
pgcrypto crypt()/md5 and hmac() leak memory when compiled against
OpenSSL as openssl.c digest -reset will do two DigestInit calls
against a context. This happened to work with OpenSSL 0.9.6
but not with 0.9.7+.
Reason for the messy code was that I tried to avoid creating
wrapper structure to transport algorithm info and tried to use
OpenSSL context for it. The fix is to create wrapper structure.
It also uses newer digest API to avoid memory allocations
on reset with newer OpenSSLs.
Attached are one patch for 7.3, 7.4, 8.0 branches and another
for 8.1 and HEAD.
Thanks to Daniel Blaisdell for reporting it.
--
marko
Index: contrib/pgcrypto/openssl.c
===
RCS file: /opt/cvs/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.12.4.1
diff -u -c -r1.12.4.1 openssl.c
*** contrib/pgcrypto/openssl.c 12 Mar 2005 06:55:14 - 1.12.4.1
--- contrib/pgcrypto/openssl.c 17 Feb 2006 23:31:29 -
***
*** 36,95
#include openssl/evp.h
/*
* Hashes
*/
static unsigned
digest_result_size(PX_MD * h)
{
! return EVP_MD_CTX_size((EVP_MD_CTX *) h-p.ptr);
}
static unsigned
digest_block_size(PX_MD * h)
{
! return EVP_MD_CTX_block_size((EVP_MD_CTX *) h-p.ptr);
}
static void
digest_reset(PX_MD * h)
{
! EVP_MD_CTX *ctx = (EVP_MD_CTX *) h-p.ptr;
! const EVP_MD *md;
! md = EVP_MD_CTX_md(ctx);
!
! EVP_DigestInit(ctx, md);
}
static void
digest_update(PX_MD * h, const uint8 *data, unsigned dlen)
{
! EVP_MD_CTX *ctx = (EVP_MD_CTX *) h-p.ptr;
! EVP_DigestUpdate(ctx, data, dlen);
}
static void
digest_finish(PX_MD * h, uint8 *dst)
{
! EVP_MD_CTX *ctx = (EVP_MD_CTX *) h-p.ptr;
! const EVP_MD *md = EVP_MD_CTX_md(ctx);
!
! EVP_DigestFinal(ctx, dst, NULL);
! /*
! * Some builds of 0.9.7x clear all of ctx in EVP_DigestFinal.
! * Fix it by reinitializing ctx.
! */
! EVP_DigestInit(ctx, md);
}
static void
digest_free(PX_MD * h)
{
! EVP_MD_CTX *ctx = (EVP_MD_CTX *) h-p.ptr;
! px_free(ctx);
px_free(h);
}
--- 36,124
#include openssl/evp.h
/*
+ * Backwards compatibility code for digest.
+ */
+ #if OPENSSL_VERSION_NUMBER 0x00907000L
+
+ static void EVP_MD_CTX_init(EVP_MD_CTX *ctx)
+ {
+ memset(ctx, 0, sizeof(*ctx));
+ }
+
+ static int EVP_MD_CTX_cleanup(EVP_MD_CTX *ctx)
+ {
+ memset(ctx, 0, sizeof(*ctx));
+ return 1;
+ }
+
+ static int EVP_DigestInit_ex(EVP_MD_CTX *ctx, const EVP_MD *md, void *engine)
+ {
+ EVP_DigestInit(ctx, md);
+ return 1;
+ }
+
+ static int EVP_DigestFinal_ex(EVP_MD_CTX *ctx, unsigned char *res, unsigned int *len)
+ {
+ EVP_DigestFinal(ctx, res, len);
+ return 1;
+ }
+
+ #endif
+
+ /*
* Hashes
*/
+
+ typedef struct OSSLDigest {
+ const EVP_MD *algo;
+ EVP_MD_CTX ctx;
+ } OSSLDigest;
+
static unsigned
digest_result_size(PX_MD * h)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! return EVP_MD_CTX_size(digest-ctx);
}
static unsigned
digest_block_size(PX_MD * h)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! return EVP_MD_CTX_block_size(digest-ctx);
}
static void
digest_reset(PX_MD * h)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! EVP_DigestInit_ex(digest-ctx, digest-algo, NULL);
}
static void
digest_update(PX_MD * h, const uint8 *data, unsigned dlen)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! EVP_DigestUpdate(digest-ctx, data, dlen);
}
static void
digest_finish(PX_MD * h, uint8 *dst)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! EVP_DigestFinal_ex(digest-ctx, dst, NULL);
}
static void
digest_free(PX_MD * h)
{
! OSSLDigest *digest = (OSSLDigest *)h-p.ptr;
! EVP_MD_CTX_cleanup(digest-ctx);
! px_free(digest);
px_free(h);
}
***
*** 101,108
px_find_digest(const char *name, PX_MD ** res)
{
const EVP_MD *md;
- EVP_MD_CTX *ctx;
PX_MD *h;
if (!px_openssl_initialized)
{
--- 130,137
px_find_digest(const char *name, PX_MD ** res)
{
const EVP_MD *md;
PX_MD *h;
+ OSSLDigest *digest;
if (!px_openssl_initialized)
{
***
*** 114,121
if (md == NULL)
return -1;
! ctx = px_alloc(sizeof(*ctx));
! EVP_DigestInit(ctx, md);
h = px_alloc(sizeof(*h));
h-result_size = digest_result_size;
--- 143,154
if (md == NULL)
return -1;
! digest = px_alloc(sizeof(*digest));
! digest-algo = md;
!
! EVP_MD_CTX_init(digest-ctx);
! if (EVP_DigestInit_ex(digest-ctx, digest-algo, NULL) == 0)
! return -1;
h = px_alloc(sizeof(*h));
h-result_size = digest_result_size;
***
*** 124,130
h-update = digest_update;
h-finish = digest_finish;
h-free = digest_free;
! h-p.ptr = (void *) ctx;
*res = h;
return 0;
--- 157,163
h-update = digest_update;
h-finish = digest_finish;
h-free = digest_free;
! h-p.ptr = (void *) digest;
*res = h;
return 0;
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/7/06, Bruce Momjian pgman@candle.pha.pa.us wrote:
Marko Kreen wrote:
The above table seem bit messy, but I see it as much easier to explain
to somebody.
I am confused about your list above, so I can't see how that would be
easy to explain.
Easy as in use GRANT USAGE, forget about rest. You are confused
because you know the old way and look them together.
I would have liked to say the rest are for fine-grained access control,
but with Tom's final proposal, the explanation would continue SELECT,
UPDATE are for backwards compatibility.
Attached is a patch that fixes tablename-seqname and puts USAGE
as first in list to show it's the preferred way. I think it should
be mentioned somewhere explicitly, but I cant find proper place for
it. In the Compatibility section for GRANT?
--
marko
Index: pgsql/doc/src/sgml/ref/grant.sgml
===
*** pgsql.orig/doc/src/sgml/ref/grant.sgml
--- pgsql/doc/src/sgml/ref/grant.sgml
*** GRANT { { SELECT | INSERT | UPDATE | DEL
*** 25,33
ON [ TABLE ] replaceable class=PARAMETERtablename/replaceable [, ...]
TO { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...] [ WITH GRANT OPTION ]
! GRANT { { SELECT | USAGE | UPDATE }
[,...] | ALL [ PRIVILEGES ] }
! ON SEQUENCE replaceable class=PARAMETERtablename/replaceable [, ...]
TO { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { { CREATE | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] }
--- 25,33
ON [ TABLE ] replaceable class=PARAMETERtablename/replaceable [, ...]
TO { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...] [ WITH GRANT OPTION ]
! GRANT { { USAGE | SELECT | UPDATE }
[,...] | ALL [ PRIVILEGES ] }
! ON SEQUENCE replaceable class=PARAMETERsequencename/replaceable [, ...]
TO { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { { CREATE | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] }
Index: pgsql/doc/src/sgml/ref/revoke.sgml
===
*** pgsql.orig/doc/src/sgml/ref/revoke.sgml
--- pgsql/doc/src/sgml/ref/revoke.sgml
*** REVOKE [ GRANT OPTION FOR ]
*** 28,36
[ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]
! { { SELECT | UPDATE }
[,...] | ALL [ PRIVILEGES ] }
! ON SEQUENCE replaceable class=PARAMETERtablename/replaceable [, ...]
FROM { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...]
[ CASCADE | RESTRICT ]
--- 28,36
[ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]
! { { USAGE | SELECT | UPDATE }
[,...] | ALL [ PRIVILEGES ] }
! ON SEQUENCE replaceable class=PARAMETERsequencename/replaceable [, ...]
FROM { replaceable class=PARAMETERusername/replaceable | GROUP replaceable class=PARAMETERgroupname/replaceable | PUBLIC } [, ...]
[ CASCADE | RESTRICT ]
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Bruno Wolff III wrote: It might be nice to split nextval and currval access as well. nextval access corresponds to INSERT and currval access to SELECT. Uh, that is already in the code. nextval()/setval() is UPDATE, and currval() is SELECT. This seems weird. Shouldn't nextval/currval go together and setval separately? Considering there's no currval() without nextval(), what point is disallowing currval() when user is able to call nextval()? I rather want to allow nextval/currval and disable setval as it allows regular user to DoS the database. -- marko [removing Tom from CC as he bounces gmail] ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Marko Kreen wrote: On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Bruno Wolff III wrote: It might be nice to split nextval and currval access as well. nextval access corresponds to INSERT and currval access to SELECT. Uh, that is already in the code. nextval()/setval() is UPDATE, and currval() is SELECT. This seems weird. Shouldn't nextval/currval go together and setval separately? Uh, logically, yes, but practially currval just reads/SELECTs, while nextval modifies/UPDATEs. Yeah, thats the mechanics behind it, but the currval() only works if the user was already able to call nextval(), so I see no point in separating them. In other words: there is nothing to do with only access to currval(), and with access to nextval() but not to currval() user loses only in convinience. Considering there's no currval() without nextval(), what point is disallowing currval() when user is able to call nextval()? Not sure. I think SET SESSION AUTHORIZATION would make it possible. /me confused, looks at docs... Huh? I really hope you are mistaken. This would mean the sequence state for currval() is kept per-user not per-backend. This would make impossible to make several connections as same user. Is Postgres really that broken? I rather want to allow nextval/currval and disable setval as it allows regular user to DoS the database. Oh, interesting. We could easily have INSERT control that if we wanted, but I think you have to make a clear use case to override the risk of breaking applications. I'd turn it around: is there any use-case for setval() for regular user? IMHO it's a admin-level operation, dangerous, and not needed for regular work. -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Uh, logically, yes, but practially currval just reads/SELECTs, while nextval modifies/UPDATEs. Yeah, thats the mechanics behind it, but the currval() only works if the user was already able to call nextval(), so I see no point in separating them. You are completely wrong on this, because not all the code in a session necessarily executes at the same privilege level. For instance, the nextval() might be executed inside a SECURITY DEFINER function. It might be reasonable to give code outside that function the right to see what had been assigned (by executing currval()) without also saying that it could do further nextvals(). Ah, I did not think of this. Indeed, it's useful to keep them separate. I just wanted to point out that I see much more use to keep setval() separate from nextval/currval. (that is - always) I do agree that it would be a good idea to support a privilege distinction between nextval() and setval(). I tried to imagine a usage scenario for setval() but only single-user bulk data load comes in mind. Is there any actual scenario where it could be useful in multi-user setting? Oh, interesting. We could easily have INSERT control that if we wanted, but I think you have to make a clear use case to override the risk of breaking applications. There is no backwards-compatibility risk, because we'd still have the old GRANT ON TABLE syntax grant both underlying rights. You'd have to use the new syntax to get to a state where you had nextval but not setval privilege or vice versa. Good idea. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Bruno Wolff III [EMAIL PROTECTED] wrote: On Fri, Jan 06, 2006 at 19:11:27 +0200, Marko Kreen [EMAIL PROTECTED] wrote: On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Considering there's no currval() without nextval(), what point is disallowing currval() when user is able to call nextval()? I rather want to allow nextval/currval and disable setval as it allows regular user to DoS the database. What I was thinking with this, is that you might allow someone the ability to insert records into a table which would make use of nextval, but not allow them to run nextval directly. But after inserting a record allow them to use currval to see what value was assigned. People could still mess with things by doing INSERTs and aborting the transaction, so this may not be the best example for why you would want this. This is similar to Tom's scenario. I'm not against keeping them separate. But my question is rather - is there any scenario where setval() should go with nextval()? It seems that their pairing is an accident and should be fixed. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: But my question is rather - is there any scenario where setval() should go with nextval()? It seems that their pairing is an accident and should be fixed. I think the original argument for the current design was that with enough nextval's you can duplicate the effect of a setval. This is only strictly true if the sequence is CYCLE mode, and even then it'd take a whole lot of patience to wrap an int8 sequence around ... but the distinction between them is not so large as you make it out to be. With bigserial this is more like CPU DoS, while other users can work normally. In any case I think we are wasting our time discussing it, and instead should be looking through the SQL2003 spec to see what it requires. Bruce couldn't find anything in it about this but I can't believe the info isn't there somewhere. Google tells that Oracle has ALTER and SELECT; DB2 has ALTER and USAGE. I found SQL2003 pdf's too ... from my reading it has only USAGE. 5WD-02-Foundation-2003-09.pdf: page 724 - General Rules - #2 page 740 - Syntax rules - #3 Everything combined: SELECT: currval UPDATE: nextval USAGE: currval, nextval ALTER: setval Confusing? -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/6/06, Bruce Momjian pgman@candle.pha.pa.us wrote: Marko Kreen wrote: I found SQL2003 pdf's too ... from my reading it has only USAGE. 5WD-02-Foundation-2003-09.pdf: page 724 - General Rules - #2 page 740 - Syntax rules - #3 I admit I am terrible at understanding the standard, but I can't find anything relevant on the page numbers you mentioned. Are those the document pages or the page numbers displayed by the PDF viewer? What is the section heading? I am using the same filename you have. Those are print page numbers. (In case you have dead-tree variant :) And I got them here: http://www.wiscorp.com/SQLStandards.html Uh, and they are bit wrong. Ok here are they fully: 11.62 sequence generator definition General rules (page 727 printed/751 real) point #2 12.3 privileges Syntax rules (page 740 printed/764 real) point #3 Everything combined: SELECT: currval UPDATE: nextval USAGE: currval, nextval ALTER: setval Confusing? I see USAGE in the standard, but not ALTER. We don't support USAGE so I am guessing our SELECT/UPDATE behavior is OK. No, we still want to separate setval from nextval. Does this mean we should only allow owners to do setval(), rather than binding it to INSERT? My first reaction is that it should be grantable, although I can't find any reasons for it, except backwards compatibility. How about this: SELECT: currval INSERT: nextval USAGE: currval, nextval UPDATE: setval -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] Inconsistent syntax in GRANT
On 1/7/06, Tom Lane [EMAIL PROTECTED] wrote: Bruce Momjian pgman@candle.pha.pa.us writes: FYI, we could support USAGE just on sequences, and have it map to UPDATE, but pg_dump it out as USAGE. It seems the spec doesn't cover setval() and currval(), which is not too surprising given those aren't standard. Here is a proposal: SELECT priv - allows currval() and SELECT * FROM seq USAGE priv - allows nextval() (required by SQL2003) UPDATE priv - allows setval() and nextval() I was originally thinking of a separate privilege bit for setval(), but that's sort of silly, as you can get (approximately) the effect of nextval() via setval(). Not much point in prohibiting nextval() to someone who can do setval(). This is 100% upward compatible with our current definition, and it meets both the SQL spec and Marko's desire to have a way of granting only nextval() privilege. Good point about compatibility. But makes the common case ugly. For regular usage you need to grant SELECT, USAGE ... Huh? :) How about this: SELECT: currval INSERT: nextval UPDATE: nextval, setval USAGE: nextval, currval With this the user needs only to remember SQL2003 syntax to cover 99.9% use cases. And when he wants to play more finegrained then he can combine with the SELECT, INSERT, UPDATE. The above table seem bit messy, but I see it as much easier to explain to somebody. BTW, what about lastval()? I'm not sure we can usefully associate any privilege check with that, since it's not clear which sequence it applies to. Does it make sense to remember what sequence the value came from and privilege-check against that, or is that just too weird? Hmm. So it means with lastval() user can see the state of sequences he has no access to? Seems like the privilege check would be good idea. -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] pgcrypto: bug in gen_salt (md5/xdes)
There is a signedness bug in Openwall gen_salt code that
pgcrypto uses. This makes the salt space for md5 and xdes
algorithms a lot smaller.
Salts for blowfish and standard des are unaffected.
Attached is upstream fix for it. This applies all the
way from 7.2 to 8.1 and HEAD. Please apply this to all
active branches.
--
marko
Index: contrib/pgcrypto/crypt-gensalt.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/crypt-gensalt.c,v
retrieving revision 1.6
diff -u -c -r1.6 crypt-gensalt.c
*** contrib/pgcrypto/crypt-gensalt.c 15 Oct 2005 02:49:06 - 1.6
--- contrib/pgcrypto/crypt-gensalt.c 2 Jan 2006 22:27:35 -
***
*** 62,70
output[2] = _crypt_itoa64[(count 6) 0x3f];
output[3] = _crypt_itoa64[(count 12) 0x3f];
output[4] = _crypt_itoa64[(count 18) 0x3f];
! value = (unsigned long) input[0] |
! ((unsigned long) input[1] 8) |
! ((unsigned long) input[2] 16);
output[5] = _crypt_itoa64[value 0x3f];
output[6] = _crypt_itoa64[(value 6) 0x3f];
output[7] = _crypt_itoa64[(value 12) 0x3f];
--- 62,70
output[2] = _crypt_itoa64[(count 6) 0x3f];
output[3] = _crypt_itoa64[(count 12) 0x3f];
output[4] = _crypt_itoa64[(count 18) 0x3f];
! value = (unsigned long)(unsigned char) input[0] |
! ((unsigned long)(unsigned char) input[1] 8) |
! ((unsigned long)(unsigned char) input[2] 16);
output[5] = _crypt_itoa64[value 0x3f];
output[6] = _crypt_itoa64[(value 6) 0x3f];
output[7] = _crypt_itoa64[(value 12) 0x3f];
***
*** 90,98
output[0] = '$';
output[1] = '1';
output[2] = '$';
! value = (unsigned long) input[0] |
! ((unsigned long) input[1] 8) |
! ((unsigned long) input[2] 16);
output[3] = _crypt_itoa64[value 0x3f];
output[4] = _crypt_itoa64[(value 6) 0x3f];
output[5] = _crypt_itoa64[(value 12) 0x3f];
--- 90,98
output[0] = '$';
output[1] = '1';
output[2] = '$';
! value = (unsigned long)(unsigned char) input[0] |
! ((unsigned long)(unsigned char) input[1] 8) |
! ((unsigned long)(unsigned char) input[2] 16);
output[3] = _crypt_itoa64[value 0x3f];
output[4] = _crypt_itoa64[(value 6) 0x3f];
output[5] = _crypt_itoa64[(value 12) 0x3f];
***
*** 101,109
if (size = 6 output_size = 3 + 4 + 4 + 1)
{
! value = (unsigned long) input[3] |
! ((unsigned long) input[4] 8) |
! ((unsigned long) input[5] 16);
output[7] = _crypt_itoa64[value 0x3f];
output[8] = _crypt_itoa64[(value 6) 0x3f];
output[9] = _crypt_itoa64[(value 12) 0x3f];
--- 101,109
if (size = 6 output_size = 3 + 4 + 4 + 1)
{
! value = (unsigned long)(unsigned char) input[3] |
! ((unsigned long)(unsigned char) input[4] 8) |
! ((unsigned long)(unsigned char) input[5] 16);
output[7] = _crypt_itoa64[value 0x3f];
output[8] = _crypt_itoa64[(value 6) 0x3f];
output[9] = _crypt_itoa64[(value 12) 0x3f];
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
[PATCHES] pgcrypto doc polish
Few small things: - Mention pgcrypto.sql - Mention asciidoc. To hint that occasional weird formatting is not random. - Clarify few senctences. - Remove anything related to MySQL password(). It was interesting to look at, but it should not be mentioned in serious conversation. Also, they have replaced it in 4.1 with SHA1, so the crap implementation is best forgotten. -- marko Index: contrib/pgcrypto/README.pgcrypto === RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/README.pgcrypto,v retrieving revision 1.13 diff -u -c -r1.13 README.pgcrypto *** contrib/pgcrypto/README.pgcrypto13 Aug 2005 02:06:20 - 1.13 --- contrib/pgcrypto/README.pgcrypto1 Nov 2005 22:56:42 - *** *** 3,8 --- 3,10 = Marko Kreen marko@l-t.ee + // Note: this document is in asciidoc format. + 1. Installation - *** *** 17,22 --- 19,31 for the module. They make sure the functions here produce correct results. + Next, to get the function into particular database, run commands in + file pgcrypto.sql, which should be installed into shared files directory. + + Example using psql: + + psql -d DBNAME -f pgcrypto.sql + 2. Notes -- *** *** 248,254 crypt-des 3628377 days 19 years sha1 5902234 days 12 years md5 2345086 1 day 3 years - password 143781000 25 mins 18 days * The machine used is 1.5GHz Pentium 4. --- 257,262 *** *** 256,263 John the Ripper v1.6.38 `-test` output. * MD5 numbers are from mdcrack 1.2. * SHA1 numbers are from lcrack-20031130-beta. - * MySQL password() numbers are from my own tests. - (http://grue.l-t.ee/~marko/src/mypass/) * `crypt-bf` numbers are taken using simple program that loops over 1000 8-character passwords. That way I can show the speed with different number of rounds. For reference: `john -test` shows 213 --- 264,269 *** *** 268,276 Note that the try all combinations is not a realistic exercise. Usually password cracking is done with the help of dictionaries, which contain both regular words and various mutations of them. So, even ! somewhat word-like passwords will be cracked much faster than the above numbers suggest, and a 6-character non-word like password may escape ! cracking. Or may not. 5. PGP encryption --- 274,282 Note that the try all combinations is not a realistic exercise. Usually password cracking is done with the help of dictionaries, which contain both regular words and various mutations of them. So, even ! somewhat word-like passwords could be cracked much faster than the above numbers suggest, and a 6-character non-word like password may escape ! cracking. Or not. 5. PGP encryption ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] [patch 0/7] more patches for pgcrypto
On Fri, Aug 12, 2005 at 10:06:21PM -0400, Bruce Momjian wrote: Thanks. All seven applied. Good, but you missed one step: 'cvs remove API' -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [patch 0/7] more patches for pgcrypto
On Sat, Aug 13, 2005 at 11:12:18AM -0400, Bruce Momjian wrote: Marko Kreen wrote: On Fri, Aug 12, 2005 at 10:06:21PM -0400, Bruce Momjian wrote: Thanks. All seven applied. Good, but you missed one step: 'cvs remove API' Uh, I reread your emails and couldn't find what file to remove. Which ones? Heh. The file API. See last patch. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] [patch 4/7] small fixes
o pgp_mpi_free: Accept NULLs
o pgp_mpi_cksum: result should be 16bit
o Remove function name from error messages - to be similar to other
SQL functions, and it does not match anyway the called function
o remove couple junk lines
Index: pgsql/contrib/pgcrypto/pgp-mpi.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-mpi.c
--- pgsql/contrib/pgcrypto/pgp-mpi.c
*** int pgp_mpi_create(uint8 *data, int bits
*** 66,71
--- 66,73
int pgp_mpi_free(PGP_MPI *mpi)
{
+ if (mpi == NULL)
+ return 0;
memset(mpi, 0, sizeof(*mpi) + mpi-bytes);
px_free(mpi);
return 0;
*** unsigned pgp_mpi_cksum(unsigned cksum, P
*** 129,134
for (i = 0; i n-bytes; i++)
cksum += n-data[i];
! return cksum;
}
--- 131,136
for (i = 0; i n-bytes; i++)
cksum += n-data[i];
! return cksum 0x;
}
Index: pgsql/contrib/pgcrypto/pgp-pubkey.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-pubkey.c
--- pgsql/contrib/pgcrypto/pgp-pubkey.c
***
*** 34,41
#include mbuf.h
#include pgp.h
- #define PXE_PGP_BAD_KEY -90
-
int pgp_key_alloc(PGP_PubKey **pk_p)
{
PGP_PubKey *pk;
--- 34,39
Index: pgsql/contrib/pgcrypto/pgp-pgsql.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-pgsql.c
--- pgsql/contrib/pgcrypto/pgp-pgsql.c
*** encrypt_internal(int is_pubenc, int is_t
*** 496,502
mbuf_free(dst);
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(pgp_encrypt error: %s,
px_strerror(err;
}
/* res_len includes VARHDRSZ */
--- 496,502
mbuf_free(dst);
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(%s, px_strerror(err;
}
/* res_len includes VARHDRSZ */
*** out:
*** 591,597
mbuf_free(dst);
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(pgp_decrypt error: %s,
px_strerror(err;
}
res_len = mbuf_steal_data(dst, restmp);
--- 591,597
mbuf_free(dst);
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(%s, px_strerror(err;
}
res_len = mbuf_steal_data(dst, restmp);
*** pg_dearmor(PG_FUNCTION_ARGS)
*** 879,885
if (res_len 0)
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(dearmor: %s, px_strerror(res_len;
if (res_len guess_len)
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
--- 879,885
if (res_len 0)
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
!errmsg(%s, px_strerror(res_len;
if (res_len guess_len)
ereport(ERROR,
(errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
*** pgp_key_id_w(PG_FUNCTION_ARGS)
*** 909,917
buf = create_mbuf_from_vardata(data);
res = palloc(VARHDRSZ + 17);
- px_set_debug_handler(show_debug);
res_len = pgp_get_keyid(buf, VARDATA(res));
- px_set_debug_handler(NULL);
mbuf_free(buf);
if (res_len 0)
ereport(ERROR,
--- 909,915
Index: pgsql/contrib/pgcrypto/px.c
===
*** pgsql.orig/contrib/pgcrypto/px.c
--- pgsql/contrib/pgcrypto/px.c
*** static const struct error_desc px_err_li
*** 72,85
{PXE_PGP_SHORT_ELGAMAL_KEY, Elgamal keys must be at least 1024 bits
long},
{PXE_PGP_RSA_UNSUPPORTED, pgcrypto does not support RSA keys},
{PXE_PGP_UNKNOWN_PUBALGO, Unknown public-key encryption algorithm},
! {PXE_PGP_WRONG_KEYID, Data is not encrypted with this key},
{PXE_PGP_MULTIPLE_KEYS,
Several keys given - pgcrypto does not handle keyring},
{PXE_PGP_EXPECT_PUBLIC_KEY, Refusing to encrypt with secret key},
{PXE_PGP_EXPECT_SECRET_KEY, Cannot decrypt with public key},
{PXE_PGP_NOT_V4_KEYPKT, Only V4 key packets are supported},
{PXE_PGP_KEYPKT_CORRUPT, Corrupt key packet},
!
[PATCHES] [patch 3/7] Elgamal speedup
I was bit hasty making the random exponent 'k' a prime. Further researh shows that Elgamal encryption has no specific needs in respect to k, any random number is fine. It is bit different for signing, there it needs to be 'relatively prime' to p - 1, that means GCD(k, p-1) == 1, which is also a lot lighter than full primality. As we don't do signing, this can be ignored. This brings major speedup to Elgamal encryption. Index: pgsql/contrib/pgcrypto/pgp-mpi-openssl.c === *** pgsql.orig/contrib/pgcrypto/pgp-mpi-openssl.c --- pgsql/contrib/pgcrypto/pgp-mpi-openssl.c *** pgp_elgamal_encrypt(PGP_PubKey *pk, PGP_ *** 120,126 * generate k */ k_bits = decide_k_bits(BN_num_bits(p)); ! if (!BN_generate_prime(k, k_bits, 0, NULL, NULL, NULL, NULL)) goto err; /* --- 120,126 * generate k */ k_bits = decide_k_bits(BN_num_bits(p)); ! if (!BN_rand(k, k_bits, 0, 0)) goto err; /* -- ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[PATCHES] [patch 6/7] small updates for README
o Tom stuck a CVS id into file. I doubt the usefulness of it, but if it needs to be in the file then rather at the end. Also tag it as comment for asciidoc. o Mention bytea vs. text difference o Couple clarifications Index: pgsql/contrib/pgcrypto/README.pgcrypto === *** pgsql.orig/contrib/pgcrypto/README.pgcrypto --- pgsql/contrib/pgcrypto/README.pgcrypto *** *** 1,4 - $PostgreSQL: pgsql/contrib/pgcrypto/README.pgcrypto,v 1.12 2005/07/18 17:17:12 tgl Exp $ pgcrypto - cryptographic functions for PostgreSQL = --- 1,3 *** cracking. Or may not. *** 278,284 --- The functions here implement the encryption part of OpenPGP (RFC2440) ! standard. 5.1. Overview --- 277,283 --- The functions here implement the encryption part of OpenPGP (RFC2440) ! standard. Supported are both symmetric-key and public-key encryption. 5.1. Overview *** Options are described in section 5.7. *** 334,339 --- 333,342 Decrypt a symmetric-key encrypted PGP message. + Decrypting bytea data with `pgp_sym_decrypt` is disallowed. + This is to avoid outputting invalid character data. Decrypting + originally textual data with `pgp_sym_decrypt_bytea` is fine. + Options are described in section 5.7. *** key is password-protected, you must give *** 362,367 --- 365,374 there is no password, but you want to specify option for function, you need to give empty password. + Decrypting bytea data with `pgp_pub_decrypt` is disallowed. + This is to avoid outputting invalid character data. Decrypting + originally textual data with `pgp_pub_decrypt_bytea` is fine. + Options are described in section 5.7. *** cipher-algo:: *** 422,428 Default: aes128 Applies: pgp_sym_encrypt, pgp_pub_encrypt - compress-algo:: Which compression algorithm to use. Needs building with zlib. --- 429,434 *** s2k-cipher-algo:: *** 492,498 Which cipher to use for encrypting separate session key. Values: bf, aes, aes128, aes192, aes256 ! Default: same as cipher-algo. Applies: pgp_sym_encrypt unicode-mode:: --- 498,504 Which cipher to use for encrypting separate session key. Values: bf, aes, aes128, aes192, aes256 ! Default: use cipher-algo. Applies: pgp_sym_encrypt unicode-mode:: *** Generate a new key: *** 513,519 gpg --gen-key ! You need to pick DSA and Elgamal key type, others are sign-only. List keys: --- 519,528 gpg --gen-key ! The preferred key type is DSA and Elgamal. ! ! For RSA encryption you must create either DSA or RSA sign-only key ! as master and then add RSA encryption subkey with `gpg --edit-key`. List keys: *** You need to use `dearmor()` on them befo *** 531,536 --- 540,548 pgp_pub_* functions. Or if you can handle binary data, you can drop -a from gpg. + For more details see `man gpg`, http://www.gnupg.org/gph/en/manual.html[ + The GNU Privacy Handbook] and other docs on http://www.gnupg.org[] site. + 5.10. Limitations of PGP code ~~~ *** pgp_pub_* functions. Or if you can hand *** 538,546 - No support for signing. That also means that it is not checked whether the encryption subkey belongs to master key. ! - No support for RSA keys. Only Elgamal encryption keys are supported ! - No support for several encryption subkeys. 6. Raw encryption --- 550,562 - No support for signing. That also means that it is not checked whether the encryption subkey belongs to master key. ! - No support for encryption key as master key. As such practice ! is generally discouraged, it should not be a problem. ! - No support for several subkeys. This may seem like a problem, as this ! is common practice. On the other hand, you should not use your regular ! GPG/PGP keys with pgcrypto, but create new ones, as the usage scenario ! is rather different. 6. Raw encryption *** I have used code from following sources: *** 631,636 --- 647,655 9.1. Useful reading ~ + http://www.gnupg.org/gph/en/manual.html[]:: + The GNU Privacy Handbook + http://www.openwall.com/crypt/[]:: Describes the crypt-blowfish algorithm. *** http://jlcooke.ca/random/[]:: *** 673,675 --- 692,698 http://www.cs.ut.ee/~helger/crypto/[]:: Collection of cryptology pointers. + + + // $PostgreSQL: pgsql/contrib/pgcrypto/README.pgcrypto,v 1.12 2005/07/18 17:17:12 tgl Exp $ + -- ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] [patch 1/7] remove unnecessary libs
The SHLIB section was copy-pasted from somewhere and contains several unnecessary libs. This cleans it up a bit. -lcrypt we don't use system crypt() -lssl, -lssleay32 no SSL here -lz in win32 section already added on previous line -ldes The chance anybody has it is pretty low. And the chance pgcrypto works with it is even lower. Also trim the win32 section. Index: pgsql/contrib/pgcrypto/Makefile === *** pgsql.orig/contrib/pgcrypto/Makefile --- pgsql/contrib/pgcrypto/Makefile *** endif *** 56,72 # Add libraries that pgcrypto depends (or might depend) on into the # shared library link. (The order in which you list them here doesn't # matter.) ! SHLIB_LINK += $(filter -lcrypt -ldes -lcrypto -lssl -lz, $(LIBS)) ifeq ($(PORTNAME), win32) ! SHLIB_LINK += $(filter -leay32 -lssleay32 -lz, $(LIBS)) ! endif ! ! # to make ws2_32.lib the last library (must occur after definition of PORTNAME) ! ifeq ($(PORTNAME),win32) SHLIB_LINK += -lwsock32 -lws2_32 endif - rijndael.o: rijndael.tbl rijndael.tbl: --- 56,68 # Add libraries that pgcrypto depends (or might depend) on into the # shared library link. (The order in which you list them here doesn't # matter.) ! SHLIB_LINK += $(filter -lcrypto -lz, $(LIBS)) ifeq ($(PORTNAME), win32) ! SHLIB_LINK += $(filter -leay32, $(LIBS)) ! # those must be at the end SHLIB_LINK += -lwsock32 -lws2_32 endif rijndael.o: rijndael.tbl rijndael.tbl: -- ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] [patch 5/7] support for RSA, pubkey reorg
o Support for RSA encryption
o Big reorg to better separate generic and algorithm-specific code.
o Regression tests for RSA.
Index: pgsql/contrib/pgcrypto/pgp-mpi-openssl.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-mpi-openssl.c
--- pgsql/contrib/pgcrypto/pgp-mpi-openssl.c
*** pgp_elgamal_encrypt(PGP_PubKey *pk, PGP_
*** 104,112
int res = PXE_PGP_MATH_FAILED;
int k_bits;
BIGNUM *m = mpi_to_bn(_m);
! BIGNUM *p = mpi_to_bn(pk-elg_p);
! BIGNUM *g = mpi_to_bn(pk-elg_g);
! BIGNUM *y = mpi_to_bn(pk-elg_y);
BIGNUM *k = BN_new();
BIGNUM *yk = BN_new();
BIGNUM *c1 = BN_new();
--- 104,112
int res = PXE_PGP_MATH_FAILED;
int k_bits;
BIGNUM *m = mpi_to_bn(_m);
! BIGNUM *p = mpi_to_bn(pk-pub.elg.p);
! BIGNUM *g = mpi_to_bn(pk-pub.elg.g);
! BIGNUM *y = mpi_to_bn(pk-pub.elg.y);
BIGNUM *k = BN_new();
BIGNUM *yk = BN_new();
BIGNUM *c1 = BN_new();
*** pgp_elgamal_decrypt(PGP_PubKey *pk, PGP_
*** 159,166
int res = PXE_PGP_MATH_FAILED;
BIGNUM *c1 = mpi_to_bn(_c1);
BIGNUM *c2 = mpi_to_bn(_c2);
! BIGNUM *p = mpi_to_bn(pk-elg_p);
! BIGNUM *x = mpi_to_bn(pk-elg_x);
BIGNUM *c1x = BN_new();
BIGNUM *div = BN_new();
BIGNUM *m = BN_new();
--- 159,166
int res = PXE_PGP_MATH_FAILED;
BIGNUM *c1 = mpi_to_bn(_c1);
BIGNUM *c2 = mpi_to_bn(_c2);
! BIGNUM *p = mpi_to_bn(pk-pub.elg.p);
! BIGNUM *x = mpi_to_bn(pk-sec.elg.x);
BIGNUM *c1x = BN_new();
BIGNUM *div = BN_new();
BIGNUM *m = BN_new();
*** err:
*** 195,197
--- 195,259
return res;
}
+ int
+ pgp_rsa_encrypt(PGP_PubKey *pk, PGP_MPI *_m, PGP_MPI **c_p)
+ {
+ int res = PXE_PGP_MATH_FAILED;
+ BIGNUM *m = mpi_to_bn(_m);
+ BIGNUM *e = mpi_to_bn(pk-pub.rsa.e);
+ BIGNUM *n = mpi_to_bn(pk-pub.rsa.n);
+ BIGNUM *c = BN_new();
+ BN_CTX *tmp = BN_CTX_new();
+
+ if (!m || !e || !n || !c || !tmp)
+ goto err;
+
+ /*
+* c = m ^ e
+*/
+ if (!BN_mod_exp(c, m, e, n, tmp))
+ goto err;
+
+ *c_p = bn_to_mpi(c);
+ if (*c_p)
+ res = 0;
+ err:
+ if (tmp) BN_CTX_free(tmp);
+ if (c) BN_clear_free(c);
+ if (n) BN_clear_free(n);
+ if (e) BN_clear_free(e);
+ if (m) BN_clear_free(m);
+ return res;
+ }
+
+ int
+ pgp_rsa_decrypt(PGP_PubKey *pk, PGP_MPI *_c, PGP_MPI **m_p)
+ {
+ int res = PXE_PGP_MATH_FAILED;
+ BIGNUM *c = mpi_to_bn(_c);
+ BIGNUM *d = mpi_to_bn(pk-sec.rsa.d);
+ BIGNUM *n = mpi_to_bn(pk-pub.rsa.n);
+ BIGNUM *m = BN_new();
+ BN_CTX *tmp = BN_CTX_new();
+
+ if (!m || !d || !n || !c || !tmp)
+ goto err;
+
+ /*
+* m = c ^ d
+*/
+ if (!BN_mod_exp(m, c, d, n, tmp))
+ goto err;
+
+ *m_p = bn_to_mpi(m);
+ if (*m_p)
+ res = 0;
+ err:
+ if (tmp) BN_CTX_free(tmp);
+ if (m) BN_clear_free(m);
+ if (n) BN_clear_free(n);
+ if (d) BN_clear_free(d);
+ if (c) BN_clear_free(c);
+ return res;
+ }
+
Index: pgsql/contrib/pgcrypto/pgp-pubdec.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-pubdec.c
--- pgsql/contrib/pgcrypto/pgp-pubdec.c
*** control_cksum(uint8 *msg, int msglen)
*** 91,96
--- 91,145
return 0;
}
+ static int
+ decrypt_elgamal(PGP_PubKey *pk, PullFilter *pkt, PGP_MPI **m_p)
+ {
+ int res;
+ PGP_MPI *c1 = NULL;
+ PGP_MPI *c2 = NULL;
+
+ if (pk-algo != PGP_PUB_ELG_ENCRYPT)
+ return PXE_PGP_WRONG_KEY;
+
+ /* read elgamal encrypted data */
+ res = pgp_mpi_read(pkt, c1);
+ if (res 0)
+ goto out;
+ res = pgp_mpi_read(pkt, c2);
+ if (res 0)
+ goto out;
+
+ /* decrypt */
+ res = pgp_elgamal_decrypt(pk, c1, c2, m_p);
+
+ out:
+ pgp_mpi_free(c1);
+ pgp_mpi_free(c2);
+ return res;
+ }
+
+ static int
+ decrypt_rsa(PGP_PubKey *pk, PullFilter *pkt, PGP_MPI **m_p)
+ {
+ int res;
+ PGP_MPI *c;
+
+ if (pk-algo != PGP_PUB_RSA_ENCRYPT
+pk-algo != PGP_PUB_RSA_ENCRYPT_SIGN)
+ return PXE_PGP_WRONG_KEY;
+
+ /* read rsa encrypted data */
+ res = pgp_mpi_read(pkt, c);
+ if (res 0)
+ return res;
+
+ /* decrypt */
+ res = pgp_rsa_decrypt(pk, c, m_p);
+
+ pgp_mpi_free(c);
+ return res;
+ }
+
/* key id is missing - user is expected to try all keys */
static const uint8
any_key[] = {0, 0, 0, 0, 0, 0, 0, 0};
*** pgp_parse_pubenc_sesskey(PGP_Context *ct
*** 102,108
int
[PATCHES] [patch 2/7] remove last pieces of system crypt()
It is already disabled in Makefile, remove code too.
Index: pgsql/contrib/pgcrypto/px-crypt.c
===
*** pgsql.orig/contrib/pgcrypto/px-crypt.c
--- pgsql/contrib/pgcrypto/px-crypt.c
***
*** 35,42
#include px-crypt.h
- #ifndef PX_SYSTEM_CRYPT
-
static char *
run_crypt_des(const char *psw, const char *salt,
char *buf, unsigned len)
--- 35,40
*** px_crypt(const char *psw, const char *sa
*** 107,130
return c-crypt(psw, salt, buf, len);
}
- #else /* PX_SYSTEM_CRYPT */
-
- extern char *crypt(const char *psw, const char *salt);
-
- char *
- px_crypt(const char *psw, const char *salt,
-char *buf, unsigned len)
- {
- char *res;
-
- res = crypt(psw, salt);
- if (!res || strlen(res) = len)
- return NULL;
- strcpy(buf, res);
- return buf;
- }
- #endif
-
/*
* salt generators
*/
--- 105,110
Index: pgsql/contrib/pgcrypto/px-crypt.h
===
*** pgsql.orig/contrib/pgcrypto/px-crypt.h
--- pgsql/contrib/pgcrypto/px-crypt.h
*** char *_crypt_gensalt_md5_rn(unsigned lon
*** 73,80
char *_crypt_gensalt_blowfish_rn(unsigned long count,
const char *input, int size, char *output, int
output_size);
- #ifndef PX_SYSTEM_CRYPT
-
/* disable 'extended DES crypt' */
/* #define DISABLE_XDES */
--- 73,78
*** char *px_crypt_des(const char *key, c
*** 88,93
/* crypt-md5.c */
char *px_crypt_md5(const char *pw, const char *salt,
char *dst, unsigned dstlen);
- #endif /* !PX_SYSTEM_CRYPT */
#endif /* _PX_CRYPT_H */
--- 86,90
--
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [PATCHES] Changes for AIX buildfarm
On Mon, Jul 18, 2005 at 12:32:21PM -0400, Tom Lane wrote: Rocco Altier [EMAIL PROTECTED] writes: There are C++ style comments in contrib/pgcrypto which is giving cc problems. Definitely contrary to project policy, will fix. Don't, they are fixed in small cleanups patch that I sent. -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Fri, Jul 15, 2005 at 08:06:15PM -0500, Kris Jurka wrote: On Fri, 15 Jul 2005, Marko Kreen wrote: [buildfarm machine dragonfly] On Tue, Jul 12, 2005 at 01:06:46PM -0500, Kris Jurka wrote: Well the buildfarm machine kudu is actually the same machine just building with the Sun compiler and it works fine. It links all of libz.a into libpgcrypto.so while gcc refuses to. I googled a bit and found two suggestions: 1. http://curl.haxx.se/mail/lib-2002-01/0092.html (Use -mimpure-text on linking line) The attached patch does #1. Could you try it and see if it fixes it? This patch works, pgcrypto links and passes its installcheck test now. Kris Jurka Thanks. Here is the patch with a little comment. It should not break anything as it just disables a extra argument -assert pure-text to linker. Linking static libraries into shared one is bad idea, as the static parts wont be shared between processes, but erroring out is worse, especially if another compiler for a platform allows it. This makes gcc act same way as Sun's cc. -- marko Index: src/Makefile.shlib === RCS file: /opt/arc/cvs2/pgsql/src/Makefile.shlib,v retrieving revision 1.95 diff -u -c -r1.95 Makefile.shlib *** src/Makefile.shlib 13 Jul 2005 17:00:44 - 1.95 --- src/Makefile.shlib 16 Jul 2005 09:59:18 - *** *** 188,194 ifeq ($(PORTNAME), solaris) ifeq ($(GCC), yes) ! LINK.shared = $(CC) -shared else LINK.shared = $(CC) -G endif --- 188,196 ifeq ($(PORTNAME), solaris) ifeq ($(GCC), yes) ! # -mimpure-text disables passing '-assert pure-text' to linker, ! # to allow linking static library into shared one, like Sun's cc does. ! LINK.shared = $(CC) -shared -mimpure-text else LINK.shared = $(CC) -G endif ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[PATCHES] [patch 0/3] last large update to pgcrypto
Those 3 patches hopefully end my pgcrypto patch-bombing. (Well, at least I am offline next couple weeks...) 1. Small fixes 2. Fortuna fixes 3. New README There are 3 important points about new readme: * It is formatted for use by asciidoc. To be able to generate html from it. As I consider the text version to be main target, I tried to as few formatting cruft as possible. Here are both versions online: http://grue.l-t.ee/~marko/src/pgcrypto/README.html http://grue.l-t.ee/~marko/src/pgcrypto/README.txt * I've seen people suggest to use MD5 for passwords. Now there is special section which compares crypt algorithms to md5 and sha1. * I announce the removal of (digest/cipher/hmac)_exists functions in 8.2. Reason for it is that they are useless: - The informational question can be answered by calling the main function and looking for error. - They are useless for user programs, as you cannot replace one algorithm with another on the run - missing algorithm will be always hard error. - The question 'does it exists' even should not be asked, user should rather use algorithms that are _always_ there. The situation would be different for functions that would return a _list_ of supported algorithms. But I doubt even the usefulness of such functions. -- marko ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] [patch 1/3] small cleanups
- Fix couple comments.
- internal.c didnt clean hash contexts when freeing
- zero the system randomness buffer too
- Google tells that we can use /dev/urandom on Cygwin,
HPUX and AIX. Add them to random.c
- remove a debug reference from pgp.h
Index: pgsql/contrib/pgcrypto/fortuna.c
===
*** pgsql.orig/contrib/pgcrypto/fortuna.c
--- pgsql/contrib/pgcrypto/fortuna.c
*** static void init_state(FState *st)
*** 174,181
}
/*
! * Must not reseed more ofter than RESEED_PER_SEC
! * times per second.
*/
static int too_often(FState *st)
{
--- 174,181
}
/*
! * The time between reseed must be at least RESEED_INTERVAL
! * microseconds.
*/
static int too_often(FState *st)
{
*** static void reseed(FState *st)
*** 241,247
memset(key_md, 0, sizeof(key_md));
memset(buf, 0, BLOCK);
- n = k = 0;
}
/*
--- 241,246
Index: pgsql/contrib/pgcrypto/internal.c
===
*** pgsql.orig/contrib/pgcrypto/internal.c
--- pgsql/contrib/pgcrypto/internal.c
*** int_md5_free(PX_MD * h)
*** 127,132
--- 127,133
{
MD5_CTX*ctx = (MD5_CTX *) h-p.ptr;
+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*** int_sha1_free(PX_MD * h)
*** 174,179
--- 175,181
{
SHA1_CTX *ctx = (SHA1_CTX *) h-p.ptr;
+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*** int_sha256_free(PX_MD * h)
*** 221,226
--- 223,229
{
SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*** int_sha384_free(PX_MD * h)
*** 267,272
--- 270,276
{
SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*** int_sha512_free(PX_MD * h)
*** 314,319
--- 318,324
{
SHA512_CTX *ctx = (SHA512_CTX *) h-p.ptr;
+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*** init_md5(PX_MD * md)
*** 326,331
--- 331,337
MD5_CTX*ctx;
ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));
md-p.ptr = ctx;
*** init_sha1(PX_MD * md)
*** 345,350
--- 351,357
SHA1_CTX *ctx;
ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));
md-p.ptr = ctx;
*** init_sha256(PX_MD * md)
*** 364,369
--- 371,377
SHA256_CTX *ctx;
ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));
md-p.ptr = ctx;
*** init_sha384(PX_MD * md)
*** 383,388
--- 391,397
SHA384_CTX *ctx;
ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));
md-p.ptr = ctx;
*** init_sha512(PX_MD * md)
*** 402,407
--- 411,417
SHA512_CTX *ctx;
ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));
md-p.ptr = ctx;
*** static void system_reseed(void)
*** 829,834
--- 839,845
fortuna_add_entropy(SYSTEM_ENTROPY, buf, n);
seed_time = t;
+ memset(buf, 0, sizeof(buf));
}
int
Index: pgsql/contrib/pgcrypto/pgp-compress.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-compress.c
--- pgsql/contrib/pgcrypto/pgp-compress.c
*** restart:
*** 270,276
dec-stream.avail_out = dec-buf_len;
dec-pos = dec-buf;
! // Z_NO_FLUSH, Z_SYNC_FLUSH,
flush = dec-stream.avail_in ? Z_SYNC_FLUSH : Z_FINISH;
res = inflate(dec-stream, flush);
if (res != Z_OK res != Z_STREAM_END)
--- 270,280
dec-stream.avail_out = dec-buf_len;
dec-pos = dec-buf;
! /*
!* Z_SYNC_FLUSH is tell zlib to output as much as possible.
!* It should do in anyway (Z_NO_FLUSH), but seems to reserve
!* the right not to. So lets follow the API.
!*/
flush = dec-stream.avail_in ? Z_SYNC_FLUSH : Z_FINISH;
res = inflate(dec-stream, flush);
if (res != Z_OK res != Z_STREAM_END)
Index: pgsql/contrib/pgcrypto/pgp-decrypt.c
===
*** pgsql.orig/contrib/pgcrypto/pgp-decrypt.c
--- pgsql/contrib/pgcrypto/pgp-decrypt.c
*** static void mdc_free(void *priv)
*** 339,345
ctx-mdc_ctx = NULL;
}
- // fixme: clarify
static int mdc_finish(PGP_Context *ctx, PullFilter *src,
int len, uint8 **data_p)
{
--- 339,344
*** static int mdc_finish(PGP_Context *ctx,
*** 364,369
[PATCHES] [patch 3/3] new documentation
- reformat in asciidoc syntax
- clarify various parts
- add high-level description of PGP structure
- add a comparison of crypt vs. regular hashes
Index: pgsql/contrib/pgcrypto/README.pgcrypto
===
*** pgsql.orig/contrib/pgcrypto/README.pgcrypto
--- pgsql/contrib/pgcrypto/README.pgcrypto
***
*** 1,362
! pgcrypto 0.4 - cryptographic functions for PostgreSQL.
! ==
! by Marko Kreen marko@l-t.ee
! INSTALLATION
!
! Edit makefile, if you want to use any external library.
! NB! Default randomness source is libc random() function. This
! is so only to get pgcrypto build everywhere. Randomness is
! needed for gen_salt() and pgp_encrypt() functions. So if you plan
! using those, you should definitely change that by editing Makefile.
! You can should use urandom device if your OS supports it, otherwise
! link pgcrypto against OpenSSL library and use its PRNG.
! After editing Makefile:
- make
- make install
! To run regression tests, install both PostgreSQL and pgcrypto
! and then run
! make installcheck
! SQL FUNCTIONS
! =
! If any of arguments are NULL they return NULL.
! digest(data::bytea, type::text)::bytea
! Type is here the algorithm to use. E.g. 'md5', 'sha1', ...
! Returns binary hash.
! digest_exists(type::text)::bool
! Returns BOOL whether given hash exists.
! hmac(data::bytea, key::bytea, type::text)::bytea
- Calculates Hashed MAC over data. type is the same as
- in digest(). Returns binary hash. Similar to digest()
- but noone can alter data and re-calculate hash without
- knowing key. If the key is larger than hash blocksize
- it will first hashed and the hash will be used as key.
-
- [ HMAC is described in RFC2104. ]
! hmac_exists(type::text)::bool
! Returns BOOL. It is separate function because all hashes
! cannot be used in HMAC.
! crypt(password::text, salt::text)::text
- Calculates UN*X crypt(3) style hash. Useful for storing
- passwords. For generating salt you should use the
- gen_salt() function. Usage:
! New password:
!
! UPDATE .. SET pswhash = crypt(new_psw, gen_salt('md5'));
!
! Authentication:
! SELECT pswhash = crypt(given_psw, pswhash) WHERE .. ;
!
! returns BOOL whether the given_psw is correct. DES crypt
! has max key of 8 bytes, MD5 has max key at least 2^32-1
! bytes but may be larger on some platforms...
- Builtin crypt() supports DES, Extended DES, MD5 and Blowfish
- (variant 2a) algorithms.
! gen_salt(type::text)::text
! Generates a new random salt for usage in crypt(). Type
!
! 'des' - Old UNIX, not recommended
! 'md5' - md5-based crypt()
! 'xdes' - 'Extended DES'
! 'bf'- Blowfish-based, variant 2a
! When you use --enable-system-crypt then note that system
! libcrypt may not support them all.
! gen_salt(type::text, rounds::int4)::text
- same as above, but lets user specify iteration count
- for algorithm. Number is algorithm specific:
! typedefault min max
! -
! xdes725 1 16777215
! bf 6 4 31
! In case of xdes there is a additional limitation that the
! count must be a odd number.
! The higher the count, the more time it takes to calculate
! crypt and therefore the more time to break it. But beware!
! With too high count it takes a _very_long_ time to
! calculate it.
! For maximum security, you should choose the 'bf' crypt
! and use maximum number of rounds you can still tolerate.
! armor(bytea)::text
! dearmor(text)::bytea
! Those wrap/unwrap data into PGP Ascii Armor which
! is basically Base64 with CRC and additional formatting.
! pgp_sym_encrypt(data::text, key::text)::bytea
! pgp_sym_encrypt(data::text, key::text, arg::text)::bytea
! pgp_sym_encrypt_bytea(data::bytea, key::text)::bytea
! pgp_sym_encrypt_bytea(data::bytea, key::text, arg::text)::bytea
- pgp_sym_decrypt(data::bytea, key::text)::text
- pgp_sym_decrypt(data::bytea, key::text, arg::text)::text
- pgp_sym_decrypt_bytea(data::text, key::text)::bytea
- pgp_sym_decrypt_bytea(data::text, key::text, arg::text)::bytea
! Encrypt data into OpenPGP Symmetrically Encrypted Data
! message. And decrypt it from it.
! Note that the pgp_sym_encrypt_bytea functions tag the data
! as binary, as the pgp_sym_encrypt will tag the data as text.
! You can not decrypt the binary data as text. But you can
! decrypt text data as binary. This rule avoids having
! broken textual data in PostgreSQL.
! Both encrypt
[PATCHES] [patch 2/3] Fortuna fixes
After studying Fortuna more, I found out that I missed
some of the details.
- reseeding should happen only if pool #0 has aquired additional
entropy.
- a 'rekeying' operation should happend after each request and
also after 1M of extracted data. That means taking next two
blocks and using it as new key.
- Fortuna _really_ wants entropy sources to be somewhat unpredictible.
So roll dice when adding it and also add them to pools randomly,
not sequentially.
This hopefully makes harder for someone to doctor with the
internal state (as in our case user can directly control
what goes into it).
That also drops the idea of several sources - which really
fits more to hardware backed event sources.
- add a really obvious obfuscation: take the absolutely first
block be initial counter value. If Fortuna (AES to be exact)
is secure with known counter value, then it should be also
secure with unknown counter value. This does not go against
the important property of counter - that the bit-pattern repeat
period should be as long as possible.
- S2K functions should use px_get_pseudo_random_bytes not
px_get_random_bytes.
Index: pgsql/contrib/pgcrypto/fortuna.c
===
*** pgsql.orig/contrib/pgcrypto/fortuna.c
--- pgsql/contrib/pgcrypto/fortuna.c
***
*** 94,107
/* for one big request, reseed after this many bytes */
#define RESEED_BYTES (1024*1024)
/*
* Algorithm constants
*/
- /* max sources */
- #define MAX_SOURCES 8
-
/* Both cipher key size and hash result size */
#define BLOCK 32
--- 94,109
/* for one big request, reseed after this many bytes */
#define RESEED_BYTES (1024*1024)
+ /*
+ * Skip reseed if pool 0 has less than this many
+ * bytes added since last reseed.
+ */
+ #define POOL0_FILL(256/8)
/*
* Algorithm constants
*/
/* Both cipher key size and hash result size */
#define BLOCK 32
*** struct fortuna_state {
*** 118,126
uint8 key[BLOCK];
MD_CTX pool[NUM_POOLS];
CIPH_CTXciph;
- unsignedsource_pos[MAX_SOURCES];
unsignedreseed_count;
struct timeval last_reseed_time;
};
typedef struct fortuna_state FState;
--- 120,130
uint8 key[BLOCK];
MD_CTX pool[NUM_POOLS];
CIPH_CTXciph;
unsignedreseed_count;
struct timeval last_reseed_time;
+ unsignedpool0_bytes;
+ unsignedrnd_pos;
+ int counter_init;
};
typedef struct fortuna_state FState;
*** static void md_result(MD_CTX *ctx, uint8
*** 161,167
memset(tmp, 0, sizeof(tmp));
}
-
/*
* initialize state
*/
--- 165,170
*** static void init_state(FState *st)
*** 174,179
--- 177,208
}
/*
+ * Endianess does not matter.
+ * It just needs to change without repeating.
+ */
+ static void inc_counter(FState *st)
+ {
+ uint32 *val = (uint32*)st-counter;
+ if (++val[0])
+ return;
+ if (++val[1])
+ return;
+ if (++val[2])
+ return;
+ ++val[3];
+ }
+
+ /*
+ * This is called 'cipher in counter mode'.
+ */
+ static void encrypt_counter(FState *st, uint8 *dst)
+ {
+ ciph_encrypt(st-ciph, st-counter, dst);
+ inc_counter(st);
+ }
+
+
+ /*
* The time between reseed must be at least RESEED_INTERVAL
* microseconds.
*/
*** static void reseed(FState *st)
*** 207,215
MD_CTX key_md;
uint8 buf[BLOCK];
! /* check frequency */
! if (too_often(st))
! return;
/*
* Both #0 and #1 reseed would use only pool 0.
--- 236,243
MD_CTX key_md;
uint8 buf[BLOCK];
! /* set pool as empty */
! st-pool0_bytes = 0;
/*
* Both #0 and #1 reseed would use only pool 0.
*** static void reseed(FState *st)
*** 244,292
}
/*
* update pools
*/
! static void add_entropy(FState *st, unsigned src_id, const uint8 *data,
unsigned len)
{
unsigned pos;
uint8 hash[BLOCK];
MD_CTX md;
- /* just in case there's a bug somewhere */
- if (src_id = MAX_SOURCES)
- src_id = USER_ENTROPY;
-
/* hash given data */
md_init(md);
md_update(md, data, len);
md_result(md, hash);
! /* update pools round-robin manner */
! pos = st-source_pos[src_id];
md_update( st-pool[pos], hash, BLOCK);
! if (++pos = NUM_POOLS)
! pos = 0;
! st-source_pos[src_id] = pos;
memset(hash, 0, BLOCK);
[PATCHES] pgcrypto BYTE_ORDER fix for stable branches
Recently was uncovered that pgcrypto does not include
right header file to get BYTE_ORDER define on Cygwin
and MINGW, and the missing define does not result in
build failures, but random combinations of little and
big-endian code sections.
This patch adds missing sys/param.h include for md5.c,
sha1.c and rijndael.c plus also check to catch undefined
BYTE_ORDER.
The patch applies cleanly and is tested on branches
for versions 7.2, 7.3, 7.4 and 8.0. There are also
no failures on buildfarm for HEAD, where this patch
is already included.
--
marko
Index: contrib/pgcrypto/md5.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/md5.c,v
retrieving revision 1.9
diff -u -c -r1.9 md5.c
*** contrib/pgcrypto/md5.c 29 Nov 2001 19:40:37 - 1.9
--- contrib/pgcrypto/md5.c 13 Jul 2005 14:14:07 -
***
*** 31,40
*/
#include postgres.h
- #include px.h
#include md5.h
#define SHIFT(X, s) (((X) (s)) | ((X) (32 - (s
#define F(X, Y, Z) (((X) (Y)) | ((~X) (Z)))
--- 31,47
*/
#include postgres.h
+ #include sys/param.h
+
+ #include px.h
#include md5.h
+ /* sanity check */
+ #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN BYTE_ORDER !=
BIG_ENDIAN)
+ #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
+ #endif
+
#define SHIFT(X, s) (((X) (s)) | ((X) (32 - (s
#define F(X, Y, Z) (((X) (Y)) | ((~X) (Z)))
Index: contrib/pgcrypto/rijndael.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/rijndael.c,v
retrieving revision 1.9
diff -u -c -r1.9 rijndael.c
*** contrib/pgcrypto/rijndael.c 5 Sep 2002 21:08:26 - 1.9
--- contrib/pgcrypto/rijndael.c 13 Jul 2005 14:15:43 -
***
*** 38,48
*/
! #include postgres.h
#include px.h
#include rijndael.h
#define PRE_CALC_TABLES
#define LARGE_TABLES
--- 38,56
*/
! #include postgres.h
!
! #include sys/param.h
!
#include px.h
#include rijndael.h
+ /* sanity check */
+ #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN BYTE_ORDER !=
BIG_ENDIAN)
+ #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
+ #endif
+
#define PRE_CALC_TABLES
#define LARGE_TABLES
Index: contrib/pgcrypto/sha1.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sha1.c,v
retrieving revision 1.11
diff -u -c -r1.11 sha1.c
*** contrib/pgcrypto/sha1.c 29 Nov 2001 19:40:37 - 1.11
--- contrib/pgcrypto/sha1.c 13 Jul 2005 14:15:05 -
***
*** 36,53
*/
#include postgres.h
#include px.h
#include sha1.h
/* sanity check */
! #if BYTE_ORDER != BIG_ENDIAN
! #if BYTE_ORDER != LITTLE_ENDIAN
! #define unsupported 1
#endif
- #endif
-
- #ifndef unsupported
/* constant table */
static uint32 _K[] = {0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6};
--- 36,52
*/
#include postgres.h
+
+ #include sys/param.h
+
#include px.h
#include sha1.h
/* sanity check */
! #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN BYTE_ORDER !=
BIG_ENDIAN)
! #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
#endif
/* constant table */
static uint32 _K[] = {0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6};
***
*** 347,350
#endif
}
- #endif /* unsupported */
--- 346,348
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [PATCHES] pgp encrypt v4
On Sat, Jul 09, 2005 at 11:58:14PM -0400, Bruce Momjian wrote: Patch applied. Thanks. Great! Small problem with the 3 patches: you forgot to 'cvs add' new files? -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[PATCHES] [patch 2/2] Fortuna PRNG
- Add Fortuna PRNG to pgcrypto.
- Move openssl random provider to openssl.c and builtin provider
to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
of Fortuna. There is ATM 2 functions for Windows,
reader from /dev/urandom and the regular time()/getpid() silliness.
Index: pgsql/contrib/pgcrypto/fortuna.c
===
*** /dev/null
--- pgsql/contrib/pgcrypto/fortuna.c
***
*** 0
--- 1,365
+ /*
+ * fortuna.c
+ *Fortuna-like PRNG.
+ *
+ * Copyright (c) 2005 Marko Kreen
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $PostgreSQL$
+ */
+
+ #include postgres.h
+ #include sys/time.h
+ #include time.h
+
+ #include rijndael.h
+ #include sha2.h
+
+ #include fortuna.h
+
+
+ /*
+ * Why Fortuna-like: There does not seem to be any definitive reference
+ * on Fortuna in the net. Instead this implementation is based on
+ * following references:
+ *
+ * http://en.wikipedia.org/wiki/Fortuna_(PRNG)
+ * - Wikipedia article
+ * http://jlcooke.ca/random/
+ * - Jean-Luc Cooke Fortuna-based /dev/random driver for Linux.
+ */
+
+ /*
+ * There is some confusion about whether and how to carry forward
+ * the state of the pools. Seems like original Fortuna does not
+ * do it, resetting hash after each request. I guess expecting
+ * feeding to happen more often that requesting. This is absolutely
+ * unsuitable for pgcrypto, as nothing asynchronous happens here.
+ *
+ * J.L. Cooke fixed this by feeding previous hash to new re-initialized
+ * hash context.
+ *
+ * Fortuna predecessor Yarrow requires ability to query intermediate
+ * 'final result' from hash, without affecting it.
+ *
+ * This implementation uses the Yarrow method - asking intermediate
+ * results, but continuing with old state.
+ */
+
+
+ /*
+ * Algorithm parameters
+ */
+
+ /*
+ * How many pools.
+ *
+ * Original Fortuna uses 32 pools, that means 32'th pool is
+ * used not earlier than in 13th year. This is a waste in
+ * pgcrypto, as we have very low-frequancy seeding. Here
+ * is preferable to have all entropy usable in reasonable time.
+ *
+ * With 23 pools, 23th pool is used after 9 days which seems
+ * more sane.
+ *
+ * In our case the minimal cycle time would be bit longer
+ * than the system-randomness feeding frequency.
+ */
+ #define NUM_POOLS 23
+
+ /* in microseconds */
+ #define RESEED_INTERVAL 10 /* 0.1 sec */
+
+ /* for one big request, reseed after this many bytes */
+ #define RESEED_BYTES (1024*1024)
+
+
+ /*
+ * Algorithm constants
+ */
+
+ /* max sources */
+ #define MAX_SOURCES 8
+
+ /* Both cipher key size and hash result size */
+ #define BLOCK 32
+
+ /* cipher block size */
+ #define CIPH_BLOCK16
+
+ /* for internal wrappers */
+ #define MD_CTXSHA256_CTX
+ #define CIPH_CTX rijndael_ctx
+
+ struct fortuna_state {
+ uint8 counter[CIPH_BLOCK];
+ uint8 result[CIPH_BLOCK];
+ uint8 key[BLOCK];
+ MD_CTX pool[NUM_POOLS];
+ CIPH_CTXciph;
+ unsignedsource_pos[MAX_SOURCES];
+ unsignedreseed_count;
+ struct timeval last_reseed_time;
+ };
+ typedef struct fortuna_state FState;
+
+
+ /*
+ * Use our own wrappers here.
+ * - Need to get intermediate result from digest, without affecting it.
+ * - Need re-set key on a cipher context
[PATCHES] [patch 0/2] Add Fortuna PRNG to pgcrypto
Following two patches add Fortuna PRNG to pgcrypto. I think this is needed to really complete the 'no hand-config' direction of pgcrypto. Patch #1 adds implementation of SHA2 hashes (SHA256/384/512). They are required for Fortuna. As they are replacements for SHA1 (which was replacement for MD5), they would be good to have anyway. Patch #2 adds implementation of Fortuna, plus reorganizes random functions a bit. random.c is made 'system randomness' provider, for initial seeding of PRNG. There I put 2 randomness sources for Windows, /dev/urandom reader, and a fallback provider, that does the getpid/gettimeofday stuff. The idea is to initially seed Fortuna with randomness from system and later feed SHA1 of user data into it too. Just to keep it from degenerating into pure PRNG. I looked at various PRNGs, and Fortuna seemed to best fit pgcrypto situation. Fortuna predecessor, from same authors, Yarrow - does not fit here as it needs precise accounting of entropy. PRNGs based on stream-ciphers also do not fit as they cannot handle feeding of dubious quality entropy. (And using just a PRNG without entropy feeding is not secure enough for session keys.) Fortuna does not need entropy accounting, in exchange it wastes entropy - keeps some of away from current output. -- I know, this is past the feature freeze, but in a way its not a new feature, rather is enables a already accepted (?) feature on a rather common configuration (no-OpenSSL). In particular, I am thinking about win32 port. Also, the goal for the PGP work was to replace the current encrypt() code, thus is needs also work everywhere encrypt() works. Without strong PRNG included, this goal is not filled. -- marko PS. If the inline patches bother, I can send them attached. Just that I use 'quilt' for preparing them, and it does not have option to send them as attachments. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[PATCHES] [patch 1/2] Add implementation of SHA256/384/512
This patch adds implementation of SHA2 to pgcrypto.
New hashes: SHA256, SHA384, SHA512.
Code from OpenBSD.
Index: pgsql/contrib/pgcrypto/Makefile
===
*** pgsql.orig/contrib/pgcrypto/Makefile
--- pgsql/contrib/pgcrypto/Makefile
***
*** 6,19
INT_CFLAGS = -DRAND_SILLY
#INT_CFLAGS = -DRAND_DEV=\/dev/urandom\
! INT_SRCS = md5.c sha1.c internal.c blf.c rijndael.c
OSSL_CFLAGS = -DRAND_OPENSSL
OSSL_SRCS = openssl.c
OSSL_TESTS = des 3des cast5
CF_SRCS = $(if $(subst no,,$(with_openssl)), $(OSSL_SRCS), $(INT_SRCS))
! CF_TESTS = $(if $(subst no,,$(with_openssl)), $(OSSL_TESTS))
CF_CFLAGS = $(if $(subst no,,$(with_openssl)), $(OSSL_CFLAGS), $(INT_CFLAGS))
PG_CPPFLAGS = $(CF_CFLAGS)
--- 6,20
INT_CFLAGS = -DRAND_SILLY
#INT_CFLAGS = -DRAND_DEV=\/dev/urandom\
! INT_SRCS = md5.c sha1.c sha2.c internal.c blf.c rijndael.c
! INT_TESTS = sha2
OSSL_CFLAGS = -DRAND_OPENSSL
OSSL_SRCS = openssl.c
OSSL_TESTS = des 3des cast5
CF_SRCS = $(if $(subst no,,$(with_openssl)), $(OSSL_SRCS), $(INT_SRCS))
! CF_TESTS = $(if $(subst no,,$(with_openssl)), $(OSSL_TESTS), $(INT_TESTS))
CF_CFLAGS = $(if $(subst no,,$(with_openssl)), $(OSSL_CFLAGS), $(INT_CFLAGS))
PG_CPPFLAGS = $(CF_CFLAGS)
Index: pgsql/contrib/pgcrypto/internal.c
===
*** pgsql.orig/contrib/pgcrypto/internal.c
--- pgsql/contrib/pgcrypto/internal.c
***
*** 36,41
--- 36,42
#include md5.h
#include sha1.h
+ #include sha2.h
#include blf.h
#include rijndael.h
***
*** 56,61
--- 57,65
static void init_md5(PX_MD * h);
static void init_sha1(PX_MD * h);
+ static void init_sha256(PX_MD * h);
+ static void init_sha384(PX_MD * h);
+ static void init_sha512(PX_MD * h);
struct int_digest
{
*** static const struct int_digest
*** 67,72
--- 71,79
int_digest_list[] = {
{ md5, init_md5 },
{ sha1, init_sha1 },
+ { sha256, init_sha256 },
+ { sha384, init_sha384 },
+ { sha512, init_sha512 },
{ NULL, NULL }
};
*** int_sha1_free(PX_MD * h)
*** 164,169
--- 171,316
px_free(h);
}
+ /* SHA256 */
+
+ static unsigned
+ int_sha256_len(PX_MD * h)
+ {
+ return SHA256_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha256_block_len(PX_MD * h)
+ {
+ return SHA256_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha256_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha256_reset(PX_MD * h)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Init(ctx);
+ }
+
+ static void
+ int_sha256_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ SHA256_Final(dst, ctx);
+ }
+
+ static void
+ int_sha256_free(PX_MD * h)
+ {
+ SHA256_CTX *ctx = (SHA256_CTX *) h-p.ptr;
+
+ px_free(ctx);
+ px_free(h);
+ }
+ /* SHA384 */
+
+ static unsigned
+ int_sha384_len(PX_MD * h)
+ {
+ return SHA384_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha384_block_len(PX_MD * h)
+ {
+ return SHA384_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha384_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha384_reset(PX_MD * h)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Init(ctx);
+ }
+
+ static void
+ int_sha384_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ SHA384_Final(dst, ctx);
+ }
+
+ static void
+ int_sha384_free(PX_MD * h)
+ {
+ SHA384_CTX *ctx = (SHA384_CTX *) h-p.ptr;
+
+ px_free(ctx);
+ px_free(h);
+ }
+
+ /* SHA512 */
+
+ static unsigned
+ int_sha512_len(PX_MD * h)
+ {
+ return SHA512_DIGEST_LENGTH;
+ }
+
+ static unsigned
+ int_sha512_block_len(PX_MD * h)
+ {
+ return SHA512_BLOCK_LENGTH;
+ }
+
+ static void
+ int_sha512_update(PX_MD * h, const uint8 *data, unsigned dlen)
+ {
+ SHA512_CTX *ctx = (SHA512_CTX *) h-p.ptr;
+
+ SHA512_Update(ctx, data, dlen);
+ }
+
+ static void
+ int_sha512_reset(PX_MD * h)
+ {
+ SHA512_CTX *ctx = (SHA512_CTX *) h-p.ptr;
+
+ SHA512_Init(ctx);
+ }
+
+ static void
+ int_sha512_finish(PX_MD * h, uint8 *dst)
+ {
+ SHA512_CTX *ctx = (SHA512_CTX *) h-p.ptr;
+
+ SHA512_Final(dst, ctx);
+ }
+
+ static void
+ int_sha512_free(PX_MD * h)
+ {
+ SHA512_CTX *ctx = (SHA512_CTX *) h-p.ptr;
+
+ px_free(ctx);
+ px_free(h);
+ }
+
/* init functions */
static void
*** init_sha1(PX_MD * md)
*** 204,209
--- 351,413
md-reset(md);
}
+ static void
+ init_sha256(PX_MD * md)
+ {
+
Re: [PATCHES] [patch 0/2] Add Fortuna PRNG to pgcrypto
On Fri, Jul 08, 2005 at 02:10:47PM -0500, Bruno Wolff III wrote: On Fri, Jul 08, 2005 at 20:54:40 +0300, Marko Kreen marko@l-t.ee wrote: The idea is to initially seed Fortuna with randomness from system and later feed SHA1 of user data into it too. Just to keep it from degenerating into pure PRNG. How is fortuna getting entropy? - From /dev/urandom or /dev/random on Linux, *BSD, Darwin and Solaris. - On Windows it uses CryptGenRandom function from Microsoft CryptoAPI and additionally a high-resolution timer output. (I must admit I haven't even compile tested them, I hope that win32 guys will look into it.) - On other OS'es, or in case reading /dev/random fails, the regular silly random functions: getpid, gettimeofday, etc. The system randomness is aquired on first call, and then after every 3h or so. In addition SHA1 of user data is feeded into it just to keep things moving. Actually, this is the preferred usage for /dev/random - to seed a strong PRNG, then use that. Unless you are generating secret keys, that is. But we aren't. Basically, the hardware entropy the kernel has aquired is precious, and the user should not waste it. This leaves OS'es that don't have /dev/random and aren't Windows out cold, but I really can't do anything for them. With the mixing of user data, the random output should not immidiately fail for them too. Ofcourse, if anyone knows any system-specific entropy gathering tricks, please tell. Wouldn't this be better placed in the OS as the source of /dev/urandom rather than as part of Postgres? Well, those OS'es that already have urandom/random, don't need it. And those that don't - I really don't feel responsibility to write one... -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [patch 0/2] Add Fortuna PRNG to pgcrypto
On Fri, Jul 08, 2005 at 03:12:59PM -0500, Bruno Wolff III wrote: On Fri, Jul 08, 2005 at 23:03:49 +0300, Marko Kreen marko@l-t.ee wrote: Well, those OS'es that already have urandom/random, don't need it. And those that don't - I really don't feel responsibility to write one... But fortuna is essentially a high quality /dev/urandom. It doesn't make seem to much sense to implement it on top on /dev/urandom. Ah, the chain /dev/urandom - Fortuna - result irritates you? The main reason for Fortuna is to cover OS'es that don't have /dev/random. Currently only Win32 is adequately covered as it provides it's own hardware random for seeding. On others the output is theoretically weak, but the silly randomness plus hashing user data (which is supposedly secret) and key (ditto) should provide 'Good Enough for 99% cases' randomness. Now - for /dev/random OS'es: ofcourse, if the /dev/urandom works, I could skip all of the Fortuna and use that directly, but I don't see the need for that. I even see disadvantage in doing that. First, I prefer /dev/urandom to /dev/random (in Linux terms). The /dev/random advantade is too theoretical for me - not worth the practical disadvantage. But following applies to it too. While there is hardware-backed entropy available the urandom gives true randomness - no way to predict it. If the entropy is drained, it degenerates into PRNG (although very good one). The danger is, that if it runs as PRNG, some resourceful 'interested party' can read some of the output and then predict what the next user will get or what some other user got before. So it is pretty important not to drain all entropy from it. /dev/random avoids this with blocking, but IMHO urandom has state large enough to allow running as a PRNG some time. Now consider following scenario: you have 30 clients connected to the server, each encryption something occasionally. The current reseed delay is 3h - the result is that on average, after every 6 seconds someone asks 256 bits from /dev/urandom. This pretty much guarantees that entropy will be drained - especially on an unattanded server. The situation would be much worse if they all read directly from /dev/urandom. And here's the key difference - if they read directly from /dev/urandom they all have PRNG with same state. If someone observes enough of the output from there to predict what others get, he can. But when using Fortuna inside pgcrypto: - Each user (connection) will have different state. - There is bigger chance that seeding from /dev/urandom has true entropy in it. So even if someone predicts one state, it has no effect on other users. Ok, this is not exatly true when using pooled connections - attacker could predict what the previous user got or what next one will, but the fact that other states are not compromised still stays. OK, all this is rather theorethical. Practically speaking, if we need to have Fortuna anyway, we may as well be better OS citizens by using it on /dev/urandom too. -- marko ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] pgcrypto: OpenSSL/DES cleanup
As Kris Jurka found out, pgcrypto does not work with
OpenSSL 0.9.6x. The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.
I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.
Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL 0.9.7.
I chose this route because:
- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0
Tested with OpenSSL 0.9.6c and 0.9.7e.
--
marko
PS. It's nice to see that the 'autoconfiguration' already pays
back.
Index: contrib/pgcrypto/openssl.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.20
diff -u -c -r1.20 openssl.c
*** contrib/pgcrypto/openssl.c 5 Jul 2005 18:15:36 - 1.20
--- contrib/pgcrypto/openssl.c 7 Jul 2005 09:18:37 -
***
*** 48,53
--- 48,73
#endif
/*
+ * Compatibility with older OpenSSL API for DES.
+ */
+ #if OPENSSL_VERSION_NUMBER 0x00907000L
+ #define DES_key_schedule des_key_schedule
+ #define DES_cblock des_cblock
+ #define DES_set_key(k, ks) \
+ des_set_key((k), *(ks))
+ #define DES_ecb_encrypt(i, o, k, e) \
+ des_ecb_encrypt((i), (o), *(k), (e))
+ #define DES_ncbc_encrypt(i, o, l, k, iv, e) \
+ des_ncbc_encrypt((i), (o), (l), *(k), (iv), (e))
+ #define DES_ecb3_encrypt(i, o, k1, k2, k3, e) \
+ des_ecb3_encrypt((des_cblock *)(i), (des_cblock *)(o), \
+ *(k1), *(k2), *(k3), (e))
+ #define DES_ede3_cbc_encrypt(i, o, l, k1, k2, k3, iv, e) \
+ des_ede3_cbc_encrypt((i), (o), \
+ (l), *(k1), *(k2), *(k3), (iv), (e))
+ #endif
+
+ /*
* Hashes
*/
static unsigned
***
*** 175,185
} bf;
struct
{
! des_key_schedule key_schedule;
} des;
struct
{
! des_key_schedule k1, k2, k3;
} des3;
CAST_KEYcast_key;
#ifdef GOT_AES
--- 195,205
} bf;
struct
{
! DES_key_schedule key_schedule;
} des;
struct
{
! DES_key_schedule k1, k2, k3;
} des3;
CAST_KEYcast_key;
#ifdef GOT_AES
***
*** 315,325
ossl_des_init(PX_Cipher * c, const uint8 *key, unsigned klen, const uint8 *iv)
{
ossldata *od = c-ptr;
! des_cblock xkey;
memset(xkey, 0, sizeof(xkey));
memcpy(xkey, key, klen 8 ? 8 : klen);
! des_set_key(xkey, od-u.des.key_schedule);
memset(xkey, 0, sizeof(xkey));
if (iv)
--- 335,345
ossl_des_init(PX_Cipher * c, const uint8 *key, unsigned klen, const uint8 *iv)
{
ossldata *od = c-ptr;
! DES_cblock xkey;
memset(xkey, 0, sizeof(xkey));
memcpy(xkey, key, klen 8 ? 8 : klen);
! DES_set_key(xkey, od-u.des.key_schedule);
memset(xkey, 0, sizeof(xkey));
if (iv)
***
*** 338,346
ossldata *od = c-ptr;
for (i = 0; i dlen / bs; i++)
! des_ecb_encrypt((des_cblock *) (data + i * bs),
! (des_cblock *) (res + i * bs),
! od-u.des.key_schedule, 1);
return 0;
}
--- 358,366
ossldata *od = c-ptr;
for (i = 0; i dlen / bs; i++)
! DES_ecb_encrypt((DES_cblock *) (data + i * bs),
! (DES_cblock *) (res + i * bs),
! od-u.des.key_schedule, 1);
return 0;
}
***
*** 353,361
ossldata *od = c-ptr;
for (i = 0; i dlen / bs; i++)
! des_ecb_encrypt((des_cblock *) (data + i * bs),
! (des_cblock *) (res + i * bs),
! od-u.des.key_schedule, 0);
return 0;
}
--- 373,381
ossldata *od = c-ptr;
for (i = 0; i dlen / bs; i++)
! DES_ecb_encrypt((DES_cblock *) (data + i * bs),
! (DES_cblock *) (res + i * bs),
! od-u.des.key_schedule, 0);
return 0;
}
***
***
Re: [HACKERS] [PATCHES] [PATCH] pgcrypto: pgp_encrypt v3
On Tue, Jul 05, 2005 at 10:20:17AM +1000, Neil Conway wrote: Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list That is not the latest version of Marko's patch. Bruce got v3, thats indeed the latest. Also, http://momjian.postgresql.org/cgi-bin/pgpatches shows v3. But in any case, the patch is not yet ready for application: http://archives.postgresql.org/pgsql-patches/2005-07/msg00077.php Now I did fresh rebuild of CVS and played with it. Result is that it is only partly my error. It just happens that I did initdb with option '-E unicode'. Now, can anybody explain the following difference: $ psql -c '\l' template1; psql -c select 'a\nxx'::text as x; template1 List of databases Name| Owner | Encoding +---+--- contrib_regression | marko | SQL_ASCII postgres | marko | SQL_ASCII template0 | marko | SQL_ASCII template1 | marko | SQL_ASCII (4 rows) x -- a xx (1 row) $ psql -c '\l' template1; psql -c select 'a\nxx'::text as x; template1 List of databases Name| Owner | Encoding +---+-- contrib_regression | marko | UTF8 postgres | marko | UTF8 template0 | marko | UTF8 template1 | marko | UTF8 (4 rows) x --- a xx (1 row) = I can send new regression test for SQL_ASCII, but it still would not work for all users. -- marko ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] contrib/pgcrypto patch for OpenSSL 0.9.8
On Mon, Jul 04, 2005 at 12:22:03PM +1000, Neil Conway wrote: Bruce Momjian wrote: Patch applied. Thanks. Should either or both of these patches be applied to back branches? That would be a good idea. -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [PATCH] pgcrypto: pgp_encrypt (v2)
On Mon, Jul 04, 2005 at 04:59:58PM +1000, Neil Conway wrote: Marko Kreen wrote: Please use following updated patch instead. It implements utf8 conversion, fixes couple of bugs and has many code and comment cleanups. The regression tests don't pass on my box. With the default Makefile, there are a lot of errors WRT no strong random source. After editing the Makefile to make use the random device, I get the attached regression.diffs. I don't understand the regression of the dashes in armor test, kinda seems you have different psql than in CVS. But for the regression with 'random = dev', I'd say you did not do 'make clean' after changing Makefile. It only changes flags to random.c, so if you have existing random.o, it wont be recompiled. While I understand the need to make sure people use a reasonably strong crypto source, it would be nice if the regression tests passed out of the box. Look for the 'pgp_encrypt v3' mail I sent. There I use special empty tests for disabled functionality, that succeed, but hopefully show user through naming that the functionality is missing. -- marko ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] contrib/pgcrypto patch for OpenSSL 0.9.8
On Mon, Jul 04, 2005 at 10:31:21AM -0400, Bruce Momjian wrote: OK, I have applied the last two diffs for pgcrypto includes to 8.0.X, attached. *** contrib/pgcrypto/openssl.c12 Mar 2005 06:54:31 - 1.13.4.1 --- contrib/pgcrypto/openssl.c4 Jul 2005 14:30:12 - *** *** 35,40 --- 35,54 #include openssl/evp.h + #include openssl/evp.h Why second time? -- marko ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] contrib/pgcrypto patch for OpenSSL 0.9.8
On Sat, Jul 02, 2005 at 10:33:05PM -0400, Bruce Momjian wrote: Patch applied. Thanks. --- Michael Fuhr wrote: This patch allows contrib/pgcrypto to build with OpenSSL 0.9.8 (currently in beta) when cryptolib = openssl. According to the following checkin message from several years ago, OpenSSL application developers should no longer rely on openssl/evp.h to include everything they need: http://cvs.openssl.org/chngview?cn=9888 This patch adds the necessary header files. It doesn't appear to break anything when building against OpenSSL 0.9.7. BTW, core appears to build and work fine with OpenSSL 0.9.8. I've built 7.3 through HEAD against 0.9.8-beta6 without noticing any problems. Thanks for the patch. Bruce, please apply this additional patch, that fixes the auto-detection of AES. Now openssl.c just checks OpenSSL version. Whoever compiles newer OpenSSL without AES is on his own. -- marko Index: contrib/pgcrypto/openssl.c === RCS file: /projects/cvsroot/pgsql/contrib/pgcrypto/openssl.c,v retrieving revision 1.18 diff -p -u -r1.18 openssl.c --- contrib/pgcrypto/openssl.c 3 Jul 2005 02:32:56 - 1.18 +++ contrib/pgcrypto/openssl.c 3 Jul 2005 05:17:29 - @@ -39,11 +39,12 @@ #include openssl/des.h /* - * Is OpenSSL compiled with AES? + * Does OpenSSL support AES? */ #undef GOT_AES -#ifdef AES_ENCRYPT +#if OPENSSL_VERSION_NUMBER = 0x00907000L #define GOT_AES +#include openssl/aes.h #endif /* ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] [PATCH] pgcrypto: pgp_encrypt (v2)
On Mon, Jun 27, 2005 at 09:08:05PM -0400, Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. Please use following updated patch instead. It implements utf8 conversion, fixes couple of bugs and has many code and comment cleanups. -- marko pgp-v2.diff.gz Description: Binary data ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [patch 0/6] pgcrypto update
On Fri, May 06, 2005 at 11:49:43AM -0400, Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. Neil applied all those some time ago, no? Yes. -- marko ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[PATCHES] entab vs. gcc4 again
There was a patch applied against entab/halt.c that converts
it away from KR varargs. But it's buggy, its makes halt()
use second argment as format string, not first.
Following patch fixes it, but also converts entab/entab.c away
from KR.
That is from my own patch against entab for the same
problem, which got lost somewhere in -patches.
--
marko
Index: src/tools/entab/entab.c
===
RCS file: /opt/arc/cvs2/pgsql/src/tools/entab/entab.c,v
retrieving revision 1.13
diff -u -c -r1.13 entab.c
*** src/tools/entab/entab.c 7 Oct 2003 17:40:09 - 1.13
--- src/tools/entab/entab.c 5 May 2005 12:36:27 -
***
*** 29,37
extern intoptind;
int
! main(argc, argv)
! int argc;
! char**argv;
{
int tab_size = 8,
min_spaces = 2,
--- 29,35
extern intoptind;
int
! main(int argc, char **argv)
{
int tab_size = 8,
min_spaces = 2,
Index: src/tools/entab/halt.c
===
RCS file: /opt/arc/cvs2/pgsql/src/tools/entab/halt.c,v
retrieving revision 1.6
diff -u -c -r1.6 halt.c
*** src/tools/entab/halt.c 15 Apr 2005 04:29:32 - 1.6
--- src/tools/entab/halt.c 5 May 2005 12:36:00 -
***
*** 19,33
/*VARARGS*/
void
! halt(const char *path, ...)
{
va_list arg_ptr;
! char *format,
! *pstr;
void(*sig_func) ();
! va_start(arg_ptr, path);
! format = va_arg(arg_ptr, char *);
if (strncmp(format, PERROR, 6) != 0)
vfprintf(stderr, format, arg_ptr);
else
--- 19,31
/*VARARGS*/
void
! halt(const char *format, ...)
{
va_list arg_ptr;
! const char *pstr;
void(*sig_func) ();
! va_start(arg_ptr, format);
if (strncmp(format, PERROR, 6) != 0)
vfprintf(stderr, format, arg_ptr);
else
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
