Re: [HACKERS] suspicious pointer/integer coersion
Andrew Dunstan <[EMAIL PROTECTED]> writes: >> Attached is a patch that fixes (I hope) both a recently introduced >> problem with recursion and a problem with array returns that became >> evident as a result of not throwing away non-fatal warnings (thanks to >> David Fetter for noticing this). Regression test updates to include >> both cases are included in the patch. Applied, thanks. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen wrote: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&dt=2005-07-11%2003:30:04 Linking problem with zlib on Solaris 9/x86. I am clueless about this. I can anyone look into it? It appears to be finding the static /usr/local/lib/libz.a instead of the dynamic /usr/lib/libz.so. Kris Jurka ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Vacuum summary?
Tom Lane wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: What do you guys think of having a vacuum summary command? E.g.; VACUUM [FULL] [ANALYZE] SUMMARY ... that will turn SUMMARY into a fully reserved word ... Hmmm... good point. Other options would be: brief short terse? nutshell ;) review report -- This one is interesting although sufffers from the same problem. Sincerely, Joshua D. Drake regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Possible better pg_ctl start/stop handling?
Tom Lane wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: Does the dev machine run more than one postmaster? No. Hmm, it should be pretty impossible to see this if the machine's just been rebooted It wasn't a reboot it was a total power loss and then startup. and there are no other postmasters running. If you can replicate it, could you send along the output of "ipcs -m -a" along with the contents of the postmaster.pid file? I will give it a shot a little later today. Also, what's the platform exactly? FC3. J regards, tom lane -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Possible better pg_ctl start/stop handling?
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: >> Does the dev machine run more than one postmaster? > No. Hmm, it should be pretty impossible to see this if the machine's just been rebooted and there are no other postmasters running. If you can replicate it, could you send along the output of "ipcs -m -a" along with the contents of the postmaster.pid file? Also, what's the platform exactly? regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Vacuum summary?
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > What do you guys think of having a vacuum summary command? E.g.; > VACUUM [FULL] [ANALYZE] SUMMARY ... that will turn SUMMARY into a fully reserved word ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Vacuum summary?
Hello, What do you guys think of having a vacuum summary command? E.g.; VACUUM [FULL] [ANALYZE] SUMMARY where summary would only print the last lines of a vacuum verbose? foo=> VACUUM SUMMARY; INFO: free space map contains 17 pages in 12 relations DETAIL: A total of 192 page slots are in use (including overhead). 192 page slots are required to track all free space. Current limits are: 2 page slots, 1000 relations, using 182 KB. VACUUM foo=> Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Possible better pg_ctl start/stop handling?
Power off on a dev machine ;) Does the dev machine run more than one postmaster? No. I've occasionally seen similar issues when restarting a clutch of dev postmasters --- the kernel may assign a shmem id to one of them that belonged to another one in the previous cycle, and if you already started that other one then the second gets confused. 8.0 and up have a test that should deal correctly with this; what version did you see failing exactly? This is on my personal dev machine and I am running 8.1Dev. Sincerely, Joshua D. Drake regards, tom lane -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Possible better pg_ctl start/stop handling?
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > I was just wondering if we could make pg_ctl a little smarter as all. > If pg_ctl can't start because the pid file exists, test for the > existence of the pid, if the pid does not exist test for the existence > of **any** postgres process (grep? egad...), if none exists overwrite > the pid file and start? This cannot be any smarter than the existing test in the postmaster, and is most likely to be much stupider. >> How did you get into this state anyway? > Power off on a dev machine ;) Does the dev machine run more than one postmaster? I've occasionally seen similar issues when restarting a clutch of dev postmasters --- the kernel may assign a shmem id to one of them that belonged to another one in the previous cycle, and if you already started that other one then the second gets confused. 8.0 and up have a test that should deal correctly with this; what version did you see failing exactly? regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Possible better pg_ctl start/stop handling?
Tom Lane wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: FATAL: pre-existing shared memory block (key 5432001, ID 19202077) is still in use HINT: If you're sure there are no old server processes still running, remove the shared memory block with the command "ipcclean", "ipcrm", or just delete the file "postmaster.pid". As we can see pg_ctl knows that the PID does not exist. If the PID does not exist is it safe to assume that we can remove the file? So that we may start again? The error message is warning you that there appear to still be live backends in the data directory, even though the original postmaster process is gone (crashed?). Yes I am aware of that. My actual point was that pg_ctl test to see if the process is alive when you issue the stop. It comes back with the error that the PID is no longer available to kill. I was just wondering if we could make pg_ctl a little smarter as all. If pg_ctl can't start because the pid file exists, test for the existence of the pid, if the pid does not exist test for the existence of **any** postgres process (grep? egad...), if none exists overwrite the pid file and start? If that is the case, forcibly starting a new postmaster is a fine recipe for creating unrecoverable data corruption. So having pg_ctl auto-remove the file is horribly dangerous and is NOT going to happen. Please understand my thought was not coming lightly. I recognize very well (as I have had to deal with customers who have done it) the dangers here. How did you get into this state anyway? Power off on a dev machine ;) Sincerely, Joshua D. Drake regards, tom lane -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Hide source code
> On Mon, Jul 11, 2005 at 07:59:44AM -0400, Merlin Moncure wrote: > Interesting. Again, this is pretty fragile with respect to, for > example, pg_dump(all). No, it isn't. If you revoke permission on pg_proc, /df in psql will fail, as will pg_dump. Try it (you have to revoke all on pg_proc form public). See the log below: C:\svn\rcsi\Etc>pg_dump -U test xxx pg_dump: SQL command failed pg_dump: Error message from server: ERROR: permission denied for relation pg_proc pg_dump: The command was: SELECT tableoid, oid, proname, prolang, pronargs, proargtypes, prorettyp proacl, pronamespace, (select usename from pg_user where proowner = usesysid) as usename FROM pg_ oc WHERE NOT proisagg AND pronamespace != (select oid from pg_namespace where nspname = 'pg_catalo ) > > Regardless of your philosophical standpoint, many larger > > organizations will feel uncomfortable with having unprivileged users > > having access to all the database procedure source code. > > That some "larger organizations" choose to use the known-unsafe method > of security by obscurity is not a reason for anybody here to expend > any effort helping them persist in this illusion: quite the opposite, > in fact. "Larger organizations" are likely to have security needs > which they actually need to address, not to pretend they've addressed > while actually making things easy for attackers. I think you are confusing 'security by obscurity' with 'security by policy'. The reasons for not allowing non-privileged users within your company access to your source codes are highly nuanced. Generally, though, on the systems I admin information is doled out on a 'need to know' basis. The data in the database is managed in a highly controlled fashion, why not the code? Non technical users have no business looking at the code I write because their bosses (not me) prefer it that way. End of story. observe: C:\svn\rcsi\Etc>pg_dump -U test xxx pg_dump: SQL command failed pg_dump: Error message from server: ERROR: permission denied for relation pg_proc pg_dump: The command was: SELECT tableoid, oid, proname, prolang, pronargs, proargtypes, prorettyp proacl, pronamespace, (select usename from pg_user where proowner = usesysid) as usename FROM pg_ oc WHERE NOT proisagg AND pronamespace != (select oid from pg_namespace where nspname = 'pg_catalo ) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Possible better pg_ctl start/stop handling?
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > FATAL: pre-existing shared memory block (key 5432001, ID 19202077) is > still in use > HINT: If you're sure there are no old server processes still running, > remove the shared memory block with the command "ipcclean", "ipcrm", or > just delete the file "postmaster.pid". > As we can see pg_ctl knows that the PID does not exist. If the PID does > not exist is it safe to assume that we can remove the file? So that we > may start again? The error message is warning you that there appear to still be live backends in the data directory, even though the original postmaster process is gone (crashed?). If that is the case, forcibly starting a new postmaster is a fine recipe for creating unrecoverable data corruption. So having pg_ctl auto-remove the file is horribly dangerous and is NOT going to happen. How did you get into this state anyway? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Hide source code
On Mon, Jul 11, 2005 at 09:27:19AM -0700, Josh Berkus wrote: > David, > > > That some "larger organizations" choose to use the known-unsafe > > method of security by obscurity is not a reason for anybody here > > to expend any effort helping them persist in this illusion: quite > > the opposite, in fact. "Larger organizations" are likely to have > > security needs which they actually need to address, not to pretend > > they've addressed while actually making things easy for attackers. > > Hmmm, I agree with Merlin, I think. It would be nice if users who > didn't have permission to EXECUTE functions couldn't view their > code, either. Why? > This would probably carry a performance penalty, though. > Users with EXECUTE permission not being able to see code just isn't > practical; we support too many interpreted languages. If this is a > concern, use C functions and compile binaries. That's secure. With all due respect, it's not even *close* to secure. There are plenty of tools out there that allow a person to de-compile a shared library. A lot of people have learned the hard way over the decades that any security measure that depends on the attacker's not knowing the implementation details is fragile, often disastrously so, e.g. the Enigma machine & friends. There is no good reason for us to help perpetuate the myth of security by obscurity, and plenty of good reasons for us *not* to do so. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote! ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Possible better pg_ctl start/stop handling?
Hello, Interesting problem with pg_ctl. We have ran into this consistently as I am sure a lot of other people have. If PostgreSQL does not get shutdown correctly, the postmaster.pid file is still in PGDATA. This of course causing problems starting up (and it should). However it seems that pg_ctl if issued a stop should be able to remove the file. Below is a speicifc example: bash-3.00$ bin/pg_ctl -D data start pg_ctl: another postmaster may be running; trying to start postmaster anyway LOG: could not load root certificate file "root.crt": No such file or directory DETAIL: Will not verify client certificates. FATAL: pre-existing shared memory block (key 5432001, ID 19202077) is still in use HINT: If you're sure there are no old server processes still running, remove the shared memory block with the command "ipcclean", "ipcrm", or just delete the file "postmaster.pid". pg_ctl: could not start postmaster Examine the log output. bash-3.00$ bin/pg_ctl -D data stop pg_ctl: could not send stop signal (PID: 10180): No such process bash-3.00$ As we can see pg_ctl knows that the PID does not exist. If the PID does not exist is it safe to assume that we can remove the file? So that we may start again? Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] fetch_search_path() and elog.c
On Mon, Jul 11, 2005 at 06:22:26PM +0200, Ferruccio Zamuner wrote: > Now I've found a simpler solution: > > case 'S': >/* estrae il search_path */ > if (namespace_search_path != NULL) >appendStringInfo(buf, "%s ", namespace_search_path); > break; > > It seems to work. > > But I would like to understand why previous code was wrong. I think it was because you were doing catalog lookups (the get_namespace_name() function, etc), and these don't work outside a transaction. -- Alvaro Herrera () "¿Cómo puedes confiar en algo que pagas y que no ves, y no confiar en algo que te dan y te lo muestran?" (Germán Poo) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > Another build failure from buildfarm. Seems like > I forgot to update win32 code when doing a renaming > in random.c Applied. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Another build failure from buildfarm. Seems like I forgot to update win32 code when doing a renaming in random.c -- marko Index: contrib/pgcrypto/random.c === RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/random.c,v retrieving revision 1.12 diff -u -c -r1.12 random.c *** contrib/pgcrypto/random.c 11 Jul 2005 15:07:59 - 1.12 --- contrib/pgcrypto/random.c 11 Jul 2005 16:40:16 - *** *** 126,132 if (!res) return dst; ! res = CryptGenRandom(h, NUM_BYTES, dst); if (res == TRUE) dst += len; --- 126,132 if (!res) return dst; ! res = CryptGenRandom(h, RND_BYTES, dst); if (res == TRUE) dst += len; ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Hide source code
David, > That some "larger organizations" choose to use the known-unsafe method > of security by obscurity is not a reason for anybody here to expend > any effort helping them persist in this illusion: quite the opposite, > in fact. "Larger organizations" are likely to have security needs > which they actually need to address, not to pretend they've addressed > while actually making things easy for attackers. Hmmm, I agree with Merlin, I think. It would be nice if users who didn't have permission to EXECUTE functions couldn't view their code, either. This would probably carry a performance penalty, though. Users with EXECUTE permission not being able to see code just isn't practical; we support too many interpreted languages. If this is a concern, use C functions and compile binaries. That's secure. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] fetch_search_path() and elog.c
Tom Lane wrote: Ferruccio Zamuner <[EMAIL PROTECTED]> writes: I need to have in the log_line_prefix the search_path where the query has run. So last week I've started to read elog.c and I was thinking about a small patch there using a new "%S" option. There's no chance of that code working when not inside a transaction; which means that in most of the scenarios where you really want a log entry to be made, it will fail. regards, tom lane Now I've found a simpler solution: case 'S': /* estrae il search_path */ if (namespace_search_path != NULL) appendStringInfo(buf, "%s ", namespace_search_path); break; It seems to work. But I would like to understand why previous code was wrong. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC
Simon, Tom, > > Will do. Results in a few days. Actually, between the bad patch on the 5th and ongoing STP issues, I don't think I will have results before I leave town.Will e-mail you offlist to give you info to retrieve results. > Any chance you'd be able to do this with > > ext3 and a filesystem blocksize of 4096 bytes > > PostgreSQL built with a blocksize of 4096 Hmmm ... I'm not sure the current test supports these kinds of options. Will ask Mark. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Hide source code
On Mon, Jul 11, 2005 at 07:59:44AM -0400, Merlin Moncure wrote: > > On Sun, Jul 10, 2005 at 04:26:26AM -0300, M?rcio A. Sepp wrote: > > > Hi, > > > > > > I'm looking for a way to hide the souce code of my system > > > (functions). > > > > > > In Oracle, I can wrap it. Is there something that I can use to > > > hide and/or wrap my source code? > > > > If you have code that you don't want people to take, use the > > copyright and license system, which works, not the obfuscation > > system, which is amazingly fragile. And besides, what's so > > embarrassing about this code that you don't want people to see it? > > This was discusses extensively in the archives about a month ago. > Actually it is possible to 'hide' the source code so that a database > user can't view it (or anything else) in psql by manipulating system > schema permissions. Interesting. Again, this is pretty fragile with respect to, for example, pg_dump(all). > Regardless of your philosophical standpoint, many larger > organizations will feel uncomfortable with having unprivileged users > having access to all the database procedure source code. That some "larger organizations" choose to use the known-unsafe method of security by obscurity is not a reason for anybody here to expend any effort helping them persist in this illusion: quite the opposite, in fact. "Larger organizations" are likely to have security needs which they actually need to address, not to pretend they've addressed while actually making things easy for attackers. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote! ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 11:46:29AM -0400, Tom Lane wrote: > Marko Kreen writes: > > Here is the bcopy, bzero removal patch. > > Applied. > > I'm seeing the following build failure on HPUX: > > /usr/ccs/bin/ld +h libpgcrypto.sl.0 -b +b /home/postgres/testversion/lib > pgcrypto.o px.o px-hmac.o px-crypt.o misc.o crypt-gensalt.o crypt-blowfish.o > crypt-des.o crypt-md5.o md5.o sha1.o sha2.o internal.o blf.o rijndael.o > fortuna.o random.o pgp-mpi-internal.o mbuf.o pgp.o pgp-armor.o pgp-cfb.o > pgp-compress.o pgp-decrypt.o pgp-encrypt.o pgp-info.o pgp-mpi.o pgp-pubdec.o > pgp-pubenc.o pgp-pubkey.o pgp-s2k.o pgp-pgsql.o -L../../src/port `gcc > -L../../src/port -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib > -print-libgcc-file-name` -lz -o libpgcrypto.sl.0 > /usr/ccs/bin/ld: Can't find library for -lz > make: *** [libpgcrypto.sl.0] Error 1 > > I believe the issue is that libz.sl is in /usr/local/lib/, which is not > searched by default by HP's linker. It *is* searched by default by gcc, > which is why -lz works without any explicit -L in the pg_dump/pg_restore > builds. But here we are invoking a different tool with a different > default search path. > > Possibly there's something similar happening on that Solaris buildfarm > machine? No, there it finds the libz (in /usr/local/lib - the link command is gcc with -L/usr/local/lib), but in not satisfied with .. something. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 06:41:35PM +0300, Marko Kreen wrote: > When I saw that only 0.9.7[efg] have new signature I even > considered macrofying that. But now with 0.9.8 again different > I really would like to not to touch it, as I have no idea which > one will be the stable signature. > > Comments? Sounds like a question for the OpenSSL developers. If a search through their list archives or CVS repository doesn't yield the answer, then maybe asking the question on one of their lists will. -- Michael Fuhr http://www.fuhr.org/~mfuhr/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > Here is the bcopy, bzero removal patch. Applied. I'm seeing the following build failure on HPUX: /usr/ccs/bin/ld +h libpgcrypto.sl.0 -b +b /home/postgres/testversion/lib pgcrypto.o px.o px-hmac.o px-crypt.o misc.o crypt-gensalt.o crypt-blowfish.o crypt-des.o crypt-md5.o md5.o sha1.o sha2.o internal.o blf.o rijndael.o fortuna.o random.o pgp-mpi-internal.o mbuf.o pgp.o pgp-armor.o pgp-cfb.o pgp-compress.o pgp-decrypt.o pgp-encrypt.o pgp-info.o pgp-mpi.o pgp-pubdec.o pgp-pubenc.o pgp-pubkey.o pgp-s2k.o pgp-pgsql.o -L../../src/port `gcc -L../../src/port -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib -print-libgcc-file-name` -lz -o libpgcrypto.sl.0 /usr/ccs/bin/ld: Can't find library for -lz make: *** [libpgcrypto.sl.0] Error 1 I believe the issue is that libz.sl is in /usr/local/lib/, which is not searched by default by HP's linker. It *is* searched by default by gcc, which is why -lz works without any explicit -L in the pg_dump/pg_restore builds. But here we are invoking a different tool with a different default search path. Possibly there's something similar happening on that Solaris buildfarm machine? regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 09:19:37AM -0600, Michael Fuhr wrote: > On Mon, Jul 11, 2005 at 10:39:26AM -0400, Tom Lane wrote: > > Marko Kreen writes: > > > They won't matter on older OpenSSL, as the macros will recast > > > again. But on 0.9.7e the signature is: > > > > > void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output, > > > DES_key_schedule *ks1,DES_key_schedule *ks2, > > > DES_key_schedule *ks3, int enc); > > > > > so it seems to me that with your patch the warnings will appear > > > on newer OpenSSL. (Confirmed) > > > > Grumble --- you're right. It's probably not worth ifdef'ing the code to > > suppress the warnings on 0.9.7a ... > > Hmmm...in 0.9.8 the signature is back to what it was in 0.9.7[a-d]: > > void DES_ecb3_encrypt(const_DES_cblock *input, DES_cblock *output, > DES_key_schedule *ks1,DES_key_schedule *ks2, > DES_key_schedule *ks3, int enc); Ugh. As I see the old signature goes up to 0.9.7d, and only 0.9.7[e,f,g] have the new signature. 0.9.7e is released on Oct 2004. There is a chance that the 0.9.8 serie was branched before that and later 0.9.8x releases will also change signature. Or the change was mistake, and it was reversed in 0.9.8 - but then why release 0.9.7[f,g] with new signature? When I saw that only 0.9.7[efg] have new signature I even considered macrofying that. But now with 0.9.8 again different I really would like to not to touch it, as I have no idea which one will be the stable signature. Comments? -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 11:09:06AM -0400, Tom Lane wrote:
> Marko Kreen writes:
> > New sha2 code on Solaris 2.8 / SPARC. Seems like it has
> > problems memcpy'ing to a non-8-byte-aligned uint64 *.
> > ...
> > Attached patch includes sys/param.h, where I found them on
> > MINGW, and puts stricter checks into all files.
>
> Applied.
I see you also cleaned the includes. Thanks.
Here is the bcopy, bzero removal patch.
--
marko
Index: contrib/pgcrypto/sha2.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sha2.c,v
retrieving revision 1.2
diff -u -c -r1.2 sha2.c
*** contrib/pgcrypto/sha2.c 11 Jul 2005 15:07:59 - 1.2
--- contrib/pgcrypto/sha2.c 11 Jul 2005 15:20:33 -
***
*** 42,52
#include "sha2.h"
- #undef bcopy
- #undef bzero
- #define bcopy(src, dst, len) memcpy((dst), (src), (len))
- #define bzero(ptr, len) memset((ptr), 0, (len))
-
/*
* UNROLLED TRANSFORM LOOP NOTE:
* You can define SHA2_UNROLL_TRANSFORM to use the unrolled transform
--- 42,47
***
*** 281,288
{
if (context == NULL)
return;
! bcopy(sha256_initial_hash_value, context->state, SHA256_DIGEST_LENGTH);
! bzero(context->buffer, SHA256_BLOCK_LENGTH);
context->bitcount = 0;
}
--- 276,283
{
if (context == NULL)
return;
! memcpy(context->state, sha256_initial_hash_value, SHA256_DIGEST_LENGTH);
! memset(context->buffer, 0, SHA256_BLOCK_LENGTH);
context->bitcount = 0;
}
***
*** 466,479
if (len >= freespace) {
/* Fill the buffer completely and process it */
! bcopy(data, &context->buffer[usedspace], freespace);
context->bitcount += freespace << 3;
len -= freespace;
data += freespace;
SHA256_Transform(context, context->buffer);
} else {
/* The buffer is not yet full */
! bcopy(data, &context->buffer[usedspace], len);
context->bitcount += len << 3;
/* Clean up: */
usedspace = freespace = 0;
--- 461,474
if (len >= freespace) {
/* Fill the buffer completely and process it */
! memcpy(&context->buffer[usedspace], data, freespace);
context->bitcount += freespace << 3;
len -= freespace;
data += freespace;
SHA256_Transform(context, context->buffer);
} else {
/* The buffer is not yet full */
! memcpy(&context->buffer[usedspace], data, len);
context->bitcount += len << 3;
/* Clean up: */
usedspace = freespace = 0;
***
*** 489,495
}
if (len > 0) {
/* There's left-overs, so save 'em */
! bcopy(data, context->buffer, len);
context->bitcount += len << 3;
}
/* Clean up: */
--- 484,490
}
if (len > 0) {
/* There's left-overs, so save 'em */
! memcpy(context->buffer, data, len);
context->bitcount += len << 3;
}
/* Clean up: */
***
*** 514,533
if (usedspace <= SHA256_SHORT_BLOCK_LENGTH) {
/* Set-up for the last transform: */
! bzero(&context->buffer[usedspace],
SHA256_SHORT_BLOCK_LENGTH - usedspace);
} else {
if (usedspace < SHA256_BLOCK_LENGTH) {
! bzero(&context->buffer[usedspace],
SHA256_BLOCK_LENGTH - usedspace);
}
/* Do second-to-last transform: */
SHA256_Transform(context, context->buffer);
/* And set-up for the last transform: */
! bzero(context->buffer,
SHA256_SHORT_BLOCK_LENGTH);
}
} else {
/* Set-up for the last transform: */
! bzero(context->buffer, SHA256_SHORT_BLOCK_LENGTH);
/* Begin padding with a 1 bit: */
*context->buffer = 0x80;
--- 509,528
if (usedspace <= SHA256_SHORT_BLOCK_LENGTH) {
/* Set-up for the last transform: */
! memset(&context->buffer[usedspace], 0,
SHA256_SHORT_BLOCK_LENGTH - usedspace);
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 10:39:26AM -0400, Tom Lane wrote: > Marko Kreen writes: > > They won't matter on older OpenSSL, as the macros will recast > > again. But on 0.9.7e the signature is: > > > void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output, > > DES_key_schedule *ks1,DES_key_schedule *ks2, > > DES_key_schedule *ks3, int enc); > > > so it seems to me that with your patch the warnings will appear > > on newer OpenSSL. (Confirmed) > > Grumble --- you're right. It's probably not worth ifdef'ing the code to > suppress the warnings on 0.9.7a ... Hmmm...in 0.9.8 the signature is back to what it was in 0.9.7[a-d]: void DES_ecb3_encrypt(const_DES_cblock *input, DES_cblock *output, DES_key_schedule *ks1,DES_key_schedule *ks2, DES_key_schedule *ks3, int enc); -- Michael Fuhr http://www.fuhr.org/~mfuhr/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > New sha2 code on Solaris 2.8 / SPARC. Seems like it has > problems memcpy'ing to a non-8-byte-aligned uint64 *. > ... > Attached patch includes sys/param.h, where I found them on > MINGW, and puts stricter checks into all files. Applied. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > On Mon, Jul 11, 2005 at 10:10:12AM -0400, Tom Lane wrote: >> The following addition to the patch shuts up gcc with openssl 0.9.7a, >> but I'm not sure if it will break anything with older openssl --- >> comments? > They won't matter on older OpenSSL, as the macros will recast > again. But on 0.9.7e the signature is: > void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output, > DES_key_schedule *ks1,DES_key_schedule *ks2, > DES_key_schedule *ks3, int enc); > so it seems to me that with your patch the warnings will appear > on newer OpenSSL. (Confirmed) Grumble --- you're right. It's probably not worth ifdef'ing the code to suppress the warnings on 0.9.7a ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 10:13:22AM -0400, Tom Lane wrote: > Marko Kreen writes: > > (I redefined bzero and bcopy but now I think they should be > > replaced directly - patch later.) > > Please. We do not use those old functions in the Postgres code; > memcpy, memmove, memset, etc are the project standard. Indeed. But I'll wait until the previous sha2 patch is applied as they would conflict. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 10:10:12AM -0400, Tom Lane wrote: > Marko Kreen writes: > > Result is - it's not so bad. As I used rijndael.c to provide > > OpenSSL's own interface, I even got rid of all the ifdefs inside > > the code. > > Looks good, but I'm still getting these compile warnings: > > openssl.c: In function `ossl_des3_ecb_encrypt': > openssl.c:484: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible > pointer type > openssl.c:484: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible > pointer type > openssl.c: In function `ossl_des3_ecb_decrypt': > openssl.c:498: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible > pointer type > openssl.c:498: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible > pointer type > > The following addition to the patch shuts up gcc with openssl 0.9.7a, > but I'm not sure if it will break anything with older openssl --- > comments? They won't matter on older OpenSSL, as the macros will recast again. But on 0.9.7e the signature is: void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output, DES_key_schedule *ks1,DES_key_schedule *ks2, DES_key_schedule *ks3, int enc); so it seems to me that with your patch the warnings will appear on newer OpenSSL. (Confirmed) -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > (I redefined bzero and bcopy but now I think they should be > replaced directly - patch later.) Please. We do not use those old functions in the Postgres code; memcpy, memmove, memset, etc are the project standard. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen writes: > Result is - it's not so bad. As I used rijndael.c to provide > OpenSSL's own interface, I even got rid of all the ifdefs inside > the code. Looks good, but I'm still getting these compile warnings: openssl.c: In function `ossl_des3_ecb_encrypt': openssl.c:484: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type openssl.c:484: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type openssl.c: In function `ossl_des3_ecb_decrypt': openssl.c:498: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type openssl.c:498: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type The following addition to the patch shuts up gcc with openssl 0.9.7a, but I'm not sure if it will break anything with older openssl --- comments? regards, tom lane *** /home/postgres/pgsql/contrib/pgcrypto/openssl.c Sun Jul 10 12:35:38 2005 --- new/openssl.c Mon Jul 11 10:06:30 2005 *** *** 446,452 ossldata *od = c->ptr; for (i = 0; i < dlen / bs; i++) ! DES_ecb3_encrypt(data + i * bs, res + i * bs, &od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 1); return 0; } --- 480,487 ossldata *od = c->ptr; for (i = 0; i < dlen / bs; i++) ! DES_ecb3_encrypt((const_DES_cblock *) (data + i * bs), !(DES_cblock *) (res + i * bs), &od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 1); return 0; } *** *** 460,466 ossldata *od = c->ptr; for (i = 0; i < dlen / bs; i++) ! DES_ecb3_encrypt(data + i * bs, res + i * bs, &od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 0); return 0; } --- 495,502 ossldata *od = c->ptr; for (i = 0; i < dlen / bs; i++) ! DES_ecb3_encrypt((const_DES_cblock *) (data + i * bs), !(DES_cblock *) (res + i * bs), &od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 0); return 0; } ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] suspicious pointer/integer coersion
Andrew Dunstan wrote:
Andrew Dunstan wrote:
Looking further ... we already do this implicitly for prodesc in the
call handler - we would just need to do the same thing for per-call
structures and divorce them from prodesc, which can be repeated on
the implicit stack.
I'll work on that - changes should be quite small.
Attached is a patch that fixes (I hope) both a recently introduced
problem with recursion and a problem with array returns that became
evident as a result of not throwing away non-fatal warnings (thanks to
David Fetter for noticing this). Regression test updates to include
both cases are included in the patch.
I will start looking at putting the procedure descriptors in a dynahash.
and here's the patch this time.
cheers
andrew
Index: plperl.c
===
RCS file: /projects/cvsroot/pgsql/src/pl/plperl/plperl.c,v
retrieving revision 1.84
diff -c -r1.84 plperl.c
*** plperl.c 10 Jul 2005 16:13:13 - 1.84
--- plperl.c 11 Jul 2005 13:08:26 -
***
*** 90,98
FmgrInfo arg_out_func[FUNC_MAX_ARGS];
bool arg_is_rowtype[FUNC_MAX_ARGS];
SV *reference;
- FunctionCallInfo caller_info;
- Tuplestorestate *tuple_store;
- TupleDesc tuple_desc;
} plperl_proc_desc;
--- 90,95
***
*** 106,113
static bool plperl_use_strict = false;
! /* this is saved and restored by plperl_call_handler */
static plperl_proc_desc *plperl_current_prodesc = NULL;
/**
* Forward declarations
--- 103,113
static bool plperl_use_strict = false;
! /* these are saved and restored by plperl_call_handler */
static plperl_proc_desc *plperl_current_prodesc = NULL;
+ static FunctionCallInfo plperl_current_caller_info;
+ static Tuplestorestate *plperl_current_tuple_store;
+ static TupleDesc plperl_current_tuple_desc;
/**
* Forward declarations
***
*** 577,586
--- 577,592
{
Datum retval;
plperl_proc_desc *save_prodesc;
+ FunctionCallInfo save_caller_info;
+ Tuplestorestate *save_tuple_store;
+ TupleDesc save_tuple_desc;
plperl_init_all();
save_prodesc = plperl_current_prodesc;
+ save_caller_info = plperl_current_caller_info;
+ save_tuple_store = plperl_current_tuple_store;
+ save_tuple_desc = plperl_current_tuple_desc;
PG_TRY();
{
***
*** 592,602
--- 598,614
PG_CATCH();
{
plperl_current_prodesc = save_prodesc;
+ plperl_current_caller_info = save_caller_info;
+ plperl_current_tuple_store = save_tuple_store;
+ plperl_current_tuple_desc = save_tuple_desc;
PG_RE_THROW();
}
PG_END_TRY();
plperl_current_prodesc = save_prodesc;
+ plperl_current_caller_info = save_caller_info;
+ plperl_current_tuple_store = save_tuple_store;
+ plperl_current_tuple_desc = save_tuple_desc;
return retval;
}
***
*** 897,902
--- 909,915
SV *perlret;
Datum retval;
ReturnSetInfo *rsi;
+ SV* array_ret = NULL;
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "could not connect to SPI manager");
***
*** 904,912
prodesc = compile_plperl_function(fcinfo->flinfo->fn_oid, false);
plperl_current_prodesc = prodesc;
! prodesc->caller_info = fcinfo;
! prodesc->tuple_store = 0;
! prodesc->tuple_desc = 0;
perlret = plperl_call_perl_func(prodesc, fcinfo);
--- 917,925
prodesc = compile_plperl_function(fcinfo->flinfo->fn_oid, false);
plperl_current_prodesc = prodesc;
! plperl_current_caller_info = fcinfo;
! plperl_current_tuple_store = 0;
! plperl_current_tuple_desc = 0;
perlret = plperl_call_perl_func(prodesc, fcinfo);
***
*** 958,967
}
rsi->returnMode = SFRM_Materialize;
! if (prodesc->tuple_store)
{
! rsi->setResult = prodesc->tuple_store;
! rsi->setDesc = prodesc->tuple_desc;
}
retval = (Datum)0;
}
--- 971,980
}
rsi->returnMode = SFRM_Materialize;
! if (plperl_current_tuple_store)
{
! rsi->setResult = plperl_current_tuple_store;
! rsi->setDesc = plperl_current_tuple_desc;
}
retval = (Datum)0;
}
***
*** 1006,1012
{
/* Return a perl string converted to a Datum */
char *val;
- SV* array_ret;
if (prodesc->fn_retisarray && SvTYPE(SvRV(perlret)) == SVt_PVAV)
--- 1019,1024
***
*** 1024,1030
Int32GetDatum(-1));
}
! SvREFCNT_dec(perlret);
return retval;
}
--- 1036,1044
Int32GetDatum(-1));
}
! if (array_ret == NULL)
! SvREFCNT_dec(perlret);
!
return retval;
}
***
*** 1526,1532
plperl_return_next(SV *sv)
{
plperl_proc_desc *prodesc = plperl_current_prodesc;
! FunctionCallInfo
Re: [HACKERS] fetch_search_path() and elog.c
Ferruccio Zamuner <[EMAIL PROTECTED]> writes: > I need to have in the log_line_prefix the search_path where the query > has run. > So last week I've started to read elog.c and I was thinking about a > small patch there using a new "%S" option. There's no chance of that code working when not inside a transaction; which means that in most of the scenarios where you really want a log entry to be made, it will fail. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [BUGS] BUG #1745: Unable to delete data from the
On Sun, 10 Jul 2005, Sivaraman K.G wrote: > The error in the log file is as follows : > > ERROR : xlog flush request 0/D17B00 is not satisfied --- > flushed only to > 0/C31ED0 > CONTEXT : writing block 0 of relation 17231/17232/17249 > WARNING : could not write block 0 of 17231/17232/17249 > DETAIL : Multiple failures --- write error may be > permanent. Check your kernel log for disk errors. It could very well be that your disk is about to die (or already have). -- /Dennis Björklund ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [BUGS] BUG #1745: Unable to delete data from the database
Hi, Since I didn't get any response, I am just posting in pgsql-hackers. Can any one help ?. Thanks Siva > "K.G.Sivaraman" wrote: The following bug has been logged online: Bug reference: 1745 Logged by: K.G.Sivaraman Email address: [EMAIL PROTECTED] PostgreSQL version: 8.0 Operating system: Solaris Description: Unable to delete data from the database Details: When try to delete the existing name in the database (having ID and Name as the table parameters and both are indexed), it returns success. But when queried again, then the deleted name appeared. When we analysed the problem, the basic delete command itself not working. Normally DELETE command will return the number of records deleted (ex) DELETE 1 for 1 record. But now, it returns DELETE 0, even though the record is there and it is not got deleted. The error in the log file is as follows : ERROR : xlog flush request 0/D17B00 is not satisfied --- flushed only to 0/C31ED0 CONTEXT : writing block 0 of relation 17231/17232/17249 WARNING : could not write block 0 of 17231/17232/17249 DETAIL : Multiple failures --- write error may be permanent. But when I add a new record, it is getting added and I am able to delete the newly added record. Why this error coming ?. Can some one can help ?. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- K.G.Sivaraman, Axes Technologies (I) Pvt Ltd "The only SERVICE to be done for our lower classes is to give them EDUCATION, to develop their lost individuality" - Swami Vivekananda
Re: [HACKERS] suspicious pointer/integer coersion
Andrew Dunstan wrote: Looking further ... we already do this implicitly for prodesc in the call handler - we would just need to do the same thing for per-call structures and divorce them from prodesc, which can be repeated on the implicit stack. I'll work on that - changes should be quite small. Attached is a patch that fixes both a recently introduced problem with recursion and a problem with array returns that became evident as a result of not throwing away non-fatal warnings (thanks to David Fetter for noticing this). Regression test updates to include both cases are included in the patch. I will start looking at putting the procedure descriptors in a dynahash. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [BUGS] BUG #1744: Directory Missing
Hi, Since I didn't get any response, I am just posting in pgsql-hackers. Can any one help ?. Thanks Siva > "K.G.Sivaraman" wrote: The following bug has been logged online: Bug reference: 1744 Logged by: K.G.Sivaraman Email address: [EMAIL PROTECTED] PostgreSQL version: 8.0 Operating system: Solaris Description: Directory Missing Details: Postmaster is successfully running after the install. But after sometimes, I am facing the following problem. Can some one suggests what is the problem ?. ERROR : FATAL: database "CTBLISTDB" does not exist. DETAIL : The database subdirectory "/usr/local/pgsql/data/pg_tblspc/17231/17232" is missing. Note : CTBLISTDB is the table space created in the PostGRESQL, where we create table and store the data. I am starting the Postmaster when the Solaris system is coming up. This problem is not happening always. But once the problem is happened I have to manually clean the database and setup again or the backup to be restore, which is not a correct solution. I am not sure whether this is a bug or not. But I am getting this problem frequently. What is the cause and what is the solution. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- K.G.Sivaraman, Axes Technologies (I) Pvt Ltd "The only SERVICE to be done for our lower classes is to give them EDUCATION, to develop their lost individuality" - Swami Vivekananda
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 02:59:54PM +0300, Marko Kreen wrote:
> On Mon, Jul 11, 2005 at 05:50:32AM -0500, Andrew Dunstan wrote:
> > Marko Kreen said:
> > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00>
> > > NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
> > > AES, but most of PGP tests use it as it is the preferred cipher.
> > > And the AES tests fails anyway. I guess it can stay as expected
> > > failure.
> >
> > Please try to avoid expected failures if possible. If you must have them,
> > move them into a test file of their own. Consider the possibility of using
> > alternative .out files.
>
> I need either to use included rijndael.c for AES with older
> OpenSSL or rerun all tests to be Blowfish-only.
>
> I want to standardise on AES so the former is preferred.
>
> Now there's a choice:
>
> 1. Check OpenSSL version in main configure
> 2. #include "rijndael.c" in openssl.c
>
> I guess 1. is nicer. I try to hack something together.
I tried 1. but that was messing with main build system for no
good reason. As the openssl.c would still be mess, so I went
with 2.
Result is - it's not so bad. As I used rijndael.c to provide
OpenSSL's own interface, I even got rid of all the ifdefs inside
the code.
--
marko
Index: contrib/pgcrypto/openssl.c
===
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.22
diff -u -c -r1.22 openssl.c
*** contrib/pgcrypto/openssl.c 10 Jul 2005 13:54:34 - 1.22
--- contrib/pgcrypto/openssl.c 11 Jul 2005 13:02:00 -
***
*** 44,53
/*
* Does OpenSSL support AES?
*/
- #undef GOT_AES
#if OPENSSL_VERSION_NUMBER >= 0x00907000L
! #define GOT_AES
#include
#endif
/*
--- 44,89
/*
* Does OpenSSL support AES?
*/
#if OPENSSL_VERSION_NUMBER >= 0x00907000L
!
! /* Yes, it does. */
#include
+
+ #else
+
+ /*
+ * No, it does not. So use included rijndael code to emulate it.
+ */
+ #include "rijndael.c"
+
+ #define AES_ENCRYPT 1
+ #define AES_DECRYPT 0
+ #define AES_KEY rijndael_ctx
+
+ #define AES_set_encrypt_key(key, kbits, ctx) \
+ aes_set_key((ctx), (key), (kbits), 1)
+
+ #define AES_set_decrypt_key(key, kbits, ctx) \
+ aes_set_key((ctx), (key), (kbits), 0)
+
+ #define AES_ecb_encrypt(src, dst, ctx, enc) \
+ do { \
+ memcpy((dst), (src), 16); \
+ if (enc) \
+ aes_ecb_encrypt((ctx), (dst), 16); \
+ else \
+ aes_ecb_decrypt((ctx), (dst), 16); \
+ } while (0)
+
+ #define AES_cbc_encrypt(src, dst, len, ctx, iv, enc) \
+ do { \
+ memcpy((dst), (src), (len)); \
+ if (enc) \
+ aes_cbc_encrypt((ctx), (iv), (dst), (len)); \
+ else \
+ aes_cbc_decrypt((ctx), (iv), (dst), (len)); \
+ } while (0)
+
#endif
/*
***
*** 205,213
DES_key_schedule k1, k2, k3;
} des3;
CAST_KEYcast_key;
- #ifdef GOT_AES
AES_KEY aes_key;
- #endif
} u;
uint8 key[EVP_MAX_KEY_LENGTH];
uint8 iv[EVP_MAX_IV_LENGTH];
--- 241,247
***
*** 549,556
/* AES */
- #ifdef GOT_AES
-
static int
ossl_aes_init(PX_Cipher * c, const uint8 *key, unsigned klen, const uint8 *iv)
{
--- 583,588
***
*** 642,648
AES_cbc_encrypt(data, res, dlen, &od->u.aes_key, od->iv, AES_DECRYPT);
return 0;
}
- #endif
/*
* aliases
--- 674,679
***
*** 711,717
64 / 8, 128 / 8, 0
};
- #ifdef GOT_AES
static const struct ossl_cipher ossl_aes_ecb = {
ossl_aes_init, ossl_aes_ecb_encrypt, ossl_aes_ecb_decrypt,
128 / 8, 256 / 8, 0
--- 742,747
***
*** 721,727
ossl_aes_init, ossl_aes_cbc_encrypt, ossl_aes_cbc_decrypt,
128 / 8, 256 / 8, 0
};
- #endif
/*
* Special handlers
--- 751,756
***
*** 742,751
{"des3-cbc", &ossl_des3_cbc},
{"cast5-ecb", &ossl_cast_ecb},
{"cast5-cbc", &ossl_cast_cbc},
- #ifdef GOT_AES
{"aes-ecb", &ossl_aes_ecb},
{"aes-cbc", &ossl_aes_cbc},
- #endif
{NULL}
};
--- 771,778
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
On Mon, Jul 11, 2005 at 05:50:32AM -0500, Andrew Dunstan wrote: > Marko Kreen said: > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00> > > NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have > > AES, but most of PGP tests use it as it is the preferred cipher. > > And the AES tests fails anyway. I guess it can stay as expected > > failure. > > Please try to avoid expected failures if possible. If you must have them, > move them into a test file of their own. Consider the possibility of using > alternative .out files. I need either to use included rijndael.c for AES with older OpenSSL or rerun all tests to be Blowfish-only. I want to standardise on AES so the former is preferred. Now there's a choice: 1. Check OpenSSL version in main configure 2. #include "rijndael.c" in openssl.c I guess 1. is nicer. I try to hack something together. -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Hide source code
> On Sun, Jul 10, 2005 at 04:26:26AM -0300, M?rcio A. Sepp wrote: > > Hi, > > > > I'm looking for a way to hide the souce code of my system > > (functions). > > > > In Oracle, I can wrap it. Is there something that I can use to hide > > and/or wrap my source code? > > If you have code that you don't want people to take, use the copyright > and license system, which works, not the obfuscation system, which is > amazingly fragile. And besides, what's so embarrassing about this > code that you don't want people to see it? This was discusses extensively in the archives about a month ago. Actually it is possible to 'hide' the source code so that a database user can't view it (or anything else) in psql by manipulating system schema permissions. Regardless of your philosophical standpoint, many larger organizations will feel uncomfortable with having unprivileged users having access to all the database procedure source code. Merlin ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved
Marko Kreen said: > > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00> > NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have > AES, but most of PGP tests use it as it is the preferred cipher. > And the AES tests fails anyway. I guess it can stay as expected > failure. > Please try to avoid expected failures if possible. If you must have them, move them into a test file of their own. Consider the possibility of using alternative .out files. This doesn't matter to Buildfarm quite so much for contrib as it does for main or PL regression sets, as contrib is the last thing checked, so a failure there doesn't block any other steps in the checking process. Still, a contrib failure will show up as yellow rather than green. Buildfarm does not currently have a way of knowing what failure is expected and what is not - it currently treats any regression failure as unexpected. Changing that is on my TODO list, but it's not going to happen any time soon. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC
On Fri, 2005-07-08 at 09:34 +0200, Zeugswetter Andreas DAZ SD wrote: > >>> The point here is that fsync-off is only realistic for development > or > >>> playpen installations. You don't turn it off in a production > >>> machine, and I can't see that you'd turn off the full-page-write > >>> option either. So we have not solved anyone's performance problem. > > > >> Yes, this is basically another fsync-like option that isn't for > >> production usage in most cases. Sad but true. > > > > Just to make my position perfectly clear: I don't want to see > > this option shipped in 8.1. > > Why not ? If your filesystem buffer size matches your pg page size, > and you have a persistent write cache, the option makes perfect sense. I think this point needs expansion: It is possible to run with matching page sizes, in which case the option to not-log full page images is desirable. The only type of crash that can cause torn pages is an OS crash, such as a power outage. If you have an expensive server with battery backup, persistent write cache etc then this reduces that risk considerably. However, as long as PostgreSQL can't tell the difference between any crash and an OS crash, we must assume the worst. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] 4 pgcrypto regressions failures - 1 unsolved
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=potorooo&dt=2005-07-10%2022:30:03
New sha2 code on Solaris 2.8 / SPARC. Seems like it has
problems memcpy'ing to a non-8-byte-aligned uint64 *.
Attached patch fixes it by simplifying the _Final code and
getting rid of the pointer.
(I redefined bzero and bcopy but now I think they should be
replaced directly - patch later.)
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=goose&dt=2005-07-11%2006:00:04
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ferret&dt=2005-07-10%2018:25:11
The new sha2.c checks not only whether BYTE_ENDIAN is
LITTLE_ENDIAN or BIG_ENDIAN but also whether it is set.
And the test fails on both Cygwin and MINGW.
As gcc evaluates "#if UNDEF1 == UNDEF2" as true and there
were no compile failures reported with current code, that
means currently internal AES, SHA1 and MD5 used randomly
big-endian, little-endian or both variants of code.
If there was no regression failures on those platforms,
it must be only by dumb luck.
Attached patch includes sys/param.h, where I found them on
MINGW, and puts stricter checks into all files.
After I see successful run in pgbuildfarm, I send it for
stable branches too.
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00
NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
AES, but most of PGP tests use it as it is the preferred cipher.
And the AES tests fails anyway. I guess it can stay as expected
failure.
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&dt=2005-07-11%2003:30:04
Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?
Error message:
ld: fatal: relocations remain against allocatable but non-writable sections
--
marko
Index: pgsql/contrib/pgcrypto/sha2.c
===
*** pgsql.orig/contrib/pgcrypto/sha2.c
--- pgsql/contrib/pgcrypto/sha2.c
*** SHA256_Update(SHA256_CTX *context, const
*** 496,502
void
SHA256_Final(uint8 digest[], SHA256_CTX *context)
{
- uint32 *d = (uint32 *)digest;
unsigned intusedspace;
/* If no digest buffer is passed, we don't bother doing this: */
--- 496,501
*** SHA256_Final(uint8 digest[], SHA256_CTX
*** 542,553
int j;
for (j = 0; j < 8; j++) {
REVERSE32(context->state[j],context->state[j]);
- *d++ = context->state[j];
}
}
- #else
- bcopy(context->state, d, SHA256_DIGEST_LENGTH);
#endif
}
/* Clean up state data: */
--- 541,550
int j;
for (j = 0; j < 8; j++) {
REVERSE32(context->state[j],context->state[j]);
}
}
#endif
+ bcopy(context->state, digest, SHA256_DIGEST_LENGTH);
}
/* Clean up state data: */
*** SHA512_Last(SHA512_CTX *context)
*** 823,830
void
SHA512_Final(uint8 digest[], SHA512_CTX *context)
{
- uint64 *d = (uint64 *)digest;
-
/* If no digest buffer is passed, we don't bother doing this: */
if (digest != NULL) {
SHA512_Last(context);
--- 820,825
*** SHA512_Final(uint8 digest[], SHA512_CTX
*** 836,847
int j;
for (j = 0; j < 8; j++) {
REVERSE64(context->state[j],context->state[j]);
- *d++ = context->state[j];
}
}
- #else
- bcopy(context->state, d, SHA512_DIGEST_LENGTH);
#endif
}
/* Zero out state data */
--- 831,840
int j;
for (j = 0; j < 8; j++) {
REVERSE64(context->state[j],context->state[j]);
}
}
#endif
+ bcopy(context->state, digest, SHA512_DIGEST_LENGTH);
}
/* Zero out state data */
*** SHA384_Update(SHA384_CTX *context, const
*** 869,876
void
SHA384_Final(uint8 digest[], SHA384_CTX *context)
{
- uint64 *d = (uint64 *)digest;
-
/* If no digest buffer is passed, we don't bother doing this: */
if (digest != NULL) {
SHA512_Last((SHA512_CTX *)context);
--- 862,867
*** SHA384_Final(uint8 digest[], SHA384_CTX
*** 882,893
int j;
for (j = 0; j < 6; j++) {
REVERSE64(context->state[j],context->state[j]);
- *d++ = context->state[j];
}
}
- #else
- bcopy(context->state, d, SHA384_DIGEST_LENGTH);
Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC
On Fri, 2005-07-08 at 14:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I don't think we should care too much about indexes. We can rebuild > > them...but losing heap sectors means *data loss*. > > If you're so concerned about *data loss* then none of this will be > acceptable to you at all. We are talking about going from a system > that can actually survive torn-page cases to one that can only tell > you whether you've lost data to such a case. Arguing about the > probability with which we can detect the loss seems beside the point. In all of this, I see that turning off full page images would be an option that defaults to "yes, take page images". PITR was originally discussed (in 2002, see the archives) as a mechanism that would allow full page images to be avoided. Since we now have PITR, we can discuss more sensibly taking that option. If there are some circumstances where we don't know the state of the server and need to recover, that is OK, as long as we *can* recover. BUT only if we have a fairly low chance of needing to use it. (Rebuilding an index is preferable to a full system recovery.) So I am interested in the probability of us knowing whether the system is damaged or not. It may then become an acceptable risk for a production system to take in order to gain 50% performance. To that end, I am willing to consider various heuristics that would allow us to reduce the risk. I have suggested some, but am happy to hear others (or, as you say, corrections to them) to make that idea more viable. ISTM that Recovery could tell us: 1. Fully recovered, provably correct state of all data blocks 2. Fully recovered, unknown data correctness of some data blocks 3. Fully recovered, provably incorrect state of some data blocks as well as: a) no indexes require rebuilding b) the following indexes require an immediate REINDEX... Result 1a requires no further action 1b requires some index rebuild after system becomes operational Results 2 and 3 would require some form of system recovery Since currently there are no tests performed to show correctness, we won't ever know we're in state 1 and so would need to presume we are in state 2 and recover. My view is that if enough heuristics can be found to increase the potential for ending a recovery in state 1 then turning off full page images may become viable as a realistic cost/benefit. Though that is never an option that I would suggest should be disabled by default. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] fetch_search_path() and elog.c
Hi,
I need to have in the log_line_prefix the search_path where the query
has run.
So last week I've started to read elog.c and I was thinking about a
small patch there using a new "%S" option.
First I've introduced a small code:
case 'S':
{
List *search_path = fetch_search_path(false);
if (search_path != NIL) {
ListCell *l;
foreach(l, search_path)
{
char *nspname;
nspname = get_namespace_name(lfirst_oid(l));
if (nspname)/* watch out for
deleted namespace */
{
appendStringInfo(buf, "%s ", nspname);
pfree(nspname);
}
}
list_free(search_path);
}
break;
}
but in this way postgres was starting with a core dump writing on log.
Then I've add some code, and now it starts but it kills the postgres
process as soon as a query has sent to log:
$ diff -u elog.c elog_new.c
--- elog.c Sat Mar 12 02:55:15 2005
+++ elog_new.c Sun Jul 10 10:16:35 2005
@@ -67,7 +67,8 @@
#include "tcop/tcopprot.h"
#include "utils/memutils.h"
#include "utils/guc.h"
-
+#include "catalog/namespace.h"
+#include "catalog/pg_type.h"
/* Global variables */
ErrorContextCallback *error_context_stack = NULL;
@@ -1444,6 +1445,32 @@
case '%':
appendStringInfoChar(buf, '%');
break;
+ /* */
+ case 'S':
+ /* estrae il search_path */
+ if (MyProcPort && (MyProcPort->commandTag !=
NULL)) {
+ char *cmd=MyProcPort->commandTag;
+ if ((strcasecmp(cmd,"SELECT")== 0) ||
(strcasecmp(cmd,"INSERT")== 0) || (strcasecmp(cmd,"UPDATE")== 0) ||
(strcasecmp(cmd,"DELETE")== 0)) {
+
+ }
+ }
+ break;
+List *search_path = fetch_search_path(false);
+ if (search_path != NIL) {
+ ListCell *l;
+ foreach(l, search_path)
+ {
+ char *nspname;
+
+ nspname =
get_namespace_name(lfirst_oid(l));
+ if (nspname)/*
watch out for deleted namespace */
+ {
+ appendStringInfo(buf, "%s ",
nspname);
+ pfree(nspname);
+ }
+ }
+ list_free(search_path);
+ }
default:
/* format error - ignore it */
break;
And here there is the client output:
bash-2.05a$ psql prova -U pgsql
Welcome to psql 8.0.3, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit
prova=# select * from prova;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>
Where can I start to understand which checks I've missed and how to gain
the output I need?
Thank you in advance,\ferz
---
NonSoLoSoft - http://www.nonsolosoft.com/
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
