Re: [HACKERS] Why we still see some reports of could not access transaction status
Tom Lane wrote: Having seen a couple recent reports of could not access status of transaction for old, not-obviously-corrupt transaction numbers, I went looking to see if I could find a way that the system could truncate CLOG before it's really marked all occurrences of old transaction numbers as known-dead or known-good. I found one. I was starting to wonder about those reports, too. Actually I was thinking about bringing this up as soon as I would find time. So I am glad you picked that up yourself -- and found a problem already. I think what we ought to do to solve this problem permanently is to stop ... Comments? Well, I am not able to comment here, but I can say I usually trust your judgement. Best Regards, Michael Paesold ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] Embedded postgresql
Dear Sirs, I would like to know if there are any discussions about creating an embedded version on postgresql. My thoughts go towards building/porting a sqlite equivalent of pg. Regards, GB. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] plans for bitmap indexes?
create index people_male_gay_ix on people (city) where gender = 'male' and orientation = 'gay'; You've forgotten part of my premise (based on a real case I discussed on IRC) that there are EIGHTEEN criteria columns. That is why I said maybe :-) Whether it helps depends on the number of actually (often) used access patterns. That would require, by the method you have above, roughly 18(3rd factorial) indexes, times the number of values allowed by each column, which if it averaged, say 5 values, would be 24,480 Well, an index only needs to reduce the cost enough so that you can afford your workload and have reasonable response times, so you might only need to create a few of them. I was actually only trying to help optimize this example without the bitmap index feature, not trying to say that for this example partial indexes are better. Especially since the first example, that mentioned partial indexes was not the way to do it for a value that represents a large part of your table (here approx 50%). (don't do: create index people_male on people (gender) where gender='male';) Andreas ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Embedded postgresql
[EMAIL PROTECTED] wrote: I would like to know if there are any discussions about creating an embedded version on postgresql. My thoughts go towards building/porting a sqlite equivalent of pg. Yes, there have been several. Peruse the archives: http://archives.postgresql.org/ -- dave ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Why we still see some reports of could not access transaction status
On Wed, Oct 13, 2004 at 12:18:08PM -0400, Tom Lane wrote: I think what we ought to do to solve this problem permanently is to stop making the callers of the HeapTupleSatisfiesFoo() routines responsible for checking for hint bit updates. It would be a lot safer, and AFAICS not noticeably less efficient, for those routines to call SetBufferCommitInfoNeedsSave for themselves. This would require adding to their parameter lists, because they aren't currently told which buffer the tuple is in, but that's no big deal considering we get to simplify the calling logic in all the places that are faithfully doing the t_infomask update check. Comments? I remember seeing this code when coding the phantom Xid idea and wondering why such an error-prone style was used. It never ocurred to me to change it (or maybe have the guts to do it), but now that you mention it it certainly seems a good idea. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) Tulio: oh, para qué servirá este boton, Juan Carlos? Policarpo: No, aléjense, no toquen la consola! Juan Carlos: Lo apretaré una y otra vez. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] plperl Safe restrictions
It has just been brought to my attention that we are being very restrictive about what we allow to be done in trusted plperl. Basically we allow the :default and :base_math set of operations (run perldoc Opcode or see http://www.perldoc.com/perl5.8.0/lib/Opcode.html for details of what these mean). In particular, we do not allow calls to perl's builtin sort, which is unpleasant, and on reviewing the list it seems to me we could quite reasonably allow access to pack and unpack also. bless and sprintf are also likely candidates for inclusion - I have not finished reviewing the list, and would welcome advice from perl gurus on this. On the other side, I think we should exclude the :base_io set which is part of the :default set (we don't want trusted plperl writing to stdout, for example - all IO should be forbidden). I know changing this at this stage of the dev cycle is bad, but I think it ought to be done. Unless there are loud squawks I will submit a patch RSN. It should be very low risk - one or two lines. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Problem with CIDR data type restrictions
Added to TODO: * Prevent inet cast to cidr if the unmasked bits are not zero, or zero bits --- Andrew Dunstan wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Not sure how serious this is since we have gotten few complaints about it but clearly it should be fixed. Personally I'm inclined to leave it for 8.1. The inet/cidr code is really designed around the assumption that these datatypes are interchangeable, and I suspect that enforcing a stronger distinction will actually take much more wide-ranging changes than just this. Do all of the functions on inet/cidr take care to deliver a value that is both correctly marked and declared as the correct type? I doubt it. It needs some thought not just a band-aid ... Yeah. I am not sure I understand the intention, but I should have thought there was a good case for clearing the bits past the mask on conversion from either text or inet, rather than rejecting or invalidly copying. As you say, it needs some thought. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] plperl Safe restrictions
On Thu, 14 Oct 2004, Andrew Dunstan wrote: It has just been brought to my attention that we are being very restrictive about what we allow to be done in trusted plperl. Basically we allow the :default and :base_math set of operations (run perldoc Opcode or see http://www.perldoc.com/perl5.8.0/lib/Opcode.html for details of what these mean). In particular, we do not allow calls to perl's builtin sort, which is unpleasant, and on reviewing the list it seems to me we could quite reasonably allow access to pack and unpack also. bless and sprintf are also likely candidates for inclusion - I have not finished reviewing the list, and would welcome advice from perl gurus on this. On the other side, I think we should exclude the :base_io set which is part of the :default set (we don't want trusted plperl writing to stdout, for example - all IO should be forbidden). That makes sense. Allowing rand would be nice too. Jon -- Jon Jensen End Point Corporation http://www.endpoint.com/ Software development with Interchange, Perl, PostgreSQL, Apache, Linux, ... ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] plperl Safe restrictions
Jon Jensen wrote: On Thu, 14 Oct 2004, Andrew Dunstan wrote: It has just been brought to my attention that we are being very restrictive about what we allow to be done in trusted plperl. Basically we allow the :default and :base_math set of operations (run perldoc Opcode or see http://www.perldoc.com/perl5.8.0/lib/Opcode.html for details of what these mean). In particular, we do not allow calls to perl's builtin sort, which is unpleasant, and on reviewing the list it seems to me we could quite reasonably allow access to pack and unpack also. bless and sprintf are also likely candidates for inclusion - I have not finished reviewing the list, and would welcome advice from perl gurus on this. On the other side, I think we should exclude the :base_io set which is part of the :default set (we don't want trusted plperl writing to stdout, for example - all IO should be forbidden). That makes sense. Allowing rand would be nice too. You can now - it's part of :base_math. What we should do, however, is disallow is calling srand, since pg goes to quite a bit of trouble to seed the PRNG. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] plperl Safe restrictions
On 14. okt 2004, at 21:09, Andrew Dunstan wrote: It has just been brought to my attention that we are being very restrictive about what we allow to be done in trusted plperl. Basically we allow the :default and :base_math set of operations (run perldoc Opcode or see http://www.perldoc.com/perl5.8.0/lib/Opcode.html for details of what these mean). In particular, we do not allow calls to perl's builtin sort, which is unpleasant, and on reviewing the list it seems to me we could quite reasonably allow access to pack and unpack also. bless and sprintf are also likely candidates for inclusion - I have not finished reviewing the list, and would welcome advice from perl gurus on this. pack and unpack are unfortunately not safe. Very useful, but they allow write/read access to random memory. It's really a shame perl doesn't have a pragma to make them safe or have safe versions of them. Bless should be OK, unless sensitive objects are provided to the procedures. A postgres question I don't know the answer to is whether allowing the user to trigger a segfault is a security problem. If it isn't, several opcodes may probably be allowed, including sort and sprintf. If it is, well, you need only follow the perl5-porters list to know that there's banal perl structures are continuously being found that will segfault perl, some at compile time, other at runtime. On the other side, I think we should exclude the :base_io set which is part of the :default set (we don't want trusted plperl writing to stdout, for example - all IO should be forbidden). Definitely out :) I know changing this at this stage of the dev cycle is bad, but I think it ought to be done. Unless there are loud squawks I will submit a patch RSN. It should be very low risk - one or two lines. Take care what you open. Perl may be a fairly secure, but the complex implementation means that problematic corner cases are regularly found. d. -- David Helgason, Business Development et al., Over the Edge I/S (http://otee.dk) Direct line +45 2620 0663 Main line +45 3264 5049 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] plperl Safe restrictions
On Thu, 14 Oct 2004, Andrew Dunstan wrote: That makes sense. Allowing rand would be nice too. You can now - it's part of :base_math. Oh, ok. I saw it's not included in :base_core despite being in :base_math, but didn't realize explicitly including :base_math would bring it back. What we should do, however, is disallow is calling srand, since pg goes to quite a bit of trouble to seed the PRNG. That sounds reasonable. Jon -- Jon Jensen End Point Corporation http://www.endpoint.com/ Software development with Interchange, Perl, PostgreSQL, Apache, Linux, ... ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [BUGS] BUG #1270: stack overflow in thread in fe_getauthname
One idea would be to use malloc() to allocate storage for the thread-safe buffers when compiled with thread-safety, rather than using the stack. --- Peter Davie wrote: Hi Tom, How many of these platforms you use are POSIX-compliant? This information came from the POSIX web site (NOT THE DIGITAL/COMPAQ/HP/... WEBSITE). Sysconf(_SC_GETPW_R_SIZE_MAX) is supported by Solaris 2.5, SCO UNIX (circa 1999!), Digital UNIX/Compaq Tru64 UNIX, FreeBSD, AIX, HP-UX, and probably many more. Maybe the non-compliant platforms should be catered for as legacy with support code in the ports area, and those that do adhere to open standards can be accommodated without breaking existing production applications. Thanks Peter Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: OK, we got a report. I just thinkg 8192 is excessive for that structure, and if someone is having a problem, others might as well. On Tru64 UNIX, sysconf(_SC_GETPW_R_SIZE_MAX) returns 1024. I'd be more impressed by this line of reasoning if _SC_GETPW_R_SIZE_MAX were defined on more than one of the platforms I use... regards, tom lane -- Relevance... because results count Relevance Phone: +61 (0)2 6241 6400 A.B.N. 86 063 403 690 Fax:+61 (0)2 6241 6422 Unit 11, Mitchell Commercial Centre,Mobile: +61 (0)417 265 175 cnr Brookes and Heffernan Sts., E-mail: [EMAIL PROTECTED] Mitchell ACT 2911 Web:http://www.relevance.com.au -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] plans for bitmap indexes?
Hi, On Sat, Oct 09, 2004 at 01:31:36PM -0400, Chris Browne wrote: The most nearly comparable thing is be the notion of partial indexes, where, supposing you had 60 region codes (e.g. - 50 US states, 10 Canadian provinces), you might set up indices thus: [...] The partial indexes will not ALWAYS be useful; in cases where they aren't, it is not inconceivable that there are improvements to be made in the query optimizer... So what you are suggesting here is the tree-fashioned-static way of real bitmap indexes. I.E. each time a new value is inserted vor any kind of thus indexes column you have to create a new index which is not very usefull as you can think of. In addition nothing about the real granularity is known to the optimizer to let it guess the best execution plan, i.e. to do a full table scan or use an index. That means if one attributes value is representative for 80 percent it is usefull to do a full table scan whereas if its value is representative for only 5 percent the index might be better. But as I understood the partial index concept, no statistics for value representation are available. Therefore I started to do read some articles and books about bitmap index implementations to maby contribute... we will see... BTW: Is there any more documented CVS-version available? I mean it would be really nice to read some comments from time to time or at least more comments about each function/method's purpose or functionality. Regards, Yann ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] plans for bitmap indexes?
On Thu, Oct 14, 2004 at 11:08:54PM +0200, Yann Michel wrote: BTW: Is there any more documented CVS-version available? I mean it would be really nice to read some comments from time to time or at least more comments about each function/method's purpose or functionality. Huh, the code is reasonably commented. Certainly not following Javadoc-like standards, but it can be read. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) La rebeldía es la virtud original del hombre (Arthur Schopenhauer) ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] libpq and prepared statements progress for 8.0
(I apologise for the delayed response.) At 2004-10-07 01:23:56 -0400, [EMAIL PROTECTED] wrote: So why is this part of the patch ok? Isn't it going to make libpq get confused every time a PQExecPrepared sends a v3.0 prepare message? I thought about that for a while, but I couldn't find anything that is actually broken or confused by the patch. I could be missing something obvious, though, so I'd appreciate another set of eyes looking at it. Does anyone have any ideas? -- ams ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Strange code in initdb
I have modified these reports so they print the full path used.
Typical pg_dumpall output is:
The program pg_dump is needed by pg_dumpall but was not found in the
same directory as /usr/local/postgres/bin/pg_dumpall.
Check your installation.
---
Peter Eisentraut wrote:
What is this trying to tell us?
if (ret == -1)
fprintf(stderr,
_(The program \postgres\ is needed by %s
but was not found in the same directory as \%s\.\n
Check your installation.\n),
progname, progname);
else
fprintf(stderr,
_(The program \postgres\ was found by %s
but was not the same version as \%s\.\n
Check your installation.\n),
progname, progname);
Shouldn't the second progname be argv[0] or something else that contains
the full path?
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/bin/initdb/initdb.c
===
RCS file: /cvsroot/pgsql/src/bin/initdb/initdb.c,v
retrieving revision 1.61
diff -c -c -r1.61 initdb.c
*** src/bin/initdb/initdb.c 12 Oct 2004 21:54:42 - 1.61
--- src/bin/initdb/initdb.c 15 Oct 2004 04:16:47 -
***
*** 2268,2285
if ((ret = find_other_exec(argv[0], postgres, PG_VERSIONSTR,
backend_exec)) 0)
{
if (ret == -1)
fprintf(stderr,
_(The program \postgres\ is needed by %s
! but was not found in the same directory as
\%s\.\n
Check your installation.\n),
! progname, progname);
else
fprintf(stderr,
! _(The program \postgres\ was found by %s
! but was not the same version as \%s\.\n
Check your installation.\n),
! progname, progname);
exit(1);
}
--- 2268,2291
if ((ret = find_other_exec(argv[0], postgres, PG_VERSIONSTR,
backend_exec)) 0)
{
+ char full_path[MAXPGPATH];
+
+ if (find_my_exec(argv[0], full_path) 0)
+ StrNCpy(full_path, progname, MAXPGPATH);
+
if (ret == -1)
fprintf(stderr,
_(The program \postgres\ is needed by %s
! but was not found in the\n
! same directory as \%s\.\n
Check your installation.\n),
! progname, full_path);
else
fprintf(stderr,
! _(The program \postgres\ was found by
\%s\\n
! but was not the same version as %s.\n
Check your installation.\n),
! full_path, progname);
exit(1);
}
Index: src/bin/pg_ctl/pg_ctl.c
===
RCS file: /cvsroot/pgsql/src/bin/pg_ctl/pg_ctl.c,v
retrieving revision 1.35
diff -c -c -r1.35 pg_ctl.c
*** src/bin/pg_ctl/pg_ctl.c 13 Oct 2004 10:35:05 - 1.35
--- src/bin/pg_ctl/pg_ctl.c 15 Oct 2004 04:16:50 -
***
*** 524,540
if ((ret = find_other_exec(argv0, postmaster, PM_VERSIONSTR,
postmaster_path))
0)
{
if (ret == -1)
write_stderr(_(The program \postmaster\ is needed
by %s
! but was not found in the same
directory as
!
[HACKERS] Networking feature for postgresql...
Hi, I'm trying to add a -project specific- networking feature to my postgres build (or database as function). What I want to do is to send a Query instance (as a String-retrieved through an SPI function) to other machines and (after they have executed it) to receive result tuples. It's about a mediator-wrapper project. My first thought was to write 2 SPI functions (one for the server (concurrent) and the other for client) but I'm not sure if this is going to work. I'm worried about setting up the server process running on the background while other SPI calls are made. Can this be done? Should I consider something else first, before writting any code? Any suggestions would be appreciated! Best regards, Ntinos Katsaros ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [GENERAL] Networking feature for postgresql...
Katsaros Kwn/nos wrote: Hi, I'm trying to add a -project specific- networking feature to my postgres build (or database as function). What I want to do is to send a Query instance (as a String-retrieved through an SPI function) to other machines and (after they have executed it) to receive result tuples. It's about a mediator-wrapper project. My first thought was to write 2 SPI functions (one for the server (concurrent) and the other for client) but I'm not sure if this is going to work. I'm worried about setting up the server process running on the background while other SPI calls are made. Have you looked at the dblink code? -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [GENERAL] Networking feature for postgresql...
Well, actually no :) ! Thanks for the hint! But just from curiosity, would the scenario I described work? I mean is it possible for an SPI process to run in the background while other SPI calls are made? Ntinos Katsaros On Thu, 2004-10-14 at 11:15, Richard Huxton wrote: Katsaros Kwn/nos wrote: Hi, I'm trying to add a -project specific- networking feature to my postgres build (or database as function). What I want to do is to send a Query instance (as a String-retrieved through an SPI function) to other machines and (after they have executed it) to receive result tuples. It's about a mediator-wrapper project. My first thought was to write 2 SPI functions (one for the server (concurrent) and the other for client) but I'm not sure if this is going to work. I'm worried about setting up the server process running on the background while other SPI calls are made. Have you looked at the dblink code? ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [GENERAL] Networking feature for postgresql...
Katsaros Kwn/nos wrote: Well, actually no :) ! Thanks for the hint! But just from curiosity, would the scenario I described work? I mean is it possible for an SPI process to run in the background while other SPI calls are made? I don't think so, you're running in a backend process, so you'd need to fork the backend itself. On Thu, 2004-10-14 at 11:15, Richard Huxton wrote: Katsaros Kwn/nos wrote: Hi, I'm trying to add a -project specific- networking feature to my postgres build (or database as function). What I want to do is to send a Query instance (as a String-retrieved through an SPI function) to other machines and (after they have executed it) to receive result tuples. It's about a mediator-wrapper project. My first thought was to write 2 SPI functions (one for the server (concurrent) and the other for client) but I'm not sure if this is going to work. I'm worried about setting up the server process running on the background while other SPI calls are made. Have you looked at the dblink code? -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
