Re: [HACKERS] psql variables

2004-02-09 Thread Neil Conway
Tom Lane [EMAIL PROTECTED] writes: [ blinks... ] This is historical revisionism. Psql variables were invented to provide user-defined variables; it is the predefined ones that are a wart added to the mechanism, not vice versa. The historical origins of the feature are no excuse for its

Re: [HACKERS] session persistent data for plperl

2004-02-09 Thread Josh Berkus
Adnrew, The attached tiny patch (not intended for application yet) provides a space for plperl functions to create and share session persistent data, which I should think would increase the utility of plperl. You want to really make PL/Perl more useful? Add an SPI interface, and work out

[HACKERS] Timestamps

2004-02-09 Thread Slavisa Garic
Hi all, I am not sure if this should be sent to this mailing list. If i am wrong could someone please direct me to the correct one so I can subscribe there. I wanted to ask a simple question. Say I have a table with the timestamp field. What is the best way to say get all the records that were

Re: [HACKERS] RFC: Security documentation

2004-02-09 Thread Josh Berkus
Alex, As such, I would like to see some documentation about securing the database at a data and application level. It would be nice to have some general guidelines, as well as being able to cite documentation when setting up a security policy for a database application. I'd be happy to

Re: [HACKERS] Timestamps

2004-02-09 Thread Christopher Kings-Lynne
I wanted to ask a simple question. Say I have a table with the timestamp field. What is the best way to say get all the records that were created say 2 hours before the query. One of the options would be to generate the timestamp in the correct format and then send a query in the format SELECT *

Re: [HACKERS] psql tab completion USERSET vars

2004-02-09 Thread Neil Conway
Tom Lane [EMAIL PROTECTED] writes: I'm not by any means wedded to the USERSET and possibly SUSET policy, but I would like to stop somewhere short of include everything. Any thoughts? Perhaps we could have two sets of variables: all the GUC vars (that can be displayed via SHOW), and a subset

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Josh Berkus
Alex, I find myself wondering what other people are doing with postgres that this doesn't seem to have come up. When one searches for postgres clustering on google, they will find lots of HA products. However, nobody seems to be attempting to create very high throughput clusters. Have you

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Chris
That's what I said, and what I meant. Ten billion transactions equates to 115,740 transactions per second. Have you tried to look at the scientific comunity? CERN has setups that produce such large amounts of data - try searching google for

Re: [HACKERS] Advice regarding configuration parameters

2004-02-09 Thread Peter Eisentraut
Tom Lane wrote: Given all the work Peter put into GUC (for very good reasons), I was a tad astonished to read him proposing to develop a non-GUC mechanism for configuring PLs. I for one was a tad astonished to read that there is already support for adding variables at run-time, given that we

Re: [HACKERS] psql variables

2004-02-09 Thread Peter Eisentraut
Neil Conway wrote: Recently, I was surprised to learn that psql variables are case sensitive. like shell variables Furthermore, there is no error when one attempts to '\set' a non-existent variable Well, how are you going to set a new variable if not this way? One possible justification

Re: [HACKERS] session persistent data for plperl

2004-02-09 Thread Andrew Dunstan
Josh Berkus wrote: Adnrew, Jsoh :-) You want to really make PL/Perl more useful? Add an SPI interface, and work out the kinks in error-trapping via eval{}. This would be far more useful that session variables. Just a thought. I don't intend to stop there. I admit that the gain in

Re: [HACKERS] Transaction aborts on syntax error.

2004-02-09 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: What it comes down to is that a lot of code in the backend assumes that transaction abort can be relied on to do any post-elog cleanup needed, such as releasing locks or reclaiming leaked memory. I don't think we can afford to give up that assumption; the

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Keith Bottner
Alex, I agree that this is something that is worth spending time on. This resembles the Oracle RAC (Real Application Cluster). While other people may feel that the amount of data is unreasonable I have a similar problem that will only be solved using such a solution. In regards to how your

Re: [HACKERS] Transaction aborts on syntax error.

2004-02-09 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: So you picture the backend automatically introducing a mini-nested-transaction for every request and automatically rolling that back on any error. So the application or user wouldn't have to do anything to continue processing ignoring the error? You're

Re: [HACKERS] [pgsql-hackers-win32] Sync vs. fsync during checkpoint

2004-02-09 Thread Jan Wieck
Bruce Momjian wrote: Jan Wieck wrote: Tom Lane wrote: Zeugswetter Andreas SB SD [EMAIL PROTECTED] writes: So Imho the target should be to have not much IO open for the checkpoint, so the fsync is fast enough, even if serial. The best we can do is push out dirty pages with write() via the

Re: [HACKERS] RFC: Security documentation

2004-02-09 Thread Alex J. Avriette
On Sun, Feb 08, 2004 at 09:34:15PM -0500, Tom Lane wrote: Is this nothing? http://www.postgresql.org/docs/7.4/static/libpq-exec.html#LIBPQ-EXEC-ESCAPE-STRING I don't think the docs are nearly as bereft of security-related items as you claim. They may be scattered and poorly indexed, but

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Andreas Pflug
Keith Bottner wrote: Alex, I agree that this is something that is worth spending time on. This resembles the Oracle RAC (Real Application Cluster). While other people may feel that the amount of data is unreasonable I have a similar problem that will only be solved using such a solution. In

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Keith Bottner
I always enjoy how everyone wants to talk about using different solutions prior to understanding the complete problem. I would say that a *real* issue is any perceived issue whether a current solution exists or not. If current solutions are applicable and would work then great we have all gained;

Re: [HACKERS] RFC: Very large scale postgres support

2004-02-09 Thread Andreas Pflug
Keith Bottner wrote: I understand your position Andreas and respect your opinion; maybe what I have identified as requirements is what you are specifying as *real* issues. I hope so, because I to would like to avoid unnecessary dbms efforts. You got me very right. I didn't mean to declare high

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread markw
On 6 Feb, To: [EMAIL PROTECTED] wrote: On 5 Jan, Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: 2) DEVELOP BETTER PLANS FOR OR GROUP QUERIES Summary: Currently, queries with complex or group criteria get devolved by the planner into canonical and-or filters resulting in very poor

Re: [HACKERS] psql variables

2004-02-09 Thread Peter Eisentraut
Neil Conway wrote: Perhaps you're suggesting shell variables were used as the design model for psql's variables (although I can't be sure, you didn't elaborate). If so, what I'm saying is that this model is not very friendly for setting psql-internal options, and we'd be better changing it as

Re: [HACKERS] [pgsql-hackers-win32] Sync vs. fsync during checkpoint

2004-02-09 Thread Tom Lane
Jan Wieck [EMAIL PROTECTED] writes: The whole sync() vs. fsync() discussion is in my opinion nonsense at this point. The sync vs fsync discussion is not about performance, it is about correctness. You can't simply dismiss the fact that we don't know whether a checkpoint is really complete

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread Tom Lane
[EMAIL PROTECTED] writes: I'll see what I can do about the explain and explain analyze results. I remember in the past that someone said it would be most interesting to execute the latter while the test while running, as opposed to before or after a test. Should I do that here too? If

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread markw
On 9 Feb, Tom Lane wrote: [EMAIL PROTECTED] writes: http://developer.osdl.org/markw/dbt3-pgsql/ There's a short summary of the tests I ran over the weekend, with links to detailed retults. Comparing runs 43 (7.4) and 52 (7.5devel), it looks like query #7 had the only significant

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread Tom Lane
[EMAIL PROTECTED] writes: http://developer.osdl.org/markw/dbt3-pgsql/ There's a short summary of the tests I ran over the weekend, with links to detailed retults. Comparing runs 43 (7.4) and 52 (7.5devel), it looks like query #7 had the only significant improvement. Oprofile data should be

Re: [HACKERS] 7.4.1 release status - Turkish Locale

2004-02-09 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: It seems to me that that's too narrow a definition of the problem. I think we should state our goal as we don't want bizarre locale definitions to interfere with downcasing of the basic ASCII letters. If we put in a special case for 'I' we will fix the

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread markw
On 9 Feb, Josh Berkus wrote: Mark, Ok, I've found that the kit does capture explain results and I've added a Query Plans links under the query time charts on each of the pages. Um, but I did notice a couple of problems. It looks liks one of the 22 queries is missing and they're not

[HACKERS] Question on pg_dump

2004-02-09 Thread Michael Brusser
I'm running Postgres v.7.3.4. In my database dump file I see this: CREATE FUNCTION plpgsql_call_handler () RETURNS language_handler AS '/home/tmichael/build/relipg21/syncinc/lib.sol2/plpgsql', 'plpgsql_call_handler' LANGUAGE c; The hardcoded library path may become an obstacle when

Re: [HACKERS] psql variables

2004-02-09 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Shell variables did serve as a design model, mostly because I found it better to use *some* model rather than inventing behavior out of thin air. Consequently, I am sort of biased on this. It does seem worth pointing out that shell variables have

Re: [HACKERS] 7.4.1 release status - Turkish Locale

2004-02-09 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: If it matches an SQL keyword after being downcased the old fashioned way, then it's an SQL keyword. If not then the locale-aware tolower() would be appropriate for tables, columns, etc. That's exactly what we do already. The complaint was that the

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: If #19 is missing it's because Oleg I could not get it to complete. That was also the query which we are most interested in testing. Q19 doesn't seem to be particularly slow in either the 7.4 or 7.5 tests --- there are many others with longer runtimes.

Re: [HACKERS] Question on pg_dump

2004-02-09 Thread Tom Lane
Michael Brusser [EMAIL PROTECTED] writes: I'm running Postgres v.7.3.4. In my database dump file I see this: CREATE FUNCTION plpgsql_call_handler () RETURNS language_handler AS '/home/tmichael/build/relipg21/syncinc/lib.sol2/plpgsql', 'plpgsql_call_handler' LANGUAGE c; The

Re: [HACKERS] Preventing duplicate vacuums?

2004-02-09 Thread Robert Treat
On Sat, 2004-02-07 at 02:07, Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: Don't know if I would agree for sure, but i the second vacuum could see that it is being blocked by the current vacuum, exiting out would be a bonus, since in most scenarios you don't need to run that second

Re: [HACKERS] [pgsql-hackers-win32] Sync vs. fsync during checkpoint

2004-02-09 Thread Greg Stark
Jan Wieck [EMAIL PROTECTED] writes: The whole sync() vs. fsync() discussion is in my opinion nonsense at this point. Without the ability to limit the amount of files to a reasonable number, by employing tablespaces in the form of larger container files, the risk of forcing excessive head

[HACKERS] BYTE_ORDER for contribs

2004-02-09 Thread strk
Is there a quick way to use the BYTE_ORDER define as set by pgsql ? I can't find an entry point include for it. It's needed for postgis (problems with Solaris BYTE_ORDER). --strk; ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands

Re: [HACKERS] Two-phase commit

2004-02-09 Thread Jeroen T. Vermeulen
On Mon, Feb 09, 2004 at 10:09:34PM +0200, Heikki Linnakangas wrote: However, if this gets into 7.5, I guess you could just check for the version of the backend instead with SELECT version(). Hey, that works? That's very good news, because I was getting a bit worried about all the things I

Re: [HACKERS] CVS HEAD compile failure on Freebsd 4.9

2004-02-09 Thread Vsevolod Lobko
Yes, here too... There are no headers included for struct timeval and function select in miscadmin.h adding #include fixes the problem on freebsd, but I'm sure it's not a portable solution... Index: miscadmin.h === RCS file:

Re: [HACKERS] 7.4.1 release status - Turkish Locale

2004-02-09 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Greg Stark [EMAIL PROTECTED] writes: If it matches an SQL keyword after being downcased the old fashioned way, then it's an SQL keyword. If not then the locale-aware tolower() would be appropriate for tables, columns, etc. That's exactly what we do

Re: [HACKERS] 7.4.1 release status - Turkish Locale

2004-02-09 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: But the example given was SERIAL. serial is an English word, not a Turkish word. It shouldn't really be subject to Turkish locale effects at all. SERIAL is not a keyword according to the grammar. Neither are PUBLIC, VOID, INT4, and numerous other examples.

Re: [HACKERS] Proposed Query Planner TODO items

2004-02-09 Thread Josh Berkus
Jenny, For 19, we moved the common conditions out of the big ORs, for 20, we added distinct. We can change the query back if the optimizer can handle it now. Well, we want to test if it can. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of

Re: [HACKERS] [pgsql-hackers-win32] Sync vs. fsync during checkpoint

2004-02-09 Thread Tom Lane
Jan Wieck [EMAIL PROTECTED] writes: Doing this is not just what you call it. In a system with let's say 500 active backends on a database with let's say 1000 things that are represented as a file, you'll need half a million virtual file descriptors. [shrug] We've been dealing with virtual

Re: [HACKERS] CVS HEAD compile failure on Freebsd 4.9

2004-02-09 Thread Bruce Momjian
Tom Lane wrote: Vsevolod Lobko [EMAIL PROTECTED] writes: Yes, here too... There are no headers included for struct timeval and function select in miscadmin.h Certain parties who shall remain nameless have been gratuitously scattering dependencies on select() into lots of modules that

Re: [HACKERS] CVS HEAD compile failure on Freebsd 4.9

2004-02-09 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: What Win32 has done is to centralize all delay calls around that function, now renamed PG_USLEEP/PG_MSLEEP. I was going to add the missing includes but if you want to centralize it, I think that makes more sense. Yeah, I'm

Re: [HACKERS] Transaction aborts on syntax error.

2004-02-09 Thread Andrej Czapszys
Gavin Sherry wrote: Its not that there's a rationale behind it. Rather, the existing error handling code *has* to abort the current transaction because an error has taken place. In a multi statement transaction block (ie, BEGIN; ...; ...; ... COMMIT;) each statement piggy backs on onto the whole

Re: [PATCHES] [HACKERS] dollar quoting

2004-02-09 Thread Andrew Dunstan
Tom Lane wrote: A bigger problem here: + else if (!dol_quote line[i] == '$' + !isdigit(line[i + thislen]) + (dol_end = strchr(line+i+1,'$')) != NULL + (i == 0 || + ! ((line[i-1] 0x80) !=

Re: [PATCHES] [HACKERS] dollar quoting

2004-02-09 Thread Andrew Dunstan
I think the attached patch addresses Tom's comments. I ended up not using a regex, which seemed to be a little heavy handed, but just writing a small custom recognition function, that should (and I think does) mimic the pattern recognition for these tokens used by the backend lexer. This patch

Re: [HACKERS] Linking the previously separated documentation

2004-02-09 Thread Peter Eisentraut
Michael Glaesemann wrote: If I'm understanding you correctly, that's what I'm doing. Here's an example of the change: Original: Please familiarize yourself with the citerefentryrefentrytitlepg_dump/ reference page. Revised: Please familiarize yourself with the

Re: [HACKERS] Linking the previously separated documentation

2004-02-09 Thread Michael Glaesemann
On Feb 9, 2004, at 6:23 PM, Peter Eisentraut wrote: Michael Glaesemann wrote: If I'm understanding you correctly, that's what I'm doing. Here's an example of the change: Original: Please familiarize yourself with the citerefentryrefentrytitlepg_dump/ reference page. Revised: Please