Re: [HACKERS] massive quotes?

2003-09-11 Thread Andrew Dunstan


Tom Lane wrote:

Andreas Pflug [EMAIL PROTECTED] writes:
 

Bruce Momjian wrote:
   

I assume we never came to a final conclusion on how to do CREATE
FUNCTION without double-quoting.
 

 

Many discussions, but no final conclusion in sight, it seems. That 
\beginliteral stuff is psql centric, where a sql syntax solution is needed.
   

Some people think a sql syntax solution is needed, and some do not.

 

So does this get resolved by a vote?

cheers

andrew

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug

How is that relevant? It's still parseable with parameter placeholders in
place of literal parameters.
*NO PARSING*
The script must be stuffable into PQexec in total, backend does the rest.
 

Presumably \beginliteral \endliteral would be psql's way of specifying
parameters to ship over as parameters.
Again: not psql, but sql language itself must provide this.

It probably is, but that's not what I was thinking of. I was thinking it
wouldn't have to poke around inside the string at all, it would pass it as an
out-of-band parameter using the new FE protocol.
No out-of-band, because this would require splitting the script in pieces.

This helps with DBI too, since you can already do that.

$dbh-do(CREATE FUNCTION foo as ? LANGUAGE SQL, $func);

There may be a huge number of CREATE FUNCTION within the same script!

Regards,
Andreas
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Andrew Dunstan wrote:



Tom Lane wrote:

Some people think a sql syntax solution is needed, and some do not. 

So does this get resolved by a vote?

A  non-sql-syntax solution is useless.

Regards,
Andreas
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Richard Huxton
On Thursday 11 September 2003 09:33, Andreas Pflug wrote:
 *NO PARSING*
 The script must be stuffable into PQexec in total, backend does the rest.

 Again: not psql, but sql language itself must provide this.

 No out-of-band, because this would require splitting the script in pieces.

What's wrong with re-using the COPY FROM format?

CREATE FUNCTION foo(int4) RETURNS int4 BODY FROM stdin LANGUAGE 'plpgsql';
BEGIN
  RAISE NOTICE 'param %',$1;
  RETURN $1;
END;
\.

The only real change is that the body of your function comes after the 
LANGUAGE specification.

Obviously, this is a special case where we are only looking for one value, so 
suppress delimiters on \t and \n.

If that's not liked then something like Perl's qq[] quoting system, which we 
could use anywhere.

Since the options boil down to:
1. Add some syntax to the SQL parser
2. Rely on client capabilities
3. Add syntax to privileged client - psql

I'll volunteer to help out if people decide #1 is the answer. The only places 
this quoting seems to be a real issue are when defining functions, so the 
COPY FROM format above seems to be adequate.

Is there anywhere else people need to quote large blocks of text that will 
almost certainly contain other quotes?

-- 
  Richard Huxton
  Archonet Ltd

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Richard Huxton wrote:

On Thursday 11 September 2003 09:33, Andreas Pflug wrote:
 

*NO PARSING*
The script must be stuffable into PQexec in total, backend does the rest.
Again: not psql, but sql language itself must provide this.

No out-of-band, because this would require splitting the script in pieces.
   

What's wrong with re-using the COPY FROM format?

I must be writing a completely ununderstandable english, sorry for that.

*NO PARSING*, I don't know how to express this differently.

What I'm saying all the time that a single sql script file containing a 
huge number of statements including function creation must be executable 
without parsing it, splitting it into parts and stuffing the data into 
different functions.

When creating a new instance of a database for an app, I don't create 
each object one by one interactively, but use a script for this. To 
create the script, I certainly won't use psql and some arbitrary editor, 
but a frontend allowing for editing and executing that script.

Regards,
Andreas




---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jon Jensen
On Thu, 10 Sep 2003, Doug McNaught wrote:

 But Perl/DBI does escaping for you, so all you'd have to do is:
 
 $sth = $dbh-prepare
(CREATE FUNCTION foo(x text) RETURNS text AS ? LANGUAGE 'plpgsql');
 $sth-execute($function_body);
 
 where $function_body is the unescaped form of the function.  So
 there's no need for a COPY-style mechanism, you can use the current
 CREATE FUNCTION syntax without having to escape everything yourself.

Right; I'm not saying there's no reasonable way to deal with it in DBI
right now. I think the biggest benefit to a block-string quoting mechanism
is for scripts to feed to psql, but would prefer having a consistent SQL
solution that I could use when desired in libpq/DBI for times I build the
entire SQL statement in a string and call it with do().

Jon

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Christopher Browne
[EMAIL PROTECTED] (Matthew T. O'Connor) wrote:
 On Wed, 2003-09-10 at 15:57, Bruce Momjian wrote:
 I assume the attached patch is what you want done to fix this.  Applied.
 
 It quotes table names for vacuum and analyze, and uppercases the
 keywords for clarity.

 Yeah, this is basically what I meant, sorry I didn't get to it quicker. 

 However, I tested it out a little and the patch you made doesn't work
 because it produces commands like:

 VACUUM ANALYZE public.FooBar

 Which doesn't work, so I made my own patch that creates commands like:

 VACUUM ANALYZE public.FooBar

 This allows for mixed case schema names as well as tables.

 Adam, can you please give this a test as you are the person who caught
 the bug in the first place.

Something I am feeling a little suspicious of is that I haven't seen,
in the logs, pg_autovacuum looking at pg_ tables.  

I know that if we don't periodically vacuum such system tables as
pg_class, pg_attribute, pg_statistic, and pg_type, they can get to
pretty evil size.

[Rummaging around...]  These tables are being added for template1, but
apparently not for main databases.  That looks like a bit of a fly
in the ointment...
-- 
(format nil [EMAIL PROTECTED] cbbrowne ntlug.org)
http://www.ntlug.org/~cbbrowne/rdbms.html
Signs  of a   Klingon Programmer  -  16.  Klingon programs   don't do
accountancy. For that, you need a Ferengi.

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] massive quotes?

2003-09-11 Thread Richard Huxton
On Thursday 11 September 2003 10:40, Andreas Pflug wrote:
 Richard Huxton wrote:
 On Thursday 11 September 2003 09:33, Andreas Pflug wrote:
 *NO PARSING*
 The script must be stuffable into PQexec in total, backend does the rest.
 
 Again: not psql, but sql language itself must provide this.
 
 No out-of-band, because this would require splitting the script in
  pieces.
 
 What's wrong with re-using the COPY FROM format?

 I must be writing a completely ununderstandable english, sorry for that.

 *NO PARSING*, I don't know how to express this differently.

I think it's my problem - I thought COPY mytable FROM source_file *was* 
something handled by the SQL parser in the backend, i.e. you could use it via 
psql / DBI / jdbc / whatever.

As it turns out, if the source file is an actual file (/tmp/foo.sql) then you 
can, but not if it's stdin. Presumably, there's some magic going on if you 
use psql/pg_restore (NOTE-should this be mentioned in the docs).

 What I'm saying all the time that a single sql script file containing a
 huge number of statements including function creation must be executable
 without parsing it, splitting it into parts and stuffing the data into
 different functions.

 When creating a new instance of a database for an app, I don't create
 each object one by one interactively, but use a script for this. To
 create the script, I certainly won't use psql and some arbitrary editor,
 but a frontend allowing for editing and executing that script.

I agree completely with you - a single file containing statements that I can 
run into PG via any convenient connection, not something that works one way 
and not another.

-- 
  Richard Huxton
  Archonet Ltd

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Vote: Adding flex/bison derived files in WIN32_DEV

2003-09-11 Thread Matthew T. O'Connor
On Wed, 2003-09-10 at 12:03, Bruce Momjian wrote:
 Because MinGW/Msys doesn't come with flex/bison by default, I have added
 those derived files to the WIN32_DEV branch in CVS.  

I'm confused.  Right on the MinGW download page is a link for
bison-1.875.


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Vote: Adding flex/bison derived files in WIN32_DEV

2003-09-11 Thread Darko Prenosil
On Thursday 11 September 2003 17:30, Matthew T. O'Connor wrote:
 On Wed, 2003-09-10 at 12:03, Bruce Momjian wrote:
  Because MinGW/Msys doesn't come with flex/bison by default, I have added
  those derived files to the WIN32_DEV branch in CVS.

 I'm confused.  Right on the MinGW download page is a link for
 bison-1.875.


Yep, but I had problems with it. Author confirmed that there could be some 
problems creating processes (for example M4). However if You make it work, 
I'll be interested to know how. Check the MinGW mailing list for last few 
weeks, and see what I'm talking about. By the way the last tip of the 
author : reconfigure  make, was not working for me :-(

Regards !

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


[HACKERS] Problem with function permission test in a view

2003-09-11 Thread Bruce Momjian
Someone asked me a question about view and function permissions.  I
assumed all object access done by a view would be based on the
permissions on the view, and not the permissions of the objects.

While table access done in a view follows this outline, function call
access does not.  In my tests below, I create a function ftest(), and
allow only 'postgres' to execute it.  I then create a view and put the
function call in the view, and grant permissions on the view to others. 
However, in the test, you will see that the function call in the view
fails.  

Is this a bug?

---

CREATE USER dummyuser;
CREATE USER
SET SESSION AUTHORIZATION postgres;
SET
CREATE OR REPLACE FUNCTION ftest() RETURNS INTEGER
EXTERNAL SECURITY DEFINER
AS 'SELECT 1' LANGUAGE 'sql';
CREATE FUNCTION
REVOKE EXECUTE ON FUNCTION ftest() FROM PUBLIC;
REVOKE
SELECT ftest();
 ftest
---
 1
(1 row)

SET SESSION AUTHORIZATION dummyuser;
SET
SELECT ftest();
ERROR:  permission denied for function ftest
SET SESSION AUTHORIZATION postgres;
SET
CREATE VIEW vv AS SELECT ftest();
ERROR:  relation vv already exists
SELECT * FROM vv;
 ftest
---
 1
(1 row)

GRANT ALL ON vv TO PUBLIC;
GRANT
SET SESSION AUTHORIZATION dummyuser;
SET
SELECT * FROM vv;
ERROR:  permission denied for function ftest

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Greg Stark
Jon Jensen [EMAIL PROTECTED] writes:

 On Thu, 10 Sep 2003, Doug McNaught wrote:
 
  But Perl/DBI does escaping for you, so all you'd have to do is:

Only because the FE protocol is new and the DBD driver hasn't switched to
using it.

  $sth = $dbh-prepare
 (CREATE FUNCTION foo(x text) RETURNS text AS ? LANGUAGE 'plpgsql');
  $sth-execute($function_body);
  
  where $function_body is the unescaped form of the function.  So
  there's no need for a COPY-style mechanism, you can use the current
  CREATE FUNCTION syntax without having to escape everything yourself.

Well that will only work for as long as DBD actually does do the quoting and
interpolating. Presumably soon the driver will be converted to the new FE
protocol and ship the parameter out-of-band. Will the CREATE FUNCTION handle a
string argument shipping separately like this?

If so then all that has to happen is psql has to have a syntax that allows the
user to specify parameters.

Something like 

CREATE FUNCTION foo(text) RETURNS text as ? LANGUAGE 'plpgsql'
$1EOF
 .
 .
 .
EOF

then plsql would be able to read in the parameters into buffers without having
to dig inside looking for quotes. And execute the query passing the
parameters.


It could even support alternate sources of data for parameters:

CREATE FUNCTION foo(text) RETURNS text as ? LANGUAGE 'plpgsql'
$1'foo.func'
;



Another nice thing about this is that it would help not just psql, but any
front-end using any driver that supports the new FE protocol.

It would also help any other query you want to do with big text parameters.
For example:

INSERT INTO message (header,body) values (?,?)
$1EOF
From: [EMAIL PROTECTED]
EOF

$2EOF
Big long message
EOF
;


I don't see any advantage to inventing a new quoting syntax for sql. In fact
doing it in sql would only increase the amount of parsing psql and other
front-ends would have to do and limit future options. They would still have to
parse to find the end of the statement which is the same parsing they have to
do to pass the arguments as separate parameters.


There's a security issue here too. If the data is already available in
alternate storage, such as in an external file or in a separate variable then
the last thing you want to have to do is interpolate the data into the sql
query only to have the backend parse it out all over again. One bug in the
interpolation or the parsing and you have a security hole.

Consider what happens if you do the above query but somebody passes a text of:


foo
EOF
;
DELETE FROM message
;


If the front-end is shipping it over to the backend whole the backend will
parse it and execute the DELETE statement. If the front-end is shipping it
over as parameters and receives this from a file or from a separate variable,
then it will be inserted as text into the table.


-- 
greg


---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Vote: Adding flex/bison derived files in WIN32_DEV

2003-09-11 Thread Zeugswetter Andreas SB SD

  I'm confused.  Right on the MinGW download page is a link for bison-1.875.
 
 
 Yep, but I had problems with it. Author confirmed that there could be some 
 problems creating processes (for example M4). However if You make it work, 
 I'll be interested to know how. Check the MinGW mailing list for last few 
 weeks, and see what I'm talking about. By the way the last tip of the 
 author : reconfigure  make, was not working for me :-(

Download the bin package at 
http://prdownloads.sf.net/mingw/bison-1.875.0-2003.02.10-1.exe?download
and use it. You do not need to compile bison yourself.

Andreas

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Vote: Adding flex/bison derived files in WIN32_DEV

2003-09-11 Thread Darko Prenosil
On Thursday 11 September 2003 18:00, Zeugswetter Andreas SB SD wrote:
   I'm confused.  Right on the MinGW download page is a link for
   bison-1.875.
 
  Yep, but I had problems with it. Author confirmed that there could be
  some problems creating processes (for example M4). However if You make it
  work, I'll be interested to know how. Check the MinGW mailing list for
  last few weeks, and see what I'm talking about. By the way the last tip
  of the author : reconfigure  make, was not working for me :-(

 Download the bin package at
 http://prdownloads.sf.net/mingw/bison-1.875.0-2003.02.10-1.exe?download
 and use it. You do not need to compile bison yourself.

 Andreas

Did You actually succeed to make output from postgresql yacc definition files 
with that package ? Can  You explain how?

Regards !


---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Andrew Dunstan [EMAIL PROTECTED] writes:
  Tom Lane wrote:
  Some people think a sql syntax solution is needed, and some do not.
  
  So does this get resolved by a vote?
 
 A vote is a little premature when we don't have fully-developed
 alternatives to choose from.  The psql literal proposal is fairly
 squishy around the edges (no one's written down an exact spec), while
 the camp that wants a sql-syntax solution has not gotten further than
 wishing.  The COPY camp is touting their answer as applicable to more
 than psql, but I haven't seen any explanation of exactly how it would be
 useful (or better than existing approaches) in non-psql frontends.  The
 only specific use-case that's been presented for it is psql scripts.
 
 The discussion so far today seems to be entirely a rehash of arguments
 already made (and in many cases already rebutted).  Rather than wasting
 list bandwidth with this, I think each camp ought to go off and do their
 homework.  Give us *details* of how your solution would work.

Another idea would be to enable another set of quoting characters, like:

CREATE FUNCTION xx ...
--
x = 'fred';
...
--

and have the lexer understand those new quoting characters.  We just use
'' too much in function bodies to use that also for quoting the function
text.  Of course, '--' would have no special meaning inside a quoted
string, so we are only eliminating their use as custom operators, and I
think that is reasonable.

Having heard all the other proposals, I think this will be the clearest.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Vote: Adding flex/bison derived files in WIN32_DEV

2003-09-11 Thread Darko Prenosil
On Thursday 11 September 2003 18:08, Darko Prenosil wrote:
 On Thursday 11 September 2003 18:00, Zeugswetter Andreas SB SD wrote:
I'm confused.  Right on the MinGW download page is a link for
bison-1.875.
  
   Yep, but I had problems with it. Author confirmed that there could be
   some problems creating processes (for example M4). However if You make
   it work, I'll be interested to know how. Check the MinGW mailing list
   for last few weeks, and see what I'm talking about. By the way the last
   tip of the author : reconfigure  make, was not working for me :-(
 
  Download the bin package at
  http://prdownloads.sf.net/mingw/bison-1.875.0-2003.02.10-1.exe?download
  and use it. You do not need to compile bison yourself.
 
  Andreas

 Did You actually succeed to make output from postgresql yacc definition
 files with that package ? Can  You explain how?

I forgot to mention that I succeed to make that output, but only from shell, 
not from makefile. Maybe there is something wrong with my MinGW setup ?
However  I do not like Idea that people might give up MinGW bison only because 
I misinformed the group.

Regards !

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Bruce Momjian wrote:

Tom Lane wrote:
 

The discussion so far today seems to be entirely a rehash of arguments
already made (and in many cases already rebutted).  Rather than wasting
list bandwidth with this, I think each camp ought to go off and do their
homework.  Give us *details* of how your solution would work.
   

Another idea would be to enable another set of quoting characters, like:

CREATE FUNCTION xx ...
--
x = 'fred';
...
--
and have the lexer understand those new quoting characters.  We just use
'' too much in function bodies to use that also for quoting the function
text.  Of course, '--' would have no special meaning inside a quoted
string, so we are only eliminating their use as custom operators, and I
think that is reasonable.
Having heard all the other proposals, I think this will be the clearest.

 

This looks quite similar to my proposal. I called it function body is 
enclosed in keywords, while Bruce will enclose it in new quote strings. 
This is obviously very different for the lexer/parser, while identical 
for the user. Sounds good to me.
The quoting strings seem a bit suspicious to me, I can imagine comments 
like this converting abc -- def, which would certainly break the 
function definition (I scanned a part of my sources, I found two 
occurrences of --, one of them in SQL code...)

How about quoting strings that look like keywords, e.g. FUNCTIONBODY and 
ENDFUNCTIONBODY?

CREATE FUNCTION foo() RETURNS int4 AS
FUNCTIONBODY
DECLARE bar int4;
BEGIN
  RETURN bar;
END;
ENDFUNCTIONBODY
LANGUAGE 'plpgsql';
Regards,
Andreas
Regards,
Andreas
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Another idea would be to enable another set of quoting characters, like:

Yeah, I was toying with that also; it would be nearly the same as the
psql literal proposal, but pushed into the backend.  I am not sure what
the quoting symbols should look like though.  '--' will not do, it
conflicts with SQL comments.

Another issue that no one has really addressed is that all these
proposals only solve the basic need to double quotes and backslashes in
function bodies.  Yes, that would be a huge step forward, but if you
look at the situations where the plpgsql quoting recipe recommends six
or eight or ten quotes in a row, it's still gonna be messy.  Seems like
you might want another layer of special quoting within plpgsql to aid
in constructing dynamic SQL commands.  Can that be handled with the same
notation, or do we need another one?  This line of thought suggests that
we might want a convention that supports multiple end-markers, like
shell here-documents do, so that you can nest uses of special quoting.

Perhaps we could do something like this: a special quoted string is
started by a line like
$$FOO
where FOO represents any string of uppercase ASCII letters (we allow
only A-Z here, and nothing else not even whitespace on the line, so
as to minimize the potential for conflicts).  Then the string extends
to the first exactly-matching line.  This would allow nesting:

$$FOO
...
$$BAR
...
$$BAR
...
$$FOO

I'm not by any means wedded to this particular syntax, but it came to
mind as unlikely to conflict with any existing or planned SQL notations.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 
 Tom Lane wrote:
   
 
 The discussion so far today seems to be entirely a rehash of arguments
 already made (and in many cases already rebutted).  Rather than wasting
 list bandwidth with this, I think each camp ought to go off and do their
 homework.  Give us *details* of how your solution would work.
 
 
 
 Another idea would be to enable another set of quoting characters, like:
 
  CREATE FUNCTION xx ...
  --
  x = 'fred';
  ...
  --
 
 and have the lexer understand those new quoting characters.  We just use
 '' too much in function bodies to use that also for quoting the function
 text.  Of course, '--' would have no special meaning inside a quoted
 string, so we are only eliminating their use as custom operators, and I
 think that is reasonable.
 
 Having heard all the other proposals, I think this will be the clearest.
 
   
 
 This looks quite similar to my proposal. I called it function body is 
 enclosed in keywords, while Bruce will enclose it in new quote strings. 
 This is obviously very different for the lexer/parser, while identical 
 for the user. Sounds good to me.
 The quoting strings seem a bit suspicious to me, I can imagine comments 
 like this converting abc -- def, which would certainly break the 
 function definition (I scanned a part of my sources, I found two 
 occurrences of --, one of them in SQL code...)
 
 How about quoting strings that look like keywords, e.g. FUNCTIONBODY and 
 ENDFUNCTIONBODY?
 
 CREATE FUNCTION foo() RETURNS int4 AS
 FUNCTIONBODY
 DECLARE bar int4;
 BEGIN
RETURN bar;
 END;
 ENDFUNCTIONBODY
 LANGUAGE 'plpgsql';

Uh, the problem with long keywords is that you are then requiring the
_parser_ to identify those keywords, and at that point, the entire text
between the keywords has been sliced up by the lexer, which will
certainly make it a mess.  I might be wrong that we can even use more
then two characters for the start of quote string because I don't think
flex supports more than one character of look-ahead.   Again, lookahead
is the key because you have to flag that text as a quoted string before
it gets processed by the lexer.

I was picking -- -- out of the air.  We would have to choose other
character sequences, and probably only two-character ones.  However,
looking at scan.l, it seems you could try putting a literal keyword in
there.  Of course, it would also match 'identifier' code, but if it is
earlier, I think it gets matched first.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Uh, the problem with long keywords is that you are then requiring the
 _parser_ to identify those keywords, and at that point, the entire text
 between the keywords has been sliced up by the lexer, which will
 certainly make it a mess.  I might be wrong that we can even use more
 then two characters for the start of quote string

You're wrong.  We can use anything we like for the start of the quote
string; flex is quite capable of recognizing fixed strings, and even
variable ones.  I'd prefer to avoid expecting it to handle
up/downcasing, I think, but there is no technical reason that the
delimiter couldn't look like a keyword.

My objection to the proposal FUNCTIONBODY is that it makes it look like
the feature is only useful in CREATE FUNCTION.  In point of fact, the
quoting facility could be used to construct any SQL string literal.  The
comparison points I am thinking about are shell here-documents and
Perl quoting conventions, both of which are used for many things.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Uh, the problem with long keywords is that you are then requiring the
  _parser_ to identify those keywords, and at that point, the entire text
  between the keywords has been sliced up by the lexer, which will
  certainly make it a mess.  I might be wrong that we can even use more
  then two characters for the start of quote string
 
 You're wrong.  We can use anything we like for the start of the quote
 string; flex is quite capable of recognizing fixed strings, and even
 variable ones.  I'd prefer to avoid expecting it to handle
 up/downcasing, I think, but there is no technical reason that the
 delimiter couldn't look like a keyword.
 
 My objection to the proposal FUNCTIONBODY is that it makes it look like
 the feature is only useful in CREATE FUNCTION.  In point of fact, the
 quoting facility could be used to construct any SQL string literal.  The
 comparison points I am thinking about are shell here-documents and
 Perl quoting conventions, both of which are used for many things.

Sounds good.  I just think keywords in general are weird to use for
quoting.  We use ' for quoting, so something similar like another
operator combination would be nice.  I have never been fond of the
here-document approach, though I can see the value of doing
here-documents in here-documents, though if we make the open and close
of the quote string different --, --, we can still do that, no?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Problem with function permission test in a view

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Someone asked me a question about view and function permissions.  I
 assumed all object access done by a view would be based on the
 permissions on the view, and not the permissions of the objects.

Table references are checked according to the owner of the view, but use
in a view does not change the execution context for function or operator
calls.  This is how it's always been done.

 Is this a bug?

Changing it would be a major definitional change (and a pretty major
implementation change too).  It might be better, but please don't
pre-judge the issue by labeling it a bug.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Tom Lane wrote:

Bruce Momjian [EMAIL PROTECTED] writes:
 

Uh, the problem with long keywords is that you are then requiring the
_parser_ to identify those keywords, and at that point, the entire text
between the keywords has been sliced up by the lexer, which will
certainly make it a mess.  I might be wrong that we can even use more
then two characters for the start of quote string
   

You're wrong.  We can use anything we like for the start of the quote
string; flex is quite capable of recognizing fixed strings, and even
variable ones.  I'd prefer to avoid expecting it to handle
up/downcasing, I think, but there is no technical reason that the
delimiter couldn't look like a keyword.
My objection to the proposal FUNCTIONBODY is that it makes it look like
the feature is only useful in CREATE FUNCTION.  In point of fact, the
quoting facility could be used to construct any SQL string literal.  The
comparison points I am thinking about are shell here-documents and
Perl quoting conventions, both of which are used for many things.
 

Anyway,
a viable solution seems to be very near, the general direction is quite 
clear.
Problem with pure literal quote strings is that they couldn't be 
immediately adjacent to literal strings, so they should start end end 
with special chars:

$QUOTE1$This is my string$ENDQUOTE1$

Regards,
Andreas




---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] Problem with function permission test in a view

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Someone asked me a question about view and function permissions.  I
  assumed all object access done by a view would be based on the
  permissions on the view, and not the permissions of the objects.
 
 Table references are checked according to the owner of the view, but use
 in a view does not change the execution context for function or operator
 calls.  This is how it's always been done.
 
  Is this a bug?
 
 Changing it would be a major definitional change (and a pretty major
 implementation change too).  It might be better, but please don't
 pre-judge the issue by labeling it a bug.

Well, it sure sounds like a bug.  What logic is there that table access
use the view permissions, but not function access?  Could we just use
SECURITY DEFINER for function calls in views?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Adam Kavan
At 12:03 AM 9/11/03 -0400, Matthew T. O'Connor wrote:
Adam, can you please give this a test as you are the person who caught
the bug in the first place.
Thanks,

Matthew T. O'Connor
I applied your patch and it works fine for me.

--- Adam Kavan 

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Andreas Pflug wrote:
 Bruce Momjian wrote:
 
 You mean if the special quotes are -- and --, - would be the
 same as '-'?
   
 
 
 If they work as the standard ' quote (and that seems to be Toms 
 intention), obviously.
 
 Besides, we have to care specially about --. Remember the complaints about
 select 1--1,
 behaving differently between MySQL and PostgreSQL. -- should remain 
 comment under all circumstances.
 
 Too sad, all special chars are used up for operators

Oh, I never intended --, -- to be used --- they were just examples.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Sergio A. Kessler wrote:


Too sad, all special chars are used up for operators


also '{' '}' are used ?

I've only seen this in ACLs, so it might be usable. Tom, Bruce?

Regards,
Andreas
---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Andreas Pflug wrote:
 Sergio A. Kessler wrote:
 
 
 
  Too sad, all special chars are used up for operators
 
 
  also '{' '}' are used ?
 
 I've only seen this in ACLs, so it might be usable. Tom, Bruce?

Something that includes ' would be clearest. I thought of ' and ',
but this would break:

if var 'yes'

I think {' and '} might work.  Arrays are specified as '{val, val}',
which is safe because it is opposite of the suggested syntax.  I can't
think of any case where you would use an opening brace, then a single
quote.  Interestingly, it looks like a C braces:

CREATE FUNCTION test() ...
{'
x = 'no';
'}

Of course, this brings up a problem.  What if we do:


CREATE FUNCTION test() ...
{'
x = '}text';
'}

Oops, two closing mega-quotes.

One clean way would be to use {' to start a quote, and }' to end it, so
we have:


CREATE FUNCTION test() ...
{'
x = 'text';
}'

which looks even better and this is safe because both braces in '}text}'
are seen in a quoted string:

CREATE FUNCTION test() ...
{'
x = '}text}';
}'

Also, I can't imagine anyone defining those as operators.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Bruce Momjian

Patch applied.  You might want to look at pg_dump/dumputils.c::fmtId()
for a function that does smart quoting.

---

Matthew T. O'Connor wrote:
 On Wed, 2003-09-10 at 15:57, Bruce Momjian wrote:
  I assume the attached patch is what you want done to fix this.  Applied.
  
  It quotes table names for vacuum and analyze, and uppercases the
  keywords for clarity.
 
 Yeah, this is basically what I meant, sorry I didn't get to it quicker. 
 
 However, I tested it out a little and the patch you made doesn't work
 because it produces commands like:
 
 VACUUM ANALYZE public.FooBar
 
 Which doesn't work, so I made my own patch that creates commands like:
 
 VACUUM ANALYZE public.FooBar
 
 This allows for mixed case schema names as well as tables.
 
 Adam, can you please give this a test as you are the person who caught
 the bug in the first place.
 
 Thanks, 
 
 Matthew T. O'Connor
 

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 3: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andrew Dunstan
Bruce Momjian wrote:

Something that includes ' would be clearest. I thought of ' and ',
but this would break:
 

I'm not sure that using a quote is necessarily clearest. But it's a 
matter of taste. I had thought of {{ and }} as maybe working.

[snip]

One clean way would be to use {' to start a quote, and }' to end it, so
we have:
CREATE FUNCTION test() ...
{'
x = 'text';
}'
which looks even better and this is safe because both braces in '}text}'
are seen in a quoted string:
CREATE FUNCTION test() ...
{'
x = '}text}';
}'
Also, I can't imagine anyone defining those as operators.

 

Quite cute. I like it better than the here-document style.

Is this proposed as a general quoting mechanism, or only in the context 
of create function? (I favor a general mechanism, if that matters :-) 
- I got caught three times in the last 2 weeks with embedded quotes in 
comment on statements.)

cheers

andrew

---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Something that includes ' would be clearest. I thought of ' and ',
 but this would break:
   if var 'yes'

People seem to be assuming that this feature needs to be impervious to
whitespace and being adjacent to other things.  I believe we could make
it a good deal more robust if both the opening and closing markers
(whatever they are) are required to stand alone on a line.  For example

{'
}'

represents the empty string, and

{'
x = 'text';
}'

represents '\tx = \'text\';'.  I think the rule would need to be that
the newline just after the opening marker, and the newline just before
the closing marker, are not included in the resulting string literal.

The point is we don't have to make case like {'xxx}' be recognized as an
occurrence of this construct, and on the whole I think it's a lot safer
if we don't.  Keep in mind that the string you are trying to quote is
often code in a language that is not SQL.  Therefore, choosing markers
on the grounds that they won't appear in SQL is not good enough.

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jon Jensen
On Thu, 11 Sep 2003, Tom Lane wrote:

 People seem to be assuming that this feature needs to be impervious to
 whitespace and being adjacent to other things.  I believe we could make
 it a good deal more robust if both the opening and closing markers
 (whatever they are) are required to stand alone on a line.  For example
 
   {'
   }'
 
 represents the empty string, and
 
   {'
   x = 'text';
   }'
 
 represents '\tx = \'text\';'.  I think the rule would need to be that
 the newline just after the opening marker, and the newline just before
 the closing marker, are not included in the resulting string literal.
 
 The point is we don't have to make case like {'xxx}' be recognized as an
 occurrence of this construct, and on the whole I think it's a lot safer
 if we don't.  Keep in mind that the string you are trying to quote is
 often code in a language that is not SQL.  Therefore, choosing markers
 on the grounds that they won't appear in SQL is not good enough.

Agreed. But I think it would be very nice to not invent yet another block
quoting mechanism if possible. I don't understand the resistance to here
documents, since they solve the problem Tom brought up -- the token will
by definition never be a problem because it's alone on a line, and the
user chooses it and can avoid problematic ones. Or to combine COPY style
(but not its under-the-hood operation) with here documents, why not:

INSERT INTO sometable (5, \.
a
very long
string
\.
);

Is there a reason not to use here documents?

Jon

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] Problem with function permission test in a view

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Well, it sure sounds like a bug.  What logic is there that table access
 use the view permissions, but not function access?  Could we just use
 SECURITY DEFINER for function calls in views?

You're confusing two distinct questions, I think.  One is how we decide
whether it's allowed to call the function (ie, whose USAGE right do we
check).  The other is whose ID does the function run under.  If the
function is SECURITY DEFINER then the second question is determined by
the function itself, but otherwise it's not; and in any case SECURITY
DEFINER doesn't speak to the first question.

A case can be made that the answer to both questions should be the view
owner, rather than the view user as it is now, but I am not sure that
this is an open-and-shut issue.  Both the user and the owner can affect
whether such a function gets called and what arguments it gets called
with (maybe not so much in a view rule, but definitely in ON INSERT and
other action rules).  It seems possible that we'd just be switching from
one set of security issues to another.

regards, tom lane

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andrew Dunstan
Tom Lane wrote:

I believe we could make
it a good deal more robust if both the opening and closing markers
(whatever they are) are required to stand alone on a line.  

Hard to detect whitespace might trip things up. I wish I had a $ for 
every time that has made my life difficult.

cheers

andrew

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 I believe we could make
 it a good deal more robust if both the opening and closing markers
 (whatever they are) are required to stand alone on a line.  
 
 Hard to detect whitespace might trip things up. I wish I had a $ for 
 every time that has made my life difficult.

I could accept a definition that allowed trailing whitespace on the same
line as the marker, but not any other visible characters.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Matthew T. O'Connor
On Thu, 2003-09-11 at 15:02, Bruce Momjian wrote:
 Patch applied.  You might want to look at pg_dump/dumputils.c::fmtId()
 for a function that does smart quoting.

OK, thanks.


---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Matthew T. O'Connor
On Thu, 2003-09-11 at 08:12, Christopher Browne wrote:
 Something I am feeling a little suspicious of is that I haven't seen,
 in the logs, pg_autovacuum looking at pg_ tables.  
 
 I know that if we don't periodically vacuum such system tables as
 pg_class, pg_attribute, pg_statistic, and pg_type, they can get to
 pretty evil size.
 
 [Rummaging around...]  These tables are being added for template1, but
 apparently not for main databases.  That looks like a bit of a fly
 in the ointment...

I designed it that way.  It was my understanding that all of the system
tables pg_class etc... are shared tables, available in all databases,
but actually stored as only one central set of real tables.  Hence
vacuuming pg_class from template1 helps every database that accesses
pg_class.

Did I make a design error?



---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Jon Jensen [EMAIL PROTECTED] writes:
 Is there a reason not to use here documents?

The $$FOO proposal I put forward earlier was consciously modeled on
here-documents.  We cannot use exactly the shell syntax for
here-documents, though, mainly because we already have meaning assigned
to strings like ' ( is already a standard operator, and the ' could
be the start of an ordinary literal).

I would definitely like to see us adopt a proposal that is like
here-documents to the extent that there's a family of possible
terminator markers and not only one.  But we'll have to adjust the
syntax a little bit.  If you don't like $$FOO, what else comes to mind?

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Bruce Momjian
Matthew T. O'Connor wrote:
 On Thu, 2003-09-11 at 08:12, Christopher Browne wrote:
  Something I am feeling a little suspicious of is that I haven't seen,
  in the logs, pg_autovacuum looking at pg_ tables.  
  
  I know that if we don't periodically vacuum such system tables as
  pg_class, pg_attribute, pg_statistic, and pg_type, they can get to
  pretty evil size.
  
  [Rummaging around...]  These tables are being added for template1, but
  apparently not for main databases.  That looks like a bit of a fly
  in the ointment...
 
 I designed it that way.  It was my understanding that all of the system
 tables pg_class etc... are shared tables, available in all databases,
 but actually stored as only one central set of real tables.  Hence
 vacuuming pg_class from template1 helps every database that accesses
 pg_class.
 
 Did I make a design error?

Oops, no.  Only a few pg_* tables are global.  pg_class isn't.  In
fact, I am not sure how someone tells which are global.   A grep in
/src/include/catalog shows:

$ grep BKI_SHARED_RELATION *.h
pg_database.h:CATALOG(pg_database) BOOTSTRAP BKI_SHARED_RELATION
pg_group.h:CATALOG(pg_group) BOOTSTRAP BKI_SHARED_RELATION BKI_WITHOUT_OIDS
pg_shadow.h:CATALOG(pg_shadow) BOOTSTRAP BKI_SHARED_RELATION BKI_WITHOUT_OIDS

so those are the only shared ones.  I found a query to do it too:

test= select * from pg_class where relisshared = 't' and relkind = 'r';
   relname   | relnamespace | reltype | relowner | relam | relfilenode | 
relpages | reltuples | reltoastrelid | reltoastidxid | relhasindex | relisshared | 
relkind | relnatts | relchecks | reltriggers | relukeys | relfkeys | relrefs | 
relhasoids | relhaspkey | relhasrules | relhassubclass |   relacl

-+--+-+--+---+-+--+---+---+---+-+-+-+--+---+-+--+--+-+++-++
 pg_shadow   |   11 |  86 |1 | 0 |1260 |   
 1 | 1 | 16677 | 0 | t   | t   | r   | 
   8 | 0 |   1 |0 |0 |   0 | f  | f
  | f   | f  | {postgres=a*r*w*d*R*x*t*/postgres}
 pg_database |   11 |  88 |1 | 0 |1262 |   
 1 | 2 | 16662 | 0 | t   | t   | r   | 
  11 | 0 |   0 |0 |0 |   0 | t  | f
  | f   | f  | {=r/postgres}
 pg_group|   11 |  87 |1 | 0 |1261 |   
 0 | 0 | 16668 | 0 | t   | t   | r   | 
   3 | 0 |   1 |0 |0 |   0 | f  | f
  | f   | f  | {=r/postgres}
(3 rows)

so those are the only ones that should be template1-only.  All other
pg_* tables should be vacuumed in individual database.

I will wait for a patch.  :-)

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jan Wieck


Bruce Momjian wrote:

Sounds good.  I just think keywords in general are weird to use for
quoting.  We use ' for quoting, so something similar like another
operator combination would be nice.  I have never been fond of the
here-document approach, though I can see the value of doing
here-documents in here-documents, though if we make the open and close
of the quote string different --, --, we can still do that, no?
The beauty of here-documents is that you specify your closing tag on a 
per usage base and can vary that depending on the content you need to 
enclose. Keep in mind that this literal mechanism is not only used for 
PL/pgSQL, but for other languages like PL/Tcl and PL/Perl as well. 
Imagine the pain for a Tcl programmer if we'd go with curly braces!

Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andreas Pflug
Tom Lane wrote:

Andrew Dunstan [EMAIL PROTECTED] writes:
 

Tom Lane wrote:
   

I believe we could make
it a good deal more robust if both the opening and closing markers
(whatever they are) are required to stand alone on a line.  

 

Hard to detect whitespace might trip things up. I wish I had a $ for 
every time that has made my life difficult.
   

I could accept a definition that allowed trailing whitespace on the same
line as the marker, but not any other visible characters.
 

This contradicts a wish you made earlier, using this extended quoting 
for further reducing quote-quoting. IMHO, such a single-line quote 
marker would be totally sufficient, it would reduce  to , 
and if you write \'\' (what I prefer) it gets even more distinguishable 
(actually, this is the way pgAdmin3's function wizard works).
But if we're back to a single line, it's somehow like a keyword, 
separated by space-chars. FUNCTIONBODY might be a bit function-centric, 
so how about QUOTE and ENDQUOTE? In case another quote level makes 
sense, it could be QUOTE(1) and ENDQUOTE(1), or only a pair of QUOTE(n).

Anyway, I think such a here-documents or single line quote marker 
solution would be a great enhancement, details are now merely a question 
of taste.

While this sounds nearly done for me, as there seems some fundamental 
consense, I'll will probably have to wait for 7.5?

Regards,
Andreas


---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
  http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Tom Lane
Matthew T. O'Connor [EMAIL PROTECTED] writes:
 On Thu, 2003-09-11 at 08:12, Christopher Browne wrote:
 [Rummaging around...]  These tables are being added for template1, but
 apparently not for main databases.  That looks like a bit of a fly
 in the ointment...

 I designed it that way.  It was my understanding that all of the system
 tables pg_class etc... are shared tables, available in all databases,
 but actually stored as only one central set of real tables.

You are very badly mistaken.

Only the tables marked relisshared in pg_class (currently pg_shadow,
pg_group, pg_database, and their indexes and toast tables) are shared
across a cluster.  The rest have separate copies per-database.

regards, tom lane

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Andreas Pflug wrote:
 solution would be a great enhancement, details are now merely a question 
 of taste.
 
 While this sounds nearly done for me, as there seems some fundamental 
 consense, I'll will probably have to wait for 7.5?

Yes, has to wait for 7.5.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Jan Wieck wrote:
 
 
 Bruce Momjian wrote:
 
  Sounds good.  I just think keywords in general are weird to use for
  quoting.  We use ' for quoting, so something similar like another
  operator combination would be nice.  I have never been fond of the
  here-document approach, though I can see the value of doing
  here-documents in here-documents, though if we make the open and close
  of the quote string different --, --, we can still do that, no?
  
 
 The beauty of here-documents is that you specify your closing tag on a 
 per usage base and can vary that depending on the content you need to 
 enclose. Keep in mind that this literal mechanism is not only used for 
 PL/pgSQL, but for other languages like PL/Tcl and PL/Perl as well. 
 Imagine the pain for a Tcl programmer if we'd go with curly braces!

Oh, non-SQL languages.  OK, I agree, we need HERE documents, and I think
we all agree it has to be done in the lexer.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Jon Jensen [EMAIL PROTECTED] writes:
  Is there a reason not to use here documents?
 
 The $$FOO proposal I put forward earlier was consciously modeled on
 here-documents.  We cannot use exactly the shell syntax for
 here-documents, though, mainly because we already have meaning assigned
 to strings like ' ( is already a standard operator, and the ' could
 be the start of an ordinary literal).
 
 I would definitely like to see us adopt a proposal that is like
 here-documents to the extent that there's a family of possible
 terminator markers and not only one.  But we'll have to adjust the
 syntax a little bit.  If you don't like $$FOO, what else comes to mind?

Couldn't we allow  at the beginning of the line to mean 'here' document?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jan Wieck
Bruce Momjian wrote:

Jan Wieck wrote:


Bruce Momjian wrote:

 Sounds good.  I just think keywords in general are weird to use for
 quoting.  We use ' for quoting, so something similar like another
 operator combination would be nice.  I have never been fond of the
 here-document approach, though I can see the value of doing
 here-documents in here-documents, though if we make the open and close
 of the quote string different --, --, we can still do that, no?
 

The beauty of here-documents is that you specify your closing tag on a 
per usage base and can vary that depending on the content you need to 
enclose. Keep in mind that this literal mechanism is not only used for 
PL/pgSQL, but for other languages like PL/Tcl and PL/Perl as well. 
Imagine the pain for a Tcl programmer if we'd go with curly braces!
Oh, non-SQL languages.  OK, I agree, we need HERE documents, and I think
we all agree it has to be done in the lexer.
Not only. psql by default sends the edit buffer when it encounters the 
;. You don't want that to happen inside of the here document. So it 
has to duplicate that logic.

Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #
---(end of broadcast)---
TIP 6: Have you searched our list archives?
  http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 The $$FOO proposal I put forward earlier was consciously modeled on
 here-documents.

 Couldn't we allow  at the beginning of the line to mean 'here' document?

No; you could easily be breaking existing queries, for example

regression=# select f1, f1  8 from int4_tbl;
 f1  | ?column?
-+---
   0 | 0
  123456 |  31604736
 -123456 | -31604736
  2147483647 |  -256
 -2147483647 |   256
(5 rows)

is only one unfortunate choice of line break away from being eaten by
such a proposal.

I suggested $$ because AFAIK we don't currently have any valid syntax
that would allow that to appear at the start of a line (it helps a great
deal that we just removed $ from the set of characters allowed in
operators ;-)).  If you consider my $$FOO proposal to include the
possibility of a zero-length FOO string, then the shortest legal
alternative would be

$$
string contents here
$$

but adding a string to that reduces the odds of conflict (especially
when you consider languages like plperl; IIRC, $$ is something or other
useful in Perl).  Also you can use mnemonically-chosen strings; maybe
Andreas will like

$$FUNCTIONBODY
text here
$$FUNCTIONBODY

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Jan Wieck wrote:
  The beauty of here-documents is that you specify your closing tag on a 
  per usage base and can vary that depending on the content you need to 
  enclose. Keep in mind that this literal mechanism is not only used for 
  PL/pgSQL, but for other languages like PL/Tcl and PL/Perl as well. 
  Imagine the pain for a Tcl programmer if we'd go with curly braces!
  
  Oh, non-SQL languages.  OK, I agree, we need HERE documents, and I think
  we all agree it has to be done in the lexer.
  
 
 Not only. psql by default sends the edit buffer when it encounters the 
 ;. You don't want that to happen inside of the here document. So it 
 has to duplicate that logic.

Yep, good point.  It already understands quoting and comments, so it
will have to understand this too.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Tom Lane wrote:
  The $$FOO proposal I put forward earlier was consciously modeled on
  here-documents.
 
  Couldn't we allow  at the beginning of the line to mean 'here' document?
 
 No; you could easily be breaking existing queries, for example

Yes, that is true.  I didn't like the beginning-of-line requirement for
here documents for that reason.  However, we are already requiring the
thing to be at the beginning of the line.  You are saying it is safer to
make it at the beginnning of a line _and_ have it be something that
isn't used in SQL, but $$ is used in Perl, so I don't see the big
advantage either way --- once you say X has to begin at the beginning of
the line, we are already making things breakable by a newline, no?

You could make the point that using $$ makes only the text inside the
quote as newline-sensitive, while using  makes any SQL
newline-sensitive, and that might be the major advantage of something
like $$.

To me, however, the dollar sign is just too overloaded for function
arguments, which you will likely see in the function text.

 I suggested $$ because AFAIK we don't currently have any valid syntax
 that would allow that to appear at the start of a line (it helps a great
 deal that we just removed $ from the set of characters allowed in
 operators ;-)).  If you consider my $$FOO proposal to include the
 possibility of a zero-length FOO string, then the shortest legal
 alternative would be
 
   $$
   string contents here
   $$
 
 but adding a string to that reduces the odds of conflict (especially
 when you consider languages like plperl; IIRC, $$ is something or other
 useful in Perl).  Also you can use mnemonically-chosen strings; maybe

Of course, every operator combination is used by Perl.  :-)

 Andreas will like

   $$FUNCTIONBODY
   text here
   $$FUNCTIONBODY

So you are requiring the identical text to appear at the beginning and
end of the quote, rather than a here document that would be:

END
...
END

or in your example:

$$END
...
END

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Christopher Browne
[EMAIL PROTECTED] (Matthew T. O'Connor) writes:
 On Thu, 2003-09-11 at 08:12, Christopher Browne wrote:
 Something I am feeling a little suspicious of is that I haven't seen,
 in the logs, pg_autovacuum looking at pg_ tables.  
 
 I know that if we don't periodically vacuum such system tables as
 pg_class, pg_attribute, pg_statistic, and pg_type, they can get to
 pretty evil size.
 
 [Rummaging around...]  These tables are being added for template1, but
 apparently not for main databases.  That looks like a bit of a fly
 in the ointment...

 I designed it that way.  It was my understanding that all of the system
 tables pg_class etc... are shared tables, available in all databases,
 but actually stored as only one central set of real tables.  Hence
 vacuuming pg_class from template1 helps every database that accesses
 pg_class.

 Did I make a design error?

[rummaging around...  Where's a suitable system?  There it is...]

database3=# vacuum verbose pg_class;
NOTICE:  --Relation pg_class--
NOTICE:  Index pg_class_oid_index: Pages 11399; Tuples 165: Deleted 1408.
CPU 2.00s/0.41u sec elapsed 2.51 sec.
NOTICE:  Index pg_class_relname_index: Pages 30604; Tuples 171: Deleted 1408.
CPU 5.81s/1.07u sec elapsed 7.07 sec.
NOTICE:  Removed 1408 tuples in 24 pages.
CPU 0.01s/0.02u sec elapsed 0.20 sec.
NOTICE:  Pages 360: Changed 1, Empty 0; Tup 165: Vac 1408, Keep 0, UnUsed 21110.
Total CPU 7.82s/1.50u sec elapsed 9.79 sec.
VACUUM
database3=# \c database1
You are now connected to database database1
database1=# vacuum verbose pg_class;
NOTICE:  --Relation pg_class--
NOTICE:  Pages 22768: Changed 0, Empty 0; Tup 158: Vac 0, Keep 0, UnUsed 1434149.
Total CPU 1.35s/0.97u sec elapsed 2.38 sec.
VACUUM
database1=# \c template1
You are now connected to database template1.
template1=# vacuum verbose pg_class;
NOTICE:  --Relation pg_class--
NOTICE:  Pages 2: Changed 0, Empty 0; Tup 101: Vac 0, Keep 0, UnUsed 27.
Total CPU 0.00s/0.00u sec elapsed 0.00 sec.
VACUUM

No, they aren't shared.  (And why yes, some of those databases could
indeed use more regular vacuuming. :-))

Note, that was a 7.2.4 database, so pg_autovacuum won't work there
yet.  C'est la vie.

At any rate, template1.pg_class seems pretty distinct from
database1.pg_class which is distinct from database3.pg_class.  I think
there's a bit of an error there.
-- 
(reverse (concatenate 'string ofni.smrytrebil @ enworbbc))
http://dev6.int.libertyrms.com/
Christopher Browne
(416) 646 3304 x124 (land)

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jon Jensen
On Thu, 11 Sep 2003, Bruce Momjian wrote:

  I would definitely like to see us adopt a proposal that is like
  here-documents to the extent that there's a family of possible
  terminator markers and not only one.  But we'll have to adjust the
  syntax a little bit.  If you don't like $$FOO, what else comes to mind?
 
 Couldn't we allow  at the beginning of the line to mean 'here' document?

Even if that worked, it diverges so much from shell and Perl here document 
syntax (where it's customary to have it at the end of the line, or even in 
the middle), it wouldn't be any more familiar than anything else.

I think Tom's $$FOO suggestion is fine -- it just takes getting used to.

Jon

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Yes, that is true.  I didn't like the beginning-of-line requirement for
 here documents for that reason.  However, we are already requiring the
 thing to be at the beginning of the line.  You are saying it is safer to
 make it at the beginnning of a line _and_ have it be something that
 isn't used in SQL, but $$ is used in Perl, so I don't see the big
 advantage either way --- once you say X has to begin at the beginning of
 the line, we are already making things breakable by a newline, no?

But as Jan pointed out, you can choose your string so as not to conflict
with whatever is in the text to be quoted.  So you'd probably *not* use
plain $$ as marker if working with a plperl function.  $$FUNCTION would
have a reasonably good chance of not conflicting, and if by some
chance it did match a line you want in the text, you pick another.

It is possible that we could allow the start marker to be not at the
beginning of its line, which would create structures very very close
to shell here-documents:

CREATE FUNCTION foo() RETURNS int AS $$FUNCTION
... text here ...
$$FUNCTION

This would be a little more open to typos --- if you leave out the space
so that it reads

CREATE FUNCTION foo() RETURNS int AS$$FUNCTION

then you wrote an identifier, not AS followed by a here-document marker.
But it might be worth defining it that way anyway because of the
similarity to shell notation.  Or maybe I'm assuming too much about
whether the average SQL programmer has ever heard of shell
here-documents.

regards, tom lane

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 So you are requiring the identical text to appear at the beginning and
 end of the quote, rather than a here document that would be:

   END
   ...
   END

 or in your example:

   $$END
   ...
   END

Yes, I was thinking of requiring the $$ to appear at both beginning and
end.  This is perhaps not critical, but it seems more symmetric that
way.  Also, we might even be able to get away with allowing SQL text to
resume on the same line as the terminator, for example

CREATE FUNCTION foo() RETURNS int AS $$FUNCTION
... text here ...
$$FUNCTION LANGUAGE plpgsql;

I would not want to risk that with a plain word as terminator, but $$
helps a lot to make it distinctive.

regards, tom lane

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Jon Jensen wrote:
 On Thu, 11 Sep 2003, Bruce Momjian wrote:
 
   I would definitely like to see us adopt a proposal that is like
   here-documents to the extent that there's a family of possible
   terminator markers and not only one.  But we'll have to adjust the
   syntax a little bit.  If you don't like $$FOO, what else comes to mind?
  
  Couldn't we allow  at the beginning of the line to mean 'here' document?
 
 Even if that worked, it diverges so much from shell and Perl here document 
 syntax (where it's customary to have it at the end of the line, or even in 
 the middle), it wouldn't be any more familiar than anything else.
 
 I think Tom's $$FOO suggestion is fine -- it just takes getting used to.

Oh, sorry, I forgot 'here' documents don't have to start the line.  I
was looking at the manual page and not thinking of the code I write with
here documents.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tilo Schwarz
Bruce Momjian writes:
 Tom Lane wrote:
  Jon Jensen [EMAIL PROTECTED] writes:
   Is there a reason not to use here documents?
 
  The $$FOO proposal I put forward earlier was consciously modeled on
  here-documents.  We cannot use exactly the shell syntax for
  here-documents, though, mainly because we already have meaning assigned
  to strings like ' ( is already a standard operator, and the ' could
  be the start of an ordinary literal).
 
  I would definitely like to see us adopt a proposal that is like
  here-documents to the extent that there's a family of possible
  terminator markers and not only one.  But we'll have to adjust the
  syntax a little bit.  If you don't like $$FOO, what else comes to mind?

 Couldn't we allow  at the beginning of the line to mean 'here' document?

What about the Python approach: The literal text is enclosed either in a pair 
of three single quotes or three double quotes. So you can do (e.g. in the 
python shell)

 s = 
This is
'one'
string.


to set string s or 

 s = '''
This is
'one'
string.
'''

Even 

 s = 
This is 'one'
string with '''triple single quotes'''
and some double quotes.

works:

 print s

This is 'one'
string with '''triple single quotes'''
and some double quotes.

Bye,

Tilo

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] massive quotes?

2003-09-11 Thread Jon Jensen
On Thu, 11 Sep 2003, Tom Lane wrote:

 It is possible that we could allow the start marker to be not at the
 beginning of its line, which would create structures very very close
 to shell here-documents:
 
   CREATE FUNCTION foo() RETURNS int AS $$FUNCTION
   ... text here ...
   $$FUNCTION
 
 This would be a little more open to typos --- if you leave out the space
 so that it reads
 
   CREATE FUNCTION foo() RETURNS int AS$$FUNCTION
 
 then you wrote an identifier, not AS followed by a here-document marker.
 But it might be worth defining it that way anyway because of the
 similarity to shell notation.  Or maybe I'm assuming too much about
 whether the average SQL programmer has ever heard of shell
 here-documents.

Perl's adopting shell here-document syntax has made them pretty well-known
even outside of Unix, but if we diverge at all we'd might as well do what 
makes the most sense for PostgreSQL. I think your proposal above is good, 
at least enough to write code for and test with.

Jon

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Matthew T. O'Connor
On Thu, 2003-09-11 at 17:11, Tom Lane wrote:
 Matthew T. O'Connor [EMAIL PROTECTED] writes:
  I designed it that way.  It was my understanding that all of the system
  tables pg_class etc... are shared tables, available in all databases,
  but actually stored as only one central set of real tables.
 
 You are very badly mistaken.
 
 Only the tables marked relisshared in pg_class (currently pg_shadow,
 pg_group, pg_database, and their indexes and toast tables) are shared
 across a cluster.  The rest have separate copies per-database.

hrm OK.  Patch forthcoming



---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Tilo Schwarz [EMAIL PROTECTED] writes:
 What about the Python approach: The literal text is enclosed either in a pair
 of three single quotes or three double quotes.

That might be okay if we were working in a vacuum, but we aren't.  Among
other things, the SQL spec tells us what 'quote''' ... means, and it's
not Python-compatible.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Tom Lane
Matthew T. O'Connor [EMAIL PROTECTED] writes:
 On Thu, 2003-09-11 at 17:11, Tom Lane wrote:
 Matthew T. O'Connor [EMAIL PROTECTED] writes:
 I designed it that way.  It was my understanding that all of the system
 tables pg_class etc... are shared tables, available in all databases,
 but actually stored as only one central set of real tables.
 
 Only the tables marked relisshared in pg_class (currently pg_shadow,
 pg_group, pg_database, and their indexes and toast tables) are shared
 across a cluster.  The rest have separate copies per-database.

 hrm OK.  Patch forthcoming

BTW, I am not sure it is a good idea to suppress redundant vacuuming
of shared tables in the first place.  The trouble with doing so is that
if you only vacuum pg_shadow through template1, then only template1 will
ever have up-to-date statistics about it.  That's not good.

You might be able to get away with doing actual vacuums only through
template1, and doing just ANALYZEs every so often in other DBs.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] massive quotes?

2003-09-11 Thread Andrew Dunstan


Tom Lane wrote:

Bruce Momjian [EMAIL PROTECTED] writes:
 

So you are requiring the identical text to appear at the beginning and
end of the quote, rather than a here document that would be:
   

 

	END
	...
	END
   

 

or in your example:
   

 

	$$END
	...
	END
   

Yes, I was thinking of requiring the $$ to appear at both beginning and
end.  This is perhaps not critical, but it seems more symmetric that
way.  Also, we might even be able to get away with allowing SQL text to
resume on the same line as the terminator, for example
CREATE FUNCTION foo() RETURNS int AS $$FUNCTION
... text here ...
$$FUNCTION LANGUAGE plpgsql;
I would not want to risk that with a plain word as terminator, but $$
helps a lot to make it distinctive.
 

This discussion now seems to be on the right track - thanks for 
recognising a need, meeting which will clearly improve the useability of 
the product.

cheers

andrew

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Another small bug (pg_autovacuum)

2003-09-11 Thread Matthew T. O'Connor
On Thu, 2003-09-11 at 18:25, Tom Lane wrote:
 BTW, I am not sure it is a good idea to suppress redundant vacuuming
 of shared tables in the first place.  The trouble with doing so is that
 if you only vacuum pg_shadow through template1, then only template1 will
 ever have up-to-date statistics about it.  That's not good.
 
 You might be able to get away with doing actual vacuums only through
 template1, and doing just ANALYZEs every so often in other DBs.

ok I will see what I can do about that.  So I assume that the vacuumdb
script handle this just does redundant vacuums / analyzes on shared
tables so that it doesn't have a problem with this.

If we can supress redundant vacuuming I think that would be a good
thing as pg_autovacuum is supposed to make the required vacuuming as
efficient as possible.


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 ... You are saying it is safer to
 make it at the beginnning of a line _and_ have it be something that
 isn't used in SQL, but $$ is used in Perl, so I don't see the big
 advantage either way --- once you say X has to begin at the beginning of
 the line, we are already making things breakable by a newline, no?

Keep in mind that we have two different requirements: the quote start
marker has to be recognizable while we are parsing SQL (or possibly
plpgsql) code.  The quote end marker has to be recognizable while we are
scanning text that could be almost anything.

The cute thing about the here-document solution to this problem is that
you can choose the quote end marker on a case-by-case basis.  So you can
always pick something that won't conflict with anything that's actually
in the text you need to quote.

If we try to go with fixed markers (like {' ... }' and some other ideas
that were floated today), then we lose that flexibility, and we're up
against the losing game of trying to pick an end-marker that won't cause
problems in any programming language anywhere (not to mention possible
uses of the quoting mechanism for arbitrary string literals that aren't
even function bodies).  

I'm not wedded to the $$FOO idea in particular, but I do think we want
to go with a solution that allows a variable end-marker.

regards, tom lane

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Win32 native port

2003-09-11 Thread Bruce Momjian

Below is the email that prompted me to add the derived files to
WIN32_DEV CVS.

However, most people don't want them in there, so I have removed them,
and updated the web page to recommend the nightly snapshots (which have
the derived files), and mentioned the tools that will be needed for a
CVS build:

http://momjian.postgresql.org/main/writings/pgsql/win32.html


---

steve novick wrote:
 That would not only be a quick fix but enable folks that work on sections
 of the tree not depending the bison/flex input to have a simpler build
 process.
 
 
 
 -Original Message-
 From: [EMAIL PROTECTED]
 [mailto:[EMAIL PROTECTED] Behalf Of Bruce Momjian
 Sent: Thursday, September 04, 2003 6:18 PM
 To: Peter Eisentraut
 Cc: Doug McNaught; Dann Corbit; Joerg Hessdoerfer;
 [EMAIL PROTECTED]; PostgreSQL Win32 port list
 Subject: Re: [HACKERS] Win32 native port
 
 
 Peter Eisentraut wrote:
  Bruce Momjian writes:
 
   As you can see from the new Compiling web page, I just normally
   compile under Unix, distclean, then Win32 compile via Samba.
 
  That isn't very efficient unless you have two machines or use something
  like vmware.
 
 One quick solution would be to add the bison/flex output files to the
 WIN32_DEV CVS tree.  Do people want that?
 
 --
   Bruce Momjian|  http://candle.pha.pa.us
   [EMAIL PROTECTED]   |  (610) 359-1001
   +  If your life is a hard drive, |  13 Roberts Road
   +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073
 
 ---(end of broadcast)---
 TIP 5: Have you checked our extensive FAQ?
 
http://www.postgresql.org/docs/faqs/FAQ.html
 

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] Examining the output of: ldd `which postgres`

2003-09-11 Thread Bruce Momjian
Sean Chittenden wrote:
  Backend only forks().  I think you would be better off using
  Makefile macros to _remove_ those two libraries.
  
  I see this:
  
$(filter crypt.o getaddrinfo.o inet_aton.o snprintf.o strerror.o path.o 
  thread.o, $(LIBOBJS))
  
  Seems you need the reverse.
 
 Ah, well, if it fork()'s, then I don't really care.  The best
 remaining argument for this would be to reduce the total size of a
 machine's VM page table size and possibly the expense of switching
 contexts between procs, but that's a pretty weak argument for only .5M
 of shared RAM.  For some reason I thought it exec()'ed a child with
 the args necessary for it to read in a postgresql.conf.  Looks like
 the comment in backend/storage/ipc/ipci.c is out of date then:
 
  * AttachSharedMemoryAndSemaphores
  *  Attaches to the existing shared resources when exec()'d off
  *  by the postmaster.

I have updated this comment.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] massive quotes?

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Keep in mind that we have two different requirements: the quote start
 marker has to be recognizable while we are parsing SQL (or possibly
 plpgsql) code.  The quote end marker has to be recognizable while we are
 scanning text that could be almost anything.
 
 The cute thing about the here-document solution to this problem is that
 you can choose the quote end marker on a case-by-case basis.  So you can
 always pick something that won't conflict with anything that's actually
 in the text you need to quote.
 
 If we try to go with fixed markers (like {' ... }' and some other ideas
 that were floated today), then we lose that flexibility, and we're up
 against the losing game of trying to pick an end-marker that won't cause
 problems in any programming language anywhere (not to mention possible
 uses of the quoting mechanism for arbitrary string literals that aren't
 even function bodies).  
 
 I'm not wedded to the $$FOO idea in particular, but I do think we want
 to go with a solution that allows a variable end-marker.

Agreed --- with multiple languages, we have to allow users to specify a
unique end marker.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Marc G. Fournier


On Fri, 12 Sep 2003, Bruce Momjian wrote:

 Marc G. Fournier wrote:
 
 
  On Thu, 11 Sep 2003, Bruce Momjian wrote:
 
   Well, the problem was that we defined HAS_TEST_AND_SET inside the ports.
   I guess we could splatter a test for Itanium and Opterion in every port
   that could possibly use it, but then again, if we fall back to not
   finding it for some reason, we don't get a report because we silently
   fall back to semaphores.  That's what has me worried, that if we don't
   do it, we will not know what platforms really aren't working properly.
 
  From what I understand, not working properly means slow, not broken, no?
  Which means ppl could submit a problem report and it could be fixed for
  v7.4.1 ... its not so much  'not working properly' as it is 'not optimal
  performance' ...

 Right, though I am not sure people will know _slow_ configuration vs.
 PostgreSQL is slow.

No, but definitely something for those discussion performance to add
to their checklist :)

BTW, post-compile, running system ... how do you check this?  Or can you?

For instance, having a checklist, of sorts, that ppl can go through when
trying to investigate performance issues could include stuff like:

check swap usage (albeit obvious to alot of ppl, not to all)
check disk usage using iostat
check spinlocks in use using ... ??



---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Marc G. Fournier [EMAIL PROTECTED] writes:
 Right, though I am not sure people will know _slow_ configuration vs.
 PostgreSQL is slow.

 No, but definitely something for those discussion performance to add
 to their checklist :)

 BTW, post-compile, running system ... how do you check this?  Or can you?

If we force people to give a --without-spinlocks config option to build
that way, then `pg_config --configure' will reveal the dirty deed ...

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Marc G. Fournier wrote:
   From what I understand, not working properly means slow, not broken, no?
   Which means ppl could submit a problem report and it could be fixed for
   v7.4.1 ... its not so much  'not working properly' as it is 'not optimal
   performance' ...
 
  Right, though I am not sure people will know _slow_ configuration vs.
  PostgreSQL is slow.
 
 No, but definitely something for those discussion performance to add
 to their checklist :)
 
 BTW, post-compile, running system ... how do you check this?  Or can you?
 
 For instance, having a checklist, of sorts, that ppl can go through when
 trying to investigate performance issues could include stuff like:
 
 check swap usage (albeit obvious to alot of ppl, not to all)
 check disk usage using iostat
 check spinlocks in use using ... ??

We can add the compiler test that will throw an error if they aren't
using spinlocks, and they have to use a configure flag to enable it. 
The only issue there is that this is going to throw up perhaps many
cases we haven't gotten working, and they might dribble out for weeks or
after final, while a clean solution could break more platforms in the
short term, but could catch more in the long term.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


[HACKERS] Heads up: beta3 planned for Monday

2003-09-11 Thread Tom Lane
Subject says it all... if you've got something to get done, now's the
time...

regards, tom lane

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] massive quotes?

2003-09-11 Thread Tom Lane
Sean Chittenden [EMAIL PROTECTED] writes:
 Let me jump in for half a second here (no pun intended), but what
 about the use of back quotes? ` `?  Use a very limited escaping policy
 of \` = ` and \\ = \ .

Actually, having to double backslashes is one of the things I want to
get rid of.  The here-document-based ideas seem to allow that.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] constraint modification on todo list

2003-09-11 Thread Jeroen Ruigrok/asmodai
-On [20030911 15:43], Tom Lane ([EMAIL PROTECTED]) wrote:
We can't ALTER a table that's already in use when the first ALTER
starts, either --- its attempt to exclusive-lock the table will fail.
But once you get the exclusive lock, you can (in Postgres) perform
a series of operations without fear that subsequently-started
transactions will be able to see the incompletely changed state of the
table.  Evidently Oracle can't handle that.  That's why they need to
invent combination operations like MODIFY CONSTRAINT.

As my colleague says:

it is indeed a lazy choice, but super safe and that's the goal.

-- 
Jeroen Ruigrok van der Werven asmodai(at)wxs.nl / asmodai / kita no mono
PGP fingerprint: 2D92 980E 45FE 2C28 9DB7  9D88 97E6 839B 2EAC 625B
http://www.tendra.org/   | http://www.in-nomine.org/~asmodai/diary/
Man inagines that it is death he fears; but what he fears is the unforeseen,
the explosion.  What man fears is himself...

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Prompted by confusion over Itanium/Opterion, I have written a patch to
  improve the way we define spinlocks for platforms and cpu's.
 
 The main.c part of the patch strikes me as irrelevant to the claimed
 purpose and unlikely to accomplish anything except breaking things.
 Do you have a system the main.c __alpha code is relevant to, on which
 to test that you did not break it?
 
 Other than that, it looks like basically a good idea.  But...

I was going to have an alpha guy test it --- that was the one change I
wasn't sure about.  We did test for __alpha__ all over the port/*.h
files, so it wasn't clear which alpha's were being hit.  I can throw in
a comment and skip that part --- not sure.

  I plan to apply this to 7.4.
 
 This seems like living dangerously.  You do realize that this patch will
 invalidate our trust that the code works on every single supported
 platform?  I think beta3 may be a bit late in the game for this sort of
 thing, because we've already gotten a good bit of the testing we can
 expect to get for lesser-used platforms during this beta cycle.
 
 At the very least I'd like to see the decision discussed on -hackers
 and not buried in -patches.

The problem with waiting for 7.5 is that we will have no error reporting
when our non-spinlock code is being executed, and with Opteron/Itanium,
it seems like a good time to get it working.  We had the FreeBSD report
with not finding Opteron/Itanium, and that's what got me going.  Also,
if it doesn't find the spinlock code, it will report an error, so we
should flesh this all out as we collect supported platforms, which we
haven't started yet.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 The problem with waiting for 7.5 is that we will have no error reporting
 when our non-spinlock code is being executed, and with Opteron/Itanium,
 it seems like a good time to get it working.

Well, as long as you're prepared to reduce the list of known supported
platforms to zero as of 7.4beta3, and issue a fresh call for port reports.

But it seems to me that this is mostly a cosmetic cleanup and therefore
not the kind of thing to be doing late in beta.  Couldn't we do
something that affects only Opteron/Itanium and doesn't take a chance
on breaking everything else?

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  The problem with waiting for 7.5 is that we will have no error reporting
  when our non-spinlock code is being executed, and with Opteron/Itanium,
  it seems like a good time to get it working.
 
 Well, as long as you're prepared to reduce the list of known supported
 platforms to zero as of 7.4beta3, and issue a fresh call for port reports.

I haven't collected any known reports yet.  That happens later, I
thought, nearer to RC1.

 But it seems to me that this is mostly a cosmetic cleanup and therefore
 not the kind of thing to be doing late in beta.  Couldn't we do
 something that affects only Opteron/Itanium and doesn't take a chance
 on breaking everything else?

Yes, but to throw an error if spinlocks aren't found, we need this
patch.  We would have to test for Opteron in all the platforms that test
for specific CPU's but don't test for opteron, and might support
opterion/itanium, but even then, we don't have any way of getting a
report of a failure.

Yea, I should have thought of this before beta1, but now that I see the
bug reports, seems we have to go this way.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Marc G. Fournier


On Thu, 11 Sep 2003, Tom Lane wrote:

 Bruce Momjian [EMAIL PROTECTED] writes:
  The problem with waiting for 7.5 is that we will have no error reporting
  when our non-spinlock code is being executed, and with Opteron/Itanium,
  it seems like a good time to get it working.

 Well, as long as you're prepared to reduce the list of known supported
 platforms to zero as of 7.4beta3, and issue a fresh call for port reports.

I didn't think we had done that yet ... had we?  called for port reports,
that is ... ?

 But it seems to me that this is mostly a cosmetic cleanup and therefore
 not the kind of thing to be doing late in beta.  Couldn't we do
 something that affects only Opteron/Itanium and doesn't take a chance
 on breaking everything else?

I just went through the whole patch myself, and as much as I like the
overall simplification, I tend to agree with Tom here on questioning the
requirement to do suck a massive change so late in the end cycle ... is
there no smaller bandaid that can be applied to handle the Opteron/Itanium
issue for v7.4, with the cleanup patch being applied right away after
v7.4?


---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Marc G. Fournier wrote:
  But it seems to me that this is mostly a cosmetic cleanup and therefore
  not the kind of thing to be doing late in beta.  Couldn't we do
  something that affects only Opteron/Itanium and doesn't take a chance
  on breaking everything else?
 
 I just went through the whole patch myself, and as much as I like the
 overall simplification, I tend to agree with Tom here on questioning the
 requirement to do suck a massive change so late in the end cycle ... is
 there no smaller bandaid that can be applied to handle the Opteron/Itanium
 issue for v7.4, with the cleanup patch being applied right away after
 v7.4?

Well, the problem was that we defined HAS_TEST_AND_SET inside the ports.
I guess we could splatter a test for Itanium and Opterion in every port
that could possibly use it, but then again, if we fall back to not
finding it for some reason, we don't get a report because we silently
fall back to semaphores.  That's what has me worried, that if we don't
do it, we will not know what platforms really aren't working properly.

Take FreeBSD for example, that couldn't find Opteron. It lists every
cpu like this:

#if defined(__i386__)
#define NEED_I386_TAS_ASM
#define HAS_TEST_AND_SET
typedef unsigned char slock_t;
#endif

#if defined(__sparc__)
#define NEED_SPARC_TAS_ASM
#define HAS_TEST_AND_SET
typedef unsigned char slock_t;
#endif

We would have to add an opteron/itanium to port that does this, but if
we miss some opteron/itanium define, we might never know because of the
silent fallback.

I don't care if we save it for 7.5 --- I just don't know how we will be
sure we have things working properly without it.

We could apply it tomorrow and see how things look on Monday.


-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Larry Rosenman


--On Thursday, September 11, 2003 23:46:56 -0300 Marc G. Fournier 
[EMAIL PROTECTED] wrote:



On Thu, 11 Sep 2003, Tom Lane wrote:

Bruce Momjian [EMAIL PROTECTED] writes:
 The problem with waiting for 7.5 is that we will have no error
 reporting when our non-spinlock code is being executed, and with
 Opteron/Itanium, it seems like a good time to get it working.
Well, as long as you're prepared to reduce the list of known supported
platforms to zero as of 7.4beta3, and issue a fresh call for port
reports.
I didn't think we had done that yet ... had we?  called for port reports,
that is ... ?
But it seems to me that this is mostly a cosmetic cleanup and therefore
not the kind of thing to be doing late in beta.  Couldn't we do
something that affects only Opteron/Itanium and doesn't take a chance
on breaking everything else?
I just went through the whole patch myself, and as much as I like the
overall simplification, I tend to agree with Tom here on questioning the
requirement to do suck a massive change so late in the end cycle ... is
there no smaller bandaid that can be applied to handle the Opteron/Itanium
issue for v7.4, with the cleanup patch being applied right away after
v7.4?
Bruce sent me a copy of the patch, and it BREAKS UnixWare (If y'all 
care).

LER

--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED]
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


pgp0.pgp
Description: PGP signature


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Marc G. Fournier

On Thu, 11 Sep 2003, Bruce Momjian wrote:

 Yes, but to throw an error if spinlocks aren't found, we need this
 patch.  We would have to test for Opteron in all the platforms that test
 for specific CPU's but don't test for opteron, and might support
 opterion/itanium, but even then, we don't have any way of getting a
 report of a failure.

'K, but apparently right now we are broken on Opteron/Itanium without this
patch ... so, to fix, we either:

a. add appropriate tests to the individual port files based on individual
failure reports (albeit not clean, definitely safer), or:

b. we do massive, sweeping changes to the whole HAVE_TEST_AND_SET
detection code (definitely cleaner, but has potential of breaking more
then it fixes) :(

personally, as late in the cycle as we are, I think that a. is the wiser
move for v7.4, with b. being something that should happen as soon as
possible once we've branched and start working on v7.5 ...



---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Marc G. Fournier [EMAIL PROTECTED] writes:
 On Thu, 11 Sep 2003, Tom Lane wrote:
 Well, as long as you're prepared to reduce the list of known supported
 platforms to zero as of 7.4beta3, and issue a fresh call for port reports.

 I didn't think we had done that yet ... had we?  called for port reports,
 that is ... ?

We hadn't, no.  My point is that in the past we've continued to list
platforms as supported if we've had a successful report in the past
release or two.  Fooling with the spinlock code is delicate enough
that I'd want to insist on moving everything to the unsupported
category until we get a success report with the modified code.

Maybe we should just do that.  It's likely that the only platforms
that end up marked unsupported are ones that no one cares about any
more anyway.  But I think we have to realize that this is not a
trivial set of changes, even if it looks like it should work.
(Which it does, just for the record.  I'm just feeling paranoid
because of where we are in the release cycle.)

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 I guess we could splatter a test for Itanium and Opterion in every port
 that could possibly use it, but then again, if we fall back to not
 finding it for some reason, we don't get a report because we silently
 fall back to semaphores.  That's what has me worried, that if we don't
 do it, we will not know what platforms really aren't working properly.

Agreed, the silent fallback to semaphores isn't such a hot idea in
hindsight.  But the part of the patch that requires a configure option
to use that code path could be applied without touching anything else.

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Larry Rosenman [EMAIL PROTECTED] writes:
 Bruce sent me a copy of the patch, and it BREAKS UnixWare (If y'all=
 =20
 care).

Unfixably?  Or just a small oversight?

I'm actually not worried about platforms that are actively being tested.
It's the stuff that hasn't been confirmed recently that is going to be
at risk.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Larry Rosenman [EMAIL PROTECTED] writes:
  Bruce sent me a copy of the patch, and it BREAKS UnixWare (If y'all=
  =20
  care).
 
 Unfixably?  Or just a small oversight?
 
 I'm actually not worried about platforms that are actively being tested.
 It's the stuff that hasn't been confirmed recently that is going to be
 at risk.

I sent him a new patch just now.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Larry Rosenman


--On Thursday, September 11, 2003 23:13:54 -0400 Tom Lane 
[EMAIL PROTECTED] wrote:

Larry Rosenman [EMAIL PROTECTED] writes:
Bruce sent me a copy of the patch, and it BREAKS UnixWare (If
y'all= =20
care).
Unfixably?  Or just a small oversight?

I'm actually not worried about platforms that are actively being tested.
It's the stuff that hasn't been confirmed recently that is going to be
at risk.
I'm seeing failures with the 2nd patch as well.  Seems like it's not liking 
UnixWare's
cc defines.

the documentation is at:

http://www.lerctr.org:8458/

the cc man page is at:

http://www.lerctr.org:8458/en/man/html.1/cc.1.html

Tom, You still have an account here.

Bruce, if you'd like an account, that is easily arranged.

LER

			regards, tom lane


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED]
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


pgp0.pgp
Description: PGP signature


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  I guess we could splatter a test for Itanium and Opterion in every port
  that could possibly use it, but then again, if we fall back to not
  finding it for some reason, we don't get a report because we silently
  fall back to semaphores.  That's what has me worried, that if we don't
  do it, we will not know what platforms really aren't working properly.
 
 Agreed, the silent fallback to semaphores isn't such a hot idea in
 hindsight.  But the part of the patch that requires a configure option
 to use that code path could be applied without touching anything else.

Yes, we could do just the configure warning, then plaster tests into the
port files to try to hit all the opteron/itanium cases.  I am a little
concerned that this might throw up a bunch of problem cases that we will
patching for a while.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Yes, we could do just the configure warning, then plaster tests into the
 port files to try to hit all the opteron/itanium cases.  I am a little
 concerned that this might throw up a bunch of problem cases that we will
 patching for a while.

Probably so --- but we'd only be breaking new platforms that people are
starting to use, not old ones that might not be getting tested
regularly.

Understand that I'm not dead set against applying this patch for 7.4.
(On a code-cleanliness point of view I favor it.)  What I want is some
open discussion about the risks and benefits before we decide.

regards, tom lane

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Yes, we could do just the configure warning, then plaster tests into the
  port files to try to hit all the opteron/itanium cases.  I am a little
  concerned that this might throw up a bunch of problem cases that we will
  patching for a while.
 
 Probably so --- but we'd only be breaking new platforms that people are
 starting to use, not old ones that might not be getting tested
 regularly.

Looking at the code, I wonder if we already have folks not using
spinlocks, and not even knowing it.  I don't think problem reports will
be limited to new platforms.

 Understand that I'm not dead set against applying this patch for 7.4.
 (On a code-cleanliness point of view I favor it.)  What I want is some
 open discussion about the risks and benefits before we decide.

Sure, and I am not pushing the patch.  I am just saying it would have
been ideal a few weeks ago --- I am not sure if we are worse off with or
without it.

I just learned from Larry that Unixware defines intel as i386, not
__i386 or __i386__, at least of the native SCO compiler that he uses.

What the code used to do is define NEED_I386_TAS_ASM unconditionally on
some platforms (negating the need to test for a compiler symbol) or test
for each platform compiler symbol (and not test all possible ways it
could be specified), like FreeBSD did.  That's why things are so messy. 
I am going to test for __cpu, __cpu__, and cpu on non-gcc compiler for
consistency.  It is only done in one place in the patch, so that should
be good.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Looking at the code, I wonder if we already have folks not using
 spinlocks, and not even knowing it.  I don't think problem reports will
 be limited to new platforms.

Very likely --- I heard from someone recently who was trying to run
HPUX/Itanium.  After we got past the hard-wired assumption that HPUX
== HPPA, it was still giving lousy performance for lack of spinlocks.
I like the part of the patch that is in-your-face about that.

 I just learned from Larry that Unixware defines intel as i386, not
 __i386 or __i386__, at least of the native SCO compiler that he uses.

[blink]  Namespace infringement 'r us, huh?

 I am going to test for __cpu, __cpu__, and cpu on non-gcc compiler for
 consistency.  It is only done in one place in the patch, so that should
 be good.

Please, only the first two.  Make the Unixware template add __i386__.
Don't add assumptions about valid user-namespace symbols.

regards, tom lane

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Looking at the code, I wonder if we already have folks not using
  spinlocks, and not even knowing it.  I don't think problem reports will
  be limited to new platforms.
 
 Very likely --- I heard from someone recently who was trying to run
 HPUX/Itanium.  After we got past the hard-wired assumption that HPUX
 == HPPA, it was still giving lousy performance for lack of spinlocks.
 I like the part of the patch that is in-your-face about that.
 
  I just learned from Larry that Unixware defines intel as i386, not
  __i386 or __i386__, at least of the native SCO compiler that he uses.
 
 [blink]  Namespace infringement 'r us, huh?
 
  I am going to test for __cpu, __cpu__, and cpu on non-gcc compiler for
  consistency.  It is only done in one place in the patch, so that should
  be good.
 
 Please, only the first two.  Make the Unixware template add __i386__.
 Don't add assumptions about valid user-namespace symbols.

Roger!

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Larry Rosenman


--On Thursday, September 11, 2003 23:42:53 -0400 Tom Lane 
[EMAIL PROTECTED] wrote:

Bruce Momjian [EMAIL PROTECTED] writes:
Looking at the code, I wonder if we already have folks not using
spinlocks, and not even knowing it.  I don't think problem reports will
be limited to new platforms.
Very likely --- I heard from someone recently who was trying to run
HPUX/Itanium.  After we got past the hard-wired assumption that HPUX
== HPPA, it was still giving lousy performance for lack of spinlocks.
I like the part of the patch that is in-your-face about that.
I just learned from Larry that Unixware defines intel as i386, not
__i386 or __i386__, at least of the native SCO compiler that he uses.
[blink]  Namespace infringement 'r us, huh?
Yeah.  I **DO** have SCO's ear on it, but I don't know how far I'll get, 
plus there are
TONS of pre-whenever-we-get-it-fixed out there.



I am going to test for __cpu, __cpu__, and cpu on non-gcc compiler for
consistency.  It is only done in one place in the patch, so that should
be good.
Please, only the first two.  Make the Unixware template add __i386__.
Don't add assumptions about valid user-namespace symbols.
that's reasonable.  At least until 64-bit UnixWare. :-)

(announced at SCOForum).


			regards, tom lane

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
   http://www.postgresql.org/docs/faqs/FAQ.html



--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED]
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


pgp0.pgp
Description: PGP signature


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Larry Rosenman [EMAIL PROTECTED] writes:
 Please, only the first two.  Make the Unixware template add __i386__.
 Don't add assumptions about valid user-namespace symbols.

 that's reasonable.  At least until 64-bit UnixWare. :-)

Even then, I'd prefer to put the necessary kluge into template/unixware
or Makefile.unixware or port/unixware.h, rather than add a risky
assumption globally.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Tom Lane
Larry Rosenman [EMAIL PROTECTED] writes:
 I've already sent a whine-a-gram to the compiler guys at SCO.

Prolly you thought of this already, but: getting them to *add*
an implicit #define of __i386__ should be plenty easy compared
to getting them to *remove* the one for i386.  And while I think
they should remove the latter, the immediate problem would be
solved as soon as they add the former.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Larry Rosenman


--On Friday, September 12, 2003 00:06:49 -0400 Tom Lane [EMAIL PROTECTED] 
wrote:

Larry Rosenman [EMAIL PROTECTED] writes:
I've already sent a whine-a-gram to the compiler guys at SCO.
Prolly you thought of this already, but: getting them to *add*
an implicit #define of __i386__ should be plenty easy compared
to getting them to *remove* the one for i386.  And while I think
they should remove the latter, the immediate problem would be
solved as soon as they add the former.
sure, and I expect that's what they may do.  We'll see what they say.

LER

			regards, tom lane


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: [EMAIL PROTECTED]
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


pgp0.pgp
Description: PGP signature


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Marc G. Fournier wrote:
 
 
 On Thu, 11 Sep 2003, Bruce Momjian wrote:
 
  Well, the problem was that we defined HAS_TEST_AND_SET inside the ports.
  I guess we could splatter a test for Itanium and Opterion in every port
  that could possibly use it, but then again, if we fall back to not
  finding it for some reason, we don't get a report because we silently
  fall back to semaphores.  That's what has me worried, that if we don't
  do it, we will not know what platforms really aren't working properly.
 
 From what I understand, not working properly means slow, not broken, no?
 Which means ppl could submit a problem report and it could be fixed for
 v7.4.1 ... its not so much  'not working properly' as it is 'not optimal
 performance' ...

Right, though I am not sure people will know _slow_ configuration vs.
PostgreSQL is slow.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Marc G. Fournier

On Thu, 11 Sep 2003, Bruce Momjian wrote:

 I just learned from Larry that Unixware defines intel as i386, not
 __i386 or __i386__, at least of the native SCO compiler that he uses.

could we put something in the various port files to standardize this?  ie.
in unixware.h, add somethinglike:

#ifdef i386
#define __i386__
#endif

just so that 'special cases' are centralized in the ports file, and the
mainstream code doesn't have:

#if defined(i386) || defined(__i386) || defined(__i386__)

?


---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Marc G. Fournier wrote:
 
 On Thu, 11 Sep 2003, Bruce Momjian wrote:
 
  I just learned from Larry that Unixware defines intel as i386, not
  __i386 or __i386__, at least of the native SCO compiler that he uses.
 
 could we put something in the various port files to standardize this?  ie.
 in unixware.h, add somethinglike:
 
 #ifdef i386
 #define __i386__
 #endif
 
 just so that 'special cases' are centralized in the ports file, and the
 mainstream code doesn't have:
 
 #if defined(i386) || defined(__i386) || defined(__i386__)

Yep, that's what Tom wants and I am doing that now.  I sent a patch to
Larry for testing.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] [PATCHES] Reorganization of spinlock defines

2003-09-11 Thread Bruce Momjian
Tom Lane wrote:
 Larry Rosenman [EMAIL PROTECTED] writes:
  Bruce sent me a copy of the patch, and it BREAKS UnixWare (If y'all=
  =20
  care).
 
 Unfixably?  Or just a small oversight?

Updated patch now works on Unixware.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster