Re: [HACKERS] RC1?
Bruce Momjian <[EMAIL PROTECTED]> writes: > Are we ready for RC1 yet? I think so. The NO_MKTIME_BEFORE_1970 issue was bothering me, but I feel that's resolved now. (It'd be nice to hear a crosscheck from some AIX users though...) regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] RC1?
Are we ready for RC1 yet? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] the map file between gb18030 and utf8 is error
> hello,
> recently i downloaded postgresql-7.3b5,i found the conversion between
> gb18030 and utf was mistake because the map file bwteen them wasn't
> obviously right.the first byte of utf8 encoding with two bytes shoule
> between 0xc0 with 0xfd,the map file didn't accord with this condition
> .please check it ,i wished that postgresql-7.3 can support the GB18030 and
> can run in China.
> best regards
> jenny wang
Thanks for testing GB18030 support. Yes, the map file is completely
broken. I have attached fixed map generation perl script. Please test
it (I do not understand Chinese).
(1) save the perl script in the postgresql-7.3b5 source tree as:
src/backend/utils/mb/Unicode/UCS_to_GB18030.pl.
(2) run it.
cd src/backend/utils/mb/Unicode
./UCS_to_GB18030.pl
(3) the script will generate
src/backend/utils/mb/Unicode/utf8_to_gb18030.map
and
src/backend/utils/mb/Unicode/gb18030_to_utf8.map
(4) If they look good, rebuild PostgreSQL and test it.
--
Tatsuo Ishii
#! /usr/bin/perl
#
# Copyright 2002 by Bill Huang
#
# $Id: UCS_to_GB18030.pl,v 1.1 2002/06/13 08:28:55 ishii Exp $
#
# Generate UTF-8 <--> GB18030 code conversion tables from
# map files provided by Unicode organization.
# Unfortunately it is prohibited by the organization
# to distribute the map files. So if you try to use this script,
# you have to obtain ISO10646-GB18030.TXT from
# the organization's ftp site.
#
# ISO10646-GB18030.TXT format:
#GB18030 code in hex
#UCS-2 code in hex
## and Unicode name (not used in this script)
require "ucs2utf.pl";
# first generate UTF-8 --> GB18030 table
$in_file = "ISO10646-GB18030.TXT";
open( FILE, $in_file ) || die( "cannot open $in_file" );
while( ){
chop;
if( /^#/ ){
next;
}
( $u, $c, $rest ) = split;
$ucs = hex($u);
$code = hex($c);
if( $code >= 0x80 && $ucs >= 0x0080 ){
$utf = &ucs2utf($ucs);
if( $array{ $utf } ne "" ){
printf STDERR "Warning: duplicate unicode: %04x\n",$ucs;
next;
}
$count++;
$array{ $utf } = $code;
}
}
close( FILE );
#
# first, generate UTF8 --> GB18030 table
#
$file = "utf8_to_gb18030.map";
open( FILE, "> $file" ) || die( "cannot open $file" );
print FILE "static pg_utf_to_local ULmapGB18030[ $count ] = {\n";
for $index ( sort {$a <=> $b} keys( %array ) ){
$code = $array{ $index };
$count--;
if( $count == 0 ){
printf FILE " {0x%04x, 0x%04x}\n", $index, $code;
} else {
printf FILE " {0x%04x, 0x%04x},\n", $index, $code;
}
}
print FILE "};\n";
close(FILE);
#
# then generate GB18030 --> UTF8 table
#
reset 'array';
open( FILE, $in_file ) || die( "cannot open $in_file" );
while( ){
chop;
if( /^#/ ){
next;
}
( $c, $u, $rest ) = split;
$ucs = hex($u);
$code = hex($c);
if( $code >= 0x80 && $ucs >= 0x0080 ){
$utf = &ucs2utf($ucs);
if( $array{ $code } ne "" ){
printf STDERR "Warning: duplicate code: %04x\n",$ucs;
next;
}
$count++;
$array{ $code } = $utf;
}
}
close( FILE );
$file = "gb18030_to_utf8.map";
open( FILE, "> $file" ) || die( "cannot open $file" );
print FILE "static pg_local_to_utf LUmapGB18030[ $count ] = {\n";
for $index ( sort {$a <=> $b} keys( %array ) ){
$utf = $array{ $index };
$count--;
if( $count == 0 ){
printf FILE " {0x%04x, 0x%04x}\n", $index, $utf;
} else {
printf FILE " {0x%04x, 0x%04x},\n", $index, $utf;
}
}
print FILE "};\n";
close(FILE);
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] Unique functional index and FK constraints
Kris Jurka <[EMAIL PROTECTED]> writes: > When you restructured the unique index location and validation for > foreign key constraints around 9/22 you added the restriction that the > supporting unique index may not be functional. I believe that this > restriction is not necessary. Hmm ... I'm not convinced. What about functions that return NULL for some inputs? A unique index will not reject multiple NULL entries. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] Unique functional index and FK constraints
Tom, When you restructured the unique index location and validation for foreign key constraints around 9/22 you added the restriction that the supporting unique index may not be functional. I believe that this restriction is not necessary. Suppose I had a unique index on LOWER(login). That would imply that login is unique as well. Any function which returns different results given the same input is no good for a functional index anyway. http://archives.postgresql.org/pgsql-committers/2002-09/msg00293.php Kris Jurka ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Idea for better handling of cntxDirty
Vadim,
In LockBuffer() you wrote
else if (mode == BUFFER_LOCK_EXCLUSIVE)
{
LWLockAcquire(buf->cntx_lock, LW_EXCLUSIVE);
/*
* This is not the best place to set cntxDirty flag (eg indices do
* not always change buffer they lock in excl mode). But please
* remember that it's critical to set cntxDirty *before* logging
* changes with XLogInsert() - see comments in BufferSync().
*/
buf->cntxDirty = true;
}
The comments in BufferSynx are
/*
* We can check bufHdr->cntxDirty here *without* holding any lock
* on buffer context as long as we set this flag in access methods
* *before* logging changes with XLogInsert(): if someone will set
* cntxDirty just after our check we don't worry because of our
* checkpoint.redo points before log record for upcoming changes
* and so we are not required to write such dirty buffer.
*/
Wouldn't it work for cntxDirty to be set not by LockBuffer, but by
XLogInsert for each buffer that is included in its argument list?
This would avoid setting the flag for pages that are not modified
after being locked. XLogInsert would of course set the flag before
doing the actual WAL insertion, so it seems to me that the condition
we want is met, and we still have only a single place that needs to
remember to set the flag.
regards, tom lane
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] 500 tpsQL + WAL log implementation
"Curtis Faith" <[EMAIL PROTECTED]> writes: > Using a raw file partition and a time-based technique for determining the > optimal write position, I am able to get 8K writes physically written to disk > synchronously in the range of 500 to 650 writes per second using FreeBSD raw > device partitions on IDE disks (with write cache disabled). What can you do *without* using a raw partition? I dislike that idea for two reasons: portability and security. The portability disadvantages are obvious. And in ordinary system setups Postgres would have to run as root in order to write on a raw partition. It occurs to me that the same technique could be used without any raw device access. Preallocate a large WAL file and apply the method within it. You'll have more noise in the measurements due to greater variability in the physical positioning of the blocks --- but it's rather illusory to imagine that you know the disk geometry with any accuracy anyway. Modern drives play a lot of games under the hood. > The obvious problem with the above mechanism is that the WAL log needs to be > able to read from the log file in transaction order during recovery. This > could be provided for using an abstraction that prepends the logical order > for each block written to the disk and makes sure that the log blocks contain > either a valid logical order number or some other marker indicating that the > block is not being used. This scares me quite a bit too. The reason that the existing implementation maxes out at one WAL write per rotation is that for small transactions it's having to repeatedly write the same disk sector. You could only get around that by writing multiple versions of the same WAL page at different disk locations. Reliably reconstructing what data to use is not something that I'm prepared to accept on a handwave... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] "Uninitialized page" bug mechanism identified
We have seen a few reports (eg from Hervé Piedvache) of VACUUM FULL in 7.2 producing messages like dbfr=# VACUUM FULL VERBOSE ANALYZE pg_class ; NOTICE: --Relation pg_class-- NOTICE: Rel pg_class: Uninitialized page 9 - fixing NOTICE: Rel pg_class: Uninitialized page 10 - fixing NOTICE: Rel pg_class: Uninitialized page 11 - fixing NOTICE: Rel pg_class: Uninitialized page 12 - fixing NOTICE: Rel pg_class: Uninitialized page 13 - fixing NOTICE: Rel pg_class: Uninitialized page 14 - fixing NOTICE: Rel pg_class: Uninitialized page 15 - fixing NOTICE: Rel pg_class: Uninitialized page 16 - fixing NOTICE: Rel pg_class: Uninitialized page 17 - fixing NOTICE: Rel pg_class: Uninitialized page 18 - fixing NOTICE: Rel pg_class: Uninitialized page 19 - fixing NOTICE: Rel pg_class: Uninitialized page 20 - fixing NOTICE: Rel pg_class: Uninitialized page 21 - fixing NOTICE: Rel pg_class: Uninitialized page 22 - fixing NOTICE: Rel pg_class: Uninitialized page 23 - fixing ... I had originally suspected hardware problems, but Hervé told me today that he was still seeing this behavior after moving to a new machine. So I went digging for an explanation --- and I found one. I've been able to reproduce the above behavior by issuing repeated table creations in one backend while another backend does occasional VACUUM FULLs on pg_class. The fundamental problem is that for nailed-in-cache relations like pg_class, RelationClearRelation() does not want to release the cache entry. In 7.2 it doesn't do anything except close the smgr file for the relation and return. But RelationClearRelation is what gets called to implement a relcache flush from an SI message. This means that nothing much happens in other backends when a VACUUM transmits a relcache flush message for a nailed-in-cache relation. In particular, they fail to update their rd_targblock and rd_nblocks fields. So the scenario goes like this: 1. Backend A has done a lot of inserts/deletes in pg_class. Its rd_targblock field points out somewhere near the end of the table. 2. Backend B does a VACUUM FULL, gets rid of lots of space, and shrinks pg_class. 3. Backend A does nothing in response to B's SI message, so its rd_targblock field now points past the end of the table. 4. Backend A now tries to insert another pg_class row. In RelationGetBufferForTuple(), it reads the rd_targblock page, locks it, checks it for free space. md.c will allow the read to occur even though it's past current EOF of the table; it will return a zeroed page. The check for free space will act as though there is zero free space available, so RelationGetBufferForTuple releases the buffer and goes to find another page where there's space. No problem ... yet. 5. The trouble is that the bufmgr now has a live buffer for a page that's past the end of pg_class. What's more, it thinks the page is dirty (because the mere act of obtaining an exclusive buffer lock on the page sets cntxDirty). Eventually, the bufmgr will want to recycle that buffer for some other use, and at that point it writes out the buffer. Presto, a page of zeroes. In fact possibly many pages of zeroes --- if the rd_targblock was more than one block past the new actual EOF, standard Unixen will accept the write and will silently fill the intervening file space with zeroes (or make it look like they did, anyway). There isn't any serious consequence of this problem, other than that the next VACUUM will issue some "Uninitialized page" messages, so I'm not feeling that we need a 7.2.4 to fix it in the 7.2 series. But it needs to be fixed. The good news is that it is partly fixed already in 7.3, because in 7.3 RelationClearRelation does reset rd_targblock for nailed-in relations. So I believe the problem cannot occur in this form anymore. But I am also thinking that it's a really bad idea for mdread to allow reads from beyond EOF --- that's just asking for trouble. Can anyone see a reason not to remove the special-case at line 440 in md.c? It'd probably also be a good idea to decouple setting cntxDirty from acquiring exclusive buffer lock. As things stand, when RelationGetBufferForTuple finds there's not enough space on a target page, it's still set cntxDirty, thereby triggering an unnecessary write of that page. In many cases the page would be dirty already, but it's ugly nonetheless ... and it is a contributing factor in this bug. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] geometry test failed (beta5 on Debian)
On Tue, Nov 12, 2002 at 03:20:52AM +0200, [EMAIL PROTECTED] wrote: > validated, but i'm not absolutely sure. So i've attached the diff. Oops , forgot to attach it in the first message. This is the diff. *** ./expected/geometry.out Thu Nov 29 20:57:31 2001 --- ./results/geometry.out Tue Nov 12 02:08:46 2002 *** *** 443,454 FROM CIRCLE_TBL; six | polygon -+- ! | ((-3,0),(-2.59807621135076,1.500442),(-1.499116,2.59807621135842),(1.53102359078377e-11,3),(1.501768,2.59807621134311),(2.59807621136607,1.49779),(3,-3.06204718156754e-11),(2.59807621133545,-1.503094),(1.496464,-2.59807621137373),(-4.59307077235131e-11,-3),(-1.50442,-2.5980762113278),(-2.59807621138138,-1.495138)) | ((-99,2),(-85.6025403783588,52.01473),(-48.97054,88.602540378614),(1.051034,102),(51.05893,88.6025403781036),(87.6025403788692,51.92634),(101,1.897932),(87.6025403778485,-48.10313),(50.88214,-84.6025403791243),(0.8468976,-98),(-49.14732,-84.6025403775933),(-85.6025403793795,-47.83795)) | ((-4,3),(-3.33012701891794,5.500737),(-1.498527,7.3301270189307),(1.002552,8),(3.502946,7.33012701890518),(5.33012701894346,5.496317),(6,2.994897),(5.33012701889242,0.4948437),(3.494107,-1.33012701895622),(0.9923449,-2),(-1.507366,-1.33012701887966),(-3.33012701896897,0.5081028)) | ((-2,2),(-1.59807621135076,3.500442),(-0.4991161,4.59807621135842),(1.001531,5),(2.501768,4.59807621134311),(3.59807621136607,3.49779),(4,1.996938),(3.59807621133545,0.4969062),(2.496464,-0.59807621137373),(0.9954069,-1),(-0.5044197,-0.598076211327799),(-1.59807621138138,0.5048617)) | ((90,200),(91.3397459621641,205.0015),(95.00295,208.660254037861),(100.0051,210),(105.0059,208.66025403781),(108.660254037887,204.9926),(110,199.9898),(108.660254037785,194.9897),(104.9882,191.339745962088),(99.98469,190),(94.98527,191.339745962241),(91.3397459620621,195.0162)) ! | ((0,0),(13.3974596216412,50.01473),(50.02946,86.602540378614),(100.051,100),(150.0589,86.6025403781036),(186.602540378869,49.92634),(200,-1.02068239385585e-09),(186.602540377848,-50.10313),(149.8821,-86.6025403791243),(99.8469,-100),(49.85268,-86.6025403775933),(13.3974596206205,-49.83795)) (6 rows) -- convert the circle to an 8-point polygon --- 443,454 FROM CIRCLE_TBL; six | polygon -+- ! | ((-3,0),(-2.59807621135076,1.500442),(-1.499116,2.59807621135842),(1.53102359027555e-11,3),(1.501768,2.59807621134311),(2.59807621136607,1.49779),(3,-3.06204718131343e-11),(2.59807621133545,-1.503094),(1.496464,-2.59807621137373),(-4.59307077057254e-11,-3),(-1.50442,-2.5980762113278),(-2.59807621138138,-1.495138)) | ((-99,2),(-85.6025403783588,52.01473),(-48.97054,88.602540378614),(1.051034,102),(51.05893,88.6025403781036),(87.6025403788692,51.92634),(101,1.897932),(87.6025403778485,-48.10313),(50.88214,-84
[HACKERS] geometry test failed (beta5 on Debian)
Hi all! PostgreSQL 7.3 beta5 Debian GNU/Linux 2.2r5 ( x86 ) . All regression tests passed, except geometry. The differences don't seem to be big. I think the test can be validated, but i'm not absolutely sure. So i've attached the diff. . On the other hand, in psql , tab-completion doesn't seem to work for certain commands : ALTER DATABASE ; ALTER TRIGGER ; CHECKPOINT ; CREATE CAST ; CREATE CONSTRAINT TRIGGER ; CREATE CONVERSION ; CREATE DOMAIN ; CREATE LANGUAGE ; DEALLOCATE ; DROP CAST ; DROP CONVERSION ; DROP DOMAIN ; DROP LANGUAGE ; EXECUTE ; PREPARE ( this could be considered a low priority todo item , though ) . Regards, Adrian Maier ([EMAIL PROTECTED]) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] 500 tpsQL + WAL log implementation
I have been experimenting with empirical tests of file system and device level writes to determine the actual constraints in order to speed up the WAL logging code. Using a raw file partition and a time-based technique for determining the optimal write position, I am able to get 8K writes physically written to disk synchronously in the range of 500 to 650 writes per second using FreeBSD raw device partitions on IDE disks (with write cache disabled). I will be testing it soon under linux with 10,00RPM SCSI which should be even better. It is my belief that the mechanism used to achieve these speeds could be incorporated into the existing WAL logging code as an abstraction that looks to the WAL code just like the file level access currently used. The current speeds are limited by the speed of a single disk rotation. For a 7,200 RPM disk this is 120/second, for a 10,000 RPM disk this is 166.66/second The mechanism works by adjusting the seek offset of the write by using gettimeofday to determine approximately where the disk head is in its rotation. The mechanism does not use any AIO calls. Assuming the following: 1) Disk rotation time is 8.333ms or 8333us (7200 RPM). 2) A write at offset 1,500K completes at system time 103s 000ms 000us 3) A new write is requested at system time 103s 004ms 166us 4) A 390K per rotation alignment of the data on the disk. 5) A write must be sent at least 20K ahead of the current head position to ensure that it is written in less than one rotation. It can be determined from the above that a write for an offset of something slightly more than 195K past the last write, or offset 1,695K will be ahead of the current location of the head and will therefore complete in less than a single rotation's time. The disk specific metrics (rotation speed, bytes per rotation, base write time, etc.) can be derived empirically through a tester program that would take a few minutes to run and which could be run at log setup time. The obvious problem with the above mechanism is that the WAL log needs to be able to read from the log file in transaction order during recovery. This could be provided for using an abstraction that prepends the logical order for each block written to the disk and makes sure that the log blocks contain either a valid logical order number or some other marker indicating that the block is not being used. A bitmap of blocks that have already been used would be kept in memory for quickly determining the next set of possible unused blocks but this bitmap would not need to be written to disk except during normal shutdown since in the even of a failure the bitmaps would be reconstructed by reading all the blocks from the disk. Checkpointing and something akin to log rotation could be handled using this mechanism as well. So, MY REAL QUESTION is whether or not this is the sort of speed improvement that warrants the work of writing the required abstraction layer and making this very robust. The WAL code should remain essentially unchanged, with perhaps new calls for the five or six routines used to access the log files, and handle the equivalent of log rotation for raw device access. These new calls would either use the current file based implementation or the new logging mechanism depending on the configuration. I anticipate that the extra work required for a PostgreSQL administrator to use the proposed logging mechanism would be to: 1) Create a raw device partition of the appropriate size 2) Run the metrics tester for that device partition 3) Set the appropriate configuration parameters to indicate raw WAL logging I anticipate that the additional space requirements for this system would be on the order of 10% to 15% beyond the current file-based implementation's requirements. So, is this worth doing? Would a robust implementation likely be accepted for 7.4 assuming it can demonstrate speed improvements in the range of 500tps? - Curtis ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Implicit coercions, choosing types for constants, etc
Tom Lane writes:
> One way to fix this is to make cross-category coercions to text not be
> implicit casts. (I've opined before that that's a bad idea, but not
> gotten a lot of support for changing it. Now that we have a distinction
> between implicit and assignment casts, perhaps we could consider making
> coercions to text be assignment casts, as a compromise?)
I thought we had agreed to make those explicit. In fact, I vaguely recall
you not liking that notion ...
> I suppose we could fix this particular case by eliminating
> to_hex(integer), or adding to_hex(smallint), but that seems a very
> unsatisfying answer. I am wondering about adding some notion of
> "conversion distance" associated with casts, and preferring choices
> that require a smaller conversion distance; perhaps this could replace
> the concept of "preferred type", too. But again I don't have a specific
> proposal to make. Any thoughts?
A couple of months ago I played around with the notion of adding a numeric
preference ("distance", as you call it) to the casts, but in the end this
solved only a small number of cases and created a big mess at the same
time. When you have to pick arbitrary distances, any algorithm will give
you arbitrary answers, after all.
I think we can construct a type precedence list using the existing catalog
information. Considering the example of choosing between int2->int4 and
int2->int8, the system could notice that there is an implicit cast
int4->int8 (and no implicit cast the other way), so int4 is "less" than
int8. (If there were an implicit cast int8->int4, then the system would
have to consider int4 and int8 to be equivalent and picking one at random
would be reasonable.)
--
Peter Eisentraut [EMAIL PROTECTED]
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Problem with 7.3 on Irix with dates before 1970
Dear Tom, I have removed the NO_MKTIME_BEFORE_1970 symbol from irix5.h, rebuilt 7.3b2, and reran the regression. The three time tests (tinterval, horology, abstime) now match the Solaris expected files. I checked the timezone files, and the system does not appear to have savings time defined for 1947, but it does report it as such in the PostgreSQL regression tests. WRT your old message about the mktime workaround, I must have missed your message. However, in the future, if you want something tested on Irix, please let me know. I use PostgreSQL a fair amount at Bristol-Myers Squibb, and I will spend some time to keep it working. Sincerely, Bob > > > "Robert E. Bruccoleri" <[EMAIL PROTECTED]> writes: > > There are differences in the regression tests for PostgreSQL > > 7.3b2 with handling of times before 1970. I recently sent out a set of > > diffs for the changes. I've looked through the datetime.c code, but > > it's not obvious to me what the cause of the change is. PostgreSQL 7.2 > > works fine on Irix for these cases. > > Waitasec ... are you using src/include/port/irix5.h as the port-specific > config file? (Check the symlink src/include/pg_config_os.h to find > out.) If so, try removing "#define NO_MKTIME_BEFORE_1970" from irix5.h > and see if things get better. I asked awhile ago if that symbol was > still needed given the mktime workaround, but no one got back to me on > it. > > My current theory is that once you remove that symbol, you will get > results matching the Solaris expected files --- ie, I suspect that your > system believes there was DST in 1947. Do you have a way of verifying > that theory by looking at the system timezone database? > > regards, tom lane > +-++ | Robert E. Bruccoleri, Ph.D. | email: [EMAIL PROTECTED]| | P.O. Box 314| URL: http://www.congen.com/~bruc | | Pennington, NJ 08534|| +-++ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Implicit coercions, choosing types for constants, etc (yet again)
=?ISO-8859-1?Q?Dennis_Bj=F6rklund?= <[EMAIL PROTECTED]> writes: > It seems to me that one would like to have a polymorphic typesystem with > constraints to handle overloading, subtyping or whatever is needed in > SQL. What we have now is indeed pretty ad-hoc, but a full-blown type inference system might be overkill. Not sure. It would be interesting to look into it. > Is there anywhere I can read about the typesystem in SQL in general and > postgresql in particular? There's the user's guide http://developer.postgresql.org/docs/postgres/typeconv.html and there's the source code (see src/backend/parser/, particularly parse_coerce.c, parse_func.c, parse_oper.c). Not much in between I'm afraid, but the source code is reasonably well-commented. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PQescapeBytea v 7.2.3 BUG?
Reid Thompson wrote:
should
[...snip...]
result in the following output?
char[0] is [\]
char[1] is [\]
char[2] is [\]
char[3] is [\]
esclen is [5]
buffer2Ptr is []
OR should it result in
char[0] is [\]
char[1] is [\]
esclen is [3]
buffer2Ptr is [\\]
It should result in the former:
test=# select ''::bytea as string, length(''::bytea) as length;
string | length
+
\\ | 1
(1 row)
HTH,
Joe
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] Implicit coercions, choosing types for constants, etc
On Sun, 10 Nov 2002, Tom Lane wrote: > In the last episode of this long-running issue, we decided that life > would be better if we make small-enough integer constants be initially > typed as int2 rather than int4, and then auto-promote them up to int4 > when necessary. What kind of type system do postgresql (or SQL in general) use? I don't know much about the postgresql codebase but I do know something about type inference of functional languages. It seems to me that one would like to have a polymorphic typesystem with constraints to handle overloading, subtyping or whatever is needed in SQL. > This would solve problems with, for example, > SELECT ... WHERE smallintcol = 42 > not using an index. Using a suitable typesystem 42 could be said to have a type like isInt z => z which should be read that z is the type and isInt z is a constraint on z saying that z is an integer type (that means for example that z can never be instantiated to Bool). Then the use of smallintcol = 42 where smallintcol has type int2 and where equality = is overloaded for types Int2 -> Int2 -> Bool, Int4 -> Int4 -> Bool (and so on) would force 42 to be the type Int2 in this case, since the first argument of = had type Int2. Is there anywhere I can read about the typesystem in SQL in general and postgresql in particular? There are a number of type systems in the functional world with properties like this. Some very advanced and some simpler. I have a feeling from reading this list that the type inference in postgresql as a bit ad hook. But i've not read the source at all so it might be unfair to say such a thing. In the functional language Haskell there is also a defaulting rule that is used if you end up with constants like 42 still with type isInt z => z. If the type was left like this it just imply that any numeric type for 42 would do. In this case maybe z is defaulted to Int4. In most cases they way you use the constant would force it to be a particular type. Of the examples you gave in the letter I don't see anything that shouldn't work with a more advanced typesystem like this. But I'm sure there are other strange constructs in postgresql that I don't know about. I would love to make some small toy implementation to try out things but in the nearest future I don't have time for that. But this interests me so maybe I'll give it a try some day (like next summer :-). At least I would like to know more about how it works in postgresql today. It's possible that what I talk about do not apply to SQL or that postgresql already implements similar things. -- /Dennis ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PQescapeBytea v 7.2.3 BUG?
Reid Thompson <[EMAIL PROTECTED]> writes:
> should
> sprintf(buffer, "%c", 0x5C);
> readsz = 1;
> buffer2Ptr =(unsigned char *) PQescapeBytea(buffer, readsz, &esclen);
> for (ctr = 0; ctr < strlen(buffer2Ptr); ctr++)
> {
> printf("char[%d] is [%c]\n", ctr, buffer2Ptr[ctr]);
> }
> printf("esclen is [%d]\n", esclen);
> printf("buffer2Ptr is [%s]\n", buffer2Ptr);
> result in the following output?
> char[0] is [\]
> char[1] is [\]
> char[2] is [\]
> char[3] is [\]
> esclen is [5]
> buffer2Ptr is []
Looks okay to me. Note the 7.2 manual mistakenly claims that esclen
doesn't include the trailing null in the output. According to the
current manual, it does.
regards, tom lane
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
[HACKERS] PQescapeBytea v 7.2.3 BUG?
should
sprintf(buffer, "%c", 0x5C);
readsz = 1;
buffer2Ptr =(unsigned char *) PQescapeBytea(buffer, readsz, &esclen);
for (ctr = 0; ctr < strlen(buffer2Ptr); ctr++)
{
printf("char[%d] is [%c]\n", ctr, buffer2Ptr[ctr]);
}
printf("esclen is [%d]\n", esclen);
printf("buffer2Ptr is [%s]\n", buffer2Ptr);
result in the following output?
char[0] is [\]
char[1] is [\]
char[2] is [\]
char[3] is [\]
esclen is [5]
buffer2Ptr is []
OR should it result in
char[0] is [\]
char[1] is [\]
esclen is [3]
buffer2Ptr is [\\]
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] The database system is in recovery mode
Tom Lane wrote: > "Iavor Raytchev" <[EMAIL PROTECTED]> writes: > > Before the crash is this one - > > FATAL 1: Sorry, too many clients already > > That should be harmless --- I doubt it's relevant to the crash, unless > you have timestamps that prove it happened just before the crash. True, after the recovery we had this several times without a crash. > > IpcSemaphoreLock: semop(id=-1) failed: Invalid argument > > This is a new one on me. AFAICT it must mean that something clobbered > the PROC array in shared memory (the semId argument to IpcSemaphoreLock > is always taken from MyProc->sem.semId). Never heard of that happening > before. > > > DEBUG: pq_flush: send() failed: Bad file descriptor > > And that seems odd too; it suggests something overwrote MyProcPort->sock, > which is another un-heard-of failure mode. > > > Shall we simply restart? > > Yeah, I'd try that, but I'd also suggest looking for system-wide > problems. Normally the postmaster can recover by itself from any sort > of failure in child processes ... the fact that it didn't seem to do so > is another strikingly odd behavior. I'm starting to wonder about > possible hardware flakiness. Bad RAM maybe? There was something strange - one (old?) db connection that refused to be killed. After we managed to shut down the postmaster (not sure anymore, but I think we had to kill it) - this db connection could be killed as well. After new start of the postmaster all was OK. Seems there has been something stuck that has been old and now it was just triggered... We still investigate the surrounding sofware and hardware. Thanks for the help. Iavor ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
