Re: [PATCHES] [HACKERS] snprintf() argument reordering not working under Windows in 8.1
2005/12/4, Andrew Dunstan [EMAIL PROTECTED]: Tom said: Would it work to modify c.h so that it #include's libintl.h, then #undefs these macros, then #includes port.h to define 'em the way we want? Some or all of this might need to be #ifdef WIN32, but that seems like a reasonably noninvasive solution if it can work. IIRC last time I tried this it didn't work too well ;-( I will have another go. I think it's the best way to go. Very well, I will try to put up a patch to implement it in a couple of days. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] snprintf() argument reordering not working under Windows in 8.1
2005/12/6, Nicolai Tufar [EMAIL PROTECTED]: IIRC last time I tried this it didn't work too well ;-( I will have another go. I think it's the best way to go. Very well, I will try to put up a patch to implement it in a couple of days. Oh boy, it is already fixed. Sorry folks, my error. Many thanks to Bruce, Tom and Andrew! Turksh Windows user can breathe easier now. Sincerely, Nicolai Tufar ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] snprintf() argument reordering not working under Windows in 8.1
Greetings, Last April we have made some changes to src/ports/snprintf.c so that it would support argument reordering like %2$s, %1$d and such on platforms where original snprintf() does not support it, like Windows, HP-UX or NetBSD. NLS messages of some languages, like Turkish, rely heavily on argument reordering because of the language structure. In 8.1 Turkish messages in Windows version are all broken because argument reordering is not there. I examined commit logs and came to conclusion that src/port/snprintf.c is not included in server when compiling under Windows because of change to src/port/Makefile made in revision 1.28 by Bruce Momjian. See here: http://developer.postgresql.org/cvsweb.cgi/pgsql/src/port/Makefile Comment to the commit says: `No server version of snprintf needed, so remove Makefile rule.' In fact I think we need snprintf in server because NLS messages are printed by the server. Kindest regards, Nicolai Tufar ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] DTrace Probes?
On 6/17/05, Josh Berkus josh@agliodbs.com wrote: Hey, Folks, I need to find someone who's really interesed in working with DTrace. Sun has offered to help put DTrace probes into PostgreSQL for advanced profiling, but need to know where to probe. Anyone? I'm afraid that I won't get around to this quickly enough. I played a little with DTrace probes when Solaris 10 just came out. DTrace is useful when you have no source code of application or when you are collecting statistics on a live system. Otherwise it is not much different from gprof apart, maybe, that it can collect statistics about kernel syscalls. Anyways, DTrace is a very powerful yet lightweight tool. Creating a strace program to attach to a running PostgreSQL instance and collect statistics will be a nice thing to do. We may even find some bottlenecks in our code. I can volunteer to do it but I do not have a through understanding of PostgreSQL internals. --Josh Regards, Nicolai ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Wed, 16 Mar 2005 01:00:21 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: I have applied a modified version of your patch, attached. I am so sorry, I sent untested patch again. Thank you very much for patience in fixing it. The patch looks perfectly fine and works under Solaris. Under win32 I am still struggling with build environment. In many directories link fails with undefined reference to `pg_snprintf' in other it fails with undefined reference to `_imp__libintl_sprintf'. In yet another directory it fails with both, like in src/interfaces/ecpg/pgtypeslib: dlltool --export-all --output-def pgtypes.def numeric.o datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o dllwrap -o libpgtypes.dll --dllname libpgtypes.dll --def pgtypes.def numeric.o datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o -L../../../../src/port -lm numeric.o(.text+0x19ea):numeric.c: undefined reference to `_imp__libintl_sprintf' datetime.o(.text+0x476):datetime.c: undefined reference to `pg_snprintf' common.o(.text+0x1cd):common.c: undefined reference to `pg_snprintf' common.o(.text+0x251):common.c: undefined reference to `pg_snprintf' dt_common.o(.text+0x538):dt_common.c: undefined reference to `_imp__libintl_sprintf' dt_common.o(.text+0x553):dt_common.c: undefined reference to `_imp__libintl_sprintf' dt_common.o(.text+0x597):dt_common.c: undefined reference to `_imp__libintl_sprintf' dt_common.o(.text+0x5d5):dt_common.c: undefined reference to `_imp__libintl_sprintf' dt_common.o(.text+0x628):dt_common.c: undefined reference to `_imp__libintl_sprintf' dt_common.o(.text+0x7e8):dt_common.c: more undefined references to `_imp__libintl_sprintf' follow c:\MinGW\bin\dllwrap.exe: c:\MinGW\bin\gcc exited with status 1 make: *** [libpgtypes.a] Error 1 Could someone with a better grasp of configure and win32 environment check it? Aparently no one regularily compiles source code under win32 during development cycle these days. Best regards, Nicolai ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
Resubmission of yesterday's patch so that it would
cont conflict with Bruce's cvs commit. Pleas apply.
Best regards,
Nicolai.
On Sat, 12 Mar 2005 01:58:15 +0200, Nicolai Tufar [EMAIL PROTECTED] wrote:
On Thu, 10 Mar 2005 19:21:41 -0500 (EST), Bruce Momjian
pgman@candle.pha.pa.us wrote:
The CVS-tip implementation is fundamentally broken and won't work even
for our internal uses. I've not wasted time complaining about it
because I thought we were going to replace it. If we can't find a
usable replacement then we're going to have to put a lot of effort
into fixing what's there. On the whole I think the effort would be
better spent importing someone else's solution.
Oh, so our existing implementation doesn't even meet our needs. OK.
Which made me wander why did I not aggree with
Tom Lane's suggestion to make do three passes
instead of two. Tom was right, as usual. It happened to
be much easier than I expected. The patch is attached.
Please apply.
Tom, what do you think? Will it be fine with you?
Best regards,
Nicolai
*** ./src/port/snprintf.c.orig Sat Mar 12 09:13:43 2005
--- ./src/port/snprintf.c Sat Mar 12 10:01:44 2005
***
*** 195,200
--- 195,202
int pointflag;
char func;
int realpos;
+ int longflag;
+ int longlongflag;
} *fmtpar, **fmtparptr;
/* Create enough structures to hold all arguments */
***
*** 264,275
--- 266,279
realpos = position;
len = 0;
goto nextch;
+ /*
case '*':
if (pointflag)
maxwidth = va_arg(args, int);
else
len = va_arg(args, int);
goto nextch;
+ */
case '.':
pointflag = 1;
goto nextch;
***
*** 301,316
#endif
case 'u':
case 'U':
! /* fmtnum(value,base,dosign,ljust,len,zpad,output) */
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 305,312
#endif
case 'u':
case 'U':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 325,340
break;
case 'o':
case 'O':
! /* fmtnum(value,base,dosign,ljust,len,zpad,output) */
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 321,328
break;
case 'o':
case 'O':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 349,365
break;
case 'd':
case 'D':
! if (longflag)
! {
! if (longlongflag)
! {
! value = va_arg(args, int64);
! }
! else
! value = va_arg(args, long);
! }
! else
! value = va_arg(args, int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 337,344
break;
case 'd':
case 'D':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 373,387
fmtpos++;
break;
case 'x':
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 352,359
fmtpos++;
break;
case 'x':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 395,409
fmtpos++;
break;
case 'X':
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Thu, 10 Mar 2005 19:21:41 -0500 (EST), Bruce Momjian
pgman@candle.pha.pa.us wrote:
The CVS-tip implementation is fundamentally broken and won't work even
for our internal uses. I've not wasted time complaining about it
because I thought we were going to replace it. If we can't find a
usable replacement then we're going to have to put a lot of effort
into fixing what's there. On the whole I think the effort would be
better spent importing someone else's solution.
Oh, so our existing implementation doesn't even meet our needs. OK.
Which made me wander why did I not aggree with
Tom Lane's suggestion to make do three passes
instead of two. Tom was right, as usual. It happened to
be much easier than I expected. The patch is attached.
Please apply.
Tom, what do you think? Will it be fine with you?
Best regards,
Nicolai
*** ./src/port/snprintf.c.orig Sat Mar 12 01:28:49 2005
--- ./src/port/snprintf.c Sat Mar 12 01:08:30 2005
***
*** 195,200
--- 195,202
int pointflag;
char func;
int realpos;
+ int longflag;
+ int longlongflag;
} *fmtpar, **fmtparptr;
/* Create enough structures to hold all arguments */
***
*** 263,274
--- 265,278
realpos = position;
len = 0;
goto nextch;
+ /*
case '*':
if (pointflag)
maxwidth = va_arg(args, int);
else
len = va_arg(args, int);
goto nextch;
+ */
case '.':
pointflag = 1;
goto nextch;
***
*** 300,315
#endif
case 'u':
case 'U':
! /* fmtnum(value,base,dosign,ljust,len,zpad,output) */
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 304,311
#endif
case 'u':
case 'U':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 324,339
break;
case 'o':
case 'O':
! /* fmtnum(value,base,dosign,ljust,len,zpad,output) */
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 320,327
break;
case 'o':
case 'O':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 348,364
break;
case 'd':
case 'D':
! if (longflag)
! {
! if (longlongflag)
! {
! value = va_arg(args, int64);
! }
! else
! value = va_arg(args, long);
! }
! else
! value = va_arg(args, int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 336,343
break;
case 'd':
case 'D':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 372,386
fmtpos++;
break;
case 'x':
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 351,358
fmtpos++;
break;
case 'x':
! fmtpar[fmtpos].longflag = longflag;
! fmtpar[fmtpos].longlongflag = longlongflag;
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
***
*** 394,408
fmtpos++;
break;
case 'X':
! if (longflag)
! {
! if (longlongflag)
! value = va_arg(args, uint64);
! else
! value = va_arg(args, unsigned long);
! }
! else
! value = va_arg(args, unsigned int);
fmtpar[fmtpos].fmtbegin = fmtbegin;
fmtpar[fmtpos].fmtend = format;
fmtpar[fmtpos].numvalue = value;
--- 366,373
fmtpos++;
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Wed, 9 Mar 2005 22:51:27 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: What do you think about it? Shall I abandon FreeBSD and go ahead Incorporating Trio? Yes, maybe just add the proper %$ handling from Trio to what we have now. Adding proper %$ from Trio will require too much effort. I would rather not do it. Not because I am lazy but because: 1) Trio team seem to be very serious about standards, update the library as soon as new standards come out: quote Trio fully implements the C99 (ISO/IEC 9899:1999) and UNIX98 (the Single Unix Specification, Version 2) standards, as well as many features from other implementations, e.g. the GNU libc and BSD4. /quote 2) If we integrate the whole library in source code we will not have to maintain it and will rely on Trio team for bug fixes and updates. Integrating it will be very easy since all of the functions begin with trio_. I used it instead of the src/port/snrpintf.c one and it passes regression tests under Win32 just fine. The downside is that Trio library is rather big. It is 3 .c and 6 .h files totalling 11556 lines. Compiled it is 71224 bytes not stripped and 56204 bytes stripped on Solaris 10 for x86, 32-bit. Even for a shared library it will probably be too much. Trio has a lot of string handling functions which are probably not necessary. Would you like me to try to remove everything unnecessary from it or we will settle with the full version? Regards, Nicolai Tufar ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests
On Thu, 10 Mar 2005 16:26:47 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Please see my posting about using a macro for snprintf. If the current implementation of snprintf is enough for our existing translation users we probably don't need to add anything more to it because snprintf will not be exported to client applications. Oh, Bruce. It will be the best solution. I was worried about the problems with my modifications to snprintf.c Tom Lane pointed out. But if we really separate snprintf() used by messages and snprintf() used by the like of src/backend/utils/adt/int8.c then we are safe. We can claim current release safe and I will modify src/port/snprintf.c at my leisure later. I will try out your modifications tomorrow. It is late here and I have a PostgreSQL class to to teach tomorrow ;) I still think that it is more convenient to rip off current implementation of snprintf.c and replace it with a very much stripped down of Trio's one. I will work on it and try to get a patch in one week's time. Thank you all for your patience. Best regards, Nicolai Tufar -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
Tom Lane wrote: The CVS-tip implementation is fundamentally broken and won't work even for our internal uses. I've not wasted time complaining about it because I thought we were going to replace it. If we can't find a usable replacement then we're going to have to put a lot of effort into fixing what's there. On the whole I think the effort would be better spent importing someone else's solution. Bruce Momjian wrote: Oh, so our existing implementation doesn't even meet our needs. OK. Very well, I too, tend to think that importing some else's solution is the way to go. Tom, have you looked at Trio? If it is fine with you too, I will strip it to the bare minimum needed for snprintf(), vsnprintf() and printf() by Saturday. Regards, Nicolai Tufar ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Fri, 11 Mar 2005 01:14:31 -0500, Tom Lane wrote: Nicolai Tufar [EMAIL PROTECTED] writes: Very well, I too, tend to think that importing some else's solution is the way to go. Tom, have you looked at Trio? I have not seen it ... but if it looks reasonable to you, have a go at it. It looks reasonable to me. It claims to: fully implement C99 (ISO/IEC 9899:1999) and UNIX98 (the Single Unix Specification, Version 2) standards, as well as many features from other implementations, e.g. the GNU libc and BSD4. I compiled and run regression tests with it used instead of our current implementation and it worked fine under win32 and Solaris x86. The only problem is that it is 11000 lines as oposed to our curent implementation's 600. I will remove everything unnecessary and submit a patch for consideration. Regards, Nicolai Tufar ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
Dear all, After struggling for one week to to integrate FreeBSD's vfprintf.c into PostgreSQL I finally gave up. It is too dependent on underlying FreeBSD system functions. To incorporate it into PostgreSQL we need to move vfprintf.c file itself, two dozen files form gdtoa and a half a dozen __XXtoa.c files scattered in apparently random fashion all around FreeBSD source tree. Instead I researched some other implementations of snprintf on the web released under a license compatible with PostgreSQL's. The most suitable one I have come upon is Trio [http://daniel.haxx.se/projects/trio/]. It is distributed under a MIT-like license which, I think will be compatible with us. What do you think about it? Shall I abandon FreeBSD and go ahead ncorporatng Tro? And by the way, what s the concluson of snprntf() vs. pg_snprintf() and UNIX libraries discussion a week ago? Which one shall I implement? Regards, Nicolai Tufar ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Wed, 2 Mar 2005 09:24:32 +0100, Joerg Hessdoerfer [EMAIL PROTECTED] wrote: don't know if PG borrowed the code from here, but according to the manpage FreeBSD 5.3 seems to have a quite complete implementation, see http://www.freebsd.org/cgi/man.cgi?query=snprintfapropos=0sektion=3manpath=FreeBSD+5.3-RELEASE+and+Portsformat=html Would this help? Yes, it would. With Tom Lane's blessing I am starting on incorporating it into PG now. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef :( Regards, Nick ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
I spent all day debugging it. Still have absolutely no idea what could possibly go wrong. Does anyone have a slightest clue what can it be and why it manifests itself only on win32? On Tue, 1 Mar 2005 09:29:07 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Nicolai Tufar wrote: On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef Sure, in src/backend/utils/adt/int8.c, there is a call in int8out(): if ((len = snprintf(buf, MAXINT8LEN, INT64_FORMAT, val)) 0) and that calls port/snprintf.c. I have added a puts() in snprintf.c to make sure it is getting the long/long specifier. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Tue, 1 Mar 2005 15:38:58 -0500 (EST), [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Is there a reason why we don't use the snprintf that comes with the various C compilers? snprintf() is usually buried in OS libraries. We implement our own snprintf to make things like this: snprintf(buf,%2$s %1$s,world,Hello); which is not supported on some platforms work. We do it for national language translation of messages. In some languages you may need to change order of parameters to make a meaningful sentence. Another question is why we are using it for printing values from database. I am not too good on function overriding in standard C but maybe there is a way to usage of standard snprintf() in a particular place. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression
On Tue, 1 Mar 2005 16:20:50 -0500 (EST), [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Well, here is a stupid question, do we even know which snprintf we are using on Win32? May it be possible that we are using the MingW version which may be broken? Defenitely not. I checked it by placing debugging print statements in code. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [pgsql-hackers-win32] snprintf causes regression tests to fail
On Tue, 01 Mar 2005 17:45:31 -0500, Tom Lane Just out of curiosity, do either HAVE_INT64 or HAVE_UINT64 get set in pg_config.h? pg_config.h is attached. What drew my attention is the following declaration: /* Define to 1 if `long long int' works and is 64 bits. */ #define HAVE_LONG_LONG_INT_64 is it normal? should it not be like this: #define HAVE_LONG_LONG_INT_64 1 /* src/include/pg_config.h. Generated by configure. */ /* src/include/pg_config.h.in. Generated from configure.in by autoheader. */ /* Define to the type of arg 1 of 'accept' */ #define ACCEPT_TYPE_ARG1 unsigned int /* Define to the type of arg 2 of 'accept' */ #define ACCEPT_TYPE_ARG2 struct sockaddr * /* Define to the type of arg 3 of 'accept' */ #define ACCEPT_TYPE_ARG3 int /* Define to the return type of 'accept' */ #define ACCEPT_TYPE_RETURN unsigned int PASCAL /* The alignment requirement of a `double'. */ #define ALIGNOF_DOUBLE 8 /* The alignment requirement of a `int'. */ #define ALIGNOF_INT 4 /* The alignment requirement of a `long'. */ #define ALIGNOF_LONG 4 /* The alignment requirement of a `long long int'. */ #define ALIGNOF_LONG_LONG_INT 8 /* The alignment requirement of a `short'. */ #define ALIGNOF_SHORT 2 /* Define to the default TCP port number on which the server listens and to which clients will try to connect. This can be overridden at run-time, but it's convenient if your clients have the right default compiled in. (--with-pgport=PORTNUM) */ #define DEF_PGPORT 5432 /* Define to the default TCP port number as a string constant. */ #define DEF_PGPORT_STR 5432 /* Define to 1 if you want National Language Support. (--enable-nls) */ /* #undef ENABLE_NLS */ /* Define to 1 to build client libraries as thread-safe code. (--enable-thread-safety) */ /* #undef ENABLE_THREAD_SAFETY */ /* Define to 1 if getpwuid_r() takes a 5th argument. */ /* #undef GETPWUID_R_5ARG */ /* Define to 1 if gettimeofday() takes only 1 argument. */ /* #undef GETTIMEOFDAY_1ARG */ #ifdef GETTIMEOFDAY_1ARG # define gettimeofday(a,b) gettimeofday(a) #endif /* Define to 1 if you have the `atexit' function. */ #define HAVE_ATEXIT 1 /* Define to 1 if you have the `cbrt' function. */ #define HAVE_CBRT 1 /* Define to 1 if you have the `class' function. */ /* #undef HAVE_CLASS */ /* Define to 1 if you have the `crypt' function. */ /* #undef HAVE_CRYPT */ /* Define to 1 if you have the crypt.h header file. */ /* #undef HAVE_CRYPT_H */ /* Define to 1 if you have the declaration of `fdatasync', and to 0 if you don't. */ #define HAVE_DECL_FDATASYNC 0 /* Define to 1 if you have the declaration of `snprintf', and to 0 if you don't. */ #define HAVE_DECL_SNPRINTF 1 /* Define to 1 if you have the declaration of `vsnprintf', and to 0 if you don't. */ #define HAVE_DECL_VSNPRINTF 1 /* Define to 1 if you have the dld.h header file. */ /* #undef HAVE_DLD_H */ /* Define to 1 if you have the `dlopen' function. */ /* #undef HAVE_DLOPEN */ /* Define to 1 if you have the editline/history.h header file. */ /* #undef HAVE_EDITLINE_HISTORY_H */ /* Define to 1 if you have the editline/readline.h header file. */ /* #undef HAVE_EDITLINE_READLINE_H */ /* Define to 1 if you have the endian.h header file. */ /* #undef HAVE_ENDIAN_H */ /* Define to 1 if you have the `fcvt' function. */ #define HAVE_FCVT 1 /* Define to 1 if you have the `fdatasync' function. */ /* #undef HAVE_FDATASYNC */ /* Define to 1 if you have finite(). */ #define HAVE_FINITE 1 /* Define to 1 if you have the `fpclass' function. */ /* #undef HAVE_FPCLASS */ /* Define to 1 if you have the `fp_class' function. */ /* #undef HAVE_FP_CLASS */ /* Define to 1 if you have the `fp_class_d' function. */ /* #undef HAVE_FP_CLASS_D */ /* Define to 1 if you have the fp_class.h header file. */ /* #undef HAVE_FP_CLASS_H */ /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */ /* #undef HAVE_FSEEKO */ /* Define to 1 if your compiler understands __func__. */ #define HAVE_FUNCNAME__FUNC 1 /* Define to 1 if your compiler understands __FUNCTION__. */ /* #undef HAVE_FUNCNAME__FUNCTION */ /* Define to 1 if you have the `getaddrinfo' function. */ /* #undef HAVE_GETADDRINFO */ /* Define to 1 if you have the `gethostbyname_r' function. */ /* #undef HAVE_GETHOSTBYNAME_R */ /* Define to 1 if you have the `gethostname' function. */ /* #undef HAVE_GETHOSTNAME */ /* Define to 1 if you have the `getopt' function. */ #define HAVE_GETOPT 1 /* Define to 1 if you have the getopt.h header file. */ #define HAVE_GETOPT_H 1 /* Define to 1 if you have the `getopt_long' function. */ #define HAVE_GETOPT_LONG 1 /* Define to 1 if you have the `getpeereid' function. */ /* #undef HAVE_GETPEEREID */ /* Define to 1 if you have the `getpwuid_r' function. */ /* #undef HAVE_GETPWUID_R */ /* Define to 1 if you have the `getrusage' function. */ /* #undef HAVE_GETRUSAGE */ /* Define to 1 if you have the history.h header file. */ /* #undef HAVE_HISTORY_H */ /* Define to 1 if
Re: [HACKERS] snprintf causes regression tests to fail
Linux and Solaris 10 x86 pass regression tests fine when I force the use of new snprintf(). The problem should be win32 - specific. I will investigate it throughly tonight. Can someone experienced in win32 what can possibly be the problem? Nick On Sun, 27 Feb 2005 19:07:16 +0100, Magnus Hagander [EMAIL PROTECTED] wrote: Hi! The new snpritnf code appears not to deal with 64-bit ints. I'm getting failures on win32 for int8 as well as all the time related tests (win32 uses int8 for tinmestamps). Removing the snprintf code and falling back to the OS code makes everything pass again. I would guess this affects int8 etc on other platforms as well (assuming they use our snprintf and not the libc one), but I haven't checked it. //Magnus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] snprintf causes regression tests to fail
After some extensive debugging with Magnus's
help we finally managed to a kind of isolate the
problem. We placed snprintf.c in a separate
file, added necessary #includes and wrote
a simple main() function:
main()
{
unsigned long long ull=4567890123456789ULL;
static char buf[1024];
mysnprintf(buf,1024,%lld\n,ull);
printf(buf);
}
When compiled with -D HAVE_LONG_LONG_INT_64=1
which declares long_long and ulong_long like:
typedef long long long_long;
typedef unsigned long long ulong_long;
It compiles fine and produces desired result. If not,
it produces -869367531 as in regression tests.
Amazingly enough HAVE_LONG_LONG_INT_64 is
defined when compilation comes to src/port/snprintf.c
but the result is still wrong. I looked into configure.in
but the check for HAVE_LONG_LONG_INT_64 is too
complicated for me to understand. Bruce, could you
take a look at this? I am 90% sure it is an issue with
some configure definitions.
Best regards,
Nicolai
On Mon, 28 Feb 2005 19:58:15 +0200, Nicolai Tufar [EMAIL PROTECTED] wrote:
Regression test diff is attached.
It fails on the following tests:
int8
subselect
union
sequence
It fails to display correctly number 4567890123456789.
In output is shows -869367531. Apparent overflow or
interpreting int8 as int4.
while rewriting snprintf() I did not touch the actual functions
that convert number to ASCII digit string. In truth, if you
force PostgreSQL to use snprintf.c without my patch applied
it produces the same errors.
What can be wrong? GCC bug? The one I use is:
gcc.exe (GCC) 3.3.1 (mingw special 20030804-1)
Any thoughts?
On Mon, 28 Feb 2005 09:17:20 -0500 (EST), Bruce Momjian
pgman@candle.pha.pa.us wrote:
Nicolai Tufar wrote:
Linux and Solaris 10 x86 pass regression tests fine when I force the use
of new
snprintf(). The problem should be win32 - specific. I will
investigate it throughly
tonight. Can someone experienced in win32 what can possibly be the
problem?
Yea, I am confused too because my BSD uses the new snprintf.c code was
well. Magnus, what failures are you seeing on Win32?
--
Bruce Momjian| http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] snprintf causes regression tests to fail
Neither Bruce's nor subsequent Tom's patch did not fix the issue. The command used is: make maintainer-clean ./configure make make install make check It should have be fine to recompile the source code completely. I attach the resulting config.log. May be it will give a clue. Regression test failure are in the same places as previous ones. Best regards, Nicolai On Mon, 28 Feb 2005 16:29:57 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Ah, the problem was introduced here: Right, it was my fault. The problem is that the PGAC_FUNC_PRINTF_ARG_CONTROL call was moved below the printf 64-bit tests. This commited patch moves PGAC_FUNC_PRINTF_ARG_CONTROL which is after we know AC_TRY_RUN works and just before printf 64-bit args are tested. This patch breaks things in a different way: you should not have moved the AC_LIBOBJ(snprintf) step. Also you randomly placed the arg-control test between a chunk of code and the comment describing same. Thanks. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Thread-safe snprintf() vsnprintf() and printf()
And while we are on it, I would like to submit minor
changes to make snprintf() vsnprintf() and printf()
functions in src/port/snprintf.c thread-safe.
Best regards,
Nicolai Tufar
Index: src/port/snprintf.c
===
RCS file: /projects/cvsroot/pgsql/src/port/snprintf.c,v
retrieving revision 1.7
diff -c -r1.7 snprintf.c
*** src/port/snprintf.c 28 Feb 2005 14:16:16 - 1.7
--- src/port/snprintf.c 28 Feb 2005 23:11:22 -
***
*** 82,105
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nast effects.
**/
! /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.7 2005/02/28 14:16:16 momjian Exp $;*/
! static char *end;
! static int SnprfOverflow;
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args);
int
printf(const char *fmt,...)
{
int len;
va_list args;
! static char* buffer[4096];
char* p;
va_start(args, fmt);
--- 82,103
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nasty effects.
**/
! /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args, char *end);
int
printf(const char *fmt,...)
{
int len;
va_list args;
! char* buffer[4096];
char* p;
va_start(args, fmt);
***
*** 127,136
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
str[0] = '\0';
end = str + count - 1;
! SnprfOverflow = 0;
! dopr(str, fmt, args);
if (count 0)
end[0] = '\0';
return strlen(str);
--- 125,134
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
+ char *end;
str[0] = '\0';
end = str + count - 1;
! dopr(str, fmt, args, end);
if (count 0)
end[0] = '\0';
return strlen(str);
***
*** 140,150
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag);
! static void dostr(char *str, int cut);
! static void dopr_outch(int c);
static char *output;
--- 138,148
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth, char *end);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad, char *end);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag, char *end);
! static void dostr(char *str, int cut, char *end);
! static void dopr_outch(int c, char *end);
static char *output;
***
*** 154,160
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args)
{
int ch;
long_long value;
--- 152,158
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args, char *end)
{
int ch;
long_long value;
***
*** 417,427
case '%':
break;
default:
! dostr(???, 0);
}
break;
default:
! dopr_outch(ch);
break;
}
}
--- 415,425
case '%':
break;
default:
! dostr(???, 0, end);
}
break;
default:
! dopr_outch(ch, end);
break;
}
}
***
*** 448,474
case FMTSTR:
fmtstr(fmtparptr[i]-value, fmtparptr[i]-ljust,
fmtparptr[i]-len, fmtparptr[i]-zpad,
! fmtparptr[i]-maxwidth);
break;
case FMTNUM:
fmtnum(fmtparptr[i]-numvalue, fmtparptr[i]-base,
fmtparptr[i]-dosign, fmtparptr[i]-ljust,
! fmtparptr[i]-len, fmtparptr[i]-zpad);
break;
case FMTFLOAT:
fmtfloat(fmtparptr[i]-fvalue, fmtparptr[i]-type,
fmtparptr[i]-ljust, fmtparptr[i]-len,
! fmtparptr[i]-precision, fmtparptr[i]-pointflag);
break;
case FMTCHAR:
! dopr_outch(fmtparptr[i]-charvalue);
break;
}
format = fmtpar[i].fmtend;
goto nochar;
}
}
! dopr_outch(ch);
nochar:
/* nothing */
; /* semicolon required
Re: [HACKERS] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c
On Thu, 24 Feb 2005 22:18:11 -0500, Tom Lane [EMAIL PROTECTED] wrote: Didn't we do that already? No :( I promised to do it a couple of weeks ago but could not get to do it. Now with Magnus's help I finaly did it. The last patch should be fine. regards, tom lane Nicolai ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Repleacement for src/port/snprintf.c
On Mon, Feb 21, 2005 at 10:53:08PM -0500, Bruce Momjian wrote:
Applied.
Thanks a lot. The patch attached solves the tread
safety problem. Please review it before applying,
I am not sure I am doing the right thing
On Tue, 22 Feb 2005 19:57:15 +0100, Kurt Roeckx [EMAIL PROTECTED] wrote:
The configure test is a little broken. It needs to quote the
$'s.
I've rewritten the test a little.
This one needs applying too. $'s do get scrambled.
Best regards,
Nicolai.
*** ./src/port/snprintf.c.orig sali Şub 22 20:02:03 2005
--- ./src/port/snprintf.c sali Şub 22 21:59:48 2005
***
*** 80,96
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nast effects.
**/
/*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
- static char *end;
- static int SnprfOverflow;
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args);
int
printf(const char *fmt,...)
--- 80,94
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
! * causing nasty effects.
**/
/*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.6 2005/02/22 04:57:24 momjian Exp $;*/
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt, va_list args);
int printf(const char *format, ...);
! static void dopr(char *buffer, const char *format, va_list args, char *end);
int
printf(const char *fmt,...)
***
*** 97,103
{
int len;
va_list args;
! static char* buffer[4096];
char* p;
va_start(args, fmt);
--- 95,101
{
int len;
va_list args;
! char* buffer[4096];
char* p;
va_start(args, fmt);
***
*** 125,134
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
str[0] = '\0';
end = str + count - 1;
! SnprfOverflow = 0;
! dopr(str, fmt, args);
if (count 0)
end[0] = '\0';
return strlen(str);
--- 123,132
int
vsnprintf(char *str, size_t count, const char *fmt, va_list args)
{
+ char *end;
str[0] = '\0';
end = str + count - 1;
! dopr(str, fmt, args, end);
if (count 0)
end[0] = '\0';
return strlen(str);
***
*** 138,148
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag);
! static void dostr(char *str, int cut);
! static void dopr_outch(int c);
static char *output;
--- 136,146
* dopr(): poor man's version of doprintf
*/
! static void fmtstr(char *value, int ljust, int len, int zpad, int maxwidth, char *end);
! static void fmtnum(long_long value, int base, int dosign, int ljust, int len, int zpad, char *end);
! static void fmtfloat(double value, char type, int ljust, int len, int precision, int pointflag, char *end);
! static void dostr(char *str, int cut, char *end);
! static void dopr_outch(int c, char *end);
static char *output;
***
*** 152,158
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args)
{
int ch;
long_long value;
--- 150,156
#define FMTCHAR 4
static void
! dopr(char *buffer, const char *format, va_list args, char *end)
{
int ch;
long_long value;
***
*** 415,425
case '%':
break;
default:
! dostr(???, 0);
}
break;
default:
! dopr_outch(ch);
break;
}
}
--- 413,423
case '%':
break;
default:
! dostr(???, 0, end);
}
break;
default:
! dopr_outch(ch, end);
break;
}
}
***
*** 446,465
case FMTSTR:
fmtstr(fmtparptr[i]-value, fmtparptr[i]-ljust,
fmtparptr[i]-len, fmtparptr[i]-zpad,
! fmtparptr[i]-maxwidth);
break;
case FMTNUM:
fmtnum(fmtparptr[i]-numvalue, fmtparptr[i]-base,
fmtparptr[i]-dosign, fmtparptr[i]-ljust,
! fmtparptr[i]-len, fmtparptr[i]-zpad);
break;
case FMTFLOAT:
fmtfloat(fmtparptr[i]-fvalue, fmtparptr[i]-type,
fmtparptr[i]-ljust, fmtparptr[i]-len,
! fmtparptr[i]-precision, fmtparptr[i]-pointflag);
break;
case FMTCHAR:
! dopr_outch(fmtparptr[i]-charvalue);
break;
}
Re: [HACKERS] [pgsql-hackers-win32] Repleacement for src/port/snprintf.c
On Sun, 13 Feb 2005 19:06:34 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Anyway, this is too large to put into 8.0, but I am attaching a patch for 8.1 that has the proper configure tests to check if the C library supports this behavior. If it does not, the build will use our port/snprintf.c. One problem with that is that our snprintf.c is not thread-safe. Seems the increases use of it will require us to fix this soon. I have added to TODO: * Make src/port/snprintf.c thread-safe Okay, I am applying your patch to CVS HEAD and getting hands on making snprintf.c thread-safe. I will submit a roll up pathch in a day or two. Regards, Nicolai ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Repleacement for src/port/snprintf.c
Hello all,
I would like to submit my changes to src/port/snprintf.c to
enable %n$ format placeholder replacement in snprintf() and
vsnprintf(). Additionally I implemented a trivial printf().
I also attach a diff for configure.in to include snprintf.o
in pgport but I am sure it is not the right thing to do.
Could someone give a hint on where I need to place such a
definition.
Please review my patch. as Tom Lane pointed out there
are 150 messages in the following files that do not print
properly:
src/backend/po/pt_BR.po
src/backend/po/de.po
src/backend/po/es.po
src/backend/po/zh_CN.po
src/backend/po/tr.po
src/bin/pg_dump/po/zh_CN.po
src/bin/pg_dump/po/tr.po
src/bin/psql/po/zh_CN.po
src/bin/psql/po/zh_TW.po
src/bin/psql/po/tr.po
src/bin/scripts/po/zh_CN.po
we need to fix snprintf.c before next release
Best regards,
Nicolai Tufar
*** ./src/port/snprintf.c.orig 2005-01-20 01:27:22.0 +0200
--- ./src/port/snprintf.c 2005-01-24 03:09:31.0 +0200
***
*** 57,62
--- 57,66
typedef unsigned long ulong_long;
#endif
+ #ifndef NL_ARGMAX
+ #define NL_ARGMAX 4096
+ #endif
+
/*
**SNPRINTF, VSNPRINT -- counted versions of printf
**
***
*** 85,93
--- 89,115
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt,
va_list args);
+ int printf(const char *format, ...);
static void dopr(char *buffer, const char *format, va_list args);
int
+ printf(const char *fmt,...)
+ {
+ int len;
+ va_list args;
+ static char*buffer[4096];
+ char* p;
+
+ va_start(args, fmt);
+ len = vsnprintf((char*)buffer, (size_t)4096, fmt, args);
+ va_end(args);
+ p = (char*)buffer;
+ for(;*p;p++)
+ putchar(*p);
+ return len;
+ }
+
+ int
snprintf(char *str, size_t count, const char *fmt,...)
{
int len;
***
*** 124,129
--- 146,155
static char *output;
+ #define FMTSTR 1
+ #define FMTNUM 2
+ #define FMTFLOAT3
+ #define FMTCHAR 4
static void
dopr(char *buffer, const char *format, va_list args)
***
*** 139,145
--- 165,200
int ljust;
int len;
int zpad;
+ int i;
+ const char* format_save;
+ const char* fmtbegin;
+ int fmtpos = 1;
+ int realpos = 0;
+ int position;
+ static struct{
+ const char* fmtbegin;
+ const char* fmtend;
+ union{
+ void* value;
+ long_long numvalue;
+ double fvalue;
+ int charvalue;
+ };
+ int ljust;
+ int len;
+ int zpad;
+ int maxwidth;
+ int base;
+ int dosign;
+ chartype;
+ int precision;
+ int pointflag;
+ charfunc;
+ int realpos;
+ }fmtpar[NL_ARGMAX+1], *fmtparptr[NL_ARGMAX+1];
+
+ format_save = format;
output = buffer;
while ((ch = *format++))
{
***
*** 148,161
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! dostr(**end of format**, 0);
! *output = '\0';
! return;
case '-':
ljust = 1;
goto nextch;
--- 203,217
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
+ fmtbegin = format - 1;
+ realpos = 0;
+ position = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! goto performpr
[HACKERS] Repleacement for src/port/snprintf.c
Greetings,
I would like to submit my changes to src/port/snprintf.c to
enable %n$ format placeholder replacement in snprintf() and
vsnprintf(). Additionally I implemented a trivial printf().
I also attach a diff for configure.in to include snprintf.o
in pgport but I am sure it is not the right thing to do.
Could someone give a hint on where I need to place such a
definition.
Best regards,
Nicolai Tufar
*** ./src/port/snprintf.c.orig 2005-01-20 01:27:22.0 +0200
--- ./src/port/snprintf.c 2005-01-24 03:09:31.0 +0200
***
*** 57,62
--- 57,66
typedef unsigned long ulong_long;
#endif
+ #ifndef NL_ARGMAX
+ #define NL_ARGMAX 4096
+ #endif
+
/*
**SNPRINTF, VSNPRINT -- counted versions of printf
**
***
*** 85,93
--- 89,115
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt,
va_list args);
+ int printf(const char *format, ...);
static void dopr(char *buffer, const char *format, va_list args);
int
+ printf(const char *fmt,...)
+ {
+ int len;
+ va_list args;
+ static char*buffer[4096];
+ char* p;
+
+ va_start(args, fmt);
+ len = vsnprintf((char*)buffer, (size_t)4096, fmt, args);
+ va_end(args);
+ p = (char*)buffer;
+ for(;*p;p++)
+ putchar(*p);
+ return len;
+ }
+
+ int
snprintf(char *str, size_t count, const char *fmt,...)
{
int len;
***
*** 124,129
--- 146,155
static char *output;
+ #define FMTSTR 1
+ #define FMTNUM 2
+ #define FMTFLOAT3
+ #define FMTCHAR 4
static void
dopr(char *buffer, const char *format, va_list args)
***
*** 139,145
--- 165,200
int ljust;
int len;
int zpad;
+ int i;
+ const char* format_save;
+ const char* fmtbegin;
+ int fmtpos = 1;
+ int realpos = 0;
+ int position;
+ static struct{
+ const char* fmtbegin;
+ const char* fmtend;
+ union{
+ void* value;
+ long_long numvalue;
+ double fvalue;
+ int charvalue;
+ };
+ int ljust;
+ int len;
+ int zpad;
+ int maxwidth;
+ int base;
+ int dosign;
+ chartype;
+ int precision;
+ int pointflag;
+ charfunc;
+ int realpos;
+ }fmtpar[NL_ARGMAX+1], *fmtparptr[NL_ARGMAX+1];
+
+ format_save = format;
output = buffer;
while ((ch = *format++))
{
***
*** 148,161
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! dostr(**end of format**, 0);
! *output = '\0';
! return;
case '-':
ljust = 1;
goto nextch;
--- 203,217
case '%':
ljust = len = zpad = maxwidth = 0;
longflag = longlongflag = pointflag = 0;
+ fmtbegin = format - 1;
+ realpos = 0;
+ position = 0;
nextch:
ch = *format++;
switch (ch)
{
case 0:
! goto performpr;
case '-':
ljust = 1;
goto nextch;
***
*** 174,180
--- 230,243
if (pointflag)
maxwidth = maxwidth *
10 + ch - '0';
else
Re: [HACKERS] Goals for 8.1
On Sun, 23 Jan 2005 21:43:17 -0800, Benjamin Arai [EMAIL PROTECTED] wrote: What are the goals for 8.1? Fix %n$ format string argument placement in platforms that do not support it, like HP-UX, Win32 Best regards, Nicolai ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
Greetings,
I would like to submit a new version of src/port/snprintf.c
It passes regression tests on Linux and Win32 and
prints all of %n$ messages finely.
I added printf() function too because --help usage-type
output is printed with printf().
I have no experience with autoconf so I would ask you
for help on what changes to do to include libpgport to
src/Makefile.global and src/Makefile.shlib.
PostgreSQL uses snprintf() in more places than I expected.
So these changes need a through testing. I have no 64-bit
to able to run regression test on it would be nice to run
it on a 64-bit platform too.
Best regards,
Nicolai Tufar
/*
* Copyright (c) 1983, 1995, 1996 Eric P. Allman
* Copyright (c) 1988, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
*notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
*notice, this list of conditions and the following disclaimer in the
*documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
*must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
*may be used to endorse or promote products derived from this software
*without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
/* might be in either frontend or backend */
#include postgres_fe.h
#ifdef ENABLE_THREAD_SAFETY
#error The replacement snprintf() is not thread-safe. \
Your platform must have a thread-safe snprintf() to compile with threads.
#endif
#if !defined(WIN32) !defined(__CYGWIN__)
#include sys/ioctl.h
#endif
#include sys/param.h
/*
* We do all internal arithmetic in the widest available integer type,
* here called long_long (or ulong_long for unsigned).
*/
#ifdef HAVE_LONG_LONG_INT_64
typedef long long long_long;
typedef unsigned long long ulong_long;
#else
typedef long long_long;
typedef unsigned long ulong_long;
#endif
#ifndef NL_ARGMAX
#define NL_ARGMAX 4096
#endif
/*
** SNPRINTF, VSNPRINT -- counted versions of printf
**
** These versions have been grabbed off the net. They have been
** cleaned up to compile properly and support for .precision and
** %lx has been added.
*/
/**
* Original:
* Patrick Powell Tue Apr 11 09:48:21 PDT 1995
* A bombproof version of doprnt (dopr) included.
* Sigh. This sort of thing is always nasty do deal with. Note that
* the version here does not include floating point. (now it does ... tgl)
*
* snprintf() is used instead of sprintf() as it does limit checks
* for string length. This covers a nasty loophole.
*
* The other functions are there to prevent NULL pointers from
* causing nast effects.
**/
/*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.4 2004/08/29
05:07:02 momjian Exp $;*/
static char *end;
static int SnprfOverflow;
int snprintf(char *str, size_t count, const char *fmt,...);
int vsnprintf(char *str, size_t count, const char *fmt,
va_list args);
int printf(const char *format, ...);
static void dopr(char *buffer, const char *format, va_list args);
int
printf(const char *fmt,...)
{
int len;
va_list args;
static char*buffer[4096];
char* p;
va_start(args, fmt);
len = vsnprintf((char*)buffer, (size_t)4096, fmt, args);
va_end(args);
p = (char*)buffer;
for(;*p;p++)
putchar(*p);
return len;
}
int
snprintf(char *str, size_t count, const char
Re: [HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
Greetings, for a couple of days I have been hacking on src/port/snprintf.c. With Magnus' help I have managed to implement argument replacement in snprintf(). The code is very crude and not quite optimised, any suggestions will be more than welcome. Here is what I did: 1. I renamed snprintf() to pg_snprintf(), vsnprintf() to pg_vsnprintf() and introduced pg_printf() that calls pg_vsnprintf(). 2. After running configure I manually added snprintf.o to src/Makefile.global's LIBOBJ declaration and -lpgport to Makefile.shlib's DLLWRAP declaration 3. To make sure these functions are used everywhere I introduced the following lines at the end of src/include/c.h: #define snprintf pg_snprintf #define vsnprintf pg_vsnprintf #define printf pg_printf 4. I introduced a volatile static char[] variable in snprintf.c code so I can grep executables for this string and be sure that it is included. 5. Before running regression test I always ran make install, apparently because libpq is read from /usr/local/. During compilation the following warnings were reported: ../../../src/include/utils/elog.h:121: warning: `pg_printf' is an unrecognized format function type which is perfectly fine because we replace printf with pg_printf and gcc's format() does know anything about it. On Linux, PostgreSQL passed regression tests with flying colours and prints messages with %n$ just fine. On win32: int8, timestamp, timestamptz, abstime, horology, constraints, vacuum, and many others failed. To check my code, I reverted snprintf.c to the original one from CVS and forced win32 port to use these functions and it fails in same places. After examining regression tests diff I came to conclusion that problem is in fmtnum() function when it operates with particularly long integers. In snprintf() file I changed only and only dopr() function, neve touching fmtnum(), fmtstr() or fmtfolat(). I would like to kindly ask these questions: 1. Am I on the right to implement %n$ ? Can it be accepted? 2. Why would we not just take FreeBSD's vfprintf() and use it instead? 3. What is wrong with fmtnum() on Win32? 4. What %m format string is used for? And where is it handled. Do I need to implement it? I am attaching my version if snprintf.c (because it is too different from the original to make a patch) and regression.diff of a failed Win32 regression test produced wither with my or with original snprintf.c. Best regards, Nicolai Tufar ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
On Sat, 22 Jan 2005 15:31:39 +0100, Peter Eisentraut [EMAIL PROTECTED] wrote: Nicolai Tufar wrote: 1. I renamed snprintf() to pg_snprintf(), vsnprintf() to pg_vsnprintf() and introduced pg_printf() that calls pg_vsnprintf(). This is not necessary. You will notice that we already have configure tests of snprintf format specifiers (%lld etc.), so using the original function names is OK even in that case. how about a test for the thing like: printf(%2$s %1$s!\n, world, Hello); It is what I am trying to solve :( 5. Before running regression test I always ran make install, apparently because libpq is read from /usr/local/. That's because of the -rpath. I see. Thanks for information. 2. Why would we not just take FreeBSD's vfprintf() and use it instead? Try it. It's painful. src/port/snprintf.c is not not a particularly pretty. And after my rework it got even uglier. I have looked at FreeBDS's one and it is not *that* complicated. Would you like me to try to incorporate into PostgreSQL. NetBSD's one is somewhat simplier but does not support %n$ feature by the way. 4. What %m format string is used for? And where is it handled. Do I need to implement it? It's only used in the error reporting functions in the server and is handled there. You don't need to worry about it. Oh, that is a relief. Peter Eisentraut Nicolai ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [pgsql-hackers-win32] [HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
On Sat, 22 Jan 2005 17:05:22 -0500, Tom Lane [EMAIL PROTECTED] wrote: src/port/snprintf.c is not not a particularly pretty. And after my rework it got even uglier. I have looked at FreeBDS's one and it is not *that* complicated. If you can get it to work then it'd be a better bet than writing (and having to maintain) our own. Very well, I am starting to work on it. I will put everything necessary in on file. So far it is 1500-odd lines and will probably grow a little bit. Plus we would probably need to include Double-to-ASCII package too: http://cvsup.pt.freebsd.org/cgi-bin/cvsweb/cvsweb.cgi/src/contrib/gdtoa/ A daunting task indeed. I will do it but I am afraid it would not be too portable. I was wandering if reimplementing print formatting is the right thing to do. The problem I have is in fmtnum() and probably in fmtfloat() functions which are very platform-dependent. The code to shuffle xxprinf() arguments based on %n$ formatting stirngs is ready why don't shuffle formatting placeholders too and call OS's vsnprintf() with modified formatting formatting strings and va_list? Or, a similar solution, for every parameter extract formatting placeholder and value, call snprintf() and the combine the resulting string? The first solution requires doing nasty manipulations with va_list, the second will be slower because of multiple calls to snprintf(). Which one is better? Best regards, Nicolai Tufar regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: Translations at pgfoundry (was Re: [HACKERS] [PATCHES] Latest Turkish translation updates)
On Thu, 20 Jan 2005 14:08:20 +0100, Peter Eisentraut [EMAIL PROTECTED] wrote: Peter Eisentraut wrote: Maybe we should have a pgfoundry project where all translations were kept, and from which the main CVS could be updated semi-automatically. Then we wouldn't have Peter checking out and committing all the time. That sounds like a fine idea. My only concern would be the not-maintained-here syndrome, which occurs every time some CVS tree contains a file that is actually maintained by an external group, thus blocking the maintainers of the former CVS tree from applying necessary fixes at times. Nevertheless, I think this is a winner. Let's consider it when we start the 8.1 cycle. OK, is anyone opposed to this idea? I would register a pgfoundry project (name suggestions? translations?), give most established translators commit access, and move the statistics pages there. Also, some translation groups seem to have their own mailing lists or web pages, which could optionally also be hosted there. We could then sync the translations either regularly (e.g., once a week) or only at release time. Of course we would need to mirror all the branches there. Comments? Perfectly fine. Please go ahead. -- Peter Eisentraut Nicolai Tufar Turkish Language Translation Team. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Latest Turkish translation updates
Wow, Turkish seem to be the first translation to report 100% translation completion for 8.0 release. Congratulations for great work! And thanks to Peter for being patient with us all this time. We can't reproduce it with msgfmt -v. How do you get those errors? The scripts that produce these tables do not use the standard gettext tools; they use hand-crafted Perl scripts. In some cases, these catch more errors. In all cases that I have analyzed further, this was because %m was not identified as a format specifier by msgfmt. Could you share these scripts with us? Many update submissions we made were beacause of these %m errors. Also would it be easier to you Peter if we give you login to our CVS poject (on sf.net) so that you just run cvs up every time you package a new update and not bother with emails. Thank you all folks for your efforts. I will open a bottle of chamgne tonight to celebrate 8.0. Peter Eisentraut http://developer.postgresql.org/~petere/ Best regards, Nicolai Tufar ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [PATCHES] Latest Turkish translation updates
Alvaro Herrera wrote: Maybe we should have a pgfoundry project where all translations were kept, and from which the main CVS could be updated semi-automatically. Then we wouldn't have Peter checking out and committing all the time. Peter Eisentraut wrote: That sounds like a fine idea. My only concern would be the not-maintained-here syndrome, which occurs every time some CVS tree contains a file that is actually maintained by an external group, thus blocking the maintainers of the former CVS tree from applying necessary fixes at times. Nevertheless, I think this is a winner. Let's consider it when we start the 8.1 cycle. It is definitely a winner. And I don't think anyonwone from core group will object since Peter will be the only one who updates main CVS tree. Just appoint a maintainer or two with cvs commit priveleges for each language. It will ease your routine workload greatly. Best regards, Nicolai ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
Sorry for such a late submission. I just downloaded the latest postgresql-8.0.0-rc5-3.zip installer for windows and it appears that Windows' printf() does not support placeholder replacement as described in http://developer.postgresql.org/docs/postgres/nls.html#AEN57284 I searched list archives but found no explanation for this. Original string is %s at or near \%s\ and we replaced it in Turkish for \%2$s\ yerinde %1$s. It works perfectly fine on Linux but fails miserably under Windows with something like: HATA: $s yerinde $sat character 1 while on Linux the same version works fine: HATA: sdfassfsdfasd yerinde söz dizim hatasý at character 1 What is it? I suppose that Windows' printf() does not support changing the order of placeholder characters with %2$ %1$. Did some one else deal with it? Is it because of compilation flags? In any case PostgreSQL version in current pginstaller is broken. Best regards, Nicolai ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] %2$, %1$ gettext placeholder replacement is not working under Win32
On Mon, 17 Jan 2005 16:17:33 -0300, Fx [EMAIL PROTECTED] wrote: unix/win32 libc doesnt support $n variables.. What can we do then? ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [pgsql-hackers-win32] %2$, %1$ gettext placeholder replacement is not working under Win32
On Mon, 17 Jan 2005 14:54:44 -0500, Tom Lane [EMAIL PROTECTED] wrote: Also, it looks like src/port/snprintf.c is not %n$ capable either. I'm not sure which platforms that affects. A possible route to a solution is to upgrade snprintf.c and then use it on platforms that don't have this support. This only fixes those cases where we go through snprintf, which is probably not all of the affected messages, but it might be enough. It's not happening for 8.0.0 though. Bad news for the Turks. Turkish language syntax is lake Latin: the verb is always at the end. Because of that almost all messages that have two or more placeholders are broken under Windows. The solution you propose seem to be a trivial exercise: 1. write a replacement function, say, pg_snprintf() that would shuffle input arguments, replace %x$ with % and then call libc's snprintf(); 2. replace snprintf() in all source files with pg_snprintf(); 3. Update Makefiles to use our custom function on broken platforms or default to snprinf(). (Or maybe pg_snprintf() should be a #define); 4. Tell everyone to use pg_snprintf() instead of snprintf() from now on. I would like volunteer to implement this for 8.1 and would be honoured if I get a chance to contribute. regards, tom lane Best regards, Nicolai Tufar ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] ARC patent
Greetings, I would like to contribute my $.02 to this issue. I speak as not a lawyer but as someone tho worked one and a half year in a patent bureau and even got a certificate from WIPO (http://academy.wipo.int/ those who interested may attend the course too, it is free). First, the whole point of USPTO's publishing patents which are pending is to get it publicly reviewed and collect objections before final decision. So, those of you who live in US file and objection based on USENIX File Storage Technologies Conference (FAST), March 31, 2003, San Francisco, CA mentioned by Jan Wieck. Filing and objection should be not be too expensive though you may need help of professional lawyer form a patent bureau co compose a solid objection. I will call my old friends from Patent Bureau tomorrow to get a professional advise on this matter. Second, a pending patent is not a granted patent, one is not infringing anything by distributing technology based in a pending patent. As soon as patent is granted AND Cease and Desist letter form IBM is received removing offending code, removing offending versions from download and and notifying customers to upgrade to a new version is sufficient. I am not sure about CVS, apparently it need to be cleared out too. A vaguely similar issue happened between Pixar, the developer of Renderman and Exluna the developer of BMRT, a free (but not open source) raytracing 3D renderer. Pixar sued Exluna for willful patent infringement. Exluna released a new version of BMRT - 2.6 without offending technology and ensured that version 2.5 is removed from all mirrors. For quite a lot of time -and even now- one of the most valuable things a 3D designer may own is a copy of BMRT version 2.5. Exluna was intended to defend themselves in court but soon ran out of money, settled with Pixar and was swallowed by nVidia. A sad story indeed. A story of how a big company squashes a small one using patents. Read more at: http://www.renderman.org/RMR/OtherLinks/blackSIGGRAPH.html The point here is that IBM may force PostgreSQL Global Development Group to remove offending version if patent is granted. But, lastly, as it was pointed out before it would be a very bad publicity for IBM and, in my opinion, very good publicity for PostgreSQL. IBM will admit that PostgreSQL is a worthy competitor. Thus, in my personal opinion IBM will never threat PostgreSQL. We can remove offending code but host patches to introduce the code in a country that does accept software patents. It would be even better for publicity. IBM can NEVER sue customers for using infringing code before first informing them of infringement and giving reasonable time to upgrade to uninfringing version. So, in short my advise is: 1. File an objection with USPTO. And maybe an informative letter to IBM legal department mentioning USENIX paper. 2. If patent is granted, contact IBM and request an unlimited, perpetual license to use the technology 3. If IBM refuses, remove the offending code, clean up CVS and shout from the rooftops about the hypocrisy of IBM. Hope it helps make up your mind, Best regards, Nicolai Tufar P.S. But if filing date really is 2002 and there is no prior art me may skip step 1. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] ARC patent
On Mon, 17 Jan 2005 14:02:14 -0800, Joshua D. Drake [EMAIL PROTECTED] wrote: IBM can NEVER sue customers for using infringing code before first informing them of infringement and giving reasonable time to upgrade to uninfringing version. I can see it now: We won't sue you (customer) but you have to upgrade to DB2 ;) More like downgrading, actually ;) Sincerely, Joshua D. Drake ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] US Patents vs Non-US software ...
Greetings, Patents do not transcend international border. They need to be applied for in each country separately. To ease the process of applying for patents in many countries at once Patent Cooperation Treaty (PCT) was formed. When you file a patent application with WIPO head office under PCT you specify a list countries of designation from list of countries members of PCT. Filing like this takes significantly less in paperwork and application fees than filing application in each country separately. Many countries do not grant software patents so it is not likely that IBM applied through PCT since a refusal in one country may cause to patent to be refused in all countries. Hope it helps, Nicolai Tufar ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] apparent problem on linux/s390
On Tue, 07 Dec 2004 13:51:36 -0500, Tom Lane [EMAIL PROTECTED] wrote: FWIW, I get clean regression test passes on a real z900 at Red Hat, in both s390 and s390x (32- and 64-bit) modes. I'm not sure what that means --- it could be that Red Hat Linux doesn't use the hardware floating point, or it could be that their version of glibc knows how to deal with corner cases like NaNs on this hardware. Or it could be that Hercules is broken. Too bad :( I had great confidence in Hercules emulator. This is from official Hercules FAQ: ``The following standard feature has been partially implemented: * Binary Floating-Point instructions '' I am playing with -msoft-float gcc option which disables hawdware floating-piont instructions and uses glibc's ones instead but I get the following error: /tmp/ccOWrBZJ.o: In function `main': /tmp/ccOWrBZJ.o(.text+0x40): undefined reference to `__extendsfdf2' collect2: ld returned 1 exit status Doeas somebody know in what library `__extendsfdf2' is? Thanks in advance, Nicolai ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] apparent problem on linux/s390
Hello, On Mon, 06 Dec 2004 19:15:23 -0500, Andrew Dunstan [EMAIL PROTECTED] wrote: problems apparently with NaNs, infinities and negative zeros. I am the owner of this baby. It is not an actual mainframe, it is Hercules emulator running on AthlonXP 2500+ producing 20-25 MIPS of pure mainframe horsepower! On bootup Linux reports 130 BogoMIPS and OS/390 reports 40 MIPS. We all know that mainframe MIPS != mini MIPS. Anyways, a full compile of PostgreSQL source takes about 300 minutes. I run it overnight. Hercules is a tested and proved emulator but I have a vague doubt that it's IEEE FPU emulation may be slightly different from the real iron's one. Are there test suites to check if a linux's math functions confirm to the standards. I may need to run one of them first. Also if the is something to do with the code I am more than willing to help. Seems like nobody else has a mainframe handy for testing. I will do what I can but bear in mind the compile time. In case anyone wants to try it at home I will help to set it up, it is really easy once it is up and running it is the same old Linux to you. As for urgency of bugfixes for S390 platform I am not aware of anyone running PostgreSQL on IFL (integrated facility for Linux). And I am not aware of anyone who would even consider this. Mainframe users are all so scared by IBM FUD. Nevertheless running PostgreSQL in iron is an important sales pitch for me and Devrim. Imagine: Of course it runs on mainframe or Save $2,573,000.00 in monthly software licence fees, use PostgreSQL!. see: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=fantaildt=2004-12-06%2011:05:24 Meanwhile another fresh build completed with absolutely the same result: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=fantaildt=2004-12-06%2023:04:43 Best regards, Nicolai Tufar ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] apparent problem on linux/s390
On Tue, 7 Dec 2004 06:38:25 -0600 (CST), Andrew Dunstan [EMAIL PROTECTED] wrote: Maybe we should mark the buildfarm member as s390 emulator ? It could be. But we'd rather use the full specification, Hercules 2.37, ESAME S390 architecture, 31 bit mode, running Debian Linux Woody or something like that. Don't quote it now though, I do not remember what was the version Hercules I used. But in my opinion what runs on Hercules runs on Mainframe too, we can safely claim that we compiled and tested on a mainframe. Meanwhile, if anyone knows of real big iron that could join the buildfarm that would also be good. It will be great. There should be some universities that still use mainframe and some of them even installed Linux. Compiling PostgreSQL is extremely resource hungry process. I doubt a business will lend its mainframe for such a purpose. Also the used to be P390 - personal mainframes machines produced by IBM a decade ago. They still can be bought on eBay for something like $5,000. They can run Linux too. Regards, Nicolai Tufar ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] 7.4.1 release status - Turkish Locale
-Original Message- From: Tom Lane Did you try running the regression tests under tr_TR locale? It seems a few bricks short of fine yet :-( I run regression tests under tr_TR locale. To do this I hardcoded Turkish locale in initdb in pg_regress.sh. Three tests failed, I attached resulting diff. With days of the week, the same problem is with downcasting occurs. I think it is not that crucial, but the rest of the differences in the file seem to be important. I was not able to interpret them. Thanks, Nicolai regards, tom lane regression.diffs Description: Binary data ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 7.4.1 release status - Turkish Locale
Sorry for rising up old issue again but the problem still persists. And database cluster is not being created with Turkish locale If you have any doubts about how Turkish users will react to the fact that both I and I WITH DOT will be treated as same character, rest assured that this behavior is de-facto standard when it comes to file names, identifiers and commands. Greg Stark and Devrim Gunduz will confirm that, no doubt. Please review and apply this patch I send you for the third time. You will not regret and many users will be grateful. Please note that to my knowledge it will not break any other locales. Best regards, Nicolai tr20040203.diff Description: Binary data ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PITR Dead horse?
-Original Message- From: Dave Page [mailto:[EMAIL PROTECTED] Sent: Thursday, February 05, 2004 11:02 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: [HACKERS] PITR Dead horse? Of course, but I would argue that my claim that PostgreSQL is reliable is backed up by the lack of people posting messages like 'we had a powercut and now my DB is hosed'. It's not like that. It's more like 'what will happen if we had a powercut/ disk failure/cpu failure/memory failure, etc, etc.' and that answer I have to give is 'why, there is PITR of course!'. No other answer will pass in enterprise world. Those people are not open-minded, they'd rather be safe than sorry. Do they have specific problems with the reliability of PostgreSQL then? Perhaps you could post details of how things have gone wrong for them (assuming you haven't already - I don't recall anything on -hackers recently). Nothing remarkable. PostgreSQL just works. Bu as I said before, In enterprise world, good sleep at night is treasured above all. Regards, Dave ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] PITR Dead horse?
-Original Message- From: Dave Page [mailto:[EMAIL PROTECTED] My SQL2K servers give me far more sleepless nights than PostgreSQL ever did! You bet! I totally agree with you. Technicians like you, me and most people on this list Already know that PostgreSQL is stable and reliable. It is management that needs to be convinced, and for this we need to have PITR in feature list. Regards, Dave. Regards, Nicolai ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] PITR Dead horse?
I would like to join this effort too. I was afraid that people at RedHat are already halfway though and were to release their work shortly. But it does not seem to be the case. Regards, Nicolai -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Koichi Suzuki Sent: Wednesday, February 04, 2004 11:25 AM To: Tatsuo Ishii Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] PITR Dead horse? Hi, This is Suzuki from NTT DATA Intellilink. I told Bruce Momjan that I and my colleagues are interested in implementing PITR in BOF in NY LW2004. NTT's laboratory is very interested in this issue and I'm planning to work with them. I hope we could cooperate. Tatsuo Ishii wrote: Has this been beaten to death now? Just curious if PITR was in Dev tree yet. Been out of the loop. TIA. I and my co workers are very interested in implementing PITR. We will tackle this for 7.5 if no one objects. -- Tatsuo Ishii ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] PITR Dead horse?
Totally agree. Robustness and rock-solidness are the only things missing for PostgreSQL to become the killer of certain commercial enterprise databases out there. And the only thing that is missing in this respect is PITR. PITR should be there INGRES had it in '84 and some people as why PostgreSQL does not have it. I am well versed in the internals of PITR features of a certain leading enterprise-class database out there. And would like to contribute (write code) to this effort as much as I can. Best regards, Nicolai Tufar -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Simon Riggs Sent: Thursday, February 05, 2004 1:33 AM To: 'Tom Lane'; 'Marc G. Fournier' Cc: 'Tatsuo Ishii'; [EMAIL PROTECTED]; [EMAIL PROTECTED]; pgsql- [EMAIL PROTECTED] Subject: Re: [HACKERS] PITR Dead horse? Tom Lane wrote Marc G. Fournier [EMAIL PROTECTED] writes: Is this something large enough, like the win32 stuff, that having a side list for discussions is worth setting up? In terms of the amount of code to be written, I expect it's larger than the win32 porting effort. And it should be mostly pretty separate from hacking the core backend, since most of what remains to do is writing external management utilities (I think). Yes it is! I'd like to start the discussion about PITR and try to go through some functional requirements and how those might be implemented. The Win32 port has a self-evident set of functional requirements; I'm not sure that the PITR stuff is as clear - so I couldn't pass any judgement at all (even if I did know the code well enough) on how big a coding task that is, but I can see that the analysis and discussion is large indeed. I've been dissatisfied with having the separate pgsql-hackers-win32 list; I feel it just fragments the discussion, and people tend to end up crossposting to -hackers anyway. But a separate list for PITR work might be a good idea despite that experience, since it seems like it'd be a more separable project. I'd vote for a new list dedicated to discussing Robustness issues, such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the Functionality and Performance, it just needs some rock-solid analysis of where-things-can-go-wrong with it, so that the business data centre people will be able to use it with absolute confidence...even if the answer is we've got every base covered. For me, the issues about robustness are as much to do with risk reduction and confidence building as they are about specific features in that area. [Wow, I expect some flames on those comments!] The list probably would remain clearly differentiated, in the same way [Performance] covers lots of areas not discussed in [Hackers]. Not hung up on the name either, just something that indicates breadth-of-scope, e.g. Availability or Data Protection or Resilience etc..; maybe the Advocates would like to name it? It might even be a press-release: PostgreSQL community focuses new efforts towards Robustness features for its next major release. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Turkish Locale in Identifiers (contd.)
Tom Lane [EMAIL PROTECTED] wrote: Nicolai Tufar [EMAIL PROTECTED] writes: A possible compromise is to apply ASCII downcasing (same as in keywords.c) for 7-bit-ASCII characters, and apply tolower() only for character codes above 127. In other words If we go this way why not make a special case only and only for 'I' Character and not all 7-bit ASCII: It seems to me that that's too narrow a definition of the problem. I think we should state our goal as we don't want bizarre locale definitions to interfere with downcasing of the basic ASCII letters. If we put in a special case for 'I' we will fix the known problem with Turkish, but what other strange locales might be out there? And if we don't trust tolower() for 'I', why should we trust it for 'A'-'Z'? Since nobody commented on the issue I may suggest a patch that implements 'I' special case solution. 'A'-'Z' ASCII-only downcasting idea was rejected before on basis of SQL99 compliance. I hope I would have more luck with this one. Because PostgreSQL just does not work with Turkish locale, and it was so since 7.4.0. initdb just chokes on VOID identifier and quits. Devrim Gunduz will second me on this, I am sure. With my knowledge of Russian, Arabic and -to some degree- Hebrew encodings I claim that this patch will not break them. If someone who uses far eastern Encodings would also check it, I think it would be pretty safe to apply this patch to the source. Thanks, Nicolai Tufar ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Turkish Locale in Identifiers (contd.)
Oops, forgot the patch :) -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Nicolai Tufar Sent: Tuesday, February 03, 2004 9:31 PM To: [EMAIL PROTECTED] Cc: 'Tom Lane'; [EMAIL PROTECTED] Subject: [HACKERS] Turkish Locale in Identifiers (contd.) Tom Lane [EMAIL PROTECTED] wrote: Nicolai Tufar [EMAIL PROTECTED] writes: A possible compromise is to apply ASCII downcasing (same as in keywords.c) for 7-bit-ASCII characters, and apply tolower() only for character codes above 127. In other words If we go this way why not make a special case only and only for 'I' Character and not all 7-bit ASCII: It seems to me that that's too narrow a definition of the problem. I think we should state our goal as we don't want bizarre locale definitions to interfere with downcasing of the basic ASCII letters. If we put in a special case for 'I' we will fix the known problem with Turkish, but what other strange locales might be out there? And if we don't trust tolower() for 'I', why should we trust it for 'A'-'Z'? Since nobody commented on the issue I may suggest a patch that implements 'I' special case solution. 'A'-'Z' ASCII-only downcasting idea was rejected before on basis of SQL99 compliance. I hope I would have more luck with this one. Because PostgreSQL just does not work with Turkish locale, and it was so since 7.4.0. initdb just chokes on VOID identifier and quits. Devrim Gunduz will second me on this, I am sure. With my knowledge of Russian, Arabic and -to some degree- Hebrew encodings I claim that this patch will not break them. If someone who uses far eastern Encodings would also check it, I think it would be pretty safe to apply this patch to the source. Thanks, Nicolai Tufar ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) tr20040203.diff Description: Binary data ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Q: How ORDER BY is being done inetrnally?
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Text sorting depends on strcoll() and nothing but. See varstr_cmp(). I see, apparently sort done for ORDER BY clause is case-sensitive. But problem is still there. It is about I-with-dot and I-without-dot in Turkish again. While all UNIX programs put I-without-dot before I-with-dot, as it should be, PostgreSQL puts it in reverse order. I examine the code for any possible gotchas, but I am confused about what function is being called by what. Especially that all those sort methods and functions are not hard-coded but stored in pg_am* catalogue tables. Could someone please explain -very briefly- what exactly is happening when a sort is performed. A kind of stack trace: which function calls which one would be very appreciated. Best regards, Nicolai Tufar ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Q: How ORDER BY is being done inetrnally? - solved, in a way
Okay, I figured out what was the problem. glibc's LC_COLLATE file under /usr/lib/locale/tr_TR is wrong! And it has been wrong for many years now. And nobody noticed it. PostgreSQL is innocent here. I checked it many times over and over again with test programs and different environment settings and it is wrong! I will contact glibc team now. Thanks a lot for help. Regards, Nicolai Tufar ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] 7.4.1 release status - Turkish Locale
Tom Lane [EMAIL PROTECTED] wrote: Nicolai Tufar [EMAIL PROTECTED] writes: A possible compromise is to apply ASCII downcasing (same as in keywords.c) for 7-bit-ASCII characters, and apply tolower() only for character codes above 127. In other words If we go this way why not make a special case only and only for 'I' Character and not all 7-bit ASCII: It seems to me that that's too narrow a definition of the problem. I think we should state our goal as we don't want bizarre locale definitions to interfere with downcasing of the basic ASCII letters. If we put in a special case for 'I' we will fix the known problem with Turkish, but what other strange locales might be out there? And if we don't trust tolower() for 'I', why should we trust it for 'A'-'Z'? To my knowledge no other locale have similar problems. At least nobody complained so far while Turk users are rising their voices for many years now. Let try and put this very special case, together with an extensive explanation in comment and see if someone complains. And by the way, national characters in table, column, index or function names is something that never happens in production databases. As for 'A'-'Z'^, it was pointed to me that SQL99 standard states that identifier names need to be downcasted in locale-dependent manner. Would you like me to create a patch that would touch only src/backend/parser/scan.l, introduce a special case for 'I' and include an explanation in comment? What it comes down to is that by training and experience, I always expect that any bug might be just one example of a whole class of bugs. You have to look for the related cases that might happen in future, not only fix the case that's under your nose. regards, tom lane Regards, Nicolai Tufar ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Q: How ORDER BY is being done inetrnally?
Greetings, We are trying to fix another problem we have with Turkish locale. Order in which tuples are returned by SELECT .. ORDER BY . statement differs from the one returned by UNIX sort utility or what strcoll() function will suggest. It is specific to a locale so you will not likely be able to reproduce it. I understand that with ORDER BY clause, values are compared in case-insensitive manner as well as it should be the case with the values stored in indices. And I suspect that problem is hidden somewhere there. I tried to find where this conversion and comparison is being done in source code but could not, despite all my efforts. I would very much appreciate if someone would point out what functions are being called while sorting data for return for ORDER BY clause. Thanks in advance, Nicolai Tufar ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 7.4.1 release status - Turkish Locale
We might think that the Turkish-locale problem Devrim Gunduz pointed out is a must-fix, too. But I'm not convinced yet what to do about it.
Here is a first try
to fix what
Devrim Gunduz talked about.
Please be patient with me for
it is the first major patch
I submit and I realize that I blatantly violated many rules
of good style in PostgreSQL source code.
First, about the problem. Turkish language has two letters i.
One is with dot on top and the other is without.
Simply as that.
The one with
dot has the dot both as capital
and lower-case and
the one without
dot has no dot in both upper and
lower case...
as opposed to English where i has a
dot when lower-case and
has no dot when upper-case.
Problem arise when PostgreSQL, while running with tr_TR locale
converts to lower-case an identifier
as a table, an index or
a column name. If it is written with capital
I, tolower() with
'I' as argument will return Turkish specific character:
'i'-without-a-dot what I am afraid
will not be shown correctly
in your e-mail readers.
Let me give
some examples.
initdb script runs apparently innocent script in file
src/backend/utils/mb/conversion_procs/conversion_create.sql
to create a couple of functions whose only fault
was
to declare it their return parameters
as VOID. Backend
returns error message that type
vo d is not found
and
initdb fails.
A nothing suspecting novice user was
excited about
SERIAL data type he was tail is present
in PostgreSQL.
It took us with Devrim a lot of time to explain why
he
need to type
SERIAL as SERiAL for now till a workaround
is developed.
Another case happened with me
when I wanted to restore
a pg_dump dump. Restore failed because script was creating
scripts that belong to PUBLIC.
For the solution,
after some research we found
out that
offender is tolower() call in src/backend/parser/scan.l
in {identifier} section. tolower() works fine with any
locale and with any character
save for the Turkish locale
and capital 'I' character. So, the obvious solution
is
to put a check for Turkish locale
and 'I' character.
Something like this:
if( locale is Turkish ident[i]
== 'I' )
ident[i] =
'i';
else
ident[i] =
tolower((unsigned char) ident[i]);
Looks rather simple but the hard part was to
figure out
what is the current locale. To do this I added
const char *get_locale_category(const char *category);
to src/backend/utils/adt/pg_locale.c that would
return
locale identifier for the category
specified or LC_ALL
if category is NULL. I could not find any other function
that will return what I need.
Please help me to find
one because I would hate to
introduce a new function.
I realize that {identifier} section is very performance
critical so I introduced a global variable
static int isturkishlocale = -1;
at the beginning of src/backend/parser/scan.l
It is set to -1 when not yet initialized, 0 if
locale is not Turkish and 1 if locale
is Turkish.
It might not be the way it is usually
done in PostgreSQL
source code. Could you pleas
advise if the name I chose
is appropriate and whether there is a more appropriate
place to put declaration and initialization.
Best regards,
Nicolai Tufar Devrim Gunduz
trpatch.diff
Description: Binary data
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Patch to make Turks happy.
Bruce Momjian wrote:
I am not going to apply this patch because I think it will mess up the
handling of other locales.
As far as I figured from the source code this function only deals with
cleaning up
locale names and nothing else. Since all the locale names are in plain
ASCII I think
it will be safe to use ASCII-only lower-case conversion.
By the way, I noticed only after sending the patch that compiler
complains about
ambiguous `else' so it can be rewritten as:
if (*p = 'A' *p = 'Z'){
*np++ = *p + 'a' - 'A';
}else{
*np++ = *p;
}
Regards,
Nicolai
---
Nicolai Tufar wrote:
Hi,
Yet another problem with Turkish encoding. clean_encoding_name()
in src/backend/utils/mb/encnames.c uses tolower() to convert locale
names to lower-case. This causes errors if locale name contains
capital I and current olcale is Turkish. Some examples:
aaa=# \l
List of databases
Name| Owner | Encoding
---+---+--
aaa | pgsql | LATIN5
bbb | pgsql | LATIN5
template0 | pgsql | LATIN5
template1 | pgsql | LATIN5
(4 rows)
aaa=# CREATE DATABASE ccc ENCODING='LATIN5';
ERROR: LATIN5 is not a valid encoding name
aaa=# \encoding
SQL_ASCII
aaa=# \encoding SQL_ASCII
SQL_ASCII: invalid encoding name or conversion procedure not found
aaa=# \encoding LATIN5
LATIN5: invalid encoding name or conversion procedure not found
Patch, is a simple change to use ASCII-only lower-case conversion
instead of locale-dependent tolower()
Best regards,
Nic.
*** ./src/backend/utils/mb/encnames.c.orig Mon Dec 2 15:58:49 2002
--- ./src/backend/utils/mb/encnames.c Mon Dec 2 18:13:23 2002
***
*** 407,413
for (p = key, np = newkey; *p != '\0'; p++)
{
if (isalnum((unsigned char) *p))
! *np++ = tolower((unsigned char) *p);
}
*np = '\0';
return newkey;
--- 407,416
for (p = key, np = newkey; *p != '\0'; p++)
{
if (isalnum((unsigned char) *p))
! if (*p = 'A' *p = 'Z')
! *np++ = *p + 'a' - 'A';
! else
! *np++ = *p;
}
*np = '\0';
return newkey;
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] 7.4 Wishlist
my wish: * error codes. It's very interesting that nobody other wants it... I do :-) Me too. It is a must in my opinion.. Regards, Nic. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] Patch to make Turks happy.
Hi,
Yet another problem with Turkish encoding. clean_encoding_name()
in src/backend/utils/mb/encnames.c uses tolower() to convert locale
names to lower-case. This causes errors if locale name contains
capital I and current olcale is Turkish. Some examples:
aaa=# \l
List of databases
Name| Owner | Encoding
---+---+--
aaa | pgsql | LATIN5
bbb | pgsql | LATIN5
template0 | pgsql | LATIN5
template1 | pgsql | LATIN5
(4 rows)
aaa=# CREATE DATABASE ccc ENCODING='LATIN5';
ERROR: LATIN5 is not a valid encoding name
aaa=# \encoding
SQL_ASCII
aaa=# \encoding SQL_ASCII
SQL_ASCII: invalid encoding name or conversion procedure not found
aaa=# \encoding LATIN5
LATIN5: invalid encoding name or conversion procedure not found
Patch, is a simple change to use ASCII-only lower-case conversion
instead of locale-dependent tolower()
Best regards,
Nic.
*** ./src/backend/utils/mb/encnames.c.orig Mon Dec 2 15:58:49 2002
--- ./src/backend/utils/mb/encnames.c Mon Dec 2 18:13:23 2002
***
*** 407,413
for (p = key, np = newkey; *p != '\0'; p++)
{
if (isalnum((unsigned char) *p))
! *np++ = tolower((unsigned char) *p);
}
*np = '\0';
return newkey;
--- 407,416
for (p = key, np = newkey; *p != '\0'; p++)
{
if (isalnum((unsigned char) *p))
! if (*p = 'A' *p = 'Z')
! *np++ = *p + 'a' - 'A';
! else
! *np++ = *p;
}
*np = '\0';
return newkey;
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Locale-dependent case conversion in {identifier}
- Original Message -
From: Hannu Krosing [EMAIL PROTECTED]
To: Nicolai Tufar [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Saturday, November 30, 2002 5:41 PM
Subject: Re: [HACKERS] Locale-dependent case conversion in {identifier}
[ ... ]
could it just be that we store identifiers in lower case, whereas most
others
(including SQL spec IIRC)have them in upper case ?
That seem to be the case. All the databases I used, automaticaly convert
identifiers to upper case.
And they all do it using ASCII-only conversion.
Could you try the grant in both databases also in lower case ?
i.e.:
grant select on a to public;
The statement works in both databases. But the problem is that it was
pg_dumpall who created SQL statements with PUBLIC. Why
pg_dumpall does not enclose identifiers in quotes, like:
REVOKE ALL ON TABLE tamara2 FROM public;
insted of
REVOKE ALL ON TABLE tamara2 FROM PUBLIC;
as it does now.
I will make an attempt to modify pg_dump accordingly, and will send a
patch to the list.
Regards,
Nic
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Hard-coded PUBLIC in pg_dump
src/bin/pg_dump/pg_dump.c happen to have hard-coded PUBLIC role name.
It completly breaks dumps when run with Turksh locale setting. In my
opinion making it lower-case would do much good and no harm. A mini
patch is given below.
On the other hand, I was thinking about wrapping all the identifiers in
dump files in single quotes. It is done in SET SESSION AUTHORIZATION
clause. Is there a reason for not doing this with table and colum names?
Regards,
Nic
*** ./src/bin/pg_dump/pg_dump.c.origSun Dec 1 03:23:56 2002
--- ./src/bin/pg_dump/pg_dump.c Sun Dec 1 03:24:48 2002
***
*** 4918,4924
* wire-in knowledge about the default public privileges for different
* kinds of objects.
*/
! appendPQExpBuffer(sql, REVOKE ALL ON %s %s FROM PUBLIC;\n,
type, name);
/* Make a working copy of acls so we can use strtok */
--- 4918,4924
* wire-in knowledge about the default public privileges for different
* kinds of objects.
*/
! appendPQExpBuffer(sql, REVOKE ALL ON %s %s FROM public;\n,
type, name);
/* Make a working copy of acls so we can use strtok */
***
*** 4980,4986
if (eqpos == tok)
{
/* Empty left-hand side means PUBLIC */
! appendPQExpBuffer(sql, PUBLIC;\n);
}
else if (strncmp(tok, group , strlen(group )) == 0)
appendPQExpBuffer(sql, GROUP %s;\n,
--- 4980,4986
if (eqpos == tok)
{
/* Empty left-hand side means PUBLIC */
! appendPQExpBuffer(sql, public;\n);
}
else if (strncmp(tok, group , strlen(group )) == 0)
appendPQExpBuffer(sql, GROUP %s;\n,
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Hard-coded PUBLIC in pg_dump
- Original Message -
From: Christopher Kings-Lynne [EMAIL PROTECTED]
To: Nicolai Tufar [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Sunday, December 01, 2002 4:05 AM
Subject: Re: [HACKERS] Hard-coded PUBLIC in pg_dump
H...does putting double quotes (eg. PUBLIC) around the public word
fix
it?
No:
apb= GRANT SELECT ON TABLE maras2.esya TO PUBLIC;
ERROR: user PUBLIC does not exist
apb= GRANT SELECT ON TABLE maras2.esya TO 'PUBLIC';
ERROR: parser: parse error at or near 'PUBLIC' at character 38
apb= GRANT SELECT ON TABLE maras2.esya TO public;
GRANT
apb=
The problem here is case conversion from capital I to lower-case i.
In Turkish locale tolower('I') is not equal to 'i'. So, since public role
is lower-case internally, why would we not make it lower-case in dump file.
You can't put single quotes around table and column names. You need to
use
double quotes as they are identifiers rather than literals.
Bear in mind that some improvements have been made in Postgres 7.3 with
regards to quoting, so have you checked 7.3?
I stand corrected. It is indeed has to be double-quoted.
7.3 is quoting only SET SESSION AUTHORIZATION 'role' clause in my dump.
Possibly,
because it has been added recently. Old code does not quote anything.
Chris
Regards,
Nic.
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] Hard-coded PUBLIC in pg_dump
- Original Message - From: Tom Lane [EMAIL PROTECTED] ... but considering that SQL92 clearly lists it as a reserved word, there's not a lot of ground for that complaint to stand on. I'd prefer shifting PUBLIC back to the true-keyword category over any of the other workarounds you've suggested ... It will work for me. But why not change PUBLIC in pg_dump output to lower-case as well? regards, tom lane Nic. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [GENERAL] Segmentation fault while COPY in 7.3
- Original Message - From: Tom Lane [EMAIL PROTECTED] Ohhh ... Nicolai, are you running with a client encoding different from server encoding? Got it! Gentlemen, thank you very much for assistance. The body of evidence was slowly growing, then, finaly Tom Lan's message have enlightened me. It all started with initdb's warning that it can not set client's connection to 'LATIN5'. Okay, I said, maybe some system files are not installed. Then I tried to create a database with ENCODING='LATIN5'. I did not pay much attention either since my template1 and template0 are in LATIN5. Then on Tom's suggestion I tried to change client encoding: aaa=# \encoding LATIN5 aaa=# \encoding SQL_ASCII SQL_ASCII: invalid encoding name or conversion procedure not found aaa=# \encoding LATIN5 LATIN5: invalid encoding name or conversion procedure not found aaa=# \encoding SQL_ASCii aaa=# \encoding SQL_ASCII aaa=# \encoding LATiN5 aaa=# \encoding LATIN5 aaa=# So it all falls back to Turkish I problem I mentioned earlier. To perform COPY successfully I had to set client locale to 'LATiN5' (note quotes and lower-case i). Any other combinations result in segmentation fault. Now, would it be right to change locale name handling to use ASCII-only lower and upper-case conversions? Many thanks to Tom Lane and Joe Conway. I've got to get some sleep :-) Regards, Nic. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Locale-dependent case conversion in {identifier}
Comment in {identifier} section in src/backend/parser/scan.l states:
[...]
* Note: here we use a locale-dependent case conversion,
* which seems appropriate under SQL99 rules, whereas
* the keyword comparison was NOT locale-dependent.
*/
And in ScanKeywordLookup() in src/backend/parser/keywords.c:
/*
* Apply an ASCII-only downcasing. We must not use tolower()
since it
* may produce the wrong translation in some locales (eg, Turkish),
* and we don't trust isupper() very much either. In an ASCII-based
* encoding the tests against A and Z are sufficient, but we also
* check isupper() so that we will work correctly under EBCDIC. The
* actual case conversion step should work for either ASCII or
EBCDIC.
*/
And I happen to have bad luck to use PostgreSQL with Turkish locale. And, as
you
may know our I is not your I:
pgsql=# create table a(x char(1));
CREATE TABLE
pgsql=# grant SELECT ON a to PUBLIC;
ERROR: user public does not exist
pgsql=#
Oracle, the second best database I have does seem to convert relation names
in
locale-dependent fassion:
SQL alter session set NLS_LANGUAGE='TURKISH';
Session altered.
SQL create table a(x char(1));
Table created.
SQL grant select on a to PUBLIC;
Grant succeeded.
Further, if I try to create a table in oracle using Turkish-specific
characters,
it is creating it alright, without trying to make them upper-case as it
usually does.
So I have changed lower-case conversion code in scan.l to make it purely
ASCII-based
as in keywords.c. Mini-patch is given below. Please bear in mind that it is
my first
attempt at hacking PostgreSQL code, so there can be some mistakes.
Regards,
Nick
diff -Nur src/backend/parser/scan.l.orig src/backend/parser/scan.l
--- src/backend/parser/scan.l.orig Sat Nov 30 02:54:06 2002
+++ src/backend/parser/scan.l Sat Nov 30 02:57:45 2002
@@ -551,9 +551,12 @@
ident = pstrdup(yytext);
for (i = 0; ident[i]; i++)
{
- if (isupper((unsigned char)
ident[i]))
- ident[i] =
tolower((unsigned char) ident[i]);
+ charch =
ident[i];
+ if (ch = 'A' ch = 'Z'
isupper((unsigned char) ch))
+ ch += 'a' - 'A';
+ ident[i] = ch;
}
+ ident[i] = '\0';
if (i = NAMEDATALEN)
{
int len;
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Locale-dependent case conversion in {identifier}
By no means I would try to convince that your reading of the SQL standards is wrong. What I am trying to tell is that Turkish alphabet is broken beyond repair. And since there is absolutely no way to change our alphabet, we may can code a workaround in the code. So i do not claim that your code is wrong. It is behaviang according to specification. But unfortunately folks at SQL99 probably were not aware of the woes of Turkish I. The very special case of letter I in Turkish is not only PostgreSQL's problem. Many java programs have failed miserably trying to open files with Is in pathnames. So basically, there are two letters I in Trukish. The wone is with dot on top and another is without. The with dot on top walways has the dot and the one without never has it. Simple. The problem is with the standard Latin I. So why small i does have a dot and capital I does not? Standard conversion is Lower: I - y' and Y' - i. Upper: y' - I and i - Y'. (font may not be displayed correctly in your mail reader) Historically programs that operate in Turkish locale have chosen to hardcode the capitalisation of i in system messages and identifier names like this: Lower: I - i and Y' - i. Upper: y' - I and i - I. With this, no matter what kind of I you used in names, it is always going to end up a valid ASCII character. Would it be acceptable if I submit a path that applies this special logic in src/backend/parser/scan.l if the locale is tr_TR? Because for many folks setting locale to Turkish would render their database unusable. For, god forbid, if your sql has a column name written in capitlas including I. It is not working. So I deeply believe that PostgreSQL community have to provide a workaround for this problem. So what should I do? Best regards, Nick Tom Lane wrote: Nicolai Tufar [EMAIL PROTECTED] writes: So I have changed lower-case conversion code in scan.l to make it purely ASCII-based. as in keywords.c. Mini-patch is given below. Rather than offering a patch, you need to convince us why our reading of the SQL standard is wrong. (Oracle does it that way is not an argument that will carry a lot of weight.) SQL99 states that identifier case conversions are done on the basis of the Unicode upper/lower case equivalences, so it seems clear that they intend more than ASCII-only conversion for identifiers. Locale-based conversion might not be an exact implementation of the spec, but it's surely closer than ASCII-only. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] [HACKERS] Realtime VACUUM, was: performance of insert/delete/update
I always wandered if VACUUM is the right name for the porcess. Now, when PostgreSQL is actively challenging in Enterprise space, it might be a good idea to give it a more enterprise-like name. Try to think how it is looking for an outside person to see us, database professionals hold lenghty discussions about the ways we vacuum a database. Why should you need to vacuum a database? Is it dirty? In my personal opinion, something like space reclaiming daemon, free-list organizer, tuple recyle job or segment coalesce process would sound more business-like . Regards, Nick - Original Message - From: Bruce Momjian [EMAIL PROTECTED] To: Curtis Faith [EMAIL PROTECTED] Cc: Tom Lane [EMAIL PROTECTED]; Ron Johnson [EMAIL PROTECTED]; PgSQL Performance ML [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, November 26, 2002 9:09 PM Subject: Re: [PERFORM] [HACKERS] Realtime VACUUM, was: performance of insert/delete/update Good ideas. I think the master solution is to hook the statistics daemon information into an automatic vacuum that could _know_ which tables need attention. -- - Curtis Faith wrote: tom lane wrote: Sure, it's just shuffling the housekeeping work from one place to another. The thing that I like about Postgres' approach is that we put the housekeeping in a background task (VACUUM) rather than in the critical path of foreground transaction commit. Thinking with my marketing hat on, MVCC would be a much bigger win if VACUUM was not required (or was done automagically). The need for periodic VACUUM just gives ammunition to the PostgreSQL opponents who can claim we are deferring work but that it amounts to the same thing. A fully automatic background VACUUM will significantly reduce but will not eliminate this perceived weakness. However, it always seemed to me there should be some way to reuse the space more dynamically and quickly than a background VACUUM thereby reducing the percentage of tuples that are expired in heavy update cases. If only a very tiny number of tuples on the disk are expired this will reduce the aggregate performance/space penalty of MVCC into insignificance for the majority of uses. Couldn't we reuse tuple and index space as soon as there are no transactions that depend on the old tuple or index values. I have imagined that this was always part of the long-term master plan. Couldn't we keep a list of dead tuples in shared memory and look in the list first when deciding where to place new values for inserts or updates so we don't have to rely on VACUUM (even a background one)? If there are expired tuple slots in the list these would be used before allocating a new slot from the tuple heap. The only issue is determining the lowest transaction ID for in-process transactions which seems relatively easy to do (if it's not already done somewhere). In the normal shutdown and startup case, a tuple VACUUM could be performed automatically. This would normally be very fast since there would not be many tuples in the list. Index slots would be handled differently since these cannot be substituted one for another. However, these could be recovered as part of every index page update. Pages would be scanned before being written and any expired slots that had transaction ID's lower than the lowest active slot would be removed. This could be done for non-leaf pages as well and would result in only reorganizing a page that is already going to be written thereby not adding much to the overall work. I don't think that internal pages that contain pointers to values in nodes further down the tree that are no longer in the leaf nodes because of this partial expired entry elimination will cause a problem since searches and scans will still work fine. Does VACUUM do something that could not be handled in this realtime manner? - Curtis ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
