Re: [HACKERS] src/tools/msvc usage instructions
That's on my TODO list to write, but I haven't had the time yet. The basic is that if you disable everything in the config.pl file, you can run with almost no external dependencies. You'll need flex+bison if buliding off CVS. Oh, and it requires Visual C++ 2005. Should work fine with the Express version. I grabbed flex and bison from GNUwin32 (http://gnuwin32.sourceforge.net/packages/bison.htm) This appears to not work out well. If I copy the generated files from bison from a linux box, then they are ok, but if I try to use ones generated using that version of bison, it does not compile. I'll look around for a different one. That's the onw I'm using. However, be sure to get version 1.875-4, and *not* version 2.1. All right, I started off by commenting out almost all of the options in config.pl, and then ran build.bat from my visual studio 2005 command prompt (the express version). It said that Pthreads is required. Which pthreads implementation is this, and where can I get it? Also, which zlib build for windows is the correct one? There are apparently many different ways to build it. I grabbed the zlib dll from www.zlib.net, and pthreads from ftp://sources.redhat.com/pub/pthreads-win32/ and these seem to compile. Those are the ones. Should work fine to use the binaries as well, no need to compile yourself. I ran into an issue where if I tried to compile using the batch file it would fail with a bunch of errors about not finding windows.h, even when I loaded the environment file from the platform sdk. I had more success opening the solution with the gui, but it is not entirely clear which projects I may need to build in which order. I got the postgres project to compile, so I guess that's a start. If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. In my environment, that gets set when I start the Visual Studio command prompt - that's the whole point abou tusing the VS commandprompt and not a normal one. I think you get a question about integrating the Platform SDK with Visual studio when you install it - any chance you missed that one? I already have ActivePerl and ActivePython installed, so those should work out. I am not really concerned about krb5 and ldap, so as long as commenting them out will disable them, that is good. You can safely leave LDAP in, because it uses only the builtin functionality in the OS and no external dependencies. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Win32 hard crash problem
IIRC there is no real SIGINT on Windows, so it can only come from a postgres program. The windows shutdown could be calling pg_ctl to stop the service, of course. Well, not quite that, but it will send a service command to the running pg_ctl (which is our service supervisor), which *will* respond with a SIGINT to the postmaster. //Magnus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
I grabbed flex and bison from GNUwin32
(http://gnuwin32.sourceforge.net/packages/bison.htm)
This appears to not work out well. If I copy the generated files from
bison from a linux box, then they are ok, but if I try to use ones
generated using that version of bison, it does not compile.
Hm, have you tried diff'ing the output files from the two cases? This
is really probably a question for the flex and bison maintainers, not
us, but it seems like it should work for moderately up-to-date versions
of those tools. What compile failures do you get exactly?
I was just going to chalk it up to a bad matching of tool ports or
something and try to find a different bison, but if you are really
interested...
I get errors on any bison generated file. For simplicity of the diff, I
did not use the first failure I got, which was gram.c, but instead used
the much smaller bootparse.c file. I grabbed the bootparse.c files
generated on windows and on linux, did a diff -cw between them, and tarred
up the three files, which you can get from
http://www.jdrake.com/postgresql/bison-files-win32.tar.gz
The errors I got on this file were:
1-- Build started: Project: postgres, Configuration: Release Win32 --
1Compiling...
1bootparse.c
1bootparse.tab.c(1065) : error C2449: found '{' at file scope (missing
function header?)
1bootparse.tab.c(1858) : error C2059: syntax error : '}'
and then a whole lot of random, uninteresting errors of the kind you get
when the compiler is so confused it no longer knows what it is doing.
I am currently trying to build a newer version of bison using mingw and
use it, but I am running into issues with that also.
Oh, I just got the email from Magnus which says do not use v2.1, but
1.875, so I guess that's what I did wrong. Oops!
regards, tom lane
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
--
Drive defensively. Buy a tank.
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Magnus Hagander wrote: This appears to not work out well. If I copy the generated files from bison from a linux box, then they are ok, but if I try to use ones generated using that version of bison, it does not compile. I'll look around for a different one. That's the onw I'm using. However, be sure to get version 1.875-4, and *not* version 2.1. Oops, that was it. If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. If I do build solution it tells me Project not selected to build for this solution configuration for all projects, then 55 skipped at the end. Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. In my environment, that gets set when I start the Visual Studio command prompt - that's the whole point abou tusing the VS commandprompt and not a normal one. I think you get a question about integrating the Platform SDK with Visual studio when you install it - any chance you missed that one? Well, it works in the gui, so I thought I got that integrated correctly. One of the deals with the visual c express thing is that it does not come with the headers and libraries and that you have to use the platform sdk instead. I already have ActivePerl and ActivePython installed, so those should work out. I am not really concerned about krb5 and ldap, so as long as commenting them out will disable them, that is good. You can safely leave LDAP in, because it uses only the builtin functionality in the OS and no external dependencies. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- I don't know what you mean by `glory,' Alice said Humpty Dumpty smiled contemptuously. Of course you don't -- till I tell you. I meant `there's a nice knock-down argument for you!' But glory doesn't mean `a nice knock-down argument,' Alice objected. When I use a word, Humpty Dumpty said, in a rather scornful tone, it means just what I choose it to mean -- neither more nor less. The question is, said Alice, whether you can make words mean so many different things. The question is, said Humpty Dumpty, which is to be master-- that's all. -- Lewis Carroll, Through the Looking Glass ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] src/tools/msvc usage instructions
On Sun, 1 Oct 2006, Jeremy Drake wrote: On Mon, 2 Oct 2006, Magnus Hagander wrote: If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. If I do build solution it tells me Project not selected to build for this solution configuration for all projects, then 55 skipped at the end. I clicked around a little, selected the postgres project in the project list, and switched to the release configuration, and now build solution works. Hmm. Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. I switched to short paths in the INCLUDE env var, but it seems to just ignore it. I'll have to look around for how to deal with this, but for now perhaps the gui will work ok. Is there anything that needs to happen post-compile that may not get done if I use the gui? -- H. L. Mencken suffers from the hallucination that he is H. L. Mencken -- there is no cure for a disease of that magnitude. -- Maxwell Bodenheim ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/msvc usage instructions
If you do build solution it should build all project sin the correct order - there are dependency references set between them that should take care of this automatically. If I do build solution it tells me Project not selected to build for this solution configuration for all projects, then 55 skipped at the end. I clicked around a little, selected the postgres project in the project list, and switched to the release configuration, and now build solution works. Hmm. Very interesting. Never done that for me :-) I need to do some more checking. Let me know if you figure out exactly why this happens... Do you have any idea how to get the environment to know where windows.h is? I even explicitly added the directory to the INCLUDE environment variable, but it did not work. I will try switching to short paths in there in case it is an issue of paths with spaces. I switched to short paths in the INCLUDE env var, but it seems to just ignore it. I'll have to look around for how to deal with this, but for now perhaps the gui will work ok. Is there anything that needs to happen post-compile that may not get done if I use the gui? Nope. build.bat pretty much just calls mkvcbuild.pl (to generate the project files), and then calls msbuild to build all the project files - the exactly same way the GUI does it. (Or it's supposed to be exactly the same way). Nothing more. So GUI building is just fine, just not automated. //Magnus ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] src/tools/msvc usage instructions
Hi. I think that it has forgotten for VS2005-express to add path of SDK by myself. http://www.winpg.jp/~saito/VS2005/VS2005_Include.png http://www.winpg.jp/~saito/VS2005/VS2005_Library.png Do I mistake your meaning? Regards, Hiroshi Saito I switched to short paths in the INCLUDE env var, but it seems to just ignore it. I'll have to look around for how to deal with this, but for now perhaps the gui will work ok. Is there anything that needs to happen post-compile that may not get done if I use the gui? Nope. build.bat pretty much just calls mkvcbuild.pl (to generate the project files), and then calls msbuild to build all the project files - the exactly same way the GUI does it. (Or it's supposed to be exactly the same way). Nothing more. So GUI building is just fine, just not automated. //Magnus ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Please build --with-ldap for Win32 beta1 binaries
Can we have the beta1 binaries for Win32 with --with-ldap? On Windows, the LDAP library is part of the OS, so we won't incur any undesirable dependencies. Yours, Laurenz Albe ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Please build --with-ldap for Win32 beta1 binaries
Can we have the beta1 binaries for Win32 with --with-ldap? On Windows, the LDAP library is part of the OS, so we won't incur any undesirable dependencies. Oops, that's an oversight, it's supposed to be there. --with-ldap will be enabled in beta2. Sorry 'bout that. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Another idea for dealing with cmin/cmax
Ühel kenal päeval, E, 2006-10-02 kell 01:30, kirjutas Tom Lane: Jim C. Nasby [EMAIL PROTECTED] writes: ... place a limit on the number of transactions that can be live in a table at once. Urk, well maybe, but ... you could shrink all the visibility info to 1 byte if you wanted to. ... 256 of 'em is surely not an acceptable limit. I have been thinking about this, and it seems that especially for OLAP loads it would be much better to keep tuple visibility info in a separate file, lets call it Tuple Visibility Map (TVM) TVM would have the following benefits: 1) TVM could be uses for index-only lookups as well as heap-only lookups, also other index lookups could be filtered against it fast before going to heap. 2) TVM could be heavily compressed, especially for bulk loads something like a single (xmin, xmax,cmin,cmax) tuple plus RLE-encoded list of pointers to it will do. 3) In case TVM space is needed in in any page, it would be easy to just throw away cmin/cmax from tuples from committed/aborted transactions. 4) First pass of VACUUM would be much faster, as it has to scan only TVM. Pages with no expired tuples would not need to be touched. If we can come up with a good design for TVM, it may also be an overall win for many kinds of OLTP queries, as it may result in less writes to disk and almost the same amount of writing to WAL. Maybe bitmap or btree index would be something to use as a starting point when designing TVM. Another idea to consider would be to merge FSM and TVM and then use them also for keeping data in cluster order. -- Hannu Krosing Database Architect Skype Technologies OÜ Akadeemia tee 21 F, Tallinn, 12618, Estonia Skype me: callto:hkrosing Get Skype for free: http://www.skype.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Replication hooks discussion
On Friday 29 September 2006 20:02, Andrew Sullivan wrote: At the beginning of the month, in http://archives.postgresql.org/pgsql-hackers/2006-09/msg00453.php, I said that I'd be willing to try to do any sort of co-ordination, document writing, c. for a project that might define common back-end resources necessary for the various kinds of replication systems people seem to want. There seems to be a widespread agreement that there is more than one sort of replication facilities that are desired, and that none of the systems on offer satisfies all of those desires. There also seems to be a hope that we could come to some sort of agreement on what the necessary conditions for any of these facilties are. If we could, then we could build the necessary framework to provide those conditions, and it could be made available in the back end without every replication project having to be shipped with the main PostgreSQL code. We at the GORDA project strongly agree with this approach. I'll try to summarize our proposals on the new list. Regards, -- Jose Orlando Pereira ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [PATCHES] Bad bug in fopen() wrapper code
Now, I still twist my head around the lines: if ((fd = _open_osfhandle((long) h, fileFlags O_APPEND)) 0 || (fileFlags (O_TEXT | O_BINARY) (_setmode(fd, fileFlags (O_TEXT | O_BINARY)) 0))) Without having studied it closely, it might also highlight a bug on failure of the second clause -- if the _setmode fails, shouldn't _close be called instead of CloseHandle, and -1 returned? (CloseHandle would still be called on failure of the _open_osfhandle, obviously) I agree that this code is both wrong and unreadable (although in practice the _setmode will probably never fail, which is why our attention hasn't been drawn to it). Is someone going to submit a patch? I'm hesitant to change the code myself since I'm not in a position to test it. I can look at fixing that. Is it something we want to do for 8.2, or wait until 8.3? If there's a hidden bug, I guess 8.2? //Magnus ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] In which module does query rewriting takes place
Hi,We are trying to introduce access control. For this we have to rewrite the input query by replacing each relation by its corresponding authorized view. Which part of the code should we modify for this. Till now we have thought of the following: 1. We take as input the parse tree generated by the parser.2. We implement a new module which rewrites the parse tree into the format we want.3. We feed this new parse tree into the optimizer. Does a module exist which rewrites the parse tree generated. We saw that there exist some rewrite module in /src/backend/rewrite. Is that the correct module to make changes to!ThanksShaunak -- It is not the aptitude but the attitude that decides a persons altitudeShaunak GodboleSenior UndergraduateComputer Science Dept.IIT Powai, Mumbai 400076Ph no: 9819841537
Re: [HACKERS] [PATCHES] Bad bug in fopen() wrapper code
Magnus Hagander [EMAIL PROTECTED] writes: I agree that this code is both wrong and unreadable (although in practice the _setmode will probably never fail, which is why our attention hasn't been drawn to it). Is someone going to submit a patch? I'm hesitant to change the code myself since I'm not in a position to test it. I can look at fixing that. Is it something we want to do for 8.2, or wait until 8.3? If there's a hidden bug, I guess 8.2? Yeah, I think it's 8.2 material. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] In which module does query rewriting takes place
Shaunak Godbole wrote: Hi, We are trying to introduce access control. For this we have to rewrite the input query by replacing each relation by its corresponding authorized view. This sounds like a pretty ugly hack. There are already extensive access controls available in postgres. What is there about the existing access controls that does not do what you want? cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] In which module does query rewriting takes place
Andrew Dunstan [EMAIL PROTECTED] writes: Shaunak Godbole wrote: We are trying to introduce access control. This sounds like a pretty ugly hack. Perhaps more to the point, it's already been done: http://veil.projects.postgresql.org/ regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/msvc usage instructions
Magnus Hagander [EMAIL PROTECTED] writes: Certainly not --- we work with both of those (I have 1.875 on one of my devel machines and 2.1 on two others). Well, it *does* break with 2.1 on win32-native. Could be something simple, could be that it's just generating msvc-incompatible code. Hm, how does it break exactly? There may not be much we can do about it except inform the bison guys, but seems like we should look into it. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Another idea for dealing with cmin/cmax
Tom Lane [EMAIL PROTECTED] writes: Jim C. Nasby [EMAIL PROTECTED] writes: ... place a limit on the number of transactions that can be live in a table at once. Urk, well maybe, but ... you could shrink all the visibility info to 1 byte if you wanted to. ... 256 of 'em is surely not an acceptable limit. The plan Gavin Sherry and I were discussing at the Code Sprint was to store a single most common xmin xmin in the per-page special area. Then have a bit on each tuple indicating that the xmin isn't present in the tuple and instead to use the xmin from the page footer. Any tuples with other values of xmin would have to store that xmin normally. The use case here is primarily tables loaded in large batch jobs that have large swaths of the table, possibly the entire table, loaded in the same transaction. My thinking was that most common xmin could be very approximate. In fact my my plan was to just store the first xmin the page sees there. This catches the common use cases of pg_restore or COPY loading many records and even catches most cases of large inserts into existing tables whenever they extend the table. A further refinement could be to have vacuum look for the actual most common xmin and rewrite the page using it. If there's enough free space it may be worthwhile storing InvalidTransactionId and allowing the next insert to set the most common xmin. However I'm not convinced of the importance of this refinement. The thing to remember is that shaving bits off tuples is not a goal in itself. It's a means to an end, namely packing more tuples on a page. If we shave off 4 bytes off every tuple when we're loading thousands of tuples then we get to put more of them on a page. If we save space on tuples when we're running vacuum that just gives us more free space in the free space map and we only see a benefit down the line if we end up eventually filling up that space. -- greg ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] In which module does query rewriting takes place
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Shaunak Godbole wrote: We are trying to introduce access control. This sounds like a pretty ugly hack. Perhaps more to the point, it's already been done: http://veil.projects.postgresql.org/ Ah, the missing piece is that case is row level. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/msvc usage instructions
Jeremy Drake [EMAIL PROTECTED] writes:
The errors I got on this file were:
1bootparse.tab.c(1065) : error C2449: found '{' at file scope (missing
function header?)
I looked at this. Line 1065 is the left brace starting yyparse(). On
my Fedora Core 5 box with Bison 2.1 installed, the stuff leading up to
it is
#ifdef YYPARSE_PARAM
... some uninteresting stuff, since we don't define YYPARSE_PARAM ...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
#endif
#endif
{
But lookee here, your Windows-built version has
#ifdef YYPARSE_PARAM
...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
;
#endif
#endif
{
So that semicolon is the source of the trouble. That's clearly a bison
bug, and in fact digging in Red Hat's SRPM shows that they are carrying
a patch for it:
2005-10-05 Paul Eggert [EMAIL PROTECTED]
* data/m4sugar/m4sugar.m4 (_m4_map): New macro.
(m4_map, m4_map_sep): Use it. Handle the empty list correctly.
--- bison-2.1/data/m4sugar/m4sugar.m4
+++ bison-2.1/data/m4sugar/m4sugar.m4
@@ -398,8 +398,11 @@ m4_define([m4_cdr],
# of LIST (which can be lists themselves, for multiple arguments MACROs).
m4_define([m4_fst], [$1])
m4_define([m4_map],
+[m4_if([$2], [[]], [],
+ [_m4_map([$1], [$2])])])
+m4_define([_m4_map],
[m4_ifval([$2],
- [$1(m4_fst($2))[]m4_map([$1], m4_cdr($2))])])
+ [$1(m4_fst($2))[]_m4_map([$1], m4_cdr($2))])])
# m4_map_sep(MACRO, SEPARATOR, LIST)
@@ -408,8 +411,8 @@ m4_define([m4_map],
# are the elements of LIST (which can be lists themselves, for multiple
# arguments MACROs).
m4_define([m4_map_sep],
-[m4_ifval([$3],
- [$1(m4_fst($3))[]m4_map([$2[]$1], m4_cdr($3))])])
+[m4_if([$3], [[]], [],
+ [$1(m4_fst($3))[]_m4_map([$2[]$1], m4_cdr($3))])])
## ##
Presumably bison 2.2 includes this fix. But I guess the real question
is why the devil doesn't MSVC define __STDC__ ? Are they that far
removed from spec compliance?
regards, tom lane
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Faster StrNCpy
Mark, Thanks for attaching the C code for your test. I ran a few tests on a 3Ghz Intel Xeon Paxville (dual core) system. I hope the formatting of this table survives: Method Size N=1024*1024 N=1 MEMCPY 63 6964927 us 582494 us MEMCPY 32 7102497 us 582467 us MEMCPY 16 7116358 us 582538 us MEMCPY 8 6965239 us 582796 us MEMCPY 4 6964722 us 583183 us STRNCPY 6310131174 us8843010 us STRNCPY 3210648202 us9563868 us STRNCPY 16 9187398 us7969947 us STRNCPY 8 9275353 us8042777 us STRNCPY 4 9067570 us8058532 us STRLCPY 6315045507 us 14379702 us STRLCPY 32 8960303 us8120471 us STRLCPY 16 7393607 us4915457 us STRLCPY 8 7222983 us3211931 us STRLCPY 4 7181267 us1725546 us LENCPY 63 7608932 us4416602 us LENCPY 32 7252849 us3807535 us LENCPY 1611680927 us 10331487 us LENCPY 8 10409685 us9660616 us LENCPY 4 10824632 us9525082 us The first column is the copy method, the second column is the source string size (size of -DSTRING), the 3rd and 4th columns are the different -DN settings. The memcpy () call is the clear winner, at all source string sizes. The strncpy () call is better than strlcpy (), until the size of the string decreases. This is probably due to the zero padding effect of strncpy. The lencpy () call starts out strong, but degrades as the size of the string decreases. This was a little surprising and I don't have an explanation for this behavior at this time. The AMD optimization manuals have some interesting examples for optimizations for memcpy, along the lines of cache line copies and prefetching: http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/25112.PDF#search=%22amd%20optimization%20manual%22 h http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/22007.pdf#search=%22amd%20optimization%20manual%22 ttp://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/22007.pdf#search=%22amd%20optimization%20manual%22 There also used to be an interesting article on the SGI web site called Optimizing CPU to Memory Accesses on the SGI Visual Workstations 320 and 540, but this seems to have been pulled. I did find a copy of the article here: http://eunchul.com/database/board/cat.php?data=Win32_APIboard_group=D42a8ff5c3a9b9 Obviously, different copy mechanisms suit different data sizes. So, I added a little debug to the strlcpy () function that was added to Postgres the other day. I ran a test against Postgres for ~15 minutes that used 2 client backends and the BG writer - 8330804 calls to strlcpy () were generated by the test. Out of the 8330804 calls, 6226616 calls used a maximum copy size of 2213 bytes e.g. strlcpy (dest, src, 2213) and 2104074 calls used a maximum copy size of 64 bytes. I know the 2213 size calls come from the set_ps_display () function. I don't know where the 64 size calls come from, yet. In the 64 size case, with the exception of 35 calls, calls for size 64 are only copying 1 byte - I would assume this is a NULL. In the 2213 size case, 1933027 calls copy 20 bytes; 2189415 calls copy 5 bytes; 85550 calls copy 6 bytes and 2018482 calls copy 7 bytes. Based on this data, it would seem that either memcpy () or strlcpy () calls would be better due to the source string size. Call originating from the set_ps_display () function might be able to use the memcpy () call as the size of the source string should be known. The other calls probably need something like strlcpy () as the source string might not be known, although using memcpy () to copy in XX byte blocks might be interesting. David From: [EMAIL PROTECTED] on behalf of [EMAIL PROTECTED] Sent: Fri 9/29/2006 2:59 PM To: Tom Lane Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Faster StrNCpy On Fri, Sep 29, 2006 at 05:34:30PM -0400, Tom Lane wrote: [EMAIL PROTECTED] writes: If anybody is curious, here are my numbers for an AMD X2 3800+: You did not show your C code, so no one else can reproduce the test on other hardware. However, it looks like your compiler has unrolled the memcpy into straight-line 8-byte moves, which makes it pretty hard for anything operating byte-wise to compete, and is a bit dubious for the general case anyway (since it requires assuming that the size and alignment are known at compile time). I did show the .s code. I call into x_memcpy(a, b), meaning that the compiler can't assume anything. It may happen to be aligned. Here are results over 64 Mbytes of memory, to ensure that every call is a cache miss: $ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x NONE:767243 us MEMCPY: 6044137 us STRNCPY: 10741759 us STRLCPY: 12061630 us LENCPY: 9459099 us $ gcc -O3
Re: [HACKERS] Select for update with outer join broken?
Tom, Josh, you don't know what you're talking about. The backend's capabilities for this have not moved an inch since 8.1 (transient bugs in its error checking do not represent an advance in capability), Hmmm ... was this an unapplied patch? We certainly had it working on the benchmark machine. Researching ... and furthermore I'm not seeing anyplace in the SQL spec that suggests the nullable side of an outer join should be updatable. Oh, no, what should happen is that the outer join portion of the query doesn't get locked, rather than a fatal exception. That behavior is expected by the J2EE certification, so it's at least somewhat industry-standard. -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Select for update with outer join broken?
Josh Berkus josh@agliodbs.com writes: Oh, no, what should happen is that the outer join portion of the query doesn't get locked, rather than a fatal exception. That behavior is expected by the J2EE certification, so it's at least somewhat industry-standard. Really? Please cite chapter and verse. And why should it be that the unmatched outer-side rows don't get locked (which is what I think you just said)? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [BUGS] Update using sub-select table in schema
On Mon, 2006-10-02 at 11:38 -0400, Tom Lane wrote: Chris Dunlop [EMAIL PROTECTED] writes: I'm not sure if this is a bug or if it's displaying my ignorance of this corner of SQL... update a set name = ( select name from temp.a where temp.a.id = a.id ) Postgres treats FROM temp.a the same as FROM temp.a AS a, and then the a.id references that rather than the outer query's A. Try just select name from temp.a where temp.a.id = a.id; to see this in action. [Which AFAICS doesn't follow SQL:2003, since a reference to temp.a should not be allowed following its redefinition as a.] Looking at the SQL spec, I'm having a hard time finding any verbiage that either confirms or denies this interpretation. It says that a FROM item without a correlation name (ie, an AS alias) exposes the table name, and then a column reference can refer to the table name, but it's not at all clear whether the table name must be qualified or not in the reference. Comments anyone? In my SQL:2003 draft, there is a Language Opportunity here: 242 [From London] The following Opportunity exists: For language consistency, a correlation name should be permitted for the modified table in positioned and searched update and delete statements. However, somebody claiming to be Joe Celko is quoted here as saying that is not part of the SQL:2003 standard http://www.thescripts.com/forum/thread65819.html ISTM the most obvious route in this situation differs from normal usage: treat any unqualified names that match the target table as a reference to the target table, rather than potentially another table. i.e. treat this situation as a correlated sub-query rather than as an independent query. No correlation name on the target table is allowed, so there is no possibility of writing a correlation name to allow the query to be more readable: update a correlationname set name = ( select name from temp.a where temp.a.id = correlationname.a.id ) (which is definitely not allowed by SQL:2003) Having said all of that, its clearly a grey area so no need to change this as part of beta, since we could easily cause more wierdness than we solve. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] undescribe
unsubscribepgsql-hackers
Re: [HACKERS] Faster StrNCpy
[EMAIL PROTECTED] writes:
Here is the cache hit case including your strlen+memcpy as 'LENCPY':
$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected
to be very slow.' -DN=1 -o x x.c y.c strlcpy.c ; ./x
NONE:696157 us
MEMCPY: 825118 us
STRNCPY:7983159 us
STRLCPY: 10787462 us
LENCPY: 6048339 us
It appears that these results are a bit platform-dependent; on my x86_64
(Xeon) Fedora 5 box, I get
$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to
be very slow.' -DN=1 x.c y.c strlcpy.c
$ ./a.out
NONE:358679 us
MEMCPY: 619255 us
STRNCPY:8932551 us
STRLCPY:9212371 us
LENCPY:13910413 us
I'm not sure why the lencpy method sucks so badly on this machine :-(.
Anyway, I looked at glibc's strncpy and determined that on this machine
the only real optimization that's been done to it is to unroll the data
copying loop four times. I did the same to strlcpy (attached) and got
numbers like these:
$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to
be very slow.' -DN=1 x.c y.c strlcpy.c
$ ./a.out
NONE:359317 us
MEMCPY: 619636 us
STRNCPY:8933507 us
STRLCPY:7644576 us
LENCPY:13917927 us
$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to
be very slow.' -DN=(1024*1024) x.c y.c strlcpy.c
$ ./a.out
NONE:502960 us
MEMCPY: 5382528 us
STRNCPY:9733890 us
STRLCPY:8740892 us
LENCPY:15358616 us
$ gcc -O3 -std=c99 -DSTRING='short' -DN=1 x.c y.c strlcpy.c
$ ./a.out
NONE:358426 us
MEMCPY: 618533 us
STRNCPY:6704926 us
STRLCPY: 867336 us
LENCPY:10115883 us
$ gcc -O3 -std=c99 -DSTRING='short' -DN=(1024*1024) x.c y.c strlcpy.c
$ ./a.out
NONE:502746 us
MEMCPY: 5365171 us
STRNCPY:7983610 us
STRLCPY:5557277 us
LENCPY:11533066 us
So the unroll seems to get us to the point of not losing compared to the
original strncpy code for any string length, and so I propose doing
that, if it holds up on other architectures.
regards, tom lane
size_t
strlcpy(char *dst, const char *src, size_t siz)
{
char *d = dst;
const char *s = src;
size_t n = siz;
/* Copy as many bytes as will fit */
if (n != 0) {
while (n 4) {
if ((*d++ = *s++) == '\0')
goto done;
if ((*d++ = *s++) == '\0')
goto done;
if ((*d++ = *s++) == '\0')
goto done;
if ((*d++ = *s++) == '\0')
goto done;
n -= 4;
}
while (--n != 0) {
if ((*d++ = *s++) == '\0')
goto done;
}
}
/* Not enough room in dst, add NUL and traverse rest of src */
if (siz != 0)
*d = '\0'; /* NUL-terminate dst */
while (*s++)
;
done:
return(s - src - 1);/* count does not include NUL */
}
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Faster StrNCpy
Mark, On 9/29/06 2:59 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Here are results over 64 Mbytes of memory, to ensure that every call is a cache miss: On my Mac OSX intel laptop (Core Duo, 2.16 GHz, 2GB RAM, gcc 4.01): Luke-Lonergans-Computer:~/strNcpy-perf-test lukelonergan$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x /usr/bin/ld: warning multiple definitions of symbol _strlcpy /var/tmp//cc1eMVq7.o definition of _strlcpy in section (__TEXT,__text) /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib(strlcpy.So) definition of _strlcpy NONE:416677 us MEMCPY: 5649587 us STRNCPY:5806591 us STRLCPY: 12865010 us LENCPY:17801485 us Luke-Lonergans-Computer:~/strNcpy-perf-test lukelonergan$ gcc -O3 -std=c99 -DSTRING='Short sentence.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x /usr/bin/ld: warning multiple definitions of symbol _strlcpy /var/tmp//ccOZl9R6.o definition of _strlcpy in section (__TEXT,__text) /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib(strlcpy.So) definition of _strlcpy NONE:416652 us MEMCPY: 5830540 us STRNCPY:6207594 us STRLCPY:5582607 us LENCPY: 7887703 us Luke-Lonergans-Computer:~/strNcpy-perf-test lukelonergan$ gcc -O3 -std=c99 -DSTRING='' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x /usr/bin/ld: warning multiple definitions of symbol _strlcpy /var/tmp//ccBUsIdR.o definition of _strlcpy in section (__TEXT,__text) /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib(strlcpy.So) definition of _strlcpy NONE:417210 us MEMCPY: 5506346 us STRNCPY:5769302 us STRLCPY:5424234 us LENCPY: 5609338 us - Luke ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Faster StrNCpy
Thanks for the results on your machines Dave, Luke, and Tom. Interesting. On Mon, Oct 02, 2006 at 02:30:11PM -0400, Tom Lane wrote: It appears that these results are a bit platform-dependent; on my x86_64 (Xeon) Fedora 5 box, I get *nod* Anyway, I looked at glibc's strncpy and determined that on this machine the only real optimization that's been done to it is to unroll the data copying loop four times. I did the same to strlcpy (attached) and got numbers like these: So the unroll seems to get us to the point of not losing compared to the original strncpy code for any string length, and so I propose doing that, if it holds up on other architectures. Sounds reasonable. strlcpy() is the best match. I'm not sure why GLIBC doesn't come with strlcpy(). Apparently some philosophical differences against it by the GLIBC maintainers... :-) memcpy() makes too many assumptions, that would be difficult to prove in each case, or may be difficult to ensure that they remain true. I only mentioned it as a possibility, to keep the options open, and to show the sort of effects we are dealing with. Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __ . . _ ._ . . .__. . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/|_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Faster StrNCpy
Mark, Tom, Just the test on IA64 (Itanium2, 1.6Ghz, 8Gb memory). The results seem to be quite different: [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x NONE:825671 us MEMCPY: 4533637 us STRNCPY: 10959380 us STRLCPY: 34258306 us LENCPY:13939373 us [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=1 -o x x.c y.c strlcpy.c ; ./x NONE:938815 us MEMCPY: 3441330 us STRNCPY:5881628 us STRLCPY: 20167825 us LENCPY:11326259 us [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='Short sentence.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x NONE:831920 us MEMCPY: 4550683 us STRNCPY:9339254 us STRLCPY: 16871433 us LENCPY:21332076 us [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='Short sentence.' -DN=1 -o x x.c y.c strlcpy.c ; ./x NONE:938104 us MEMCPY: 3438871 us STRNCPY:3438969 us STRLCPY:6042829 us LENCPY: 6022909 us [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x NONE:825472 us MEMCPY: 4547456 us STRNCPY:8591618 us STRLCPY:6818957 us LENCPY: 8264340 us [EMAIL PROTECTED]:~/test$ gcc -O3 -std=c99 -DSTRING='' -DN=1 -o x x.c y.c strlcpy.c ; ./x NONE:937878 us MEMCPY: 3439101 us STRNCPY:3188791 us STRLCPY: 750437 us LENCPY: 2751238 us Regards, Sergey *** Sergey E. Koposov Max Planck Institute for Astronomy/Sternberg Astronomical Institute Tel: +49-6221-528-349 Web: http://lnfm1.sai.msu.ru/~math E-mail: [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Faster StrNCpy
Sergey E. Koposov [EMAIL PROTECTED] writes: Just the test on IA64 (Itanium2, 1.6Ghz, 8Gb memory). The results seem to be quite different: What libc are you using exactly? Can you try it with the unrolled strlcpy I posted? In glibc-2.4.90, there seem to be out-of-line assembly code implementations of strncpy for: sparc64 sparc32 s390x s390 alpha ia64 and an inlined assembler version for i386. So the x86_64 case is nearly the only popular architecture that doesn't seem to have a hand-hacked implementation ... which throws some doubt on Mark's and my results as possibly not being very representative. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Ready for pgindent?
Are people ready for me to run pgindent? -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] Ready for beta2?
Are we ready to think about a beta2? Seems beta1 was quiet. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Faster StrNCpy
On Mon, 2 Oct 2006, Tom Lane wrote: Sergey E. Koposov [EMAIL PROTECTED] writes: Just the test on IA64 (Itanium2, 1.6Ghz, 8Gb memory). The results seem to be quite different: What libc are you using exactly? Can you try it with the unrolled strlcpy I posted? glibc 2.3.5 , gcc 3.4.4 my results were obtained already with your unrolled version (but when I first ran it with the 'default' strlcpy the results were the same). In glibc-2.4.90, there seem to be out-of-line assembly code implementations of strncpy for: sparc64 sparc32 s390x s390 alpha ia64 and an inlined assembler version for i386. So the x86_64 case is nearly the only popular architecture that doesn't seem to have a hand-hacked implementation ... which throws some doubt on Mark's and my results as possibly not being very representative. Regards, Sergey *** Sergey E. Koposov Max Planck Institute for Astronomy/Sternberg Astronomical Institute Tel: +49-6221-528-349 Web: http://lnfm1.sai.msu.ru/~math E-mail: [EMAIL PROTECTED] ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Faster StrNCpy
I did a couple more tests using x86 architectures. On a rather old Pentium-4 machine running Fedora 5 (gcc 4.1.1, glibc-2.4-11): $ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=(1024*1024) -o x x.c y.c strlcpy.c NONE:786305 us MEMCPY: 9887843 us STRNCPY: 15045780 us STRLCPY: 1763 us U_STRLCPY: 14994639 us LENCPY:19700346 us $ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=1 -o x x.c y.c strlcpy.c NONE:562001 us MEMCPY: 2026546 us STRNCPY: 11149134 us STRLCPY: 13747827 us U_STRLCPY: 12467527 us LENCPY:17672899 us (STRLCPY is our CVS HEAD code, U_STRLCPY is the unrolled version) On a Mac Mini (Intel Core Duo, OS X 10.4.8, gcc 4.0.1), the system has a version of strlcpy, but it seems to suck :-( $ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=(1024*1024) -o x x.c y.c strlcpy.c ; ./x NONE:480298 us MEMCPY: 7857291 us STRNCPY: 10485948 us STRLCPY: 16745154 us U_STRLCPY: 18337286 us S_STRLCPY: 20920213 us LENCPY:22878114 us $ gcc -O3 -std=c99 -DSTRING='This is a very long sentence that is expected to be very slow.' -DN=1 -o x x.c y.c strlcpy.c ; ./x NONE:480269 us MEMCPY: 1858974 us STRNCPY:5405618 us STRLCPY: 16364452 us U_STRLCPY: 16439753 us S_STRLCPY: 19134538 us LENCPY:22873141 us It's interesting that the unrolled version is actually slower here. I didn't dig into the assembly code, but maybe Apple's compiler isn't doing a very good job with it? Anyway, these results make me less excited about the unrolled version. In any case, I don't think we should put too much emphasis on the long-source-string case. In essentially all cases, the true source string length will be much shorter than the target buffer (were this not so, we'd probably be needing to make the buffer bigger), and strlcpy will certainly beat out strncpy in those situations. The memcpy numbers look attractive, but they ignore the problem that in practice we usually don't know the source string length in advance --- so I don't think those represent something achievable. One thing that seems real clear is that the LENCPY method loses across the board, which surprised me, but it's hard to argue with numbers. I'm still interested to experiment with MemSet-then-strlcpy for namestrcpy, but given the LENCPY results this may be a loser too. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Faster StrNCpy
Strong, David [EMAIL PROTECTED] writes: Obviously, different copy mechanisms suit different data sizes. So, I added a little debug to the strlcpy () function that was added to Postgres the other day. I ran a test against Postgres for ~15 minutes that used 2 client backends and the BG writer - 8330804 calls to strlcpy () were generated by the test. Out of the 8330804 calls, 6226616 calls used a maximum copy size of 2213 bytes e.g. strlcpy (dest, src, 2213) and 2104074 calls used a maximum copy size of 64 bytes. I know the 2213 size calls come from the set_ps_display () function. I don't know where the 64 size calls come from, yet. Prepared-statement and portal hashtable lookups, likely. Were your clients using V3 extended query protocol? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Faster StrNCpy
Tom, Yes, the clients are using the V3 protocol and prepared statements. David From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Mon 10/2/2006 2:09 PM To: Strong, David Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Faster StrNCpy Strong, David [EMAIL PROTECTED] writes: Obviously, different copy mechanisms suit different data sizes. So, I added a little debug to the strlcpy () function that was added to Postgres the other day. I ran a test against Postgres for ~15 minutes that used 2 client backends and the BG writer - 8330804 calls to strlcpy () were generated by the test. Out of the 8330804 calls, 6226616 calls used a maximum copy size of 2213 bytes e.g. strlcpy (dest, src, 2213) and 2104074 calls used a maximum copy size of 64 bytes. I know the 2213 size calls come from the set_ps_display () function. I don't know where the 64 size calls come from, yet. Prepared-statement and portal hashtable lookups, likely. Were your clients using V3 extended query protocol? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] on-disk bitmap indexes
Hello, I carefully check all mail threads of Re: On-disk bitmap index patch http://secure.linuxports.com/pgsql-hackers/2006-07/msg01514.php I am testing 8.2 postgresql features and wonder if on-disk bitmap indexes are included in beta version. It could be great to test their performance. Sincerely, Radovan Jablonovsky Database Architect/DBA Arrow Transportation Systems Inc. Tel: (250) 571-7773 Email: [EMAIL PROTECTED] WWW: www.arrowtransportation.com
Re: [HACKERS] on-disk bitmap indexes
Radovan Jablonov wrote: Hello, I carefully check all mail threads of Re: http://secure.linuxports.com/pgsql-hackers/2006-07/msg01507.php On-disk bitmap index patch http://secure.linuxports.com/pgsql-hackers/2006-07/msg01514.php I am testing 8.2 postgresql features and wonder if on-disk bitmap indexes are included in beta version. It could be great to test their performance. No, the patch was not completed in time for 8.2. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] NULL and plpgsql rows
I'm looking at how NULLs are handled in relation to plpgsql row types.
Looking at exec_assign_value, it appears that we're supposed to be able
to handle setting a row variable to NULL:
if (*isNull)
{
/* If source is null, just assign nulls to the row */
exec_move_row(estate, NULL, row, NULL, NULL);
}
However, the test right above that means that we'll fail if the user
tries something like row_variable := NULL;:
if (!(valtype == RECORDOID ||
get_typtype(valtype) == 'c'))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg(cannot assign non-composite value to a row
variable)));
Presumably, I can just add code to that test to allow for *isNull.
Of course, setting a row variable to null is a lot more useful if we can
actually test for it after the fact, and I'm not really sure how to make
that happen.
--
Jim Nasby[EMAIL PROTECTED]
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] initdb ignores invalid locale names
Is this a TODO item? --- Greg Sabino Mullane wrote: [ There is text before PGP section. ] -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 When initdb is given an invalid (possibly mistyped) locale name, it just ... /* should we exit here? */ ... I obviously think we should. Why shouldn't we? +1. This has been on my long-term, low-priority todo list for a while. Raising the issue that it, not necessarily fixing it. I gotta get more organized... :) - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key: 0x14964AC8 200609221704 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQAKvJuQZxSWSsgRAmSzAJ0Uz4RqQEBvsn4S3uGwbslfw/j8+QCg/uzd FFa7vbTh6YnF2gsdSxUCdzM= =tc4X -END PGP SIGNATURE- ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] timestamptz alias
Hi, I'm happy that the rather verbose timestamp with time zone has the much nicer alias timestamptz, however it seems that this alias is not documented, neither at http://developer.postgresql.org/pgdocs/postgres/datatype-datetime.html nor at http://www.postgresql.org/docs/8.1/interactive/datatype-datetime.html I see it mentioned at http://www.postgresql.org/docs/8.1/interactive/datatype.html but that's possibly not where people look at first, when they search for the timestamp type. (At least I found it only when grepping for timestamptz in the docs. :-) Should the alias be mentioned on the datetime page? The same for timetz? What do you think? Thanks, Markus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Tom Lane wrote:
Jeremy Drake [EMAIL PROTECTED] writes:
The errors I got on this file were:
1bootparse.tab.c(1065) : error C2449: found '{' at file scope (missing
function header?)
I looked at this. Line 1065 is the left brace starting yyparse(). On
my Fedora Core 5 box with Bison 2.1 installed, the stuff leading up to
it is
#ifdef YYPARSE_PARAM
... some uninteresting stuff, since we don't define YYPARSE_PARAM ...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
#endif
#endif
{
But lookee here, your Windows-built version has
#ifdef YYPARSE_PARAM
...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus)
int
yyparse (void)
#else
int
yyparse ()
;
#endif
#endif
{
So that semicolon is the source of the trouble. That's clearly a bison
bug, and in fact digging in Red Hat's SRPM shows that they are carrying
a patch for it:
2005-10-05 Paul Eggert [EMAIL PROTECTED]
* data/m4sugar/m4sugar.m4 (_m4_map): New macro.
(m4_map, m4_map_sep): Use it. Handle the empty list correctly.
snip patch
Presumably bison 2.2 includes this fix. But I guess the real question
is why the devil doesn't MSVC define __STDC__ ? Are they that far
removed from spec compliance?
In the bison 2.2 generated code, the #if check is
#if (defined __STDC__ || defined __C99__FUNC__ \
|| defined __cplusplus || defined _MSC_VER)
which looks like they figured out that they needed to check for MicroSoft
C explicitly. I have no idea why they do not define __STDC__ however.
regards, tom lane
--
A person is just about as big as the things that make him angry.
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] Broken link in PG docs
I added the citeseer URL as an SGML comment. Neil is trying to contact the author. --- Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: I agree the link should be fixed, but I can't see another canonical location for the document online: linking to CiteSeer (which itself is generated from the mention in our online docs) is probably not wise. citeseer's cache still has the paper, and in it I find the authors' email addresses ... could try writing to them ... regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Ready for pgindent?
Bruce Momjian [EMAIL PROTECTED] writes: Are people ready for me to run pgindent? I don't see a reason to hold off. There aren't any more pending patches we intend to apply before 8.2 ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] horo(r)logy test fail on solaris (again and
Thanks for the analysis. I have removed mention of the -fast option from the Solaris FAQ. --- Kenneth Marshall wrote: On Wed, Sep 27, 2006 at 04:09:18PM +0200, Zdenek Kotala wrote: Tom Lane napsal(a): Zdenek Kotala [EMAIL PROTECTED] writes: The problem was generated, because -fast option was set only for the compiler and not for the linker. Linker takes wrong version of libraries. If -fast is set for both then horology test is OK, but question was if float optimalization should generate some problems. So FAQ_Solaris needs to tell people to put -fast in both CFLAGS and LDFLAGS? Exactly, but I want to sure, that float optimalization is safe and should be applied for postgres, because -fast breaks IEE754 standard. If it is OK I will adjust FAQ_Solaris. Zdenek Unless the packager understands the floating point usage of every piece and module included and the effect that the -fast option will have on them, please do not recommend it for anything but extremely well tested dedicated use-cases. When it causes problems, it can be terrible if the problems are not detected immediately. Massive data corruption could occur. Given these caveats, in a well tested use-case the -fast option can squeeze a bit more from the CPU and could be used. I have had to debug the fallout from the -fast option in other software in the past. Let's just say, backups are a good thing. I would vote not to recommend it without very strong cautions similar to was Sun includes in the compiler manual pages. Ken ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Sane error messages for SSL retry cases
Is this a TODO? I don't see how it is a new problem, meaning it probably is for 8.3. --- Tom Lane wrote: As per a recent discussion in pgsql-admin, http://archives.postgresql.org/pgsql-admin/2006-09/msg00297.php libpq doesn't cope well with the situation where the server is configured to allow only SSL connections (or only non-SSL connections) and there is some unrelated-to-SSL connection problem such as wrong password. The reason is that libpq is set up to retry with the other kind of connection (either dropping or adding SSL) for just about any sort of error returned by the server. This may lead to reporting no pg_hba.conf entry, or some such, rather than the more useful password authentication failed. I am tempted to propose that libpq should only retry in the other mode when the server specifically returns no pg_hba.conf entry, and not for other server errors (beyond the initial do-you-do-SSL-at-all handshake of course). This would save a useless fork() cycle on the server as well as make it more likely that we return a useful error message. There are some corner cases where this might fail to connect when a blind retry would have succeeded, but they all involve the server offering different auth methods depending on SSL or not --- an example is hostssl + ident and hostnossl + password, and you fail the ident test but could have produced the correct password. ISTM that is a scenario where the user should use the sslmode parameter to control which method is tried first. One problem with implementing this proposal is that we currently use the generic INVALID_AUTHORIZATION_SPECIFICATION sqlstate for a bunch of distinct conditions including no pg_hba.conf entry. Looking directly at the error string is of course not localization-proof, so we'd have to break down that errcode into some more-specific categories. Which is probably not a bad idea anyway, but it would mean that the nicer behavior would only happen when talking to an 8.2 or later server. Thoughts? Is this something to tackle during beta, or must we put it off till 8.3? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Faster StrNCpy
Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: A wholesale replacement of strncpy() calls is probably worth doing -- replacing them with strlcpy() if the source string is NUL-terminated, and I suppose memcpy() otherwise. What I'd like to do immediately is put in strlcpy() and hit the two or three places I think are performance-relevant. I agree with trying to get rid of StrNCpy/strncpy calls over the long run, but it's just code beautification and probably not appropriate for beta. Added to TODO: * Use strlcpy() rather than our StrNCpy() macro -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] src/tools/msvc usage instructions
On Mon, 2 Oct 2006, Jeremy Drake wrote: In the bison 2.2 generated code, the #if check is #if (defined __STDC__ || defined __C99__FUNC__ \ || defined __cplusplus || defined _MSC_VER) which looks like they figured out that they needed to check for MicroSoft C explicitly. I have no idea why they do not define __STDC__ however. From msdn ( http://msdn2.microsoft.com/en-us/library/b0084kay.aspx ): __STDC__ Indicates full conformance with the ANSI C standard. Defined as the integer constant 1 only if the /Za compiler option is given and you are not compiling C++ code; otherwise is undefined. Regards, Sergey *** Sergey E. Koposov Max Planck Institute for Astronomy/Sternberg Astronomical Institute Tel: +49-6221-528-349 Web: http://lnfm1.sai.msu.ru/~math E-mail: [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] src/tools/msvc usage instructions
I now get things to compile, but now I get linker errors on any dll which needs to access symbols from postgres.exe via postgres.lib. For example: 1-- Build started: Project: autoinc, Configuration: Release Win32 -- 1Generate DEF file 1Not re-generating AUTOINC.DEF, file already exists. 1Linking... 1 Creating library Release\autoinc\autoinc.lib and object Release\autoinc\autoinc.exp 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_modifytuple referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _pfree referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _pg_detoast_datum referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _nextval referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _DirectFunctionCall1 referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _textin referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_getbinval referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_gettypeid referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errfinish referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol ___msvc_errcode referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errmsg referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _errstart referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_fnumber referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _MemoryContextAlloc referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _SPI_getrelname referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _elog_finish referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _elog_start referenced in function _autoinc 1.\Release\autoinc\autoinc.dll : fatal error LNK1120: 17 unresolved externals I checked the project properties for linker options and it does list Release\postgres\postgres.lib in the additional dependencies list. Any ideas? Am I missing something? -- A penny saved is ridiculous. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Incrementally Updated Backup
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. --- Simon Riggs wrote: On Tue, 2006-09-19 at 12:13 -0400, Tom Lane wrote: Also, I'm not sold that the concept is even useful. Apparently the idea is to offload the expense of taking periodic base backups from a master server, by instead backing up a PITR slave's fileset --- which is fine. Good. That's the key part of the idea and its a useful one, so I was looking to document it for 8.2 I thought of this idea separately, then, as usual, realised that this idea has a long heritage: Change Accumulation has been in production use with IMS for at least 20 years. But why in the world would you want to stop the slave to do it? ISTM we would want to arrange things so that you can copy the slave's files while it continues replicating, just as with a standard base backup. You can do that, of course, but my thinking was that people would regard the technique as unsupported, so I added a quick flag as a prototype. On Tue, 2006-09-19 at 12:13 -0400, Tom Lane wrote: This patch has obviously been thrown together with no thought and even less testing. It breaks the normal case (I think the above if-test is backwards), and I don't believe that it works for the advertised purpose either (because nothing gets done to force a checkpoint before aborting, thus the files on disk are not up to date with the end of WAL). Yes, it was done very quickly and submitted to ensure it could be considered yesterday for inclusion. It was described by me as rushed, which it certainly was because of personal time pressure yesterday: I thought that made it clear that discussion was needed. Heikki mentions to me it wasn't clear, so those criticisms are accepted. On Tue, 2006-09-19 at 16:05 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: + + if (startupAfterRecovery) + ereport(ERROR, + (errmsg(recovery ends normally with startup_after_recovery=false))); + I find this part of the patch a bit ugly. Me too. Overall, my own thoughts and Tom's and Heikki's comments indicate I should withdraw the patch rather than fix it. Patch withdrawn. Enclose a new doc patch to describe the capability, without s/w change. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com [ Attachment, skipping... ] ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] Incrementally Updated Backup
Documentation patch applied. Thanks. Your documentation changes can be viewed in five minutes using links on the developer's page, http://www.postgresql.org/developer/testing. --- Simon Riggs wrote: On Tue, 2006-09-19 at 12:13 -0400, Tom Lane wrote: Also, I'm not sold that the concept is even useful. Apparently the idea is to offload the expense of taking periodic base backups from a master server, by instead backing up a PITR slave's fileset --- which is fine. Good. That's the key part of the idea and its a useful one, so I was looking to document it for 8.2 I thought of this idea separately, then, as usual, realised that this idea has a long heritage: Change Accumulation has been in production use with IMS for at least 20 years. But why in the world would you want to stop the slave to do it? ISTM we would want to arrange things so that you can copy the slave's files while it continues replicating, just as with a standard base backup. You can do that, of course, but my thinking was that people would regard the technique as unsupported, so I added a quick flag as a prototype. On Tue, 2006-09-19 at 12:13 -0400, Tom Lane wrote: This patch has obviously been thrown together with no thought and even less testing. It breaks the normal case (I think the above if-test is backwards), and I don't believe that it works for the advertised purpose either (because nothing gets done to force a checkpoint before aborting, thus the files on disk are not up to date with the end of WAL). Yes, it was done very quickly and submitted to ensure it could be considered yesterday for inclusion. It was described by me as rushed, which it certainly was because of personal time pressure yesterday: I thought that made it clear that discussion was needed. Heikki mentions to me it wasn't clear, so those criticisms are accepted. On Tue, 2006-09-19 at 16:05 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: + + if (startupAfterRecovery) + ereport(ERROR, + (errmsg(recovery ends normally with startup_after_recovery=false))); + I find this part of the patch a bit ugly. Me too. Overall, my own thoughts and Tom's and Heikki's comments indicate I should withdraw the patch rather than fix it. Patch withdrawn. Enclose a new doc patch to describe the capability, without s/w change. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com [ Attachment, skipping... ] ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Sane error messages for SSL retry cases
Bruce Momjian [EMAIL PROTECTED] writes: Is this a TODO? I don't see how it is a new problem, meaning it probably is for 8.3. It's definitely not a new problem, but we hadn't recognized it before, so it qualifies as a new bug. The question at hand was whether anyone was excited enough about it to try to push a fix in for 8.2. Seems like nobody is, so let's just put it on TODO: * Fix libpq SSL retry to avoid useless repeat connection attempts and ensuing misleading error messages regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Sane error messages for SSL retry cases
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is this a TODO? I don't see how it is a new problem, meaning it probably is for 8.3. It's definitely not a new problem, but we hadn't recognized it before, so it qualifies as a new bug. The question at hand was whether anyone was excited enough about it to try to push a fix in for 8.2. Seems like nobody is, so let's just put it on TODO: * Fix libpq SSL retry to avoid useless repeat connection attempts and ensuing misleading error messages Thanks, added. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Faster StrNCpy
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: You'll notice that it iterates once per char. Between that and the strlen() call in Tom's version, not sure which is the lesser evil. Yeah, I was wondering that too. My code would require two scans of the source string (one inside strlen and one in memcpy), but in much of our usage the source and dest should be reasonably well aligned and one could expect memcpy to be using word rather than byte operations, so you might possibly make it back on the strength of fewer write cycles. And on the third hand, for short source strings none of this matters and the extra function call involved for strlen/memcpy probably dominates. I'm happy to just use the OpenBSD version as a src/port module. Any objections? I found this URL about the function history of strlcpy(): http://www.gratisoft.us/todd/papers/strlcpy.html I added the URL to port/strlcpy.c. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Faster StrNCpy
Tom, FWIW, Tom Daly did some SpecJAppserver runs on the latest snapshot and didn't show any reduction in text parsing overhead. Unfortunately, he's gone on vacation now so I can't get details. I'm going to try to set up some tests using TPCE to see if it's affected. --Josh ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TM modificator don't work? to_char
Can someone confirm this features works or fails? --- Pavel Stehule wrote: Hello, I try 8.2 features. I tested to_char from doc, but without success. postgres=# select to_char(now(), 'TMDay, DD TMMonth '); to_char - Thursday, 28 September 2006 (1 row) I expected translated names :-(. What can be wrong. lc_collate | cs_CZ.UTF-8 | Shows the collation order locale. lc_ctype| cs_CZ.UTF-8 | Shows the character classification and case conversion locale. lc_messages | cs_CZ.UTF-8 | Sets the language in which messages are displayed. lc_monetary | cs_CZ.UTF-8 | Sets the locale for formatting monetary amounts. lc_numeric | cs_CZ.UTF-8 | Sets the locale for formatting numbers. lc_time | cs_CZ.UTF-8 Regards Pavel Stehule _ Chcete sdilet sve obrazky a hudbu s prateli? http://messenger.msn.cz/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Darwin stuff is getting deprecated
My guess is that we are going to have to live with the warnings for 8.2 and save this issue for 8.3. --- Tom Lane wrote: With the latest Apple developers' tools, I get some warnings that weren't there before: dynloader.c: In function 'pg_dlsym': dynloader.c:44: warning: 'NSIsSymbolNameDefined' is deprecated (declared at /usr/include/mach-o/dyld.h:150) dynloader.c:46: warning: 'NSLookupAndBindSymbol' is deprecated (declared at /usr/include/mach-o/dyld.h:158) postmaster.c: In function 'PostmasterMain': postmaster.c:779: warning: 'DNSServiceRegistrationCreate' is deprecated (declared at /usr/include/DNSServiceDiscovery/DNSServiceDiscovery.h:114) Anyone want to look into this and see what we're supposed to do instead? The postmaster.c message is in the USE_BONJOUR code. There was already some discussion about how we were using an obsolete Bonjour API and we ought to rewrite to support using Bonjour on non-Darwin platforms. Perhaps Apple is now using a compatible API? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TM modificator don't work? to_char
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Can someone confirm this features works or fails? It works if you have NLS translations for Thursday, September, etc ... which none of the backend .po files do yet. But they will when we have the po files updated for this release, I suppose. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TM modificator don't work? to_char
On Mon, 2 Oct 2006, Bruce Momjian wrote: Can someone confirm this features works or fails? As I see from the original commit: http://groups.google.com/group/pgsql.committers/browse_frm/thread/4ba64c906fca7211/0e3b4d495ec7ecf8?lnk=gstq=to_charrnum=8#0e3b4d495ec7ecf8 http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/adt/formatting.c.diff?r1=1.103r2=1.104 the patch for to_char wasn't written at all for looking into the locale settings... It uses gettext() to localize the output of to_char() which seems to be pretty useless behaviour... Regards, Sergey *** Sergey E. Koposov Max Planck Institute for Astronomy/Sternberg Astronomical Institute Tel: +49-6221-528-349 Web: http://lnfm1.sai.msu.ru/~math E-mail: [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TM modificator don't work? to_char
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: It works if you have NLS translations for Thursday, September, etc ... which none of the backend .po files do yet. But they will when we have the po files updated for this release, I suppose. We should probably list translation updates as one of the open items for the release. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TM modificator don't work? to_char
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: It works if you have NLS translations for Thursday, September, etc ... which none of the backend .po files do yet. But they will when we have the po files updated for this release, I suppose. We should probably list translation updates as one of the open items for the release. Added. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Patch: Tie stats options to autovacuum in
On Oct 2, 2006, at 8:41 PM, Bruce Momjian wrote: David E. Wheeler wrote: On Sep 28, 2006, at 16:39, Jim C. Nasby wrote: +1. I was just at a client today that had run into this problem. Actually, I'm in favor of refusing to start if autovac is on but the proper stats settings aren't. I'd rather that then people ending up with bloated databases and crappy performance. I agree, but I figured that this was a start, at least. The problem with refusing to start is that autovacuum is sighup, so you might modify postgresql.conf and do a server restart, and then the server doesn't start. Are people OK with that? I did apply the postgresql.conf comment addition. Hrm... how about if the options are incompatible on HUP we refuse to pick up any new settings and complain loudly? Perhaps it would be easier to just override stats_row_level if autovac is on. Doesn't necessarily meet the least surprise test, but it does protect newbies which I feel is more important in this case. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Patch: Tie stats options to autovacuum in
Jim Nasby wrote: On Oct 2, 2006, at 8:41 PM, Bruce Momjian wrote: David E. Wheeler wrote: On Sep 28, 2006, at 16:39, Jim C. Nasby wrote: +1. I was just at a client today that had run into this problem. Actually, I'm in favor of refusing to start if autovac is on but the proper stats settings aren't. I'd rather that then people ending up with bloated databases and crappy performance. I agree, but I figured that this was a start, at least. The problem with refusing to start is that autovacuum is sighup, so you might modify postgresql.conf and do a server restart, and then the server doesn't start. Are people OK with that? I did apply the postgresql.conf comment addition. Hrm... how about if the options are incompatible on HUP we refuse to pick up any new settings and complain loudly? We don't read postgresql.conf as a test and then set values. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Bad bug in fopen() wrapper code
Magnus Hagander wrote:
Now, I still twist my head around the lines:
if ((fd = _open_osfhandle((long) h, fileFlags O_APPEND)) 0
||
(fileFlags (O_TEXT | O_BINARY) (_setmode(fd, fileFlags
(O_TEXT
| O_BINARY)) 0)))
Without having studied it closely, it might also highlight a bug
on
failure of the second clause -- if the _setmode fails, shouldn't
_close be called instead of CloseHandle, and -1 returned?
(CloseHandle would still be called on failure of the
_open_osfhandle,
obviously)
I agree that this code is both wrong and unreadable (although in
practice the _setmode will probably never fail, which is why our
attention hasn't been drawn to it). Is someone going to submit a
patch? I'm hesitant to change the code myself since I'm not in a
position to test it.
I can look at fixing that. Is it something we want to do for 8.2, or
wait until 8.3? If there's a hidden bug, I guess 8.2?
Magnus, is this the right fix?
--
Bruce Momjian [EMAIL PROTECTED]
EnterpriseDBhttp://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: src/port/open.c
===
RCS file: /cvsroot/pgsql/src/port/open.c,v
retrieving revision 1.15
diff -c -c -r1.15 open.c
*** src/port/open.c 24 Sep 2006 17:19:53 - 1.15
--- src/port/open.c 3 Oct 2006 01:16:43 -
***
*** 105,113
}
/* _open_osfhandle will, on error, set errno accordingly */
! if ((fd = _open_osfhandle((long) h, fileFlags O_APPEND)) 0 ||
! (fileFlags (O_TEXT | O_BINARY) (_setmode(fd, fileFlags (O_TEXT | O_BINARY)) 0)))
CloseHandle(h); /* will not affect errno */
return fd;
}
--- 105,119
}
/* _open_osfhandle will, on error, set errno accordingly */
! if ((fd = _open_osfhandle((long) h, fileFlags O_APPEND)) 0)
CloseHandle(h); /* will not affect errno */
+ else if (fileFlags (O_TEXT | O_BINARY)
+ _setmode(fd, fileFlags (O_TEXT | O_BINARY)) 0)
+ {
+ _close(fd)/* will not affect errno */
+ return -1;
+ }
+
return fd;
}
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] NULL and plpgsql rows
On Oct 2, 2006, at 6:28 PM, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: However, the test right above that means that we'll fail if the user tries something like row_variable := NULL;: The patch you seem to have in mind would allow row_variable := int_variable; to succeed if the int_variable chanced to contain NULL, which is surely not very desirable. Hrm... is there any reasonable way to catch that? The real issue here is that the bare NULL has type UNKNOWN and we're not making any effort to cast it. I'm not sure whether it'd work to simply apply exec_cast_value --- that looks like it's only meant to handle scalars, where in general you'd need something close to ExecEvalConvertRowtype(). Of course, setting a row variable to null is a lot more useful if we can actually test for it after the fact, and I'm not really sure how to make that happen. Doesn't IS NULL work (as of CVS HEAD)? Ahh, so it does. Doesn't work with RECORD, though... which seems a bit surprising. I can't really think of a good reason why they should differ. ERROR: record v_row is not assigned yet DETAIL: The tuple structure of a not-yet-assigned record is indeterminate. CONTEXT: PL/pgSQL function test line 4 at return -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] timestamptz alias
On Oct 2, 2006, at 6:22 PM, AgentM wrote: On Oct 2, 2006, at 18:15 , Markus Schaber wrote: I'm happy that the rather verbose timestamp with time zone has the much nicer alias timestamptz, however it seems that this alias is not documented, neither at http://developer.postgresql.org/pgdocs/postgres/datatype- datetime.html nor at http://www.postgresql.org/docs/8.1/interactive/datatype- datetime.html I see it mentioned at http://www.postgresql.org/docs/8.1/interactive/datatype.html but that's possibly not where people look at first, when they search for the timestamp type. (At least I found it only when grepping for timestamptz in the docs. :-) Should the alias be mentioned on the datetime page? The same for timetz? What do you think? I am pleased that the documentation promotes database-independent (standard) SQL. There's a difference between promoting and withholding info. I'd rather see us explicitly state which is preferred and why. BTW, another confusing example is all the string functions that are essentially the same, such as substring and substr. (http:// www.postgresql.org/docs/8.1/interactive/functions-string.html) -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Patch: Tie stats options to autovacuum in
On Oct 2, 2006, at 9:17 PM, Bruce Momjian wrote: Jim Nasby wrote: On Oct 2, 2006, at 8:41 PM, Bruce Momjian wrote: David E. Wheeler wrote: On Sep 28, 2006, at 16:39, Jim C. Nasby wrote: +1. I was just at a client today that had run into this problem. Actually, I'm in favor of refusing to start if autovac is on but the proper stats settings aren't. I'd rather that then people ending up with bloated databases and crappy performance. I agree, but I figured that this was a start, at least. The problem with refusing to start is that autovacuum is sighup, so you might modify postgresql.conf and do a server restart, and then the server doesn't start. Are people OK with that? I did apply the postgresql.conf comment addition. Hrm... how about if the options are incompatible on HUP we refuse to pick up any new settings and complain loudly? We don't read postgresql.conf as a test and then set values. IMHO we should... if something got botched in the config file I'd rather have it complain and not change anything instead of taking just some of the changes. This will be even more important once the code goes in to return to default values if something gets commented out in the config. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] src/tools/msvc usage instructions
Looks like the gendef script is failing. Check the contents of release\postgres\postgres.def - it should have thousands of symbols, but I'm willing to bet it's empty... //Magnus -Original Message- From: Jeremy Drake [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 03, 2006 1:28 AM To: Magnus Hagander Cc: PostgreSQL Hackers Subject: Re: [HACKERS] src/tools/msvc usage instructions I now get things to compile, but now I get linker errors on any dll which needs to access symbols from postgres.exe via postgres.lib. For example: 1-- Build started: Project: autoinc, Configuration: Release Win32 1-- Generate DEF file Not re-generating AUTOINC.DEF, file already 1exists. 1Linking... 1 Creating library Release\autoinc\autoinc.lib and object 1Release\autoinc\autoinc.exp autoinc.obj : error LNK2019: unresolved 1external symbol _SPI_modifytuple referenced in function _autoinc 1autoinc.obj : error LNK2019: unresolved external symbol _pfree 1referenced in function _autoinc autoinc.obj : error LNK2019: 1unresolved external symbol _pg_detoast_datum referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1_nextval referenced in function _autoinc autoinc.obj : error LNK2019: 1unresolved external symbol _DirectFunctionCall1 referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1_textin referenced in function _autoinc autoinc.obj : error LNK2019: 1unresolved external symbol _SPI_getbinval referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1_SPI_gettypeid referenced in function _autoinc autoinc.obj : error 1LNK2019: unresolved external symbol _errfinish referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1___msvc_errcode referenced in function _autoinc autoinc.obj : error 1LNK2019: unresolved external symbol _errmsg referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1_errstart referenced in function _autoinc autoinc.obj : error LNK2019: 1unresolved external symbol _SPI_fnumber referenced in function 1_autoinc autoinc.obj : error LNK2019: unresolved external symbol 1_MemoryContextAlloc referenced in function _autoinc autoinc.obj : 1error LNK2019: unresolved external symbol _SPI_getrelname referenced 1in function _autoinc autoinc.obj : error LNK2019: unresolved external 1symbol _elog_finish referenced in function _autoinc autoinc.obj : 1error LNK2019: unresolved external symbol _elog_start referenced in 1function _autoinc .\Release\autoinc\autoinc.dll : fatal error LNK1120: 117 unresolved externals I checked the project properties for linker options and it does list Release\postgres\postgres.lib in the additional dependencies list. Any ideas? Am I missing something? -- A penny saved is ridiculous. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] src/tools/msvc usage instructions
Jeremy Drake [EMAIL PROTECTED] writes:
The errors I got on this file were:
1bootparse.tab.c(1065) : error C2449: found '{' at file scope
1(missing function header?)
I looked at this. Line 1065 is the left brace starting
yyparse(). On
my Fedora Core 5 box with Bison 2.1 installed, the stuff
leading up to
it is
#ifdef YYPARSE_PARAM
... some uninteresting stuff, since we don't define
YYPARSE_PARAM ...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus) int yyparse (void)
#else int yyparse ()
#endif
#endif
{
But lookee here, your Windows-built version has
#ifdef YYPARSE_PARAM
...
#else /* ! YYPARSE_PARAM */
#if defined (__STDC__) || defined (__cplusplus) int yyparse (void)
#else int yyparse ()
;
#endif
#endif
{
So that semicolon is the source of the trouble. That's clearly a
bison bug, and in fact digging in Red Hat's SRPM shows that
they are
carrying a patch for it:
2005-10-05 Paul Eggert [EMAIL PROTECTED]
* data/m4sugar/m4sugar.m4 (_m4_map): New macro.
(m4_map, m4_map_sep): Use it. Handle the empty list correctly.
snip patch
Presumably bison 2.2 includes this fix. But I guess the
real question
is why the devil doesn't MSVC define __STDC__ ? Are they that far
removed from spec compliance?
In the bison 2.2 generated code, the #if check is
#if (defined __STDC__ || defined __C99__FUNC__ \
|| defined __cplusplus || defined _MSC_VER)
which looks like they figured out that they needed to check
for MicroSoft C explicitly. I have no idea why they do not
define __STDC__ however.
Can we just define __STDC__ when compiling that file (or rather, when
compiling any bison-generated output file)? Or is that likely to cause
*other* issues?
//Magnus
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
