Re: [HACKERS] SQL99 Hierarchical queries
Ok, I'm started porting it to 8.0.1 and will fix this also. By the way, did you know any test suit for such queries? To make some regression test. Hello, I can find some examples on internet and prepare regression tests. I think PostgreSQL can support all syntax H.Q. Is more easy created question via Oracle syntax, and processing is faster (maybe better optimalisation now), than ANSI WITH syntax. Can You add support for clausule VALUE? Pavel Stehule On this weekend I'll see to DB2 reference and if there some description on what is it, I'll try to implement it. Regards, Evgen. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] SQL99 Hierarchical queries
Thanks very much for doing this work. Is there some way you can summarize what you did so others can join you in working on it? If it is easier for you to write this in some language other than English, please do, and we'll find translators :) I noticed that the patch touches the parser, the optimizer and the executor. What does it to with each? What did I miss? Cheers, D Ok, i'll make some doc on this topic. Regards, Evgen ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] bitmap AM design
I don't think we really need any more fundamentally nonconcurrent index types :-( Tom, I posted a message about a week ago (I forget the name) about a persistent reference index, sort of like CTID, but basically a table lookup. The idea is to simulate a structure that ISAM sort of techniques can work in PostgreSQL. Victor had emailed me and basically said he needed a similar sort of thing for this bitmap index. Eliminating the bitmap index issue for a moment, how hard would it be to create a reference table like index? I'm sure given that construct, the bitmap index becomes easier to construct. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef :( Regards, Nick ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] mysterious log output
Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: I keep seeing the following log output around every 5 minutes: 2005-02-28 23:25:05 [8646] LOG: 0: QUERY STATISTICS This has to be coming from exec_simple_query(): if (save_log_statement_stats) ShowUsage(QUERY STATISTICS); so *something* is turning on log_statement_stats. I wonder if some statistics were turned on at postmaster start and even though it was turned off and 'pg_ctl reload' was done the checkpoint/bgwriter process isn't reloading for some reason. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests
Nicolai Tufar wrote: On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef Sure, in src/backend/utils/adt/int8.c, there is a call in int8out(): if ((len = snprintf(buf, MAXINT8LEN, INT64_FORMAT, val)) 0) and that calls port/snprintf.c. I have added a puts() in snprintf.c to make sure it is getting the long/long specifier. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] index file detached from data file
I have a server running 7.4 that has an index file that's kind of floating in limbo, not attached to anything. When I try to drop it, I get the following: ERROR: could not open relation with OID 2293160133 The OID of the table that it should be attached to is 2300667425, so it might still think it's attached to an old copy of that table -- that table has been dropped and subsequently restored from a backup, which is how I found out there was a problem. The backup was trying to create an index with the same name that the limbo index had, which caused an error. The index still shows up in pg_class and its file still exists, it's just not attached to anything. My most immediate concern is how can I safely clean up this entry for the index? Also, is it really possible to drop a table while it's being indexed or otherwise used to cause a situation like this? Or am I misreading what probably happened? -- Jeff Hoffmann [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] mysterious log output
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: This has to be coming from exec_simple_query(): I wonder if some statistics were turned on at postmaster start and even though it was turned off and 'pg_ctl reload' was done the checkpoint/bgwriter process isn't reloading for some reason. The checkpoint/bgwriter never executes exec_simple_query(). I think the 5-minute cycle is driven by something in Robert's application, rather than being tied to checkpoints. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] bitmap AM design
* Tom Lane [EMAIL PROTECTED] [01.03.2005 09:37]: The other stuff you mentioned implies that an insertion therefore requires exclusive lock on the index (because you may have to relocate everything in sight to add one more CTID slot). No, exclusive lock on index is worst thing to do. All lists (list of ctids, bitmaps) will only grow, no data will be deleted, as deletes will require relocation and possibly exclusive lock on the index. Extending lists will need only a short-term exclusive locks on the pages in the tails of each list. I can't believe you are seriously suggesting that it's OK to force every VACUUM to rebuild the index from scratch. We already get far too many complaints about the time needed for VACUUM. I don't think we really need any more fundamentally nonconcurrent index types :-( Well, I misunderstood the purpose of ambulkdelete function, my fault. Of course, no index rebuild will take place, instead, only flags in the list of CTIDs will be updated for deleted tuples. I have counter question for you: you've mentioned, that you want bitmaps in the 8.1. What kind of bitmaps you were speaking about? On-disk bitmaps (this is how I call new index access method I'm working on) or in-memory bitmaps, as in here http://archives.postgresql.org/pgsql-hackers/2005-01/msg01001.php Thanks for your reply. -- Victor Y. Yegorov ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch for Postmaster
Hi *, I have submitted a patch which does exactly this. Somebody edited it and Bruce put it on the queue for 8.1. Now we both did the same work twice. This is a good example how a public accessible Bugtracker and TODO-List where one can claim things (or get assigned to some workitem) can save valuable time. Image someone likes to get in touch with the codebase by fixing some bugs first. To my understanding, there is no Bug Database for Postgres (correct me, if I am wrong). Where can this volunteer lookup Bugs? I don't want to restart the Bugzilla Discussion, since I personally don't have any preference which Trouble-Ticket System to use. But I think the idea of having one is worth considering. cheers, Matthias Am 25.02.2005 um 03:51 schrieb Bruce Momjian: Eric Crampton wrote: On Feb 24, 2005, at 5:38 PM, Tom Lane wrote: Eric Crampton [EMAIL PROTECTED] writes: I noticed in the TODO that there is an item requesting a function which returns the uptime of the postmaster. I've wanted that as well. So, I've added just such a thing: server_start_time(). This function returns the time when the postmaster was started; a simple now() - server_start_time() returns the uptime. Isn't there one of these in the patch queue already? There could be, but I didn't see one because perhaps I'm looking in the wrong place. I referenced the TODO list here: http://developer.postgresql.org/todo.php and I referenced the unapplied patches here: http://momjian.postgresql.org/cgi-bin/pgpatches Where should I be looking? Sure: http://momjian.postgresql.org/cgi-bin/pgpatches2 -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org -- Matthias Schmidt Viehtriftstr. 49 67346 Speyer GERMANY Tel.: +49 6232 4867 Fax.: +49 6232 640089 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] bitmap AM design
[EMAIL PROTECTED] writes: Tom, I posted a message about a week ago (I forget the name) about a persistent reference index, sort of like CTID, but basically a table lookup. The idea is to simulate a structure that ISAM sort of techniques can work in PostgreSQL. Eliminating the bitmap index issue for a moment, how hard would it be to create a reference table like index? I didn't see the point. You cannot actually use CTID that way (at least not without fundamentally redesigning our MVCC machinery) and anything else you might come up with is effectively just a random unique ID that has to be mapped through an index to find the row. I don't see the advantage compared to any ordinary application-defined primary key. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] bitmap AM design
[EMAIL PROTECTED] writes: Tom, I posted a message about a week ago (I forget the name) about a persistent reference index, sort of like CTID, but basically a table lookup. The idea is to simulate a structure that ISAM sort of techniques can work in PostgreSQL. Eliminating the bitmap index issue for a moment, how hard would it be to create a reference table like index? I didn't see the point. You cannot actually use CTID that way (at least not without fundamentally redesigning our MVCC machinery) and anything else you might come up with is effectively just a random unique ID that has to be mapped through an index to find the row. I don't see the advantage compared to any ordinary application-defined primary key. I have a search engine product which does use primary key based number. The search engine also uses an external bitmap index. Here is the process: Parse incoming text into discrete words. Look up each word and retrieve its bitmap. Combine all the bitmaps using the appropriate logical functions (AND, OR, etc) list out all the 1s from the bitmaps as an entry into a table which points to the primary key number. Find all the records in the database with all the primary keys, sometimes hundreds or thousands of entries in a WHERE IN (...) clause. Now, after I've done all this logical work getting document numbers, I have to do an index lookup for each one (or, god forbid, a full table scan!) This can be a long process, longer than actually doing the text search with the bitmaps in the first place. A persistent reference index would be like almost any other index except that as new items are added to a table a new entry is added to the end of the index. When a row is updated, its CTID is updated in the table. When you run vacuum, you just update the CTID in the table as if it were any other index. When you delete an item, you clear the CDID value of the table entry. You can do VACUUM COMPRESS INDEX myindex which will re-order and compact the persistent reference. I know from a purely SQL standpoint, it sounds whacky, but from a VAR or consultants point of view, it can really increase performance of some products. That last WHERE IN clause of my search engine can take several tens of seconds to run, but the actual search engine only took 0.03 seconds to find the documents. A persistent reference system would eliminate tens ro thousands of index lookups per search query. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Execute and PortalSuspended needs explicit transaction
--- Oliver Jowett [EMAIL PROTECTED] escreveu: Francisco Figueiredo Jr. wrote: After some testing, I could send an Execute message with 2 as the manx number of rows. After the second execute I get the following: portal does not exist Severity: ERROR Code: 34000 I noticed that I could only get it working if I explicitly create a transaction. I thought it could be some Sync() messages I was sending after the first execute, but when I removed them, I still get the problems. If you're sending any Sync messages at all between the two Executes, it will indeed cause problems as Sync causes any implicitly-opened transaction to be closed, which will in turn invalidate any non-holdable portals. Ok. The strange part to me is that even after removing the sync calls, I still got the problem :( Do you have a trace of all the messages sent? I'm not at my devel box right now, but when I get home I will send you. Thanks in advance, Oliver. Regards, Francisco Figueiredo Jr. ___ Yahoo! Acesso Grátis - Instale o discador do Yahoo! agora. http://br.acesso.yahoo.com/ - Internet rápida e grátis ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch for Postmaster
Have you read the developer's FAQ. It is a requirement to get involved. --- Matthias Schmidt wrote: Hi *, I have submitted a patch which does exactly this. Somebody edited it and Bruce put it on the queue for 8.1. Now we both did the same work twice. This is a good example how a public accessible Bugtracker and TODO-List where one can claim things (or get assigned to some workitem) can save valuable time. Image someone likes to get in touch with the codebase by fixing some bugs first. To my understanding, there is no Bug Database for Postgres (correct me, if I am wrong). Where can this volunteer lookup Bugs? I don't want to restart the Bugzilla Discussion, since I personally don't have any preference which Trouble-Ticket System to use. But I think the idea of having one is worth considering. cheers, Matthias Am 25.02.2005 um 03:51 schrieb Bruce Momjian: Eric Crampton wrote: On Feb 24, 2005, at 5:38 PM, Tom Lane wrote: Eric Crampton [EMAIL PROTECTED] writes: I noticed in the TODO that there is an item requesting a function which returns the uptime of the postmaster. I've wanted that as well. So, I've added just such a thing: server_start_time(). This function returns the time when the postmaster was started; a simple now() - server_start_time() returns the uptime. Isn't there one of these in the patch queue already? There could be, but I didn't see one because perhaps I'm looking in the wrong place. I referenced the TODO list here: http://developer.postgresql.org/todo.php and I referenced the unapplied patches here: http://momjian.postgresql.org/cgi-bin/pgpatches Where should I be looking? Sure: http://momjian.postgresql.org/cgi-bin/pgpatches2 -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org -- Matthias Schmidt Viehtriftstr. 49 67346 Speyer GERMANY Tel.: +49 6232 4867 Fax.: +49 6232 640089 -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] bitmap AM design
[EMAIL PROTECTED] writes: A persistent reference index would be like almost any other index except that as new items are added to a table a new entry is added to the end of the index. When a row is updated, its CTID is updated in the table. This *does not work* under MVCC: it can't cope with referencing multiple simultaneously existing versions of a row. In general, any index design that includes the words update in place can be rejected out of hand. In any case I still fail to see the advantage compared to an ordinary serial primary key. You could have made your bitmaps reference the serial numbers directly, instead of an intermediate table. (Either way still fails to handle MVCC updates, unless the indexable attributes cannot be changed by updates; but the intermediate table isn't helping or hurting that.) A bitmap that indexes CTIDs directly could work, because it doesn't need to update any entries in-place when a table row is updated. I didn't care for the details of Victor's design because (a) the intermediate list of CTIDs looks bulky and (b) it seemed to require a lot of data shuffling to cope with growth of the table. But in principle it would work. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] bitmap AM design
Victor Y. Yegorov [EMAIL PROTECTED] writes: All lists (list of ctids, bitmaps) will only grow, no data will be deleted, as deletes will require relocation and possibly exclusive lock on the index. Extending lists will need only a short-term exclusive locks on the pages in the tails of each list. Hmm, you seem to be envisioning that these are actually lists, not arrays --- that is, to find the N'th page in a list requires traversing list links starting at the first page. That doesn't sound workable. If you can't access all the entries in roughly constant time then the thing is going to have problems with big tables. I have counter question for you: you've mentioned, that you want bitmaps in the 8.1. What kind of bitmaps you were speaking about? In-memory is what I intend to work on. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] bitmap AM design
* Tom Lane [EMAIL PROTECTED] [01.03.2005 21:07]: Hmm, you seem to be envisioning that these are actually lists, not arrays --- that is, to find the N'th page in a list requires traversing list links starting at the first page. That doesn't sound workable. If you can't access all the entries in roughly constant time then the thing is going to have problems with big tables. Bitmaps are arays, you're right. But they are only accessed either when data is inserted and gets added to the end (there're direct pointers to the last page in each bitmap in the list of values), or during index scan. Scanning index implies visiting all pages that forms the bitmap. After scanning the bitmaps (and performing all logical operations as requested), we end up with bit positions and need to return CTID from the list, that resides in the given position in the list-of-CTIDs (yes, it's actually one huge array, that occupies many pages). List-of-CTIDs is organized into extents, each extent contains a known number of pages and all pages for the extent are allocated sequentially. ID of the first page of each extent is stored in the metapage. Also, it is known at compile time how many CTIDs can be stored into one page. Given all that, it is possible to compute ID of the page and CTID offset inside that page by CTID offset, obtained at bitmap scan step. Each extent has 2,4,8,16,32,etc. pages, so the metapage has enough space to store an array of ID's for the first page of each extent. Updating list-of-CTIDs is also cheap operation, as direct reference to the last page in the list-of-CTIDs is stored in metapage. The only list, that have drawbacks you've mentioned, is list-of-values. But, as it contains only attributes' related data and pointers to start page of corresponding bitmap, there won't be many pages in this list. Maybe, there are some more insufficiencies I've missed? -- Victor Y. Yegorov ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] bitmap AM design
OK, lets step back a bit and see if there is a solution that fits what we think we need and PostgreSQL. Lets talk about FTSS, its something I can discuss easily. It is a two stage system with an indexer and a server. Only the data to be indexed is in the database, all the FTSS data structures are in external files. The indexer creates a number of data structures. A table of document references, one entry per document. A table of words parsed, one word per entry A table of compressed bitmaps, one (or more) bitmap(s) per word. The relation of bits in the word bitmaps is one bit per document as ordered by the document table, i.e. if bit 5 is set high, then the fith document is selected. (Let's not discuss phrase analysis at this point.) When the indexer runs, it executes a query that produces a set of results. Each result has a document reference which is stored in the FTSS document table. The results are parsed out as discrete words, new words are added to the word table, previously used word's reference counts are incremented. A bitmap is created for each new word. The bit of the current document is set to 1. This procedure runs for each record in the query. The server runs as follows: accepts an HTTP request for search Parses out the discrete words. The word is found in the word table. The word's bitmap is retrieved from the bitmap table. A series of logical functions are performed on the retrieved bitmaps. The resulting bitmap contains all the relevant documents in the form of bits correlating to offsets into the document reference table. The list of document references is returned to the database and found using a WHERE IN clause. Now, it occurs to me that if my document reference table can refer to something other than an indexed primary key, I can save a lot of index processing time in PostgreSQL if I can have a safe analogy to CTID. I should be able to know more about a particular row (document) being referenced, because I have already been through the table once. I need to be able to know which rows are newer than my FTSS index so I can search those rows more dynamically. I currently do this by saving the highest value during indexing. [EMAIL PROTECTED] writes: A persistent reference index would be like almost any other index except that as new items are added to a table a new entry is added to the end of the index. When a row is updated, its CTID is updated in the table. This *does not work* under MVCC: it can't cope with referencing multiple simultaneously existing versions of a row. In general, any index design that includes the words update in place can be rejected out of hand. In any case I still fail to see the advantage compared to an ordinary serial primary key. You could have made your bitmaps reference the serial numbers directly, instead of an intermediate table. (Either way still fails to handle MVCC updates, unless the indexable attributes cannot be changed by updates; but the intermediate table isn't helping or hurting that.) A bitmap that indexes CTIDs directly could work, because it doesn't need to update any entries in-place when a table row is updated. I didn't care for the details of Victor's design because (a) the intermediate list of CTIDs looks bulky and (b) it seemed to require a lot of data shuffling to cope with growth of the table. But in principle it would work. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
I spent all day debugging it. Still have absolutely no idea what could possibly go wrong. Does anyone have a slightest clue what can it be and why it manifests itself only on win32? On Tue, 1 Mar 2005 09:29:07 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Nicolai Tufar wrote: On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef Sure, in src/backend/utils/adt/int8.c, there is a call in int8out(): if ((len = snprintf(buf, MAXINT8LEN, INT64_FORMAT, val)) 0) and that calls port/snprintf.c. I have added a puts() in snprintf.c to make sure it is getting the long/long specifier. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression
Nicolai Tufar wrote: On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef Sure, in src/backend/utils/adt/int8.c, there is a call in int8out(): if ((len = snprintf(buf, MAXINT8LEN, INT64_FORMAT, val)) 0) and that calls port/snprintf.c. I have added a puts() in snprintf.c to make sure it is getting the long/long specifier. Just a question, and don't mind me if I am being rude, isn't this the WRONG PLACE for a printf function? Wouldn't an itoa function be more efficient and be less problematic? ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests
[EMAIL PROTECTED] writes: Just a question, and don't mind me if I am being rude, isn't this the WRONG PLACE for a printf function? Wouldn't an itoa function be more efficient and be less problematic? This particular call could be so replaced, but it wouldn't solve the general problem. snprintf has to work. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression
I spent all day debugging it. Still have absolutely no idea what could possibly go wrong. Does anyone have a slightest clue what can it be and why it manifests itself only on win32? It may be that the CLIB has badly broken support for 64bit integers on 32 bit platforms. Does anyone know of any Cygwin/Ming issues? Is this only with the new snprintf code in Win32? Is this a problem with snprintf as implemented in src/port? Is there a reason why we don't use the snprintf that comes with the various C compilers? On Tue, 1 Mar 2005 09:29:07 -0500 (EST), Bruce Momjian pgman@candle.pha.pa.us wrote: Nicolai Tufar wrote: On Tue, 1 Mar 2005 00:55:20 -0500 (EST), Bruce Momjian My next guess is that Win32 isn't handling va_arg(..., long long int) properly. I am trying various combination of number and types of parameters in my test program and everything prints fine. When it comes to pg, it fails :( template1=# select * from test where x 1000::int8; x -869367531 (1 row) I am not too fluent in source code, could someone point me to there actual call to snprintf() is being done when a query like this is executed. I could not find it myslef Sure, in src/backend/utils/adt/int8.c, there is a call in int8out(): if ((len = snprintf(buf, MAXINT8LEN, INT64_FORMAT, val)) 0) and that calls port/snprintf.c. I have added a puts() in snprintf.c to make sure it is getting the long/long specifier. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
I spent all day debugging it. Still have absolutely no idea what could possibly go wrong. Does anyone have a slightest clue what can it be and why it manifests itself only on win32? It may be that the CLIB has badly broken support for 64bit integers on 32 bit platforms. Does anyone know of any Cygwin/Ming issues? Is this only with the new snprintf code in Win32? Yes. Is this a problem with snprintf as implemented in src/port? Yes. Only. It works with the snprintf() in the runtime (this particular part). Is there a reason why we don't use the snprintf that comes with the various C compilers? It does not support positional parameters (I think it's called) which is required for proper translations. We do use that one when it works... //Magnus ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
On Tue, 1 Mar 2005 15:38:58 -0500 (EST), [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Is there a reason why we don't use the snprintf that comes with the various C compilers? snprintf() is usually buried in OS libraries. We implement our own snprintf to make things like this: snprintf(buf,%2$s %1$s,world,Hello); which is not supported on some platforms work. We do it for national language translation of messages. In some languages you may need to change order of parameters to make a meaningful sentence. Another question is why we are using it for printing values from database. I am not too good on function overriding in standard C but maybe there is a way to usage of standard snprintf() in a particular place. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression
On Tue, 1 Mar 2005 15:38:58 -0500 (EST), [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Is there a reason why we don't use the snprintf that comes with the various C compilers? snprintf() is usually buried in OS libraries. We implement our own snprintf to make things like this: snprintf(buf,%2$s %1$s,world,Hello); which is not supported on some platforms work. We do it for national language translation of messages. In some languages you may need to change order of parameters to make a meaningful sentence. Another question is why we are using it for printing values from database. I am not too good on function overriding in standard C but maybe there is a way to usage of standard snprintf() in a particular place. Well, here is a stupid question, do we even know which snprintf we are using on Win32? May it be possible that we are using the MingW version which may be broken? ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression tests to fail
[EMAIL PROTECTED] writes: Well, here is a stupid question, do we even know which snprintf we are using on Win32? May it be possible that we are using the MingW version which may be broken? The regression tests were not failing until we started using the port/ version ... regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 8.0.X and the ARC patent
On Sat, Feb 19, 2005 at 12:57:52AM -0500, Tom Lane wrote: So far I've not been able to measure any consistent difference, but you know how much I trust pgbench ;-). I hope that Mark Wong can give us some results on the OSDL setup soon. Sorry for the delay, broken laptop, vacation, etc. Here's a baseline against 8.0.1: http://www.osdl.org/projects/dbt2dev/results/dev4-010/309/ throughput 3639.97 Against CVS from 20050217 and the 2Q patch: http://www.osdl.org/projects/dbt2dev/results/dev4-010/311/ throughput 3875.84 About a 6% increase, but if you look at the performance over time, it's degrading steadily. The latter throughput peaks near 5000. Mark ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 8.0.X and the ARC patent
Mark Wong [EMAIL PROTECTED] writes: About a 6% increase, but if you look at the performance over time, it's degrading steadily. The latter throughput peaks near 5000. Curious. The immediate question is does it ever flatten out, and if so at what TPM rate compared to 8.0.1? Could you run the same test for a longer duration? Also, I suppose the large spike at 30 minutes is from a checkpoint? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] [HACKERS] WAL: O_DIRECT and multipage-writer (+ memory leak)
On Thu, Feb 03, 2005 at 07:25:55PM +0900, ITAGAKI Takahiro wrote: Hello everyone. I fixed two bugs in the patch that I sent before. Check and test new one, please. Ok, finally got back into the office and was able to run 1 set of tests. So the new baseline result with 8.0.1: http://www.osdl.org/projects/dbt2dev/results/dev4-010/309/ Throughput: 3639.97 Results with the patch but open_direct not set: http://www.osdl.org/projects/dbt2dev/results/dev4-010/308/ Throughput: 3494.72 Results with the patch and open_direct set: http://www.osdl.org/projects/dbt2dev/results/dev4-010/312/ Throughput: 3489.69 You can verify that the wall_sync_method is set to open_direct under the database parameters link, but I'm wondering if I missed something. It looks a little odd the the performance dropped. Mark ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] 8.0.X and the ARC patent
On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: About a 6% increase, but if you look at the performance over time, it's degrading steadily. The latter throughput peaks near 5000. Curious. The immediate question is does it ever flatten out, and if so at what TPM rate compared to 8.0.1? Could you run the same test for a longer duration? The comparison was against 8.0.1, or did you mean 8.0.1 with the 2Q patch? I can run a longer duration and see how it looks. Also, I suppose the large spike at 30 minutes is from a checkpoint? Yes, checkpoint_timeout is set to 1800 seconds. Mark ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.0.X and the ARC patent
Mark Wong [EMAIL PROTECTED] writes: On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: Curious. The immediate question is does it ever flatten out, and if so at what TPM rate compared to 8.0.1? Could you run the same test for a longer duration? The comparison was against 8.0.1, or did you mean 8.0.1 with the 2Q patch? I can run a longer duration and see how it looks. My point was that unpatched 8.0.1 seems to have a pretty level TPM rate. If the patched version levels out at something not far below that, I'll be satisfied. If it continues to degrade then I won't be satisfied ... but the test stops short of telling what will happen. If you could run it for 2 hours then we'd probably know enough. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] logging as inserts
Hello, I am looking at having one of our guys write up the code to allow logging as insert statements. I have a couple of questions. What would we like the postgresql.conf option to be? I was thinking log_statements_as_inserts = (t/f) However I thought that was getting a little silly. Also do we want to allow logging as inserts for all options? There is a lot of different types of logging we can do. Sincerely, Joshua D. Drake -- Command Prompt, Inc., Your PostgreSQL solutions company. 503-667-4564 Custom programming, 24x7 support, managed services, and hosting Open Source Authors: plPHP, pgManage, Co-Authors: plPerlNG Reliable replication, Mammoth Replicator - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] snprintf causes regression
On Tue, 1 Mar 2005 16:20:50 -0500 (EST), [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Well, here is a stupid question, do we even know which snprintf we are using on Win32? May it be possible that we are using the MingW version which may be broken? Defenitely not. I checked it by placing debugging print statements in code. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [pgsql-hackers-win32] snprintf causes regression tests to fail
Nicolai Tufar [EMAIL PROTECTED] writes: Amazingly enough HAVE_LONG_LONG_INT_64 is defined when compilation comes to src/port/snprintf.c but the result is still wrong. I looked into configure.in but the check for HAVE_LONG_LONG_INT_64 is too complicated for me to understand. Bruce, could you take a look at this? I am 90% sure it is an issue with some configure definitions. Just out of curiosity, do either HAVE_INT64 or HAVE_UINT64 get set in pg_config.h? The observed symptoms would be explained if typedef int64 were ending up as long rather than long long. Looking at the #ifdef nest in include/c.h, there are a couple of ways that could happen, including importing a definition from system header files. If this were happening, it would presumably break all int8 math not only snprintf, so I'm not sure it's the story. As far as I've seen, no one has actually posted the regression diffs seen in this failure, so most of us are in the dark about the details of the problem. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [pgsql-hackers-win32] snprintf causes regression tests to fail
On Tue, 01 Mar 2005 17:45:31 -0500, Tom Lane Just out of curiosity, do either HAVE_INT64 or HAVE_UINT64 get set in pg_config.h? pg_config.h is attached. What drew my attention is the following declaration: /* Define to 1 if `long long int' works and is 64 bits. */ #define HAVE_LONG_LONG_INT_64 is it normal? should it not be like this: #define HAVE_LONG_LONG_INT_64 1 /* src/include/pg_config.h. Generated by configure. */ /* src/include/pg_config.h.in. Generated from configure.in by autoheader. */ /* Define to the type of arg 1 of 'accept' */ #define ACCEPT_TYPE_ARG1 unsigned int /* Define to the type of arg 2 of 'accept' */ #define ACCEPT_TYPE_ARG2 struct sockaddr * /* Define to the type of arg 3 of 'accept' */ #define ACCEPT_TYPE_ARG3 int /* Define to the return type of 'accept' */ #define ACCEPT_TYPE_RETURN unsigned int PASCAL /* The alignment requirement of a `double'. */ #define ALIGNOF_DOUBLE 8 /* The alignment requirement of a `int'. */ #define ALIGNOF_INT 4 /* The alignment requirement of a `long'. */ #define ALIGNOF_LONG 4 /* The alignment requirement of a `long long int'. */ #define ALIGNOF_LONG_LONG_INT 8 /* The alignment requirement of a `short'. */ #define ALIGNOF_SHORT 2 /* Define to the default TCP port number on which the server listens and to which clients will try to connect. This can be overridden at run-time, but it's convenient if your clients have the right default compiled in. (--with-pgport=PORTNUM) */ #define DEF_PGPORT 5432 /* Define to the default TCP port number as a string constant. */ #define DEF_PGPORT_STR 5432 /* Define to 1 if you want National Language Support. (--enable-nls) */ /* #undef ENABLE_NLS */ /* Define to 1 to build client libraries as thread-safe code. (--enable-thread-safety) */ /* #undef ENABLE_THREAD_SAFETY */ /* Define to 1 if getpwuid_r() takes a 5th argument. */ /* #undef GETPWUID_R_5ARG */ /* Define to 1 if gettimeofday() takes only 1 argument. */ /* #undef GETTIMEOFDAY_1ARG */ #ifdef GETTIMEOFDAY_1ARG # define gettimeofday(a,b) gettimeofday(a) #endif /* Define to 1 if you have the `atexit' function. */ #define HAVE_ATEXIT 1 /* Define to 1 if you have the `cbrt' function. */ #define HAVE_CBRT 1 /* Define to 1 if you have the `class' function. */ /* #undef HAVE_CLASS */ /* Define to 1 if you have the `crypt' function. */ /* #undef HAVE_CRYPT */ /* Define to 1 if you have the crypt.h header file. */ /* #undef HAVE_CRYPT_H */ /* Define to 1 if you have the declaration of `fdatasync', and to 0 if you don't. */ #define HAVE_DECL_FDATASYNC 0 /* Define to 1 if you have the declaration of `snprintf', and to 0 if you don't. */ #define HAVE_DECL_SNPRINTF 1 /* Define to 1 if you have the declaration of `vsnprintf', and to 0 if you don't. */ #define HAVE_DECL_VSNPRINTF 1 /* Define to 1 if you have the dld.h header file. */ /* #undef HAVE_DLD_H */ /* Define to 1 if you have the `dlopen' function. */ /* #undef HAVE_DLOPEN */ /* Define to 1 if you have the editline/history.h header file. */ /* #undef HAVE_EDITLINE_HISTORY_H */ /* Define to 1 if you have the editline/readline.h header file. */ /* #undef HAVE_EDITLINE_READLINE_H */ /* Define to 1 if you have the endian.h header file. */ /* #undef HAVE_ENDIAN_H */ /* Define to 1 if you have the `fcvt' function. */ #define HAVE_FCVT 1 /* Define to 1 if you have the `fdatasync' function. */ /* #undef HAVE_FDATASYNC */ /* Define to 1 if you have finite(). */ #define HAVE_FINITE 1 /* Define to 1 if you have the `fpclass' function. */ /* #undef HAVE_FPCLASS */ /* Define to 1 if you have the `fp_class' function. */ /* #undef HAVE_FP_CLASS */ /* Define to 1 if you have the `fp_class_d' function. */ /* #undef HAVE_FP_CLASS_D */ /* Define to 1 if you have the fp_class.h header file. */ /* #undef HAVE_FP_CLASS_H */ /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */ /* #undef HAVE_FSEEKO */ /* Define to 1 if your compiler understands __func__. */ #define HAVE_FUNCNAME__FUNC 1 /* Define to 1 if your compiler understands __FUNCTION__. */ /* #undef HAVE_FUNCNAME__FUNCTION */ /* Define to 1 if you have the `getaddrinfo' function. */ /* #undef HAVE_GETADDRINFO */ /* Define to 1 if you have the `gethostbyname_r' function. */ /* #undef HAVE_GETHOSTBYNAME_R */ /* Define to 1 if you have the `gethostname' function. */ /* #undef HAVE_GETHOSTNAME */ /* Define to 1 if you have the `getopt' function. */ #define HAVE_GETOPT 1 /* Define to 1 if you have the getopt.h header file. */ #define HAVE_GETOPT_H 1 /* Define to 1 if you have the `getopt_long' function. */ #define HAVE_GETOPT_LONG 1 /* Define to 1 if you have the `getpeereid' function. */ /* #undef HAVE_GETPEEREID */ /* Define to 1 if you have the `getpwuid_r' function. */ /* #undef HAVE_GETPWUID_R */ /* Define to 1 if you have the `getrusage' function. */ /* #undef HAVE_GETRUSAGE */ /* Define to 1 if you have the history.h header file. */ /* #undef HAVE_HISTORY_H */ /* Define to 1 if
Re: [HACKERS] logging as inserts
On Tue, 1 Mar 2005, Joshua D. Drake wrote: Hello, I am looking at having one of our guys write up the code to allow logging as insert statements. I have a couple of questions. What would we like the postgresql.conf option to be? I was thinking log_statements_as_inserts = (t/f) However I thought that was getting a little silly. Also do we want to allow logging as inserts for all options? There is a lot of different types of logging we can do. Seems to me that a better approach is a script which, given the log file format, is able to parse and allow the user to format the insert themselves. The reason I say this is that users are almost always going to want a human readable log. As such, post processing the log outside of the database system seems to make sense. Sincerely, Joshua D. Drake Gavin ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch
Bruce Momjian wrote: Have you read the developer's FAQ. It is a requirement to get involved. How does the developer's FAQ address this concern? -Neil ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch
I have a feeling Bruce was referring to item 1.4: http://developer.postgresql.org/readtext.php?src/FAQ/ FAQ_DEV.html+Developers-FAQ#1.4 -tfo -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-260-0005 On Mar 1, 2005, at 5:18 PM, Neil Conway wrote: Bruce Momjian wrote: Have you read the developer's FAQ. It is a requirement to get involved. How does the developer's FAQ address this concern? -Neil ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [pgsql-hackers-win32] snprintf causes regression tests to fail
Nicolai Tufar wrote: On Tue, 01 Mar 2005 17:45:31 -0500, Tom Lane [EMAIL PROTECTED] wrote: Nicolai Tufar [EMAIL PROTECTED] writes: Just out of curiosity, do either HAVE_INT64 or HAVE_UINT64 get set in pg_config.h? The observed symptoms would be explained if typedef int64 were ending up as long rather than long long. Looking at the #ifdef nest in include/c.h, there are a couple of ways that could happen, including importing a definition from system header files. If this were happening, it would presumably break all int8 math not only snprintf, so I'm not sure it's the story. I am looking into it. Will report if find something of importance. As far as I've seen, no one has actually posted the regression diffs seen in this failure, so most of us are in the dark about the details of the problem. Regression diiff is attached. The problem is reported by Magnus and me. Probably nobody else compiles pg under Win32 these days. I am testing the failure here. I will keep at it until I find the cause. The only downside is that Win32 compiles are much slower than Unix. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] snprintf causes regression tests to fail
I have some new information. If I add the attached patch to snprintf.c, I should see see snprintf() being called printing 0, vsnprintf() printing 1 and dopr(), 2. However, I only see 0 printing in the server logs. I think this means it is finding our /port/snprintf(), but when it calls vsnprintf, it must be using some other version, probably the operating system version that doesn't support %lld. I am also attaching the 'nm' output from libpgport_srv.a which does show vsnprintf as being defined. Win32 doesn't like multiply defined symbols so we use -Wl,--allow-multiple-definition to allow multiple symbols. I bet if I define LONG_LONG_INT_FORMAT as '%I64d' it would pass the regression tests. (I will test now.) Our config/c-library.m4 file confirms that format for MinGW: # MinGW uses '%I64d', though gcc throws an warning with -Wall, The big question is why our own vsnprintf() is not being called from snprintf() in our port file. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 *** snprintf.c Tue Mar 1 19:02:13 2005 --- /laptop/tmp/snprintf.c Tue Mar 1 20:27:21 2005 *** *** 96,101 --- 96,102 int len; va_list args; + puts(0); va_start(args, fmt); len = vsnprintf(str, count, fmt, args); va_end(args); *** *** 109,114 --- 110,116 char *end; str[0] = '\0'; end = str + count - 1; + puts(1); dopr(str, fmt, args, end); if (count 0) end[0] = '\0'; *** *** 178,183 --- 180,186 int realpos; } fmtpar[NL_ARGMAX+1], *fmtparptr[NL_ARGMAX+1]; + puts(2); format_save = format; output = buffer; crypt.o: b .bss d .data t .text 0060 b _a64toi 2ce0 b _CF6464 03c0 t _CIFP 34e0 b _constdatablock 0441 T _crypt 34f0 b _cryptresult 0760 t _des_cipher d _des_ready 06c2 t _des_setkey 00c0 t _ExpandTr 18e0 b _IE3264 0a84 t _init_des 0f0f t _init_perm 0080 t _IP 0400 t _itoa64 3510 b _KS 03a0 t _P32Tr 0100 t _PC1 00e0 b _PC1ROT 0160 t _PC2 08e0 b _PC2ROT b _perm.0 t _permute 0138 t _Rotates 01a0 t _S 1ce0 b _SPE 0040 b _tmp32.1 fseeko.o: b .bss d .data t .text getrusage.o: b .bss d .data t .text U ___udivdi3 U ___umoddi3 U __dosmaperr U __errno U [EMAIL PROTECTED] U [EMAIL PROTECTED] U [EMAIL PROTECTED] T _getrusage inet_aton.o: b .bss d .data t .text U __impmb_cur_max U __imp___pctype U __isctype U [EMAIL PROTECTED] T _inet_aton random.o: b .bss d .data t .text U _lrand48 T _random srandom.o: b .bss d .data t .text U _srand48 T _srandom unsetenv.o: b .bss d .data t .text U _getenv U _malloc U _putenv U _sprintf 0004 T _unsetenv getaddrinfo_srv.o: b .bss d .data t .text U _atoi U _free U [EMAIL PROTECTED] U [EMAIL PROTECTED] U [EMAIL PROTECTED] U _inet_aton U [EMAIL PROTECTED] U _malloc U _memcpy U [EMAIL PROTECTED] 025a T _pg_freeaddrinfo 02ca T _pg_gai_strerror T _pg_getaddrinfo 02f6 T _pg_getnameinfo U _snprintf U [EMAIL PROTECTED] copydir.o: b .bss d .data t .text t ___func__.0 U _AllocateDir 00a5 T _copydir U [EMAIL PROTECTED] U _errcode_for_file_access U _errfinish U _errmsg U _errstart U _FreeDir U _mkdir U _readdir U _snprintf gettimeofday.o: b .bss d .data t .text U ___udivdi3 t _epoch U [EMAIL PROTECTED] 0008 T _gettimeofday U [EMAIL PROTECTED] kill.o: b .bss d .data t .text U __errno U [EMAIL PROTECTED] U [EMAIL PROTECTED] 0015 T _pgkill U _wsprintfA open.o: b .bss d .data t .text U __assert U __errno U __open_osfhandle U __setmode U [EMAIL PROTECTED] U [EMAIL PROTECTED] U [EMAIL PROTECTED] t _openFlagsToCreateFileFlags 0160 T _win32_open rand.o: b .bss d .data t .text t __dorand48 000c D
Re: [HACKERS] snprintf causes regression tests to fail
Bruce Momjian pgman@candle.pha.pa.us writes: I think this means it is finding our /port/snprintf(), but when it calls vsnprintf, it must be using some other version, probably the operating system version that doesn't support %lld. Ya know, I was wondering about that but dismissed it because the routines were all declared in the same file. Windows' linker must behave very oddly to do this. Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] snprintf causes regression tests to fail
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I think this means it is finding our /port/snprintf(), but when it calls vsnprintf, it must be using some other version, probably the operating system version that doesn't support %lld. I can confirm that using %I64d for the printf format allows the regression tests to pass for int8. Ya know, I was wondering about that but dismissed it because the routines were all declared in the same file. Windows' linker must behave very oddly to do this. Yep, quite unusual. Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? Testing now. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] snprintf causes regression tests to fail
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I think this means it is finding our /port/snprintf(), but when it calls vsnprintf, it must be using some other version, probably the operating system version that doesn't support %lld. Ya know, I was wondering about that but dismissed it because the routines were all declared in the same file. Windows' linker must behave very oddly to do this. Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? Yes, it fixes the problem and I have applied the reordering with a comment. I will start working on fixing the large fmtpar allocations now. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] snprintf causes regression tests to fail
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? Yes, it fixes the problem and I have applied the reordering with a comment. Fascinating. I will start working on fixing the large fmtpar allocations now. Quite honestly, this code is not worth micro-optimizing because it is fundamentally broken. See my other comments in this thread. Can we find a BSD-license implementation that follows the spec? regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] snprintf causes regression tests to fail
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? Yes, it fixes the problem and I have applied the reordering with a comment. Fascinating. I will start working on fixing the large fmtpar allocations now. Quite honestly, this code is not worth micro-optimizing because it is fundamentally broken. See my other comments in this thread. I am working on something that just counts the '%' characters in the format string and allocates an array that size. Can we find a BSD-license implementation that follows the spec? I would think NetBSD would be our best bet. I will download it and take a look. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] snprintf causes regression tests to fail
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Does it help if you flip the order of the snprintf and vsnprintf functions in snprintf.c? Yes, it fixes the problem and I have applied the reordering with a comment. Fascinating. I will start working on fixing the large fmtpar allocations now. Quite honestly, this code is not worth micro-optimizing because it is fundamentally broken. See my other comments in this thread. I am working on something that just counts the '%' characters in the format string and allocates an array that size. Can we find a BSD-license implementation that follows the spec? I would think NetBSD would be our best bet. I will download it and take a look. Oops, I remember now that NetBSD doesn't support it. I think FreeBSD does. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] logging as inserts
On Tuesday 01 March 2005 18:16, Gavin Sherry wrote:
On Tue, 1 Mar 2005, Joshua D. Drake wrote:
Hello,
I am looking at having one of our guys write up the code to allow
logging as insert statements. I have a couple of questions.
What would we like the postgresql.conf option to be? I was thinking
log_statements_as_inserts = (t/f)
However I thought that was getting a little silly. Also do we want
to allow logging as inserts for all options? There is a lot of different
types of logging we can do.
Seems to me that a better approach is a script which, given the log file
format, is able to parse and allow the user to format the insert
themselves.
Doesn't the Practical Query Analysis project already do this? (It's up on
pgfoundry for anyone interested)
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] mysterious log output
On Tuesday 01 March 2005 12:46, Tom Lane wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
Tom Lane wrote:
This has to be coming from exec_simple_query():
I wonder if some statistics were turned on at postmaster start and even
though it was turned off and 'pg_ctl reload' was done the
checkpoint/bgwriter process isn't reloading for some reason.
The checkpoint/bgwriter never executes exec_simple_query(). I think
the 5-minute cycle is driven by something in Robert's application,
rather than being tied to checkpoints.
Actually looking at it now, it happens every 10 minutes, so you're right in
that it doesn't coincide with the bgwriter. Actually I am pretty sure this
is slony related... the output is always on the same pid, and everything else
output on that pid is slony commands. Is there some way I could see what
that pid thinks the GUC variables are set to? I have a wild theory that
slony is preventing the modification of SUSET level variables, but haven't
really got anything solid to back that up.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
