Re: [HACKERS] 8.3 beta timing
On Sun, 2007-09-30 at 00:25 -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I think we need another week to get things ready for beta. Why? Other than the lack of release notes, we could wrap on Monday. +1 The full release notes aren't really required for Beta. We can just say: Here are the main changes in 8.3, please focus your testing in these areas first. A full set of release notes will be available here link as Beta progresses. That's actually desirable because we want Beta to test the whole system, not just the individual parts of it. That way we're more likely to hit performance regressions and multi-feature integration issues. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 beta timing
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I think we need another week to get things ready for beta. Why? Other than the lack of release notes, we could wrap on Monday. OK, Monday is fine. It seemed to me there were was a lot of activity in recent days so I wasn't sure everything was nailed down but if others think we are OK, we must be OK then. FYI, pgindent is usually run later in the beta process. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 beta timing
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I think we need another week to get things ready for beta. Why? Other than the lack of release notes, we could wrap on Monday. OK, Monday is fine. It seemed to me there were was a lot of activity in recent days so I wasn't sure everything was nailed down but if others think we are OK, we must be OK then. Well, the only stuff left on the pending-patches page is documentation issues and the question of what to do about tsearch contrib modules; neither of those seem to be beta-stoppers, especially not when we have no commitment from Oleg and Teodor about when they might do something towards the proposed new module. I'm not happy with taking still another week's slip for no reason. If there's some specific task that should be done and we have a commitment from someone to get it done by a particular time, then okay, but otherwise it's past time to push this puppy out the door. FYI, pgindent is usually run later in the beta process. Right, that's traditionally a late-in-beta item. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Something's been bugging me
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I'd be inclined to make the second byte be the length and have VARSIZE_1B_E depend on that --- any objection? On one hand it offends me since it's hard coding an assumption that the size of a pointer decides what it contains and vice versa. There's nothing saying we won't have two possible special meanings for a one-byte datum. Well, what you're proposing is to treat the second byte of a 1B_E datum as an enum value, requiring every piece of code that examines it to know exactly what all the possible values are. That doesn't sound like a good idea to me. If it does turn out to be a good idea, we could still redefine it that way --- we'd just have assigned 18 not 0 as the enumeration value for basic TOAST pointers. The key point in my mind is that there is lots of performance-critical code (the inner loops of heap_deform_tuple and friends) that needs to determine the physical size of a datum quickly. Interpreting the content of a toasted datum is a completely separate and much less performance-critical task. If it turns out that the size is not sufficient to tell the difference between two types of 1B_E values, we can go over to examining the contents instead. I note that basic TOAST pointers start with va_rawsize which can't exceed 1G, so there are two free bits that could be exploited in exactly the same way as TOAST and now varvarlena have done with 4-byte datum length words. And it forecloses any possibility of having a type whose size is at all variable. Au contraire, I think it makes it easier, at least for sizes up to 255 bytes --- you need not introduce any more complexity into VARSIZE_ANY to have that. On the other hand I suppose you're concerned about the time to do a few comparisons before knowing which length to skip over? I'm not entirely sure cycle-counting at that level leads to the correct conclusions. I am. I have spent many many hours examining PG profiling results, and stuff in and around the tuple-decoding loops is almost always interesting from a performance standpoint. Cycles spent in interpreting a toast pointer never are (not least because you probably have to go off and do I/O after you interpret the pointer). I'm willing to push almost any amount of work onto toast_fetch_datum if it'll save cycles in VARSIZE_ANY. But in this case you haven't even demonstrated a reason to think that any complexity will be added there. The likely uses for this, in my mind, are toast pointers with wider valueid fields and toast pointers with indicators of different compression methods, and those seem like they'd naturally be different sizes anyway. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql: Applied another patch by ITAGAKI Takahiro
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Michael Meskes wrote: Applied another patch by ITAGAKI Takahiro [EMAIL PROTECTED] to get memory allocation thread-safe. He also did some cleaning up. this patch seems to break every single buildfarm member out there: http://buildfarm.postgresql.org/cgi-bin/show_status.pl Looks like Michael forgot a cvs add? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [COMMITTERS] pgsql: Applied another patch by ITAGAKI Takahiro
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Michael Meskes wrote: Applied another patch by ITAGAKI Takahiro [EMAIL PROTECTED] to get memory allocation thread-safe. He also did some cleaning up. this patch seems to break every single buildfarm member out there: http://buildfarm.postgresql.org/cgi-bin/show_status.pl I guess Michael's not on-line at the moment. In hopes of un-breaking the buildfarm, I took the missing file from Takahiro-san's patch and committed it. If there were any edits needed, Michael should commit his version ... regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [PATCHES] [HACKERS] Add function for quote_qualified_identifier?
On 9/29/07, Bruce Momjian [EMAIL PROTECTED] wrote: I think we need more than one person's request to add this function. Well, I don't expect it would get requested. Most DBAs would likely look for the function in the docs, see it's not there and then just implement it themselves. Obviously it's not critical. But anticipating those little requirements and providing for them is one of the things that makes a piece of software a pleasure to use. Batteries included and all that. Anyway, I seem to be flogging a horse which, if not dead, is surely mortally wounded. If quote_qualified_ident isn't desired, perhaps you can still use the regression test I included for quote_ident in the patch. The test is functional as a standalone item, and seems to fill a gap. Thanks for your time, BJ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] First steps with 8.3 and autovacuum launcher
Tom Lane wrote: Guillaume Smet [EMAIL PROTECTED] writes: So a total of: 16 minutes for 8.2 compared to 53 minutes for 8.3 to have the database in the same state. Please try that experiment with all three configurations on both versions: * autovacuum off * autovacuum on, autovacuum_vacuum_cost_delay = 0 * autovacuum on, autovacuum_vacuum_cost_delay = 20 Comparing apples and oranges isn't real helpful in determining what's happening. for the record I'm seeing similiar dramatic slowdowns(ie takes a few additional hours) during restoration of a 120GB database here - with the default autovacuum settings the restore is taking ages to go through the ALTER TABLE foo ADD PRIMARY KEY bar steps because autovacuum is analyzing those tables at a very slow rate (ie the box is more or less idle and the ALTER TABLE is waiting for the lock). I have not yet experimented with different autovacuum_vacuum_cost_delay settings but this seems like a step backwards :-( Stefan ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Something's been bugging me
Gregory Stark [EMAIL PROTECTED] writes: Here's a patch that does all of the above. Applied with tweak to use the added byte as an actual length word. I ran into an interesting failure here on HPPA: the code the compiler generated for copying unaligned toast pointers into aligned local variables failed, because it was assuming halfword (2-byte) alignment of the data to be copied! (Instead of a memcpy call it was generating an inline loop of ldh/sth instructions.) Apparently gcc's thought process is the pointer is declared as struct varlena *, therefore must be at least 4-aligned, therefore the data at offset 2 is at least 2-aligned. The intermediate cast to varattrib_1b_e * did not prevent this; I had to assign the datum pointer into a separate local variable of that type to suppress the optimization. I'm not sure if the gcc boys would consider this a bug or not; I kinda suspect the behavior is intentional, because otherwise they'd not be able to optimize constructs like memcpy((char *) foo, ...) which is a pretty darn widespread locution. Anyway, it seems to work now, I just thought I'd put something in the archives about what that VARATT_EXTERNAL_GET_POINTER macro is for. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Something's been bugging me
Tom Lane [EMAIL PROTECTED] writes: Apparently gcc's thought process is the pointer is declared as struct varlena *, therefore must be at least 4-aligned, therefore the data at offset 2 is at least 2-aligned. The intermediate cast to varattrib_1b_e * did not prevent this; I had to assign the datum pointer into a separate local variable of that type to suppress the optimization. Fascinating. Why do you cast arguments to memcmp to char* ? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Latest ecpg patch broke MSVC build
This morning's ecpg patch certainly seems to have been snake-bit. Although the Windows gcc buildfarm members seem happy, the MSVC ones are all failing with Linking... Creating library Release\libecpg\libecpg.lib and object Release\libecpg\libecpg.exp libecpg.exp : error LNK2001: unresolved external symbol [EMAIL PROTECTED] .\Release\libecpg\libecpg.dll : fatal error LNK1120: 1 unresolved externals I see that DllMain() got added to misc.c, so it's not obvious what's wrong here. Some adjustment needed in the MSVC build scripts maybe? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Something's been bugging me
Gregory Stark [EMAIL PROTECTED] writes: Why do you cast arguments to memcmp to char* ? Well, *I* haven't done it in a long time, but it used to be a fairly standard thing. I imagine that back before memcpy was usually declared with void * arguments, it was necessary to avoid compiler warnings. The problem would still exist if the code were written memcpy((void *) foo, ...) so I suspect that an in-line cast to void * wouldn't work any better. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Something's been bugging me
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Why do you cast arguments to memcmp to char* ? Well, *I* haven't done it in a long time, I'm referring to tuptoaster.c:488 -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Something's been bugging me
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Why do you cast arguments to memcmp to char* ? Well, *I* haven't done it in a long time, I'm referring to tuptoaster.c:488 Oh, I'm sorry, I thought you were talking about my hypothetical memcpy example. The casts in the memcmp call were because I was worried about some compiler trying to optimize the memcmp into a word-wide operation on the strength of the theory that struct varlena pointers must be word-aligned. Given the later discovery about the copying problem, those casts may be unhelpful as far as gcc is concerned; but I'm still inclined to leave them there in case they make a difference for some other compiler. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] adding operators
Dear PostgreSQL developers I am a graduate student in the University of Calgary. I want to add some new operators to PostgreSQL to perform some specific tasks in a project I am working in. My problem is that I cannot find my way into the code, where should I start and where to find the documentation for the code. Regards Islam Hegazy
Re: [HACKERS] adding operators
On 10/1/07, Islam Hegazy [EMAIL PROTECTED] wrote: I am a graduate student in the University of Calgary. I want to add some new operators to PostgreSQL to perform some specific tasks in a project I am working in. My problem is that I cannot find my way into the code, where should I start and where to find the documentation for the code. There's no need to hack Postgres to add operators. You can do so by defining functions using CREATE FUNCTION, and then hooking operators up to them using CREATE OPERATOR. http://www.postgresql.org/docs/8.2/static/xoper.html http://www.postgresql.org/docs/8.2/static/sql-createoperator.html Regards, BJ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 beta timing
[EMAIL PROTECTED] (Bruce Momjian) writes: I think we need another week to get things ready for beta. I will have the release notes done mid-week and hopefully we can close out all open items by the end of the week. It's worth noting that Greg Smith has collected release note information into the wiki; that is hopefully a useful resource for organizing them topically and such, perhaps even permitting the effort to get split up a bit... http://developer.postgresql.org/index.php/8.3_Changelog http://developer.postgresql.org/index.php/8.3release -- output = reverse(ofni.sesabatadxunil @ enworbbc) http://www3.sympatico.ca/cbbrowne/internet.html Shin: A device for finding furniture in the dark. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Latest ecpg patch broke MSVC build
Tom Lane wrote: This morning's ecpg patch certainly seems to have been snake-bit. Although the Windows gcc buildfarm members seem happy, the MSVC ones are all failing with Linking... Creating library Release\libecpg\libecpg.lib and object Release\libecpg\libecpg.exp libecpg.exp : error LNK2001: unresolved external symbol [EMAIL PROTECTED] .\Release\libecpg\libecpg.dll : fatal error LNK1120: 1 unresolved externals I see that DllMain() got added to misc.c, so it's not obvious what's wrong here. Some adjustment needed in the MSVC build scripts maybe? It is building with thread.c but it should not be unless I am misreading the Makefile. The makefile processing in Project.pm doesn't look nearly powerful enough to handle this: # thread.c is needed only for non-WIN32 implementation of path.c ifneq ($(PORTNAME), win32) OBJS += thread.o endif It will ignore the if and endif lines and process the OBJS line :-( A quick fix is probably to put some whitespace in front of OBJS, although that seems horribly fragile. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Latest ecpg patch broke MSVC build
Andrew Dunstan [EMAIL PROTECTED] writes: It is building with thread.c but it should not be unless I am misreading the Makefile. The makefile processing in Project.pm doesn't look nearly powerful enough to handle this: # thread.c is needed only for non-WIN32 implementation of path.c ifneq ($(PORTNAME), win32) OBJS += thread.o endif Hmm, sounds like a problem, but why was it not a problem before? regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Latest ecpg patch broke MSVC build
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: It is building with thread.c but it should not be unless I am misreading the Makefile. The makefile processing in Project.pm doesn't look nearly powerful enough to handle this: # thread.c is needed only for non-WIN32 implementation of path.c ifneq ($(PORTNAME), win32) OBJS += thread.o endif Hmm, sounds like a problem, but why was it not a problem before? Good point. I don't know. I guess something else must be causing the build failure. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
