Re: [HACKERS] damage control mode
Robert Haas writes: >>> ... The affected patches are: >>> - Listen/Notify Rewrite. >>> - Writeable CTEs. >>> - more frame options for window functions >>> - knngist >>> - rbtree > I think it might be time to revisit this issue. SR is in, and we have > a week left in the CF, and we have all of the above patches plus 5 > small ones left to deal with. rbtree is close to being committable, I > think; knngist has not been reviewed yet; you (Tom) have claimed the > frame options patch but I haven't seen any update on it in a while; I > doubt either of the other two are ready to commit but I'm not sure how > far they have to go. Yeah, I claimed the window frame patch and then got distracted on the vacuum full vs HS issue. Current status on that: I expect to be able to commit the core relation-mapping patch tomorrow (if the power stays on here, which is no sure thing given the weather). There is something like another day or two's worth of work involved to rip out VACUUM FULL INPLACE. I suppose that part could get put off till later, but I'm the kind of person who prefers to finish a project once it's started. I feel it has to get done before we release the final alpha in any case. Given that we have a week still to go in the CF, I feel fairly confident of still getting the window frame patch in on time (assuming that there are indeed no major problems with it). I have not let go of it for that reason, and also because I doubt anybody else is qualified to commit it --- AFAIR only Hitoshi-san and I were really neck-deep in the original window patch. However, with the deadline fast approaching, I don't feel that I can also promise to handle writeable CTEs, which is another one that I'd really like to be the committer for. Maybe we had better make a management decision about which of those two is higher priority to get into 9.0. Also: I haven't been following either one terribly closely lately. If there's reason to think that one is more likely to be committable than the other, that ought to get factored into the choice of which to go to first; but I'm not sure whether that's the case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify - what to do when the queue is full
On Sun, 2010-02-07 at 00:18 -0500, Robert Haas wrote: > Jeff, do you think this patch is ready for committer? If so, please > mark it as such on commitfest.postgresql.org - otherwise, please > clarify what you think the action items are. I'll post an update tomorrow. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On Fri, 2010-02-05 at 09:19 -0500, Bruce Momjian wrote: > My son has brought to my attention that our current crop of Python > client libraries is inadequate/confusing. I took a look myself, and > asked on our IRC channel, and am now convinced this area needs > attention. I have written up a set of guidelines for driver development based on what I learned working on ruby-pg: http://wiki.postgresql.org/wiki/Driver_development Whether we take one of the existing projects and improve upon it, or start a complete rewrite, I hope these guidelines will be a useful destination. Note that the ruby-pg driver doesn't 100% adhere to those standards (encoding is the primary problem, and that will be fixed). I would appreciate comments by anyone (Greg Sabino Mullane: I included you in the CC because I thought you may have some input). Even if the python driver doesn't go in that direction, it will help me improve ruby-pg. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Robert, > I think it might be time to revisit this issue. SR is in, and we have > a week left in the CF, and we have all of the above patches plus 5 > small ones left to deal with. rbtree is close to being committable, I > think; knngist has not been reviewed yet; you (Tom) have claimed the > frame options patch but I haven't seen any update on it in a while; I > doubt either of the other two are ready to commit but I'm not sure how > far they have to go. I think, as previously discussed, that we should bounce knngist.It's a complex patch and nobody saw anything of it until Jan 15, so I don't feel bad about it. Mark Cave-Ayland was going to review it, but apparently felt that rbtree was the higher priority. Also, this one: Provide rowcount for utility SELECTs ... based on your last review does not look anywhere near ready. We know the following because of endless discussion on -hackers; what these patches seem to be suffering from is endless arguments over relatively minor points, it just requires someone to make a decision on implementation method: Add on_trusted_init and on_untrusted_init to plperl Faster CREATE DATABASE by delaying fsync For the rest, can we just have reviewer reports on readiness? Writeable CTEs Package namespace and Safe init cleanup for plperl More frame options in window functions Fix large object support in pg_dump Actually, looking at that list, I think we're in pretty darned good shape. That's a pretty small list of things left to commit. Keep in mind that last year at this time (week 3 of CF-last) we still had over a dozen patches completely unreviewed! --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Thu, Jan 7, 2010 at 10:20 PM, Tom Lane wrote: > Robert Haas writes: >> OK, we have a proposal on the table to bump some patches from this >> CommitFest to free up more committer resources, particularly Tom, to >> work on Hot Standby and Streaming Replication and attempt to >> accelerate the process of getting 8.5 out the door. This proposal >> needs input from the community. The affected patches are: > >> - Listen/Notify Rewrite. >> - Writeable CTEs. >> - more frame options for window functions >> - knngist >> - rbtree > > Looking at this list again, it strikes me that the listen/notify rewrite > might need to go in so that we have a sane framework for listen/notify > with HS. Treating pg_listener as a replicatable table isn't sane at > all, whereas with a message-driven notify mechanism we have at least got > the possibility of shipping the messages to the standby (via WAL) and > having listeners there. I don't want to say we'd actually implement > that in 8.5, but shipping pg_listener tuple updates is just completely > nuts. > > The other four things have no apparent connection to HS/SR so I think > they could be punted without creating technical issues. Whether this > is really necessary from a schedule viewpoint is not clear yet. > > My thought about it would be to put these four on the back burner; > not necessarily bounce them right away, but not put any effort into > them until we have dealt with the other stuff in the January CF. > At that point we should have a feel for where we are schedule-wise, > and in particular we'll know whether SR is in or not ;-) I think it might be time to revisit this issue. SR is in, and we have a week left in the CF, and we have all of the above patches plus 5 small ones left to deal with. rbtree is close to being committable, I think; knngist has not been reviewed yet; you (Tom) have claimed the frame options patch but I haven't seen any update on it in a while; I doubt either of the other two are ready to commit but I'm not sure how far they have to go. Thoughts? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify - what to do when the queue is full
On Wed, Feb 3, 2010 at 4:34 AM, Joachim Wieland wrote: > On Wed, Feb 3, 2010 at 2:05 AM, Jeff Davis wrote: >>> Thanks, very well spotted... Actually the same is true for LISTEN... I >>> have reworked the patch to do the changes to listenChannels only in >>> the post-commit functions. >> >> I'm worried that this creates the opposite problem: that a LISTEN >> transaction might commit before a NOTIFY transaction, and yet miss the >> notification. > > See the following comment and let me know if you agree... > > ! /* > ! * Exec_ListenBeforeCommit --- subroutine for AtCommit_NotifyBeforeCommit > ! * > ! * Note that we do only set our pointer here and do not yet add the channel > to > ! * listenChannels. Since our transaction could still roll back we do this > only > ! * after commit. We know that our tail pointer won't move between here and > ! * directly after commit, so we won't miss a notification. > ! */ > > However this introduces a new problem when an initial LISTEN aborts: > Then we are not listening to anything but for other backends it looks > like we were. This is tracked by the boolean variable > backendExecutesInitialListen and gets cleaned up in AtAbort_Notify(). > > >> It seems safest to me to add a backend (LISTEN) to the list before >> commit, and remove a backend (UNLISTEN) after commit. That way we are >> sure to only receive spurious notifications, and can't miss any. > > If a LISTEN aborted we would not only receive a few spurious > notifications from it but would receive notifications on this channel > forever even though we have never executed LISTEN on it successfully. Jeff, do you think this patch is ready for committer? If so, please mark it as such on commitfest.postgresql.org - otherwise, please clarify what you think the action items are. Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Sat, Feb 6, 2010 at 7:03 AM, Andres Freund wrote: > On Saturday 06 February 2010 06:03:30 Greg Smith wrote: >> Andres Freund wrote: >> > On 02/03/10 14:42, Robert Haas wrote: >> >> Well, maybe we should start with a discussion of what kernel calls >> >> you're aware of on different platforms and then we could try to put an >> >> API around it. >> > >> > In linux there is sync_file_range. On newer Posixish systems one can >> > emulate that with mmap() and msync() (in batches obviously). >> > >> > No idea about windows. >> The effective_io_concurrency feature had proof of concept test programs >> that worked using AIO, but actually following through on that >> implementation would require a major restructuring of how the database >> interacts with the OS in terms of reads and writes of blocks. It looks >> to me like doing something similar to sync_file_range on Windows would >> be similarly difficult. > Looking a bit arround it seems one could achieve something approximediately > similar to pg_prepare_fsync() by using > CreateFileMapping && MapViewOfFile && FlushViewOfFile > > If I understand it correctly that will flush, but not wait. Unfortunately you > cant event make it wait, so its not possible to implement sync_file_range or > similar fully. Well it seems that what we're trying to implement is more like it_would_be_nice_if_you_would_start_syncing_this_file_range_but_its_ok_if_you_dont(), so maybe that would work. Anyway, is there something that we can agree on and get committed here for 9.0, or should we postpone this to 9.1? It seems simple enough that we ought to be able to get it done, but we're running out of time and we don't seem to have a clear vision here yet... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Provide rowcount for utility SELECTs
On Tue, Feb 2, 2010 at 4:03 AM, Boszormenyi Zoltan wrote: > Thanks for testing it, with the attached patch your test case also > returns SELECT N. Thoughts: 1. Looks like you've falsified the last comment block in PortalRunMulti(). 2. I don't like the duplication of code in PortalRun() between the first and second branches of the switch statement. 3. You've falsified the comment preceding that code, too. 4. Is there any reason to use pg_strcasecmp() instead of plain old strcmp()? Someone who knows the overall structure of the code better than I do will have to comment on whether there are any code paths to worry about that do not go through PortalRun(). A general concern I have is that this what we're basically doing here is handling the most common case in ProcessQuery() and then installing fallback mechanisms to pick up any stragglers: but the fallback mechanisms only guarantee that we'll add a number to the command tag, not that it will be meaningful. Unfortunately, my limited imagination can't quite figure out in what cases we'll get a SELECT command tag back other than (1) plain old SELECT, (2) SELECT INTO, and (3) CTAS, so I'm not sure what to go test. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On Feb 6, 2010, at 5:51 PM, Josh Berkus wrote: >> Finally, I just don't see the existing (often PG specific) goals that I have >> in mind for it appealing to the majority of [web framework/abstraction] >> users. > > What are those goals? I think the most interesting one that has yet to be implemented is fast, multiple destination COPY support. Currently, COPY is supported, but a bytes() object is allocated for each row, so it's currently not efficient for moving data(pg-to-pg ETL sans the T? =). While some C is still needed to make it properly efficient, it's primarily to keep track of the COPY's state and to update stats. This is pretty useless to a django user... Well, I suppose it might be interesting if COPY OUT could target [or transform into] JSON, but idk... The general, ongoing goal is to implement and document *convenient* Python interfaces to PostgreSQL features. A, perhaps uninteresting, case being "supporting" advisory locks. I was thinking a context manager, but it might just be something as trivial as an additional method on the connection(very simple/direct binding). Some "fuzzy" goals: twisted support, some asynchronous interfaces, and greater user control over type I/O. The first, twisted, mostly interests me as an exercise. The second, async interfaces, scares me as it took me some time just to feel "not unhappy" with the blocking APIs. The third will probably happen, but it's going to be a while. I also have some goals not directly related to a driver. postgresql.unittest is currently only used internally, but I hope to document it some day soon so that people can write Python unittest.TestCase's that auto-magically build out a target cluster(~pg_regress/pgTap for Python?). Well, it works, but it's not documented and the APIs haven't been given much thought. Generally, basic cluster management tools for Python. (At one point I tried to write a programmer's HBA editor, but I think I hurt myself trying to figure out rule reduction.. That is, it was trying to be smarter than "insert/delete rule at position x".) Well, these are the ones that come to mind, atm, but I don't think there's much beyond them. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Kevin, > Of course all of this is from the perspective of Python users. Of > course, you have your own features that you want from your end (from > PostgreSQL's perspective). Perhaps this info would help you to know > which avenue to pursue. That's invaluable. Thanks for chiming in! --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Confusion over Python drivers
I saw this on reddit and thought I might drop a line. I went through this same issue, trying to find a postgresql driver. Mostly, I had no intention of using psycopg because of its copyleft licensing. (Of course, I don't need to go into why.) Anyways, here's some info that might help on three alternatives: Py-postgresql, PyGreSQL, and PG3000 Py-postgresql is active (but I only see 1 person committing) MIT/BSD style license It uses some some C for speed. It has DB-API and PG-API bindings. Problem: is it Python 3 only! PyGreSQL (looking to not be very active in the last few months) MIT/BSD style license I'm vague on it's features, whether it has C optimizations, or if it is a version limited to Python 3 or 2.x PG8000 is active at the moment MIT/BSD style license It works on 2.x and Python 3 Problem: it is pure Python, so it is slower If you plan on pursuing this, I'd recommend talking to people on on 3 projects. Here's what I would describe as ideal in a project: * MIT/BSD style license * works on Python 2.x and 3 * C optimizations for speed (or whatever for speed) -- That's that part. Now for some problems that would need solving in order for the db api to be extremely useful/userfriendly in the Python world: -- * SQLAlchemy bindings/support -- as of right now only psycopyg works in SQLAlchemy without problems. PG8000 will work, but you need an experimental version of boht PG8000 and SQLAlchemy. The other two do not work. SQLAlchemy support is probably most needed of anything. SQLAlchemy support should not only be possible but default. * Django bindings (these are unique to Django and nobody but psycopg has them, which is not good). Django support should not only be possible but default. * Needs 2.x and Python 3 compatability * It is inevitable that if the library uses C optimizations, then that means they need to provide lots of different binaries -- think 32bit vs 64bit and then 2.4, 2.5, 3.0 etc Some combinations will often be left out making it hard for someone to use/install it. Of course all of this is from the perspective of Python users. Of course, you have your own features that you want from your end (from PostgreSQL's perspective). Perhaps this info would help you to know which avenue to pursue. _ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. http://clk.atdmt.com/GBL/go/201469229/direct/01/
Re: [HACKERS] Confusion over Python drivers
On Sat, Feb 6, 2010 at 7:48 PM, Marko Kreen wrote: > This is long-term todo item for psycopg, seems offtopic > to the "driver situation". [...] > This is routine bug in either app or psycopg, we have no reason > to touch it. The guy should report to appropriate lists. [...] > Long-term todo item for psycopg2, offtopic for "driver situation". [...] > psycopg2 has array support, I'd like to have tuple/record also. > > Minor todo item for psycopg, mostly but not completely offtopic > for "driver situation". I'm not a Python user myself, but I have trouble understanding how you can describe bugs in one of the Python drivers as off-topic to the Python driver situation. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
> Finally, I just don't see the existing (often PG specific) goals that I have > in mind for it appealing to the majority of [web framework/abstraction] users. What are those goals? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On 2/7/10, Greg Smith wrote: > Marko Kreen wrote: > > Psycopg was the leader, especially in web-environments, > > but it has non-obvious license and with dead website it does not > > seem that attractive. Although it is well-maintained still. > > Best path forward would be to talk with Psycopg guys about > > license clarification/change. > > Agreed. A relicensed Psycopg, with a more professional looking > introduction/documentation page (like the budding page on our Wiki) than > what the initd web site has, seems like the best platform to hack on top of > to me as well. The fact that they've moved to git recently makes it that > much easier for another branch to exist even outside of their somewhat > troubled infrastructure. > > To summarize what I saw on this thread, the primary wishlist of changes to > it are: > > -License change Note that as long they use non-BSD licence, they need the exception for the horror that is the OpenSSL licence. So you cannot 100% satisfy the one-click-lawyers. > -Consider refactoring to better follow standard driver practices, such as > using PQExecParams This is long-term todo item for psycopg, seems offtopic to the "driver situation". > -Improvement in transaction control to resolve issues that cause idle > transactions This is routine bug in either app or psycopg, we have no reason to touch it. The guy should report to appropriate lists. > -Possible simplifications in how it's implemented async operations, to > improve robustness/reduce code complexity Long-term todo item for psycopg2, offtopic for "driver situation". > -Confirm/add multi-threaded support. Seems psycopg2 already has good enough threading. > -Confirm/add support for the most common standard types (such as array) psycopg2 has array support, I'd like to have tuple/record also. Minor todo item for psycopg, mostly but not completely offtopic for "driver situation". > > PyGreSQL is the oldest, older than DB-API, and so it's DB-API > > interface seems an afterthought and is untested/underused - eg. > > it does not support bytea. > > > > > And if Psycopg can't be relicensed happily and/or improved as above, as the > only other native Python driver PyGreSQL looks like the next candidate to > build on top of. Its major issues are: > > -Test/complete/refactor for full DB-API 2.0 support > -Add bytea support timestamp/timestamptz also... > -Add extension support, perhaps modeled on what Psycopg. > -Build a COPY extension They seems to have the low-level parts, just they are missing from DB-API interface. > -Confirm/add multi-threaded support > -Confirm/add support for the most common standard types (such as array) > > Any other suggestions before I turn the above into a roadmap page on the > wiki? I think we should concentrate on the PR problem and technical issues related to that, keep the other low-level and non-user-visible issues out. Or at least separate. (PsycopgTodo wiki page?) -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Greg Smith wrote: > To summarize what I saw on this thread, the primary wishlist of changes > to it are: > > -License change > -Consider refactoring to better follow standard driver practices, such > as using PQExecParams > -Improvement in transaction control to resolve issues that cause idle > transactions > -Possible simplifications in how it's implemented async operations, to > improve robustness/reduce code complexity > -Confirm/add multi-threaded support > -Confirm/add support for the most common standard types (such as array) > > > PyGreSQL is the oldest, older than DB-API, and so it's DB-API > > interface seems an afterthought and is untested/underused - eg. > > it does not support bytea. > > > And if Psycopg can't be relicensed happily and/or improved as above, as > the only other native Python driver PyGreSQL looks like the next > candidate to build on top of. Its major issues are: > > -Test/complete/refactor for full DB-API 2.0 support > -Add bytea support > -Add extension support, perhaps modeled on what Psycopg. > -Build a COPY extension > -Confirm/add multi-threaded support > -Confirm/add support for the most common standard types (such as array) > > Any other suggestions before I turn the above into a roadmap page on the > wiki? Great summary! This is what I was hoping for. One suggestion on the license issue is that the LGPL seems like the type of license intended by the author, and I think it is an acceptable license to most client application programmers. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Marko Kreen wrote: The pg8000 / bpgsql seem to be toy projects, and anyway you dont want to use pure-Python drivers in high-performance environments. We are not talking about C#/java here. Right, and the comments from James reinforce this general idea: there is little value to the people who most want Python+PostgreSQL support in working on any of the pure Python implementations, because best case performance is still half or less of the ones that more directly wrap libpq. Even the best case with psycopg is enough of a performance hit as it is. py-postgresql seems to be more serious, but as it's python3 only which makes it irrelevant today. Also true. Psycopg was the leader, especially in web-environments, but it has non-obvious license and with dead website it does not seem that attractive. Although it is well-maintained still. Best path forward would be to talk with Psycopg guys about license clarification/change. Agreed. A relicensed Psycopg, with a more professional looking introduction/documentation page (like the budding page on our Wiki) than what the initd web site has, seems like the best platform to hack on top of to me as well. The fact that they've moved to git recently makes it that much easier for another branch to exist even outside of their somewhat troubled infrastructure. To summarize what I saw on this thread, the primary wishlist of changes to it are: -License change -Consider refactoring to better follow standard driver practices, such as using PQExecParams -Improvement in transaction control to resolve issues that cause idle transactions -Possible simplifications in how it's implemented async operations, to improve robustness/reduce code complexity -Confirm/add multi-threaded support -Confirm/add support for the most common standard types (such as array) PyGreSQL is the oldest, older than DB-API, and so it's DB-API interface seems an afterthought and is untested/underused - eg. it does not support bytea. And if Psycopg can't be relicensed happily and/or improved as above, as the only other native Python driver PyGreSQL looks like the next candidate to build on top of. Its major issues are: -Test/complete/refactor for full DB-API 2.0 support -Add bytea support -Add extension support, perhaps modeled on what Psycopg. -Build a COPY extension -Confirm/add multi-threaded support -Confirm/add support for the most common standard types (such as array) Any other suggestions before I turn the above into a roadmap page on the wiki? -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [GENERAL] FM format modifier does not remove leading zero from year
Guy Rouillier wrote:
> On 1/6/2010 3:29 PM, Tom Lane wrote:
> > Guy Rouillier writes:
> >> Oracle states clearly in the SQL Reference manual:
> >
> >> "A modifier can appear in a format model more than once. In such a case,
> >> each subsequent occurrence toggles the effects of the modifier."
> >
> > *Toggles* the effect of the modifier? Egad, what drunken idiot chose
> > that specification?
>
> Eh, tomato, tomahto. If you assume that someone will strip leading
> zeroes consistently, the Oracle approach makes sense. That would be a
> reasonable assumption to make; why would I strip the zero off the month
> but leave it on the day? So, in the unusual case that you want to do
> such a thing, you are asked to use a second occurrence of FM to turn
> zero suppression back off.
I have developed the attached patch which implements FM control of YYY,
YY, and Y specifications. I also documented that we do not match
Oracle's toggle behavior. There are a few effects on regression test
output which are part of this patch.
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/func.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/func.sgml,v
retrieving revision 1.500
diff -c -c -r1.500 func.sgml
*** doc/src/sgml/func.sgml 1 Feb 2010 15:38:21 - 1.500
--- doc/src/sgml/func.sgml 6 Feb 2010 21:14:41 -
***
*** 5174,5180
FM suppresses leading zeroes and trailing blanks
that would otherwise be added to make the output of a pattern be
!fixed-width.
--- 5174,5184
FM suppresses leading zeroes and trailing blanks
that would otherwise be added to make the output of a pattern be
!fixed-width. In PostgreSQL,
!FM modifies only the next specification, while in
!Oracle FM affects all subsequent
!specifications, and repeated FM modifiers
!toggle fill mode on and off.
Index: src/backend/utils/adt/formatting.c
===
RCS file: /cvsroot/pgsql/src/backend/utils/adt/formatting.c,v
retrieving revision 1.162
diff -c -c -r1.162 formatting.c
*** src/backend/utils/adt/formatting.c 2 Jan 2010 16:57:53 - 1.162
--- src/backend/utils/adt/formatting.c 6 Feb 2010 21:14:42 -
***
*** 515,520
--- 515,521
#define S_th(_s) (((_s) & DCH_S_th) ? 1 : 0)
#define S_TH_TYPE(_s) (((_s) & DCH_S_TH) ? TH_UPPER : TH_LOWER)
+ /* Oracle toggles FM behavior, we don't; see docs. */
#define S_FM(_s) (((_s) & DCH_S_FM) ? 1 : 0)
#define S_SP(_s) (((_s) & DCH_S_SP) ? 1 : 0)
#define S_TM(_s) (((_s) & DCH_S_TM) ? 1 : 0)
***
*** 2411,2438
break;
case DCH_YYY:
case DCH_IYY:
! snprintf(buff, sizeof(buff), "%03d",
n->key->id == DCH_YYY ?
ADJUST_YEAR(tm->tm_year, is_interval) :
ADJUST_YEAR(date2isoyear(tm->tm_year,
tm->tm_mon, tm->tm_mday),
is_interval));
i = strlen(buff);
! strcpy(s, buff + (i - 3));
if (S_THth(n->suffix))
str_numth(s, s, S_TH_TYPE(n->suffix));
s += strlen(s);
break;
case DCH_YY:
case DCH_IY:
! snprintf(buff, sizeof(buff), "%02d",
n->key->id == DCH_YY ?
ADJUST_YEAR(tm->tm_year, is_interval) :
ADJUST_YEAR(date2isoyear(tm->tm_year,
tm->tm_mon, tm->tm_mday),
is_interval));
i = strlen(buff);
! strcpy(s, buff + (i - 2));
if (S_THth(n->suffix))
str_numth(s, s, S_TH_TYPE(n->suffix));
s += strlen(s);
--- 2412,2441
break;
case DCH_YYY:
case DCH_IYY:
! snprintf(buff, sizeof(buff), "%0*d",
! S_FM(n->suffix) ? 0 : 3,
n->key->id == DCH_YYY ?
ADJUST_YEAR(tm->tm_year, is_interval) :
ADJUST_YEAR(date2isoyear(tm->tm_year,
tm->tm_mon, tm->tm_mday),
is_interval));
i = strlen(buff);
! strcpy(s, buff + (i > 3 ? i - 3 : 0));
if (S_THth(n->suffix))
str_numth(s, s, S_TH_TYPE(n->suffix));
s += strlen(s);
break;
case DCH_YY:
case DCH_IY:
! snprintf(buff, sizeof(buff), "%0*d",
! S_FM(n->suffix) ? 0 : 2,
n->key->id == DCH_YY ?
ADJUST_YEAR(tm->tm_year, is_interval) :
ADJUST_YEAR(date2isoyear(tm->tm_year,
tm->tm_mon, tm->tm_mday),
is_interval));
i = strlen(buff);
! strcpy(s, buff + (i > 2 ? i - 2 : 0));
if (S_THth(n->suffix))
str_numth(s, s, S_TH_TYPE(n->suffix));
s += strlen(s);
***
*** 2446,2452
tm->tm_mon, tm->tm_mday),
is_interval));
i = strlen(buff);
! strcpy(s, buff + (i - 1));
if (S_THth(n->suffi
Re: [HACKERS] remove contrib/xml2
M Z writes: > The thing is, why it doesn't crash on 8.3.8 but crash on 8.4.2? Any idea? Pure luck. Memory-clobber bugs like these are notoriously nondeterministic. Any minor, logically unrelated change could make them visible or not visible, because the clobber happens to clobber data that is or is not in active use at the moment. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql tab completion for DO blocks
Robert Haas wrote:
> On Sat, Jan 2, 2010 at 7:47 PM, Tom Lane wrote:
> > Peter Eisentraut writes:
> >> On l?r, 2010-01-02 at 17:34 -0500, Tom Lane wrote:
> >>> As for the overhead, these queries are not zero-maintenance. ?I still
> >>> think that the usefulness of tab completion here is pretty darn
> >>> minimal,
> >>> since most people are more likely to rely on default_do_language;
> >
> >> We really don't have any data on that, and it doesn't seem all that
> >> likely to me.
> >
> > I'm not really objecting to putting in the patch entirely. ?I'm objecting
> > to carrying an extra completion query for it. ?I don't think hiding
> > languages with laninline=0 improves its usefulness at all, let alone
> > enough to justify extra maintenance burden.
>
> As a practical matter there aren't that many languages in the first
> place, and many of them begin with the same few letters. If you have
> both plperl and plpython loaded (and they both have inline handlers)
> you'll have to type four characters to disambiguate, and by that time
> (especially for plperl) you might as well just finish typing it by
> hand.
>
> Having said that, I don't see much value in deliberately making the
> tab-completion list fail to match the set of arguments that will
> actually work. The maintenance burden of an additional query strikes
> me as not worth worrying about. If David finds it useful and/or has
> users who want it, I think we should just do it.
Where are we on this patch? We should at least implement the completion
for 'LANGUAGE' in 'DO', and use the existing pg_language query for
completion. I am attaching a patch that does exactly this.
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: src/bin/psql/tab-complete.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/tab-complete.c,v
retrieving revision 1.192
diff -c -c -r1.192 tab-complete.c
*** src/bin/psql/tab-complete.c 25 Jan 2010 18:23:09 - 1.192
--- src/bin/psql/tab-complete.c 6 Feb 2010 19:09:26 -
***
*** 618,624
static const char *const sql_commands[] = {
"ABORT", "ALTER", "ANALYZE", "BEGIN", "CHECKPOINT", "CLOSE", "CLUSTER",
"COMMENT", "COMMIT", "COPY", "CREATE", "DEALLOCATE", "DECLARE",
! "DELETE FROM", "DISCARD", "DROP", "END", "EXECUTE", "EXPLAIN", "FETCH",
"GRANT", "INSERT", "LISTEN", "LOAD", "LOCK", "MOVE", "NOTIFY", "PREPARE",
"REASSIGN", "REINDEX", "RELEASE", "RESET", "REVOKE", "ROLLBACK",
"SAVEPOINT", "SELECT", "SET", "SHOW", "START", "TABLE", "TRUNCATE", "UNLISTEN",
--- 618,624
static const char *const sql_commands[] = {
"ABORT", "ALTER", "ANALYZE", "BEGIN", "CHECKPOINT", "CLOSE", "CLUSTER",
"COMMENT", "COMMIT", "COPY", "CREATE", "DEALLOCATE", "DECLARE",
! "DELETE FROM", "DISCARD", "DO", "DROP", "END", "EXECUTE", "EXPLAIN", "FETCH",
"GRANT", "INSERT", "LISTEN", "LOAD", "LOCK", "MOVE", "NOTIFY", "PREPARE",
"REASSIGN", "REINDEX", "RELEASE", "RESET", "REVOKE", "ROLLBACK",
"SAVEPOINT", "SELECT", "SET", "SHOW", "START", "TABLE", "TRUNCATE", "UNLISTEN",
***
*** 1532,1537
--- 1532,1555
COMPLETE_WITH_LIST(list_DISCARD);
}
+ /* DO */
+ /*
+ * Complete DO with LANGUAGE.
+ */
+ else if (pg_strcasecmp(prev_wd, "DO") == 0)
+ {
+ static const char *const list_DO[] =
+ {"LANGUAGE", NULL};
+
+ COMPLETE_WITH_LIST(list_DO);
+ }
+ /*
+ * Complete DO LANGUAGE with in-line-able languages.
+ */
+ else if (pg_strcasecmp(prev2_wd, "DO") == 0 &&
+ pg_strcasecmp(prev_wd, "LANGUAGE") == 0)
+ COMPLETE_WITH_QUERY(Query_for_list_of_languages);
+
/* DROP (when not the previous word) */
/* DROP AGGREGATE */
else if (pg_strcasecmp(prev3_wd, "DROP") == 0 &&
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove contrib/xml2
The thing is, why it doesn't crash on 8.3.8 but crash on 8.4.2? Any idea? A
patch was applied to 8.3 but not to 8.4.2?
Thanks,
M Z
On Fri, Feb 5, 2010 at 1:45 PM, Robert Haas wrote:
> On Wed, Feb 3, 2010 at 8:49 AM, Alvaro Herrera
> wrote:
> > Robert Haas escribió:
> >> On Mon, Feb 1, 2010 at 5:23 PM, Andrew Dunstan
> wrote:
> >> > Robert Haas wrote:
> >> >> (2) add a very, very large warning that this will crash if you do
> >> >> almost anything with it.
> >> >
> >> > I think that's an exaggeration. Certain people are known to be using
> it
> >> > quite successfully.
> >>
> >> Hmm. Well, all I know is that the first thing I tried crashed the
> server.
> >>
> >> CREATE TABLE xpath_test (id integer NOT NULL, t xml);
> >> INSERT INTO xpath_test VALUES (1, '1');
> >> SELECT * FROM xpath_table('id', 't', 'xpath_test', '/doc/int', 'true')
> >> as t(id int4);
> >
> > This trivial patch lingering on my system fixes this crasher (this is
> > for the 8.3 branch). It makes the "problem in alloc set ExprContext"
> > warning show up instead.
> >
> > There are still lotsa other holes, but hey, this is a start ...
>
> Interestingly M Z found he couldn't reproduce this crash on 8.3. Can
> you? If so, +1 for applying this and backpatching it as far as make
> sense.
>
> ...Robert
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
Re: [HACKERS] pg_class has no toast table?
I wrote: > Still fooling with VACUUM FULL on catalogs ... I find that a sanity > check I put in is barfing on "VACUUM FULL pg_class", because the > transient table is built with a toast table, whereas pg_class hasn't got > one. It seems like it probably ought to have one, because either relacl > or reloptions could in principle be too big to fit without toasting > (which is exactly why AlterTableCreateToastTable thinks it should make > one for the transient table). > I have a vague feeling that we intentionally omitted a toast table for > pg_class, but I don't remember why. Comments? BTW, I decided not to touch this issue in the current patch --- it turns out there are quite a few system catalogs that lack a toast table but AlterTableCreateToastTable's rules would say to create one. The one that made me stop adding them was pg_largeobject. That has a bytea column but there are usage rules that limit the possible width of the column, and of course AlterTableCreateToastTable doesn't know that. So I tweaked the CLUSTER/VAC FULL logic to never add a toast table if the original table hasn't got one. (It can still *remove* a toast table, as might happen after dropping a wide column for instance.) We might still want to consider toast-ifying pg_class if anyone ever complains about not having room for wide relacl values; but CLUSTER shouldn't be a forcing function for such decisions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql 8.4 \c repeats version banner
Peter Eisentraut wrote:
> In 8.3, running \c from a file prints something like
>
> You are now connected to database "postgres".
>
> In 8.4 it prints
>
> psql (8.4.1)
> You are now connected to database "postgres".
>
> Is it intentional/sensible to repeat the startup banner every time the
> connection changes, or was this unintentionally introduced while the
> startup banner was reshuffled in 8.4?
I did some reseach on this. I bet this behavior was added when we
decided to print the backend version warning banner on \c as well as
startup, because it is possible for the backend to be different version
from the backend originally used for psql startup. The code that prints
the psql banner and the warning banner are in the same function and
share the same output line.
What I did in the attached patch is to add a boolean to
connection_warnings() to indicate whether it was being called on psql
startup or via \c, and to supress the psql banner on \c if the client
and server versions match:
$ psql test
psql (8.5devel)
Type "help" for help.
test=> \c test
You are now connected to database "test".
test=>
Any version mismatch will still print the psql banner for \c, which is
what I think we want.
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: src/bin/psql/command.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/command.c,v
retrieving revision 1.214
diff -c -c -r1.214 command.c
*** src/bin/psql/command.c 5 Feb 2010 03:09:05 - 1.214
--- src/bin/psql/command.c 6 Feb 2010 17:22:47 -
***
*** 1331,1337
PQsetNoticeProcessor(n_conn, NoticeProcessor, NULL);
pset.db = n_conn;
SyncVariables();
! connection_warnings(); /* Must be after SyncVariables */
/* Tell the user about the new connection */
if (!pset.quiet)
--- 1331,1337
PQsetNoticeProcessor(n_conn, NoticeProcessor, NULL);
pset.db = n_conn;
SyncVariables();
! connection_warnings(false); /* Must be after SyncVariables */
/* Tell the user about the new connection */
if (!pset.quiet)
***
*** 1357,1363
void
! connection_warnings(void)
{
if (!pset.quiet && !pset.notty)
{
--- 1357,1363
void
! connection_warnings(bool in_startup)
{
if (!pset.quiet && !pset.notty)
{
***
*** 1383,1389
printf(_("%s (%s, server %s)\n"),
pset.progname, PG_VERSION, server_version);
}
! else
printf("%s (%s)\n", pset.progname, PG_VERSION);
if (pset.sversion / 100 != client_ver / 100)
--- 1383,1390
printf(_("%s (%s, server %s)\n"),
pset.progname, PG_VERSION, server_version);
}
! /* For version match, only print psql banner on startup. */
! else if (in_startup)
printf("%s (%s)\n", pset.progname, PG_VERSION);
if (pset.sversion / 100 != client_ver / 100)
Index: src/bin/psql/command.h
===
RCS file: /cvsroot/pgsql/src/bin/psql/command.h,v
retrieving revision 1.33
diff -c -c -r1.33 command.h
*** src/bin/psql/command.h 2 Jan 2010 16:57:59 - 1.33
--- src/bin/psql/command.h 6 Feb 2010 17:22:47 -
***
*** 34,40
printQueryOpt *popt,
bool quiet);
! extern void connection_warnings(void);
extern void SyncVariables(void);
--- 34,40
printQueryOpt *popt,
bool quiet);
! extern void connection_warnings(bool in_startup);
extern void SyncVariables(void);
Index: src/bin/psql/startup.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/startup.c,v
retrieving revision 1.160
diff -c -c -r1.160 startup.c
*** src/bin/psql/startup.c 5 Feb 2010 03:09:05 - 1.160
--- src/bin/psql/startup.c 6 Feb 2010 17:22:47 -
***
*** 294,300
if (!options.no_psqlrc)
process_psqlrc(argv[0]);
! connection_warnings();
if (!pset.quiet && !pset.notty)
printf(_("Type \"help\" for help.\n\n"));
if (!pset.notty)
--- 294,300
if (!options.no_psqlrc)
process_psqlrc(argv[0]);
! connection_warnings(true);
if (!pset.quiet && !pset.notty)
printf(_("Type \"help\" for help.\n\n"));
if (!pset.notty)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby and DROP DATABASE
On Saturday 06 February 2010 17:32:43 Andres Freund wrote: > On Saturday 06 February 2010 02:25:33 Tatsuo Ishii wrote: > > Hi, > > > > While testing Hot Standby, I have encountered strange behavior with > > DROP DATABASE command. > > > > 1) connect to "test" database at standby via psql > > 2) issue DROP DATABASE test command to primary > > 3) session #1 works fine > > 4) close session #1 > > 5) "test" database dropped on standby > > > > Fromt the manual: > > Running DROP DATABASE, ALTER DATABASE ... SET TABLESPACE, or ALTER > > DATABASE ... RENAME on primary will generate a log message that will > > cause all users connected to that database on the standby to be > > forcibly disconnected. This action occurs immediately, whatever the > > setting of max_standby_delay. > > > > So it seems at least the behavior is quite different from what the > > docs stats. Am I missing something here? > > Its a small bug/typo in standby.c:ResolveRecoveryConflictWithDatabase > > The line: > CancelDBBackends(dbid, PROCSIG_RECOVERY_CONFLICT_TABLESPACE, > true); For the case it should not be clear, the reason that PROCSIG_RECOVERY_CONFLICT_TABLESPACE did not kill the session is that currently all tablespace conflicts are valid only inside a transaction, so when receiving the recovery conflict checks whether its not inside a transaction block "anymore" and continues happily. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby and DROP DATABASE
On Saturday 06 February 2010 02:25:33 Tatsuo Ishii wrote: > Hi, > > While testing Hot Standby, I have encountered strange behavior with > DROP DATABASE command. > > 1) connect to "test" database at standby via psql > 2) issue DROP DATABASE test command to primary > 3) session #1 works fine > 4) close session #1 > 5) "test" database dropped on standby > > Fromt the manual: > > Running DROP DATABASE, ALTER DATABASE ... SET TABLESPACE, or ALTER > DATABASE ... RENAME on primary will generate a log message that will > cause all users connected to that database on the standby to be > forcibly disconnected. This action occurs immediately, whatever the > setting of max_standby_delay. > > So it seems at least the behavior is quite different from what the > docs stats. Am I missing something here? Its a small bug/typo in standby.c:ResolveRecoveryConflictWithDatabase The line: CancelDBBackends(dbid, PROCSIG_RECOVERY_CONFLICT_TABLESPACE, true); has to be CancelDBBackends(dbid, PROCSIG_RECOVERY_CONFLICT_DATABASE, true); Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backup history file should be replicated in Streaming Replication?
Fujii Masao wrote:
> On Mon, Jan 4, 2010 at 9:55 PM, Heikki Linnakangas
> wrote:
> > Heikki Linnakangas wrote:
> >> Here's is modified patch that adds a new backupStartPoint field to
> >> pg_control for that + some other minor editorialization.
> >
> > I've committed this now.
>
> Thanks a lot!
>
> src/backend/access/transam/xlog.c
> > else
> > {
> > XLogRecPtr InvalidXLogRecPtr = {0, 0};
> > ControlFile->minRecoveryPoint = InvalidXLogRecPtr;
> > }
>
> In my original patch, the above is for the problem discussed in
> http://archives.postgresql.org/pgsql-hackers/2009-12/msg02039.php
>
> Since you've already fixed the problem, that code is useless.
> How about getting rid of that code?
Has this been addressed?
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] archive_timeout behavior for no activity
Kevin Grittner wrote: > [resend, because of apparent failure to hit the list] > > Bruce Momjian wrote: > > > I am dismayed that we are using a 16MB file for monitoring archive > > activity. Can't you use pg_current_xlog_location() and only check > > for an archive file when that location changes? > > Hmmm Let me think about that. The intent was to check the > end-to-end health of the PITR backups. The current process ensures > that the archive command is working, the crontab scripts to copy the > files are working (they get copied from the database server to > multiple locations), and that the one copied to our central location > applies cleanly to a warm standby (thereby providing confirmation of > the health of that process). I'd have to think about how much we > would lose with the change you suggest, and how much we'd care about > that. I guess in a pinch we could always use a crontab job to force > something to the WAL files periodically, but now that the system is > proven and "settled in", perhaps constant validation of some of > those points is overkill. > > By the way a near-empty WAL file is only 16KB by the time > pg_clearxlogtail and gzip get done chewing on it, and we have a > parallel stream of data from our application which allows us to keep > the archive frequency to once per hour. Not everyone is going to be > in this position, though, so I can understand the motivation to > change it. > > > Is there a TODO here? > > Well, if I'm the only one who likes the status quo, I'm not sure > that should preclude a change that would benefit others. I'm sure > we can code around it one way or another. Perhaps some of the new > monitoring functions in 9.0 will help. I'll have to take a look. > > Seriously, if there would be significant benefit to others, don't > let me be the spoiler here; we'll sort it out. Seems there is enough concern about the existing behavior that I have added a TODO item: Consider avoiding WAL switching via archive_timeout if there has been no database activity * http://archives.postgresql.org/pgsql-hackers/2010-01/msg01469.php * http://archives.postgresql.org/pgsql-hackers/2010-02/msg00395.php -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Saturday 06 February 2010 06:03:30 Greg Smith wrote: > Andres Freund wrote: > > On 02/03/10 14:42, Robert Haas wrote: > >> Well, maybe we should start with a discussion of what kernel calls > >> you're aware of on different platforms and then we could try to put an > >> API around it. > > > > In linux there is sync_file_range. On newer Posixish systems one can > > emulate that with mmap() and msync() (in batches obviously). > > > > No idea about windows. > The effective_io_concurrency feature had proof of concept test programs > that worked using AIO, but actually following through on that > implementation would require a major restructuring of how the database > interacts with the OS in terms of reads and writes of blocks. It looks > to me like doing something similar to sync_file_range on Windows would > be similarly difficult. Looking a bit arround it seems one could achieve something approximediately similar to pg_prepare_fsync() by using CreateFileMapping && MapViewOfFile && FlushViewOfFile If I understand it correctly that will flush, but not wait. Unfortunately you cant event make it wait, so its not possible to implement sync_file_range or similar fully. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] archive_timeout behavior for no activity
[resend, because of apparent failure to hit the list] Bruce Momjian wrote: > I am dismayed that we are using a 16MB file for monitoring archive > activity. Can't you use pg_current_xlog_location() and only check > for an archive file when that location changes? Hmmm Let me think about that. The intent was to check the end-to-end health of the PITR backups. The current process ensures that the archive command is working, the crontab scripts to copy the files are working (they get copied from the database server to multiple locations), and that the one copied to our central location applies cleanly to a warm standby (thereby providing confirmation of the health of that process). I'd have to think about how much we would lose with the change you suggest, and how much we'd care about that. I guess in a pinch we could always use a crontab job to force something to the WAL files periodically, but now that the system is proven and "settled in", perhaps constant validation of some of those points is overkill. By the way a near-empty WAL file is only 16KB by the time pg_clearxlogtail and gzip get done chewing on it, and we have a parallel stream of data from our application which allows us to keep the archive frequency to once per hour. Not everyone is going to be in this position, though, so I can understand the motivation to change it. > Is there a TODO here? Well, if I'm the only one who likes the status quo, I'm not sure that should preclude a change that would benefit others. I'm sure we can code around it one way or another. Perhaps some of the new monitoring functions in 9.0 will help. I'll have to take a look. Seriously, if there would be significant benefit to others, don't let me be the spoiler here; we'll sort it out. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch for system-catalog vacuuming via a relation map
Tom Lane wrote: > Josh Berkus writes: >>> Barring objections I'm going to press ahead with completing and >>> committing this; then in a separate patch remove VACUUM FULL INPLACE. > >> Was it our determination that we could remove VFI if we eliminated the >> system catalogs? I'm fine with it, I just thought some people had a >> marginal use case for VFI. > > I thought the consensus was to remove it if possible. There may still > be some "marginal" use cases, but they don't justify the work that'd > be needed to make it play safely with HS; let alone fixing the other > longstanding gotchas with it, like the double-commit risk. I think part of the plan was to also provide an online reorg tool that works by doing dummy UPDATEs, which means that you can get serialization errors in serializable mode, but doesn't need to lock the table. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On Fri, 2010-02-05 at 10:35 -0800, Josh Berkus wrote: > I'm not as concerned about "confusion" as the fact that *all* of the > various Python drivers suck in different, and crippling, ways. I don't > care how many drivers we have, as long as we have at least one 1st-class > driver. Absolutely. And I would prefer that it can provide all (or nearly all) of the capabilities of libpq. PyGreSQL apparently doesn't even offer parameterized queries! http://pygresql.org/pg.html That was based on a quick glance at the document; I hope I'm mistaken. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
