Re: [HACKERS] Encoding issues in console and eventlog on win32
2009/10/7 Itagaki Takahiro :
>
> Magnus Hagander wrote:
>> Per your own comments earlier, and in the code, what will happen if
>> pg_do_encoding_conversion() calls ereport()? Didn't you say we need a
>> non-throwing version of it?
>
> We are hard to use encoding conversion functions in logging routines
> because they could throw errors if there are some unconvertable characters.
> Non-throwing version will convert such characters into '?' or escaped form
> (something like \888 or \xFF). If there where such infrastructure, we can
> support "log_encoding" settings and convert messages in platform-dependent
> encoding before writing to syslog or console.
Right, which we don't have at this point. That would be very useful on
unix, i believe.
>> pgwin32_toUTF16() needs error checking on the API calls, and needs to
>> do something reasonable if it fails.
>
> Now it returns NULL and caller writes messages in the original encoding.
Seems reasonable. If encoding fails, I think that's the best we can do.
> Also I added the following error checks before calling pgwin32_toUTF16()
> (errordata_stack_depth < ERRORDATA_STACK_SIZE - 1)
> to avoid recursive errors, but I'm not sure it is really meaningful.
> Please remove or rewrite this part if it is not a right way.
I'm not entirely sure either, but it looks like it could protect us
from getting into a tight loop on an error here.. Tom (or someone else
who knows that for sure :P),comments?
>> The encoding_to_codepage array needs to go in encnames.c, where other
>> such tables are. Perhaps it can even be integrated in pg_enc2name_tbl
>> as a separate field?
>
> I added pg_enc2name.codepage. Note that this field is needed only
> on Windows, but now exported for all platforms. If you don't like
> the useless field, the following macro could be a help.
> #ifdef WIN32
> #define def_enc2name(name, codepage) { #name, PG_##name, codepage }
> #else
> #define def_enc2name(name, codepage) { #name, PG_##name }
> #endif
> pg_enc2name pg_enc2name_tbl[] =
> {
> def_enc2name(SQL_ASCII),
> def_enc2name(EUC_JP),
> ...
Yeah, I think that makes sense. It's not much data, but it's
completely unnecessary :-) I can make that change at commit.
One other question - you note that WriteConsoleW() "could fail if
stderr is redirected". Are you saying that it will always fail when
stderr is redirected, or only sometimes? If ony sometimes, do you know
under which conditions it happens?
If it's always, I assume this just means that the logfile will be in
the database encoding and not in UTF16? Is this what we want, or would
we like the logfile to also be in UTF16? If we can convert it to
UTF16, that would fix the case when you have different databases in
different encodings, wouldn't it? (Even if your editor, unlike the
console subsystem, can view the individual encoding you need, I bet it
can't deal with multiple encodings in the same file)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Using results from INSERT ... RETURNING
Marko Tiikkaja writes: > Tom Lane wrote: >> Could you pull out a patch that includes those changes, please? > Sorry for the delay, my master was a bit behind :-( . I moved the > trigger code to nodeDml.c with minor changes and removed unused > resultRelation stuff from DML nodes completely. This also has the > README stuff in it. Applied with a moderate amount of editorialization. Notably, I didn't like what you'd done with the EvalPlanQual stuff, and after a bit of reflection decided that the right thing was to teach EvalPlanQual to execute just the desired subplan. Also, I put back the marking of the ModifyTuple node with its target relations, which you'd removed in the v4 patch --- I'm convinced that that will be necessary in some form or other later, so taking it out now seemed like moving backward. I did not do anything about changing EXPLAIN's output of trigger information. The stumbling block there is that EXPLAIN executes queued AFTER trigger events only after finishing the main plan tree execution. The part of that that is driven by ModifyTable is just the *queuing* of the triggers, not their *execution*. So my previous claim that all the trigger execution would now be part of ModifyTable was wrong. There are several things we could do here: 1. Nothing. I don't care for this, though, because it will lead to the inconsistent behavior that BEFORE triggers count as part of the displayed runtime for ModifyTuple and AFTER triggers don't. 2. Move actual execution of (non-deferred) AFTER triggers inside ModifyTuple. This might be a good idea in order to have the most consistent results for a series of WITH queries, but I'm not sure. 3. Have EXPLAIN show BEFORE triggers as associated with ModifyTuple while still showing AFTER triggers as "free standing". Seems a bit inconsistent. Comments? Also, working on this patch made me really want to pull SELECT FOR UPDATE/SHARE locking out as a separate node too. We've talked about that before but never got round to it. It's really necessary to do that in order to have something like INSERT INTO foo SELECT * FROM bar FOR UPDATE; behave sanely. I don't recall at the moment whether that worked sanely before, but it is definitely broken as of CVS tip. Perhaps I'll work on that this weekend. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idle connection timeout
2009/10/10 Alvaro Herrera > > > Our TODO has: > > > > > >Add idle_in_transaction_timeout GUC so locks are not held for > > >long periods of time > > > > Awesome! Thanks for the info Bruce. > > This is quite a different thing though. You were talking about idle > connections; the TODO item is about idle-in-transaction connections. > (Also, keep in mind that something being in the TODO list does not mean > anyone is planning to work on it.) > > Oh, I see. Yes, that is different. In which case, I'd still like to see such a feature implemented as I imagine it could be useful for auto-killing connections not being used that are continuously taking some of the resources. Thom
Re: [HACKERS] Concurrency testing
On Fri, Oct 09, 2009 at 08:34:55PM +0200, Markus Schiltknecht wrote: > Hi, > > David Fetter wrote: >> 1. Test situations which require more than one connection, possibly >> to different clusters, for example in the cases of Hot Standby and >> Streaming Replication. This further divides into event-based and >> time-based tests. It's this situation I had in mind when I posted, >> and it was inspired by bugs I've seen, the most recent being in >> writeable CTEs. > > Hm.. not sure what you mean by time-base tests, Kick off x. Wait time t1 Kick off y. Wait time t2. etc., etc. > but to me that distinction sounds like regression vs. performance > testing. Or do you think of time-based regression tests? I > certainly favor regression tests that are event-based, as I've run > into too many false errors due to unexpected timings already. Totally agree about regression tests. There might be other kinds of test--stress- or performance tests, for example--that would make more sense time-based. >> 2. Test situations which require *many* connections in order to >> find issues caused, in essence, by load. >> >> Tsung seems well-suited to the latter. > > Agreed. I would also note here, that Tsung seems especially well > suited for performance testing of clustered solutions, as you very > likely have to cluster the testing agents as well to put a decent > load on the SUT. I'd *love* to see PostgreSQL solutions that would outrun a single decent Tsung box, but I suspect that's at least a year or two away. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Looking for funding
I am looking for funding for two Postgres community projects: 1) enhancements to full text search and GiST 2) proofreading of our documentation I have people ready to work on these, and the items would be completed for Postgres 8.5. If you or your company are interested in funding either of these, please contact me privately. I will handle the funding transfer details, and payment would not be necessary until the items are completed. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idle connection timeout
Thom Brown escribió: > 2009/10/9 Bruce Momjian > > > Thom Brown wrote: > > > I saw a recent blog entry where someone came up with a solution > > > for expiring idle connections ( > > > > > http://dividebyzeroexception.blogspot.com/2009/10/implementing-waittimeout-in-postgresql.html > > ) > > > and I was surprised there wasn't a config option for this already. > > > Is this something that can be considered for a future release? > > > > Our TODO has: > > > >Add idle_in_transaction_timeout GUC so locks are not held for > >long periods of time > > Awesome! Thanks for the info Bruce. This is quite a different thing though. You were talking about idle connections; the TODO item is about idle-in-transaction connections. (Also, keep in mind that something being in the TODO list does not mean anyone is planning to work on it.) -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idle connection timeout
2009/10/9 Bruce Momjian > Thom Brown wrote: > > I saw a recent blog entry where someone came up with a solution for > expiring > > idle connections ( > > > http://dividebyzeroexception.blogspot.com/2009/10/implementing-waittimeout-in-postgresql.html > ) > > and I was surprised there wasn't a config option for this already. Is > this > > something that can be considered for a future release? > > Our TODO has: > >Add idle_in_transaction_timeout GUC so locks are not held for >long periods of time > > > Awesome! Thanks for the info Bruce.
Re: [HACKERS] Idle connection timeout
Thom Brown wrote: > I saw a recent blog entry where someone came up with a solution for expiring > idle connections ( > http://dividebyzeroexception.blogspot.com/2009/10/implementing-waittimeout-in-postgresql.html) > and I was surprised there wasn't a config option for this already. Is this > something that can be considered for a future release? Our TODO has: Add idle_in_transaction_timeout GUC so locks are not held for long periods of time -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] one line comment style
Dan Colish wrote: > I'm going through a few files and trying to clean them up for style mostly > and a > bit of refactoring. I am curious about the preferred style for a one line > comment. I see them in both of these forms and I would like to keep it > consistent. > > /* a one line comment */ > > or > > /* > * a one line comment > */ I use single-line of I just want to comment the next line, and multi-line if I want to comment on the next block of lines, e.g a section comment. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unicode UTF-8 table formatting for psql text output
Peter Eisentraut wrote: > I think the setting ought be called linestyle unicode (instead of > utf8), since the same setting would presumably work in case we ever > implement UTF-16 support on the client side. Yeah, anytime one gets sloppy with the distinction between a character set and a character encoding scheme, one tends to regret it, sooner or later. Here's we're talking about which glyphs to show -- that's based on a character set. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unicode UTF-8 table formatting for psql text output
On Tue, 2009-10-06 at 19:35 +0100, Roger Leigh wrote: > This patch included a bit of code not intended for inclusion > (setting of client encoding based on locale), which the attached > (and hopefully final!) revision of the patch excludes. Well, the documentation still claims that this is dependent on the locale. This should be updated to match the code. I think the setting ought be called linestyle unicode (instead of utf8), since the same setting would presumably work in case we ever implement UTF-16 support on the client side. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
On Mon, 2009-09-14 at 18:58 +0900, Itagaki Takahiro wrote: > Itagaki Takahiro wrote: > > > Ok, the attached patch implements standard-compliant version of > > column trigger. > > Here is an updated version of column-level trigger patch. > I forgot to adjust pg_get_triggerdef() in the previous version. > pg_dump also uses pg_get_triggerdef() instead of building > CREATE TRIGGER statements to avoid duplicated codes if the > server version is 8.5 or later. I have committed the parts involving pg_get_triggerdef and pg_dump. I will get to the actual column trigger functionality next. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Writeable CTEs and side effects
Added to TODO: Allow INSERT/UPDATE/DELETE ... RETURNING in common table expressions * http://archives.postgresql.org/pgsql-hackers/2009-10/msg00472.php --- Marko Tiikkaja wrote: > I've made progress in implementing writeable CTEs (repo at > git://git.postgresql.org/git/writeable_cte.git , branch actually_write) > and I've hit a few corner-cases which have lead me to think that we > should be handling DML inside CTEs a bit differently. Before I go on > implementing this, I'd like to hear your input. > > 1)WITH t AS > (UPDATE foo SET bar = bar+1 RETURNING *) > SELECT * FROM t LIMIT 1; > > What's problematic here is that only 1 row is read from the CTE, meaning > also that only one row is updated which, at least how I see it, is not > what we want. The CTE should only store one row and return that after > it has completely processed the UPDATE statement. > > 2)WITH t1 AS > (UPDATE foo SET bar=bar+1 RETURNING *), > t2 AS > (UPDATE foo SET bar=bar+1 RETURNING *) > SELECT * FROM t1 > UNION ALL > SELECT * FROM t2; > > This is probably not the most common scenario, but is still very > surprising if you for some reason happen to hit it. Both of the updates > actually have the same transaction ID and command ID, so the rows are > first updated by t1, but when t2 is processed, it looks at the rows and > thinks that it already updated them. > > 3)WITH t1 AS > (UPDATE foo SET bar=bar+1 RETURNING *), > t2 AS > (UPDATE baz SET bat=bat+1 RETURNING *) > VALUES (true); > > This isn't probably the most common situation either, but I think it's > worth looking at; the user wants to update two different tables, but > ignore the RETURNING data completely. On IRC, this has been requested > multiple times. Even if we wouldn't agree that this feature is useful, > it pretty much follows the semantics of example #1. > > > Trying to tackle all of these at once, I've come up with this kind of > execution strategy: > > Before starting the execution of the main plan tree, for every CTE which > is a DML query, do the following: > >1) Get a new CID >2a) If there are no references to the CTE (example #3), run the DML > query to the end but ignore the results of the RETURNING query, >or >2b) If there are references, run the DML query to the end but store > either as many as rows as you need to to answer the outer query (example > #1) or if we can't determine the number of rows we need (most cases, > example #2) run the query and store all of its results. > > Then, if required, get a new CID for the main execution tree and execute > it using the data we now have inside the CTEs. This way we can avoid > storing useless rows in memory without unexpected behaviour and caveats. > > > Regards, > Marko Tiikkaja > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrency testing
Markus Wanner wrote: Hi, Dimitri Fontaine wrote: I even started a DBT2 implementation as tsung modules, back when returning from pgcon 2006: http://tapoueh.org/misc.html#sec7 darcs get http://pgsql.tapoueh.org/dbt2-tsung/ Now *that* is very cool! I definitely need to have a look at it. Have you tried Sun Faban? It allows to manage test runs, compare test results and configurations and features a web interface. Pretty nice as well. But it's all Java, which I like even less than Perl. However, as far as I know, neither of the two support downloading code from a repository and building automatically, before testing. While the buildfarm already does that (partly, testing single patches would be a nice to have, too). It does, sort of. It has a switch called --from-source which lets you specify a local source repository, which can be something you've applied a patch to. Use of this switch disables fetching the source code, checking it for cleanliness, and uploading the results to the central server. It was designed for testing patches. Here's how I use it: pushd srcroot/HEAD cp -a pgsql pgsql.thistest cd pgsql.thistest cvs update patch -p 0 < /path/to/patch_to_test popd ./run_build.pl --verbose --from-source=`pwd`/srcroot/HEAD/pgsql.thistest cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] contrib/plantuner - enable PostgreSQL planner hints
On Fri, 9 Oct 2009, Bruce Momjian wrote: Oleg Bartunov wrote: =# set enable_seqscan=off; =# set plantuner.forbid_index='id_idx2'; The genius of this module is the line above -- a more fine-grained way to control the optimizer, with specific index disabling. . Are you planning to submit this as a /contrib module? No plan yet. Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrency testing
Hi, Dimitri Fontaine wrote: I even started a DBT2 implementation as tsung modules, back when returning from pgcon 2006: http://tapoueh.org/misc.html#sec7 darcs get http://pgsql.tapoueh.org/dbt2-tsung/ Now *that* is very cool! I definitely need to have a look at it. Have you tried Sun Faban? It allows to manage test runs, compare test results and configurations and features a web interface. Pretty nice as well. But it's all Java, which I like even less than Perl. However, as far as I know, neither of the two support downloading code from a repository and building automatically, before testing. While the buildfarm already does that (partly, testing single patches would be a nice to have, too). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Concurrency testing
Hi, David Fetter wrote: 1. Test situations which require more than one connection, possibly to different clusters, for example in the cases of Hot Standby and Streaming Replication. This further divides into event-based and time-based tests. It's this situation I had in mind when I posted, and it was inspired by bugs I've seen, the most recent being in writeable CTEs. Hm.. not sure what you mean by time-base tests, but to me that distinction sounds like regression vs. performance testing. Or do you think of time-based regression tests? I certainly favor regression tests that are event-based, as I've run into too many false errors due to unexpected timings already. 2. Test situations which require *many* connections in order to find issues caused, in essence, by load. Tsung seems well-suited to the latter. Agreed. I would also note here, that Tsung seems especially well suited for performance testing of clustered solutions, as you very likely have to cluster the testing agents as well to put a decent load on the SUT. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby 0.2.1
Andrew Dunstan wrote: > > > Robert Haas wrote: > > But at least for simple features I think that there would be a value > > in separating the patch author's work from the committer's adjustments. > > > > > > That is just going to make life harder for committers. > > There are plenty of things with my name on them that are not exactly > what I submitted. I think that's true of just about everybody. Mostly > things changed hae improved, but not always. I don't think we should be > too proprietary about patches. As far as I'm concerned, credit goes to > the submitter and blame if any to the committer. Agreed. Simon is right that if only his name is on the commit, there is an assumption that the committer made no changes, or only cosmetic ones. For hot standby, I think the committer is making significant changes (that could lead to bugs) and hence the committer's name should be in the commit. Sometimes we say "adjusted by" committer, but in this case I think Heikki is doing more than adjustmants --- nothing wrong with that --- it should just be documented in the commit message. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby 0.2.1
On Fri, 2009-10-09 at 14:05 -0400, Andrew Dunstan wrote: > > Robert Haas wrote: > > But at least for simple features I think that there would be a value > > in separating the patch author's work from the committer's adjustments. > > > > > > That is just going to make life harder for committers. > > There are plenty of things with my name on them that are not exactly > what I submitted. I think that's true of just about everybody. Mostly > things changed hae improved, but not always. I don't think we should be > too proprietary about patches. As far as I'm concerned, credit goes to > the submitter and blame if any to the committer. +1 > > cheers > > andrew > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering If the world pushes look it in the eye and GRR. Then push back harder. - Salamander -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby 0.2.1
Robert Haas wrote: But at least for simple features I think that there would be a value in separating the patch author's work from the committer's adjustments. That is just going to make life harder for committers. There are plenty of things with my name on them that are not exactly what I submitted. I think that's true of just about everybody. Mostly things changed hae improved, but not always. I don't think we should be too proprietary about patches. As far as I'm concerned, credit goes to the submitter and blame if any to the committer. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby 0.2.1
On Oct 9, 2009, at 1:21 PM, Bruce Momjian wrote: Simon Riggs wrote: You have posted patches that I have said I don't agree with. My name is going to be on this when it goes in, so I don't think it makes any sense to force that commit to include changes I don't agree with. I cannot prevent you making changes afterwards, nor would I wish to. I'd like you to respond sensibly to comments on those. We should work together on a consensus basis, especially since I know you have not fully tested your changes (either). Your error rate might be lower than mine, but it is non-zero. The commit message and release notes mention might have just Simon's name, or multiple people. The hot patch commit is going to have multiple people involved before it is committed, so if Simon is worried that the patch will have ideas in it he does not agree with, perhaps we can make sure the commit and release note items include Heikki's name as well. Normally if a committer makes signficant changes to a patch, the committer's name is also added to the commmit message, and I suggest we do the same thing here with hot standby. I think this is a weakness of our current style of heavy-weight commits. I don't have a great suggestion for fixing it, though. Even if we move to git, a major feature like this has such a complex development history that I'm queasy about slurping it in unsquashed. But at least for simple features I think that there would be a value in separating the patch author's work from the committer's adjustments. I realize (now) that this would complicate the release note generation process somewhat, based on our current process, and there might be other downsides as well. All the same, I think it has enough value to make it worth thinking about whether there's some way to make it work. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
Simon Riggs wrote: >> Tom Lane wrote: >>> [ scratches head ... ] Why is hot standby messing with this sort of >>> thing at all? It sounds like a performance optimization that should >>> be considered separately, and *later*. >> Yeah, I too considered just ripping it out. Simon is worried that >> locking all the lock partitions and scanning the locks table can take a >> long time. We do that in the master, while holding both ProcArrayLock >> and XidGenLock in exclusive mode (hmm, why is shared not enough?), so >> there is some grounds for worry. OTOH, it's only done once per checkpoint. > > I could live with ripping it out, but what we have now doesn't make > sense, to me. Ok, let's just rip it out for now. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby on git
While playing with conflict resolution, I bumped into this: postgres=# begin ISOLATION LEVEL SERIALIZABLE; BEGIN postgres=# SELECT * FROM foo; id | data +-- 12 | (1 row) postgres=# SELECT * FROM foo; id | data +-- 12 | (1 row) postgres=# SELECT * FROM foo; id | data +-- 12 | (1 row) postgres=# SELECT * FROM foo; id | data +-- 12 | (1 row) postgres=# SELECT * FROM foo; id | data +-- 12 | (1 row) postgres=# SELECT * FROM foo; ERROR: canceling statement due to conflict with recovery postgres=# SELECT * FROM foo; id | data +-- 13 | (1 row) postgres=# SELECT * FROM foo; id | data +-- 13 | (1 row) postgres=# begin ISOLATION LEVEL SERIALIZABLE; id | data +-- 13 | (1 row) postgres=# SELECT * FROM foo; BEGIN postgres=# SELECT * FROM foo; id | data +-- 13 | (1 row) The backend and the frontend seem to go out of sync, when a conflict happens in idle-in-transaction mode. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby 0.2.1
Simon Riggs wrote: > You have posted patches that I have said I don't agree with. My name is > going to be on this when it goes in, so I don't think it makes any sense > to force that commit to include changes I don't agree with. I cannot > prevent you making changes afterwards, nor would I wish to. I'd like you > to respond sensibly to comments on those. We should work together on a > consensus basis, especially since I know you have not fully tested your > changes (either). Your error rate might be lower than mine, but it is > non-zero. The commit message and release notes mention might have just Simon's name, or multiple people. The hot patch commit is going to have multiple people involved before it is committed, so if Simon is worried that the patch will have ideas in it he does not agree with, perhaps we can make sure the commit and release note items include Heikki's name as well. Normally if a committer makes signficant changes to a patch, the committer's name is also added to the commmit message, and I suggest we do the same thing here with hot standby. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [GENERAL] contrib/plantuner - enable PostgreSQL planner hints
Oleg Bartunov wrote: > =# set enable_seqscan=off; > =# set plantuner.forbid_index='id_idx2'; The genius of this module is the line above -- a more fine-grained way to control the optimizer, with specific index disabling. > =# explain select id from test where id=1; >QUERY PLAN > -- > Bitmap Heap Scan on test (cost=4.34..15.03 rows=12 width=4) > Recheck Cond: (id = 1) > -> Bitmap Index Scan on id_idx (cost=0.00..4.34 rows=12 width=0) > Index Cond: (id = 1) > (4 rows) Are you planning to submit this as a /contrib module? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Idle connection timeout
I saw a recent blog entry where someone came up with a solution for expiring idle connections ( http://dividebyzeroexception.blogspot.com/2009/10/implementing-waittimeout-in-postgresql.html) and I was surprised there wasn't a config option for this already. Is this something that can be considered for a future release? Thanks Thom
Re: [HACKERS] COPY enhancements
On Fri, 9 Oct 2009, Tom Lane wrote: what do we do with rows that fail encoding conversion? For logging to a file we could/should just decree that we write out the original, allegedly-in-the-client-encoding data. I'm not sure what we do about logging to a table though. The idea of storing bytea is pretty unpleasant but there might be little choice. I think this detail can get punted as documented and the error logged, but not actually handled perfectly. In most use cases I've seen here, saving the rows to the "reject" file/table is a convenience rather than a hard requirement anyway. You can always dig them back out of the original again if you see an encoding error in the logs, and it's rare you can completely automate that anyway. The main purpose of the reject file/table is to accumulate things you might fix by hand or systematic update (i.e. add ",\N" for a missing column when warranted) before trying a re-import for review. I suspect the users of this feature would be OK with knowing that can't be 100% accurate in the face of encoding errors. It's more important that in the usual case, things like bad delimiters and missing columns, that you can easily manipulate the rejects as simple text. Making that harder just for this edge case wouldn't match the priorities of the users of this feature I've encountered. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
Hannu Krosing writes: > On Thu, 2009-10-08 at 11:32 -0400, Robert Haas wrote: >> Another possible approach, which isn't perfect either, is the idea of >> allowing COPY to generate a single column of output of type text[]. >> That greatly reduces the number of possible error cases, > maybe make it bytea[] to further reduce error cases caused by charset > incompatibilities ? That seems likely to be considerably less convenient and more error prone, as it now puts it on the user to do the correct conversion. It does bring up an interesting point for error handling though, which is what do we do with rows that fail encoding conversion? For logging to a file we could/should just decree that we write out the original, allegedly-in-the-client-encoding data. I'm not sure what we do about logging to a table though. The idea of storing bytea is pretty unpleasant but there might be little choice. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
Simon Riggs writes: > Another thing that has occurred to me is that RI checks are currently > resolved at end of statement and could end up rejecting any/all rows > loaded. If we break down the load into subtransaction pieces we would > really want the RI checks on the rows to be performed during the > subtransaction that makes them. The good thing about that is that it > would lend itself to holding successful checks in a hash table to allow > a fast path optimization of continual re-checks of same values. If we did that it would guarantee that cases like self-referential RI would fail. (Think parent-child links in a tree, for example.) I see the point about wishing that such checks would be part of the per-row error handling, but we can't just unconditionally do things that way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
Simon Riggs writes: > On Thu, 2009-10-08 at 12:21 -0400, Tom Lane wrote: >> So really we have to find some way to only expend one XID per failure, >> not one per row. > I discovered a few days back that ~550 subtransactions is sufficient to > blow max_stack_depth. 1 subtransaction per error doesn't allow many > errors. I assume you are talking about 550 nested levels, not 550 sequential subtransactions, so that doesn't seem particularly relevant. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Writeable CTEs and side effects
Jeff Davis writes: > On Fri, 2009-10-09 at 02:23 +0300, Peter Eisentraut wrote: >> I think I'd want "writable subqueries" instead of only "writable CTEs". > I think the original motivation was that it's more clear that a CTE is > separated and can only be executed once (if it has side effects). > Depending on how the query is written, it might be less obvious how many > times the subquery should be executed, and it might change based on the > plan. Right. The behavior would be entirely unpredictable, and usually undesirable, if the RETURNING query is underneath a join, or an aggregate, or a LIMIT, yadda yadda. Tying it to WITH provides a convenient way, from both the user-visible and implementation sides, of saying "this is an independent query that we will execute once and then make the RETURNING results available for use in this other query". regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Following the discussions in http://archives.postgresql.org/pgsql-hackers/2009-09/msg01766.php and http://archives.postgresql.org/pgsql-hackers/2009-10/msg00025.php , here are patches for a) a hook in backend/commands/user.c that allows one to add password checking functions b) a contrib module that makes use of the hook and c) documentation for the contrib module. I'll add links to the November commitfest page. Yours, Laurenz Albe pwdcheck-hook.patch Description: pwdcheck-hook.patch pwdcheck-contrib.patch Description: pwdcheck-contrib.patch pwdcheck-contrib-doc.patch Description: pwdcheck-contrib-doc.patch -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to use eclipse when debugging postgreSQL backend
This whole discussion looked helpful to save, I put a summary of the issue and the suggested solution at https://wiki.postgresql.org/wiki/Working_with_Eclipse#Debugging_with_child_processes so nobody has to reinvent this. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COPY enhancements
On Thu, 2009-10-08 at 11:32 -0400, Robert Haas wrote: > Another possible approach, which isn't perfect either, is the idea of > allowing COPY to generate a single column of output of type text[]. > That greatly reduces the number of possible error cases, maybe make it bytea[] to further reduce error cases caused by charset incompatibilities ? > and at least > gets the data into the DB where you can hack on it. But it's still > going to be painful for some use cases. > > ...Robert -- Hannu Krosing http://www.2ndQuadrant.com PostgreSQL Scalability and Availability Services, Consulting and Training -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
