Re: [HACKERS] Synchronous replication: Admin command for replication_timeout_action
Hi, On Tue, May 5, 2009 at 2:37 AM, K, Niranjan (NSN - IN/Bangalore) wrote: > Hi, > > This is to support an admin command or utility which can trigger the > server to be taken to a standalone mode if there a connection failure > detection between Primary and server. It need not be always, that the > replication_timeout needs to be accomplished to detect the connection > failure because it could happen that cluster/hearbeat framework might > detect the connection failure earlier to the replication_timeout. So the > admin command, which will abstract the implementation details will > assist in taking the server to standalone mode earlier to > replication_timeout. > > Are there any suggestions from your side with respect to this? Yes. Since walsender is treated as special backend, we can use pg_terminate_backend() to terminate replication and let the server standalone. This feature is simple but very useful, so I'll address it (my previous patch has not provided this completely yet). Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pg_migrator and an 8.3-compatible tsvector data type
I found out at PGCon that the internal format of tsvector changed slightly from 8.3 to 8.4. Teodor gave me a conversion function and I have adjusted pg_migrator to install a v8_3_tsvector data type to be used during the load so the old user tables use that data type. You can see the code here at the bottom of this file: http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pg-migrator/pg_migrator/src/pg_migrator.c?rev=1.36&content-type=text/x-cvsweb-markup I am not done yet because eventually v8_3_tsvector will be moved into a separate schema and the tsvector data type restored. The problem I have is that while I have created v8_3_tsvector (with casts), I have not created index routines for it, so I get this error when restoring a database that uses a GIN index on tsvector: Restoring database schema psql:/u/postgres/pg_migrator_dump_db.sql:7006: ERROR: data type tsvector has no default operator class for access method "gin" HINT: You must specify an operator class for the index or define a default operator class for the data type. So, how do I create a compatible set of hooks for v8_3_tsvector? Because tsvector is a native data type I can't figure out how to set up a similar type. Also, will the indexes be OK if they are created from types that don't have the 8.3 format? What about new indexes created after the migration is done? Ideally the cast would just be called during index activity and the 8.4 tsvector index routines used. I am told by Teodor that GiST indexes would be fine with the 8.3 data format, and we are forcing the reindexing of GIN indexes anyway in 8.4. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 8:03 PM, Tom Lane wrote: > Greg Stark writes: >> On Mon, May 25, 2009 at 11:32 PM, Tom Lane wrote: >>> Admittedly this is a bit inconvenient, but the point is that the >>> functionality does exist. There is no need to have a built-in >>> version of this function unless we get significant advantages >>> from having it built-in, and right now I'm not seeing those. > >> I assume people don't want the *text* of the current output format but >> the actual values in separate columns. > > Well, I notice that everyone is carefully dodging the subject of exactly > what columns they want, I had a try at this upthread, actually, but it's not a real easy problem. > but my example would clearly scale easily to any > specific set of output columns that EXPLAIN might return instead of one > text column. Since we were previously told that any particular release > of PG need only offer one set of possible output columns, I figured the > problem was solved ;-) I was totally unconvinced by that argument. I actually think that the best data structure for this would be something like hstore. It would sure be nice to be able to manipulate this data using SQL: I am sure there are people on this mailing list who hate XML and maybe a few who hate JSON, but if they hate SQL then they're off my list of people I care about making happy. :-) At the same time, the variable number of output columns is problematic for a flat table representation. It may not be so problematic that we can't work around it, but it's definitely not great. It's really the pits to think that our data model is so impoverished that it can't in a reasonable way handle the output of our EXPLAIN command. It would be awfully sweet to be able to do things like: show me all the plan nodes where the expected and actual row counts differed by more than a factor of 10. And why should I need an external XML/JSON parser to do that, rather than just a WHERE clause? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] from_collapse_limit vs. geqo_threshold
On Mon, May 25, 2009 at 6:15 PM, Tom Lane wrote: > Now I'm still not exactly happy with GEQO, but it's surely a lot better > than it was in the fall of 2000. So on the whole it does seem that the > current relationships between from_collapse_limit, join_collapse_limit, > and geqo_threshold are based on obsolete information and should be > revisited. I don't have any data at hand to suggest specific new > default values, though. For 8.4, I'd be happy to just improve the documentation. I think this sentence could just be deleted from the section on from_collapse_limit: It is usually wise to keep this less than . We could put some other explanation in place of that sentence, but I'm not exactly sure what that explanation would say. I guess the point is that setting from_collapse_limit < geqo_threshold may delay GEQO planning considerably in the face of complex subqueries, because pulling up subqueries increases the size of the FROM list (I think). That could be good if you want your query plans to be more deterministic, but there's no guarantee they'll be good. Setting from_collapse_limit > geqo_threshold is basically saying that the standard planner will always have subqueries pulled up, so from_collapse_limit should be based on what the pain point will be for GEQO. I'm not sure there's a lot of point in spelling all that out, though. It more or less follows from the definition of the parameters. So, I'd be just as happy to delete the misleading hint and call it good. But I could go either way. For 8.5, it sounds like we need to do some testing to determine an appropriate set of values, but I'm not exactly sure what to test. As a practical matter, the correct level of effort depends a lot on how long the query figures to run. For OLAP queries, planning times of more than 50 ms or so start to add noticeably to the overall runtime of the query, but if the query is expected to run for several minutes, we'd presumably be happy to spend several seconds planning it, which might make it feasible to use the standard planner even for very, very big queries. I'm not 100% convinced of the value of join_collapse_limit for anything other than explicit control over the join order. I have yet to meet a PostgreSQL who thought that it was intuitive that it might matter whether you wrote A JOIN B ON P1 JOIN C ON P2 JOIN D ON P3 [etc] or A, B, C, D, [etc] WHERE P1, P2, P3. I suspect there are many people who, if they knew that the latter might optimize better than the former in some circumstances, would simply always write it in the latter fashion, which makes the whole thing look a lot like a concealed foot-gun, since whether or not it actually protects you against exponential planning-time growth has a lot to do with how you happen to like to write your queries (myself, I've switched styles in the last few years). ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] usability of pg_get_function_arguments
That would be more work, not less, for the known existing users of the function (namely pg_dump and psql). It's a bit late to be redesigning the function's API anyway. I agree. The recommended way to do that is to use pg_get_expr --- it'd certainly be a bad idea to try to parse that string from client code. I experimented with your example and noticed that pg_get_expr requires a hack --- it insists on having a relation OID argument, because all previous use-cases for it involved expressions that might possibly refer to a particular table. So you have to do something like regression=# select pg_get_expr(proargdefaults,'pg_proc'::regclass) from pg_proc where proname='f13'; pg_get_expr --- 10, 'hello'::character varying, '2009-01-01 00:00:00'::timestamp without time zone, 'comma here ,'::character varying (1 row) Unfortunately, there is no way to know to which argument(s) the values above belongs to. After some searching, it looks like PgAdmin does the trick by hand parsing the string. Fortunately the result of pg_get_expr from above is ordered --- Perhaps by doing some find and replace, I can determine to which argument the returned default value belongs to. Thank you for your help :) -- Regards, Gevik -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
On 26/05/2009, at 10:25 AM, Tom Lane wrote: Andrew McNamara writes: Are there any other cases where the binary receive functions are missing sanity checks? Possibly --- you want to go looking? Uh. I'd be lying if I said I "wanted to" - I got enough of a taste of those functions when trying to work out what they expect when doing my ocpgdb module. At the moment, however, I have a good excuse for wimping out - every waking hour is being expended on swine flu related work (although I'd almost welcome the distraction of a good *virtual* bug hunt). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
Andrew McNamara writes: > Are there any other cases where the binary receive functions are > missing sanity checks? Possibly --- you want to go looking? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
On 26/05/2009, at 5:41 AM, Tom Lane wrote: The only place I can find where an oversize time value behaves in a seriously bogus fashion is in time_out, or more specifically EncodeTimeOnly(): it fails to initialize its output string at all. So you could easily get garbage text output, though in my quick tests you seem to usually get an empty string instead. The odds of an actual crash seem pretty small, but not quite zero (if somehow there was no zero byte up to the end of the stack). I'm seeing all sorts of odd stuff - typically the last column value output, but occasionally other snippets of random data that don't seem related to the query. My feeling is that the error check in EncodeTimeOnly is just stupid and should be removed. That code will work fine with oversize times (and no, it won't overrun the output buffers either). The callers aren't bothering to check for error returns anyway... I'm not sure it's postgresql's job to police things like this, but returning values greater than 24 hours may violate assumptions in user code, and I would be worried about potentially causing silent failures. Of course, it should no longer be possible to get an illegal value into the database, so the risk is low - either a database that predates the fix, or database corruption. Are there any other cases where the binary receive functions are missing sanity checks? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
Greg Stark writes: > On Mon, May 25, 2009 at 11:32 PM, Tom Lane wrote: >> Admittedly this is a bit inconvenient, but the point is that the >> functionality does exist. There is no need to have a built-in >> version of this function unless we get significant advantages >> from having it built-in, and right now I'm not seeing those. > I assume people don't want the *text* of the current output format but > the actual values in separate columns. Well, I notice that everyone is carefully dodging the subject of exactly what columns they want, but my example would clearly scale easily to any specific set of output columns that EXPLAIN might return instead of one text column. Since we were previously told that any particular release of PG need only offer one set of possible output columns, I figured the problem was solved ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] usability of pg_get_function_arguments
Gevik Babakhani writes: > Perhaps it would be much better if pg_get_function_arguments returned > the data is some kind of a structure than a blob of string like the above. That would be more work, not less, for the known existing users of the function (namely pg_dump and psql). It's a bit late to be redesigning the function's API anyway. > In order to make the data above usable, one has to write a custom parser > to hopefully be able to make any use of the return data. Of course > another option is to parse the pg_proc.proargdefaults > which in turn is a challenge on its own. The recommended way to do that is to use pg_get_expr --- it'd certainly be a bad idea to try to parse that string from client code. I experimented with your example and noticed that pg_get_expr requires a hack --- it insists on having a relation OID argument, because all previous use-cases for it involved expressions that might possibly refer to a particular table. So you have to do something like regression=# select pg_get_expr(proargdefaults,'pg_proc'::regclass) from pg_proc where proname='f13'; pg_get_expr --- 10, 'hello'::character varying, '2009-01-01 00:00:00'::timestamp without time zone, 'comma here ,'::character varying (1 row) where it doesn't matter which table you name, as long as you name one. It would probably be cleaner to allow pg_get_expr to accept a zero OID, for use when you are asking it to deparse an expression that's expected to be Var-free. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 11:32 PM, Tom Lane wrote: > Admittedly this is a bit inconvenient, but the point is that the > functionality does exist. There is no need to have a built-in > version of this function unless we get significant advantages > from having it built-in, and right now I'm not seeing those. I assume people don't want the *text* of the current output format but the actual values in separate columns. So you could do things like accumulate the data in a table and later use sql to search for queries using specific indexes or where estimates are off etc. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 6:32 PM, Tom Lane wrote:
> Robert Haas writes:
>> I agree with this, but there is a lot of sentiment (which I share)
>> that it should be possible to capture EXPLAIN output using subselect
>> or CTAS syntax, regardless of exactly what that output ends up being.
>
> Well, it should be possible to capture the output, but let's not
> prejudge the syntax.
>
>> That seems to require that EXPLAIN be a fully-reserved keyword, so I
>> wonder what we think about that.
>
> Nonstarter, especially when it's so obviously possible to do it without
> that. The threshold for reserving words that aren't reserved by SQL
> spec has to be really high, because you will break innocent applications
> that way.
>
> Before anyone gets overly excited about having special syntax for this,
> I should point out that you can do it today, for instance like so:
>
> regression=# create function expl(q text) returns setof text as $$
> regression$# declare r record;
> regression$# begin
> regression$# for r in execute 'explain ' || q loop
> regression$# return next r."QUERY PLAN";
> regression$# end loop;
> regression$# end$$ language plpgsql;
> CREATE FUNCTION
>
> regression=# select * from expl('select * from tenk1');
> expl
> -
> Seq Scan on tenk1 (cost=0.00..458.00 rows=1 width=244)
> (1 row)
>
> Admittedly this is a bit inconvenient, but the point is that the
> functionality does exist. There is no need to have a built-in
> version of this function unless we get significant advantages
> from having it built-in, and right now I'm not seeing those.
The only problem I see with this is that there's no convenient way of
specifying the options you want. Granted, it wouldn't be all that
difficult to add a couple of boolean options to specify the state of
EXPLAIN and ANALYZE, but if we have more options, it starts to get a
bit complicated, especially if they do things like change the set of
output columns. I'm still liking that idea, but even if we don't end
up implementing that particular thing, I'm really doubtful that the
need to make EXPLAIN do more things is likely to go away.
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] usability of pg_get_function_arguments
I am trying to extract function argument information using the pg_get_function_arguments() and it strikes me that despite of this function generating very useful information, it is actually not so user friendly. Consider the following: - create or replace function f13(int=10,varchar='hello',inout complex timestamp='01-01-2009'::timestamp,varchar='comma here ,') as $$ begin end; $$ language plpgsql; where the pg_get_function_arguments generates the following string: - "integer DEFAULT 10, character varying DEFAULT 'hello'::character varying, INOUT complex timestamp without time zone DEFAULT '2009-01-01 00:00:00'::timestamp without time zone, character varying DEFAULT 'comma here ,'::character varying" (good luck parsing that, I thought) - In order to make the data above usable, one has to write a custom parser to hopefully be able to make any use of the return data. Of course another option is to parse the pg_proc.proargdefaults which in turn is a challenge on its own. Perhaps it would be much better if pg_get_function_arguments returned the data is some kind of a structure than a blob of string like the above. BTW: The same goes for pg_get_function_identity_arguments. Any thoughts? -- Regards, Gevik -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
Robert Haas writes:
> I agree with this, but there is a lot of sentiment (which I share)
> that it should be possible to capture EXPLAIN output using subselect
> or CTAS syntax, regardless of exactly what that output ends up being.
Well, it should be possible to capture the output, but let's not
prejudge the syntax.
> That seems to require that EXPLAIN be a fully-reserved keyword, so I
> wonder what we think about that.
Nonstarter, especially when it's so obviously possible to do it without
that. The threshold for reserving words that aren't reserved by SQL
spec has to be really high, because you will break innocent applications
that way.
Before anyone gets overly excited about having special syntax for this,
I should point out that you can do it today, for instance like so:
regression=# create function expl(q text) returns setof text as $$
regression$# declare r record;
regression$# begin
regression$# for r in execute 'explain ' || q loop
regression$# return next r."QUERY PLAN";
regression$# end loop;
regression$# end$$ language plpgsql;
CREATE FUNCTION
regression=# select * from expl('select * from tenk1');
expl
-
Seq Scan on tenk1 (cost=0.00..458.00 rows=1 width=244)
(1 row)
Admittedly this is a bit inconvenient, but the point is that the
functionality does exist. There is no need to have a built-in
version of this function unless we get significant advantages
from having it built-in, and right now I'm not seeing those.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] from_collapse_limit vs. geqo_threshold
Robert Haas writes: > On Thu, May 21, 2009 at 7:50 AM, Tom Lane wrote: >> Josh Berkus writes: >>> ... trying to remember why I wrote that ... what would happen if >>> FROM_COLLAPSE_LIMIT was *more* than GEQO_THRESHOLD? >> >> I think I wrote it, not you. The point of the advice is to keep >> subquery collapsation (hm, what's the right noun form? Need caffeine) >> from turning a non-GEQO query into a GEQO one, and thus subjecting >> you to unpredictable plans. Maybe the resulting plans would be better >> on average, or maybe they wouldn't, but in any case they'd be >> unpredictable. > That's more or less what I figured, but my real world experience is > that pulling up subqueries and using GEQO leads to plans that are > random but tolerable, whereas not pulling up subqueries leads to plans > that are almost uniformly bad. I went back and looked at the CVS history to try to refresh my memory about how we got here. As best I can find, there were two steps: 1. The original commit of the ability to have subqueries at all, during 7.1 development: 2000-09-29 14:21 tgl Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED! This introduced the ability to pull up subqueries, but with an arbitrary limit of geqo_threshold/2 on the number of relations that would be collected into a single planning problem. 2. During 7.4 development, we did this: 2003-01-25 18:10 tgl Allow the planner to collapse explicit inner JOINs together, rather than necessarily following the JOIN syntax to develop the query plan. The old behavior is still available by setting GUC variable JOIN_COLLAPSE_LIMIT to 1. Also create a GUC variable FROM_COLLAPSE_LIMIT to control the similar decision about when to collapse sub-SELECT lists into their parent lists. (This behavior existed already, but the limit was always GEQO_THRESHOLD/2; now it's separately adjustable.) The excuse for join_collapse_limit to exist at all is largely one of backwards compatibility. Up to then, we had not-infrequently suggested that people could force a desired join order by writing an explicit JOIN nest, and eliminating that escape hatch altogether didn't seem like a good idea. I think from_collapse_limit was added largely on grounds of symmetry. Now, as to why the original commit had the geqo_threshold/2 restriction: it was obviously not based on field experience with flattening, because we didn't have any. What I think it *was* based on was that GEQO sucked really badly back then, and I wanted to avoid having it kick in for queries that it had never kicked in for in previous releases. Some quick comparisons say that 7.1 in GEQO mode was about 5X slower than HEAD (despite its planning being a lot more simplistic), and tended to find considerably worse plans. Some of the significant improvements since then: 2004-01-23 18:54 tgl Revise GEQO planner to make use of some heuristic knowledge about SQL, namely that it's good to join where there are join clauses rather than where there are not. Also enable it to generate bushy plans at need, so that it doesn't fail in the presence of multiple IN clauses containing sub-joins. 2004-01-21 18:33 tgl Repair error apparently introduced in the initial coding of GUC: the default value for geqo_effort is supposed to be 40, not 1. The actual 'genetic' component of the GEQO algorithm has been practically disabled since 7.1 because of this mistake. Also, up to 7.0 there were some nasty memory leaks in the planner and especially in GEQO, because we didn't have the memory context mechanism. I think those were actually fixed as of 2000-09-29, but GEQO still had a reputation for blowing out backend memory. Now I'm still not exactly happy with GEQO, but it's surely a lot better than it was in the fall of 2000. So on the whole it does seem that the current relationships between from_collapse_limit, join_collapse_limit, and geqo_threshold are based on obsolete information and should be revisited. I don't have any data at hand to suggest specific new default values, though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 11:22 AM, Tom Lane wrote:
> Joshua Tolley writes:
>> I'm not sure I see why it would be less flexible. I'm imagining we define
>> some
>> record type, and a function that returns a set of those records.
>
> I'm unimpressed by the various proposals to change EXPLAIN into a
> function. Quoting the command-to-explain is going to be a pain in the
> neck.
I agree with this, but there is a lot of sentiment (which I share)
that it should be possible to capture EXPLAIN output using subselect
or CTAS syntax, regardless of exactly what that output ends up being.
That seems to require that EXPLAIN be a fully-reserved keyword, so I
wonder what we think about that.
(The way I tested this quickly is to make '(' ExplainStmt ')' a third
production for select_with_parens. I'm not 100% sure that's the right
place for it, but a couple of other reasonable-looking places produced
non-obvious parsing conflicts.)
> And can you really imagine using it manually, especially if it
> returns so many fields that you *have to* write out the list of fields
> you actually want, else the result is unreadable? It's going to be just
> as much of something you can only use through a helper application as
> the XML way would be.
Nothing could possibly be as bad as XML. I'm with Josh: if we produce
table-formatted output, someone can always turn it into XML or JSON or
whatever they want. The reverse figures to be a whole lot more
difficult.
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
Hi Tom, On 05/25/2009 08:04 PM, Tom Raney wrote: So, why not put ALL interesting data in the EXPLAIN XML feed? I'm not suggesting for this discussion that we include discarded plans, but that we include every piece of data that may be of interest to folks building connecting tools. The parsers can pick and choose what they use easily and, because the feed isn't positional, won't break when addition data is added. A GUC parameter could govern the data included in this variant of EXPLAIN, but even that seems unnecessary. This approach will allow the standard EXPLAIN to evolve in whatever way pleases the humans without interfering with the machines. Well, there is the problem Robert Haas described - some stats may be too expensive to gather (like the io-stats) for regular use, but still be quite usefull. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I'm not entirely sure why we put a range limit on time values at all, >> but given that we do, it'd probably be a good idea to check the range >> in the recv functions. I'm inclined to fix this for 8.4, but not >> back-patch because of compatibility considerations. Any objections >> out there? > Are we confident it can't be abused to impact other clients connecting > or break the back-end in some way? More specifically, could it be a > security issue? Havn't looked at it yet, but getting what sounded like > corrupted data back out could be bad.. The only place I can find where an oversize time value behaves in a seriously bogus fashion is in time_out, or more specifically EncodeTimeOnly(): it fails to initialize its output string at all. So you could easily get garbage text output, though in my quick tests you seem to usually get an empty string instead. The odds of an actual crash seem pretty small, but not quite zero (if somehow there was no zero byte up to the end of the stack). My feeling is that the error check in EncodeTimeOnly is just stupid and should be removed. That code will work fine with oversize times (and no, it won't overrun the output buffers either). The callers aren't bothering to check for error returns anyway... On the whole the argument that it could be a security problem seems pretty thin. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A couple of gripes about the gettext plurals patch
Peter Eisentraut writes: > On Sunday 26 April 2009 21:29:20 Tom Lane wrote: >> This is bogus: errmsg expects that it should itself feed its first >> argument through gettext(), not receive an argument that is already >> translated. That's at the least a waste of cycles, and it's not >> entirely impossible that double translation could end up with a just >> plain wrong result. > I think we can live with this for now, and we have lived with this sort of > issue in other places for a while. We should consider reshuffling the > interfaces a bit as you describe to reduce the problem. The issue of double translation is really a minor point; what is bothering me is that we've got such an ad-hoc, non-compile-time-checkable approach here. Zdenek's discovery today that some of the format strings are flat-out wrong http://archives.postgresql.org/pgsql-hackers/2009-05/msg00946.php surprises me not in the least. I think that we need to have a more carefully designed interface before we allow any more plural translations to be put in place, not after. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 11:22:24AM -0400, Tom Lane wrote: > Joshua Tolley writes: > > I'm not sure I see why it would be less flexible. I'm imagining we define > > some > > record type, and a function that returns a set of those records. > > I'm unimpressed by the various proposals to change EXPLAIN into a > function. Quoting the command-to-explain is going to be a pain in the > neck. Yeah, that's been bugging me, despite my recent support of that plan. > And can you really imagine using it manually, especially if it > returns so many fields that you *have to* write out the list of fields > you actually want, else the result is unreadable? It's going to be just > as much of something you can only use through a helper application as > the XML way would be. Good point. The reason, as I remember it, that we wanted to be able to specify what fields are returned was so that fields that are expensive to calculate are calculated only when the user wants them. If that's the only consideration, perhaps we should have a standard version and a "FULL" version, e.g. EXPLAIN [ANALYZE] [FULL] ...where FULL would indicate the user wanted the all available statistics, not just the cheap ones. Somewhere in there we'd also need an indicator to say we wanted an output format other than the usual text version (such as the "WITH XML" clause I think someone suggested); I maintain it's all just a table of data, and should be represented the same way we represent any other table of data. - Josh / eggyknap signature.asc Description: Digital signature
Re: [HACKERS] generic options for explain
On May 25, 2009, at 0:47 , Joshua Tolley wrote: On Sun, May 24, 2009 at 06:53:29PM -0400, Tom Lane wrote: Greg Smith writes: On Sun, 24 May 2009, Pavel Stehule wrote: we should have a secondary function explain_query(query_string, option) that returns setof some. +1. The incremental approach here should first be adding functions that actually do the work required. Then, if there's a set of those that look to be extremely useful, maybe at that point it's worth talking about how to integrate them into the parser. Starting with the parser changes rather than the parts that actually do the work is backwards. If you do it the other way around, at all times you have a patch that actually provides immediate useful value were it to be committed. Something that returns a setof can also be easily used to implement the "dump EXPLAIN to a table" feature Josh Tolley brought up (which is another common request in this area). A serious problem with EXPLAIN via a function returning set, or with putting the result into a table, is that set results are logically unordered, just as table contents are. So from a strict point of view this only makes sense when the output format is designed to not depend on row ordering to convey information. We could certainly invent such a format, but I think it's a mistake to go in this direction for EXPLAIN output that is similar to the current output. The Oracle version, as it fills the table of explain results, gives each number an id and the id of its parent row, which behavior we could presumably copy. Or some other schema that allows us to preserve the tree. Michael Glaesemann grzm seespotcode net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
David Fetter wrote: On Mon, May 25, 2009 at 11:02:53AM -0400, Tom Lane wrote: Robert Haas writes: This is all much more complicated than what I proposed, and I fail to see what it buys us. I'd say that you're just reinforcing the point I made upthread, which is that insisting that XML is the only way to get more detailed information will just create a cottage industry of beating that XML output format into submission. The impression I have is that (to misquote Churchill) XML is the worst option available, except for all the others. We need something that can represent a fairly complex data structure, easily supports addition or removal of particular fields in the structure (including fields not foreseen in the original design), is not hard for programs to parse, and is widely supported --- ie, "not hard" includes "you don't have to write your own parser, in most languages". How many realistic alternatives are there? JSON for one, and it's *much* lighter in just about every way. Cheers, David. For what it's worth, if this revised output form is destined for consumption by a machine, it really doesn't matter what protocol is used and how 'readable' it is by humans, as long as the protocol can express all the present and anticipated variations of the data without breaking parsers along the way. While building the Visual Planner tool, I selected XML output for no other reason than it was easy to parse on the receiving end and was hierarchical, making it perfect for representing a plan tree - or thousands. I'm sure other alternatives would have been fine as well. But, once that decision was made, I never had any reason again to look at the XML stream. If we're worried about the excess 'weight' of XML, I found this to be a non-issue in practice. The output generated by the Visual-Planner-Enabled Postgres server contains MUCH more information that one would typically see with standard EXPLAIN. The tool returns not only the most-optimal plan, but all discarded plans as well. A four way join results in output of 24k lines of XML. While it parses nearly instantly, the biggest delay is in the network. And, even this is minimal. So, why not put ALL interesting data in the EXPLAIN XML feed? I'm not suggesting for this discussion that we include discarded plans, but that we include every piece of data that may be of interest to folks building connecting tools. The parsers can pick and choose what they use easily and, because the feed isn't positional, won't break when addition data is added. A GUC parameter could govern the data included in this variant of EXPLAIN, but even that seems unnecessary. This approach will allow the standard EXPLAIN to evolve in whatever way pleases the humans without interfering with the machines. Regards, Tom Raney -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Andres Freund writes: >> You can define that a subset (or all) files use a specific "diff driver" >> in the repository - unfortunately the definition of that driver has to >> be done locally. Defining it currently involves installing a wrapper >> like the one on http://wiki.postgresql.org/wiki/Talk:Working_with_Git >> and doing > Ugh, hit the wrong key: > and executing > `git config --global diff.context.command "git-external-diff"` Okay, so it will more or less have to be a local option. That's okay ... all I really insist on is being able to get a readable diff out of it. I grant that not everyone may have the same opinion about what's readable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On 05/25/2009 07:53 PM, Andres Freund wrote: On 05/25/2009 07:31 PM, Tom Lane wrote: David Fetter writes: On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: If you'd like to accomplish something *useful* about this, how about pestering git upstream to support diff -c output format? It looks like this is doable with a suitable git configuration file such as $HOME/.gitconfig or (finer grain) a .git/config for the repository :) Cool, let's see one. If we were to put it into a repository config file, that would more or less have the effect of enforcing a project style for diffs, no? Yes and no. You can define that a subset (or all) files use a specific "diff driver" in the repository - unfortunately the definition of that driver has to be done locally. Defining it currently involves installing a wrapper like the one on http://wiki.postgresql.org/wiki/Talk:Working_with_Git and doing Ugh, hit the wrong key: and executing `git config --global diff.context.command "git-external-diff"` Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On 05/25/2009 07:31 PM, Tom Lane wrote: David Fetter writes: On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: If you'd like to accomplish something *useful* about this, how about pestering git upstream to support diff -c output format? It looks like this is doable with a suitable git configuration file such as $HOME/.gitconfig or (finer grain) a .git/config for the repository :) Cool, let's see one. If we were to put it into a repository config file, that would more or less have the effect of enforcing a project style for diffs, no? Yes and no. You can define that a subset (or all) files use a specific "diff driver" in the repository - unfortunately the definition of that driver has to be done locally. Defining it currently involves installing a wrapper like the one on http://wiki.postgresql.org/wiki/Talk:Working_with_Git and doing Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On 05/25/2009 07:20 PM, Alvaro Herrera wrote: David Fetter wrote: On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: If you'd like to accomplish something *useful* about this, how about pestering git upstream to support diff -c output format? It looks like this is doable with a suitable git configuration file such as $HOME/.gitconfig or (finer grain) a .git/config for the repository :) Can you be more specific on the necessary contents of such file? A very sketchy notion of it is at: http://wiki.postgresql.org/wiki/Talk:Working_with_Git I will try to correct the wording + windows information after eating. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
David Fetter writes: > On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: >> If you'd like to accomplish something *useful* about this, how about >> pestering git upstream to support diff -c output format? > It looks like this is doable with a suitable git configuration file > such as $HOME/.gitconfig or (finer grain) a .git/config for the > repository :) Cool, let's see one. If we were to put it into a repository config file, that would more or less have the effect of enforcing a project style for diffs, no? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
David Fetter wrote: > On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: > > If you'd like to accomplish something *useful* about this, how about > > pestering git upstream to support diff -c output format? > > It looks like this is doable with a suitable git configuration file > such as $HOME/.gitconfig or (finer grain) a .git/config for the > repository :) Can you be more specific on the necessary contents of such file? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
Michael Meskes writes: > On Mon, May 25, 2009 at 12:10:49PM -0400, Tom Lane wrote: >> Consider >> >> if (...) >> macro; >> else >> something-else; > Sure, but some/most/all macros are called as > MACRO; > No real reason there it seems. Well, they are called that way right now. The point of this discussion is making the code safe against easily-foreseeable future changes. Now, I'm privately of the opinion that those macros were a terrible idea to begin with, because of the fact that they contain continue and break statements; not only does that make them not act like self-contained code, but they will break --- silently --- if anyone tries to put them inside nested loops or switch statements. However, that doesn't seem nearly as likely as trying to put them inside if-statements; so I'll just grumble to myself while insisting that we at least keep them safe against that case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Zdenek Kotala writes: > Tom Lane pÃÅ¡e v ne 24. 05. 2009 v 18:46 -0400: >> In any case, the barriers to implementing 8.3-style hash indexes in 8.4 >> are pretty huge: you'd need to duplicate not only the hash AM code, but >> also all the hash functions, and therefore all of the hash pg_amop and >> pg_amproc entries. > I'm not sure if I need duplicate functions. Generally yes but It seems > to me that hash index does not changed functions behavior and they could > be shared at this moment. No, the behavior of the hash functions themselves changed during 8.4. Twice, even: 2008-04-06 12:54 tgl * contrib/dblink/expected/dblink.out, contrib/dblink/sql/dblink.sql, src/backend/access/hash/hashfunc.c, src/include/catalog/catversion.h, src/test/regress/expected/portals.out, src/test/regress/sql/portals.sql: Improve hash_any() to use word-wide fetches when hashing suitably aligned data. This makes for a significant speedup at the cost that the results now vary between little-endian and big-endian machines; which forces us to add explicit ORDER BYs in a couple of regression tests to preserve machine-independent comparison results. Also, force initdb by bumping catversion, since the contents of hash indexes will change (at least on big-endian machines). Kenneth Marshall and Tom Lane, based on work from Bob Jenkins. This commit does not adopt Bob's new faster mix() algorithm, however, since we still need to convince ourselves that that doesn't degrade the quality of the hashing. 2009-02-09 16:18 tgl * src/: backend/access/hash/hashfunc.c, include/catalog/catversion.h, test/regress/expected/polymorphism.out, test/regress/expected/union.out, test/regress/sql/polymorphism.sql: Adopt Bob Jenkins' improved hash function for hash_any(). This changes the contents of hash indexes (again), so bump catversion. Kenneth Marshall So as far as I can see, you need completely separate copies of both hash_any() and the SQL-level functions that call it. I'm not really seeing that the proposed refactoring makes this any easier. You might as well just copy-and-paste all that old code into a separate set of files, and not worry about what is in access/hash.h. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication: Promotion of Standby to Primary
Hello Fujii, my name is Harald Kolb, I'm a colleague of Niranjan and I will continue his community work since he's currently busy with other topics. We are looking for a fast mechanism to activate the switchover. Therfore we prefer to use a signal to trigger the standby to become primary, since in our case the postmaster will be started via fork by a watchdog. The originally intended solution via SIGTERM should be ok. In the case the mechanism will be still based on the trigger file concept, we would require to have in addition a signal to speed up the trigger file detection, since the periodic check might be not fast enough. It should be possible to reuse the SIGHUP signal. If it occurs, the first action should be to check for the existence of the trigger file. In the case it now exists, the switch to master mode is performed. Otherwise the usual re-reading of the configuration files is performed. Are there currently some concrete plannings how the promotion to primary will be solved in the final solution ? BR, Harald. -Original Message- From: ext Fujii Masao [mailto:masao.fu...@gmail.com] Sent: Thursday, May 14, 2009 1:39 PM To: K, Niranjan (NSN - IN/Bangalore) Cc: Kolb, Harald (NSN - DE/Munich); Czichy, Thoralf (NSN - FI/Helsinki); PostgreSQL-development Subject: Re: Synchronous replication: Promotion of Standby to Primary Hi, On Thu, May 14, 2009 at 8:16 PM, Fujii Masao wrote: > Hi, > > On Mon, May 4, 2009 at 11:07 PM, K, Niranjan (NSN - IN/Bangalore) > wrote: >> Hi, >> >> Re-opening the discussion related to triggers to promote standby server. >> In the earlier dicussion, there were 2 proposals, Trigger based on file >> and trigger based on signals. I think there was no conclusion on this. >> http://archives.postgresql.org/pgsql-hackers/2008-12/msg01231.php >> >> According to the proposal, it seems to be better if signals are used as >> we can avoid checking the file existance in the loop. Or the other >> approach is to use inotify (http://en.wikipedia.org/wiki/Inotify). But >> portability to other unix like OS is an issue to look for. >> >> Could you please get back which is being implemented/ considered for >> promotion of standby? > > I had to choose the trigger file approach last time since the patch > was also using warm-standby in part. But, in 8.5, synch-rep is > going to work without warm-standby. So, we can choose also the > trigger based on signals. In this case, In this case, I'm not sure which signal can be used for the trigger. SIGINT/QUIT/TERM/USR1 are already used in postmaster. SIGUSR2 is already reserved for children according to the source comment. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
On Mon, May 25, 2009 at 12:10:49PM -0400, Tom Lane wrote:
> Consider
>
> if (...)
> macro;
> else
> something-else;
> ...
Sure, but some/most/all macros are called as
MACRO;
No real reason there it seems.
> [ thinks for a bit... ] What might be both safe and warning-free
> is to code an explicit empty statement, viz macro body as
>
> if (1) { ... } else ((void) 0)
Will try, but probably not now. :-)
michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: mes...@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: > If you'd like to accomplish something *useful* about this, how about > pestering git upstream to support diff -c output format? It looks like this is doable with a suitable git configuration file such as $HOME/.gitconfig or (finer grain) a .git/config for the repository :) Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On Mon, May 25, 2009 at 12:24:05PM -0400, Tom Lane wrote: > David Fetter writes: > > On Mon, May 25, 2009 at 11:45:33AM -0400, Stephen Frost wrote: > >> I'm all for moving to git, but not until at least the core folks are > >> more familiar with it and have been using it. > > > Which ones aren't familiar and haven't been using it for at least > > the past year? I count two. > > I'm not familiar with it, and neither is Bruce, and frankly that's > entirely sufficient reason not to change now. > > What was more or less agreed to at the developer's meeting was that > we would move towards git in an orderly fashion. The rest have already been moving to it in "an orderly fashion," some for over than a year. > I'm thinking something like six months to a year before cutting over > the core repository. What would gate that? > If you'd like to accomplish something *useful* about this, how about > pestering git upstream to support diff -c output format? I've been pestering them :) Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
David Fetter writes: > On Mon, May 25, 2009 at 11:45:33AM -0400, Stephen Frost wrote: >> I'm all for moving to git, but not until at least the core folks are >> more familiar with it and have been using it. > Which ones aren't familiar and haven't been using it for at least the > past year? I count two. I'm not familiar with it, and neither is Bruce, and frankly that's entirely sufficient reason not to change now. What was more or less agreed to at the developer's meeting was that we would move towards git in an orderly fashion. I'm thinking something like six months to a year before cutting over the core repository. If you'd like to accomplish something *useful* about this, how about pestering git upstream to support diff -c output format? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
Michael Meskes writes:
> On Mon, May 25, 2009 at 11:27:27AM -0400, Tom Lane wrote:
>> You can't just remove the "else", or it's unsafe;
> But why? What does this empty else accomplish?
Consider
if (...)
macro;
else
something-else;
Without the "else" in the macro, this code would be parsed
in a surprising fashion, ie else bound to the wrong if.
I'm afraid that "else {}" might not be any better --- it
might fail outright in this context.
[ thinks for a bit... ] What might be both safe and warning-free
is to code an explicit empty statement, viz macro body as
if (1) { ... } else ((void) 0)
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] problem with plural-forms
I tried to run msgfmt -v ... on solaris and I got following error: Processing file "psql-cs.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "psql-cs.po"... Lines 1311, 1312 (psql-cs.po): incompatible printf-format. 0 format specifier(s) in "msgid", but 1 format specifier(s) in "msgstr". ... ... Problem is in: #: print.c:2351 #, c-format msgid "(1 row)" msgid_plural "(%lu rows)" msgstr[0] "(%lu řádka)" msgstr[1] "(%lu řádky)" msgstr[2] "(%lu řádek)" The problem here is (1 row) instead of (%lu row). When I run msgfmt without -v everything works fine but I think we should fixed it (there are more occurrences of this issue). Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
David Fetter wrote: On Mon, May 25, 2009 at 09:59:14AM -0400, Tom Lane wrote: Dimitri Fontaine writes: Tom Lane writes: The rearrangement might be marginally nicer from a code beautification point of view --- right now we're a bit inconsistent about whether datatype-specific hash functions live in hashfunc.c or in the datatype's utils/adt/ file. But I'm not sure that removing hashfunc.c altogether is an appropriate solution to that, not least because of the loss of CVS history for the functions. I'd be inclined to leave the core hash_any() code where it is, if not all of these functions altogether. I guess someone has to talk about it: git will follow the code even when the file hosting it changes. That might possibly be relevant a year from now; it is 100% irrelevant to a change being proposed for 8.4. It's pretty relevant as far as the schedule goes. I'm not alone thinking that the appropriate place to make this change, given buildfarm support, is at the transition to 8.5. CVS is dead. Long live git! :) That still misses Tom's point, since the change is proposed for 8.4 and at the earliest we would not change SCCMs until after 8.4 is released (and, notwithstanding your eagerness, I suspect it will be rather later). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On Mon, May 25, 2009 at 11:45:33AM -0400, Stephen Frost wrote: > David, > > * David Fetter (da...@fetter.org) wrote: > > It's pretty relevant as far as the schedule goes. I'm not alone > > thinking that the appropriate place to make this change, given > > buildfarm support, is at the transition to 8.5. > > > > CVS is dead. Long live git! :) > > I'm all for moving to git, but not until at least the core folks are > more familiar with it and have been using it. Which ones aren't familiar and haven't been using it for at least the past year? I count two. > I don't believe that experience will be there by the time we open > for 8.5 and a forced march when we have numerous big things > hopefully hitting on the first commitfest seems like a bad idea. Your portrayal of a rough and complicated transition is not terribly well supported by other projects' switches to git. > I would encourage core, committers and contributors to start > becoming familiar with git on the expectation that we'll be making > that move when we open for 8.6/9.0. > > Ideally, there could be an official decision made about when it's > going to happen followed by an announcment when 8.4 is released. > > Thoughts? Here's mine: Git delayed is git denied. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
On Mon, May 25, 2009 at 11:27:27AM -0400, Tom Lane wrote:
> Oh, right, that was Bruce's "improvement" of the COPY code. I was less
> than thrilled with it, but didn't have an easy alternative.
>
> You can't just remove the "else", or it's unsafe; and I'm afraid that
But why? What does this empty else accomplish? I'd like to understand the need
for it.
> changing the macros into "else {}" would still leave us with some
> warnings about empty statements ...
Hmm, apparently no, at least not on my gcc 4.3. As soon as I add the empty
braces, the warning is gone. But without really understanding the need for this
I certainly don't want to make that change. Maybe Bruce can comment.
Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: mes...@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
David, * David Fetter (da...@fetter.org) wrote: > It's pretty relevant as far as the schedule goes. I'm not alone > thinking that the appropriate place to make this change, given > buildfarm support, is at the transition to 8.5. > > CVS is dead. Long live git! :) I'm all for moving to git, but not until at least the core folks are more familiar with it and have been using it. I don't believe that experience will be there by the time we open for 8.5 and a forced march when we have numerous big things hopefully hitting on the first commitfest seems like a bad idea. I would encourage core, committers and contributors to start becoming familiar with git on the expectation that we'll be making that move when we open for 8.6/9.0. Ideally, there could be an official decision made about when it's going to happen followed by an announcment when 8.4 is released. Thoughts? Stephen signature.asc Description: Digital signature
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Tom Lane píše v ne 24. 05. 2009 v 18:46 -0400: > Kenneth Marshall writes: > > On Sat, May 23, 2009 at 02:52:49PM -0400, Zdenek Kotala wrote: > >>> Attached patch cleanups hash index headers to allow compile hasham for > >>> 8.3 version. It helps to improve pg_migrator with capability to migrate > >>> database with hash index without reindexing. > > > How does that work with the updated hash functions without a reindex? > > I looked at this patch and I don't see how it helps pg_migrator at all. > It's just pushing some code and function declarations around. The main important thing is move hash_any/hash_uint32 function from hash am. It should reduce amount of duplicated code necessary for old hash index implementation. Rest is only rearrangement and cleanup. > The rearrangement might be marginally nicer from a code beautification > point of view --- right now we're a bit inconsistent about whether > datatype-specific hash functions live in hashfunc.c or in the datatype's > utils/adt/ file. I personally prefer to keep it in type definition. AM should be independent on types which are installed and data type code should be self contained. > But I'm not sure that removing hashfunc.c altogether is > an appropriatera solution to that, not least because of the loss of CVS > history for the functions. I'd be inclined to leave the core hash_any() > code where it is, if not all of these functions altogether. Until we will have better version control system, hashfunc.c can stay here, but what I need is hashfunc.h. Minimalistic version of this patch is to create hashfuct.h and modified related #include in C and header files. > What does seem useful is to refactor the headers so that datatype hash > functions don't need to include all of the AM's implementation details. > But this patch seems to do both more and less than that --- I don't > think it's gotten rid of all external #includes of access/hash.h, and > in any case moving the function code is not necessary to that goal. Agree, I will prepare minimalistic version with hashfunc.h only. > In any case, the barriers to implementing 8.3-style hash indexes in 8.4 > are pretty huge: you'd need to duplicate not only the hash AM code, but > also all the hash functions, and therefore all of the hash pg_amop and > pg_amproc entries. I'm not sure if I need duplicate functions. Generally yes but It seems to me that hash index does not changed functions behavior and they could be shared at this moment. > Given the close-to-zero usefulness of hash indexes > in production installations, I don't think it's worth the trouble. It > would be much more helpful to look into supporting 8.3-compatible GIN > indexes. Agree, I wanted to quickly verify function naming collision problem and HASH index seems to me better candidate for this basic test. GIN has priority. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
On Mon, May 25, 2009 at 09:59:14AM -0400, Tom Lane wrote: > Dimitri Fontaine writes: > > Tom Lane writes: > >> The rearrangement might be marginally nicer from a code > >> beautification point of view --- right now we're a bit > >> inconsistent about whether datatype-specific hash functions live > >> in hashfunc.c or in the datatype's utils/adt/ file. But I'm not > >> sure that removing hashfunc.c altogether is an appropriate > >> solution to that, not least because of the loss of CVS history > >> for the functions. I'd be inclined to leave the core hash_any() > >> code where it is, if not all of these functions altogether. > > > I guess someone has to talk about it: git will follow the code > > even when the file hosting it changes. > > That might possibly be relevant a year from now; it is 100% > irrelevant to a change being proposed for 8.4. It's pretty relevant as far as the schedule goes. I'm not alone thinking that the appropriate place to make this change, given buildfarm support, is at the transition to 8.5. CVS is dead. Long live git! :) Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 11:02:53AM -0400, Tom Lane wrote: > Robert Haas writes: > > This is all much more complicated than what I proposed, and I fail > > to see what it buys us. I'd say that you're just reinforcing the > > point I made upthread, which is that insisting that XML is the > > only way to get more detailed information will just create a > > cottage industry of beating that XML output format into > > submission. > > The impression I have is that (to misquote Churchill) XML is the > worst option available, except for all the others. We need > something that can represent a fairly complex data structure, easily > supports addition or removal of particular fields in the structure > (including fields not foreseen in the original design), is not hard > for programs to parse, and is widely supported --- ie, "not hard" > includes "you don't have to write your own parser, in most > languages". How many realistic alternatives are there? JSON for one, and it's *much* lighter in just about every way. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
Michael Meskes writes:
> On Mon, May 25, 2009 at 10:19:40AM -0400, Tom Lane wrote:
>> That sounds both dangerous and against our coding conventions. The
>> standard way to do that is "do { ... } while (0)"
> Which won't work here as the macros have continue and break commands in them.
Oh, right, that was Bruce's "improvement" of the COPY code. I was less
than thrilled with it, but didn't have an easy alternative.
You can't just remove the "else", or it's unsafe; and I'm afraid that
changing the macros into "else {}" would still leave us with some
warnings about empty statements ...
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
Joshua Tolley writes: > I'm not sure I see why it would be less flexible. I'm imagining we define some > record type, and a function that returns a set of those records. I'm unimpressed by the various proposals to change EXPLAIN into a function. Quoting the command-to-explain is going to be a pain in the neck. And can you really imagine using it manually, especially if it returns so many fields that you *have to* write out the list of fields you actually want, else the result is unreadable? It's going to be just as much of something you can only use through a helper application as the XML way would be. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
On Mon, May 25, 2009 at 10:19:40AM -0400, Tom Lane wrote:
> Michael Meskes writes:
> > - some combination of signed and unsigned: ~ 600
> > Are we really sure that *all* compilers out there do handle this
> > correctly?
>
> The behavior is spelled out in the C spec, and always has been. You
> might as well worry if they handle "if" correctly.
Well this is probably because I got bitten by this once. Okay, granted it was
very long ago and the compiler was not state of the art.
> > There are some #defines of the form
> > #define foo if(1) { ... } else
> > that are called as foo;
>
> > I see the need for the macro to expand as block, but what use hase the
> > empty
> > else?
>
> That sounds both dangerous and against our coding conventions. The
> standard way to do that is "do { ... } while (0)"
Which won't work here as the macros have continue and break commands in them.
Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: mes...@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
> The impression I have is that (to misquote Churchill) XML is the worst > option available, except for all the others. We need something that can > represent a fairly complex data structure, easily supports addition or > removal of particular fields in the structure (including fields not > foreseen in the original design), is not hard for programs to parse, > and is widely supported --- ie, "not hard" includes "you don't have to > write your own parser, in most languages". How many realistic > alternatives are there? > > One realistic alternative may be JSON: parsers for nearly all languages are freely available; everything web-affected speeks it perfectly; it's easier on the eye then XML, less bytes per information, addition and removal of fields as well as complex structures are possible. Harald -- GHUM Harald Massa persuadere et programmare Harald Armin Massa Spielberger Straße 49 70435 Stuttgart 0173/9409607 no fx, no carrier pigeon - LASIK good, steroids bad?
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 10:55:48AM -0400, Tom Lane wrote: > Joshua Tolley writes: > > The Oracle version, as it fills the table of explain results, gives > > each number an id and the id of its parent row, which behavior we > > could presumably copy. I'm definitely keen to keep a human-readable > > EXPLAIN such as we have now, to augment the table-based proposal, but > > a table would provide the more flexible output we'd need for more > > detailed reporting, a simple interface for applications to consume the > > EXPLAIN data without human intervention, and a convenient platform > > from whence the data can be transformed to XML, JSON, etc. for those > > that are so inclined. > > I would think a table would be considerably *less* flexible --- you > could not easily change the output column set. Unless you're imagining > just dumping something equivalent to the current output into a text > column. Which would be flexible, but it'd hardly have any of the > other desirable properties you list. I'm not sure I see why it would be less flexible. I'm imagining we define some record type, and a function that returns a set of those records. The fields in the record would include data element this version of explain could possibly return. Then you call a function to explain a query, passing it some options to select which of those data elements you're interested in. The function returns the same data type at each call, filling with NULLs the fields you've told it you're uninterested in. Changes between versions would modify this data type, but provided consumer applications have specified the columns they're interested in (rather than, say, SELECT *) that shouldn't bother anyone. The functions to change this into some other format would probably need to be more intelligent than just that. That seems a fair price to pay. - Josh / eggyknap signature.asc Description: Digital signature
Re: [HACKERS] generic options for explain
Robert Haas writes: > This is all much more complicated than what I proposed, and I fail to > see what it buys us. I'd say that you're just reinforcing the point I > made upthread, which is that insisting that XML is the only way to get > more detailed information will just create a cottage industry of > beating that XML output format into submission. The impression I have is that (to misquote Churchill) XML is the worst option available, except for all the others. We need something that can represent a fairly complex data structure, easily supports addition or removal of particular fields in the structure (including fields not foreseen in the original design), is not hard for programs to parse, and is widely supported --- ie, "not hard" includes "you don't have to write your own parser, in most languages". How many realistic alternatives are there? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I'm not entirely sure why we put a range limit on time values at all, > but given that we do, it'd probably be a good idea to check the range > in the recv functions. I'm inclined to fix this for 8.4, but not > back-patch because of compatibility considerations. Any objections > out there? Are we confident it can't be abused to impact other clients connecting or break the back-end in some way? More specifically, could it be a security issue? Havn't looked at it yet, but getting what sounded like corrupted data back out could be bad.. Stephen signature.asc Description: Digital signature
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 07:14:56AM -0400, Robert Haas wrote:
> Many people who responded to this
> thread were fine with the idea of some sort of options syntax, but we
> had at least four different proposals for how to implement it:
>
> Robert Haas: EXPLAIN (foo 'bar', baz 'bletch', ...) query
> Pavel Stehule: explain_query(query, options...) [exact format of
> options not specified]
> Andrew Dunstan: SET explain_format = 'foo, baz'; EXPLAIN query
> Josh Tolley: EXPLAIN (foo, baz, ...) query [also suggested by me as an
> idea I rejected]
I hadn't actually caught that there were two ideas on the table with syntax
similar to "EXPLAIN (...) ", and don't mean to champion either of the
two specifically (at least until I've read closely enough to note the
difference). I just kinda liked the "EXPLAIN (some options of some sort)
" syntax better than other proposals.
That said, I think I'm changing my vote in favor of Pavel. It's my guess that
some variant of his version would be the easiest to make compliant with the
bit I'm most interested in, which is not being limited to, say,
XML/JSON/YAML/etc. in the output. Applications that use PostgreSQL will of
necessity need to know how to handle data presented in tables, so let's
present our explain results as a table. As has been said in other branches of
this thread, that way we don't force applications also to support
XML/JSON/YAML/etc. We might consider providing functions to convert the
tabular result to one or more of those formats, but at its inception, the data
should live as tuples in a relation.
In other messages, I've advocated actually inserting the data into a table. I
think that was a mistake. Who makes the table? What's it called? What schema
is it in? Who cleans it up when we're done with it? ...etc. I'd much rather
see a bunch of rows returned as a set, which I can then insert into a table,
pass into a function for reformatting, or just consume in an application.
All of which leads me to this variant of the functional approach as my answer:
SELECT * FROM pg_explain_query("", );
I could then do things like this:
CREATE TABLE explain_results AS SELECT * FROM pg_explain_query(...);
and this:
SELECT xmlify_a_record(pg_explain_query(...));
- Josh / eggyknap
signature.asc
Description: Digital signature
Re: [HACKERS] generic options for explain
Joshua Tolley writes: > The Oracle version, as it fills the table of explain results, gives > each number an id and the id of its parent row, which behavior we > could presumably copy. I'm definitely keen to keep a human-readable > EXPLAIN such as we have now, to augment the table-based proposal, but > a table would provide the more flexible output we'd need for more > detailed reporting, a simple interface for applications to consume the > EXPLAIN data without human intervention, and a convenient platform > from whence the data can be transformed to XML, JSON, etc. for those > that are so inclined. I would think a table would be considerably *less* flexible --- you could not easily change the output column set. Unless you're imagining just dumping something equivalent to the current output into a text column. Which would be flexible, but it'd hardly have any of the other desirable properties you list. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] No sanity checking performed on binary TIME parameters.
Andrew McNamara writes: > When submitting a query via the V3 binary protocol (PQexecParams, > paramFormats[n]=1), it appears the PostgreSQL server performs no range > checking on the passed values. A quick look at time_recv() shows this is true, and timetz_recv() checks neither the time nor the zone component. > Passing values greater than 24 hours > results in unpredictable results (dumps that cannot be restored, > strange output when printing the column in psql, etc). I'm not entirely sure why we put a range limit on time values at all, but given that we do, it'd probably be a good idea to check the range in the recv functions. I'm inclined to fix this for 8.4, but not back-patch because of compatibility considerations. Any objections out there? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Compiler warning cleanup
Peter Eisentraut píše v ne 24. 05. 2009 v 00:40 +0300: > I think this is not the best way to do it, because it will confuse pgindent > and editors and such. The DATA() macros in include/catalog have this solved; > see include/catalog/genbki.sh. Attached variant with faked extern without ; which is what genbki.sh uses. Zdenek diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c pgsql.orig/src/backend/utils/adt/tsquery_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.594747865 +0200 --- pgsql.orig/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.599072981 +0200 *** *** 188,194 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! } CMPFUNC(tsquery_lt, res < 0); CMPFUNC(tsquery_le, res <= 0); --- 188,196 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! }\ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable CMPFUNC(tsquery_lt, res < 0); CMPFUNC(tsquery_le, res <= 0); diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c pgsql.orig/src/backend/utils/adt/tsvector_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.596082053 +0200 --- pgsql.orig/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.599263998 +0200 *** *** 172,178 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } TSVECTORCMPFUNC(lt, <, BOOL); TSVECTORCMPFUNC(le, <=, BOOL); --- 172,180 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } \ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable TSVECTORCMPFUNC(lt, <, BOOL); TSVECTORCMPFUNC(le, <=, BOOL); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Warnings in compile
Michael Meskes writes:
> - some combination of signed and unsigned: ~ 600
> Are we really sure that *all* compilers out there do handle this correctly?
The behavior is spelled out in the C spec, and always has been. You
might as well worry if they handle "if" correctly.
> There are some #defines of the form
> #define foo if(1) { ... } else
> that are called as foo;
> I see the need for the macro to expand as block, but what use hase the empty
> else?
That sounds both dangerous and against our coding conventions. The
standard way to do that is "do { ... } while (0)"
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Doubt
On May 25, 2009, at 9:41 AM, jibin jose wrote: which tool is used as profiler in postgresql There is some code in the backend to support gprof; I know that some people have had good luck with oprofile, too, which doesn't require any special support. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Doubt
which tool is used as profiler in postgresql
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Dimitri Fontaine writes: > Tom Lane writes: >> The rearrangement might be marginally nicer from a code beautification >> point of view --- right now we're a bit inconsistent about whether >> datatype-specific hash functions live in hashfunc.c or in the datatype's >> utils/adt/ file. But I'm not sure that removing hashfunc.c altogether is >> an appropriate solution to that, not least because of the loss of CVS >> history for the functions. I'd be inclined to leave the core hash_any() >> code where it is, if not all of these functions altogether. > I guess someone has to talk about it: git will follow the code even when > the file hosting it changes. That might possibly be relevant a year from now; it is 100% irrelevant to a change being proposed for 8.4. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feedback on writing extensible modules
Dimitri Fontaine writes:
> And currently calling SPI_connect() from _PG_init will crash the
> backend. I'll try to obtain a gdb backtrace, I've just been told about
> pre_auth_delay and post_auth_delay parameters.
Here we go:
(gdb) handle SIGABRT nopass
SignalStop Print Pass to program Description
SIGABRT Yes Yes No Aborted
(gdb) continue
Program received signal SIGABRT, Aborted.
0xb802d424 in __kernel_vsyscall ()
(gdb) bt
#0 0xb802d424 in __kernel_vsyscall ()
#1 0xb7e7c640 in raise () from /lib/i686/cmov/libc.so.6
#2 0xb7e7dfa1 in abort () from /lib/i686/cmov/libc.so.6
#3 0x082dadde in ExceptionalCondition (conditionName=0x83cbfe0 "!(((context)
!= ((void *)0) && (Node*)((context)))->type) == T_AllocSetContext",
errorType=0x830bc09 "BadArgument", fileName=0x83be166 "mcxt.c",
lineNumber=507) at assert.c:57
#4 0x082f8abb in MemoryContextAlloc (context=0x0, size=448) at mcxt.c:507
#5 0x081a93a3 in SPI_connect () at spi.c:81
#6 0xb582cf15 in _PG_init () at pre_prepare.c:150
#7 0x082df913 in internal_load_library (libname=0x9808da4
"/home/dim/pgsql/8.3/lib/plugins/pre_prepare.so") at dfmgr.c:296
#8 0x082dfc38 in load_file (filename=0x9809d00 "$libdir/plugins/pre_prepare",
restricted=1 '\001') at dfmgr.c:153
#9 0x082e7554 in load_libraries (libraries=,
gucname=0x9809d00 "$libdir/plugins/pre_prepare", restricted=1 '\001') at
miscinit.c:1185
#10 0x08233ce2 in PostgresMain (argc=4, argv=0x9807fb8, username=0x9807f90
"dim") at postgres.c:3314
#11 0x0820054c in ServerLoop () at postmaster.c:3207
#12 0x0820124b in PostmasterMain (argc=3, argv=0x97f1bd8) at postmaster.c:1029
#13 0x081b2b39 in main (argc=3, argv=0x97f1bd8) at main.c:188
And I'm runnin a CVS version of 8.3 I'm not sure is the last update in
the branch, so here's what I have at mcxt.c:507
504 void *
505 MemoryContextAlloc(MemoryContext context, Size size)
506 {
507 AssertArg(MemoryContextIsValid(context));
508
509 if (!AllocSizeIsValid(size))
510 elog(ERROR, "invalid memory alloc request size %lu",
511 (unsigned long) size);
That's with attached patch to pre_prepare.c from pgfoundry:
http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/preprepare/preprepare/
If you need any more information from me, or for me to rerun with
another server version, please ask. I'm very interrested in being able
to prepare a query at local_preload_libraries time, if possible in 8.3
and following releases.
Regards,
--
dim
Index: pre_prepare.c
===
RCS file: /cvsroot/preprepare/preprepare/pre_prepare.c,v
retrieving revision 1.1
diff -p -u -r1.1 pre_prepare.c
--- pre_prepare.c 13 May 2009 20:54:04 - 1.1
+++ pre_prepare.c 25 May 2009 13:37:52 -
@@ -35,6 +35,7 @@
PG_MODULE_MAGIC;
+static bool pre_prepare_at_init = NULL;
static char *pre_prepare_relation = NULL;
void _PG_init(void);
@@ -125,6 +126,15 @@ int pre_prepare_all() {
*/
void
_PG_init(void) {
+ DefineCustomBoolVariable("preprepare.at_init",
+ "Do we prepare the statements at backend init start",
+ "You have to setup local_preload_libraries too",
+ &pre_prepare_at_init,
+ PGC_USERSET,
+ NULL,
+ NULL);
+ EmitWarningsOnPlaceholders("prepare.at_init");
+
DefineCustomStringVariable("preprepare.relation",
"Table name where to find statements to prepare",
"Can be schema qualified, must have columns \"name\" and \"statement\"",
@@ -132,8 +142,21 @@ _PG_init(void) {
PGC_USERSET,
NULL,
NULL);
-
EmitWarningsOnPlaceholders("prepare.relation");
+
+ if( pre_prepare_at_init ) {
+int err;
+
+err = SPI_connect();
+if (err != SPI_OK_CONNECT)
+ elog(ERROR, "SPI_connect: %s", SPI_result_code_string(err));
+
+pre_prepare_all();
+
+err = SPI_finish();
+if (err != SPI_OK_FINISH)
+ elog(ERROR, "SPI_finish: %s", SPI_result_code_string(err));
+ }
}
/*
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Hi, Tom Lane writes: > The rearrangement might be marginally nicer from a code beautification > point of view --- right now we're a bit inconsistent about whether > datatype-specific hash functions live in hashfunc.c or in the datatype's > utils/adt/ file. But I'm not sure that removing hashfunc.c altogether is > an appropriate solution to that, not least because of the loss of CVS > history for the functions. I'd be inclined to leave the core hash_any() > code where it is, if not all of these functions altogether. I guess someone has to talk about it: git will follow the code even when the file hosting it changes. It's not all magic though: http://kerneltrap.org/node/11765 "And when using git, the whole 'keep code movement separate from changes' has an even more fundamental reason: git can track code movement (again, whether moving a whole file or just a function between files), and doing a 'git blame -C' will actually follow code movement between files. It does that by similarity analysis, but it does mean that if you both move the code *and* change it at the same time, git cannot see that 'oh, that function came originally from that other file', and now you get worse annotations about where code actually originated." Having better tools maybe could help maintain the high quality standards that are established code wise, too. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication: Promotion of Standby to Primary
Hi, On Mon, May 25, 2009 at 7:10 PM, Kolb, Harald (NSN - DE/Munich) wrote: > Hello Fujii, > > my name is Harald Kolb, I'm a colleague of Niranjan and I will continue > his community work since he's currently busy with other topics. > We are looking for a fast mechanism to activate the switchover. Therfore > we prefer to use a signal to trigger the standby to become primary, > since in our case the postmaster will be started via fork by a watchdog. > The originally intended solution via SIGTERM should be ok. > In the case the mechanism will be still based on the trigger file > concept, we would require to have in addition a signal to speed up the > trigger file detection, since the periodic check might be not fast > enough. To be honest, I don't think that such quick detection shortens the failover time so much (probably it would be shortened only a few ms). Instead, we should reduce the time of checkpoint which occurs at the end of recovery/failover. It might take a few minutes to end. So, I'd like to emphasize user-friendliness rather than the failover time in the choice of the way of activation. What is your opinion? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] forward declaration in c
2009/5/25 Martijn van Oosterhout : > On Mon, May 25, 2009 at 01:20:05PM +0200, Pavel Stehule wrote: >> Hello >> >> I can't to find fine syntax for cyclic declaration: > > If you mean forward declaraions of a type, just: > > struct ParseState > > will do, then: > >> typedef Node *(*TransformColumnRef_hook_type) (struct ParseState *pstate, >> ColumnRef *cref); > > With the "struct" keyword added. > >> with empty declaration typedef struct ParseState; > yes, good advice thank you Pavel > It might work with: > > typedef struct ParseState ParseState; > > But then you can't use typedef in the actual declaraion (AFAIK you can > only forward declare structs, enums and such, but not typedefs). > > Have a nice day, > -- > Martijn van Oosterhout http://svana.org/kleptog/ >> Please line up in a tree and maintain the heap invariant while >> boarding. Thank you for flying nlogn airlines. > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.9 (GNU/Linux) > > iD8DBQFKGoOMIB7bNG8LQkwRApDrAKCKLSPWKoDVu0xemsbijTef/wTo3QCgjzAM > WPgB/FGqCXiANvrEM+rePfA= > =nRZn > -END PGP SIGNATURE- > > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] forward declaration in c
On Mon, May 25, 2009 at 01:20:05PM +0200, Pavel Stehule wrote: > Hello > > I can't to find fine syntax for cyclic declaration: If you mean forward declaraions of a type, just: struct ParseState will do, then: > typedef Node *(*TransformColumnRef_hook_type) (struct ParseState *pstate, > ColumnRef *cref); With the "struct" keyword added. > with empty declaration typedef struct ParseState; It might work with: typedef struct ParseState ParseState; But then you can't use typedef in the actual declaraion (AFAIK you can only forward declare structs, enums and such, but not typedefs). Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
[HACKERS] forward declaration in c
Hello
I can't to find fine syntax for cyclic declaration:
.
typedef Node *(*TransformColumnRef_hook_type) (ParseState *pstate,
ColumnRef *cref);
typedef struct ParseState
{
struct ParseState *parentParseState;/* stack link */
const char *p_sourcetext; /* source text, or NULL if not
available */
List *p_rtable; /* range table so far */
List *p_joinexprs;/* JoinExprs for RTE_JOIN
p_rtable entries */
List *p_joinlist; /* join items so far (will
become FromExpr
*
node's fromlist) */
List *p_relnamespace; /* current namespace for relations */
List *p_varnamespace; /* current namespace for columns */
List *p_ctenamespace; /* current namespace for common
table exprs */
List *p_future_ctes; /* common table exprs not yet
in namespace */
List *p_windowdefs; /* raw representations of
window clauses */
Oid*p_paramtypes; /* OIDs of types for
$n parameter symbols */
int p_numparams;/* allocated size of
p_paramtypes[] */
int p_next_resno; /* next targetlist
resno to assign */
List *p_locking_clause; /* raw FOR UPDATE/FOR
SHARE info */
Node *p_value_substitute; /* what to replace
VALUE with, if any */
boolp_variableparams;
boolp_hasAggs;
boolp_hasWindowFuncs;
boolp_hasSubLinks;
boolp_is_insert;
boolp_is_update;
Relationp_target_relation;
RangeTblEntry *p_target_rangetblentry;
TransformColumnRef_hook_typetransformColumnRef_hook;
} ParseState;
with empty declaration typedef struct ParseState;
I got errors:
In file included from ../../../src/include/catalog/heap.h:18,
from parse_clause.c:20:
../../../src/include/parser/parse_node.h:79: warning: useless storage
class specifier in empty declaration
../../../src/include/parser/parse_node.h:81: error: expected ‘)’
before ‘*’ token
../../../src/include/parser/parse_node.h:109: error: expected
specifier-qualifier-list before ‘TransformColumnRef_hook_type’
thank you
Pavel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
On Mon, May 25, 2009 at 6:24 AM, Dimitri Fontaine wrote: > I think the summary here is to say that we want two modes of operations: > - the current one, which continues to get refinements > > - a new one conveying all possible information in machine readable > formats, possibly with some tools to handle it easily: XML output and > maybe XSLT stylesheets I don't agree with that summary. Many people who responded to this thread were fine with the idea of some sort of options syntax, but we had at least four different proposals for how to implement it: Robert Haas: EXPLAIN (foo 'bar', baz 'bletch', ...) query Pavel Stehule: explain_query(query, options...) [exact format of options not specified] Andrew Dunstan: SET explain_format = 'foo, baz'; EXPLAIN query Josh Tolley: EXPLAIN (foo, baz, ...) query [also suggested by me as an idea I rejected] Tom Lane was the only person to suggest that we only ever need one more option to EXPLAIN and that it should be called XML. Even though I prefer my format to the other options suggested (which I would probably rank in order of descending preference Josh-Andrew-Pavel), I am actually someone encouraged that we might have some kind of fragile consensus that an extensible options syntax is useful (a point that Andres Freund and Greg Smith also seemed to agree with). >> Anyway, I'm suprised by the reaction to this patch, but I'll drop it. >> I would like to make the EXPLAIN syntax more powerful for command-line >> use, and I'd implement XML format and JSON along the way just for >> completeness. But I don't have much interest in creating an XML >> output format that is the ONLY way of getting more information, >> because I'm a command-line user and it does me no good at all. :-( > > That's only because you seem to be thinking that having core PostgreSQL > do the first half of the work means you as a user will have to do the > second part. I assume pgadmin and phppgadmin developers will offer their > users some graphical approach to the output reading, with dynamic > filtering, eg. > > I don't see anything stopping you to provide a simple way to have the > same facility into psql. You can already have the query output filtered > by any script you want this way: > =# \o |my_presentation_script
Re: [HACKERS] generic options for explain
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [Sent by mistake to Robert Haas only at first try. No cure for fat fingers, I guess] On Sun, May 24, 2009 at 04:05:18PM -0400, Robert Haas wrote: [...] > I think XML output format is a complete distraction from the real > issue here, which is that there are certain pieces of information that > are sometimes useful but are not useful enough to justify including > them in the EXPLAIN output 100% of the time. By just punting all that > stuff to EXPLAIN XML, we're just saying that we're not interested in > creating a workable set of options to allow users to pick and choose > the information they care about - so instead we're going to dump a > huge chunk of unreadable XML and then make it the user's problem to > find a tool that will extract the details that they care about. Boo, > hiss. +1 In my experience, this happens with "XML the data description language" many times, but I haven' seen the problem as well-stated as in your mail. Hard for humans to read, hard for machines to read, and often trying to solve a problem it can't (in this case, selecting the needed information _beforehand_). Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKGntjBcgs9XrR2kYRAk1/AJ4rnZFnU4PFM8AJkaYYYLRInYHJDQCbBbt2 lTwxydBBnXP1MgDxz+vcpM8= =o2qW -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] generic options for explain
Hi, After having read all the followups I already received, I prefer to answer to this particular message. Robert Haas writes: > On Sun, May 24, 2009 at 3:09 PM, Tom Lane wrote: >> On the other side of the coin, I'm strongly against inventing more than >> one new output format for EXPLAIN, and so any argument that depends on >> examples such as "xml vs json" is falling on deaf ears here. I think >> that we only need an XML option, and so EXPLAIN ANALYZE XML ... doesn't >> seem untenable. What other options than those do you really need? >> Not ones to add or remove output fields; we'd expect the client to >> ignore fields it doesn't care about. > > It's not just about me; we've had many requests for new EXPLAIN > features. [...] > I think XML output format is a complete distraction from the real > issue here, which is that there are certain pieces of information that > are sometimes useful but are not useful enough to justify including > them in the EXPLAIN output 100% of the time. I think the summary here is to say that we want two modes of operations: - the current one, which continues to get refinements - a new one conveying all possible information in machine readable formats, possibly with some tools to handle it easily: XML output and maybe XSLT stylesheets > Anyway, I'm suprised by the reaction to this patch, but I'll drop it. > I would like to make the EXPLAIN syntax more powerful for command-line > use, and I'd implement XML format and JSON along the way just for > completeness. But I don't have much interest in creating an XML > output format that is the ONLY way of getting more information, > because I'm a command-line user and it does me no good at all. :-( That's only because you seem to be thinking that having core PostgreSQL do the first half of the work means you as a user will have to do the second part. I assume pgadmin and phppgadmin developers will offer their users some graphical approach to the output reading, with dynamic filtering, eg. I don't see anything stopping you to provide a simple way to have the same facility into psql. You can already have the query output filtered by any script you want this way: =# \o |my_presentation_script
Re: [HACKERS] proposal: SQL parser integration to PL/pgSQL
2009/5/25 Pavel Stehule : > 2009/5/24 Tom Lane : >> Pavel Stehule writes: >>> ==Steps== >>> 1. add hook to analyser (transform stage) to substitute unknown >>> columnref by param - when analyser detect unknown columnref, then call >>> callback, that returns possible para node or NULL (when external >>> environment doesn't have a variable). Returned param should be typed >>> or unknown (for polymorphic params). >> >> IMHO the hook definition should support both the case of external >> variables taking precedence over query variables and vice versa. >> I don't think the core parser should be forcing that decision. In any >> case we'd probably need both options for plpgsql, so as to be able to >> support both traditional and Oracle-compatible behavior. > > good idea > the problem is place for hook variable - it should be in ParseState because we need call hooked functions from environments that install hook. Pavel >> >> I'd be inclined to do that by letting the hook function interpose >> itself between the parser and the regular transformColumnRef processing, >> so that it can call the regular transformColumnRef processing either >> before or after doing its external lookups. Giving it control only >> after the regular processing fails would mean there's no way to let >> external variables take precedence. >> > > ok > >>> 2. add special modes to sql parser: >> >> None of those seem like a good idea to me. The only part that seems >> useful is warning about conflicts between external variables and query >> variables. That can be implemented by the hook function itself, if we >> define the hook behavior as above. >> > > there is minimal one necessary - for polymorphic variables, we know > name, but we don't know type. And without types, we can't to transform > correctly functions. > > regards > Pavel Stehule > > >> regards, tom lane >> > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Warnings in compile
Hi,
sitting here on my flight back I went through the list of all warnings gcc
spits out when using -Wextra. There are a whole lot of them (~ 1700) that
mostly (except one) fall into one of four classes:
- unused parameters: ~ 600
- some combination of signed and unsigned: ~ 600
Are we really sure that *all* compilers out there do handle this correctly?
- missing initializer: ~ 500
Probably coming from us initialising structures only partially.
- empty body in else statement: 14 all in backend/commands/copy.c
There are some #defines of the form
#define foo if(1) { ... } else
that are called as foo;
I see the need for the macro to expand as block, but what use hase the empty
else?
I assume that the only warning outside these classes is a false positive.
Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: mes...@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] search_path vs extensions
Hi, Preliminary note: I'm using the term "extension" as if it's what we already agree to call them, feel free to ignore this and use whatever term you see fit. We'll have the naming issue tackled, please not now though. Following-up to discussions we had at the Developer Meeting and subsequent pub events, I'd like us to agree upon the relations of extensions and search_path. We basically have to choose one of those: Proposal: do nothing What's good about it: it's already there, folks! What's not good about it: Users are alone on deciding where to put what, and the system won't help them: either public is a complete mess, or they have to manually care about search_path for their extensions and their own application needs. Installations where DBA and application folks are separate teams will suffer, ones where the application is heavily using schemas will suffer too. Proposal: pg_extension, a new dedicated system schema for extensions Good: It's easy to see SQL objects (\df) of extensions (think contribs) you installed, and as extension developpers are required to use it, you don't have to care about it any more. As you have only one namespace for everyone, the collisions are detected early. Not good: As you have only one namespace for everyone, collisions prevent users from installing several extensions using the same SQL object name, so we'd need a way for extension authors to share a catalog of free names, like internally we do for systems OIDs in the bootstrap, IIUC. But in a distributed fashion. We would have to add ways for the user to see which extension which object belongs to, so you'd have extension | schema | object_name columns in all \dX things, e.g. Proposal: allow user schema to behave the same as pg_catalog Good: Tell the system your schema is implicit and be done with it, object searching won't need users to manage search_path explicitly. Not good: Breaking existing application code by adding an implicit schema in an existing database is damn too easy. And how to choose if the implicit schemas are to be searched in before or after the search_path? Proposal: Separate search_path into components: pre_search_path, search_path, post_search_path Good: This allows to easily separate who changes what: typically DBAs will edit pre and post search_path components while application will care about search_path the same way as now. Not good: 2 new GUCs (but no new semantics, and defaults to empty) My vote is to go with the pre/post search_path components proposal as it's the one allowing the more flexibility, and we tend to value this a lot around here. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
