Re: [HACKERS] WIP: URI connection string support for libpq
On Mon, Dec 12, 2011 at 5:05 PM, David E. Wheeler da...@justatheory.com wrote: On Dec 12, 2011, at 3:55 PM, Peter van Hardenberg wrote: only a nearly insurmountable mailing list thread prevents it. What happened to SexQL? Case in point. -- Peter van Hardenberg San Francisco, California Everything was beautiful, and nothing hurt. -- Kurt Vonnegut -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Mon, Dec 12, 2011 at 9:25 PM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-12-12 at 16:51 -0800, Peter van Hardenberg wrote: You don't need a new PL to do that. The existing PLs can also parse JSON. So that's not nearly enough of a reason to consider adding this new PL. PL/V8 is interesting because it is very fast, sandboxed, and well embedded with little overhead. My experience with PL/Python and PL/Perl has not been thus, and although they are handy if you want to break out and run system work, they're not the kind of thing I'd consider for defining performant operators with. I feel PL/V8 has promise in that area. -- Peter van Hardenberg San Francisco, California Everything was beautiful, and nothing hurt. -- Kurt Vonnegut -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Mon, Dec 5, 2011 at 3:12 PM, Bruce Momjian br...@momjian.us wrote: Where are we with adding JSON for Postgres 9.2? We got bogged down in the data representation last time we discussed this. We should probably have a wiki page titled JSON datatype status to help break the cycle we're in: * Someone asks about the status of JSON * Various ideas are suggested * Patches are posted (maybe) * More discussion about fundamental issues ensues * Nothing is accomplished (as far as adding JSON to Postgres core) There are several JSON implementations for Postgres floating around, including: * http://pgxn.org/dist/pg-json/ : Mentioned in previous posts; a JSON library based on Jansson supporting path subscript and equality testing * http://git.postgresql.org/gitweb/?p=json-datatype.git;a=summary : The JSON datatype I implemented for Google Summer of Code 2010. It has the most features of any implementation I'm aware of, but: * Is in the form of a contrib module * Preserves input text verbatim, a guarantee that will be broken by more efficient implementations * http://git.postgresql.org/gitweb/?p=json-datatype.git;a=shortlog;h=refs/heads/json2 : My rewrite of the JSON module that condenses input (but still stores it as text) and addresses the issue of JSON when either the server or client encoding is not UTF-8. Needs more features and documentation, but like my other implementation, may not be quite what we want. Issues we've encountered include: * Should JSON be stored as binary or as text? * How do we deal with Unicode escapes and characters if the server or client encoding is not UTF-8? Some (common!) character encodings have code points that don't map to Unicode. Also, the charset conversion modules do not provide fast entry points for converting individual characters; each conversion involves a funcapi call. --- In an application I'm working on, I store JSON-encoded objects in a PostgreSQL database (using TEXT). I do so because it allows me to store non-relational data that is easy for my JavaScript code to work with. However, I fail to see much benefit of a JSON type. When I need to work with the data in PHP, C, or Haskell, I use JSON parsing libraries available in each programming language. Although being able to transform or convert JSON data within SQL might be convenient, I can't think of any compelling reason to do it in my case. Can someone clarify why a JSON type would be useful, beyond storage and validation? What is a real-world, *concrete* example of a problem where JSON manipulation in the database would be much better than: * Using the application's programming language to manipulate the data (which it does a lot already) ? * Using CouchDB or similar instead of PostgreSQL? - Joey -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server
Tom Lane wrote: Shigeru Hanada shigeru.han...@gmail.com writes: (2011/12/12 22:59), Robert Haas wrote: ... I feel like we might need a system here that allows for more explicit user control about what to push down vs. not, rather than assuming we'll be able to figure it out behind the scenes. Agreed. How about to add a per-column boolean FDW option, say pushdown, to pgsql_fdw? Users can tell pgsql_fdw that the column can be pushed down safely by setting this option to true. [ itch... ] That doesn't seem like the right level of granularity. ISTM the problem is with whether specific operators have the same meaning at the far end as they do locally. If you try to attach the flag to columns, you have to promise that *every* operator on that column means what it does locally, which is likely to not be the case ever if you look hard enough. Plus, having to set the flag on each individual column of the same datatype seems pretty tedious. I don't have a better idea to offer at the moment though. Trying to attach such a property to operators seems impossibly messy too. If it weren't for the collations issue, I might think that labeling datatypes as being compatible would be a workable approximation. Maybe I'm missing something, but if pushdown worked as follows: - Push down only system functions and operators on system types. - Only push down what is guaranteed to work. then the only things we would miss out on are encoding- or collation-sensitive string operations. Is that loss so big that it warrants a lot of effort? Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] review: CHECK FUNCTION statement
Pavel Stehule wrote: One thing I forgot to mention: I thought there was a consensus to add a WITH() or OPTIONS() clause to pass options to the checker function: http://archives.postgresql.org/message-id/12568.1322669...@sss.pgh.pa.us I think this should be there so that the API does not have to be changed in the future. there is just one question - how propagate options to check functions I am thinking about third parameter - probably text array Either that, or couldn't you pass an option List as data type internal? I don't know what is most natural or convenient. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
Hi, I've revived the old moribund / bitrotten OpenOffice driver for PostgreSQL (originally at http://dba.openoffice.org/drivers/postgresql/index.html). As you may already know, OpenOffice was forked into LibreOffice. Barring any unforeseen disaster, LibreOffice 3.5 (to be released in early February 2011) will bundle that driver. As part of this integration, I have a few questions / remarks. LibreOffice can be configured at build-time to use Mozilla LDAP or OpenLDAP. We patched postgresql's configure.in to build libpq with Mozilla LDAP when requested to do so. I wanted to check that this is not by any chance known to produce a more-or-less subtly broken libpq. I'd also be happy to extract from our patch the parts that are relevant for integration in postgresql proper, and massage them into the right modularity / form. Are you interested? -- Lionel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] LibreOffice driver 2: MIT Kerberos vs Microsoft Kerberos
(See part 1 for general introduction) LibreOffice currently - when it rebuilds libpq, such as for our official MS Windows binaries - links libpq against only the MS Windows built-in SSPI stuff, which if I understand well is an embrace and extend Kerberos 5 implementation. I wanted to understand the limitations (if any) of building libpq on Windows only with the MS Windows-builtin Kerberos and not additionally with MIT Kerberos. The gsslib parameter in the connection string won't work, but will that keep users from authenticating to some Kerberos domains, and/or are there other (interoperability?) issues that make it strongly desirable to link libpq with *both* SSPI *and* MIT krb5 (and its gssapi_krb5 library)? -- Lionel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch to allow users to kill their own queries
The submission from Edward Muller I'm replying to is quite similar to
what the other raging discussion here decided was the right level to
target. There was one last year from Josh Kupershmidt with similar
goals: http://archives.postgresql.org/pgsql-admin/2010-02/msg00052.php
A good place to start is the concise summary of the new specification
goal that Tom made in the other thread:
If allowing same-user cancels is enough to solve 95% or 99% of the
real-world use cases, let's just do that.
Same-user cancels, but not termination. Only this, and nothing more.
Relative to that goal, Ed's patch was too permissive for termination,
and since he's new to this code it didn't check all the error conditions
possible here. Josh's patch had many of the right error checks, but it
was more code than I liked for his slightly different permissions
change. And its attempts to be helpful leaked role information. (That
may have been just debugging debris left for review purposes) I mashed
the best bits of both together, tried to simplify the result, then
commented heavily upon the race conditions and design decisions the code
reflects. Far as I can tell the patch is feature complete, including
documentation.
Appropriate credits here would go Josh Kupershmidt, Edward Muller, and
then myself; everyone did an equally useful chunk of this in that
order. It's all packaged up for useful gitsumption at
https://github.com/greg2ndQuadrant/postgres/tree/cancel_backend too. I
attached it to the next CommitFest:
https://commitfest.postgresql.org/action/patch_view?id=722 but would
enjoy seeing a stake finally put through its evil heart before then, as
I don't think there's much left to do now.
To demo I start with a limited user and a crazy, must be stopped backend:
$ createuser test
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
$ psql -U test
test= select pg_sleep(100);
Begin another session, find and try to terminate; get rejected with a
hint, then follow it to cancel:
test= select procpid,usename,current_query from pg_stat_activity;
-[ RECORD 1 ]-+
procpid | 28154
usename | test
current_query | select pg_sleep(100);
test= select pg_terminate_backend(28154);
ERROR: must be superuser to terminate other server processes
HINT: you can use pg_cancel_backend() on your own processes
test= select pg_cancel_backend(28154);
-[ RECORD 1 ]-+--
pg_cancel_backend | t
And then this is shown on the first one:
test= select pg_sleep(100);
ERROR: canceling statement due to user request
Victory over the evil sleeping backend is complete, without a superuser
in sight.
There's one obvious and questionable design decision I made to
highlight. Right now the only consumers of pg_signal_backend are the
cancel and terminate calls. What I did was make pg_signal_backend more
permissive, adding the idea that role equivalence = allowed, and
therefore granting that to anything else that might call it. And then I
put a stricter check on termination. This results in a redundant check
of superuser on the termination check, and the potential for mis-using
pg_signal_backend. I documented all that and liked the result; it feels
better to me to have pg_signal_backend provide an API that is more
flexible here. Pushback to structure this differently is certainly
possible though, and I'm happy to iterate the patch to address that. It
might drift back toward something closer to Josh's original design.
--
Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e7f7fe0..f145c3f 100644
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
*** SELECT set_config('log_statement_stats',
*** 14244,14251
para
The functions shown in xref
linkend=functions-admin-signal-table send control signals to
! other server processes. Use of these functions is restricted
! to superusers.
/para
table id=functions-admin-signal-table
--- 14244,14251
para
The functions shown in xref
linkend=functions-admin-signal-table send control signals to
! other server processes. Use of these functions is usually restricted
! to superusers, with noted exceptions.
/para
table id=functions-admin-signal-table
*** SELECT set_config('log_statement_stats',
*** 14262,14268
literalfunctionpg_cancel_backend(parameterpid/parameter typeint/)/function/literal
/entry
entrytypeboolean/type/entry
!entryCancel a backend's current query/entry
/row
row
entry
--- 14262,14271
Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server
On 13.12.2011 11:57, Albe Laurenz wrote: Tom Lane wrote: Shigeru Hanadashigeru.han...@gmail.com writes: (2011/12/12 22:59), Robert Haas wrote: ... I feel like we might need a system here that allows for more explicit user control about what to push down vs. not, rather than assuming we'll be able to figure it out behind the scenes. Agreed. How about to add a per-column boolean FDW option, say pushdown, to pgsql_fdw? Users can tell pgsql_fdw that the column can be pushed down safely by setting this option to true. [ itch... ] That doesn't seem like the right level of granularity. ISTM the problem is with whether specific operators have the same meaning at the far end as they do locally. If you try to attach the flag to columns, you have to promise that *every* operator on that column means what it does locally, which is likely to not be the case ever if you look hard enough. Plus, having to set the flag on each individual column of the same datatype seems pretty tedious. I don't have a better idea to offer at the moment though. Trying to attach such a property to operators seems impossibly messy too. If it weren't for the collations issue, I might think that labeling datatypes as being compatible would be a workable approximation. Maybe I'm missing something, but if pushdown worked as follows: - Push down only system functions and operators on system types. - Only push down what is guaranteed to work. then the only things we would miss out on are encoding- or collation-sensitive string operations. Is that loss so big that it warrants a lot of effort? The SQL/MED spec handles this with the concept of routine mappings. There is syntax for defining which remote routines, meaning functions, correspond local functions: CREATE ROUTINE MAPPING routine mapping name FOR specific routine designator SERVER foreign server name [ generic options ] generic options is FDW-specific, I'd imagine the idea is to give the name of the corresponding function in the remote server. It doesn't say anything about collations, but you could have extra options to specify that a function can only be mapped under C collation, or whatever. It seems tedious to specify that per-server, though, so we'll probably still want to have some smarts in the pgsql_fdw to handle the built-in functions and types that we know to be safe. I've been talking about functions here, not operators, on the assumption that we can look up the function underlying the operator and make the decisions based on that. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] LibreOffice driver 3: pg_config and linking statically to libpq
(See part 1 for general introduction) LibreOffice currently - when it rebuilds libpq, such as for most of our official binaries - links statically against libpq. I noticed that pg_config does not give the information of what additional libraries to include in the link to resolve all symbols that libpq needs: * On the one hand, it gives too much since LIBS is filtered to only a subset in src/interface/libpq/Makefile. * On the other hand, it does not give enough, since it does not give the value of LDAP_LIBS_FE anywhere, nor say if it is necessary to add PTHREAD_LIBS. This is not an immediate problem for LibreOffice: I export the value of SHLIB_EXPORTS from src/interface/libpq/Makefile as a Makefile snippet that gets imported in our build system or (on Microsoft Windows) we just proceeded by trial and error until the link succeeds. However, I suggest it would be cleaner to give that kind of information in pg_config, so that one can basically do something like: $LINK_COMMAND -lpq $(pg_config --libpq-dep-libs) and have it work automatically. You could also provide a pq.pc file for pkgconfig, which would give nice nearly-automatic integration for projects using e.g. autoconf and friends. -- Lionel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server
(2011/12/13 20:04), Heikki Linnakangas wrote: The SQL/MED spec handles this with the concept of routine mappings. There is syntax for defining which remote routines, meaning functions, correspond local functions: CREATE ROUTINE MAPPING routine mapping name FOR specific routine designator SERVER foreign server name [ generic options ] generic options is FDW-specific, I'd imagine the idea is to give the name of the corresponding function in the remote server. It doesn't say anything about collations, but you could have extra options to specify that a function can only be mapped under C collation, or whatever. I considered ROUTINE MAPPING for other RDBMS before, and thought that having order of parameter in generic options would be necessary. It's also useful for pgsql_fdw to support pushing down user-defined functions. Maybe built-in format() function suits for this purpose? It seems tedious to specify that per-server, though, so we'll probably still want to have some smarts in the pgsql_fdw to handle the built-in functions and types that we know to be safe. One possible idea is having default mapping with serverid = InvalidOid, and override them with entries which has valid server oid. Such default mappings can be loaded during CREATE EXTENSION. I've been talking about functions here, not operators, on the assumption that we can look up the function underlying the operator and make the decisions based on that. It's interesting viewpoint to think operator notation is syntax sugar of function notation, e.g. A = B - int4eq(A, B). Routine mappings seem to work for operators too. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server
(2011/12/13 18:57), Albe Laurenz wrote: Maybe I'm missing something, but if pushdown worked as follows: - Push down only system functions and operators on system types. - Only push down what is guaranteed to work. Oh, I didn't care whether system data types. Indeed user defined types would not be safe to push down. then the only things we would miss out on are encoding- or collation-sensitive string operations. Is that loss so big that it warrants a lot of effort? It depends on the definition of collation-sensitive. If we define it as all operations which might handle any collation-sensitive element, all functions/operators which take any of character data types (text, varchar, bpchar, sql_identifier, etc.) are unable to be pushed down. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
Compare: int PQntuples(const PGresult *res) Reasonable: doesn't modify 'res'. With: char *PQcmdStatus(PGresult *res); char *PQcmdTuples(PGresult *res); Unreasonable: a. What, these two can modify 'res' I pass in?.. b. Oh, yes, because they return 'char *' pointing to 'res-cmdStatus+n', so, a libpq user may write: char *s = PQcmdStatus(res); *s = 'x'; and have 'res' modified. (Would be the user's fault, of course.) The non-const-ness of 'PGresult *' for these two functions seems to stand out among the functions covered in the 30.3.2. Retrieving Query Result Information manual section and inhibits writing the strict client code. I would suggest to change the signatures by applying this trivial patch (and changing the documentation): == diff orig/postgresql-9.1.1/src/interfaces/libpq/libpq-fe.h ./postgresql-9.1.1/src/interfaces/libpq/libpq-fe.h 450c450 extern char *PQcmdStatus(PGresult *res); --- extern const char *PQcmdStatus(const PGresult *res); 453c453 extern char *PQcmdTuples(PGresult *res); --- extern const char *PQcmdTuples(const PGresult *res); == diff orig/postgresql-9.1.1/src/interfaces/libpq/fe-exec.c ./postgresql-9.1.1/src/interfaces/libpq/fe-exec.c 2665,2666c2665,2666 char * PQcmdStatus(PGresult *res) --- const char * PQcmdStatus(const PGresult *res) 2736,2737c2736,2737 char * PQcmdTuples(PGresult *res) --- const char * PQcmdTuples(const PGresult *res) 2739,2740c2739 char *p, *c; --- const char *p, *c; (The above was obtained in 9.1.1; the subsequent build with GCC 4.1.2 succeeds without warnings.) If the above change causes a warning in a client code, so much the better: the client code is doing something unreasonable like the *s assignment in my example above. -- Alex -- alex-goncha...@comcast.net -- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] review: CHECK FUNCTION statement
2011/12/13 Albe Laurenz laurenz.a...@wien.gv.at: Pavel Stehule wrote: One thing I forgot to mention: I thought there was a consensus to add a WITH() or OPTIONS() clause to pass options to the checker function: http://archives.postgresql.org/message-id/12568.1322669...@sss.pgh.pa.us I think this should be there so that the API does not have to be changed in the future. there is just one question - how propagate options to check functions I am thinking about third parameter - probably text array Either that, or couldn't you pass an option List as data type internal? this is question - internal is most simply solution, but then we cannot to call check function directly Regards Pavel I don't know what is most natural or convenient. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On 12/12/2011 11:32 AM, Robert Haas wrote: I haven't yet thought about your specific proposal here in enough to have a fully-formed opinion, but I am a little nervous that this may turn out to be one of those cases where the obvious API ends up working less well than might have been supposed. There's another cautionary tale from the sepgsql history worth mentioning here, which surely I don't have to remind you about. Making the goal for a first shippable subset include proof you can solve the hardest problems in that area can lead to a long road without committing anything. With sepgsql, that was focusing on the worst of the ALTER TABLE issues. As Dimitri was pointing out, the name change to Command Triggers includes a sort of admission that DDL Triggers are too hard to solve in all cases yet. We shouldn't be as afraid to introduce APIs that are aimed at developers who currently have none. Yes, there's a risk that will end with ...and this one has to be broken in the next release because of this case we didn't see. We can't be so afraid of that we don't do anything, especially when the users who would be impacted by that theoretical case are currently suffering from an even worse problem than that. To provide the big picture infrastructure tools that people are desperate for now, PostgreSQL needs to get a lot more agile when it comes to revving hooks whose main consumers are not regular application programs. They're the administrators of the system instead. I know what I was just rallying against is not what you were arguing for, you just triggered a stored rant of mine. [Bad trigger joke goes here] Regardless, thoughts on where the holes are here are appreciated. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On 12/13/2011 03:15 AM, Joey Adams wrote: We should probably have a wiki page titled JSON datatype status to help break the cycle we're in I was about to point you to http://wiki.postgresql.org/wiki/JSON_API_Brainstorm , only to realize you created that thing in the first place. There's http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010 too. I don't think it's completely stuck in a cycle yet--every pass around seems to accumulate some better informed ideas than the last still. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Tue, Dec 13, 2011 at 12:25 AM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-12-12 at 16:51 -0800, Peter van Hardenberg wrote: Because we haven't heard from him in a while we've been using PL/V8 to validate a JSON datatype simulated by a DOMAIN with a simple acceptance function. (See below.) This is not ideally performant but thanks to V8's JIT the JSON parser is actually reasonably good. I think releasing something simple and non-performant with reasonable semantics would be the best next step. If it were up to me, I'd probably even try to just land PL/V8 as PL/JavaScript for 9.2 if the crash bugs and deal breakers can be sifted out. You don't need a new PL to do that. The existing PLs can also parse JSON. So that's not nearly enough of a reason to consider adding this new PL. Just because all our languages are Turing-complete doesn't mean they are all equally well-suited to every task. Of course, that doesn't mean we'd add a whole new language just to get a JSON parser, but I don't think that's really what Peter was saying. Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
2011/12/13 Robert Haas robertmh...@gmail.com: On Tue, Dec 13, 2011 at 12:25 AM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-12-12 at 16:51 -0800, Peter van Hardenberg wrote: Because we haven't heard from him in a while we've been using PL/V8 to validate a JSON datatype simulated by a DOMAIN with a simple acceptance function. (See below.) This is not ideally performant but thanks to V8's JIT the JSON parser is actually reasonably good. I think releasing something simple and non-performant with reasonable semantics would be the best next step. If it were up to me, I'd probably even try to just land PL/V8 as PL/JavaScript for 9.2 if the crash bugs and deal breakers can be sifted out. You don't need a new PL to do that. The existing PLs can also parse JSON. So that's not nearly enough of a reason to consider adding this new PL. Just because all our languages are Turing-complete doesn't mean they are all equally well-suited to every task. Of course, that doesn't mean we'd add a whole new language just to get a JSON parser, but I don't think that's really what Peter was saying. Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. it is true - but there is a few questions * will be JSON supported from SQL? * what Javascript engine will be supported? * will be integrated JSON supported from PLPerl? I like to see Javacript's in pg, but I don't like Javascript just for JSON. JSON should be independent on javascript. Regards Pavel -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On 12/13/2011 08:44 AM, Robert Haas wrote: Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. Quite. I hate Javascript with a passion, wish it would just go away and stop meddling with my life. And even with that context, I think in-core PL/V8 would be a huge advocacy win. PostgreSQL has this great developer-oriented PL interface, it just doesn't work out of the box with any of the pop languages right now. Personal story on this. When my book came out, I was trying to take the #1 spot on Packt's bestseller list, even if it was just for a day. Never made it higher than #2. The #1 spot the whole time was jQuery 1.4 Reference Guide, discussing the most popular JavaScript library out there. And you know what? Over a year later, it's *still there*. At no point did it over drop out of that top spot. The number of people who would consider server-side programming suddenly feasible if PL/V8 were easy to do is orders of magnitude larger than the current PostgreSQL community. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 2: MIT Kerberos vs Microsoft Kerberos
* Lionel Elie Mamane (lio...@mamane.lu) wrote: The gsslib parameter in the connection string won't work, but will that keep users from authenticating to some Kerberos domains, and/or are there other (interoperability?) issues that make it strongly desirable to link libpq with *both* SSPI *and* MIT krb5 (and its gssapi_krb5 library)? The MIT KRB5 library on Windows is more-or-less defunct now, as I understand it. pgAdmin3 hasn't been linking against it due to unfixed security bugs (that don't seem likely to ever be fixed) and because it's horribly painful to maintain. The gist of the limitation is this- if you need to support decent encryption in a cross-realm environment on Windows XP-age systems, you need MIT KRB5. If you're on Windows 7 or something else recent, the built-in Windows stuff w/ AES works fine. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] JSON for PG 9.2
On Tue, Dec 13, 2011 at 8:11 AM, Greg Smith g...@2ndquadrant.com wrote: On 12/13/2011 08:44 AM, Robert Haas wrote: Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. Quite. I hate Javascript with a passion, wish it would just go away and stop meddling with my life. And even with that context, I think in-core PL/V8 would be a huge advocacy win. PostgreSQL has this great developer-oriented PL interface, it just doesn't work out of the box with any of the pop languages right now. Personal story on this. When my book came out, I was trying to take the #1 spot on Packt's bestseller list, even if it was just for a day. Never made it higher than #2. The #1 spot the whole time was jQuery 1.4 Reference Guide, discussing the most popular JavaScript library out there. And you know what? Over a year later, it's *still there*. At no point did it over drop out of that top spot. The number of people who would consider server-side programming suddenly feasible if PL/V8 were easy to do is orders of magnitude larger than the current PostgreSQL community. Yeah -- javascript is making strides server-side with technologies like node.js. Like you I have really mixed feelings about javascript -- there's a lot of nastiness but the asynchronous style of coding javascript developers tend to like is a great fit for postgres both inside the backend and in database clients. This is on top of the already nifty type system synergy I mentioned upthread. Postgres would in fact make a wonderful 'nosql' backend with some fancy json support -- document style transmission to/from the backend without sacrificing relational integrity in storage. Properly done this would be a fabulous public relations coup (PostgreSQL = better nosql). merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On 12/13/2011 09:11 AM, Greg Smith wrote: On 12/13/2011 08:44 AM, Robert Haas wrote: Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. Quite. I hate Javascript with a passion, wish it would just go away and stop meddling with my life. And even with that context, I think in-core PL/V8 would be a huge advocacy win. PostgreSQL has this great developer-oriented PL interface, it just doesn't work out of the box with any of the pop languages right now. Personal story on this. When my book came out, I was trying to take the #1 spot on Packt's bestseller list, even if it was just for a day. Never made it higher than #2. The #1 spot the whole time was jQuery 1.4 Reference Guide, discussing the most popular JavaScript library out there. And you know what? Over a year later, it's *still there*. At no point did it over drop out of that top spot. The number of people who would consider server-side programming suddenly feasible if PL/V8 were easy to do is orders of magnitude larger than the current PostgreSQL community. I think your passion is probably somewhat misdirected. I've long thought JS would be a good fit for Postgres. It's naturally sandboxed and its type system fits ours quite well. And, as you say, it's massively popular and getting a major second wind thanks to things like JQuery, Ext-JS and node.js. This last one has certainly convinced lots of people that JS is not just for browsers any more. Having said that, don't underestimate the complexity of trying to build in PLV8. In its current incarnation the interface is written in C++ (and of course so is the underlying V8 engine). I have been doing some development work for it, even though my C++ is rather rusty (that's an understatement, folks), which is why I haven't got a lot more done - I'm just having to go slowly and with reference books by my side. So either we'd need to rewrite the glue code entirely in C (and it's littered with C++isms) and handle the difficulties of embedding a C++ library, or we'd have a major new build infrastructure dependency which could well give us a major case of developmental indigestion. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign key locks, 2nd attempt
On Mon, Dec 12, 2011 at 05:20:39PM -0300, Alvaro Herrera wrote:
Excerpts from Noah Misch's message of dom dic 04 09:20:27 -0300 2011:
Second, I tried a SELECT FOR SHARE on a table of 1M tuples; this might incur
some cost due to the now-guaranteed use of pg_multixact for FOR SHARE. See
attached fklock-test-forshare.sql. The median run slowed by 7% under the
patch, albeit with a rather brief benchmark run. Both master and patched
PostgreSQL seemed to exhibit a statement-scope memory leak in this test
case:
to lock 1M rows, backend-private memory grew by about 500M. When trying 10M
rows, I cancelled the query after 1.2 GiB of consumption. This limited the
duration of a convenient test run.
I found that this is caused by mxid_to_string being leaked all over the
place :-( I fixed it by making the returned string be a static that's
malloced and then freed on the next call. There's still virtsize growth
(not sure it's a legitimate leak) with that, but it's much smaller.
Great. I'll retry that benchmark with the next patch version. I no longer
see a leak on master, so I probably messed up that part of the test somehow.
By the way, do you have a rapid procedure for finding the call site behind a
leak like this?
This being a debugging aid, I don't think there's any need to backpatch
this.
Agreed.
diff --git a/src/backend/access/transam/multixact.c
b/src/backend/access/transam/multixact.c
index ddf76b3..c45bd36 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -1305,9 +1305,14 @@ mxstatus_to_string(MultiXactStatus status)
static char *
mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members)
{
- char *str = palloc(15 * (nmembers + 1) + 4);
+ static char*str = NULL;
int i;
+ if (str != NULL)
+ free(str);
+
+ str = malloc(15 * (nmembers + 1) + 4);
Need a check for NULL return.
+
snprintf(str, 47, %u %d[%u (%s), multi, nmembers, members[0].xid,
mxstatus_to_string(members[0].status));
Thanks,
nm
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On Tue, Dec 13, 2011 at 8:25 AM, Greg Smith g...@2ndquadrant.com wrote: There's another cautionary tale from the sepgsql history worth mentioning here, which surely I don't have to remind you about. Making the goal for a first shippable subset include proof you can solve the hardest problems in that area can lead to a long road without committing anything. With sepgsql, that was focusing on the worst of the ALTER TABLE issues. As Dimitri was pointing out, the name change to Command Triggers includes a sort of admission that DDL Triggers are too hard to solve in all cases yet. We shouldn't be as afraid to introduce APIs that are aimed at developers who currently have none. Yes, there's a risk that will end with ...and this one has to be broken in the next release because of this case we didn't see. Well, the problem is that just because something better comes along doesn't mean we'll actually deprecate and remove the old functionality. We still have contrib/xml2, even though the docs say we're planning to remove it in 8.4. Tom even rewrote the memory handling, because it was easier to rewrite a module he probably doesn't intrinsically care much about than to convince people we should remove something that was both planned for deprecation anyway and a huge security risk because it crashed if you looked at it sideways. And we still have rules, so people read the documentation and say to themselves hmm, should i use triggers or rules for this project?. And elsewhere we're discussing whether and under what conditions it would be suitable to get rid of recovery.conf, which almost everyone seems to agree is a poor design, largely AFAICT because third-party tools find recovery.conf a convenient way to circumvent the need to rewrite postgresql.conf, which is a pain in the neck because of our insistence that it has to contain arbitrary user comments. In other words, more often than not, we are extremely reluctant to remove or modify features of even marginal utility because there will certainly be somebody, somewhere who is relying on the old behavior. Of course, it does go the other way sometimes: we removed old-style VACUUM FULL (which was useful if you were short of disk space and long on time), flat-file authentication (which was used by third party tools), and made removing OIDs require a table rewrite. Still, I think it's entirely appropriate to be cautious about adding new features that might not turn out to be the design we really want to have. Odds are good that we will end up supporting them for backward compatibility reasons for many, many years. Now, all that having been said, I also agree that the perfect can be the enemy of the good, and we go there frequently. The question I'm asking is not whether the feature is perfect, but whether it's adequate for even the most basic things people might want to do with it. Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. In particular, you won't have the index or constraint names, and you might not have the schema or tablespace information either. But maybe we ought to put the question to the intended audience for the feature - is there a Slony developer in the house? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
Merlin Moncure wrote: Postgres would in fact make a wonderful 'nosql' backend with some fancy json support -- document style transmission to/from the backend without sacrificing relational integrity in storage. Properly done this would be a fabulous public relations coup (PostgreSQL = better nosql). PostSQL? ;-) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Tue, Dec 13, 2011 at 8:44 AM, Robert Haas robertmh...@gmail.com wrote: Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. +1 to that. I'm not a huge fan of JS; wish that one of the Scheme variations had made it instead. But it's clear that a LOT of fairly successful work has gone into making JS implementations performant, and it's clearly heavily used. JS+hstore would probably draw in a bunch of users, and tempt them to the SQL dark side :-). Wanting a JSON processor isn't quite a good enough reason to add C++ support in order to draw in a JS interpreter. But I don't imagine things are restricted to just 1 JS implementation, and JSON isn't the only reason to do so. -- When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_cancel_backend by non-superuser
Hi Greg 2011/12/13 Greg Smith g...@2ndquadrant.com: On 12/11/2011 05:29 PM, Torello Querci wrote: I will try to adjust the patch and submit for the next Commit Fest if this is ok for you. I don't think we'll need this, it will take a bit to explain why though. First, thanks for returning this topic to discussion and keeping up with all the controversy around it. You said back in February this was your first post here, and I doubt you expected that 10 months later this would still be active and argued over. The fact that you're still here and everyone knows your name now is itself an accomplishment, many people just give up on their submission ideas under far less negative feedback. Why. I need one feature, can spend some time to try to get it and I try. This is only way to lean that I know. I just took a long look at all three of the submissions in this area we've gotten. The central idea that made yours different was making the database owner the person allowed to cancel things. That hadn't been suggested as a cancellation requisite before that I know of, and this code may wander in that direction one day. It's just a bit too much to accept right now. You seem to need that specific feature for your environment. If that's the case, you might want to develop something that works that way, but handles the concerns raised here. The fact that it's not acceptable for a database owner to cancel a superuser query is the biggest objection, there were some others too. Ultimately it may take a reworking of database permissions to really make this acceptable, which is a larger job than I think you were trying to get involved with. Probably you have right :( Unfortunately, when I look at the new spec we have now, I don't see anything from what you did that we can re-use. It's too specific to the owner-oriented idea. The two other patches that have been submitted both are closer to what we've decided we want now. What I'm going to do here is mark your submission returned with feedback. Again no problem. The only thing that I need (not only me obviusly) is give the permission to one or more users to kill session and query owned by other users. Have a kind of ACL where is specify who can kill and which is the right way. My problem is related to production environment where an application server access the database server and I am the database owner, not the DBA. So I need to kill the application server sessions (again I not the root of application server so I not able to stop and restart it). I hope to explain my scenario if not before. Rather than wait for something new from you, I'm going to review and rework the other two submissions. That I can start on right now. It's taken so long to reach this point that I don't want to wait much longer for another submission here, certainly not until over a month from now when the next CF starts. We need to get the arguments around a new version started earlier than that. Thanks for offering to work on this more, and I hope there's been something about this long wandering discussion that's been helpful to you. As I said, you did at least make a good first impression, and that is worth something when it comes to this group. Thanks Greg. I hope to meet you at Fosdem if you wil go there. Best Regards, Torello -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] review: CHECK FUNCTION statement
Pavel Stehule pavel.steh...@gmail.com writes: 2011/12/13 Albe Laurenz laurenz.a...@wien.gv.at: Either that, or couldn't you pass an option List as data type internal? this is question - internal is most simply solution, but then we cannot to call check function directly Yeah, one of the proposals for allowing people to specify complicated conditions about what to check was to tell them to do select checker(oid) from pg_proc where any-random-condition; If the checker isn't user-callable then we lose that escape hatch, and the only selection conditions that will ever be possible are the ones we take the trouble to shoehorn into the CHECK FUNCTION statement. Doesn't seem like a good thing to me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Dec 13, 2011, at 9:15 AM, Kevin Grittner wrote: Merlin Moncure wrote: Postgres would in fact make a wonderful 'nosql' backend with some fancy json support -- document style transmission to/from the backend without sacrificing relational integrity in storage. Properly done this would be a fabulous public relations coup (PostgreSQL = better nosql). PostSQL? ;-) I think you meant to say Postgre... ;P -- Jim C. Nasby, Database Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
Lionel Elie Mamane lio...@mamane.lu writes: I've revived the old moribund / bitrotten OpenOffice driver for PostgreSQL (originally at http://dba.openoffice.org/drivers/postgresql/index.html). As you may already know, OpenOffice was forked into LibreOffice. Barring any unforeseen disaster, LibreOffice 3.5 (to be released in early February 2011) will bundle that driver. Cool ... As part of this integration, I have a few questions / remarks. LibreOffice can be configured at build-time to use Mozilla LDAP or OpenLDAP. We patched postgresql's configure.in to build libpq with Mozilla LDAP when requested to do so. Um, if I interpret this correctly, you're intending to make OpenOffice include a bundled copy of libpq.so. With my other hat on (the red fedora) that sounds like a truly awful idea. Distros do not like packages that include bundled copies of other packages' libraries, because they're a nightmare for updates (cf recent discussions about static libraries, which are the same thing). I strongly suggest you find a way to not need to do this. I wanted to check that this is not by any chance known to produce a more-or-less subtly broken libpq. No, we are not going to make any promises about that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
On Tue, Dec 13, 2011 at 7:55 AM, Alex Goncharov alex-goncha...@comcast.net wrote: If the above change causes a warning in a client code, so much the better: the client code is doing something unreasonable like the *s assignment in my example above. Or they just haven't bothered to decorate their entire code-base with const declarations. I suppose it's probably worth doing this, but I reserve the right to be unexcited about it. At a minimum, I dispute the application of the term painless to any change involving const. If you want this patch to be considered for application, you should post an updated patch which includes the necessary doc changes and add a link to it here: https://commitfest.postgresql.org/action/commitfest_view/open -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On Tue, Dec 13, 2011 at 10:53 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Dec 13, 2011 at 8:59 AM, Robert Haas robertmh...@gmail.com wrote: Now, all that having been said, I also agree that the perfect can be the enemy of the good, and we go there frequently. The question I'm asking is not whether the feature is perfect, but whether it's adequate for even the most basic things people might want to do with it. Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. In particular, you won't have the index or constraint names, and you might not have the schema or tablespace information either. But, you could query all that out from the system catalogs right? You could probably get a lot of it that way, although first you'll have to figure out which schema to look up the name in. It seems likely that everyone who uses the trigger will need to write that code, though, and they'll all have different implementations with different bugs, because many of them probably really want the facility that you write in your next sentence: Maybe a better facility should exist to convert a table name to a create table statement than hand rolling it or invoking pg_dump, but that's a separate issue. This feature fills an important niche given that you can't hook RI triggers to system catalogs...it comes up (in short, +1). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On Tue, Dec 13, 2011 at 8:59 AM, Robert Haas robertmh...@gmail.com wrote: Now, all that having been said, I also agree that the perfect can be the enemy of the good, and we go there frequently. The question I'm asking is not whether the feature is perfect, but whether it's adequate for even the most basic things people might want to do with it. Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. In particular, you won't have the index or constraint names, and you might not have the schema or tablespace information either. But, you could query all that out from the system catalogs right? Maybe a better facility should exist to convert a table name to a create table statement than hand rolling it or invoking pg_dump, but that's a separate issue. This feature fills an important niche given that you can't hook RI triggers to system catalogs...it comes up (in short, +1). merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
On Tue, Dec 13, 2011 at 10:43:26AM -0500, Tom Lane wrote:
Lionel Elie Mamane lio...@mamane.lu writes:
LibreOffice can be configured at build-time to use Mozilla LDAP or
OpenLDAP. We patched postgresql's configure.in to build libpq with
Mozilla LDAP when requested to do so.
Um, if I interpret this correctly, you're intending to make
OpenOffice include a bundled copy of libpq.so. With my other hat on
(the red fedora) that sounds like a truly awful idea.
In the traditional Unix world (GNU/Linux, Free/Net/Open/MirBSD,
...), libpq.so.5 is in the dynamic linker's default search path, so we
can conceivably link against it and tell ours users install package
libpq5 / postgresql-client-lib / ... if you want PostgreSQL support.
On MacOS X and Microsoft Windows, the world is far more messy. There
are several ways to install libpq (one-click installer, fink,
MacPorts, ...), and each of these ways allows the user to select an
install location (or a prefix thereof) freely. How we are supposed to
instruct the dynamic linker to find libpq on the user's system, you
tell me, I have no better idea than bundling it (or linking
statically); running
find / -name libpq.5.dylib
(or libpq.dll on Windows) and then dynamically running otool (the
MacOS X equivalent of chrpath/patchelf/...) on ourselves is *not*
practical, to say the least...
Distros do not like packages that include bundled copies of other
packages' libraries,
Traditional Unix distros are free to (and I expect typically will)
configure LibreOffice with --with-system-libs or
--with-system-postgresql (which will use pg_config in the PATH) or
--with-libpq-path=/foo/bar, all of which will just link against libpq
and not bundle it, and expect to find it at runtime at the same
location than at compile-time or in the dynamic linker's default
search path. Their RPM/DEB/... will declare a dependency on the
RPM/DEB/... that ships an ABI-compatible libpq.so.5, so it will be
there and all will be well.
With my Debian GNU/{Linux,kFreeBSD,Hurd,...} hat on, obviously I would
not have it any other way. With my LibreOffice hat on, I have to take
care of other platforms' needs, too.
because they're a nightmare for updates (cf recent discussions about
static libraries, which are the same thing). I strongly suggest you
find a way to not need to do this.
Personally, I don't have a beginning of a clue of a way to not need
to do this for MacOS X and Microsoft Windows. The MacOS X Windows
guys within LibreOffice tell me bundling the lib (or linking
statically) is the least insane thing to do on these
platforms. shrug
--
Lionel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign key locks, 2nd attempt
Excerpts from Noah Misch's message of mar dic 13 11:44:49 -0300 2011: On Mon, Dec 12, 2011 at 05:20:39PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of dom dic 04 09:20:27 -0300 2011: Second, I tried a SELECT FOR SHARE on a table of 1M tuples; this might incur some cost due to the now-guaranteed use of pg_multixact for FOR SHARE. See attached fklock-test-forshare.sql. The median run slowed by 7% under the patch, albeit with a rather brief benchmark run. Both master and patched PostgreSQL seemed to exhibit a statement-scope memory leak in this test case: to lock 1M rows, backend-private memory grew by about 500M. When trying 10M rows, I cancelled the query after 1.2 GiB of consumption. This limited the duration of a convenient test run. I found that this is caused by mxid_to_string being leaked all over the place :-( I fixed it by making the returned string be a static that's malloced and then freed on the next call. There's still virtsize growth (not sure it's a legitimate leak) with that, but it's much smaller. Great. I'll retry that benchmark with the next patch version. I no longer see a leak on master, so I probably messed up that part of the test somehow. Maybe you recompiled without the MULTIXACT_DEBUG symbol defined? By the way, do you have a rapid procedure for finding the call site behind a leak like this? Not really ... I tried some games with GDB (which yielded the first report: I did some call MemoryContextStats(TopMemoryContext) to see where the bloat was, and then stepped with breaks on MemoryContextAlloc, also with a watch on CurrentMemoryContext and noting when it was pointing to the bloated context. But since I'm a rookie with GDB I didn't find a way to only break when MemoryContextAlloc was pointing at that context. I know there must be a way.) and then went to do some code inspection instead. I gather some people use valgrind successfully. +if (str != NULL) +free(str); + +str = malloc(15 * (nmembers + 1) + 4); Need a check for NULL return. Yeah, thanks ... I changed it to MemoryContextAlloc(TopMemoryContext), because I'm not sure that a combination of malloc plus palloc would end up in extra memory fragmentation. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign key locks, 2nd attempt
On Tue, Dec 13, 2011 at 01:09:46PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of mar dic 13 11:44:49 -0300 2011: On Mon, Dec 12, 2011 at 05:20:39PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of dom dic 04 09:20:27 -0300 2011: Second, I tried a SELECT FOR SHARE on a table of 1M tuples; this might incur some cost due to the now-guaranteed use of pg_multixact for FOR SHARE. See attached fklock-test-forshare.sql. The median run slowed by 7% under the patch, albeit with a rather brief benchmark run. Both master and patched PostgreSQL seemed to exhibit a statement-scope memory leak in this test case: to lock 1M rows, backend-private memory grew by about 500M. When trying 10M rows, I cancelled the query after 1.2 GiB of consumption. This limited the duration of a convenient test run. I found that this is caused by mxid_to_string being leaked all over the place :-( I fixed it by making the returned string be a static that's malloced and then freed on the next call. There's still virtsize growth (not sure it's a legitimate leak) with that, but it's much smaller. Great. I'll retry that benchmark with the next patch version. I no longer see a leak on master, so I probably messed up that part of the test somehow. Maybe you recompiled without the MULTIXACT_DEBUG symbol defined? Neither my brain nor my shell history recall that, but it remains possible. By the way, do you have a rapid procedure for finding the call site behind a leak like this? Not really ... I tried some games with GDB (which yielded the first report: I did some call MemoryContextStats(TopMemoryContext) to see where the bloat was, and then stepped with breaks on MemoryContextAlloc, also with a watch on CurrentMemoryContext and noting when it was pointing to the bloated context. But since I'm a rookie with GDB I didn't find a way to only break when MemoryContextAlloc was pointing at that context. I know there must be a way.) and then went to do some code inspection instead. I gather some people use valgrind successfully. Understood. Incidentally, the GDB command in question is condition. +if (str != NULL) +free(str); + +str = malloc(15 * (nmembers + 1) + 4); Need a check for NULL return. Yeah, thanks ... I changed it to MemoryContextAlloc(TopMemoryContext), because I'm not sure that a combination of malloc plus palloc would end up in extra memory fragmentation. Sounds good. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [REVIEW] Patch for cursor calling with named parameters
On 12.12.2011 21:55, Kevin Grittner wrote: Yeb Havinga wrote: Forgot to copy regression output to expected - attached v7 fixes that. This version addresses all of my concerns. It applies cleanly and compiles without warning against current HEAD and performs as advertised. I'm marking it Ready for Committer. This failed: postgres=# do $$ declare foocur CURSOR (insane /* arg int4) IS SELECT generate_series(1, insane /* arg); begin OPEN foocur(insane /* arg := 10); end; $$; ERROR: unterminated /* comment at or near /* insane /* arg := */ 10; LINE 1: SELECT /* insane /* arg := */ 10; ^ QUERY: SELECT /* insane /* arg := */ 10; CONTEXT: PL/pgSQL function inline_code_block line 5 at OPEN I don't have much sympathy for anyone who uses argument names like that, but it nevertheless ought to not fail. A simple way to fix that is to constuct the query as: value AS argname, instead of /* argname := */ value. Then you can use the existing quote_identifier() function to do the necessary quoting. I replaced the plpgsql_isidentassign() function with a more generic plpgsql_peek2() function, which allows you to peek ahead two tokens in the input stream, without eating them. It's implemented using the pushdown stack like plpgsql_isidentassign() was, but the division of labor between pl_scanner.c and gram.y seems more clear this way. I'm still not 100% happy with it. plpgsql_peek2() behaves differently from plpgsql_yylex(), in that it doesn't perform variable or unreserved keyword lookup. It could do that, but it would be quite pointless since the only current caller doesn't want variable or unreserved keyword lookup, so it would just have to work harder to undo them. Attached is a patch with those changes. I also I removed a few of the syntax error regression tests, that seemed excessive, plus some general naming and comment fiddling. I'll apply this tomorrow, if it still looks good to me after sleeping on it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com *** a/doc/src/sgml/plpgsql.sgml --- b/doc/src/sgml/plpgsql.sgml *** *** 2823,2833 OPEN curs1 FOR EXECUTE 'SELECT * FROM ' || quote_ident(tabname) /para /sect3 ! sect3 titleOpening a Bound Cursor/title synopsis ! OPEN replaceablebound_cursorvar/replaceable optional ( replaceableargument_values/replaceable ) /optional; /synopsis para --- 2823,2833 /para /sect3 ! sect3 id=plpgsql-open-bound-cursor titleOpening a Bound Cursor/title synopsis ! OPEN replaceablebound_cursorvar/replaceable optional ( optional replaceableargname/replaceable := /optional replaceableargument_value/replaceable optional, .../optional ) /optional; /synopsis para *** *** 2847,2856 OPEN replaceablebound_cursorvar/replaceable optional ( replaceableargume --- 2847,2867 /para para + Argument values can be passed using either firsttermpositional/firstterm + or firsttermnamed/firstterm notation. In positional + notation, all arguments are specified in order. In named notation, + each argument's name is specified using literal:=/literal to + separate it from the argument expression. Similar to calling + functions, described in xref linkend=sql-syntax-calling-funcs, it + is also allowed to mix positional and named notation. + /para + + para Examples (these use the cursor declaration examples above): programlisting OPEN curs2; OPEN curs3(42); + OPEN curs3(key := 42); /programlisting /para *** *** 3169,3175 COMMIT; synopsis optional lt;lt;replaceablelabel/replaceablegt;gt; /optional ! FOR replaceablerecordvar/replaceable IN replaceablebound_cursorvar/replaceable optional ( replaceableargument_values/replaceable ) /optional LOOP replaceablestatements/replaceable END LOOP optional replaceablelabel/replaceable /optional; /synopsis --- 3180,3186 synopsis optional lt;lt;replaceablelabel/replaceablegt;gt; /optional ! FOR replaceablerecordvar/replaceable IN replaceablebound_cursorvar/replaceable optional ( optional replaceableargname/replaceable := /optional replaceableargument_value/replaceable optional, .../optional ) /optional LOOP replaceablestatements/replaceable END LOOP optional replaceablelabel/replaceable /optional; /synopsis *** *** 3179,3186 END LOOP optional replaceablelabel/replaceable /optional; commandFOR/ statement automatically opens the cursor, and it closes the cursor again when the loop exits. A list of actual argument value expressions must appear if and only if the cursor was declared to take ! arguments. These values will be substituted in the query, in just ! the same way as during an commandOPEN/. The variable
Re: [HACKERS] WIP: SP-GiST, Space-Partitioned GiST
I wrote: ... the leaf tuple datatype is hard-wired to be After another day's worth of hacking, I now understand the reason for the above: when an index is less than a page and an incoming value would still fit on the root page, the incoming value is simply dumped into a leaf tuple without ever calling any opclass-specific function at all. Exactly. To allow the leaf datatype to be different from the indexed column, we'd need at least one more opclass support function, and it's not clear that the potential gain is worth any extra complexity. Agree, all opclasses which I could imagine for sp-gist use the same type. Without clear example I don't like an idea to add one more support function and it could be easily added later as an optional support function as it's already done for distance function for GiST However, I now have another question: what is the point of the allTheSame mechanism? It seems to add quite a great deal of complexity, I thought about two options: separate code path in core to support a-lot-of-the-same-values with minimal support in support functions and move all logic about this case to support functions. Second option is demonstrated in k-d-tree implementation, where split axis is contained by each half-plane. May be it is a simpler solution although it moves responsibility to opclass developers. one thing, it's giving me fits while attempting to fix the limitation on storing long indexed values. There's no reason why a suffix tree representation shouldn't work for long strings, but you have to be willing to cap the length of any given inner tuple's prefix to something I don't see clear interface for now: let we have an empty index and we need to insert a long string (more than even several page). So, it's needed to have support function to split input value to several ones. I supposed that sp-gist is already complex enough for first step to add support for this non very useful case. Of course, for future we have a plans to add support of long values, NULLs/IS NULL, knn-search at least. I'm also still wondering what your thoughts are on storing null values versus full-index-scan capability. I'm leaning towards getting rid of the dead code, but if you have an idea how to remove the limitation, maybe we should do that instead. I didn't have a plan to support NULLs in first stage, because it's not clear for me how and where to store them. It seems to me that it should be fully separated from normal path, like a linked list of pages with only ItemPointer data (similar to leaf data pages in GIN) I missed that planner will not create qual-free scan, because I thought it's still possible with NOT NULL columns. If not, this code could be removed/commented/ifdefed. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
,--- I/Alex (Tue, 13 Dec 2011 07:55:45 -0500) * | If the above change causes a warning in a client code, so much the | better: the client code is doing something unreasonable like the *s | assignment in my example above. ,--- Robert Haas (Tue, 13 Dec 2011 10:51:54 -0500) * | Or they just haven't bothered to decorate their entire code-base with | const declarations. They don't have to, for the conceptually correct code. I.e. one can write (with the old and new code): /* no: const */ PGresult *res; const char *readout; readout = PQxxx(res,...); /* no: *readout = 'x'; */ all right and have no compilation warnings. But one can also (reasonably) const-qualify the 'res' above (const-correct and const-consistent code is a good thing.) | If you want this patch to be considered for application, you should | post an updated patch which includes the necessary doc changes and add | a link to it here: | | https://commitfest.postgresql.org/action/commitfest_view/open OK, I could do it... ,--- Alvaro Herrera (Tue, 13 Dec 2011 13:01:13 -0300) * | Do we really need a 100% complete patch just to discuss whether we're | open to doing it? IMHO it makes sense to see a WIP patch and then | accept or reject based on that; if we accept the general idea, then a | complete patch would presumably be submitted. `-* ... but I like this more. I.e., can one tell me to bother or not with the complete patch, based on the general idea, which wouldn't change for the complete patch? -- Alex -- alex-goncha...@comcast.net -- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
On Tue, Dec 13, 2011 at 11:01 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mar dic 13 12:51:54 -0300 2011: If you want this patch to be considered for application, you should post an updated patch which includes the necessary doc changes and add a link to it here: https://commitfest.postgresql.org/action/commitfest_view/open Do we really need a 100% complete patch just to discuss whether we're open to doing it? Of course not. You may notice that I also offered an opinion on the substance of the patch. In the course of doing that, I don't see why I shouldn't point out that it's the patch author's responsibility to provide docs. YMMV, of course. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] review: CHECK FUNCTION statement
2011/12/13 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2011/12/13 Albe Laurenz laurenz.a...@wien.gv.at: Either that, or couldn't you pass an option List as data type internal? this is question - internal is most simply solution, but then we cannot to call check function directly Yeah, one of the proposals for allowing people to specify complicated conditions about what to check was to tell them to do select checker(oid) from pg_proc where any-random-condition; If the checker isn't user-callable then we lose that escape hatch, and the only selection conditions that will ever be possible are the ones we take the trouble to shoehorn into the CHECK FUNCTION statement. Doesn't seem like a good thing to me. yes, it is reason why I thinking just about string array. I have not idea about other PL, but options for plpgsql can be one word and checker function can simply parse two or more words options. Now I would to implement flags quite - ignore NOTIFY messages and fatal_errors to stop on first error. Regards Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace
2011/12/13 Jaime Casanova ja...@2ndquadrant.com: On Mon, Dec 12, 2011 at 10:54 AM, Julien Tachoires jul...@gmail.com wrote: 2011/12/10 Jaime Casanova ja...@2ndquadrant.com: On Mon, Nov 28, 2011 at 1:32 PM, Julien Tachoires jul...@gmail.com wrote: 2) after CLUSTER the index of the toast table gets moved to the same tablespace as the main table there is still a variant of this one, i created 3 tablespaces (datos_tblspc): create table t1 ( i serial primary key, t text ) tablespace datos_tblspc; ALTER TABLE t1 SET TOAST TABLESPACE pg_default; CLUSTER t1 USING t1_pkey; I am not able to reproduce this case, could you show me exactly how to reproduce it ? just as that... - create a table in a certain tablespace (diferent from pg_default), the toast table will be in the same tablespace, - then change the tablespace to pg_default and - then cluster the table... the toast table will be again in the same tablespace as the main table Right, it seems to happen when the destination tablespace is the same as the database's tbs, because, in this case, relation's tbs is set to InvalidOid : src/backend/commands/tablecmds.c line 8342 + rd_rel-reltablespace = (newTableSpace == MyDatabaseTableSpace) ? InvalidOid : newTableSpace; Why don't just asign newTableSpace value here ? Thanks, -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace
On Tue, Dec 13, 2011 at 12:02 PM, Julien Tachoires jul...@gmail.com wrote: Right, it seems to happen when the destination tablespace is the same as the database's tbs, because, in this case, relation's tbs is set to InvalidOid : src/backend/commands/tablecmds.c line 8342 + rd_rel-reltablespace = (newTableSpace == MyDatabaseTableSpace) ? InvalidOid : newTableSpace; Why don't just asign newTableSpace value here ? When a relation is stored in the default tablespace, we always record that in the system catalogs as InvalidOid. Otherwise, if the database's default tablespace were changed, things would break. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: SP-GiST, Space-Partitioned GiST
Teodor Sigaev teo...@sigaev.ru writes: However, I now have another question: what is the point of the allTheSame mechanism? It seems to add quite a great deal of complexity, I thought about two options: separate code path in core to support a-lot-of-the-same-values with minimal support in support functions and move all logic about this case to support functions. Second option is demonstrated in k-d-tree implementation, where split axis is contained by each half-plane. May be it is a simpler solution although it moves responsibility to opclass developers. I eventually realized that you have to have it to ensure that you can split a leaf-tuple list across pages even when the opclass picksplit function thinks the values are all the same. What made no sense to me was (a) having the property forcibly inherit to child inner tuples, and (b) suppressing node labels in allTheSame tuples. That could make it impossible for the opclass to reconstruct values. In my local copy I've changed this behavior a bit and improved the documentation about what opclasses have to do with the flag. one thing, it's giving me fits while attempting to fix the limitation on storing long indexed values. There's no reason why a suffix tree representation shouldn't work for long strings, but you have to be willing to cap the length of any given inner tuple's prefix to something I don't see clear interface for now: let we have an empty index and we need to insert a long string (more than even several page). So, it's needed to have support function to split input value to several ones. I supposed that sp-gist is already complex enough for first step to add support for this non very useful case. Well, I have it working well enough to satisfy me locally. The picksplit function can handle the prefixing perfectly well, as long as it's not surprised by getting called on a single oversized leaf tuple. (I changed the format of leaf tuples to let the size field be 30 bits, so we can have an oversized leaf tuple in memory even if we can't store it. This doesn't cost anything space-wise because of alignment rules.) Of course, for future we have a plans to add support of long values, NULLs/IS NULL, knn-search at least. I think if we're going to do nulls we can't wait; that will necessarily change the on-disk representation, which is going to be a hard sell once 9.2 is out the door. Neither leaf nor inner tuples have any good way to deal with nulls at present. Maybe if you invent a completely separate representation for nulls it could be added on after the fact, but that seems like an ugly answer. I missed that planner will not create qual-free scan, because I thought it's still possible with NOT NULL columns. If not, this code could be removed/commented/ifdefed. Well, it won't do so because pg_am.amoptionalkey is not set. But we can't set that if we don't store NULLs. Right at the moment, my local copy has completely broken handling of WAL, because I've been focusing on the opclass interface and didn't worry about WAL while I was whacking the picksplit code around. I'll try to clean that up today and then post a new version of the patch. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
On 12/13/2011 11:07 AM, Lionel Elie Mamane wrote: On MacOS X and Microsoft Windows, the world is far more messy. There are several ways to install libpq (one-click installer, fink, MacPorts, ...), and each of these ways allows the user to select an install location (or a prefix thereof) freely. This is less controversial. If you yell at Tom about something messy that must be done only to support Windows and Mac OS X, he does something completely different with his RedHat Fedora. It's kind of like http://www.youtube.com/watch?v=kLbOMb7F40k , only more red. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 2: MIT Kerberos vs Microsoft Kerberos
On 12/13/2011 09:18 AM, Stephen Frost wrote: The gist of the limitation is this- if you need to support decent encryption in a cross-realm environment on Windows XP-age systems, you need MIT KRB5. If you're on Windows 7 or something else recent, the built-in Windows stuff w/ AES works fine. This answers Lionel's question, but I'm curious for a more user impact opinion from you. Given that pgAdmin III has given up on MIT KRB5, would you feel doing the same is appropriate for LibreOffice too? It sounds like they really shouldn't take on either the build cruft or the potential security issues of pulling that in at this point. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On Tue, Dec 13, 2011 at 9:59 AM, Robert Haas robertmh...@gmail.com wrote: Now, all that having been said, I also agree that the perfect can be the enemy of the good, and we go there frequently. The question I'm asking is not whether the feature is perfect, but whether it's adequate for even the most basic things people might want to do with it. Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. In particular, you won't have the index or constraint names, and you might not have the schema or tablespace information either. But maybe we ought to put the question to the intended audience for the feature - is there a Slony developer in the house? Yeah, I'm not certain yet what is being provided, and the correspondence with what would be needed. - It's probably not sufficient to capture the raw statement, as that gets invoked within a context replete with GUC values, and you may need to duplicate that context/environment on a replica. Mind you, a command trigger is doubtless capable of querying GUCs to duplicate relevant portions of the environment. - What I'd much rather have is a form of the query that is replete with Full Qualification, so that create table foo (id serial primary key, data text not null unique default 'better replace this', dns_data dnsrr not null); may be transformed into a safer form like: create table public.foo (id serial primary key, data text not null unique default 'better replace this'::text, dns_data dns_rr.dnsrr not null); What's not clear, yet, is which transformations are troublesome. For instance, if there's already a sequence called foo_id_seq, then the sequence defined for that table winds up being foo_id_seq1, and it's not quite guaranteed that *that* would be identical across databases. But perhaps it's sufficient to implement what, of COMMAND TRIGGERS, can be done, and we'll see, as we proceed, whether or not it's enough. It's conceivable that a first implementation won't be enough to implement DDL triggers for Slony, and that we'd need to ask for additional functionality that doesn't make it in until 9.3. That seems better, to me, than putting it on the shelf, and having functionality in neither 9.2 nor 9.3... -- When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace
2011/12/13 Robert Haas robertmh...@gmail.com: On Tue, Dec 13, 2011 at 12:02 PM, Julien Tachoires jul...@gmail.com wrote: Right, it seems to happen when the destination tablespace is the same as the database's tbs, because, in this case, relation's tbs is set to InvalidOid : src/backend/commands/tablecmds.c line 8342 + rd_rel-reltablespace = (newTableSpace == MyDatabaseTableSpace) ? InvalidOid : newTableSpace; Why don't just asign newTableSpace value here ? When a relation is stored in the default tablespace, we always record that in the system catalogs as InvalidOid. Otherwise, if the database's default tablespace were changed, things would break. OK, considering that, I don't see any way to handle the case raised by Jaime :( -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
Excerpts from Alex Goncharov's message of mar dic 13 13:43:35 -0300 2011: I.e., can one tell me to bother or not with the complete patch, based on the general idea, which wouldn't change for the complete patch? So let's see the patch. In context diff format please, and also include in-tree changes to the callers of those functions, if any are necessary. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [REVIEW] Patch for cursor calling with named parameters
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Attached is a patch with those changes. I also I removed a few of the syntax error regression tests, that seemed excessive, plus some general naming and comment fiddling. I'll apply this tomorrow, if it still looks good to me after sleeping on it. The code looks reasonably clean now, although I noted one comment thinko: + * bool:trim trailing whitespace ITYM + * trim:trim trailing whitespace However, I'm still concerned about whether this approach gives reasonable error messages in cases where the error would be detected during parse analysis of the rearranged statement. The regression test examples don't cover such cases, and I'm too busy right now to apply the patch and check for myself. What happens for example if a named parameter's value contains a misspelled variable reference, or a type conflict? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On Tue, Dec 13, 2011 at 12:29 PM, Christopher Browne cbbro...@gmail.com wrote: But perhaps it's sufficient to implement what, of COMMAND TRIGGERS, can be done, and we'll see, as we proceed, whether or not it's enough. It's conceivable that a first implementation won't be enough to implement DDL triggers for Slony, and that we'd need to ask for additional functionality that doesn't make it in until 9.3. That seems better, to me, than putting it on the shelf, and having functionality in neither 9.2 nor 9.3... The thing is, I don't really see the approach Dimitri is taking as being something that we can extend to meet the requirement you just laid out. So it's not like, OK, let's do this, and we'll improve it later. It's, let's do this, and then later do something completely different, and that other thing will be the one that really solves the problem. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
Greg Smith g...@2ndquadrant.com writes: On 12/13/2011 11:07 AM, Lionel Elie Mamane wrote: On MacOS X and Microsoft Windows, the world is far more messy. There are several ways to install libpq (one-click installer, fink, MacPorts, ...), and each of these ways allows the user to select an install location (or a prefix thereof) freely. This is less controversial. Yeah, I'm aware that sane handling of library dependencies is practically impossible under Windows, but I didn't see how that would result in wanting to change the configure script. I wasn't thinking about OSX though. (You're aware that Apple ships a perfectly fine libpq.so in Lion, no?) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xlog location arithmetic
On Dec 6, 2011, at 12:06 PM, Robert Haas wrote: On Tue, Dec 6, 2011 at 1:00 PM, Euler Taveira de Oliveira eu...@timbira.com wrote: On 06-12-2011 13:11, Robert Haas wrote: On Tue, Dec 6, 2011 at 5:14 AM, Magnus Hagander mag...@hagander.net wrote: I've been considering similar things, as you can find in the archives, but what I was thinking of was converting the number to just a plain bigint, then letting the user apply whatever arithmetic wanted at the SQL level. I never got around to acutally coding it, though. It could easily be extracted from your patch of course - and I think that's a more flexible approach. Is there some advantage to your method that I'm missing? I went so far as to put together an lsn data type. I didn't actually get all that far with it, which is why I haven't posted it sooner, but here's what I came up with. It's missing indexing support and stuff, but that could be added if people like the approach. It solves this problem by implementing -(lsn,lsn) = numeric (not int8, that can overflow since it is not unsigned), which allows an lsn = numeric conversion by just subtracting '0/0'::lsn. Interesting approach. I don't want to go that far. If so, you want to change all of those functions that deal with LSNs and add some implicit conversion between text and lsn data types (for backward compatibility). As of int8, I'm not aware of any modern plataform that int8 is not 64 bits. I'm not against numeric use; I'm just saying that int8 is sufficient. The point isn't that int8 might not be 64 bits - of course it has to be 64 bits; that's why it's called int8 i.e. 8 bytes. The point is that a large enough LSN, represented as an int8, will come out as a negative values. int8 can only represent 2^63 *non-negative* values, because one bit is reserved for sign. I've often wondered about adding uint2/4/8... I suspect it's actually pretty uncommon for people to put negative numbers into int fields, since one of their biggest uses seems to be surrogate keys. I realize that this opens a can of worms with casting, but perhaps that can be kept under control by not doing any implicit casting between int and uint... that just means that we'd have to be smart about casting from unknown, but hopefully that's doable since we already have a similar concern with casting unknown to int2/4/8 vs numeric? -- Jim C. Nasby, Database Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Avoiding repeated snapshot computation
On Sat, Nov 26, 2011 at 5:49 PM, Andres Freund and...@anarazel.de wrote: On Saturday, November 26, 2011 11:39:23 PM Robert Haas wrote: On Sat, Nov 26, 2011 at 5:28 PM, Andres Freund and...@anarazel.de wrote: On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: I'd just as soon keep the fields in a logical order. Btw, I don't think the new order is necessarily worse than the old one. You forget to attach the benchmark results. My impression is that cache lines on modern hardware are 64 or 128 *bytes*, in which case this wouldn't matter a bit. All current x86 cpus use 64bytes. The 2nd 128bit reference was a typo. Sorry for that. And why is 72=56 *bytes* (I even got that one right) not relevant for 64byte cachelines? OK, so I got around to looking at this again today; sorry for the delay. I agree that 72 - 56 bytes is very relevant for 64-byte cache lines. I had not realized the structure was as big as that. And yea. I didn't add benchmark results. I don't think I *have* to do that when making suggestions to somebody trying to improve something specific. I also currently don't have hardware where I can sensibly run at a high enough concurrency to see that GetSnapshotData takes ~40% of runtime. Additional cacheline references around synchronized access can hurt to my knowledge... I tested this on Nate Boley's 32-core box today with the 32 clients doing a select-only pgbench test. Results of individual 5 minute runs: results.master.32:tps = 171701.947919 (including connections establishing) results.master.32:tps = 22.430112 (including connections establishing) results.master.32:tps = 218257.478461 (including connections establishing) results.master.32:tps = 226425.964855 (including connections establishing) results.master.32:tps = 218687.662373 (including connections establishing) results.master.32:tps = 219819.451605 (including connections establishing) results.master.32:tps = 216800.131634 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 181997.531357 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 171505.145440 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 226970.348605 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 169725.115763 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 219548.690522 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 175440.705722 (including connections establishing) results.snapshotdata-one-cacheline.32:tps = 217154.173823 (including connections establishing) There's no statistically significant difference between those two data sets; if anything, the results with the patch look like they might be worse, although I believe that's an artifact - some runs seem to mysteriously come out in the 170-180 rangeinstead of the 215-225 range, with nothing in between. But even if you only average the good runs out of each set there's no material difference. Having said that, I am coming around to the view that we should apply this patch anyway, just because it reduces memory consumption. Since the size change crosses a power-of-two boundary, I believe it will actually cut down the size of a palloc'd chunk for a SnapshotData object from 128 bytes to 64 bytes. I doubt we can measure the benefit of that even on a microbenchmark unless someone has a better idea for making PostgreSQL take ridiculous numbers of snapshots than I do. It still seems like a good idea, though: a penny saved is a penny earned. With response to Tom's objections downthread, I don't think that the new ordering is significantly more confusing than the old one. xcnt/subxcnt/xip/subxip doesn't seem measurably less clear than xcnt/xip/subxcnt/subxip, and we're not normally in the habit of letting such concerns get in the way of squeezing out alignment padding. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On 12/13/2011 09:59 AM, Robert Haas wrote: Well, the problem is that just because something better comes along doesn't mean we'll actually deprecate and remove the old functionality. The examples you gave fall into three groups, and I think it's useful to demarcate how they're different. Please pardon me going wildly off topic before returning back. If you drop xml2 or rules, people lose something. It's primarily the PostgreSQL developers who gain something. You can make a case that people who won't get sucked into doing something wrong with rules one day will gain something, but those people are a future speculation; they're not here asking to be saved for a problem they don't know will happen yet. This sort of deprecation battle is nearly impossible to win. One of the reasons I placed a small bet helping sponsor PGXN is that I hope it allows some of this should be deprecated stuff to move there usefully. Let the people who use it maintain it moving forward if they feel it's important. The recovery.conf change and other attempts to reorganize the postgresql.conf are in a second category. These break scripts, without providing an immediate and obvious gain to everyone. Some say it's better, some say it's worse, from the list traffic it seems like a zero-sum game. The burden is on the person advocating the change to justify it if there's not a clear win for everyone. You might note that my latest attitude toward this area is to provide the mechanism I want as a new option, and not even try to argue about removing the old thing anymore. This lets implementation ideas battle it out in the wild. Let's say a year from now everyone who hasn't switched to using a conf.d dirctory approach looks like an old stick in the mud. Then maybe I pull the sheet I have an enormous bikeshed hidden behind, waiting for just that day.[1] When VACUUM FULL was rewritten, it took a recurring large problem that has impacted a lot of people, and replaced with a much better thing for most cases. A significantly smaller number of people lost something that was slightly useful. There weren't as many complaints because the thing that was lost was replaced with something better by most metrics. Different, but better. This third category of changes are much easier to sell. We have another such discontinuity coming with pg_stat_activity. The changes Scott Mead's patch kicked off make it different and better. Anyone who has a tool using the old thing can look at the new design and say well, that makes the whole 'what state is the connection in' logic I used to worry about go away; that's progress even if it breaks my old scripts. People need to get something that offsets the breakage to keep griping down. Anyone who argues against those sort of changes has a challenging battle on their hands. If there is a Command Trigger implementation that Slony etc. use, and we discover a limitation that requires an API break, that's OK so long as it's expected that will fall into the last category. Breakage to add support for something new should be a feature clearly gained, something lost, and with a net benefit to most consumers of that feature. People accept it or block obvious forward progress. We don't want to get too confused between what makes for good progress on that sort of thing with the hard deprecation problems. (Not that I'm saying you are here, just pointing out it happens) Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. These are all good things to look into, please keep those test set ideas coming and hopefully we'll get some more input on this. But let's say this rolls out seeming good enough, and later someone discovers some weird index thing that isn't supported. If that's followed by here's a new API; it breaks your old code, but it allows supporting that index you couldn't deal with before, that's unlikely to get shot down by that API's consumers. What you wouldn't be able to do is say this new API doesn't work right, let's just yank it out. Your concerns about making sure at least the fundamentals hold here are on point though. [1] Look at that, I can now say that 100% of the programs I compose e-mail with now have bikeshed added to their dictionary. I don't bother with this often, but there's entries for PostgreSQL and committer there too.[2] [2] Would you believe a Google search for committer shows the PostgreSQL wiki page as its second hit? That's only behind the Wikipedia link, and ahead of the FreeBSD, Chromium, Apache, and Mozilla pages on that topic. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training,
Re: [HACKERS] JSON for PG 9.2
On Dec 12, 2011, at 7:42 PM, Alvaro Herrera wrote: I remember there was the idea of doing something like this for regexes -- have a specialized data type that saves the trouble of parsing it. I imagine this is pretty much the same. Nobody got around to doing anything about it though. (regex data type)++ David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch for type privileges
On 2011-12-12 20:53, Peter Eisentraut wrote: On sön, 2011-12-11 at 21:21 +0200, Peter Eisentraut wrote: * Cannot restrict access to array types. After revoking usage from the element type, the error is perhaps a bit misleading. (smallint[] vs smallint) postgres= create table a (a int2[]); ERROR: permission denied for type smallint[] OK, that error message should be improved. Fixing this is easy, but I'd like to look into refactoring this a bit. Let's ignore that for now; it's easy to do later. My experience with ignoring things for now is not positive. * The information schema view 'attributes' has this additional condition: AND (pg_has_role(t.typowner, 'USAGE') OR has_type_privilege(t.oid, 'USAGE')); What happens is that attributes in a composite type are shown, or not, if the current user has USAGE rights. The strange thing here, is that the attribute in the type being show or not, doesn't match being able to use it (in the creation of e.g. a table). Yeah, that's a bug. That should be something like AND (pg_has_role(c.relowner, 'USAGE') OR has_type_privilege(c.reltype, 'USAGE')); And fix for that included. Confirmed that this now works as expected. I have no remarks on the other parts of the patch code. After puzzling a bit more with the udt and usage privileges views, it is clear that they should complement each other. That might be reflected by adding to the 'usage_privileges' section a link back to the 'udt_privileges' section. I have no further comments on this patch. regards, Yeb Havinga -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
On Tue, Dec 13, 2011 at 12:48:01PM -0500, Tom Lane wrote: Greg Smith g...@2ndquadrant.com writes: On 12/13/2011 11:07 AM, Lionel Elie Mamane wrote: On MacOS X and Microsoft Windows, the world is far more messy. There are several ways to install libpq (one-click installer, fink, MacPorts, ...), and each of these ways allows the user to select an install location (or a prefix thereof) freely. This is less controversial. Yeah, I'm aware that sane handling of library dependencies is practically impossible under Windows, (...). I wasn't thinking about OSX though. (You're aware that Apple ships a perfectly fine libpq.so in Lion, no?) No, I was not aware of that. Thank you for that information. Is it installed by default? Anyway, LibreOffice wishes to be compatible back to MacOS X 10.4 (probably to have some sort of MacOSX/PowerPC support), so that's not yet useful to us, but it is good news! As I use PQconnectdbParams, I also need version 9.0 or later (but I'd be willing to work around that if it were useful). -- Lionel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuration include directory
On Mon, Dec 12, 2011 at 01:34:24PM -0500, Greg Smith wrote: [various things I agree with] -Don't bother trying to free individual bits of memory now that it's all in the same context. Saves some lines of code, and I do not miss the asserts I am no longer triggering. In the postmaster, this context is the never-reset PostmasterContext. Thus, these yield permanent leaks. The rest of the ProcessConfigFile() code makes an effort to free everything it allocates, so let's do the same here. (I'd favor, as an independent effort, replacing some of the explicit pfree() activity in guc-file.l with a temporary memory context create/delete.) The only review suggestion I failed to incorporate was this one from Noah: + if (strcmp(de-d_name + strlen(de-d_name) - 5, .conf) != 0) + continue; That may as well be memcmp(). While true, his idea bothers both my abstraction and unexpected bug concern sides for reasons I can't really justify. I seem to have received too many past beatings toward using the string variants of all these functions whenever operating on things that are clearly strings. I'll punt this one toward whoever looks at this next to decide, both strcmp and strncmp are used in this section now. Okay. diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index d1e628f..5df214e 100644 *** a/doc/src/sgml/config.sgml --- b/doc/src/sgml/config.sgml *** SET ENABLE_SEQSCAN TO OFF; *** 178,184 any desired selection condition. It also contains more information about what values are allowed for the parameters. /para ! /sect1 sect1 id=runtime-config-file-locations titleFile Locations/title --- 161,273 any desired selection condition. It also contains more information about what values are allowed for the parameters. /para ! ! sect2 id=config-includes ! titleConfiguration file includes/title Our section names use title case. !para !indexterm ! primaryliteralinclude//primary ! secondaryin configuration file/secondary !/indexterm !In addition to parameter settings, the filenamepostgresql.conf/ !file can contain firstterminclude directives/, which specify !another file to read and process as if it were inserted into the !configuration file at this point. Include directives simply look like: ! programlisting ! include 'filename' ! /programlisting !If the file name is not an absolute path, it is taken as relative to !the directory containing the referencing configuration file. !All types of inclusion directives can be nested. ! /para ! ! para !indexterm ! primaryliteralinclude_dir//primary ! secondaryin configuration file/secondary !/indexterm !The filenamepostgresql.conf/ file can also contain !firstterminclude_dir directives/, which specify an entire directory !of configuration files to include. It is used similarly: ! programlisting ! include_dir 'directory' ! /programlisting !Non-absolute directory names follow the same rules as single file include !directives: they are relative to the directory containing the referencing !configuration file. Within that directory, only files whose names end Consider the wording Within that directory, only non-directory files whose names ... to communicate that we ignore all subdirectories. !with the suffix literal.conf/literal will be included. File names !that start with the literal./literal character are also excluded, !to prevent mistakes as they are hidden on some platforms. Multiple files !within an include directory are ordered by an alphanumeric sorting, so !that ones beginning with numbers are considered before those starting !with letters. ! /para ! ! para ! Include files or directories can be used to logically separate portions ! of the database configuration, rather than having a single large ! filenamepostgresql.conf/ file. Consider a company that has two ! database servers, each with a different amount of memory. There are likely ! elements of the configuration both will share, for things such as logging. ! But memory-related parameters on the server will vary between the two. And ! there might be server specific customizations too. One way to manage this I suggest the punctuation server-specific customizations, too. ! situation is to break the custom configuration changes for your site into ! three files. You could add this to the end of your !filenamepostgresql.conf/ file to include them: ! programlisting ! include 'shared.conf' ! include 'memory.conf' ! include 'server.conf' !
Re: [HACKERS] logging in high performance systems.
On 11/24/2011 11:33 AM, Theo Schlossnagle wrote: I see the next steps being: 1) agreeing that a problem exists (I know one does, but I suppose consensus is req'd) 2) agreeing that hooks are the right approach, if not propose a different approach. (fwiw, it's incredible common) 3) reworking the implementation to fit in the project; I assume the implementation I proposed will, at best, vaguely resemble anything that gets integrated. It was just a PoC. With this idea still being pretty new, and several of the people popping out opinions in this thread being local--Theo, Stephen, myself--we've decided to make our local Baltimore/Washington PUG meeting this month be an excuse to hash some of this early stuff out a bit more in person, try to speed things up . See http://www.meetup.com/Baltimore-Washington-PostgreSQL-Users-Group/events/44335672/ if any other locals would like to attend, it's a week from today. (Note that the NYC PUG is also having its meeting at the same time this month) -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved
Excerpts from Robert Haas's message of mar dic 13 12:51:54 -0300 2011: If you want this patch to be considered for application, you should post an updated patch which includes the necessary doc changes and add a link to it here: https://commitfest.postgresql.org/action/commitfest_view/open Do we really need a 100% complete patch just to discuss whether we're open to doing it? IMHO it makes sense to see a WIP patch and then accept or reject based on that; if we accept the general idea, then a complete patch would presumably be submitted. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuration include directory
On tis, 2011-11-15 at 23:53 -0500, Greg Smith wrote: -Called by specifying includedir directory. No changes to the shipped postgresql.conf yet. -Takes an input directory name -If it's not an absolute path, considers that relative to the -D option (if specified) or PGDATA, the same logic used to locate the postgresql.conf (unless a full path to it is used) -Considers all names in that directory that end with *.conf [Discussion concluded more flexibility here would be of limited value relative to how it complicates the implementation] -Loops over the files found in sorted order by name I can see some potential confusion here in one case. Let's say someone specifies a full path to their postgresql.conf file. They might assume that the includedir was relative to the directory that file is in. Let's say configfile is /etc/sysconfig/pgsql/postgresql.conf ; a user might think that includedir conf.d from there would reference /etc/sysconfig/pgsql/conf.d instead of the $PGDATA/conf.d you actually get. Wavering on how to handle that is one reason I didn't try documenting this yet, the decision I made here may not actually be the right one. Well, the existing include directive works relative to the directory the including file is in. If includedir works differently from that, that would be highly confusing. I would actually just extend include to accept wildcards instead of inventing a slightly new and slightly different mechanism. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On tis, 2011-12-13 at 00:06 -0800, Peter van Hardenberg wrote: On Mon, Dec 12, 2011 at 9:25 PM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-12-12 at 16:51 -0800, Peter van Hardenberg wrote: You don't need a new PL to do that. The existing PLs can also parse JSON. So that's not nearly enough of a reason to consider adding this new PL. PL/V8 is interesting because it is very fast, sandboxed, and well embedded with little overhead. My experience with PL/Python and PL/Perl has not been thus, and although they are handy if you want to break out and run system work, they're not the kind of thing I'd consider for defining performant operators with. Some performance numbers comparing a valid_json() functions implemented in different ways would clarify this. I wouldn't be surprised if PL/V8 won, but we need to work with some facts. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why do we need two snapshots per query?
On Sat, Nov 26, 2011 at 2:50 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: + /* Done with the snapshot used for parameter I/O and parsing/planning */ + if (snapshot_set) + PopActiveSnapshot(); This comment needs adjusting. I thought about adjusting it, but I didn't see what it made sense to adjust it to. It still is the parameter used for parameter I/O and parsing/planning, so the existing text isn't wrong. It will possibly also get reused for execution, but the previous statement has a lengthy comment on that, so it didn't seem worth recapitulating here. You need to be editing the comments for this function. To be specific you didn't update this text: * The caller can optionally pass a snapshot to be used; pass InvalidSnapshot * for the normal behavior of setting a new snapshot. This parameter is * presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended * to be used for cursors). Actually, I did, but the change was in the second patch file attached to the same email, which maybe you missed? Combined patch attached. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company fewer-snapshots.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On tis, 2011-12-13 at 09:11 -0500, Greg Smith wrote: Personal story on this. When my book came out, I was trying to take the #1 spot on Packt's bestseller list, even if it was just for a day. Never made it higher than #2. The #1 spot the whole time was jQuery 1.4 Reference Guide, discussing the most popular JavaScript library out there. And you know what? Over a year later, it's *still there*. I would guess that that's largely because there are a lot more people developing web sites than people tuning databases, and also because the on-board documentation of javascript and jquery is poor, at least for their audience. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On Tue, Dec 13, 2011 at 2:33 PM, Peter Eisentraut pete...@gmx.net wrote: On tis, 2011-12-13 at 09:11 -0500, Greg Smith wrote: Personal story on this. When my book came out, I was trying to take the #1 spot on Packt's bestseller list, even if it was just for a day. Never made it higher than #2. The #1 spot the whole time was jQuery 1.4 Reference Guide, discussing the most popular JavaScript library out there. And you know what? Over a year later, it's *still there*. I would guess that that's largely because there are a lot more people developing web sites than people tuning databases, and also because the on-board documentation of javascript and jquery is poor, at least for their audience. jquery being used in as much as 40%+ of all websites by some estimates is surely a contributing factor. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xlog location arithmetic
On Tue, Dec 13, 2011 at 12:48 PM, Jim Nasby j...@nasby.net wrote: I've often wondered about adding uint2/4/8... I suspect it's actually pretty uncommon for people to put negative numbers into int fields, since one of their biggest uses seems to be surrogate keys. I realize that this opens a can of worms with casting, but perhaps that can be kept under control by not doing any implicit casting between int and uint... that just means that we'd have to be smart about casting from unknown, but hopefully that's doable since we already have a similar concern with casting unknown to int2/4/8 vs numeric? I've wondered about it too, but it seems like too large a can of worms to open just to address this case. Returning the value as numeric is hardly a disaster; the user can always downcast to int8 if they really want, and as long as it's 2^63 (which in practice it virtually always will be) it will work. It's not clear what the point of this is since for typical values numeric is going to take up less storage anyway (e.g. 101 is 7 bytes on disk as a numeric), not to mention that it only requires 4-byte alignment rather than 8-byte alignment, and probably no one does enough arithmetic with LSN values for any speed penalty to matter even slightly, but it should work. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
On tis, 2011-12-13 at 08:44 -0500, Robert Haas wrote: Just because all our languages are Turing-complete doesn't mean they are all equally well-suited to every task. Of course, that doesn't mean we'd add a whole new language just to get a JSON parser, but I don't think that's really what Peter was saying. That was in fact what I was saying. Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. If JavaScript (trademark of Oracle, btw.; be careful about calling anything PL/JavaScript) had a near-canonical implementation with a stable shared library and a C API, then this might be a no-brainer. But instead we have lots of implementations, and the one being favored here is written in C++ and changes the soname every 3 months. I don't think that's the sort of thing we want to carry around. The way forward here is to maintain this as an extension, provide debs and rpms, and show that that is maintainable. I can see numerous advantages in maintaining a PL outside the core; especially if you are still starting up and want to iterate quickly. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] JSON for PG 9.2
Robert Haas wrote:
On Mon, Dec 12, 2011 at 4:08 PM, Simon Riggs si...@2ndquadrant.com wrote:
On Mon, Dec 12, 2011 at 8:54 PM, Robert Haas robertmh...@gmail.com wrote:
There are way too many places that assume that the typmod can
just be discarded.
If true, that probably ought to be documented cos it sounds fairly
important.
Where and when is it true?
I'm not going to go compile an exhaustive list, since that would take
a week and I don't have any particular desire to invest that much time
in it, but just to take a couple of simple examples:
rhaas=# create or replace function wuzzle(numeric(5,2)) returns int as
$$select 1$$ language sql;
CREATE FUNCTION
rhaas=# \df wuzzle
List of functions
Schema | Name | Result data type | Argument data types | Type
++--+-+
public | wuzzle | numeric | | normal
public | wuzzle | integer | numeric | normal
(2 rows)
rhaas=# select pg_typeof(1.23::numeric(5,2));
pg_typeof
---
numeric
(1 row)
There are a very large number of others. Possibly grepping for places
where we do getBaseType() rather than getBaseTypeAndTypmod() would be
a way to find some of them.
I think the most common one I see is with concatentation:
test= select 'abc'::varchar(3) || 'def'::varchar(3);
?column?
--
abcdef
(1 row)
It is not really clear how the typmod length should be passed in this
example, but passing it unchanged seems wrong:
test= select ('abc'::varchar(3) || 'def'::varchar(3))::varchar(3);
varchar
-
abc
(1 row)
--
Bruce Momjian br...@momjian.ushttp://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)
Hi! On Mon, Dec 12, 2011 at 10:41 PM, Jeff Davis pg...@j-davis.com wrote: Thank you. I have attached a patch that's mostly just cleanup to this one. Thanks a lot for cleanup. Path with applied cleanup is attached. Comments: * You use the term ordinal range quite a lot, which I haven't heard before. Is that a mathematical term, or do you mean something more like ordinary? Actually I meant ordinal range to be finite, non-empty and non-contain-empty range. It's not mathematical term. Probably there is some better word for that, but my english is not strong enough :). * There's a lot of code for range_gist_penalty. Rather than having special cases for all combinations of properties in the new an original, is it possible to use something a little simpler? Maybe just start the penalty at zero, and add something for each property of the predicate range that must be changed. The penalties added might vary, e.g., if the original range has an infinite lower bound, changing it to have an infinite upper bound might be a higher penalty. I belive it's possible to make it simplier. I've coded quite intuitively. Probably, we should select some representive datasets in order to determine which logic is reasonable by tests. * It looks like LIMIT_RATIO is not always considered. Should it be? Yes, it's so. In this part I repeat logic of GiST with NULLs. It makes NULLs to be separated from non-NULLs even if it's produce worse ratio. I'm not sure about how it should be. It seems to be tradeoff between having some excess pages and having slightly worse tree. * You defined get/set_range_contain_empty, but didn't use them. I think this was a merge error, but I removed them. So now there are no changes in rangetypes.c. Ok, thanks. -- With best regards, Alexander Korotkov. rangetypegist-0.5.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)
On Sat, Dec 10, 2011 at 6:14 PM, Greg Smith g...@2ndquadrant.com wrote: On 12/02/2011 06:48 AM, Alexander Korotkov wrote: Rebased with head. Could you comment a little more on what changed? There were a couple of areas Tom commented on: -General code fixes Expensibe usage of Max macro is fixed in 0.5 version of patch. -pull out and apply the changes related to the RANGE_CONTAIN_EMPTY flag, and also remove the opclass entry It's already done by Tom. -Subdiff issues The third one sounded hard to deal with, so presumably nothing there. As I wrote before, I believe there is some limitation of current GiST interface. Most likely we're not going to change GiST interface now and have to do will solution of tradeoff. I think good way to do it is to select representive datasets and do some tests which will show which logic is more reasonable. Actually, I need some help with that, because I don't have enough of datasets. -- With best regards, Alexander Korotkov.
Re: [HACKERS] JSON for PG 9.2
On Tue, Dec 13, 2011 at 2:41 PM, Peter Eisentraut pete...@gmx.net wrote: On tis, 2011-12-13 at 08:44 -0500, Robert Haas wrote: Just because all our languages are Turing-complete doesn't mean they are all equally well-suited to every task. Of course, that doesn't mean we'd add a whole new language just to get a JSON parser, but I don't think that's really what Peter was saying. That was in fact what I was saying. Rather, I think the point is that embedded Javascript is *extremely* popular, lots and lots of people are supporting it, and we ought to seriously consider doing the same. It's hard to think of another PL that we could add that would give us anywhere near the bang for the buck that Javascript would. If JavaScript (trademark of Oracle, btw.; be careful about calling anything PL/JavaScript) had a near-canonical implementation with a stable shared library and a C API, then this might be a no-brainer. But instead we have lots of implementations, and the one being favored here is written in C++ and changes the soname every 3 months. I don't think that's the sort of thing we want to carry around. Mozilla SpiderMonkey seems like a good fit: it compiles to a dependency free .so, has excellent platform support, has a stable C API, and while it's C++ internally makes no use of exceptions (in fact, it turns them off in the c++ compiler). ISTM to be a suitable foundation for an external module, 'in core' parser, pl, or anything really. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] logging in high performance systems.
On Dec 13, 2011, at 13:57, Greg Smith wrote: With this idea still being pretty new, and several of the people popping out opinions in this thread being local--Theo, Stephen, myself--we've decided to make our local Baltimore/Washington PUG meeting this month be an excuse to hash some of this early stuff out a bit more in person, try to speed things up . See http://www.meetup.com/Baltimore-Washington-PostgreSQL-Users-Group/events/44335672/ if any other locals would like to attend, it's a week from today. (Note that the NYC PUG is also having its meeting at the same time this month) What time? I'd potentially like to attend. Philadelphia, represent! Michael Glaesemann grzm seespotcode net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] static or dynamic libpgport
On 12/12/2011 02:59 PM, Tom Lane wrote: Peter Eisentrautpete...@gmx.net writes: On lör, 2011-12-10 at 20:26 -0500, Tom Lane wrote: Right now, libpq laboriously rebuilds all the .o files it needs from src/port/ so as to get them with -fpic. It would be nice if we could clean that up while we're doing this. It might be all right to always build the client-side version of libpgport with -fpic, though I'd be sad if that leaked into the server-side build. So would we continue to build the client binaries (psql, pg_dump, etc.) against the static libpgport.a, thus keeping it invisible there, or would we dynamically link them, thus creating a new dependency. I think that if possible we should avoid creating a new dependency for either the client binaries or libpq.so itself; what I suggest above is only a simplification of the build process for libpq. If we create a new dependency we risk packagers breaking things by forgetting to include it. The Fedora/RHEL rule against static libraries is meant to prevent situations where changes in a library would require rebuilding other packages to get the fixes in place. If we had to make a quick security fix in libpq, for example, it would suck if dozens of other packages had to be rebuilt to propagate the change everywhere. However, I don't think that concern applies to programs that are in the same source package as the library --- they'd get rebuilt anyway. So I see nothing wrong with continuing to statically link these .o files into files belonging to the postgresql package. It's just that I can't export them in a .a file for *other* source packages to use. (Whether a security issue in libpgport is really likely to happen is not a question that this policy concerns itself with ...) OK, my possibly naïve approach is shown in the attached. Essentially it builds libpgport-shared.so and then installs it as libpgport.so. That ensures that the library is not used in building any postgres binaries or libraries. Places such as libpq that formerly symlinked and recompiled the sources in a way that is suitable for a shared library now just link in the already built object files. Is there a better way to do this? cheers andrew *** src/port/Makefile 2011-12-03 17:21:59.944509111 -0500 --- src/port/GNUmakefile 2011-12-12 22:32:50.875312294 -0500 *** *** 37,47 # foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND OBJS_SRV = $(OBJS:%.o=%_srv.o) ! all: libpgport.a libpgport_srv.a ! # libpgport is needed by some contrib install: all installdirs ! $(INSTALL_STLIB) libpgport.a '$(DESTDIR)$(libdir)/libpgport.a' installdirs: $(MKDIR_P) '$(DESTDIR)$(libdir)' --- 37,55 # foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND OBJS_SRV = $(OBJS:%.o=%_srv.o) ! NAME = pgport-shared ! SO_MAJOR_VERSION= 1 ! SO_MINOR_VERSION= 1 ! include $(top_srcdir)/src/Makefile.shlib ! ! all: all-lib libpgport_srv.a ! rm -f libpgport.a $(LN_S) libpgport-shared.a libpgport.a ! ! # libpgport is needed by any exe built with pgxs install: all installdirs ! rm -f libpgport.so* $(LN_S) libpgport-shared.so.$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION) libpgport.so.$(SO_MAJOR_VERSION).$(SO_MINOR_VERSION) ! $(MAKE) -f GNUmakefile.installshared install installdirs: $(MKDIR_P) '$(DESTDIR)$(libdir)' *** *** 49,57 uninstall: rm -f '$(DESTDIR)$(libdir)/libpgport.a' - libpgport.a: $(OBJS) - $(AR) $(AROPT) $@ $^ - # thread.o needs PTHREAD_CFLAGS (but thread_srv.o does not) thread.o: thread.c $(CC) $(CFLAGS) $(CPPFLAGS) $(PTHREAD_CFLAGS) -c $ --- 57,62 *** *** 64,70 $(AR) $(AROPT) $@ $^ %_srv.o: %.c ! $(CC) $(CFLAGS) $(subst -DFRONTEND,, $(CPPFLAGS)) -c $ -o $@ $(OBJS_SRV): | submake-errcodes --- 69,75 $(AR) $(AROPT) $@ $^ %_srv.o: %.c ! $(CC) $(subst $(CFLAGS_SL),,$(CFLAGS)) $(subst -DFRONTEND,, $(CPPFLAGS)) -c $ -o $@ $(OBJS_SRV): | submake-errcodes *** *** 97,100 echo #define MANDIR \$(mandir)\ $@ clean distclean maintainer-clean: ! rm -f libpgport.a libpgport_srv.a $(OBJS) $(OBJS_SRV) pg_config_paths.h --- 102,105 echo #define MANDIR \$(mandir)\ $@ clean distclean maintainer-clean: ! rm -f libpgport.so* libpgport.a libpgport_srv.a $(OBJS) $(OBJS_SRV) pg_config_paths.h *** /dev/null 2011-12-12 18:04:00.41099 -0500 --- src/port/GNUmakefile.installshared 2011-12-12 22:29:29.508512245 -0500 *** *** 0 --- 1,11 + subdir = src/port + top_builddir = ../.. + include $(top_builddir)/src/Makefile.global + + NAME = pgport + SO_MAJOR_VERSION= 1 + SO_MINOR_VERSION= 1 + + include $(top_srcdir)/src/Makefile.shlib + + install: install-lib *** src/interfaces/libpq/Makefile 2011-11-02 18:40:03.040342172 -0400 --- src/interfaces/libpq/GNUmakefile 2011-12-13 15:42:16.477592793 -0500 *** *** 88,94 # For some libpgport
Re: [HACKERS] WIP: URI connection string support for libpq
On Mon, Dec 12, 2011 at 6:55 PM, Peter van Hardenberg p...@pvh.ca wrote: I'd like to make the controversial proposal that the URL prefix should be postgres: instead of postgresql:. Postgres is a widely accepted nickname for the project, and is eminently more pronounceable. Once the url is established it will be essentially impossible to change later, but right now only a nearly insurmountable mailing list thread prevents it. That, and the fact the JDBC is already doing it the other way. A reasonable compromise might be to accept either one. AIUI, part of what Alexander was aiming for here was to unite the clans, so to speak, and it would seem a bit unfriendly (and certainly counter-productive as regards that goal) to pull the rug out from him by refusing to support that syntax over what is basically a supermassive bikeshed. However, being generous in what we accept won't cost anything, so why not? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign key locks, 2nd attempt
Excerpts from Noah Misch's message of dom dic 04 09:20:27 -0300 2011:
+/*
+ * If the tuple we're updating is locked, we need to preserve this in
the
+ * new tuple's Xmax as well as in the old tuple. Prepare the new xmax
+ * value for these uses.
+ *
+ * Note there cannot be an xmax to save if we're changing key columns;
in
+ * this case, the wait above should have only returned when the locking
+ * transactions finished.
+ */
+if (TransactionIdIsValid(keep_xmax))
+{
+if (keep_xmax_multi)
+{
+keep_xmax_old = MultiXactIdExpand(keep_xmax,
+ xid, MultiXactStatusUpdate);
+keep_xmax_infomask = HEAP_XMAX_KEYSHR_LOCK |
HEAP_XMAX_IS_MULTI;
Not directly related to this line, but is the HEAP_IS_NOT_UPDATE bit getting
cleared where needed?
AFAICS it's reset along the rest of the HEAP_LOCK_BITS when the tuple is
modified.
@@ -2725,11 +2884,20 @@ l2:
oldtup.t_data-t_infomask = ~(HEAP_XMAX_COMMITTED |
HEAP_XMAX_INVALID |
HEAP_XMAX_IS_MULTI |
- HEAP_IS_LOCKED |
+ HEAP_LOCK_BITS |
HEAP_MOVED);
+oldtup.t_data-t_infomask2 = ~HEAP_UPDATE_KEY_INTACT;
HeapTupleClearHotUpdated(oldtup);
/* ... and store info about transaction updating this tuple */
-HeapTupleHeaderSetXmax(oldtup.t_data, xid);
+if (TransactionIdIsValid(keep_xmax_old))
+{
+HeapTupleHeaderSetXmax(oldtup.t_data, keep_xmax_old);
+oldtup.t_data-t_infomask |= keep_xmax_old_infomask;
+}
+else
+HeapTupleHeaderSetXmax(oldtup.t_data, xid);
+if (key_intact)
+oldtup.t_data-t_infomask2 |= HEAP_UPDATE_KEY_INTACT;
HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);
/* temporarily make it look not-updated */
oldtup.t_data-t_ctid = oldtup.t_self;
Shortly after this, we release the content lock and go off toasting the tuple
and finding free space. When we come back, could the old tuple have
accumulated additional KEY SHARE locks that we need to re-copy?
Yeah, I've been wondering about this: do we have a problem already with
exclusion constraints? I mean, if a concurrent inserter doesn't see the
tuple that we've marked here as deleted while we toast it, it could
result in a violated constraint, right? I haven't built a test case to
prove it.
@@ -3231,30 +3462,70 @@ l3:
{
TransactionId xwait;
uint16infomask;
+uint16infomask2;
+boolrequire_sleep;
/* must copy state data before unlocking buffer */
xwait = HeapTupleHeaderGetXmax(tuple-t_data);
infomask = tuple-t_data-t_infomask;
+infomask2 = tuple-t_data-t_infomask2;
LockBuffer(*buffer, BUFFER_LOCK_UNLOCK);
/*
- * If we wish to acquire share lock, and the tuple is already
- * share-locked by a multixact that includes any subtransaction of
the
- * current top transaction, then we effectively hold the desired
lock
- * already. We *must* succeed without trying to take the tuple
lock,
- * else we will deadlock against anyone waiting to acquire
exclusive
- * lock. We don't need to make any state changes in this case.
+ * If we wish to acquire share or key lock, and the tuple is
already
+ * key or share locked by a multixact that includes any
subtransaction
+ * of the current top transaction, then we effectively hold the
desired
+ * lock already (except if we own key share lock and now desire
share
+ * lock). We *must* succeed without trying to take the tuple lock,
This can now apply to FOR UPDATE as well.
For the first sentence, I suggest the wording If any subtransaction of the
current top transaction already holds a stronger lock, we effectively hold the
desired lock already.
I settled on this:
/*
* If any subtransaction of the current top transaction already
holds a
* lock as strong or stronger than what we're requesting, we
* effectively hold the desired lock already. We *must* succeed
* without trying to take the tuple lock, else we will deadlock
against
* anyone wanting to acquire a stronger lock.
*/
if (infomask HEAP_XMAX_IS_MULTI)
{
int i;
int nmembers;
MultiXactMember *members;
Re: [HACKERS] Configuration include directory
On 12/13/2011 01:28 PM, Noah Misch wrote: !para ! Another possibility for this same sort of organization is to create a ! configuration file directory and put this information into files there. ! Other programs such asproductnameApache/productname use a !filenameconf.d/ directory for this purpose. And using numbered names This specific use of conf.d is a distribution-driven pattern; the upstream Apache HTTP Server distribution never suggests it directly... ... Overall, I'd probably just remove these comparisons to other projects. I hadn't realized that distinction; will have to look into that some more. Thanks again for the thorough review scrubbings, I can see I have another night of getting cozy with mmgr/README ahead. I've gotten more than a fair share of feedback time for this CF, I'm going to close this patch for now, keep working on it for a bit more, and re-submit later. My hope with this new section is that readers will realize the flexibility and options possible with the include and include_dir commands, and inspire PostgreSQL users to adopt familiar conventions from other programs if they'd like to. I've made no secret of the fact that I don't like the way most people are led toward inefficiently managing their postgresql.conf files, that I feel the default configurations both encourages bad practices and makes configuration tool authoring a mess. I would really like to suggest some possible alternatives here and get people to consider them, see if any gain adoption. I thought that mentioning the examples are inspired by common setups of other programs, ones that people are likely to be familiar with, enhanced that message. That's not unprecedented; doc/src/sgml/client-auth.sgml draws a similar comparison with Apache in regards to how parts of the pg_hba.conf are configured. No argument here that I need to clean that section up still if I'm going to make this argument though. I didn't expect to throw out 85 new lines of docs and get them perfect the first time. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 2: MIT Kerberos vs Microsoft Kerberos
* Greg Smith (g...@2ndquadrant.com) wrote: This answers Lionel's question, but I'm curious for a more user impact opinion from you. Given that pgAdmin III has given up on MIT KRB5, would you feel doing the same is appropriate for LibreOffice too? It sounds like they really shouldn't take on either the build cruft or the potential security issues of pulling that in at this point. Yes, I'd encourage LibreOffice to drop MIT Kerberos for Windows from their configure/install of libpq on Windows. It's just too painful and evil and, today, it might almost be better to just use the built-in Windows stuff (even on XP with the crappy encryption..) than deal with the headaches and known security flaws in the ancient MIT KfW build. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] WIP: URI connection string support for libpq
Excerpts from Robert Haas's message of Tue Dec 13 23:31:32 +0200 2011: On Mon, Dec 12, 2011 at 6:55 PM, Peter van Hardenberg p...@pvh.ca wrote: I'd like to make the controversial proposal that the URL prefix should be postgres: instead of postgresql:. Postgres is a widely accepted nickname for the project, and is eminently more pronounceable. Once the url is established it will be essentially impossible to change later, but right now only a nearly insurmountable mailing list thread prevents it. That, and the fact the JDBC is already doing it the other way. A reasonable compromise might be to accept either one. AIUI, part of what Alexander was aiming for here was to unite the clans, so to speak, and it would seem a bit unfriendly (and certainly counter-productive as regards that goal) to pull the rug out from him by refusing to support that syntax over what is basically a supermassive bikeshed. However, being generous in what we accept won't cost anything, so why not? (oops, misfired... now sending to the list) I was going to put a remark about adding to the soup here, but realized that if this is actually committed, the soup is gonna be like this: libpq-supported syntax vs. everything else (think JDBC, or is there any other driver in the wild not using libpq?) This is in the ideal world, where every binding is updated to embrace the new syntax and users have updated all of their systems, etc. Before that, why don't also accept psql://, pgsql://, postgre:// and anything else? Or wait, aren't we adding to the soup again (or rather putting the soup right into libpq?) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configuration include directory
On 12/13/2011 03:22 PM, Peter Eisentraut wrote: Well, the existing include directive works relative to the directory the including file is in. If includedir works differently from that, that would be highly confusing. Right, and that's gone now; latest update matches the regular include behavior. I would actually just extend include to accept wildcards instead of inventing a slightly new and slightly different mechanism. That's one of the ideas thrown out during the first round of discussion around this patch. Tom's summary of why that wasn't worth doing hits the highlights: http://archives.postgresql.org/pgsql-hackers/2009-10/msg01628.php -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] NOTIFY with tuples
Hi, I've used LISTEN/NOTIFY in a few apps with great success (both the new and the old implementation) but I've found myself wondering why I couldn't use a richer payload, and wondered if anyone already had plans in this direction. It seems there are number of academic and commercial systems (StreamSQL [1], CQL [2], ...) which provide powerful queryable streams of tuples, including windowing, grouping, joining and pipelining facilities, all of which are far beyond what I have been picturing. I imagine a very simple system like this, somehow built on top of the existing NOTIFY infrastructure: CREATE STREAM foo (sensor INTEGER, temperature NUMERIC); In session A: INSERT INTO foo VALUES (42, 99.0); INSERT INTO foo VALUES (99, 100.0); COMMIT; Meanwhile in session B: SELECT * FROM foo; And perhaps even some simple filtering: SELECT * FROM foo WHERE sensor = 42; I don't know how you would first signal your interest in foo before you can start SELECTing from it... perhaps with LISTEN. I suppose running the SELECT query on a stream would return only tuples that are queued up and ready to fetch, without blocking to wait for more, and a client could execute the query repeatedly, using select() on the file descriptor to wait for data to be ready (the same way people currently wait between calls to PGnotifies). As for implementation details, I haven't done much research yet into how something like this would be done and am very new to the source tree, but I thought I'd present this idea and see if it's a duplicate effort, or someone has a much better idea, or it is instantly shot down in flames for technical or other reasons, before investing further in it. Thanks! Thomas Munro [1] http://en.wikipedia.org/wiki/StreamSQL [2] http://de.wikipedia.org/wiki/Continuous_Query_Language -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: URI connection string support for libpq
On 12/13/2011 05:45 PM, Alexander Shulgin wrote: Before that, why don't also accept psql://, pgsql://, postgre:// and anything else? Or wait, aren't we adding to the soup again (or rather putting the soup right into libpq?) There are multiple URI samples within PostgreSQL drivers in the field, here are two I know of what I believe to be a larger number of samples that all match in this regard: http://sequel.rubyforge.org/rdoc/files/doc/opening_databases_rdoc.html http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html These two are using postgres. One of the hopes in adding URI support was to make it possible for the libpq spec to look similar to the ones already floating around, so that they'd all converge. Using a different prefix than the most popular ones have already adopted isn't a good way to start that. Now, whenever the URI discussion wanders off into copying the JDBC driver I wonder again why that's relevant. But making the implementation look like what people have already deployed surely is, particularly if there's no downside to doing that. Initial quick review of your patch: you suggested this as the general form: psql -d postgresql://user@pw:host:port/dbname?param1=value1param2=value2... That's presumably supposed to be: psql -d postgresql://user:pw@host:port/dbname?param1=value1param2=value2... This variation worked here: $ psql -d postgresql://gsmith@localhost:5432/gsmith If we had to pick one URI prefix, it should be postgres. But given the general name dysfunction around this project, I can't see how anyone would complain if we squat on postgresql too. Attached patch modifies yours to prove we can trivially support both, in hopes of detonating this argument before it rages on further. Tested like this: $ psql -d postgres://gsmith@localhost:5432/gsmith And that works too now. I doubt either of us like what I did to the handoff between conninfo_uri_parse and conninfo_uri_parse_options to achieve that, but this feature is still young. After this bit of tinkering with the code, it feels to me like this really wants a split() function to break out the two sides of a string across a delimiter, eating it in the process. Adding the level of paranoia I'd like around every bit of code I see that does that type of operation right now would take a while. Refactoring in terms of split and perhaps a few similarly higher-level string parsing operations, targeted for this job, might make it easier to focus on fortifying those library routines instead. For example, instead of the gunk I just added that moves past either type of protocol prefix, I'd like to just say split(buf,://,left,right) and then move on with processing the right side. I agree with your comment that we need to add some sort of regression tests for this. Given how the parsing is done right now, we'd want to come up with some interesting invalid strings too. Making sure this fails gracefully (and not in a buffer overflow way) might even use something like fuzz testing too. Around here we've just been building some Python scripting to do that sort of thing, tests that aren't practical to do with pg_regress. Probably be better from the project's perspective if such things were in Perl instead; so far no one has ever paid me enough to stomach writing non-trivial things in Perl. Perhaps you are more diverse. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 50f3f83..9c10abf 100644 *** a/src/interfaces/libpq/fe-connect.c --- b/src/interfaces/libpq/fe-connect.c *** static const PQEnvironmentOption Environ *** 282,287 --- 282,291 } }; + /* The recognized connection URI must start with one of the following designators: */ + static const char uri_designator[] = postgresql://; + static const char short_uri_designator[] = postgres://; + static bool connectOptions1(PGconn *conn, const char *conninfo); static bool connectOptions2(PGconn *conn); *** static PQconninfoOption *conninfo_parse( *** 297,303 static PQconninfoOption *conninfo_array_parse(const char *const * keywords, const char *const * values, PQExpBuffer errorMessage, bool use_defaults, int expand_dbname); ! static char *conninfo_getval(PQconninfoOption *connOptions, const char *keyword); static void defaultNoticeReceiver(void *arg, const PGresult *res); static void defaultNoticeProcessor(void *arg, const char *message); --- 301,326 static PQconninfoOption *conninfo_array_parse(const char *const * keywords, const char *const * values, PQExpBuffer errorMessage, bool use_defaults, int expand_dbname); ! static PQconninfoOption *conninfo_uri_parse(const char *uri, ! PQExpBuffer errorMessage); ! static bool
Re: [HACKERS] WIP: URI connection string support for libpq
On 12/13/2011 04:54 PM, Greg Smith wrote: On 12/13/2011 05:45 PM, Alexander Shulgin wrote: Before that, why don't also accept psql://, pgsql://, postgre:// and anything else? Or wait, aren't we adding to the soup again (or rather putting the soup right into libpq?) There are multiple URI samples within PostgreSQL drivers in the field, here are two I know of what I believe to be a larger number of samples that all match in this regard: http://sequel.rubyforge.org/rdoc/files/doc/opening_databases_rdoc.html http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html These two are using postgres. One of the hopes in adding URI support was to make it possible for the libpq spec to look similar to the ones already floating around, so that they'd all converge. Using a different prefix than the most popular ones have already adopted isn't a good way to start that. Now, whenever the URI discussion wanders off into copying the JDBC driver I wonder again why that's relevant. Because the use of Java/JDBC dwarfs both of your examples combined. Don't get me wrong, I love Python (everyone knows this) but in terms of where the work is being done it is still in Java for the most part, by far. That said, I am not really arguing against your other points except to answer your question. Sincerely, Joshua D. Drake -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development The PostgreSQL Conference - http://www.postgresqlconference.org/ @cmdpromptinc - @postgresconf - 509-416-6579 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Command Triggers
On 12/13/2011 9:59 AM, Robert Haas wrote: it. Dimitri says that he wants it so that we can add support for CREATE TABLE, ALTER TABLE, and DROP TABLE to Slony, Bucardo, and Londiste. My fear is that it won't turn out to be adequate to that task, because there won't actually be enough information in the CREATE TABLE statement to do the same thing on all servers. In particular, you won't have the index or constraint names, and you might not have the schema or tablespace information either. But maybe we ought to put the question to the intended audience for the feature - is there a Slony developer in the house? I agree. While it is one of the most asked for features among the trigger based replication systems, I fear that an incomplete solution will cause more problems than it solves. It is far easier to tell people DDL doesn't propagate automatically, do this instead ... than to try to support a limited list of commands, that may or may not propagate as intended. And all sorts of side effects, like search_path, user names and even the existing schema in the replica can cause any given DDL string to actually do something completely different than what happened on the origin. On top of that, the PostgreSQL main project has a built in replication solution that doesn't need any of this. There is no need for anyone, but us trigger replication folks, to keep command triggers in sync with all other features. I don't think it is going to be reliable enough any time soon to make this the default for any of the trigger based replication systems. Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] NOTIFY with tuples
On Tue, Dec 13, 2011 at 6:30 PM, Thomas Munro mu...@ip9.org wrote: It seems there are number of academic and commercial systems (StreamSQL [1], CQL [2], ...) which provide powerful queryable streams of tuples, including windowing, grouping, joining and pipelining facilities, all of which are far beyond what I have been picturing. I imagine a very simple system like this, somehow built on top of the existing NOTIFY infrastructure: CREATE STREAM foo (sensor INTEGER, temperature NUMERIC); In session A: INSERT INTO foo VALUES (42, 99.0); INSERT INTO foo VALUES (99, 100.0); COMMIT; Meanwhile in session B: SELECT * FROM foo; And perhaps even some simple filtering: SELECT * FROM foo WHERE sensor = 42; I don't know how you would first signal your interest in foo before you can start SELECTing from it... perhaps with LISTEN. I suppose running the SELECT query on a stream would return only tuples that are queued up and ready to fetch, without blocking to wait for more, and a client could execute the query repeatedly, using select() on the file descriptor to wait for data to be ready (the same way people currently wait between calls to PGnotifies). As for implementation details, I haven't done much research yet into how something like this would be done and am very new to the source tree, but I thought I'd present this idea and see if it's a duplicate effort, or someone has a much better idea, or it is instantly shot down in flames for technical or other reasons, before investing further in it. I'm not sure whether we'd want something like this in core, so for a first go-around, you might want to consider building it as an extension. It might work to just decree that each stream must be built around a composite type. Then you could do this: pg_create_stream(regclass) - create a stream based on the given composite type pg_destroy_stream(regclass) - nuke the stream pg_subscribe_stream(regclass) - current backend wants to read from the stream pg_unsubscribe_stream(regclass) - current backend no longer wants to read from the stream The function pg_create_stream() could create reader and writer functions for the stream. For example, if the composite type were called foo, then you'd end up with foo_read() returning SETOF foo and foo_write(foo) returning void. The C functions would look at the argument types to figure out which stream they were operating on. The writer function store all the tuples written to the stream into a temp file with a name based on the composite type OID. The reader function would return all tuples added to the temp file since the last read. You'd want the last read locations for all the subscribers stored in the file (or another file) somewhere so that when the furthest-back reader read the data, it could figure out which data was no longer need it and arrange for it to be truncated away. I'm not sure you need NOTIFY for anything anywhere in here. All in all, this is probably a pretty complicated project, but I'm sure there are people who would use it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Race condition in HEAD, possibly due to PGPROC splitup
If you add this Assert to lock.c: diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 3ba4671..d9c15e0 100644 *** a/src/backend/storage/lmgr/lock.c --- b/src/backend/storage/lmgr/lock.c *** GetRunningTransactionLocks(int *nlocks) *** 3195,3200 --- 3195,3202 accessExclusiveLocks[index].dbOid = lock-tag.locktag_field1; accessExclusiveLocks[index].relOid = lock-tag.locktag_field2; + Assert(TransactionIdIsNormal(accessExclusiveLocks[index].xid)); + index++; } } then set wal_level = hot_standby, and run the regression tests repeatedly, the Assert will trigger eventually --- for me, it happens within a dozen or so parallel iterations, or rather longer if I run the tests serial style. Stack trace is unsurprising, since AFAIK this is only called in the checkpointer: #2 0x0073461d in ExceptionalCondition ( conditionName=value optimized out, errorType=value optimized out, fileName=value optimized out, lineNumber=value optimized out) at assert.c:57 #3 0x0065eca1 in GetRunningTransactionLocks (nlocks=0x7fffa997de8c) at lock.c:3198 #4 0x006582b8 in LogStandbySnapshot (nextXid=0x7fffa997dee0) at standby.c:835 #5 0x004b0b97 in CreateCheckPoint (flags=32) at xlog.c:7761 #6 0x0062bf92 in CheckpointerMain () at checkpointer.c:488 #7 0x004cf465 in AuxiliaryProcessMain (argc=2, argv=0x7fffa997e110) at bootstrap.c:424 #8 0x006261f5 in StartChildProcess (type=CheckpointerProcess) at postmaster.c:4487 The actual value of the bogus xid (which was pulled from allPgXact[proc-pgprocno]-xid just above here) is zero. What I believe is happening is that somebody is clearing his pgxact-xid entry asynchronously to GetRunningTransactionLocks, and since that clearly oughta be impossible, something is broken. Without the added assert, you'd only notice this if you were running a standby slave --- the zero xid results in an assert failure in WAL replay on the slave end, which is how I found out about this to start with. But since we've not heard reports of such before, I suspect that this is a recently introduced bug; and personally I'd bet money that it was the PGXACT patch that broke it. I have other things to do than look into this right now myself. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] NOTIFY with tuples
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 13, 2011 at 6:30 PM, Thomas Munro mu...@ip9.org wrote: I imagine a very simple system like this, somehow built on top of the existing NOTIFY infrastructure: I'm not sure whether we'd want something like this in core, so for a first go-around, you might want to consider building it as an extension. ... I'm not sure you need NOTIFY for anything anywhere in here. Actually, what I'd suggest is just some code to serialize and deserialize tuples and transmit 'em via the existing NOTIFY payload facility. I agree that presenting it as some functions would be a lot less work than inventing bespoke syntax, but what you sketched still involves writing a lot of communications infrastructure from scratch, and I'm not sure it's worth doing that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: URI connection string support for libpq
On 12/13/2011 08:11 PM, Joshua D. Drake wrote: Because the use of Java/JDBC dwarfs both of your examples combined. Don't get me wrong, I love Python (everyone knows this) but in terms of where the work is being done it is still in Java for the most part, by far. I was talking about better targeting a new userbase, and I think that one is quite a bit larger than the current PostgreSQL+JDBC one. I just don't see any value in feeding them any Java inspired cruft. As for total size, Peter's comment mentioned having 250,000 installations using URIs already. While they support other platforms now, I suspect the majority of those are still running Heroku's original Ruby product offering. The first link I pointed at was one of the Ruby URI examples. While I do still have more Java-based customers here, there's enough Rails ones mixed in that I wouldn't say JDBC dwarfs them anymore even even for me. As for the rest of the world, I direct you toward https://github.com/erh/mongo-jdbc as a sign of the times. experimental because there's not much demand for JDBC in web app land anymore. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] NOTIFY with tuples
On 14 December 2011 04:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 13, 2011 at 6:30 PM, Thomas Munro mu...@ip9.org wrote: I imagine a very simple system like this, somehow built on top of the existing NOTIFY infrastructure: I'm not sure whether we'd want something like this in core, so for a first go-around, you might want to consider building it as an extension. ... I'm not sure you need NOTIFY for anything anywhere in here. Actually, what I'd suggest is just some code to serialize and deserialize tuples and transmit 'em via the existing NOTIFY payload facility. I agree that presenting it as some functions would be a lot less work than inventing bespoke syntax, but what you sketched still involves writing a lot of communications infrastructure from scratch, and I'm not sure it's worth doing that. Thank you both for your feedback! Looking at commands/async.c, it seems as thought it would be difficult for function code running in the backend to get its hands on the payload containing the serialized tuple, since the notification is immediately passed to the client in NotifyMyFrontEnd and there is only one queue for all notifications, you can't just put things back or not consume some of them yet IIUC. Maybe the code could changed to handle payloads holding serialized tuples differently, and stash them somewhere backend-local rather than sending to the client, so that a function returning SETOF (or a new executor node type) could deserialize them when the user asks for them. Or did you mean that libpq could support deserializing tuples on the client side? Thinking about Robert's suggestion for extension-only implementation, maybe pg_create_stream could create an unlogged table with a monotonically increasing primary key plus the columns from the composite type, and a high-water mark table to track subscribers, foo_write could NOTIFY foo to wake up subscribed clients only (ie not use the payload for the data, but clients need to use regular LISTEN to know when to call foo_read), and foo_read could update the per-subscriber high water mark and delete rows if the current session is the slowest reader. That does sound hideously heavyweight... I guess that wouldn't be anywhere near as fast as a circular buffer in a plain old file and/or a bit of shared memory. A later version could use files as suggested, bit I do want these streams to participate in transactions, and that sounds incompatible to me (?). I'm going to prototype that and see how it goes. I do like the idea of using composite types to declare the stream structure, and the foo_read function returning the SETOF composite type seems good because it could be filtered and incorporated into arbitrary queries with joins and so forth. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] review: CHECK FUNCTION statement
Hello
2011/12/12 Albe Laurenz laurenz.a...@wien.gv.at:
Pavel Stehule wrote:
there is merged patch
Works fine, except that there are still missing const qualifiers
in copyfuncs.c and equalfuncs.c that lead to compiler warnings.
One thing I forgot to mention:
I thought there was a consensus to add a WITH() or OPTIONS() clause
to pass options to the checker function:
http://archives.postgresql.org/message-id/12568.1322669...@sss.pgh.pa.us
I think this should be there so that the API does not have to be
changed in the future.
changes:
* fixed warnings
* support for options - actually only two options are supported -
quite and fatal_errors
these options are +/- useful - main reason for their existence is
testing of support of options - processing on CHECK ... stmt side and
processing on checker function side.
options are send as 2d text array - some like
'{{quite,on},{fatal_errors,on}} - so direct call of checker function
is possible
* regress test for multi check
Regards
Pavel
Yours,
Laurenz Albe
check_function-2011-12-14-1.diff.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] proposal: bytea_agg aggregate function
Hello For join of encoded text should be useful fast concat aggregation. The behave should be very similar to string_agg, only separator is useless in this case. a) This allow a fast only sql expressions on encoded texts b) our interface will be more orthogonal Regards Pavel -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] LibreOffice driver 1: Building libpq with Mozilla LDAP instead of OpenLDAP
Hello, Tom. You wrote: TL Greg Smith g...@2ndquadrant.com writes: On 12/13/2011 11:07 AM, Lionel Elie Mamane wrote: On MacOS X and Microsoft Windows, the world is far more messy. There are several ways to install libpq (one-click installer, fink, MacPorts, ...), and each of these ways allows the user to select an install location (or a prefix thereof) freely. This is less controversial. TL Yeah, I'm aware that sane handling of library dependencies is TL practically impossible under Windows, but I didn't see how that would TL result in wanting to change the configure script. I wasn't thinking TL about OSX though. (You're aware that Apple ships a perfectly fine TL libpq.so in Lion, no?) Is it true? Really? Where can we read about it? TL regards, tom lane -- With best wishes, Pavel mailto:pa...@gf.microolap.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
