Re: [HACKERS] mixed, named notation support
--On Montag, August 03, 2009 12:38:48 -0400 Robert Haas robertmh...@gmail.com wrote: Let's get back to focusing on the patch that is actually under consideration here. Status Report: I will finish documentation and review tomorrow and will mark this patch for committer review. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
One other stylistic gripe: I don't much like inserting a GUC variable definition into builtins.h --- that file has traditionally only contained function extern declarations. The best alternative I can think of is to move the bytea-related stuff into a new include file include/utils/bytea.h. Has anyone got an objection or a better idea? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Greg Williamson wrote: KaiGai -- I was pulled away from any work on this for a few days ... what can I do to help, if anything ? (Keeping in mind that my knowledge of the internals of postgres is, alas, minimal.) ... I had a quick look at this new page and want to take another, more careful, look. I got a few reasonable sugegstions at the last week. The one is it is important to provide a design specification from the viewpoint of developers to make clear what features are provided by SE-PostgreSQL, and it will be a good source for the official documentation for users. The other is a suggestion corresponding to the way to implement it. Its conclusion was to inject an abstraction layer to support multiple security models at first. I guess Robert concerned about my English quality in the documentation patch contained in the proposed patch set at first. In same time, lack of design specification was pointed out. It also should be provided prior to the patch to make clear what is identical to/different from the native database privilege mechanism. Both of them are documentations. But these have their own purpose, target and style to be required. It might be a causion of the confusion. In my current opinion, we should have a discussion corresponding to the design specifications and internals, and implement it prior to the user documentation. So, I would like you to give me the time to conclude the design specifications and to implement patches for the next commit fest. The sheer scope of this proposal undoubtedly gives pause to many, so I'd urge a certain minimalism at first to prove that the concept works and doesn't damage the core functionality at all when it is not used (fairly straight forward). Eventually rough measures of the performance hit when it is used will be useful, but users will expect something of a slow-down, I suspect, in exchange foe the greater security. Are you talking about what the user documentation should include, aren't you? Indeed, I also think these items should be introduced. To that end, I am wondering if there is test data yet designed ? Are there prescribed tests (I remember seeing some but they seem to be buried in months/threads that I have not yet re-dicsovered) ? I have some experience doing QA and could perhaps also help in these, a little. I also provided a patch for the testcases of SE-PostgreSQL. For example: http://code.google.com/p/sepgsql/source/browse/tags/sepgsql/src/test/sepgsql Thanks, And let me add, I am in awe of your efforts on this ! G - Original Message From: KaiGai Kohei kai...@ak.jp.nec.com To: Stephen Frost sfr...@snowman.net Cc: KaiGai Kohei kai...@kaigai.gr.jp; Robert Haas robertmh...@gmail.com; pgsql-hackers@postgresql.org; Greg Williamson gwilliamso...@yahoo.com; Sam Mason s...@samason.me.uk; Joshua Brindle met...@manicmethod.com Sent: Monday, August 3, 2009 12:09:45 AM Subject: Re: [HACKERS] SE-PostgreSQL Specifications Stephen Frost wrote: I think what I should do on the next is ... - To check up whether it is really possible to implement SELinux's model. - To describe the list of the security functions in the new abstraction layer. - To discuss the list of permission at: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development#Mandatory_access_controls That sounds like a good approach. As we define the security functions to go into the abstraction layer, I would also say we should identify the exact pieces of existing code which are going to move. I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction In my current impression, it indeed requires a few kilo lines of changes, but it is not impossible scale. I now plans to submit two patches for the next commit fest. The one is implementation of the abstraction layer. The other is basic implementation of the SE-PostgreSQL. So, I would like to fix external specification at least. The specifications for developer notes definitions of permissions: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development As Robert suggested before, I plans to support access controls on the following database objects and permissions at the first stage. * databases * schemas * tables * columns * sequences * functions * tablespaces Do you have any comment for the directions? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
On Mon, Aug 03, 2009 at 11:21:43AM -0400, Tom Lane wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Over the weekend I ran 40 restores of Milwaukee County's production data using Friday's snapshot with and without the patch. I alternated between patched and unpatched. It appears that this latest version is slightly slower for our production database on the same machine and configuration where the previous patch appeared to be 1% to 2% faster than unpatched (although I had fewer samples of that). I think we can conclude that for this particular test case, the effects of the patch are pretty much masked by noise. I definitely see no way that the latest version of the patch could really be slower than the original; it has the same job-scheduling behavior and strictly less list-munging overhead. Now the patch could be slower than unpatched as a result of different job-scheduling behavior ... but there's no evidence here of a consistently measurable benefit or loss from that. IIRC daveg was volunteering to do some tests with his own data; maybe we should wait for those results. I have run extensive tests with three trials of each configuration on two hosts with a variety of db sizes from 3GB to 142GB. These just finished, and I will send a more detailed summary later, but at the moment I don't see any significant difference between the patched and vanilla pg_restore. -dg -- David Gould da...@sonic.net 510 536 1443510 282 0869 If simplicity worked, the world would be overrun with insects. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Adding alter column syntax into postgres
Please suggest how best to propose that the following feature be added to PostgreSQL's roadmap? Ability to Alter column position as described in the section Adding alter column syntax into postgres of http://wiki.postgresql.org/wiki/Alter_column_position (and the two links in that section). Once implemented in tables, I would suggest a command perhaps similar to that in mysql (http://trebleclick.blogspot.com/2009/02/reorder-mysql-table-columns.html) as well as adding the feature to pgadmin and to PostgreSQL views. I would suggest that addition of this feature should be considered common sense. = -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote: One other stylistic gripe: I don't much like inserting a GUC variable definition into builtins.h --- that file has traditionally only contained function extern declarations. The best alternative I can think of is to move the bytea-related stuff into a new include file include/utils/bytea.h. Has anyone got an objection or a better idea? The other guc that controls default i/o formats for a data type is DateStyle. I can't say I expected to find that in miscadmin.h though. Perhaps move both of them into a utils/adt.h or something like that? -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Greg Stark gsst...@mit.edu writes: On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote: One other stylistic gripe: I don't much like inserting a GUC variable definition into builtins.h --- that file has traditionally only contained function extern declarations. The best alternative I can think of is to move the bytea-related stuff into a new include file include/utils/bytea.h. Has anyone got an objection or a better idea? The other guc that controls default i/o formats for a data type is DateStyle. I can't say I expected to find that in miscadmin.h though. Perhaps move both of them into a utils/adt.h or something like that? Hmm, actually now that you mention it there's a bunch of GUC variables in miscadmin.h. Surprise factor aside, I'm inclined to just shove bytea_output in there along with DateStyle/IntervalStyle/etc. I did try the new-include-file approach, and unsurprisingly found three or four files that had to be modified to include it, because they'd been expecting to find byteain and byteaout declared in builtins.h. I still think that way is a bit cleaner, but I'm not sure it's enough cleaner to risk breaking third-party code for. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding alter column syntax into postgres
JwexlerAt MailDotCom wrote: Please suggest how best to propose that the following feature be added to PostgreSQL's roadmap? Ability to Alter column position as described in the section Adding alter column syntax into postgres of http://wiki.postgresql.org/wiki/Alter_column_position (and the two links in that section). It is already on the roadmap. You either need to do the actual work, or convince someone else to do it for you. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding alter column syntax into postgres
Jwexler, Please suggest how best to propose that the following feature be added to PostgreSQL's roadmap? Ability to Alter column position as described in the section Adding alter column syntax into postgres of http://wiki.postgresql.org/wiki/Alter_column_position (and the two links in that section). Fortunately, there's already a specification discussed for this. We'd like to have the ability for each column to have both a logical position and a physical position which are separate. This would also allow us to dynamically re-arrange the columns at CREATE time for best storage efficiency. It would also provide a foundation for column aliases. However, I don't think there's much code for this yet. Are you in a position to either write code, or fund work? Once implemented in tables, I would suggest a command perhaps similar to that in mysql (http://trebleclick.blogspot.com/2009/02/reorder-mysql-table-columns.html) as well as adding the feature to pgadmin and to PostgreSQL views. H. It *might* be worth supporting that for MySQL compatibility, but really it hardly seems adequate to a large table where you want to rearrange most of the columns. Seems like we'd need a better syntax. I would suggest that addition of this feature should be considered common sense. Actually, it's not. Column ordering is supposed to be arbitrary and implementation-dependent, so we're on our own here. However, it would help people administer their applications ... even if it encourages bad application writers to continue using SELECT *. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Tue, Aug 4, 2009 at 12:18 AM, Tom Lanet...@sss.pgh.pa.us wrote: One other stylistic gripe: I don't much like inserting a GUC variable definition into builtins.h --- that file has traditionally only contained function extern declarations. �The best alternative I can think of is to move the bytea-related stuff into a new include file include/utils/bytea.h. �Has anyone got an objection or a better idea? The other guc that controls default i/o formats for a data type is DateStyle. I can't say I expected to find that in miscadmin.h though. Perhaps move both of them into a utils/adt.h or something like that? Hmm, actually now that you mention it there's a bunch of GUC variables in miscadmin.h. Surprise factor aside, I'm inclined to just shove bytea_output in there along with DateStyle/IntervalStyle/etc. I vote for a new bytea.h file that does not slurp in byteain/byteaout, to avoid breaking 3rd party code. miscadmin.h seems the worst solution, since it's already included in 210 other files. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support Index: src/pl/plpgsql/src/pl_exec.c === RCS file: /home/alvherre/Code/cvs/pgsql/src/pl/plpgsql/src/pl_exec.c,v retrieving revision 1.246 diff -c -p -r1.246 pl_exec.c *** src/pl/plpgsql/src/pl_exec.c 22 Jul 2009 02:31:38 - 1.246 --- src/pl/plpgsql/src/pl_exec.c 4 Aug 2009 01:00:06 - *** *** 23,28 --- 23,29 #include executor/spi_priv.h #include funcapi.h #include lib/stringinfo.h + #include miscadmin.h #include nodes/nodeFuncs.h #include parser/scansup.h #include storage/proc.h Index: src/pl/plpgsql/src/pl_handler.c === RCS file: /home/alvherre/Code/cvs/pgsql/src/pl/plpgsql/src/pl_handler.c,v retrieving revision 1.44 diff -c -p -r1.44 pl_handler.c *** src/pl/plpgsql/src/pl_handler.c 18 Feb 2009 11:33:04 - 1.44 --- src/pl/plpgsql/src/pl_handler.c 4 Aug 2009 00:59:54 - *** *** 18,23 --- 18,24 #include catalog/pg_proc.h #include catalog/pg_type.h #include funcapi.h + #include miscadmin.h #include utils/builtins.h #include utils/guc.h #include utils/lsyscache.h Index: src/pl/plpgsql/src/plpgsql.h === RCS file: /home/alvherre/Code/cvs/pgsql/src/pl/plpgsql/src/plpgsql.h,v retrieving revision 1.114 diff -c -p -r1.114 plpgsql.h *** src/pl/plpgsql/src/plpgsql.h 22 Jul 2009 02:31:38 - 1.114 --- src/pl/plpgsql/src/plpgsql.h 4 Aug 2009 00:59:06 - *** *** 20,26 #include access/xact.h #include fmgr.h - #include miscadmin.h #include commands/trigger.h #include executor/spi.h #include utils/tuplestore.h --- 20,25 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Alvaro Herrera alvhe...@commandprompt.com writes: I vote for a new bytea.h file that does not slurp in byteain/byteaout, to avoid breaking 3rd party code. miscadmin.h seems the worst solution, since it's already included in 210 other files. Well, unless you want to leave *all* the bytea functions in builtins.h there will still be some risk there. I'd actually sooner break calls of byteaout than other things, because in reality every caller of byteaout is going to need to be inspected to see if it's expecting the old-style output format. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
2009/8/3 KaiGai Kohei kai...@ak.jp.nec.com: I now plans to submit two patches for the next commit fest. The one is implementation of the abstraction layer. The other is basic implementation of the SE-PostgreSQL. Is this a good idea, or would it be better to focus on the aclcheck stuff (which is what I understand you to mean here by abstraction layer) first? You will be much happier getting one patch committed than two patches not committed... getting two patches of this size in one CommitFest seems very unlikely, and I worry that the SE-PostgreSQL patch will distract your time and reviewing time from the aclcheck refactoring that must get done first, and well. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: I vote for a new bytea.h file that does not slurp in byteain/byteaout, to avoid breaking 3rd party code. miscadmin.h seems the worst solution, since it's already included in 210 other files. Well, unless you want to leave *all* the bytea functions in builtins.h there will still be some risk there. I'd actually sooner break calls of byteaout than other things, because in reality every caller of byteaout is going to need to be inspected to see if it's expecting the old-style output format. Hmm, good point ... why avoid the breakage then? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: Well, unless you want to leave *all* the bytea functions in builtins.h there will still be some risk there. I'd actually sooner break calls of byteaout than other things, because in reality every caller of byteaout is going to need to be inspected to see if it's expecting the old-style output format. Hmm, good point ... why avoid the breakage then? Maybe we shouldn't. Okay, back to plan A (separate bytea.h file). (BTW, so far as I can tell there isn't anything in the backend that will be broken in that way. pg_dump, however, is a different story... it knows way too much about pg_trigger.tgargs.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding alter column syntax into postgres
Fortunately, there's already a specification discussed for this. We'd like to have the ability for each column to have both a logical position and a physical position which are separate. This would also allow us to dynamically re-arrange the columns at CREATE time for best storage efficiency. It would also provide a foundation for column aliases. However, I don't think there's much code for this yet. Are you in a position to either write code, or fund work? I am glad to here that there is a specification discussed. This would be a great help in administration and organization tasks (for example when working with a very large database using the pgadmin tool). I wish I were in a position for assisting with the code or funding but unfortunately am not. Thank you for considering and I am hoping that interest in this increases and over the near-term horizon. = 700% Massive Penny Stock Gains We Find Penny Stocks that go up! Were the No1 Stock Newsletter Online. http://a8-asy.a8ww.net/a8-ads/adftrclick?redirectid=6d0b4885a30374506bbb3552d9331caf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for 8.5, transformationHook
On Thu, Jul 30, 2009 at 1:22 AM, Pavel Stehulepavel.steh...@gmail.com wrote: 2009/7/30 Robert Haas robertmh...@gmail.com: On Sun, Jul 26, 2009 at 9:29 AM, Pavel Stehulepavel.steh...@gmail.com wrote: Hello new patch add new contrib transformations with three modules anotation, decode and json. These modules are ported from my older work. Before applying this patch, please use named-fixed patch too. The hook doesn't need it, but modules anotation and json depend on it. These are pretty good examples, but the whole thing still feels a bit grotty to me. The set of syntax transformations that can be performed with a hook of this type is extremely limited - in particular, it's the set of things where the parser thinks it's valid and that the structure is reasonably similar to what you have in mind, but the meaning is somewhat different. The fact that two of your three examples require your named and mixed parameters patch seems to me to be evidence of that. I see the main hook using as open door to functionality like decode and json. Anotation is little bit corner use case. We don't need a change of syntax or rules in parser. But I need to get some info for functions from parser stage - like JSON or replace standard coercion rules like decode. Hook is the most simple and general technique for it (what I found). I thing, so there are other techniques - but it needs more invasive patch and are not too general - what is values of any hooking. I doesn't thing, so there will be any real extended parser based on bison in near or far future. I thing, so this is theoretically possible, but nobody work on it. What more - with extensible parser we still need the transformation hook, because we need the change in transformation - decode, json. The JSON transformation provides functionality which is very similar to what we also offer for XML. I sort of think we ought to just provide that, rather than making it an add-on. I have found it to be a tremendously attractive alternative to XML. The JSON is only one use case (there should be output to any format), and I agree, so this should be in core. But every integration similar function to core needs one or two years. Hook allows do this things faster and from external library. It's little bit lighter process to put your project to pgfoundry than to PostgreSQL core. I don't really believe that JSON is only one use case. XML and JSON are in a class of their own; there's nothing else out there that is really comparable. So I guess I'm back to feeling like this is of pretty marginal benefit. But I would still like some opinions from others. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
In reference to:
http://archives.postgresql.org/message-id/6534f7ae0811181547v1dc1f096g6222e8273b461...@mail.gmail.com
Had to fix a lot of bit rot, but otherwise looks good. My updated patch
attached. Will commit in a day or so if no objections.
BTW, some commitfest procedural comments/questions:
1) I couldn't figure out how to attach my patch to the commitfest page
short of cut-n-paste into the small comment text box -- is that my only
choice?
2) It would be nice if the mail archive web page of the original message
had a Reply To button. Otherwise I guess I can do what I've done above.
3) I couldn't see any way to assign myself as the committer.
Joe
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iQIcBAEBCAAGBQJKd5K8AAoJEDfy90M199hlDbIQAIYeLR1B3JKkXvoBCZUkzVNj
hkj67d38ygU/Q4RDsnPT0N96PbPvzBJGCueaCLfwzbdpEQb4Si37R8xBMXPzT/df
SEoWrx/2r1/cERyImxHVGlj0kBWBa7K42hcEotD0mNqWqX6rqByLDpVTSlpZZdfM
a/b04iXfPcIvOLSpX6PJOb047SeHQrzOmcnurBqc1HE81XiiBNQlJjOd0Zi+y+pT
zGJChGSRO1lXriOI1Pu2K18daqY8vLLZA0LlaZ3SD6UeS7Uayeo+8cOpAk6N1b7H
EKqYAwWfmRrSO6bUCmgkqlHa/TiaIL0uJb0me68YcUy08DAt1O8iRmUnAE+cxXU5
HeZgGqJI2G19Ts5i9R2YaHVe8kTmPD88zztv8giiAw01m9h6azkiM342CzNuyQTw
8TbZnjWdCkb4KuH9en4C/puIWUpCOd2OkVju3ZUJCvzaO/jS/HVf2fc08K9TddPK
OWpsytXCNS6ojM/Em5lZzfsZZ9sDcrP29dSHZEJqOIkMhFoEGqq4DVqtJnLMHsLw
3PtSfgz0SbXcuAA0vv/VJSDm+cAl+aJN93hbwNybwrT14XEotllFdhyJrSPqFgYm
4b7P29dFKP+aF90C6klxr5Mq/nRYiJVdTepqjfyikVBsdZoMwQXvG+ROiwbzjl9g
eEmht8ysuxn2Ju8FcDYc
=AJfk
-END PGP SIGNATURE-
Index: contrib/dblink/dblink.c
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v
retrieving revision 1.82
diff -c -r1.82 dblink.c
*** contrib/dblink/dblink.c 11 Jun 2009 14:48:50 - 1.82
--- contrib/dblink/dblink.c 4 Aug 2009 01:05:49 -
***
*** 1635,1640
--- 1635,1681
PG_RETURN_DATUM(current_query(fcinfo));
}
+ /*
+ * Retrieve async notifications for a connection.
+ *
+ * Returns an array of notifications, or NULL if none recieved.
+ * Can optionally take a named connection as parameter, but uses the unnamed connection per default.
+ *
+ */
+ PG_FUNCTION_INFO_V1(dblink_get_notify);
+ Datum
+ dblink_get_notify(PG_FUNCTION_ARGS)
+ {
+ PGconn *conn = NULL;
+ remoteConn *rconn = NULL;
+ ArrayBuildState *astate = NULL;
+ PGnotify *notify;
+
+ DBLINK_INIT;
+ if (PG_NARGS() == 1)
+ DBLINK_GET_NAMED_CONN;
+ else
+ conn = pconn-conn;
+
+ PQconsumeInput(conn);
+
+ while ((notify = PQnotifies(conn)) != NULL)
+ {
+ /* stash away current value */
+ astate = accumArrayResult(astate,
+ PointerGetDatum(cstring_to_text(notify-relname)),
+ false, TEXTOID, CurrentMemoryContext);
+ PQfreemem(notify);
+ PQconsumeInput(conn);
+ }
+
+ if (astate)
+ PG_RETURN_ARRAYTYPE_P(makeArrayResult(astate,
+ CurrentMemoryContext));
+ else
+ PG_RETURN_NULL();
+ }
+
/*
* internal functions
*/
Index: contrib/dblink/dblink.h
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v
retrieving revision 1.22
diff -c -r1.22 dblink.h
*** contrib/dblink/dblink.h 9 Jun 2009 17:41:02 - 1.22
--- contrib/dblink/dblink.h 4 Aug 2009 00:56:56 -
***
*** 57,61
--- 57,62
extern Datum dblink_build_sql_delete(PG_FUNCTION_ARGS);
extern Datum dblink_build_sql_update(PG_FUNCTION_ARGS);
extern Datum dblink_current_query(PG_FUNCTION_ARGS);
+ extern Datum dblink_get_notify(PG_FUNCTION_ARGS);
#endif /* DBLINK_H */
Index: contrib/dblink/dblink.sql.in
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.sql.in,v
retrieving revision 1.18
diff -c -r1.18 dblink.sql.in
*** contrib/dblink/dblink.sql.in 9 Jun 2009 17:41:02 - 1.18
--- contrib/dblink/dblink.sql.in 4 Aug 2009 00:58:52 -
***
*** 202,204
--- 202,214
RETURNS text
AS 'MODULE_PATHNAME', 'dblink_error_message'
LANGUAGE C STRICT;
+
+ CREATE OR REPLACE FUNCTION dblink_get_notify()
+ RETURNS text[]
+ AS 'MODULE_PATHNAME', 'dblink_get_notify'
+ LANGUAGE C STRICT;
+
+ CREATE OR REPLACE FUNCTION dblink_get_notify(conname text)
+ RETURNS text[]
+ AS 'MODULE_PATHNAME', 'dblink_get_notify'
+ LANGUAGE C STRICT;
Index: contrib/dblink/uninstall_dblink.sql
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/uninstall_dblink.sql,v
retrieving revision 1.7
diff -c -r1.7 uninstall_dblink.sql
*** contrib/dblink/uninstall_dblink.sql 5 Apr 2008 02:26:14 - 1.7
--- contrib/dblink/uninstall_dblink.sql 4 Aug 2009 00:56:56 -
***
*** 76,78
--- 76,82
DROP FUNCTION dblink_is_busy(text);
DROP FUNCTION
[HACKERS] pg_proc.probin should become text?
pg_proc.probin is currently declared as being type bytea. Perhaps that had some real utility once upon a time, but no currently defined procedural language stores binary data there. In fact, the only thing that gets stored there is the shared-library file name for C functions. To make things even more fun, none of the code touching probin is doing anything to honor the special escaping conventions for bytea. This was pretty broken already, and might perhaps explain some of the weirder error reports we've gotten. It is now obvious to me that no shlib file name containing non-ASCII characters will correctly survive a dump/reload, in any existing PG release. Backslashes accidentally fail to fail, at least for some values of standard_conforming_strings. However, with the hex bytea output patch in place, it's *completely* broken. I offer the following pg_dump output: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '\\x2f686f6d652f706f7374677265732f706773716c2f7372632f746573742f726567726573732f726567726573732e736c', 'interpt_pp'; which should of course have looked like this: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '/home/postgres/testversion/src/test/regress/regress.sl', 'interpt_pp'; I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Comments? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/3 Brendan Jurd dire...@gmail.com: Okay, so Oracle just forces the output wider to accomodate the exponent (i.e., you can't rely on it being fixed-width). I can adjust the patch to imitate this behaviour; should be able to post a new revision within 24 hours. Please find attached version 5 of the patch, which allows for exponents with any number of digits (up to a total string length of MAXDOUBLEWIDTH). Cheers, BJ _5.diff.bz2 Description: BZip2 compressed data _4-to-5.diff Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Robert Haas wrote: 2009/8/3 KaiGai Kohei kai...@ak.jp.nec.com: I now plans to submit two patches for the next commit fest. The one is implementation of the abstraction layer. The other is basic implementation of the SE-PostgreSQL. Is this a good idea, or would it be better to focus on the aclcheck stuff (which is what I understand you to mean here by abstraction layer) first? You will be much happier getting one patch committed than two patches not committed... getting two patches of this size in one CommitFest seems very unlikely, and I worry that the SE-PostgreSQL patch will distract your time and reviewing time from the aclcheck refactoring that must get done first, and well. Needless to say, the security abstraction layer shall have higher priority than SE-PostgreSQL which depends on the layer. If we can focus on the SE-PostgreSQL feature in the third commit fest, it will be better than all the facilities from the scratch. (BTW, I also have a WIP patch to support largeobject permissions.) So, we may be able to modify the development plan as follows: * 2nd CommitFest (15-Sep) - security abstraction layer (- largeobject permission) * 3rd CommitFest (15-Nov) - basic functionality of SE-PostgreSQL * 4th CommitFest (15-Jan) - full functionality of SE-PostgreSQL (row-level controls, filesystem permissions, ...) Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/3 Tom Lane t...@sss.pgh.pa.us:
Euler Taveira de Oliveira eu...@timbira.com writes:
As I said in a prior e-mail, Oracle has a diferent overflow limit (-84 to
127).
In PostgreSQL, the numeric datatype can have up to 1000 digits (ie 1e+999)
and
the double precision datatype can have up to 309 digits (ie 1e-307 or
1e+308).
We should support up to 3 exponent digits so all of our native datatypes are
covered by the to_char() function.
Uh, no, we had better support more. The actual limit of the current
numeric format is 1e+131072.
As long as we consider that should emit as many exponent digits
as needed, this isn't particularly critical. But it would be if we
try to specify an exact number of output digits.
Well, I tried this and as it turns out the patch casts the value to a
float8 in order to pass it on to snprintf for sci-notation formatting.
See the following chunk in numeric_to_char():
else if (IS_(Num))
{
float8 val;
val = DatumGetFloat8(DirectFunctionCall1(numeric_float8,
NumericGetDatum(value)));
numstr = orgnum = (char *) palloc(MAXDOUBLEWIDTH + 1);
len = snprintf(orgnum, MAXDOUBLEWIDTH + 1, %.*e, Num.post,
val);
}
This leads to the following behaviour:
=# select to_char(1e+1000::numeric, '9.99');
ERROR:
1
is out of range for type double precision
If we want to support the full range of numeric values with , I
guess we would need to write our own implementation of scientific
notation rather than depend on sprintf()'s?
Cheers,
BJ
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
Joe Conway m...@joeconway.com writes: BTW, some commitfest procedural comments/questions: 1) I couldn't figure out how to attach my patch to the commitfest page short of cut-n-paste into the small comment text box -- is that my only choice? No, what you should do is first send the patch to -hackers, then put the message-ID of that email into the place provided. The comment box isn't really meant for much more than a one-line summary of the message being linked to. 3) I couldn't see any way to assign myself as the committer. Yeah, the webapp doesn't explicitly record the committer for a patch. What I've been doing is adding a comment saying that I'm taking a patch to commit. A separate field would probably be better though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: So, we may be able to modify the development plan as follows: * 2nd CommitFest (15-Sep) - security abstraction layer (- largeobject permission) * 3rd CommitFest (15-Nov) - basic functionality of SE-PostgreSQL * 4th CommitFest (15-Jan) - full functionality of SE-PostgreSQL (row-level controls, filesystem permissions, ...) Not to throw water on this right from the get-go, but I think getting the security abstraction and basic SE-PostgreSQL functionality (based on existing PG permissions) into 8.5 will be enough of a stretch. row-level security needs to be implement in PG proper first, before we can add the SE-PG hooks for it. That's going to be a serious amount of work by itself, and is something which is extremely unlikely to make sense to commit that late in the cycle. Let's focus on improving aclchk.c to the point where SE-PG can be easily added without dropping hooks all over the place. If we can get that into 8.5 it will be a huge success. We can then work on row-level permissions for 8.6, first as a PG-native feature, and then with SE-PG hooks. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] async notification patch for dblink
Joe Conway escribió: 2) It would be nice if the mail archive web page of the original message had a Reply To button. Otherwise I guess I can do what I've done above. I totally agree, but this is not workable given our current software. I've been giving some time to reworking the email archives using something completely different to allow this kind of trick, but this is to be considered longish-term. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alvaro Herrera wrote: Joe Conway escribió: 2) It would be nice if the mail archive web page of the original message had a Reply To button. Otherwise I guess I can do what I've done above. I totally agree, but this is not workable given our current software. I've been giving some time to reworking the email archives using something completely different to allow this kind of trick, but this is to be considered longish-term. OK, good to know it isn't just me ;-) Thanks, Joe -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iQIcBAEBCAAGBQJKd5zzAAoJEDfy90M199hlCuoP/0ZDDBSjShDeZXf5ReMEl/cw 7W5N0J78HJoM9Jrq587i/iqc//hU35V9Jq2GxHXbwYo+ByWd86KCwmhoY7fEDMTZ bQKd26ZPmTdxME/JXF+R/6XWa5cDhzOyaMNsaJJkSrT1CvJTk6H10R5JF8ibrz2H gcAPFYprd7IYGzXamq8cwOnsu3pTPBXDu6Z7HxSd3d1AoZfb3YwPsN+1Y18U5ffj XCy/MbL8EmO/Vvs3cFTG2AP04tU6w63w1zENN0FHaWc6zyDQBbgrUhg9dudnIrNz wlFigvECwPFST0pi6pNJie+0gOkgwTZq3ePWUb1J3qIVycCezS7dWNFgLRO2t8XH zrlnYF2V9QouMfwDU+oYdS95mUNcFIKq65nHLi6skswNPh+04iX54TX85XaR9wge MDn0vCtWV1gqC8SFrkkerNRo1pGhxIuAzjxlRrIaRkPzjcQLh/DAtYPyHNnv7LFp hETT7AL0MUmvXlL+eHAQ+cotV98sF/Jw6Lb94e8wUABcgxoyknbUQxUCn0k//Wrz irBRKLPLDS28EdZayV1VzyeNoNLtDH5X6vjp2Y6yqLkn+hQQNpjEPhUIFh+aaDLv 4TXe53oTnHOkle1PbhPLjE+MbRLTBxeHODNia5t3yY4mNHbiGvwS0ASPOzs5MPNX wyfwBeLCpxSAgZsYU7SS =Wy56 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
Joe Conway m...@joeconway.com writes: In reference to: http://archives.postgresql.org/message-id/6534f7ae0811181547v1dc1f096g6222e8273b461...@mail.gmail.com Had to fix a lot of bit rot, but otherwise looks good. My updated patch attached. Will commit in a day or so if no objections. After a quick look-over, here are some comments on the patch itself: 1. Sooner or later, hopefully sooner, we will have payload strings in notifications, not just the relname string. libpq and the protocol already have support for that (see the extra field in PGnotify). It would be a shame to have this function's API become obsolete immediately. Can we fix the API to return the extra string too? 2. By the same line of reasoning, the be_pid field might be useful. Back when I was doing a lot of LISTEN/NOTIY coding, you really needed that field in order to distinguish your own notifies coming back from other sessions' notifies. I don't think we've done anything to eliminate the need for that. 3. I see the function returns NULL when there's nothing to report, but maybe an empty array would be more sensible. [ thinks for awhile... ] Actually, it seems to me that the present patch's definition of the function would be very hard to work with. You would normally want to work with the events one at a time. There isn't much you could do with the array result except unnest() it, and I'm a bit worried that careless usage could result in multiple evaluation of the function and hence loss of events. I wonder whether it would be better to have the function return setof record. Which, not incidentally, would greatly simplify adding in those extra result fields. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
On Mon, Aug 3, 2009 at 10:03 PM, Tom Lanet...@sss.pgh.pa.us wrote: pg_proc.probin is currently declared as being type bytea. Perhaps that had some real utility once upon a time, but no currently defined procedural language stores binary data there. In fact, the only thing that gets stored there is the shared-library file name for C functions. To make things even more fun, none of the code touching probin is doing anything to honor the special escaping conventions for bytea. This was pretty broken already, and might perhaps explain some of the weirder error reports we've gotten. It is now obvious to me that no shlib file name containing non-ASCII characters will correctly survive a dump/reload, in any existing PG release. Backslashes accidentally fail to fail, at least for some values of standard_conforming_strings. However, with the hex bytea output patch in place, it's *completely* broken. I offer the following pg_dump output: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '\\x2f686f6d652f706f7374677265732f706773716c2f7372632f746573742f726567726573732f726567726573732e736c', 'interpt_pp'; which should of course have looked like this: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '/home/postgres/testversion/src/test/regress/regress.sl', 'interpt_pp'; I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Comments? Will that require a special hack in pg_migrator? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Tom Lane wrote: [ thinks for awhile... ] Actually, it seems to me that the present patch's definition of the function would be very hard to work with. You would normally want to work with the events one at a time. There isn't much you could do with the array result except unnest() it, and I'm a bit worried that careless usage could result in multiple evaluation of the function and hence loss of events. I wonder whether it would be better to have the function return setof record. Which, not incidentally, would greatly simplify adding in those extra result fields. Sure that makes sense. I'll take a stab at it. Joe -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iQIcBAEBCAAGBQJKd54DAAoJEDfy90M199hleEoP/i9JAI3c+5LMrz90ntAoEthf PzC6+QpjKIEeU5vF/NFZl8r7yWiCqlw/015clGQ4bzZfoCVwKiabGl9ziH8G1xzz ruuM/dr3H8gZfl69LdN65t+t0P1QLSLeNeOQbtLjm0n9439lCt8r8q3joxw2sKIK HHN6BBpyINHrlgkfw0CjRYLm0kEFW2Jj76AQyOs0V4HYtK4d+Zmr0Ut3q9aTHg7h 1MYtkPHvzq0ploKmMtx7+zpqEI3JPzyWA2hxeCZJfHHM5Y7j2eadDeN+CqWaDjs5 2T1HrtXIgtzeQWBV7J8q3rGTFc3YXTzv0mCYveHULUByl/vIx6Lind6ErbPd7gig sTUdTo77JK7J4oV4PAZfJDRMIjUiKZGHoOPMeCIIXPyuCIYCFv/YTR3lFlEllwws 3ocY/0BZzNtUiCvH7CLD0BiSNF2sSfG6bC1I9FbHwoezlPCLKInjUoRyknGSnHAV i2W5IIdmHiwxR5sSy+zNAUASFaK2shcvn2SX0hLbPAsDAMnPa1nYVNuqoojb1HWG uvYXtRHoBrrtQkXl2F8NzSBmiaxfG02YY5Y2o6zkv8C/UG5+eaUIbF7SKcZMmHwJ Ar/Zdz/eY0c/Fcy3ttfHc4C03E6qn1aDKHD+sXgDMDHfbGafDGfGhntW92ipngP1 CXbtfEYLgWcO4eGHusSB =/Q5q -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
Brendan Jurd dire...@gmail.com writes: Well, I tried this and as it turns out the patch casts the value to a float8 in order to pass it on to snprintf for sci-notation formatting. Well, that's pretty dumb. Quite aside from the range problem, that would mean that you lose everything past the sixteenth or so digit. I think that's sufficient grounds for bouncing the patch back for rework right there. What I'd consider instead is calling numeric_out and then working with the result of that. It would always be f-format, so you'd have to do your own conversion to e-format, but you could do it without any risk of precision or range loss. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
Robert Haas robertmh...@gmail.com writes: On Mon, Aug 3, 2009 at 10:03 PM, Tom Lanet...@sss.pgh.pa.us wrote: I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Will that require a special hack in pg_migrator? No, pg_dump (and hence pg_migrator) wouldn't care. If we thought we were going to try to back-patch a fix for the non-ASCII-char problem into older releases, then some other approach would be preferable. But given the fact that this hasn't gotten complained of (at least not enough to be identified) in all the years since Berkeley, I can't see doing the pushups that would be needed to back-patch a fix. It looks to me like everyone has been effectively assuming that probin stores text, and we should just make it really truly do that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Stephen Frost wrote: KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: So, we may be able to modify the development plan as follows: * 2nd CommitFest (15-Sep) - security abstraction layer (- largeobject permission) * 3rd CommitFest (15-Nov) - basic functionality of SE-PostgreSQL * 4th CommitFest (15-Jan) - full functionality of SE-PostgreSQL (row-level controls, filesystem permissions, ...) Not to throw water on this right from the get-go, but I think getting the security abstraction and basic SE-PostgreSQL functionality (based on existing PG permissions) into 8.5 will be enough of a stretch. row-level security needs to be implement in PG proper first, before we can add the SE-PG hooks for it. That's going to be a serious amount of work by itself, and is something which is extremely unlikely to make sense to commit that late in the cycle. It seems to me it is a bit early to conclude what feature will be included into 8.5 and what feature is not so. The above plan is a very rough sketch. Anyway, what I would like to say is I can agree to focus on the security abstraction layer earlier than SE-PostgreSQL feature. -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
Tom Lane wrote: 3) I couldn't see any way to assign myself as the committer. Yeah, the webapp doesn't explicitly record the committer for a patch. What I've been doing is adding a comment saying that I'm taking a patch to commit. A separate field would probably be better though. +1. I raised this before, iirc. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/4 Tom Lane t...@sss.pgh.pa.us: Brendan Jurd dire...@gmail.com writes: Well, I tried this and as it turns out the patch casts the value to a float8 in order to pass it on to snprintf for sci-notation formatting. Well, that's pretty dumb. Quite aside from the range problem, that would mean that you lose everything past the sixteenth or so digit. I think that's sufficient grounds for bouncing the patch back for rework right there. I agree. What I'd consider instead is calling numeric_out and then working with the result of that. It would always be f-format, so you'd have to do your own conversion to e-format, but you could do it without any risk of precision or range loss. Yeah, I figured as much. I'll see what I can do about reworking the numeric case. I should be able to post a new revision in the next day or so, but I certainly won't cry foul if this gets punted to September. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
Brendan Jurd dire...@gmail.com writes: 2009/8/4 Tom Lane t...@sss.pgh.pa.us: What I'd consider instead is calling numeric_out and then working with the result of that. It would always be f-format, so you'd have to do your own conversion to e-format, but you could do it without any risk of precision or range loss. Yeah, I figured as much. I'll see what I can do about reworking the numeric case. I should be able to post a new revision in the next day or so, but I certainly won't cry foul if this gets punted to September. BTW, there's no rule saying you have to fix that strictly within to_char(). It might make more sense to have numeric.c export a function that is like numeric_out but produces e-style output. Your choice as the patch writer, but keep it in mind ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
On Mon, Aug 03, 2009 at 10:03:11PM -0400, Tom Lane wrote: However, with the hex bytea output patch in place, it's *completely* broken. I offer the following pg_dump output: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '\\x2f686f6d652f706f7374677265732f706773716c2f7372632f746573742f726567726573732f726567726573732e736c', 'interpt_pp'; which should of course have looked like this: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '/home/postgres/testversion/src/test/regress/regress.sl', 'interpt_pp'; Oh, of course! How could I have been so dense? ;) I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Comments? +1 on changing it to text. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction I'm not really a huge fan of 'security_' as a prefix for these functions, but I don't have a better suggestion right now. The initial abstraction patch shouldn't include the security context pieces. I realize that will be needed eventually, but the patch to do the abstraction and to formally move permissions checking to aclchk.c needs to stand alone. I'm also not sure that the API of having the security context be returned as a Datum makes sense.. Doesn't security_table_permissions() need to know if the query is an UPDATE or an INSERT? Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] SE-PostgreSQL Specifications
On Mon, Aug 3, 2009 at 10:19 PM, Stephen Frostsfr...@snowman.net wrote: KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: So, we may be able to modify the development plan as follows: * 2nd CommitFest (15-Sep) - security abstraction layer (- largeobject permission) * 3rd CommitFest (15-Nov) - basic functionality of SE-PostgreSQL * 4th CommitFest (15-Jan) - full functionality of SE-PostgreSQL (row-level controls, filesystem permissions, ...) Not to throw water on this right from the get-go, but I think getting the security abstraction and basic SE-PostgreSQL functionality (based on existing PG permissions) into 8.5 will be enough of a stretch. row-level security needs to be implement in PG proper first, before we can add the SE-PG hooks for it. That's going to be a serious amount of work by itself, and is something which is extremely unlikely to make sense to commit that late in the cycle. +1. Optimism is good, realism is better. Let's focus on improving aclchk.c to the point where SE-PG can be easily added without dropping hooks all over the place. If we can get that into 8.5 it will be a huge success. We can then work on row-level permissions for 8.6, first as a PG-native feature, and then with SE-PG hooks. Row-level security is going to be a very, very difficult feature to implement properly. A lot of thought is needed here to design something good. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
On Mon, Aug 3, 2009 at 10:44 PM, Andrew Dunstanand...@dunslane.net wrote: Tom Lane wrote: 3) I couldn't see any way to assign myself as the committer. Yeah, the webapp doesn't explicitly record the committer for a patch. What I've been doing is adding a comment saying that I'm taking a patch to commit. A separate field would probably be better though. +1. I raised this before, iirc. Yes, I just haven't had enough round tuits yet. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
On Mon, Aug 3, 2009 at 10:40 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Aug 3, 2009 at 10:03 PM, Tom Lanet...@sss.pgh.pa.us wrote: I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Will that require a special hack in pg_migrator? No, pg_dump (and hence pg_migrator) wouldn't care. If we thought we were going to try to back-patch a fix for the non-ASCII-char problem into older releases, then some other approach would be preferable. But given the fact that this hasn't gotten complained of (at least not enough to be identified) in all the years since Berkeley, I can't see doing the pushups that would be needed to back-patch a fix. It looks to me like everyone has been effectively assuming that probin stores text, and we should just make it really truly do that. Sounds good to me, then. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Stephen Frost wrote: KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction I'm not really a huge fan of 'security_' as a prefix for these functions, but I don't have a better suggestion right now. If so, 'pgsec_' (PostGresql SECutiry) instead? The initial abstraction patch shouldn't include the security context pieces. I realize that will be needed eventually, but the patch to do the abstraction and to formally move permissions checking to aclchk.c needs to stand alone. I'm also not sure that the API of having the security context be returned as a Datum makes sense.. OK, I'll add pieces corresponding to the security context on the second patch (SE-PostgreSQL patch). Doesn't security_table_permissions() need to know if the query is an UPDATE or an INSERT? Either ACL_UPDATE or ACL_INSERT should be set on the required_perms. Both of them are never set in same time. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/4 Tom Lane t...@sss.pgh.pa.us: BTW, there's no rule saying you have to fix that strictly within to_char(). It might make more sense to have numeric.c export a function that is like numeric_out but produces e-style output. Your choice as the patch writer, but keep it in mind ... Ah, thanks for the suggestion. I think you're right, numeric.c would be a nice place for this code. Going directly from the guts of a NumericVar to a scientific notation seems more elegant than performing cut-and-paste style surgery on the string from numeric_out(). It does require a bit more effort and headspace than I was expecting. I'm still keen to do it, but my next day or so timeframe may not be so realistic. It's looking more like by the end of the week. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
On Mon, Aug 03, 2009 at 11:18:55PM -0400, Stephen Frost wrote: KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction I'm not really a huge fan of 'security_' as a prefix for these functions, but I don't have a better suggestion right now. Just generally, access control is a great way to describe what's actually happening here. That people conflate access control with security has resulted in a number of disasters :P Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
David Fetter wrote: On Mon, Aug 03, 2009 at 11:18:55PM -0400, Stephen Frost wrote: KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction I'm not really a huge fan of 'security_' as a prefix for these functions, but I don't have a better suggestion right now. Just generally, access control is a great way to describe what's actually happening here. That people conflate access control with security has resulted in a number of disasters :P My concern is access_control_ is a bit long for prefixes, but ac_ is too short to represent what it is doing. -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Tom Lane wrote:
[ thinks for awhile... ] Actually, it seems to me that the present
patch's definition of the function would be very hard to work with.
You would normally want to work with the events one at a time.
There isn't much you could do with the array result except unnest() it,
and I'm a bit worried that careless usage could result in multiple
evaluation of the function and hence loss of events. I wonder whether
it would be better to have the function return setof record. Which, not
incidentally, would greatly simplify adding in those extra result
fields.
OK, how's this look?
Joe
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iQIcBAEBCAAGBQJKd7diAAoJEDfy90M199hlhlUP/0nsiVPY7wCRdGGs7LsTmnQR
o4Sd9f7R4XlZdakZLKHPf61Qxe33/Af9OdosLToBjssdDW4rrER1rql8+MwddKld
+H/5VZkMRTA91BLbt8kgSZzBj3sGtGpi4zCTgYrFNfTpvCNWK/YLb5rlmbyoCbST
AlnIr/MvcFGNj/JAzQFcoA+YHjEinMnnOA/VS03hbbzBUj2F3Q2uIhsx+YxxZpEQ
jeW54YMOolpsnmQBGIY/NKbU379zWdKtscgKiDO+OLM5OkowaKPbeAZTUBx4+OGR
juOfHH7A5bLZ9APPO/N1yLNHPOLr49DrsYKdkY0Ho97NBEFZhZSqKZQgUemMB+5Z
PNjxFrC2y6HTRabeV+yKQOM/jL8ZmSiSMOwrsdmomAjLNSYi2r2o+XTTDQbNwMqQ
MqHHXRNslooJft2iNWp8iF1L/wX5URroTP+7aZbdTqUqNp/ITJu4BFZTjajZP2zQ
waAAEIVz740yVL3V8mOWyHnHgH1vQEIZ7zMqd4ss0Nn+V9Yltby2hG2Cy9/MRMxt
AkzmE2H+f794mIiyp/jydLiqqgxzbSVOv3m2cx76srSkKY7/C/wZGm2DqjbXqzb/
Dcfs3TgOYozUD02lcnrC4tncdexru/sr6iiAsprslQtzsFxWlHqYqqIh1LfwThba
SJnCszfPpDHx98RA/9b8
=7koq
-END PGP SIGNATURE-
Index: contrib/dblink/dblink.c
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v
retrieving revision 1.82
diff -c -r1.82 dblink.c
*** contrib/dblink/dblink.c 11 Jun 2009 14:48:50 - 1.82
--- contrib/dblink/dblink.c 4 Aug 2009 03:48:33 -
***
*** 1635,1640
--- 1635,1723
PG_RETURN_DATUM(current_query(fcinfo));
}
+ /*
+ * Retrieve async notifications for a connection.
+ *
+ * Returns an setof record of notifications, or an empty set if none recieved.
+ * Can optionally take a named connection as parameter, but uses the unnamed connection per default.
+ *
+ */
+ #define DBLINK_NOTIFY_COLS 3
+
+ PG_FUNCTION_INFO_V1(dblink_get_notify);
+ Datum
+ dblink_get_notify(PG_FUNCTION_ARGS)
+ {
+ PGconn *conn = NULL;
+ remoteConn *rconn = NULL;
+ PGnotify *notify;
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo-resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+
+ DBLINK_INIT;
+ if (PG_NARGS() == 1)
+ DBLINK_GET_NAMED_CONN;
+ else
+ conn = pconn-conn;
+
+ /* create the tuplestore */
+ per_query_ctx = rsinfo-econtext-ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ tupdesc = CreateTemplateTupleDesc(DBLINK_NOTIFY_COLS, false);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, notify_name,
+ TEXTOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, be_pid,
+ INT4OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, extra,
+ TEXTOID, -1, 0);
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo-returnMode = SFRM_Materialize;
+ rsinfo-setResult = tupstore;
+ rsinfo-setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ PQconsumeInput(conn);
+ while ((notify = PQnotifies(conn)) != NULL)
+ {
+ Datum values[DBLINK_NOTIFY_COLS];
+ bool nulls[DBLINK_NOTIFY_COLS];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ if (notify-relname != NULL)
+ values[0] = CStringGetTextDatum(notify-relname);
+ else
+ nulls[0] = true;
+
+ values[1] = Int32GetDatum(notify-be_pid);
+
+ if (notify-extra != NULL)
+ values[2] = CStringGetTextDatum(notify-extra);
+ else
+ nulls[2] = true;
+
+ /* switch to appropriate context while storing the tuple */
+ MemoryContextSwitchTo(per_query_ctx);
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ MemoryContextSwitchTo(oldcontext);
+
+ PQfreemem(notify);
+ PQconsumeInput(conn);
+ }
+
+ /* clean up and return the tuplestore */
+ tuplestore_donestoring(tupstore);
+
+ return (Datum) 0;
+ }
+
/*
* internal functions
*/
Index: contrib/dblink/dblink.h
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v
retrieving revision 1.22
diff -c -r1.22 dblink.h
*** contrib/dblink/dblink.h 9 Jun 2009 17:41:02 - 1.22
--- contrib/dblink/dblink.h 4 Aug 2009 02:35:59 -
***
*** 57,61
--- 57,62
extern Datum dblink_build_sql_delete(PG_FUNCTION_ARGS);
extern Datum dblink_build_sql_update(PG_FUNCTION_ARGS);
extern Datum dblink_current_query(PG_FUNCTION_ARGS);
+ extern Datum dblink_get_notify(PG_FUNCTION_ARGS);
#endif
Re: [HACKERS] Patch for 8.5, transformationHook
2009/8/4 Robert Haas robertmh...@gmail.com: On Thu, Jul 30, 2009 at 1:22 AM, Pavel Stehulepavel.steh...@gmail.com wrote: 2009/7/30 Robert Haas robertmh...@gmail.com: On Sun, Jul 26, 2009 at 9:29 AM, Pavel Stehulepavel.steh...@gmail.com wrote: Hello new patch add new contrib transformations with three modules anotation, decode and json. These modules are ported from my older work. Before applying this patch, please use named-fixed patch too. The hook doesn't need it, but modules anotation and json depend on it. These are pretty good examples, but the whole thing still feels a bit grotty to me. The set of syntax transformations that can be performed with a hook of this type is extremely limited - in particular, it's the set of things where the parser thinks it's valid and that the structure is reasonably similar to what you have in mind, but the meaning is somewhat different. The fact that two of your three examples require your named and mixed parameters patch seems to me to be evidence of that. I see the main hook using as open door to functionality like decode and json. Anotation is little bit corner use case. We don't need a change of syntax or rules in parser. But I need to get some info for functions from parser stage - like JSON or replace standard coercion rules like decode. Hook is the most simple and general technique for it (what I found). I thing, so there are other techniques - but it needs more invasive patch and are not too general - what is values of any hooking. I doesn't thing, so there will be any real extended parser based on bison in near or far future. I thing, so this is theoretically possible, but nobody work on it. What more - with extensible parser we still need the transformation hook, because we need the change in transformation - decode, json. The JSON transformation provides functionality which is very similar to what we also offer for XML. I sort of think we ought to just provide that, rather than making it an add-on. I have found it to be a tremendously attractive alternative to XML. The JSON is only one use case (there should be output to any format), and I agree, so this should be in core. But every integration similar function to core needs one or two years. Hook allows do this things faster and from external library. It's little bit lighter process to put your project to pgfoundry than to PostgreSQL core. I don't really believe that JSON is only one use case. XML and JSON are in a class of their own; there's nothing else out there that is really comparable. XML and JSON are well known formats. But everybody can use it for custom formats, for binary formats, for direct communication, for loging, ... Pavel So I guess I'm back to feeling like this is of pretty marginal benefit. But I would still like some opinions from others. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] question about the _SPI_save_plan() and plan cache
Tom Lane t...@sss.pgh.pa.us writes:
Tao Ma feng_e...@163.com writes:
I knew that the delete_function() will reclaim the memory context
allocated for the function. But I did not find any code for removing
the plan(SPI plan memory context), saved by calling _SPI_save_plan.
Hmmm ... good point, those probably won't get cleaned up. In an
environment where functions are getting changed constantly, that
might be worth doing.
regards, tom lane
Hi, I just paste a re-produce sql script. Is it possible to cache the
SPI plan under the function cache context?
Thanks.
begin 666 spi_plan_leak_eg.sql
M0U)%051%($Q!3D=504=%('!L=S6P[#0H-BTM(=E;F5R871E($@:'5G
M92!F=6YC=EO;@T*0U)%051%($]2(%)%4$q!...@1e5.0u1)3...@9g5n8u]g
M96YEF%T;W(H*2!215154DY3(%1%...@05,@)0-D1%0TQ!4D4-B @W1M
M=!415A4.PT*(!I($E.5#L-D)%1TE.#0H@('-T;7...@.ct@)T-214%412!/
M4B!215!,04-%($953D-424].(8H*2!215154DY3(%1%...@05,@)$$D1$5#
M3$%212 G.PT*(!3U(@:2!)3B Q+BXQ,# P($Q/3U -B @(!S=UT(#H]
M('-T;7...@?'P@)R!V87)?=5X=@?'P@:2!\? G(%1%...@1$5055,5!#
M55)214Y47U1)344[)SL-B @(!S=UT(#H]('-T;7...@?'P@)R!V87)?:6YT
M)R!\?!I('Q\(@24Y4($1%1D%53%0@,3 P.R[#0H@( @W1M= Z/2!S
M=UT('Q\(@8W5R7R@?'P@:2!\? G($-54E-/4B H#...@24y4+!P,B!4
M15A4*2!)4R!314Q%0U0@#(-B @( @( @( @( @( @( @( @( @
M( @( @( @( @1E)/32!D=6%L(%=(15)%(%)/5TY532 \(' Q.R[#0H@
M($5.1!,3T]0.PT*(!S=UT(#H]('-T;7...@?'P@)T)%1TE.(%)%5%523B!V
M87)?=5X=#$[($5.1#L@)$$D($Q!3D=504=%('!L=S6PG.PT*(!%6$5#
M551%('-T;70[#0H@(%)%5%523B G1E5.0U1)3...@1t5.15)!5$]2)SL-D5.
M1#L-B0D($Q!3D=504=%('!Lw%l.pt*#0i314q%...@9g5n8u]g96yeF%T
M;W(H*3L-E-%3$5#5!F*D[(TM(-O;G-U;65S(%B;W5T(#(P34(@;65M
M;W)y...@t*#0i$4d]0($953D-424].(8h...@+2t@4U!)('!L86X@;5A:PT*
M1%)/4!54Y#5$E/3B!F=6YC7V=E;F5R871Ob...@i.pt*#0h-E-%3$5#5!F
M=6YC7V=E;F5R871Ob...@i.pt*4t5,14-4(8h...@+2t@8V]NW5M97,@86YO
1=AEB R,$U(UE;6]R2X`
`
end
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
2009/8/4 Tom Lane t...@sss.pgh.pa.us: pg_proc.probin is currently declared as being type bytea. Perhaps that had some real utility once upon a time, but no currently defined procedural language stores binary data there. In fact, the only thing that gets stored there is the shared-library file name for C functions. To make things even more fun, none of the code touching probin is doing anything to honor the special escaping conventions for bytea. This was pretty broken already, and might perhaps explain some of the weirder error reports we've gotten. It is now obvious to me that no shlib file name containing non-ASCII characters will correctly survive a dump/reload, in any existing PG release. Backslashes accidentally fail to fail, at least for some values of standard_conforming_strings. However, with the hex bytea output patch in place, it's *completely* broken. I offer the following pg_dump output: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '\\x2f686f6d652f706f7374677265732f706773716c2f7372632f746573742f726567726573732f726567726573732e736c', 'interpt_pp'; which should of course have looked like this: CREATE FUNCTION interpt_pp(path, path) RETURNS point LANGUAGE c AS '/home/postgres/testversion/src/test/regress/regress.sl', 'interpt_pp'; I think that the least painful solution might be to change pg_proc.probin to be declared as text. Otherwise we're going to need version-specific klugery in pg_dump and who knows where else. Comments? -1 correct solution is moving the path to prosrc col or create new colum externalproc. I thing so probin should be useful for plpgsql - sometime we should to store parser result from plpgsql compilation stage in this column. So my option is don't change native sense of this column. If we actually have not using, the we should to drop it, not create some breaks in future. Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
Joe Conway escribió: OK, how's this look? Hmm, is it possible to use OUT parameters in the function instead of declaring a new type for the result? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
Pavel Stehule pavel.steh...@gmail.com writes: 2009/8/4 Tom Lane t...@sss.pgh.pa.us: I think that the least painful solution might be to change pg_proc.probin to be declared as text. correct solution is moving the path to prosrc col or create new colum externalproc. I thing so probin should be useful for plpgsql - sometime we should to store parser result from plpgsql compilation stage in this column. So my option is don't change native sense of this column. If we actually have not using, the we should to drop it, not create some breaks in future. [ shrug... ] Can't get excited about it. What you propose would be a significant amount of work and added complexity, because pg_dump or somebody would have to take care to translate existing function definitions properly. And the benefit is purely speculative. I seriously doubt that serializing plpgsql compilation trees into a bytea representation would be worth anyone's time. The original Berkeley authors probably had some idea of storing compiled machine code in probin, but nobody's done that either, and I'll bet long odds that nobody ever will. (The advantage compared to external C functions implemented as we currently know them seems negligible. The Berkeley folk probably did not foresee the prevalence of loadable-shared-library support, nor the general security-driven move away from allowing execution of writable memory.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
2009/8/4 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2009/8/4 Tom Lane t...@sss.pgh.pa.us: I think that the least painful solution might be to change pg_proc.probin to be declared as text. correct solution is moving the path to prosrc col or create new colum externalproc. I thing so probin should be useful for plpgsql - sometime we should to store parser result from plpgsql compilation stage in this column. So my option is don't change native sense of this column. If we actually have not using, the we should to drop it, not create some breaks in future. [ shrug... ] Can't get excited about it. What you propose would be a significant amount of work and added complexity, because pg_dump or somebody would have to take care to translate existing function definitions properly. And the benefit is purely speculative. I seriously doubt that serializing plpgsql compilation trees into a bytea representation would be worth anyone's time. The original Berkeley authors probably had some idea of storing compiled machine code in probin, but nobody's done that either, and I'll bet long odds that nobody ever will. (The advantage compared to external C functions implemented as we currently know them seems negligible. The Berkeley folk probably did not foresee the prevalence of loadable-shared-library support, nor the general security-driven move away from allowing execution of writable memory.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
2009/8/4 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2009/8/4 Tom Lane t...@sss.pgh.pa.us: I think that the least painful solution might be to change pg_proc.probin to be declared as text. correct solution is moving the path to prosrc col or create new colum externalproc. I thing so probin should be useful for plpgsql - sometime we should to store parser result from plpgsql compilation stage in this column. So my option is don't change native sense of this column. If we actually have not using, the we should to drop it, not create some breaks in future. [ shrug... ] Can't get excited about it. What you propose would be a significant amount of work and added complexity, because pg_dump or somebody would have to take care to translate existing function definitions properly. And the benefit is purely speculative. I seriously doubt that serializing plpgsql compilation trees into a bytea representation would be worth anyone's time. The original Berkeley authors probably had some idea of storing compiled machine code in probin, but nobody's done that either, and I'll bet long odds that nobody ever will. (The advantage compared to external C functions implemented as we currently know them seems negligible. The Berkeley folk probably did not foresee the prevalence of loadable-shared-library support, nor the general security-driven move away from allowing execution of writable memory.) I agree, so information about patch would be store in text field. But I am not sure, if your fix isn't too simply. I haven't plan to compile plpgsql to C or to binary code. But could be interesting link postgres with some virtual machine like parrot or lua vm, and translate plpgsql to p code. It's maybe far future. Early future is integration main SQL parser to plpgsql. I am not sure, but maybe we will need some persistent cache for store parametrized sql queries. I though about store it in probin column. Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... ALTER COLUMN ... SET DISTINCT
Are there plans to make a small follow-up patch to make CREATE UNIQUE INDEX on one column (and variants in CREATE TABLE ... PRIMARY KEY) automatically do SET STATISTICS DISTINCT? It might not be as perfect a solution as teaching the planner to know about unique indexes, but it is better than nothing. Good work! Regards, Kim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
--On Sonntag, August 02, 2009 11:38:22 -0400 Robert Haas robertmh...@gmail.com wrote: What's the current status of this patch? Is someone (either Pavel or Bernd) planning to update it further, or should it be marked Ready for Committer? I will incorporate some additional docs adjustments per Pavels suggestion and will then mark it ready for committer review. Please note that there are actually two patches involved here: one is Pavels' patch the other is Steve Prentice' enhancement to allow named notation in plpgsql. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Stephen Frost wrote: I think what I should do on the next is ... - To check up whether it is really possible to implement SELinux's model. - To describe the list of the security functions in the new abstraction layer. - To discuss the list of permission at: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development#Mandatory_access_controls That sounds like a good approach. As we define the security functions to go into the abstraction layer, I would also say we should identify the exact pieces of existing code which are going to move. I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction In my current impression, it indeed requires a few kilo lines of changes, but it is not impossible scale. I now plans to submit two patches for the next commit fest. The one is implementation of the abstraction layer. The other is basic implementation of the SE-PostgreSQL. So, I would like to fix external specification at least. The specifications for developer notes definitions of permissions: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development As Robert suggested before, I plans to support access controls on the following database objects and permissions at the first stage. * databases * schemas * tables * columns * sequences * functions * tablespaces Do you have any comment for the directions? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest Status Summary - 2009-08-03
Personally I was thinking of multi-threaded pgbench as being Tatsuo-san's to commit, since that's his code originally. Ah see well that's why I post these summaries... :-) Thanks. I consider it a great honor to be allowed to commit the pacthes. -- Tatsuo Ishii SRA OSS, Inc. Japan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
2009/8/3 Bernd Helmle maili...@oopsware.de: --On Sonntag, August 02, 2009 11:38:22 -0400 Robert Haas robertmh...@gmail.com wrote: What's the current status of this patch? Is someone (either Pavel or Bernd) planning to update it further, or should it be marked Ready for Committer? I will incorporate some additional docs adjustments per Pavels suggestion and will then mark it ready for committer review. Please note that there are actually two patches involved here: one is Pavels' patch the other is Steve Prentice' enhancement to allow named notation in plpgsql. I should to wait with Steve patch - I would to add main sql parser into plpgsql - than Steve's patch is unnecessary. But if there will be some problems, then we can use Steve's patch. It is simple - so there are not big problems with commit. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 win32 shared memory patch
On Sat, Aug 1, 2009 at 20:30, Kevin Fieldkevinjamesfi...@gmail.com wrote: The event viewer says: The description for Event ID ( 0 ) in Source ( PostgreSQL ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: pg_ctl: could not find postgres program executable And yes, I renamed it correctly... Check permissions on it. If you moved it at some point, it may have the wrong permissions. They should be the same as for the other .EXEs in that directory. The two files (new and old exe) have identical permissions. That's just weird. It could be that the postgres executable won't work - maybe because of some DLL issue. Can you run postgres -V on the executable, or does that give you some error? -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
KaiGai -- I was pulled away from any work on this for a few days ... what can I do to help, if anything ? (Keeping in mind that my knowledge of the internals of postgres is, alas, minimal.) ... I had a quick look at this new page and want to take another, more careful, look. The sheer scope of this proposal undoubtedly gives pause to many, so I'd urge a certain minimalism at first to prove that the concept works and doesn't damage the core functionality at all when it is not used (fairly straight forward). Eventually rough measures of the performance hit when it is used will be useful, but users will expect something of a slow-down, I suspect, in exchange foe the greater security. To that end, I am wondering if there is test data yet designed ? Are there prescribed tests (I remember seeing some but they seem to be buried in months/threads that I have not yet re-dicsovered) ? I have some experience doing QA and could perhaps also help in these, a little. And let me add, I am in awe of your efforts on this ! G - Original Message From: KaiGai Kohei kai...@ak.jp.nec.com To: Stephen Frost sfr...@snowman.net Cc: KaiGai Kohei kai...@kaigai.gr.jp; Robert Haas robertmh...@gmail.com; pgsql-hackers@postgresql.org; Greg Williamson gwilliamso...@yahoo.com; Sam Mason s...@samason.me.uk; Joshua Brindle met...@manicmethod.com Sent: Monday, August 3, 2009 12:09:45 AM Subject: Re: [HACKERS] SE-PostgreSQL Specifications Stephen Frost wrote: I think what I should do on the next is ... - To check up whether it is really possible to implement SELinux's model. - To describe the list of the security functions in the new abstraction layer. - To discuss the list of permission at: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development#Mandatory_access_controls That sounds like a good approach. As we define the security functions to go into the abstraction layer, I would also say we should identify the exact pieces of existing code which are going to move. I began to describe the list of abstraction layer functions (but not completed yet): http://wiki.postgresql.org/wiki/SEPostgreSQL_Abstraction In my current impression, it indeed requires a few kilo lines of changes, but it is not impossible scale. I now plans to submit two patches for the next commit fest. The one is implementation of the abstraction layer. The other is basic implementation of the SE-PostgreSQL. So, I would like to fix external specification at least. The specifications for developer notes definitions of permissions: http://wiki.postgresql.org/wiki/SEPostgreSQL_Development As Robert suggested before, I plans to support access controls on the following database objects and permissions at the first stage. * databases * schemas * tables * columns * sequences * functions * tablespaces Do you have any comment for the directions? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Alpha releases: How to tag
As we are approaching the first alpha release, let's think about how to tag and label it and how to schedule those two operations with respect to one another. The typical process for, say, a beta release is - apply version number change to source tree - commit - tag (repeat for next beta) The problem, arguably, with this is that the source tree would then read 8.5alpha1 until alpha2, instead of the 8.5devel that we are used to. The fix could be: - apply version number change to source tree - commit - tag - revert version number change in source tree - commit (repeat for next alpha) or more sophisticatedly - branch - apply version number change to source tree - commit - tag or alternatively no tag at all, just apply version number and build tarball. Comments? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
* Peter Eisentraut (pete...@gmx.net) wrote: - branch - apply version number change to source tree - commit - tag If this wasn't CVS, this would certainly be the way to go. or alternatively no tag at all, just apply version number and build tarball. As this *is* CVS, I'm thinking we should probably go with this approach. Just my 2c. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Alpha releases: How to tag
Peter Eisentraut wrote: As we are approaching the first alpha release, let's think about how to tag and label it and how to schedule those two operations with respect to one another. The typical process for, say, a beta release is - apply version number change to source tree - commit - tag (repeat for next beta) The problem, arguably, with this is that the source tree would then read 8.5alpha1 until alpha2, instead of the 8.5devel that we are used to. The fix could be: - apply version number change to source tree - commit - tag - revert version number change in source tree - commit (repeat for next alpha) or more sophisticatedly - branch - apply version number change to source tree - commit - tag or alternatively no tag at all, just apply version number and build tarball. Comments? Does it need a version number change? Maybe just a tag (no branch) is all that is required. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Aug 3, 2009, at 7:57 AM, Stephen Frost sfr...@snowman.net wrote: * Peter Eisentraut (pete...@gmx.net) wrote: - branch - apply version number change to source tree - commit - tag If this wasn't CVS, this would certainly be the way to go. or alternatively no tag at all, just apply version number and build tarball. As this *is* CVS, I'm thinking we should probably go with this approach. Sounds right to me. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... ALTER COLUMN ... SET DISTINCT
Kim Bisgaard kim...@alleroedderne.adsl.dk writes: Are there plans to make a small follow-up patch to make CREATE UNIQUE INDEX on one column (and variants in CREATE TABLE ... PRIMARY KEY) automatically do SET STATISTICS DISTINCT? No. It would be pointless for a single column and wrong for multiple columns. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] CVS Head parser error?
I got following with CVS Head parser. I'm using bison 2.1 and flex 2.5.35. Am I missing something? make[3]: Entering directory `/usr/local/src/pgsql/current/pgsql/src/backend/parser' gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -I. -I../../../src/include -D_GNU_SOURCE -c -o gram.o gram.c In file included from gram.y:11211: scan.c:14473: error: conflicting types for `base_yylex' ../../../src/include/parser/gramparse.h:122: error: previous declaration of `base_yylex' scan.l: In function `base_yylex': scan.l:354: error: `yylloc' undeclared (first use in this function) scan.l:354: error: (Each undeclared identifier is reported only once scan.l:354: error: for each function it appears in.) scan.l:386: warning: implicit declaration of function `yyerror' scan.l:404: error: `yylval' undeclared (first use in this function) scan.l:448: error: too few arguments to function `ScanKeywordLookup' scan.l:474: error: too few arguments to function `scanner_errposition' scan.l:522: error: too few arguments to function `scanner_errposition' scan.l:823: error: too few arguments to function `ScanKeywordLookup' scan.l: At top level: scan.l:864: error: conflicting types for `scanner_errposition' ../../../src/include/parser/gramparse.h:123: error: previous declaration of `scanner_errposition' scan.l:889: warning: no previous prototype for `yyerror' scan.l:889: warning: type mismatch with previous implicit declaration scan.l:748: warning: previous implicit declaration of `yyerror' scan.l:889: warning: `yyerror' was previously implicitly declared to return `int' scan.l: In function `yyerror': scan.l:890: error: `yylloc' undeclared (first use in this function) scan.l: At top level: scan.l:916: error: conflicting types for `scanner_init' ../../../src/include/parser/gramparse.h:119: error: previous declaration of `scanner_init' scan.l:947: error: conflicting types for `scanner_finish' ../../../src/include/parser/gramparse.h:120: error: previous declaration of `scanner_finish' scan.l: In function `check_unicode_value': scan.l:1024: error: `yylloc' undeclared (first use in this function) scan.l: In function `litbuf_udeescape': scan.l:1041: error: `yylloc' undeclared (first use in this function) scan.l: In function `check_string_escape_warning': scan.l:1132: error: `yylloc' undeclared (first use in this function) scan.l: In function `check_escape_warning': scan.l:1157: error: `yylloc' undeclared (first use in this function) make[3]: *** [gram.o] Error 1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
Andrew Dunstan and...@dunslane.net writes: Does it need a version number change? Maybe just a tag (no branch) is all that is required. I think that we do want the alpha releases to identify themselves as such. And we want a marker in CVS as to what state the alpha release corresponds to. Peter's label-and-undo approach seems like a kluge; and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. I feel that making a branch is the way to go. If we try to get away with a shortcut, we'll probably regret it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CVS Head parser error?
Sorry for noise. I regenerated scan.c and gram.c and now everything works fine. -- Tatsuo Ishii SRA OSS, Inc. Japan I got following with CVS Head parser. I'm using bison 2.1 and flex 2.5.35. Am I missing something? make[3]: Entering directory `/usr/local/src/pgsql/current/pgsql/src/backend/parser' gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -I. -I../../../src/include -D_GNU_SOURCE -c -o gram.o gram.c In file included from gram.y:11211: scan.c:14473: error: conflicting types for `base_yylex' ../../../src/include/parser/gramparse.h:122: error: previous declaration of `base_yylex' scan.l: In function `base_yylex': scan.l:354: error: `yylloc' undeclared (first use in this function) scan.l:354: error: (Each undeclared identifier is reported only once scan.l:354: error: for each function it appears in.) scan.l:386: warning: implicit declaration of function `yyerror' scan.l:404: error: `yylval' undeclared (first use in this function) scan.l:448: error: too few arguments to function `ScanKeywordLookup' scan.l:474: error: too few arguments to function `scanner_errposition' scan.l:522: error: too few arguments to function `scanner_errposition' scan.l:823: error: too few arguments to function `ScanKeywordLookup' scan.l: At top level: scan.l:864: error: conflicting types for `scanner_errposition' ../../../src/include/parser/gramparse.h:123: error: previous declaration of `scanner_errposition' scan.l:889: warning: no previous prototype for `yyerror' scan.l:889: warning: type mismatch with previous implicit declaration scan.l:748: warning: previous implicit declaration of `yyerror' scan.l:889: warning: `yyerror' was previously implicitly declared to return `int' scan.l: In function `yyerror': scan.l:890: error: `yylloc' undeclared (first use in this function) scan.l: At top level: scan.l:916: error: conflicting types for `scanner_init' ../../../src/include/parser/gramparse.h:119: error: previous declaration of `scanner_init' scan.l:947: error: conflicting types for `scanner_finish' ../../../src/include/parser/gramparse.h:120: error: previous declaration of `scanner_finish' scan.l: In function `check_unicode_value': scan.l:1024: error: `yylloc' undeclared (first use in this function) scan.l: In function `litbuf_udeescape': scan.l:1041: error: `yylloc' undeclared (first use in this function) scan.l: In function `check_string_escape_warning': scan.l:1132: error: `yylloc' undeclared (first use in this function) scan.l: In function `check_escape_warning': scan.l:1157: error: `yylloc' undeclared (first use in this function) make[3]: *** [gram.o] Error 1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
On Aug 3, 2009, at 1:41 AM, Pavel Stehule wrote: I should to wait with Steve patch - I would to add main sql parser into plpgsql - than Steve's patch is unnecessary. But if there will be some problems, then we can use Steve's patch. It is simple - so there are not big problems with commit. I was hoping we could get the small patch into plpgsql during this commitfest. This makes plpgsql recognize 'AS' and not replace named parameter labels with the variable reference. I understand there is an effort underway to redo the plpgsql parser, but getting these two patches in together will allow people to start playing with plpgsql + named parameters at the end the of commitfest when the first alpha is released. (You can use named parameters + plpgsql without this patch, but not without some pretty serious limitations.) Without this patch, this will fail: create function create_user(alias text, display_name text) returns void as $$ BEGIN perform create_alias(alias AS alias); ... END $$ language plpgsql; This is a common pattern for many of the stored procedures we are porting and I'd imagine it's common elsewhere too. If the plpgsql parser patch lands, this patch won't be needed, but it's hard to predict when it will land. As an aside, this pattern really shows how confusing the AS syntax can be for named parameters. Which side is the label and which is the value? Thanks, -Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
2009/8/3 Steve Prentice prent...@cisco.com: On Aug 3, 2009, at 1:41 AM, Pavel Stehule wrote: I should to wait with Steve patch - I would to add main sql parser into plpgsql - than Steve's patch is unnecessary. But if there will be some problems, then we can use Steve's patch. It is simple - so there are not big problems with commit. I was hoping we could get the small patch into plpgsql during this commitfest. This makes plpgsql recognize 'AS' and not replace named parameter labels with the variable reference. I understand there is an effort underway to redo the plpgsql parser, but getting these two patches in together will allow people to start playing with plpgsql + named parameters at the end the of commitfest when the first alpha is released. (You can use named parameters + plpgsql without this patch, but not without some pretty serious limitations.) Without this patch, this will fail: create function create_user(alias text, display_name text) returns void as $$ BEGIN perform create_alias(alias AS alias); ... END $$ language plpgsql; This is a common pattern for many of the stored procedures we are porting and I'd imagine it's common elsewhere too. If the plpgsql parser patch lands, this patch won't be needed, but it's hard to predict when it will land. As an aside, this pattern really shows how confusing the AS syntax can be for named parameters. Which side is the label and which is the value? ok - I don't though about it. My goal is integration main parser next commitfest, but it is true, so somebody would to play with named params now. Commiting of Steve's patch doesn't break anything. Pavel Thanks, -Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CVS Head parser error?
Tatsuo Ishii is...@postgresql.org writes: I got following with CVS Head parser. I'm using bison 2.1 and flex 2.5.35. Am I missing something? Hmm, are you sure that scan.c got remade? I didn't check in detail, but the errors look a bit like what would happen if the older scanner code got recompiled against current headers. I'd try make maintainer-clean and start over before investing additional thought. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Does it need a version number change? Maybe just a tag (no branch) is all that is required. I think that we do want the alpha releases to identify themselves as such. And we want a marker in CVS as to what state the alpha release corresponds to. Peter's label-and-undo approach seems like a kluge; and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. I feel that making a branch is the way to go. If we try to get away with a shortcut, we'll probably regret it. Yes, if that's what we want then definitely branch. I guess the branch will (almost) only ever have exactly one commit on it. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
I wrote: Tom Lane t...@sss.pgh.pa.us wrote: Attached is a further small improvement that gets rid of the find_ready_items() scans. After re-reading the patch I realized that it wasn't *really* avoiding O(N^2) behavior ... but this version does. I'll run a fresh set of benchmarks. Over the weekend I ran 40 restores of Milwaukee County's production data using Friday's snapshot with and without the patch. I alternated between patched and unpatched. It appears that this latest version is slightly slower for our production database on the same machine and configuration where the previous patch appeared to be 1% to 2% faster than unpatched (although I had fewer samples of that). Although the actual runs were interleaved, I've separated them for this email, because it seems easier to look them over this way: 8.5devel pg_restore -j2 real77m13.737s real78m21.701s real78m21.536s real77m37.541s real79m10.068s real81m37.111s real77m52.252s real80m49.110s real76m50.093s real78m0.701s real77m30.674s real77m22.875s real80m43.914s real78m51.525s real80m46.329s real76m56.703s real78m44.960s real82m37.757s real84m12.938s real82m27.591s 8.5devel-alt pg_restore -j2 real78m10.846s real78m5.584s real78m13.673s real79m43.939s real84m39.593s real80m25.240s real78m10.045s real82m34.320s real78m43.528s real77m12.211s real77m39.171s real79m58.421s real84m50.816s real85m56.248s real79m17.361s real79m0.778s real77m3.525s real77m22.750s real78m22.882s real78m51.617s That's about 0.52% slower with the patch. Because there was over 10% variation in the numbers with the patch, I tried leaving out the four highest outliers on both, in case it was the result of some other activity on the system (even though this machine should have been pretty quiet over the weekend) and the difference fell to 0.09%. I don't know if this difference is enough to worry about; it might depend on whether we're comparing to the unpatched version or the previous patch. If it comes to choosing between a 1% to 2% performance gain for a normal case versus elimination of O(N^2) behavior for a worst-case scenario, I'm not sure which should win -- especially in the absence of benchmarks showing the pessimal case. What would be the most productive focus for benchmarks at this point? The artificial pessimal case? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: ... it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. Yes, if that's what we want then definitely branch. I guess the branch will (almost) only ever have exactly one commit on it. Yeah, I'd expect so; but it seems like a good idea to leave room for the possibility of more than one commit. I was first thinking that we'd just release a new alpha from CVS HEAD if the evil-bug situation comes up. However, if we've already done a catversion bump or some other incompatible change in HEAD, that wouldn't be very practical. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
That's about 0.52% slower with the patch. Because there was over 10% variation in the numbers with the patch, I tried leaving out the four highest outliers on both, in case it was the result of some other activity on the system (even though this machine should have been pretty quiet over the weekend) and the difference fell to 0.09%. I don't know if this difference is enough to worry about; it might depend on whether we're comparing to the unpatched version or the previous patch. If it comes to choosing between a 1% to 2% performance gain for a normal case versus elimination of O(N^2) behavior for a worst-case scenario, I'm not sure which should win -- especially in the absence of benchmarks showing the pessimal case. What would be the most productive focus for benchmarks at this point? The artificial pessimal case? My instinct says that the variation is probably just noise, of no great significance. I'm personally not terribly worried about the O(n^2) case, but I think the patch is probably useful anyway, as a basis for other people to try to improve the item selection algorithm further. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Monday 03 August 2009 17:44:32 Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Does it need a version number change? Maybe just a tag (no branch) is all that is required. I think that we do want the alpha releases to identify themselves as such. And we want a marker in CVS as to what state the alpha release corresponds to. Peter's label-and-undo approach seems like a kluge; and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. I feel that making a branch is the way to go. If we try to get away with a shortcut, we'll probably regret it. Another more lightweight alternative is to tag and then change the version number and build the tarball, without another commit. This assumes that the version number stamping is automated and reproducible, so you don't have to record it in history. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 03, 2009 at 10:44:32AM -0400, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Does it need a version number change? Maybe just a tag (no branch) is all that is required. I think that we do want the alpha releases to identify themselves as such. And we want a marker in CVS as to what state the alpha release corresponds to. Peter's label-and-undo approach seems like a kluge; Right. and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. Is this a use case? I truly hope nobody will try using a beta, let alone an alpha, in production. Do we need to provide for such a possibility? I don't recall that we've ever back-patched a beta, or even a release candidate. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
Kevin Grittner kevin.gritt...@wicourts.gov writes: Over the weekend I ran 40 restores of Milwaukee County's production data using Friday's snapshot with and without the patch. I alternated between patched and unpatched. It appears that this latest version is slightly slower for our production database on the same machine and configuration where the previous patch appeared to be 1% to 2% faster than unpatched (although I had fewer samples of that). I think we can conclude that for this particular test case, the effects of the patch are pretty much masked by noise. I definitely see no way that the latest version of the patch could really be slower than the original; it has the same job-scheduling behavior and strictly less list-munging overhead. Now the patch could be slower than unpatched as a result of different job-scheduling behavior ... but there's no evidence here of a consistently measurable benefit or loss from that. IIRC daveg was volunteering to do some tests with his own data; maybe we should wait for those results. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest Status Summary - 2009-08-03
Personally I was thinking of multi-threaded pgbench as being Tatsuo-san's to commit, since that's his code originally. Ah see well that's why I post these summaries... :-) Thanks. I consider it a great honor to be allowed to commit the pacthes. Patch committed. -- Tatsuo Ishii SRA OSS, Inc. Japan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 10:44:32AM -0400, Tom Lane wrote: and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. Is this a use case? I truly hope nobody will try using a beta, let alone an alpha, in production. Do we need to provide for such a possibility? I don't recall that we've ever back-patched a beta, or even a release candidate. I don't really know if it's a use-case or not; I just have a feeling that if we use a release procedure that guarantees we can't do it, we'll live to regret that. The bug-fixing situation for betas and RCs is a bit different because it's expected that there will be a compatible update available shortly. So you can usually assume that updating to the next beta/RC/release will fix whatever problems got found. Alphas are going to be out there on their own with absolutely no expectation that the next alpha is catversion-compatible. And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. I agree with you that it'd be insane to run anything mission-critical on an alpha build; but I could see someone having loaded up quite a lot of test data on one. (If we aren't hoping for some pretty serious testing of these releases, I am not sure what is the point of doing them at all.) So it seems to me that having the ability to fix show-stopper bugs without forcing a migration to a later alpha would be a good thing. Maybe we'll never need it, or maybe we will. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 17:44:32 Tom Lane wrote: I feel that making a branch is the way to go. If we try to get away with a shortcut, we'll probably regret it. Another more lightweight alternative is to tag and then change the version number and build the tarball, without another commit. This assumes that the version number stamping is automated and reproducible, so you don't have to record it in history. The stamping is reproducible enough, I think, but don't we have the tarball build process set up so that it pulls the tarball directly from a particular CVS tag? Anyway, the reason I'm for a branch is not that. It's so we have the option to issue an 8.5alpha1.1 update if we need to. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest Status Summary - 2009-08-03
On Mon, Aug 3, 2009 at 11:22 AM, Tatsuo Ishiiis...@postgresql.org wrote: Personally I was thinking of multi-threaded pgbench as being Tatsuo-san's to commit, since that's his code originally. Ah see well that's why I post these summaries... :-) Thanks. I consider it a great honor to be allowed to commit the pacthes. Patch committed. Can you go here and change the status to Committed? https://commitfest.postgresql.org/action/patch_view?id=123 Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 03, 2009 at 11:32:48AM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 10:44:32AM -0400, Tom Lane wrote: and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. Is this a use case? I truly hope nobody will try using a beta, let alone an alpha, in production. Do we need to provide for such a possibility? I don't recall that we've ever back-patched a beta, or even a release candidate. I don't really know if it's a use-case or not; I just have a feeling that if we use a release procedure that guarantees we can't do it, we'll live to regret that. I can see being cautious. I'm just questioning the value of this precaution in particular. The bug-fixing situation for betas and RCs is a bit different because it's expected that there will be a compatible update available shortly. So you can usually assume that updating to the next beta/RC/release will fix whatever problems got found. Alphas are going to be out there on their own with absolutely no expectation that the next alpha is catversion-compatible. We've bumped catversion in beta and even RC, if I recall right. And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. That's an interesting idea. Shouldn't pg_migrator be mandated to work for *any* catversion bump? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 11:32:48AM -0400, Tom Lane wrote: And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. That's an interesting idea. Shouldn't pg_migrator be mandated to work for *any* catversion bump? Oh, are you volunteering to make that happen? That code is going to be ugly enough just dealing with pairs of major releases ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 3, 2009 at 17:32, Tom Lanet...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 10:44:32AM -0400, Tom Lane wrote: and it doesn't scale to consider the possibility that we might want to re-release an alpha after fixing some particularly evil bug. A tag without a branch won't handle that either. Is this a use case? I truly hope nobody will try using a beta, let alone an alpha, in production. Do we need to provide for such a possibility? I don't recall that we've ever back-patched a beta, or even a release candidate. I don't really know if it's a use-case or not; I just have a feeling that if we use a release procedure that guarantees we can't do it, we'll live to regret that. Agreed. The bug-fixing situation for betas and RCs is a bit different because it's expected that there will be a compatible update available shortly. So you can usually assume that updating to the next beta/RC/release will fix whatever problems got found. Alphas are going to be out there on their own with absolutely no expectation that the next alpha is catversion-compatible. And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. I agree with you that it'd be insane to run anything mission-critical on an alpha build; but I could see someone having loaded up quite a lot of test data on one. (If we aren't hoping for some pretty serious testing of these releases, I am not sure what is the point of doing them at all.) So it seems to me that having the ability to fix show-stopper bugs without forcing a migration to a later alpha would be a good thing. Maybe we'll never need it, or maybe we will. I think the alpha-alpha migration would actually be very useful to these testers. That'll get them onto the new alpha. I think that's actually a lot more important than alpha-release or release-next alpha. I haven't actually looked into pg_migrator enough to know how likely it is that it'll just work going alpha-alpha when there have only been normal changes? How invasive are the changes that actually require pg_migrator to be touched at all? -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Split-up ECPG patches
Boszormenyi Zoltan z...@cybertec.at writes: new versions attached, updated to apply to the current CVS cleanly. Why is this messing with the core grammar? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
On Mon, Aug 3, 2009 at 10:51 AM, Pavel Stehulepavel.steh...@gmail.com wrote: 2009/8/3 Steve Prentice prent...@cisco.com: On Aug 3, 2009, at 1:41 AM, Pavel Stehule wrote: I should to wait with Steve patch - I would to add main sql parser into plpgsql - than Steve's patch is unnecessary. But if there will be some problems, then we can use Steve's patch. It is simple - so there are not big problems with commit. I was hoping we could get the small patch into plpgsql during this commitfest. This makes plpgsql recognize 'AS' and not replace named parameter labels with the variable reference. I understand there is an effort underway to redo the plpgsql parser, but getting these two patches in together will allow people to start playing with plpgsql + named parameters at the end the of commitfest when the first alpha is released. (You can use named parameters + plpgsql without this patch, but not without some pretty serious limitations.) Without this patch, this will fail: create function create_user(alias text, display_name text) returns void as $$ BEGIN perform create_alias(alias AS alias); ... END $$ language plpgsql; This is a common pattern for many of the stored procedures we are porting and I'd imagine it's common elsewhere too. If the plpgsql parser patch lands, this patch won't be needed, but it's hard to predict when it will land. As an aside, this pattern really shows how confusing the AS syntax can be for named parameters. Which side is the label and which is the value? ok - I don't though about it. My goal is integration main parser next commitfest, but it is true, so somebody would to play with named params now. Commiting of Steve's patch doesn't break anything. I'm a little confused here. We are 19 days into a 31 day CommitFest; you are almost three weeks too late to add a patch to the queue. Unless you can convince a friendly committer to pick this up out of sequence, I think the only patch that is under consideration here is the one that has been being discussed on this thread for the last 17 days. I sent several notes adding for all patches to be added to commitfest.postgresql.org prior to the start of CommitFest; AFAIK, this one was never added. You haven't actually specified which other patch of Steve's you're talking about anyway, but I'm guessing it's this one here: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00801.php I don't think that patch would get applied even if HAD been added to the CommitFest page in time, see Tom's remarks here: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00802.php Let's get back to focusing on the patch that is actually under consideration here. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
Magnus Hagander mag...@hagander.net writes: I haven't actually looked into pg_migrator enough to know how likely it is that it'll just work going alpha-alpha when there have only been normal changes? How invasive are the changes that actually require pg_migrator to be touched at all? To my mind there are three categories of stuff that pg_migrator has to do: * catalog changes (handled by pg_dump/reload) * on-disk data layout changes (not handled yet anyway) * random weird unclassifiable stuff It's the third category that is the problem. An example from the 8.3 to 8.4 migration is the need to specially handle sequences because they're not compatible on-disk. Any pair of releases you might pick is going to have its own little quirks like that. Our intention (at least as I see it) is to minimize pg_migrator's complexity by decreeing that any given release of pg_migrator deals with exactly one pair of PG releases. I don't think that works if alpha-alpha has to be supported --- in particular, 8.5alpha1 to 8.5alpha2 might be quite different from what will be needed later for 8.4 to 8.5, so you would end up with multiple live branches of pg_migrator, or else a lot of spaghetti code trying to track which actions to take for a given case. The other little problem is who's going to do this work and when. The alpha-release idea was sold to us on the basis of being a small amount of incremental work: just tag an alpha release right after CommitFest and kick it out there. If we're waiting around for someone to produce and test a compatible pg_migrator, that's not going to be how it works. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Split-up ECPG patches
Tom Lane írta: Boszormenyi Zoltan z...@cybertec.at writes: new versions attached, updated to apply to the current CVS cleanly. Why is this messing with the core grammar? regards, tom lane It was explained in earlier mails, let me explain it again but a bit more verbosely. This was the evolution of my approaches adding the dynamic cursorname: At first, I changed ECPG grammar only. This meant adding one new rule for every rule dealing with FETCH or MOVE in the ECPG grammar. It turned out quickly, that these two rules below: FETCH fetch_direction from_in cursor_name ecpg_into MOVE fetch_direction from_in cursor_name created shift/reduce conflicts in fetch_direction. This conflict could have been solved by decreasing factorization of fetch_direction, pulling BACKWARD and FORWARD (without the row number indicator) up to the rules using fetch_direction. Which could be solved by two ways. 1. Leave the original (auto-generated) rules alone and add the new ones together with a new fetch_direction2 rule used by the dynamic cursorname FETCH/MOVE rules. At this point the FETCH/MOVE ruleset was so proliferated that my eyes started aching just by looking at it. And as Michael Meskes said, it was a great step back in the ECPG auto-generated grammar introduced in 8.4. 2. With the first approach being a dead-end, I tried to unify the two ruleset (FETCH/MOVE with and without dynamic cursorname variable). But it only worked if I add these changes: - add the cursor_name rule to the main grammar, so it can be picked up by the auto-generated ECPG grammar - cursor_name has a new subrule dealing with host variables in ECPG - the above solution for solving the shift/reduce problems in *ECPG*, needed the change in the main grammar, so auto-generation still works, and both the main grammar and the ECPG grammar are clean. I hope I explained it well, the first approach would have made the autogenerated ECPG grammar very unclean, you would have rejected immediately seeing that proposed. Best regards, Zoltán Böszörményi -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
ok - I don't though about it. My goal is integration main parser next commitfest, but it is true, so somebody would to play with named params now. Commiting of Steve's patch doesn't break anything. I'm a little confused here. We are 19 days into a 31 day CommitFest; you are almost three weeks too late to add a patch to the queue. Unless you can convince a friendly committer to pick this up out of sequence, I think the only patch that is under consideration here is the one that has been being discussed on this thread for the last 17 days. I sent several notes adding for all patches to be added to commitfest.postgresql.org prior to the start of CommitFest; AFAIK, this one was never added. You haven't actually specified which other patch of Steve's you're talking about anyway, but I'm guessing it's this one here: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00801.php I don't think that patch would get applied even if HAD been added to the CommitFest page in time, see Tom's remarks here: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00802.php Let's get back to focusing on the patch that is actually under consideration here. There are two patches - named and mixed parameters and protecting AS as keyword. First patch is pl independed, second patch is related to plpgsql. Both patches was in queue and Bernard did reviewing both I thing. Second patch fix some unwanted behave in plpgsql and it does on level that was possible on code base one month old. I hope, so we will be able to integrate main parser to core. I hope so this will be in next commitfest. If we doesn't release code after this commitfest, then I'll prefer wait for integration, but now, when we will release code, I thing, so for somebody should be nice to use fixed plpgsql. I thing so Steve fixed issues mentioned by Tom http://archives.postgresql.org/pgsql-hackers/2009-05/msg00804.php Integration of main parser to plpgsql will be very significant change and it needs separate patch. This solve some others big problems plpgsql and I would to solve it separately. I'll be glad to answer all questions. Pavel ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 03, 2009 at 12:19:40PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 11:32:48AM -0400, Tom Lane wrote: And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. That's an interesting idea. Shouldn't pg_migrator be mandated to work for *any* catversion bump? Oh, are you volunteering to make that happen? That code is going to be ugly enough just dealing with pairs of major releases ... With all due respect, if pg_migrator does not function in such a way as to have data-driven composable changes, it's going to bang us up in the long haul much worse than not branching alphas ever could. We require that people supply docs with their changes, and it is totally unreasonable to let them send in catalog changes which do not include need migration changes. That's how it works in every other RDBMS outfit that has changes on disk, and we do not need to be the exception. If pg_migrator doesn't have this ability, we need to refactor it until it does, or go with something that can. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Monday 03 August 2009 21:07:00 David Fetter wrote: We require that people supply docs with their changes, and it is totally unreasonable to let them send in catalog changes which do not include need migration changes. That's how it works in every other RDBMS outfit that has changes on disk, and we do not need to be the exception. Well, blocker number one for that is that pg_migrator is not even in the PostgreSQL CVS repository, but is more like an endorsed third-party product. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Bernd Helmle maili...@oopsware.de writes: --On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut pete...@gmx.net wrote: OK, here is an updated patch. It has the setting as enum, completed documentation, and libpq support. I'll add it to the commit fest in the hope that someone else can look it over in detail. I've attached a slightly edited patch which fixes a compiler warning in encode.c, too. I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
On Aug 3, 2009, at 9:38 AM, Robert Haas wrote: I sent several notes adding for all patches to be added to commitfest.postgresql.org prior to the start of CommitFest; AFAIK, this one was never added. Hi Robert, The patch for plpgsql was added as a comment to Pavel's patch. I added it as a comment because it wouldn't make since to commit it or even review it separately. This was done on the wiki before the migration. Perhaps that was not the correct way to add it to the commitfest. If not, my apologies. -Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
--On Montag, August 03, 2009 12:38:48 -0400 Robert Haas robertmh...@gmail.com wrote: I'm a little confused here. We are 19 days into a 31 day CommitFest; you are almost three weeks too late to add a patch to the queue. Unless you can convince a friendly committer to pick this up out of sequence, I think the only patch that is under consideration here is the one that has been being discussed on this thread for the last 17 days. I sent several notes adding for all patches to be added to commitfest.postgresql.org prior to the start of CommitFest; AFAIK, this one was never added. Well, i already noted that in the named and mixed notation thread actually two patches were involved, see http://archives.postgresql.org/pgsql-rrreviewers/2009-07/msg00016.php. Since Steve's changes were so small, i considered it to be an extension to Pavels patch, without treating it separately to commit. In my opinion it wouldn't make sense to commit those changes to plpgsql _without_ having Pavels functionality. I have to admit that i saw Tom's complaint about making AS a special keyword in plpgsql, but decided to leave the decision wether we want to commit this or not up to a committer. I took the efforts to get the attached patches there in a reviewable state then instead. Please note that Steve's suggestion is linked into the commitfest since 2009-05-21, too. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
--On Montag, August 03, 2009 15:11:08 -0400 Tom Lane t...@sss.pgh.pa.us wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I would prefer it to be the default at least for pg_dump, if we can get some significant performance improvement for both, dump and restore from it. However, here are some current performance numbers (taken from today, since yesterday i had some trouble to get on the machine): I did some restore testing based on the following flow: BEGIN; TRUNCATE ... ; COPY testtable FROM ... ; ROLLBACK; with bytea_output = 'escape' i get Time: 1478801,770 ms where bytea_output = 'hex' gives: Time: 1448871,566 ms So 'hex' is slightly faster on this machine, but not in the numbers i would have expected. The hex-based restore gives the following profile: Each sample counts as 0.01 seconds. % cumulative self self total time seconds secondscalls s/call s/call name 37.81157.22 157.2297847 0.00 0.00 pglz_compress 20.25241.4384.21 141398 0.00 0.00 CopyReadLine 14.44301.4860.05 3605691992 0.00 0.00 get_hex 8.29335.9634.48 141397 0.00 0.00 hex_decode 7.99369.2033.24133.24 398.14 DoCopy 3.95385.6316.43 esc_enc_len 0.71388.58 2.95 137268286 0.00 0.00 _bt_compare 0.54390.81 2.23 7209863 0.00 0.00 XLogInsert 0.48392.81 2.00 49329221 0.00 0.00 hash_search_with_hash_value 0.43394.59 1.78 91132579 0.00 0.00 LWLockAcquire 0.42396.34 1.75 92250421 0.00 0.00 LWLockRelease 0.42398.08 1.75 30477526 0.00 0.00 ReadBuffer_common 0.20398.93 0.85 28686690 0.00 0.00 PinBuffer 0.18399.67 0.74 21541372 0.00 0.00 _bt_binsrch 0.16400.34 0.67 39278753 0.00 0.00 AllocSetAlloc -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
--On Montag, August 03, 2009 12:18:13 -0700 Steve Prentice prent...@cisco.com wrote: I added it as a comment because it wouldn't make since to commit it or even review it separately. That was exactly i was understanding it. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
Bernd Helmle maili...@oopsware.de wrote: Please note that Steve's suggestion is linked into the commitfest since 2009-05-21, too. Yeah: http://wiki.postgresql.org/index.php?title=CommitFest_2009-Firstdiff=6391oldid=6250 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 3, 2009 at 2:07 PM, David Fetterda...@fetter.org wrote: On Mon, Aug 03, 2009 at 12:19:40PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Mon, Aug 03, 2009 at 11:32:48AM -0400, Tom Lane wrote: And I doubt we'd bother generating pg_migrator builds that work for pairs of alpha releases. That's an interesting idea. Shouldn't pg_migrator be mandated to work for *any* catversion bump? Oh, are you volunteering to make that happen? That code is going to be ugly enough just dealing with pairs of major releases ... With all due respect, if pg_migrator does not function in such a way as to have data-driven composable changes, it's going to bang us up in the long haul much worse than not branching alphas ever could. We require that people supply docs with their changes, and it is totally unreasonable to let them send in catalog changes which do not include need migration changes. That's how it works in every other RDBMS outfit that has changes on disk, and we do not need to be the exception. If pg_migrator doesn't have this ability, we need to refactor it until it does, or go with something that can. I don't disagree with this, but we don't have anyone to do the work atm, even if it were exactly clear what to do, which it isn't. What we could do, though, is try to figure out whether it will work between 8.4.0 and 8.5alpha1. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Mon, Aug 03, 2009 at 09:22:52PM +0300, Peter Eisentraut wrote: On Monday 03 August 2009 21:07:00 David Fetter wrote: We require that people supply docs with their changes, and it is totally unreasonable to let them send in catalog changes which do not include need migration changes. That's how it works in every other RDBMS outfit that has changes on disk, and we do not need to be the exception. Well, blocker number one for that is that pg_migrator is not even in the PostgreSQL CVS repository, but is more like an endorsed third-party product. I'm not entirely sure that pg_migrator should be tied to releases of PostgreSQL, given what it does. Or did you mean that it's not been given the same scrutiny that the PostgreSQL code base has? Thoughts? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Bernd Helmle maili...@oopsware.de writes: --On Montag, August 03, 2009 15:11:08 -0400 Tom Lane t...@sss.pgh.pa.us wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I would prefer it to be the default at least for pg_dump, Well, we could have pg_dump force the output format to hex regardless of what the default is. A disadvantage of doing that is there wouldn't be any convenient way to get pg_dump to *not* set the output format (unless we add a switch, which seems way overkill). Which would mean there would be no good way to get pg_dump to produce backwards-compatible output. But considering how many other backwards-incompatible changes we have put into pg_dump without blinking, I'm not sure this argument outweighs the probability of breaking a lot of applications. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Alpha Releases: Docs?
Peter, There's another question for alpha releases: are we going to build docs? Either for www.postgresql.org, or for PGDATA/docs? I think we need some kind of docs up, otherwise we'll get little actual testing. As previously discussed, building the docs yourself from pure source involves several complicated dependancies which aren't available on all platforms. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
