Re: [HACKERS] Cancelling idle in transaction state
On Dec 5, 2009, at 12:25 PM, Simon Riggs wrote: ... I'm not volunteering here, but having worked with the protocol, I do have a suggestion: This allows locks to be released, but it is complex to report the cancellation back to the client. I think the answer here is that the server should *not* report the cancellation. Rather, it should mark the transaction as failed and let the client eventually sync its state on subsequent requests that will result in InFailedTransaction ERRORs. With such a solution, COMMITs issued to administrator cancelled transactions should result in an ERROR. Well, I suppose that would only be a requirement when: BEGIN; ... some work ... idle admin zapped this transaction more idle COMMIT; -- client needs to know that this failed, and it should be something louder than a ROLLBACK tag. :P So, if a command were issued to a cancelled transaction prior to a COMMIT: BEGIN; ... some work ... idle admin zapped this transaction SELECT * FROM something; -- fails, IFX ERROR emitted to client COMMIT; -- client was already notified of the xact failure by a prior command's error, so the normal ROLLBACK would be fine. Also, if immediate notification is seen as a necessity, a WARNING with a special code could be leveraged. Oh, or maybe use a dedicated LISTEN/NOTIFY channel? LISTEN pg_darn_admin_zapped_my_xact; to opt-in for transaction cancellation events that occur in *this* backend.. [Note: this is in addition to COMMITs emitting ERRORs] I can't see immediate notification being useful excepting some rather strange situations where the client left the transaction idle to go do other expensive operations that should be immediately interrupted if this particular transaction were to be cancelled for some reason.. Such a situation might even make sense if those expensive operations somehow depended on the locks held by the transaction, but I think that's a stretch. Not to mention that the client could just occasionally poll the transaction with 'SELECT 1's; no special WARNING or NOTIFY's would be necessary. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 20, 2009, at 12:02 AM, Peter Eisentraut wrote: Is there any precedent for the sort of behavior that you are implementing, that is, automatic sharing of variables between independent executions of the same source container? import foo # bar is a regular, def'd function. foo.bar() ... # even in another thread, doesn't matter.. foo.bar() In either call, foo.bar()'s globals() is the same dictionary object(the foo module's dictionary). A plpython3 function *is* a Python module. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 20, 2009, at 1:26 AM, Peter Eisentraut wrote: because this is the same execution Hrm, not necessarily. foo could be imported by another, completely independent part of the program. foo is cached in sys.modules. bar() is executed and it's still the same globals(). shared. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 19, 2009, at 3:12 AM, Peter Eisentraut wrote: The other approach, which is what James Pye's new implementation proposes (as I understand it), is to convert PostgreSQL types into specially made Python objects, such as Postgres.types.record or Postgres.types.timestamp. Convert is not a good word choice. The Datum of the parameter is stored inside a new Python object(that only holds a Datum). So more like copied into Python memory, and associated with its respective type. Wrapped in a Python object? One cool thing about doing it this way, is that if you just pass parameters forward to a prepared statement, there's no type I/O overhead. Not a huge performance win for common cases, but if someone were passing larger arrays around, it could be quite beneficial. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 19, 2009, at 11:32 AM, Peter Eisentraut wrote: But you wouldn't, for example, get away with breaking SQL (or even improving it incompatibly) to facilitate a better elog. This doesn't fit the situation. I'm not breaking PL/Python. I'm trying to add PL/Python3. =) I think of a PL/Python function as a Python script file stored in the database. For Python, I think that's a mistake. Python scripts are independent applications. [tho, I think this does illuminate our perspectives...] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 18, 2009, at 8:37 AM, Peter Eisentraut wrote: The question is whether it helps the user, not the implementer. Sure, but do you have a patch waiting to implement tracebacks? I'd argue the reason it's never been done is due to the way procedures are currently managed in PL/Python. And *without some significant refactoring*, any patch fully implementing tracebacks is going to be a seriously ugly hack. What helped the implementer here would help the user. As far as I can tell, it just creates more typing for no benefit whatsoever. def main(*args): is annoying, but not entirely lamentable... It's explicit, as well(no need to document munging that occurs behind the scenes). Also, compare the cases where you need to cache some initialized data: if 'key' not in SD: ... SD['key'] = my_newly_initialized_data ... With function modules, SD is not needed as you have your module globals to keep your locally cached data in: ... data = my_newly_initialized_data def main(*args): ... Also, it's inconsistent with normal Python script files Hinges on whether normal is actually normal. I often use the __name__ convention in script files myself: if __name__ == '__main__': main(...) That is, using that convention, the script can be import'd and used without executing the script functionality. (It has proven to be very handy a few times now) and with other PLs. I don't understand why that's a significant enough interest to note. I don't need another PostgreSQL implementation on top of Python. Indeed, and I do understand that. That is, I have removed some features with that very thought in mind. (OTOH, I consider the date_part properties on datetime types to be special: too likely useful.) [tho, PostgreSQL implementation? I think I understand what you were getting at, but..] The maintenance effort required to keep those two consistent aside. I don't think there are many consistency issues here. What did you have in mind? Again, I'm only one user. But so far I haven't seen anyone else speak up here, and clearly accepting this for inclusion will need nontrivial convincing. Agreed. It would seem quite doomed. At this point, I'm not going to try getting it into PG. (apparent futility and such) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 18, 2009, at 1:36 PM, James Pye wrote: At this point, I'm not going to try getting it into PG. (apparent futility and such) ugh, on second thought, I think I've written a bit too much code to stop now. I'm going to get plpython3 as far as I can and submit it to the next commitfest. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 15, 2009, at 6:37 AM, Peter Eisentraut wrote: but these two features don't excite me at all, hrm.. at all? I can see how function modules might look like a half-step backwards from function fragments at first, but the benefits of a *natural* initialization section (the module body) was enough to convince me. The added value on the PL developer's side was also compelling. Tracebacks were trivial to implement, and there is no need to munge the function's source. It seemed like a win all around... AFA native typing is concerned, I think the flexibility and potential it offers is useful, no? Already, plpython3 provides properties on PG's datetime types to access the date_part()'s of the object. OTOH, for folk who primarily use the PL to access functionality in Python modules(bindings), native typing may be of no direct utility as they will likely need to convert anyways. (If that's your common use-case, then the absence of interest in native typing is quite understandable.) [looking at the PL/Python todo list..] Excepting DB-API and trusted, I believe all the current PL/Python TODOs are fulfilled or N/A in plpython3... ugh, the docs are not yet complete, but I like to think of them as better anyways. :P the pain of dealing with a second implementation. What pain are you anticipating? Maintenance? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 13, 2009, at 4:47 AM, Peter Eisentraut wrote: Has this list of gripes ever been brought up and discussed in this forum? Some are TODOs, so in part by other people. Some were briefly touched on in the recent past discussions(around the time that I announced the WIP). Native typing vs conversion, function fragments vs function modules. I don't think native typing has seen any actual discussion, but I do recall mentioning it as something that I wanted to do(implicitly griped?). ... There is a difference in the situation from the discussion before. Prior, it was, I would like to implement a new PL for Python 3 with these features, and now, it is, I have implemented a new PL for Python 3 with these features. Simply, -hackers can choose among moving forward with Python 3 support in plpython or going with plpython3 or even both, I suppose(?). Naturally, I'm biased toward something that involves plpython3, so I don't think I can(should?) be of much help to -hackers as a Python PG user in any such decision. Of course, excepting the provision of justifications for my implementation/design choices... I would really love to see some input from Python users. I certainly don't want to waste time trying to get something into pgsql that Python users don't want. [here's a gripe that I haven't brought up as I think it is a matter of taste] I find (plpython) trigger functions to be a bit odd. I think it's the way in which manipulation/suppression decisions are made in BEFORE ROW triggers(return OK, SKIP, etc).. [label this as opinion at this point as I have yet to be able to nail down what, specifically, is wrong or un-pythonic about them.] Also, having distinct entry points to handle trigger events helps reduce potential errors by forcing the user to explicitly state the events that the trigger function can handle. Currently, in plpython, users *should* do sanity checks. Function modules opened the door for implementing this in a natural way, multiple functions(entry points) in the function module. http://python.projects.postgresql.org/pldocs/plpython3-programming.html#PLPYTHON3-FUNCTIONS-TRIGGERS -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.1 support
On Nov 12, 2009, at 12:54 PM, Peter Eisentraut wrote: Here's the patch to support Python =3.1 with PL/Python. :\ I was hoping to be able to use Python 3 to draw a clear distinction between plpython and the would-be plpython3 that I've been working on. I understand that you're not in favor of a brand new implementation for Python 3. Despite my dislike for that position(well, it would seem to be in opposition to my initiative, so naturally =), I don't entirely disagree with your rationale[wrt doing things more incrementally]. For me, plpython has never been what I would call a pleasure to use, and many of the gripes that I have with it are, IMO, entrenched far enough into the implementation that any efforts to change it would(should? =) cause unacceptable breakage in user applications(?). Well, as far as additional Python interfaces are concerned, a lot of redundant functionality, but that's not even the half of it. [I was hoping to get to a status message this weekend, but it seems like I should follow-up here. =] So here's where I'm at: -- Mostly documentation improvements since I last pinged -hackers. Still, *sigh*, filling in documentation and fighting bugs as I go. Currently resolving a bug instantiating MD arrays from nested lists. Once I'm finished with the docs, I'm going to start looking for refcount leaks. No major additions or changes are planned, but I have been making some minor additions as I write more docs. Overview/Features: http://wiki.postgresql.org/wiki/WIP:plpython3 Documentation: http://python.projects.postgresql.org/pldocs/plpython3.html git repo[see the plpython3 branch]: http://git.postgresql.org/gitweb?p=plpython3.git;a=summary Most of the documented interfaces have tests. I only have two platforms at my disposal, so I do fear that this will not just work on all of PG's supported platforms. Specifically, I've ran the tests on freebsd/amd64 and Mac10.6/intel(of course 10.5 as well for some earlier revisions). [err, actually, it's been a while since I ran the tests on freebsd.] -- plpython3 is turning out to be kinda beefy(~974K diff[eh, there is some fluff in there]), and I can't say that I've seen much interest in it, so I can't really blame anyone if -hackers ends up taking a pass on it. (python3 is too far away for most folk to care? folk are content with plpython?) eh, cheers, either way. =) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Typed tables
On Nov 5, 2009, at 10:24 AM, Peter Eisentraut wrote: One thing I'm not sure of is whether to keep the implicit row type in that case. That is, would the above command sequence still create a persons type? We could keep that so as to preserve the property a table always has a row type of the same name +1 for keeping it. Thoughts? Any plans to allow the specification of multiple types to define the table? CREATE TABLE employee OF employee_data_type, persons_data_type; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Tightening binary receive functions
On Aug 31, 2009, at 1:12 AM, Heikki Linnakangas wrote:
...
Is the new date_recv() constraint actually correct?
[looking at the result 0 part, at least]
src/backend/utils/adt/date.c
...
+ /* Limit to the same range that date_in() accepts. */
+ if (result 0 || result JULIAN_MAX)
+ ereport(ERROR,
+ (errcode
(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
+errmsg(date out of range)));
+
+ PG_RETURN_DATEADT(result);
}
postgres=# SELECT date_send('500-01-01'::date);
date_send
\xfff7a3e9
(1 row)
...
struct.unpack(!l, b'\xff\xf7\xa3\xe9')
(-547863,)
Perhaps 'result' needs to be adjusted by the postgres epoch for the
comparison?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RfD: more powerful any types
On Sep 9, 2009, at 4:44 AM, Peter Eisentraut wrote: That's beginning to sound a bit like a generics feature. E.g., CREATE FUNCTION the_sameT(arg_a T, arg_b T) RETURNS bool AS $$ SELECT arg_a IS DISTINCT FROM arg_b; $$; mmm, yeah... ISTM that expansion in this area should probably head toward generics.. Does SQL spec such a thing? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RfD: more powerful any types
On Sep 8, 2009, at 10:48 AM, David Fetter wrote: I'd like to see pseudo-types like ANYNUMERIC, and allow it to take an array decorator, which would really help for math-ish functions. Not sure where that fits in this discussion. Perhaps typcategory could be leveraged here? ..Tho, if I understand the general direction, I think it would be along the lines of type classes/interfaces.. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
On Sep 8, 2009, at 7:38 AM, Kevin Grittner wrote: David Fetter da...@fetter.org wrote: CREATE TRIGGER trig BEFORE UPDATE ON tbl FOR EACH ROW WHEN (NEW.col IS DISTINCT FROM OLD.col) EXECUTE PROCEDURE trigger_func(); How much does that buy you versus including this at the start of trigger_func: On the face, it buys nothing, IMO. ISTM, looking at the examples, that the above syntax would lead to redundant logic if the particular trigger_func() were used by multiple TRIGGERs. That is, assuming the precondition is necessary for proper functionality, it would have to be repeated on all the TRIGGERs that trigger_func() would be executed by. [..moving away from the isolated use-case of the example] However, if trigger_func() were a generalized trigger function, which would likely be the case if it were used by multiple TRIGGERs[;)]. The necessary precondition would probably be inconsistent across the TRIGGERs, and thus the feature could be quite useful. Currently, such a generalized trigger function *could* be crafted using trigger arguments, but I'd be inclined to think that that would require more exercise than it would be worth( that is, I'm imagining something that would be dirty, and much less convenient than WHEN =). Personally, I think WHEN () would be pretty sweet. =) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
On Sep 8, 2009, at 5:33 PM, Itagaki Takahiro wrote:
WHEN clause in other times [1][2][3]. (All of them use WHEN for the
syntax; that's why I proposed WHEN but not WHERE.)
Well, looks like WHEN is, or is going to be standard:
triggered action ::=
[ FOREACH { ROW | STATEMENT } ]
[ WHENleft parensearch condition right paren ]
triggered SQL statement
(page 653 from 5CD2-02-Foundation-2006-01)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] still a wip, plpython3
Thought another message might be appropriate as I've made some progress: finished up PEP302 interfaces on PyPgFunction objects(provides linecache support for tracebacks), IST support, DB error handling(pg error - pyexc - pg error), and, recently, some annoying type interface improvements.. Some of those features in action: IST tests[1], error tests[2]. Also, I started on a WIP wiki page covering my progress and a small amount of documentation for anyone curious enough to play with it[3]. [some notes; feel free to swing the cluebat =] In order to implement IST support and error handling, the PL has to keep track of the number of open ISTs, and whether or not the code executed by the PL caused a database error. Tracking the error state is needed so that some (most) Postgres interfaces will not be invoked inside a failed transaction, and so that the PL will not exit without indicating an error state(elog/ ereport), and so that the PL will not try to commit a failed subtransaction on IST __exit__. Tracking the number of open ISTs is needed so that the PL can properly clean up after any misused Postgres.Transaction objects. When this situation is presented to the PL, all the excess ISTs that remained open will be aborted and an error will be thrown. At this point, the major items remaining are: SPI interfaces, polymorphic functions, array composite interfaces(the objects can be created and passed around, but the item/element access methods are completely untested, ATM), whatever random bugs popup/minor cleanupS, and, of course, documentation. Still lots to do. [1] http://github.com/jwp/postgresql-plpython3/blob/67c75e7e80c853d12cd279ecd86fd0a409d4007d/src/pl/plpython3/expected/plpython3_xact.out [2] http://github.com/jwp/postgresql-plpython3/blob/67c75e7e80c853d12cd279ecd86fd0a409d4007d/src/pl/plpython3/expected/plpython3_error.out [3] http://wiki.postgresql.org/wiki/WIP:plpython3 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] plpythonu datatype conversion improvements
On Aug 15, 2009, at 4:44 PM, Peter Eisentraut wrote: What's needed here, I think, is an API that takes a datum plus type information and checks whether the datum is valid within the domain. /agree =) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath not a good replacement for xpath_string
On Jul 29, 2009, at 12:12 PM, Andrew Dunstan wrote: As I said upthread, I think we can easily provide some extra convenience functions which will do what you want. The only thing I was really arguing about was the function name for such a gadget. +1. xpath_string does seem unfortunate, but I'm not offended by it. =) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: plpython3
On Jul 24, 2009, at 1:21 AM, Peter Eisentraut wrote: While various of these ideas may be good, I think you are setting yourself up for a rejection. Right, I supposed that that may be the case or at least that you would feel this way based on your messages from the prior thread. There is a lot of plpython code already out there, and many years have gone into debugging plpython to work well, so rewriting everything and setting everyone up for a flag day, or requiring the parallel maintenance of old and new versions of plpython is not going to work. Does this mean that you are no longer of the opinion that a separate implementation is acceptable under the circumstances that it provides major advantages? Or are you of the opinion that the listed features do not provide major advantages? Or, perhaps, more appropriately, that the transitional features do not provide major advantages? [transitional features being native typing and reworked function structure] As far as I can tell, most of the features you list above could very well be implemented in the current language handler, using separate, isolated patches. I don't see why everything needs to be written from scratch. That's why I tried to highlight native typing and the reworked function structure. Those two features, not to mention Python 3, make it a distinct-enough beast to justify a different code base, IMO. The rest are icing. Icing is delicious. I see Python 3 as a good opportunity to change the interfaces and fix the design of the PL. I dunno. I have time to give it some TLC, and I'm not terribly excited about trying to tack features onto something that I find kinda gross. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: plpython3
On Jul 24, 2009, at 7:08 PM, Stuart Bishop wrote: I'm not particularly interested in Python 3.x support yet (we are still back on 2.4, soon to hop to 2.5 or 2.6. For us 3.1 is probably 2 years away at the earliest). I am interested in improved plpython though. Two years would hopefully be enough time to work out most of the new bugs. =) This way I can pass in a mock object. This is also useful outside of the test suite - the same module can be used as a stored procedure or by your Python application - your web application can use the same validators as your check constraints for instance. Hmm. import sys sys.modules[Postgres] = mock_pg_module Would that not suffice? I'd like a way to avoid initialization on module import if possible. Calling an initialization function after module import, if it exists, would do this. CREATE FUNCTION foo(int) RETURNS in LANGUAGE plpythonu AS $python$ [initialization on module import] def pg_init(pg): [initialization after module import] def pg_main(pg, i): return i $python$; I do like this idea. However, it may already be possible under the current design with some explicit main() management: CREATE ... $python$ import Postgres def usual(*args): ... def init(*args): global main ... main = usual return usual(*args) main = init $python$; Perhaps ugly, but I imagine a construct could be created to clean it up: CREATE ... $python$ import Postgres def usual(*args): ... def init(*args): ... return usual(*args) main = call_once_then(init, lambda: globals()['main'] = usual) $python$; Hmm, still ugly tho, no? Well, the above examples aren't actually consistent with your design, but perhaps it achieves the desired result? I tend to dislike magic function names, but perhaps it is the most usable solution. Indeed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] WIP: plpython3
http://github.com/jwp/postgresql-plpython3/tree/plpython3 [branch name: plpython3] [src/pl/plpython3] (Yeah, I'm going to try to move it to git.postgresql.org soon-ish) In a recent thread[1], Peter said: That also means that maintaining a separate, parallel code base for a Python 3 variant can only be acceptable if it gives major advantages. Here are the features that I plan/hope to implement before submitting any patch: * Native Typing [Python types that represent Postgres types] * Reworked function structure (Python modules, not function fragments) * Improved SQL interfaces (prepared statement objects[2]) * Better SRF support(?) (uses iterators, will support composites, vpc mat) * Direct function calls (to other Postgres functions) * IST support (with xact(): ...) * Full tracebacks for Python exceptions(CONTEXT support) * Cached bytecode (presuming a procache attributes patch would be acceptable[3]) The first two features are why a new PL should be incorporated. Native typing alone is that desirable because it allows for Postgres type semantics to be retained inside Python. Using conversion for some types--the existing solution in plpython--may not be desirable due to potential inconsistencies in value. A notable example is that Python's datetime.timedelta cannot support interval's month field. And from a performance perspective, creating Python objects representing a parameter is approximately the cost of allocating memory for a Python object and datumCopy. The second feature, function structure, is actually new to the PL. Originally PL/Py took a pl/python-like approach to triggers and functions. *Currently*, I want to change procedures to be Python modules with specific entry points used to handle an event. Mere invocation: main. Or, a trigger event: before_insert, after_insert, before_update, etc. So, a regular function might look like: CREATE OR REPLACE FUNCTION foo(int) RETURNS int LANGUAGE plpython3u AS $python$ import Postgres def main(i): return i $python$; Despite the signature repetition, this is an improvement for the user and the developer. The user now has an explicit initialization section that is common to Python(it's a module). The PL developer no longer needs to munge the source, and can work with common Python APIs to manage and introspect the procedure's module(...thinking: procedure settings..). A trigger function might look like: CREATE OR REPLACE FUNCTION trig() RETURNS TRIGGER LANGUAGE plpython3u AS $python$ import Postgres def check(i): ... def before_insert(new): ... def before_update(new, old): # The default action is for the manipulation to occur, # so users must explicitly raise FilterEvent in order to # stop a row from being inserted, updated, deleted. if check(new[column_name]): raise StopEvent() def after_delete(old): ... $python$; Thoughts? [...it still has a *long* ways to go =] [1] http://archives.postgresql.org/pgsql-hackers/2009-05/msg01376.php [2] http://python.projects.postgresql.org/docs/0.9/driver.html#prepared-statement-interface-points [3] http://archives.postgresql.org/pgsql-hackers/2006-05/ msg01160.php (I think a new column would be wise) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git.postgresql.org vs. REL8_1_STABLE
On Jul 10, 2009, at 11:03 AM, Joshua Tolley wrote: Am I the only one having problems building 8.1 from git? (Am I the only one building 8.1 from git?) In a clean repository, I've checked out REL8_1_STABLE, configured with only one argument, to set --prefix, and make gives me this: While not this exact issue, I have had issues compiling older _STABLE branches from git. At the time, I wasn't sure if I somehow borked my clone or not, so I just moved on. [iirc, my problem was with dbcommand.c. the repo seemed to have checked out an older header file..] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.0 does not work with PL/Python
On May 29, 2009, at 1:17 AM, Peter Eisentraut wrote: On Friday 29 May 2009 04:06:14 Andrew Dunstan wrote: Otherwise, I'm not too keen simply to throw Python 2.x overboard until it's no longer common on platforms people are likely to want to install Postgres on, if that's what's implied by the original question. My guess is that we will need to keep around a Python 2.x version for at least another three years, meaning two or three major PostgreSQL releases. Yeah, I wasn't meaning to imply tossing 2.x out... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.0 does not work with PL/Python
On May 3, 2009, at 11:02 PM, Peter Eisentraut wrote: http://www.joelonsoftware.com/articles/fog69.html Good read. =) However, complete rewrite being relative in this case: WIP: http://github.com/jwp/postgresql-plpython3/tree/c804e693b6a0df98c0e5c465e75ba2e9014ebf37/src/pl/plpython3 That is, from pgsql's perspective it would be a complete rewrite. From my perspective it would be a refactored version of plpy for Python 3.x and pgsql 8.5/head. Additionally, plpy originally came from plpythonu(7.3 or 7.4), but was *massively* refactored(read: effectively rewritten). I wouldn't want to start from scratch, actually. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Array types
On Apr 7, 2009, at 12:54 PM, John Lister wrote: Cheers, nice to know it is possible... Now to see if i can get java/ python to do the same :) or to use a modified libpq somehow... http://python.projects.postgresql.org will do it for Python. =D tho, only supports Python 3, which is still quite new. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Python 3.0 does not work with PL/Python
On Apr 5, 2009, at 8:54 AM, Tom Lane wrote: Hm, did you read the link I cited? It's not so surprising that 3.0 should have broken distutils, but what I found distressing is that they fixed distutils and then 3.0.1 broke it *again*. I stand by my opinion that Python 3 isn't stable yet. Yeah, actually. From some of the talk I've seen on python-dev, it sounds like 3.0.2 will be the last 3.0 release. 3.1 is in alpha, and ready to start cleaning things up, afaict. This means that users of PL/Python should not expect PL/Python to automatically work with 3.0. Supporting 3.0 will be a new feature. So it's OK to drop it from 8.4. One other thing that we'll have to seriously consider is whether we should package python3 as a separate PL, so that people can keep using their 2.x plpython functions without fear of breakage. I know that the Fedora guys are currently debating whether to treat it that way, and I suppose other distros are having or will soon have the same conversation. Six months from now, there will be some precedents and some track record for us to look at in making that choice. I think this would be wise. Any thoughts on the acceptability of a complete rewrite for Python 3? I've been fiddling with a HEAD branch including the plpy code in a github repo. (nah it dunt compile yet: bitrot and been busy with a 3.x driver. ;) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
On Mar 20, 2009, at 8:56 AM, Andrew Dunstan wrote: Andrew Dunstan wrote: A / at the beginning of a path expression is an abbreviation for the initial step fn:root(self::node()) treat as document-node()/ (however, if the / is the entire path expression, the trailing / is omitted from the expansion.) The effect of this initial step is to begin the path at the root node of the tree that contains the context node. If the context item is not a node, a type error is raised [err:XPTY0020]. At evaluation time, if the root node above the context node is not a document node, a dynamic error is raised [err:XPDY0050]. The problem is that we certainly do have to provide a context node (the standard is clear about that), and unless we want to convert a non-document to a node-set as James suggested and then apply the xpath expression to each node in the node-set, we have no way of sanely specifying the context node. libxml2 only supports xpath1. Why are you referencing xpath20? And, if SQL-XML requires an xpath20 conforming xpath() function, we have bigger problems than '/x' + xpath_str. ;) Although, this is not terribly important to me as: No-one has come up with an answer to this, I don't feel fragment()/node-set() is a good idea from a usability standpoint alone. I only mentioned it because that's how I've always worked with fragments in the xslt1 world.. Mere curiosity drove most of the remaining interest I had in it. so I propose to remove the hackery. I think this would probably be best for the core xpath() function. However, it may be wise to relocate the munging functionality into another function so users don't have invent their own when they feel so inclined to work with fragments. raise an error? +1, xpath requires a well-formed document -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
sigh.. I got curious. :P
On Feb 27, 2009, at 7:19 PM, James Pye wrote:
Well, that or force the user to call it explicitly.
Attached is the patch that I used to get the results below..
This is just a proof of concept, so it's quite lacking. Notably, it
doesn't even try to identify well-formed documents.
Purpose/idea being, give the user access to the poorly-formed document
as a node-set via the fragment function instead of mangling the
xpath and xml:
postgres=# SELECT xpath('fragment()//*', 'blehfoo/bar/'::xml);
xpath
---
{}
(1 row)
postgres=# SELECT xpath('fragment()//*', 'blehmehsub//mehfoo/
bar/'::xml);
xpath
--
{sub/}
(1 row)
postgres=# SELECT xpath('fragment()/*', 'blehmehsub//mehfoo/
bar/'::xml);
xpath
--
{sub/}
(1 row)
postgres=# SELECT xpath('fragment()', 'blehmehsub//mehfoo/bar/
'::xml);
xpath
{bleh,meh
sub/
/meh,foo/,bar/}
(1 row)
postgres=# SELECT xpath('/*', 'blehmehsub//mehfoo/bar/'::xml);
xpath
---
{}
(1 row)
postgres=# SELECT xpath('fragment()[local-name()=foo]/@att',
'blehmehsub//mehfoo att=sometin/bar/'::xml);
xpath
---
{sometin}
(1 row)
postgres=# SELECT xpath('fragment()[local-name()=meh]/*',
'blehmehsub//mehfoo att=sometin/bar/'::xml);
xpath
--
{sub/}
(1 row)
postgres=# SELECT xpath('fragment()[local-name()=meh or local-
name()=bar]', 'blehmehsub//mehfoo att=sometin/bar/'::xml);
xpath
-
{meh
sub/
/meh,bar/}
(1 row)
postgres=# SELECT xpath('fragment()[local-name()=bar]',
'blehmehsub//mehfoo att=sometin/bar/'::xml);
xpath
--
{bar/}
(1 row)
postgres=# SELECT xpath('fragment()[...@*]',
'blehmehsub//mehothertextfoo att=sometin/bar/'::xml);
xpath
{foo att=\sometin\/}
(1 row)
Can't say that I've ever been thrilled with using node-sets, but
*shrug*.
I'm sleepy now..
git.diff
Description: Binary data
xmlc_file.diff
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
On Feb 28, 2009, at 7:53 AM, Andrew Dunstan wrote: This is entirely out of the question for 8.3, as it's a significant change of behaviour. Yep. Even with implicit prefixing, the semantics are very different. What got me thinking about it was this: http://archives.postgresql.org/pgsql-bugs/2008-07/msg00058.php If it's desirable to avoid prefixing, what options remain? (At least I find it desirable to avoid prefixing =) I'd also want to see this usage blessed by some xpath guru ... I'm not sure it meets the standard's requirements, but I could be wrong. Oh, the context node question you raised? I think it would be easy to expect that the standard is expecting a well-formed document to query against in the first place, so I *do* think it's a very valid concern. http://www.w3.org/TR/xpath http://www.w3.org/TR/xpath#data-model http://www.w3.org/TR/xpath#infoset Curious, if we constructed an actual document fragment node from the node list and set it as the document's root, would that be enough to satisfy any requirements? It does appear to talk about nodes quite generally. In the current case, we're shaving the corners of the square peg so it will fit in the round hole. In fragment()'s case, it seems we would be trying to circumvent the round hole altogether.. I don't really like either way. :P -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
On Feb 26, 2009, at 7:03 PM, Andrew Dunstan wrote:
can you point me at any call in libxml2 which will evaluate an xpath
expression in the context of a nodeset instead of a document?
No, I can't. node-sets are XPath objects not xmlNode objects, so I
don't think it would be as simple as modifying:
xml.c:xpath() {
...
xpathctx-node = xmlDocGetRootElement(doc);
with the result of xmlXPathNewNodeSet..
[snip other questions]
My *guess* would be that if we were to use a node-set instead, we'd
still have to prefix the XPath query. In this case, with a function
call to an xpath extension function that creates the NodeSet from the
content fragment(s?) of the document created by xml_parse(ie, more or
less, a re-implementation of exsl:node-set() tailored for our use-
case). Well, that or force the user to call it explicitly. Possible or
not--wrt using a content fragment/document as the context node, I find
this less desirable than the current mangling, so I'm becoming quite
indifferent. :)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
On Feb 26, 2009, at 9:37 AM, Peter Eisentraut wrote: It's not about ill-formed pieces, it is about (well-formed) content fragments that are not full documents (exactly one root element). libxml2 doesn't support xpath on content fragments. exslt:node-set() to the rescue? Or is that/equivalent functionality not easily accessed at the C-level with libxml2? http://www.exslt.org/exsl/functions/node-set/index.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath processing brain dead
On Feb 26, 2009, at 5:41 PM, Andrew Dunstan wrote: http://www.exslt.org/exsl/functions/node-set/index.html A node-set isn't a document. yes.. :) I guess what I'm saying is that if: tinman=# SELECT XML'foo/bar/'; xml -- foo/bar/ (1 row) is considered to be valid per *SQL-XML*, then it should probably be treated as a node-set in the context of xpath, not mangled with x.../x.. Certainly, I would expect an implicit node-set() call long before wrapping the fragment in x.../x and prefixing my xpath query. However, I find the validity of the above, XML'foo/bar/', a bit disturbing to begin with. :P In any case, this functionality doesn't appear to be in libxml2, it's in libxslt according to the reference you provided. I think that's *just* referencing the list of xslt implementations that the extension function is known to be available in.. I doubt that means to imply that the function or equivalent functionality is not available in libxml2 itself. I'd wager that equivalent functionality could be implemented if it weren't directly/already available.. =) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] plpython
Greetings, I've recently been spending some quality time with the plpython module, and I think I'm well on the road to an improved version of it(although, nothing about a trusted variant). By improved, I mostly mean cleaned up, and reorganized.. Here are some of the changes that I have made in my own version: Compilation and execution have been greatly simplified and should be faster(at least execution should be). Caching of compiled code no longer references a Python dictionary(PLyProcedureCache). The handler keeps its own vector of procedure structs(should be faster, and is trivial). Removal of plpython generated dictionaries SD and GD. They don't seem be very useful, as they are forgotten when the postmaster exits and not remembered when a new one starts. SD is questionable, does/did anyone find SD very useful? GD seems almost pointless as the global keyword should be sufficient. Although, I do think there was a mention of GD being safe globals, but I don't know why it would be safer than global var. Removal of the built-in plpy python module that plpython creates. This is done because it provides interfaces to pgsql functions that I feel should be located elsewhere; elsewhere being another python module. I've already generated a preliminary interface to elog and SPI_* with SWIG that at first glance seems quite functional(it links, and is at least able to properly call elog, I haven't really tested SPI). Improvement to tracebacks, as it now NOTICE's the python tracebacks(There is already an ERROR, so I don't think WARNING is necessary). PLy_traceback, originally, seemed to ignore the tb of the PyErr_Fetch. Removal of plpython type conversion routines and data structures. This was done because I felt that there was a better way to do it. Not sure what yet, as it is one of my questions to the list, but it will probably end up being a similar implementation. I also plan to make some changes to trigger handling, but I haven't done anything worth mentioning yet.. Type conversion plpython's current type conversion implementation appears to be dependent on strings as the common format. This is fine, but not very extensible as is, unless you don't mind explicitly parsing strings inside each function that takes an unsupported data type. I was thinking that a better solution would be creating a python object type inside the database. Thus allowing users to write casts to and from non-standard or unimplemented data types with little difficulty(well, maybe some :). This would allow conversion in an extensible way, which doesn't require modification to plpython. Storage could be easily achieved by pickling the object. Another thought would be to just pass valid PyObject pointers in and out of conversion procedures, effectively disallowing storage(outside the process in which the object was created in), unless it is possible to have a persistent storage mechanism that makes it possible to go through pickle?.?..(yeah, I'm new to pgsql dev). Python PostgreSQL Interface plpython, currently, implements its own built-in module to interface with a few pgsql routines, and it works, but I feel it should be located elsewhere, as I said before. For the most part, I can only see most people using elog, and SPI within plpy, but perhaps that is too narrow of a view. Perhaps it would be useful to many to have access to some backend routines through plpy, but I'm not sure and that is why I'm asking the list. How far should such an PostgreSQL interface module go? What should its name be if full/semi-full interface is created? I was thinking simply py-pgsql as the package name, and the module name, of course, would be pgsql. What should the name be if it was only elog and SPI? py-pgspi? I'm leaning towards py-pgsql, a partial interface consisting of elog and SPI and perhaps a few other useful routines. But have the module as a package as to allow easy extensions to the package as subpackages.. From this interface, a DB-API 2.0 compatible SPI interface will come as well. My version has a short ways to go before it is ready for usage, but if you want to see what I've done, just drop me an e-mail. Comments? Criticisms? Feature suggestions? Anyone else doing significant work on plpython? -James pgp0.pgp Description: PGP signature
[HACKERS] persistant psql feature suggestion
Greets, Just a thought for a psql enhancement, afiak, it is not easily possible for persistent connections to a database in a shell script.. The ability for psql to remain in the background reading from stdin and writing to stdout until explicitly killed. More specifically, so a shell scriptor can have persistent connections to the database by calling psql once(leaving it in the bg), and redirecting stdio through a fifo(mkfifo)(sending queries by echo fifo, and fetching results by cat fifo). When I have tried this in the past it will read the query, execute it, and exit when the results are cat'd from the fifo. Just wanted to make sure nothing similar enough was already in existence, and if anyone could easily implement this. If no one wants to, I suppose I'll look into doing it if it's worth doing. :) Cheers, James pgp0.pgp Description: PGP signature
[HACKERS] FROM ONLY limitation in RICs
Does the SQL(99?) spec specify that referential integrity checks should have the FROM ONLY limitation? I ask because(afiak; I've googled and such, only found 92 which apparently does not specify anything about inheritance.) I have no access to the specs without spending money that needs to go other places.. I have a hierarchy of tables that reference base tables that may never have insertions, but its children do, which of course creates the problem of broken referential integrity checks due to FROM ONLY. Of course I know I can edit the source, but I figured I'd see if this is specified in the spec, and hopefully induce a change if it isn't. Thanks for you time, James btw, I'm not on the list(yet, perhaps), so please CC me. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
