Included here is my attempt a reconstructing our recent thread on this
subject. Given that my mail system is Lotus Notes, I have no idea how this
will appear to you.
>From my perspective a few workarounds have been suggested that I am happy
with. I do also rather like JC's latest suggestion since it removes a
burden from the programmer. However (perhaps for backward compatibility
reasons and also perhaps for a bit more flexibility), it might be better to
introduce a new storage creation method: openpersistentstorage(). I don't
know, since I'm unsure of what all the consequences would be an whether it
would be more confusing to have to "types" of storage.
--------------------------------------
Gary H. Merrill
Director and Principal Scientist, New Applications
Data Exploration Sciences
GlaxoSmithKline Inc.
(919) 483-8456
----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:43 -----
----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:44 -----
"Jean-Claude
Wippler"
<[EMAIL PROTECTED]>
To: "Brian Kelley" <[EMAIL
PROTECTED]>
21-Nov-2003 02:49 cc: [EMAIL PROTECTED]
Subject: Re: [Metakit] Passing views in
Python
Brian Kelley wrote:
> There is a cheap way of keeping the storage alive for the length of
> your program.
>
> make an empty python file like "globals.py"
>
> import metakit
> import globals
> storage = globals.storage = metakit.storage(...)
>
> now it will stay alive for the entire run of your program and be
> closed at the end. Not pretty, but fairly effective. You can always
> close the storage with
>
> globals.storage = None
>
> or
> del globals.storage
Couldn't the above lead to a trivial wrapper which could perhaps be
added to metakit.py, which lets you do:
storage = metakit.openstorage(...)
...
storage.close()
It really would be nice to have this discussion on [EMAIL PROTECTED] -
for one, I'd like Gordon McMillan, Mk4py's author for all practical
purposes, to see this discussion. It would allow me to understand what
is best *and* how far we are from implementing that choice.
The current behavior is the best in C++ - but of course that does not
imply anything for Python.
Gary, would you be willing to copy the relevant parts to the mailing
list?
-jcw
"Brian Kelley"
<[EMAIL PROTECTED]
du>
To: "Jean-Claude Wippler" <[EMAIL
PROTECTED]>
20-Nov-2003 22:41 cc: [EMAIL PROTECTED]
Subject: Re: [Metakit] Passing views in
Python
There is a cheap way of keeping the storage alive for the length of your
program.
make an empty python file like "globals.py"
import metakit
import globals
storage = globals.storage = metakit.storage(...)
now it will stay alive for the entire run of your program and be closed
at the end. Not pretty, but fairly effective. You can always close the
storage with
globals.storage = None
or
del globals.storage
Brian
"Brian Kelley"
<[EMAIL PROTECTED]
du>
To: "Jean-Claude Wippler" <[EMAIL
PROTECTED]>
20-Nov-2003 22:37 cc: [EMAIL PROTECTED]
Subject: Re: [Metakit] Passing views in
Python
Jean-Claude Wippler wrote:
>> So as long as view.copy() will do the trick, I'm
>> inclined to use that approach and live with whatever performance
>> consequences there may be with it . If I have to do memory
>> management, I
>> may as well do it in C :-) and reap corresponding performance
>> enhancements.
>> But I'm happy with the view.copy() approach and understanding what the
>> issues are.
>
I tend to think of this as resource management and not memory
management, although they are related. It is fairly easy to write a
proxy for the storage and views that will reference count correctly and
have the behavior that you expect. I posted one a while back, I think
it had one bug in it though. If you would like it, I'll send the fixed
one. It can be used transparently with the metakit interface and has
two nice extensions.
1) a view keeps a reference to its storage (and parent view)
2) all views are named, i.e. you can tell if a view was generated from a
given table or combinations of tables.
> Put the storage objects inside the objects which you want to hold the
> storage open, and you'll be all set. No need for cheap shots at C
> (which in this context is pretty far off, given how much C++ does
> extra with reference counts and clean-up) ... :)
I have to agree with Gary here that this isn't really the pythonic way.
f = open("test")
read = f.read
del f
f will not get garbage collected because we have a reference to it with
the read function.
The big difference here is that f has a close function
f.close()
while the storage object does not. It just turns out that python
currently garbage collects and closes the storage object "soon" after it
goes out of scope. The jython (JNI) binding to metakit would not. I'd
feel much safer with more explicit control over closing a storage. If
we could call storage.close() and then the views would get upset I'd be
all for that. I actually don't like having to rely on the garbage
collector to close the storage, it make me feel dirty inside :)
Here is the question, if we had storage.close() would you care if open
views kept the storage alive?
>
>
> -jcw
>
----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:41 -----
Gary H Merrill
DES Centennial Campus Laboratory
Venture Building II, 920 Main Campus
Drive , Suite 120
20-Nov-2003 12:08 Raleigh, NC 27606
To: [EMAIL PROTECTED]
cc:
Subject: Passing views in Python
I've now run into this difficulty a couple of times, but I'm still a bit
puzzled about it. There are circumstances when I will pass a view as a
parameter to a class constructor and things work fine. The callee gets the
view and can access it without difficulty.
There are other cases where I do (almost) the same thing, the callee gets
the view, but a later access to it fails. On entry to the callee, the view
is okay. But at some point apparently some "part" of the view is garbage
collected, dereferenced, goes out of scope, or whatever and *some*
properties of the view become inaccessible (determined by the dreaded
internal Python interpreter error exception), though some *other*
properties of the view are still accessible. For example, 'len(view)'
works, but 'view[0].propertyName' does not. My original thought was that
in the callee (constructor for the class being instantiated) all I would
need to do was to create a self.view to hold a reference to the incoming
view parameter. Then any of that class's methods should have a reliable
reference to the view. But this is not sufficient.
I have discovered that if instead I do 'self.view = view.copy()', then
things go well. What I'm puzzled about is *when* I need to do the copy()
and when I do not. I think there are some arcane Python scoping issues
here, but I wonder if there are any guidelines (beyond, "If it doesn't work
the way you think it should, make a copy."). I'm also puzzled because I
don't seem to have the problem with other (i.e., non-view) objects in
general. I also wonder if this is in fact what should reasonably be
expected, or if there is some possibility of a subtle bug (or at least
incompatibility) between the underlying C++ implementation and Python's
approach to memory management and scoping.
Any thoughts?
--------------------------------------
Gary H. Merrill
Director and Principal Scientist, New Applications
Data Exploration Sciences
GlaxoSmithKline Inc.
(919) 483-8456
_______________________________________________
metakit mailing list - [EMAIL PROTECTED]
http://www.equi4.com/mailman/listinfo/metakit
