Re: [Tutor] String Encoding problem
Le Mon, 20 Apr 2009 10:46:47 -0400,
Matt hellzfury+pyt...@gmail.com s'exprima ainsi:
Hey everyone,
I'm hoping someone here can help me solve an odd problem (bug?). I'm having
trouble with string encoding, object deletion, and the xml.etree library. If
this isn't the right list to be posting this question, please let me know.
I'm new to Python and don't know of any other help me Python mailing
lists. I have tried debugging this ad-infinitem. Anyway, at the bottom of
this e-mail you will find the code of a python file. This is a gross
over-simplification of my code, with little exception handling so that the
errors are obvious.
Running this interactively, if you finish off with 'del db', it exits fine
and creates a skeleton xml file called 'db.xml' with text 'root /'.
However, if you instead CTRL-D, it throws at exception while quitting and
then leaves an empty 'db.xml' which won't work. Can anyone here help me
figure out why this is?
Stuff I've done:
I've traced this down to the self.commit() call in __del__. The stacktrace
and a few print statements injected into xml.etree leads me to the call
'root'.encode('us-ascii') throwing a LookupError on line 751 of
xml.etree.ElementTree. This makes no sense to me, since it works fine
normally.
Thank you very much. Any and all help or pointers are appreciated.
~Matt
db.py ###
from xml.etree import ElementTree as ET
import os
class Database(object):
def __init__(self, path):
self.__dbpath = path## Path to the database
self.load()
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a
LookupError
##and empty xml file
from StringIO import StringIO
from traceback import print_exc
trace = StringIO()
try:
print 5
self.commit()
print 7
except Exception:
print_exc(100, trace)
print trace.getvalue()
def load(self):
if os.path.exists(self.__dbpath):
self.root = ET.parse(self.__dbpath).getroot()
else:
self.root = ET.Element(root)
def commit(self):
ET.ElementTree(self.root).write(self.__dbpath)
db = Database('db.xml')
Actually, it all runs well for me -- after the following modification:
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a LookupError
##and empty xml file
try:
print 5
self.commit()
print 7
except Exception:
raise
Notes:
* I don't know for what reason you needed such a complicated traceback
construct.
* Before I did this modif, I indeed had a weird exception about stringIO.
* __del__() seems to do the contrary: it writes back to file through commit()???
* del db works fine, anyway
* When I run without any bd.xml, it properly creates one with text root /.
* When I run with an ampty db.xml, I have the following exception message:
Traceback (most recent call last):
File xmlTree.py, line 29, in module
db = Database('db.xml')
File xmlTree.py, line 10, in __init__
self.load()
File xmlTree.py, line 24, in load
self.root = ET.parse(self.__dbpath).getroot()
File /usr/lib/python2.5/xml/etree/ElementTree.py, line 862, in parse
tree.parse(source, parser)
File /usr/lib/python2.5/xml/etree/ElementTree.py, line 587, in parse
self._root = parser.close()
File /usr/lib/python2.5/xml/etree/ElementTree.py, line 1254, in close
self._parser.Parse(, 1) # end of data
xml.parsers.expat.ExpatError: no element found: line 2, column 0
5
Exception exceptions.AttributeError: AttributeError('Database' object has no
attribute 'root',) in bound method Database.__del__ of __main__.Database
object at 0xb7e78fec ignored
--
la vita e estrany
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
From: spir denis.s...@free.fr
Date: Mon, 20 Apr 2009 12:22:59 -0500
To: Python Tutor tutor@python.org
Subject: Re: [Tutor] String Encoding problem
Le Mon, 20 Apr 2009 10:46:47 -0400,
Matt hellzfury+pyt...@gmail.com s'exprima ainsi:
Hey everyone,
I'm hoping someone here can help me solve an odd problem (bug?). I'm having
trouble with string encoding, object deletion, and the xml.etree library. If
this isn't the right list to be posting this question, please let me know.
I'm new to Python and don't know of any other help me Python mailing
lists. I have tried debugging this ad-infinitem. Anyway, at the bottom of
this e-mail you will find the code of a python file. This is a gross
over-simplification of my code, with little exception handling so that the
errors are obvious.
Running this interactively, if you finish off with 'del db', it exits fine
and creates a skeleton xml file called 'db.xml' with text 'root /'.
However, if you instead CTRL-D, it throws at exception while quitting and
then leaves an empty 'db.xml' which won't work. Can anyone here help me
figure out why this is?
Stuff I've done:
I've traced this down to the self.commit() call in __del__. The stacktrace
and a few print statements injected into xml.etree leads me to the call
'root'.encode('us-ascii') throwing a LookupError on line 751 of
xml.etree.ElementTree. This makes no sense to me, since it works fine
normally.
Thank you very much. Any and all help or pointers are appreciated.
~Matt
db.py ###
from xml.etree import ElementTree as ET
import os
class Database(object):
def __init__(self, path):
self.__dbpath = path## Path to the database
self.load()
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a
LookupError
##and empty xml file
from StringIO import StringIO
from traceback import print_exc
trace = StringIO()
try:
print 5
self.commit()
print 7
except Exception:
print_exc(100, trace)
print trace.getvalue()
def load(self):
if os.path.exists(self.__dbpath):
self.root = ET.parse(self.__dbpath).getroot()
else:
self.root = ET.Element(root)
def commit(self):
ET.ElementTree(self.root).write(self.__dbpath)
db = Database('db.xml')
Actually, it all runs well for me -- after the following modification:
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a LookupError
##and empty xml file
try:
print 5
self.commit()
print 7
except Exception:
raise
I must be missing something I run the following code (in DB.py) without any
other files in the current directory:
from xml.etree import ElementTree as ET import os class Database(object):
def __init__(self, path):self.dbpath = path## Path to the database
self.load()def __del__(self):try:print 5
self.commit()print 7except Exception:raise
def load(self):if os.path.exists(self.dbpath):self.root =
ET.parse(self.dbpath).getroot()else:self.root =
ET.Element(root)def commit(self):
ET.ElementTree(self.root).write(self.dbpath) db = Database('db.xml')
Output:
5 Exception LookupError: LookupError('unknown encoding: us-ascii',) in bound
method Database.__del__ of __main__.Database object at 0x87870 ignored
If you're not getting the same output, please let me know what your environment
is. Perhaps this is an implementation difference across platforms.
Notes:
* I don't know for what reason you needed such a complicated traceback
construct.
That was only to demonstrate the error. Without that, you see a LookupError
without any trace.
* Before I did this modif, I indeed had a weird exception about stringIO.
Top-level imports are not consistently available in __del__. That shouldn't be
necessary with the code I have above.
* __del__() seems to do the contrary: it writes back to file through
commit()???
Yes, I know. In my actual code, there is a flag that is set when certain
run-time conditions are met or when the user wants the DB to be saved on quit.
Most of the time, however, modifications to the database need to be done in
memory because they are not intended to be saved.
* del db works fine, anyway
* When I run without any bd.xml, it properly creates one with text root /.
* When I run with an ampty db.xml, I have the following exception message:
Traceback (most recent call last):
File xmlTree.py, line 29, in module
db
Re: [Tutor] String Encoding problem
Matt wrote:
Hey everyone,
I'm hoping someone here can help me solve an odd problem (bug?). I'm
having trouble with string encoding, object deletion, and the xml.etree
library. If this isn't the right list to be posting this question,
please let me know. I'm new to Python and don't know of any other help
me Python mailing lists. I have tried debugging this ad-infinitem.
Anyway, at the bottom of this e-mail you will find the code of a python
file. This is a gross over-simplification of my code, with little
exception handling so that the errors are obvious.
Running this interactively, if you finish off with 'del db', it exits
fine and creates a skeleton xml file called 'db.xml' with text 'root
/'. However, if you instead CTRL-D, it throws at exception while
quitting and then leaves an empty 'db.xml' which won't work. Can anyone
here help me figure out why this is?
Stuff I've done:
I've traced this down to the self.commit() call in __del__. The
stacktrace and a few print statements injected into xml.etree leads me
to the call 'root'.encode('us-ascii') throwing a LookupError on line 751
of xml.etree.ElementTree. This makes no sense to me, since it works fine
normally.
The environment available to __del__ methods during program termination
is wonky, and apparently not very consistent either. I can't say that I
completely understand it myself, perhaps someone else can provide a
better explanation for both of us, but some of the causes are described
in the documentation:
http://docs.python.org/reference/datamodel.html#object.__del__
What is your rationale for using __del__? Are you trying to force a
'commit()' call on Database instances when your program terminates -- in
the case of an unhandled exception, for example?
HTH,
Marty
Thank you very much. Any and all help or pointers are appreciated.
~Matt
db.py ###
from xml.etree import ElementTree as ET
import os
class Database(object):
def __init__(self, path):
self.__dbpath = path## Path to the database
self.load()
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a
LookupError
##and empty xml file
from StringIO import StringIO
from traceback import print_exc
trace = StringIO()
try:
print 5
self.commit()
print 7
except Exception:
print_exc(100, trace)
print trace.getvalue()
def load(self):
if os.path.exists(self.__dbpath):
self.root = ET.parse(self.__dbpath).getroot()
else:
self.root = ET.Element(root)
def commit(self):
ET.ElementTree(self.root).write(self.__dbpath)
db = Database('db.xml')
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
On Mon, Apr 20, 2009 at 10:46 AM, Matt hellzfury+pyt...@gmail.com wrote:
Running this interactively, if you finish off with 'del db', it exits fine
and creates a skeleton xml file called 'db.xml' with text 'root /'.
However, if you instead CTRL-D, it throws at exception while quitting and
then leaves an empty 'db.xml' which won't work. Can anyone here help me
figure out why this is?
Stuff I've done:
I've traced this down to the self.commit() call in __del__. The stacktrace
and a few print statements injected into xml.etree leads me to the call
'root'.encode('us-ascii') throwing a LookupError on line 751 of
xml.etree.ElementTree. This makes no sense to me, since it works fine
normally.
Please show the exact error message and stack trace when you post
errors, it can be very helpful.
What you are doing with __del__ is unusual and not common practice. A
better way to ensure cleanup is to use a close() method which a client
must call, or to use a context manager and 'with' statement.
I think the reason your code is failing is because some module needed
by the encode() call has already been unloaded before your __del__()
method is called.
Thank you very much. Any and all help or pointers are appreciated.
If you defined a close() method, you could write client code like this:
from contextlib import closing
with closing(Database('db.xml')) as db:
# do something with db
# when this block exits db will be closed
It's also not too hard to make an openDatabase() function so you could write
with (openDatabase('db.xml')) as db:
# etc
though that is not really a beginner challenge. Some notes and further
pointers here:
http://personalpages.tds.net/~kent37/kk/00015.html
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
Sorry about that. Hopefully this is better:
In operator:
def __init__(self, path, saveDB=True, cleanUp=True):
'''Constructor'''
## Calculate filesystem paths
self.WORK_DIR= path + '.tmp'
DB_PATH= path + '.xml'
self.SAVE_DB= saveDB## finish(): Delete unnecessary files created
by run?
self.CLEANUP= cleanUp## finish(): Delete database at end of run?
## Make sure we have a working directory (exception on failed write)
if not os.path.isdir(self.WORK_DIR):
os.mkdir(self.WORK_DIR)
self._db = DB.Database(DB_PATH)
## SOME OTHER ENVIRONMENT SETUP STUFF
def _cleanUpEnvironment(self):
'''Delete temp files created for this run'''
try:
for path,dirs,files in os.walk(self.WORK_DIR, topdown=False):
for f in files:os.unlink(os.path.join(path,f))
for d in dirs:os.rmdir(os.path.join(path,d))
os.rmdir(self.WORK_DIR)
except:
print sys.stderr, 'Could not delete temp files; left at:'
print sys.stderr, self.WORK_DIR
def finish(self):
'''Clean up and finish the run (write out to the database)'''
if self.SAVE_DB:self._db.commit()
if self.CLEANUP:self._cleanUpEnvironment()
def __del__(self):
## FIXME: Known bug:
## del t at command line works properly
## Ctrl-D, when there is no db file present, results in a LookupError
self.finish()
if __name__ == '__main__':
printHelp()
## Provide tab completion to the user
import readline, rlcompleter
readline.parse_and_bind('tab: complete')
t= OperatorClassName(os.path.splitext(__file__)[0])
In database:
def __init__(self, path):
'''Constructor'''
self.__dbpath = path## Path to the database
self.load()
def load(self):
'''Read the database out from the file'''
from xml.parsers.expat import ExpatError
if os.path.exists(self.__dbpath):
## Noticed exceptions: IOError, ExpatError
try:
self.root = ET.parse(self.__dbpath).getroot()
except ExpatError:
raise ExpatError('Invalid XML in ' + self.__dbpath)
else:
self.root = ET.Element(root)
def commit(self):
'''Write the database back to the file'''
## Noticed exceptions: IOError
ET.ElementTree(self.root).write(self.__dbpath)
--
~Matthew Strax-Haber
National Aeronautics and Space Administration
Langley Research Center (LaRC)
Co-op, Safety-Critical Avionics Systems Branch
W: 757-864-7378; C: 561-704-0029
Mail Stop 130
matthew.strax-ha...@nasa.gov
From: Martin Walsh mwa...@mwalsh.org
Date: Mon, 20 Apr 2009 16:05:01 -0500
To: Python Tutor tutor@python.org
Cc: Strax-Haber, Matthew (LARC-D320) matthew.strax-ha...@nasa.gov
Subject: Re: [Tutor] String Encoding problem
Forwarding to the list. Matt, perhaps you can repost in plain text, my
mail client seems to have mangled your source ...
Strax-Haber, Matthew (LARC-D320) wrote:
*From: *Martin Walsh mwa...@mwalsh.org
The environment available to __del__ methods during program termination
is wonky, and apparently not very consistent either. I can't say that I
completely understand it myself, perhaps someone else can provide a
better explanation for both of us, but some of the causes are described
in the documentation:
http://docs.python.org/reference/datamodel.html#object.__del__
What is your rationale for using __del__? Are you trying to force a
'commit()' call on Database instances when your program terminates -- in
the case of an unhandled exception, for example?
Perhaps I oversimplified a bit. In my actual code, there is a database
class and an operator class. The actual structure is this:
In operator:
def __init__(self, path, saveDB=True, cleanUp=True):
'''Constructor'''## Calculate filesystem paths
self.WORK_DIR= path + '.tmp'DB_PATH= path
+ '.xml'self.SAVE_DB= saveDB## finish(): Delete
unnecessary files created by run?self.CLEANUP= cleanUp##
finish(): Delete database at end of run?## Make sure we
have a working directory (exception on failed write)if not
os.path.isdir(self.WORK_DIR):os.mkdir(self.WORK_DIR)
self._db = DB.Database(DB_PATH)
## SOME OTHER ENVIRONMENT SETUP STUFF
def _cleanUpEnvironment(self): try:## Delete
temp files created for this runfor path,dirs,files in
os.walk(self.WORK_DIR, topdown=False):for f in files:
os.unlink(os.path.join(path,f))for d in dirs:
os.rmdir(os.path.join(path,d))os.rmdir(self.WORK_DIR)
except:print sys.stderr, 'Could not delete temp
files; left at:'print sys.stderr, self.WORK_DIRdef
finish(self):'''Clean up and finish the run (write out to
the database)'''if self.SAVE_DB
Re: [Tutor] String Encoding problem
I've solved the problem by passing on the work of deciding when to commit to client code. This isn't ideal but it will do what is necessary and unfortunately I don't have any more time to dedicate to this. I hate not being able to find a reasonable workaround :/. -- ~Matthew Strax-Haber National Aeronautics and Space Administration Langley Research Center (LaRC) Co-op, Safety-Critical Avionics Systems Branch W: 757-864-7378; C: 561-704-0029 Mail Stop 130 matthew.strax-ha...@nasa.gov From: Kent Johnson ken...@tds.net Date: Mon, 20 Apr 2009 16:55:16 -0500 To: Strax-Haber, Matthew (LARC-D320) matthew.strax-ha...@nasa.gov Cc: Python Tutor tutor@python.org Subject: Re: [Tutor] String Encoding problem Can you give us a simple description of what you are trying to do? And if you can post in plain text instead of HTML that would be helpful. Maybe this will give you some ideas - you can trap the control-D and do your cleanup: http://openbookproject.net/pybiblio/tips/wilson/simpleExceptions.php Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
