Re: [Tutor] String Encoding problem
I've solved the problem by passing on the work of deciding when to commit to client code. This isn't ideal but it will do what is necessary and unfortunately I don't have any more time to dedicate to this. I hate not being able to find a reasonable workaround :/. -- ~Matthew Strax-Haber National Aeronautics and Space Administration Langley Research Center (LaRC) Co-op, Safety-Critical Avionics Systems Branch W: 757-864-7378; C: 561-704-0029 Mail Stop 130 matthew.strax-ha...@nasa.gov From: Kent Johnson Date: Mon, 20 Apr 2009 16:55:16 -0500 To: "Strax-Haber, Matthew (LARC-D320)" Cc: Python Tutor Subject: Re: [Tutor] String Encoding problem Can you give us a simple description of what you are trying to do? And if you can post in plain text instead of HTML that would be helpful. Maybe this will give you some ideas - you can trap the control-D and do your cleanup: http://openbookproject.net/pybiblio/tips/wilson/simpleExceptions.php Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
Can you give us a simple description of what you are trying to do? And if you can post in plain text instead of HTML that would be helpful. Maybe this will give you some ideas - you can trap the control-D and do your cleanup: http://openbookproject.net/pybiblio/tips/wilson/simpleExceptions.php Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
Sorry about that. Hopefully this is better:
In operator:
def __init__(self, path, saveDB=True, cleanUp=True):
'''Constructor'''
## Calculate filesystem paths
self.WORK_DIR= path + '.tmp'
DB_PATH= path + '.xml'
self.SAVE_DB= saveDB## finish(): Delete unnecessary files created
by run?
self.CLEANUP= cleanUp## finish(): Delete database at end of run?
## Make sure we have a working directory (exception on failed write)
if not os.path.isdir(self.WORK_DIR):
os.mkdir(self.WORK_DIR)
self._db = DB.Database(DB_PATH)
## SOME OTHER ENVIRONMENT SETUP STUFF
def _cleanUpEnvironment(self):
'''Delete temp files created for this run'''
try:
for path,dirs,files in os.walk(self.WORK_DIR, topdown=False):
for f in files:os.unlink(os.path.join(path,f))
for d in dirs:os.rmdir(os.path.join(path,d))
os.rmdir(self.WORK_DIR)
except:
print >>sys.stderr, 'Could not delete temp files; left at:'
print >>sys.stderr, self.WORK_DIR
def finish(self):
'''Clean up and finish the run (write out to the database)'''
if self.SAVE_DB:self._db.commit()
if self.CLEANUP:self._cleanUpEnvironment()
def __del__(self):
## FIXME: Known bug:
## del t at command line works properly
## Ctrl-D, when there is no db file present, results in a LookupError
self.finish()
if __name__ == '__main__':
printHelp()
## Provide tab completion to the user
import readline, rlcompleter
readline.parse_and_bind('tab: complete')
t= OperatorClassName(os.path.splitext(__file__)[0])
In database:
def __init__(self, path):
'''Constructor'''
self.__dbpath = path## Path to the database
self.load()
def load(self):
'''Read the database out from the file'''
from xml.parsers.expat import ExpatError
if os.path.exists(self.__dbpath):
## Noticed exceptions: IOError, ExpatError
try:
self.root = ET.parse(self.__dbpath).getroot()
except ExpatError:
raise ExpatError('Invalid XML in ' + self.__dbpath)
else:
self.root = ET.Element("root")
def commit(self):
'''Write the database back to the file'''
## Noticed exceptions: IOError
ET.ElementTree(self.root).write(self.__dbpath)
--
~Matthew Strax-Haber
National Aeronautics and Space Administration
Langley Research Center (LaRC)
Co-op, Safety-Critical Avionics Systems Branch
W: 757-864-7378; C: 561-704-0029
Mail Stop 130
matthew.strax-ha...@nasa.gov
From: Martin Walsh
Date: Mon, 20 Apr 2009 16:05:01 -0500
To: Python Tutor
Cc: "Strax-Haber, Matthew (LARC-D320)"
Subject: Re: [Tutor] String Encoding problem
Forwarding to the list. Matt, perhaps you can repost in plain text, my
mail client seems to have mangled your source ...
Strax-Haber, Matthew (LARC-D320) wrote:
>> *From: *Martin Walsh
>>
>> The environment available to __del__ methods during program termination
>> is wonky, and apparently not very consistent either. I can't say that I
>> completely understand it myself, perhaps someone else can provide a
>> better explanation for both of us, but some of the causes are described
>> in the documentation:
>>
>> http://docs.python.org/reference/datamodel.html#object.__del__
>>
>> What is your rationale for using __del__? Are you trying to force a
>> 'commit()' call on Database instances when your program terminates -- in
>> the case of an unhandled exception, for example?
>
> Perhaps I oversimplified a bit. In my actual code, there is a database
> class and an operator class. The actual structure is this:
>
> In operator:
> def __init__(self, path, saveDB=True, cleanUp=True):
>'''Constructor'''## Calculate filesystem paths
>self.WORK_DIR= path + '.tmp'DB_PATH= path
> + '.xml'self.SAVE_DB= saveDB## finish(): Delete
> unnecessary files created by run?self.CLEANUP= cleanUp##
> finish(): Delete database at end of run?## Make sure we
> have a working directory (exception on failed write)if not
> os.path.isdir(self.WORK_DIR):os.mkdir(self.WORK_DIR)
>
>self._db = DB.Database(DB_PATH)
> ## SOME OTHER ENVIRONMENT SETUP STUFF
> def _cleanUpEnvironment(self): try:## Delete
> temp files created for this runfor path,dirs,files in
> os.walk(
Re: [Tutor] String Encoding problem
Forwarding to the list. Matt, perhaps you can repost in plain text, my
mail client seems to have mangled your source ...
Strax-Haber, Matthew (LARC-D320) wrote:
>> *From: *Martin Walsh
>>
>> The environment available to __del__ methods during program termination
>> is wonky, and apparently not very consistent either. I can't say that I
>> completely understand it myself, perhaps someone else can provide a
>> better explanation for both of us, but some of the causes are described
>> in the documentation:
>>
>> http://docs.python.org/reference/datamodel.html#object.__del__
>>
>> What is your rationale for using __del__? Are you trying to force a
>> 'commit()' call on Database instances when your program terminates -- in
>> the case of an unhandled exception, for example?
>
> Perhaps I oversimplified a bit. In my actual code, there is a database
> class and an operator class. The actual structure is this:
>
> In operator:
> def __init__(self, path, saveDB=True, cleanUp=True):
>'''Constructor'''## Calculate filesystem paths
>self.WORK_DIR= path + '.tmp'DB_PATH= path
> + '.xml'self.SAVE_DB= saveDB## finish(): Delete
> unnecessary files created by run?self.CLEANUP= cleanUp##
> finish(): Delete database at end of run?## Make sure we
> have a working directory (exception on failed write)if not
> os.path.isdir(self.WORK_DIR):os.mkdir(self.WORK_DIR)
>
>self._db = DB.Database(DB_PATH)
> ## SOME OTHER ENVIRONMENT SETUP STUFF
> def _cleanUpEnvironment(self): try:## Delete
> temp files created for this runfor path,dirs,files in
> os.walk(self.WORK_DIR, topdown=False):for f in files:
>os.unlink(os.path.join(path,f))for d in dirs:
>os.rmdir(os.path.join(path,d))os.rmdir(self.WORK_DIR)
>except:print >>sys.stderr, 'Could not delete temp
> files; left at:'print >>sys.stderr, self.WORK_DIRdef
> finish(self):'''Clean up and finish the run (write out to
> the database)'''if self.SAVE_DB:self._db.commit()
>if self.CLEANUP:self._cleanUpEnvironment()
> def __del__(self):## FIXME: Known bug:## del t at
> command line works properly## Ctrl-D, when there is no db file
> present, results in a LookupErrorself.finish()
> if __name__ == '__main__':
> printHelp()
> ## Provide tab completion to the userimport readline,
> rlcompleterreadline.parse_and_bind('tab: complete')
> t= OperatorClassName(os.path.splitext(__file__)[0])
>
> In database:
> def commit(self):'''Write the database back to the file'''
>## Noticed exceptions: IOError
>ET.ElementTree(self.root).write(self.__dbpath)
>
> The operator also has a series of functions that collect data and enter
> it into the database. Here’s the usage pattern I want to have:
>
> User calls Operator.py. User runs a bunch of functions of the form
> t.(). When the user quits, the database will save or not
> save depending on the status of t.SAVE_DB (which some functions in the
> operator will programmatically change).
>
>>
>> HTH,
>> Marty
>>
>>>
>>> Thank you very much. Any and all help or pointers are appreciated.
>>>
>>> ~Matt
>>>
>>> db.py ###
>>> from xml.etree import ElementTree as ET
>>> import os
>>>
>>> class Database(object):
>>> def __init__(self, path):
>>> self.__dbpath = path## Path to the database
>>> self.load()
>>> def __del__(self):
>>> ## FIXME: Known bug:
>>> ## del db at command line works properly
>>> ## Ctrl-D, when there is no db file present, results in a
>>> LookupError
>>> ##and empty xml file
>>> from StringIO import StringIO
>>> from traceback import print_exc
>>> trace = StringIO()
>>> try:
>>> print 5
>>> self.commit()
>>> print 7
>>> except Exception:
>>> print_exc(100, trace)
>>> print trace.getvalue()
>>> def load(self):
>>> if os.path.exists(self.__dbpath):
>>> self.root = ET.parse(self.__dbpath).getroot()
>>> else:
>>> self.root = ET.Element("root")
>>> def commit(self):
>>> ET.ElementTree(self.root).write(self.__dbpath)
>>> db = Database('db.xml')
>>
>> ___
>> Tutor maillist - Tutor@python.org
>> http://mail.python.org/mailman/listinfo/tutor
>
> --
> ~Matthew Strax-Haber
> National Aeronautics and Space Administration
> Langley Research Center (LaRC)
> Co-op, Safety-Critical Avionics Systems Branch
> W: 757-864-7378; C: 561-704-0029
> Mail Stop 130
> matthew.strax-ha...@nasa.gov
___
Tutor maillist - Tutor@python.org
http://mail.pyt
Re: [Tutor] String Encoding problem
On Mon, Apr 20, 2009 at 10:46 AM, Matt wrote:
> Running this interactively, if you finish off with 'del db', it exits fine
> and creates a skeleton xml file called 'db.xml' with text ''.
> However, if you instead CTRL-D, it throws at exception while quitting and
> then leaves an empty 'db.xml' which won't work. Can anyone here help me
> figure out why this is?
>
> Stuff I've done:
> I've traced this down to the self.commit() call in __del__. The stacktrace
> and a few print statements injected into xml.etree leads me to the call
> 'root'.encode('us-ascii') throwing a LookupError on line 751 of
> xml.etree.ElementTree. This makes no sense to me, since it works fine
> normally.
Please show the exact error message and stack trace when you post
errors, it can be very helpful.
What you are doing with __del__ is unusual and not common practice. A
better way to ensure cleanup is to use a close() method which a client
must call, or to use a context manager and 'with' statement.
I think the reason your code is failing is because some module needed
by the encode() call has already been unloaded before your __del__()
method is called.
>
> Thank you very much. Any and all help or pointers are appreciated.
If you defined a close() method, you could write client code like this:
from contextlib import closing
with closing(Database('db.xml')) as db:
# do something with db
# when this block exits db will be closed
It's also not too hard to make an openDatabase() function so you could write
with (openDatabase('db.xml')) as db:
# etc
though that is not really a beginner challenge. Some notes and further
pointers here:
http://personalpages.tds.net/~kent37/kk/00015.html
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
Matt wrote:
> Hey everyone,
>
> I'm hoping someone here can help me solve an odd problem (bug?). I'm
> having trouble with string encoding, object deletion, and the xml.etree
> library. If this isn't the right list to be posting this question,
> please let me know. I'm new to Python and don't know of any other "help
> me" Python mailing lists. I have tried debugging this ad-infinitem.
> Anyway, at the bottom of this e-mail you will find the code of a python
> file. This is a gross over-simplification of my code, with little
> exception handling so that the errors are obvious.
>
> Running this interactively, if you finish off with 'del db', it exits
> fine and creates a skeleton xml file called 'db.xml' with text ' />'. However, if you instead CTRL-D, it throws at exception while
> quitting and then leaves an empty 'db.xml' which won't work. Can anyone
> here help me figure out why this is?
>
> Stuff I've done:
> I've traced this down to the self.commit() call in __del__. The
> stacktrace and a few print statements injected into xml.etree leads me
> to the call 'root'.encode('us-ascii') throwing a LookupError on line 751
> of xml.etree.ElementTree. This makes no sense to me, since it works fine
> normally.
The environment available to __del__ methods during program termination
is wonky, and apparently not very consistent either. I can't say that I
completely understand it myself, perhaps someone else can provide a
better explanation for both of us, but some of the causes are described
in the documentation:
http://docs.python.org/reference/datamodel.html#object.__del__
What is your rationale for using __del__? Are you trying to force a
'commit()' call on Database instances when your program terminates -- in
the case of an unhandled exception, for example?
HTH,
Marty
>
> Thank you very much. Any and all help or pointers are appreciated.
>
> ~Matt
>
> db.py ###
> from xml.etree import ElementTree as ET
> import os
>
> class Database(object):
> def __init__(self, path):
> self.__dbpath = path## Path to the database
> self.load()
> def __del__(self):
> ## FIXME: Known bug:
> ## del db at command line works properly
> ## Ctrl-D, when there is no db file present, results in a
> LookupError
> ##and empty xml file
> from StringIO import StringIO
> from traceback import print_exc
> trace = StringIO()
> try:
> print 5
> self.commit()
> print 7
> except Exception:
> print_exc(100, trace)
> print trace.getvalue()
> def load(self):
> if os.path.exists(self.__dbpath):
> self.root = ET.parse(self.__dbpath).getroot()
> else:
> self.root = ET.Element("root")
> def commit(self):
> ET.ElementTree(self.root).write(self.__dbpath)
> db = Database('db.xml')
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Encoding problem
> From: spir
> Date: Mon, 20 Apr 2009 12:22:59 -0500
> To: Python Tutor
> Subject: Re: [Tutor] String Encoding problem
>
> Le Mon, 20 Apr 2009 10:46:47 -0400,
> Matt s'exprima ainsi:
>
>> Hey everyone,
>>
>> I'm hoping someone here can help me solve an odd problem (bug?). I'm having
>> trouble with string encoding, object deletion, and the xml.etree library. If
>> this isn't the right list to be posting this question, please let me know.
>> I'm new to Python and don't know of any other "help me" Python mailing
>> lists. I have tried debugging this ad-infinitem. Anyway, at the bottom of
>> this e-mail you will find the code of a python file. This is a gross
>> over-simplification of my code, with little exception handling so that the
>> errors are obvious.
>>
>> Running this interactively, if you finish off with 'del db', it exits fine
>> and creates a skeleton xml file called 'db.xml' with text ''.
>> However, if you instead CTRL-D, it throws at exception while quitting and
>> then leaves an empty 'db.xml' which won't work. Can anyone here help me
>> figure out why this is?
>>
>> Stuff I've done:
>> I've traced this down to the self.commit() call in __del__. The stacktrace
>> and a few print statements injected into xml.etree leads me to the call
>> 'root'.encode('us-ascii') throwing a LookupError on line 751 of
>> xml.etree.ElementTree. This makes no sense to me, since it works fine
>> normally.
>>
>> Thank you very much. Any and all help or pointers are appreciated.
>>
>> ~Matt
>>
>> db.py ###
>> from xml.etree import ElementTree as ET
>> import os
>>
>> class Database(object):
>> def __init__(self, path):
>> self.__dbpath = path## Path to the database
>> self.load()
>> def __del__(self):
>> ## FIXME: Known bug:
>> ## del db at command line works properly
>> ## Ctrl-D, when there is no db file present, results in a
>> LookupError
>> ##and empty xml file
>> from StringIO import StringIO
>> from traceback import print_exc
>> trace = StringIO()
>> try:
>> print 5
>> self.commit()
>> print 7
>> except Exception:
>> print_exc(100, trace)
>> print trace.getvalue()
>> def load(self):
>> if os.path.exists(self.__dbpath):
>> self.root = ET.parse(self.__dbpath).getroot()
>> else:
>> self.root = ET.Element("root")
>> def commit(self):
>> ET.ElementTree(self.root).write(self.__dbpath)
>> db = Database('db.xml')
>
> Actually, it all runs well for me -- after the following modification:
>
> def __del__(self):
> ## FIXME: Known bug:
> ## del db at command line works properly
> ## Ctrl-D, when there is no db file present, results in a LookupError
> ##and empty xml file
> try:
> print 5
> self.commit()
> print 7
> except Exception:
> raise
I must be missing something I run the following code (in DB.py) without any
other files in the current directory:
from xml.etree import ElementTree as ET import os class Database(object):
def __init__(self, path):self.dbpath = path## Path to the database
self.load()def __del__(self):try:print 5
self.commit()print 7except Exception:raise
def load(self):if os.path.exists(self.dbpath):self.root =
ET.parse(self.dbpath).getroot()else:self.root =
ET.Element("root")def commit(self):
ET.ElementTree(self.root).write(self.dbpath) db = Database('db.xml')
Output:
5 Exception LookupError: LookupError('unknown encoding: us-ascii',) in > ignored
If you're not getting the same output, please let me know what your environment
is. Perhaps this is an implementation difference across platforms.
>
> Notes:
> * I don't know for what reason you needed such a complicated traceback
> construct.
That was only to demonstrate the error. Without that, you see a LookupError
without any trace.
> * Before I did this modif, I indeed had a weird exception about stringIO.
Top-level imports are not consistently available in __del__. That shouldn't be
necessary with the code I have
Re: [Tutor] String Encoding problem
Le Mon, 20 Apr 2009 10:46:47 -0400,
Matt s'exprima ainsi:
> Hey everyone,
>
> I'm hoping someone here can help me solve an odd problem (bug?). I'm having
> trouble with string encoding, object deletion, and the xml.etree library. If
> this isn't the right list to be posting this question, please let me know.
> I'm new to Python and don't know of any other "help me" Python mailing
> lists. I have tried debugging this ad-infinitem. Anyway, at the bottom of
> this e-mail you will find the code of a python file. This is a gross
> over-simplification of my code, with little exception handling so that the
> errors are obvious.
>
> Running this interactively, if you finish off with 'del db', it exits fine
> and creates a skeleton xml file called 'db.xml' with text ''.
> However, if you instead CTRL-D, it throws at exception while quitting and
> then leaves an empty 'db.xml' which won't work. Can anyone here help me
> figure out why this is?
>
> Stuff I've done:
> I've traced this down to the self.commit() call in __del__. The stacktrace
> and a few print statements injected into xml.etree leads me to the call
> 'root'.encode('us-ascii') throwing a LookupError on line 751 of
> xml.etree.ElementTree. This makes no sense to me, since it works fine
> normally.
>
> Thank you very much. Any and all help or pointers are appreciated.
>
> ~Matt
>
> db.py ###
> from xml.etree import ElementTree as ET
> import os
>
> class Database(object):
> def __init__(self, path):
> self.__dbpath = path## Path to the database
> self.load()
> def __del__(self):
> ## FIXME: Known bug:
> ## del db at command line works properly
> ## Ctrl-D, when there is no db file present, results in a
> LookupError
> ##and empty xml file
> from StringIO import StringIO
> from traceback import print_exc
> trace = StringIO()
> try:
> print 5
> self.commit()
> print 7
> except Exception:
> print_exc(100, trace)
> print trace.getvalue()
> def load(self):
> if os.path.exists(self.__dbpath):
> self.root = ET.parse(self.__dbpath).getroot()
> else:
> self.root = ET.Element("root")
> def commit(self):
> ET.ElementTree(self.root).write(self.__dbpath)
> db = Database('db.xml')
Actually, it all runs well for me -- after the following modification:
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a LookupError
##and empty xml file
try:
print 5
self.commit()
print 7
except Exception:
raise
Notes:
* I don't know for what reason you needed such a complicated traceback
construct.
* Before I did this modif, I indeed had a weird exception about stringIO.
* __del__() seems to do the contrary: it writes back to file through commit()???
* "del db" works fine, anyway
* When I run without any bd.xml, it properly creates one with text "".
* When I run with an ampty db.xml, I have the following exception message:
Traceback (most recent call last):
File "xmlTree.py", line 29, in
db = Database('db.xml')
File "xmlTree.py", line 10, in __init__
self.load()
File "xmlTree.py", line 24, in load
self.root = ET.parse(self.__dbpath).getroot()
File "/usr/lib/python2.5/xml/etree/ElementTree.py", line 862, in parse
tree.parse(source, parser)
File "/usr/lib/python2.5/xml/etree/ElementTree.py", line 587, in parse
self._root = parser.close()
File "/usr/lib/python2.5/xml/etree/ElementTree.py", line 1254, in close
self._parser.Parse("", 1) # end of data
xml.parsers.expat.ExpatError: no element found: line 2, column 0
5
Exception exceptions.AttributeError: AttributeError("'Database' object has no
attribute 'root'",) in > ignored
--
la vita e estrany
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] String Encoding problem
Hey everyone,
I'm hoping someone here can help me solve an odd problem (bug?). I'm having
trouble with string encoding, object deletion, and the xml.etree library. If
this isn't the right list to be posting this question, please let me know.
I'm new to Python and don't know of any other "help me" Python mailing
lists. I have tried debugging this ad-infinitem. Anyway, at the bottom of
this e-mail you will find the code of a python file. This is a gross
over-simplification of my code, with little exception handling so that the
errors are obvious.
Running this interactively, if you finish off with 'del db', it exits fine
and creates a skeleton xml file called 'db.xml' with text ''.
However, if you instead CTRL-D, it throws at exception while quitting and
then leaves an empty 'db.xml' which won't work. Can anyone here help me
figure out why this is?
Stuff I've done:
I've traced this down to the self.commit() call in __del__. The stacktrace
and a few print statements injected into xml.etree leads me to the call
'root'.encode('us-ascii') throwing a LookupError on line 751 of
xml.etree.ElementTree. This makes no sense to me, since it works fine
normally.
Thank you very much. Any and all help or pointers are appreciated.
~Matt
db.py ###
from xml.etree import ElementTree as ET
import os
class Database(object):
def __init__(self, path):
self.__dbpath = path## Path to the database
self.load()
def __del__(self):
## FIXME: Known bug:
## del db at command line works properly
## Ctrl-D, when there is no db file present, results in a
LookupError
##and empty xml file
from StringIO import StringIO
from traceback import print_exc
trace = StringIO()
try:
print 5
self.commit()
print 7
except Exception:
print_exc(100, trace)
print trace.getvalue()
def load(self):
if os.path.exists(self.__dbpath):
self.root = ET.parse(self.__dbpath).getroot()
else:
self.root = ET.Element("root")
def commit(self):
ET.ElementTree(self.root).write(self.__dbpath)
db = Database('db.xml')
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
