Re: monitoring friendly applications
Thanks a lot. Your suggestions lead me to pypi, (I knew it but didnt remember the exact spelling, and no obvious link from www.python.org), and from there to supervisord, that answers pretty well my problem. Thanks again. Tim Roberts wrote: Imbaud Pierre pierre.imb...@gmail.com wrote: I have A LOT of batch applications to monitor, on linux machines, mostly written in python. I have to know: - which are active, at a given moment? - when did the last run occur? How long did it last? - for some daemons: are they stuck? generally, waiting for i/o, or lost in some C call. ... By any chance, does something like this exist? Would someone be interested with this development? http://www.letmegooglethatforyou.com?q=python+daemon+tools -- http://mail.python.org/mailman/listinfo/python-list
monitoring friendly applications
I have A LOT of batch applications to monitor, on linux machines, mostly written in python. I have to know: - which are active, at a given moment? - when did the last run occur? How long did it last? - for some daemons: are they stuck? generally, waiting for i/o, or lost in some C call. I could do this (I partly did, btw) thru external processes tools. But most processes are python programs, making slight changes is not a problem. Thats why I think a python library could help me here; let us call it mfa (for monitor friendly application); then one call to mfa.in() at start (or at import?), mfa.out() at exit (maybe even on exception?), and possibly mfa.loop(), for loop managed daemons. the library would open a pipe, or a flow (to a file, a centralizing process, a database, a log), and every call make a dated entry. Ideally, the application should not be aware of the implementation, especially the repository being a file or a db. Another aspect of process management can be endorsed by the library: sometimes only one instance of the process should be active, then withdraw if already running. Not a lot of code to write, but careful design, to keep things simple, reliable, only add a low overhead, etc. By any chance, does something like this exist? Would someone be interested with this development? -- http://mail.python.org/mailman/listinfo/python-list
iso 8601, quality criteria
for each library/module or even application, a note in [0:10] in front of every quality criterium. criteria?: completeness robustness how well tested? simplicity documentation maintenance team responsiveness usage: how many developpers picked it up and still use it? how many picked it up but gave it up? The list is an obvious oversimplification, each criterium could be discussed for hours. robustness, for instance, means: how well does it behave when illegal data is fed? A LOT of actual modules (comprising mine (:-) raise obscure exceptions. the iso8601 module is simple enough, easy to install, but fails on legal data. I guess the fix would be useful, but is it maintained? Is it in use? I used xml.dom.minidom, recently. Works fine, but I found the interface awfully complicated. Right or wrong, when I had to write some xml, I wrote my own code: better be simple, although untested, undocumented, etc, than using a module so complicated U never finished the learning curve... So, my questions would be: - how do You, other developpers, cope with this question? - is there such a base, helping to pick up existing modules? - if no, dont U think such an initiative might be useful? - Those who have to deal with iso8601 dates, how do U proceed? Ill have a look to mxdatetime, but is it the right answer? -- http://mail.python.org/mailman/listinfo/python-list
Re: iso 8601, quality criteria
Imbaud Pierre a écrit :
cutnpaste error in this posting, here is the complete message:
Context:
I am writing an application that accesses XMP-coded files. Some
fields contain dates, and comply to iso 8601. I installed the iso8601
python module (http://cheeseshop.python.org/pypi/iso8601), it fails
on the simplest forms:
ipdb parse_date('2005-01-01')
*** TypeError: expected string or buffer
(behaves fine with complete strings, but this one is legal!)
Choose a module
Python interfaces to the world: very few libs or protocols have no
python module interfacing them. But they sometimes have many, and it
aint easy to pick up the right one. And if there is only one, it aint
obvious wether it will fill your quality criteria.
My dream: a table with, for each library/module or even application,
a note in [0:10] in front of every quality criterium. criteria?:
completeness
robustness
how well tested?
simplicity
documentation
maintenance team responsiveness
usage: how many developpers picked it up and still use it? how many
picked it up but gave it up?
The list is an obvious oversimplification, each criterium could be
discussed for hours. robustness, for instance, means: how well does
it behave when illegal data is fed? A LOT of actual modules
(comprising mine (:-) raise obscure exceptions.
the iso8601 module is simple enough, easy to install, but fails on
legal data. I guess the fix would be useful, but is it maintained? Is
it in use?
I used xml.dom.minidom, recently. Works fine, but I found the
interface awfully complicated. Right or wrong, when I had to write
some xml, I wrote my own code: better be simple, although untested,
undocumented, etc, than using a module so complicated U never finished
the learning curve...
So, my questions would be:
- how do You, other developpers, cope with this question?
- is there such a base, helping to pick up existing modules?
- if no, dont U think such an initiative might be useful?
- Those who have to deal with iso8601 dates, how do U proceed?
Ill have a look to mxdatetime, but is it the right answer?
--
http://mail.python.org/mailman/listinfo/python-list
pyYaml community
I began using pyYaml. I found no place where pyYaml users exchange ideas, know-how, etc (as here for python). There is a wiki, a bug tracker, documentation, but such a place (mailing list, newsgroup, forum, or even IRC) is a must (IMHO) to smooth the learning curve. Does someone know better? My concern: yaml allows complex data as keys to dicts. I need tuples as keys, pyYaml turns my yaml sequences into lists, and then yells list cant be used as keys. No way to turn my yaml sequences into tuples? Oh, I found! RTFM, as usual! (the first question sticks) -- http://mail.python.org/mailman/listinfo/python-list
Re: data design
James Stroud a écrit : Szabolcs Nagy wrote: Hurray for yaml! A perfect fit for my need! And a swell tool! Thanks a lot! i warn you against yaml I feel both thanful, and sorry, for your warning. And not convinced yet, but Ill be cautious. it looks nice, but the underlying format is imho too complex (just look at their spec.) complex indeed, but real powerful. Is it not true that: if I used yaml, sticking to what .ini allows, yaml files would be simple? you said you don't want python source because that's too complex for the users. i must say that yaml is not easier to use than python data structures. Easier to read and write, U must agree. Surround strings with quotes is a python requirement, to distinguish them from identifiers. This only makes data input for python somewhat clumsy. Granted, its a new format to learn. But sharing this format with a much wider community than python, aint this worth the effort? (well, if yaml succeeds and spreads...) if you want userfriedly config files then ConfigParser is the way to go. Granted. for END users. I rather target administrators, programmers, integrators: make customization an easy process, and allowing this customization to go much farther than changing simple values, aint this the REAL challenge for new applications? if you want somthing really simple and fast then i'd recommend s- expressions of lisp lisp is more powerful than python. its syntax deterred many programmers, who adopted python, it will deter my targeted customizers. And the process to translate to python structure, I have no idea. involves a python or lisp translater... also here is an identation based xml-like tree/hierarchical data structure syntax: http://www.scottsweeney.com/projects/slip/ Pretty nice, too! James, have a look at this! I've been spending the last 2 days weighing ConfigParser and yaml, with much thought and re-organizing of each file type. The underlying difference is that, conceptually, ini files are an absurdly limited subset of yaml in that ini files are basically limited to a map of a map. U have a point here. For instance, I have a copy_files section of a configuration. In order to know what goes with what you have to resort to gymnastics with the option names [copy_files] files_dir1 = this.file that.file path_dir1 = /some/path files_dir2 = the_other.file yet_another.file path_dir2 = /some/other/path In yaml, it might look thus. copy_files : - files : [this.file, that.file] path : /some/path - files : [the_other.file, yet_another.file] path : /some/other/path Both are readable (though I like equals signs in appearance over colons), but yaml doesn't require a lot of string processing to group the files with the paths. I don't even want to think the coding gymnastics required to split all of the option names and then group those with common suffixes. Now if the config file were for copying only, ini would be okay, because one could just have sections that group paths and dirs: [dir1] files = this.file, that.file path = /some/path [dir2] ... But if you need different kinds of sections, you have outgrown ini. In essence, ini is limited to a single dictionary of dictionaries while yama can express pretty much arbitrary complexity. James, this single formula makes things real clear. As we both work on the subject, maybe we could continue to exchange ideas, and information? Have a look at the link Szabolcs Nagy [EMAIL PROTECTED] gives: http://www.scottsweeney.com/projects/slip/ Ill further dig yaml, with 2 questions: - how do I translate to python? - how do I express and/or enforce rules the data should follow? (avoid the classic: configuration data error raise some obscure exception). Big thanks to Szabolcs Nagy (hungarian, my friend? I love this country), although I seem to disagree, your statements are pretty clear and helpful, and... maybe U are right, and I am a fool... Pierre -- http://mail.python.org/mailman/listinfo/python-list
data design
The applications I write are made of, lets say, algorithms and data. I mean constant data, dicts, tables, etc: to keep algorithms simple, describe what is peculiar, data dependent, as data rather than case statements. These could be called configuration data. The lazy way to do this: have modules that initialize bunches of objects, attributes holding the data: the object is somehow the row of the table, attribute names being the column. This is the way I proceeded up to now. Data input this way are almost configuration data, with 2 big drawbacks: - Only a python programmer can fix the file: this cant be called a configuration file. - Even for the author, these data aint easy to maintain. I feel pretty much ready to change this: - make these data true text data, easier to read and fix. - write the module that will make python objects out of these data: the extra cost should yield ease of use. 2 questions arise: - which kind of text data? - csv: ok for simple attributes, not easy for lists or complex data. - xml: the form wont be easier to read than python code, but an xml editor could be used, and a formal description of what is expected can be used. - how can I make the data-to-object transformation both easy, and able to spot errors in text data? Last, but not least: is there a python lib implementing at least part of this dream? -- http://mail.python.org/mailman/listinfo/python-list
Re: data design
Szabolcs Nagy a écrit : The lazy way to do this: have modules that initialize bunches of objects, attributes holding the data: the object is somehow the row of the table, attribute names being the column. This is the way I proceeded up to now. Data input this way are almost configuration data, with 2 big drawbacks: - Only a python programmer can fix the file: this cant be called a configuration file. - Even for the author, these data aint easy to maintain. I feel pretty much ready to change this: - make these data true text data, easier to read and fix. - write the module that will make python objects out of these data: the extra cost should yield ease of use. 2 questions arise: - which kind of text data? - csv: ok for simple attributes, not easy for lists or complex data. - xml: the form wont be easier to read than python code, but an xml editor could be used, and a formal description of what is expected can be used. - how can I make the data-to-object transformation both easy, and able to spot errors in text data? Last, but not least: is there a python lib implementing at least part of this dream? there is a csv parser and multiple xml parsers in python (eg xml.etree) I used both. both are ok, but only bring a low layer parsing. also there is a ConfigParser module (able to parse .ini like config files) Used this years ago, I had forgotten. Another fine data text format. i personally like the python module as config file the most eg if you need a bunch of key-value pairs or lists of data: * python's syntax is pretty nice (dict, tuples and lists or just key=value) But only python programmer editable! * xml is absolutely out of question * csv is very limited * .ini like config file for more complex stuff is not bad but then you can use .py as well. Thanks a lot for your advices. -- http://mail.python.org/mailman/listinfo/python-list
Re: data design
Larry Bates a écrit : Imbaud Pierre wrote: The applications I write are made of, lets say, algorithms and data. I mean constant data, dicts, tables, etc: to keep algorithms simple, describe what is peculiar, data dependent, as data rather than case statements. These could be called configuration data. The lazy way to do this: have modules that initialize bunches of objects, attributes holding the data: the object is somehow the row of the table, attribute names being the column. This is the way I proceeded up to now. Data input this way are almost configuration data, with 2 big drawbacks: - Only a python programmer can fix the file: this cant be called a configuration file. - Even for the author, these data aint easy to maintain. I feel pretty much ready to change this: - make these data true text data, easier to read and fix. - write the module that will make python objects out of these data: the extra cost should yield ease of use. 2 questions arise: - which kind of text data? - csv: ok for simple attributes, not easy for lists or complex data. - xml: the form wont be easier to read than python code, but an xml editor could be used, and a formal description of what is expected can be used. - how can I make the data-to-object transformation both easy, and able to spot errors in text data? Last, but not least: is there a python lib implementing at least part of this dream? Use the configurations module. It was built to provide a way to parse configuration files that provide configuration data to program. It is VERY fast so the overhead to parse even thousands of lines of config data is extremely small. I use it a LOT and it is very flexible and the format of the files is easy for users/programmers to work with. -Larry Bates U mean configParser? Otherwise be more specific (if U dont mind...) -- http://mail.python.org/mailman/listinfo/python-list
Re: data design
Paddy a écrit : On Jan 30, 2:34 pm, Imbaud Pierre [EMAIL PROTECTED] wrote: The applications I write are made of, lets say, algorithms and data. I mean constant data, dicts, tables, etc: to keep algorithms simple, describe what is peculiar, data dependent, as data rather than case statements. These could be called configuration data. The lazy way to do this: have modules that initialize bunches of objects, attributes holding the data: the object is somehow the row of the table, attribute names being the column. This is the way I proceeded up to now. Data input this way are almost configuration data, with 2 big drawbacks: - Only a python programmer can fix the file: this cant be called a configuration file. - Even for the author, these data aint easy to maintain. I feel pretty much ready to change this: - make these data true text data, easier to read and fix. - write the module that will make python objects out of these data: the extra cost should yield ease of use. 2 questions arise: - which kind of text data? - csv: ok for simple attributes, not easy for lists or complex data. - xml: the form wont be easier to read than python code, but an xml editor could be used, and a formal description of what is expected can be used. - how can I make the data-to-object transformation both easy, and able to spot errors in text data? Last, but not least: is there a python lib implementing at least part of this dream? Google for YAML and JSON formats too. http://www.yaml.org/ http://www.json.org/ -Paddy Hurray for yaml! A perfect fit for my need! And a swell tool! Thanks a lot! -- http://mail.python.org/mailman/listinfo/python-list
module file
I am willing to retrieve the file an imported module came from module.__file__, or inspect.getfile(module) only gives me the relative file name. How do I determine the path? Its obviously possible from python: ipython displays the information (interactively: *module?*). NB: I saw the discussion about the python path, find a .py path, this is different! Its not the script, but the module I want to retrieve! -- http://mail.python.org/mailman/listinfo/python-list
closed: module file
Imbaud Pierre a écrit : I am willing to retrieve the file an imported module came from; module.__file__, or inspect.getfile(module) only gives me the relative file name. How do I determine the path? Its obviously possible from python: ipython displays the information (interactively: *module?*). Python 2.4 on Suse 9.3 (clueless, I guess) Apologies: module.__file__, or inspect.getfile(module) only *gave* me the relative file name, because the module was found in the current directory! Changing the current directory, and having the module be found thru the path, changed the returned path to an absolute directory! Hence the simplest form at hand was used. Thank you, python! -- http://mail.python.org/mailman/listinfo/python-list
module file
I am willing to retrieve the file an imported module came from; module.__file__, or inspect.getfile(module) only gives me the relative file name. How do I determine the path? Its obviously possible from python: ipython displays the information (interactively: *module?*). Python 2.4 on Suse 9.3 (clueless, I guess) -- http://mail.python.org/mailman/listinfo/python-list
closed issue
I submitted a bug, to sourceforge. Was answered (pretty fast) the file I dealt with was the buggy part. I then submitted a bug to the file author, who agreed, and fixed. End of the story. All I could complain about, with the xml.dom library, is how obscure the exception context was: I did violate SOME xml rule, ideally the exception should show the rule, and the faulty piece of data. But I know this has a cost, both runtime cost and developper-s time cost. Imbaud Pierre a écrit : I am using the standard xml library to create another library able to read, and maybe write, xmp files. Then an xml library bug popped out: xml.dom.minidom was unable to parse an xml file that came from an example provided by an official organism.(http://www.iptc.org/IPTC4XMP) The parsed file was somewhat hairy, but I have been able to reproduce the bug with a simplified version, that goes: ?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'? x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 3.0-28, framework 1.6' rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/' rdf:Description rdf:about='uuid:f5b64178-9394-11d9-bb8e-a67e6693b6e9' xmlns:xmpPLUS='XMP Photographic Licensing Universal System (xmpPLUS, http://ns.adobe.com/xap/1.0/PLUS/)' xmpPLUS:CreditLineReqFalse/xmpPLUS:CreditLineReq xmpPLUS:ReuseAllowedFalse/xmpPLUS:ReuseAllowed /rdf:Description /rdf:RDF /x:xmpmeta ?xpacket end='w'? The offending part is the one that goes: xmpPLUS='' it triggers an exception: ValueError: too many values to unpack, in _parse_ns_name. Some debugging showed an obvious mistake in the scanning of the name argument, that goes beyond the closing ' . Im aware I dont give here enough matter to allow full understanding of the bug. But thats not the place for this, and thats not my point. Now my points are: - how do I spot the version of a given library? There is a __version__ attribute of the module, is that it? - How do I access to a given library buglist? Maybe this one is known, about to be fixed, it would then be useless to report it. - How do I report bugs, on a standard lib? - I tried to copy the lib somewhere, put it BEFORE the official lib in the path (that is:sys.path), the stack shown by the traceback still shows the original files being used. Is there a special mechanism bypassing the sys.path search, for standard libs? (I may be wrong on this, it seems hard to believe...) - does someone know a good tool to validate an xml file? btw, my code: from nxml.dom import minidom ... class whatever: def __init__(self, inStream): xmldoc = minidom.parse(inStream) Thanks for any help... -- http://mail.python.org/mailman/listinfo/python-list
Re: xml bug?
Martin v. Löwis a écrit : Imbaud Pierre schrieb: - how do I spot the version of a given library? There is a __version__ attribute of the module, is that it? Contrary to what others have said: for modules included in the standard library (and if using these modules, rather than using PyXML), you should use sys.version_info to identify a version. - How do I access to a given library buglist? Maybe this one is known, about to be fixed, it would then be useless to report it. Others have already pointed you to SF. - How do I report bugs, on a standard lib? Likewise. - I tried to copy the lib somewhere, put it BEFORE the official lib in the path (that is:sys.path), the stack shown by the traceback still shows the original files being used. Is there a special mechanism bypassing the sys.path search, for standard libs? (I may be wrong on this, it seems hard to believe...) Which lib? minidom.py? Well, you are likely importing xml.dom.minidom, not minidom. So adding another minidom.py to a directory in sys.path won't help. Regards, Martin I did import xml! Maybe my mistake came from copying the whole tree from the standard lib: comprising .pyc, .pyo... maybe the .pyc contained references to previous sources? Got rid of these, did reload ALL the modules, then exited/re-entered the interpreter (ipython, btw...), and it eventually accessed the new modules... Btw, I pushed debugging further, the bug seem to stem from C code, hence nothing easy to fix... Ill indeed submit a bug. Thanks for your help! I obviously screamed for help before being helpless, apologies... -- http://mail.python.org/mailman/listinfo/python-list
xml bug?
I am using the standard xml library to create another library able to read, and maybe write, xmp files. Then an xml library bug popped out: xml.dom.minidom was unable to parse an xml file that came from an example provided by an official organism.(http://www.iptc.org/IPTC4XMP) The parsed file was somewhat hairy, but I have been able to reproduce the bug with a simplified version, that goes: ?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'? x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 3.0-28, framework 1.6' rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/' rdf:Description rdf:about='uuid:f5b64178-9394-11d9-bb8e-a67e6693b6e9' xmlns:xmpPLUS='XMP Photographic Licensing Universal System (xmpPLUS, http://ns.adobe.com/xap/1.0/PLUS/)' xmpPLUS:CreditLineReqFalse/xmpPLUS:CreditLineReq xmpPLUS:ReuseAllowedFalse/xmpPLUS:ReuseAllowed /rdf:Description /rdf:RDF /x:xmpmeta ?xpacket end='w'? The offending part is the one that goes: xmpPLUS='' it triggers an exception: ValueError: too many values to unpack, in _parse_ns_name. Some debugging showed an obvious mistake in the scanning of the name argument, that goes beyond the closing ' . Im aware I dont give here enough matter to allow full understanding of the bug. But thats not the place for this, and thats not my point. Now my points are: - how do I spot the version of a given library? There is a __version__ attribute of the module, is that it? - How do I access to a given library buglist? Maybe this one is known, about to be fixed, it would then be useless to report it. - How do I report bugs, on a standard lib? - I tried to copy the lib somewhere, put it BEFORE the official lib in the path (that is:sys.path), the stack shown by the traceback still shows the original files being used. Is there a special mechanism bypassing the sys.path search, for standard libs? (I may be wrong on this, it seems hard to believe...) - does someone know a good tool to validate an xml file? btw, my code: from nxml.dom import minidom ... class whatever: def __init__(self, inStream): xmldoc = minidom.parse(inStream) Thanks for any help... -- http://mail.python.org/mailman/listinfo/python-list
Re: xml bug?
Erik Johnson a écrit : Imbaud Pierre [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Now my points are: - how do I spot the version of a given library? There is a __version__ attribute of the module, is that it? Yes, the module maintainer should be incrementing this version for each new release and so it should properly correspond to the actual revision of code. - How do I access to a given library buglist? Maybe this one is known, about to be fixed, it would then be useless to report it. Not exactly sure, but this is probably a good place to start: http://docs.python.org/modindex.html But python.org was the right entry point, it sent me to the bug tracker: http://sourceforge.net/tracker/?group_id=5470atid=105470 Its a bit short on explanations... And I found unsolved issues, 3 years old! this indexes the modules, not the buglist! - How do I report bugs, on a standard lib? I found this link: http://sourceforge.net/tracker/?group_id=5470atid=105470 Right! Same place to fetch and to submit. Fair. by looking under the help item at www.python.org (an excellent starting place for all sorts of things). - I tried to copy the lib somewhere, put it BEFORE the official lib in the path (that is:sys.path), the stack shown by the traceback still shows the original files being used. Is there a special mechanism bypassing the sys.path search, for standard libs? (I may be wrong on this, it seems hard to believe...) My understanding is sys.path is searched in order. The first entry is usually the empty string, interpreted to mean the current directory. If you modify sys.path to put the directory containing your modified code in front of where the standard library is found, your code should be the one used. That is not the case? I put it in front, as for the unix PATH... - does someone know a good tool to validate an xml file? Typing XML validator into google returns a bunch. I think I would start with the one at w3.org: http://validator.w3.org/ Ill try this. Thanks a lot, my friend! -- http://mail.python.org/mailman/listinfo/python-list
rdf, xmp
I have to add access to some XMP data to an existing python application. XMP is built on RDF, RDF is built on XML. I try to reuse as much of possible of existing code. btw, dont mistake XMP (http://www.adobe.com/products/xmp/) with XMPP (http://www.faqs.org/rfcs/rfc3920.html), backed by PyXMPP (http://pyxmpp.jajcus.net/). XMP is adobe's standard for storing metadata in files (jpg, pdf). Are there people with the same concern out there? It seemed logical to use existing rdf libraries. I found two, RDFLib (http://rdflib.net) and pyrple (http://infomesh.net/pyrple/). My first contact with RDFLib is disappointing: real intricate lib (lot of modules, lot of methods), almost no documentation (an almost empty Epydoc generated documentation frame), puzzling experiments. I guess XMP uses a real tiny subset of RDF possibilities, and maybe RDFLib is fine for ambitious design, but too heavy in this case? Ill now dig a little on pyrple. Feedback on these matters, please? -- http://mail.python.org/mailman/listinfo/python-list
Re: rdf, xmp
Andy Dingley a écrit : Imbaud Pierre wrote: I have to add access to some XMP data to an existing python application. XMP is built on RDF, RDF is built on XML. RDF is _NOT_ built on top of XML. Thinking that it is causes a lot of trouble in the architecture of big RDF projects. RDF is a data model, not a serialisation. The data model is also a graph (more than XML can cope with) and can span multiple documents. It's only RDF/XML that's the serialisation of RDF into XML and good architectures start from thinking about the RDF data model, not this RDF/XML serialisation. Granted, I oversimplified, my statement was misleading. I tried to help unknowledgeable reader understand what it was about. As to RDF handling, then the usual toolset is Jena (in Java) and Redland has a Python binding although Redland is fairly aged now. I'm unfamiliar with XMP and won't have a chance to look at it until Monday. However if XMP is strongly XML like despite claiming to be RDF, then you might find that handling a pure XMP problem is quite easily done with XML tools. This was my wild guess: the data model I deal with (XMP data, I mean) is hardly more than a bunch of key-value pairs - with control for vocabulary, and some typing. Famously RDF/XML is unprocessable with XSLT if it's sophisticated, but quite easy if it's restricted to only a simple XML-like RDF model. XMP could well be similar. Still unclear. Thanks for your help! -- http://mail.python.org/mailman/listinfo/python-list
tempfile.NamedTemporaryFile wont work
On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected. (I found a permanent workaround, so I dont ask for help) I expected to write to a file, and access it thru a shell command. This code, in a loop: tf = tempfile.NamedTemporaryFile() tfName = tf.name #tf.seek(0) # rewind the file tf.write(chunk); tf.flush() print sys.stderr, '%s: %s' % (tfName, ['no', 'yes'][os.path.exists(tfName)]) subprocess.Popen(['strings', tfName]) Symptom: the file does not always exist, after the call to NamedTemporaryFile(). Or at least its not seen by the strings command, or by os.path.exists. I guess the bug is pretty much os dependent, or even filesystem dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the python interface. Or did I miss something? Shame, I didnt even try to check for a python bug tracker. -- http://mail.python.org/mailman/listinfo/python-list
Re: tempfile.NamedTemporaryFile wont work
Steven D'Aprano a écrit : On Sun, 19 Nov 2006 13:18:39 +0100, Bjoern Schliessmann wrote: Imbaud Pierre wrote: tf = tempfile.NamedTemporaryFile() tfName = tf.name [...] print sys.stderr, '%s: %s' % (tfName, ['no', 'yes'][os.path.exists(tfName)]) subprocess.Popen(['strings', tfName]) Just out of curiosity: Why did you assign tf.name to tfname? Hypothetically, if tf.name changed, tfname wouldn't follow since strings are immutable. Well, yes, but if tf.name changed, that won't change the file name on disk either: tf = tempfile.NamedTemporaryFile() tf.name '/tmp/tmpYVV1Ij' os.path.exists(tf.name) True oldname = tf.name tf.name = /tmp/something os.path.exists(tf.name) False os.path.exists(oldname) True I'm guessing that binding tf.name to tfName is a micro-optimization. indeed. And I dont see why tf.name would change. In a very tight loop, name lookups can take considerable time, and one optimization can be to reduce the number of lookups: method = something.method while 1: something.method # needs at least two lookups method # needs a single lookup -- http://mail.python.org/mailman/listinfo/python-list
Re: tempfile.NamedTemporaryFile wont work
Peter Otten a écrit : Steven D'Aprano wrote: On Sun, 19 Nov 2006 13:11:13 +0100, Imbaud Pierre wrote: On suse 9.3, tempfile.NamedTemporaryFile() doesnt work as expected. [snip] Symptom: the file does not always exist, after the call to NamedTemporaryFile(). Or at least its not seen by the strings command, or by os.path.exists. I guess the bug is pretty much os dependent, or even filesystem dependent (Im on reiserfs). Maybe the os is buggy, maybe, somehow, the python interface. Or did I miss something? Shame, I didnt even try to check for a python bug tracker. I can verify this problem occurs on Fedora Core 5 too: import os import sys import tempfile import subprocess def test(n): chunk = ': +++ abcd +++' for i in xrange(n): tf = tempfile.NamedTemporaryFile() tfName = tf.name tf.seek(0) tf.write(str(i) + chunk) tf.flush() if not os.path.exists(tfName): print 'pre-check: %s not there' % tfName subprocess.Popen(['strings', tfName]) if not os.path.exists(tfName): print 'post-check: %s not there' % tfName And here is a typical run, with the boring bits removed for ease of reading: test(30) 0: +++ abcd +++ 1: +++ abcd +++ [ more of the same ] 14: +++ abcd +++ strings: '/tmp/tmpOALbx9': No such file 16: +++ abcd +++ 17: +++ abcd +++ 18: +++ abcd +++ [ more of the same ] 27: +++ abcd +++ strings: /tmp/tmpdc52Nz: No such file or directory 29: +++ abcd +++ Curiouser and curiouser... not only does os.path.exist always report the temp file as existing (at least in my tests), even when strings can't find it, but strings returns different error messages. Is it possible this is a bug in strings? What /you/ are seeing is not a bug, I think. Popen() is asynchronous, therefore you may enter the second iteration -- which implicitly closes the temporary file -- before strings actually tries to access it. Use call() and everything should be fine. Thanks A LOT, works fine, I feel kind of silly; your diagnostic is pretty obvious, afterward... I felt uneasy not closing the Popen, but it worked, so why bother? Its so easy to make ugly code! -- http://mail.python.org/mailman/listinfo/python-list
