Re: Problem with minidom and special chars in HTML
Horst Gutmann wrote:
I currently have quite a big problem with minidom and special chars
(for example uuml;) in HTML.
Yes. Ignoring the issue of the wrong doctype, minidom is a pure XML
parser and knows nothing of XHTML and its doctype's entities 'uuml' and
the like. Only the built-in entities (amp; etc.) will work.
Unfortunately the parser minidom uses won't read external entities -
including the external subset of the DTD (which is where all the stuff
about what uuml; means is stored). And because minidom does not
support EntityReference nodes, the information that there was an entity
reference there at all gets thrown away as it is replaced with the
empty string. Which is kind of bad.
Possible workarounds:
1. pass minidom a different parser to use, one which supports external
entities and which will parse all the DTD stuff. I don't know if there
is anything suitable available, though...
2. use a DOM implementation with the option to support external
entities. For example, with pxdom, one can use DOM Level 3 LS methods,
or pxdom.parse(f, {'pxdom-external-entities': True}).
However note that reading and parsing an external entity will introduce
significant slowdown, especially in the case of the rather complex
multi-file XHTML DTD. Other possibilities:
3. hack the content on the way into the parser to replace the DOCTYPE
declaration with one including entity definitions in the internal
subset:
!DOCTYPE html PUBLIC ... ... [
!ENTITY uuml #252;
...
]
html...
4. hack the content on the way into the parser to replace entity
references with character references, eg. uuml; - #252;. This is
'safe' for simple documents without an internal subset; charrefs and
entrefs can be used in the same places with the same meaning, except
for some issues in the internal subset.
5. use a DOM implementation that supports EntityReference nodes, such
as pxdom. Entity references with no replacement text (or all entity
references if the DOM Level 3 LS parameter 'entities' is set) will
exist as EntityReference DOM objects instead of being flattened to
text. They can safely be reserialized as uuml; without the
implementation having to know what text they represent.
Entities are a big source of complication and confusion, which I wish
had not made it into XML!
--
Andrew Clover
mailto:[EMAIL PROTECTED]
http://www.doxdesk.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problem with minidom and special chars in HTML
Jarek Zgoda wrote: Horst Gutmann napisa(a): I currently have quite a big problem with minidom and special chars (for example uuml;) in HTML. Let's say I have following input file: -- ?xml version=1.0? !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN http://www.w3.org/TR/html4/strict.dtd; HTML4 is not an XML application. Even if minidom will fetch this DTD and be able to parse character entities, it may not be able to parse the document. Any idea how I could solve this problem? Don't use minidom or convert HTML4 to XHTML and change declaration of doctype. This was just a bad example :-) I get the same problem with XHTML in the doctype. The funny thing here IMO is, that the special chars are simply removed. No warning, no nothing :-( MfG, Horst -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with minidom and special chars in HTML
Horst Gutmann napisa(a): Don't use minidom or convert HTML4 to XHTML and change declaration of doctype. This was just a bad example :-) I get the same problem with XHTML in the doctype. The funny thing here IMO is, that the special chars are simply removed. No warning, no nothing :-( As Fredrik pointed out, it's minidom that cann't fetch DTD from remote location. Download this DTD file to your local machine (it lies at exactly this URI), try changing PUBLIC identifier to SYSTEM and give local path to this file. -- Jarek Zgoda http://jpa.berlios.de/ | http://www.zgodowie.org/ -- http://mail.python.org/mailman/listinfo/python-list
Problem with minidom and special chars in HTML
Hi :-)
I currently have quite a big problem with minidom and special chars (for
example uuml;) in HTML.
Let's say I have following input file:
--
?xml version=1.0?
!DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
body
uuml;
/body
/html
--
And following python script:
--
from xml.dom import minidom
if __name__ == '__main__':
doc = minidom.parse('test2.html')
f = open('test3.html','w+')
f.write(doc.toxml())
f.close()
--
test3.html only has a blank line where should be the uuml; It is simply
removed.
Any idea how I could solve this problem?
MfG, Horst
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problem with minidom and special chars in HTML
Horst Gutmann wrote: I currently have quite a big problem with minidom and special chars (for example uuml;) in HTML. Let's say I have following input file: -- ?xml version=1.0? !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN http://www.w3.org/TR/html4/strict.dtd; html body uuml; /body /html -- test3.html only has a blank line where should be the uuml; It is simply removed. Any idea how I could solve this problem? umm. doesn't that doctype point to an SGML DTD? even if minidom did fetch external DTD's (I don't think it does), it would probably choke on that DTD. running your documents through tidy -asxml -numeric before parsing them as XML might be a good idea... http://tidy.sourceforge.net/ (command-line binaries, library) http://utidylib.berlios.de/ (python bindings) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with minidom and special chars in HTML
Fredrik Lundh wrote: umm. doesn't that doctype point to an SGML DTD? even if minidom did fetch external DTD's (I don't think it does), it would probably choke on that DTD. running your documents through tidy -asxml -numeric before parsing them as XML might be a good idea... http://tidy.sourceforge.net/ (command-line binaries, library) http://utidylib.berlios.de/ (python bindings) /F Thanks, but the problem is, that I can't use the numeric representations of these special chars. I will probably simply play findreplace before feeding the document into minidom and change the output back afterwards :-) MfG, Horst -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with minidom and special chars in HTML
Horst Gutmann napisa(a): I currently have quite a big problem with minidom and special chars (for example uuml;) in HTML. Let's say I have following input file: -- ?xml version=1.0? !DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN http://www.w3.org/TR/html4/strict.dtd; HTML4 is not an XML application. Even if minidom will fetch this DTD and be able to parse character entities, it may not be able to parse the document. Any idea how I could solve this problem? Don't use minidom or convert HTML4 to XHTML and change declaration of doctype. -- Jarek Zgoda http://jpa.berlios.de/ | http://www.zgodowie.org/ -- http://mail.python.org/mailman/listinfo/python-list
