This is a heads-up that I am about to start fixing bug 1223383 Garbled text with BOM-marked files. https://bugs.launchpad.net/leo-editor/+bug/1223383
**Important**: the bug report discusses several issues regarding Python and utf-16 and utf-32 encodings. Please read this bug report if you have any interest in these constant-width encodings. Fixing this bug will require substantial changes throughout the read/write code in leoAtFile.py: 1. The read logic will call the new (tested) g.stripBOM method to detect the encodings for files that start with a BOM (Byte Order Mark). Any such encodings will set at.bom_encoding. That encoding will take precedence over encodings set any other way. 2. The write logic (at.os) writes to at.output_file, which will now *always* be a g.fileLikeObject. at.os will ensure that *only* unicode strings are written to at.output_file. This will ensure that strings added by the output logic are treated exactly like strings from the user's file. 3. at.replaceTargetFileIfDifferent will convert at.output_file to an encoded string using g.toEncodedString. This will be the *only* call to g.toEncodedString in the entire output logic. This will ensure that at most one BOM will ever be written to an external file, and that the BOM bytes (if present) are the first bytes of the file. 4. at.compareFiles will be changed to use the contents of at.output_file rather than opening a file on the file system. All these changes should simplify and speed up the code, but it's possible other complications will arise. *Warning*: the write code is different for @file, @edit, @nosent and @shadow: subtle errors could creep in. But this work must be done. I have spent about 6 hours working to understand the issues here. It's essential that I complete the work now, while all the details are at hand. I'll do as much testing as I can before upping any code, and I'll warn you when the new code is live. Edward -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/leo-editor. For more options, visit https://groups.google.com/groups/opt_out.
