https://github.com/jljusten/edk2.git utf8-v4
v4:
* Reject characters in the range 0xd800 - 0xdfff since they are
reserved for UTF-16 Surrogate Pairs. (lersek)
* Add more tests to verify the Surrogate Pairs range is rejected, and
that UTF-8 BOM is allowed. (lersek)
v3:
* v2 fixed the USC-2 issue with UTF-16 file by 'accident'. Now this
is done in separate patches. (Patches 3 & 4)
* Validate entire file by loading the entire contents (mdkinney)
* Add stub version of ucs-2 codec to verify unicode file contents are
valid USC-2 characters.
v2:
* Drop .utf8 extension. Use .uni file for UTF-8 data (mdkinney)
The UTF-16 .uni files are fairly annoying to work with:
* They must be checked in as 'binary' files
* It is difficult to produce a diff of changes
* UTF-8 is more likely to be supported by text editors
This series allows .uni files to contain UTF-8 (or, as before, UTF-16)
string data. If the UTF-16 LE or BE BOM is found, then the file is
read as UTF-16. Otherwise, it is treated as UTF-8.
Jordan Justen (10):
BaseTools/Tests: Always add BaseTools source to import path
BaseTools/EdkLogger: Support unit tests with a SILENT log level
BaseTools/Tests: Add unit test for AutoGen.UniClassObject
BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni
files
BaseTools/Tests: Verify unsupported UTF-16 are rejected
BaseTools/UniClassObject: Support UTF-8 string data in .uni files
BaseTools/Tests: Verify 32-bit UTF-8 chars are rejected
BaseTools/Tests: Verify unsupported UTF-8 data is rejected
BaseTools/Tests: Verify supported UTF-8 data is allowed
OvmfPkg/PlatformDxe: Convert Platform.uni to UTF-8
BaseTools/Source/Python/AutoGen/UniClassObject.py | 91 ++++++++++-
BaseTools/Source/Python/Common/EdkLogger.py | 9 +-
BaseTools/Tests/CheckUnicodeSourceFiles.py | 181 ++++++++++++++++++++++
BaseTools/Tests/PythonToolsTests.py | 4 +-
BaseTools/Tests/RunTests.py | 2 -
BaseTools/Tests/TestTools.py | 9 +-
OvmfPkg/PlatformDxe/Platform.uni | Bin 3298 -> 1648 bytes
7 files changed, 289 insertions(+), 7 deletions(-)
create mode 100644 BaseTools/Tests/CheckUnicodeSourceFiles.py
--
2.1.4
------------------------------------------------------------------------------
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/edk2-devel
