On Friday, 1 December 2017 at 12:21:22 UTC, A Guy With a Question
wrote:
On Friday, 1 December 2017 at 06:07:07 UTC, Patrick Schluter
wrote:
On Thursday, 30 November 2017 at 19:37:47 UTC, Steven
Schveighoffer wrote:
On 11/30/17 1:20 PM, Patrick Schluter wrote:
[...]
iopipe handles this:
http://schveiguy.github.io/iopipe/iopipe/textpipe/ensureDecodeable.html
It was only to give an example. With UTF-8 people who
implement the low level code in general think about the
multiple codeunits at the buffer boundary. With UTF-16 it's
often forgotten. In UTF-16 there are also 2 other common
pitfalls, that exist also in UTF-8 but are less consciously
acknowledged, overlong encoding and isolated codepoints. So
UTF-16 has the same issues as UTF-8, plus some more,
endianness and size.
Most problems with UTF16 is applicable to UTF8. The only issue
that isn't, is if you are just dealing with ASCII it's a bit of
a waste of space.
That's what I said. UTF-16 and UTF-8 have the same issues, but
UTF-16 has even 2 more: endianness and bloat for ASCII. All 3
encodings have their pluses and minuses, that's why D supports
all 3 but with a preference for utf-8.
