A few suggestions
There is a reason why the C standard library function fgetc(FILE*) returns an unsigned int instead of a char, because the constant EOF (end of file) must be outside of the definition area of a char.
Some encodings like Base64 or Quoted-printable use the escape character =, but make sure that you can still encode this escape character in another way.
Another possible encoding would be using a "continue" flag. For example you could use the least significant bit to signal if a stream ends or is continued, this allows you to encode 7 bits per byte and is used for arbitrary length integers or other variable length structures where terminator characters like 0x00 may be part of the data.
Gesendet: Mittwoch, 03. Juli 2019 um 10:49 Uhr
Von: "Philippe Verdy via Unicode" <unicode@unicode.org>
An: "Sławomir Osipiuk" <sosip...@gmail.com>
Cc: "unicode Unicode Discussion" <unicode@unicode.org>
Betreff: Re: Unicode "no-op" Character?
Von: "Philippe Verdy via Unicode" <unicode@unicode.org>
An: "Sławomir Osipiuk" <sosip...@gmail.com>
Cc: "unicode Unicode Discussion" <unicode@unicode.org>
Betreff: Re: Unicode "no-op" Character?
Le mer. 3 juil. 2019 à 06:09, Sławomir Osipiuk <sosip...@gmail.com> a écrit :
I don’t think you understood me at all. I can packetize a string with any character that is guaranteed not to appear in the text.
Your goal is **impossible** to reach with Unicode. Assume sich character is "added" to the UCS, then it can appear in the text. Your goal being that it should be "warrantied" not to be used in any text, means that your "character" cannot be encoded at all. Unicode and ISO **require** that the any proposed character can be used in text without limitation. Logivally it would be rejected becauyse your character would not be usable at all from the start.
So you have no choice: you must use some transport format for your "packeting", jsut like what is used in MIME for emails, in HTTP(S) for streaming, or in internationalized domain names.
For your escaping mechanism you have a very large choice already of characters considered special only for your chosen transport syntax.
Your goal shows a chicken and egg problem. It is not solvable without creating self-contradictions immediately (and if you attempt to add some restriction to avoid the contradiction, then you'll fall on cases where you can no longer transport your message and your protocol will become unusable.
