variation selectors are within the subset of characters that should never be permitted in programming identifiers; they could cause surprizing results such as adding new APIs or backdoors that would not be detected by code reviewers looking at the code.
But you allow them in the language, the first thing you'll need to integrate in your project is a source code scanner that will work on detecting unsafe characters (including checking the list of confusables, and enforcing the normalization of that source code before compiling it, as text editors may break these normalizations unexpectedly). Such tool should run in a routine, just like there are tools that perform reindentation of code and enforce some common conventions for its presentation; in order to ease exploration/searches in the source code, review, and facilitate use of regexps in editors as well. There are various tools that will also inspect how well the code is documented or if documentation os missing about publicly exposed variables and API and try to infer dependencies. Such tools fall in the same categories as the old "lint" tool for C (almost deprecated the way it was since now most of its rules are integrated in the language itself and by compilers to ensure type safety). The risk however is higher in untyped or weakly typed languages like Javascript/ECMAScript where all objects can be surcharged freely, that confusing identifiers could create unseen security risks. Note that identifiers are not just within programming languages; they exist as well on other types of APIs (and notably within web APIs within protocols transmitting data such as encoded web forms, even if these identifiers will be used and exposed isolately in a true language such as JSON or HTML or XML or CSS, possibly also with some escaping mechanisms). Also "identifiers" should be interpreted broadly to include symbolic operators (e.g. operators) if the language or API allow their extension or surcharge or derivations (Unicode identifiers or identifiers used in classic languages like HML, XML, C/C++, Java, Cobol, Fortran, Ada, PHP, Python... or assembly languages. are more restricted in their allowed repertoire and all other extensions require explicit escaping whose decoding should not weaken the security). Identifiers for data may be very liberal (e.g. if we want to allow toponyms or people names or trademarks) as they will frequently need to use significnat punctuation or symbols as well as spacing or word separation. This is even more critical for work names/titles, pagenames and filenames in an open collection: these identifiers or names should resolve unambiguously to the document or data intended (and generally this implies develping naming conventions and some required classification system to get an accurate invorory of available data; and make it possible to inspect this inventory and detect undesirable/conflicting items). I am then convinced than for such open inventories or collections, normalization should never be an option, it should be enforced and automated as early as possible even if we admit input in non-normalized forms. Any programming language or protocol that considers using a large repertoire from the UCS should seriously look at the specification of security in the Unicode standard and its annexes, and conside what has been made and discussed for maintaining the security of the worldwide DNS within the IDNA. The risks coming from instability of normalizations if you allow unassigned codepoints are real because they they can be easily used by automated tools and human reviewers will not detect these attacks easily without using tools to check these normalizations. Code checkers should immediately alarm about usage of codepoints not assigned in a known version of the UCS; and if they upgrade that version, thay should make sure that all other tools in that chain will check the same version. But some identifiers are not always found in source code but are generated at runtime using dyna,ic language features (dynamic binder libraries or reflection APIs should also perform their own check and will then need to integrate a minmum database of known assigned code points for that UCS version, and this could cause some complications for maintaining compatibility; notably having a version negociation mechanism and integrating the version property of assigned codepoints) 2014-06-09 4:46 GMT+02:00 Norbert Lindenberg <[email protected]> : > It does allow some usage that may surprise code reviewers – for example, > this is a valid Swift program: > > let s = "😄" > let s︀ = "😞" > let ︀ = "😉" > let all = s + s︀ + ︀ > > The value of the constant “all” is "😄😞😉". Or at least it is as long as > mail software doesn’t harm the variation selectors… > > Norbert > > > On Jun 5, 2014, at 9:06 , Mark Davis ☕️ <[email protected]> wrote: > > > I haven't done any analysis, but on first glance it looks like it is > based on > > > > http://www.unicode.org/reports/tr31/#Alternative_Identifier_Syntax > > > > > > Mark > > > > — Il meglio è l’inimico del bene — > > > > > > On Thu, Jun 5, 2014 at 5:46 PM, Jeff Senn <[email protected]> wrote: > > Has anyone figured out whether character sequences that are > non-canonical (de)compositions but could be recomposed to the same result > > are the same identifier or not? > > > > That is: are identifiers merely sequences of characters or intended to > be comparable as “Unicode strings” (under some sort of compatibility rule)? > > > > On Jun 5, 2014, at 11:27 AM, Martin v. Löwis <[email protected]> wrote: > > > > > Am 04.06.14 11:28, schrieb Andre Schappo: > > >> The restrictions seem a little like IDNA2008. Anyone have links to > > >> info giving a detailed explanation/tabulation of allowed and non > > >> allowed Unicode chars for Swift Variable and Constant names? > > > > > > The language reference is at > > > > > > > https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/Swift_Programming_Language/LexicalStructure.html > > > > > > For reference, the definition of identifier-character is (read each > > > line as an alternative) > > > > > > identifier-character → Digit 0 through 9 > > > identifier-character → U+0300–U+036F, U+1DC0–U+1DFF, U+20D0–U+20FF, or > > > U+FE20–U+FE2F > > > identifier-character → identifier-head > > > > > > where identifier-head is > > > > > > identifier-head → Upper- or lowercase letter A through Z > > > identifier-head → U+00A8, U+00AA, U+00AD, U+00AF, U+00B2–U+00B5, or > > > U+00B7–U+00BA > > > identifier-head → U+00BC–U+00BE, U+00C0–U+00D6, U+00D8–U+00F6, or > > > U+00F8–U+00FF > > > identifier-head → U+0100–U+02FF, U+0370–U+167F, U+1681–U+180D, or > > > U+180F–U+1DBF > > > identifier-head → U+1E00–U+1FFF > > > identifier-head → U+200B–U+200D, U+202A–U+202E, U+203F–U+2040, U+2054, > > > or U+2060–U+206F > > > identifier-head → U+2070–U+20CF, U+2100–U+218F, U+2460–U+24FF, or > > > U+2776–U+2793 > > > identifier-head → U+2C00–U+2DFF or U+2E80–U+2FFF > > > identifier-head → U+3004–U+3007, U+3021–U+302F, U+3031–U+303F, or > > > U+3040–U+D7FF > > > identifier-head → U+F900–U+FD3D, U+FD40–U+FDCF, U+FDF0–U+FE1F, or > > > U+FE30–U+FE44 > > > identifier-head → U+FE47–U+FFFD > > > identifier-head → U+10000–U+1FFFD, U+20000–U+2FFFD, U+30000–U+3FFFD, or > > > U+40000–U+4FFFD > > > identifier-head → U+50000–U+5FFFD, U+60000–U+6FFFD, U+70000–U+7FFFD, or > > > U+80000–U+8FFFD > > > identifier-head → U+90000–U+9FFFD, U+A0000–U+AFFFD, U+B0000–U+BFFFD, or > > > U+C0000–U+CFFFD > > > identifier-head → U+D0000–U+DFFFD or U+E0000–U+EFFFD > > > > > > As the construction principle for this list, they say > > > > > > "Identifiers begin with an upper case or lower case letter A through Z, > > > an underscore (_), a noncombining alphanumeric Unicode character in the > > > Basic Multilingual Plane, or a character outside the Basic Multilingual > > > Plan that isn’t in a Private Use Area. After the first character, > digits > > > and combining Unicode characters are also allowed." > > > > > > Regards, > > > Martin > > > _______________________________________________ > > > Unicode mailing list > > > [email protected] > > > http://unicode.org/mailman/listinfo/unicode > > > > > > _______________________________________________ > > Unicode mailing list > > [email protected] > > http://unicode.org/mailman/listinfo/unicode > > > > _______________________________________________ > > Unicode mailing list > > [email protected] > > http://unicode.org/mailman/listinfo/unicode > > > _______________________________________________ > Unicode mailing list > [email protected] > http://unicode.org/mailman/listinfo/unicode >
_______________________________________________ Unicode mailing list [email protected] http://unicode.org/mailman/listinfo/unicode
