Kieran Beltran wrote: > Sam / Gavin thank-you. > > So, in the case I am receiving UTF-32 input, I would need to preprocess > (using UTF-32-->UTF-16 algorithim) for characters in the 10000 to 10FFFF > ranges and convert them into surrogate pairs, passing that input to > ANTLRInputStream.
Or just use 'new ANTLRInputStream(inputstream, "UTF-32")' (which converts UTF-32 to UTF-16, at least in the current version of ANTLR). > In my lexer definition, where appropriate, I would define the tokens to > recognize the surrogate pairs for example: > > fragment ARITHMOS: '\uD835\uDD38'; // recognize UTF-32 (0001 D538) arithmos > fragment FINSET: '\uD835\uDD3D'; // recognize UTF-32 (0001 D53D) finite > set > > As indicated this is only for a Java targets. > > Have I got it right? Yes. -- David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en -~----------~----~----~----~------~----~------~--~---
