Aleksey Plekhanov created IGNITE-14677:
------------------------------------------

             Summary: Calcite engine. Add 32-bit Unicode characters support
                 Key: IGNITE-14677
                 URL: https://issues.apache.org/jira/browse/IGNITE-14677
             Project: Ignite
          Issue Type: Improvement
            Reporter: Aleksey Plekhanov


Currently, all SQL string length-dependent functions ({{length}}, {{substr}}, 
{{right}}, {{left}}, {{lpad}}, {{rpad}}, etc) work correctly only with up to 
16-bit Unicode characters. These functions mapped to java {{String}} class 
functions (length, substring, etc), which operates with an array of 16-bit 
{{char}}'s. For example, the result of {{"🦆".length()}} will be 2, and the 
result of {{SELECT CHAR_LENGTH('🦆}}') will be 2. 

Java {{String}} class has method {{offsetByCodePoints}} which can be used to 
implement functions that correctly work with 32-bit characters, but there will 
be a performance drop if this method will be used for all strings. Perhaps we 
need dedicated functions with 32-bit characters support.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to