[
https://issues.apache.org/jira/browse/LUCENE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616271#comment-14616271
]
Dawid Weiss commented on LUCENE-6563:
-------------------------------------
bq. As a matter of fact, a filename is just a sequence of bytes, it's not even
a string.
In the end most things are just a sequence of bytes, Ramkumar :) And seriously,
standard C didn't have any unicode-related utilities for a looong time (because
there was no unicode); strings were/are 0-byte terminated byte regions. The
interpretation of which characters they constitute is a higher-level concept.
bq. LANG=C touch 中国
The question is how does the terminal know how to decode your input above into
an argument (itself being a byte*)... and how did it know what you typed in
(and which glyphs to pick in order to display it)... I'm guessing the terminal
accepts unicode on input then if it sees C locale it blindly passes the input
bytes without any conversion at all. The unicode is very likely UTF-8, which
was specifically designed to be an identity conversion coding page (so that C
"strings" just work with it) and it just happens to be the default filesystem
encoding... That's why it works, it just performs no conversion at all...
It's no magic, really. But trying to understand how and where
character-to-byte(-to-glyph) conversions occur will drive you nuts because
there is no consistency here.
> MockFileSystemTestCase.testURI should be improved to handle cases where
> OS/JVM cannot create non-ASCII filenames
> ----------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-6563
> URL: https://issues.apache.org/jira/browse/LUCENE-6563
> Project: Lucene - Core
> Issue Type: Wish
> Reporter: Christine Poerschke
> Assignee: Dawid Weiss
> Priority: Minor
>
> {{ant test -Dtestcase=TestVerboseFS -Dtests.method=testURI
> -Dtests.file.encoding=UTF-8}} fails (for example) with 'Oracle Corporation
> 1.8.0_45 (64-bit)' when the default {{sun.jnu.encoding}} system property is
> (for example) {{ANSI_X3.4-1968}}
> [details to follow]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
