>> Tatsuo Ishii <is...@postgresql.org> writes:
>>>> So far as I can see, the only LCPRVn marker code that is actually in
>>>> use right now is 0x9d --- there are no instances of 9a, 9b, or 9c
>>>> that I can find.
>>>>
>>>> I also read in the xemacs internals doc, at
>>>> http://www.xemacs.org/Documentation/21.5/html/internals_26.html#SEC145
>>>> that XEmacs thinks the marker code for private single-byte charsets
>>>> is 0x9e (only) and that for private multi-byte charsets is 0x9f (only);
>>>> moreover they think 0x9a-0x9d are potential future official multibyte
>>>> charset codes. I don't know how we got to the current state of using
>>>> 0x9a-0x9d as private charset markers, but it seems pretty inconsistent
>>>> with XEmacs.
>>
>>> At the time when mule internal code was introduced to PostgreSQL,
>>> xemacs did not have multi encoding capabilty and mule (a patch to
>>> emacs) was the only implementation allowed to use multi encoding. So I
>>> used the specification of mule documented in the URL I wrote.
>>
>> I see. Given that upstream has decided that a simpler definition is
>> more appropriate, is there any reason not to follow their lead, to the
>> extent that we can do so without breaking existing on-disk data?
>
> Please let me spend week end to understand the their latest spec.
This is an intermediate report on the internal multi-byte charset
implementation of emacen. I have read the link Tom showed. Also I made
a quick scan on xemacs-21.4.0 source code, especially
xemacs-21.4.0/src/mule-charset.h. It seems the web document is
essentially a copy of the comments in the file. Also I looked into
other place of xemacs code and I think I can conclude that xeamcs
21.4's multi-byte implementation is based on the doc on the web.
Next I looked into emacs 24.1 source code because I could not find any
doc regarding emacs's(not xemacs's) implementation of internal
multi-byte charset. I found followings in src/charset.h:
/* Leading-code followed by extended leading-code. DIMENSION/COLUMN */
#define EMACS_MULE_LEADING_CODE_PRIVATE_11 0x9A /* 1/1 */
#define EMACS_MULE_LEADING_CODE_PRIVATE_12 0x9B /* 1/2 */
#define EMACS_MULE_LEADING_CODE_PRIVATE_21 0x9C /* 2/2 */
#define EMACS_MULE_LEADING_CODE_PRIVATE_22 0x9D /* 2/2 */
And these are used like this:
/* Read one non-ASCII character from INSTREAM. The character is
encoded in `emacs-mule' and the first byte is already read in
C. */
static int
read_emacs_mule_char (int c, int (*readbyte) (int, Lisp_Object), Lisp_Object
readcharfun)
{
:
:
else if (len == 3)
{
if (buf[0] == EMACS_MULE_LEADING_CODE_PRIVATE_11
|| buf[0] == EMACS_MULE_LEADING_CODE_PRIVATE_12)
{
charset = CHARSET_FROM_ID (emacs_mule_charset[buf[1]]);
code = buf[2] & 0x7F;
}
As far as I can tell, this is exactly the same way how PostgreSQL
handles single private character sets: they consist of 3 bytes, and
leading byte is either 0x9a or 0x9b. Other examples regarding single
byte/multi-byte private charsets can be seen in coding.c.
As far as I can tell, it seems emacs and xemacs employes different
implementations of multi-byte charaset regarding "private"
charsets. Emacs's is same as PostgreSQL, while xemacs is not. I am
contacting to the original Mule author if he knows anything about
this.
BTW, while looking into emacs's source code, I found their charset
definitions are in lisp/international/mule-conf.el. According to the
file several new charsets has been added. Included is the patch to
follow their changes. This makes no changes to current behavior, since
the patch just changes some comments and non supported charsets.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 1bcdfbc..e44749e 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -108,7 +108,7 @@ typedef unsigned int pg_wchar;
#define LC_KOI8_R 0x8b /* Cyrillic KOI8-R */
#define LC_ISO8859_5 0x8c /* ISO8859 Cyrillic */
#define LC_ISO8859_9 0x8d /* ISO8859 Latin 5 (not supported yet) */
-/* #define FREE 0x8e free (unused) */
+#define LC_ISO8859_15 0x8e /* ISO8859 Latin 15 (not supported yet) */
/* #define CONTROL_1 0x8f control characters (unused) */
/* Is a leading byte for "official" single byte encodings? */
@@ -119,14 +119,13 @@ typedef unsigned int pg_wchar;
* 0x9a-0x9d are free. 0x9e and 0x9f are reserved.
*/
#define LC_JISX0208_1978 0x90 /* Japanese Kanji, old JIS (not supported) */
-/* #define FREE 0x90 free (unused) */
#define LC_GB2312_80 0x91 /* Chinese */
#define LC_JISX0208 0x92 /* Japanese Kanji (JIS X 0208) */
#define LC_KS5601 0x93 /* Korean */
#define LC_JISX0212 0x94 /* Japanese Kanji (JIS X 0212) */
#define LC_CNS11643_1 0x95 /* CNS 11643-1992 Plane 1 */
#define LC_CNS11643_2 0x96 /* CNS 11643-1992 Plane 2 */
-/* #define FREE 0x97 free (unused) */
+#define LC_JISX0213-1 0x97 /* Japanese Kanji (JIS X 0213 Plane 1) (not supported) */
#define LC_BIG5_1 0x98 /* Plane 1 Chinese traditional (not supported) */
#define LC_BIG5_2 0x99 /* Plane 1 Chinese traditional (not supported) */
@@ -184,6 +183,12 @@ typedef unsigned int pg_wchar;
* (not supported) */
#define LC_TIBETAN_1_COLUMN 0xf1 /* Tibetan 1-column width glyphs
* (not supported) */
+#define LC_UNICODE_SUBSET_2 0xf2 /* Unicode characters of the range U+2500..U+33FF.
+ * (not supported) */
+#define LC_UNICODE_SUBSET_3 0xf3 /* Unicode characters of the range U+E000..U+FFFF.
+ * (not supported) */
+#define LC_UNICODE_SUBSET 0xf4 /* Unicode characters of the range U+0100..U+24FF.
+ * (not supported) */
#define LC_ETHIOPIC 0xf5 /* Ethiopic characters (not supported) */
#define LC_CNS11643_3 0xf6 /* CNS 11643-1992 Plane 3 */
#define LC_CNS11643_4 0xf7 /* CNS 11643-1992 Plane 4 */
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
