So what status are we on? Are we going to release 9.2.2 as it is?
Or withdraw current 9.2.2?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
> Andres Freund writes:
>> On 2012-12-04 21:27:34 -0500, Tom Lane wrote:
&
> exceptions that you noticed).
So, what you wanted to say here is:
"If the conversion to this encoding is not avaliable then fall back to
UTF-8"
Am I correct?
Also is it possible to completely disable the feature?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraos
a
>> disaster to me.
>>
> Ok, maybe the time of real universal encoding has not yet come. Then
> we maybe just should add a new parameter "log_encoding" (UTF-8 by
> default) to postgresql.conf. And to use this encoding consistently
> within logging_collector.
> If t
> Hello,
>>
>> Implementing any of these isn't trivial - especially making sure
>> messages emitted to stderr from things like segfaults and dynamic
>> linker messages are always correct. Ensuring that the logging
>> collector knows when setlocale() has been called to change the
>> encoding and tra
quot; unify ABC to D. This is a great
space saving:-) The price of this is inablity of
round-trip-conversion. You can convert A, B or C to D, but you cannot
convert D to A/B/C.
BTW, I'm not stick with mule-internal encoding. What we need here is a
"super" encoding which could includ
> Tatsuo Ishii writes:
>> My idea is using mule-internal encoding for the log file instead of
>> UTF-8. There are several advantages:
>
>> 1) Converion to mule-internal encoding is cheap because no conversion
>>table is required. Also no information loss ha
ernal encoding can be handled by emacs, one of the most
popular editors in the world.
3) No need to worry about locale. Mule-internal encoding has enough
information about language.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.c
. pgpool-II disconnects the connection to PostgreSQL upon failure
including network problem, rather than trying to resend packet to
PostgreSQL due to the reason stated above.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
--
Sent
customer's problem today...
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
ases
at the earliest. The realistic solution available today is replacing
default conversion for EUC-JP and UTF-8.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make
tween EUC-JP and UTF-8 itself
break backward compatibility?
I think what we can do best here is, adding new encoding and default
conversion.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
--
Sent via pgsql-bugs mailing list (pgsql-
H. So if we want to do a round trip conversion between
EUC-JP and UTF-8, we have to choose either U+FF5E OR U+301C. We have
chosen U+FF5E. If we change the mapping, many existing applications
would break.
Same thing can be said to MINUS sign.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http:
while (isspace((unsigned char) *p))
p++;
IMO this should be something like:
while (isspace((unsigned char) *p))
p += pg_mblen(p);
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://ww
Unicode.
For example the glyph for 0xe7a281 == U+7881 is defined in page 43 of
http://unicode.org/charts/PDF/U4E00.pdf. So we need to make sure that the
particular kanji character defined in Big5 0xf9d6 has the same glyph
as the one defined in Unicode(U+7881). Same thing can be said to rest
of the
> Tatsuo Ishii wrote:
> > Why do you think that an UTF-8 encoded string starting with 0x92 is
> > valid?
> >
> > 0x92 can appear in the second, third or fourth octet, but should never
> > appear in the first octet.
> > --
> > Tatsuo Ishii
> > SR
> Then how then does Postgres expect to be able to store text that goes
> beyond SQL_ASCII.. ??? Because what you are saying in effect is that
If you need to store an arbitrary byte sequences, BYTEA is your
friend.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
---(
> Tatsuo Ishii wrote:
> > Why do you think that an UTF-8 encoded string starting with 0x92 is
> > valid?
> >
> > 0x92 can appear in the second, third or fourth octet, but should never
> > appear in the first octet.
> > --
> > Tatsuo Ishii
> > SR
Why do you think that an UTF-8 encoded string starting with 0x92 is
valid?
0x92 can appear in the second, third or fourth octet, but should never
appear in the first octet.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
> The following bug has been logged online:
>
> Bug reference: 3638
&g
into utils/mb/Unicode/euc_cn_to_utf8.map, the
translation above seems to be correct. BTW, who does the translation
from EUC-CN to UTF-8? Maybe gettext()?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
> But anyway, if this is Chinese then it's
> hardly surprising that there would be no LATIN1 equivalent. A
ffected by this.
>
> utf8_and_iso8859.c is certainly broken, and I'm wondering what else
> might be. Tatsuo, can you think of any other places to look?
I will look into this.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
---(end of broadcast)---
TIP 6: explain analyze is your friend
ring to
> > fix the tables that are affected by this.
> >
> > utf8_and_iso8859.c is certainly broken, and I'm wondering what else
> > might be. Tatsuo, can you think of any other places to look?
>
> I will look into this.
Quick check reveals that ISO-8859-5 to ISO-885
> Having blank lines in -f scripts was causing silent failures. This
> fixes it, for some value of "fixes." If it's OK, please apply to 8.1
> CURRENT and CVS TIP :)
Thanks. I have committed your patches to current and 8.1 stable.
--
Tatsuo I
converted iso data from the orte_de view, as converted
> names that contain "ß" are not shown...
Let me make sure. You store UTF-8 data in ISO 8859-1 DB?
--
Tatsuo Ishii
---(end of broadcast)---
TIP 6: explain analyze is your friend
plz, loc_id, ort, convert(ort using utf_8_to_iso_8859_1) as
> ort_conv
> from orte_de
> where plz between 2 and 3
> order by ort_conv
Problem here is the result of convert(ort using utf_8_to_iso_8859_1)
is ISO-8859-1 but your database encoding is UTF-8, so the terminal
here's no
reason the conversion for german sharp s does not work.
Marcus,
Can you give me the exact error message from PostgreSQL when the
conversio failed?
--
Tatsuo Ishii
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, ple
rs are "Unified" in Unicode. To solve the problem, we can use
convert UTF8 to EUC_JP using CONVERT. See archives for more details.
Or you can use Unicode locale only if your platform's locale database
is not broken and you only use single locale.
--
Tatsuo Ishii
TABLESPACE = pg_default;
> 2. insert data with Chinese words
> INSERT into node set title='1 中文'
>
> 3. SELECT title from node where title > '1.1 '
> would return '1 中文'
>
> 4. Both SELECT '1 中文' > '1.1'
mb/Unicode/big5_to_utf8.map) does not have an
entry for 0xf9d8. There are two possible solutions:
1) If there is any "standard confirmance" map including an entry for
0xf9d8, we are happy to replace it with the one we have.
2) you could create your own con
CONVERSION
command documents for more details.
--
Tatsuo Ishii
From: "PostgreSQL Bugs List" <[EMAIL PROTECTED]>
Subject: [BUGS] BUG #1091: Localization in EUC_TW Can't decode Big5 0xFA40--0xFEF0.
Date: Wed, 3 Mar 2004 22:08:47 -0400 (AST)
Message-ID: <[EMAIL PROTECTED]>
&
You need to create database "test1" with encoding EUC_JP or UNICODE.
(BFor example:
(B
(B/usr/local/pgsql/bin/createdb -E EUC_JP test1
(B--
(BTatsuo Ishii
(B
(BFrom: "PostgreSQL Bugs List" <[EMAIL PROTECTED]>
(BSubject: [BUGS] BUG #1060: psql encoding bug : sjis
(BDate: Thu, 22 Jan 2004 04
pect you did something wrong with update and/or build procedure for
PHP. Hard to tell without knowing how you did your PHP installation,
though.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister comma
, plane 2
> > > > CNS 11643-1993, plane 15
> > > >
> > > > Would you like to have support for rest of CNS 11643-1993 planes:
> > > >
> > > > CNS 11643-1993, plane 3
> > > > CNS 11643-1993, plane 4
> > > > CNS 11643-1993, plane 5
> > > > CNS 11643-1993, plane 6
> > > > CNS 11643-1993, plane 7
> > > >
> > > > support for upcoming 7.4?
--
Tatsuo Ishii
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
BUGS list, could you let me know the status of that BUG? Thanks!
Sorry for the delay. In EUC-JP <--> Unicode translation, WAVE DASH is
always a problem since there are several different mappings among
different vendors/standards. I think I need more time to solve this.
--
Tatsuo Ishii
---
ducts where name like '??~'
> Query failed: ERROR: Invalid EUC_JP character sequence found (0xba7e)
This is definitly a bad EUC_JP.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
> > > I reported bug #943 (I found in 7.3.2) and you checked in some change against
> > > integer overflow.
> > > Now I upgraded to 7.3.3 and I'm not happy with this.
> > > The exact error as I described is fixed, but I found new errors in conversion
> > > UTF-8 <-> EUC_TW and BIG5:
> > >
> > > C
e supports todays BIG5.
>
> I'm not looking forward to add any conversion entries confirmed by
> standards. Can some one explain me the current status of the
Oops. above should be:
I'm not looking forward to add any conversion entries NOT confirmed by
standards.
> conversi
ndards. Can some one explain me the current status of the
conversion maps between BIG5 and Unicode? The only info I could found
so far is in www.unicode.org.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
ot convert UTF-8 (0xe7b2a7). Ignored
>
> to EUC_TW is ok!
BIG5 and EUC_TW have different code points. So this is not very strange.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
> Sample Code
> insert into t2 (names) values ('xxx');
What did you insert as xxx? I would like to see them as hex or octal
value.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please sen
> zzhb ([EMAIL PROTECTED]) reports a bug with a severity of 3
> The lower the number the more severe it is.
>
> Short Description
> client_encoding can't support GB18030 encoding in postgresql 7.3.1
>
> Long Description
> When I insert records using files, psql -f install.sql
>
> if I set PGCLIE
> Tatsuo Ishii <[EMAIL PROTECTED]> writes:
> > Thanks. I have confirmed that at least following reports are
> > correct. Will fix them.
>
> I already committed fixes in HEAD and 7.3 branch.
Thanks.
> I wanted to ask you if this might explain some of the od
E]
> X [UNKNOWN]
> X [BROKE]
> X [SKIP]
> /u2/yxie/postgresql-7.3.1/src/backend/utils/mb/conversion_procs/euc_tw_a
> nd_big5/big5.c:371:CNStoBIG5: ERROR:BUFFER:371:371:Array bounds error
--
Tatsuo Ishii
---(end of broadcast)---
TIP 3: if posting/r
> > I got a sig 11 in an apache child process:
It's a well known problem with PHP 4.0.6. Upgrade to php 4.2.2.
(you could upgrade to 4.2.3 if you do not care about mbstring module
bugs).
--
Tatsuo Ishii
---(end of broadcast)---
TIP 4:
d by the
goverment in Taiwan. Is there any technical superiority in Big5?
Or maybe "don't know why but just many peole use Big5":-)
--
Tatsuo Ishii
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appro
) corresponding to
Big5 0xc05c. That's why PostgreSQL complains.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
en using psql.
>
> I¡¦ve found the problem exists since 7.1.x till 7.2.*.
What is the encoding for "chinese char"? You need to give us more
info.
--
Tatsuo Ishii
P.S. Please don't post with non-ascii chars. If you need to show
non-ascii chars, you should give
I suspect that data stored in your database was not EUC_CN. GB2312(or
GBK) is different from EUC_CN. Can you show me the INSERT statement
text in gzipped form to prevent accidental changes while relaying the
mail chanin?
--
Tatsuo Ishii
> Hello,
>
> I am using Postgresql 7.1
data you actually want to store, but
for PostgreSQL \3003 means "octal 300 + ascii '3'" == 0xc033, which is
apparently wrong UTF-8 sequence. So PostgreSQL behaves as expected.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
everal ways to set the client encoding to Big5:
SQL: set client_encoding to 'Big5';
from psql: \encoding Big5
using environment variable: export PGCLIENTENCODING=Big5 (example for bash)
Hope this helps,
--
Tatsuo Ishii
---(end of broadcast)--
en the 8th bit of following three
bytes must be set
(2) else if the first byte is 0x8f, then the 8th bit of following two
bytes must be set
(3) else if the 8th bit of the first byte is set, then the 8th bit of
following one bytes must be set
(4) else (that means th
_TW and Big5 mixed data, then you have a serious
problem. You probably have to fix the the dump data by hand.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
; to its lower case if you are sure that 'X'
could be converted to ISO-8859-1.
Of course the problem with this method is:
Someone has suggested me a fix using UTF-8 locales, but I'm worried
about usage of UTF-8 and am waiting for the test result with my
Japanese data.
--
Tatsuo Ishii
> > > > There are "full width alphabets" in Japanese. Thoes include not only
> > > > ASCII letters but also some European characters.
> > >
> > > Are these ASCII and European characters uppercased in some
> > > Japanese-specific way ?
> >
> > Probably not, but I'm not sure since my Linux box does
re are "full width alphabets" in Japanese. Thoes include not only
> > ASCII letters but also some European characters.
>
> Are these ASCII and European characters uppercased in some
> Japanese-specific way ?
Probably not, but I'm not sure since
that say, de_DE.utf8 locale produce meaningful results
for any other languages? If so, why are there so many *.utf8 locales?
> btw, does Japanese language have distinct upper and lower case letters ?
There are "full width alphabets" in Japanese. Thoes include not only
ASCII letters
,-- English message
germany text,-- Germany message
japanese text-- Japanese message
);
If you have set the local to, say de_DE, then:
select lower(japanese) from t1;
would be executed in de_DE.utf8 locale, and I doubt it produces any
meaningfull results for Japanese.
--
Tatsu
icular locale and the encoding. For example,
you know that for de_DE locale using LATIN1 encoding is appropreate,
but PostgreSQL does not.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
might be:
lower(convert('your_utf8_string', 'LATIN1'))
Of course this would only work if you use only LATIN1 characters in
UTF-8.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an app
em, but you need to escape bytes >
0x7f (PQescapebytea has been fixed in that way in current). Or you
could cast them as bytea.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
f9d6(BIG5) to
0xea4d(UTF-8). Take a look at
src/backend/utils/mb/Unicode/big5_to_utf8.map.
BTW, if you really want to the conversion between 0xf9d6 and 0xea4d,
you could add it to big5_to_utf8.map and utf8_to_big5.map.
Hope this helps,
--
Tatsuo Ishii
---(end of broadcast
I guess you are inserting correct EUC Traditional Chinese (EUC-TW)
characters but hard to tell what is happening unless you are showing
us the character sequences in hexa decimal format.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
> When useing COPY FROM 'file' DELIMITER '\254' copyfrom reads past the
> delimiter and ends up with parse errors when trying to do the insert
>
>
> What the ?? why dind' tthat go through with the body of the text.. *sigh*
> I'll resend in the AM..
Good catch. It's definitely a bug in copy com
Not sure what version of PostgreSQL or what kind of platform you are
using, I would say this may be a bug with the locale database of your
OS. Try disable locale support of PostgreSQL.
--
Tatsuo Ishii
> Young ([EMAIL PROTECTED]) reports a bug with a severity of 2
> The lower the number th
tat4+AIX 5L compiles fine for me.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Seems a known bug with Mac OS 10.1. Work around: try 7.2beta3.
--
Tatsuo Ishii
> Curtis Hermann ([EMAIL PROTECTED]) reports a bug with a severity of 1
> The lower the number the more severe it is.
>
> Short Description
> Cannot Create Database
>
> Long Description
> I a
This is a known problem with PHP 4.0.6. You might want to upgrade to
4.1.0RC2, or try patches made by one of the PHP developers:
ftp://ftp.sra.co.jp/pub/cmd/postgres/php/php-4.0.6-patches.tar.gz
> Mike Rogers wrote:
>
> > Sorry:
> > PHP 4.0.6 (with memory leak patch [download listed right b
> Tatsuo Ishii <[EMAIL PROTECTED]> writes:
> > Note that EUC_TW or any multibyte encodings that are allowed for
> > backend side, do not contain such ASCII special characters as "|" and
> > should be safe for the parser and the regexp routines.
>
>
tion to PostgreSQL).
Note that EUC_TW or any multibyte encodings that are allowed for
backend side, do not contain such ASCII special characters as "|" and
should be safe for the parser and the regexp routines.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Hum. Why don't you enable --enable-multibyte and
--enable-unicode-conversion and set client_encoding to UNICODE? That
would do a conversion from/to UTF-8 for Tcl 8.x (x > 9) clients?
--
Tatsuo Ishii
> Eugene Faukin ([EMAIL PROTECTED]) reports a bug with a severity of 2
> The lower
762 1
What is \261\\? It does not seem to be correct EUC_TW. Probably you
want to enter BIG5 from psql? The you should do like this:
\encoding BIG5
insert into a values('some big5 characters');
For the inserting value you should not use octal exporession but
sh
b). If you want to use
traditional chinese do:
initdb --encoding=EUC_TW
instead.
--
Tatsuo Ishii
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
nding fields.
> I don't know why and I must use ';' as the delimiter since I have no choice.
> All the information abc,def,hij,mnx are BIG5 characters (I only try BIG5)
> Do anyone have report this strange behaviour?
Copying BIG5 data to/from UNICODE
> Le jeu, 11 jan 2001, Tatsuo Ishii a écrit :
> > > I use PostgreSQL 7.0.2 on linux.
> > > The base was set with initdb -E UNICODE.
> > >
> > > I have many Strings with accents (french language).
> > > Some of them aren't supporte
from element_attribute where java_lang_string like 'Scholtè_s';
Are you sure that the letter (LATIN SMALL LETTER E WITH GRAVE) is
encoded in UTF-8? It's 2 bytes long and starting with 0xc...
--
Tatsuo Ishii
anges to ignore empty-string pgport
paramter of PQsetdbLogin(). Same thing should be applied to PGPORT
environment variable too, I think.
--
Tatsuo Ishii
7;d be glad to send them, but I'm sure
> you can replicate this pretty easily now if required.
Of course regexp should not crash in this situation above. Thanks for
the info. I will dig into the problem.
--
Tatsuo Ishii
problem. See the
> following four examples for representative errors:
Neither \df or \dS problem reproduces here (I have exactly same
configuration as you).
Instead, I have another problem already reported at hackers list:
creatdb/dropdb does not work
See the posting "Solaris 2.6 problems" in the archives.
--
Tatsuo Ishii
77 matches
Mail list logo
