#30477 [NEW]: UTF-8 characters are changed by mysql_query

[stian at grytoyr dot net]

From:             stian at grytoyr dot net
Operating system: SuSE Linux 9.1 Professional
PHP version:      5.0.2
PHP Bug Type:     MySQL related
Bug description:  UTF-8 characters are changed by mysql_query

Description:
------------
I'm using PHP 5.0.2, MySQL 4.1.6gamma and Apache 2.0.49-27.16 on SuSE
Linux Professional 9.1. PHP has been compiled by me, MySQL is a binary
distribution from MySQL, Apache is the SuSE default.

When I submit a query containing a UTF-8 representation (hex
value 0xC48D) of the Unicode character &#x010D (lowercase c with caron) to
the MySQL server using mysql_query, sniffing the network traffic reveals
that the character is sent as 0xC43F, which is not a valid UTF-8
character. 

It's important to note that this happens BEFORE the data reaches the MySQL
server, so I believe it must be a PHP problem.

Some other Unicode characters are also problematic, including &#x0110 and
&#x00C1, but they are both sent as 0xC33F.

The MySQL table has been created with CHARACTER SET utf8.


I don't think this bug is possible to reproduce without running
mysql_query, so the code below requires a connection to a MySQL database.

This _may_ be a duplicate of Bug #28231, but I'm not entirely sure.

Reproduce code:
---------------
header("Content-Type: text/html; charset=UTF-8");
// connect to a mysql database which supports UTF-8 here
$testStr = chr(196) . chr(141); // Unicode character &#x010D (c caron)
echo '$testStr before query: ' . $testStr . '<br />'; // will output the
correct UTF-8 character (c caron)
echo 'hex of $testStr before query: ' . bin2hex($testStr) . '<br />'; //
will output c48d

$updateResult = mysql_query("update klara_test set klara_text = '$testStr'
where klara_id = 1", $conn);
$fetchResult = mysql_query("select klara_text from klara_test", $conn);

while ($myrow = mysql_fetch_array($fetchResult)) {
  echo '$testStr from database: ' . $myrow['klara_text'] . '<br />'; //
will output gibberish
  echo 'hex of $testStr from database: ' . bin2hex($myrow['klara_text']) .
'<br />'; // will output c43f
}


Expected result:
----------------
I would expect the UTF-8 characters to remain unchanged.

Actual result:
--------------
Certain Unicode characters are altered by mysql_query, as far as I can
tell.

-- 
Edit bug report at http://bugs.php.net/?id=30477&edit=1
-- 
Try a CVS snapshot (php4):   http://bugs.php.net/fix.php?id=30477&r=trysnapshot4
Try a CVS snapshot (php5.0): http://bugs.php.net/fix.php?id=30477&r=trysnapshot50
Try a CVS snapshot (php5.1): http://bugs.php.net/fix.php?id=30477&r=trysnapshot51
Fixed in CVS:                http://bugs.php.net/fix.php?id=30477&r=fixedcvs
Fixed in release:            http://bugs.php.net/fix.php?id=30477&r=alreadyfixed
Need backtrace:              http://bugs.php.net/fix.php?id=30477&r=needtrace
Need Reproduce Script:       http://bugs.php.net/fix.php?id=30477&r=needscript
Try newer version:           http://bugs.php.net/fix.php?id=30477&r=oldversion
Not developer issue:         http://bugs.php.net/fix.php?id=30477&r=support
Expected behavior:           http://bugs.php.net/fix.php?id=30477&r=notwrong
Not enough info:             http://bugs.php.net/fix.php?id=30477&r=notenoughinfo
Submitted twice:             http://bugs.php.net/fix.php?id=30477&r=submittedtwice
register_globals:            http://bugs.php.net/fix.php?id=30477&r=globals
PHP 3 support discontinued:  http://bugs.php.net/fix.php?id=30477&r=php3
Daylight Savings:            http://bugs.php.net/fix.php?id=30477&r=dst
IIS Stability:               http://bugs.php.net/fix.php?id=30477&r=isapi
Install GNU Sed:             http://bugs.php.net/fix.php?id=30477&r=gnused
Floating point limitations:  http://bugs.php.net/fix.php?id=30477&r=float
MySQL Configuration Error:   http://bugs.php.net/fix.php?id=30477&r=mysqlcfg