Re: Fixing broken UTF-8 handling with MySQL driver

[Felix Zandanel]

 Am 12.03.2015 um 14:55 schrieb Timo Sirainen t...@iki.fi:
 
 On 11 Mar 2015, at 21:31, Felix Zandanel fe...@zandanel.me wrote:
 
 Although my MySQL installation is set up to use UTF-8 as the default charset 
 on every level (config, database, table and field), and the character_set_* 
 runtime variables all yield the value utf8, it still seems that the mysql 
 client library must be instructed to actually use UTF-8 explicitly. Adding 
 the following statement to driver_mysql_connect() fixes the issue for me:
 
 mysql_options(db-mysql, MYSQL_SET_CHARSET_NAME, utf8);
 
 I think you can also add to /etc/my.cnf :
 
 [client]
 default-character-set = utf8

Sorry for the late reply. You were so right, that simple line did the trick. My 
fault, I didn't read the whole charset documentation of MySQL. It's a shame 
that UTF-8 isn't the default setting.

Anyway, as dovecot's internals expect all input strings to be UTF-8, wouldn't 
it be useful to enforce UTF-8 in the database drivers? Using anything else than 
ASCII / UTF-8 for dovecot's MySQL connections doesn't really make sense, I 
think. Also, a default-character-set = utf8 line in my.cnf is a system wide 
configuration, which might break other software interacting with MySQL—in 
theory.




signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Fixing broken UTF-8 handling with MySQL driver

[Timo Sirainen]

On 11 Mar 2015, at 21:31, Felix Zandanel fe...@zandanel.me wrote:
 
 Although my MySQL installation is set up to use UTF-8 as the default charset 
 on every level (config, database, table and field), and the character_set_* 
 runtime variables all yield the value utf8, it still seems that the mysql 
 client library must be instructed to actually use UTF-8 explicitly. Adding 
 the following statement to driver_mysql_connect() fixes the issue for me:
 
 mysql_options(db-mysql, MYSQL_SET_CHARSET_NAME, utf8);

I think you can also add to /etc/my.cnf :

[client]
default-character-set = utf8

Fixing broken UTF-8 handling with MySQL driver

[Felix Zandanel]

Hi,

I noticed that dovecot's MySQL driver doesn't play well with UTF-8 encoded 
strings. I presume this issue has been around for a while. However, it still 
fails with the current release candidate 2.2.16-rc1.

I have been using the sieve extdata plugin with a MySQL-backed dict to retrieve 
a folder name that is then passed to fileinto:

fileinto :create ${extdata.folder_name_drafts};

Whenever this dict returned a string with non-7byte characters, a line such as 
the following popped up in sieve.log. (This example query should return 
Entwürfe.)

error: folder name specified for fileinto command is not utf-8: Entw?rfe.

Although my MySQL installation is set up to use UTF-8 as the default charset on 
every level (config, database, table and field), and the character_set_* 
runtime variables all yield the value utf8, it still seems that the mysql 
client library must be instructed to actually use UTF-8 explicitly. Adding the 
following statement to driver_mysql_connect() fixes the issue for me:

mysql_options(db-mysql, MYSQL_SET_CHARSET_NAME, utf8);

I checked this against mysql 5.5.41.

Regards,
Felix



mysql_utf8.patch
Description: Binary data