Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-17 Thread Reinier Olislagers
On 17/07/2013 01:18, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 16/07/2013 17:18, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 15-7-2013 18:43, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: Another

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-17 Thread Graeme Geldenhuys
On 2013-07-17 00:18, Hans-Peter Diettrich wrote: As for sorting etc, there are various unicode collation standards. A DB implementing these standards has to offer Unicode fields in the first place. Firebird does, and so does many other database servers.

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-17 Thread Graeme Geldenhuys
On 2013-07-17 02:44, waldo kitty wrote: should one be allowed to place [chinese glyph][greekglyph][cyrillic glyph][latin glyph] all in the same string? what codepage will that string then contain? Of course, and that is exactly why the Unicode standard was developed. But getting

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-17 Thread Graeme Geldenhuys
On 2013-07-16 16:41, Hans-Peter Diettrich wrote: I know that, the question is whether the user and DB understand that, too. If you tell the database server (eg: Firebird) to use Unicode (UTF-8 for example), it will understand that. This will then affect storage size, up/lower case conversion,

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-17 Thread Reinier Olislagers
On 17/07/2013 11:36, Graeme Geldenhuys wrote: But getting back to the point. SqlDB seems to always define strings in byte length, yet the DB-aware components use character lengths. This is what is causing the trouble. Exactly. Other database components, as far as I remember, have .Size and

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Reinier Olislagers
On 15-7-2013 18:43, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: Workaround: make your field size as large as the maximum number of UTF8 bytes you expect. Another workaround: use the appropriate codepage for storing strings in

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Graeme Geldenhuys
On 2013-07-15 17:43, Hans-Peter Diettrich wrote: Another workaround: use the appropriate codepage for storing strings in the database, so that all characters are single bytes. You should know by now that not all characters can be represented in a single byte. Also Unicode was developed to

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Hans-Peter Diettrich
Graeme Geldenhuys schrieb: On 2013-07-15 17:43, Hans-Peter Diettrich wrote: Another workaround: use the appropriate codepage for storing strings in the database, so that all characters are single bytes. You should know by now that not all characters can be represented in a single byte. I

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Hans-Peter Diettrich
Reinier Olislagers schrieb: On 15-7-2013 18:43, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: Workaround: make your field size as large as the maximum number of UTF8 bytes you expect. Another workaround: use the appropriate codepage

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Reinier Olislagers
On 16/07/2013 17:18, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 15-7-2013 18:43, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: Workaround: make your field size as large as the maximum number of UTF8 bytes you expect.

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread Hans-Peter Diettrich
Reinier Olislagers schrieb: On 16/07/2013 17:18, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 15-7-2013 18:43, Hans-Peter Diettrich wrote: Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: Workaround: make your field size as large as the maximum

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-16 Thread waldo kitty
On 7/16/2013 13:14, Reinier Olislagers wrote: No it isn't. Why shouldn't a user be able to enter Chinese, Greek, Cyrillic and Latin characters? should one be allowed to place [chinese glyph][greekglyph][cyrillic glyph][latin glyph] all in the same string? what codepage will that string

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-15 Thread Graeme Geldenhuys
On 2013-07-14 19:00, Daniel Simoes de Ameida wrote: When the FieldDefs.Size properties is defined and we are trying to save UTF8 characters with accented words, the Data is Truncated. Yeah, I have experienced that too. There really should be a .Size and .DataSize property - or something

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-15 Thread Reinier Olislagers
On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: When the FieldDefs.Size properties is defined and we are trying to save UTF8 characters with accented words, the Data is Truncated. Could be because UTF8 with accented characters may result in multiple bytes per character. If the size is too

Re: [Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-15 Thread Hans-Peter Diettrich
Reinier Olislagers schrieb: On 14-7-2013 8:00, Daniel Simoes de Ameida wrote: When the FieldDefs.Size properties is defined and we are trying to save UTF8 characters with accented words, the Data is Truncated. Could be because UTF8 with accented characters may result in multiple bytes per

[Lazarus] Losing data when saving Database fileds with Size defined and UTF8 chars

2013-07-14 Thread Daniel Simoes de Ameida
Some one can confirm this problem ? When the FieldDefs.Size properties is defined and we are trying to save UTF8 characters with accented words, the Data is Truncated. This affect all Database components, including third party components like ZeosDB