Hello, well, first it was hard to figure out where the problem was.
I played around with different charsets on My Linux-Box, on the database server and the webserver (with my PHP frontend for the same data). After some (unsuccessful) attempts, I figured out, that everything was ok on the client and on the server side, also with the conversion from/to ustring<->strings in My Gtkmm application. The solution was just be a conversion from html-entities wich are also utf-8 characters to be valid c-lib utf-8 chars. I simply used the multibyte-conversion function from php to convert the data from my html-entities-utf-8 PHP-app to real utf-8 data and stored them in My DB. The embrassing thing is, that I should know, that http-form data are not stored as the same character data, as it is in real applications. (Never trust what your browser shows you. :-)) ) I think that have to do with the character-conversion-table in PHP. So I just simply store and load them from My PHP-app as multibyte-converted-strings. With this conversion, they are also compatible to My Gtkmm application. I'm not sure if this is a good way, but it works. :D Thanks to your suggestions about the operand "<<" ">>" some posts ago. If someone has a better or other way, feel free, to show Me another point of view. Best regards Bernd Am Samstag, den 03.01.2009, 15:59 +0100 schrieb Milosz Derezynski: > How did you fix it? After all this back-and-forth it would be really > interesting to know. > > Regards, > M. > > On Sat, Jan 3, 2009 at 11:03 AM, Bernd Robertz > <[email protected]> wrote: > Hello again, > > I fixed the problem. > > I know why I don't receive an answer about that. The solution > was so > easy that it was a bit embrassing. :-) > > Sorry for bothering and thanks to everyone who helped Me out. > > Greetings > > Bernd > > > Am Freitag, den 02.01.2009, 14:30 +0100 schrieb Bernd Robertz: > > > Hello all, > > > > I hope everyone is well arrived in the new year. > > > > I come back to my last post. > > It seems to be, that I have some weired missconfiguration > with my > > charsets. > > If I'm inserting a row from My gtkmm-app, it will be shown > correctly in > > My app, but not in other apps like my PHP webfrontend or > phpMyAdmin. > > > > I think I know how to fix it. > > > > Has anyone suggestions wich charset should generally be used > i. e. for > > client and server apps (Gtkmm, webserver, db, etc.)? > > > > Thanks everyone who give Me the flashlight to get the > problem. > > > > Many thanks > > > > Bernd > > > > > > Am Montag, den 29.12.2008, 13:09 +0100 schrieb Bernd > Robertz: > > > Hello, > > > > > > thanks for the code. > > > > > > result is an object of mysqlpp::StoreQueryResult in the > sql-libs. > > > It stores the query data. > > > But in the docs of libmysql++ is described, that are the > stored data > > > handeld as a derived std::string. Also compatible to the > original > > > std::string. > > > > > > I inserted your code as you described and it returns the > following > > > output (2 textviews as example, one with shown data, one > with empty) > > > The output is taken from console: > > > > > > :::::::::::::::::: SNIP first example > data ::::::::::::::::: > > > > > > ** Message: UTF-8 valid: 0: Demozugang ist der gleiche. > > > > > > Auch f\xfcr gesch\xe4ftliche Aktivit\xe4ten (Streeting > > > Unternehmensvertrieb, siehe Unterlagen). > > > > > > Vertriebsnummer: 00028 > > > > > > Vertriebsseite: > > > URL: http://vertrieb.streeting.com > > > Benutzer: xxxx > > > Kennwort: xxxx > > > > > > Abrechnungssystem (im Vertriebslogin): > > > Benutzer: xxxx > > > Kennwort: xxxx > > > > > > :::::::::::::::::: SNIP second example > data ::::::::::::::::: > > > > > > ** Message: UTF-8 valid: 1: Daten CySoTec: > > > -------------- > > > > > > Jabba (XMPP) Messenger Client: > > > Benutzer: xxxx > > > Kennwort: xxxx > > > Server: cysotec.info > > > Jabba ID: [email protected] > > > [...] > > > > > > :::::::::::::::::: SNIP example data end ::::::::::::::::: > > > > > > As far as I can see, the one with empty data is not UTF-8 > valid > > > and the other is. > > > So, what can I do, to convert the Data from the SQL server > to My app in > > > a valid UTF-8, or on the other hand, > > > what collation should I use in on My SQL server? > > > > > > Thank you alot > > > > > > Bernd > > > > > > > > > Am Montag, den 29.12.2008, 00:47 +0100 schrieb Milosz > Derezynski: > > > > Of what type is 'result'? > > > > > > > > Also, could you add, before this line: > > > > > > > > tmp.push_back((std::string)result[row][i]); > > > > > > > > this code: > > > > > > > > Glib::ustring utf8_str(result[row][i]); > > > > g_message("UTF-8 valid: %d: %s", > int(utf8_str.validate()), > > > > utf8_str.c_str()); > > > > > > > > and see what it gives? > > > > > > > > > > > > M. > > > > > > > > P.S.: > > > > Yes std::string and ustring are generally compatible, > with the main > > > > differences being that ustring works by-character not > by-byte (e.g. > > > > find(), operator[], etc), ustring has methods for > validation of UTF-8 > > > > and related functionality; the main potential culprit > here is > > > > operator<< and operator>> of ustring. I had really big > problems with > > > > these using boost::format() until i realised that > internally, it's a > > > > stream to which the args are fed using operator<<() of > ustring. > > > > > > > > > > > > On Mon, Dec 29, 2008 at 12:01 AM, Bernd Robertz > > > > <[email protected]> wrote: > > > > Thanks alot for your fast reply, > > > > > > > > I wrote a simple MySQL-Class wich is derived > from the libsql++ > > > > class. > > > > > > > > In this class I used std::string to get the > columns of a row. > > > > The columns (strings) are stored in a > vector<string> wich I > > > > return to > > > > Glib::ustring in My treeview-data-window. > > > > > > > > Hiere are some snippets: > > > > > > > > :::::::::::::::::::::::: > SNIP :::::::::::::::::::::: > > > > database.cpp > > > > (My simple MySQL database class.) > > > > > > > > DataBase::Database() > > > > { > > > > // db-connect > > > > // query-setup > > > > // other connect related stuff > > > > > > > > mysqlpp::StoreQueryResult result; // > Setup the result > > > > object > > > > int rows; // Keep track of result counts > > > > } > > > > > > > > void DataBase::returnrow(int dataid) > > > > { > > > > // Setup db-query with dataid to get > requested db-row > > > > // > > > > mysqlpp::Query query = > dbcon.query(queries.alldata); // > > > > DB Query > > > > result = query.store(); // Store the > results > > > > rows = result.num_rows(); // Set the > number of results > > > > } > > > > > > > > std::vector<std::string> DataBase::fetchdata(int > row) > > > > { > > > > std::vector<std::string> tmp; // Temp > vector for return > > > > > > > > for (int i = 0; i < > result.field_names()->size(); ++i) > > > > { > > > > // Every column will be stored in > the temp > > > > vector > > > > > tmp.push_back((std::string)result[row][i]); > > > > } > > > > return tmp; // Return the temp-vector > with the > > > > [columns]=data > > > > } > > > > > > > > ------------ > > > > > > > > datawindow.cpp > > > > (The window in where the data is displayed after > dbl-click on > > > > treeview) > > > > > > > > DataWindow::DataWindow() > > > > { > > > > // Widgets setup > > > > // Database connect via DataBase-Object > > > > } > > > > > > > > void DataWindow::getid(int dataid) > > > > { > > > > // From the treeview I call this method > to set the > > > > // requested row-id > > > > mysql.returnrow(dataid); > > > > > > > > // Call this method to setup the entries > and textview > > > > setdata(); > > > > } > > > > > > > > void DataWindow::setdata() > > > > { > > > > l_dataid.set_text(mysql.fetchdata(0)[0]); > > > > Gtk::TreeModel::Row selrow = > *(ls_cbox->prepend()); > > > > selrow[c_mod.c_cat] = > mysql.fetchdata(0)[1]; > > > > cbox.set_active(0); > > > > > > > > // Here comes the problem, I get every > column from My > > > > // database-object wich I store in the > entries. > > > > e_name.set_text(mysql.fetchdata(0)[2]); > > > > e_number.set_text(mysql.fetchdata(0)[3]); > > > > e_user.set_text(mysql.fetchdata(0)[4]); > > > > e_pass.set_text(mysql.fetchdata(0)[5]); > > > > e_mail.set_text(mysql.fetchdata(0)[6]); > > > > e_url.set_text(mysql.fetchdata(0)[7]); > > > > e_other.set_text(mysql.fetchdata(0)[8]); > > > > > > > > // Here we have the main problem: the > textview-buffer > > > > // Some rows are shown here, some not. > > > > b_descr->set_text(mysql.fetchdata(0)[9]); > > > > } > > > > > > > > ::::::::::::::::::::::::: SNIP > > > > END :::::::::::::::::::::::::::: > > > > > > > > That is the main code where the problem occours. > > > > > > > > The DataWindow is called on doubleclick on a > treeview-row. > > > > With the call of the DataWindow I call the > DataView::getid() > > > > wich receive the database row-id. > > > > Then I fill with the DataView::setdata() method > the widgets > > > > in the new window. > > > > > > > > Some chars are, as I told, glyphs, but the > textbuffer is the > > > > main > > > > problem. > > > > > > > > As far as I know, the entries and textbuffers > are expecting > > > > Glib::ustring, but I thought the conversion from > std::string > > > > to > > > > Glib::ustring is quite compatible between these > types. > > > > I used std::string in the database class, > because I wanted to > > > > isolated > > > > it from Gtkmm for other not Gtkmm-related > projects. > > > > > > > > Thank you for helping. > > > > > > > > Bernd > > > > > > > > > > > > Am Sonntag, den 28.12.2008, 22:26 +0100 schrieb > Milosz > > > > Derezynski: > > > > > > > > > Are you using Glib::ustring anywhere in the > process of > > > > getting the > > > > > data from the SQL database to the view? > > > > > > > > > > Glib::ustring implicitly (and in my opinion > still at least > > > > not well > > > > > enough documented/hinted at) converts from > UTF-8 > > > > > to the current locale with operators << and > >>. If you are > > > > using a > > > > > stringstream anywhere in the process, or are > > > > > using operator<< or operator>> in any other > way, this could > > > > then be > > > > > the cause of the problem. > > > > > > > > > > Otherwise, and in the above described case too > actually it > > > > would help > > > > > very much if you could post (if possible, all > of) > > > > > the relevant code online. > > > > > > > > > > Regards, > > > > > M. > > > > > > > > > > On Sun, Dec 28, 2008 at 8:53 PM, Bernd Robertz > > > > > <[email protected]> wrote: > > > > > Hello list, > > > > > > > > > > I'm not really sure, but I hope My > question is in > > > > the right > > > > > place here. > > > > > > > > > > I wrote a simple database app like > this: > > > > > > > > > > Connect to a MySQL server via libmysql > ++, > > > > > and then list the rows in a treeview. > > > > > Doubleclick on a row shows a window > with some > > > > entries and a > > > > > textview > > > > > with the content of the selected row. > > > > > The textview is associated with a type > text in the > > > > DB. > > > > > > > > > > Everything is fine. > > > > > > > > > > So but I ran really into problems with > the charsets. > > > > > german umlaute (ö, ä, etc.) and some > other > > > > characters are > > > > > glyphs in the > > > > > treeview. Ok, thats not the main > problem. > > > > > > > > > > That is: The entries in the detailed > view still > > > > shows the > > > > > glyphs for > > > > > some chars like ö, ä. > > > > > But on some rows, the represented > textbuffer for > > > > textview is > > > > > not > > > > > visible. Wich means the textview is in > some rows > > > > empty. > > > > > > > > > > I run My app from a console and I got > these error > > > > messages: > > > > > > > > > > ::::::::: > > > > > Gtk-CRITICAL **: > gtk_text_buffer_emit_insert: > > > > assertion > > > > > `g_utf8_validate > > > > > (text, len, NULL)' failed > > > > > ::::::::: > > > > > > > > > > In other rows, the textbuffer is > correct shown, > > > > exepct some > > > > > glyphs. > > > > > > > > > > If I print the textbuffer as a > simple .c_str() in a > > > > console it > > > > > will be > > > > > shown correctly (inklusive ä, ö, > ü...). > > > > > (Tried to show the textview as > a .c_str(), without > > > > any > > > > > effect.) > > > > > > > > > > The SQL connection is > serverside-configured as UTF-8 > > > > Unicode. > > > > > The SQL table collation is UTF-8 > General CI (Case > > > > > Insensitive). > > > > > The Application runs on a de...@euro > (ISO-8859-15). > > > > > > > > > > All characters are shown correctly in > a PHP app wich > > > > does the > > > > > same, just > > > > > only online. :-) > > > > > > > > > > I hope someone could turn on a light > above my head, > > > > why I > > > > > don't get the > > > > > right charactes. > > > > > > > > > > Thanks alot in advance > > > > > > > > > > Bernd > > > > > > > > > > > > > > > > _______________________________________________ > > > > > gtkmm-list mailing list > > > > > [email protected] > > > > > > http://mail.gnome.org/mailman/listinfo/gtkmm-list > > > > > > > > > > > > > > > > > > > > -- > > > > > Please note that according to the German law > on data > > > > retention, > > > > > information on every electronic information > exchange with me > > > > is > > > > > retained for a period of six months. > > > > > [Bitte beachten Sie, dass dem Gesetz zur > > > > Vorratsdatenspeicherung > > > > > zufolge > > > > > jeder elektronische Kontakt mit mir sechs > Monate lang > > > > gespeichert > > > > > wird.] > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Please note that according to the German law on data > retention, > > > > information on every electronic information exchange > with me is > > > > retained for a period of six months. > > > > [Bitte beachten Sie, dass dem Gesetz zur > Vorratsdatenspeicherung > > > > zufolge > > > > jeder elektronische Kontakt mit mir sechs Monate lang > gespeichert > > > > wird.] > > > > > > _______________________________________________ > > > gtkmm-list mailing list > > > [email protected] > > > http://mail.gnome.org/mailman/listinfo/gtkmm-list > > > > _______________________________________________ > > gtkmm-list mailing list > > [email protected] > > http://mail.gnome.org/mailman/listinfo/gtkmm-list > > _______________________________________________ > gtkmm-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/gtkmm-list > > > > > -- > Please note that according to the German law on data retention, > information on every electronic information exchange with me is > retained for a period of six months. > [Bitte beachten Sie, dass dem Gesetz zur Vorratsdatenspeicherung > zufolge > jeder elektronische Kontakt mit mir sechs Monate lang gespeichert > wird.] _______________________________________________ gtkmm-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/gtkmm-list
