> What console font are you using, and which codepage? The default OEM > codepage is quite likely 437, and you may need to change it to 850. > > Codepage 437 is the original IBM ANSI OEM CodePage that contains line drawing > characters. Codepage 850 is the multilingual codepage that has multilingual > characters in it. I believe that the current shell is translating to MBCS > (ie, the Windows ACP, usually 1252 for Western European languages) so you > might want to try that codepage too.
I must have switched to 65001 in that command line window - because the SELECT query does display correctly with code page 437 and code page 850 (default) Results of tests Active code page: 437 SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .open test.restorednew.db sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; Japón (2002)|4A6170C3B36E20283230303229 sqlite> ---------------------- Active code page: 850 SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .open test.restorednew.db sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; Japón (2002)|4A6170C3B36E20283230303229 sqlite> ---------------------- Active code page: 1252 SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .open test.restorednew.db sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; Jap¢n (2002)|4A6170C3B36E20283230303229 sqlite> ----------------------- Active code page: 65001 SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .open test.restorednew.db sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; Jap�n (2002)|4A6170C3B36E20283230303229 Regards Simon -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Keith Medcalf Sent: 13 July 2016 22:34 To: SQLite mailing list <[email protected]> Subject: Re: [sqlite] [Windows] 3.13.0 recreate db from .dump file corrupts records with extended characters What console font are you using, and which codepage? The default OEM codepage is quite likely 437, and you may need to change it to 850. Codepage 437 is the original IBM ANSI OEM CodePage that contains line drawing characters. Codepage 850 is the multilingual codepage that has multilingual characters in it. I believe that the current shell is translating to MBCS (ie, the Windows ACP, usually 1252 for Western European languages) so you might want to try that codepage too. Also, the console font will affect the output. The raster fonts do not contain Unicode characters -- try using the Consolas or Lucida Console font (both of which look like crap, though Consolas is the least crappy of the two.) > One thing that concerns me though is that using windows command line > window with the sqlite3.exe v3.13.0 shell to interrogate a database > and display records, is that the display is corrupted - the database > is ok but display is not > SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage > hints. > Connected to a transient in-memory database. > Use ".open FILENAME" to reopen on a persistent database. > sqlite> .open test.restorednew.db > sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; > Jap n (2002)|4A6170C3B36E20283230303229 > sqlite> > whereas version 3.8.6 does display the record ok. No change in select > output with 3.13.0 even after `chcp 65001` > And also tried the following > >or > > > > (echo .output dump.sql && echo .dump) | sqlite3 old.db > > sqlite3 -init dump.sql new.db > > But this sequence corrupted the database records > > SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage > hints. > Connected to a transient in-memory database. > Use ".open FILENAME" to reopen on a persistent database. > sqlite> .open test.restorednew2.db > sqlite> .tables > spellfix_metadata_titles test-directories > spellfix_tag_titles test-media_parts > sqlite> SELECT path,hex(path) FROM "test-directories" WHERE id=68; > Jap├│n (2002)|4A6170E2949CE294826E20283230303229 > sqlite> > > The sequence I used > (echo .output newdump2.sql && echo .dump) | sqlite3_v3.13.0 > test.original.db > sqlite3_v3.13.0 -init newdump2.sql test.restorednew2.db > > > Simon > > -----Original Message----- > From: [email protected] > [mailto:sqlite-users- [email protected]] On Behalf Of > Clemens Ladisch > Sent: 13 July 2016 18:26 > To: [email protected] > Subject: Re: [sqlite] [Windows] 3.13.0 recreate db from .dump file > corrupts records with extended characters > > S.Ajina wrote: > > Can confirm that sqlite3.exe version 3.8.6 works ok restoring > > database from a .dump dump.sql file using these commands > > > > echo .dump | sqlite3_v3.8.6 test.original.db > dump_v3.8.6.sql > > sqlite3_v3.8.6 -init dump_v3.8.6.sql test.restored.v3.8.6.db > > > > Doing the same with the current version of sqlite3.exe (version > > 3.13.0) corrupts the database records when there are extended > > characters > > > > echo .dump | sqlite3_v3.13.0 test.original.db > dump_v3.13.0.sql > > sqlite3_v3.13.0 -init dump_v3.13.0.sql test.restored.v3.13.0.db > > > > Release Notes show changes in this area in version 3.10.0 which may > > have made the change Translate between MBCS and UTF8 when running in > > cmd.exe on Windows > > The old version of the sqlite3 shell assumes that screen and keyboard > I/O could be done with UTF-8. This assumption was wrong. > > The new version converts screen and keyboard I/O between UTF-8 and the > console's code page. So the output redirection with > no longer uses > UTF- 8. > > To fix this, use the same mechanism for both output and input: > > sqlite3 old.db .dump > dump.sql > sqlite3 new.db < dump.sql > > or > > (echo .output dump.sql && echo .dump) | sqlite3 old.db > sqlite3 -init dump.sql new.db > > > Regards, > Clemens > _______________________________________________ > sqlite-users mailing list > [email protected] > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > _______________________________________________ > sqlite-users mailing list > [email protected] > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list [email protected] http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list [email protected] http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
