This might be a representation error only.
I followed the OP's method of importing the file section (to a
encoding=UTF8 DB) using the command-line facility to import the
tab-separated file and then selecting the values from it.
File content of utf8test.txt:
no Newcastle 1 NULL NULL NULL
uk ??????? 1 NULL NULL NULL
ja ??????? 1 NULL NULL NULL
In the command-line facility I got all weird characters, the usual kind
of thing when you try to represent Unicode / UTF8 in ANSI or ASCII text.
C:\Users\R.Smith\Desktop>sqlite3 TestDB2.db
SQLite version 3.8.8.1 2015-01-20 16:51:25
Enter ".help" for usage hints.
sqlite> PRAGMA encoding=UTF8;
sqlite> CREATE TABLE cities(ct TEXT,nm TEXT,i INT,v1,v2,v3);
sqlite> .separator ' '
sqlite> .import utf8test.txt cities
sqlite> SELECT * FROM cities;
???no|Newcastle|1|NULL|NULL|NULL
uk|??????????????|1|NULL|NULL|NULL
ja|?????????????????????|1|NULL|NULL|NULL
sqlite>
(That might also just be my console output not being UTF8 friendly.)
However, when I used another DB tool to open the database, it looked
perfect:
SELECT * FROM "cities" WHERE 1;
-- ct nm i v1 v2 v3
-- -- -- - -- -- --
-- ?no Newcastle 1 NULL NULL NULL
-- uk ??????? 1 NULL NULL NULL
-- ja ??????? 1 NULL NULL NULL
-- Item Stats: Item No: 2 Query Size
(Chars): 33
-- Result Columns: 6 Result
Rows: 3
-- VM Work Steps: 32 Rows
Modified: 0
-- Full Query Time: -- --- --- --- --.----
-- Query Result: Success.
Conclusion: The importing works fine, the display of data might be
missing a trick.
Please ask the OP to test the DB with another viewer, and if still wrong
please supply the imported file and resulting DB file from his import on
some file-share site and note the SQLite version used.
On 2015-02-26 10:43 AM, Adam Podstawczy?ski wrote:
> Hi all,
>
> I experienced an issue with character encoding in sqlite3 following an import
> from a CSV file.
>
> The issue is described here:
> http://stackoverflow.com/questions/28719413/sqlite3-corrupts-some-chars-after-importing?noredirect=1#comment45732717_28719413
>
> In short: after importing a UTF-8 file via .import, only some non-lating
> chars, mostly at the end of strings, get corrupted.
>
> Please advise,
