[sqlite] Characters corrupt after importing a CSV file

[R.Smith]

While it is a presentation issue in the end, it still is an issue, so 
thank you for happening upon it and bringing it to our attention, and no 
need to apologize for lack of investigation.

I think it is possibly the command line utility falling into the 
character-length trap which Hick mentioned when enforcing column widths.


Also, it is of course no longer needed to share the import file, but I 
wouldn't mind having a list of country names in localized format - if 
you are happy to share.


On 2015-02-26 12:58 PM, Adam Podstawczy?ski wrote:
> Thank you Ryan. I will provide the files if still necessary, but I have just 
> discovered it has to do with column width.
>
> Please compare:
>
> .width 5 5 20
> select * from countrynameslocalized where iso="AE";
> AE     gsw    Ver?inigti Arabisch
> AE     gu     ???????
> AE     he     ????? ?????
> AE     hi     ???????
> AE     hr     Ujedinjeni Arapski E
> AE     hu     Egyes?lt Arab Emir?
> AE     hy     ??????? ???
> AE     ia     Emiratos Arabe Unite
> AE     id     Uni Emirat Arab
> AE     is     Sameinu?u arab?sku
> AE     it     Emirati Arabi Uniti
> AE     ja     ???????
> AE     ka     ???????
> AE     km     ???????
> AE     kn     ???????
> AE     ko     ?????? ?
> AE     lo     ???????
> AE     lt     Jungtiniai Arab? Em
>
> .width 5 5 150
> select * from countrynameslocalized where iso="AE?;
> AE     gsw    Ver?inigti Arabischi Emir??t
> AE     gu     ???????? ??? ????????
> AE     he     ????? ????????? ???????
> AE     hi     ??????? ??? ??????
> AE     hr     Ujedinjeni Arapski Emirati
> AE     hu     Egyes?lt Arab Emir?tus
> AE     hy     ??????? ???????? ?????????
> AE     ia     Emiratos Arabe Unite
> AE     id     Uni Emirat Arab
> AE     is     Sameinu?u arab?sku furstad?min
> AE     it     Emirati Arabi Uniti
> AE     ja     ????????
> AE     ka     ???????? ????????????? ?????????
> AE     km     ??????????????????
> AE     kn     ??????? ???? ????????
> AE     ko     ?????? ??
> AE     lo     ????????????????????
> AE     lt     Jungtiniai Arab? Emyratai
>
> So, it looks like this is a presentation issue in the end. I must have not 
> investigated it properly.
>
> While this solves the issue for me, I still believe this behavior is 
> confusing ? truncated characters should be handled more gracefully.
>
> Again, thank you for helping out and regards,