Thank you Ryan. I will provide the files if still necessary, but I have just 
discovered it has to do with column width.

Please compare:

.width 5 5 20
select * from countrynameslocalized where iso="AE";
AE     gsw    Ver?inigti Arabisch
AE     gu     ???????
AE     he     ????? ?????
AE     hi     ???????
AE     hr     Ujedinjeni Arapski E
AE     hu     Egyes?lt Arab Emir?
AE     hy     ??????? ???
AE     ia     Emiratos Arabe Unite
AE     id     Uni Emirat Arab     
AE     is     Sameinu?u arab?sku
AE     it     Emirati Arabi Uniti 
AE     ja     ???????
AE     ka     ???????
AE     km     ???????
AE     kn     ???????
AE     ko     ?????? ?
AE     lo     ???????
AE     lt     Jungtiniai Arab? Em

.width 5 5 150
select * from countrynameslocalized where iso="AE?;
AE     gsw    Ver?inigti Arabischi Emir??t                                      
                                                                                
 
AE     gu     ???????? ??? ????????                                             
                                              
AE     he     ????? ????????? ???????                                           
                                                               
AE     hi     ??????? ??? ??????                                                
                                                    
AE     hr     Ujedinjeni Arapski Emirati                                        
                                                                                
    
AE     hu     Egyes?lt Arab Emir?tus                                            
                                                                                
  
AE     hy     ??????? ???????? ?????????                                        
                                                            
AE     ia     Emiratos Arabe Unite                                              
                                                                                
    
AE     id     Uni Emirat Arab                                                   
                                                                                
    
AE     is     Sameinu?u arab?sku furstad?min                                    
                                                                                
 
AE     it     Emirati Arabi Uniti                                               
                                                                                
    
AE     ja     ????????                                                          
                                                                    
AE     ka     ???????? ????????????? ?????????                                  
                        
AE     km     ??????????????????                                                
                                                
AE     kn     ??????? ???? ????????                                             
                                              
AE     ko     ?????? ??                                                         
                                                                    
AE     lo     ????????????????????                                              
                                            
AE     lt     Jungtiniai Arab? Emyratai           

So, it looks like this is a presentation issue in the end. I must have not 
investigated it properly.

While this solves the issue for me, I still believe this behavior is confusing 
? truncated characters should be handled more gracefully.

Again, thank you for helping out and regards,
-- 
adam

> On 26 Feb 2015, at 10:29, R.Smith <rsmith at rsweb.co.za> wrote:
> 
> Thanks Adam,
> 
> Could you kindly post an example file somewhere (the kind which you import) 
> for us to test with?  (The list does not allow attachments).
> 
> Also, could you try a later version of SQLite so that we could see if this 
> problem still exists on your platform. Updated versions obtainable from:
> http://www.sqlite.org/downloads/
> 
> Thanks.
> Ryan
> 
> On 2015-02-26 12:10 PM, Adam Podstawczy?ski wrote:
>> Hi all,
>> 
>> Thanks for trying to help. Answering all suggestions and questions:
>> 
>> 1. I don?t think this is a presentation issue: I exported the data which I 
>> had imported earlier, and the issue is there in the exported files.
>> 
>> 2. Version information:
>> 
>> Last login: Thu Feb 26 09:06:41 on ttys004
>> Hg:locs adampodstawczynski$ sqlite3 --version
>> -- Loading resources from /Users/adampodstawczynski/.sqliterc
>> 
>> 3.8.7.1 2014-10-29 13:59:56 3b7b72c4685aa5cf5e675c2c47ebec10d9704221
>> Hg:locs adampodstawczynski$ cat ~/.sqliterc
>> .headers on
>> .mode column
>> Hg:locs adampodstawczynski$ uname -a
>> Darwin Hg 14.1.0 Darwin Kernel Version 14.1.0: Mon Dec 22 23:10:38 PST 2014; 
>> root:xnu-2782.10.72~2/RELEASE_X86_64 x86_64
>> Hg:locs adampodstawczynski$
>> 
>> This is on Mac OS 10.10.2 (Yosemite).
>> 
>> 3. I?m not doing any length() operation anywhere in the process, so there is 
>> no risk of truncation because of that. I just import the data and the char 
>> corruption issue is immediately there.
>> 
>> Thanks,
> 
> _______________________________________________
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Reply via email to