[sqlite] Cleaning unicode text

2011-10-03 Thread BareFeetWare
Hi SQLiters, I am trying to clean up some text in a database that has some weird non-printable unicode characters. For instance: .mode insert select distinct Name, length (Name), substr(Name,-1,1) from My Table gives: INSERT INTO table VALUES('Roundup Ready®',15,'') As you can see, the

Re: [sqlite] Cleaning unicode text

2011-10-03 Thread Pavel Ivanov
I've tried: replace(Name, x'f87f', '') but it doesn't seem to match the weird character. Any ideas? I guess x'f87f' will match your character only if your database is in UTF-16. For UTF-8 you probably should try x'efa1bf'. And maybe explicit conversion to text is needed. Pavel On Mon, Oct