If anyone can unravel this mystery, it would be much appreciated. For now, I inserted a comma - 一日、耶羅波安出 and it works. I suspect it must be somehow that the sequence of bytes encodes another character, which throws the tokenizer out of whack or maybe the fts4aux table.
一 19968 %E4%B8%80 日 26085 %E6%97%A5 耶 32822 %E8%80%B6 羅 32645 %E7%BE%85 波 27874 %E6%B3%A2 安 23433 %E5%AE%89 出 20986 %E5%87%BA On Mon, Jun 18, 2012 at 12:59 PM, E. Timothy Uy <t...@loqu8.com> wrote: > Thanks for writing back Dan. Using charCodeAt() in Javascript, I have the > following for 一日耶羅波安出: > > 19968 > 26085 > 32822 > 32645 > 27874 > 23433 > 20986 > > I tried entering subsets of the data: > > 一日耶羅波安出 - Error: SQL logic error or missing database <-- target > 一日耶羅波安 - Ok > 日耶羅波安出 - Ok > 耶羅波安出 - Ok > 一日耶羅波安出x - Error: SQL logic error or missing database > 一日耶羅波安x出 - Error: SQL logic error or missing database > 一日耶羅波x安出 - Error: SQL logic error or missing database > 一日耶羅x波安出 - Ok > 一日耶x羅波安出 - Ok > 一日x耶羅波安出 - Ok > 一x日耶羅波安出 - Ok > x一日耶羅波安出 - Ok > > I'm a bit concerned that this might be an indicator for a deeper issue. > Running Ubuntu Linux x64. > > Respectfully, > Tim > > > On Mon, Jun 18, 2012 at 12:29 PM, Dan Kennedy <danielk1...@gmail.com>wrote: > >> On 06/19/2012 02:11 AM, E. Timothy Uy wrote: >> > I recompiled ICU using U_CHARSET_IS_UTF8 and the error persists. >> > >> > On Mon, Jun 18, 2012 at 11:45 AM, E. Timothy Uy<t...@loqu8.com> wrote: >> > >> >> Hopefully someone has some insight on this. I am using FTS4 with >> >> tokenize=icu (and PRAGMA encoding="UTF-8"). I'm getting getting an >> error >> >> inserting the following into my virtual table: >> >> >> >> 一日耶羅波安出 >> >> Can you post the list of codepoints in this text? Or the hex >> of the utf-16 or utf-8 encoding of the same? >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> > > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
