Okay, found the fault - the fix is available in commit 1d116fb046.
On 02/04/2013, at 8:36 AM, hipertracker wrote:
> I have charset_table which covers UTF-8 characters. I used it for TS2 earlier.
>
> For MySQL I have
>
> development: &shared
> mem_limit: 256M
> min_word_len: 2
> enable_star: true
> min_infix_len: 3
> html_strip: true
> sql_query_pre:
> - "SET CHARACTER_SET_RESULTS=utf8"
> - "SET SESSION query_cache_type=OFF"
> charset_table: 0..9, A..Z->a..z, a..z, U+0143->U+0144, U+0104->U+0105,
> U+0106->U+0107, U+0118->U+0119, U+0141->U+0142, U+00D3->U+00F3,
> U+015A->U+015B, U+0179->U+017A, U+017B->U+017C, U+0105, U+0107, U+0119,
> U+0142, U+00F3, U+015B, U+017A, U+017C, U+0144, U+41..U+5a->U+61..U+7a,
> U+61..U+7a, U+aa, U+b5, U+ba, U+c0..U+d6->U+e0..U+f6, U+d8..U+de->U+f8..U+fe,
> U+df..U+f6, U+f8..U+ff, U+100..U+12f/2, U+130->U+69, U+131, U+132..U+137/2,
> U+138, U+139..U+148/2, U+149, U+14a..U+177/2, U+178->U+ff, U+179..U+17e/2,
> U+17f..U+180, U+181->U+253, U+182..U+185/2, U+186->U+254, U+187..U+188/2,
> U+189..U+18a->U+256..U+257, U+18b..U+18c/2, U+18d, U+18e->U+1dd,
> U+18f->U+259, U+190->U+25b, U+191..U+192/2, U+193->U+260, U+194->U+263,
> U+195, U+196->U+269, U+197->U+268, U+198..U+199/2, U+19a..U+19b,
> U+19c->U+26f, U+19d->U+272, U+19e, U+19f->U+275, U+1a0..U+1a5/2,
> U+1a6->U+280, U+1a7..U+1a8/2, U+1a9->U+283, U+1aa..U+1ab, U+1ac..U+1ad/2,
> U+1ae->U+288, U+1af..U+1b0/2, U+1b1..U+1b2->U+28a..U+28b, U+1b3..U+1b6/2,
> U+1b7->U+292, U+1b8..U+1b9/2, U+1ba..U+1bb, U+1bc..U+1bd/2, U+1be..U+1c3,
> U+1c4->U+1c6, U+1c5..U+1c6/2, U+1c7->U+1c9, U+1c8..U+1c9/2, U+1ca->U+1cc,
> U+1cb..U+1dc/2, U+1dd, U+1de..U+1ef/2, U+1f0, U+1f1->U+1f3, U+1f2..U+1f5/2,
> U+1f6->U+195, U+1f7->U+1bf, U+1f8..U+21f/2, U+220->U+19e, U+221,
> U+222..U+233/2, U+234..U+23a, U+23b..U+23c/2, U+23d->U+19a, U+23e..U+240,
> U+241->U+294, U+250..U+2c1, U+2c6..U+2d1, U+2e0..U+2e4, U+2ee,
> U+1d00..U+1dbf, U+1e00..U+1e95/2, U+1e96..U+1e9b, U+1ea0..U+1ef9/2, U+37a,
> U+386..U+389->U+3ac..U+3af, U+38c..U+38e->U+3cc..U+3ce, U+390,
> U+391..U+3a1->U+3b1..U+3c1, U+3a3..U+3ab->U+3c3..U+3cb, U+3ac..U+3ce,
> U+3d0..U+3d7, U+3d8..U+3ef/2, U+3f0..U+3f3, U+3f4->U+3b8, U+3f5,
> U+3f7..U+3f8/2, U+3f9->U+3f2, U+3fa..U+3fb/2, U+3fc..U+3ff,
> U+400..U+40f->U+450..U+45f, U+410..U+42f->U+430..U+44f, U+430..U+45f,
> U+460..U+481/2, U+48a..U+4bf/2, U+4c0, U+4c1..U+4ce/2, U+4d0..U+4f9/2,
> U+500..U+50f/2, U+5D0..U+5EA, U+5F0..U+5F2, U+5BE, U+5C0, U+5C3, U+5F3,
> U+5F4, U+621..U+63a, U+640..U+64a, U+66e..U+66f, U+671..U+6d3, U+6d5,
> U+6e5..U+6e6, U+6ee..U+6ef, U+6fa..U+6fc, U+6ff
>
> For PostgreSQL I have commented out
>
> # sql_query_pre:
> # - "SET CHARACTER_SET_RESULTS=utf8"
> # - "SET SESSION query_cache_type=OFF"
>
> And my indices files have also
>
> set_property :utf8? => true
>
> All works fine, except matching utf-8 words in excerpts.
>
> --
> JZ
>
> On Monday, April 1, 2013 1:04:40 AM UTC+1, Pat Allan wrote:
> What's the contents of your config/thinking_sphinx.yml file, if it exists?
> Most importantly, do you have a charset_table value that covers these UTF-8
> characters?
> --
> Pat
>
> On 01/04/2013, at 4:53 AM, hipertracker wrote:
>
> > Now encoding is fine, but still excerpts does not match UTF-8 based words.
> > Only words without UTF-8 characters are matched
> >
> > rs = VersePl.search('pozdrawia && łukasz && lekarz')
> > rs.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> > puts rs.first.excerpts.text
> > # => <span class="match">Pozdrawia</span> was Łukasz, umiłowany <span
> > class="match">lekarz</span>, i Demas.
> >
> > --
> > JZ
> >
> > On Sunday, March 31, 2013 2:25:50 PM UTC+1, Pat Allan wrote:
> > Just fixed this with commit 87e22abd7e - Sphinx isn't very good at
> > responding with UTF-8, but it's easy enough to work around with Ruby 1.9
> > encodings, thankfully.
> > --
> > Pat
> >
> > On 01/04/2013, at 12:15 AM, hipertracker wrote:
> >
> > > er, It should be "It still does not work with UTF-8 characters."
> > >
> > > I've just checked it for MySQL and PostgreSQL. All UTF-8 characters
> > > returned from excertps.fieldname method are corrupted.
> > >
> > > --
> > > JZ
> > >
> > > On Sunday, March 31, 2013 1:01:07 PM UTC+1, hipertracker wrote:
> > > I can see excerpts method now, but it still does work with UTF-8
> > > characters.
> > >
> > > rs = VersePl.search('pozdrawia && łukasz')
> > > rs.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> > > puts rs.first.text # that's correct:
> > > # => Pozdrawia was Łukasz, lekarz miły i Demas.
> > >
> > > puts rs.first.excerpts.text
> > > <span class="match">Pozdrawia</span> was Šukasz, lekarz miły i Demas.
> > >
> > > It matches only non-UTF-8 words and all UTF-8 characters are corrupted.
> > >
> > > --
> > > JZ
> > >
> > > On Saturday, March 30, 2013 11:53:27 PM UTC, Pat Allan wrote:
> > > That's a bug - and I've just fixed it now. If you use the following in
> > > your Gemfile, it should work fine:
> > > gem 'thinking-sphinx', '~> 3.0.2',
> > > :git => 'git://github.com/pat/thinking-sphinx.git',
> > > :ref => 'b50e7d3f4c'
> > >
> > > --
> > > Pat
> > >
> > > On 31/03/2013, at 10:44 AM, hipertracker wrote:
> > >
> > > > But when I repeat that with a1 different phrase I have a side effect.
> > > > search.context[:panes] array is groving with every new search. Is it
> > > > intentional or an error?
> > > >
> > > > search = MyModel.search 'phrase2'
> > > > search.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> > > > p search.context[:panes]
> > > > # => [ThinkingSphinx::Panes::ExcerptsPane,
> > > > ThinkingSphinx::Panes::ExcerptsPane]
> > > > search = MyModel.search 'phrase3'
> > > > search.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> > > > p search.context[:panes]
> > > > # => [ThinkingSphinx::Panes::ExcerptsPane,
> > > > ThinkingSphinx::Panes::ExcerptsPane,
> > > > ThinkingSphinx::Panes::ExcerptsPane]
> > > >
> > > > --
> > > > JZ
> > > >
> > > > On Saturday, March 30, 2013 2:46:09 AM UTC, Pat Allan wrote:
> > > > I've actually been working on updating documentation for TS v3 over the
> > > > last 24 hours… though the changes are not online yet (as they're not
> > > > complete).
> > > > The excerpts method isn't added to search results by default, but it's
> > > > easily enabled:
> > > >
> > > > search = MyModel.search 'phrase'
> > > > search.context[:panes] << ThinkingSphinx::Panes::ExcerptsPane
> > > > search.first.excerpts
> > > >
> > > > Cheers
> > > >
> > > > --
> > > > Pat
> > > >
> > > > On 30/03/2013, at 1:39 PM, hipertracker wrote:
> > > >
> > > > > How to use excerpts in TS3? http://pat.github.com/ts/en/excerpts.html
> > > > > is old and does not work in newer TS3.
> > > > >
> > > > > Thinking Sphinx 2.0.11:
> > > > >
> > > > > MyModel.search('phrase').first.excerpts
> > > > > # => #<ThinkingSphinx::Excerpter:0x3fd79290348c>
> > > > >
> > > > > Thinking Sphinx 3.0.2:
> > > > >
> > > > > MyModel.search('phrase').first.excerpts
> > > > > NoMethodError: undefined method `excerpts' for
> > > > > #<Verse:0x007fa5dde723d8>
> > > > >
> > > > > --
> > > > > JZ
> >
> >
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Thinking Sphinx" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [email protected].
> > To post to this group, send email to [email protected].
> > Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Thinking Sphinx" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
