Created an issue for this:
https://issues.apache.org/jira/browse/SOLR-882
and added the patch that adds a trigger to disable padding and (!) fixes a bug
in the current code -- hex. entities are not properly padded.
Dawid
Dawid Weiss wrote:
Hi. Let met do this -- I'll provide the first patch that:
- addds entities which be written all in uppercase and are displayed
correctly by browsers (amp, gt, lt, copy),
- an optional argument that would prevent emitting additional spaces if
these are not needed.
Then I'll look at the CharFilter/CharStream and see what can be done
about it.
Dawid
Yonik Seeley wrote:
Perhaps the new CharFilter/CharStream stuff would work for this (i.e.
HTMLStripReader modified to implement corrective offsets instead of
inserting whitespace)?
https://issues.apache.org/jira/browse/SOLR-822
-Yonik
On Sat, Nov 22, 2008 at 4:31 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
Thanks Grant. You mean this issue:
https://issues.apache.org/jira/browse/SOLR-42, I see now. This is a
problem
for me only, I guess, because I use HTMLStripReader independently of the
Lucene architecture. This class is public, would it make sense if I
provided
a patch that would switch the whitespace emitting functionality on
and off,
depending on a particular person's use case?
Dawid
Grant Ingersoll wrote:
It is an attempt at making things work properly with the highlighter
(such
that offsets are correct). I believe it works most of the time, but
there
still might be a few issues, check JIRA.
-Grant
On Nov 21, 2008, at 5:29 PM, Dawid Weiss wrote:
Hi folks. What's the motivation to add exactly the number of white
spaces
after an entity declaration in HTMLStripReader? It basically looks
like
this:
"lód"
(UTF: lód, "ice" in Polish) is translated into:
"ló d"
This happens both with numeric entities and named entities.
Needless to
say, these added spaces in the character stream do no good as they
effectively split a single term "lód" into two meaningless terms
"l" and
"d".
I can fix this in the code easily, but it looks like it was
intentional,
so before I write test cases and commit a JIRA issue I would like to
understand what the original reasons might have been (I really
don't see
anything this would be useful for). Apologies if I'm being dim here.
Dawid
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
