Hi! Erik,
Yes basic seems to be working.
a) My problem is there is a chances that query is not present in
stored content of a file so sometimes I am getting empty strings at
line#106 so I have to put a special check at line#109 and line#126. I
guess this is not a problem. What you think?
b) When I click on a doc path that was generated by line#120 and
line#121 The files that it open do not have a searched query
highlighted. Any suggestion for this? How I can do?
-----Original Message-----
From: Erik Hatcher [mailto:[EMAIL PROTECTED]
Sent: Monday, April 04, 2005 8:45 PM
To: java-user@lucene.apache.org
Subject: Re: HTML pages highlighter
On Apr 4, 2005, at 5:35 PM, Yagnesh Shah wrote:
I end up purchasing your book "Lucene in Action". I have downloaded
your code samples. I am able to retrieve "result" only some time.
Below is the code I have taken from Search.jhtml in lucene demo. I
have 2 problem
a) I am unable to display "result" using
b) When I click on the title to retrieve document I do not see my
query highlighted.
First things first.... get something very very simple working and
expand from there. Here is the simple code from our HighlightIt.java:
TermQuery query = new TermQuery(new Term("f", "ipsum"));
QueryScorer scorer = new QueryScorer(query);
SimpleHTMLFormatter formatter =
new SimpleHTMLFormatter("<span class=\"highlight\">",
"</span>");
Highlighter highlighter = new Highlighter(formatter, scorer);
Fragmenter fragmenter = new SimpleFragmenter(50);
highlighter.setTextFragmenter(fragmenter);
TokenStream tokenStream = new StandardAnalyzer()
.tokenStream("f", new StringReader(text));
String result =
highlighter.getBestFragments(tokenStream, text, 5, "...");
One trick is that you must ensure the query you are passing to
QueryScorer has been rewritten. In our simple TermQuery case, that is
not necessary, but in a general application it is. You can call
query.rewrite(reader) where reader is your IndexReader instance. This
ensures that range, fuzzy, and wildcard queries are expanded and
highlightable.
I'm not sure what is wrong with the code you are trying. But again,
start simple, just try out our HighlightIt or our HighlightTest. If
those work fine for you then move on to integrating further with your
index. Besides the Query.rewrite() trick, you have to be sure that the
text you want to highlight is available. If you're pulling it from the
index, it must be in a stored field, otherwise you need to retrieve it
from elsewhere.
Erik
<java>
Searcher searcher = new IndexSearcher(getReader(indexName));
// get query from request
String queryString = request.getParameter("query");
query = QueryParser.parse(queryString, "contents", analyzer);
Hits hits = searcher.search(query);
SimpleHTMLFormatter formatter =
new SimpleHTMLFormatter();
Highlighter highlighter = new Highlighter(formatter, new
QueryScorer(query));
highlighter.setTextFragmenter(new SimpleFragmenter(50));
String FIELD_NAME = "contents";
for (int i = start; i < end; i++) { // display the hits
Document doc = hits.doc(i);
String text = hits.doc(i).get(FIELD_NAME);
int maxNumFragmentsRequired = 5;
String fragmentSeparator = "...";
if ( text != null){
TokenStream tokenStream = new
StandardAnalyzer().tokenStream(FIELD_NAME, new
java.io.StringReader(text));
String result =
highlighter.getBestFragments
(tokenStream,text,maxNumFragmentsRequired,fragmentSeparator);
System.out.println("result=" +result);
}
String title = doc.get("title");
if (title.equals("")) // use url for docs
w/o title
title = doc.get("path");
</java>
<p><b><java type=print>(int)(hits.score(i) * 100.0f)</java>%
<a href="`doc.get("path")`">
<java type=print>Entities.encode(title)</java>
</b></a>
<java>
if (showSummaries) { // maybe show
summary
</java>
<ul><i>Summary</i>:
<java type=print>Entities.encode(doc.get("summary"))</java>
</ul>
<java>
}
}
</java>
-----Original Message-----
From: Erik Hatcher [mailto:[EMAIL PROTECTED]
Sent: Thursday, March 31, 2005 8:04 PM
To: java-user@lucene.apache.org
Subject: Re: HTML pages highlighter
On Mar 31, 2005, at 6:36 PM, Yagnesh Shah wrote:
try {
fis = new FileInputStream(f);
HTMLParser parser = new HTMLParser(fis);
// Add the tag-stripped contents as a Reader-valued Text field
so it will
// get tokenized and indexed.
// doc.add(new Field("contents", parser.getReader()));
LineNumberReader reader = new
LineNumberReader(parser.getReader());
for (String l = reader.readLine(); l != null; l =
reader.readLine())
// System.out.println(l);
doc.add(Field.Text("contents", l));
Notice that your loop here is adding a "contents" field for *every*
line read since that is where the first semi-colon is.
Look at using Luke to explore your index. Try indexing just a dummy
String:
doc.add(Field.Text("contents", "some dummy text"));
to show that it works. Always always always simplify a complicated
situation by doing the most obvious thing that _should_ work.
Also, the demo Lucene code is not really designed to be used in a
production application (sadly), so you're better off borrowing code
from the many articles or our book to begin with.
Erik
// Add the summary as a field that is stored and returned with
// hit documents for display.
doc.add(new Field("summary", parser.getSummary(),
Field.Store.YES, Field.Index.NO));
// Add the title as a field that it can be searched and that is
stored.
doc.add(new Field("title", parser.getTitle(), Field.Store.YES,
Field.Index.TOKENIZED));
}
-----Original Message-----
From: Erik Hatcher [mailto:[EMAIL PROTECTED]
Sent: Wednesday, March 30, 2005 7:38 PM
To: java-user@lucene.apache.org
Subject: Re: HTML pages highlighter
On Mar 30, 2005, at 4:46 PM, Yagnesh Shah wrote:
Hi! Eric,
Erik - with a 'k' - Sorry, I let it slide once though :)
I try to modified that with this but I get compile error. Do you
have
any code snippet of highlighting code to pull the contents from the
original source?
I have a whole book full of code examples :)
http://www.lucenebook.com - Grab the source code and look in
src/lia/tools at Highlight*.java
or Do you know how I can do field store?
doc.add(new Field("contents", parser.getReader(),
Field.Store.YES, Field.Index.NO));
You cannot store it with a Reader. You need to use
Field.Text(String,
String), or one of the other variations.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
