[
https://issues.apache.org/jira/browse/LUCENE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985420#action_12985420
]
Doron Cohen commented on LUCENE-1540:
-------------------------------------
Hi Shai, thanks for reviewing!
I agree about making read() public, and same for parseDate().
As we discussed offline the interface with TrecParser is not ideal - I looked
at the option we discussed to have the TrecContentSource just read everything
between <Doc> and </Doc> and then let the TrecDocParser do everything - in one
call to it - but as this would mean going through the data 3 times (comparing
to 2 times today) this is not appealing either and I rather stay with the two
methods interface for now.
> Improvements to contrib.benchmark for TREC collections
> ------------------------------------------------------
>
> Key: LUCENE-1540
> URL: https://issues.apache.org/jira/browse/LUCENE-1540
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Affects Versions: 2.4
> Reporter: Tim Armstrong
> Assignee: Doron Cohen
> Priority: Minor
> Attachments: LUCENE-1540.patch
>
>
> The benchmarking utilities for TREC test collections (http://trec.nist.gov)
> are quite limited and do not support some of the variations in format of
> older TREC collections.
> I have been doing some benchmarking work with Lucene and have had to modify
> the package to support:
> * Older TREC document formats, which the current parser fails on due to
> missing document headers.
> * Variations in query format - newlines after <title> tag causing the query
> parser to get confused.
> * Ability to detect and read in uncompressed text collections
> * Storage of document numbers by default without storing full text.
> I can submit a patch if there is interest, although I will probably want to
> write unit tests for the new functionality first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
