[
https://issues.apache.org/jira/browse/LUCENE-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509288
]
Michael McCandless commented on LUCENE-848:
-------------------------------------------
> I think Mike mentioned not doing the one file per article. I'll try to look
> at that ...
I'm actually [slowly] working through a patch to contrib/benchmark
that adds a LineDocMaker that will open a single file and make one
document per line. (This is the follow-through on LUCENE-843 to merge
in the benchmarking tool that I used there, into contib/benchmark).
This is in order to do tests that aren't affected by the time to
decompress files/walk trees/open new files/etc. to build their
documents.
I will also include in the patch some way to run an existing DocMaker,
pull its documents, and store them into a single line file. It's
probably still worthwhile to have a DocMaker that can read the single
wikipedia XML file and produce documents directly from that to save
creating file-per-document in a large dir tree.
> Add supported for Wikipedia English as a corpus in the benchmarker stuff
> ------------------------------------------------------------------------
>
> Key: LUCENE-848
> URL: https://issues.apache.org/jira/browse/LUCENE-848
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/benchmark
> Reporter: Steven Parkes
> Assignee: Grant Ingersoll
> Priority: Minor
> Attachments: LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt,
> LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt,
> WikipediaHarvester.java, xerces.jar, xerces.jar, xml-apis.jar
>
>
> Add support for using Wikipedia for benchmarking.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
