[ https://issues.apache.org/jira/browse/LUCENE-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509288 ]
Michael McCandless commented on LUCENE-848: ------------------------------------------- > I think Mike mentioned not doing the one file per article. I'll try to look > at that ... I'm actually [slowly] working through a patch to contrib/benchmark that adds a LineDocMaker that will open a single file and make one document per line. (This is the follow-through on LUCENE-843 to merge in the benchmarking tool that I used there, into contib/benchmark). This is in order to do tests that aren't affected by the time to decompress files/walk trees/open new files/etc. to build their documents. I will also include in the patch some way to run an existing DocMaker, pull its documents, and store them into a single line file. It's probably still worthwhile to have a DocMaker that can read the single wikipedia XML file and produce documents directly from that to save creating file-per-document in a large dir tree. > Add supported for Wikipedia English as a corpus in the benchmarker stuff > ------------------------------------------------------------------------ > > Key: LUCENE-848 > URL: https://issues.apache.org/jira/browse/LUCENE-848 > Project: Lucene - Java > Issue Type: New Feature > Components: contrib/benchmark > Reporter: Steven Parkes > Assignee: Grant Ingersoll > Priority: Minor > Attachments: LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt, > LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt, LUCENE-848.txt, > WikipediaHarvester.java, xerces.jar, xerces.jar, xml-apis.jar > > > Add support for using Wikipedia for benchmarking. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]