Where can I find any sample code or documentation about merging a set of small indexes into one big index?
Lixin -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Monday, June 30, 2003 10:24 AM To: Lucene Users List Subject: Re: parallizing index building Marc Dumontier wrote: > I'm indexing 500 XML files each ~150Mb on an 8 CPU machine. > > I'm wondering what the best strategy for making maximum use of resources is. I have the tweaked the single process indexer to index 5000 records (not files) in memory before writing out to disk. > > Should i create an IndexThread and share the IndexWriter object across 5 threads..then monitor when one ends to start another, etc. Or should i create difference indexes then to a series of merges. Creating multiple indexes in parallel and then merging them at the end will probably be fastest. Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
