Re: Compound file format file size question
Otis, Thanks for the response. Yeah, I was copying the file to a brand new hard-drive and it was formatted to FAT32 by default, which is probably why it couldn't handle the 13GB file. I'm converting the drive to NTFS now, which should get me through temporarily. In the future though, I may break the index up into smaller sub-indexes so that I can distribute them across seperate physical disks for better disk IO. Thanks for your help! Jim --- Otis Gospodnetic <[EMAIL PROTECTED]> wrote: > Hello, > > --- James Dunn <[EMAIL PROTECTED]> wrote: > > Hello all, > > > > I have an index that's about 13GB on disk. I'm > using > > 1.4 rc3 which uses the compound file format by > > default. > > > > Once I run optimize on my index, it creates one > 13GB > > ..cfs file. This isn't a problem on Linux (yet), > but > > I'm having some trouble copying the file over to > my > > Windows XP box. > > What is the exact problem? The sheer size of it or > something else? > Just curious... > > > Is there some way using the compound file format > to > > set the maximum file size and have Lucene break > the > > index into multiple files once it hits that limit? > > Can't be done with Lucene, but I seem to recall some > discussion about > it. Nothing concrete, though. > > > Or do I need to go back to using the non-compound > file > > format? > > The total size should be (about) the same, but you > could certainly do > that, if having more smaller files is better for > you. > > Otis > > > Another solution, I suppose, would be to break up > my > > index into seperate smaller indexes. This would > be my > > second choice, however. > > > > Thanks a lot, > > > > Jim > > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Compound file format file size question
Hello, --- James Dunn <[EMAIL PROTECTED]> wrote: > Hello all, > > I have an index that's about 13GB on disk. I'm using > 1.4 rc3 which uses the compound file format by > default. > > Once I run optimize on my index, it creates one 13GB > ..cfs file. This isn't a problem on Linux (yet), but > I'm having some trouble copying the file over to my > Windows XP box. What is the exact problem? The sheer size of it or something else? Just curious... > Is there some way using the compound file format to > set the maximum file size and have Lucene break the > index into multiple files once it hits that limit? Can't be done with Lucene, but I seem to recall some discussion about it. Nothing concrete, though. > Or do I need to go back to using the non-compound file > format? The total size should be (about) the same, but you could certainly do that, if having more smaller files is better for you. Otis > Another solution, I suppose, would be to break up my > index into seperate smaller indexes. This would be my > second choice, however. > > Thanks a lot, > > Jim - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Compound file format file size question
Hello all, I have an index that's about 13GB on disk. I'm using 1.4 rc3 which uses the compound file format by default. Once I run optimize on my index, it creates one 13GB .cfs file. This isn't a problem on Linux (yet), but I'm having some trouble copying the file over to my Windows XP box. Is there some way using the compound file format to set the maximum file size and have Lucene break the index into multiple files once it hits that limit? Or do I need to go back to using the non-compound file format? Another solution, I suppose, would be to break up my index into seperate smaller indexes. This would be my second choice, however. Thanks a lot, Jim - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
