Why don't you check out Hadoop and Nutch? It should provide what you are looking for.
Zach Bailey wrote: > > Hi, > > It's been a couple of days now and I haven't heard anything on this > topic, while there has been substantial list traffic otherwise. > > Am I asking in the wrong place? Was I unclear? > > I know there are people out there that have used/are using Lucene in a > clustered environment. I am just looking for any sort of feedback > (general or specific) about clustering lucene as well as filesystem > compatibility (windows shares, NFS, etc.). > > Thanks again, > -Zach > > Zach Bailey wrote: >> Hello all, >> >> First a little background - we are developing a clustered application >> that will in part leverage Lucene to provide index and search >> capabilities. We have already spent time investigating various index >> storage implementations (database vs. filesystem) and we've decided for >> performance reasons to go with a filesystem index storage scheme. >> >> That said, I have read back through the archives a bit and noticed that >> the support for index storage on NFS is still experimental (e.g. the >> latest bugfixes have not made it out to an official, stable release). I >> realize most of the issues related to using a shared file system revolve >> around locking, and I haven't seen much about the maturity of locking >> for other network filesystems. >> >> I was wondering if anyone has tried any other networked filesystems or >> had any recommendations. We have clients who would be doing this on both >> Windows and Unix/Linux so any insight there would be appreciated as well >> - it can be assumed that across any cluster the operating system use >> would be homogeneous (i.e. all nodes are on windows and would use >> windows shares, or all nodes are on linux and would use xyz filesystem). >> >> Thanks in advance, >> -Zach Bailey >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/Clustered-Indexing-on-common-network-filesystem-tf4194135.html#a11966423 Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
