2007/4/3, Chun Wei Ho <[EMAIL PROTECTED]>: > As the index have been growing in size, we have been noticing that the > search response time on the searcher machine increases drastically > when an index (about 15GB) is being copied from the crawler to the > searcher. Both machines run Fedora Core 4 and are on a gbps lan. > > We've tried a number of ways to reduce the impact of the copy over NFS > on searching performance, such as "nice"ing the copy process, but to > no avail. I wonder if anyone is running a lucene search service over a > similar architecture and how you are managing the updates to the > lucene index.
Have you tried limiting the transfer rate? If you don't mind the additional index refresh latency, it might be a simple way to give more CPU and disk IO time to the search service...you could pull (wget --limit-rate=1M index) instead of push (cp src dest) the index and in doing so, get the added flexibility of controling the copy rate. Cheers, t.n.a. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
