[ http://issues.apache.org/jira/browse/NUTCH-79?page=comments#action_12364496 ]
Piotr Kosiorowski commented on NUTCH-79: ---------------------------------------- I think it should work without changes I suggested in previous comment - they would be simply useful additions. I was not using it for quite a while so I would get back to it to make sure it works with latest code (I hope sooner than later) - but no promises at the moment > Fault tolerant searching. > ------------------------- > > Key: NUTCH-79 > URL: http://issues.apache.org/jira/browse/NUTCH-79 > Project: Nutch > Type: New Feature > Components: searcher > Reporter: Piotr Kosiorowski > Attachments: patch > > I have finally managed to prepare first version of fault tolerant searching I > have promised long time ago. > It reads server configuration from search-groups.txt file (in startup > directory or directory specified by searcher.dir) if no search-servers.txt > file is present. If search-servers.txt is presentit would be read and > handled as previously. > --------------------------------------------------- > Format of search-groups.txt: > * <pre> > * search.group.count=[int] > * search.group.name.[i]=[string] (for i=0 to count-1) > * > * For each name: > * [name].part.count=[int] partitionCount > * [name].part.[i].host=[string] (for i=0 to partitionCount-1) > * [name].part.[i].port=int (for i=0 to partitionCount-1) > * > * Example: > * search.group.count=2 > * search.group.name.0=master > * search.group.name.1=backup > * > * master.part.count=2 > * master.part.0.host=host1 > * master.part.0.port=7777 > * master.part.1.host=host2 > * master.part.1.port=7777 > * > * backup.part.count=2 > * backup.part.0.host=host3 > * backup.part.0.port=7777 > * backup.part.1.host=host4 > * backup.part.1.port=7777 > * </pre>. > ------------------------------------------------ > If more than one search group is defined in configuration file requests are > distributed among groups in round-robin fashion. If one of the servers from > the group fails to respond the whole group is treated as inactive and removed > from the pool used to distributed requests. There is a separate recovery > thread that every "searcher.recovery.delay" seconds (default 60) tries to > check if inactive became alive and if so adds it back to the pool of active > groups. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
