[jira] Commented: (NUTCH-79) Fault tolerant searching.
[ http://issues.apache.org/jira/browse/NUTCH-79?page=comments#action_12364496 ] Piotr Kosiorowski commented on NUTCH-79: I think it should work without changes I suggested in previous comment - they would be simply useful additions. I was not using it for quite a while so I would get back to it to make sure it works with latest code (I hope sooner than later) - but no promises at the moment Fault tolerant searching. - Key: NUTCH-79 URL: http://issues.apache.org/jira/browse/NUTCH-79 Project: Nutch Type: New Feature Components: searcher Reporter: Piotr Kosiorowski Attachments: patch I have finally managed to prepare first version of fault tolerant searching I have promised long time ago. It reads server configuration from search-groups.txt file (in startup directory or directory specified by searcher.dir) if no search-servers.txt file is present. If search-servers.txt is presentit would be read and handled as previously. --- Format of search-groups.txt: * pre * search.group.count=[int] * search.group.name.[i]=[string] (for i=0 to count-1) * * For each name: * [name].part.count=[int] partitionCount * [name].part.[i].host=[string] (for i=0 to partitionCount-1) * [name].part.[i].port=int (for i=0 to partitionCount-1) * * Example: * search.group.count=2 * search.group.name.0=master * search.group.name.1=backup * * master.part.count=2 * master.part.0.host=host1 * master.part.0.port= * master.part.1.host=host2 * master.part.1.port= * * backup.part.count=2 * backup.part.0.host=host3 * backup.part.0.port= * backup.part.1.host=host4 * backup.part.1.port= * /pre. If more than one search group is defined in configuration file requests are distributed among groups in round-robin fashion. If one of the servers from the group fails to respond the whole group is treated as inactive and removed from the pool used to distributed requests. There is a separate recovery thread that every searcher.recovery.delay seconds (default 60) tries to check if inactive became alive and if so adds it back to the pool of active groups. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-79) Fault tolerant searching.
[ http://issues.apache.org/jira/browse/NUTCH-79?page=comments#action_12364357 ] byron miller commented on NUTCH-79: --- Piotr, Any update on this? Have you been able to run with this or still working out the kinks? Fault tolerant searching. - Key: NUTCH-79 URL: http://issues.apache.org/jira/browse/NUTCH-79 Project: Nutch Type: New Feature Components: searcher Reporter: Piotr Kosiorowski Attachments: patch I have finally managed to prepare first version of fault tolerant searching I have promised long time ago. It reads server configuration from search-groups.txt file (in startup directory or directory specified by searcher.dir) if no search-servers.txt file is present. If search-servers.txt is presentit would be read and handled as previously. --- Format of search-groups.txt: * pre * search.group.count=[int] * search.group.name.[i]=[string] (for i=0 to count-1) * * For each name: * [name].part.count=[int] partitionCount * [name].part.[i].host=[string] (for i=0 to partitionCount-1) * [name].part.[i].port=int (for i=0 to partitionCount-1) * * Example: * search.group.count=2 * search.group.name.0=master * search.group.name.1=backup * * master.part.count=2 * master.part.0.host=host1 * master.part.0.port= * master.part.1.host=host2 * master.part.1.port= * * backup.part.count=2 * backup.part.0.host=host3 * backup.part.0.port= * backup.part.1.host=host4 * backup.part.1.port= * /pre. If more than one search group is defined in configuration file requests are distributed among groups in round-robin fashion. If one of the servers from the group fails to respond the whole group is treated as inactive and removed from the pool used to distributed requests. There is a separate recovery thread that every searcher.recovery.delay seconds (default 60) tries to check if inactive became alive and if so adds it back to the pool of active groups. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
