I'm curious about strategies for managing Solr indexing failures. If there's an indexing problem, I'd like my application to retry the indexing operation.
I noticed that Chef uses RabbitMQ for (apparently) all Solr indexing operations. http://docs.opscode.com/server_components.html has this to say: "Messages are sent to the Search Index using the following components: 1. RabbitMQ is used as the message queue for the Chef server. All items that will be added to the search index repository are first added to a queue. 2. chef-expander is used to pull messages from the RabbitMQ queue, process them into the required format, and then post them to chef-solr for indexing." Is using a message broker such as RabbitMQ a recommended or common approach to managing Solr indexing failures? How about a job queue in a database table instead? Other ideas? If there's an indexing problem, I'd like to retry a reasonable number of times and have some reporting of these problems. Thanks in advance for any thoughts or pointers on this topic, Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin
