re: StreamingUpdateSolrServer - connection refused

[Poulton, Gareth | Gareth | DU]

Hi all,
I’m having basically the exact same problem someone described in this email to 
the list from just over a year ago (see below). The only suggested solution 
given on the thread at the time was to ping the server before sending an add, 
which I’m not particularly keen on; least of all because I don’t know that 
there couldn’t be situations where a ping might succeed where an add wouldn’t.

My current solution is to switch to CommonsHttpSolrServer, which is slower, and 
I’d rather not have to reinvent the wheel by implementing my own 
threading/buffering solution with it when one theoretically already exists in 
solr.

So, any thoughts on how to proceed? I see an UpdateResponse.getStatus() in the 
3.5 documentation but it’s not described in any detail and also doesn’t appear 
in my current 4.0 build.

Cheers,
Gareth


On 2010-12-30, 16:10, Christopher Gross wrote:
Hi all.

  I have designed a synchronizer that goes out to various databases,
extracts some data, does some processing, and then uses the
StreamingUpdateSolrServer to send the records to a Solr index.  When
everything is up, it works just fine.

  Now I'm trying to account for problems, like if the Solr index is down for
some reason.  I'm trying to get it so that I can keep track of items that
were missed for synchronization, so that they can get replayed when the
index comes back up.  Before I was using the StreamingUpdateSolrServer, I'd
get a message back immediately that there was a problem.  Now that it
streams, the error comes outside of the thread that I'm using to run the
updates, so I can't catch the exception, so my thread isn't aware of the
error, and just chugs along.

  Has anyone run into this problem?  Is there anything built into the server
so that I can know which records it was unable to send, and then pull them
out in order to try running them again later?  Any insight that anyone has
would be greatly appreciated.

  Thanks! Christopher Gross

-- Christopher Gross


Gareth Poulton | ポルトン ガレフ
開発部・次世代サーチグループ
gareth.poul...@mail.rakuten.com

Re: StreamingUpdateSolrServer - connection refused

[Christopher Gross]

My current solution is to use the ping() function -- which doesn't run in a
thread -- to test the connection before trying to send the data to the Solr
index.  It isn't elegant, but it works.

If anyone has a better idea, I'd like to hear it.

-- Chris


On Thu, Dec 30, 2010 at 11:10 AM, Christopher Gross wrote:

> Hi all.
>
>   I have designed a synchronizer that goes out to various databases,
> extracts some data, does some processing, and then uses the
> StreamingUpdateSolrServer to send the records to a Solr index.  When
> everything is up, it works just fine.
>
>   Now I'm trying to account for problems, like if the Solr index is down
> for some reason.  I'm trying to get it so that I can keep track of items
> that were missed for synchronization, so that they can get replayed when the
> index comes back up.  Before I was using the StreamingUpdateSolrServer, I'd
> get a message back immediately that there was a problem.  Now that it
> streams, the error comes outside of the thread that I'm using to run the
> updates, so I can't catch the exception, so my thread isn't aware of the
> error, and just chugs along.
>
>   Has anyone run into this problem?  Is there anything built into the
> server so that I can know which records it was unable to send, and then pull
> them out in order to try running them again later?  Any insight that anyone
> has would be greatly appreciated.
>
>   Thanks!
>
> -- Christopher Gross
>