DIH Commit Issue

[AJ Lemke]

Hi All,

I have a DIH issue where the index will commit after 1 and 2 minutes then will 
not commit again until the end.
We would like the commit to happen at the end so the index does not lose 75% or 
more of the records until the end of the process.
We went from 370,000+ records to around 27,000 records then back to 370,000+ 
when the process ended.
I have change the update handler to the following.



${solr.ulog.dir:}






Is there something else that I should do?

Thanks All!
AJ

Re: DIH Commit Issue

[Erick Erickson]

I would set the times in the autoCommit to a large number (or -1 I
think). It's possible that there's a default there if the autocommit
section is found but nothing specified, you'll have to look at the
code to be sure.

But what I would do is use aliasing (either core if you're in
stand-alone or collection if you're in SolrCloud). Index to the
offline collection, and when you're satisfied switch the alias. That
way these updates are all atomic.

Best,
Erick

On Thu, Dec 22, 2016 at 2:38 PM, AJ Lemke  wrote:
> Hi All,
>
> I have a DIH issue where the index will commit after 1 and 2 minutes then 
> will not commit again until the end.
> We would like the commit to happen at the end so the index does not lose 75% 
> or more of the records until the end of the process.
> We went from 370,000+ records to around 27,000 records then back to 370,000+ 
> when the process ended.
> I have change the update handler to the following.
>
> 
> 
> ${solr.ulog.dir:}
> 
> 
> 
> 
>
>
> Is there something else that I should do?
>
> Thanks All!
> AJ