Sorry for late reply. I was trying to change our indexing pipeline and do
explicit intermediate commits for each core. That turned out to be a bit
more work that I have time for.
So, I do want to explore hard commits. I tried
<solr-host>:<port>/solr/<core>/*update?commit=true* . But there is no
impact on Txn Log size, so I feel, it must be getting ignored.
So can someone tell me, how to do the Hard Commits ?
@Shawn : openSearcher = false is not an option. On Each commit, index will
be replicated to Slaves which will have a searcher on it immediately and
can intermediate state. The longer term and better solution is changing
indexing pipeline and doing explicit commits, but I can't implement that
right now.
On 18 Apr 2013 00:35, "Shawn Heisey" <s...@elyograg.org> wrote:
> On 4/17/2013 11:56 AM, Mark Miller wrote:
>
>> There is one additional caveat - when you disable the updateLog, you have
>>> to switch to MMapDirectoryFactory instead of NRTCachingDirectoryFactory.
>>> The NRT directory implementation will cache a portion of a commit
>>> (including hard commits) into RAM instead of onto disk. On the next
>>> commit, the previous one is persisted completely to disk. Without a
>>> transaction log, you can lose data.
>>>
>>
>> I don't think this is true? NRTCachingDirectoryFactory should not cache
>> hard commits and should be as safe as MMapDirectoryFactory is - neither of
>> which is as safe as using a tran log.
>>
>
> This is based on observations of what happens with my segment files when I
> do a full-import, using autoCommit with openSearcher disabled. I see that
> each autoCommit results in a full segment being written, the part of
> another segment. On the next autoCommit, the rest of the files for the
> last segment are written, another full segment is written, I get another
> partial segment. I asked about this on the list some time ago, and what I
> just told Umesh is a rehash of what I understood from Yonik's response.
>
> If I'm wrong, I hope someone who knows for sure can correct me.
>
> Thanks,
> Shawn
>
>
