Hi Hendrik,
Thanks for your input. Previously I was using the hard commit
(SolrClient.commit()) but then I got some error when there are concurrent
real time index requests from my app. The error was "Exceeded limit of
maxWarmingSearchers=2, try again later", then i changed the code to use
only solrserver.add(docs) method and configured autoSoftCommit timeout and
autoCommit timeout in solrConfig.
I think, i'll get the same error when I use the method you described
(SolrClient.commit(String
collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)),
Anyway i'll have a look at that method.
Best regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893 <+65%209389%206893>
Blog : techreadme.blogspot.com
On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <hendrik.hadd...@gmx.net>
wrote:
> Hi,
>
> the SolrJ API has this method: SolrClient.commit(String collection,
> boolean waitFlush, boolean waitSearcher, boolean softCommit).
> My assumption so far was that when you set waitSearcher to true that the
> method call only returns once a search would find the new data, which
> sounds what you want. I used this already and it seemed to work just fine.
>
> regards,
> Hendrik
>
>
> On 19.12.2016 04:09, Lasitha Wattaladeniya wrote:
>
>> Hi all,
>>
>> Thanks for your replies,
>>
>> @dorian : the requirement is, we are showing a list of entries on a page.
>> For each user there's a read / unread flag. The data for listing is
>> fetched from solr. And you can see the entry was previously read or not.
>> So
>> when a user views an entry by clicking. We are updating the database flag
>> to READ and use real time indexing to update solr entry. So when the user
>> close the full view of the entry and go back to entry listing page, the
>> data fetched from solr should be updated to READ. That's the use case we
>> are trying to fix.
>>
>> @eric : thanks for the lengthy reply. So let's say I increase the
>> autosoftcommit time out to may be 100 ms. In that case do I have to wait
>> much that time from client side before calling search ?. What's the
>> correct way of achieving this?
>>
>> Regards,
>> Lasitha
>>
>> On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote:
>>
>> 1 ms autocommit is far too frequent. And it's not
>>> helping you anyway.
>>>
>>> There is some lag between when a commit happens
>>> and when the docs are really available. The sequence is:
>>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>>> 2> a new searcher is opened and autowarming starts
>>> 3> until the new searcher is opened, queries continue to be served by
>>> the old searcher
>>> 4> the new searcher is fully opened
>>> 5> _new_ requests are served by the new searcher.
>>> 6> the last request is finished by the old searcher and it's closed.
>>>
>>> So what's probably happening is that you send docs and then send a
>>> query and Solr is still in step <3>. You can look at your admin UI
>>> pluginst/stats page or your log to see how long it takes for a
>>> searcher to open and adjust your expectations accordingly.
>>>
>>> If you want to fetch only the document (not try to get it by a
>>> search), Real Time Get is designed to insure that you always get the
>>> most recent copy whether it's searchable or not.
>>>
>>> All that said, Solr wasn't designed for autocommits that are that
>>> frequent. That's why the documentation talks about _Near_ Real Time.
>>> You may need to adjust your expectations.
>>>
>>> Best,
>>> Erick
>>>
>>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com>
>>> wrote:
>>>
>>>> There's a very high probability that you're using the wrong tool for the
>>>> job if you need 1ms softCommit time. Especially when you always need it
>>>>
>>> (ex
>>>
>>>> there are apps where you need commit-after-insert very rarely).
>>>>
>>>> So explain what you're using it for ?
>>>>
>>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>>>>
>>> watt...@gmail.com>
>>>
>>>> wrote:
>>>>
>>>> Hi Furkan,
>>>>>
>>>>> Thanks for the links. I had read the first one but not the second one.
>>>>> I
>>>>> did read it after you sent. So in my current solrconfig.xml settings
>>>>>
>>>> below
>>>
>>>> are the configurations,
>>>>>
>>>>> <autoSoftCommit>
>>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>>>>> </autoSoftCommit>
>>>>>
>>>>>
>>>>> <autoCommit>
>>>>> <maxTime>15000</maxTime>
>>>>> <openSearcher>false</openSearcher>
>>>>> </autoCommit>
>>>>>
>>>>> The problem i'm facing is, just after adding the documents to solr
>>>>> using
>>>>> solrj, when I retrieve data from solr I am not getting the updated
>>>>>
>>>> results.
>>>
>>>> This happens time to time. Most of the time I get the correct data but
>>>>>
>>>> in
>>>
>>>> some occasions I get wrong results. so as you suggest, what the best
>>>>> practice to use here ? , should I wait 1 mili second before calling for
>>>>> updated results ?
>>>>>
>>>>> Regards,
>>>>> Lasitha
>>>>>
>>>>> Lasitha Wattaladeniya
>>>>> Software Engineer
>>>>>
>>>>> Mobile : +6593896893
>>>>> Blog : techreadme.blogspot.com
>>>>>
>>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> Hi Lasitha,
>>>>>>
>>>>>> First of all, did you check these:
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/solr/Near+
>>>>>>
>>>>> Real+Time+Searching
>>>>>
>>>>>> https://lucidworks.com/blog/2013/08/23/understanding-
>>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/
>>>>>>
>>>>>> after that, if you cannot adjust your configuration you can give more
>>>>>> information and we can find a solution.
>>>>>>
>>>>>> Kind Regards,
>>>>>> Furkan KAMACI
>>>>>>
>>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>>>>>>
>>>>> watt...@gmail.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> Hi furkan,
>>>>>>>
>>>>>>> Thanks for your reply, it is generally a query heavy system. We are
>>>>>>>
>>>>>> using
>>>>>
>>>>>> realtime indexing for editing the available data
>>>>>>>
>>>>>>> Regards,
>>>>>>> Lasitha
>>>>>>>
>>>>>>> Lasitha Wattaladeniya
>>>>>>> Software Engineer
>>>>>>>
>>>>>>> Mobile : +6593896893 <+65%209389%206893>
>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>
>>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
>>>>>>>
>>>>>> furkankam...@gmail.com>
>>>
>>>> wrote:
>>>>>>>
>>>>>>> Hi Lasitha,
>>>>>>>>
>>>>>>>> What is your indexing / querying requirements. Do you have an index
>>>>>>>> heavy/light - query heavy/light system?
>>>>>>>>
>>>>>>>> Kind Regards,
>>>>>>>> Furkan KAMACI
>>>>>>>>
>>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>>>>>>> watt...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello devs,
>>>>>>>>>
>>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a
>>>>>>>>>
>>>>>>>> commit
>>>
>>>> (soft
>>>>>>>>
>>>>>>>>> commit) through solrj and just after the commit, retrieve the data
>>>>>>>>>
>>>>>>>> from
>>>>>
>>>>>> solr (requirement is to get updated data list).
>>>>>>>>>
>>>>>>>>> I'm using soft commit instead of the hard commit, is previously I
>>>>>>>>>
>>>>>>>> got
>>>
>>>> an
>>>>>>>>
>>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>>>>>>>>
>>>>>>>> because of
>>>>>>>>
>>>>>>>>> too many commit requests. Now I have removed the explicit commit
>>>>>>>>>
>>>>>>>> and
>>>
>>>> has
>>>>>>>>
>>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili
>>>>>>>>>
>>>>>>>> second)*
>>>
>>>> and
>>>>>>>>
>>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>>>>>>>>
>>>>>>>> errors
>>>>>>>>
>>>>>>>>> when i'm committing frequently.
>>>>>>>>>
>>>>>>>>> The problem i'm facing now is, I'm not getting the updated data
>>>>>>>>>
>>>>>>>> when
>>>
>>>> I
>>>>>
>>>>>> fetch from solr just after the soft commit. So in this case what
>>>>>>>>>
>>>>>>>> are
>>>
>>>> the
>>>>>>>>
>>>>>>>>> best practices to use ? to wait 1 mili second before retrieving
>>>>>>>>>
>>>>>>>> data
>>>
>>>> after
>>>>>>>>
>>>>>>>>> soft commit ? I don't feel like waiting from client side is a good
>>>>>>>>>
>>>>>>>> option.
>>>>>>>>
>>>>>>>>> Please give me some help from your expert knowledge
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Lasitha Wattaladeniya
>>>>>>>>> Software Engineer
>>>>>>>>>
>>>>>>>>> Mobile : +6593896893
>>>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>
