Hi Hendrik, Thanks for your input. Previously I was using the hard commit (SolrClient.commit()) but then I got some error when there are concurrent real time index requests from my app. The error was "Exceeded limit of maxWarmingSearchers=2, try again later", then i changed the code to use only solrserver.add(docs) method and configured autoSoftCommit timeout and autoCommit timeout in solrConfig.
I think, i'll get the same error when I use the method you described (SolrClient.commit(String collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)), Anyway i'll have a look at that method. Best regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 <+65%209389%206893> Blog : techreadme.blogspot.com On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <hendrik.hadd...@gmx.net> wrote: > Hi, > > the SolrJ API has this method: SolrClient.commit(String collection, > boolean waitFlush, boolean waitSearcher, boolean softCommit). > My assumption so far was that when you set waitSearcher to true that the > method call only returns once a search would find the new data, which > sounds what you want. I used this already and it seemed to work just fine. > > regards, > Hendrik > > > On 19.12.2016 04:09, Lasitha Wattaladeniya wrote: > >> Hi all, >> >> Thanks for your replies, >> >> @dorian : the requirement is, we are showing a list of entries on a page. >> For each user there's a read / unread flag. The data for listing is >> fetched from solr. And you can see the entry was previously read or not. >> So >> when a user views an entry by clicking. We are updating the database flag >> to READ and use real time indexing to update solr entry. So when the user >> close the full view of the entry and go back to entry listing page, the >> data fetched from solr should be updated to READ. That's the use case we >> are trying to fix. >> >> @eric : thanks for the lengthy reply. So let's say I increase the >> autosoftcommit time out to may be 100 ms. In that case do I have to wait >> much that time from client side before calling search ?. What's the >> correct way of achieving this? >> >> Regards, >> Lasitha >> >> On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote: >> >> 1 ms autocommit is far too frequent. And it's not >>> helping you anyway. >>> >>> There is some lag between when a commit happens >>> and when the docs are really available. The sequence is: >>> 1> commit (soft or hard-with-opensearcher=true doesn't matter). >>> 2> a new searcher is opened and autowarming starts >>> 3> until the new searcher is opened, queries continue to be served by >>> the old searcher >>> 4> the new searcher is fully opened >>> 5> _new_ requests are served by the new searcher. >>> 6> the last request is finished by the old searcher and it's closed. >>> >>> So what's probably happening is that you send docs and then send a >>> query and Solr is still in step <3>. You can look at your admin UI >>> pluginst/stats page or your log to see how long it takes for a >>> searcher to open and adjust your expectations accordingly. >>> >>> If you want to fetch only the document (not try to get it by a >>> search), Real Time Get is designed to insure that you always get the >>> most recent copy whether it's searchable or not. >>> >>> All that said, Solr wasn't designed for autocommits that are that >>> frequent. That's why the documentation talks about _Near_ Real Time. >>> You may need to adjust your expectations. >>> >>> Best, >>> Erick >>> >>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com> >>> wrote: >>> >>>> There's a very high probability that you're using the wrong tool for the >>>> job if you need 1ms softCommit time. Especially when you always need it >>>> >>> (ex >>> >>>> there are apps where you need commit-after-insert very rarely). >>>> >>>> So explain what you're using it for ? >>>> >>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya < >>>> >>> watt...@gmail.com> >>> >>>> wrote: >>>> >>>> Hi Furkan, >>>>> >>>>> Thanks for the links. I had read the first one but not the second one. >>>>> I >>>>> did read it after you sent. So in my current solrconfig.xml settings >>>>> >>>> below >>> >>>> are the configurations, >>>>> >>>>> <autoSoftCommit> >>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime> >>>>> </autoSoftCommit> >>>>> >>>>> >>>>> <autoCommit> >>>>> <maxTime>15000</maxTime> >>>>> <openSearcher>false</openSearcher> >>>>> </autoCommit> >>>>> >>>>> The problem i'm facing is, just after adding the documents to solr >>>>> using >>>>> solrj, when I retrieve data from solr I am not getting the updated >>>>> >>>> results. >>> >>>> This happens time to time. Most of the time I get the correct data but >>>>> >>>> in >>> >>>> some occasions I get wrong results. so as you suggest, what the best >>>>> practice to use here ? , should I wait 1 mili second before calling for >>>>> updated results ? >>>>> >>>>> Regards, >>>>> Lasitha >>>>> >>>>> Lasitha Wattaladeniya >>>>> Software Engineer >>>>> >>>>> Mobile : +6593896893 >>>>> Blog : techreadme.blogspot.com >>>>> >>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com >>>>> > >>>>> wrote: >>>>> >>>>> Hi Lasitha, >>>>>> >>>>>> First of all, did you check these: >>>>>> >>>>>> https://cwiki.apache.org/confluence/display/solr/Near+ >>>>>> >>>>> Real+Time+Searching >>>>> >>>>>> https://lucidworks.com/blog/2013/08/23/understanding- >>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/ >>>>>> >>>>>> after that, if you cannot adjust your configuration you can give more >>>>>> information and we can find a solution. >>>>>> >>>>>> Kind Regards, >>>>>> Furkan KAMACI >>>>>> >>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya < >>>>>> >>>>> watt...@gmail.com> >>>>> >>>>>> wrote: >>>>>> >>>>>> Hi furkan, >>>>>>> >>>>>>> Thanks for your reply, it is generally a query heavy system. We are >>>>>>> >>>>>> using >>>>> >>>>>> realtime indexing for editing the available data >>>>>>> >>>>>>> Regards, >>>>>>> Lasitha >>>>>>> >>>>>>> Lasitha Wattaladeniya >>>>>>> Software Engineer >>>>>>> >>>>>>> Mobile : +6593896893 <+65%209389%206893> >>>>>>> Blog : techreadme.blogspot.com >>>>>>> >>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI < >>>>>>> >>>>>> furkankam...@gmail.com> >>> >>>> wrote: >>>>>>> >>>>>>> Hi Lasitha, >>>>>>>> >>>>>>>> What is your indexing / querying requirements. Do you have an index >>>>>>>> heavy/light - query heavy/light system? >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> Furkan KAMACI >>>>>>>> >>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya < >>>>>>>> watt...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello devs, >>>>>>>>> >>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a >>>>>>>>> >>>>>>>> commit >>> >>>> (soft >>>>>>>> >>>>>>>>> commit) through solrj and just after the commit, retrieve the data >>>>>>>>> >>>>>>>> from >>>>> >>>>>> solr (requirement is to get updated data list). >>>>>>>>> >>>>>>>>> I'm using soft commit instead of the hard commit, is previously I >>>>>>>>> >>>>>>>> got >>> >>>> an >>>>>>>> >>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later" >>>>>>>>> >>>>>>>> because of >>>>>>>> >>>>>>>>> too many commit requests. Now I have removed the explicit commit >>>>>>>>> >>>>>>>> and >>> >>>> has >>>>>>>> >>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili >>>>>>>>> >>>>>>>> second)* >>> >>>> and >>>>>>>> >>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any >>>>>>>>> >>>>>>>> errors >>>>>>>> >>>>>>>>> when i'm committing frequently. >>>>>>>>> >>>>>>>>> The problem i'm facing now is, I'm not getting the updated data >>>>>>>>> >>>>>>>> when >>> >>>> I >>>>> >>>>>> fetch from solr just after the soft commit. So in this case what >>>>>>>>> >>>>>>>> are >>> >>>> the >>>>>>>> >>>>>>>>> best practices to use ? to wait 1 mili second before retrieving >>>>>>>>> >>>>>>>> data >>> >>>> after >>>>>>>> >>>>>>>>> soft commit ? I don't feel like waiting from client side is a good >>>>>>>>> >>>>>>>> option. >>>>>>>> >>>>>>>>> Please give me some help from your expert knowledge >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Lasitha Wattaladeniya >>>>>>>>> Software Engineer >>>>>>>>> >>>>>>>>> Mobile : +6593896893 >>>>>>>>> Blog : techreadme.blogspot.com >>>>>>>>> >>>>>>>>> >>>>>>> >