You probably need a database instead of a search engine. What requirement makes you want to do this with a search engine?
wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 19, 2016, at 6:34 PM, Lasitha Wattaladeniya <watt...@gmail.com> wrote: > > Hi Hendrik, > > Thanks for your input. Previously I was using the hard commit > (SolrClient.commit()) but then I got some error when there are concurrent > real time index requests from my app. The error was "Exceeded limit of > maxWarmingSearchers=2, try again later", then i changed the code to use > only solrserver.add(docs) method and configured autoSoftCommit timeout and > autoCommit timeout in solrConfig. > > I think, i'll get the same error when I use the method you described > (SolrClient.commit(String > collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)), > Anyway i'll have a look at that method. > > Best regards, > Lasitha > > Lasitha Wattaladeniya > Software Engineer > > Mobile : +6593896893 <+65%209389%206893> > Blog : techreadme.blogspot.com > > On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <hendrik.hadd...@gmx.net> > wrote: > >> Hi, >> >> the SolrJ API has this method: SolrClient.commit(String collection, >> boolean waitFlush, boolean waitSearcher, boolean softCommit). >> My assumption so far was that when you set waitSearcher to true that the >> method call only returns once a search would find the new data, which >> sounds what you want. I used this already and it seemed to work just fine. >> >> regards, >> Hendrik >> >> >> On 19.12.2016 04:09, Lasitha Wattaladeniya wrote: >> >>> Hi all, >>> >>> Thanks for your replies, >>> >>> @dorian : the requirement is, we are showing a list of entries on a page. >>> For each user there's a read / unread flag. The data for listing is >>> fetched from solr. And you can see the entry was previously read or not. >>> So >>> when a user views an entry by clicking. We are updating the database flag >>> to READ and use real time indexing to update solr entry. So when the user >>> close the full view of the entry and go back to entry listing page, the >>> data fetched from solr should be updated to READ. That's the use case we >>> are trying to fix. >>> >>> @eric : thanks for the lengthy reply. So let's say I increase the >>> autosoftcommit time out to may be 100 ms. In that case do I have to wait >>> much that time from client side before calling search ?. What's the >>> correct way of achieving this? >>> >>> Regards, >>> Lasitha >>> >>> On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote: >>> >>> 1 ms autocommit is far too frequent. And it's not >>>> helping you anyway. >>>> >>>> There is some lag between when a commit happens >>>> and when the docs are really available. The sequence is: >>>> 1> commit (soft or hard-with-opensearcher=true doesn't matter). >>>> 2> a new searcher is opened and autowarming starts >>>> 3> until the new searcher is opened, queries continue to be served by >>>> the old searcher >>>> 4> the new searcher is fully opened >>>> 5> _new_ requests are served by the new searcher. >>>> 6> the last request is finished by the old searcher and it's closed. >>>> >>>> So what's probably happening is that you send docs and then send a >>>> query and Solr is still in step <3>. You can look at your admin UI >>>> pluginst/stats page or your log to see how long it takes for a >>>> searcher to open and adjust your expectations accordingly. >>>> >>>> If you want to fetch only the document (not try to get it by a >>>> search), Real Time Get is designed to insure that you always get the >>>> most recent copy whether it's searchable or not. >>>> >>>> All that said, Solr wasn't designed for autocommits that are that >>>> frequent. That's why the documentation talks about _Near_ Real Time. >>>> You may need to adjust your expectations. >>>> >>>> Best, >>>> Erick >>>> >>>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com> >>>> wrote: >>>> >>>>> There's a very high probability that you're using the wrong tool for the >>>>> job if you need 1ms softCommit time. Especially when you always need it >>>>> >>>> (ex >>>> >>>>> there are apps where you need commit-after-insert very rarely). >>>>> >>>>> So explain what you're using it for ? >>>>> >>>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya < >>>>> >>>> watt...@gmail.com> >>>> >>>>> wrote: >>>>> >>>>> Hi Furkan, >>>>>> >>>>>> Thanks for the links. I had read the first one but not the second one. >>>>>> I >>>>>> did read it after you sent. So in my current solrconfig.xml settings >>>>>> >>>>> below >>>> >>>>> are the configurations, >>>>>> >>>>>> <autoSoftCommit> >>>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime> >>>>>> </autoSoftCommit> >>>>>> >>>>>> >>>>>> <autoCommit> >>>>>> <maxTime>15000</maxTime> >>>>>> <openSearcher>false</openSearcher> >>>>>> </autoCommit> >>>>>> >>>>>> The problem i'm facing is, just after adding the documents to solr >>>>>> using >>>>>> solrj, when I retrieve data from solr I am not getting the updated >>>>>> >>>>> results. >>>> >>>>> This happens time to time. Most of the time I get the correct data but >>>>>> >>>>> in >>>> >>>>> some occasions I get wrong results. so as you suggest, what the best >>>>>> practice to use here ? , should I wait 1 mili second before calling for >>>>>> updated results ? >>>>>> >>>>>> Regards, >>>>>> Lasitha >>>>>> >>>>>> Lasitha Wattaladeniya >>>>>> Software Engineer >>>>>> >>>>>> Mobile : +6593896893 >>>>>> Blog : techreadme.blogspot.com >>>>>> >>>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com >>>>>>> >>>>>> wrote: >>>>>> >>>>>> Hi Lasitha, >>>>>>> >>>>>>> First of all, did you check these: >>>>>>> >>>>>>> https://cwiki.apache.org/confluence/display/solr/Near+ >>>>>>> >>>>>> Real+Time+Searching >>>>>> >>>>>>> https://lucidworks.com/blog/2013/08/23/understanding- >>>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/ >>>>>>> >>>>>>> after that, if you cannot adjust your configuration you can give more >>>>>>> information and we can find a solution. >>>>>>> >>>>>>> Kind Regards, >>>>>>> Furkan KAMACI >>>>>>> >>>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya < >>>>>>> >>>>>> watt...@gmail.com> >>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi furkan, >>>>>>>> >>>>>>>> Thanks for your reply, it is generally a query heavy system. We are >>>>>>>> >>>>>>> using >>>>>> >>>>>>> realtime indexing for editing the available data >>>>>>>> >>>>>>>> Regards, >>>>>>>> Lasitha >>>>>>>> >>>>>>>> Lasitha Wattaladeniya >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Mobile : +6593896893 <+65%209389%206893> >>>>>>>> Blog : techreadme.blogspot.com >>>>>>>> >>>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI < >>>>>>>> >>>>>>> furkankam...@gmail.com> >>>> >>>>> wrote: >>>>>>>> >>>>>>>> Hi Lasitha, >>>>>>>>> >>>>>>>>> What is your indexing / querying requirements. Do you have an index >>>>>>>>> heavy/light - query heavy/light system? >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> Furkan KAMACI >>>>>>>>> >>>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya < >>>>>>>>> watt...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hello devs, >>>>>>>>>> >>>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a >>>>>>>>>> >>>>>>>>> commit >>>> >>>>> (soft >>>>>>>>> >>>>>>>>>> commit) through solrj and just after the commit, retrieve the data >>>>>>>>>> >>>>>>>>> from >>>>>> >>>>>>> solr (requirement is to get updated data list). >>>>>>>>>> >>>>>>>>>> I'm using soft commit instead of the hard commit, is previously I >>>>>>>>>> >>>>>>>>> got >>>> >>>>> an >>>>>>>>> >>>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later" >>>>>>>>>> >>>>>>>>> because of >>>>>>>>> >>>>>>>>>> too many commit requests. Now I have removed the explicit commit >>>>>>>>>> >>>>>>>>> and >>>> >>>>> has >>>>>>>>> >>>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili >>>>>>>>>> >>>>>>>>> second)* >>>> >>>>> and >>>>>>>>> >>>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any >>>>>>>>>> >>>>>>>>> errors >>>>>>>>> >>>>>>>>>> when i'm committing frequently. >>>>>>>>>> >>>>>>>>>> The problem i'm facing now is, I'm not getting the updated data >>>>>>>>>> >>>>>>>>> when >>>> >>>>> I >>>>>> >>>>>>> fetch from solr just after the soft commit. So in this case what >>>>>>>>>> >>>>>>>>> are >>>> >>>>> the >>>>>>>>> >>>>>>>>>> best practices to use ? to wait 1 mili second before retrieving >>>>>>>>>> >>>>>>>>> data >>>> >>>>> after >>>>>>>>> >>>>>>>>>> soft commit ? I don't feel like waiting from client side is a good >>>>>>>>>> >>>>>>>>> option. >>>>>>>>> >>>>>>>>>> Please give me some help from your expert knowledge >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Lasitha Wattaladeniya >>>>>>>>>> Software Engineer >>>>>>>>>> >>>>>>>>>> Mobile : +6593896893 >>>>>>>>>> Blog : techreadme.blogspot.com >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>
