Hi, If your tlogs grow, It looks like hard commit are never executed.
Are you seeing hard commits in logs (grep 'softCommit=false') ? What is it happening if you force a hard commit ? Are tlog truncated ? http://localhost:8983/solr/[collection_name]/update?commit=true Did you read this article ? https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Regards Dominique Le dim. 21 mars 2021 à 05:22, Ritvik Sharma <[email protected]> a écrit : > Hi Dominique > > Heavy query means high query rate on solr. > > Honestly for 2-3 days we have stopped the queries on solr only we are > doing indexing !!! > As you see we have enabled Hard commit also to decrease tlog log size or > more and the same is written on solr docs. Still the same behaviour is > occurring on replicas. In some replicas, tlog size is of 300 GB. > > We may increase softcommit timeout but as per my understanding it only > helps in searching availability prior to commit to indexes. > > On Sun, 21 Mar 2021 at 4:29 AM, Dominique Bejean < > [email protected]> wrote: > >> Hi, >> >> By heavy query, do you mean you have a high query rate and/or you need >> index update be available fast after few seconds (NRT search) ? >> >> Do you see the hard commits in logs ? >> Can you try to increase autoSoftCommit to 30 seconds or more ? >> >> Regards >> >> Dominique >> >> Le sam. 20 mars 2021 à 18:53, Ritvik Sharma <[email protected]> a >> écrit : >> >> > Hi Guys >> > >> > We have faced an issue where tlog size is increasing unnecessarily. We >> are >> > using a "heavy indexing, heavy query" approach. We enabled hard commit >> > also, >> > >> > solr cloud: 6.6 >> > zk: 3.4.10 >> > shards: 2, replication factor= 2 >> > >> > >> > solrconfig, >> > >> > <autoCommit> >> > <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> >> > <maxDocs>10000</maxDocs> >> > <openSearcher>false</openSearcher> >> > >> > >> > </autoCommit> >> > >> > <!-- softAutoCommit is like autoCommit except it causes a >> > 'soft' commit which only ensures that changes are visible >> > but does not ensure that data is synced to disk. This is >> > faster and more near-realtime friendly than a hard commit. >> > --> >> > >> > <autoSoftCommit> >> > <maxTime>${solr.autoSoftCommit.maxTime:15000}</maxTime> --> >> > </autoSoftCommit> >> > >> > In every replica, tlog is increasing for more than 200GB which exhaust >> disk >> > space. Please suggest someting >> > >> > On Fri, 19 Mar 2021 at 16:36, Ritvik Sharma <[email protected]> >> wrote: >> > >> > > Hi Guys >> > > >> > > We have faced an issue where tlog size is increasing unnecessarily. We >> > are >> > > using a "heavy indexing, heavy query" approach. We enabled hard >> commit >> > > also, >> > > >> > > solr cloud: 6.6 >> > > zk: 3.4.10 >> > > shards: 2, replication factor= 2 >> > > >> > > >> > > solrconfig, >> > > >> > > <autoCommit> >> > > <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> >> > > <maxDocs>10000</maxDocs> >> > > <openSearcher>false</openSearcher> >> > > >> > > >> > > </autoCommit> >> > > >> > > <!-- softAutoCommit is like autoCommit except it causes a >> > > 'soft' commit which only ensures that changes are visible >> > > but does not ensure that data is synced to disk. This is >> > > faster and more near-realtime friendly than a hard commit. >> > > --> >> > > >> > > <autoSoftCommit> >> > > <maxTime>${solr.autoSoftCommit.maxTime:15000}</maxTime> --> >> > > </autoSoftCommit> >> > > >> > > >> > >> >
