Just Add java memory parameters in solr config which should not be more than 75% of total RAM. and use G1GC.
On Tue, 5 Jul 2022 at 4:59 PM, Deepak Goel <deic...@gmail.com> wrote: > I would also suggest you look at which GC mechanism you use. Increasing RAM > and Heap-Size might result in the application freezed for a long time > (during GC). > > Deepak > "The greatness of a nation can be judged by the way its animals are treated > - Mahatma Gandhi" > > +91 73500 12833 > deic...@gmail.com > > Facebook: https://www.facebook.com/deicool > LinkedIn: www.linkedin.com/in/deicool > > "Plant a Tree, Go Green" > > Make In India : http://www.makeinindia.com/home > > > On Tue, Jul 5, 2022 at 4:43 PM Dave <hastings.recurs...@gmail.com> wrote: > > > Exactly. You could have the best engineer on your continent but the end > > result is the same, more metal. I could build a very fast search server > > for less than a week of my salary so what’s the point of wasting two > weeks > > trying to solve a problem when the solution is literally just right > there, > > a big ssd and a lot of memory, then I could work on things that are > > actually important rather than try to squeeze blood from a turnip. > > > > > On Jul 5, 2022, at 6:11 AM, Charlie Hull < > > ch...@opensourceconnections.com> wrote: > > > > > > I think you're missing my point. > > > > > > Good engineers, even mediocre ones, are expensive, and the great ones > > are rare. It's a tedious task chasing tiny performance gains when you > know > > you're limited by the hardware and a bored engineer might just go and > look > > for another job. So if you fail to realise that a capital expense for > > hardware or hosting is necessary, you run the risk of losing the people > > that make your search engine work (even if they stay they could also be > > doing something more useful to your business). > > > > > > Charlie > > > > > >> On 05/07/2022 10:49, Deepak Goel wrote: > > >> If you are tearing your hair out on 'Number of Hours' required for > > tuning > > >> your software, it's time you switch to a better quality performance > > >> engineer. > > >> > > >> Deepak > > >> "The greatness of a nation can be judged by the way its animals are > > treated > > >> - Mahatma Gandhi" > > >> > > >> +91 73500 12833 > > >> deic...@gmail.com > > >> > > >> Facebook:https://www.facebook.com/deicool > > >> LinkedIn:www.linkedin.com/in/deicool > > >> > > >> "Plant a Tree, Go Green" > > >> > > >> Make In India :http://www.makeinindia.com/home > > >> > > >> > > >> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull< > > ch...@opensourceconnections.com> > > >> wrote: > > >> > > >>> Equally it's not a good management practice to burn engineering hours > > >>> trying to optimise performance to avoid spending (often much less) > > money > > >>> on sufficient hardware to do the job. I've seen this happen many > times, > > >>> sadly. > > >>> > > >>> Charlie > > >>> > > >>> On 05/07/2022 10:33, Deepak Goel wrote: > > >>>> Not a good software engineering practice to beef up the hardware > > blindly. > > >>>> Of Course when you have tuned the software to a point where you > can't > > >>> tune > > >>>> anymore, you can then turn your eyes to hardware. > > >>>> > > >>>> Deepak > > >>>> "The greatness of a nation can be judged by the way its animals are > > >>> treated > > >>>> - Mahatma Gandhi" > > >>>> > > >>>> +91 73500 12833 > > >>>> deic...@gmail.com > > >>>> > > >>>> Facebook:https://www.facebook.com/deicool > > >>>> LinkedIn:www.linkedin.com/in/deicool > > >>>> > > >>>> "Plant a Tree, Go Green" > > >>>> > > >>>> Make In India :http://www.makeinindia.com/home > > >>>> > > >>>> > > >>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<hastings.recurs...@gmail.com> > > >>> wrote: > > >>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which helps a > > lot. > > >>> It > > >>>>> comes to a point where money on hardware will outweigh money on > > >>> engineering > > >>>>> man power hours, and still come to the same conclusion. As much ram > > as > > >>> your > > >>>>> rack can take and as big and fast of a raid ssd drive it can take. > > >>> Remember > > >>>>> since solr is always meant to be destroyed and recreated you don’t > > have > > >>> to > > >>>>> worry much about hardware failure if you just buy two of everything > > and > > >>>>> have a backup server ready and waiting to take over while the > > original > > >>>>> fails and is reconstructed. > > >>>>> > > >>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<apa...@elyograg.org> > > wrote: > > >>>>>> > > >>>>>> On 7/4/22 03:01, Mike wrote: > > >>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. Solr > > eats > > >>> up > > >>>>> all > > >>>>>>> the memory and because of that PHP works very, very slowly. What > > can I > > >>>>> do? > > >>>>>> Solr is a Java program. A Java program will never directly use > more > > >>>>> memory than you specify for the max heap size. We cannot make any > > >>> general > > >>>>> recommendations about what heap size you need, because there is a > > good > > >>>>> chance that any recommendation we make would be completely wrong > for > > >>> your > > >>>>> install. I did see that someone recommended not going above 31G > ... > > and > > >>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers > > >>> instead of > > >>>>> 32-bit. So a heap size of 32 GB actually has LESS memory available > > >>> than a > > >>>>> heap size of 31 GB. > > >>>>>> The OS will use additional memory beyond the heap for caching the > > index > > >>>>> data, but that is completely outside of Solr's control. Note that > > 64GB > > >>>>> total memory for a 500GB index is almost certainly not enough > memory, > > >>>>> ESPECIALLY if the same server is used for things other than Solr. > I > > >>> wrote > > >>>>> the following wiki page: > > >>> > > https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems > > >>>>>> Others have recommended that you run Solr on dedicated hardware > > that is > > >>>>> not used for any other purpose. I concur with that recommendation. > > >>>>>> Thanks, > > >>>>>> Shawn > > >>>>>> > > >>> -- > > >>> Charlie Hull - Managing Consultant at OpenSource Connections Limited > > >>> Founding member of The Search Network< > http://www.thesearchnetwork.com> > > >>> and co-author of Searching the Enterprise > > >>> < > > >>> > > > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > > >>> tel/fax: +44 (0)8700 118334 > > >>> mobile: +44 (0)7767 825828 > > >>> > > >>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > > >>> Amtsgericht Charlottenburg | HRB 230712 B > > >>> Geschäftsführer: John M. Woodell | David E. Pugh > > >>> Finanzamt: Berlin Finanzamt für Körperschaften II > > >>> > > >>> -- > > >>> This email has been checked for viruses by AVG. > > >>> https://www.avg.com > > >>> > > > -- > > > Charlie Hull - Managing Consultant at OpenSource Connections Limited > > > Founding member of The Search Network <http://www.thesearchnetwork.com > > > > and co-author of Searching the Enterprise < > > > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > > > > > > tel/fax: +44 (0)8700 118334 > > > mobile: +44 (0)7767 825828 > > > > > > OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > > > Amtsgericht Charlottenburg | HRB 230712 B > > > Geschäftsführer: John M. Woodell | David E. Pugh > > > Finanzamt: Berlin Finanzamt für Körperschaften II > > > > > > -- > > > This email has been checked for viruses by AVG. > > > https://www.avg.com > > >
