Do you mind telling us which java version are you using? On Tue, 5 Jul 2022, 17:55 Dave, <hastings.recurs...@gmail.com> wrote:
> Also a good rule of thumb I found is set your xmx and xms, maximum and > minimum memory for the heap to be exactly the same, you don’t want Java to > try to figure it out, > > > On Jul 5, 2022, at 7:52 AM, Ritvik Sharma <ritvik.s...@gmail.com> wrote: > > > > Just Add java memory parameters in solr config which should not be more > > than 75% of total RAM. and use G1GC. > > > > > >> On Tue, 5 Jul 2022 at 4:59 PM, Deepak Goel <deic...@gmail.com> wrote: > >> > >> I would also suggest you look at which GC mechanism you use. Increasing > RAM > >> and Heap-Size might result in the application freezed for a long time > >> (during GC). > >> > >> Deepak > >> "The greatness of a nation can be judged by the way its animals are > treated > >> - Mahatma Gandhi" > >> > >> +91 73500 12833 > >> deic...@gmail.com > >> > >> Facebook: https://www.facebook.com/deicool > >> LinkedIn: www.linkedin.com/in/deicool > >> > >> "Plant a Tree, Go Green" > >> > >> Make In India : http://www.makeinindia.com/home > >> > >> > >>> On Tue, Jul 5, 2022 at 4:43 PM Dave <hastings.recurs...@gmail.com> > wrote: > >>> > >>> Exactly. You could have the best engineer on your continent but the end > >>> result is the same, more metal. I could build a very fast search > server > >>> for less than a week of my salary so what’s the point of wasting two > >> weeks > >>> trying to solve a problem when the solution is literally just right > >> there, > >>> a big ssd and a lot of memory, then I could work on things that are > >>> actually important rather than try to squeeze blood from a turnip. > >>> > >>>> On Jul 5, 2022, at 6:11 AM, Charlie Hull < > >>> ch...@opensourceconnections.com> wrote: > >>>> > >>>> I think you're missing my point. > >>>> > >>>> Good engineers, even mediocre ones, are expensive, and the great ones > >>> are rare. It's a tedious task chasing tiny performance gains when you > >> know > >>> you're limited by the hardware and a bored engineer might just go and > >> look > >>> for another job. So if you fail to realise that a capital expense for > >>> hardware or hosting is necessary, you run the risk of losing the people > >>> that make your search engine work (even if they stay they could also be > >>> doing something more useful to your business). > >>>> > >>>> Charlie > >>>> > >>>>> On 05/07/2022 10:49, Deepak Goel wrote: > >>>>> If you are tearing your hair out on 'Number of Hours' required for > >>> tuning > >>>>> your software, it's time you switch to a better quality performance > >>>>> engineer. > >>>>> > >>>>> Deepak > >>>>> "The greatness of a nation can be judged by the way its animals are > >>> treated > >>>>> - Mahatma Gandhi" > >>>>> > >>>>> +91 73500 12833 > >>>>> deic...@gmail.com > >>>>> > >>>>> Facebook:https://www.facebook.com/deicool > >>>>> LinkedIn:www.linkedin.com/in/deicool > >>>>> > >>>>> "Plant a Tree, Go Green" > >>>>> > >>>>> Make In India :http://www.makeinindia.com/home > >>>>> > >>>>> > >>>>> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull< > >>> ch...@opensourceconnections.com> > >>>>> wrote: > >>>>> > >>>>>> Equally it's not a good management practice to burn engineering > hours > >>>>>> trying to optimise performance to avoid spending (often much less) > >>> money > >>>>>> on sufficient hardware to do the job. I've seen this happen many > >> times, > >>>>>> sadly. > >>>>>> > >>>>>> Charlie > >>>>>> > >>>>>> On 05/07/2022 10:33, Deepak Goel wrote: > >>>>>>> Not a good software engineering practice to beef up the hardware > >>> blindly. > >>>>>>> Of Course when you have tuned the software to a point where you > >> can't > >>>>>> tune > >>>>>>> anymore, you can then turn your eyes to hardware. > >>>>>>> > >>>>>>> Deepak > >>>>>>> "The greatness of a nation can be judged by the way its animals are > >>>>>> treated > >>>>>>> - Mahatma Gandhi" > >>>>>>> > >>>>>>> +91 73500 12833 > >>>>>>> deic...@gmail.com > >>>>>>> > >>>>>>> Facebook:https://www.facebook.com/deicool > >>>>>>> LinkedIn:www.linkedin.com/in/deicool > >>>>>>> > >>>>>>> "Plant a Tree, Go Green" > >>>>>>> > >>>>>>> Make In India :http://www.makeinindia.com/home > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<hastings.recurs...@gmail.com> > >>>>>> wrote: > >>>>>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which helps a > >>> lot. > >>>>>> It > >>>>>>>> comes to a point where money on hardware will outweigh money on > >>>>>> engineering > >>>>>>>> man power hours, and still come to the same conclusion. As much > ram > >>> as > >>>>>> your > >>>>>>>> rack can take and as big and fast of a raid ssd drive it can take. > >>>>>> Remember > >>>>>>>> since solr is always meant to be destroyed and recreated you don’t > >>> have > >>>>>> to > >>>>>>>> worry much about hardware failure if you just buy two of > everything > >>> and > >>>>>>>> have a backup server ready and waiting to take over while the > >>> original > >>>>>>>> fails and is reconstructed. > >>>>>>>> > >>>>>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<apa...@elyograg.org> > >>> wrote: > >>>>>>>>> > >>>>>>>>> On 7/4/22 03:01, Mike wrote: > >>>>>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. Solr > >>> eats > >>>>>> up > >>>>>>>> all > >>>>>>>>>> the memory and because of that PHP works very, very slowly. What > >>> can I > >>>>>>>> do? > >>>>>>>>> Solr is a Java program. A Java program will never directly use > >> more > >>>>>>>> memory than you specify for the max heap size. We cannot make any > >>>>>> general > >>>>>>>> recommendations about what heap size you need, because there is a > >>> good > >>>>>>>> chance that any recommendation we make would be completely wrong > >> for > >>>>>> your > >>>>>>>> install. I did see that someone recommended not going above 31G > >> ... > >>> and > >>>>>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers > >>>>>> instead of > >>>>>>>> 32-bit. So a heap size of 32 GB actually has LESS memory > available > >>>>>> than a > >>>>>>>> heap size of 31 GB. > >>>>>>>>> The OS will use additional memory beyond the heap for caching the > >>> index > >>>>>>>> data, but that is completely outside of Solr's control. Note that > >>> 64GB > >>>>>>>> total memory for a 500GB index is almost certainly not enough > >> memory, > >>>>>>>> ESPECIALLY if the same server is used for things other than Solr. > >> I > >>>>>> wrote > >>>>>>>> the following wiki page: > >>>>>> > >>> > https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems > >>>>>>>>> Others have recommended that you run Solr on dedicated hardware > >>> that is > >>>>>>>> not used for any other purpose. I concur with that > recommendation. > >>>>>>>>> Thanks, > >>>>>>>>> Shawn > >>>>>>>>> > >>>>>> -- > >>>>>> Charlie Hull - Managing Consultant at OpenSource Connections Limited > >>>>>> Founding member of The Search Network< > >> http://www.thesearchnetwork.com> > >>>>>> and co-author of Searching the Enterprise > >>>>>> < > >>>>>> > >>> > >> > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > >>>>>> tel/fax: +44 (0)8700 118334 > >>>>>> mobile: +44 (0)7767 825828 > >>>>>> > >>>>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 > Berlin > >>>>>> Amtsgericht Charlottenburg | HRB 230712 B > >>>>>> Geschäftsführer: John M. Woodell | David E. Pugh > >>>>>> Finanzamt: Berlin Finanzamt für Körperschaften II > >>>>>> > >>>>>> -- > >>>>>> This email has been checked for viruses by AVG. > >>>>>> https://www.avg.com > >>>>>> > >>>> -- > >>>> Charlie Hull - Managing Consultant at OpenSource Connections Limited > >>>> Founding member of The Search Network < > http://www.thesearchnetwork.com > >>> > >>> and co-author of Searching the Enterprise < > >>> > >> > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > >>>> > >>>> tel/fax: +44 (0)8700 118334 > >>>> mobile: +44 (0)7767 825828 > >>>> > >>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > >>>> Amtsgericht Charlottenburg | HRB 230712 B > >>>> Geschäftsführer: John M. Woodell | David E. Pugh > >>>> Finanzamt: Berlin Finanzamt für Körperschaften II > >>>> > >>>> -- > >>>> This email has been checked for viruses by AVG. > >>>> https://www.avg.com > >>> > >> >
