Exactly. You could have the best engineer on your continent but the end result is the same, more metal. I could build a very fast search server for less than a week of my salary so what’s the point of wasting two weeks trying to solve a problem when the solution is literally just right there, a big ssd and a lot of memory, then I could work on things that are actually important rather than try to squeeze blood from a turnip.
> On Jul 5, 2022, at 6:11 AM, Charlie Hull <ch...@opensourceconnections.com> > wrote: > > I think you're missing my point. > > Good engineers, even mediocre ones, are expensive, and the great ones are > rare. It's a tedious task chasing tiny performance gains when you know you're > limited by the hardware and a bored engineer might just go and look for > another job. So if you fail to realise that a capital expense for hardware or > hosting is necessary, you run the risk of losing the people that make your > search engine work (even if they stay they could also be doing something more > useful to your business). > > Charlie > >> On 05/07/2022 10:49, Deepak Goel wrote: >> If you are tearing your hair out on 'Number of Hours' required for tuning >> your software, it's time you switch to a better quality performance >> engineer. >> >> Deepak >> "The greatness of a nation can be judged by the way its animals are treated >> - Mahatma Gandhi" >> >> +91 73500 12833 >> deic...@gmail.com >> >> Facebook:https://www.facebook.com/deicool >> LinkedIn:www.linkedin.com/in/deicool >> >> "Plant a Tree, Go Green" >> >> Make In India :http://www.makeinindia.com/home >> >> >> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull<ch...@opensourceconnections.com> >> wrote: >> >>> Equally it's not a good management practice to burn engineering hours >>> trying to optimise performance to avoid spending (often much less) money >>> on sufficient hardware to do the job. I've seen this happen many times, >>> sadly. >>> >>> Charlie >>> >>> On 05/07/2022 10:33, Deepak Goel wrote: >>>> Not a good software engineering practice to beef up the hardware blindly. >>>> Of Course when you have tuned the software to a point where you can't >>> tune >>>> anymore, you can then turn your eyes to hardware. >>>> >>>> Deepak >>>> "The greatness of a nation can be judged by the way its animals are >>> treated >>>> - Mahatma Gandhi" >>>> >>>> +91 73500 12833 >>>> deic...@gmail.com >>>> >>>> Facebook:https://www.facebook.com/deicool >>>> LinkedIn:www.linkedin.com/in/deicool >>>> >>>> "Plant a Tree, Go Green" >>>> >>>> Make In India :http://www.makeinindia.com/home >>>> >>>> >>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<hastings.recurs...@gmail.com> >>> wrote: >>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which helps a lot. >>> It >>>>> comes to a point where money on hardware will outweigh money on >>> engineering >>>>> man power hours, and still come to the same conclusion. As much ram as >>> your >>>>> rack can take and as big and fast of a raid ssd drive it can take. >>> Remember >>>>> since solr is always meant to be destroyed and recreated you don’t have >>> to >>>>> worry much about hardware failure if you just buy two of everything and >>>>> have a backup server ready and waiting to take over while the original >>>>> fails and is reconstructed. >>>>> >>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<apa...@elyograg.org> wrote: >>>>>> >>>>>> On 7/4/22 03:01, Mike wrote: >>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. Solr eats >>> up >>>>> all >>>>>>> the memory and because of that PHP works very, very slowly. What can I >>>>> do? >>>>>> Solr is a Java program. A Java program will never directly use more >>>>> memory than you specify for the max heap size. We cannot make any >>> general >>>>> recommendations about what heap size you need, because there is a good >>>>> chance that any recommendation we make would be completely wrong for >>> your >>>>> install. I did see that someone recommended not going above 31G ... and >>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers >>> instead of >>>>> 32-bit. So a heap size of 32 GB actually has LESS memory available >>> than a >>>>> heap size of 31 GB. >>>>>> The OS will use additional memory beyond the heap for caching the index >>>>> data, but that is completely outside of Solr's control. Note that 64GB >>>>> total memory for a 500GB index is almost certainly not enough memory, >>>>> ESPECIALLY if the same server is used for things other than Solr. I >>> wrote >>>>> the following wiki page: >>> https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems >>>>>> Others have recommended that you run Solr on dedicated hardware that is >>>>> not used for any other purpose. I concur with that recommendation. >>>>>> Thanks, >>>>>> Shawn >>>>>> >>> -- >>> Charlie Hull - Managing Consultant at OpenSource Connections Limited >>> Founding member of The Search Network<http://www.thesearchnetwork.com> >>> and co-author of Searching the Enterprise >>> < >>> https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf >>> tel/fax: +44 (0)8700 118334 >>> mobile: +44 (0)7767 825828 >>> >>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin >>> Amtsgericht Charlottenburg | HRB 230712 B >>> Geschäftsführer: John M. Woodell | David E. Pugh >>> Finanzamt: Berlin Finanzamt für Körperschaften II >>> >>> -- >>> This email has been checked for viruses by AVG. >>> https://www.avg.com >>> > -- > Charlie Hull - Managing Consultant at OpenSource Connections Limited > Founding member of The Search Network <http://www.thesearchnetwork.com> and > co-author of Searching the Enterprise > <https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf> > tel/fax: +44 (0)8700 118334 > mobile: +44 (0)7767 825828 > > OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin > Amtsgericht Charlottenburg | HRB 230712 B > Geschäftsführer: John M. Woodell | David E. Pugh > Finanzamt: Berlin Finanzamt für Körperschaften II > > -- > This email has been checked for viruses by AVG. > https://www.avg.com
