Joesph: Not so much after using some of the settings available on Shawn's Solr Wiki page: https://wiki.apache.org/solr/ShawnHeisey
This is what we're running with right now: -Xmx6g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 Michael Della Bitta Applications Developer o: +1 646 532 3062 appinions inc. "The Science of Influence Marketing" 18 East 41st Street New York, NY 10017 t: @appinions <https://twitter.com/Appinions> | g+: plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts> w: appinions.com <http://www.appinions.com/> On Fri, Jan 31, 2014 at 10:58 AM, Joseph Hagerty <joa...@gmail.com> wrote: > Thanks, Shawn. This information is actually not all that shocking to me. > It's always been in the back of my mind that I was "getting away with > something" in serving from the m1.large. Remarkably, however, it has served > me well for nearly two years; also, although the index has not always been > 30GB, it has always been much larger than the RAM on the box. As you > suggested, I can only suppose that usage patterns and the index schema have > in some way facilitated minimal heap usage, up to this point. > > For now, we're going to increase the heap size on the instance and see > where that gets us; if it still doesn't suffice for now, then we'll upgrade > to a more powerful instance. > > Michael, thanks for weighing in. Those i2 instances look delicious indeed. > Just curious -- have you struggled with garbage collection pausing at all? > > > > On Thu, Jan 30, 2014 at 7:43 PM, Shawn Heisey <s...@elyograg.org> wrote: > > > On 1/30/2014 3:20 PM, Joseph Hagerty wrote: > > > >> I'm using Solr 3.5 over Tomcat 6. My index has reached 30G. > >> > > > > <snip> > > > > > > - The box is an m1.large on AWS EC2. 2 virtual CPUs, 4 ECU, 7.5 GiB RAM > >> > > > > One detail that you did not provide was how much of your 7.5GB RAM you > are > > allocating to the Java heap for Solr, but I actually don't think I need > > that information, because for your index size, you simply don't have > > enough. If you're sticking with Amazon, you'll want one of the instances > > with at least 30GB of RAM, and you might want to consider more memory > than > > that. > > > > An ideal RAM size for Solr is equal to the size of on-disk data plus the > > heap space used by Solr and other programs. This means that if your java > > heap for Solr is 4GB and there are no other significant programs running > on > > the same server, you'd want a minimum of 34GB of RAM for an ideal setup > > with your index. 4GB of that would be for Solr itself, the remainder > would > > be for the operating system to fully cache your index in the OS disk > cache. > > > > Depending on your query patterns and how your schema is arranged, you > > *might* be able to get away as little as half of your index size just for > > the OS disk cache, but it's better to make it big enough for the whole > > index, plus room for growth. > > > > http://wiki.apache.org/solr/SolrPerformanceProblems > > > > Many people are *shocked* when they are told this information, but if you > > think about the relative speeds of getting a chunk of data from a hard > disk > > vs. getting the same information from memory, it's not all that shocking. > > > > Thanks, > > Shawn > > > > > > > -- > - Joe >
