Re: Best SSD block size for large SOLR indexes

2014-03-21 Thread Salman Akram
For now I am going with 64kb and results seem good. Thanks for the useful feedback. On Wed, Mar 19, 2014 at 9:30 PM, Shawn Heisey wrote: > On 3/19/2014 12:09 AM, Salman Akram wrote: > >> Thanks for the info. The articles were really useful but still seems I >> have >> to do my own testing to fi

Re: Best SSD block size for large SOLR indexes

2014-03-19 Thread Shawn Heisey
On 3/19/2014 12:09 AM, Salman Akram wrote: Thanks for the info. The articles were really useful but still seems I have to do my own testing to find the right page size? I thought for large indexes there would already be some tests done in SOLR community. Side note: We are heavily using Microsoft

Re: Best SSD block size for large SOLR indexes

2014-03-18 Thread Salman Akram
We do have couple of commodity SSDs already and they perform good. However, our user queries are very complex and quite a few of them go above a minute so we really had to do something about it. Using this beast vs putting the whole index to RAM, the beast still seemed a better option. Also we are

Re: Best SSD block size for large SOLR indexes

2014-03-18 Thread Salman Akram
Thanks for the info. The articles were really useful but still seems I have to do my own testing to find the right page size? I thought for large indexes there would already be some tests done in SOLR community. Side note: We are heavily using Microsoft technology (.NET etc) for development so by

RE: Best SSD block size for large SOLR indexes

2014-03-18 Thread Toke Eskildsen
Salman Akram [salman.ak...@northbaysolutions.net] wrote: [Hundreds of GB index] > http://www.storagereview.com/micron_p420m_enterprise_pcie_ssd_review May I ask why you have chosen a drive with such a high speed and matching cost? We have some years of experience with using SSDs for search at w

Re: Best SSD block size for large SOLR indexes

2014-03-18 Thread Shawn Heisey
On 3/18/2014 7:39 AM, Salman Akram wrote: > This SSD default size seems to be 4K not 16K (as can be seen below). > > Bytes Per Sector : 512 > Bytes Per Physical Sector : 4096 > Bytes Per Cluster : 4096 > Bytes Per FileRecord Segment: 1024 The *sector* size o

Re: Best SSD block size for large SOLR indexes

2014-03-18 Thread Salman Akram
This SSD default size seems to be 4K not 16K (as can be seen below). Bytes Per Sector : 512 Bytes Per Physical Sector : 4096 Bytes Per Cluster : 4096 Bytes Per FileRecord Segment: 1024 I will go through the articles you sent. Thanks On Tue, Mar 18, 2014 at

Re: Best SSD block size for large SOLR indexes

2014-03-18 Thread Shawn Heisey
On 3/18/2014 7:12 AM, Salman Akram wrote: > Is there a rule of thumb for ideal block size for SSDs for large indexes > (in hundreds of GBs)? Read performance is of top importance for us and we > can sacrifice the space a little... > > This is the one we just got and wanted to see if there are any