Is it correct to assume that using field compression will cause performance issues if we decide to allow search over this field?
ie: <field name="id" type="sint" indexed="true" stored="true" required="true" /> <field name="title" type="text" indexed="true" stored="true" omitNorms="true"/> <field name="file_location" type="string" indexed="false" stored="true"/> <field name="body" type="text" indexed="true" stored="false" omitNorms="true"/> if I decide to add "compressed=true" to the BODY field... and a I allow search on body... would that be a problem? At the same time: if I add compressed=true , but I never do search on this field.... ? Stu Hood-3 wrote: > > I just finished watching this talk about a column-store RDBMS, which has a > long section on column compression. Specifically, it talks about the gains > from compressing similar data together, and how lazily decompressing data > only when it must be processed is great for memory/CPU cache usage. > > http://youtube.com/watch?v=yrLd-3lnZ58 > > While interesting, its not relevant to Lucene's stored field storage. On > the other hand, it did get me thinking about stored field compression and > lazy field loading. > > Can anyone give me some pointers about compressThreshold values that would > be worth experimenting with? Our stored fields are often between 20 and > 300 characters, and we're willing to spend more time indexing if it will > make searching less IO bound. > > Thanks, > > Stu Hood > Architecture Software Developer > Mailtrust, a Rackspace Company > > > -- View this message in context: http://www.nabble.com/Field-Compression-tp15258669p23865558.html Sent from the Solr - User mailing list archive at Nabble.com.
