Hi everyone, My company currently uses SOLR to completely hydrate client objects by storing all fields (stored=true). Therefore we have 2 types of fields:
1. indexed=true | stored=true : For fields that will be used for searching, sorting, etc. 2. indexed=false | stored=true: For fields that only need hydrating for clients We are re-architecting this so that we will eventually only get the id from SOLR (fl=id) and hydrate from another data source. This means we can obviously delete all the indexed=false | stored=true fields to reduce our index size. However, when it comes to the indexed=true | stored=true fields, we are not sure whether to also set them to be stored=false and perform in-place updates or leave it as is and perform atomic updates. We've done a fair bit of research on the archives of this mailing list, but are still a bit confused: 1. Will having the fields be converted from indexed=true | stored=true -> indexed=true | stored=false cause our index size to reduce? Will it also mean that indexing will be less compute expensive due to the compression of stored field logic? 2. Are atomic updates preferred to in-place updates? Obviously if we move to index only fields, then we have to do in-place updates all the time. This isn't an issue for us, but we are a bit concerned about how SOLR's indexing speed will suffer & deleted docs increase. Currently we perform both. Some points about our SOLR usecase: - 40-60M docs with 8 shards (PULL/TLOG structure) Solr 7.4 - No need for extremely fast indexing - Need for high query throughput (thus why we only want to retrieve the id field and hydrate with a faster db store) Thanks everyone, always appreciate the good information being shared here daily :) Regards, Ash -- *P.S. We've launched a new blog to share the latest ideas and case studies from our team. Check it out here: product.canva.com <http://product.canva.com/>. *** ** <https://canva.com>Empowering the world to design Also, we're hiring. Apply here! <https://about.canva.com/careers/> <https://twitter.com/canva> <https://facebook.com/canva> <https://au.linkedin.com/company/canva> <https://instagram.com/canva>
