Hi. Compression is a good suggestion. All large dictionaries are compressed well below 1MB with GZIP. Where should this be implemented? SolrZkClient or ZkController? Which good compressor is already in Solr's lib? And what's the difference between SolrZkClient setData and create? Should it autocompress files larger than N bytes? And how should we detect if data is compressed when reading from ZooKeeper?
On Thursday 03 May 2012 14:04:31 Mark Miller wrote: > On May 3, 2012, at 5:15 AM, Markus Jelsma wrote: > > Hi, > > > > We've increased Zookeepers znode size limit to accomodate for some larger > > dictionaries and other files. It isn't the best idea to increase the > > maximum znode size. Any plans for splitting up larger files and storing > > them with multi? Does anyone have another suggestion? > > > > Thanks, > > Markus > > Patches welcome :) You can compress, you can break up the files, or you can > raise the limit - that's about the options I know of. > > You might start by creating a JIRA issue. > > - Mark Miller > lucidimagination.com -- Markus Jelsma - CTO - Openindex
