On May 5, 2012, at 8:39 AM, Jan Høydahl wrote: > ZK is not really designed for keeping large data files, > fromhttp://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#Data+Access: >> ZooKeeper was not designed to be a general database or large object >> store.....If large data storage is needed, the usually pattern of dealing >> with such data is to store it on a bulk storage system, such as NFS or HDFS, >> and store pointers to the storage locations in ZooKeeper. >
I don't really think it's that big a deal where Solr is concerned. We don't use ZooKeeper intensively. Only for state changes, and initially loading config for SolrCore starts or reloads. ZooKeeper perf should not be a big deal for Solr. I think the main issue is going to be that zk wants everything in ram - but again, a few, couple MB conf files should be no big deal AFAICT. Other large scale applications that constantly and intensively use zookeeper are a different story - when Solr is in its running state, it doesnt do anything with zookeeper other than maintain it's heartbeat. - Mark Miller lucidimagination.com
