For leader/follower, would SOLR_VAR_HOME be adequate? It would mean that, at a minimum, core.properties moves to this data volume as well. If there is a configset, it would either be there directly or better, could be referenced from SOLR_HOME/configsets via referencing a named configset. If such cores had their very own libs separate from other cores, this wouldn't mean the libs go to the data volume as well, though <lib> directives could address that.
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Aug 12, 2021 at 4:25 AM Jan Høydahl <[email protected]> wrote: > The SOLR_DATA_HOME was primarily designed for leader/follower mode, > letting the user mount a /mnt/my/data/volume into the machines to store > indices, but still keep all config locally in a different folder structure. > I'm not attached to SOLR_DATA_HOME, it has caused quite some confusion in > SolrCloud mode I have seen. So if it stands in the way for container use > cases, it should go away. > > Jan > > 12. aug. 2021 kl. 06:51 skrev David Smiley <[email protected]>: > > SOLR_DATA_HOME[1] ... yeah that. It allows you to have your cores defined > in their typical location yet have the data for all cores be relative to > this path -- a different place. You can also set this via "-t" to > bin/solr. I'm not sure why someone would want them in separate places. If > I recall, Jan was an advocate of it? Perhaps whatever its utility is might > be reduced after SOLR_VAR_HOME is implemented, to the point that it ought > to go away. One less quirky/thing to maintain / document / confuse users > with. That's my opinion. > > [1]: > https://github.com/apache/solr/blob/main/solr/solr-ref-guide/src/index-location-format.adoc#specifying-a-location-for-index-data-with-the-datadir-parameter > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Wed, Aug 11, 2021 at 10:04 AM Jan Høydahl <[email protected]> > wrote: > >> +1, I've had to work around this by copying newest config from the new >> container version into the volume on disk, and it is quite confusing. >> Immutable is good. >> >> Jan >> >> 10. aug. 2021 kl. 21:05 skrev David Smiley <[email protected]>: >> >> Folks, please let me know what you think of my proposal below. >> >> PROBLEM: >> Today in Docker (and probably the Linux installer), the solr home dir in >> /var/solr/... is initialized with solr.xml, zoo.cfg and log4j2.xml [1]. >> Consequently, if you deploy new Docker images of Solr with updated built-in >> configurations (e.g. with your company/project settings), the old >> configuration is *still* used. This is in conflict with the "immutable >> infrastructure"[2] principle in which the deployable unit (Docker image) >> contains as much configuration as possible (and immutably) to avoid issues >> like this and snowflake server configuration. Configuration in a mutable >> place like ZooKeeper or the file system is problematic for upgrades, and so >> reducing what we put in both is helpful. >> >> [1]: >> https://github.com/apache/solr/blob/main/solr/docker/scripts/init-var-solr >> [2]: https://www.bmc.com/blogs/immutable-infrastructure/ >> >> At work I worked around this by editing my Dockerfile to have >> SOLR_HOME=/opt/solr/server/solr and editing my solr.xml coreRootDirectory >> to point to /var/solr/data. Since I also use embedded ZK in tests, I >> needed to pass -DzkServerDataDir=/var/solr/zoo_data. Overall this was easy >> but I'd like to propose Solr running this way auto-magically in Docker and >> via the Unix installer script. >> >> PROPOSAL: >> Add an env var SOLR_VAR that both Docker and the Linux installer will set >> to /var/solr. When this is set, aspects of SolrXmlConfig, the loader of >> solr.xml, will resolve some defaults relative to SOLR_VAR. If the >> directory or file is present in SOLR_VAR then it will use that, if it's in >> SOLR_HOME, it will use that. Otherwise, try to create it in SOLR_VAR >> assuming SOLR_VAR exists. >> >> Furthermore, the coreRootDirectory shall have a default relative dir of >> "data", NOT empty/blank as is now. This will separate cores from other >> stuff like configSets, userFiles, fileStore. >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> >
