Will White created SOLR-16813: --------------------------------- Summary: PackageLoader not copying manifest.json file to filestore Key: SOLR-16813 URL: https://issues.apache.org/jira/browse/SOLR-16813 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: Package Manager Affects Versions: 9.2.1, 9.2 Reporter: Will White
When a new Solr node enters the cluster and the package management system recreates the package from the source node, the filestore location is set up and the JAR files are copied across correctly, but the {{manifest.json}} file isn't (and the {{.manifest.json.json}} file isn't generated). This means that when trying to interact with the package (such as {{bin/solr package list-installed}}) the command errors out because of the missing expected file. This can be reproduced by: * Setup Zookeeper cluster * Create two solr nodes ("solr1" and "solr2") in cluster * Install a package on solr1 ** {{bin/solr package add-repo data-import-handler "https://raw.githubusercontent.com/searchscale/dataimporthandler/master/repo"}} ** {{bin/solr package install data-import-handler}} ** {{find /var/solr/data/filestore}} ({{manifest.json}} and {{.manifest.json.json}} should exist) ** {{bin/solr package list-installed}} * At this point, the package exists as expected on solr2 * Stop solr1 and solr2 * Restart the cluster, now with an additional solr3 node * solr3 will have the JAR files and their {{.json}} files, but the {{manifest.json}} file won't be present ** Running {{bin/solr package list-installed}} will throw an exception ---- The difference appears to be that the {{org.apache.solr.packagemanager.RepositoryManager::installPackage}} method contains the following lines: {code:java} // org/apache/solr/packagemanager/RepositoryManager.java#182 if (release.manifest == null) { String manifestJson = PackageUtils.getFileFromJarsAsString(downloaded, "manifest.json"); if (manifestJson == null) { throw new SolrException(ErrorCode.NOT_FOUND, "No manifest found for package: " + packageName + ", version: " + version); } release.manifest = getMapper().readValue(manifestJson, SolrPackage.Manifest.class); } String manifestJson = getMapper().writeValueAsString(release.manifest); // We go on to write this file later {code} But in the equivalent space in {{org.apache.solr.pkg.SolrPackageLoader.Version()}}, the equivalent section to fetch the JAR files if missing doesn't reference the version.manifest anywhere: {code:java} // org/apache/solr/pkg/SolrPackageLoader.java#276 coreContainer.getPackageStoreAPI().validateFiles(version.files, true, s -> errs.add(s)); if (!errs.isEmpty()) { ... } for (String file : version.files) { paths.add(coreContainer.getPackageStoreAPI().getPackageStore().getRealpath(file)); } {code} Something like [this commit|https://github.com/apache/solr/commit/9fc5253eef573dc5e53edf52d98d4a2e55e00e1e] solves the problem by manually copying the file across, but I'm not sure if this is the best approach to take. I'm happy to put a full PR together, although I'd appreciate it if someone could confirm whether the {{manifest.json}} file should be transferred from the source node like this (and can be treated as mandatory), or if I've misunderstood how this should be working. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org