[
https://issues.apache.org/jira/browse/SOLR-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17335843#comment-17335843
]
Andy Throgmorton commented on SOLR-15359:
-----------------------------------------
Moving discussion from [~gerlowskija]'s GCS PR
(https://github.com/apache/solr/pull/39/files#r623428282) over to this Jira.
I mentioned there that it appears something changed from Solr 8.x to 9.x --
now, Solr expects it to be {{mkdir -p}} behavior when taking backups. As I'm
cleaning up our S3 code (as a part of SOLR-15089) and moving it to 9.x repo,
this behavior change broke our code (b/c ours throws exceptions if the parent
dir does not exist). This is the stacktrace of the Solr code that's calling
{{createDirectory}} and assuming {{mkdir -p}} behavior:
{noformat}
2021-04-29 21:49:16.472 INFO
(OverseerThreadFactory-15-thread-2-processing-n:localhost:8983_solr)
[c:collection1619732944 ] o.a.s.c.a.c.BackupCmd Starting to backup ZK data
for backupName=backup1619732944
2021-04-29 21:49:16.481 ERROR
(OverseerThreadFactory-15-thread-2-processing-n:localhost:8983_solr)
[c:collection1619732944 ] o.a.s.c.a.c.OverseerCollectionMessageHandler
Collection: collection1619732944 operation: backup failed:java.io.IOException:
org.apache.solr.blob.client.BlobException: Parent directory doesn't exist,
path=foo/backup1619732944/collection1619732944/zk_backup_0/
at
org.apache.solr.blob.backup.BlobBackupRepository.createDirectory(BlobBackupRepository.java:120)
at
org.apache.solr.core.backup.BackupManager.downloadConfigDir(BackupManager.java:249)
at
org.apache.solr.cloud.api.collections.BackupCmd.call(BackupCmd.java:124)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:117)
at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:531)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:226)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
> Clarify BackupRepository.createDirectory() semantics
> ----------------------------------------------------
>
> Key: SOLR-15359
> URL: https://issues.apache.org/jira/browse/SOLR-15359
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore
> Affects Versions: main (9.0), 8.8.2
> Reporter: Jason Gerlowski
> Priority: Minor
>
> The BackupRepository abstraction (used by backup code to plug in various
> storage backends) has a createDirectory method with ambiguous semantics. In
> short, it's unclear whether the method is intended to act as {{mkdir}} or
> {{mkdir -p}}. Should it create any missing parent directories, or should it
> error out if a parent directory doesn't exist?
> Solr's current backup code avoids running afoul of this mostly by
> coincidence. Anywhere that nested directories are used, backup code makes a
> createDirectory call for each one. But this has allowed the different
> implementations to diverge. LocalFileSystemRepository throws an error if a
> parent directory doesn't exist, but HdfsBackupRepository quietly creates any
> missing parent directories.
> We should decide how we want this to behave, so that BackupRepository will
> serve as a better general purpose API for file/directory storage. Once
> decided, existing implementations should be brought into line.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
