Gergely Harmadás created KAFKA-20820:
----------------------------------------
Summary: Broker fails to start when log.dirs is a symbolic link
Key: KAFKA-20820
URL: https://issues.apache.org/jira/browse/KAFKA-20820
Project: Kafka
Issue Type: Bug
Reporter: Gergely Harmadás
Assignee: Gergely Harmadás
The issue already came up earlier on the mailing list
[https://lists.apache.org/thread/ro8mbk70xstywz5yw7mlm4yoswcfqlp5]
The bug can be reproduced by pointing log.dirs to a symlinked path, e.g.:
_/kafka/1/dd -> /navencrypt/1/kafka/kafka/1/dd_
The storage format step attempts to write the required meta.properties file to
the configured log directory and fails with:
{code:java}
Error while writing meta.properties file /kafka/3/dd:
java.nio.file.FileAlreadyExistsException: /kafka/3/dd{code}
The broker later exits with:
{noformat}
java.lang.RuntimeException: No readable meta.properties files found.{noformat}
Based on my research the root cause is a bug in JDK's Files.createDirectories
which was addressed in [https://bugs.openjdk.org/browse/JDK-8294193] and
shipped in JDK 20.
Specific code location in Kafka
[https://github.com/apache/kafka/blob/8fc539cb383c34e44bdb31721e0643faf453b8e6/metadata/src/main/java/org/apache/kafka/metadata/storage/Formatter.java#L452]
To conclude, Kafka 4.x is committed to use JDK17 and there is no real
restriction on why symlinked paths could not be used for log dirs. Also this
bug prevents to use certain tools like NavEncrypt which could handle encryption
at rest.
I would propose a simple fix to also make symlinked log.dirs possible when on
JDK17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
