This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 759638e0b [CELEBORN-1467] celeborn.worker.storage.dirs should support
soft link
759638e0b is described below
commit 759638e0b17f0b80797964ffcab7007170134984
Author: Xianming Lei <[email protected]>
AuthorDate: Wed Jun 19 10:44:38 2024 +0800
[CELEBORN-1467] celeborn.worker.storage.dirs should support soft link
### What changes were proposed in this pull request?
`celeborn.worker.storage.dirs` supports soft link
### Why are the changes needed?
NPE will be thrown when StorageManager.createWriter if
`celeborn.worker.storage.dirs` contains soft link.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing uts.
Closes #2576 from leixm/CELEBORN-1467.
Authored-by: Xianming Lei <[email protected]>
Signed-off-by: mingji <[email protected]>
---
.../src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala
b/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala
index 0fc2f4984..29971c3cf 100644
--- a/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala
@@ -288,7 +288,9 @@ object DeviceInfo {
val deviceInfo = mountPointToDeviceInfo.get(mountPoint)
val diskInfo = new DiskInfo(
mountPoint,
- dirs.map(_._1).toList,
+ dirs.map { workingDir =>
+ new File(workingDir._1.getCanonicalPath)
+ }.toList,
deviceInfo,
conf)
val (_, maxUsableSpace, threadCount, storageType) = dirs(0)
