This is an automated email from the ASF dual-hosted git repository.
zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new a8ac18f2e [CELEBORN-299] Deprecate
`celeborn.worker.storage.baseDir.prefix` and
`celeborn.worker.storage.baseDir.number`
a8ac18f2e is described below
commit a8ac18f2e8a74095e558f57452794ebf733ba678
Author: sychen <[email protected]>
AuthorDate: Mon Oct 16 19:10:13 2023 +0800
[CELEBORN-299] Deprecate `celeborn.worker.storage.baseDir.prefix` and
`celeborn.worker.storage.baseDir.number`
### What changes were proposed in this pull request?
<img width="1460" alt="image"
src="https://github.com/apache/incubator-celeborn/assets/3898450/ac3b29be-7c39-4c18-b71d-0e243797273e";>
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
```
23/10/16 03:31:13,399 WARN
[pool-1-thread-1-ScalaTest-running-CelebornConfSuite] CelebornConf: The
configuration key 'celeborn.worker.storage.baseDir.prefix' has been deprecated
in v0.4.0 and may be removed in the future. Please use
celeborn.worker.storage.dirs
23/10/16 03:31:13,399 WARN
[pool-1-thread-1-ScalaTest-running-CelebornConfSuite] CelebornConf: The
configuration key 'celeborn.worker.storage.baseDir.number' has been deprecated
in v0.4.0 and may be removed in the future. Please use
celeborn.worker.storage.dirs
```
Closes #1993 from cxzl25/CELEBORN-299.
Authored-by: sychen <[email protected]>
Signed-off-by: zky.zhoukeyong <[email protected]>
---
.../src/main/scala/org/apache/celeborn/common/CelebornConf.scala | 9 ++++++++-
docs/migration.md | 3 +++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git
a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
index bcd462213..ae05a38ae 100644
--- a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
@@ -1088,7 +1088,14 @@ object CelebornConf extends Logging {
*/
private val deprecatedConfigs: Map[String, DeprecatedConfig] = {
val configs = Seq(
- DeprecatedConfig("none", "1.0", "None"))
+ DeprecatedConfig(
+ "celeborn.worker.storage.baseDir.prefix",
+ "0.4.0",
+ "Please use celeborn.worker.storage.dirs"),
+ DeprecatedConfig(
+ "celeborn.worker.storage.baseDir.number",
+ "0.4.0",
+ "Please use celeborn.worker.storage.dirs"))
Map(configs.map { cfg => (cfg.key -> cfg) }: _*)
}
diff --git a/docs/migration.md b/docs/migration.md
index 4840a5803..faaff3d09 100644
--- a/docs/migration.md
+++ b/docs/migration.md
@@ -39,6 +39,9 @@ license: |
| `celeborn.metrics.worker.prometheus.host` | `celeborn.worker.http.host` |
| `celeborn.metrics.worker.prometheus.port` | `celeborn.worker.http.port` |
+- Since 0.4.0, Celeborn deprecate `celeborn.worker.storage.baseDir.prefix` and
`celeborn.worker.storage.baseDir.number`.
+ Please use `celeborn.worker.storage.dirs` instead.
+
## Upgrading from 0.3.0 to 0.3.1
- Since 0.3.1, Celeborn changed the default value of
`celeborn.worker.directMemoryRatioToResume` from `0.5` to `0.7`.
