[jira] [Updated] (SAMZA-2752) DiskSpace monitor reports 0 disk usage even for a stateful job with lots of disk usage

[Lakshmi Manasa Gaduputi (Jira)]

Title: Message Title

Lakshmi Manasa Gaduputi updated an issue   Samza / SAMZA-2752 DiskSpace monitor reports 0 disk usage even for a stateful job with lots of disk usage Change By: Lakshmi Manasa Gaduputi Stateful jobs after PR (#1491): report 0 disk usage for their disk spaced used by their rocksdb kv stores.  Reason: # [PR|https://github.com/apache/samza/pull/1491] changed the init of rocksdb task stores from constructor of ContainerStorageManager to ContainerStorageManager.restoreStores() (which is invoked at run and not in the constructor). see these code changes of the PR for file ContainerStorageManager #  However, SamzaContainer creates the disk space monitor during [setup (here) |https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala#L661] (before run() which invokes restoreStores()) and SamzaContainer uses the store paths returned by ContainerStorageManager after its constructor is done. # Hence, the task stores (not side input) are not part of the disk space monitoring. # logs show "2022-06-22 20:46:41.897 [main] SamzaContainer$ [INFO] Initialized disk space monitor watch paths to: [] // basically disk space monitor is not looking at anything" after this PR but should be showing a non-empty set of paths     Add Comment   This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)