abudnik commented on a change in pull request #355: Handle EBUSY when
destroying a cgroup.
URL: https://github.com/apache/mesos/pull/355#discussion_r402256087
##########
File path: src/linux/cgroups.cpp
##########
@@ -696,13 +696,24 @@ Try<Nothing> remove(const string& hierarchy, const
string& cgroup)
string path = path::join(hierarchy, cgroup);
// Do NOT recursively remove cgroups.
- Try<Nothing> rmdir = os::rmdir(path, false);
- if (rmdir.isError()) {
- return Error(
+ // TODO The retry was added as a fix for kernel bug
+ // https://lkml.org/lkml/2020/1/15/1349
+ // where calling rmdir on a seemingly empty cgroup can fail
+ // with EBUSY while some tasks are exiting.
+ auto delay = Milliseconds(1);
+ for (auto retry = 10; ;) {
+ Try<Nothing> rmdir = os::rmdir(path, false);
+ if (!rmdir.isError()) {
+ return rmdir;
+ } else if (retry > 0) {
+ os::sleep(delay);
Review comment:
An internal helper function can be introduced to implement non-blocking
retry logic:
```
Future<Nothing> remove(const std::string& hierarchy, const
std::vector<std::string>& groups)
````
[XfsDiskIsolatorProcess::initialize](https://github.com/apache/mesos/blob/f8a3dd334934094ec44e07fa350f958d218bc78f/src/slave/containerizer/mesos/isolators/xfs/disk.cpp#L973-L982)
and
[IOSwitchboard::_connect](https://github.com/apache/mesos/blob/f8a3dd334934094ec44e07fa350f958d218bc78f/src/slave/containerizer/mesos/io/switchboard.cpp#L695-L713)
are examples of how `process::loop()` can be used to implement non-blocking
retries.
The new `remove` function can be used in both places you have mentioned
above where the blocking `cgroups::remove` function is called.
> we should check for EBUSY
Just call `::rmdir()` (no `os::` namespace)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
