Jure Pečar <[email protected]> writes:
> On Mon, 05 May 2025 16:27:08 +0200
> "Loris Bennett" <loris.bennett-j/[email protected]> wrote:
>
>> [build@dadmin eco]$ rm
>> -rf /trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0 rm: cannot
>> remove
>> '/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmptgidvyac-bazel-root/install/a9574404bbcb6906c73792d9ebf99d36/.nfs000000010101265b00000032':
>> Device or resource busy rm: cannot remove
>> '/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmptgidvyac-bazel-root/install/a9574404bbcb6906c73792d9ebf99d36/.nfs000000010101260400000033':
>> Device or resource busy rm: cannot remove
>> '/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmptgidvyac-bazel-root/f3954181a366e6f66e4bfbf5ea055ac4/server':
>> Directory not empty
>
> You have files opened from this directory on some other node.
>
> You can mv this directory to rename it and make room for new installation
> or you can trace down who is keeping it open and kill those processes.
The directory is, as far as I understand, created by EasyBuild itself
during the build process. The build in fact is successful, just the
clean-up step after the sanity check fails:
== sanity checking...
>> file 'bin/bazel' found: OK
>> loading modules: Bazel/6.1.0-GCCcore-12.3.0...
>> running command 'bazel
--output_user_root=/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmprw3ggx7v-bazel-root
--help' ...
>> result for command 'bazel
--output_user_root=/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmprw3ggx7v-bazel-root
--help': OK
== ... (took 3 secs)
== cleaning up...
== ... (took 1 min 37 secs)
== FAILED: Installation ended unsuccessfully: Failed to remove directory
/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0 even after 3
attempts.
Reasons: [OSError(39, 'Directory not empty'), OSError(16, 'Device or resource
busy'), OSError(16, 'Device or resource busy')] (took 21 mins 52 secs)
== Results of the build can be found in the log file(s)
/tmp/eb-jyf6dyxz/easybuild-Bazel-6.1.0-20250506.074415.AwxPh.log
ERROR: Installation of Bazel-6.1.0-GCCcore-12.3.0.eb failed: "Failed to
remove directory /trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0
even after 3 attempts.\nReasons: [OSError(39, 'Directory not empty'),
OSError(16, 'Device or resource busy'), OSError(16, 'Device or resource busy')]"
Initially trying to remove the directory explicitly fails:
[build@dadmin eco]$ rm -rf /trinity/shared/easybuild/build/Bazel/6.1.0/
rm: cannot remove
'/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmprw3ggx7v-bazel-root/install/62c30a383d9f99337c332af80bc94336/.nfs000000010103774c00000038':
Device or resource busy
rm: cannot remove
'/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmprw3ggx7v-bazel-root/install/62c30a383d9f99337c332af80bc94336/.nfs000000010103733d00000039':
Device or resource busy
rm: cannot remove
'/trinity/shared/easybuild/build/Bazel/6.1.0/GCCcore-12.3.0/tmprw3ggx7v-bazel-root/f3954181a366e6f66e4bfbf5ea055ac4/server':
Directory not empty
However, a few minutes later it works:
[build@dadmin eco]$ rm -rf /trinity/shared/easybuild/build/Bazel/6.1.0/
[build@dadmin eco]$
Nevertheless, if I run EasyBuild without '--disable-cleanup-builddir'
the installation fails consistently. Other ECs don't have this problem.
Cheers,
Loris
--
Dr. Loris Bennett (Herr/Mr)
FUB-IT, Freie Universität Berlin
