Anil, If you are able to reproduce this, it would be helpful to gather debug logs as described in the Lustre manual (I suggest starting with -1 debug on the client). As is, this is interesting but not something investigable without more info.
Regards, Patrick ________________________________ From: lustre-discuss <[email protected]> on behalf of Anil Mahadev Misal via lustre-discuss <[email protected]> Sent: Friday, February 6, 2026 2:19 AM To: [email protected] <[email protected]> Subject: [lustre-discuss] the -14 EFAULT error during pip installations Dear Team, I am facing error -14 while pip installation. ERROR: Could not install packages due to an OSError: [Errno 14] Bad address uname -r 5.14.0-611.13.1.el9_7.x86_64 lctl lustre_build_version Lustre version: 2.17.0 lctl get_param osc.*.max_rpcs_in_flight osc.lustrefs-OST0000-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0001-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0002-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0003-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0004-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0005-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0006-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0007-osc-MDT0000.max_rpcs_in_flight=8 pip install 'tensorflow[and-cuda]' Collecting tensorflow[and-cuda] Using cached tensorflow-2.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.5 kB) Requirement already satisfied: absl-py>=1.0.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.4.0) Requirement already satisfied: astunparse>=1.6.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.6.3) Requirement already satisfied: flatbuffers>=24.3.25 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (25.12.19) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.7.0) Requirement already satisfied: google_pasta>=0.1.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.2.0) Requirement already satisfied: libclang>=13.0.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (18.1.1) Requirement already satisfied: opt_einsum>=2.3.2 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.4.0) Requirement already satisfied: packaging in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (25.0) Requirement already satisfied: protobuf>=5.28.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (6.33.5) Requirement already satisfied: requests<3,>=2.21.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.32.5) Requirement already satisfied: setuptools in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (80.10.2) Requirement already satisfied: six>=1.12.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.17.0) Requirement already satisfied: termcolor>=1.1.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.3.0) Requirement already satisfied: typing_extensions>=3.6.6 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (4.15.0) Requirement already satisfied: wrapt>=1.11.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.1.1) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.76.0) Requirement already satisfied: tensorboard~=2.20.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.20.0) Requirement already satisfied: keras>=3.10.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.13.2) Requirement already satisfied: numpy>=1.26.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.4.2) Requirement already satisfied: h5py>=3.11.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.15.1) Requirement already satisfied: ml_dtypes<1.0.0,>=0.5.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.5.4) Collecting nvidia-cublas-cu12<13.0,>=12.5.3.2 (from tensorflow[and-cuda]) Using cached nvidia_cublas_cu12-12.9.1.4-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-cupti-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_cupti_cu12-12.9.79-py3-none-manylinux_2_25_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cuda-nvcc-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_nvcc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-nvrtc-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-runtime-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cudnn-cu12<10.0,>=9.3.0.75 (from tensorflow[and-cuda]) Using cached nvidia_cudnn_cu12-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl.metadata (1.9 kB) Collecting nvidia-cufft-cu12<12.0,>=11.2.3.61 (from tensorflow[and-cuda]) Using cached nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-curand-cu12<11.0,>=10.3.6.82 (from tensorflow[and-cuda]) Using cached nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cusolver-cu12<12.0,>=11.6.3.83 (from tensorflow[and-cuda]) Using cached nvidia_cusolver_cu12-11.7.5.82-py3-none-manylinux_2_27_x86_64.whl.metadata (1.9 kB) Collecting nvidia-cusparse-cu12<13.0,>=12.5.1.3 (from tensorflow[and-cuda]) Using cached nvidia_cusparse_cu12-12.5.10.65-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Requirement already satisfied: nvidia-nccl-cu12<3.0,>=2.25.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.29.3) Requirement already satisfied: nvidia-nvjitlink-cu12<13.0,>=12.5.82 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (12.9.86) Requirement already satisfied: charset_normalizer<4,>=2 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.4.4) Requirement already satisfied: idna<4,>=2.5 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.11) Requirement already satisfied: urllib3<3,>=1.21.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2.6.3) Requirement already satisfied: certifi>=2017.4.17 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2026.1.4) Requirement already satisfied: markdown>=2.6.8 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (3.10.1) Requirement already satisfied: pillow in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (12.1.0) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (3.1.5) Requirement already satisfied: wheel<1.0,>=0.23.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from astunparse>=1.6.0->tensorflow[and-cuda]) (0.46.3) Requirement already satisfied: rich in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (14.3.2) Requirement already satisfied: namex in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (0.1.0) Requirement already satisfied: optree in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (0.18.0) Requirement already satisfied: markupsafe>=2.1.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from werkzeug>=1.0.1->tensorboard~=2.20.0->tensorflow[and-cuda]) (3.0.3) Requirement already satisfied: markdown-it-py>=2.2.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from rich->keras>=3.10.0->tensorflow[and-cuda]) (4.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from rich->keras>=3.10.0->tensorflow[and-cuda]) (2.19.2) Requirement already satisfied: mdurl~=0.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.10.0->tensorflow[and-cuda]) (0.1.2) Using cached tensorflow-2.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (620.7 MB) Using cached nvidia_cublas_cu12-12.9.1.4-py3-none-manylinux_2_27_x86_64.whl (581.2 MB) Using cached nvidia_cuda_cupti_cu12-12.9.79-py3-none-manylinux_2_25_x86_64.whl (10.8 MB) Using cached nvidia_cuda_nvcc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (40.5 MB) Using cached nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (89.6 MB) Using cached nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB) Using cached nvidia_cudnn_cu12-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl (657.9 MB) Using cached nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (200.9 MB) Using cached nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl (68.3 MB) Using cached nvidia_cusolver_cu12-11.7.5.82-py3-none-manylinux_2_27_x86_64.whl (338.1 MB) Using cached nvidia_cusparse_cu12-12.5.10.65-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (366.5 MB) Installing collected packages: nvidia-cusparse-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-nvcc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusolver-cu12, nvidia-cudnn-cu12, tensorflow ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/11 [nvidia-cusparse-cu12]ERROR: Could not install packages due to an OSError: [Errno 14] Bad address grep "Write request failed with -14" /var/log/messages | tail -n 20 kernel: LustreError: 841:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 14867:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 859:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 812:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 Thanks & Regards, Anil Mahadev Misal 📱 +91-7045598174 📧 [email protected]
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
