Anil, Yes, but this information is also in the manual - I encourage you to take a look. You can search for "debug" or "debug_mb" to find the relevant commands.
Thanks Patrick ________________________________ From: Anil Mahadev Misal <[email protected]> Sent: Friday, February 6, 2026 5:58 PM To: Patrick Farrell <[email protected]>; [email protected] <[email protected]> Subject: Re: the -14 EFAULT error during pip installations Hi Patrick, Is is possible to share how to get debug log. Thanks & Regards, Anil Mahadev Misal Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Patrick Farrell <[email protected]> Sent: Friday, February 6, 2026 8:52:46 PM To: [email protected] <[email protected]>; Anil Mahadev Misal <[email protected]> Subject: Re: the -14 EFAULT error during pip installations Anil, If you are able to reproduce this, it would be helpful to gather debug logs as described in the Lustre manual (I suggest starting with -1 debug on the client). As is, this is interesting but not something investigable without more info. Regards, Patrick ________________________________ From: lustre-discuss <[email protected]> on behalf of Anil Mahadev Misal via lustre-discuss <[email protected]> Sent: Friday, February 6, 2026 2:19 AM To: [email protected] <[email protected]> Subject: [lustre-discuss] the -14 EFAULT error during pip installations Dear Team, I am facing error -14 while pip installation. ERROR: Could not install packages due to an OSError: [Errno 14] Bad address uname -r 5.14.0-611.13.1.el9_7.x86_64 lctl lustre_build_version Lustre version: 2.17.0 lctl get_param osc.*.max_rpcs_in_flight osc.lustrefs-OST0000-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0001-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0002-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0003-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0004-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0005-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0006-osc-MDT0000.max_rpcs_in_flight=8 osc.lustrefs-OST0007-osc-MDT0000.max_rpcs_in_flight=8 pip install 'tensorflow[and-cuda]' Collecting tensorflow[and-cuda] Using cached tensorflow-2.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.5 kB) Requirement already satisfied: absl-py>=1.0.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.4.0) Requirement already satisfied: astunparse>=1.6.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.6.3) Requirement already satisfied: flatbuffers>=24.3.25 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (25.12.19) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.7.0) Requirement already satisfied: google_pasta>=0.1.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.2.0) Requirement already satisfied: libclang>=13.0.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (18.1.1) Requirement already satisfied: opt_einsum>=2.3.2 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.4.0) Requirement already satisfied: packaging in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (25.0) Requirement already satisfied: protobuf>=5.28.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (6.33.5) Requirement already satisfied: requests<3,>=2.21.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.32.5) Requirement already satisfied: setuptools in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (80.10.2) Requirement already satisfied: six>=1.12.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.17.0) Requirement already satisfied: termcolor>=1.1.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.3.0) Requirement already satisfied: typing_extensions>=3.6.6 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (4.15.0) Requirement already satisfied: wrapt>=1.11.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.1.1) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (1.76.0) Requirement already satisfied: tensorboard~=2.20.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.20.0) Requirement already satisfied: keras>=3.10.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.13.2) Requirement already satisfied: numpy>=1.26.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.4.2) Requirement already satisfied: h5py>=3.11.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (3.15.1) Requirement already satisfied: ml_dtypes<1.0.0,>=0.5.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (0.5.4) Collecting nvidia-cublas-cu12<13.0,>=12.5.3.2 (from tensorflow[and-cuda]) Using cached nvidia_cublas_cu12-12.9.1.4-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-cupti-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_cupti_cu12-12.9.79-py3-none-manylinux_2_25_x86_64.whl.metadata (1.8 kB) Collecting nvidia-cuda-nvcc-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_nvcc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-nvrtc-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cuda-runtime-cu12<13.0,>=12.5.82 (from tensorflow[and-cuda]) Using cached nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cudnn-cu12<10.0,>=9.3.0.75 (from tensorflow[and-cuda]) Using cached nvidia_cudnn_cu12-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl.metadata (1.9 kB) Collecting nvidia-cufft-cu12<12.0,>=11.2.3.61 (from tensorflow[and-cuda]) Using cached nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Collecting nvidia-curand-cu12<11.0,>=10.3.6.82 (from tensorflow[and-cuda]) Using cached nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB) Collecting nvidia-cusolver-cu12<12.0,>=11.6.3.83 (from tensorflow[and-cuda]) Using cached nvidia_cusolver_cu12-11.7.5.82-py3-none-manylinux_2_27_x86_64.whl.metadata (1.9 kB) Collecting nvidia-cusparse-cu12<13.0,>=12.5.1.3 (from tensorflow[and-cuda]) Using cached nvidia_cusparse_cu12-12.5.10.65-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB) Requirement already satisfied: nvidia-nccl-cu12<3.0,>=2.25.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (2.29.3) Requirement already satisfied: nvidia-nvjitlink-cu12<13.0,>=12.5.82 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorflow[and-cuda]) (12.9.86) Requirement already satisfied: charset_normalizer<4,>=2 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.4.4) Requirement already satisfied: idna<4,>=2.5 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (3.11) Requirement already satisfied: urllib3<3,>=1.21.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2.6.3) Requirement already satisfied: certifi>=2017.4.17 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow[and-cuda]) (2026.1.4) Requirement already satisfied: markdown>=2.6.8 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (3.10.1) Requirement already satisfied: pillow in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (12.1.0) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from tensorboard~=2.20.0->tensorflow[and-cuda]) (3.1.5) Requirement already satisfied: wheel<1.0,>=0.23.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from astunparse>=1.6.0->tensorflow[and-cuda]) (0.46.3) Requirement already satisfied: rich in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (14.3.2) Requirement already satisfied: namex in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (0.1.0) Requirement already satisfied: optree in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from keras>=3.10.0->tensorflow[and-cuda]) (0.18.0) Requirement already satisfied: markupsafe>=2.1.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from werkzeug>=1.0.1->tensorboard~=2.20.0->tensorflow[and-cuda]) (3.0.3) Requirement already satisfied: markdown-it-py>=2.2.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from rich->keras>=3.10.0->tensorflow[and-cuda]) (4.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from rich->keras>=3.10.0->tensorflow[and-cuda]) (2.19.2) Requirement already satisfied: mdurl~=0.1 in /apps/conda/env/tf-gpu/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.10.0->tensorflow[and-cuda]) (0.1.2) Using cached tensorflow-2.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (620.7 MB) Using cached nvidia_cublas_cu12-12.9.1.4-py3-none-manylinux_2_27_x86_64.whl (581.2 MB) Using cached nvidia_cuda_cupti_cu12-12.9.79-py3-none-manylinux_2_25_x86_64.whl (10.8 MB) Using cached nvidia_cuda_nvcc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (40.5 MB) Using cached nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (89.6 MB) Using cached nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB) Using cached nvidia_cudnn_cu12-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl (657.9 MB) Using cached nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (200.9 MB) Using cached nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl (68.3 MB) Using cached nvidia_cusolver_cu12-11.7.5.82-py3-none-manylinux_2_27_x86_64.whl (338.1 MB) Using cached nvidia_cusparse_cu12-12.5.10.65-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (366.5 MB) Installing collected packages: nvidia-cusparse-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-nvcc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusolver-cu12, nvidia-cudnn-cu12, tensorflow ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/11 [nvidia-cusparse-cu12]ERROR: Could not install packages due to an OSError: [Errno 14] Bad address grep "Write request failed with -14" /var/log/messages | tail -n 20 kernel: LustreError: 841:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 14867:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 859:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 kernel: LustreError: 812:0:(osc_cache.c:2226:osc_check_rpcs()) Write request failed with -14 Thanks & Regards, Anil Mahadev Misal 📱 +91-7045598174 📧 [email protected]
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
