Reviewed: https://review.opendev.org/c/openstack/nova/+/845543 Committed: https://opendev.org/openstack/nova/commit/8f4b740ca5292556f8e953a30f2a11ed4fbc2945 Submitter: "Zuul (22348)" Branch: master
commit 8f4b740ca5292556f8e953a30f2a11ed4fbc2945 Author: Takashi Kajinami <tkaji...@redhat.com> Date: Mon Jun 13 14:48:24 2022 +0900 Retry attachment delete API call for 504 Gateway Timeout When cinder-api runs behind a load balancer(eg haproxy), the load balancer can return 504 Gateway Timeout when cinder-api does not respond within timeout. This change ensures nova retries deleting a volume attachment in that case. Also this change makes nova ignore 404 in the API call. This is required because cinder might continue deleting the attachment even if the load balancer returns 504. This also helps us in the situation where the volume attachment was accidentally removed by users. Closes-Bug: #1978444 Change-Id: I593011d9f4c43cdae7a3d53b556c6e2a2b939989 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1978444 Title: Volume can't be detached if attachment delete api call fails with 504 gateway timeout Status in OpenStack Compute (nova): Fix Released Bug description: Description =========== When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout. When this timeout occurs while detaching a volume, this results in un-detachable volume. - nova-compute calls delete attachment api in cinder - haproxy detects server timeout and returns 504 - cinder continues processing the API and removes the attachment - nova-compute immediately aborts the volume detachment and leaves the bdm - when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova See for details https://bugzilla.redhat.com/show_bug.cgi?id=2002643 Steps to reproduce ================== * Stop cinder-volume * Detach a volume from an instance * Start cinder-volume * Detach the volume again Expected result =============== * Volume can be detached after cinder-volume is recovered Actual result =============== * Volume can't be detached Environment =========== * The issue was initially found in stable/train Logs & Configs ============== * See https://bugzilla.redhat.com/show_bug.cgi?id=2002643#c1 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1978444/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp