GitHub user akrasnov-drv added a comment to the discussion: Failures while assigning public IP via enableStaticNat
I removed all VMs and network and recreated with standard isolated network offering with static nat with single VR. Each call execution time was increasing with the last one almost at timeout: ``` Wed Feb 5 04:12:44 PM UTC 2025 { "success": true } real 0m3.929s user 0m0.280s sys 0m0.024s Wed Feb 5 04:12:48 PM UTC 2025 Wed Feb 5 04:12:50 PM UTC 2025 { "success": true } real 0m3.842s user 0m0.274s sys 0m0.033s Wed Feb 5 04:12:54 PM UTC 2025 Wed Feb 5 04:12:56 PM UTC 2025 { "success": true } real 0m3.973s user 0m0.289s sys 0m0.016s Wed Feb 5 04:13:00 PM UTC 2025 ... Wed Feb 5 04:17:54 PM UTC 2025 { "success": true } real 0m9.809s user 0m0.285s sys 0m0.024s Wed Feb 5 04:18:04 PM UTC 2025 Wed Feb 5 04:18:06 PM UTC 2025 { "success": true } real 0m10.102s user 0m0.281s sys 0m0.032s Wed Feb 5 04:18:17 PM UTC 2025 Wed Feb 5 04:18:19 PM UTC 2025 Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 446, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 441, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse response.begin() File "/usr/lib/python3.10/http/client.py", line 318, in begin version, status, reason = self._read_status() File "/usr/lib/python3.10/http/client.py", line 279, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.10/socket.py", line 705, in readinto return self._sock.recv_into(b) TimeoutError: timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 756, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 534, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise raise value File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 700, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 448, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 337, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=10) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/bin/cloudstack", line 11, in <module> load_entry_point('cs==2.7.1', 'console_scripts', 'cs')() File "/usr/lib/python3/dist-packages/cs/__init__.py", line 104, in main response = getattr(cs, command)(fetch_result=fetch_result, File "/usr/lib/python3/dist-packages/cs/client.py", line 213, in handler return self._request(command, **kwargs) File "/usr/lib/python3/dist-packages/cs/client.py", line 273, in _request response = session.send(prepped, File "/usr/lib/python3/dist-packages/requests/sessions.py", line 657, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=8080): Read timed out. (read timeout=10) real 0m10.337s user 0m0.293s sys 0m0.036s Wed Feb 5 04:18:29 PM UTC 2025 ``` It managed to assign about 40 IPs on the way before failing. So, there are 2 outcomes from my tests - redundant VR really doubles (even more) the time of the call - there is some problem that each next call takes more time (and it depends on number of calls/assigned IPs, not on some extra load, if I try the call after some significant time it still fails) Attaching logs from management and from VR for the time of above. [cloud.log.gz](https://github.com/user-attachments/files/18675689/cloud.log.gz) [management-server.log.gz](https://github.com/user-attachments/files/18675690/management-server.log.gz) GitHub link: https://github.com/apache/cloudstack/discussions/10512#discussioncomment-12405017 ---- This is an automatically sent email for users@cloudstack.apache.org. To unsubscribe, please send an email to: users-unsubscr...@cloudstack.apache.org