Public bug reported: Since 18th Nov nftables jobs failing with RETRY_LIMIT and no logs are available. These jobs quickly fails and last task seen running in console is "Preparing job workspace".
This looks a regression with zuul change https://review.opendev.org/c/zuul/zuul/+/900489. @fungi checked executor logs and it fails as:- 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: [e: a4e70e9b208f479e98334c09305d2013] [build: 3c41d75624274d2e8d7fd62ae332c31d] Exception while executing job 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: Traceback (most recent call last): 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1125, in __getitem__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: return getattr(self, index) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1114, in __getattr__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: return list.__getattribute__(self, attr) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: AttributeError: 'IterableList' object has no attribute '2.3.0' 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: The above exception was the direct cause of the following exception: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: Traceback (most recent call last): 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 1185, in do_execute 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self._execute() 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 1556, in _execute 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.preparePlaybooks(args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2120, in preparePlaybooks 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.preparePlaybook(jobdir_playbook, playbook, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2177, in preparePlaybook 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.prepareRole(jobdir_playbook, role, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2367, in prepareRole 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.prepareZuulRole(jobdir_playbook, role, args, role_info) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2423, in prepareZuulRole 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: path = self.checkoutUntrustedProject(project, branch, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2295, in checkoutUntrustedProject 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: repo.getBranchHead(branch).hexsha 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 465, in getBranchHead 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: branch_head = repo.heads[branch] 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ~~~~~~~~~~^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1127, in __getitem__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: raise IndexError("No item found with id %r" % (self._prefix + index)) from e 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: IndexError: No item found with id '2.3.0' The parent job[1] which does pins neutron-tempest-plugin to 2.3.0 also do pass but not the inherited ones, same can be seen in the test patch[2] Builds:- https://zuul.openstack.org/builds?job_name=neutron-linuxbridge- tempest-plugin-scenario-nftables&job_name=neutron-ovs-tempest-plugin- scenario-iptables_hybrid- nftables&project=openstack%2Fneutron&branch=stable%2Fxena&skip=0 [1] https://zuul.openstack.org/builds?job_name=neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-xena [2] https://review.opendev.org/c/openstack/neutron/+/902296 ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2045383 Title: [xena] nftables periodic jobs fails with RETRY_LIMIT Status in neutron: New Bug description: Since 18th Nov nftables jobs failing with RETRY_LIMIT and no logs are available. These jobs quickly fails and last task seen running in console is "Preparing job workspace". This looks a regression with zuul change https://review.opendev.org/c/zuul/zuul/+/900489. @fungi checked executor logs and it fails as:- 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: [e: a4e70e9b208f479e98334c09305d2013] [build: 3c41d75624274d2e8d7fd62ae332c31d] Exception while executing job 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: Traceback (most recent call last): 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1125, in __getitem__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: return getattr(self, index) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1114, in __getattr__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: return list.__getattribute__(self, attr) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: AttributeError: 'IterableList' object has no attribute '2.3.0' 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: The above exception was the direct cause of the following exception: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: Traceback (most recent call last): 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 1185, in do_execute 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self._execute() 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 1556, in _execute 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.preparePlaybooks(args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2120, in preparePlaybooks 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.preparePlaybook(jobdir_playbook, playbook, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2177, in preparePlaybook 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.prepareRole(jobdir_playbook, role, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2367, in prepareRole 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: self.prepareZuulRole(jobdir_playbook, role, args, role_info) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2423, in prepareZuulRole 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: path = self.checkoutUntrustedProject(project, branch, args) 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/executor/server.py", line 2295, in checkoutUntrustedProject 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: repo.getBranchHead(branch).hexsha 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 465, in getBranchHead 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: branch_head = repo.heads[branch] 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: ~~~~~~~~~~^^^^^^^^ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: File "/usr/local/lib/python3.11/site-packages/git/util.py", line 1127, in __getitem__ 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: raise IndexError("No item found with id %r" % (self._prefix + index)) from e 2023-11-30 14:08:14,253 ERROR zuul.AnsibleJob: IndexError: No item found with id '2.3.0' The parent job[1] which does pins neutron-tempest-plugin to 2.3.0 also do pass but not the inherited ones, same can be seen in the test patch[2] Builds:- https://zuul.openstack.org/builds?job_name=neutron- linuxbridge-tempest-plugin-scenario-nftables&job_name=neutron-ovs- tempest-plugin-scenario-iptables_hybrid- nftables&project=openstack%2Fneutron&branch=stable%2Fxena&skip=0 [1] https://zuul.openstack.org/builds?job_name=neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-xena [2] https://review.opendev.org/c/openstack/neutron/+/902296 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2045383/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
