Re: [openstack-dev] [neutron] Stumped...need help with neutronclient job failure

[Matthew Treinish]

On Thu, Sep 24, 2015 at 10:52:45AM -0700, Kevin Benton wrote:
> Can you look to see what process tempest_lib is trying to execute?
> 
> On Wed, Sep 23, 2015 at 4:02 AM, Paul Michali  wrote:
> 
> > Hi,
> >
> > I created a pair of experimental jobs for python-neutronclient that will
> > run functional tests on core and advanced services, respectively. In the
> > python-neutronclient repo, I have a commit [1] that splits the tests into
> > two directories for core/adv-svcs, enables the VPN devstack plugin for the
> > advanced services tests, and removes the skip decorator for the VPN tests.
> >
> > When these two jobs run, the core job pass (as expected). The advanced
> > services job shows all four advanced services tests (testing REST LIST
> > requests for IKE policy, IPSec policy, IPSec site-to-site connection, and
> > VPN service resources) failing, with this T/B:
> >
> > ft1.1: 
> > neutronclient.tests.functional.adv-svcs.test_readonly_neutron_vpn.SimpleReadOnlyNeutronVpnClientTest.test_neutron_vpn_*ipsecpolicy_list*_StringException:
> >  Empty attachments:
> >   pythonlogging:''
> >   stderr
> >   stdout
> >
> > Traceback (most recent call last):
> >   File 
> > "neutronclient/tests/functional/adv-svcs/test_readonly_neutron_vpn.py", 
> > line 37, in test_neutron_vpn_ipsecpolicy_list
> > ipsecpolicy = self.parser.listing(self.neutron('vpn-ipsecpolicy-list'))
> >   File "neutronclient/tests/functional/base.py", line 78, in neutron
> > **kwargs)
> >   File 
> > "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
> >  line 292, in neutron
> > 'neutron', action, flags, params, fail_ok, merge_stderr)
> >   File 
> > "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
> >  line 361, in cmd_with_auth
> > self.cli_dir)
> >   File 
> > "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
> >  line 61, in execute
> > proc = subprocess.Popen(cmd, stdout=stdout, stderr=stderr)
> >   File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
> > errread, errwrite)
> >   File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
> > raise child_exception
> > OSError: [Errno 2] No such file or directory

So taking a blind guess without actually looking at anything besides this email
my thinking is that you aren't installing neutronclient in /usr/bin in that job.
Either it's being installed in the tox venv only or going into /usr/local/bin or
something like that. There is a parameter to give tempest-lib the bin dir where
the cli commands live. You need to make sure that's set to where ever you're
installing the CLI commands.

> >
> >
> > When I look at the other logs on this run [2], I see these things:
> > - The VPN agent is running (so the DevStack plugin started up VPN)
> > - screen-q-svc.log shows only two of the four REST GET requests
> > - Initially there was no testr results, but I modified post test hook
> > script similar to what Neutron does (so it shows results now)
> > - No other errors seen, including nothing on the StringException
> >
> > When I run this locally, all four tests pass, and I see four REST requests
> > in the screen-q-svc.log.
> >
> > I tried a hack to enable NEUTRONCLIENT_DEBUG environment variable, but no
> > additional information was shown.
> >
> > Does anyone have any thoughts on what may be going wrong here?
> > Any ideas on how to troubleshoot this issue?
> >
> > Thanks in advance!
> >
> > Paul Michali (pc_m)
> >
> > Refs
> > [1] https://review.openstack.org/#/c/214587/
> > [2]
> > http://logs.openstack.org/87/214587/8/experimental/gate-neutronclient-test-dsvm-functional-adv-svcs/5dfa152/
> >


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Stumped...need help with neutronclient job failure

[Kevin Benton]

Can you look to see what process tempest_lib is trying to execute?

On Wed, Sep 23, 2015 at 4:02 AM, Paul Michali  wrote:

> Hi,
>
> I created a pair of experimental jobs for python-neutronclient that will
> run functional tests on core and advanced services, respectively. In the
> python-neutronclient repo, I have a commit [1] that splits the tests into
> two directories for core/adv-svcs, enables the VPN devstack plugin for the
> advanced services tests, and removes the skip decorator for the VPN tests.
>
> When these two jobs run, the core job pass (as expected). The advanced
> services job shows all four advanced services tests (testing REST LIST
> requests for IKE policy, IPSec policy, IPSec site-to-site connection, and
> VPN service resources) failing, with this T/B:
>
> ft1.1: 
> neutronclient.tests.functional.adv-svcs.test_readonly_neutron_vpn.SimpleReadOnlyNeutronVpnClientTest.test_neutron_vpn_*ipsecpolicy_list*_StringException:
>  Empty attachments:
>   pythonlogging:''
>   stderr
>   stdout
>
> Traceback (most recent call last):
>   File 
> "neutronclient/tests/functional/adv-svcs/test_readonly_neutron_vpn.py", line 
> 37, in test_neutron_vpn_ipsecpolicy_list
> ipsecpolicy = self.parser.listing(self.neutron('vpn-ipsecpolicy-list'))
>   File "neutronclient/tests/functional/base.py", line 78, in neutron
> **kwargs)
>   File 
> "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
>  line 292, in neutron
> 'neutron', action, flags, params, fail_ok, merge_stderr)
>   File 
> "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
>  line 361, in cmd_with_auth
> self.cli_dir)
>   File 
> "/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
>  line 61, in execute
> proc = subprocess.Popen(cmd, stdout=stdout, stderr=stderr)
>   File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
> errread, errwrite)
>   File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
> raise child_exception
> OSError: [Errno 2] No such file or directory
>
>
> When I look at the other logs on this run [2], I see these things:
> - The VPN agent is running (so the DevStack plugin started up VPN)
> - screen-q-svc.log shows only two of the four REST GET requests
> - Initially there was no testr results, but I modified post test hook
> script similar to what Neutron does (so it shows results now)
> - No other errors seen, including nothing on the StringException
>
> When I run this locally, all four tests pass, and I see four REST requests
> in the screen-q-svc.log.
>
> I tried a hack to enable NEUTRONCLIENT_DEBUG environment variable, but no
> additional information was shown.
>
> Does anyone have any thoughts on what may be going wrong here?
> Any ideas on how to troubleshoot this issue?
>
> Thanks in advance!
>
> Paul Michali (pc_m)
>
> Refs
> [1] https://review.openstack.org/#/c/214587/
> [2]
> http://logs.openstack.org/87/214587/8/experimental/gate-neutronclient-test-dsvm-functional-adv-svcs/5dfa152/
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Stumped...need help with neutronclient job failure

[Paul Michali]

Hi,

I created a pair of experimental jobs for python-neutronclient that will
run functional tests on core and advanced services, respectively. In the
python-neutronclient repo, I have a commit [1] that splits the tests into
two directories for core/adv-svcs, enables the VPN devstack plugin for the
advanced services tests, and removes the skip decorator for the VPN tests.

When these two jobs run, the core job pass (as expected). The advanced
services job shows all four advanced services tests (testing REST LIST
requests for IKE policy, IPSec policy, IPSec site-to-site connection, and
VPN service resources) failing, with this T/B:

ft1.1: 
neutronclient.tests.functional.adv-svcs.test_readonly_neutron_vpn.SimpleReadOnlyNeutronVpnClientTest.test_neutron_vpn_*ipsecpolicy_list*_StringException:
Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "neutronclient/tests/functional/adv-svcs/test_readonly_neutron_vpn.py",
line 37, in test_neutron_vpn_ipsecpolicy_list
ipsecpolicy = self.parser.listing(self.neutron('vpn-ipsecpolicy-list'))
  File "neutronclient/tests/functional/base.py", line 78, in neutron
**kwargs)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 292, in neutron
'neutron', action, flags, params, fail_ok, merge_stderr)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 361, in cmd_with_auth
self.cli_dir)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 61, in execute
proc = subprocess.Popen(cmd, stdout=stdout, stderr=stderr)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory


When I look at the other logs on this run [2], I see these things:
- The VPN agent is running (so the DevStack plugin started up VPN)
- screen-q-svc.log shows only two of the four REST GET requests
- Initially there was no testr results, but I modified post test hook
script similar to what Neutron does (so it shows results now)
- No other errors seen, including nothing on the StringException

When I run this locally, all four tests pass, and I see four REST requests
in the screen-q-svc.log.

I tried a hack to enable NEUTRONCLIENT_DEBUG environment variable, but no
additional information was shown.

Does anyone have any thoughts on what may be going wrong here?
Any ideas on how to troubleshoot this issue?

Thanks in advance!

Paul Michali (pc_m)

Refs
[1] https://review.openstack.org/#/c/214587/
[2]
http://logs.openstack.org/87/214587/8/experimental/gate-neutronclient-test-dsvm-functional-adv-svcs/5dfa152/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev