Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Mark McClain 
On May 2, 2014, at 10:28 AM, Doug Hellmann  wrote:

> As Robert said, libraries should not be calling sys.exit() or raising
> SystemExit directly, ever.
> 
> Throwing SystemExit from a library bypasses other exception handling
> cleanup code higher in the stack that is unlikely to be looking for
> fatal exceptions like SystemExit (because well-behaved libraries don't
> throw those exceptions). Libraries should define meaningful
> exceptions, subclassed from Exception, which the main application can
> log before deciding whether to exit, retry, pick another driver, or
> whatever.

I’ll add this to the list of items to address as part of the clean up of the 
Neutron core code.

mark


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Doug Hellmann 
My guess is the rootwrap and oslo service stuff is OK, the daemon
module may be OK, but all of the plugins should be changed. That's
just a guess after a cursory review, though, and someone who knows the
neutron code better than I do will have to make the call. Some of
those plugin modules may hold the main function for independent
services, for example.

On Fri, May 2, 2014 at 11:05 AM, Paul Michali (pcm)  wrote:
> Here are the calls in Neutron:
>
> neutron/agent/l3_agent.py:raise SystemExit(msg)
> neutron/agent/l3_agent.py:raise SystemExit(msg)
> neutron/agent/l3_agent.py:raise SystemExit(msg)
> neutron/agent/linux/dhcp.py:raise SystemExit(msg)
> neutron/agent/linux/dhcp.py:raise SystemExit(msg)
> neutron/db/migration/cli.py:raise SystemExit(_('You must provide a 
> revision or relative delta'))
> neutron/openstack/common/service.py:class SignalExit(SystemExit):
> neutron/openstack/common/service.py:except SystemExit as exc:
> neutron/openstack/common/service.py:except SystemExit as exc:
> neutron/plugins/ibm/agent/sdnve_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ibm/agent/sdnve_neutron_agent.py:raise SystemExit(1)
> neutron/plugins/ml2/managers.py:raise SystemExit(msg)
> neutron/plugins/mlnx/agent/eswitch_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/mlnx/agent/utils.py:raise SystemExit(msg)
> neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
> neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
> neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
> neutron/plugins/nec/nec_router.py:raise SystemExit(1)
> neutron/plugins/nec/nec_router.py:raise SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
> SystemExit(1)
> neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:raise 
> SystemExit(1)
> neutron/services/loadbalancer/agent/agent_manager.py:raise 
> SystemExit(msg % driver)
> neutron/services/loadbalancer/agent/agent_manager.py:raise 
> SystemExit(msg % driver_name)
> neutron/services/loadbalancer/plugin.py:raise SystemExit(msg)
> neutron/services/metering/agents/metering_agent.py:raise 
> SystemExit(_('A metering driver must be specified'))
> neutron/services/metering/drivers/iptables/iptables_driver.py:
> raise SystemExit(_('An interface driver must be specified'))
> neutron/services/service_base.py:raise SystemExit(msg)
> neutron/services/vpn/device_drivers/cisco_ipsec.py:raise 
> SystemExit(_('No Cisco CSR configurations found in: %s') %
>
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_NOCOMMAND)
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_UNAUTHORIZED)
> bin/neutron-rootwrap-xen-dom0:sys.exit(RC_XENAPI_ERROR)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_NOCOMMAND)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_UNAUTHORIZED)
> bin/quantum-rootwrap-xen-dom0:sys.exit(RC_XENAPI_ERROR)
> neutron/agent/linux/daemon.py:sys.exit(1)
> neutron/agent/linux/daemon.py:sys.exit(0)
> neutron/agent/linux/daemon.py:sys.exit(1)
> neutron/agent/linux/daemon.py:sys.exit(0)
> neutron/agent/linux/daemon.py:sys.exit(1)
> neutron/agent/linux/dhcp.py:sys.exit()
> neutron/openstack/common/lockutils.py:sys.exit(main(sys.argv))
> neutron/openstack/common/rpc/amqp.py:# just before doing a 
> sys.exit(), so cleanup() only happens once and
> neutron/openstack/common/service.py:sys.exit(1)
> neutron/openstack/common/systemd.py:sys.exit(retval)
> neutron/plugins/bigswitch/agent/restproxy_agent.py:sys.exit(0)
> neutron/plugins/linuxbridge/agent/linuxbridge_neutron_agent.py:
> sys.exit(1)
> neutron/plugins/linuxbridge/agent/linuxbridge_neutron_agent.py:sys.exit(0)
> neutron/plugins/linuxbridge/lb_neutron_plugin.py:sys.exit(1)
> neutron/plugins/linuxbridge/lb_neutron_plugin.p

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Paul Michali (pcm) 
Here are the calls in Neutron:

neutron/agent/l3_agent.py:raise SystemExit(msg)
neutron/agent/l3_agent.py:raise SystemExit(msg)
neutron/agent/l3_agent.py:raise SystemExit(msg)
neutron/agent/linux/dhcp.py:raise SystemExit(msg)
neutron/agent/linux/dhcp.py:raise SystemExit(msg)
neutron/db/migration/cli.py:raise SystemExit(_('You must provide a 
revision or relative delta'))
neutron/openstack/common/service.py:class SignalExit(SystemExit):
neutron/openstack/common/service.py:except SystemExit as exc:
neutron/openstack/common/service.py:except SystemExit as exc:
neutron/plugins/ibm/agent/sdnve_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ibm/agent/sdnve_neutron_agent.py:raise SystemExit(1)
neutron/plugins/ml2/managers.py:raise SystemExit(msg)
neutron/plugins/mlnx/agent/eswitch_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/mlnx/agent/utils.py:raise SystemExit(msg)
neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
neutron/plugins/mlnx/mlnx_plugin.py:raise SystemExit(1)
neutron/plugins/nec/nec_router.py:raise SystemExit(1)
neutron/plugins/nec/nec_router.py:raise SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/ofagent/agent/ofa_neutron_agent.py:raise 
SystemExit(1)
neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:raise 
SystemExit(1)
neutron/services/loadbalancer/agent/agent_manager.py:raise 
SystemExit(msg % driver)
neutron/services/loadbalancer/agent/agent_manager.py:raise 
SystemExit(msg % driver_name)
neutron/services/loadbalancer/plugin.py:raise SystemExit(msg)
neutron/services/metering/agents/metering_agent.py:raise 
SystemExit(_('A metering driver must be specified'))
neutron/services/metering/drivers/iptables/iptables_driver.py:raise 
SystemExit(_('An interface driver must be specified'))
neutron/services/service_base.py:raise SystemExit(msg)
neutron/services/vpn/device_drivers/cisco_ipsec.py:raise 
SystemExit(_('No Cisco CSR configurations found in: %s') %

bin/neutron-rootwrap-xen-dom0:sys.exit(RC_NOCOMMAND)
bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/neutron-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/neutron-rootwrap-xen-dom0:sys.exit(RC_UNAUTHORIZED)
bin/neutron-rootwrap-xen-dom0:sys.exit(RC_XENAPI_ERROR)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_NOCOMMAND)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_BADCONFIG)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_UNAUTHORIZED)
bin/quantum-rootwrap-xen-dom0:sys.exit(RC_XENAPI_ERROR)
neutron/agent/linux/daemon.py:sys.exit(1)
neutron/agent/linux/daemon.py:sys.exit(0)
neutron/agent/linux/daemon.py:sys.exit(1)
neutron/agent/linux/daemon.py:sys.exit(0)
neutron/agent/linux/daemon.py:sys.exit(1)
neutron/agent/linux/dhcp.py:sys.exit()
neutron/openstack/common/lockutils.py:sys.exit(main(sys.argv))
neutron/openstack/common/rpc/amqp.py:# just before doing a sys.exit(), 
so cleanup() only happens once and
neutron/openstack/common/service.py:sys.exit(1)
neutron/openstack/common/systemd.py:sys.exit(retval)
neutron/plugins/bigswitch/agent/restproxy_agent.py:sys.exit(0)
neutron/plugins/linuxbridge/agent/linuxbridge_neutron_agent.py:
sys.exit(1)
neutron/plugins/linuxbridge/agent/linuxbridge_neutron_agent.py:sys.exit(0)
neutron/plugins/linuxbridge/lb_neutron_plugin.py:sys.exit(1)
neutron/plugins/linuxbridge/lb_neutron_plugin.py:sys.exit(1)
neutron/plugins/ml2/drivers/type_vlan.py:sys.exit(1)
neutron/plugins/mlnx/agent/eswitch_neutron_agent.py:sys.exit(1)
neutron/plugins/mlnx/agent/eswitch_neutron_agent.py:sys.exit(1)
neutron/plugins/mlnx/agent/eswitch_neutron_agent.py:sys.exit(0)
neutron/plugins/mlnx/mlnx_plugin.py:sys.exit(1)
neutron/plugins/mlnx/mlnx_plugin.py:sys.exit(1)
neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:
sys.exit(1)
neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:sys.exit(1

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Doug Hellmann 
As Robert said, libraries should not be calling sys.exit() or raising
SystemExit directly, ever.

Throwing SystemExit from a library bypasses other exception handling
cleanup code higher in the stack that is unlikely to be looking for
fatal exceptions like SystemExit (because well-behaved libraries don't
throw those exceptions). Libraries should define meaningful
exceptions, subclassed from Exception, which the main application can
log before deciding whether to exit, retry, pick another driver, or
whatever.

On Fri, May 2, 2014 at 6:24 AM, Paul Michali (pcm)  wrote:
> On May 1, 2014, at 1:23 PM, Yuriy Taraday  wrote:
>
>>
>> Coming back to topic, I'd prefer using standard library call because it can 
>> be mocked for testing.
>
> Yeah that’s probably the open question I still have. Does the community 
> prefer raising a SystemError exception or use the sys.exit() call?
> Should we be consistent in our use?
>
> openstack@devstack-32:/opt/stack/neutron$ git grep sys.exit | wc -l
> 56
> openstack@devstack-32:/opt/stack/neutron$ git grep SystemExit | wc -l
> 57
>
>
> Regards,
>
> PCM (Paul Michali)
>
> MAIL …..…. p...@cisco.com
> IRC ……..… pcm_ (irc.freenode.com)
> TW ………... @pmichali
> GPG Key … 4525ECC253E31A83
> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>
>
>>
>> --
>>
>> Kind regards, Yuriy.
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Paul Michali (pcm) 
On May 1, 2014, at 1:23 PM, Yuriy Taraday  wrote:

> 
> Coming back to topic, I'd prefer using standard library call because it can 
> be mocked for testing.

Yeah that’s probably the open question I still have. Does the community prefer 
raising a SystemError exception or use the sys.exit() call?
Should we be consistent in our use?

openstack@devstack-32:/opt/stack/neutron$ git grep sys.exit | wc -l
56
openstack@devstack-32:/opt/stack/neutron$ git grep SystemExit | wc -l
57


Regards,

PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83


> 
> -- 
> 
> Kind regards, Yuriy.
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Paul Michali (pcm) 
It was in init code for a device driver, which (currently, as a short term PoC 
solution) reads a config file for settings of statically configured VPN 
devices. If there are no devices at all, it will report the issue and the agent 
will exit.  In the future, will be dynamically obtaining the device settings as 
needed, so this won’t be needed (and any failure will just fail the request and 
not exit the agent).

Regards,

PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83



On May 1, 2014, at 9:12 PM, Robert Collins  wrote:

> Raising SystemExit *or* calling sys.exit() are poor ideas: only outer
> layer code should do that. Plumbing should only be raising semantic,
> normally catchable exceptions IMO.
> 
> -Rob
> 
> On 2 May 2014 07:09, Kevin L. Mitchell  wrote:
>> On Thu, 2014-05-01 at 18:41 +, Paul Michali (pcm) wrote:
>>> So, I tried to reproduce, but I actually see the same results with
>>> both of these. However, they both show the issue I was hitting,
>>> namely, I got no information on where the failure was located:
>> 
>> So, this is pretty much by design.  A SystemExit extends BaseException,
>> rather than Exception.  The tests will catch Exception, but not
>> typically BaseException, as you generally want things like ^C to work
>> (raises a different BaseException).  So, your tests that might possibly
>> trigger a SystemExit (or sys.exit()) that you don't want to actually
>> exit from must either explicitly catch the SystemExit or—assuming the
>> code uses sys.exit()—must mock sys.exit() to inhibit the normal exit
>> behavior.
>> 
>> (Also, because SystemExit is the exception that is usually raised for a
>> normal exit condition, the traceback would not typically be printed, as
>> that could confuse users; no one expects a successfully executed script
>> to print a traceback, after all :)
>> --
>> Kevin L. Mitchell 
>> Rackspace
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> -- 
> Robert Collins 
> Distinguished Technologist
> HP Converged Cloud
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Paul Michali (pcm) 
Thanks for the exception explanation - now I understand better what is going on 
there. Yuriy’s mention about looking in .testrepository gave me the needed 
piece on how to find out where the failure occurred.

Regards,


PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83



On May 1, 2014, at 3:09 PM, Kevin L. Mitchell  
wrote:

> On Thu, 2014-05-01 at 18:41 +, Paul Michali (pcm) wrote:
>> So, I tried to reproduce, but I actually see the same results with
>> both of these. However, they both show the issue I was hitting,
>> namely, I got no information on where the failure was located:
> 
> So, this is pretty much by design.  A SystemExit extends BaseException,
> rather than Exception.  The tests will catch Exception, but not
> typically BaseException, as you generally want things like ^C to work
> (raises a different BaseException).  So, your tests that might possibly
> trigger a SystemExit (or sys.exit()) that you don't want to actually
> exit from must either explicitly catch the SystemExit or—assuming the
> code uses sys.exit()—must mock sys.exit() to inhibit the normal exit
> behavior.
> 
> (Also, because SystemExit is the exception that is usually raised for a
> normal exit condition, the traceback would not typically be printed, as
> that could confuse users; no one expects a successfully executed script
> to print a traceback, after all :)
> -- 
> Kevin L. Mitchell 
> Rackspace
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-02 Thread Paul Michali (pcm) 
Ah, I looked in .testrepository and there was information on the failure:

Content-Type: text/x-traceback;charset="utf8",language="python"
traceback
114
Traceback (most recent call last):
  File "neutron/tests/unit/pcm/test_pcm.py", line 34, in test_using_SystemExit
self.assertIsNone(using_SystemExit())
  File "neutron/tests/unit/pcm/test_pcm.py", line 25, in using_SystemExit
raise SystemExit("ouch")
SystemExit: ouch
0
]

Great! That was my big concern, I would run tox and it would show failures with 
just process-returncode and I couldn’t figure out where the problem occurred. I 
forgot about the .testrepository files. At least now I can figure out what 
causes those types of failures, which I do see occasionally.


Thanks!

PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83



On May 1, 2014, at 2:57 PM, Yuriy Taraday 
mailto:yorik@gmail.com>> wrote:

On Thu, May 1, 2014 at 10:41 PM, Paul Michali (pcm) 
mailto:p...@cisco.com>> wrote:
==
FAIL: process-returncode
tags: worker-1
--
Binary content:
  traceback (test/plain; charset="utf8")
==
FAIL: process-returncode
tags: worker-0
--
Binary content:
  traceback (test/plain; charset="utf8")

process-returncode failures means that child process (subunit one) exited with 
nonzero code.

It looks like there was some traceback, but it doesn’t show it. Any ideas how 
to get around this, as it makes it hard to troubleshoot these types of failures?

Somehow traceback got MIME type "test/plain". I guess, testr doesn't push this 
type of attachments to the screen. You can try to see what's there in 
.testrepository dir but I doubt there will be anything useful there.

I think this behavior is expected. Subunit process gets terminated because of 
uncaught SystemExit exception and testr reports that as an error.

--

Kind regards, Yuriy.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Robert Collins 
Raising SystemExit *or* calling sys.exit() are poor ideas: only outer
layer code should do that. Plumbing should only be raising semantic,
normally catchable exceptions IMO.

-Rob

On 2 May 2014 07:09, Kevin L. Mitchell  wrote:
> On Thu, 2014-05-01 at 18:41 +, Paul Michali (pcm) wrote:
>> So, I tried to reproduce, but I actually see the same results with
>> both of these. However, they both show the issue I was hitting,
>> namely, I got no information on where the failure was located:
>
> So, this is pretty much by design.  A SystemExit extends BaseException,
> rather than Exception.  The tests will catch Exception, but not
> typically BaseException, as you generally want things like ^C to work
> (raises a different BaseException).  So, your tests that might possibly
> trigger a SystemExit (or sys.exit()) that you don't want to actually
> exit from must either explicitly catch the SystemExit or—assuming the
> code uses sys.exit()—must mock sys.exit() to inhibit the normal exit
> behavior.
>
> (Also, because SystemExit is the exception that is usually raised for a
> normal exit condition, the traceback would not typically be printed, as
> that could confuse users; no one expects a successfully executed script
> to print a traceback, after all :)
> --
> Kevin L. Mitchell 
> Rackspace
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Kevin L. Mitchell 
On Thu, 2014-05-01 at 18:41 +, Paul Michali (pcm) wrote:
> So, I tried to reproduce, but I actually see the same results with
> both of these. However, they both show the issue I was hitting,
> namely, I got no information on where the failure was located:

So, this is pretty much by design.  A SystemExit extends BaseException,
rather than Exception.  The tests will catch Exception, but not
typically BaseException, as you generally want things like ^C to work
(raises a different BaseException).  So, your tests that might possibly
trigger a SystemExit (or sys.exit()) that you don't want to actually
exit from must either explicitly catch the SystemExit or—assuming the
code uses sys.exit()—must mock sys.exit() to inhibit the normal exit
behavior.

(Also, because SystemExit is the exception that is usually raised for a
normal exit condition, the traceback would not typically be printed, as
that could confuse users; no one expects a successfully executed script
to print a traceback, after all :)
-- 
Kevin L. Mitchell 
Rackspace


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Yuriy Taraday 
On Thu, May 1, 2014 at 10:41 PM, Paul Michali (pcm)  wrote:

> ==
> FAIL: process-returncode
> tags: worker-1
> --
> *Binary content:*
> *  traceback (test/plain; charset="utf8")*
> ==
> FAIL: process-returncode
> tags: worker-0
> --
> *Binary content:*
> *  traceback (test/plain; charset="utf8")*
>

process-returncode failures means that child process (subunit one) exited
with nonzero code.


> It looks like there was some traceback, but it doesn’t show it. Any ideas
> how to get around this, as it makes it hard to troubleshoot these types of
> failures?
>

Somehow traceback got MIME type "test/plain". I guess, testr doesn't push
this type of attachments to the screen. You can try to see what's there in
.testrepository dir but I doubt there will be anything useful there.

I think this behavior is expected. Subunit process gets terminated because
of uncaught SystemExit exception and testr reports that as an error.

-- 

Kind regards, Yuriy.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Paul Michali (pcm) 
So, I tried to reproduce, but I actually see the same results with both of 
these. However, they both show the issue I was hitting, namely, I got no 
information on where the failure was located:

root@devstack-32:/opt/stack/neutron# tox -e py27 -v -- 
neutron.tests.unit.pcm.test_pcm
using tox.ini: /opt/stack/neutron/tox.ini
using tox-1.6.1 from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc
py27 reusing: /opt/stack/neutron/.tox/py27
  /opt/stack/neutron$ /opt/stack/neutron/.tox/py27/bin/python 
/opt/stack/neutron/setup.py --name
py27 develop-inst-nodeps: /opt/stack/neutron
  /opt/stack/neutron$ /opt/stack/neutron/.tox/py27/bin/pip install -U -e 
/opt/stack/neutron --no-deps >/opt/stack/neutron/.tox/py27/log/py27-163.log
py27 runtests: commands[0] | python -m neutron.openstack.common.lockutils 
python setup.py testr --slowest --testr-args=neutron.tests.unit.pcm.test_pcm
  /opt/stack/neutron$ /opt/stack/neutron/.tox/py27/bin/python -m 
neutron.openstack.common.lockutils python setup.py testr --slowest 
--testr-args=neutron.tests.unit.pcm.test_pcm
running testr
running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_LOG_CAPTURE=1 
${PYTHON:-python} -m subunit.run discover -t ./ 
${OS_TEST_PATH:-./neutron/tests/unit} --list
running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_LOG_CAPTURE=1 
${PYTHON:-python} -m subunit.run discover -t ./ 
${OS_TEST_PATH:-./neutron/tests/unit}  --load-list /tmp/tmpkYugPE
running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_LOG_CAPTURE=1 
${PYTHON:-python} -m subunit.run discover -t ./ 
${OS_TEST_PATH:-./neutron/tests/unit}  --load-list /tmp/tmpPoDsN3
ouch
==
FAIL: process-returncode
tags: worker-1
--
Binary content:
  traceback (test/plain; charset="utf8")
==
FAIL: process-returncode
tags: worker-0
--
Binary content:
  traceback (test/plain; charset="utf8")
Ran 4 (+4) tests in 0.348s
FAILED (id=150, failures=2 (+2))
error: testr failed (1)
ERROR: InvocationError: '/opt/stack/neutron/.tox/py27/bin/python -m 
neutron.openstack.common.lockutils python setup.py testr --slowest 
--testr-args=neutron.tests.unit.pcm.test_pcm'
__
 summary 
___
ERROR:   py27: commands failed

It looks like there was some traceback, but it doesn’t show it. Any ideas how 
to get around this, as it makes it hard to troubleshoot these types of failures?

Here is the code:

# Copyright 2014 Cisco Systems, Inc.  All rights reserved.
#
#Licensed under the Apache License, Version 2.0 (the "License"); you may
#not use this file except in compliance with the License. You may obtain
#a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#Unless required by applicable law or agreed to in writing, software
#distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
#WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
#License for the specific language governing permissions and limitations
#under the License.
#
# @author: Paul Michali, Cisco Systems, Inc.

import sys

from neutron.tests import base

def using_sys_exit():
sys.exit(1)

def using_SystemExit():
raise SystemExit("ouch")


class TestSystemExit(base.BaseTestCase):

def test_using_sys_exit(self):
self.assertIsNone(using_sys_exit())

def test_using_SystemExit(self):
self.assertIsNone(using_SystemExit())


Regards,


PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83



On May 1, 2014, at 1:23 PM, Yuriy Taraday 
mailto:yorik@gmail.com>> wrote:

On Thu, May 1, 2014 at 8:17 PM, Salvatore Orlando 
mailto:sorla...@nicira.com>> wrote:
The patch you've been looking at just changes the way in which SystemExit is 
used, it does not replace it with sys.exit.
In my experience sys.exit was causing unit test threads to interrupt abruptly, 
whereas SystemExit was being caught by the test runner and handled.

According to https://docs.python.org/2.7/library/sys.html#sys.exit , 
sys.exit(n) is an equivalent for raise SystemExit(n), it can be confirmed in 
the source code here: 
http://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l206
If there's any difference in behavior it seems to be the problem of test 
runner. For example, it can mock sys.exit somehow.

I find therefore a bit strange that you're reporting what appears to be the 
opposite behaviour.

Maybe if you could share the code you're working on we

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Yuriy Taraday 
On Thu, May 1, 2014 at 8:17 PM, Salvatore Orlando wrote:

> The patch you've been looking at just changes the way in which SystemExit
> is used, it does not replace it with sys.exit.
> In my experience sys.exit was causing unit test threads to interrupt
> abruptly, whereas SystemExit was being caught by the test runner and
> handled.
>

According to https://docs.python.org/2.7/library/sys.html#sys.exit ,
sys.exit(n) is an equivalent for raise SystemExit(n), it can be confirmed
in the source code here:
http://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l206
If there's any difference in behavior it seems to be the problem of test
runner. For example, it can mock sys.exit somehow.

 I find therefore a bit strange that you're reporting what appears to be
> the opposite behaviour.
>
> Maybe if you could share the code you're working on we can have a look at
> it and see what's going on.
>

I'd suggest finding out what's the difference in both of your cases.

Coming back to topic, I'd prefer using standard library call because it can
be mocked for testing.

-- 

Kind regards, Yuriy.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-05-01 Thread Salvatore Orlando 
The patch you've been looking at just changes the way in which SystemExit
is used, it does not replace it with sys.exit.
In my experience sys.exit was causing unit test threads to interrupt
abruptly, whereas SystemExit was being caught by the test runner and
handled.
I find therefore a bit strange that you're reporting what appears to be the
opposite behaviour.

Maybe if you could share the code you're working on we can have a look at
it and see what's going on.

Salvatore


On 30 April 2014 21:08, Paul Michali (pcm)  wrote:

>  Hi,
>
>  In Neutron I see SystemExit() being raised in some cases. Is this
> preferred over calling sys.exit()?
>
>  I ask, because I recall having a TOX failure where all I was getting was
> the return code, with no traceback or indication at all of where the
> failure occurred. In that case, I changed from SystemExit() to sys.exit()
> and I then got the traceback and was able to see what was going wrong in
> the test case (it’s been weeks, so I don’t recall where this was at).
>
>  I see currently, there is some changes to use of SystemExit() being
> reviewed (https://review.openstack.org/91185), and it reminded me of the
> concern I had.
>
>  Can anyone enlighten me?
>
>
>  Thanks!
>
>  PCM (Paul Michali)
>
>  MAIL …..…. p...@cisco.com
> IRC ……..… pcm_ (irc.freenode.com)
> TW ………... @pmichali
> GPG Key … 4525ECC253E31A83
> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra][neutron]SystemExit() vs sys.exit()?

2014-04-30 Thread Paul Michali (pcm) 
Hi,

In Neutron I see SystemExit() being raised in some cases. Is this preferred 
over calling sys.exit()?

I ask, because I recall having a TOX failure where all I was getting was the 
return code, with no traceback or indication at all of where the failure 
occurred. In that case, I changed from SystemExit() to sys.exit() and I then 
got the traceback and was able to see what was going wrong in the test case 
(it’s been weeks, so I don’t recall where this was at).

I see currently, there is some changes to use of SystemExit() being reviewed 
(https://review.openstack.org/91185), and it reminded me of the concern I had.

Can anyone enlighten me?


Thanks!

PCM (Paul Michali)

MAIL …..…. p...@cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev