On Tue, Nov 21, 2017 at 04:41:13PM +0000, Mooney, Sean K wrote: > > > > -----Original Message----- > > From: Jeremy Stanley [mailto:fu...@yuggoth.org] > > Sent: Tuesday, November 21, 2017 3:05 PM > > To: OpenStack Development Mailing List (not for usage questions) > > <openstack-dev@lists.openstack.org> > > Subject: Re: [openstack-dev] Removing internet access from unit test > > gates > > > > On 2017-11-21 09:28:20 +0100 (+0100), Thomas Goirand wrote: > > [...] > > > The only way that I see going forward, is having internet access > > > removed from unit tests in the gate, or probably just the above > > > variables set. > > [...] > > > > Historically, our projects hadn't done a great job of relegating their > > "unit test" jobs to only run unit tests, and had a number of what would > > be commonly considered functional tests mixed in. This has improved in > > recent years as many of those projects have created separate functional > > test jobs and are able to simplify their unit test jobs accordingly, so > > this may be more feasible now than it was in the past. > > > > Removing network access from the machines running these jobs won't > > work, of course, because our job scheduling and execution service needs > > to reach them over the Internet to start jobs, monitor progress and > > collect results. As you noted, faking Python out with envvars pointing > > it at nonexistent HTTP proxies might help at least where tests attempt > > to make HTTP(S) connections to remote systems. > > The Web is not all there is to the Internet however, so this wouldn't > > do much to prevent use of remote DNS, NTP, SMTP or other > > non-HTTP(S) protocols. > > > > The biggest wrinkle I see in your "proxy" idea is that most of our > > Python-based projects run their unit tests with tox, and it will use > > pip to install project and job dependencies via HTTPS prior to starting > > the test runner. As such, any proxy envvar setting would need to happen > > within the scope of tox itself so that it will be able to set up the > > virtualenv prior to configuring the proxy vars for the ensuing tests. > > It might be easiest for you to work out the tox.ini modification on one > > project (it'll be self-testing at least) and then once the pilot can be > > shown working the conversation with the community becomes a little > > easier. > [Mooney, Sean K] I may be over simplifying here but our unit tests are still > executed by > Zuul in vms provided by nodepool. Could we simply take advantage of openstack > and > use security groups to to block egress traffic from the vm except that > required to upload the logs? > e.g. don't mess with tox or proxyies within the vms and insteand do this > externally via neutron. > This would require the cloud provider to expose neutron however which may be > an issue for Rackspace but > It its only for unit test which are relatively short lived vs tempest jobs > perhaps the other providers would > Still have enough capacity? > > -- > > Jeremy Stanley > I don't think we'd need to use security groups, we could just setup a local firewall ruleset to do this on the node if we wanted.
-Paul __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev