On Thu, Oct 26, 2017 at 10:23 PM, Matt Riedemann <mriede...@gmail.com> wrote:
> Nova has had this long-standing known performance issue if you're > filtering a large number of instances by IP. The instance IPs are stored in > a JSON blob in the database so we don't do filtering in SQL. We pull the > instances out of the database, deserialize the JSON and then apply a regex > filter match in the nova-api python code. > > At the Queens PTG we talked about possible ways to fix this and came up > with this nova spec: > > https://specs.openstack.org/openstack/nova-specs/specs/queen > s/approved/improve-filter-instances-by-ip-performance.html > > The idea is to have nova get ports from neutron and apply the IP filter in > neutron to whittle down the ports, then from that list of ports get the > instances to pull out of the nova database. > > One issue that has come up with this is neutron does not currently support > regex filters when listing ports. There is an RFE for adding that: > > https://bugs.launchpad.net/neutron/+bug/1718605 > > The proposed neutron implementation is to just do SQL LIKE substring > matching in the database. > > However, one issue that has come up is that the compute API accepts a > python regex filter and uses re.match(): > > https://github.com/openstack/nova/blob/16.0.0/nova/compute/api.py#L2469 > > At least one good thing about that is match() only matches from the > beginning of the string unlike search(). > > So for example I can filter on "192.16.*[1-5]$" if I wanted to, but that's > not going to work with just a LIKE substring filter in SQL. > > The question is, does anyone actually do more than basic substring > matching with the IP filter today? Because if we started using neutron, > that behavior would be broken. We've never actually documented the match > restrictions on the IP filter, but that's not a good reason to break it. > The use-case for us is that it helps us easily identify or find VMs which we get any abuse reports for (or anything we see malicious traffic going to/from). We usually search for an *exact* match of the IP address as we are simply trying to perform a lookup of instance ID based on the IP address. Regex matching isn't important in our case. > One option is to make this configurable such that deployments which rely > on the complicated pattern matching can just use the existing nova code > despite performance issues. However, that's not interoperable, I hate > config-driven API behavior, and it would mean maintaining two code paths in > nova, which is also terrible. > > I was trying to think of a way to determine if the IP filter passed to > nova is basic or a complicated pattern match and let us decide that way, > but I'm not sure if there are good ways to detect that - maybe by simply > looking for special characters like (, ), - and $? But then there is [] and > we have an IPv6 filter, so that gets messy too... > > For now I'd just like to know if people rely on the regex match or not. > Other ideas on how to handle this are appreciated. > > -- > > Thanks, > > Matt > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators