From: Morgan Fainberg [mailto:[email protected]] Sent: Thursday, November 3, 2016 4:31 PM To: OpenStack Development Mailing List (not for usage questions) <[email protected]> Subject: Re: [openstack-dev] [all][dev][python] constructing a deterministic representation of a python data structure On Thu, Nov 3, 2016 at 1:04 PM, Amrith Kumar <[email protected] <mailto:[email protected]> > wrote: Gordon, You can see a very quick-and-dirty prototype of the kind of thing I'm looking to do in Trove at https://gist.github.com/amrith/6a89ff478f81c2910e84325923eddebe Uncommenting line 51 would simulate a bad hash. I'd be happy to propose something similar in oslo.messaging if you think that would pass muster there. -amrith -----Original Message----- From: gordon chung [mailto:[email protected] <mailto:[email protected]> ] Sent: Thursday, November 3, 2016 3:09 PM To: [email protected] <mailto:[email protected]> Subject: Re: [openstack-dev] [all][dev][python] constructing a deterministic representation of a python data structure On 03/11/16 02:24 PM, Amrith Kumar wrote: > > So, just before calling call() or cast(), I could compute the hash and > stuff it into the dictionary that is being sent over, and I can do the > same on the receiving side. But since I cannot guarantee that the > representation on the receiving side is necessarily identical to the > representation on the sending side, I have issues computing the hash. > > based on description, you're trying to sign the messages? there was some effort done in oslo.messaging[1] we do something similar in Ceilometer to sign IPC messages[2]. it does add overhead though. [1] https://review.openstack.org/#/c/205330/ [2] https://github.com/openstack/ceilometer/blob/ffc9ee99c10ede988769907fdb0594a <https://github.com/openstack/ceilometer/blob/ffc9ee99c10ede988769907fdb0594a512c890cd/ceilometer/publisher/utils.py#L43-L58> 512c890cd/ceilometer/publisher/utils.py#L43-L58 cheers, -- gord __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe <http://[email protected]?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe <http://[email protected]?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I had to solve a similar issue for deterministic key generation in dogpile (key for memcache/etc) when memoizing methods/functions with kwargs. There are a couple issues you run into, default args are not represented in **kwargs, and non-positional args can come in any order. [Amrith Kumar] This, as it turns out, is a good thing. In the specific use case that I have, the sender of the message may be on one version and the recipient may be on a later version which defaults some parameters. Therefore intercepting and handing the arguments in a decorator is perfect for me because I don’t get to see the impact of the default arguments that are added by the new version of the receiver. Also, since this is used in the context of cast() and call() in oslo.messaging, all parameters are passed as kwargs and there are no positional arguments. Here, for example, is a call to _cast() for the resize_flavor() RPC call in the Trove Taskmanager. def resize_flavor(self, instance_id, old_flavor, new_flavor): … self._cast("resize_flavor", self.version_cap, instance_id=instance_id, old_flavor=self._transform_obj(old_flavor), new_flavor=self._transform_obj(new_flavor)) If you want an example of what we did to generate the cache-key programatically you can look here: https://bitbucket.org/zzzeek/dogpile.cache/src/669582c2e5bf12b1303f50c4b7ba3dad308eb1cc/dogpile/cache/util.py?at=master <https://bitbucket.org/zzzeek/dogpile.cache/src/669582c2e5bf12b1303f50c4b7ba3dad308eb1cc/dogpile/cache/util.py?at=master&fileviewer=file-view-default#util.py-67:118> &fileviewer=file-view-default#util.py-67:118 [Amrith Kumar] Thanks, this is awesome! But, maybe a little more than I need. I’ll snag the guts of generate_key() and that should give me what I need (I think). You don't need all the namespace and probably not fn/module info, but this can look at the call and handle / ensure defaults also match (or be used to extract default kwargs if needed) for passing down to RPC. --Morgan
smime.p7s
Description: S/MIME cryptographic signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
