Hello,
I use backdoor of eventlet to enable gc.DEBUG_LEAK, and after wait a few
minutes, i can sure that there will some objects that can not be collected by
gc.collect in gc.garbage.
Those looks like this (catched in ceilometer-collector)
['_context_auth_token', 'auth_token', 'new_pass'],
(<cell at 0x363bc58: list object at 0x361c170>,
<cell at 0x363bda8: function object at 0x361a5f0>),
<function _fix_passwords at 0x361a5f0>,
<cell at 0x363bde0: list object at 0x363c680>,
<cell at 0x363bd70: function object at 0x361a668>,
['_context_auth_token', 'auth_token', 'new_pass'],
(<cell at 0x363bde0: list object at 0x363c680>,
<cell at 0x363bd70: function object at 0x361a668>),
<function _fix_passwords at 0x361a668>,
<cell at 0x363bf30: list object at 0x361b680>,
<cell at 0x363e168: function object at 0x361a758>,
['_context_auth_token', 'auth_token', 'new_pass'],
(<cell at 0x363bf30: list object at 0x361b680>,
<cell at 0x363e168: function object at 0x361a758>),
<function _fix_passwords at 0x361a758>,
<cell at 0x363e1a0: list object at 0x363cdd0>,
<cell at 0x363e130: function object at 0x361a7d0>,
and i suspect those code in oslo.messaging
def _safe_log(log_func, msg, msg_data):
"""Sanitizes the msg_data field before logging."""
SANITIZE = ['_context_auth_token', 'auth_token', 'new_pass']
def _fix_passwords(d):
"""Sanitizes the password fields in the dictionary."""
for k in d.iterkeys():
if k.lower().find('password') != -1:
d[k] = '<SANITIZED>'
elif k.lower() in SANITIZE:
d[k] = '<SANITIZED>'
elif isinstance(d[k], dict):
_fix_passwords(d[k])
return d
return log_func(msg, _fix_passwords(copy.deepcopy(msg_data)))
i can resolve this problem by add _fix_passwords = None before _safe_log
returns.
But i do not really understand why this problem happened, and in depth why the
gc can not collect those object. Although i can make those uncollectable
objects disappeared.
But this is not good enough, because if you do not understand it you will write
out some code like this in future, and then also has memory leak too.
So can some one helps me give some detailed on recursive closure used like the
code above, and on why gc can not collect them.
Thanks a lot lot ......
--
Best
Li Tianqing_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
