[389-devel] Re: Please review: DSLdapObject compare function and user compare tests

2017-04-09 Thread William Brown
On Fri, 2017-04-07 at 06:10 +, Ankit Yadav wrote:
> Hello Everyone,
> 
> I have changed the compare function, took help from Entry.__eq__ function. 
> I have also updated the user_compare_test.py, Now it includes 4 assertions. 
> Details can be found in lib389/tests/idm/user_compare_test.py
> Also removed the 'DeepDiff' dependency.
> Please review this commit: 
> https://pagure.io/fork/ankity10/lib389/c/762ef9054f01837a33219aa2412d428f5d783c52?branch=ticket-1-config-compare
> 
> Thanks

Hi there,

This is a big improvement! Thanks for taking all my feedback.

When we supply patches for review, we generally like to supply them as
1, so we can see the whole change in one, rather than over two commits.
I've just updated our contributing guide, so it may be worth you reading
over it.

http://www.port389.org/docs/389ds/contributing.html

It has a section on how to squash commits together in git, as well as
our commit message formats we prefer.

21 + # check if RDN is same
22 + if obj1.rdn != obj2.rdn:
23 + return False

I think the question is if we want to check if two objects in different
subtrees have the same attributes, or if we want *true* object equality.

*true* object equality, you want to check nsUniqueId first, because
that's always going to be different between everything.

But if we want to just compare arbitrary objects, I think the rdn check
is better, and we need to ignore nsUniqueId.

For this purpose perhaps we want the "loose" equality, just checking if
attributes are the same. IE if I had:

cn=user1,ou=a
cn=user1,ou=b

They should compare the same, even though their nsUniqueId differs. 

For now I think this code is okay, but we should leave some comments
around this discussion there, and certainly document in the docstring
that this is a loose equality, not a strict "this object *is* this other
object". 


46   # removing _compate_exclude attrs from all attrs
47   for key in self._compare_exclude:
48 - attrs_dict.pop(key.lower(), None)
49 + del attrs_dict[key.lower()]

Rather than deleting these, why not use a set and do:

compare_attrs = set(attrs_dict.keys) - set(self._compare_exclude)

The issue with python is that sometimes delete and operations like that
are destructive and have weird side effects, so it's better to take a
functional approach and make a new set of attributes you want, rather
than modifying a set you already have. 

48   testuser1.delete()
49   testuser2.delete()

The test teardown deletes the database, so you don't need to delete the
users at the end :) 


If you squash this patch and the last one together now, and send again
for a quick check, I think you would be pretty close to having a working
object compare to merge :) 

Thanks heaps! 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org


[389-devel] Re: Please review: DSLdapObject compare function and user compare tests

2017-04-04 Thread William Brown
On Tue, 2017-04-04 at 23:02 +, Ankit Yadav wrote:
> Hello Everyone, 
> 
> I was working on issue #1 lib389 cn=config comparison, I have done some work 
> in that direction. I have added a compare function on DSLdapObject and have 
> written a test for comparing user object. Currently this test is not 
> complete, I am just printing the attributes values to verify the working of 
> comparison function.
> But comparison function is working, so let me know your feedback and how we 
> can proceed further with this?
> 
> Link to commit: 
> https://pagure.io/fork/ankity10/lib389/c/44b02ba4ddb54be12041052b5b75f17b9eaf6b8f?branch=ticket-1-config-compare
> 

Hey there,

Looks like a great start. A lot of comments though, but thanks for all
your effort and I hope these comments help you improve your code a lot.



27 + obj1_attrs_json = obj1.get_compare_attrs()
28 + obj2_attrs_json = obj2.get_compare_attrs()
29 + return DeepDiff(obj1_attrs_json, obj2_attrs_json)

I don't think you should convert these to json. You should be comparing
the dictionaries from the Entry directly. The JSON is only there for our
future admin server to represent the entries in a website - it should
never be used internally for operations in this library. JSON is also
non-deterministic garbage IMO, so lets avoid it "unless we need it". 

You already have:

42 + attrs_entry = self._instance.getEntry(self._dn,
ldap.SCOPE_BASE, "(objectclass=*)", ["*", "+"])42 +
attrs_entry = self._instance.getEntry(self._dn, ldap.SCOPE_BASE,
"(objectclass=*)", ["*", "+"])

So you should probably return either the entry data dict from here
instead, then compare on that an example of this here:

As "inspiration", look at:

https://pagure.io/lib389/blob/master/f/lib389/_entry.py#_91

It's worth exploring and reading the code for what you use to find gems
like this you may be able to copy or reuse. I think in this case, it's
better to make a new compare function based on this code rather than
trying to hack entry.__eq__ to work in this case, if that makes sense. 


 5 + from deepdiff import DeepDiff

Please don't add library dependencies, unless it exists as a package on
RHEL7, we can not support extra dependencies in our code. 

"""
yum search deepdiff
...
Warning: No matches found for: deepdiff
No matches found
"""


With these lines:

12   class DSLdapObject(DSLogging):
13 + compare_exclude = []

4   class UserAccount(DSLdapObject):
5 + compare_exclude = ['nsuniqueid']
6 + 

Make sure you put these attributes in __init__() of the objects as:

def __init__()
self._compare_exclude =  

The reason for this is that the top level attribute is global to all instances. 
So if any subclass overrides
the attribute, it causes the override of the parents attribute for ALL 
subclasses: this causes horrible
horrible issues that are near impossible to resolve. You need to make these 
self. to indicate
they are on the single instance only, which makes it work correctly.

As well, because these are internal, prefix the variable with an _ to say it's 
private. Therefore:

def __init__()
self._compare_exclude =  

Remember, especially on the UserAccount, to put the self._compare_exclude 
*after* the call
to super(UserAccount).__init__! , else the super call will just flush it back 
to the DSLdapObject
value. 



With your test case, you don't need:

57 + assert (len(users.list()) == 1)

Because if the user fails to create, we throw an exception from users.create, 
so the test
will fail at that stage. :) 


74 + print("testuser1 all attributes: ")

try to use 

log.[info,debug,error]() instead. This way based on the
DEBBUGING environment variable we see more or less output. We shouldn't
have "print" anywhere in the code base. 


88 + # We want to capture std out while running this module through
pytest, hence adding explicit test failure

Run py.test with "-s" to show the stdout during the test instead :) 


Remember, to test users that are the same and also different! Also, a
user compared to itself must be the same also, and a user compared to a
different object class should also be false! IE compare a user to a
group :) 


I hope that this advice helps you: I know it's a lot to change, but
thanks for your work! 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane



signature.asc
Description: This is a digitally signed message part
___
389-devel mailing list -- 389-devel@lists.fedoraproject.org
To unsubscribe send an email to 389-devel-le...@lists.fedoraproject.org