Re: [389-devel] [lib389] Deref control advice needed
> > > > Old !! but it worked like a charm for me. I just had to do this modif > > because of change in python-ldap IIRC > > OK. But I would rather use William's version which is based on pyasn1 - > it hurts my brain to hand code BER . . . https://fedorahosted.org/389/ticket/48262 Please take a look and review. -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] [lib389] Deref control advice needed
On 09/02/2015 10:35 AM, thierry bordaz wrote: On 08/27/2015 02:31 AM, Rich Megginson wrote: On 08/26/2015 03:28 AM, William Brown wrote: In relation to ticket 47757, I have started work on a deref control for Noriko. The idea is to get it working in lib389, then get it upstreamed into pyldap. At this point it's all done, except that the actual request control doesn't appear to work. Could one of the lib389 / ldap python experts cast their eye over this and let me know where I've gone wrong? I have improved this, but am having issues with the asn1spec for ber decoding. I have attached the updated patch, but specifically the issue is in _controls.py I would appreciate if anyone could take a look at this, and let me know if there is something I have missed. Not sure, but here is some code I did without using pyasn: https://github.com/richm/scripts/blob/master/derefctrl.py This is quite old by now, and is probably bit rotted with respect to python-ldap and python3. Old !! but it worked like a charm for me. I just had to do this modif because of change in python-ldap IIRC OK. But I would rather use William's version which is based on pyasn1 - it hurts my brain to hand code BER . . . diff derefctrl.py /tmp/derefctrl_orig.py 0a1 > 151,152c152 < self.criticality,self.derefspeclist,self.entry = criticality,derefspeclist or [],None < #LDAPControl.__init__(self,DerefCtrl.controlType,criticality,derefspeclist) --- > LDAPControl.__init__(self,DerefCtrl.controlType,criticality,derefspeclist) 154c154 < def encodeControlValue(self): --- > def encodeControlValue(self,value): 156c156 < for (derefattr,attrs) in self.derefspeclist: --- > for (derefattr,attrs) in value: """ controlValue ::= SEQUENCE OF derefRes DerefRes DerefRes ::= SEQUENCE { derefAttr AttributeDescription, derefValLDAPDN, attrVals[0] PartialAttributeList OPTIONAL } PartialAttributeList ::= SEQUENCE OF partialAttribute PartialAttribute """ class DerefRes(univ.Sequence): componentType = namedtype.NamedTypes( namedtype.NamedType('derefAttr', AttributeDescription()), namedtype.NamedType('derefVal', LDAPDN()), namedtype.OptionalNamedType('attrVals', PartialAttributeList()), ) class DerefResultControlValue(univ.SequenceOf): componentType = DerefRes() def decodeControlValue(self,encodedControlValue): self.entry = {} #decodedValue,_ = decoder.decode(encodedControlValue,asn1Spec=DerefResultControlValue()) # Gets the error: TagSet(Tag(tagClass=0, tagFormat=32, tagId=16), Tag(tagClass=128, tagFormat=32, tagId=0)) not in asn1Spec: {TagSet(Tag(tagClass=0, tagFormat=32, tagId=16)): PartialAttributeList()}/{} decodedValue,_ = decoder.decode(encodedControlValue) print(decodedValue.prettyPrint()) # Pretty print yields #Sequence: <-- Sequence of # =Sequence: <-- derefRes # =uniqueMember <-- derefAttr # =uid=test,dc=example,dc=com <-- derefVal # =Sequence: <-- attrVals # =uid # =Set: #=test # For now, while asn1spec is sad, we'll just rely on it being well formed # However, this isn't good, as without the asn1spec, we seem to actually be dropping values for result in decodedValue: derefAttr, derefVal, _ = result self.entry[str(derefAttr)] = str(derefVal) -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] [lib389] Deref control advice needed
On 08/27/2015 02:31 AM, Rich Megginson wrote: On 08/26/2015 03:28 AM, William Brown wrote: In relation to ticket 47757, I have started work on a deref control for Noriko. The idea is to get it working in lib389, then get it upstreamed into pyldap. At this point it's all done, except that the actual request control doesn't appear to work. Could one of the lib389 / ldap python experts cast their eye over this and let me know where I've gone wrong? I have improved this, but am having issues with the asn1spec for ber decoding. I have attached the updated patch, but specifically the issue is in _controls.py I would appreciate if anyone could take a look at this, and let me know if there is something I have missed. Not sure, but here is some code I did without using pyasn: https://github.com/richm/scripts/blob/master/derefctrl.py This is quite old by now, and is probably bit rotted with respect to python-ldap and python3. Old !! but it worked like a charm for me. I just had to do this modif because of change in python-ldap IIRC diff derefctrl.py /tmp/derefctrl_orig.py 0a1 > 151,152c152 < self.criticality,self.derefspeclist,self.entry = criticality,derefspeclist or [],None < #LDAPControl.__init__(self,DerefCtrl.controlType,criticality,derefspeclist) --- > LDAPControl.__init__(self,DerefCtrl.controlType,criticality,derefspeclist) 154c154 < def encodeControlValue(self): --- > def encodeControlValue(self,value): 156c156 < for (derefattr,attrs) in self.derefspeclist: --- > for (derefattr,attrs) in value: """ controlValue ::= SEQUENCE OF derefRes DerefRes DerefRes ::= SEQUENCE { derefAttr AttributeDescription, derefValLDAPDN, attrVals[0] PartialAttributeList OPTIONAL } PartialAttributeList ::= SEQUENCE OF partialAttribute PartialAttribute """ class DerefRes(univ.Sequence): componentType = namedtype.NamedTypes( namedtype.NamedType('derefAttr', AttributeDescription()), namedtype.NamedType('derefVal', LDAPDN()), namedtype.OptionalNamedType('attrVals', PartialAttributeList()), ) class DerefResultControlValue(univ.SequenceOf): componentType = DerefRes() def decodeControlValue(self,encodedControlValue): self.entry = {} #decodedValue,_ = decoder.decode(encodedControlValue,asn1Spec=DerefResultControlValue()) # Gets the error: TagSet(Tag(tagClass=0, tagFormat=32, tagId=16), Tag(tagClass=128, tagFormat=32, tagId=0)) not in asn1Spec: {TagSet(Tag(tagClass=0, tagFormat=32, tagId=16)): PartialAttributeList()}/{} decodedValue,_ = decoder.decode(encodedControlValue) print(decodedValue.prettyPrint()) # Pretty print yields #Sequence: <-- Sequence of # =Sequence: <-- derefRes # =uniqueMember <-- derefAttr # =uid=test,dc=example,dc=com <-- derefVal # =Sequence: <-- attrVals # =uid # =Set: #=test # For now, while asn1spec is sad, we'll just rely on it being well formed # However, this isn't good, as without the asn1spec, we seem to actually be dropping values for result in decodedValue: derefAttr, derefVal, _ = result self.entry[str(derefAttr)] = str(derefVal) -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] [lib389] Deref control advice needed
> > > > I would appreciate if anyone could take a look at this, and let me > > know if there > > is something I have missed. > > Not sure, but here is some code I did without using pyasn: > https://github.com/richm/scripts/blob/master/derefctrl.py > This is quite old by now, and is probably bit rotted with respect to > python-ldap and python3. Yep. It appears to do exactly the same as my control, just that I'm using pyasn. Your structures look really similar to what I have defined, so that is a good start on my part. Thanks for the interesting read, very interesting to see how you do the hand encoding. Sincerely, -- William -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] [lib389] Deref control advice needed
On 08/26/2015 03:28 AM, William Brown wrote: In relation to ticket 47757, I have started work on a deref control for Noriko. The idea is to get it working in lib389, then get it upstreamed into pyldap. At this point it's all done, except that the actual request control doesn't appear to work. Could one of the lib389 / ldap python experts cast their eye over this and let me know where I've gone wrong? I have improved this, but am having issues with the asn1spec for ber decoding. I have attached the updated patch, but specifically the issue is in _controls.py I would appreciate if anyone could take a look at this, and let me know if there is something I have missed. Not sure, but here is some code I did without using pyasn: https://github.com/richm/scripts/blob/master/derefctrl.py This is quite old by now, and is probably bit rotted with respect to python-ldap and python3. """ controlValue ::= SEQUENCE OF derefRes DerefRes DerefRes ::= SEQUENCE { derefAttr AttributeDescription, derefValLDAPDN, attrVals[0] PartialAttributeList OPTIONAL } PartialAttributeList ::= SEQUENCE OF partialAttribute PartialAttribute """ class DerefRes(univ.Sequence): componentType = namedtype.NamedTypes( namedtype.NamedType('derefAttr', AttributeDescription()), namedtype.NamedType('derefVal', LDAPDN()), namedtype.OptionalNamedType('attrVals', PartialAttributeList()), ) class DerefResultControlValue(univ.SequenceOf): componentType = DerefRes() def decodeControlValue(self,encodedControlValue): self.entry = {} #decodedValue,_ = decoder.decode(encodedControlValue,asn1Spec=DerefResultControlValue()) # Gets the error: TagSet(Tag(tagClass=0, tagFormat=32, tagId=16), Tag(tagClass=128, tagFormat=32, tagId=0)) not in asn1Spec: {TagSet(Tag(tagClass=0, tagFormat=32, tagId=16)): PartialAttributeList()}/{} decodedValue,_ = decoder.decode(encodedControlValue) print(decodedValue.prettyPrint()) # Pretty print yields #Sequence: <-- Sequence of # =Sequence: <-- derefRes # =uniqueMember <-- derefAttr # =uid=test,dc=example,dc=com <-- derefVal # =Sequence: <-- attrVals # =uid # =Set: #=test # For now, while asn1spec is sad, we'll just rely on it being well formed # However, this isn't good, as without the asn1spec, we seem to actually be dropping values for result in decodedValue: derefAttr, derefVal, _ = result self.entry[str(derefAttr)] = str(derefVal) -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel
Re: [389-devel] [lib389] Deref control advice needed
> In relation to ticket 47757, I have started work on a deref control for > Noriko. > The idea is to get it working in lib389, then get it upstreamed into pyldap. > > At this point it's all done, except that the actual request control doesn't > appear to work. Could one of the lib389 / ldap python experts cast their eye > over this and let me know where I've gone wrong? I have improved this, but am having issues with the asn1spec for ber decoding. I have attached the updated patch, but specifically the issue is in _controls.py I would appreciate if anyone could take a look at this, and let me know if there is something I have missed. """ controlValue ::= SEQUENCE OF derefRes DerefRes DerefRes ::= SEQUENCE { derefAttr AttributeDescription, derefValLDAPDN, attrVals[0] PartialAttributeList OPTIONAL } PartialAttributeList ::= SEQUENCE OF partialAttribute PartialAttribute """ class DerefRes(univ.Sequence): componentType = namedtype.NamedTypes( namedtype.NamedType('derefAttr', AttributeDescription()), namedtype.NamedType('derefVal', LDAPDN()), namedtype.OptionalNamedType('attrVals', PartialAttributeList()), ) class DerefResultControlValue(univ.SequenceOf): componentType = DerefRes() def decodeControlValue(self,encodedControlValue): self.entry = {} #decodedValue,_ = decoder.decode(encodedControlValue,asn1Spec=DerefResultControlValue()) # Gets the error: TagSet(Tag(tagClass=0, tagFormat=32, tagId=16), Tag(tagClass=128, tagFormat=32, tagId=0)) not in asn1Spec: {TagSet(Tag(tagClass=0, tagFormat=32, tagId=16)): PartialAttributeList()}/{} decodedValue,_ = decoder.decode(encodedControlValue) print(decodedValue.prettyPrint()) # Pretty print yields #Sequence: <-- Sequence of # =Sequence: <-- derefRes # =uniqueMember <-- derefAttr # =uid=test,dc=example,dc=com <-- derefVal # =Sequence: <-- attrVals # =uid # =Set: #=test # For now, while asn1spec is sad, we'll just rely on it being well formed # However, this isn't good, as without the asn1spec, we seem to actually be dropping values for result in decodedValue: derefAttr, derefVal, _ = result self.entry[str(derefAttr)] = str(derefVal) From 22f1912f95e2188743525dba876cd122db27bb13 Mon Sep 17 00:00:00 2001 From: William Brown Date: Tue, 25 Aug 2015 15:38:11 +0930 Subject: [PATCH] Add dereference request control to lib389 for testing. --- lib389/__init__.py| 43 ++ lib389/_constants.py | 6 +++ lib389/_controls.py | 110 ++ tests/dereference_test.py | 81 ++ 4 files changed, 240 insertions(+) create mode 100644 lib389/_controls.py create mode 100644 tests/dereference_test.py diff --git a/lib389/__init__.py b/lib389/__init__.py index 8dfe28f..201bf32 100644 --- a/lib389/__init__.py +++ b/lib389/__init__.py @@ -73,6 +73,7 @@ MAJOR, MINOR, _, _, _ = sys.version_info if MAJOR >= 3 or (MAJOR == 2 and MINOR >= 7): from ldap.controls.simple import GetEffectiveRightsControl +from lib389._controls import DereferenceControl RE_DBMONATTR = re.compile(r'^([a-zA-Z]+)-([1-9][0-9]*)$') RE_DBMONATTRSUN = re.compile(r'^([a-zA-Z]+)-([a-zA-Z]+)$') @@ -2401,6 +2402,48 @@ class DirSrv(SimpleLDAPObject): self.set_option(ldap.OPT_SERVER_CONTROLS, []) return ldap_result +# Is there a better name for this function? +def dereference(self, deref, base=DEFAULT_SUFFIX, scope=ldap.SCOPE_SUBTREE, *args, **kwargs): +""" +Perform a search which dereferences values from attributes such as member +or unique member. +For arguments to this function, please see LDAPObject.search_s. For example: + +@param deref - Dereference query +@param base - Base DN of the suffix to check +@param scope - search scope +@param args - +@param kwargs - +@return - ldap result + +LDAPObject.search_s(base, scope[, filterstr='(objectClass=*)'[, attrlist=None[, attrsonly=0]]]) -> list|None + +A deref query is of the format: + +":,..." + +"uniqueMember:dn,objectClass" + +This will return the dn's and objectClasses of the dereferenced members of the group. +""" +if not (MAJOR >= 3 or (MAJOR == 2 and MINOR >= 7)): +raise Exception("UNSUPPORTED EXTENDED OPERATION ON THIS VERSION OF PYTHON") +ldap_result = None +# This may not be thread safe. Is there a better way to do this? +try: +drc = DereferenceControl(True, deref=deref.encode('UTF-8')) +sctrl = [drc] +self.set_option(ldap.OPT_SERVER_CONTROLS, sctrl) + +#ldap_result = self.search_s(base, scope, *args, **kwargs) +
[389-devel] [lib389] Deref control advice needed
Hi, In relation to ticket 47757, I have started work on a deref control for Noriko. The idea is to get it working in lib389, then get it upstreamed into pyldap. At this point it's all done, except that the actual request control doesn't appear to work. Could one of the lib389 / ldap python experts cast their eye over this and let me know where I've gone wrong? For one, I don't think that the DerferenceControl decodeControlValue function is ever called, as I'm never seeing the encodedControlValue printed in my logs. Second, the results I get from result3 are: [('cn=testgroup,dc=example,dc=com', {'objectClass': ['top', 'extensibleobject'], 'uniqueMember': ['uid=test,dc=example,dc=com'], 'cn': ['testgroup']})] [] Which again, doesn't seem correct, as there should be a result from the control. Additionally, any tips on how to make the code nicer would be appreciated. I've attached the complete patch, with a unit test to trigger the search, but the request control looks like: """ controlValue ::= SEQUENCE OF derefSpec DerefSpec DerefSpec ::= SEQUENCE { derefAttr attributeDescription,; with DN syntax attributes AttributeList } AttributeList ::= SEQUENCE OF attr AttributeDescription """ class AttributeList(univ.SequenceOf): componentType = AttributeDescription() class DerefSpec(univ.Sequence): componentType = namedtype.NamedTypes( namedtype.NamedType('derefAttr', AttributeDescription()), namedtype.NamedType('attributes', AttributeList()), ) class ControlValue(univ.SequenceOf): componentType = DerefSpec() class DereferenceControl(LDAPControl): """ Dereference Control """ def __init__(self, criticality, deref): LDAPControl.__init__(self, CONTROL_DEREF, criticality) self.deref = deref def encodeControlValue(self): # How does -E ask for many values? derefAttr, attributes = self.deref.split(':') attributes = attributes.split(',') al = AttributeList() i = 0 while len(attributes) > 0: al.setComponentByPosition(i, attributes.pop()) i += 1 ds = DerefSpec() ds.setComponentByName('derefAttr', derefAttr) ds.setComponentByName('attributes', al) cv = ControlValue() cv.setComponentByPosition(0, ds) print(cv.prettyPrint()) return encoder.encode(cv) def decodeControlValue(self,encodedControlValue): print(encodedControlValue) From eb3dede21afcfb930d9c45ee0d83d309683f02ce Mon Sep 17 00:00:00 2001 From: William Brown Date: Tue, 25 Aug 2015 15:38:11 +0930 Subject: [PATCH] Add dereference request control to lib389 for testing. --- lib389/__init__.py| 99 +++ lib389/_constants.py | 6 +++ tests/dereference_test.py | 75 +++ 3 files changed, 180 insertions(+) create mode 100644 tests/dereference_test.py diff --git a/lib389/__init__.py b/lib389/__init__.py index 8dfe28f..1f3e6a4 100644 --- a/lib389/__init__.py +++ b/lib389/__init__.py @@ -73,6 +73,65 @@ MAJOR, MINOR, _, _, _ = sys.version_info if MAJOR >= 3 or (MAJOR == 2 and MINOR >= 7): from ldap.controls.simple import GetEffectiveRightsControl +from ldap.controls import LDAPControl + +from pyasn1.type import namedtype,univ +from pyasn1.codec.ber import encoder,decoder +from pyasn1_modules.rfc2251 import AttributeDescription + +# Could use AttributeDescriptionList + +""" + controlValue ::= SEQUENCE OF derefSpec DerefSpec + + DerefSpec ::= SEQUENCE { + derefAttr attributeDescription,; with DN syntax + attributes AttributeList } + + AttributeList ::= SEQUENCE OF attr AttributeDescription +""" +class AttributeList(univ.SequenceOf): +componentType = AttributeDescription() + +class DerefSpec(univ.Sequence): +componentType = namedtype.NamedTypes( +namedtype.NamedType('derefAttr', AttributeDescription()), +namedtype.NamedType('attributes', AttributeList()), +) + +class ControlValue(univ.SequenceOf): +componentType = DerefSpec() + +class DereferenceControl(LDAPControl): +""" +Dereference Control +""" + +def __init__(self, criticality, deref): +LDAPControl.__init__(self, CONTROL_DEREF, criticality) +self.deref = deref + +def encodeControlValue(self): +# How does -E ask for many values? +derefAttr, attributes = self.deref.split(':') +attributes = attributes.split(',') +al = AttributeList() +i = 0 +while len(attributes) > 0: +al.setComponentByPosition(i, attributes.pop()) +i += 1