Re: Rdn and Ava cleanup
On 2/24/11 2:50 AM, Alex Karasulu wrote: On Wed, Feb 23, 2011 at 8:51 PM, Stefan Seelmannseelm...@apache.org wrote: It made me think that maybe using a hashed index for Rdn is probably a better idea, because then we won't need this comparison to be done (the equals method would be enough) and also because it would be faster (finding an element in a Hash table is an O(1) operation - at least, theorically - when looking in a BTree is an O(log2(N))) I think in the current state the RDN index doesn't need to be ordered data structure, a hash table is sufficient. However we still need to cleanup the backends. One idea I had is to get rid of the onlevel and sublevel index and use the RDN index instead. I did that in the HBase prototype where the onelevel and sublevel index are no physical indexes but backed by the RDN index table. But if we do that then I think the RDN index needs to be a ordered data structure to be able to traverse the index using a cursor. Yes we need an ordered index and hence we need to be able to compare two RDN values to sort. we can use the Rdn Hashcode to do that, no ? I never considered making the onelevel and sublevel indices backed by the RDN index. It makes sense to do this and would improve performance because there's less house keeping to do and less IO on separate indices. Interesting. To be investigated, sure. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Rdn and Ava cleanup
On 2/24/11 3:01 AM, Alex Karasulu wrote: On Wed, Feb 23, 2011 at 12:14 PM, Emmanuel Lecharnyelecha...@gmail.com wrote: Hi guys, those last 2 days I did some cleanup in the Rdn and Ava classes, in th espirit of Dn cleanup. Here is a summary of what has been done, roughly : - injection of the SchemaManager in both classes' constructors - removing of the compareTo method - made some methods private - made the classes final - removed the Comparable interface from the equation One important modification is the last one : it makes no sense to have a Rdn be comparable. First, how do we compare a cn and a jpegPhoto ? Second, how do we compare two RDN which attributeType does not have an equality matching rule ? Of course, this has an impact in the way the backend works, as the Rdn index needs to be able to do ordered comparison between Rdn, as this index is a BTree. What I did is that I replaced the Rdn.compareTo(Rdn) method by a direct String comparison in Jdbm between the Rdn's normalized name. That does work. It made me think that maybe using a hashed index for Rdn is probably a better idea, because then we won't need this comparison to be done (the equals method would be enough) and also because it would be faster (finding an element in a Hash table is an O(1) operation - at least, theorically - when looking in a BTree is an O(log2(N))) thoughts on this last point? I just replied to Stefan's post first but I thought about this after my reply. Here's what came to mind: (1) Let's presume we back the one and sublevel indices with the RDN index. (2) Do we need to sort either of these three indices when they are coupled this way or all independent? We don't need to really sort them, we just need to be able to walk the index producing candidates for one level and sublevel relationships. We could probably still do this and get away without using a sorted B+Tree. The thing is we cannot apply certain assertions any longer with a hash like using the=.=, and substring assertions. DN syntax and the ATs that use it does not provide a Substring or Ordering MR. Does not make a lot of sense, IMO (comparing DN for equality is ok, trying to order them is just a vain thing) But this is not something performed on these indices anyway. You do these with respect to an attribute type which that might be indexed and do not use these operators on one level, or sub level inquiries. So yeah I think this can be done and may have big performance benefits. But it's a big shift. This is an interesting side effect of my question :) I haven't expected such a wide modification proposal when I asked about using Hash instead of BTree, but this is were it's start to become interesting ! Definitively to be investigated, and hopefully Stefan has already started to dig in this direction with his Hashbase implementation. Fun project too. Sure is ! -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: [LDAP] Odd Protocol Codec behaviors noticed
On 2/24/11 3:24 AM, Alex Karasulu wrote: Here's more data that might shed light. Starting to maybe see what's going on now: org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException (Hexdump: 30 81 89 02 01 02 68 67 04 1E 6F 75 3D 53 74 6F 72 65 64 20 50 72 6F 63 65 64 75 72 65 73 2C 6F 75 3D 73 79 73 74 65 6D 30 45 30 19 04 02 6F 75 31 13 04 11 53 74 6F 72 65 64 20 50 72 6F 63 65 64 75 72 65 73 30 28 04 0B 6F 62 6A 65 63 74 43 6C 61 73 73 31 19 04 03 74 6F 70 04 12 6F 72 67 61 6E 69 7A 61 74 69 6F 6E 61 6C 55 6E 69 74 A0 1B 30 19 04 17 32 2E 31 36 2E 38 34 30 2E 31 2E 31 31 33 37 33 30 2E 33 2E 34 2E 32) This is a perfectly correct message : 30 81 89 02 01 02 68 67 Add request 04 1E ou=Stored Procedures,ou=system 30 45 30 19 04 02 ou 31 13 04 11 Stored Procedures 30 28 04 0B objectClass 31 19 04 03 top 04 12 organizationalUnit A0 1B 30 19 04 17 32 2E 31 36 2E 38 34 302.16.840.1.113730.3.4.2 2E 31 2E 31 31 33 37 33 ManageDSAIt 30 2E 33 2E 34 2E 32 Caused by: java.lang.NullPointerException at org.apache.directory.shared.ldap.codec.actions.controls.AddControl.action(AddControl.java:97) Message message = container.getMessage(); Control control = container.getLdapCodecService().newControl( oidValue ); message.addControl( control ); --- That means the message is null, so the container does not hold any message. This is weird, this might be because the NoD has requested the container to be cleared up. To be investigated... May be adding some logs in the LdapMessageContainer.clear() method could help... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: [LDAP] Odd Protocol Codec behaviors noticed
Yeah fixed this problem it was due to the LdapCodecServiceFactory not properly initializing. On Thu, Feb 24, 2011 at 4:07 PM, Emmanuel Lecharny elecha...@gmail.com wrote: On 2/24/11 3:24 AM, Alex Karasulu wrote: Here's more data that might shed light. Starting to maybe see what's going on now: org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException (Hexdump: 30 81 89 02 01 02 68 67 04 1E 6F 75 3D 53 74 6F 72 65 64 20 50 72 6F 63 65 64 75 72 65 73 2C 6F 75 3D 73 79 73 74 65 6D 30 45 30 19 04 02 6F 75 31 13 04 11 53 74 6F 72 65 64 20 50 72 6F 63 65 64 75 72 65 73 30 28 04 0B 6F 62 6A 65 63 74 43 6C 61 73 73 31 19 04 03 74 6F 70 04 12 6F 72 67 61 6E 69 7A 61 74 69 6F 6E 61 6C 55 6E 69 74 A0 1B 30 19 04 17 32 2E 31 36 2E 38 34 30 2E 31 2E 31 31 33 37 33 30 2E 33 2E 34 2E 32) This is a perfectly correct message : 30 81 89 02 01 02 68 67 Add request 04 1E ou=Stored Procedures,ou=system 30 45 30 19 04 02 ou 31 13 04 11 Stored Procedures 30 28 04 0B objectClass 31 19 04 03 top 04 12 organizationalUnit A0 1B 30 19 04 17 32 2E 31 36 2E 38 34 30 2.16.840.1.113730.3.4.2 2E 31 2E 31 31 33 37 33 ManageDSAIt 30 2E 33 2E 34 2E 32 Caused by: java.lang.NullPointerException at org.apache.directory.shared.ldap.codec.actions.controls.AddControl.action(AddControl.java:97) Message message = container.getMessage(); Control control = container.getLdapCodecService().newControl( oidValue ); message.addControl( control ); --- That means the message is null, so the container does not hold any message. This is weird, this might be because the NoD has requested the container to be cleared up. To be investigated... May be adding some logs in the LdapMessageContainer.clear() method could help... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
[jira] Created: (DIRKRB-81) The ReplayCacheEntry is marked as Serializable, but contains no serializable fields
The ReplayCacheEntry is marked as Serializable, but contains no serializable fields --- Key: DIRKRB-81 URL: https://issues.apache.org/jira/browse/DIRKRB-81 Project: Directory Kerberos Issue Type: Bug Reporter: Emmanuel Lecharny Assignee: Emmanuel Lecharny Priority: Critical Fix For: 2.0.0-RC1 The KerberosTime clientTime field is not serializable. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
API cleanup update
Hi guys, just a quick heads up about what I'm working on. Currently, I'm removing all the useless Serializable implementation. I think we went anal by declaring a lot of classes to be Sezializable, when there is no reason for those classes to be serialized in any way. I have removed the implementation for those classes : (DirectoryServiceOperation) - ChangePassword, GetCatalog, GetPrincipal Value BitString OID Csn [Permission] - {ItemPermission, UserPermission} [UserClass] - {AllUsers, ParentOfEntry, Subtree, ThisEntry, [NamedUserClass] - {Name, UserGroup}} (SchemaObject) - [AbstractSchemaObject] - {AttributeType, DITContentRule, DITStructureRule, LdapSyntax, [LoadableSchemaObject] -{LdapComparatorDescription, [Normalizer] - {…}, NormalizerDescription, [SyntaxChecker] - {…}, SyntaxCheckerDescription}, MatchingRule, MatchingRuleUse, NameForm, ObjectClass} (xxx) are interfaces, [xxx] are Abstract classes. I still have the LdapComparator and the classes extending it implementing Serializable, because JDBM serialize some comparators. I trully think it's a confusion we should get rid of. There is no need to serialize SchemaObject elements into the backend, when we just want to now which comparator to use. Up to a point, the Table knows which kind of SchemaObject it stores, and can grab the associated comparator using the SchemaManager. However, this is an area I'm not comfortable with so I keep those classes implementing Serializable. I'll do the same work for Externalizable, here are the classes implementing this interface : Externalizable classes : LdapPrincipal ChangeLogEvent ReplicaEventMessage ParentIdAndRdn (Entry) - DefaultEntry, ImmutableEntry, ClonedServerEntry, ClonedServerEntrySearch (EntryAttribute) - DefaultEntryAttribute (Modification) - DefaultModification (Value) - [AbstractValue] - {BinaryValue, StringValue} LdifEntry Ava Rdn AFAICT, all of them have to be externalizable, because they all get serialized at one point of another in the server (either in the backend, or in the changeLog) However, I do think that depending on the default serialization process is a *bad* thing : it's costly, and can be avoided. We cureently have a few static classes (DnSerializer, RdnSerializer, AvaSerializer, EntrySerializer) which handle the optimal serialization, we should add some more. Alex suggested to design a single Serializer class which can handle all those serialization/deserialization, I'm 100% with him on that. The target is to be able to finish the Dn/Rdn/Ava cleanup, for what it worth... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: API cleanup update
On Thu, Feb 24, 2011 at 11:12 PM, Emmanuel Lecharny elecha...@gmail.com wrote: Hi guys, just a quick heads up about what I'm working on. Currently, I'm removing all the useless Serializable implementation. I think we went anal by declaring a lot of classes to be Sezializable, when there is no reason for those classes to be serialized in any way. I have removed the implementation for those classes : (DirectoryServiceOperation) - ChangePassword, GetCatalog, GetPrincipal Value BitString OID Csn [Permission] - {ItemPermission, UserPermission} [UserClass] - {AllUsers, ParentOfEntry, Subtree, ThisEntry, [NamedUserClass] - {Name, UserGroup}} (SchemaObject) - [AbstractSchemaObject] - {AttributeType, DITContentRule, DITStructureRule, LdapSyntax, [LoadableSchemaObject] -{LdapComparatorDescription, [Normalizer] - {…}, NormalizerDescription, [SyntaxChecker] - {…}, SyntaxCheckerDescription}, MatchingRule, MatchingRuleUse, NameForm, ObjectClass} (xxx) are interfaces, [xxx] are Abstract classes. I still have the LdapComparator and the classes extending it implementing Serializable, because JDBM serialize some comparators. I trully think it's a confusion we should get rid of. There is no need to serialize SchemaObject elements into the backend, when we just want to now which comparator to use. Up to a point, the Table knows which kind of SchemaObject it stores, and can grab the associated comparator using the SchemaManager. However, this is an area I'm not comfortable with so I keep those classes implementing Serializable. I'll do the same work for Externalizable, here are the classes implementing this interface : Externalizable classes : LdapPrincipal ChangeLogEvent ReplicaEventMessage ParentIdAndRdn (Entry) - DefaultEntry, ImmutableEntry, ClonedServerEntry, ClonedServerEntrySearch (EntryAttribute) - DefaultEntryAttribute (Modification) - DefaultModification (Value) - [AbstractValue] - {BinaryValue, StringValue} LdifEntry Ava Rdn AFAICT, all of them have to be externalizable, because they all get serialized at one point of another in the server (either in the backend, or in the changeLog) However, I do think that depending on the default serialization process is a *bad* thing : it's costly, and can be avoided. We cureently have a few static classes (DnSerializer, RdnSerializer, AvaSerializer, EntrySerializer) which handle the optimal serialization, we should add some more. Alex suggested to design a single Serializer class which can handle all those serialization/deserialization, I'm 100% with him on that. having a single serializer class will force dependency on the packages that are not interested by a user trying to use just a portion of API for e.x if I want to customize a particular module and only interested in serializing Entry classes the suggested serializer class will require ldif module too, even if it is not required The target is to be able to finish the Dn/Rdn/Ava cleanup, for what it worth... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Kiran Ayyagari
[jira] Created: (DIRSERVER-1609) I am having problems starting Apache Directory Server in Windows 7
I am having problems starting Apache Directory Server in Windows 7 -- Key: DIRSERVER-1609 URL: https://issues.apache.org/jira/browse/DIRSERVER-1609 Project: Directory ApacheDS Issue Type: Task Affects Versions: 1.5.7 Environment: Windows 7 Reporter: Subhojit Roy After I install Apache Directory Server in windows 7 OS, and I go to start the service from the services control panel I get te error: The Apache Directory Server - default service failed to start due to the following error: The system cannot find the file specified. (But it does not tell me what file it cannot find) Given below is the details from the event viewer System - Provider [ Name] Service Control Manager [ Guid] {555908d1-a6d7-4695-8e1e-26931d2012f4} [ EventSourceName] Service Control Manager - EventID 7000 [ Qualifiers] 49152 Version 0 Level 2 Task 0 Opcode 0 Keywords 0x8080 - TimeCreated [ SystemTime] 2011-02-24T19:02:32.685345900Z EventRecordID 76153 Correlation - Execution [ ProcessID] 708 [ ThreadID] 3148 Channel System Computer subhroy Security - EventData param1 Apache Directory Server - default param2 %%2 What arguement vcalue does %2 take/ Can someone please help -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: API cleanup update
On Thu, Feb 24, 2011 at 8:50 PM, Kiran Ayyagari kayyag...@apache.org wrote: On Thu, Feb 24, 2011 at 11:12 PM, Emmanuel Lecharny elecha...@gmail.com wrote: Hi guys, just a quick heads up about what I'm working on. Currently, I'm removing all the useless Serializable implementation. I think we went anal by declaring a lot of classes to be Sezializable, when there is no reason for those classes to be serialized in any way. I have removed the implementation for those classes : (DirectoryServiceOperation) - ChangePassword, GetCatalog, GetPrincipal Value BitString OID Csn [Permission] - {ItemPermission, UserPermission} [UserClass] - {AllUsers, ParentOfEntry, Subtree, ThisEntry, [NamedUserClass] - {Name, UserGroup}} (SchemaObject) - [AbstractSchemaObject] - {AttributeType, DITContentRule, DITStructureRule, LdapSyntax, [LoadableSchemaObject] -{LdapComparatorDescription, [Normalizer] - {…}, NormalizerDescription, [SyntaxChecker] - {…}, SyntaxCheckerDescription}, MatchingRule, MatchingRuleUse, NameForm, ObjectClass} (xxx) are interfaces, [xxx] are Abstract classes. I still have the LdapComparator and the classes extending it implementing Serializable, because JDBM serialize some comparators. I trully think it's a confusion we should get rid of. There is no need to serialize SchemaObject elements into the backend, when we just want to now which comparator to use. Up to a point, the Table knows which kind of SchemaObject it stores, and can grab the associated comparator using the SchemaManager. However, this is an area I'm not comfortable with so I keep those classes implementing Serializable. I'll do the same work for Externalizable, here are the classes implementing this interface : Externalizable classes : LdapPrincipal ChangeLogEvent ReplicaEventMessage ParentIdAndRdn (Entry) - DefaultEntry, ImmutableEntry, ClonedServerEntry, ClonedServerEntrySearch (EntryAttribute) - DefaultEntryAttribute (Modification) - DefaultModification (Value) - [AbstractValue] - {BinaryValue, StringValue} LdifEntry Ava Rdn AFAICT, all of them have to be externalizable, because they all get serialized at one point of another in the server (either in the backend, or in the changeLog) However, I do think that depending on the default serialization process is a *bad* thing : it's costly, and can be avoided. We cureently have a few static classes (DnSerializer, RdnSerializer, AvaSerializer, EntrySerializer) which handle the optimal serialization, we should add some more. Alex suggested to design a single Serializer class which can handle all those serialization/deserialization, I'm 100% with him on that. having a single serializer class will force dependency on the packages that are not interested by a user trying to use just a portion of API Yeah I did not think of this. Kiran is right. This will cause a single point of dependency pulling in the kitchen sink. Rather we should have different serializers for different classes. Good catch Kiran, Alex
[jira] Commented: (DIRSERVER-1609) I am having problems starting Apache Directory Server in Windows 7
[ https://issues.apache.org/jira/browse/DIRSERVER-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12999085#comment-12999085 ] Stefan Seelmann commented on DIRSERVER-1609: I installed ApacheDS 1.5.7 on my Windows 7 64-bit using a 32-bit JavaVM and it worked fine. Could you please tell: - Are you using a 64-bit or 32-bit version of Windows? - Which JavaVM do you use? Is it 64-bit or 32 bit? - Could you please attach your apacheds.conf (should be in C:\Program Files (x86)\Apache Directory Server\conf) I'm not sure if the wrapper works with a 64-bit VM, so if you only have a 64-bit JavaVM could you please try with 32-bit VM? I am having problems starting Apache Directory Server in Windows 7 -- Key: DIRSERVER-1609 URL: https://issues.apache.org/jira/browse/DIRSERVER-1609 Project: Directory ApacheDS Issue Type: Task Affects Versions: 1.5.7 Environment: Windows 7 Reporter: Subhojit Roy After I install Apache Directory Server in windows 7 OS, and I go to start the service from the services control panel I get te error: The Apache Directory Server - default service failed to start due to the following error: The system cannot find the file specified. (But it does not tell me what file it cannot find) Given below is the details from the event viewer System - Provider [ Name] Service Control Manager [ Guid] {555908d1-a6d7-4695-8e1e-26931d2012f4} [ EventSourceName] Service Control Manager - EventID 7000 [ Qualifiers] 49152 Version 0 Level 2 Task 0 Opcode 0 Keywords 0x8080 - TimeCreated [ SystemTime] 2011-02-24T19:02:32.685345900Z EventRecordID 76153 Correlation - Execution [ ProcessID] 708 [ ThreadID] 3148 Channel System Computer subhroy Security - EventData param1 Apache Directory Server - default param2 %%2 What arguement vcalue does %2 take/ Can someone please help -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: API cleanup update
On 2/24/11 10:12 PM, Alex Karasulu wrote: On Thu, Feb 24, 2011 at 8:50 PM, Kiran Ayyagarikayyag...@apache.org wrote: On Thu, Feb 24, 2011 at 11:12 PM, Emmanuel Lecharnyelecha...@gmail.com wrote: Hi guys, just a quick heads up about what I'm working on. Currently, I'm removing all the useless Serializable implementation. I think we went anal by declaring a lot of classes to be Sezializable, when there is no reason for those classes to be serialized in any way. I have removed the implementation for those classes : (DirectoryServiceOperation) - ChangePassword, GetCatalog, GetPrincipal Value BitString OID Csn [Permission] - {ItemPermission, UserPermission} [UserClass] - {AllUsers, ParentOfEntry, Subtree, ThisEntry, [NamedUserClass] - {Name, UserGroup}} (SchemaObject) - [AbstractSchemaObject] - {AttributeType, DITContentRule, DITStructureRule, LdapSyntax, [LoadableSchemaObject] -{LdapComparatorDescription, [Normalizer] - {…}, NormalizerDescription, [SyntaxChecker] - {…}, SyntaxCheckerDescription}, MatchingRule, MatchingRuleUse, NameForm, ObjectClass} (xxx) are interfaces, [xxx] are Abstract classes. I still have the LdapComparator and the classes extending it implementing Serializable, because JDBM serialize some comparators. I trully think it's a confusion we should get rid of. There is no need to serialize SchemaObject elements into the backend, when we just want to now which comparator to use. Up to a point, the Table knows which kind of SchemaObject it stores, and can grab the associated comparator using the SchemaManager. However, this is an area I'm not comfortable with so I keep those classes implementing Serializable. I'll do the same work for Externalizable, here are the classes implementing this interface : Externalizable classes : LdapPrincipal ChangeLogEvent ReplicaEventMessage ParentIdAndRdn (Entry) - DefaultEntry, ImmutableEntry, ClonedServerEntry, ClonedServerEntrySearch (EntryAttribute) - DefaultEntryAttribute (Modification) - DefaultModification (Value) - [AbstractValue] - {BinaryValue, StringValue} LdifEntry Ava Rdn AFAICT, all of them have to be externalizable, because they all get serialized at one point of another in the server (either in the backend, or in the changeLog) However, I do think that depending on the default serialization process is a *bad* thing : it's costly, and can be avoided. We cureently have a few static classes (DnSerializer, RdnSerializer, AvaSerializer, EntrySerializer) which handle the optimal serialization, we should add some more. Alex suggested to design a single Serializer class which can handle all those serialization/deserialization, I'm 100% with him on that. having a single serializer class will force dependency on the packages that are not interested by a user trying to use just a portion of API Yeah I did not think of this. Kiran is right. This will cause a single point of dependency pulling in the kitchen sink. Rather we should have different serializers for different classes. Absolutely. At first, I thought it was a good idea to gather all the serializer in a single place, but Kiran has a valid point here... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Adding a new profile to the 'project' pom.xml to ease the import of the same project twice in Eclipse
On 2/23/11 1:54 PM, Pierre-Arnaud Marcelot wrote: Hi Dev, Eclipse is great but it has one major drawback for me (and others too I believe), that's it's inability to import the same project twice in the workspace. Now, by the same project I don't mean the exact same project (which would not make much sense) but I mean the same project but from various SVN branches (e.g. one version from trunk and another from a branch). Considering that we're now working in two different places ('trunks' and 'milestones' branches), I would rather have all the projects in a common workspace than having two workspaces (one for each branch). The basic problem is that 'mvn eclipse:eclipse' generates the same project names for both branches and that Eclipse only allows one project with a given name at a time in the workspace (and I can understand that). Recently, while looking at the Maven Eclipse plugin documentation I found out an interesting configuration option that we could leverage to solve this issue. I built a specific profile that will be activated if we specify a string we want to be appended to the name of the project when creating eclipse descriptors using 'mvn eclipse:eclipse -Dappend.to.project.name=[string]'. Here it is: profile idappend-to-project-name/id activation property nameappend.to.project.name/name /property /activation build plugins plugin artifactIdmaven-eclipse-plugin/artifactId configuration projectNameTemplate[artifactId]-${append.to.project.name}/projectNameTemplate /configuration /plugin /plugins /build /profile Using this profile it is now possible to import the same project twice (and more) by providing for example the name of the branch, which will be appended to the name of the project. I'd like to include this new profile in the 'project' pom to be able to reuse it across all our projects. Thoughts? Regards, Pierre-Arnaud +1. Go for it ! -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com