Oops forgot to mention. We have 1GB assigned to the JVM. It's the same arrangement we've had for months. This is the first time I've seen an out of memory error. I'm the only one on the machine so there's no load other than mine. Seeing the method below the stack trace made me wonder if it's related to the recent changes. I'm rebuilding everything from scratch again. Thanks.
at org.apache.directory.server.xdbm.search.evaluator.SubstringEvaluator.evaluate(SubstringEvaluator.java:135) Regards, Carlo Accorsi -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Kiran Ayyagari Sent: Tuesday, October 16, 2012 9:58 AM To: [email protected] Subject: Re: Possible regression for substring search on indexes? On Tue, Oct 16, 2012 at 7:24 PM, <[email protected]> wrote: > Here are my files.. no, doesn't appear that I have them. Do these look right ? > > 0.9.2342.19200300.100.1.1.db > 0.9.2342.19200300.100.1.1.lg > 0.9.2342.19200300.100.1.1-uid.txt > 0.9.2342.19200300.100.1.3.db > 0.9.2342.19200300.100.1.3.lg > 0.9.2342.19200300.100.1.3-mail.txt > 0.9.2342.19200300.100.1.6.db > 0.9.2342.19200300.100.1.6.lg > 0.9.2342.19200300.100.1.6-roomNumber.txt > 1.3.6.1.4.1.18060.0.4.1.2.3.db > 1.3.6.1.4.1.18060.0.4.1.2.3.lg > 1.3.6.1.4.1.18060.0.4.1.2.3-apachePresence.txt > 1.3.6.1.4.1.18060.0.4.1.2.5.db > 1.3.6.1.4.1.18060.0.4.1.2.5.lg > 1.3.6.1.4.1.18060.0.4.1.2.50.db > 1.3.6.1.4.1.18060.0.4.1.2.50-apacheRdn.txt > 1.3.6.1.4.1.18060.0.4.1.2.5-apacheOneAlias.txt > 1.3.6.1.4.1.18060.0.4.1.2.6.db > 1.3.6.1.4.1.18060.0.4.1.2.6.lg > 1.3.6.1.4.1.18060.0.4.1.2.6-apacheSubAlias.txt > 1.3.6.1.4.1.18060.0.4.1.2.7.db > 1.3.6.1.4.1.18060.0.4.1.2.7.lg > 1.3.6.1.4.1.18060.0.4.1.2.7-apacheAlias.txt > 1.3.6.1.4.1.42.2.27.8.1.23.db > 1.3.6.1.4.1.42.2.27.8.1.23.lg > 1.3.6.1.4.1.42.2.27.8.1.23-pwdPolicySubentry.txt > 1.3.6.1.4.1.4203.666.1.7.db > 1.3.6.1.4.1.4203.666.1.7.lg > 1.3.6.1.4.1.4203.666.1.7-entryCSN.txt > 1.3.6.1.4.1.5322.10.1.1.db > 1.3.6.1.4.1.5322.10.1.1.lg > 1.3.6.1.4.1.5322.10.1.1-krb5PrincipalName.txt > 2.16.840.1.113730.3.1.241.db > 2.16.840.1.113730.3.1.241.lg > 2.16.840.1.113730.3.1.241-displayName.txt > 2.16.840.1.113730.3.1.3.db > 2.16.840.1.113730.3.1.3.lg > 2.16.840.1.113730.3.1.3-employeeNumber.txt > 2.5.18.5.db > 2.5.18.5.lg > 2.5.18.5-administrativeRole.txt > 2.5.4.0.db > 2.5.4.0.lg > 2.5.4.0-objectClass.txt > 2.5.4.10.db > 2.5.4.10.lg > 2.5.4.10-o.txt > 2.5.4.11.db > 2.5.4.11.lg > 2.5.4.11-ou.txt > 2.5.4.13.db > 2.5.4.13.lg > 2.5.4.13-description.txt > 2.5.4.3.db > 2.5.4.3.lg > 2.5.4.31.db > 2.5.4.31.lg > 2.5.4.31-member.txt > 2.5.4.3-cn.txt > 2.5.4.4.db > 2.5.4.4.lg > 2.5.4.42.db > 2.5.4.42.lg > 2.5.4.42-givenName.txt > 2.5.4.4-sn.txt > master.db > looks fine > Out of memory when searching for sn=Thomps* > ahhh, what is the size of heap memory allocated to JVM? > jvm 1 | Server daemon died! > jvm 1 | java.lang.OutOfMemoryError: Java heap space > jvm 1 | at java.nio.CharBuffer.allocate(CharBuffer.java:312) > jvm 1 | at > java.nio.charset.CharsetEncoder.isLegalReplacement(CharsetEncoder.java:319) > jvm 1 | at > java.nio.charset.CharsetEncoder.replaceWith(CharsetEncoder.java:267) > jvm 1 | at > java.nio.charset.CharsetEncoder.<init>(CharsetEncoder.java:186) > jvm 1 | at > java.nio.charset.CharsetEncoder.<init>(CharsetEncoder.java:209) > jvm 1 | at > sun.nio.cs.SingleByteEncoder.<init>(SingleByteEncoder.java:39) > jvm 1 | at sun.nio.cs.MS1252$Encoder.<init>(MS1252.java:115) > jvm 1 | at sun.nio.cs.MS1252.newEncoder(MS1252.java:43) > jvm 1 | at > java.lang.StringCoding$StringEncoder.<init>(StringCoding.java:215) > jvm 1 | at > java.lang.StringCoding$StringEncoder.<init>(StringCoding.java:207) > jvm 1 | at java.lang.StringCoding.encode(StringCoding.java:266) > jvm 1 | at java.lang.StringCoding.encode(StringCoding.java:284) > jvm 1 | at java.lang.String.getBytes(String.java:986) > jvm 1 | at > org.tanukisoftware.wrapper.WrapperManager.sendCommand(WrapperManager.java:3685) > jvm 1 | at > org.tanukisoftware.wrapper.WrapperManager.handleSocket(WrapperManager.java:3804) > jvm 1 | at > org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:4084) > jvm 1 | at java.lang.Thread.run(Thread.java:662) > jvm 1 | [09:31:26] WARN > [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception > forcing session to close: sending disconnect notice to client. > jvm 1 | java.lang.OutOfMemoryError: Java heap space > jvm 1 | at > java.lang.StringCoding$StringEncoder.encode(StringCoding.java:232) > jvm 1 | at java.lang.StringCoding.encode(StringCoding.java:272) > jvm 1 | at java.lang.String.getBytes(String.java:946) > jvm 1 | at > org.apache.directory.shared.util.Strings.getBytesUtf8(Strings.java:1587) > jvm 1 | at > org.apache.directory.shared.ldap.model.entry.StringValue.readExternal(StringValue.java:448) > jvm 1 | at > org.apache.directory.shared.ldap.model.entry.DefaultAttribute.readExternal(DefaultAttribute.java:2064) > jvm 1 | at > org.apache.directory.server.core.partition.impl.btree.jdbm.EntrySerializer.deserialize(EntrySerializer.java:219) > jvm 1 | at jdbm.btree.BPage.deserialize(BPage.java:1255) > jvm 1 | at jdbm.btree.BPage.deserialize(BPage.java:84) > jvm 1 | at > jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:451) > jvm 1 | at > jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:264) > jvm 1 | at jdbm.btree.BPage.loadBPage(BPage.java:1017) > jvm 1 | at jdbm.btree.BPage.find(BPage.java:315) > jvm 1 | at jdbm.btree.BTree.browse(BTree.java:706) > jvm 1 | at jdbm.btree.BTree.find(BTree.java:548) > jvm 1 | at > org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:312) > jvm 1 | at > org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.lookup(AbstractBTreePartition.java:1183) > jvm 1 | at > org.apache.directory.server.xdbm.search.evaluator.SubstringEvaluator.evaluate(SubstringEvaluator.java:135) > jvm 1 | at > org.apache.directory.server.xdbm.search.evaluator.AndEvaluator.evaluate(AndEvaluator.java:109) > jvm 1 | at > org.apache.directory.server.core.partition.impl.btree.EntryCursorAdaptor.get(EntryCursorAdaptor.java:146) > jvm 1 | at > org.apache.directory.server.core.partition.impl.btree.EntryCursorAdaptor.get(EntryCursorAdaptor.java:40) > jvm 1 | at > org.apache.directory.server.core.api.filtering.BaseEntryFilteringCursor.next(BaseEntryFilteringCursor.java:475) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.readResults(SearchHandler.java:377) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.doSimpleSearch(SearchHandler.java:830) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.handleIgnoringReferrals(SearchHandler.java:1137) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.handleWithReferrals(SearchHandler.java:1220) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.handle(SearchHandler.java:227) > jvm 1 | at > org.apache.directory.server.ldap.handlers.SearchHandler.handle(SearchHandler.java:87) > jvm 1 | at > org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:209) > jvm 1 | at > org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:56) > jvm 1 | at > org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:232) > jvm 1 | at > org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:209) > > > > > Regards, > Carlo Accorsi > > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of > Kiran Ayyagari > Sent: Tuesday, October 16, 2012 9:46 AM > To: [email protected] > Subject: Re: Possible regression for substring search on indexes? > > On Tue, Oct 16, 2012 at 6:35 PM, <[email protected]> wrote: >> OK thanks for the quick response. I've built and tested (revision >> 1398757) but is still taking several minutes to return substring search. I'm >> doing my searching via Studio and have exclusive access To the directory. >> Interestingly, it somehow knows that count right away but doesn't return >> anything for several mintues. > >> >> I just swapped out the jar (with the one from a week ago) in this case bc I >> was hoping to validate the test without needing to rebuild and re-import all >> the users. >> I would rebuild everything before for production but to test this >> correction, do I need to do this? >> > better to rebuild the server and overwrite the existing jars >> Also, months ago we had an explicit index on entryUUID but we removed it >> when we didn't see it in the stock config.ldif. >> entryCSN does have an index. >> > it is not needed anymore, cause we now use entryUUID as the *ID* of entry and > master table contains these IDs as keys >> These are the indexed attrs on the partition. After the ldif import they all >> had grown in size (on disk) so I'm fairly confident they're being populated. >> Are we missing any that would work in conjunction with index searches? >> Thanks! >> >> apacheRdn >> apacheSubLevel >> apachePresence >> apacheOneLevel >> apacheOneAlias >> apacheSubAlias >> apacheAlias >> entryCSN >> o >> krb5PrincipalName >> objectClass >> ou >> uid >> employeeNumber >> displayName >> cn >> mail >> roomNumber >> pwdPolicySubEntry >> member >> description >> givenName >> sn >> administrativeRole >> >> > no, nothing is missing, otoh, do you really have the .db files related to ONE > and SUB level indices (not ONE and SUB level *aliases*) curious cause they > are no longer created/used by the server >> Regards, >> Carlo Accorsi >> >> -----Original Message----- >> From: Emmanuel Lécharny [mailto:[email protected]] >> Sent: Tuesday, October 16, 2012 7:33 AM >> To: [email protected] >> Subject: Re: Possible regression for substring search on indexes? >> >> Le 10/15/12 11:52 PM, [email protected] a écrit : >>> Hi, we're using M9 built from the trunk on Wednesday of last week. We have >>> our db setup as it has been for the past several months. There are 80k >>> users. >>> >>> (sn=Thompson) returns in miliseconds >>> (sn=Thompso*) returns after 3-4 minutes. >>> >>> This seems very much like an issue we had back and in June (and was fixed). >>> Any help is most appreciated. We need this build for all the good work >>> Karin did on the password policy. >>> >>> Thank you, Carlo Accorsi >>> >> It should be fixed with >> http://svn.apache.org/viewvc?rev=1398737&view=rev >> >> Can you give it a try ? >> >> -- >> Regards, >> Cordialement, >> Emmanuel Lécharny >> www.iktek.com >> > > > > -- > Kiran Ayyagari > http://keydap.com -- Kiran Ayyagari http://keydap.com
