Hi Markus, Good catch. Yes, this could do it. I'm going to update trunk's dependencies and see if that fixes the issue.
Karl On Fri, Mar 17, 2017 at 4:38 PM, Markus Schuch <markus_sch...@web.de> wrote: > Hi, > > i think this may caused by > > https://issues.apache.org/jira/browse/HTTPCLIENT-1715 > > which was fixed in httpclient 4.5.2 > > There is a very similar stacktrace in > > https://issues.apache.org/jira/browse/HTTPCLIENT-1686 > > which is also linked to HTTPCLIENT-1715. > > Cheers, > Markus > > Am 17.03.2017 um 19:27 schrieb Karl Wright: > > Hi Cihad, > > > > There are NTLMEngineImpl tests that exercise precisely the case that is > > failing. I'm therefore becoming convinced that there is something very > > odd about your installation. Are you using a non-standard JVM, for > > instance? > > > > Karl > > > > > > On Fri, Mar 17, 2017 at 10:28 AM, Karl Wright <daddy...@gmail.com > > <mailto:daddy...@gmail.com>> wrote: > > > > Hi Cihad, > > > > Could you also check out and build the latest 4.5.x httpclient, from > > this branch? > > > > https://svn.apache.org/repos/asf/httpcomponents/httpclient/ > branches/pull-66 > > <https://svn.apache.org/repos/asf/httpcomponents/httpclient/ > branches/pull-66> > > > > You will need maven for this but otherwise you can build it any way > > you like. Replace the "httpclient-4.5.1.jar" in the lib directory > > with the jar you build, and then you can rebuild MCF. See if you > > still get the error. If you do, it should be possible to chase it > > down more readily. > > > > Thanks, > > Karl > > > > > > On Fri, Mar 17, 2017 at 9:57 AM, Cihad Guzel <cguz...@gmail.com > > <mailto:cguz...@gmail.com>> wrote: > > > > No. I don't use any custom library. > > > > I try with manifoldcf trunk on my notebook. I install sharepoint > > 2013 on ms server 2012 for testing with default configuration. > > > > 17 Mar 2017 16:05 tarihinde "Karl Wright" <daddy...@gmail.com > > <mailto:daddy...@gmail.com>> yazdı: > > > > Hmm, I can see no way this can happen. Are you by any > > chance using a modified version of the HttpClient library? > > Karl > > > > > > On Fri, Mar 17, 2017 at 8:09 AM, Karl Wright > > <daddy...@gmail.com <mailto:daddy...@gmail.com>> wrote: > > > > Hi Cihad, > > > > This is very interesting because the problem is coming > > from Httpclient's NTLM engine. The allocated packet > > size for the Type 1 message is being exceeded, which I > > didn't think was even possible. > > > > This may be a result of credentials that you have > > supplied being strange in some way. Let me look at the > > Httpclient code and get back to you. > > > > Karl > > > > > > On Fri, Mar 17, 2017 at 7:57 AM, Cihad Guzel > > <cguz...@gmail.com <mailto:cguz...@gmail.com>> wrote: > > > > Hi, > > > > I try sharepoint connector with Active Directory in > > debug mode. I saw ArrayIndexOutOfBoundException in > > manifoldcf.log file. Any bugs? > > > > DEBUG 2017-03-17 14:30:48,386 (Worker thread '0') - > > SharePoint: Getting version of '/Documents2//Step by > > step Installation of SharePoint 2013 on Windows > > Server 2012 R2 part 1 - SharePoint Community.pdf' > > DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - > > SharePoint: Checking whether to include document > > '/Documents2/Step by step Installation of SharePoint > > 2013 on Windows Server 2012 R2 part 1 - SharePoint > > Community.pdf' > > DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - > > SharePoint: File '/Documents2/Step by step > > Installation of SharePoint 2013 on Windows Server > > 2012 R2 part 1 - SharePoint Community.pdf' exactly > > matched rule path '/Documents2/*' > > DEBUG 2017-03-17 14:30:48,467 (Worker thread '0') - > > SharePoint: Including file '/Documents2/Step by step > > Installation of SharePoint 2013 on Windows Server > > 2012 R2 part 1 - SharePoint Community.pdf' > > DEBUG 2017-03-17 14:30:48,468 (Worker thread '0') - > > SharePoint: Finding metadata to include for > > document/item '/Documents2/Step by step Installation > > of SharePoint 2013 on Windows Server 2012 R2 part 1 > > - SharePoint Community.pdf'. > > DEBUG 2017-03-17 14:30:48,510 (Worker thread '0') - > > SharePoint: In getFieldValues; > > fieldNames=[Ljava.lang.String;@69f1a61a, site='', > > docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}', > > docId='/Documents2/Step by step Installation of > > SharePoint 2013 on Windows Server 2012 R2 part 1 - > > SharePoint Community.pdf', dspStsWorks=false > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - > > SharePoint: Getting version of '/Documents2//' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - > > SharePoint: Getting version of > > '/Documents2//CXFCA3100080010.pdf' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - > > SharePoint: Checking whether to include document > > '/Documents2/CXFCA3100080010.pdf' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - > > SharePoint: File '/Documents2/CXFCA3100080010.pdf' > > exactly matched rule path '/Documents2/*' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - > > SharePoint: Including file > > '/Documents2/CXFCA3100080010.pdf' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - > > SharePoint: Checking whether to include library > > '/Documents2' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - > > SharePoint: Finding metadata to include for > > document/item '/Documents2/CXFCA3100080010.pdf'. > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - > > SharePoint: Library '/Documents2' partially matched > > file rule path '/Documents2/*' - including > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - > > SharePoint: Document identifier is a library: > > '/Documents2' > > DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - > > SharePoint: In getDocLibID; parentSite='', > > parentSiteDecoded='', docLibrary='Documents2' > > DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - > > SharePoint: Getting version of '/' > > DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - > > SharePoint: Checking whether to include site '/' > > DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - > > SharePoint: Site '/' partially matched file rule > > path '/Documents2/*' - including > > DEBUG 2017-03-17 14:30:48,548 (Worker thread '4') - > > SharePoint: In getFieldValues; > > fieldNames=[Ljava.lang.String;@6f447d2e, site='', > > docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}', > > docId='/Documents2/CXFCA3100080010.pdf', > > dspStsWorks=false > > DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - > > SharePoint: Document identifier is a site: '' > > DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - > > SharePoint: In getSites; parentSite='' > > DEBUG 2017-03-17 14:30:50,398 (Worker thread '4') - > > SharePoint: Got a remote exception getting field > > values for site library > > {1B694C45-DF1F-44E7-9814-F5096E85A126} document > > [/Documents2/CXFCA3100080010.pdf] - retrying > > AxisFault > > faultCode: > > {http://schemas.xmlsoap.org/soap/envelope/}Server. > userException > > <http://schemas.xmlsoap.org/soap/envelope/%7DServer. > userException> > > faultSubcode: > > faultString: > > java.lang.ArrayIndexOutOfBoundsException: 41 > > faultActor: > > faultNode: > > faultDetail: > > {http://xml.apache.org/axis/}stackTrace:java.lang. > ArrayIndexOutOfBoundsException > > <http://xml.apache.org/axis/%7DstackTrace:java.lang. > ArrayIndexOutOfBoundsException>: > > 41 > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:911) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addULong(NTLMEngineImpl.java:941) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1043) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > getType1Message(NTLMEngineImpl.java:148) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > generateType1Msg(NTLMEngineImpl.java:1628) > > at > > org.apache.http.impl.auth.NTLMScheme.authenticate( > NTLMScheme.java:139) > > at > > org.apache.http.impl.auth. > AuthSchemeBase.authenticate(AuthSchemeBase.java:138) > > at > > org.apache.http.impl.auth.HttpAuthenticator.doAuth( > HttpAuthenticator.java:239) > > at > > org.apache.http.impl.auth.HttpAuthenticator. > generateAuthResponse(HttpAuthenticator.java:202) > > at > > org.apache.http.impl.execchain.MainClientExec. > execute(MainClientExec.java:262) > > at > > org.apache.http.impl.execchain.ProtocolExec. > execute(ProtocolExec.java:184) > > at > > org.apache.http.impl.execchain.RedirectExec. > execute(RedirectExec.java:110) > > at org.apache.http.impl.client.In > > <http://org.apache.http.impl.client.In > >ternalHttpClient.doExecute(InternalHttpClient.java:184) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:82) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:107) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:55) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) > > > > {http://xml.apache.org/axis/}hostname:cg-DELL > > <http://xml.apache.org/axis/%7Dhostname:cg-DELL> > > > > java.lang.ArrayIndexOutOfBoundsException: 41 > > at > > org.apache.axis.AxisFault. > makeFault(AxisFault.java:101) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender.invoke(CommonsHTTPSender.java:263) > > at > > org.apache.axis.strategies.InvocationStrategy.visit( > InvocationStrategy.java:32) > > at org.apache.axis.SimpleChain.do > > <http://org.apache.axis.SimpleChain.do>Visiting( > SimpleChain.java:118) > > at org.apache.axis.SimpleChain.in > > <http://org.apache.axis.SimpleChain.in>voke( > SimpleChain.java:83) > > at > > org.apache.axis.client.AxisClient.invoke(AxisClient. > java:165) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in>vokeEngine( > Call.java:2784) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2767) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2443) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2366) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:1812) > > at > > com.microsoft.schemas.sharepoint.soap.ListsSoapStub. > getListItems(ListsSoapStub.java:1841) > > at > > org.apache.manifoldcf.crawler.connectors.sharepoint. > SPSProxyHelper.getFieldValues(SPSProxyHelper.java:2134) > > at > > org.apache.manifoldcf.crawler.connectors.sharepoint. > SharePointRepository.processDocuments(SharePointRepository.java:1433) > > at > > org.apache.manifoldcf.crawler. > system.WorkerThread.run(WorkerThread.java:399) > > Caused by: java.lang.ArrayIndexOutOfBoundsException: > 41 > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:911) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addULong(NTLMEngineImpl.java:941) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1043) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > getType1Message(NTLMEngineImpl.java:148) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > generateType1Msg(NTLMEngineImpl.java:1628) > > at > > org.apache.http.impl.auth.NTLMScheme.authenticate( > NTLMScheme.java:139) > > at > > org.apache.http.impl.auth. > AuthSchemeBase.authenticate(AuthSchemeBase.java:138) > > at > > org.apache.http.impl.auth.HttpAuthenticator.doAuth( > HttpAuthenticator.java:239) > > at > > org.apache.http.impl.auth.HttpAuthenticator. > generateAuthResponse(HttpAuthenticator.java:202) > > at > > org.apache.http.impl.execchain.MainClientExec. > execute(MainClientExec.java:262) > > at > > org.apache.http.impl.execchain.ProtocolExec. > execute(ProtocolExec.java:184) > > at > > org.apache.http.impl.execchain.RedirectExec. > execute(RedirectExec.java:110) > > at org.apache.http.impl.client.In > > <http://org.apache.http.impl.client.In > >ternalHttpClient.doExecute(InternalHttpClient.java:184) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:82) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:107) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:55) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) > > DEBUG 2017-03-17 14:30:50,398 (Worker thread '0') - > > SharePoint: Got a remote exception getting field > > values for site library > > {1B694C45-DF1F-44E7-9814-F5096E85A126} document > > [/Documents2/Step by step Installation of SharePoint > > 2013 on Windows Server 2012 R2 part 1 - SharePoint > > Community.pdf] - retrying > > AxisFault > > faultCode: > > {http://schemas.xmlsoap.org/soap/envelope/}Server. > userException > > <http://schemas.xmlsoap.org/soap/envelope/%7DServer. > userException> > > faultSubcode: > > faultString: > > java.lang.ArrayIndexOutOfBoundsException: 41 > > faultActor: > > faultNode: > > faultDetail: > > {http://xml.apache.org/axis/}stackTrace:java.lang. > ArrayIndexOutOfBoundsException > > <http://xml.apache.org/axis/%7DstackTrace:java.lang. > ArrayIndexOutOfBoundsException>: > > 41 > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:911) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:933) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1040) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > getType1Message(NTLMEngineImpl.java:148) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > generateType1Msg(NTLMEngineImpl.java:1628) > > at > > org.apache.http.impl.auth.NTLMScheme.authenticate( > NTLMScheme.java:139) > > at > > org.apache.http.impl.auth. > AuthSchemeBase.authenticate(AuthSchemeBase.java:138) > > at > > org.apache.http.impl.auth.HttpAuthenticator.doAuth( > HttpAuthenticator.java:239) > > at > > org.apache.http.impl.auth.HttpAuthenticator. > generateAuthResponse(HttpAuthenticator.java:202) > > at > > org.apache.http.impl.execchain.MainClientExec. > execute(MainClientExec.java:262) > > at > > org.apache.http.impl.execchain.ProtocolExec. > execute(ProtocolExec.java:184) > > at > > org.apache.http.impl.execchain.RedirectExec. > execute(RedirectExec.java:110) > > at org.apache.http.impl.client.In > > <http://org.apache.http.impl.client.In > >ternalHttpClient.doExecute(InternalHttpClient.java:184) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:82) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:107) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:55) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) > > > > {http://xml.apache.org/axis/}hostname:cg-DELL > > <http://xml.apache.org/axis/%7Dhostname:cg-DELL> > > > > java.lang.ArrayIndexOutOfBoundsException: 41 > > at > > org.apache.axis.AxisFault. > makeFault(AxisFault.java:101) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender.invoke(CommonsHTTPSender.java:263) > > at > > org.apache.axis.strategies.InvocationStrategy.visit( > InvocationStrategy.java:32) > > at org.apache.axis.SimpleChain.do > > <http://org.apache.axis.SimpleChain.do>Visiting( > SimpleChain.java:118) > > at org.apache.axis.SimpleChain.in > > <http://org.apache.axis.SimpleChain.in>voke( > SimpleChain.java:83) > > at > > org.apache.axis.client.AxisClient.invoke(AxisClient. > java:165) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in>vokeEngine( > Call.java:2784) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2767) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2443) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:2366) > > at org.apache.axis.client.Call.in > > <http://org.apache.axis.client.Call.in > >voke(Call.java:1812) > > at > > com.microsoft.schemas.sharepoint.soap.ListsSoapStub. > getListItems(ListsSoapStub.java:1841) > > at > > org.apache.manifoldcf.crawler.connectors.sharepoint. > SPSProxyHelper.getFieldValues(SPSProxyHelper.java:2134) > > at > > org.apache.manifoldcf.crawler.connectors.sharepoint. > SharePointRepository.processDocuments(SharePointRepository.java:1433) > > at > > org.apache.manifoldcf.crawler. > system.WorkerThread.run(WorkerThread.java:399) > > Caused by: java.lang.ArrayIndexOutOfBoundsException: > 41 > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:911) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:933) > > at > > org.apache.http.impl.auth. > NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1040) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > getType1Message(NTLMEngineImpl.java:148) > > at > > org.apache.http.impl.auth.NTLMEngineImpl. > generateType1Msg(NTLMEngineImpl.java:1628) > > at > > org.apache.http.impl.auth.NTLMScheme.authenticate( > NTLMScheme.java:139) > > at > > org.apache.http.impl.auth. > AuthSchemeBase.authenticate(AuthSchemeBase.java:138) > > at > > org.apache.http.impl.auth.HttpAuthenticator.doAuth( > HttpAuthenticator.java:239) > > at > > org.apache.http.impl.auth.HttpAuthenticator. > generateAuthResponse(HttpAuthenticator.java:202) > > at > > org.apache.http.impl.execchain.MainClientExec. > execute(MainClientExec.java:262) > > at > > org.apache.http.impl.execchain.ProtocolExec. > execute(ProtocolExec.java:184) > > at > > org.apache.http.impl.execchain.RedirectExec. > execute(RedirectExec.java:110) > > at org.apache.http.impl.client.In > > <http://org.apache.http.impl.client.In > >ternalHttpClient.doExecute(InternalHttpClient.java:184) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:82) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:107) > > at org.apache.http.impl.client.Cl > > <http://org.apache.http.impl.client.Cl > >oseableHttpClient.execute(CloseableHttpClient.java:55) > > at > > org.apache.manifoldcf.connectorcommon.common. > CommonsHTTPSender$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) > > WARN 2017-03-17 14:30:50,404 (Worker thread '4') - > > Service interruption reported for job 1489681799610 > > connection 'sp2013 crawl': Remote procedure > > exception: ; nested exception is: > > java.lang.ArrayIndexOutOfBoundsException: 41 > > WARN 2017-03-17 14:30:50,405 (Worker thread '0') - > > Service interruption reported for job 1489681799610 > > connection 'sp2013 crawl': Remote procedure > > exception: ; nested exception is: > > java.lang.ArrayIndexOutOfBoundsException: 41 > > DEBUG 2017-03-17 14:31:09,041 (Worker thread '2') - > > SharePoint: getSites xml response: <ns1:Webs > > xmlns:ns1="http://schemas. > microsoft.com/sharepoint/soap/ > > <http://schemas.microsoft.com/sharepoint/soap/>"/> > > DEBUG 2017-03-17 14:31:09,043 (Worker thread '2') - > > SharePoint: In getDocumentLibraries; parentSite='', > > parentSiteDecoded='' > > > > -- > > Teşekkürler > > Cihad Güzel > > > > > > > > > > >
