Re: SharePoint 2007 Connector - (401)HTTP/1.1 401 Unauthorized
Hi Luigi,
Others have also run into this exception, from one or more SharePoint
web services. It is a server side catch-all exception which tells us
very little.
You may get more details by looking at the server's event logs.
SharePoint also has a log you can look at which may be even more
helpful. In my experience, this is often the result of administrators
changing the system's permissions in ways that cause SharePoint's web
services to stop functioning correctly. At MetaCarta we never would
see this on fresh SharePoint installations, but only on those where
SharePoint was first installed, and then afterwards people made
adjustments to the system permissions.
I hope you have access to a competent SharePoint system administrator,
because without that, it will be very hard to resolve this problem.
Thanks,
Karl
On Thu, Dec 6, 2012 at 5:12 AM, Luigi D'Addario
luigi.dadda...@googlemail.com wrote:
Karl,
I'm trying to put into Solr my SharPoint documents from Shared Documents.
What do you think about this exception ?
Permission problems again or ?
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Getting
version of '/Shared Documents//'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Checking
whether to include library '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Library
'/Shared Documents' exactly matched rule path '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Including
library '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Processing:
'/Shared Documents//'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Document
identifier is a library: '/Shared Documents'
DEBUG 2012-12-06 11:02:09,515 (Worker thread '3') - Enter:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,000 (Worker thread '3') - Exit:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,031 (Worker thread '3') - Enter:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,406 (Worker thread '3') - Exit:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,421 (Worker thread '3') - SharePoint: Got an
unknown remote exception getting child documents for site guid
{CC072748-E1EE-4F34-B120-FAF33273A616} - axis fault = Server.Dsp.Connect,
detail = Cannot open the requested Sharepoint Site. - retrying
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.Dsp.Connect
faultSubcode:
faultString: Cannot open the requested Sharepoint Site.
faultActor:
faultNode:
faultDetail:
{http://schemas.microsoft.com/sharepoint/dsp}queryResponse:dsQueryResponse
status=failure/
Cannot open the requested Sharepoint Site.
I send you manifoldcf.log.
Thanks.
Luigi
2012/12/5 Luigi D'Addario luigi.dadda...@googlemail.com
..and I, finally, tomorrow will try to put into Solr my SharPoint
documents !
2012/12/5 Karl Wright daddy...@gmail.com
I'll have to figure out how to get this patched httpcomponents release
into the field
Re: SharePoint 2007 Connector - (401)HTTP/1.1 401 Unauthorized
If you have access to the SharePoint installation media itself, one
approach would be to try to install your own version of SharePoint on
a similar environment. Prove to yourself (and others) that you can
actually crawl on that SharePoint. Then, based on what the target
system's event logs and SharePoint logs tell you, you can start
modifying settings and module permissions to match the fresh
installation's, until it works. You can also save yourself some time
by getting the actual request being done using http wire debugging in
ManifoldCF, and then trying that request over and over with curl until
you get it to not fail.
Thanks,
Karl
On Thu, Dec 6, 2012 at 6:29 AM, Karl Wright daddy...@gmail.com wrote:
Hi Luigi,
Others have also run into this exception, from one or more SharePoint
web services. It is a server side catch-all exception which tells us
very little.
You may get more details by looking at the server's event logs.
SharePoint also has a log you can look at which may be even more
helpful. In my experience, this is often the result of administrators
changing the system's permissions in ways that cause SharePoint's web
services to stop functioning correctly. At MetaCarta we never would
see this on fresh SharePoint installations, but only on those where
SharePoint was first installed, and then afterwards people made
adjustments to the system permissions.
I hope you have access to a competent SharePoint system administrator,
because without that, it will be very hard to resolve this problem.
Thanks,
Karl
On Thu, Dec 6, 2012 at 5:12 AM, Luigi D'Addario
luigi.dadda...@googlemail.com wrote:
Karl,
I'm trying to put into Solr my SharPoint documents from Shared Documents.
What do you think about this exception ?
Permission problems again or ?
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Getting
version of '/Shared Documents//'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Checking
whether to include library '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Library
'/Shared Documents' exactly matched rule path '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Including
library '/Shared Documents'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Processing:
'/Shared Documents//'
DEBUG 2012-12-06 11:02:09,500 (Worker thread '3') - SharePoint: Document
identifier is a library: '/Shared Documents'
DEBUG 2012-12-06 11:02:09,515 (Worker thread '3') - Enter:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,000 (Worker thread '3') - Exit:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,031 (Worker thread '3') - Enter:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,406 (Worker thread '3') - Exit:
CommonsHTTPSender::invoke
DEBUG 2012-12-06 11:02:10,421 (Worker thread '3') - SharePoint: Got an
unknown remote exception getting child documents for site guid
{CC072748-E1EE-4F34-B120-FAF33273A616} - axis fault = Server.Dsp.Connect,
detail = Cannot open the requested Sharepoint Site. - retrying
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.Dsp.Connect
faultSubcode:
faultString: Cannot open the requested Sharepoint Site.
faultActor:
faultNode:
faultDetail:
{http://schemas.microsoft.com/sharepoint/dsp}queryResponse:dsQueryResponse
status=failure/
Cannot open the requested Sharepoint Site.
I send you manifoldcf.log.
Thanks.
Luigi
2012/12/5 Luigi D'Addario luigi.dadda...@googlemail.com
..and I, finally, tomorrow will try to put into Solr my SharPoint
documents !
2012/12/5 Karl Wright daddy...@gmail.com
I'll have to figure out how to get this patched httpcomponents release
into the field
RE: Cannot connect to SharePoint 2010 instance
Thanks Karl I will give it a try today -Original Message- From: Karl Wright [mailto:daddy...@gmail.com] Sent: Wednesday, December 05, 2012 6:04 PM To: user@manifoldcf.apache.org Subject: Re: Cannot connect to SharePoint 2010 instance Hi Robert, I've solved Luigi's problem - and now I want to know if it solves yours. Unfortunately, you WILL have to build ManifoldCF for this step, since I cannot modify the build process easily to accommodate the patched httpcomponents dependencies. Can you do the following: (1) Check out a trunk copy of manifoldcf sources, e.g svn co https://svn.apache.org/repos/asf/manifoldcf/trunk; . (2) Download the lib package from http://people.apache.org/~kwright/apache-manifoldcf-1.1-dev, unpack it, and install it in the lib directory as per the instructions in the lib package. (3) Run ant build to be sure you can actually build the project. If that works, download the two patched httpcomponents jars from http://people.apache.org/~kwright , and use them to overwrite lib/httpcore.jar and lib/httpclient.jar. (4) Run ant build clean (5) Start manifoldcf (it's under the dist directory), and see if you can connect to your sharepoint instance. Thanks! Karl On Thu, Nov 29, 2012 at 8:56 AM, Iannetti, Robert robert.ianne...@novartis.com wrote: Hi Karl, I have been following your thread with Luigi I look forward to testing the new release. Thanks Bob -Original Message- From: Karl Wright [mailto:daddy...@gmail.com] Sent: Thursday, November 29, 2012 3:28 AM To: user@manifoldcf.apache.org Subject: Re: Cannot connect to SharePoint 2010 instance Hi Robert, Luigi and I think we've discovered the issue, which we're going to see if we can confirm today. There is a ticket tracking it, which is CONNECTORS-572. If correct, it appears that Windows may have changed what it considers to be the name of the user at some recent time, and the httpcomponents and commons-httpclient implementations of NTLM are not resilient to this change - which isn't surprising since they are basically reverse-engineered. If correct, httpcomponents will likely need to release a patch, so the schedule will be, in part, up to them. Alternatively, we can build and patch httpcomponents as part of the ManifoldCF release process, but it would require us to have a new Maven dependency for the make-core-deps part of our release. Karl On Wed, Nov 28, 2012 at 9:01 AM, Iannetti, Robert robert.ianne...@novartis.com wrote: Karl, Here is my debug output DEBUG 2012-11-28 08:59:25,884 (Thread-479) - POST /_vti_bin/lists.asmx HTTP/1 .1 DEBUG 2012-11-28 08:59:25,899 (Thread-479) - Content-Type: text/xml; charset= utf-8 DEBUG 2012-11-28 08:59:25,899 (Thread-479) - SOAPAction: http://schemas.micr osoft.com/sharepoint/soap/GetListCollection DEBUG 2012-11-28 08:59:25,899 (Thread-479) - User-Agent: Axis/1.4 DEBUG 2012-11-28 08:59:25,899 (Thread-479) - Content-Length: 335 DEBUG 2012-11-28 08:59:25,899 (Thread-479) - Host: searchpoc.testprojects.nib r.novartis.intra DEBUG 2012-11-28 08:59:25,899 (Thread-479) - Connection: Keep-Alive DEBUG 2012-11-28 08:59:30,629 (Thread-479) - HTTP/1.1 401 Unauthorized DEBUG 2012-11-28 08:59:30,629 (Thread-479) - Server: Microsoft-IIS/7.5 DEBUG 2012-11-28 08:59:30,629 (Thread-479) - SPRequestGuid: 56647ed0-9bac-4a2 e-b61a-2d2e76ae8db0 DEBUG 2012-11-28 08:59:30,629 (Thread-479) - WWW-Authenticate: NTLM DEBUG 2012-11-28 08:59:30,629 (Thread-479) - X-Powered-By: ASP.NET DEBUG 2012-11-28 08:59:30,630 (Thread-479) - MicrosoftSharePointTeamServices: 14.0.0.6123 DEBUG 2012-11-28 08:59:30,630 (Thread-479) - X-MS-InvokeApp: 1; RequireReadOn ly DEBUG 2012-11-28 08:59:30,630 (Thread-479) - Date: Wed, 28 Nov 2012 13:59:30 GMT DEBUG 2012-11-28 08:59:30,630 (Thread-479) - Content-Length: 0 DEBUG 2012-11-28 08:59:30,663 (Thread-479) - POST /_vti_bin/lists.asmx HTTP/1.1 DEBUG 2012-11-28 08:59:30,663 (Thread-479) - Content-Type: text/xml; charset=utf-8 DEBUG 2012-11-28 08:59:30,663 (Thread-479) - SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetListCollection; DEBUG 2012-11-28 08:59:30,663 (Thread-479) - User-Agent: Axis/1.4 DEBUG 2012-11-28 08:59:30,663 (Thread-479) - Content-Length: 335 DEBUG 2012-11-28 08:59:30,663 (Thread-479) - Host: searchpoc.testprojects.nibr.novartis.intra DEBUG 2012-11-28 08:59:30,663 (Thread-479) - Connection: Keep-Alive DEBUG 2012-11-28 08:59:30,663 (Thread-479) - Authorization: NTLM TlRMTVNTUAABNQIIIAoACgBAIAAgACBJAFAALQAxADAALQAxADQANQAtAD MAMgAtADEAMgAxAE4AQQBOAEUAVAA= DEBUG 2012-11-28 08:59:30,680 (Thread-479) - HTTP/1.1 401 Unauthorized DEBUG 2012-11-28 08:59:30,680 (Thread-479) - Server: Microsoft-IIS/7.5 DEBUG 2012-11-28 08:59:30,680 (Thread-479) - SPRequestGuid: 208f5c66-7d26-4761-b578-d01645f042ed DEBUG 2012-11-28 08:59:30,680 (Thread-479) - WWW-Authenticate: NTLM
Re: Web crawl exited with an unexpected jobqueue status error under MySQL
Oh , I see... I will upgrade MCF and will try again... Thanks! Regards, Shigeki 2012/12/6 Karl Wright daddy...@gmail.com Actually, I just noticed this: I ran MCF0.6 . MCF 0.6 runs MySQL in the wrong mode, so this was a problem. It was fixed in ManifoldCF 1.0. Can you upgrade to MCF 1.01 and see if this still happens for you? Karl On Wed, Dec 5, 2012 at 9:46 PM, Shigeki Kobayashi shigeki.kobayas...@g.softbank.co.jp wrote: Hello Karl. MySQL: 5.5.24 Tomcat: 6.0.35 CentOS: 6.3 Regards, Shigeki 2012/12/5 Karl Wright daddy...@gmail.com Yes, I believe it is related, in the sense that the fix for CONNECTORS-246 was a fix to the HSQLDB database. This error makes it clear that MySQL has a similar problem with its MVCC model, and will also require a fix. However, I do not have the same kinds of leverage in the MySQL community that I do with HSQLDB. Can you give some details about the version of MySQL you are running, and on what platform? I will capture that and then maybe figure out how to open a MySQL ticket. Karl On Wed, Dec 5, 2012 at 6:57 AM, Shigeki Kobayashi shigeki.kobayas...@g.softbank.co.jp wrote: Hi. I ran MCF0.6 under MySQL5.5. I crawled WEB and the following error occurred, then MCF stopped the job: 2012/12/04 18:50:07 ERROR (Worker thread '0') - Exception tossed: Unexpected jobqueue status - record id 1354608871138, expecting active status, saw 3 org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue status - record id 1354608871138, expecting active status, saw 3 at org.apache.manifoldcf.crawler.jobs.JobQueue.updateCompletedRecord(JobQueue.java:711) at org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentCompletedMultiple(JobManager.java:2435) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:745) There was a similar ticket A file crawl exited with an unexpected jobqueue status error under HSQLDB. https://issues.apache.org/jira/browse/CONNECTORS-246 Wondering if this is related.. Regards, Shigeki
