Re: Sharepoint Crawl - Missing documents

2019-03-06 Thread Karl Wright
The SharePoint connector requests documents in chunks of size 10,000. The request you point at gets the documents from row 50,000 through 60,000. The error text (if that is related to this request) shows that the request is timing out because SharePoint is not responding in a timely manner. I

Re: Sharepoint Crawl - Missing documents

2019-03-06 Thread Gaurav G
Hi Karl, On further digging in the Manifold log, I found the following lines..Do they point to any possible reason... We are working on getting the web service specific logs enabled in Sharepoint. Also wanted to check if the Manifold sharepoint plugin prints any logs.. DEBUG

Re: Sharepoint Crawl - Missing documents

2019-03-06 Thread Karl Wright
Hi Guarav, Then I don't understand what is wrong. I've never seen this before, and that was the only thing I could think of. The only thing I can add is that the problem is taking place on the SharePoint side, so maybe (as the error suggests) it might be worth looking at the SharePoint server

Re: Sharepoint Crawl - Missing documents

2019-03-06 Thread Gaurav G
Hi Karl, The Sharepoint version is 2013. I double checked. The version of the plugin that is installed on the server and the one in the connection configuration is all 2013. Thanks, Gaurav On Wed, Mar 6, 2019 at 12:33 PM Karl Wright wrote: > Hi Guarav, > Which version of SharePoint is this?

Re: Sharepoint Crawl - Missing documents

2019-03-05 Thread Karl Wright
Hi Guarav, Which version of SharePoint is this? And, did you install the SharePoint plugin for ManifoldCF, and select the correct versions of SharePoint in the connection configuration? Versions of SharePoint after 2010 limiited the number of documents that could be returned from the Lists

Re: Sharepoint Crawl - Missing documents

2019-03-05 Thread Gaurav G
Hi Karl, There are no subsites as such. It is one big library with all documents in it in a flat structure. The same goes for the list. We enabled the logging for the connector and ran the list job. Below is the exception that it throws after it has crawled the list partially. It looks like after

Re: Sharepoint Crawl - Missing documents

2019-03-04 Thread Karl Wright
Hi Gaurav, There is no document count threshold value. If you can identify libraries or subsites that aren't being crawled, you can turn on connector debugging to see why the connector is skipping them. There could be many reasons for a library or site to be skipped, e.g. bad specification rules,

Sharepoint Crawl - Missing documents

2019-03-04 Thread Gaurav G
Hi, We are trying to crawl a Sharepoint list with about 150,000 items and a library with about 125,000 documents. We have separate jobs for both. The list job only crawls about 5 items and completes cleanly while the library job crawls about 4 documents and completes cleanly. We are