Hi users
I am using manifoldcf 2.14 Fileshare connector to crawl files from smb server which is having some millions billions of records to process and crawl. Total system memory is 64Gb out of which start options file of manifold is defined as 32GB. We have some larger files to crawl around 30 MB of file or more that than . When mentioned size in the content limiter tab is 100000 that is 1 MB job works fine but when changed to 10000000 that is 10 MB .. manifold crashes with some logs with long running query . How we can achieve or optimise job specifications to process large documents also. Do I need to increase or decrease the number of connections or number of worker thread count or something. Can anybody help me on this to crawl larger files too at least till 10 MB Thanks Ritika
