On 7/20/2015 3:03 PM, Paden wrote:
> I'm currently trying to index about 54,000 files with the Solr Data Import
> Handler and I've got a small problem. It fetches about half (28,289) of the
> 54,000 files and it process about 14,146 documents before it stops and just
> stands idle. Here's the status output
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 0
> },
> "initArgs": [
> "defaults",
> [
> "config",
> "db-data-config.xml",
> "update.chain",
> "skip-empty"
> ]
> ],
> "command": "status",
> "status": "idle",
> "importResponse": "",
> "statusMessages": {
> "Time Elapsed": "2:39:53.191",
> "Total Requests made to DataSource": "1",
> "Total Rows Fetched": "28289",
> "Total Documents Processed": "14146",
> "Total Documents Skipped": "0",
> "Full Dump Started": "2015-07-20 18:19:17"
> }
> }
>
> it has a green arrow next to the header where it says number or documents
> fetched/process but it doesn't say that it's done indexing. It also doesn't
> have the commit line that I've seen on my other core that I indexed about
> 290 documents on. This is the second time that I have tried to index these
> files. I swung by the office this last weekend to see how the index was
> going and (I didn't write the numbers down but I guess I should have) I seem
> to remember it being pretty much at this EXACT spot when the dataimport
> handler starting being idle the last time too. Is there some line in the
> solr config that I have to change to actually commit some of the documents.
> That way so it isn't all at once? Is there some doc limit I have reached
> that I don't know exists? Are the PDF's too large and killing tika (and solr
> with it). I'm really kind of stuck here.
What Solr version are you using, and if you look for the Solr logfile on
the disk, do you see any errors in it? There may be a few more
questions to ask, but they will depend on the answers to those two.
You may be on to something with the idea of a PDF document that's
killing Tika.
Thanks,
Shawn
