Solr also has a feature to stream from a local file rather than over
the network. The parameter
stream.file=/full/local/file/name.txt
means 'read this file from the local disk instead of the POST upload'.
Of course, you have to get the entire file onto the Solr indexer
machine (or a common file
Hi Eric, Shawn,
Thank you for your reply.
Luckily just on the second time itself my 13GB SOLR XML (more than a million
docs) went in fine into SOLR without any problem and I uploaded another 2
more sets of 1.2million+ docs fine without any hassle.
I will try for lesser sized more xmls next time
The error might be that your http client doesn't handle really large
files (32-bit overflow in the Content-Length header?) or something in
your network is killing your long-lived socket? Solr can definitely
accept a 13GB xml document.
I've uploaded large files into Solr successfully, including re
Don't do that. For many reasons . By trying to batch so many docs
together, you're just *asking* for trouble. Quite apart from whether it'll
work once, having *any* HTTP-based protocol work reliably with 13G is
fragile...
For instance, I don't want to have my know whether the XML parsing in
SOLR p
