Yep, going to smaller files (and possibly feeding
them through multiple clients) is the way to go
here.
You could also use a SolrJ client which would be
more efficient, here's a place to start. Admittedly
it doesn't parse JSON, but should give you an idea
of how you could go about it if you wanted
HI Erik,
Thank you for your reply. I wrote into a file. Allmy love to cat * > file.
I structured my JSON in a format that I would like to upload into solr.
defined a schema.xml and solrconfig.xml and took it from there. Initiallu I
uploaded a 1G with post then I got a bit to over zealous I guess.
Let's back up quite a ways here. Where did the 20G file come from?
Indexing files in JSON requires that they follow a very specific format,
Solr doesn't index arbitrary JSON files.
With that out of the way, yes, 20G is unlikely to work without tweaking
some parameters in both solrconfig.xml (there
Hi,
I am relatively new to Solr and have usage query. I have a 20 GB JSON file
which I want to upload into my solr. Do I have to form smaller chunks or is
there a way to upload the whole thing in one go?
I am getting the following error with bin/post:
Entering auto mode. File endings considered