you can write a simple program which streams the file from the disk to post it to Solr
On Thu, Oct 8, 2009 at 7:10 PM, Elaine Li <elaine.bing...@gmail.com> wrote: > You can increase the java heap size, e.g. java -Xms128m -Xmx8192m -jar <*.xml> > Or i split the file if it is too big. > > Elaine > > On Thu, Oct 8, 2009 at 6:47 AM, Pravin Karne > <pravin_ka...@persistent.co.in> wrote: >> Hi, >> I am new to solr. I am able to index, search and update with small >> size(around 500mb) >> But if I try to index file with 5 to 10 or more that (500mb) it gives memory >> heap exception. >> While investigation I found that post jar or post.sh load whole file in >> memory. >> >> I use one work around with dividing small file in small files..and it's >> working >> >> Is there any other way to post large file as above work around is not >> feasible for 1 TB file >> >> Thanks >> -Pravin >> >> >> DISCLAIMER >> ========== >> This e-mail may contain privileged and confidential information which is the >> property of Persistent Systems Ltd. It is intended only for the use of the >> individual or entity to which it is addressed. If you are not the intended >> recipient, you are not authorized to read, retain, copy, print, distribute >> or use this message. If you have received this communication in error, >> please notify the sender and delete all copies of this message. Persistent >> Systems Ltd. does not accept any liability for virus infected mails. >> > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com
