DIH itself may not be consuming so much memory. It also includes the memory used by Solr.
Do you have a hard limit on 400MB , is it not possible to increase it? On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar <manikumarchau...@gmail.com> wrote: > Hi ILAN: > > Only one query is required to generate a document ... > Here is my data-config.xml > > <dataConfig> > <dataSource type="JdbcDataSource" name="sp" > driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/mydb_development" > user="root" password="******" /> > <document name="items"> > <entity name="item" dataSource="sp" query="select * from items"> > <field column="id" name="id" /> > <field column="title" name="title" /> > </entity> > </document> > </dataConfig> > > and other useful info: > > mysql> select * from items > +----------+ > | count(*) | > +----------+ > | 900051 | > +----------+ > 1 row in set (0.00 sec) > > Each record consist of id and title. > > id is of type int(11) and title's avg. length is 50 chars. > > > I am using tomcat with solr. > here is the command i am using to start it > > ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M > > > Thanks! for help. I appreciate it. > > -Mani Kumar > > On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch <i...@fonz.net> wrote: > >> Depending on your dataset and how your queries look you may very likely >> need to increase to a larger heap size. How many queries and rows are >> required for each of your documents to be generated? >> >> Ilan >> >> >> On 4/13/09 12:21 PM, Mani Kumar wrote: >> >>> Hi Shalin: >>> >>> Thanks for quick response! >>> >>> By defaults it was set to 1.93 MB. >>> But i also tried it with following command: >>> >>> $ ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M >>> >>> I also tried tricks given on >>> http://wiki.apache.org/solr/DataImportHandlerFaq page. >>> >>> what should i try next ? >>> >>> Thanks! >>> Mani Kumar >>> >>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar< >>> shalinman...@gmail.com> wrote: >>> >>> On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar<manikumarchau...@gmail.com >>>> >>>>> wrote: >>>>> >>>> >>>> Hi All, >>>>> I am trying to setup a Solr instance on my macbook. >>>>> >>>>> I get following errors when m trying to do a full db import ... please >>>>> >>>> help >>>> >>>>> me on this >>>>> >>>>> java.lang.OutOfMemoryError: Java heap space >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221) >>>> >>>>> at >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351) >>>> >>>>> Caused by: java.lang.OutOfMemoryError: Java heap space >>>>> at com.mysql.jdbc.Buffer.<init>(Buffer.java:58) >>>>> at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444) >>>>> at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840) >>>>> >>>>> >>>>> How much heap size have you allocated to the jvm? >>>> >>>> Also see http://wiki.apache.org/solr/DataImportHandlerFaq >>>> >>>> -- >>>> Regards, >>>> Shalin Shekhar Mangar. >>>> >>>> >>> >> >> -- >> Ilan Rabinovitch >> i...@fonz.net >> >> --- >> SCALE 7x: 2009 Southern California Linux Expo >> Los Angeles, CA >> http://www.socallinuxexpo.org >> >> > -- --Noble Paul
