Here is the stack trace: notice in stack trace * "at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*
It looks like that its trying to read whole table into memory at a time. n thts y getting OOM. Apr 14, 2009 11:15:01 AM org.apache.solr.handler.dataimport.DataImporter doFullImport SEVERE: Full Import failed org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351) Caused by: java.lang.OutOfMemoryError: Java heap space at com.mysql.jdbc.Buffer.<init>(Buffer.java:58) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444) at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840) at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:468) at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2534) at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2159) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2548) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2477) at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:741) at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:587) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:243) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:207) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:38) at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:335) ... 5 more Apr 14, 2009 11:15:01 AM org.apache.solr.update.DirectUpdateHandler2 rollback INFO: start rollback Apr 14, 2009 11:15:01 AM org.apache.solr.update.DirectUpdateHandler2 rollback INFO: end_rollback On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar <manikumarchau...@gmail.com>wrote: > Hi ILAN: > > Only one query is required to generate a document ... > Here is my data-config.xml > > <dataConfig> > <dataSource type="JdbcDataSource" name="sp" > driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/mydb_development" > user="root" password="******" /> > <document name="items"> > <entity name="item" dataSource="sp" query="select * from items"> > <field column="id" name="id" /> > <field column="title" name="title" /> > </entity> > </document> > </dataConfig> > > and other useful info: > > mysql> select * from items > +----------+ > | count(*) | > +----------+ > | 900051 | > +----------+ > 1 row in set (0.00 sec) > > Each record consist of id and title. > > id is of type int(11) and title's avg. length is 50 chars. > > > I am using tomcat with solr. > here is the command i am using to start it > > ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M > > > Thanks! for help. I appreciate it. > > -Mani Kumar > > On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch <i...@fonz.net> wrote: > >> Depending on your dataset and how your queries look you may very likely >> need to increase to a larger heap size. How many queries and rows are >> required for each of your documents to be generated? >> >> Ilan >> >> >> On 4/13/09 12:21 PM, Mani Kumar wrote: >> >>> Hi Shalin: >>> >>> Thanks for quick response! >>> >>> By defaults it was set to 1.93 MB. >>> But i also tried it with following command: >>> >>> $ ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M >>> >>> I also tried tricks given on >>> http://wiki.apache.org/solr/DataImportHandlerFaq page. >>> >>> what should i try next ? >>> >>> Thanks! >>> Mani Kumar >>> >>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar< >>> shalinman...@gmail.com> wrote: >>> >>> On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar<manikumarchau...@gmail.com >>>> >>>>> wrote: >>>>> >>>> >>>> Hi All, >>>>> I am trying to setup a Solr instance on my macbook. >>>>> >>>>> I get following errors when m trying to do a full db import ... please >>>>> >>>> help >>>> >>>>> me on this >>>>> >>>>> java.lang.OutOfMemoryError: Java heap space >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221) >>>> >>>>> at >>>>> >>>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370) >>>> >>>>> at >>>>> >>>>> >>>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351) >>>> >>>>> Caused by: java.lang.OutOfMemoryError: Java heap space >>>>> at com.mysql.jdbc.Buffer.<init>(Buffer.java:58) >>>>> at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444) >>>>> at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840) >>>>> >>>>> >>>>> How much heap size have you allocated to the jvm? >>>> >>>> Also see http://wiki.apache.org/solr/DataImportHandlerFaq >>>> >>>> -- >>>> Regards, >>>> Shalin Shekhar Mangar. >>>> >>>> >>> >> >> -- >> Ilan Rabinovitch >> i...@fonz.net >> >> --- >> SCALE 7x: 2009 Southern California Linux Expo >> Los Angeles, CA >> http://www.socallinuxexpo.org >> >> >