why is the id field multivalued? is there a uniqueKey in the schema ? Are you sure there are no duplicates?
look at the status host:post/dataimport gives you the status it can give you some clue --Noble On Wed, Nov 12, 2008 at 4:53 AM, Giri <[EMAIL PROTECTED]> wrote: > Hi, > > I have about ~ 2 million records in a mySQL database table (about 9 fields > from a single table), and I am trying to load it to the solr using > DataImportHandler using the command=full-import option. it only indexed > about 615360 records out of 2 millions. > > here is my db-data-config.xml > <dataConfig> > <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" > url="jdbc:mysql://localhost:3306/mydb" user="ua" password="pw" batchSize > ="-1"/> > <document name="climate"> > <entity name="occurence" query="select * from mylargetable"> > <field column="id" name="id" /> > <field column="title" name="title" /> > <field column="url" name="url" /> > </entity> > </document> > </dataConfig> > > and in my solr schema.xml, i define these fields as: > > <field name="id" type="string" indexed="true" stored="true" > multiValued="true"/> > <field name="title" type="text" indexed="true" stored="true" > multiValued="true" required="false"/> > <field name="url" type="text" indexed="true" stored="true" > multiValued="true" required="false"/> > > > If I try to index just one field (id), then it indexes about 960000 records, > but if I try to index all the above three fields, it indexes only 615360 > records. > > Any help will be appreciated. > > thanks! > -- --Noble Paul
