On Tue, Sep 22, 2009 at 10:53 PM, Daniel Bradley <
daniel.brad...@adfero.co.uk> wrote:
> I appear to be getting only a small number of items imported into Solr
> when doing a full-import against an oracle data-provider. The query I'm
> running is something approximately similar to:
>
> SELECT "ID", dbms_lob.substr("Text", 4000, 1) "Text", "Date",
> "LastModified", "Type", "Created", "Available", "Parent", "Title" from
> "TheTableName" where "Available" < CURRENT_DATE and "Available" >
> add_months(current_date, -1)
>
> This retrieves the last month's items from the database (The
> dbms_lob.substr function is used to avoid Solr simply indexing the
> object name as Text is the Oracle clob type). When running this in
> oracle sql developer approximately 5600 rows are returned however
> running a full import only imports approximately 550 items.
>
> There's no visible memory use and no exceptions suggesting any problems
> with lack of memory. Is there any limiting of the number of items you
> can import in a single request? Any other thoughts on this problem would
> be much appreciated.
>
>
What is the uniqueKey in schema.xml? Is it possible that many of those 5600
rows share the same value for solr's uniqueKey field?
There are no limits on the number of items you can import. The number of
documents created should correspond to the number of rows returned by the
root level entity's query (assuming the uniqueKey for each of those
documents is actually unique).
--
Regards,
Shalin Shekhar Mangar.
