RE: Problem with DIH delta-import on JDBC

2010-04-29 Thread cbennett 
Hi,

It looks like the deltaImportQuery needs to be changed you are using
dataimporter.delta.id which is not correct, you are selecting objected in
the deltaQuery, so the deltaImportQuery should be using
dataimporter.delta.objectid

So try this:

entity name=test pk=objectid
query=select * from table
deltaImportQuery=select * from table where
objectid='${dataimporter.delta.objectid}'
deltaQuery=select objectid from table where lastupdate 
'${dataimporter.last_index_time}'
/entity

Colin.

 -Original Message-
 From: safl [mailto:s...@salamin.net]
 Sent: Wednesday, April 28, 2010 3:05 PM
 To: solr-user@lucene.apache.org
 Subject: Problem with DIH delta-import on JDBC
 
 
 Hello,
 
 I'm just new on the list.
 I searched a lot on the list, but I didn't find an answer to my
 question.
 
 I'm using Solr 1.4 on Windows with an Oracle 10g database.
 I am able to do full-import without any problem, but I'm not able to
 get
 delta-import working.
 
 I have the following in the data-config.xml:
 
 ...
 entity name=test pk=objectid
 query=select * from table
 deltaImportQuery=select * from table where
 objectid='${dataimporter.delta.id}'
 deltaQuery=select objectid from table where lastupdate 
 '${dataimporter.last_index_time}'
 /entity
 ...
 
 I update some records in the table and the try to run a delta-import.
 I track the SQL queries on DB with P6Spy, and I always see a query like
 
 select * from table where objectid=''
 
 Of course, with such an SQL query, nothing is updated in my index.
 
 It behave the same if I replace ${dataimporter.delta.id} by
 ${dataimporter.delta.objectid}.
 Can someone tell what is wrong with it?
 
 Thanks a lot,
  Florian
 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Problem-with-DIH-delta-import-on-
 JDBC-tp763469p763469.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: Problem with DIH delta-import on JDBC

2010-04-29 Thread safl 

Hi,

I did a debugger session and found that the column names are case sensitive
(at least with Oracle).
The column names are retreived from the JDBC metadatas and I found that my
objectid is in fact OBJECTID.

So now, I'm able to do an update with the following config (pay attention to
the OBJECTID):

entity name=test pk=OBJECTID
query=select * from table
deltaImportQuery=select * from table where
objectid='${dataimporter.delta.OBJECTID}'
deltaQuery=select objectid from table where lastupdate 
'${dataimporter.last_index_time}'
/entity


Is there a way to be case insensitive ?

Anyway, it works now and that's the most important thing!
:-)

Thanks to all,
 Florian



cbennett wrote:
 
 Hi,
 
 It looks like the deltaImportQuery needs to be changed you are using
 dataimporter.delta.id which is not correct, you are selecting objected in
 the deltaQuery, so the deltaImportQuery should be using
 dataimporter.delta.objectid
 
 So try this:
 
 entity name=test pk=objectid
 query=select * from table
 deltaImportQuery=select * from table where
 objectid='${dataimporter.delta.objectid}'
 deltaQuery=select objectid from table where lastupdate 
 '${dataimporter.last_index_time}'
 /entity
 
 Colin.
 
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Problem-with-DIH-delta-import-on-JDBC-tp763469p765262.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Problem with DIH delta-import on JDBC

2010-04-29 Thread Jon Baer 
All that stuff happens in the JDBC driver associated w/ the DataSource so 
probably not unless there is something which can be set in the Oracle driver 
itself.

One thing that might have helped in this case might have been if 
readFieldNames() in the JDBCDataSource dumped its return to debug log for you.  
That might be something that can be JIRA(ed).

- Jon

On Apr 29, 2010, at 9:45 AM, safl wrote:

 
 Hi,
 
 I did a debugger session and found that the column names are case sensitive
 (at least with Oracle).
 The column names are retreived from the JDBC metadatas and I found that my
 objectid is in fact OBJECTID.
 
 So now, I'm able to do an update with the following config (pay attention to
 the OBJECTID):
 
 entity name=test pk=OBJECTID
query=select * from table
deltaImportQuery=select * from table where
 objectid='${dataimporter.delta.OBJECTID}'
deltaQuery=select objectid from table where lastupdate 
 '${dataimporter.last_index_time}'
 /entity
 
 
 Is there a way to be case insensitive ?
 
 Anyway, it works now and that's the most important thing!
 :-)
 
 Thanks to all,
 Florian
 
 
 
 cbennett wrote:
 
 Hi,
 
 It looks like the deltaImportQuery needs to be changed you are using
 dataimporter.delta.id which is not correct, you are selecting objected in
 the deltaQuery, so the deltaImportQuery should be using
 dataimporter.delta.objectid
 
 So try this:
 
 entity name=test pk=objectid
query=select * from table
deltaImportQuery=select * from table where
 objectid='${dataimporter.delta.objectid}'
deltaQuery=select objectid from table where lastupdate 
 '${dataimporter.last_index_time}'
 /entity
 
 Colin.
 
 -- 
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Problem-with-DIH-delta-import-on-JDBC-tp763469p765262.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Problem with DIH delta-import on JDBC

2010-04-28 Thread Jon Baer 
You should end up w/ a file like conf/dataimport.properties  @ full import 
time, might be that it did not get written out?

- Jon

On Apr 28, 2010, at 3:05 PM, safl wrote:

 
 Hello,
 
 I'm just new on the list.
 I searched a lot on the list, but I didn't find an answer to my question.
 
 I'm using Solr 1.4 on Windows with an Oracle 10g database.
 I am able to do full-import without any problem, but I'm not able to get
 delta-import working.
 
 I have the following in the data-config.xml:
 
 ...
 entity name=test pk=objectid
query=select * from table
deltaImportQuery=select * from table where
 objectid='${dataimporter.delta.id}'
deltaQuery=select objectid from table where lastupdate 
 '${dataimporter.last_index_time}'
 /entity
 ...
 
 I update some records in the table and the try to run a delta-import.
 I track the SQL queries on DB with P6Spy, and I always see a query like
 
 select * from table where objectid=''
 
 Of course, with such an SQL query, nothing is updated in my index.
 
 It behave the same if I replace ${dataimporter.delta.id} by
 ${dataimporter.delta.objectid}.
 Can someone tell what is wrong with it?
 
 Thanks a lot,
 Florian
 -- 
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Problem-with-DIH-delta-import-on-JDBC-tp763469p763469.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Problem with DIH delta-import on JDBC

2010-04-28 Thread safl 


Jon Baer wrote:
 
 You should end up w/ a file like conf/dataimport.properties  @ full
 import time, might be that it did not get written out?
 
 - Jon
 

That's right. I get the dataimport.properties in conf directory after full
import and the deltaQuery looks like:

select objectid from table where lastupdate  '2010-04-28 10:28:04'


If I execute the statment above directly in Oracle, I get the right list of
objectids which have been modified.

The problem remains on the deltaImportQuery.

Thanks for the hint,
 Florian
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Problem-with-DIH-delta-import-on-JDBC-tp763469p763722.html
Sent from the Solr - User mailing list archive at Nabble.com.