Re: problem with data import handler delta import due to use of multiple datasource
Did you solve the problem? I'm stuck with exactly the same problem now. Let me know if you already had a solution,please. -- View this message in context: http://lucene.472066.n3.nabble.com/problem-with-data-import-handler-delta-import-due-to-use-of-multiple-datasource-tp4093698p4314273.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: problem with data import handler delta import due to use of multiple datasource
Thanks for the suggestion but that won't work as I have last_modified field
in both the parent entity and child entity as I want delta import to kick
in when either change. That other approach has the same problem since the
parent and child entity uses different datasources.
Bill
On Tue, Oct 8, 2013 at 10:18 AM, Dyer, James
wrote:
> Bill,
>
> I do not believe there is any way to tell it to use a different datasource
> for the parent delta query.
>
> If you used this approach, would it solve your problem:
> http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport ?
>
> James Dyer
> Ingram Content Group
> (615) 213-4311
>
>
> -Original Message-
> From: Bill Au [mailto:bill.w...@gmail.com]
> Sent: Tuesday, October 08, 2013 8:50 AM
> To: solr-user@lucene.apache.org
> Subject: Re: problem with data import handler delta import due to use of
> multiple datasource
>
> I am using 4.3. It is not related to bugs related to last_index_time. The
> problem is caused by the fact that the parent entity and child entity use
> different data source (different databases on different hosts).
>
> From the log output, I do see the the delta query of the child entity being
> executed correctly and found all the rows that have been modified for the
> child entity. But it fails when it executed the parentDeltaQuery because
> it is still using the database connection from the child entity (ie
> datasource ds2 in my example above).
>
> Is there a way to tell DIH to use a different datasource in the
> parentDeltaQuery?
>
> Bill
>
>
> On Sat, Oct 5, 2013 at 10:28 PM, Alexandre Rafalovitch
> wrote:
>
> > Which version of Solr and what kind of SQL errors? There were some bugs
> in
> > 4.x related to last_index_time, but it does not sound related.
> >
> > Regards,
> >Alex.
> >
> > Personal website: http://www.outerthoughts.com/
> > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> > - Time is the quality of nature that keeps events from happening all at
> > once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
> >
> >
> > On Sun, Oct 6, 2013 at 8:51 AM, Bill Au wrote:
> >
> > > Here is my DIH config:
> > >
> > >
> > > > driver="com.mysql.jdbc.Driver"
> > > url="jdbc:mysql://localhost1/dbname1" user="db_username1"
> > > password="db_password1"/>
> > > > driver="com.mysql.jdbc.Driver"
> > > url="jdbc:mysql://localhost2/dbname2" user="db_username2"
> > > password="db_password2"/>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > I am having trouble with delta import. I think it is because the main
> > > entity and the sub-entity use different data source. I have tried
> using
> > > both a delta query:
> > >
> > > deltaQuery="select id from item where id in (select item_id as id from
> > > feature where last_modified > '${dih.last_index_time}') or
> last_modified
> > > > '${dih.last_index_time}'"
> > >
> > > and a parentDeltaQuery:
> > >
> > > > > parentDeltaQuery="select ID from item where ID=${feature.ITEM_ID}"/>
> > >
> > > I ended up with an SQL error for both. Is there any way to make delta
> > > import work in my case?
> > >
> > > Bill
> > >
> >
>
>
RE: problem with data import handler delta import due to use of multiple datasource
Bill,
I do not believe there is any way to tell it to use a different datasource for
the parent delta query.
If you used this approach, would it solve your problem:
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport ?
James Dyer
Ingram Content Group
(615) 213-4311
-Original Message-
From: Bill Au [mailto:bill.w...@gmail.com]
Sent: Tuesday, October 08, 2013 8:50 AM
To: solr-user@lucene.apache.org
Subject: Re: problem with data import handler delta import due to use of
multiple datasource
I am using 4.3. It is not related to bugs related to last_index_time. The
problem is caused by the fact that the parent entity and child entity use
different data source (different databases on different hosts).
>From the log output, I do see the the delta query of the child entity being
executed correctly and found all the rows that have been modified for the
child entity. But it fails when it executed the parentDeltaQuery because
it is still using the database connection from the child entity (ie
datasource ds2 in my example above).
Is there a way to tell DIH to use a different datasource in the
parentDeltaQuery?
Bill
On Sat, Oct 5, 2013 at 10:28 PM, Alexandre Rafalovitch
wrote:
> Which version of Solr and what kind of SQL errors? There were some bugs in
> 4.x related to last_index_time, but it does not sound related.
>
> Regards,
>Alex.
>
> Personal website: http://www.outerthoughts.com/
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> - Time is the quality of nature that keeps events from happening all at
> once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
>
>
> On Sun, Oct 6, 2013 at 8:51 AM, Bill Au wrote:
>
> > Here is my DIH config:
> >
> >
> > driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost1/dbname1" user="db_username1"
> > password="db_password1"/>
> > driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost2/dbname2" user="db_username2"
> > password="db_password2"/>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > I am having trouble with delta import. I think it is because the main
> > entity and the sub-entity use different data source. I have tried using
> > both a delta query:
> >
> > deltaQuery="select id from item where id in (select item_id as id from
> > feature where last_modified > '${dih.last_index_time}') or last_modified
> > > '${dih.last_index_time}'"
> >
> > and a parentDeltaQuery:
> >
> > > parentDeltaQuery="select ID from item where ID=${feature.ITEM_ID}"/>
> >
> > I ended up with an SQL error for both. Is there any way to make delta
> > import work in my case?
> >
> > Bill
> >
>
Re: problem with data import handler delta import due to use of multiple datasource
I am using 4.3. It is not related to bugs related to last_index_time. The
problem is caused by the fact that the parent entity and child entity use
different data source (different databases on different hosts).
>From the log output, I do see the the delta query of the child entity being
executed correctly and found all the rows that have been modified for the
child entity. But it fails when it executed the parentDeltaQuery because
it is still using the database connection from the child entity (ie
datasource ds2 in my example above).
Is there a way to tell DIH to use a different datasource in the
parentDeltaQuery?
Bill
On Sat, Oct 5, 2013 at 10:28 PM, Alexandre Rafalovitch
wrote:
> Which version of Solr and what kind of SQL errors? There were some bugs in
> 4.x related to last_index_time, but it does not sound related.
>
> Regards,
>Alex.
>
> Personal website: http://www.outerthoughts.com/
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> - Time is the quality of nature that keeps events from happening all at
> once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
>
>
> On Sun, Oct 6, 2013 at 8:51 AM, Bill Au wrote:
>
> > Here is my DIH config:
> >
> >
> > driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost1/dbname1" user="db_username1"
> > password="db_password1"/>
> > driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost2/dbname2" user="db_username2"
> > password="db_password2"/>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > I am having trouble with delta import. I think it is because the main
> > entity and the sub-entity use different data source. I have tried using
> > both a delta query:
> >
> > deltaQuery="select id from item where id in (select item_id as id from
> > feature where last_modified > '${dih.last_index_time}') or last_modified
> > > '${dih.last_index_time}'"
> >
> > and a parentDeltaQuery:
> >
> > > parentDeltaQuery="select ID from item where ID=${feature.ITEM_ID}"/>
> >
> > I ended up with an SQL error for both. Is there any way to make delta
> > import work in my case?
> >
> > Bill
> >
>
Re: problem with data import handler delta import due to use of multiple datasource
Which version of Solr and what kind of SQL errors? There were some bugs in
4.x related to last_index_time, but it does not sound related.
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
On Sun, Oct 6, 2013 at 8:51 AM, Bill Au wrote:
> Here is my DIH config:
>
>
> url="jdbc:mysql://localhost1/dbname1" user="db_username1"
> password="db_password1"/>
> url="jdbc:mysql://localhost2/dbname2" user="db_username2"
> password="db_password2"/>
>
>
>
>
>
>
>
>
>
>
>
>
> I am having trouble with delta import. I think it is because the main
> entity and the sub-entity use different data source. I have tried using
> both a delta query:
>
> deltaQuery="select id from item where id in (select item_id as id from
> feature where last_modified > '${dih.last_index_time}') or last_modified
> > '${dih.last_index_time}'"
>
> and a parentDeltaQuery:
>
> parentDeltaQuery="select ID from item where ID=${feature.ITEM_ID}"/>
>
> I ended up with an SQL error for both. Is there any way to make delta
> import work in my case?
>
> Bill
>
problem with data import handler delta import due to use of multiple datasource
Here is my DIH config:
I am having trouble with delta import. I think it is because the main
entity and the sub-entity use different data source. I have tried using
both a delta query:
deltaQuery="select id from item where id in (select item_id as id from
feature where last_modified > '${dih.last_index_time}') or last_modified
> '${dih.last_index_time}'"
and a parentDeltaQuery:
I ended up with an SQL error for both. Is there any way to make delta
import work in my case?
Bill
