[jira] [Comment Edited] (SOLR-2907) java.lang.IllegalArgumentException: deltaQuery has no column to resolve to declared primary key pk='ITEM_ID, CATEGORY_ID'
[
https://issues.apache.org/jira/browse/SOLR-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151734#comment-14151734
]
Aleksandr Ivanov edited comment on SOLR-2907 at 9/29/14 2:36 PM:
-
Hi to everyone. I've had this problem. But I've solved it by adding name of
field (ITEM_ID and CATEGORY_ID) in schema.xml file on server to (just after
/fields):
uniqueKeyITEM_ID/uniqueKey
uniqueKeyCATEGORY_ID/uniqueKey
But I think the main problem is using wrong value for pk= in data-config.xml
file.
You should use TemplateTransformer, in root entity. Something like this:
entity name=account datasource=MyDS pk=id
transformer=TemplateTransformer rootEntity=true ...
...
field column=ITEM_ID name=item_id /
field column=CATEGORY_ID name=cid
field column=id name=id template=${account.item_id}_${account.cid} //
and it should be 123_232e23
...
/entity
was (Author: aleksandr.ivanov):
Hi to everyone. I've had this problem. But I've solved it by adding name of
field (ITEM_ID and CATEGORY_ID) in schema.xml file on server to (just after
/fields):
uniqueKeyITEM_ID/uniqueKey
uniqueKeyCATEGORY_ID/uniqueKey
But I think the main problem is using wrong value for pk= in data-config.xml
file.
You should use TemplateTransformer, in root entity. Something like this:
entity name=account datasource=MyDS pk=id
transformer=TemplateTransformer rootEntity=true ...
...
field column=ITEM_ID name=item_id /
field column=CATEGORY_ID name=cid
field column=id name=id template=${account.item_id}_${account.cid} //
and it should be 123_232e23
...
/entity
java.lang.IllegalArgumentException: deltaQuery has no column to resolve to
declared primary key pk='ITEM_ID, CATEGORY_ID'
-
Key: SOLR-2907
URL: https://issues.apache.org/jira/browse/SOLR-2907
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler, Schema and Analysis
Affects Versions: 3.4
Reporter: Alan Baker
We are using solr for our site and ran into this error in our own schema and
I was able to reproduce it using the dataimport example code in the solr
project. We do not get this error in SOLR 1.4 only started seeing it as we
are working to upgrade to 3.4.0. It fails when delta-importing linked tables.
Complete trace:
Nov 18, 2011 5:21:02 PM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
SEVERE: Delta Import Failed
java.lang.IllegalArgumentException: deltaQuery has no column to resolve to
declared primary key pk='ITEM_ID, CATEGORY_ID'
at
org.apache.solr.handler.dataimport.DocBuilder.findMatchingPkColumn(DocBuilder.java:849)
at
org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:900)
at
org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:879)
at
org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:285)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:179)
at
org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:390)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:429)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
I used this dataConfig from the wiki on the data import:
dataConfig
dataSource driver=org.hsqldb.jdbcDriver
url=jdbc:hsqldb:./example-DIH/hsqldb/ex user=sa /
document
entity name=item pk=ID
query=select * from item
deltaImportQuery=select * from item where
ID=='${dataimporter.delta.id}'
deltaQuery=select id from item where last_modified gt;
'${dataimporter.last_index_time}'
entity name=item_category pk=ITEM_ID, CATEGORY_ID
query=select CATEGORY_ID from item_category where
ITEM_ID='${item.ID}'
deltaQuery=select ITEM_ID, CATEGORY_ID from
item_category where last_modified '${dataimporter.last_index_time}'
parentDeltaQuery=select ID from item where
ID=${item_category.ITEM_ID}
entity name=category pk=ID
query=select DESCRIPTION as cat from category where
ID = '${item_category.CATEGORY_ID}'
deltaQuery=select ID from category where
last_modified gt; '${dataimporter.last_index_time}'
parentDeltaQuery=select ITEM_ID, CATEGORY_ID from
item_category where CATEGORY_ID=${category.ID}/
/entity
/entity
/document
/dataConfig
To reproduce use the data config from above and set the dataimport.properties
last update times to before the last_modifed date in the example
[jira] [Comment Edited] (SOLR-2907) java.lang.IllegalArgumentException: deltaQuery has no column to resolve to declared primary key pk='ITEM_ID, CATEGORY_ID'
[
https://issues.apache.org/jira/browse/SOLR-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151734#comment-14151734
]
Aleksandr Ivanov edited comment on SOLR-2907 at 9/29/14 2:37 PM:
-
Hi to everyone. I've had this problem. But I've solved it by adding name of
field (ITEM_ID and CATEGORY_ID) in schema.xml file on server to (just after
/fields):
uniqueKeyITEM_ID/uniqueKey
uniqueKeyCATEGORY_ID/uniqueKey
But I think the main problem is using wrong value for pk= in data-config.xml
file.
You should use TemplateTransformer, in root entity. Something like this:
entity name=account datasource=MyDS pk=id
transformer=TemplateTransformer rootEntity=true ...
...
field column=ITEM_ID name=item_id /
field column=CATEGORY_ID name=cid
field column=id name=id template=${account.item_id}_${account.cid} //
and it should be 123_232e23
...
/entity
was (Author: aleksandr.ivanov):
Hi to everyone. I've had this problem. But I've solved it by adding name of
field (ITEM_ID and CATEGORY_ID) in schema.xml file on server to (just after
/fields):
uniqueKeyITEM_ID/uniqueKey
uniqueKeyCATEGORY_ID/uniqueKey
But I think the main problem is using wrong value for pk= in data-config.xml
file.
You should use TemplateTransformer, in root entity. Something like this:
entity name=account datasource=MyDS pk=id
transformer=TemplateTransformer rootEntity=true ...
...
field column=ITEM_ID name=item_id /
field column=CATEGORY_ID name=cid
field column=id name=id template=${account.item_id}_${account.cid} //
and it should be 123_232e23
...
/entity
java.lang.IllegalArgumentException: deltaQuery has no column to resolve to
declared primary key pk='ITEM_ID, CATEGORY_ID'
-
Key: SOLR-2907
URL: https://issues.apache.org/jira/browse/SOLR-2907
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler, Schema and Analysis
Affects Versions: 3.4
Reporter: Alan Baker
We are using solr for our site and ran into this error in our own schema and
I was able to reproduce it using the dataimport example code in the solr
project. We do not get this error in SOLR 1.4 only started seeing it as we
are working to upgrade to 3.4.0. It fails when delta-importing linked tables.
Complete trace:
Nov 18, 2011 5:21:02 PM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
SEVERE: Delta Import Failed
java.lang.IllegalArgumentException: deltaQuery has no column to resolve to
declared primary key pk='ITEM_ID, CATEGORY_ID'
at
org.apache.solr.handler.dataimport.DocBuilder.findMatchingPkColumn(DocBuilder.java:849)
at
org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:900)
at
org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:879)
at
org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:285)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:179)
at
org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:390)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:429)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
I used this dataConfig from the wiki on the data import:
dataConfig
dataSource driver=org.hsqldb.jdbcDriver
url=jdbc:hsqldb:./example-DIH/hsqldb/ex user=sa /
document
entity name=item pk=ID
query=select * from item
deltaImportQuery=select * from item where
ID=='${dataimporter.delta.id}'
deltaQuery=select id from item where last_modified gt;
'${dataimporter.last_index_time}'
entity name=item_category pk=ITEM_ID, CATEGORY_ID
query=select CATEGORY_ID from item_category where
ITEM_ID='${item.ID}'
deltaQuery=select ITEM_ID, CATEGORY_ID from
item_category where last_modified '${dataimporter.last_index_time}'
parentDeltaQuery=select ID from item where
ID=${item_category.ITEM_ID}
entity name=category pk=ID
query=select DESCRIPTION as cat from category where
ID = '${item_category.CATEGORY_ID}'
deltaQuery=select ID from category where
last_modified gt; '${dataimporter.last_index_time}'
parentDeltaQuery=select ITEM_ID, CATEGORY_ID from
item_category where CATEGORY_ID=${category.ID}/
/entity
/entity
/document
/dataConfig
To reproduce use the data config from above and set the dataimport.properties
last update times to before the last_modifed date in the example
[jira] [Comment Edited] (SOLR-2907) java.lang.IllegalArgumentException: deltaQuery has no column to resolve to declared primary key pk='ITEM_ID, CATEGORY_ID'
[
https://issues.apache.org/jira/browse/SOLR-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13970428#comment-13970428
]
Harsha B V edited comment on SOLR-2907 at 4/16/14 5:33 AM:
---
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to fix it.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
std.last_modified,
From student std
deltaImportQuery=Select
std.studentID,
std.studentName,
From student std where
studentID='${dataimporter.delta.ID}'
deltaQuery=select studentID as ID from student where
to_char(last_modified, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
deletedPkQuery=select deleted_id as ID FROM
delete_status WHERE to_char(deleted_date, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
dataSource=ds1
transformer=DateFormatTransformer,script:removeNullDateFields
field column=STUDENTID name=id/
field column=STUDENTNAME name=name/
field column=LAST_MODIFIED name=last_modified
xpath=/RDF/item/date dateTimeFormat=-MM-dd HH:mm:ss/
/entity
/document
/dataConfig
As you can see,
-- I have used the entity's 'pk' attribute and set the value as ID (In
capital letters - uniqueKey value)
-- Use the same ID in 'deltaImportQuery' as ${dataimporter.delta.ID}
-- ID must be used as it is in 'deltaQuery' select statement as select ID
from ...
(if you different name for ID column in database, then use 'as' keyword
in select statement. In my case I had 'studentID' as primary key in student
table. So I used it as select studentID as ID from ...
-- The same applies to 'deletedPkQuery'
At present its working fine for me. Any updation in database is reflected in
Solr as well.
See if this helps. Cheers!!!
was (Author: harshrossi):
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to fix it.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
std.last_modified,
[jira] [Comment Edited] (SOLR-2907) java.lang.IllegalArgumentException: deltaQuery has no column to resolve to declared primary key pk='ITEM_ID, CATEGORY_ID'
[
https://issues.apache.org/jira/browse/SOLR-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13970428#comment-13970428
]
Harsha B V edited comment on SOLR-2907 at 4/16/14 5:36 AM:
---
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to SOLVE the problem.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
std.last_modified
From student std
deltaImportQuery=Select
std.studentID,
std.studentName,
std.last_modified
From student std where
studentID='${dataimporter.delta.ID}'
deltaQuery=select studentID as ID from student where
to_char(last_modified, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
deletedPkQuery=select deleted_id as ID FROM
delete_status WHERE to_char(deleted_date, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
dataSource=ds1
transformer=DateFormatTransformer,script:removeNullDateFields
field column=STUDENTID name=id/
field column=STUDENTNAME name=name/
field column=LAST_MODIFIED name=last_modified
xpath=/RDF/item/date dateTimeFormat=-MM-dd HH:mm:ss/
/entity
/document
/dataConfig
As you can see,
-- I have used the entity's 'pk' attribute and set the value as ID (In
capital letters - uniqueKey value)
-- Use the same ID in 'deltaImportQuery' as ${dataimporter.delta.ID}
-- ID must be used as it is in 'deltaQuery' select statement as select ID
from ...
(if you different name for ID column in database, then use 'as' keyword
in select statement. In my case I had 'studentID' as primary key in student
table. So I used it as select studentID as ID from ...
-- The same applies to 'deletedPkQuery'
At present its working fine for me. Any updation in database is reflected in
Solr as well.
See if this helps. Cheers!!!
was (Author: harshrossi):
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to fix it.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
[jira] [Comment Edited] (SOLR-2907) java.lang.IllegalArgumentException: deltaQuery has no column to resolve to declared primary key pk='ITEM_ID, CATEGORY_ID'
[
https://issues.apache.org/jira/browse/SOLR-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13970428#comment-13970428
]
Harsha B V edited comment on SOLR-2907 at 4/16/14 5:35 AM:
---
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to fix it.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
std.last_modified
From student std
deltaImportQuery=Select
std.studentID,
std.studentName,
std.last_modified
From student std where
studentID='${dataimporter.delta.ID}'
deltaQuery=select studentID as ID from student where
to_char(last_modified, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
deletedPkQuery=select deleted_id as ID FROM
delete_status WHERE to_char(deleted_date, '-MM-DD HH24:MI:SS') gt;
'${dataimporter.last_index_time}'
dataSource=ds1
transformer=DateFormatTransformer,script:removeNullDateFields
field column=STUDENTID name=id/
field column=STUDENTNAME name=name/
field column=LAST_MODIFIED name=last_modified
xpath=/RDF/item/date dateTimeFormat=-MM-dd HH:mm:ss/
/entity
/document
/dataConfig
As you can see,
-- I have used the entity's 'pk' attribute and set the value as ID (In
capital letters - uniqueKey value)
-- Use the same ID in 'deltaImportQuery' as ${dataimporter.delta.ID}
-- ID must be used as it is in 'deltaQuery' select statement as select ID
from ...
(if you different name for ID column in database, then use 'as' keyword
in select statement. In my case I had 'studentID' as primary key in student
table. So I used it as select studentID as ID from ...
-- The same applies to 'deletedPkQuery'
At present its working fine for me. Any updation in database is reflected in
Solr as well.
See if this helps. Cheers!!!
was (Author: harshrossi):
Hi all,
I am using Solr 4.8.0 and had faced the same problem but with some trial and
error I was able to fix it.
In my schema.xml I have:
field name=id type=string indexed=true stored=true required=true
multiValued=false/
field name=name type=text_general indexed=true stored=true
multiValued=false/
field name=last_modified type=date indexed=true stored=true
multiValued=false/
unique key is set as:
uniqueKeyid/uniqueKey
In my data-config.xml I have:
dataConfig
dataSource name=ds1 type=JdbcDataSource
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@blah.blah
user=blah blah
password=blah blah/
script
![CDATA[
function removeNullDateFields(row) {
var std_date = row.get('last_modified');
if (std_date === null || true === std_date.isEmpty() ||
std_date === '') {
row.remove('last_modified');
}
return row;
}
]]
/script
document name=search_doc
entity name=search pk=ID
query=Select
std.studentID,
std.studentName,
