pass custom parameters from client to solr
Hi I would like to pass a bunch of custom parameters to turn on/off or set thresholds for various custom implementations we have in our solr. What would be the best way to pass these custom parameters to solr server from the client ? The client is a Java based application and hence we are planning to use solrj. Thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/pass-custom-parameters-from-client-to-solr-tp3987511.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr with UIMA
Hi all, 2012/6/1 Jack Krupansky j...@basetechnology.com Is it failing on the first document? I see uid 5, suggests that it is not. If not, how is this document different from the others? I see the exception org.apache.uima.resource.**ResourceInitializationExceptio**n, suggesting that some file cannot be loaded. It sounds like it may be having trouble loading aePath (analysisEngine). Or maybe some other file? thanks Jack, that's correct, it's most likely what's causing the reported error. Tommaso -- Jack Krupansky -Original Message- From: debdoot Sent: Thursday, May 31, 2012 11:59 AM To: solr-user@lucene.apache.org Subject: Re: Solr with UIMA Hi Tommaso, I have followed the steps you have listed to try to deploy the example RoomNumberAnnotator with Solr 3.5. Here is the error trace that I get: org.apache.solr.common.**SolrException: processing error: null. uid=5, text=quot;Test Room HAW GN-K35...quot; at org.apache.solr.uima.**processor.**UIMAUpdateRequestProcessor.** processAdd#40;**UIMAUpdateRequestProcessor.**java:107#41; at org.apache.solr.handler.**XMLLoader.processUpdate#40;** XMLLoader.java:158#41; at org.apache.solr.handler.**XMLLoader.load#40;XMLLoader.**java:79#41; at org.apache.solr.handler.**ContentStreamHandlerBase.** handleRequestBody#40;**ContentStreamHandlerBase.java:**58#41; at org.apache.solr.handler.**RequestHandlerBase.**handleRequest#40;** RequestHandlerBase.java:129#**41; at org.apache.solr.core.SolrCore.**execute#40;SolrCore.java:**1372#41; at org.apache.solr.servlet.**SolrDispatchFilter.execute#** 40;SolrDispatchFilter.java:**356#41; at org.apache.solr.servlet.**SolrDispatchFilter.doFilter#** 40;SolrDispatchFilter.java:**252#41; at com.ibm.ws.webcontainer.**filter.FilterInstanceWrapper.**doFilter#40;** FilterInstanceWrapper.java:**192#41; at com.ibm.ws.webcontainer.**filter.WebAppFilterChain.**doFilter#40;** WebAppFilterChain.java:89#41; at com.ibm.ws.webcontainer.**filter.WebAppFilterManager.**doFilter#40;** WebAppFilterManager.java:919#**41; at com.ibm.ws.webcontainer.**filter.WebAppFilterManager.**invokeFilters#40;* *WebAppFilterManager.java:1016**#41; at com.ibm.ws.webcontainer.**webapp.WebApp.handleRequest#** 40;WebApp.java:3703#41; at com.ibm.ws.webcontainer.**webapp.WebGroup.handleRequest** #40;WebGroup.java:304#41; at com.ibm.ws.webcontainer.**WebContainer.handleRequest#** 40;WebContainer.java:953#41; at com.ibm.ws.webcontainer.**WSWebContainer.handleRequest#** 40;WSWebContainer.java:1655#**41; at com.ibm.ws.webcontainer.**channel.WCChannelLink.ready#** 40;WCChannelLink.java:195#41; at com.ibm.ws.http.channel.**inbound.impl.HttpInboundLink.** handleDiscrimination#40;**HttpInboundLink.java:452#41; at com.ibm.ws.http.channel.**inbound.impl.HttpInboundLink.** handleNewRequest#40;**HttpInboundLink.java:511#41; at com.ibm.ws.http.channel.**inbound.impl.HttpInboundLink.** processRequest#40;**HttpInboundLink.java:305#41; at com.ibm.ws.http.channel.**inbound.impl.HttpInboundLink.** ready#40;HttpInboundLink.**java:276#41; at com.ibm.ws.tcp.channel.impl.**NewConnectionInitialReadCallba** ck.sendToDiscriminators#40;**NewConnectionInitialReadCallba** ck.java:214#41; at com.ibm.ws.tcp.channel.impl.**NewConnectionInitialReadCallba** ck.complete#40;**NewConnectionInitialReadCallba**ck.java:113#41; at com.ibm.ws.tcp.channel.impl.**AioReadCompletionListener.** futureCompleted#40;**AioReadCompletionListener.**java:165#41; at com.ibm.io.async.**AbstractAsyncFuture.**invokeCallback#40;** AbstractAsyncFuture.java:217#**41; at com.ibm.io.async.**AsyncChannelFuture.**fireCompletionActions#40;** AsyncChannelFuture.java:161#**41; at com.ibm.io.async.AsyncFuture.**completed#40;AsyncFuture.** java:138#41; at com.ibm.io.async.**ResultHandler.complete#40;** ResultHandler.java:204#41; at com.ibm.io.async.**ResultHandler.**runEventProcessingLoop#40;** ResultHandler.java:775#41; at com.ibm.io.async.**ResultHandler$2.run#40;** ResultHandler.java:905#41; at com.ibm.ws.util.ThreadPool$**Worker.run#40;ThreadPool.**java:1650#41; Caused by: org.apache.uima.resource.**ResourceInitializationExceptio**n at org.apache.solr.uima.**processor.ae.**OverridingParamsAEProvider.** getAE#40;**OverridingParamsAEProvider.**java:86#41; at org.apache.solr.uima.**processor.**UIMAUpdateRequestProcessor.** processText#40;**UIMAUpdateRequestProcessor.**java:144#41; at org.apache.solr.uima.**processor.**UIMAUpdateRequestProcessor.** processAdd#40;**UIMAUpdateRequestProcessor.**java:77#41; ... 30 more Caused by: java.lang.NullPointerException at org.apache.uima.util.**XMLInputSource.lt;initgt;#** 40;XMLInputSource.java:118#**41; at org.apache.solr.uima.**processor.ae.**OverridingParamsAEProvider.** getAE#40;**OverridingParamsAEProvider.**java:58#41; ... 32 more at com.ibm.ws.webcontainer.**webapp.**WebAppDispatcherContext.**sendError(** WebAppDispatcherContext.java:**624) at
Re: Round Robin concept in distributed Solr
The main shard has some extra work to do. Namely 1 create the sub-requests 2 collate the results from all the sub-requests (including itself). But this work is generally a small amount of the actual work being done, so it's often unnoticeable. That said, I'd just put all my slaves behind a load-balancer and let that mechanism send the requests to the various slaves if for no other reason than your LB should be able to detect if one of your machines goes down and send requests to slaves still running. I even know of one situation where the users main slave has _no_ index on it, it serves solely to distribute requests/aggregate results Best Erick On Sun, Jun 3, 2012 at 11:57 PM, ratabora ratab...@gmail.com wrote: Hey Erick, It looks like the thread you mentioned talks about how to configure the shards parameter in the Solr query. I am more interested in the 'main' shard you query against when you make Solr queries (main shard being the shard you direct the query against, mainshard/select?q=*:*shards=shard1,shard2,shard3) I think Suneel's original question is still unanswered, is it better to use Scenario A or Scenario B? I suppose the 'main' shard is going to create a sub query to the rest of the shards defined in the shard parameter, but I am still wondering if you query the same main shard every time if that is going to have a load/performance impact. Suneel wrote So scenario A (round-robin): query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 2: /solr-shard-2/select?q=dog... shards=shard-1,shard2 query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2 etc. or or scenario B (fixed): query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 2: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2 Thank you for any help. Regards, Ryan Tabora -- View this message in context: http://lucene.472066.n3.nabble.com/Round-Robin-concept-in-distributed-Solr-tp3636345p3987494.html Sent from the Solr - User mailing list archive at Nabble.com.
Add HTTP-header from ResponseWriter
Hi, There has been discussion before on how to add/set a HTTP-header from a ResponseWriter. That was about adding the number of found documents for a CSVResponseWriter. We also need to set the number of found documents, in this case for the JSONResponseWriter. or any ResponseWriter. Is there any progress or open issue i am not aware of? Can the current (trunk) response framework already set or add an HTTP-header? Thanks, Markus
Re: How to use SolrJ index a file or a FileInputStream with some other attributes?
How to use SolrJ index a file or a FileInputStream with some other attributes? Please give me some example. Here are two good resources: http://www.lucidimagination.com/blog/2012/02/14/indexing-with-solrj/ http://wiki.apache.org/solr/Solrj
Re: pass custom parameters from client to solr
I would like to pass a bunch of custom parameters to turn on/off or set thresholds for various custom implementations we have in our solr. What would be the best way to pass these custom parameters to solr server from the client ? The client is a Java based application and hence we are planning to use solrj. You can use custom parameters in the URL. solr/select?q=testcustomParam=customValuerows=10 etc. SolrJ equivalent : SolrQuery query = new SolrQuery(); query.setRows(10); query.set(customParam,customValue);
Re: How to use SolrJ index a file or a FileInputStream with some other attributes?
More specifically, you probably want ContentStreamUpdateRequest: http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/request/ContentStreamUpdateRequest.html With an example here: http://wiki.apache.org/solr/ContentStreamUpdateRequestExample -- Jack Krupansky -Original Message- From: Ahmet Arslan Sent: Monday, June 04, 2012 7:29 AM To: solr-user@lucene.apache.org Subject: Re: How to use SolrJ index a file or a FileInputStream with some other attributes? How to use SolrJ index a file or a FileInputStream with some other attributes? Please give me some example. Here are two good resources: http://www.lucidimagination.com/blog/2012/02/14/indexing-with-solrj/ http://wiki.apache.org/solr/Solrj
Re: Round Robin concept in distributed Solr
Thanks Erick, While it probably does not have any major impact on performance, I think solely to avoid a single point of failure on the main shard it makes sense to load balance it. Regards, Ryan Tabora On Jun 4, 2012, at 2:51 AM, Erick Erickson [via Lucene] wrote: The main shard has some extra work to do. Namely 1 create the sub-requests 2 collate the results from all the sub-requests (including itself). But this work is generally a small amount of the actual work being done, so it's often unnoticeable. That said, I'd just put all my slaves behind a load-balancer and let that mechanism send the requests to the various slaves if for no other reason than your LB should be able to detect if one of your machines goes down and send requests to slaves still running. I even know of one situation where the users main slave has _no_ index on it, it serves solely to distribute requests/aggregate results Best Erick On Sun, Jun 3, 2012 at 11:57 PM, ratabora [hidden email] wrote: Hey Erick, It looks like the thread you mentioned talks about how to configure the shards parameter in the Solr query. I am more interested in the 'main' shard you query against when you make Solr queries (main shard being the shard you direct the query against, mainshard/select?q=*:*shards=shard1,shard2,shard3) I think Suneel's original question is still unanswered, is it better to use Scenario A or Scenario B? I suppose the 'main' shard is going to create a sub query to the rest of the shards defined in the shard parameter, but I am still wondering if you query the same main shard every time if that is going to have a load/performance impact. Suneel wrote So scenario A (round-robin): query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 2: /solr-shard-2/select?q=dog... shards=shard-1,shard2 query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2 etc. or or scenario B (fixed): query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 2: /solr-shard-1/select?q=dog... shards=shard-1,shard2 query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2 Thank you for any help. Regards, Ryan Tabora -- View this message in context: http://lucene.472066.n3.nabble.com/Round-Robin-concept-in-distributed-Solr-tp3636345p3987494.html Sent from the Solr - User mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Round-Robin-concept-in-distributed-Solr-tp3636345p3987521.html To unsubscribe from Round Robin concept in distributed Solr, click here. NAML - http://ryantabora.com -- View this message in context: http://lucene.472066.n3.nabble.com/Round-Robin-concept-in-distributed-Solr-tp3636345p3987594.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Add HTTP-header from ResponseWriter
There is some commented-out code in SolrDispatchFilter.doFilter:
// add info to http headers
//TODO: See SOLR-232 and SOLR-267.
/*try {
NamedList solrRspHeader = solrRsp.getResponseHeader();
for (int i=0; isolrRspHeader.size(); i++) {
((javax.servlet.http.HttpServletResponse) response).addHeader((Solr-
+ solrRspHeader.getName(i)), String.valueOf(solrRspHeader.getVal(i)));
}
} catch (ClassCastException cce) {
log.log(Level.WARNING, exception adding response header log
information, cce);
}*/
And there is a comment from Grant on SOLR-267 that The changes to
SolrDispatchFilter can screw up SolrJ when you have explicit=all ... so I'm
going to ... comment out #2 and put a TODO: there and someone can address it
on SOLR-232.
I did not see a separate Jira issue for arbitrarily setting HTTP headers
from response writers.
-- Jack Krupansky
-Original Message-
From: Markus Jelsma
Sent: Monday, June 04, 2012 7:10 AM
To: solr-user@lucene.apache.org
Subject: Add HTTP-header from ResponseWriter
Hi,
There has been discussion before on how to add/set a HTTP-header from a
ResponseWriter. That was about adding the number of found documents for a
CSVResponseWriter. We also need to set the number of found documents, in
this case for the JSONResponseWriter. or any ResponseWriter. Is there any
progress or open issue i am not aware of? Can the current (trunk) response
framework already set or add an HTTP-header?
Thanks,
Markus
Can't index sub-entitties in DIH
Hi folks,
I'm using DIH in order to index my Oracle database but I'm having some
issues.
Right now, I can't index fields from sub-entities. I have in my
db-data-config.xml:
dataConfig
dataSource type=JdbcDataSource name=jdbc
driver=oracle.jdbc.OracleDriver url=jdbc:oracle:thin:@localhost:1521:mydata
user=oracle password=oracle /
document
entity name=documento query=SELECT
iddocumento,nrodocumento,tipodocumento FROM documento
field column=iddocumento name=iddocumento /
field column=nrodocumento name=nrodocumento /
field column=tipodocumento name=tipodocumento /
entity name=tipodocumento query=SELECT nombre FROM
tipodocumento where idtipodocumento = '${documento.tipodocumento}'
field column=nombre name=nombre /
/entity
/entity
/document
/dataConfig
My schema.xml is:
field name=iddocumento type=string indexed=true stored=true
required=true /
field name=nrodocumento type=string indexed=true stored=true
/
field name=tipodocumento type=string indexed=true
stored=true /
field name=nombre type=string indexed=true stored=true /
I can't index NOMBRE field. Is this because it belongs to a sub-entity?
Thanks for your help
--
Rafael Taboada
Re: seeing errors during replication process on slave boxes - read past EOF
started a re-index on the master Out of curiosity, how did you do that? Did you delete files or do a deleteQuery for *:* or what? Did the delete step have time to propagate to the slaves before documents were added? I am just curious - I don't know what the proper technique is for initiating a re-index for a master-slave configuration. -- Jack Krupansky -Original Message- From: geeky2 Sent: Sunday, June 03, 2012 9:40 PM To: solr-user@lucene.apache.org Subject: seeing errors during replication process on slave boxes - read past EOF hello all, environment: solr 3.5 1 - master 2 - slave slaves are set to poll master every 10 minutes. i have had replication running on one master and two slaves - for a few weeks now. these boxes are not production boxes - just QA/test boxes. right after i started a re-index on the master - i started to see the following errors on both of the slave boxes. in previous test runs - i have not noticed any errors. can someone help me understand what is causing these errors? thank you, 2012-06-03 19:30:23,104 INFO [org.apache.solr.update.UpdateHandler] (pool-16-thread-1) start commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false) 2012-06-03 19:30:23,164 SEVERE [org.apache.solr.handler.ReplicationHandler] (pool-16-thread-1) SnapPull failed org.apache.solr.common.SolrException: Index fetch failed : at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:331) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:268) at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.RuntimeException: java.io.IOException: read past EOF: MMapIndexInput(path=/appl/solr/stress/partcatalog/index/core1/index.20120514101522/_5kgm.fdx) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1103) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:418) at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:470) at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:321) ... 11 more Caused by: java.io.IOException: read past EOF: MMapIndexInput(path=/appl/solr/stress/partcatalog/index/core1/index.20120514101522/_5kgm.fdx) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte(MMapDirectory.java:279) at org.apache.lucene.store.DataInput.readInt(DataInput.java:84) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.readInt(MMapDirectory.java:315) at org.apache.lucene.index.FieldsReader.init(FieldsReader.java:138) at org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:212) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:117) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93) at org.apache.lucene.index.DirectoryReader.init(DirectoryReader.java:235) at org.apache.lucene.index.ReadOnlyDirectoryReader.init(ReadOnlyDirectoryReader.java:34) at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:506) at org.apache.lucene.index.DirectoryReader.access$000(DirectoryReader.java:45) at org.apache.lucene.index.DirectoryReader$2.doBody(DirectoryReader.java:498) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754) at org.apache.lucene.index.DirectoryReader.doOpenNoWriter(DirectoryReader.java:493) at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:450) at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:396) at org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:520) at org.apache.lucene.index.IndexReader.reopen(IndexReader.java:697) at org.apache.solr.search.SolrIndexReader.reopen(SolrIndexReader.java:414) at org.apache.solr.search.SolrIndexReader.reopen(SolrIndexReader.java:425) at org.apache.solr.search.SolrIndexReader.reopen(SolrIndexReader.java:35) at org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:501) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1083) ... 14 more 2012-06-03 19:30:23,197 INFO
SolrCloud question: recommended technique for initiating a full re-index
If someone has a running SolrCloud with data in the index, what is the recommended technique for initiating a full-re-index? A delete by query for *:*? Deleting data files? Is the Lucene IndexWriter.deleteAll method surfaced somehow in Solr since it is supposed to be much faster than IW.deleteDocuments(“*:*”)? In case case, what is recommended and what is acceptable and supported? Thanks. -- Jack Krupansky
Re: Can't index sub-entitties in DIH
Hi folks,
I've just solved using outer joins like this:
entity name=documento query=SELECT
d.iddocumento,d.nrodocumento,td.idtipodocumento,td.nombre AS
nombretipodocumento FROM documento d LEFT OUTER JOIN tipodocumento td ON
(d.tipodocumento = td.idtipodocumento)
field column=iddocumento name=iddocumento /
field column=nrodocumento name=nrodocumento /
field column=idtipodocumento name=idtipodocumento /
field column=nombretipodocumento name=nombretipodocumento /
Any idea why I can't index using sub-entities?
Thanks in advance
On Mon, Jun 4, 2012 at 11:13 AM, Rafael Taboada
kaliman.fore...@gmail.comwrote:
Hi folks,
I'm using DIH in order to index my Oracle database but I'm having some
issues.
Right now, I can't index fields from sub-entities. I have in my
db-data-config.xml:
dataConfig
dataSource type=JdbcDataSource name=jdbc
driver=oracle.jdbc.OracleDriver
url=jdbc:oracle:thin:@localhost:1521:mydata
user=oracle password=oracle /
document
entity name=documento query=SELECT
iddocumento,nrodocumento,tipodocumento FROM documento
field column=iddocumento name=iddocumento /
field column=nrodocumento name=nrodocumento /
field column=tipodocumento name=tipodocumento /
entity name=tipodocumento query=SELECT nombre FROM
tipodocumento where idtipodocumento = '${documento.tipodocumento}'
field column=nombre name=nombre /
/entity
/entity
/document
/dataConfig
My schema.xml is:
field name=iddocumento type=string indexed=true stored=true
required=true /
field name=nrodocumento type=string indexed=true
stored=true /
field name=tipodocumento type=string indexed=true
stored=true /
field name=nombre type=string indexed=true stored=true /
I can't index NOMBRE field. Is this because it belongs to a sub-entity?
Thanks for your help
--
Rafael Taboada
--
Rafael Taboada
/*
* Phone 992 741 026
*/
Re: I got ERROR, Unable to execute query
My current suspicion is that there is activity in that table that is
preventing DIH access. I mean, like maybe the table is being updated when
DIH is failing. Maybe somebody is emptying the table and then regenerating
it and your DIH run is catching the table when it is being emptied. Or
something like that.
-- Jack Krupansky
-Original Message-
From: Jihyun Suh
Sent: Sunday, June 03, 2012 8:50 PM
To: solr-user-h...@lucene.apache.org ; solr-user@lucene.apache.org
Subject: Re: I got ERROR, Unable to execute query
I read your answer. Thank you.
But I don't get that error from same table. This time I get error from
test_5. but when I try to dataimport again, I can index test_5, but from
test_7 I get that error.
I don't know the reason. Could you help me?
--
Is test_5 created by a stored procedure? If so, is there a possibility that
the stored procedure may have done an update and not returned data - but
just sometimes?
-- Jack Krupansky
2012/6/2 Jihyun Suh jhsuh.ourli...@gmail.com
I use many tables for indexing.
During dataimport, I get errors for some tables like Unable to execute
query. But next time, when I try to dataimport for that table, I can do
successfully without any error.
[Thread-17] ERROR o.a.s.h.d.EntityProcessorWrapper - Exception in entity
:
test_5:org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query:
SELECT Title, url, synonym, description FROM test_5 WHERE status in
('1','s') Processing Document # 11046
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.init(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:596)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
I use many tables for indexing.
During dataimport, I get errors for some tables like Unable to execute
query. But next time, when I try to dataimport for that table, I can do
successfully without any error.
[Thread-17] ERROR o.a.s.h.d.EntityProcessorWrapper - Exception in entity
:
test_5:org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query:
SELECT Title, url, synonym, description FROM test_5 WHERE status in
('1','s') Processing Document # 11046
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.init(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:596)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Re: seeing errors during replication process on slave boxes - read past EOF
hello,
i have shell scripts that handle all of the operational tasks.
example:
curl -v http://${SERVER}.bogus.com:${PORT}/somecore/dataimport -F
command=full-import -F clean=${CLEAN} -F commit=${COMMIT} -F
optimize=${OPTIMIZE}
--
View this message in context:
http://lucene.472066.n3.nabble.com/seeing-errors-during-replication-process-on-slave-boxes-read-past-EOF-tp3987489p3987617.html
Sent from the Solr - User mailing list archive at Nabble.com.
ExtendedDisMax Question - Strange behaviour
I'm doing a query with edismax. When I don't tell solr which fields I want to do the search (so it does in default field), it returns 2752 documents. ex: http://000.000.0.0:/solr/select/?q=apartamento+moema+praia+churrasqueiraversion=2.2start=0rows=10indent=ondefType=dismaxmm=75%25http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueiraversion=2.2start=0rows=10indent=ondefType=dismaxmm=75%25facet=truefacet.field=bairro The same search, defining the fiels that composes the default field, it returns 1434 docs. ex: http://000.000.0.0:/solr/select/?q=apartamento+moema+praia+churrasqueiraversion=2.2start=0rows=10indent=ondefType=dismaxmm=75%25;http://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueiraversion=2.2start=0rows=10indent=ondefType=dismaxmm=75%25facet=truefacet.field=bairroqf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual qf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanualhttp://192.168.20.8:8984/solr/Index/select/?q=apartamento+moema+praia+churrasqueiraversion=2.2start=0rows=10indent=ondefType=dismaxmm=75%25facet=truefacet.field=bairroqf=agrupamentos+agrupamentos2+bairro+campanhalocalempreendimento+caracteristicas+caracteristicacomum+categoria+cep+chamada+cidade+codigoanuncio+complemento+descricaopermuta+docid+empreendimento+endereco+estado+informacoescomplementares+conteudoobservacao+sigla+subtipoimovel+tipoimovel+transacao+zapid+caminhomapa+codigooferta+segmento+anuncianteorigem+zapidcorporativo+estagiodaobra+condicoescomerciais+nomejornal+nomejornalordem+textomanual This is the important part of schema: defaultSearchFieldtextoboost/defaultSearchFieldcopyField source= agrupamentos2 dest=textoboost /copyField source=agrupamentos dest= textoboost /copyField source=bairro dest=textoboost /copyField source=campanhalocalempreendimento dest=textoboost /copyField source= caracteristicas dest=textoboost /copyField source=caracteristicacomum dest=textoboost /copyField source=categoria dest=textoboost / copyField source=cep dest=textoboost /copyField source=chamada dest =textoboost /copyField source=cidade dest=textoboost /copyField source=codigoanuncio dest=textoboost /copyField source=complemento dest=textoboost /copyField source=descricaopermuta dest=textoboost /copyField source=docid dest=textoboost /copyField source= empreendimento dest=textoboost /copyField source=endereco dest= textoboost /copyField source=estado dest=textoboost /copyField source=informacoescomplementares dest=textoboost /copyField source= conteudoobservacao dest=textoboost /copyField source=sigla dest= textoboost /copyField source=subtipoimovel dest=textoboost / copyField source=tipoimovel dest=textoboost /copyField source= transacao dest=textoboost /copyField source=zapid dest=textoboost /copyField source=caminhomapa dest=textoboost /copyField source= codigooferta dest=textoboost /copyField source=segmento dest= textoboost /copyField source=anuncianteorigem dest=textoboost / copyField source=zapidcorporativo dest=textoboost /copyField source= estagiodaobra dest=textoboost /copyField source=condicoescomerciais dest=textoboost /copyField source=nomejornal dest=textoboost / copyField source=nomejornalordem dest=textoboost / copyField source= textomanual dest=textoboost / What's the problem? Thank's * -- * *E conhecereis a verdade, e a verdade vos libertará. (João 8:32)* *andre.maldonado*@gmail.com andre.maldon...@gmail.com (11) 9112-4227 http://www.orkut.com.br/Main#Profile?uid=2397703412199036664 http://www.orkut.com.br/Main#Profile?uid=2397703412199036664 http://www.facebook.com/profile.php?id=10659376883 http://twitter.com/andremaldonado http://www.delicious.com/andre.maldonado https://profiles.google.com/105605760943701739931 http://www.linkedin.com/pub/andr%C3%A9-maldonado/23/234/4b3 http://www.youtube.com/andremaldonado
Re: Can't index sub-entitties in DIH
On 4 June 2012 22:27, Rafael Taboada kaliman.fore...@gmail.com wrote: Hi folks, I've just solved using outer joins like this: entity name=documento query=SELECT d.iddocumento,d.nrodocumento,td.idtipodocumento,td.nombre AS nombretipodocumento FROM documento d LEFT OUTER JOIN tipodocumento td ON (d.tipodocumento = td.idtipodocumento) field column=iddocumento name=iddocumento / field column=nrodocumento name=nrodocumento / field column=idtipodocumento name=idtipodocumento / field column=nombretipodocumento name=nombretipodocumento / Any idea why I can't index using sub-entities? [...] Dear Rafael, No offence meant, but please take a look at http://wiki.apache.org/solr/UsingMailingLists This is by no means meant to discourage you from asking questions on this list, but to request due diligence from your side so that people can help you better. In this particular case, given that you have changed your top-level SELECT, you might wish to inform people about whether you have made corresponding changes for sub-entities. Have you tried out the various SELECT statements in the entity, and sub-entity, outside of Solr, to ensure that they provide results? Again, with due respect, it is unlikely that a now-mature product like Solr has issues like what you have been asking about, e.g.: - Mapping column names to Solr field names - Having sub-entities indexed properly. At least from personal experience, we have worked with both of these use cases without any issues. Thankfully, Oracle was not involved in our processes, but it would be difficult to imagine that there is an issue with Oracle in this regard. Regards, Gora
Re: pass custom parameters from client to solr
Thanks for your reply, that was helpful. Is there any restriction on how many parameters i can pass ? also can i pass some data (as we pass the document to be indexed, when we call the update url) along with the parameters which the solr server will use for search. -- View this message in context: http://lucene.472066.n3.nabble.com/pass-custom-parameters-from-client-to-solr-tp3987511p3987625.html Sent from the Solr - User mailing list archive at Nabble.com.
Sorting performance
Here is the usecase: I am using synonym expansion at query time to get results. this is essentially a name search, so a search for Jim may be expanded at query time for James, Jung, Jimmy, etc. So ranking fields like TF, IDF, Norms do not mean anything to me. I just reset them to zero. so all the results which I get have the same rank. I have used a copy field to boost the weights of exact match, so Jim would be boosted to the top. However I want the other results like Jimmy, Jung, James to be sorted by Levenstein Distance with respect to word Jim (the original query). The number of results returned are quite large. So a genereal strdist sort takes 6-7 seconds. Is there any other option than applying a sort= in the query to achieve the same functionality? Any particular way to index the data to achieve the same result? any idea to boost the performance and get the intended functionality? -- View this message in context: http://lucene.472066.n3.nabble.com/Sorting-performance-tp3987632.html Sent from the Solr - User mailing list archive at Nabble.com.
Sorting performance
Here is the usecase: I am using synonym expansion at query time to get results. this is essentially a name search, so a search for Jim may be expanded at query time for James, Jung, Jimmy, etc. So ranking fields like TF, IDF, Norms do not mean anything to me. I just reset them to zero. so all the results which I get have the same rank. I have used a copy field to boost the weights of exact match, so Jim would be boosted to the top. However I want the other results like Jimmy, Jung, James to be sorted by Levenstein Distance with respect to word Jim (the original query). The number of results returned are quite large. So a genereal strdist sort takes 6-7 seconds. Is there any other option than applying a sort= in the query to achieve the same functionality? Any particular way to index the data to achieve the same result? any idea to boost the performance and get the intended functionality? -- View this message in context: http://lucene.472066.n3.nabble.com/Sorting-performance-tp3987633.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: pass custom parameters from client to solr
Thanks for your reply, that was helpful. Is there any restriction on how many parameters i can pass ? I think there are some max limits on GET and POST methods. You can use both using solrJ. POST method enables longer parameters. http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/SolrRequest.METHOD.html Also you may need to tweak your servlet container, for example: http://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests
Unable to delete by range in SOLR
I have an instance of SOLR 3.6 running, with JSON as the default
updateHandler.
I am able to delete individual documents with the following:
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:1730887464}}'
What is the right way to delete a range of IDs?
I have tried the following:
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:[* TO 1730887464]}}'
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:[0 TO 1730887464]}}'
And I get the following exception. Any help/pointers would be appreciated.
I've looked up:
http://wiki.apache.org/solr/UpdateJSON
http://lucene.apache.org/solr/api/doc-files/tutorial.html
http://wiki.apache.org/solr/UpdateXmlMessages
html
head
meta http-equiv=Content-Type content=text/html;
charset=ISO-8859-1/
titleError 500 For input string: [* TO 1730887464]
java.lang.NumberFormatException: For input string: [* TO 1730887464]
at
java.lang.NumberFormatException.forInputString(NumberFormatException.
java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:295
)
at
org.apache.solr.schema.TrieField.toInternal(TrieField.java:307)
at
org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandle
r2.java:288)
at
org.apache.solr.update.processor.RunUpdateProcessor.processDelete(Run
UpdateProcessorFactory.java:68)
at
org.apache.solr.update.processor.LogUpdateProcessor.processDelete(Log
UpdateProcessorFactory.java:137)
at
org.apache.solr.handler.JsonLoader.processUpdate(JsonLoader.java:120)
at org.apache.solr.handler.JsonLoader.load(JsonLoader.java:75)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
ntentStreamHandlerBase.java:58)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle
Request(RequestHandlers.java:244)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter
.java:365)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:260)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
99)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
66)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand
lerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.
Re: Unable to delete by range in SOLR
Delete by query.
See:
http://wiki.apache.org/solr/UpdateJSON
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {query:[* TO
1730887464]}}'
-- Jack Krupansky
-Original Message-
From: Joe
Sent: Monday, June 04, 2012 5:12 PM
To: solr-user@lucene.apache.org
Subject: Unable to delete by range in SOLR
I have an instance of SOLR 3.6 running, with JSON as the default
updateHandler.
I am able to delete individual documents with the following:
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:1730887464}}'
What is the right way to delete a range of IDs?
I have tried the following:
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:[* TO 1730887464]}}'
curl http://myURL/update?commit=true; -H
'Content-type:application/json' -d '{delete: {id:[0 TO 1730887464]}}'
And I get the following exception. Any help/pointers would be appreciated.
I've looked up:
http://wiki.apache.org/solr/UpdateJSON
http://lucene.apache.org/solr/api/doc-files/tutorial.html
http://wiki.apache.org/solr/UpdateXmlMessages
html
head
meta http-equiv=Content-Type content=text/html;
charset=ISO-8859-1/
titleError 500 For input string: [* TO 1730887464]
java.lang.NumberFormatException: For input string: [* TO 1730887464]
at
java.lang.NumberFormatException.forInputString(NumberFormatException.
java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:295
)
at
org.apache.solr.schema.TrieField.toInternal(TrieField.java:307)
at
org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandle
r2.java:288)
at
org.apache.solr.update.processor.RunUpdateProcessor.processDelete(Run
UpdateProcessorFactory.java:68)
at
org.apache.solr.update.processor.LogUpdateProcessor.processDelete(Log
UpdateProcessorFactory.java:137)
at
org.apache.solr.handler.JsonLoader.processUpdate(JsonLoader.java:120)
at org.apache.solr.handler.JsonLoader.load(JsonLoader.java:75)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
ntentStreamHandlerBase.java:58)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle
Request(RequestHandlers.java:244)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter
.java:365)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:260)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
99)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
66)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand
lerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.
java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
52)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54
2)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnectio
n.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.
java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.j
ava:582)
/title
/head
body
HTTP ERROR 500
pProblem accessing /solr/update. Reason:
preFor input string: [* TO 1730887464]
java.lang.NumberFormatException: For input string: [* TO 1730887464]
at
java.lang.NumberFormatException.forInputString(NumberFormatException.
java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:295
)
at
org.apache.solr.schema.TrieField.toInternal(TrieField.java:307)
at
org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandle
r2.java:288)
at
Re: Fuzzy Suggester
usePrefix value is set true in JaspellLookup class. This value needs to be
set to false in order to enable matchAlmost method which I believe performs
fuzzy search.
@Override
public ListLookupResult lookup(String key, boolean onlyMorePopular, int
num) {
ListLookupResult res = new ArrayListLookupResult();
ListString list;
int count = onlyMorePopular ? num * 2 : num;
if (usePrefix) {
list = trie.matchPrefix(key, count);
} else {
list = trie.matchAlmost(key, count);
}
--
View this message in context:
http://lucene.472066.n3.nabble.com/Fuzzy-Suggester-tp3355111p3987674.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unable to delete by range in SOLR
Thanks for the response Jack! That wasn't exactly right. But the following
modification does work:
curl http://myURL/update?commit=true; -H 'Content-type:application/json' -d
'{delete: {query:id:[0 TO 1730887464]}}'
--
View this message in context:
http://lucene.472066.n3.nabble.com/Unable-to-delete-by-range-in-SOLR-tp3987664p3987679.html
Sent from the Solr - User mailing list archive at Nabble.com.
Correct way to deal with source data that may include a multivalued field that needs to be used for sorting?
Greetings, I have dirty source data where some documents being indexed, although unlikely, may contain multivalued fields that are also required for sorting. In previous versions of Solr, sorting on this field worked fine (possibly because few or no multivalued fields were ever encountered?), however, as of 3.6.0, thanks to https://issues.apache.org/jira/browse/SOLR-2339 attempting to sort on this field now throws an error: [2012-06-04 17:20:01,691] ERROR org.apache.solr.common.SolrException org.apache.solr.common.SolrException: can not sort on multivalued field: f_normalizedValue The relevant bits of the schema.xml are: fieldType name=sfloat class=solr.TrieFloatField precisionStep=0 positionIncrementGap=0 sortMissingLast=true/ dynamicField name=f_* type=sfloat indexed=true stored=true required=false multiValued=true/ Assuming that the source documents being indexed cannot be changed (which, at least for now, they cannot), what would be the next best way to allow for both the possibility of multiple f_normalizedValue fields appearing in indexed documents, as wel as being able to sort by f_normalizedValue? Thank you, Aaron
Re: Can't index sub-entitties in DIH
Hi Gora.
My apologizes for this subject. It wasn't my intention to be rude or
something like that.
As I said about mapping database columns with solr fields, it works in
mysql. Maybe I'm wrong with some config file because I'm new with Solr. By
the way, it is a great product.
But I've just tried about sub-entities using mysql. And all work perfect.
It shows me that I am doing something wrong with Oracle database at work.
I attach with this mail files I used with mysql. Mapping columns is OK,
sub-entities is OK.
Thanks so much for your opinion.
Kinds regards
Rafael
db-data-config.xml
dataConfig
dataSource driver=com.mysql.jdbc.Driver
url=jdbc:mysql://localhost:3306/solr user=solr password=solr /
document
entity name=usuario query=select idusuario,nombres,apellidos
from usuario
field column=idusuario name=idusuario /
field column=nombres name=nombres111 /
field column=apellidos name=apellidos111 /
entity name=documento query=select iddocumento,nombre
from documento where usuario_idusuario = '${usuario.idusuario}'
field column=iddocumento name=iddocumento /
field column=nombre name=nombredocumento /
/entity
/entity
/document
/dataConfig
schema.xml
?xml version=1.0 encoding=UTF-8 ?
schema name=db version=1.1
types
fieldType name=string class=solr.StrField sortMissingLast=true
omitNorms=true/
fieldType name=sint class=solr.SortableIntField
sortMissingLast=true omitNorms=true/
/types
fields
field name=idusuario type=sint indexed=true stored=true
required=true /
field name=nombres111 type=string indexed=true stored=true /
field name=apellidos111 type=string indexed=true stored=true/
field name=iddocumento type=string indexed=true stored=true/
field name=nombredocumento type=string indexed=true
stored=true/
/fields
!-- Field to use to determine and enforce document uniqueness.
Unless this field is marked with required=false, it will be a
required field
--
uniqueKeyidusuario/uniqueKey
!-- field for the QueryParser to use when an explicit fieldname is absent
--
defaultSearchFieldnombres111/defaultSearchField
!-- SolrQueryParser configuration: defaultOperator=AND|OR --
solrQueryParser defaultOperator=OR/
/schema
solrconfig.xml
?xml version=1.0 encoding=UTF-8 ?
config
luceneMatchVersionLUCENE_36/luceneMatchVersion
!-- The DirectoryFactory to use for indexes.
solr.StandardDirectoryFactory, the default, is filesystem based.
solr.RAMDirectoryFactory is memory based, not persistent, and
doesn't work with replication. --
directoryFactory name=DirectoryFactory
class=${solr.directoryFactory:solr.StandardDirectoryFactory}/
updateHandler class=solr.DirectUpdateHandler2 /
requestDispatcher handleSelect=true
requestParsers enableRemoteStreaming=false
multipartUploadLimitInKB=2048 /
/requestDispatcher
requestHandler name=standard class=solr.StandardRequestHandler
default=true /
requestHandler name=/update class=solr.XmlUpdateRequestHandler /
requestHandler name=/admin/
class=org.apache.solr.handler.admin.AdminHandlers /
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdb-data-config.xml/str
/lst
/requestHandler
!-- config for the admin interface --
admin
defaultQuerysolr/defaultQuery
/admin
/config
On Mon, Jun 4, 2012 at 12:31 PM, Gora Mohanty g...@mimirtech.com wrote:
On 4 June 2012 22:27, Rafael Taboada kaliman.fore...@gmail.com wrote:
Hi folks,
I've just solved using outer joins like this:
entity name=documento query=SELECT
d.iddocumento,d.nrodocumento,td.idtipodocumento,td.nombre AS
nombretipodocumento FROM documento d LEFT OUTER JOIN tipodocumento td ON
(d.tipodocumento = td.idtipodocumento)
field column=iddocumento name=iddocumento /
field column=nrodocumento name=nrodocumento /
field column=idtipodocumento name=idtipodocumento /
field column=nombretipodocumento name=nombretipodocumento /
Any idea why I can't index using sub-entities?
[...]
Dear Rafael,
No offence meant, but please take a look at
http://wiki.apache.org/solr/UsingMailingLists
This is by no means meant to discourage you
from asking questions on this list, but to
request due diligence from your side so that
people can help you better.
In this particular case, given that you have
changed your top-level SELECT, you might
wish to inform people about whether you have
made corresponding changes for sub-entities.
Have you tried out the various SELECT
statements in the entity, and sub-entity,
outside of Solr, to ensure that they provide
results?
Again, with due respect, it is unlikely that
a now-mature product like Solr has issues
like what you have been asking about, e.g.:
- Mapping column names to Solr field
names
- Having sub-entities indexed
Re: I got ERROR, Unable to execute query
That's why I made a new DB for dataimport test. So my tables have no access
or activity.
Those are just dormant ones.
--
My current suspicion is that there is activity in that table that is
preventing DIH access. I mean, like maybe the table is being updated when
DIH is failing. Maybe somebody is emptying the table and then regenerating
it and your DIH run is catching the table when it is being emptied. Or
something like that.
-- Jack Krupansky
2012/6/4 Jihyun Suh jhsuh.ourli...@gmail.com
I read your answer. Thank you.
But I don't get that error from same table. This time I get error from
test_5. but when I try to dataimport again, I can index test_5, but from
test_7 I get that error.
I don't know the reason. Could you help me?
--
Is test_5 created by a stored procedure? If so, is there a possibility
that
the stored procedure may have done an update and not returned data - but
just sometimes?
-- Jack Krupansky
2012/6/2 Jihyun Suh jhsuh.ourli...@gmail.com
I use many tables for indexing.
During dataimport, I get errors for some tables like Unable to execute
query. But next time, when I try to dataimport for that table, I can do
successfully without any error.
[Thread-17] ERROR o.a.s.h.d.EntityProcessorWrapper - Exception in entity
:
test_5:org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query:
SELECT Title, url, synonym, description FROM test_5 WHERE status in
('1','s') Processing Document # 11046
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.init(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:596)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
I use many tables for indexing.
During dataimport, I get errors for some tables like Unable to execute
query. But next time, when I try to dataimport for that table, I can do
successfully without any error.
[Thread-17] ERROR o.a.s.h.d.EntityProcessorWrapper - Exception in entity
:
test_5:org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query:
SELECT Title, url, synonym, description FROM test_5 WHERE status in
('1','s') Processing Document # 11046
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.init(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:596)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Re: Can't index sub-entitties in DIH
On 5 June 2012 08:26, Rafael Taboada kaliman.fore...@gmail.com wrote:
Hi Gora.
My apologizes for this subject. It wasn't my intention to be rude or
something like that.
No need to apologize. I was hoping to make it easier
for people to help you.
As I said about mapping database columns with solr fields, it works in
mysql. Maybe I'm wrong with some config file because I'm new with Solr. By
the way, it is a great product.
But I've just tried about sub-entities using mysql. And all work perfect.
It shows me that I am doing something wrong with Oracle database at work.
I attach with this mail files I used with mysql. Mapping columns is OK,
sub-entities is OK.
[...]
Your configuration files look fine. It would seem that something
is going wrong with the SELECT in Oracle, or with the JDBC
driver used to access Oracle. Could you try:
* Manually doing the SELECT for the entity, and sub-entity
to ensure that things are working.
* Check the JDBC settings.
Sorry, I do not have access to Oracle so that I cannot try this
out myself.
Also, have you checked the Solr logs for any error messages?
Finally, I just noticed that you have extra quotes in:
...where usuario_idusuario = '${usuario.idusuario}'
I doubt that is the cause of your problem, but you could try
removing them.
Regards,
Gora
