Re: solrDocumentList
Set those fields to be single valued (not multivalued). And reindex :)
Erik
On Jun 2, 2012, at 0:10, gopes saraladevi.ramamoor...@gmail.com wrote:
We are using Lucid UI and solr to index our collection of xml files. I am
getting the solrDocumentList like this
[SolrDocument[{id=1331226833510, Street_Addr=[113 113TH ST], name=[113 113TH
ST SASKATOON SK S7N1V8], Municipality_Name=[SASKATOON], Province_Code=[SK],
Postal_Code=[S7N1V8]}]
But I need to have a response like this SolrDocument[{id=1330247542287,
Municipality_Name=SASKATOON, Province_Code=SK, Postal_Code=S7N3Z1,
Street_Addr=3-THE BROADWAY UNIVERSITY DR}]
Can any one help me where I am going wrong or is there any changes required
in the schema.xml files.
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/solrDocumentList-tp3987347.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Stop Words in SpellCheckComponent
Also, generally, you should have a separate field and field type for the spellcheck field **so that normal text fields can use stop words.** Now I've found a solution, although I'm not sure, if it's that what you've meant: Now I'm using a special fieldType WITHOUT stopwords for the spellcheck field. So - I think - the SpellCheckComponent doesn't find better matches for stopwords, because it has indexed the stopwords itself. Thanks for your help Matthias schema.xml . fieldType name=spellcheckType class=solr.TextField positionIncrementGap=100 analyzer tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType field name=spellcheckField type=spellcheckType indexed=true stored=false/ solrconfig.xml . searchComponent name=spellcheck class=solr.SpellCheckComponent str name=queryAnalyzerFieldTypetextSpell/str lst name=spellchecker str name=namedefault/str str name=fieldspellcheckField/str
Re: Solr Performance
Jack ,its not from Chris. --Surendra
creating SchemaField and FieldType programmatically
I'm creating a some Solr plugins that index and search documents in a
special way, and I'd like to make them as easy as possible to
configure. Ideally I'd like users to be able to just drop a jar in
place without having to copy any configuration into schema.xml, although
I suppose they will have to register the plugins in solrconfig.xml.
I tried making my UpdateProcessor core aware and creating FieldTypes
and SchemaFields in the inform(SolrCore) method. This was a good start,
but I'm running into some issues getting the types properly
initialized. One of my types, for example, derives from TextField, but
this seems to require an initialization pass in order to get its
properties set up properly. What I'm seeing is that my field values
aren't being tokenized, even though I specify TOKENIZED when I create
the SchemaField. I'm beginning to get the feeling I'm doing something
not-quite anticipated by the API designers.
My question is: is there a way to go about doing something like this
that isn't swimming upstream? Should I just give up and require users
to incorporate my schema in the xml config?
Here is a code snippet for anyone willing to dig in a little:
/** Called when each core is initialized; we ensure that lux fields
are configured. */
public void inform(SolrCore core) {
IndexSchema schema = core.getSchema();
MapString,SchemaField fields = schema.getFields();
if (fields.containsKey(lux_path)) {
return;
}
MapString,FieldType fieldTypes = schema.getFieldTypes();
FieldType luxTextWs = fieldTypes.get(lux_text_ws);
if (luxTextWs == null) {
luxTextWs = new TextField ();
luxTextWs.setAnalyzer(new WhitespaceGapAnalyzer());
luxTextWs.setQueryAnalyzer(new WhitespaceGapAnalyzer());
fieldTypes.put(lux_text_ws, luxTextWs);
}
fields.put(lux_path, new SchemaField (lux_path, luxTextWs,
0x233, )); // 0x233 = INDEXED | TOKENIZED | OMIT_NORMS |
OMIT_TF_POSITIONS | MULTIVALUED
fields.put(lux_elt_name, new SchemaField (lux_elt_name, new
StrField(), 0x231, ));// INDEXED | OMIT_NORMS | OMIT_TF_POSITIONS |
MULTIVALUED
fields.put(lux_att_name, new SchemaField (lux_att_name, new
StrField(), 0x231, ));
// must call this after making changes to the field map:
schema.refreshAnalyzers();
}
Re: creating SchemaField and FieldType programmatically
ok, never mind all is well - I had a mismatch between the
schema-declared field and my programmatic field, where I was overzealous
in using OMIT_TF_POSITIONS.
-Mike
On 6/2/2012 5:02 PM, Mike Sokolov wrote:
I'm creating a some Solr plugins that index and search documents in a
special way, and I'd like to make them as easy as possible to
configure. Ideally I'd like users to be able to just drop a jar in
place without having to copy any configuration into schema.xml,
although I suppose they will have to register the plugins in
solrconfig.xml.
I tried making my UpdateProcessor core aware and creating FieldTypes
and SchemaFields in the inform(SolrCore) method. This was a good
start, but I'm running into some issues getting the types properly
initialized. One of my types, for example, derives from TextField,
but this seems to require an initialization pass in order to get its
properties set up properly. What I'm seeing is that my field values
aren't being tokenized, even though I specify TOKENIZED when I create
the SchemaField. I'm beginning to get the feeling I'm doing something
not-quite anticipated by the API designers.
My question is: is there a way to go about doing something like this
that isn't swimming upstream? Should I just give up and require users
to incorporate my schema in the xml config?
Here is a code snippet for anyone willing to dig in a little:
/** Called when each core is initialized; we ensure that lux
fields are configured. */
public void inform(SolrCore core) {
IndexSchema schema = core.getSchema();
MapString,SchemaField fields = schema.getFields();
if (fields.containsKey(lux_path)) {
return;
}
MapString,FieldType fieldTypes = schema.getFieldTypes();
FieldType luxTextWs = fieldTypes.get(lux_text_ws);
if (luxTextWs == null) {
luxTextWs = new TextField ();
luxTextWs.setAnalyzer(new WhitespaceGapAnalyzer());
luxTextWs.setQueryAnalyzer(new WhitespaceGapAnalyzer());
fieldTypes.put(lux_text_ws, luxTextWs);
}
fields.put(lux_path, new SchemaField (lux_path, luxTextWs,
0x233, )); // 0x233 = INDEXED | TOKENIZED | OMIT_NORMS |
OMIT_TF_POSITIONS | MULTIVALUED
fields.put(lux_elt_name, new SchemaField (lux_elt_name,
new StrField(), 0x231, ));// INDEXED | OMIT_NORMS |
OMIT_TF_POSITIONS | MULTIVALUED
fields.put(lux_att_name, new SchemaField (lux_att_name,
new StrField(), 0x231, ));
// must call this after making changes to the field map:
schema.refreshAnalyzers();
}
Re: creating SchemaField and FieldType programmatically
Oh yes, final followup for the terminally curious; I also had to add
this little class in order to get analysis turned on for my programmatic
field:
class PathField extends TextField {
PathField (IndexSchema schema) {
setAnalyzer(new WhitespaceGapAnalyzer());
setQueryAnalyzer(new WhitespaceGapAnalyzer());
}
protected Field.Index getFieldIndex(SchemaField field, String
internalVal) {
return Field.Index.ANALYZED;
}
}
On 6/2/2012 5:48 PM, Mike Sokolov wrote:
ok, never mind all is well - I had a mismatch between the
schema-declared field and my programmatic field, where I was
overzealous in using OMIT_TF_POSITIONS.
-Mike
On 6/2/2012 5:02 PM, Mike Sokolov wrote:
I'm creating a some Solr plugins that index and search documents in a
special way, and I'd like to make them as easy as possible to
configure. Ideally I'd like users to be able to just drop a jar in
place without having to copy any configuration into schema.xml,
although I suppose they will have to register the plugins in
solrconfig.xml.
I tried making my UpdateProcessor core aware and creating
FieldTypes and SchemaFields in the inform(SolrCore) method. This was
a good start, but I'm running into some issues getting the types
properly initialized. One of my types, for example, derives from
TextField, but this seems to require an initialization pass in order
to get its properties set up properly. What I'm seeing is that my
field values aren't being tokenized, even though I specify TOKENIZED
when I create the SchemaField. I'm beginning to get the feeling I'm
doing something not-quite anticipated by the API designers.
My question is: is there a way to go about doing something like this
that isn't swimming upstream? Should I just give up and require
users to incorporate my schema in the xml config?
Here is a code snippet for anyone willing to dig in a little:
/** Called when each core is initialized; we ensure that lux
fields are configured. */
public void inform(SolrCore core) {
IndexSchema schema = core.getSchema();
MapString,SchemaField fields = schema.getFields();
if (fields.containsKey(lux_path)) {
return;
}
MapString,FieldType fieldTypes = schema.getFieldTypes();
FieldType luxTextWs = fieldTypes.get(lux_text_ws);
if (luxTextWs == null) {
luxTextWs = new TextField ();
luxTextWs.setAnalyzer(new WhitespaceGapAnalyzer());
luxTextWs.setQueryAnalyzer(new WhitespaceGapAnalyzer());
fieldTypes.put(lux_text_ws, luxTextWs);
}
fields.put(lux_path, new SchemaField (lux_path,
luxTextWs, 0x233, )); // 0x233 = INDEXED | TOKENIZED | OMIT_NORMS |
OMIT_TF_POSITIONS | MULTIVALUED
fields.put(lux_elt_name, new SchemaField (lux_elt_name,
new StrField(), 0x231, ));// INDEXED | OMIT_NORMS |
OMIT_TF_POSITIONS | MULTIVALUED
fields.put(lux_att_name, new SchemaField (lux_att_name,
new StrField(), 0x231, ));
// must call this after making changes to the field map:
schema.refreshAnalyzers();
}
London OSS search social - meetup 6th June
Apologies for the short notice guys, we're meeting up at The Plough in Bloomsbury on Wednesday 6th June. As usual the format is open and there's a healthy mix of experience and backgrounds. Please come and share wisdom, ask questions, geek out, etc. in the presence of beverages. -- Richard Marr
Distance Range Filtering
Hi everyone
I am trying to do distance range search using Solr.
I know its very easy to do a search for filtering within the 5km range
/q=*:*fq={!geofilt pt=45.15,-93.85 sfield=store d=5}/
What I am after is how to do the same thing if I am looking in a range of
say *5 to 10 km* ??
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/Distance-Range-Filtering-tp3987395.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Using Data Import Handler to invoke a stored procedure with output (cursor) parameter
Right, or create a view.
On Fri, Jun 1, 2012 at 8:11 PM, Michael Della Bitta
michael.della.bi...@appinions.com wrote:
Apologies for the terseness of this reply, as I'm on my mobile.
To treat the result of a function call as a table in Oracle SQL, use the
table() function, like this:
select * from table(my_stored_func())
HTH,
Michael
On Jun 1, 2012 8:01 PM, Niran Fajemisin afa...@yahoo.com wrote:
So I was able to run some additional tests today on this. I tried to use a
stored function instead of a stored procedure. The hope was that the Stored
Function would simply be a wrapper for the Store Procedure and would simply
return the cursor as the return value. This unfortunately did not work.
My test attempted to call the function from the query attribute of the
entity tag as such:
{call my_stored_func()}
It raised an error stating that: 'my_stored_func' is not a procedure or is
undefined. This makes sense because the invocation format above is
customarily reserved for a stored procedure.
So then I tried the typical approach for invoking a function which would
be:
{call ? := my_stored_function()}
And as expected this resulted in an error stating that: not all variables
bound . Again, this is expected as the ? notation would be the
placeholder parameter that would be bound to the OracleTypes.CURSOR
constant in a typical JDBC program.
Note that this function has been tested outside of DIH and it works when
properly invoked.
I think the bottom-line here is that there is no proper support for stored
procedures (or functions for that matter) in DIH. This is really
unfortunate because anyone thinking of doing any significant processing in
the source RDBMS prior to data export would have to look elsewhere. Short
of adding this functionality to the JdbcDataSource class of the DIH, I
think I'm at a dead end.
If anyone knows of any alternatives I would greatly appreciate hearing
them.
Thanks for the responses as usual.
Cheers.
From: Lance Norskog goks...@gmail.com
To: solr-user@lucene.apache.org; Niran Fajemisin afa...@yahoo.com
Sent: Thursday, May 31, 2012 3:09 PM
Subject: Re: Using Data Import Handler to invoke a stored procedure with
output (cursor) parameter
Can you add a new stored procedure that uses your current one? It
would operate like the DIH expects.
I don't remember if DB cursors are a standard part of JDBC. If they
are, it would be a great addition to the DIH if they work right.
On Thu, May 31, 2012 at 10:44 AM, Niran Fajemisin afa...@yahoo.com
wrote:
Thanks for your response, Michael. Unfortunately changing the stored
procedure is not really an option here.
From what I'm seeing, it would appear that there's really no way of
somehow instructing the Data Import Handler to get a handle on the output
parameter from the stored procedure. It's a bit surprising though that no
one has ran into this scenario but I suppose most people just work around
it.
Anyone else care to shed some more light on alternative approaches?
Thanks again.
From: Michael Della Bitta michael.della.bi...@appinions.com
To: solr-user@lucene.apache.org
Sent: Thursday, May 31, 2012 9:40 AM
Subject: Re: Using Data Import Handler to invoke a stored procedure
with output (cursor) parameter
I could be wrong about this, but Oracle has a table() function that I
believe turns the output of a function as a table. So possibly you
could wrap your procedure in a function that returns the cursor, or
convert the procedure to a function.
Michael Della Bitta
Appinions, Inc. -- Where Influence Isn’t a Game.
http://www.appinions.com
On Thu, May 31, 2012 at 8:00 AM, Niran Fajemisin afa...@yahoo.com
wrote:
Hi all,
I've seen a few questions asked around invoking stored procedures
from within Data Import Handler but none of them seem to indicate what type
of output parameters were being used.
I have a stored procedure created in Oracle database that takes a
couple input parameters and has an output parameter that is a reference
cursor. The cursor is expected to be used as a way of iterating through the
returned table rows. I'm using the following format to invoke my stored
procedure in the Data Import Handler's data config XML:
entity name=entity_name ... query={call my_stored_proc(inParam1,
inParam2)} .../entity
I have tested that this query works prior to attempting to use it
from within the DIH. But when I attempt to invoke this stored procedure, it
naturally complains that the output parameter is not specified (essentially
a mismatch in the number of parameters).
I don't know of anyway to pass in a cursor parameter (or any output
parameter for that matter) to the stored procedure invocation from within
the entity definition. I would greatly appreciate if anyone could
provide any pointers
Re: why DIH works in normal mode,error in debug mode
Search and ye shall find! http://www.lucidimagination.com/search/link?url=http://wiki.apache.org/solr/DataImportHandlerFaq On Fri, Jun 1, 2012 at 8:15 AM, Dyer, James james.d...@ingrambook.com wrote: Try setting it to 0 or -1. Or check the Mysql JDBC driver documentation about valid values for Statement.setFetchSize() I think someone else recently asked on this same list about problems with the latest Mysql driver and fetch sizes, so this driver may be particularly finicky. James Dyer E-Commerce Systems Ingram Content Group (615) 213-4311 -Original Message- From: wangjing [mailto:ppm10...@gmail.com] Sent: Friday, June 01, 2012 10:00 AM To: solr-user@lucene.apache.org Subject: Re: why DIH works in normal mode,error in debug mode In my datasource config file: dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://127.0.0.1:3306/MYSOLR?useUnicode=trueamp;characterEncoding=UTF-8 user=root password=qwertyuiop batchSize=500 / i have done it,set batchSize=500 On Fri, Jun 1, 2012 at 10:38 PM, Dyer, James james.d...@ingrambook.com wrote: I see this in your stacktrace: java.sql.SQLException: Illegal value for setFetchSize(). It must be that your JDBC driver doesn't like the default value (300) that is used. In your datasource tag, try adding a batchSize attribute of either 0 or -1 (if using -1, DIH automatically changes it to Integer.MIN_VALUE. According to the wiki this is to fix this error.) The value of batchSize is used on the java.sql.Statement objects with setFetchSize(batchSize). example: dataSource ... batchSize=0 / see: http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource James Dyer E-Commerce Systems Ingram Content Group (615) 213-4311 -Original Message- From: wangjing [mailto:ppm10...@gmail.com] Sent: Friday, June 01, 2012 9:18 AM To: solr-user@lucene.apache.org Subject: why DIH works in normal mode,error in debug mode why DIH works in normal mode,use http://localhost:8080/apache-solr-3.6.0/cn/admin/ query *:* it can find all 0 35 0 2.2 *:* on 10 测试类型1 台灯是人们生活中用来照明的一种家用电器。它一般分为两种，一种是立柱式的，一种是有夹子的。它的工作原理主要是把灯光集中在一小块区域内，便于工作和学习。一般台灯用的灯泡是白炽灯或者节能灯泡。 有的台灯还有应急功能，用于停电时无电照明已用来应急。 包含灯泡啥的 1 2012-06-01T06:15:58Z 台灯 法国台灯A234 3 100.2 100.2,USD 包含灯泡啥的法国台灯A234台灯 台灯是人们生活中用来照明的一种家用电器。它一般分为两种，一种是立柱式的，一种是有夹子的。它的工作原理主要是把灯光集中在一小块区域内，便于工作和学习。一般台灯用的灯泡是白炽灯或者节能灯泡。 有的台灯还有应急功能，用于停电时无电照明已用来应急。 10.0 BUT ERROR in debug mode,it boring me few days :_( the detail exception statck is 2012-6-1 21:51:51 org.apache.solr.common.SolrException log : Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: select * from ITEM; Processing Document # 1 at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:205) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.RuntimeException:
