Re: ' invisible ' words

[deniz]

Thank you Lance, but that German/Germany thing was just an example... I have
some other problems too... but well first will try your advice

-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/invisible-words-tp3158060p3161365.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: fst must be non null

[Robert Muir]

It just means in your schema.xml, you have a SynonymFilterFactory
configured, but it has no actual active rules (e.g. synonyms.txt is
empty or all commented out).

the reason for the warning is that its a little strange to declare to
use a synonymfilterfactory in the schema that doesn't actually have
any synonyms, so it could indicate a configuration mistake.


On Tue, Jul 12, 2011 at 12:02 AM, Stuart King  wrote:
> Sorry Robert,
>
> What does that mean? Should I be providing synonyms in my queries?
>
> Cheers
>
> Stu
>
> On Tue, Jul 12, 2011 at 1:49 PM, Robert Muir  wrote:
>
>> I just committed a fix for this, to warn that you are using an empty
>> set of synonyms instead of error.
>>
>> On Mon, Jul 11, 2011 at 10:50 PM, Stuart King  wrote:
>> > I have been building and running against trunk. In my build I have a
>> number
>> > of tests, testing solr functionality within my app.
>> >
>> > As of yesterday, I am getting build failures with this exception.
>> Something
>> > has changed on trunk. I am not sure what.
>> >
>> > ERROR [12 Jul 2011 12:49:28] org.apache.solr.core.SolrCore -
>> > java.lang.IllegalArgumentException: fst must be non-null
>> > at
>> >
>> org.apache.lucene.analysis.synonym.SynonymFilter.(SynonymFilter.java:227)
>> > at
>> >
>> org.apache.solr.analysis.FSTSynonymFilterFactory.create(FSTSynonymFilterFactory.java:59)
>> > at
>> >
>> org.apache.solr.analysis.SynonymFilterFactory.create(SynonymFilterFactory.java:46)
>> > at
>> org.apache.solr.analysis.TokenizerChain.getStream(TokenizerChain.java:71)
>> > at
>> >
>> org.apache.solr.analysis.SolrAnalyzer.reusableTokenStream(SolrAnalyzer.java:75)
>> > at
>> >
>> org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:330)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:477)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:465)
>> > at
>> >
>> org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:127)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1054)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
>> > at
>> >
>> org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:119)
>> > at
>> org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:74)
>> > at org.apache.solr.search.QParser.getQuery(QParser.java:143)
>> > at
>> >
>> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:97)
>> > at
>> >
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1404)
>> > at
>> >
>> org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:156)
>> > at
>> >
>> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
>> > at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:123)
>> > at com.wenzani.search.Search.search(Search.java:28)
>> > at
>> >
>> com.wenzani.search.user.UserIndexSearchTest.testIndexAndSearch_ByFirstName(UserIndexSearchTest.java:50)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> >
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>> > at
>> >
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> > at
>> >
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>> > at
>> >
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> > at
>> >
>> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>> > at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>> > at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> > at org.junit.runners.ParentRunner$2.evaluate(ParentRun

Re: fst must be non null

[Stuart King]

Sorry Robert,

What does that mean? Should I be providing synonyms in my queries?

Cheers

Stu

On Tue, Jul 12, 2011 at 1:49 PM, Robert Muir  wrote:

> I just committed a fix for this, to warn that you are using an empty
> set of synonyms instead of error.
>
> On Mon, Jul 11, 2011 at 10:50 PM, Stuart King  wrote:
> > I have been building and running against trunk. In my build I have a
> number
> > of tests, testing solr functionality within my app.
> >
> > As of yesterday, I am getting build failures with this exception.
> Something
> > has changed on trunk. I am not sure what.
> >
> > ERROR [12 Jul 2011 12:49:28] org.apache.solr.core.SolrCore -
> > java.lang.IllegalArgumentException: fst must be non-null
> > at
> >
> org.apache.lucene.analysis.synonym.SynonymFilter.(SynonymFilter.java:227)
> > at
> >
> org.apache.solr.analysis.FSTSynonymFilterFactory.create(FSTSynonymFilterFactory.java:59)
> > at
> >
> org.apache.solr.analysis.SynonymFilterFactory.create(SynonymFilterFactory.java:46)
> > at
> org.apache.solr.analysis.TokenizerChain.getStream(TokenizerChain.java:71)
> > at
> >
> org.apache.solr.analysis.SolrAnalyzer.reusableTokenStream(SolrAnalyzer.java:75)
> > at
> >
> org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:330)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:477)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:465)
> > at
> >
> org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:127)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1054)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
> > at
> >
> org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:119)
> > at
> org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:74)
> > at org.apache.solr.search.QParser.getQuery(QParser.java:143)
> > at
> >
> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:97)
> > at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1404)
> > at
> >
> org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:156)
> > at
> >
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
> > at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:123)
> > at com.wenzani.search.Search.search(Search.java:28)
> > at
> >
> com.wenzani.search.user.UserIndexSearchTest.testIndexAndSearch_ByFirstName(UserIndexSearchTest.java:50)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > at
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
> > at
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> > at
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> > at
> >
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> > at
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:71)
> > at
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:199)
> > at
> com.intellij.rt

Re: ' invisible ' words

[Lance Norskog]

German/Germany is being changed by an English-language stemmer.
Strip your analysis chains down the minimum and walk through what
happens when you add each step.

On Mon, Jul 11, 2011 at 8:43 PM, deniz  wrote:
> Thank you Erick, I have did what you have said but unfortunately, nothing is
> changed with the problem... Even if I use the admin interface the result is
> still the same...
>
> I have tried removing stopword and synonym files too  but still i got that
> weird result...
>
> another fact is that i can match those invisible words partially... i mean
> something like this:
>
> I work in Germany.
>
> when i make a search with the word German, I got partial match, which
> should not be matched actually... but when i make a search with Germany, no
> match
>
> -
> Zeki ama calismiyor... Calissa yapar...
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/invisible-words-tp3158060p3161306.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Lance Norskog
goks...@gmail.com

Re: fst must be non null

[Robert Muir]

I just committed a fix for this, to warn that you are using an empty
set of synonyms instead of error.

On Mon, Jul 11, 2011 at 10:50 PM, Stuart King  wrote:
> I have been building and running against trunk. In my build I have a number
> of tests, testing solr functionality within my app.
>
> As of yesterday, I am getting build failures with this exception. Something
> has changed on trunk. I am not sure what.
>
> ERROR [12 Jul 2011 12:49:28] org.apache.solr.core.SolrCore -
> java.lang.IllegalArgumentException: fst must be non-null
> at
> org.apache.lucene.analysis.synonym.SynonymFilter.(SynonymFilter.java:227)
> at
> org.apache.solr.analysis.FSTSynonymFilterFactory.create(FSTSynonymFilterFactory.java:59)
> at
> org.apache.solr.analysis.SynonymFilterFactory.create(SynonymFilterFactory.java:46)
> at org.apache.solr.analysis.TokenizerChain.getStream(TokenizerChain.java:71)
> at
> org.apache.solr.analysis.SolrAnalyzer.reusableTokenStream(SolrAnalyzer.java:75)
> at
> org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:330)
> at
> org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:477)
> at
> org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:465)
> at
> org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:127)
> at
> org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1054)
> at
> org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
> at
> org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
> at
> org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
> at
> org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
> at
> org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:119)
> at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:74)
> at org.apache.solr.search.QParser.getQuery(QParser.java:143)
> at
> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:97)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1404)
> at
> org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:156)
> at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
> at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:123)
> at com.wenzani.search.Search.search(Search.java:28)
> at
> com.wenzani.search.user.UserIndexSearchTest.testIndexAndSearch_ByFirstName(UserIndexSearchTest.java:50)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:71)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:199)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.intellij.rt.execution.application.AppMain.main(App

Re: ' invisible ' words

[deniz]

Thank you Erick, I have did what you have said but unfortunately, nothing is
changed with the problem... Even if I use the admin interface the result is
still the same... 

I have tried removing stopword and synonym files too  but still i got that
weird result... 

another fact is that i can match those invisible words partially... i mean
something like this:

I work in Germany.

when i make a search with the word German, I got partial match, which 
should not be matched actually... but when i make a search with Germany, no
match

-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/invisible-words-tp3158060p3161306.html
Sent from the Solr - User mailing list archive at Nabble.com.

fst must be non null

[Stuart King]

I have been building and running against trunk. In my build I have a number
of tests, testing solr functionality within my app.

As of yesterday, I am getting build failures with this exception. Something
has changed on trunk. I am not sure what.

ERROR [12 Jul 2011 12:49:28] org.apache.solr.core.SolrCore -
java.lang.IllegalArgumentException: fst must be non-null
at
org.apache.lucene.analysis.synonym.SynonymFilter.(SynonymFilter.java:227)
at
org.apache.solr.analysis.FSTSynonymFilterFactory.create(FSTSynonymFilterFactory.java:59)
at
org.apache.solr.analysis.SynonymFilterFactory.create(SynonymFilterFactory.java:46)
at org.apache.solr.analysis.TokenizerChain.getStream(TokenizerChain.java:71)
at
org.apache.solr.analysis.SolrAnalyzer.reusableTokenStream(SolrAnalyzer.java:75)
at
org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:330)
at
org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:477)
at
org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:465)
at
org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:127)
at
org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1054)
at
org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
at
org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
at
org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
at
org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
at
org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:119)
at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:74)
at org.apache.solr.search.QParser.getQuery(QParser.java:143)
at
org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:97)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:173)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1404)
at
org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:156)
at
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:123)
at com.wenzani.search.Search.search(Search.java:28)
at
com.wenzani.search.user.UserIndexSearchTest.testIndexAndSearch_ByFirstName(UserIndexSearchTest.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:71)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:199)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

 INFO [12 Jul 2011 12:49:28] org.apache.solr.core.SolrCore - [users]
webapp=null path=/select
params={fl=id&sort=score+desc&q=first_name%3AhKuPIBv3Ct} status=500
QTime=18

com.wenzani.search.SearchException
at com.wenzani.search.Search.search(Search.java:30)
at
com.wenzani.search.user.UserIndexSearchTest.testIndexAndSearch_ByFirstNa

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Koji Sekiguchi]

I don't think you have wrong setting in UIMA, but you may have the request 
handler
named "/update/javabin" in solrconfig.xml is not correct?

koji
--
http://www.rondhuit.com/en/

(11/07/12 0:52), Sowmya V.B. wrote:

Hi

I just added the fields which are added to the index by one of the
annotators to the index, in the fieldmappings section. I am not getting any
compilation errors and still see the admin interface. However, when I index,
i just get a SolrException,

org.apache.solr.common.SolrException: Bad Request.

On the server log, I don't see anything except for this:
127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
/apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1328

Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
original path names for privacy's sake!)

***
 

 
   
 
 
   /Users/svajjala/Documents/EnglishTok.bin.gz
   /Users/svajjala/Documents/tag.bin.gz
  english
 false
  /Users/svajjala/Documents/NewGreenline


 /Users/svajjala/Documents/ir4icallPipeline.xml
 
false
 
 
   false
   
 text
   
 

  

 
 
 
 
 
 
 
 
 
 
 

 
   
 
 
 
   

 
I don't understand where exactly can I see a more detailed log of why its
not getting indexed.

Sowmya.

On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi  wrote:


disclaimer: I'm not an expert of UIMA. I've just started using it when Solr
3.1
integrated UIMA!


  Thanks for the clarification. Now, I get it.

Should   section mention all the annotators, even if the
annotators do not add any new fields?



  For example, if I have a pipeline, starting from "parser", "tokenizer" and

"tagger", all of them operate on a field called "text"..which is the
of the document. but all these annotators do not add any new fields to the
index. Should I still write fieldmappings for these annotators inside
SolrConfig.?



I don't think so. Try it and if you got errors, come back to this list!


koji
--
http://www.rondhuit.com/en/

Re: Average PDF index time

[Markus Jelsma]

> Hi all,
> 
> Are there some kind of average indexing times or PDF's in relation to
> its size?
> I have here a 10MB PDF (50 pages) which takes about 30 seconds to index!
> Is that normal?

Depends on you hardware. PDF parsing is a lot more tedious than XML and 
besides parsing it's also analyzed and stored and maybe even committed. Is it 
a problem or do you have many thousands of files with this size?

> 
> greetings
>   alex

Re: Query grouping search terms with proximity operator

[Ahmet Arslan]

--- On Mon, 7/11/11, Jens Sieckmann  wrote:

> From: Jens Sieckmann 
> Subject: Query grouping search terms with proximity operator
> To: "solr-user@lucene.apache.org" 
> Date: Monday, July 11, 2011, 6:44 PM
> Hi folks,
> 
> I’m new to Solr and just evaluating it for my purposes.
> 
> I want to combine OR-grouped search terms by using the
> proximity
> operator with other OR-grouped terms. Something like
> (although
> this is not query parser syntax):
> 
> (term A OR term B OR term c) NEAR/5 (term D OR term E)
> 
> I didn't find a possibility to do that. Do I have to break
> it down to:
> 
> (term A NEAR/8 term D) OR
> (term A NEAR/8 term E) OR
> (term B NEAR/8 term D) OR
> (term B NEAR/8 term E) OR
> (term C NEAR/8 term D) OR
> (term C NEAR/8 term E)?
> 

With https://issues.apache.org/jira/browse/SOLR-1604 you can do that.

"(termA termB termC) (termD termE)"~5

Re: How do I specify a different analyzer at search-time?

[Mike Sokolov]

There is a syntax  that allows you to specify different analyzers to use 
for indexing and querying, in solr.xml.  But if you don't do that, it 
should use the same analyzer in both cases.


-Mike

On 07/11/2011 10:58 AM, Gabriele Kahlout wrote:

With a lucene QueryParser instance it's possible to set the analyzer in use.
I suspect Solr doesn't use the same analyzer it used at indexing, defined in
schema.xml but I cannot verify that without the queryparser instance.
 From Jan's diagram it seems this is set in the SearchHandler's init. Is it?
How?

On Sun, Apr 10, 2011 at 11:05 AM, Jan Høydahl  wrote:

   

Looks really good, but two bits that i think might confuse people are
the implications that a "Query Parser" then invokes a series of search
components; and that "analysis" (and the pieces of an analyzer chain)
are what to lookups in the underlying lucene index.

the first might just be the ambiguity of "Query" .. using the term
"request parser" might make more sense, in comparison to the "update
parsing" from the other side of hte diagram.
   

Thanks for commenting.

Yea, the purpose is more to show a conceptual rather than actual relation
between the different components, focusing on the flow. A 100% technical
correct diagram would be too complex for beginners to comprehend,
although it could certainly be useful for developers.

I've removed the arrow between QueryParser and search components to
clarify.
The boxes first and foremost show that query parsing and response writers
are within the realm of search request handler.

 

the analysis piece is a little harder to fix cleanly.  you really want
   

the
 

end of the analysis chain to feed back up to the searh components, and
then show it (most of hte search components really) talking to the Lucene
index.
   

Yea, I know. Showing how Faceting communicate with the main index and
spellchecker with its spellchecker index could also be useful, but I think
that would be for another more detailed diagram.

I felt it was more important for beginners to realize visually that
analysis happens both at index and search time, and that the analyzers
align 1:1. At this stage in the digram I often explain the importance
of matching up the analysis on both sides to get a match in the index.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com


 


   

Re: How to create a solr core if no solr cores were created before?

[Gabriele Kahlout]

have a look here [1].

[1]
https://issues.apache.org/jira/browse/SOLR-2645?focusedCommentId=13062748&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13062748

On Mon, Jul 11, 2011 at 4:46 PM, Mark Schoy  wrote:

> Hi,
>
> I tried to create a solr core but I always get "No such solr
> core:"-Exception.
>
> -
> File home = new File( pathToSolrHome );
> File f = new File( home, "solr.xml" );
>
> CoreContainer coreContainer = new CoreContainer();
> coreContainer.load( pathToSolrHome, f );
>
> EmbeddedSolrServer server = new EmbeddedSolrServer(coreContainer, "");
> CoreAdminRequest.createCore("coreName", "coreDir", server);
> -
>
> I think the problem is the "" in new EmbeddedSolrServer(coreContainer, "");
>
> Thanks.
>



-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

Re: Broken attachment link on Wiki

[Simon Wistow]

Bump?


On Mon, Jun 27, 2011 at 06:17:42PM +0100, me said:
> On the SolrJetty page 
> 
> http://wiki.apache.org/solr/SolrJetty
> 
> there's a link to a tar ball
> 
> http://wiki.apache.org/solr/SolrJetty?action=AttachFile&do=view&target=DEMO_multiple_webapps_jetty_6.1.3.tgz
> 
> which fails with the error
> 
> You are not allowed to do AttachFile on this page.
> 
> Can someone fix it somehow? Or put the file else where?
> 
> 

!frange probelm

[Rohit Gupta]

Hi,

I am trying to execute an !frange query, but I can't seem to get past the 
following error,

Bad Request

request: http://xyz.com:12397/solr/tw/select?q=(Organix)&fq={%21frange l=834925 
u=834925}id&fq={%21frange l=834345 u=834345}id&sort=createdOnGMTDate 
desc&start=0&rows=26&wt=javabin&version=2
at 
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:436)

at 
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:245)

at 
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)

I have tried URL encoding the query and which seems to run fine in a browser 
but 
not throught the application,

http://xyz.com:12397/solr/tw/select?q=(Organix)&fq={%21frange l=834925 
u=834925}id&fq={%21frange l=834345 u=834345}id&sort=createdOnGMTDate 
desc&start=0&rows=26&wt=javabin&version=2

I have even tried escaping the ! mark ,but doesnt help either

http://xyz.com:12397/solr/tw/select?q=(Organix)&fq={!frange l=834925 
u=834925}id&fq={!frange l=834345 u=834345}id&sort=createdOnGMTDate 
desc&start=0&rows=26&wt=javabin&version=2


Regards,
Rohit

Re: Increase String length

[tautilus]

You can use solr.TextField and KeywordTokenizer because it creates just a
token with the input.
bye

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Increase-String-length-tp3156430p3159454.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Restricting the Solr Posting List (retrieved set)

[Pooja Verlani]

Thanks for the reply.

I am having a very huge index, so to retrieve older documents when not
needed definitely wastes time and also at the same time I would need to do
recency boosts/ time sort. So, I am looking for a way to avoid that.
Thats why I am in need to restrict my docset  and recently added ones. I
would not prefer to use the "rows" parameter for this.

Thanks,
pooja

On Mon, Jul 11, 2011 at 5:49 PM, Bob Sandiford  wrote:

> A good answer may also depend on WHY you are wanting to restrict to 500K
> documents.
>
> Are you seeking to reduce the time spent by Solr in determining the doc
> count?  Are you just wanting to prevent people from moving too far into the
> result set?  Is it case that you can only display 6 digits for your return
> count? :)
>
> If Solr is performing adequately, you could always just artificially
> restrict the result set.  Solr doesn't actually 'return' all 5M documents -
> it only returns the number you have specified in your query (as well as
> having some cache for the next results in anticipation of a subsequent
> query).  So, if the total count returned exceeds 500K, then just report 500K
> as the number of results, and similarly restrict how far a user can page
> through the results...
>
> (And - you can (and sounds like you should) sort your results by descending
> post date so that you do in fact get the most recent ones coming back
> first...)
>
> Bob Sandiford | Lead Software Engineer | SirsiDynix
> P: 800.288.8020 X6943 | bob.sandif...@sirsidynix.com
> www.sirsidynix.com
>
>
> > -Original Message-
> > From: Ahmet Arslan [mailto:iori...@yahoo.com]
> > Sent: Monday, July 11, 2011 7:43 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Restricting the Solr Posting List (retrieved set)
> >
> >
> > > We want to search in an index in such a way that even if a
> > > clause has a long
> > > posting list - Solr should stop collecting documents for
> > > the clause
> > > after receiving X documents that match the clause.
> > >
> > > For example, if  for query "India",solr can return 5M
> > > documents, we would
> > > like to restrict the set at only 500K documents.
> > >
> > > The assumption is that since we are posting chronologically
> > > - we would like
> > > the X most recent documents to be matched for the clause
> > > only.
> > >
> > > Is it possible anyway?
> >
> > Looks like your use-case is suitable for time based sharding.
> > http://wiki.apache.org/solr/DistributedSearch
> >
> > Lets say you divide your shards according to months. You will have a
> > separate core for each month.
> > http://wiki.apache.org/solr/CoreAdmin
> >
> > When a query comes in, you will hit the most recent core. If you don't
> > obtain enough results add a new value (previous month core) to &shards=
> > parameter.
> >
>
>
>

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Sowmya V.B.]

Hi

I just added the fields which are added to the index by one of the
annotators to the index, in the fieldmappings section. I am not getting any
compilation errors and still see the admin interface. However, when I index,
i just get a SolrException,

org.apache.solr.common.SolrException: Bad Request.

On the server log, I don't see anything except for this:
127.0.0.1 - - [11/Jul/2011:17:44:04 +0200]  "POST
/apache-solr-3.3.0/update/javabin?wt=javabin&version=2 HTTP/1.1" 400 1328

Here is my UpdateRequestProcessorChain in sorlconfig.xml (Just changed the
original path names for privacy's sake!)

***



  


  /Users/svajjala/Documents/EnglishTok.bin.gz
  /Users/svajjala/Documents/tag.bin.gz
 english
false
 /Users/svajjala/Documents/NewGreenline
   

/Users/svajjala/Documents/ir4icallPipeline.xml

   false


  false
  
text
  


 














  



  


I don't understand where exactly can I see a more detailed log of why its
not getting indexed.

Sowmya.

On Mon, Jul 11, 2011 at 5:26 PM, Koji Sekiguchi  wrote:

> disclaimer: I'm not an expert of UIMA. I've just started using it when Solr
> 3.1
> integrated UIMA!
>
>
>  Thanks for the clarification. Now, I get it.
>> Should  section mention all the annotators, even if the
>> annotators do not add any new fields?
>>
>
>  For example, if I have a pipeline, starting from "parser", "tokenizer" and
>> "tagger", all of them operate on a field called "text"..which is the
>> of the document. but all these annotators do not add any new fields to the
>> index. Should I still write fieldmappings for these annotators inside
>> SolrConfig.?
>>
>
> I don't think so. Try it and if you got errors, come back to this list!
>
>
> koji
> --
> http://www.rondhuit.com/en/
>



-- 
Sowmya V.B.

Losing optimism is blasphemy!
http://vbsowmya.wordpress.com

Query grouping search terms with proximity operator

[Jens Sieckmann]

Hi folks,

I’m new to Solr and just evaluating it for my purposes.

I want to combine OR-grouped search terms by using the proximity
operator with other OR-grouped terms. Something like (although
this is not query parser syntax):

(term A OR term B OR term c) NEAR/5 (term D OR term E)

I didn't find a possibility to do that. Do I have to break it down to:

(term A NEAR/8 term D) OR
(term A NEAR/8 term E) OR
(term B NEAR/8 term D) OR
(term B NEAR/8 term E) OR
(term C NEAR/8 term D) OR
(term C NEAR/8 term E)?

Thanks in advance,
Jens

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Koji Sekiguchi]

disclaimer: I'm not an expert of UIMA. I've just started using it when Solr 3.1
integrated UIMA!


Thanks for the clarification. Now, I get it.
Should  section mention all the annotators, even if the
annotators do not add any new fields?



For example, if I have a pipeline, starting from "parser", "tokenizer" and
"tagger", all of them operate on a field called "text"..which is the
of the document. but all these annotators do not add any new fields to the
index. Should I still write fieldmappings for these annotators inside
SolrConfig.?


I don't think so. Try it and if you got errors, come back to this list!

koji
--
http://www.rondhuit.com/en/

Average PDF index time

[alexander sulz]

Hi all,

Are there some kind of average indexing times or PDF's in relation to 
its size?

I have here a 10MB PDF (50 pages) which takes about 30 seconds to index!
Is that normal?

greetings
 alex

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Sowmya V.B.]

Koji

Thanks for the clarification. Now, I get it.
Should  section mention all the annotators, even if the
annotators do not add any new fields?

For example, if I have a pipeline, starting from "parser", "tokenizer" and
"tagger", all of them operate on a field called "text"..which is the 
of the document. but all these annotators do not add any new fields to the
index. Should I still write fieldmappings for these annotators inside
SolrConfig.?

S

On Mon, Jul 11, 2011 at 4:35 PM, Koji Sekiguchi  wrote:

> Sowmya,
>
> The combination of fieldNameFeature and dynamicField can be used when
> using,
> e.g. named entity extractor that tend to produce a lot of attributes,
> organization,
> location, country, building, spot, title,... If you are going to use such
> named
> entity extractor, you don't want to define each field in schema.xml, you
> may
> want to use a dynamic field *_sm (multiValued string type) instead.
> And you want solr to map organization to organization_sm, location to
> location_sm,
> and so on. You can do it via having fieldNameFeature and dynamicField.
>
> Where "name" feature of fieldNameFeature value is used for field name in
> dynamicField.
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
> (11/07/11 21:54), Sowmya V.B. wrote:
>
>> Hi Koji
>>
>> Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
>> with my own UIMA pipeline. However, despite seeing the example
>> solrconfig.xml, I am not able to figure out how to add mine.
>>
>> In the example:
>>
>>   entity
>>
>> name
>>
>> *_sm
>>
>> I still don't understand what "fieldnamefeature" mean, in case of dynamic
>> fields.
>>
>> For example, if the annotator takes "text" field, and gives "fieldA,
>> fieldB,
>> fieldC", how should I specify that inside this?
>>
>> I was looking on the Solr pages, and on the SolrUIMA page, (
>> http://wiki.apache.org/solr/**SolrUIMA#Using_other_UIMA_**components
>> )
>> There is this example configuration, for fieldmapping specification:
>>
>> 
>> 
>> 
>>   
>> 
>> ...
>>   
>>
>>
>> Which is slightly different from the example that you used in rondhuit
>> code
>> samples.
>> So, does it mean - I can also do something like:
>> 
>> 
>> 
>> 
>> 
>>
>> 
>> 
>>
>> 
>> 
>> 
>>
>> 
>> -if I add Fields from the annotator from within the pipeline, using
>> addFStoIndexes() method?
>>
>> Sowmya.
>>
>> On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi
>>  wrote:
>>
>>  Now I've pasted sample solrconfig.xml to the project top page.
>>> Can you visit and look at it again?
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>> (11/07/09 2:29), Sowmya V.B. wrote:
>>>
>>>  Hi Koji

 Thanks. I have checked out the code and began looking at it. The code
 examples gave me an idea of what to do,though I am not fully clear,
 since
 there are no comments there, to verify my understanding. Hence, mailing
 again for clarification.

 In NamedEntity.java, you add two fields "name", "entity", to the index,
 via
 this processing pipeline "next"?
 the methods setName() and setEntity() - add two fields "name", "entity",
 to
 the index?

 If so, how should I specify this in the solrconfig.xml's<
 fieldMappings>
 section?

 
 next.NamedEntity
 
   name
   namefield   (where namefield is
 the field I declared in schema.xml, say)
 
   
   
 next.NamedEntity
 
   entity
   entityfield   (where entityfield
 is the field I declared in schema.xml, say)
 
   

 - Is this the right way to go? Can I declare 2 mappings which relate to
 the
 same class (next.NamedEntity, in this case)?

 I am sorry for repeated mails...but its a bit confusing, because there
 is
 no
 README file.
 Thankyou once again!

 Sowmya.

 On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi
  wrote:

  (11/07/08 16:19), Sowmya V.B. wrote:

>
>  Hi Koji
>
>>
>> Thanks for the mail.
>>
>> Thanks for all the clarifications. I am now using the version 3.3..
>> But,
>> another query that I have about this is:
>> How can I add an annotator that I wrote myself, in to Solr-UIMA?
>>
>> Here is what I did before I moved to Solr:
>> I wrote an annotator (which worked when I used plain vanilla lucene
>> based
>> indexer), which enriched the document with more fields (Some
>> statistics
>> about the document...all fields added were numeric fields). Those
>> fields
>> were added to the index by extending *JCasAnnotator_ImplBase* class.
>>
>> But, in Solr-UIMA, 

Edismax query parser and non existent fields

[Jamie Johnson]

I am using the Edismax query parser and have noticed that if I specify
a field that is invalid for a document I still get results which I
would not have expected.  For instance if I say first_name:kris, and
not all my documents have the field first name I can still get back
documents where the first_name matches a field value.  So for instance
if the field was a phonetic field with the value Forrest, that
matches.  Is there a way to have Edismax ignore all fielded values
that are not valid for the schema?

Re: Query Rewrite

[Jamie Johnson]

Thanks Dmitry, I think I've got this now.

On Sun, Jul 10, 2011 at 3:55 AM, Dmitry Kan  wrote:
> was typying this on-the-go from my phone, I meant LuceneQParserPlugin of
> course.
>
> On Sat, Jul 9, 2011 at 6:39 PM, Dmitry Kan  wrote:
>
>> you can try extending LuceneQParser. In its createParser method
>> (lucene 2.9.3 and solr 1.4) you can analyze the input query in the
>> param q and modify it accordingly.
>>
>> On 7/9/11, Jamie Johnson  wrote:
>> > My organization is considering a few different approaches for indexing
>> > vs query rewrite and I'm trying to figure out what would be required
>> > in order to implement some form of query rewrite.  Lets say my index
>> > has 2 fields first name and last name.  When the user does a query
>> > name:bob I'd like to transform this into first_name:bob OR
>> > last_name:bob.  I know this example is trivial but our real index is
>> > much more complex.  Are there any extension points in solr for doing
>> > such a thing?  If not are there any utilities which I could use for
>> > doing this?  We are currently using the edismax query parser so I
>> > suppose I could extend that to handle this but I'm starting from a
>> > completely blank slate so any guidance would be appreciated.
>> >
>>
>>
>> --
>> Regards,
>>
>> Dmitry Kan
>>
>
>
>
> --
> Regards,
>
> Dmitry Kan
>

How do I specify a different analyzer at search-time?

[Gabriele Kahlout]

With a lucene QueryParser instance it's possible to set the analyzer in use.
I suspect Solr doesn't use the same analyzer it used at indexing, defined in
schema.xml but I cannot verify that without the queryparser instance.
>From Jan's diagram it seems this is set in the SearchHandler's init. Is it?
How?

On Sun, Apr 10, 2011 at 11:05 AM, Jan Høydahl  wrote:

> > Looks really good, but two bits that i think might confuse people are
> > the implications that a "Query Parser" then invokes a series of search
> > components; and that "analysis" (and the pieces of an analyzer chain)
> > are what to lookups in the underlying lucene index.
> >
> > the first might just be the ambiguity of "Query" .. using the term
> > "request parser" might make more sense, in comparison to the "update
> > parsing" from the other side of hte diagram.
>
> Thanks for commenting.
>
> Yea, the purpose is more to show a conceptual rather than actual relation
> between the different components, focusing on the flow. A 100% technical
> correct diagram would be too complex for beginners to comprehend,
> although it could certainly be useful for developers.
>
> I've removed the arrow between QueryParser and search components to
> clarify.
> The boxes first and foremost show that query parsing and response writers
> are within the realm of search request handler.
>
> > the analysis piece is a little harder to fix cleanly.  you really want
> the
> > end of the analysis chain to feed back up to the searh components, and
> > then show it (most of hte search components really) talking to the Lucene
> > index.
>
> Yea, I know. Showing how Faceting communicate with the main index and
> spellchecker with its spellchecker index could also be useful, but I think
> that would be for another more detailed diagram.
>
> I felt it was more important for beginners to realize visually that
> analysis happens both at index and search time, and that the analyzers
> align 1:1. At this stage in the digram I often explain the importance
> of matching up the analysis on both sides to get a match in the index.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
>


-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

How to create a solr core if no solr cores were created before?

[Mark Schoy]

Hi,

I tried to create a solr core but I always get "No such solr core:"-Exception.

-
File home = new File( pathToSolrHome );
File f = new File( home, "solr.xml" );

CoreContainer coreContainer = new CoreContainer();
coreContainer.load( pathToSolrHome, f );

EmbeddedSolrServer server = new EmbeddedSolrServer(coreContainer, "");
CoreAdminRequest.createCore("coreName", "coreDir", server);
-

I think the problem is the "" in new EmbeddedSolrServer(coreContainer, "");

Thanks.

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Koji Sekiguchi]

Sowmya,

The combination of fieldNameFeature and dynamicField can be used when using,
e.g. named entity extractor that tend to produce a lot of attributes, 
organization,
location, country, building, spot, title,... If you are going to use such named
entity extractor, you don't want to define each field in schema.xml, you may
want to use a dynamic field *_sm (multiValued string type) instead.
And you want solr to map organization to organization_sm, location to 
location_sm,
and so on. You can do it via having fieldNameFeature and dynamicField.

Where "name" feature of fieldNameFeature value is used for field name in 
dynamicField.

koji
--
http://www.rondhuit.com/en/

(11/07/11 21:54), Sowmya V.B. wrote:

Hi Koji

Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
with my own UIMA pipeline. However, despite seeing the example
solrconfig.xml, I am not able to figure out how to add mine.

In the example:

   entity

 name

 *_sm

I still don't understand what "fieldnamefeature" mean, in case of dynamic
fields.

For example, if the annotator takes "text" field, and gives "fieldA, fieldB,
fieldC", how should I specify that inside this?

I was looking on the Solr pages, and on the SolrUIMA page, (
http://wiki.apache.org/solr/SolrUIMA#Using_other_UIMA_components)
There is this example configuration, for fieldmapping specification:


 
 
   
 
 ...
   


Which is slightly different from the example that you used in rondhuit code
samples.
So, does it mean - I can also do something like:

 
 
 


 
 

 
 



-if I add Fields from the annotator from within the pipeline, using
addFStoIndexes() method?

Sowmya.

On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi  wrote:


Now I've pasted sample solrconfig.xml to the project top page.
Can you visit and look at it again?


koji
--
http://www.rondhuit.com/en/

(11/07/09 2:29), Sowmya V.B. wrote:


Hi Koji

Thanks. I have checked out the code and began looking at it. The code
examples gave me an idea of what to do,though I am not fully clear, since
there are no comments there, to verify my understanding. Hence, mailing
again for clarification.

In NamedEntity.java, you add two fields "name", "entity", to the index,
via
this processing pipeline "next"?
the methods setName() and setEntity() - add two fields "name", "entity",
to
the index?

If so, how should I specify this in the solrconfig.xml's<**fieldMappings>
section?


 next.NamedEntity
 
   name
   namefield   (where namefield is
the field I declared in schema.xml, say)
 
   
   
 next.NamedEntity
 
   entity
   entityfield   (where entityfield
is the field I declared in schema.xml, say)
 
   

- Is this the right way to go? Can I declare 2 mappings which relate to
the
same class (next.NamedEntity, in this case)?

I am sorry for repeated mails...but its a bit confusing, because there is
no
README file.
Thankyou once again!

Sowmya.

On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi
  wrote:

  (11/07/08 16:19), Sowmya V.B. wrote:


  Hi Koji


Thanks for the mail.

Thanks for all the clarifications. I am now using the version 3.3.. But,
another query that I have about this is:
How can I add an annotator that I wrote myself, in to Solr-UIMA?

Here is what I did before I moved to Solr:
I wrote an annotator (which worked when I used plain vanilla lucene
based
indexer), which enriched the document with more fields (Some statistics
about the document...all fields added were numeric fields). Those fields
were added to the index by extending *JCasAnnotator_ImplBase* class.

But, in Solr-UIMA, I am not exactly clear on where the above setup fits
in.
I thought I would get an idea looking at the annotators that came with
the
UIMA integration of Solr, but their source was not available. So, I do
not
understand how to actually integrate my own annotator in to UIMA.



Hi Sowmya,

Please look at an example UIMA annotators that can be deployed on
Solr-UIMA
environment:

http://code.google.com/p/rondhuit-uima/






It comes with source code.


koji
--
http://www.rondhuit.com/en/

Re: ' invisible ' words

[Erick Erickson]

Try running your text and query through the admin/analysis page and see
what tokens come out the other end, that may give you a clue. Nothing
jumps out from the definitions

Also, what comes out when you attach &debugQuery=on to the
query? That sometimes helps too...

Unless you have something really strange in your stopwords
or synonyms files

Note that it is unusual to have synonyms in both the query
and analysis chains, but that's probably not the root of your
problem.. And the fact that you have ordered your filters
differently in the two chains can also lead to very "interesting"
results and is not recommended

Best
Erick

On Mon, Jul 11, 2011 at 4:28 AM, deniz  wrote:
>    
>      
>        
>
>         words="stopwords.txt"/>
>         generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>        
>         protected="protwords.txt"/>
>        
>         ignoreCase="true" expand="true"/>
>      
>      
>        
>         ignoreCase="true" expand="true"/>
>         words="stopwords.txt"/>
>         generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
>        
>         protected="protwords.txt"/>
>        
>      
>    
>
> -
> Zeki ama calismiyor... Calissa yapar...
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/invisible-words-tp3158060p3158541.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: (Solr-UIMA) Doubt regarding integrating UIMA in to solr - Configuration.

[Sowmya V.B.]

Hi Koji

Thanks a lot for the examples. Now, I was able to compile a JAR snapshot,
with my own UIMA pipeline. However, despite seeing the example
solrconfig.xml, I am not able to figure out how to add mine.

In the example:

  entity

name

*_sm

I still don't understand what "fieldnamefeature" mean, in case of dynamic
fields.

For example, if the annotator takes "text" field, and gives "fieldA, fieldB,
fieldC", how should I specify that inside this?

I was looking on the Solr pages, and on the SolrUIMA page, (
http://wiki.apache.org/solr/SolrUIMA#Using_other_UIMA_components)
There is this example configuration, for fieldmapping specification:




  

...
  


Which is slightly different from the example that you used in rondhuit code
samples.
So, does it mean - I can also do something like:








   





-if I add Fields from the annotator from within the pipeline, using
addFStoIndexes() method?

Sowmya.

On Sat, Jul 9, 2011 at 12:51 AM, Koji Sekiguchi  wrote:

> Now I've pasted sample solrconfig.xml to the project top page.
> Can you visit and look at it again?
>
>
> koji
> --
> http://www.rondhuit.com/en/
>
> (11/07/09 2:29), Sowmya V.B. wrote:
>
>> Hi Koji
>>
>> Thanks. I have checked out the code and began looking at it. The code
>> examples gave me an idea of what to do,though I am not fully clear, since
>> there are no comments there, to verify my understanding. Hence, mailing
>> again for clarification.
>>
>> In NamedEntity.java, you add two fields "name", "entity", to the index,
>> via
>> this processing pipeline "next"?
>> the methods setName() and setEntity() - add two fields "name", "entity",
>> to
>> the index?
>>
>> If so, how should I specify this in the solrconfig.xml's<**fieldMappings>
>> section?
>>
>> 
>> next.NamedEntity
>> 
>>   name
>>   namefield  (where namefield is
>> the field I declared in schema.xml, say)
>> 
>>   
>>   
>> next.NamedEntity
>> 
>>   entity
>>   entityfield  (where entityfield
>> is the field I declared in schema.xml, say)
>> 
>>   
>>
>> - Is this the right way to go? Can I declare 2 mappings which relate to
>> the
>> same class (next.NamedEntity, in this case)?
>>
>> I am sorry for repeated mails...but its a bit confusing, because there is
>> no
>> README file.
>> Thankyou once again!
>>
>> Sowmya.
>>
>> On Fri, Jul 8, 2011 at 4:07 PM, Koji Sekiguchi
>>  wrote:
>>
>>  (11/07/08 16:19), Sowmya V.B. wrote:
>>>
>>>  Hi Koji

 Thanks for the mail.

 Thanks for all the clarifications. I am now using the version 3.3.. But,
 another query that I have about this is:
 How can I add an annotator that I wrote myself, in to Solr-UIMA?

 Here is what I did before I moved to Solr:
 I wrote an annotator (which worked when I used plain vanilla lucene
 based
 indexer), which enriched the document with more fields (Some statistics
 about the document...all fields added were numeric fields). Those fields
 were added to the index by extending *JCasAnnotator_ImplBase* class.

 But, in Solr-UIMA, I am not exactly clear on where the above setup fits
 in.
 I thought I would get an idea looking at the annotators that came with
 the
 UIMA integration of Solr, but their source was not available. So, I do
 not
 understand how to actually integrate my own annotator in to UIMA.


>>> Hi Sowmya,
>>>
>>> Please look at an example UIMA annotators that can be deployed on
>>> Solr-UIMA
>>> environment:
>>>
>>> http://code.google.com/p/rondhuit-uima/
>>> 
>>> >
>>>
>>>
>>> It comes with source code.
>>>
>>>
>>> koji
>>> --
>>> http://www.rondhuit.com/en/
>>>
>>>
>>
>>
>>
>
>


-- 
Sowmya V.B.

Losing optimism is blasphemy!
http://vbsowmya.wordpress.com

Re: Increase String length

[viruslviv]

You can use TextField instead of StrField:



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Increase-String-length-tp3156430p3159059.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Restricting the Solr Posting List (retrieved set)

[Bob Sandiford]

A good answer may also depend on WHY you are wanting to restrict to 500K 
documents.

Are you seeking to reduce the time spent by Solr in determining the doc count?  
Are you just wanting to prevent people from moving too far into the result set? 
 Is it case that you can only display 6 digits for your return count? :)

If Solr is performing adequately, you could always just artificially restrict 
the result set.  Solr doesn't actually 'return' all 5M documents - it only 
returns the number you have specified in your query (as well as having some 
cache for the next results in anticipation of a subsequent query).  So, if the 
total count returned exceeds 500K, then just report 500K as the number of 
results, and similarly restrict how far a user can page through the results...

(And - you can (and sounds like you should) sort your results by descending 
post date so that you do in fact get the most recent ones coming back first...)

Bob Sandiford | Lead Software Engineer | SirsiDynix
P: 800.288.8020 X6943 | bob.sandif...@sirsidynix.com
www.sirsidynix.com


> -Original Message-
> From: Ahmet Arslan [mailto:iori...@yahoo.com]
> Sent: Monday, July 11, 2011 7:43 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Restricting the Solr Posting List (retrieved set)
> 
> 
> > We want to search in an index in such a way that even if a
> > clause has a long
> > posting list - Solr should stop collecting documents for
> > the clause
> > after receiving X documents that match the clause.
> >
> > For example, if  for query "India",solr can return 5M
> > documents, we would
> > like to restrict the set at only 500K documents.
> >
> > The assumption is that since we are posting chronologically
> > - we would like
> > the X most recent documents to be matched for the clause
> > only.
> >
> > Is it possible anyway?
> 
> Looks like your use-case is suitable for time based sharding.
> http://wiki.apache.org/solr/DistributedSearch
> 
> Lets say you divide your shards according to months. You will have a
> separate core for each month.
> http://wiki.apache.org/solr/CoreAdmin
> 
> When a query comes in, you will hit the most recent core. If you don't
> obtain enough results add a new value (previous month core) to &shards=
> parameter.
> 

Re: Restricting the Solr Posting List (retrieved set)

[Ahmet Arslan]

 
> We want to search in an index in such a way that even if a
> clause has a long
> posting list - Solr should stop collecting documents for
> the clause
> after receiving X documents that match the clause.
> 
> For example, if  for query "India",solr can return 5M
> documents, we would
> like to restrict the set at only 500K documents.
> 
> The assumption is that since we are posting chronologically
> - we would like
> the X most recent documents to be matched for the clause
> only.
> 
> Is it possible anyway?

Looks like your use-case is suitable for time based sharding.
http://wiki.apache.org/solr/DistributedSearch

Lets say you divide your shards according to months. You will have a separate 
core for each month. 
http://wiki.apache.org/solr/CoreAdmin

When a query comes in, you will hit the most recent core. If you don't obtain 
enough results add a new value (previous month core) to &shards= parameter.

RE: Error using Custom Functions for DIH

[Aviraj Chavan]

Hello,

Can anyone guide me on this issue?

Regards
Aviraj
-Original Message-
From: Aviraj Chavan [mailto:aviraj_cha...@persistent.co.in]
Sent: Saturday, July 09, 2011 2:20 PM
To: solr-user@lucene.apache.org
Subject: Error using Custom Functions for DIH

Hi,

I am trying to use DIH custom function by referring 
http://wiki.apache.org/solr/DIHCustomFunctions, I have prepared jar file 
containing my custom class and placed it in solr lib directory. For Full-import 
command, I am  getting "Full Import failed:java.lang.NoClassDefFoundError: 
org/apache/solr/handler/dataimport/Evaluator" exception.

As there is no enough documentation available I am stuck, tried adding 
dependencies didn't worked. Any pointers or suggestions is appreciated.

package com.custom;
import org.apache.solr.handler.dataimport.Evaluator;
import org.apache.solr.handler.dataimport.VariableResolver;

public class PrepareQuery extends Evaluator {
@Override
public String evaluate(VariableResolver arg0, String arg1) {
String subQueryStr = "select ID, CATEGORY_NAME 
from CATEGORY_MASTER where ID=" + arg1;
return subQueryStr.toString();
}
}


 
...




Thanks
Aviraj


DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.


DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

Re: Error using Custom Functions for DIH

[Frank Wesemann]

Aviraj Chavan schrieb:

public class PrepareQuery extends Evaluator
{
@Override
public String evaluate(VariableResolver arg0, String arg1) {
String subQueryStr = "select ID, CATEGORY_NAME from 
CATEGORY_MASTER where ID=" + arg1;
return subQueryStr.toString();
}
}
  
As stated in the wiki and the javadoc the signature of the evaluate 
Methode is:


public String evaluate(String expression, Context context) 



http://wiki.apache.org/solr/DataImportHandler offers more info on this 
subject.





Thanks
Aviraj


DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.


  



--
mit freundlichem Gruß,

Frank Wesemann
Fotofinder GmbH USt-IdNr. DE812854514
Software EntwicklungWeb: http://www.fotofinder.com/
Potsdamer Str. 96   Tel: +49 30 25 79 28 90
10785 BerlinFax: +49 30 25 79 28 999

Sitz: Berlin
Amtsgericht Berlin Charlottenburg (HRB 73099)
Geschäftsführer: Ali Paczensky

Re: Can I delete the stored value?

[Simon Willnauer]

On Mon, Jul 11, 2011 at 8:28 AM, Andrzej Bialecki  wrote:
> On 7/10/11 2:33 PM, Simon Willnauer wrote:
>>
>> Currently there is no easy way to do this. I would need to think how
>> you can force the index to drop those so the answer here is no you
>> can't!
>>
>> simon
>>
>> On Sat, Jul 9, 2011 at 11:11 AM, Gabriele Kahlout
>>   wrote:
>>>
>>> I've stored the contents of some pages I no longer need. How can I now
>>> delete the stored content without re-crawling the pages (i.e. using
>>> updateDocument ). I cannot just remove the field, since I still want the
>>> field to be indexed, I just don't want to store something with it.
>>> My understanding is that field.setValue("") won't do since that should
>>> affect the indexed value as well.
>
> You could pump the content of your index through a FilterIndexReader - i.e.
> implement a subclass of FilterIndexReader that removes stored fields under
> some conditions, and then use IndexWriter.addIndexes with this reader.
>
> See LUCENE-1812 for another practical application of this concept.

good call andrzej, to make this work I think you need to use lucene
directly so make sure you are on the right version.
simon
>
> --
> Best regards,
> Andrzej Bialecki     <><
>  ___. ___ ___ ___ _ _   __
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>

Re: Solr Configuration with 404 error

[karsten-solr]

Hi rocco,

you did not stop jetty after your first attempt.
(You have to kill the task.)

Best regards
  Karsten

btw: How to change the port 8983:
http://lucene.472066.n3.nabble.com/How-to-change-a-port-td490375.html

 Original-Nachricht 
> Datum: Sun, 10 Jul 2011 20:11:54 -0700 (PDT)
> Von: rocco2004 
> An: solr-user@lucene.apache.org
> Betreff: Solr Configuration with 404 error

> I installed Solr using:
> 
> java -jar start.jar
> 
> However I downloaded the source code and didn't compile it (Didn't pay
> attention). And the error using:
> http://localhost:8983/solr/admin/ was:
> 
> HTTP ERROR: 404 Problem accessing /solr/admin/. Reason: NOT_FOUND
> 
> I realized that it was nos configuring because the source code was not
> compiled. Then I downloaded the compiled version of solr but when trying
> to
> run the example configuration I'm getting exception: 
> 
> java.net.BindException: Address already in use
> 
> Is there a way to revert solr configuration and start from scratch? Looks
> like the configuration got messed up. I don't see anything related to it
> in
> the manual.
> 
> Here is the error:
> 
> 2011-07-10 22:41:27.631:WARN::failed SocketConnector@0.0.0.0:8983:
> java.net.BindException: Address already in use 2011-07-10
> 22:41:27.632:WARN::failed Server@c4e21db: java.net.BindException: Address
> already in use 2011-07-10 22:41:27.632:WARN::EXCEPTION
> java.net.BindException: Address already in use at
> java.net.PlainSocketImpl.socketBind(Native Method) at
> java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at
> java.net.ServerSocket.bind(ServerSocket.java:328) at
> java.net.ServerSocket.(ServerSocket.java:194) at
> java.net.ServerSocket.(ServerSocket.java:150) at
> org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
> at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73) at
> org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283) at
> org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147) at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.Server.doStart(Server.java:235) at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> org.mortbay.start.Main.invokeMain(Main.java:194) at
> org.mortbay.start.Main.start(Main.java:534) at
> org.mortbay.start.Main.start(Main.java:441) at
> org.mortbay.start.Main.main(Main.java:119) Jul 10, 2011 10:41:27 PM
> org.apache.solr.core.SolrCore registerSearcher INFO: [] Registered new
> searcher Searcher@5b6b9e62 main
> 
> 
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Configuration-with-404-error-tp3157895p3157895.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Restricting the Solr Posting List (retrieved set)

[Pooja Verlani]

Hi,

We want to search in an index in such a way that even if a clause has a long
posting list - Solr should stop collecting documents for the clause
after receiving X documents that match the clause.

For example, if  for query "India",solr can return 5M documents, we would
like to restrict the set at only 500K documents.

The assumption is that since we are posting chronologically - we would like
the X most recent documents to be matched for the clause only.

Is it possible anyway?

Regards,
Pooja

Re: ' invisible ' words

[deniz]

  








  
  







  


-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/invisible-words-tp3158060p3158541.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: ' invisible ' words

[Gora Mohanty]

On Mon, Jul 11, 2011 at 10:10 AM, deniz  wrote:
> hi all...
>
> I have a field, which is a text type:
>
>  
[...]

Please show us the analyser/tokeniser chain for the fieldType
"info".

Regards,
Gora