date:20100802


Thanks alot to all now its clear the problem was in the schema. One more
thing i would like to know is if the user queries for something does it have
to always be like q=field:monitor where field is defined in schema and
monitor is just a text in a column. 

Hando
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p1018268.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: enhancing auto complete

2010-08-02 Thread Bhavnik Gajjar

Whoops!

table still not looks ok :(

trying to send once again

loremLorem ipsum dolor sit amet
 Hieyed ddi lorem ipsum dolor
 test lorem ipsume
 test xyz lorem ipslili

lorem ipLorem ipsum dolor sit amet
 Hieyed ddi lorem ipsum dolor
 test lorem ipsume
 test xyz lorem ipslili

lorem ipsltest xyz lorem ipslili

On 8/3/2010 10:00 AM, Bhavnik Gajjar wrote:
> Avlesh,
>
> Thanks for responding
>
> The table mentioned below looks like,
>
> lorem   Lorem ipsum dolor sit amet
>   Hieyed ddi lorem ipsum
> dolor
>   test lorem ipsume
>   test xyz lorem ipslili
>
> lorem ip   Lorem ipsum dolor sit amet
>   Hieyed ddi lorem ipsum
> dolor
>   test lorem ipsume
>   test xyz lorem ipslili
>
> lorem ipsl test xyz lorem ipslili
>
>
> Yes, [http://askme.in] looks good!
>
> I would like to know its designs/solr configurations etc.. Can you
> please provide me detailed views of it?
>
> In [http://askme.in], there is one thing to be noted. Search text like,
> [business c] populates [Business Centre] which looks OK but, [Consultant
> Business] looks bit odd. But, in general the pointer you suggested is
> great to start with.
>
> On 8/2/2010 8:39 PM, Avlesh Singh wrote:
>
>>>  From whatever I could read in your broken table of sample use cases, I 
>>> think
>>>
>> you are looking for something similar to what has been done here -
>> http://askme.in; if this is what you are looking do let me know.
>>
>> Cheers
>> Avlesh
>> @avlesh   | http://webklipper.com
>>
>> On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar<
>> bhavnik.gaj...@gatewaynintec.com>   wrote:
>>
>>
>>  
>>> Hi,
>>>
>>> I'm looking for a solution related to auto complete feature for one
>>> application.
>>>
>>> Below is a list of texts from which auto complete results would be
>>> populated.
>>>
>>> Lorem ipsum dolor sit amet
>>> tincidunt ut laoreet
>>> dolore eu feugiat nulla facilisis at vero eros et
>>> te feugait nulla facilisi
>>> Claritas est etiam processus
>>> anteposuerit litterarum formas humanitatis
>>> fiant sollemnes in futurum
>>> Hieyed ddi lorem ipsum dolor
>>> test lorem ipsume
>>> test xyz lorem ipslili
>>>
>>> Consider below table. First column describes user entered value and
>>> second column describes expected result (list of auto complete terms
>>> that should be populated from Solr)
>>>
>>> lorem
>>>  *Lorem* ipsum dolor sit amet
>>> Hieyed ddi *lorem* ipsum dolor
>>> test *lorem *ipsume
>>> test xyz *lorem *ipslili
>>> lorem ip
>>>  *Lorem ip*sum dolor sit amet
>>> Hieyed ddi *lorem ip*sum dolor
>>> test *lorem ip*sume
>>> test xyz *lorem ip*slili
>>> lorem ipsl
>>>  test xyz *lorem ipsl*ili
>>>
>>>
>>>
>>> Can anyone share ideas of how this can be achieved with Solr? Already
>>> tried with various tokenizers and filter factories like,
>>> WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,
>>> ShingleFilterFactory etc. but no luck so far..
>>>
>>> Note that, It would be excellent if terms populated from Solr can be
>>> highlighted by using Highlighting or any other component/mechanism of Solr.
>>>
>>> *Note :* Standard autocomplete (like,
>>> facet.field=AutoComplete&f.AutoComplete.facet.prefix=>> term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already
>>> working fine with the application. but, nowadays, looking for enhancing
>>> the existing auto complete stuff with the above requirement.
>>>
>>> Any thoughts?
>>>
>>> Thanks in advance
>>>
>>>
>>
>



The contents of this eMail including the contents of attachment(s) are 
privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL) and 
should not be disclosed to, used by or copied in any manner by anyone other 
than the intended addressee(s). If this eMail has been received by error, 
please advise the sender immediately and delete it from your system. The views 
expressed in this eMail message are those of the individual sender, except 
where the sender expressly, and with authority, states them to be the views of 
GNPL. Any unauthorized review, use, disclosure, dissemination, forwarding, 
printing or copying of this eMail or any action taken in reliance on this eMail 
is strictly prohibited and may be unlawful. This eMail may contain viruses. 
GNPL has taken every reasonable precaution to minimize this risk, but is not 
liable for any damage you may sustain as a result of any virus in this eMail. 
You should carry out your own virus checks before opening the

Re: enhancing auto complete

2010-08-02 Thread Bhavnik Gajjar

Avlesh,

Thanks for responding

The table mentioned below looks like,

lorem   Lorem ipsum dolor sit amet
 Hieyed ddi lorem ipsum 
dolor
 test lorem ipsume
 test xyz lorem ipslili

lorem ip   Lorem ipsum dolor sit amet
 Hieyed ddi lorem ipsum 
dolor
 test lorem ipsume
 test xyz lorem ipslili

lorem ipsl test xyz lorem ipslili

Yes, [http://askme.in] looks good!

I would like to know its designs/solr configurations etc.. Can you 
please provide me detailed views of it?

In [http://askme.in], there is one thing to be noted. Search text like, 
[business c] populates [Business Centre] which looks OK but, [Consultant 
Business] looks bit odd. But, in general the pointer you suggested is 
great to start with.

On 8/2/2010 8:39 PM, Avlesh Singh wrote:
> > From whatever I could read in your broken table of sample use cases, I think
> you are looking for something similar to what has been done here -
> http://askme.in; if this is what you are looking do let me know.
>
> Cheers
> Avlesh
> @avlesh  | http://webklipper.com
>
> On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar<
> bhavnik.gaj...@gatewaynintec.com>  wrote:
>
>
>> Hi,
>>
>> I'm looking for a solution related to auto complete feature for one
>> application.
>>
>> Below is a list of texts from which auto complete results would be
>> populated.
>>
>> Lorem ipsum dolor sit amet
>> tincidunt ut laoreet
>> dolore eu feugiat nulla facilisis at vero eros et
>> te feugait nulla facilisi
>> Claritas est etiam processus
>> anteposuerit litterarum formas humanitatis
>> fiant sollemnes in futurum
>> Hieyed ddi lorem ipsum dolor
>> test lorem ipsume
>> test xyz lorem ipslili
>>
>> Consider below table. First column describes user entered value and
>> second column describes expected result (list of auto complete terms
>> that should be populated from Solr)
>>
>> lorem
>> *Lorem* ipsum dolor sit amet
>> Hieyed ddi *lorem* ipsum dolor
>> test *lorem *ipsume
>> test xyz *lorem *ipslili
>> lorem ip
>> *Lorem ip*sum dolor sit amet
>> Hieyed ddi *lorem ip*sum dolor
>> test *lorem ip*sume
>> test xyz *lorem ip*slili
>> lorem ipsl
>> test xyz *lorem ipsl*ili
>>
>>
>>
>> Can anyone share ideas of how this can be achieved with Solr? Already
>> tried with various tokenizers and filter factories like,
>> WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,
>> ShingleFilterFactory etc. but no luck so far..
>>
>> Note that, It would be excellent if terms populated from Solr can be
>> highlighted by using Highlighting or any other component/mechanism of Solr.
>>
>> *Note :* Standard autocomplete (like,
>> facet.field=AutoComplete&f.AutoComplete.facet.prefix=> term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already
>> working fine with the application. but, nowadays, looking for enhancing
>> the existing auto complete stuff with the above requirement.
>>
>> Any thoughts?
>>
>> Thanks in advance
>>  
>

The contents of this eMail including the contents of attachment(s) are 
privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL) and 
should not be disclosed to, used by or copied in any manner by anyone other 
than the intended addressee(s). If this eMail has been received by error, 
please advise the sender immediately and delete it from your system. The views 
expressed in this eMail message are those of the individual sender, except 
where the sender expressly, and with authority, states them to be the views of 
GNPL. Any unauthorized review, use, disclosure, dissemination, forwarding, 
printing or copying of this eMail or any action taken in reliance on this eMail 
is strictly prohibited and may be unlawful. This eMail may contain viruses. 
GNPL has taken every reasonable precaution to minimize this risk, but is not 
liable for any damage you may sustain as a result of any virus in this eMail. 
You should carry out your own virus checks before opening the eMail or 
attachment(s). GNPL is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt. GNPL reserves the right to monitor and review the content of all 
messages sent to or from this eMail address and may be stored on the GNPL eMail 
system. In case this eMail has reached you in error, and you  would no longer 
like to receive eMails from us, then please send an eMail to 
d...@gatewaynintec.com

Re: Solr searching performance issues, using large documents

2010-08-02 Thread Lance Norskog

Spanning won't work- you would have to make overlapping mini-documents
if you want to support this.

I don't know how big the chunks should be- you'll have to experiment.

Lance

On Mon, Aug 2, 2010 at 10:01 AM, Peter Spam  wrote:
> What would happen if the search query phrase spanned separate document chunks?
>
> Also, what would the optimal size of chunks be?
>
> Thanks!
>
>
> -Peter
>
> On Aug 1, 2010, at 7:21 PM, Lance Norskog wrote:
>
>> Not that I know of.
>>
>> The DataImportHandler has the ability to create multiple documents
>> from one input stream. It is possible to create a DIH file that reads
>> large log files and splits each one into N documents, with the file
>> name as a common field. The DIH wiki page tells you in general how to
>> make a DIH file.
>>
>> http://wiki.apache.org/solr/DataImportHandler
>>
>> From this, you should be able to make a DIH file that puts log files
>> in as separate documents. As to splitting files up into
>> mini-documents, you might have to write a bit of Javascript to achieve
>> this. There is no data structure or software that implements
>> structured documents.
>>
>> On Sun, Aug 1, 2010 at 2:06 PM, Peter Spam  wrote:
>>> Thanks for the pointer, Lance!  Is there an example of this somewhere?
>>>
>>>
>>> -Peter
>>>
>>> On Jul 31, 2010, at 3:13 PM, Lance Norskog wrote:
>>>
 Ah! You're not just highlighting, you're snippetizing. This makes it 
 easier.

 Highlighting does not stream- it pulls the entire stored contents into
 one string and then pulls out the snippet.  If you want this to be
 fast, you have to split up the text into small pieces and only
 snippetize from the most relevant text. So, separate documents with a
 common group id for the document it came from. You might have to do 2
 queries to achieve what you want, but the second query for the same
 query will be blindingly fast. Often <1ms.

 Good luck!

 Lance

 On Sat, Jul 31, 2010 at 1:12 PM, Peter Spam  wrote:
> However, I do need to search the entire document, or else the 
> highlighting will sometimes be blank :-(
> Thanks!
>
> - Peter
>
> ps. sorry for the many responses - I'm rushing around trying to get this 
> working.
>
> On Jul 31, 2010, at 1:11 PM, Peter Spam wrote:
>
>> Correction - it went from 17 seconds to 10 seconds - I was changing the 
>> hl.regex.maxAnalyzedChars the first time.
>> Thanks!
>>
>> -Peter
>>
>> On Jul 31, 2010, at 1:06 PM, Peter Spam wrote:
>>
>>> On Jul 30, 2010, at 1:16 PM, Peter Karich wrote:
>>>
 did you already try other values for hl.maxAnalyzedChars=2147483647
>>>
>>> Yes, I tried dropping it down to 21, but it didn't have much of an 
>>> impact (one search I just tried went from 17 seconds to 15.8 seconds, 
>>> and this is an 8-core Mac Pro with 6GB RAM - 4GB for java).
>>>
 ? Also regular expression highlighting is more expensive, I think.
 What does the 'fuzzy' variable mean? If you use this to query via
 "~someTerm" instead "someTerm"
 then you should try the trunk of solr which is a lot faster for fuzzy 
 or
 other wildcard search.
>>>
>>> "fuzzy" could be set to "*" but isn't right now.
>>>
>>> Thanks for the tips, Peter - this has been very frustrating!
>>>
>>>
>>> - Peter
>>>
 Regards,
 Peter.

> Data set: About 4,000 log files (will eventually grow to millions).  
> Average log file is 850k.  Largest log file (so far) is about 70MB.
>
> Problem: When I search for common terms, the query time goes from 
> under 2-3 seconds to about 60 seconds.  TermVectors etc are enabled.  
> When I disable highlighting, performance improves a lot, but is still 
> slow for some queries (7 seconds).  Thanks in advance for any ideas!
>
>
> -Peter
>
>
> -
>
> 4GB RAM server
> % java -Xms2048M -Xmx3072M -jar start.jar
>
> -
>
> schema.xml changes:
>
>  
>    
>      
>    
>     generateWordParts="0" generateNumberParts="0" catenateWords="0" 
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>
>    
>  
>
> ...
>
>  multiValued="false" termVectors="true" termPositions="true" 
> termOffsets="true" />
>   default="NOW" multiValued="false"/>
>  multiValued="false"/>
>  multiValued="false"/>
>

Re: Problems running on tomcat

2010-08-02 Thread rajini maski

I have observed this error while there is mistake in indexed fields.. i.e;
The field defined while indexing but undefined in schema... then this error
is thrown..
You can check for that missing filed in your  Catalina logs.. There it will
be written as Unknown filed..


Regards,
Rajani Maski

On Tue, Aug 3, 2010 at 2:09 AM, Claudio Devecchi wrote:

> Hi Ahmet,
>
> Works with tomcat6.
>
> Tks!
>
> On Mon, Aug 2, 2010 at 3:04 PM, Claudio Devecchi  >wrote:
>
> > Hi Ahmet,
> >
> > I'm using tomcat7 with solr 1.4.1 =\
> > If you worked fine on tomcat6 I'll try with the same version...
> >
> > Tks for helping..
> >
> >
> > On Mon, Aug 2, 2010 at 2:30 PM, Ahmet Arslan  wrote:
> >
> >> What is version of solr and tomcat?
> >>
> >> I think i saw same problem with tomcat 7 and solr 1.4.1 combination,
> thats
> >> why i am asking.
> >>
> >> I just tried to replicate this problem with tomcat 6 and solr 1.4.1, but
> >> everything went fine.
> >>
> >> --- On Fri, 7/30/10, Claudio Devecchi  wrote:
> >>
> >> > From: Claudio Devecchi 
> >> > Subject: Problems running on tomcat
> >> > To: solr-user@lucene.apache.org
> >> > Date: Friday, July 30, 2010, 10:17 PM
> >> > Hi,
> >> >
> >> > I'm new with solr and I'm doing my first installation under
> >> > tomcat, I
> >> > followed the documentation on link (
> >> > http://wiki.apache.org/solr/SolrTomcat#Installing_Tomcat_6)
> >> > but there are
> >> > some problems.
> >> > The http://localhost:8080/solr/admin works
> >> > fine, but in some cases, for
> >> > example to see my schema.xml from the admin console the
> >> > error bellow
> >> > happensHTTP
> >> > Status 404 - /solr/admin/file/index.jspSomebody already saw
> >> > this? There are
> >> > some trick to do?
> >> >
> >> > Tks
> >> >
> >> > --
> >> > Claudio Devecchi
> >> >
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Claudio Devecchi
> > flickr.com/cdevecchi
> >
>
>
>
> --
> Claudio Devecchi
> flickr.com/cdevecchi
>

Re: Multiple solr servers Vs Katta

2010-08-02 Thread Lance Norskog

This is nowhere near the volume of data where you would have to use
Katta. Katta has much more limited search and configuration features-
it is more a very distributed version of Lucene than a search server.
Solr has added a lot of features over and above Lucene.

To have 2 query servers for each index, we have 2*20*35gb/5 = 280gb of
index per server. This is a large Solr but many run much larger
systems. This should run fine on a server with 24-32G of ram and at
least 3 fast (10k SATA) disks in a raid.

On Sun, Aug 1, 2010 at 11:01 PM, Karthik K  wrote:
> Hi,
> I have about 20 indexes each of size around 30-35 GB. All of it is on one
> machine and i want to make it searchable.
> I can have about 5 solr servers each with 2-3 indexes merged and search on
> different shards or use katta.
> Please let me know which is the better option.
>
> Thanks,
> karthik
>

-- 
Lance Norskog
goks...@gmail.com

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'

2010-08-02 Thread Lance Norskog

> (PS. is there a way where I can test the current connectionstring?)

There are many jdbc-explorer apps around that take a jdbc url. You
should be able to test your jdbc url and driver with one of those.

On Mon, Aug 2, 2010 at 2:33 PM, PeterKerk  wrote:
>
> Hi Michael,
>
> Thanks, I now use that. But the same error.
>
> (PS. is there a way where I can test the current connectionstring?)
>
> Therefore I have attached 1 image with 2 screenshots of my situation.
> http://lucene.472066.n3.nabble.com/file/n1017327/Untitled.png Untitled.png
>
> Here's my current data-config:
>
> 
>  url="jdbc:sqlserver://localhost\SQLEXPRESS\Databases\wedding:1433" user="sa"
> password="123456" />
>    
>        
>            
>            
>            
>
>            
>                
>                    
>                
>            
>
>            
>                
>                    
>                
>            
>
>        
>    
> 
>
>
>
> And here's the current output in cygwin:
>
>
> INFO: Creating a connection for entity location with URL:
> jdbc:sqlserver://local
> host\SQLEXPRESS\Databases\wedding:1433
> Aug 2, 2010 11:29:25 PM org.apache.solr.handler.dataimport.DocBuilder
> buildDocum
> ent
> SEVERE: Exception while processing: location document :
> SolrInputDocument[{}]
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute
>  query: select * from locations Processing Document # 1
>        at
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
> Throw(DataImportHandlerException.java:72)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
> init>(JdbcDataSource.java:253)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
> rce.java:210)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
> rce.java:39)
>        at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEn
> tityProcessor.java:58)
>        at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEnti
> tyProcessor.java:71)
>        at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
> ityProcessorWrapper.java:237)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
> r.java:357)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
> ava:242)
>        at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
> :180)
>        at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
> rter.java:331)
>        at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
> ava:389)
>        at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
> va:370)
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for
> use
> r 'sa'.
>        at
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
> (SQLServerException.java:197)
>        at
> com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246
> )
>        at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerC
> onnection.java:2529)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConne
> ction.java:1905)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServer
> Connection.java:41)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecu
> te(SQLServerConnection.java:1893)
>        at
> com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
> rverConnection.java:1400)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLSer
> verConnection.java:1045)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConne
> ction.java:817)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerCon
> nection.java:700)
>        at
> com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.
> java:842)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
> ce.java:162)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
> ce.java:128)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcD
> ataSource.java:363)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource.access$300(JdbcData
> Source.java:39)
>        at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
> init>(JdbcDataSource.java:240)
>        ... 11 more
> Aug 2, 2010 11:29:25 PM org.apache.solr.handler.dataimport.DataImporter
> doFullIm
> port
> SEVERE: Full Import failed
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute
>  query: select * from locations Processing

QueryUtils API Change - Custom ExtendedDismaxQParserPlugin accessing QueryUtils.makeQueryable throws java.lang.IllegalAccessError

2010-08-02 Thread jayendra patil

We have a custom implementation of ExtendedDismaxQParserPlugin, which we
bundle into a jar and have it exposed in the multicore shared lib.
The custom ExtendedDismaxQParserPlugin implementation still uses QueryUtils
makeQueryable method, same as
the ExtendedDismaxQParserPlugin implementation.
However, the method calls throws an java.lang.IllegalAccessError, as it is
being called from the inner ExtendedSolrQueryParser class and the
makeQueryable has no access modifier (basically default)

Any reason for having it with default access modifier ?? or any plans making
it public ???

Regards,
Jayendra

Re: Queries with multiple wildcards failing in branch3x

2010-08-02 Thread Michael McCandless

This looks like the index corruption caused by a commit on Friday.

See the thread I sent earlier with subject "heads up -- index
corruption on Solr/Lucene trunk/3.x branch".

Mike

On Mon, Aug 2, 2010 at 6:00 PM, Paul Dlug  wrote:
> I'm running a recent build of branch3x (r981609), queries with
> multiple wildcards (e.g. a*b*c*) are failing with the exception below
> in the log. These queries worked fine for me with solr 1.4, known bug?
>
>
> SEVERE: java.lang.IndexOutOfBoundsException: Index: 114, Size: 39
>        at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>        at java.util.ArrayList.get(ArrayList.java:322)
>        at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:285)
>        at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:225)
>        at 
> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:894)
>        at 
> org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:527)
>        at org.apache.lucene.index.IndexReader.document(IndexReader.java:684)
>        at 
> org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:444)
>        at 
> org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:428)
>        at 
> org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:267)
>        at 
> org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:279)
>        at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
>        at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>        at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1322)
>        at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341)
>        at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244)
>        at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>        at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>        at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>        at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>        at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>        at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>        at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>        at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>        at 
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
>        at 
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>        at 
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>        at java.lang.Thread.run(Thread.java:619)
>

Re: Phrase search



I'm trying to match "Apple 2" but not "Apple2" using phrase search, this is why 
I have it quoted.
 
I was under the impression --when I use phrase search-- all the analyzer magic 
would not apply, but it is!!!  Otherwise, how would I search for a phrase?!
 
Using Google, when I search for "Windows 7" (with quotes), unlike Solr, I don't 
get hits on "Window7".  I want to use catenateNumbers="1" which I want it to 
take effect on other searches but no phrase searches.  Is this possible ?
 
Yes, we are in the process of planning to upgrade to Solr 1.4.1 -- it takes 
time and a lot of effort to do such an upgrade at where I work.
 
Thank you for your help and understanding.
 
-M






-Original Message-
From: Chris Hostetter 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 5:41 pm
Subject: Re: Phrase search



 I don't understand why i'm getting this behavior.  I was under the 
 impression if I search for "Apple 2" (with quotes and space before “2”) 
 it will give me different results vs. if I search for "Apple2" (with 
 quotes and no space before “2”), but I'm not!  Why?
if you search "Apple 2" in quotes, then the analyzer for your field gets 
he full string (with the space) and whatever it does with it and whatever 
erms it produces determs what Query gets executed.  If you search 
Apple2" (w/ or w/o quotes) then the analyzer for your field gets the full 
tring and whatever it does with it and whatever Terms it produces determs 
hat Query gets executed.
None of that changes based on the analyzer you use.
With that in mind: i relaly don't understand your question.  Let's step 
ack and instead of trying to explain *why* you are getting the results 
ou are getting (short answer: because that's how your analyzer works) 
et's ask the quetsion: what do you *want* to do?  What do you *want* to 
ee happen when you enter various query strings?
http://people.apache.org/~hossman/#xyproblem
Y Problem
Your question appears to be an "XY Problem" ... that is: you are dealing
ith "X", you are assuming "Y" will help you, and you are asking about "Y"
ithout giving more details about the "X" so that we can understand the
ull issue.  Perhaps the best solution doesn't involve "Y" at all?
ee Also: http://www.perlmonks.org/index.pl?node_id=542341
: I’m using Solr 1.2
PS: Solr 1.2 had numerous bugs which were really really bad and which were 
ixed in Solr 1.3.  Solr 1.3 had numerous bugs where were really really 
ad and were fixed in Solr 1.4.  Solr 1.4 had a couple of bugs where 
eally really bad and which were fixed in Solr 1.4.1 ... so even if you 
on't want any of hte new features, you should *REALLY* consider 
pgrading.

Hoss

Queries with multiple wildcards failing in branch3x

2010-08-02 Thread Paul Dlug

I'm running a recent build of branch3x (r981609), queries with
multiple wildcards (e.g. a*b*c*) are failing with the exception below
in the log. These queries worked fine for me with solr 1.4, known bug?


SEVERE: java.lang.IndexOutOfBoundsException: Index: 114, Size: 39
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:285)
at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:225)
at 
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:894)
at 
org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:527)
at org.apache.lucene.index.IndexReader.document(IndexReader.java:684)
at 
org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:444)
at 
org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:428)
at 
org.apache.solr.util.SolrPluginUtils.optimizePreFetchDocs(SolrPluginUtils.java:267)
at 
org.apache.solr.handler.component.QueryComponent.doPrefetch(QueryComponent.java:279)
at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:186)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1322)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at 
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:619)

Re: Implementing lookups while importing data


: We have a database that has numeric values for some columns, which
: correspond to text values in drop-downs on a website. We need to
: index both the numeric and text equivalents into Solr, and can do
: that via a lookup on a different table from the one holding the
: main data. We are currently doing this via a JOIN on the numeric
: field, between the main data table and the lookup table, but this
: dramatically slows down indexing.
: 
: We could try using the CachedSqlEntity processor, but there are
: some issues in doing that, as the data import handler is quite
: complicated.

wha you are describing is pretty much the exact use case of the 
CachedSqlEntity (as i understand it) so perhaps you should elaborate on 
what issues you had.

showing your DIH config is the best way ot get assistance.



-Hoss

Re: Phrase search


: I don't understand why i'm getting this behavior.  I was under the 
: impression if I search for "Apple 2" (with quotes and space before “2”) 
: it will give me different results vs. if I search for "Apple2" (with 
: quotes and no space before “2”), but I'm not!  Why?

if you search "Apple 2" in quotes, then the analyzer for your field gets 
the full string (with the space) and whatever it does with it and whatever 
Terms it produces determs what Query gets executed.  If you search 
"Apple2" (w/ or w/o quotes) then the analyzer for your field gets the full 
string and whatever it does with it and whatever Terms it produces determs 
what Query gets executed.

None of that changes based on the analyzer you use.

With that in mind: i relaly don't understand your question.  Let's step 
back and instead of trying to explain *why* you are getting the results 
you are getting (short answer: because that's how your analyzer works) 
let's ask the quetsion: what do you *want* to do?  What do you *want* to 
see happen when you enter various query strings?

http://people.apache.org/~hossman/#xyproblem
XY Problem

Your question appears to be an "XY Problem" ... that is: you are dealing
with "X", you are assuming "Y" will help you, and you are asking about "Y"
without giving more details about the "X" so that we can understand the
full issue.  Perhaps the best solution doesn't involve "Y" at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341

: I’m using Solr 1.2

PS: Solr 1.2 had numerous bugs which were really really bad and which were 
fixed in Solr 1.3.  Solr 1.3 had numerous bugs where were really really 
bad and were fixed in Solr 1.4.  Solr 1.4 had a couple of bugs where 
really really bad and which were fixed in Solr 1.4.1 ... so even if you 
don't want any of hte new features, you should *REALLY* consider 
upgrading.


-Hoss

Re: how to highlight string in jsp


: Subject: how to highlight string in jsp
: References: <8cd00862ad8eae9-12e0-2...@webmail-m057.sysops.aol.com>
:  
: In-Reply-To: 

http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists

When starting a new discussion on a mailing list, please do not reply to 
an existing message, instead start a fresh email.  Even if you change the 
subject line of your email, other mail headers still track which thread 
you replied to and your question is "hidden" in that thread and gets less 
attention.   It makes following discussions in the mailing list archives 
particularly difficult.
See Also:  http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking




-Hoss

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'


Hi Michael,

Thanks, I now use that. But the same error.

(PS. is there a way where I can test the current connectionstring?)

Therefore I have attached 1 image with 2 screenshots of my situation.
http://lucene.472066.n3.nabble.com/file/n1017327/Untitled.png Untitled.png 

Here's my current data-config:



























And here's the current output in cygwin:


INFO: Creating a connection for entity location with URL:
jdbc:sqlserver://local
host\SQLEXPRESS\Databases\wedding:1433
Aug 2, 2010 11:29:25 PM org.apache.solr.handler.dataimport.DocBuilder
buildDocum
ent
SEVERE: Exception while processing: location document :
SolrInputDocument[{}]
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute
 query: select * from locations Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
Throw(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEn
tityProcessor.java:58)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEnti
tyProcessor.java:71)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
ityProcessorWrapper.java:237)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:357)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:242)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:180)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:331)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:389)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:370)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for
use
r 'sa'.
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
(SQLServerException.java:197)
at
com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246
)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerC
onnection.java:2529)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConne
ction.java:1905)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServer
Connection.java:41)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecu
te(SQLServerConnection.java:1893)
at
com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
rverConnection.java:1400)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLSer
verConnection.java:1045)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConne
ction.java:817)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerCon
nection.java:700)
at
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.
java:842)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:162)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:128)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcD
ataSource.java:363)
at
org.apache.solr.handler.dataimport.JdbcDataSource.access$300(JdbcData
Source.java:39)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:240)
... 11 more
Aug 2, 2010 11:29:25 PM org.apache.solr.handler.dataimport.DataImporter
doFullIm
port
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute
 query: select * from locations Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
Throw(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEn
tityProcessor.java:58)

Re: SolrCore has a large number of SolrIndexSearchers retained in "infoRegistry"


: 1. QueryComponent#process
:  Is requesting for a SolrIndexSearcher twice by calling
: SolrQueryRequest#getSearcher() but is never being closed. I see several
: instances where getSearcher is being called but is never being properly
: closed - performing a quick call heirarchy of SolrQueryRequest#getSearcher()
: and SolrQueryRequest#close() will illustrate this point.

i believe you missunderstood my point -- i never said code that calls 
SolrQueryRequest#getSearcher needs to call SolrQueryRequest#close ...  i 
said that SolrIndexSearcher methods are ref counted, and that they those 
refrence counts needto be dealt with properly to ensure that the 
SOlrIndexSearchers are closed properly -- the fact that you are using 
embedded solr, and the fact that you are calling SolrCore.execute means 
that you *must* (by defintion) be instnatiating your own instances of 
SolrQueryRequest, which means thta you *must* (by definition) be "getting" 
a SolrIndexSearcher object from some method that requires you to "release" 
that SolrIndexSearcher when you are done.  If you use Solr as a server, it 
takes care of calling SolrQueryRequest.close() for you and it takes care 
of releasing the SolrIndexSearcher assocaited with that SolrQueryRequest 
object.

I realize now that i wasn't explicit in my previous response but there is 
absolutely *no* way that anyone can provide any meaninful assistance to 
you in understanding these errors w/o you showing us *exactly* how you are 
using solr, and what your custom code looksl ike that you have embedded 
solr in -- we need to see how you instantiate your SOlrCore, we need to 
see all of the code involved when you call the execute method, we need to 
see where you get your SolrQueryRequest object from, we need to see where 
you get your SolrIndexSearcher from ... all of it.

As i said: these types of resources leaks are entirely dependent on *how* 
you use the code ... if you were using Solr as a server, we would know 
exactly how the objects are being accessed/released because that code is 
in SOlr -- but when you embedded Solr in custom code we have nothing to go 
on but what you tell us.



-Hoss

Re: StatsComponent and sint?


: With an sint, it seems to have trouble if there are any documents with 
: null values for the field. It appears to decide that a null/empty/blank 
: value is -1325166535, and is thus the minimum value.

1) there is relaly no such thing as a "null" value for a field ... there 
are documents that have no value for that field -- but that's differnet 
then actually indexing a null value (Solr is not a RDBMS)

I attempted to reproduce the problem you are describing by chaning the 
solr 1.4.1 schema.xml so that the "popularity" field used type "sint" and 
then indexed all of the sample documents.  exactly one of those documents 
has no value for hte "popularity" field (id:UTF8TEST) and this is the 
results that i got from the following reuqest...

http://localhost:8983/solr/select/?wt=json&q=*%3A*%0D%0A&version=2.2&start=0&rows=00&indent=on&stats=true&stats.field=popularity
{
 "responseHeader":{
  "status":0,
  "QTime":1,
  "params":{
"indent":"on",
"start":"0",
"q":"*:*\r\n",
"stats":"true",
"stats.field":"popularity",
"wt":"json",
"version":"2.2",
"rows":"00"}},
 "response":{"numFound":19,"start":0,"docs":[]
 },
 "stats":{
  "stats_fields":{
"popularity":{
 "min":0.0,
 "max":10.0,
 "sum":102.0,
 "count":18,
 "missing":1,
 "sumOfSquares":702.0,
 "mean":5.667,
 "stddev":2.700762419587999

As you can see, it correclty recognized that the "min" value was 0.0, and 
thta 1 of the 19 total docs had no value for that field.


If you can't reproduce these types of results with your own data, then we 
need to see a lot more details about your specific sitaution (schema.xml, 
raw data, query urls, results, etc...) to try and understand what you are 
seeing.


-Hoss

RE: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'

2010-08-02 Thread Michael Griffiths

Peter:
My datasource looks like this:

From: PeterKerk [vettepa...@hotmail.com]
Sent: Monday, August 02, 2010 4:10 PM
To: solr-user@lucene.apache.org
Subject: RE: Indexing data on MSSQL failed: Caused by: 
org.apache.solr.common.SolrException: Error loading class 'com.micros 
oft.sqlserver.jdbc.SQLServerDriver'

@Michael: your suggestion to place the jar file in that folder has brought me
1 step further. Now I get a new error.
It seems as if the user cannot log in. Strange because it IS the correct
pwd.
In my MSSQL mgmt studio I provide:
server name: (local)
login: sa
password:123456

Now if you look at my dataconfig:

I also tried:

But with the same result.

Is the driver name also still correct the way I use it now?

See below my error log:

INFO: Starting Full Import
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.SolrWriter
readIndexe
rProperties
INFO: Read dataimport.properties
Aug 2, 2010 10:06:42 PM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [db] REMOVING ALL DOCUMENTS FROM INDEX
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1

commit{dir=C:\apache-solr-1.4.1\example\example-DIH\solr\db\data\index,s
egFN=segments_n,version=1279027246872,generation=23,filenames=[_l.nrm,
_l.tis, _
l.fnm, _l.prx, _l.fdx, segments_n, _l.fdt, _l.frq, _l.tii]
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1279027246872
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={command=full-import}
status=0 Q
Time=0
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call

INFO: Creating a connection for entity location with URL:
jdbc:sqlserver://local
host:1433;databaseName=wedding
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.DocBuilder
buildDocum
ent
SEVERE: Exception while processing: location document :
SolrInputDocument[{}]
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute
 query: select * from locations Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
Throw(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEn
tityProcessor.java:58)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEnti
tyProcessor.java:71)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
ityProcessorWrapper.java:237)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:357)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:242)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:180)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:331)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:389)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:370)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for
use
r 'sa'.
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
(SQLServerException.java:197)
at
com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246
)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerC
onnection.java:2529)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConne
ction.java:1905)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServer
Connection.java:41)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecu
te(SQLServerConnection.java:1893)
at
com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
rverConnection.java:1400)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLSer
verConnection.java:1045)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConne
ction.java:817)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerCon
nection.java:700)
at
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.
java:842)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:162)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:128)

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'


And found another thing, even when I have this in my data-config.xml (a
totally deformed XML file), I still get the same login failed error...where
I would expect something like a non-well formed XML document error...
Im using cygwin...dont know if that caches anything, but its still weird I
thought :)
























-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-data-on-MSSQL-failed-Caused-by-org-apache-solr-common-SolrException-Error-loading-class-com-tp1015137p1017302.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'


I just cant seem to get past this error!
So perhaps if any of you are running on Windows with SQL Server 2008, maybe
you could post your data-config?
I could have a look at it so I at least know my data-config is configured
correctly :)

Thanks a bunch!
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-data-on-MSSQL-failed-Caused-by-org-apache-solr-common-SolrException-Error-loading-class-com-tp1015137p1017295.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: min/max, StatsComponent, performance


: So one question is if there's any way to increase StatsComponent performance.
: Does it use any caches, or does it operate without caches?  My Solr is running

I believe it uses the field cache to allow fast lookup of numeric values 
for documents as it iterates through teh document set -- there's not 
really any sort of caching it can use that it isn't already.

: But it also occurs to me that the StatsComponent is doing a lot more than I
: need. I just need min/max. And the cardinality of this field is a couple
: orders of magnitude lower than the total number of documents. But

the cardnaliy of the values isn't really relevant -- it still has to check 
the value for every doc in your set to see what value it has.  

In things like faceting, term frequency can come into play becuase we can 
make optimizations to see if a given terms index wide frequency is less 
the our cut off, and if it is we can skip it completely w/o checking how 
many docs in our set contain that value -- that type of optimization isn't 
possible for min/max (although i suppose there is room for a possible 
imporvement of checking if the min we've found so far is the "global" min 
for that field, and if so don't bother checking nay docs ...  that seems 
like a really niche special case optimization, but if you want to submit a 
patch it might be useful.

Honestly: if you have a really small cardinality for these numeric 
values (ie: small enough to return every value on every request) perhaps 
you should use faceting to find the min/max values (with facet.mincount=1) 
instead of starts?

: StatsComponent is also doing a bunch of other things, like sum, median, etc.
: Perhaps if there were a way to _just_ get min/max, it would be faster. Is
: there any way to get min/max values in a result set other than StatsComponent?

I don't think so .. i belive Ryan considered this when he firsted added 
StatsComponent, but he decided it wasn't really worth the trouble -- all 
of the stats are computed in a single pass, and the majority of the time 
is spent getting the value of every doc in the set -- adding each value to 
a running total (for the sum and ultimatley computing the median) is a 
really cheap operation compared to the actaul iteration over the set.

That said: if you wanna work on a patch and can demonstrate that making 
these things configurable has performance improvements in the special 
case w/o hurting performance in the default case, i don't think anyone 
will argue against it.




-Hoss

Re: SpatialSearch: sorting by distance


: sort=dist(2,lat,lon,55.755786,37.617633) asc
: 
: In general results are sorted, but some documents are not in right order.
: I'm using DistanceUtils.getDistanceMi(...) from lucene spatial to calculate
: real distance after reading documents from Solr.

according ot the wiki for hte "dist" function...

"Return the Distance between two Vectors (points) 
in an n-dimensional space."

..and the "2" param in your call means it's going to use the Euclidean 
Distance.

where as DistanceUtils.getDistanceMi (allthough undocumented) suggests 
that it will return a distance in miles (skimming the code seems to bear 
this out ... it just calls getLLMDistance which calls p1.arcDistance( p2, 
DistanceUnits.MILES)

I know very little about the spatial package, but i know enough to know 
that a vector distance between two arbitrary points in a two dimensional 
space is not the same as the distance between two points on *earth* in 
miles.

I believe what you want to be using is the "hsin" function

In older versions of SOlr Spatial support, in order for it to work with 
"miles on earth" you had to provide the "radius of the earth in miles" as 
a param ... i believe that is still possible (and how you cna get the 
poutput in other units, like km, or feet), but on the current trunk just 
specifying the points should give you the distance in miles by default.



-Hoss

Re: Problems running on tomcat

2010-08-02 Thread Claudio Devecchi

Hi Ahmet,

Works with tomcat6.

Tks!

On Mon, Aug 2, 2010 at 3:04 PM, Claudio Devecchi wrote:

> Hi Ahmet,
>
> I'm using tomcat7 with solr 1.4.1 =\
> If you worked fine on tomcat6 I'll try with the same version...
>
> Tks for helping..
>
>
> On Mon, Aug 2, 2010 at 2:30 PM, Ahmet Arslan  wrote:
>
>> What is version of solr and tomcat?
>>
>> I think i saw same problem with tomcat 7 and solr 1.4.1 combination, thats
>> why i am asking.
>>
>> I just tried to replicate this problem with tomcat 6 and solr 1.4.1, but
>> everything went fine.
>>
>> --- On Fri, 7/30/10, Claudio Devecchi  wrote:
>>
>> > From: Claudio Devecchi 
>> > Subject: Problems running on tomcat
>> > To: solr-user@lucene.apache.org
>> > Date: Friday, July 30, 2010, 10:17 PM
>> > Hi,
>> >
>> > I'm new with solr and I'm doing my first installation under
>> > tomcat, I
>> > followed the documentation on link (
>> > http://wiki.apache.org/solr/SolrTomcat#Installing_Tomcat_6)
>> > but there are
>> > some problems.
>> > The http://localhost:8080/solr/admin works
>> > fine, but in some cases, for
>> > example to see my schema.xml from the admin console the
>> > error bellow
>> > happensHTTP
>> > Status 404 - /solr/admin/file/index.jspSomebody already saw
>> > this? There are
>> > some trick to do?
>> >
>> > Tks
>> >
>> > --
>> > Claudio Devecchi
>> >
>>
>>
>>
>>
>
>
> --
> Claudio Devecchi
> flickr.com/cdevecchi
>



-- 
Claudio Devecchi
flickr.com/cdevecchi

Re: does this indicate a commit happened for every add?


: Subject: does this indicate a commit happened for every add?
: In-Reply-To: 
: References: 

http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists

When starting a new discussion on a mailing list, please do not reply to 
an existing message, instead start a fresh email.  Even if you change the 
subject line of your email, other mail headers still track which thread 
you replied to and your question is "hidden" in that thread and gets less 
attention.   It makes following discussions in the mailing list archives 
particularly difficult.
See Also:  http://en.wikipedia.org/wiki/User:DonDiego/Thread_hijacking



-Hoss

Re: Index compatibility 1.4 Vs 3.1 Trunk

: I am trying to use the solr code from '
: https://svn.apache.org/repos/asf/lucene/dev/trunk' as my design warrants use
: of PolyType fields. My understanding is that the indexes are incompatible,
: am I right ?. I have about a million docs in my index (indexed via solr
: 1.4). Is re-indexing my only option or is there a tool of some sort to
: convert the 1.4 index to 3.1 format ?

a) the "trunk" is what will ultimately be Solr 4.x, not 3.x ... for the 
3.x line there is a 3x branch...

http://wiki.apache.org/solr/Solr3.1
http://wiki.apache.org/solr/Solr4.0

b) The 3x branch can read indexes created by Solr 1.4 -- the first time 
you add a doc and commit the new segments wil automaticly be converted to 
the new format.  I am fairly certian that as of this moment, the 4x trunk 
can also read indexes created by Solr 1.4, with the same automatic 
converstion taking place.

c)  If/When the trunk can no longer read Solr 1.4 indexes, there will be 
a tool provided for "upgrading" index versions.


-Hoss

RE: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'


@Michael: your suggestion to place the jar file in that folder has brought me
1 step further. Now I get a new error.
It seems as if the user cannot log in. Strange because it IS the correct
pwd.
In my MSSQL mgmt studio I provide:
server name: (local)
login: sa
password:123456

Now if you look at my dataconfig:



I also tried:


But with the same result.

Is the driver name also still correct the way I use it now?

See below my error log:



INFO: Starting Full Import
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.SolrWriter
readIndexe
rProperties
INFO: Read dataimport.properties
Aug 2, 2010 10:06:42 PM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [db] REMOVING ALL DOCUMENTS FROM INDEX
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
   
commit{dir=C:\apache-solr-1.4.1\example\example-DIH\solr\db\data\index,s
egFN=segments_n,version=1279027246872,generation=23,filenames=[_l.nrm,
_l.tis, _
l.fnm, _l.prx, _l.fdx, segments_n, _l.fdt, _l.frq, _l.tii]
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1279027246872
Aug 2, 2010 10:06:42 PM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={command=full-import}
status=0 Q
Time=0
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call

INFO: Creating a connection for entity location with URL:
jdbc:sqlserver://local
host:1433;databaseName=wedding
Aug 2, 2010 10:06:42 PM org.apache.solr.handler.dataimport.DocBuilder
buildDocum
ent
SEVERE: Exception while processing: location document :
SolrInputDocument[{}]
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute
 query: select * from locations Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
Throw(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:253)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:210)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSou
rce.java:39)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEn
tityProcessor.java:58)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEnti
tyProcessor.java:71)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
ityProcessorWrapper.java:237)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:357)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:242)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:180)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:331)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:389)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:370)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for
use
r 'sa'.
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError
(SQLServerException.java:197)
at
com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246
)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerC
onnection.java:2529)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConne
ction.java:1905)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServer
Connection.java:41)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecu
te(SQLServerConnection.java:1893)
at
com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
rverConnection.java:1400)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLSer
verConnection.java:1045)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConne
ction.java:817)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerCon
nection.java:700)
at
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.
java:842)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:162)
at
org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSour
ce.java:128)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcD
ataSource.java:363)
at
org.apache.solr.handler.dataimport.JdbcDataSource.access$300(JdbcData
Source.java:39)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<
init>(JdbcDataSource.java:240)
... 11 more
Aug 2, 2010 10:06:42 P

Re: Phrase search


I'm using Solr 1.2, so I don't have splitOnNumerics.  Reading that URL, is my 
use of catenateNumbers="1" causing this?  Should I set it to "0" vs. "1" as I 
have it now?
 
-M




-Original Message-
From: Markus Jelsma 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 3:54 pm
Subject: RE: Re: Phrase search


Hi,
 
Queries on an analyzed field will need to be analyzed as well or it might not 
atch. You can configure the WordDelimiterFilterFactory so it will not split 
nto multiple tokens because of numerics, see the splitOnNumerics parameter [1].
 
[1]: 
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactory
 
Cheers,


Original message-
rom: johnmu...@aol.com
ent: Mon 02-08-2010 21:29
o: solr-user@lucene.apache.org; 
ubject: Re: Phrase search


Thanks for the quick response.
Which part of my WordDelimiterFilterFactory is changing "Apple 2" to "Apple2"? 
How do I fix it?  Also, I'm really confused about this.  I was under the 
mpression a phrase search is not impacted by the analyzer, no?
-M

Original Message-
rom: Markus Jelsma 
o: solr-user@lucene.apache.org
ent: Mon, Aug 2, 2010 2:27 pm
ubject: RE: Phrase search

ell, the WordDelimiterFilterFactory in your query analyzer clearly makes "Apple 
" out of "Apple2", that's what it's for. If you're looking for an exact match, 
e a string field. Check the output with the debugQuery=true parameter.
Cheers, 
Original message-
om: johnmu...@aol.com
nt: Mon 02-08-2010 20:18
: solr-user@lucene.apache.org; 
bject: Phrase search
i All,
 don't understand why i'm getting this behavior.  I was under the impression if 
search for "Apple 2" (with quotes and space before 2 ) it will give me 
fferent results vs. if I search for "Apple2" (with quotes and no space before 
, but I'm not!  Why? 
ere is my fieldType setting from my schema.xml:
 
  







  
  







  

hat I am missing?!!  What part of my solr.WordDelimiterFilterFactory need to 
ange (if that s where the issue is)?
 m using Solr 1.2
hanks in advanced.
M

RE: Re: Phrase search

2010-08-02 Thread Markus Jelsma

Hi,

 

Queries on an analyzed field will need to be analyzed as well or it might not 
match. You can configure the WordDelimiterFilterFactory so it will not split 
into multiple tokens because of numerics, see the splitOnNumerics parameter [1].

 

[1]: 
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactory

 

Cheers,


 
-Original message-
From: johnmu...@aol.com
Sent: Mon 02-08-2010 21:29
To: solr-user@lucene.apache.org; 
Subject: Re: Phrase search





Thanks for the quick response.

Which part of my WordDelimiterFilterFactory is changing "Apple 2" to "Apple2"?  
How do I fix it?  Also, I'm really confused about this.  I was under the 
impression a phrase search is not impacted by the analyzer, no?

-M


-Original Message-
From: Markus Jelsma 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 2:27 pm
Subject: RE: Phrase search


Well, the WordDelimiterFilterFactory in your query analyzer clearly makes 
"Apple 
" out of "Apple2", that's what it's for. If you're looking for an exact match, 
se a string field. Check the output with the debugQuery=true parameter.

Cheers, 

Original message-
rom: johnmu...@aol.com
ent: Mon 02-08-2010 20:18
o: solr-user@lucene.apache.org; 
ubject: Phrase search

i All,
I don't understand why i'm getting this behavior.  I was under the impression 
if 
search for "Apple 2" (with quotes and space before 2 ) it will give me 
ifferent results vs. if I search for "Apple2" (with quotes and no space before 
), but I'm not!  Why? 
Here is my fieldType setting from my schema.xml:
  
   
     
     
     
     
     
     
     
   
   
     
     
     
     
     
     
     
   
 
What I am missing?!!  What part of my solr.WordDelimiterFilterFactory need to 
hange (if that s where the issue is)?
I m using Solr 1.2
Thanks in advanced.
-M

RE: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'

2010-08-02 Thread Michael Griffiths

I successfully used MSSQL JDBC by placing sqljdbc4.jar file in the lib 
directory, so e.g. example\lib\sqldbc4.jar.

Apart from that, my dataconfig looks very similar - I also set the CLASSPATH, 
which didn't work.

-Original Message-
From: PeterKerk [mailto:vettepa...@hotmail.com] 
Sent: Monday, August 02, 2010 5:23 AM
To: solr-user@lucene.apache.org
Subject: Indexing data on MSSQL failed: Caused by: 
org.apache.solr.common.SolrException: Error loading class 'com.micros 
oft.sqlserver.jdbc.SQLServerDriver'

I'm running MSSQL Server 2008

I did this: set CLASSPATH= C:\Program Files\Microsoft SQL Server JDBC Driver 
3.0\sqljdbc_3.0\enu\sqljdbc4.jar (I dont know where I can check if this has 
been executed correctly)

Then when approaching this URL:
http://localhost:8983/solr/db/dataimport?command=full-import

Using this data-config.xml:

I get this in my cygwin error log:

SEVERE: JdbcDataSource was not closed prior to finalize(), indicates a bug
-- PO
SSIBLE RESOURCE LEAK!!!
Aug 2, 2010 10:51:28 AM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={command=full-import}
status=0 Q
Time=0
Aug 2, 2010 10:51:28 AM org.apache.solr.handler.dataimport.DataImporter
doFullIm
port
INFO: Starting Full Import
Aug 2, 2010 10:51:28 AM org.apache.solr.handler.dataimport.SolrWriter
readIndexe
rProperties
INFO: Read dataimport.properties
Aug 2, 2010 10:51:28 AM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [db] REMOVING ALL DOCUMENTS FROM INDEX Aug 2, 2010 10:51:28 AM 
org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1

commit{dir=C:\apache-solr-1.4.1\example\example-DIH\solr\db\data\index,s
egFN=segments_n,version=1279027246872,generation=23,filenames=[_l.nrm,
_l.tis, _
l.fnm, _l.prx, _l.fdx, segments_n, _l.fdt, _l.frq, _l.tii] Aug 2, 2010 10:51:28 
AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1279027246872
Aug 2, 2010 10:51:28 AM org.apache.solr.handler.dataimport.DataImporter
doFullIm
port
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load dr
iver: com.microsoft.sqlserver.jdbc.SQLServerDriver Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAnd
Throw(DataImportHandlerException.java:72)
at
org.apache.solr.handler.dataimport.JdbcDataSource.createConnectionFac
tory(JdbcDataSource.java:114)
at
org.apache.solr.handler.dataimport.JdbcDataSource.init(JdbcDataSource
.java:62)
at
org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance
(DataImporter.java:304)
at
org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextI
mpl.java:94)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityP
rocessor.java:52)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(Entity
ProcessorWrapper.java:71)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:319)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:242)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:180)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:331)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:389)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:370)
Caused by: java.lang.ClassNotFoundException: Unable to load 
com.microsoft.sqlser ver.jdbc.SQLServerDriver or 
org.apache.solr.handler.dataimport.com.microsoft.sql
server.jdbc.SQLServerDriver
at
org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.ja
va:738)
at
org.apache.solr.handler.dataimport.JdbcDataSource.createConnectionFac
tory(JdbcDataSource.java:112)
... 11 more
Caused by: org.apache.solr.common.SolrException: Error loading class 
'com.micros oft.sqlserver.jdbc.SQLServerDriver'
at
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.
java:375)
at
org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.ja
va:728)
... 12 more
Caused by: java.lang.ClassNotFoundException:
com.microsoft.sqlserver.jdbc.SQLSer
verDriver
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.net.FactoryURLClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'


Hi Gora,

I just ran the classpath command in a DOS command prompt, that normally
works for regular PATH variable as well.
Your suggestion for the open source JTDS driver sounds good.

So if you look at my data-config.xml:




What would I have to place there under driver property?

Also Im still trying to set up my initial solr configuration, folder
structure like so:
C:\apache-solr-1.4.1\example\example-DIH\solr\db\lib

So where would I need to place the jar file? Is that in the \lib folder I
indicated above?

Thanks!

Pete
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-data-on-MSSQL-failed-Caused-by-org-apache-solr-common-SolrException-Error-loading-class-com-tp1015137p1017061.html
Sent from the Solr - User mailing list archive at Nabble.com.

heads up -- index corruption on Solr/Lucene trunk/3.x branch

2010-08-02 Thread Michael McCandless

This issue:

https://issues.apache.org/jira/browse/LUCENE-2574

which was committed 3 days ago (Friday Jul 30) can cause index corruption.

I just committed a fix for the corruption, but if you've been using
Solr/Lucene trunk or 3x branch updated after the first commit on
Friday, and you built any indexes with it, you should rebuild them!

Sorry,

Mike

Re: Phrase search





Thanks for the quick response.

Which part of my WordDelimiterFilterFactory is changing "Apple 2" to "Apple2"?  
How do I fix it?  Also, I'm really confused about this.  I was under the 
impression a phrase search is not impacted by the analyzer, no?

-M


-Original Message-
From: Markus Jelsma 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 2:27 pm
Subject: RE: Phrase search


Well, the WordDelimiterFilterFactory in your query analyzer clearly makes 
"Apple 
" out of "Apple2", that's what it's for. If you're looking for an exact match, 
se a string field. Check the output with the debugQuery=true parameter.
 
Cheers, 

Original message-
rom: johnmu...@aol.com
ent: Mon 02-08-2010 20:18
o: solr-user@lucene.apache.org; 
ubject: Phrase search

i All,
I don't understand why i'm getting this behavior.  I was under the impression 
if 
 search for "Apple 2" (with quotes and space before 2 ) it will give me 
ifferent results vs. if I search for "Apple2" (with quotes and no space before 
 ), but I'm not!  Why? 
Here is my fieldType setting from my schema.xml:
   

  
  
  
  
  
  
  


  
  
  
  
  
  
  

  
What I am missing?!!  What part of my solr.WordDelimiterFilterFactory need to 
hange (if that s where the issue is)?
I m using Solr 1.2
Thanks in advanced.
-M

Re: Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'

2010-08-02 Thread Gora Mohanty

On Mon, 2 Aug 2010 02:22:50 -0700 (PDT)
PeterKerk  wrote:

> 
> 
> I'm running MSSQL Server 2008
> 
> I did this: set CLASSPATH= C:\Program Files\Microsoft SQL Server
> JDBC Driver 3.0\sqljdbc_3.0\enu\sqljdbc4.jar (I dont know where I
> can check if this has been executed correctly)

Not familiar with Microsoft Windows, but where did you set the
CLASSPATH? I would suspect that Solr/(Tomcat|Jetty) is probably not
aware of this.

Also, the MS-SQL driver might need other DLLs. I have only checked
out installation instructions for the MS-SQL driver on Unix, and
they were not entirely clear. 

You could also consider using the more robust, and efficient
open-source JTDS driver ( http://jtds.sourceforge.net/ )

Regards,
Gora

how to highlight string in jsp

2010-08-02 Thread Ma, Xiaohui (NIH/NLM/LHC) [C]

Hello,

I am trying to display the highlight string in different color on jsp. I use 
following in servlet.

query.setHighlight(true).setHighlightSnippets(1);
query.setParam("hl.fl", "Abstract");

I wonder how I can display it in jsp

Thanks in advanced.
xm

RE: Phrase search

2010-08-02 Thread Markus Jelsma

Well, the WordDelimiterFilterFactory in your query analyzer clearly makes 
"Apple 2" out of "Apple2", that's what it's for. If you're looking for an exact 
match, use a string field. Check the output with the debugQuery=true parameter.

 

Cheers, 
 
-Original message-
From: johnmu...@aol.com
Sent: Mon 02-08-2010 20:18
To: solr-user@lucene.apache.org; 
Subject: Phrase search


Hi All,

I don't understand why i'm getting this behavior.  I was under the impression 
if I search for "Apple 2" (with quotes and space before 2 ) it will give me 
different results vs. if I search for "Apple2" (with quotes and no space before 
2 ), but I'm not!  Why? 

Here is my fieldType setting from my schema.xml:

   
     
       
       
       
       
       
       
       
     
     
       
       
       
       
       
       
       
     
   

What I am missing?!!  What part of my solr.WordDelimiterFilterFactory need to 
change (if that s where the issue is)?

I m using Solr 1.2

Thanks in advanced.

-M

Phrase search


Hi All,
 
I don't understand why i'm getting this behavior.  I was under the impression 
if I search for "Apple 2" (with quotes and space before “2”) it will give me 
different results vs. if I search for "Apple2" (with quotes and no space before 
“2”), but I'm not!  Why? 
 
Here is my fieldType setting from my schema.xml:


  







  
  







  

 
What I am missing?!!  What part of my solr.WordDelimiterFilterFactory need to 
change (if that’s where the issue is)?
 
I’m using Solr 1.2
 
Thanks in advanced.
 
-M

Re: Problems running on tomcat

2010-08-02 Thread Claudio Devecchi

Hi Ahmet,

I'm using tomcat7 with solr 1.4.1 =\
If you worked fine on tomcat6 I'll try with the same version...

Tks for helping..

On Mon, Aug 2, 2010 at 2:30 PM, Ahmet Arslan  wrote:

> What is version of solr and tomcat?
>
> I think i saw same problem with tomcat 7 and solr 1.4.1 combination, thats
> why i am asking.
>
> I just tried to replicate this problem with tomcat 6 and solr 1.4.1, but
> everything went fine.
>
> --- On Fri, 7/30/10, Claudio Devecchi  wrote:
>
> > From: Claudio Devecchi 
> > Subject: Problems running on tomcat
> > To: solr-user@lucene.apache.org
> > Date: Friday, July 30, 2010, 10:17 PM
> > Hi,
> >
> > I'm new with solr and I'm doing my first installation under
> > tomcat, I
> > followed the documentation on link (
> > http://wiki.apache.org/solr/SolrTomcat#Installing_Tomcat_6)
> > but there are
> > some problems.
> > The http://localhost:8080/solr/admin works
> > fine, but in some cases, for
> > example to see my schema.xml from the admin console the
> > error bellow
> > happensHTTP
> > Status 404 - /solr/admin/file/index.jspSomebody already saw
> > this? There are
> > some trick to do?
> >
> > Tks
> >
> > --
> > Claudio Devecchi
> >
>
>
>
>


-- 
Claudio Devecchi
flickr.com/cdevecchi

Re: Problems running on tomcat

2010-08-02 Thread Ahmet Arslan

What is version of solr and tomcat? 

I think i saw same problem with tomcat 7 and solr 1.4.1 combination, thats why 
i am asking.

I just tried to replicate this problem with tomcat 6 and solr 1.4.1, but 
everything went fine.

--- On Fri, 7/30/10, Claudio Devecchi  wrote:

> From: Claudio Devecchi 
> Subject: Problems running on tomcat
> To: solr-user@lucene.apache.org
> Date: Friday, July 30, 2010, 10:17 PM
> Hi,
> 
> I'm new with solr and I'm doing my first installation under
> tomcat, I
> followed the documentation on link (
> http://wiki.apache.org/solr/SolrTomcat#Installing_Tomcat_6)
> but there are
> some problems.
> The http://localhost:8080/solr/admin works
> fine, but in some cases, for
> example to see my schema.xml from the admin console the
> error bellow
> happensHTTP
> Status 404 - /solr/admin/file/index.jspSomebody already saw
> this? There are
> some trick to do?
> 
> Tks
> 
> -- 
> Claudio Devecchi
>

Re: Solr searching performance issues, using large documents

2010-08-02 Thread Peter Spam

What would happen if the search query phrase spanned separate document chunks?

Also, what would the optimal size of chunks be?

Thanks!


-Peter

On Aug 1, 2010, at 7:21 PM, Lance Norskog wrote:

> Not that I know of.
> 
> The DataImportHandler has the ability to create multiple documents
> from one input stream. It is possible to create a DIH file that reads
> large log files and splits each one into N documents, with the file
> name as a common field. The DIH wiki page tells you in general how to
> make a DIH file.
> 
> http://wiki.apache.org/solr/DataImportHandler
> 
> From this, you should be able to make a DIH file that puts log files
> in as separate documents. As to splitting files up into
> mini-documents, you might have to write a bit of Javascript to achieve
> this. There is no data structure or software that implements
> structured documents.
> 
> On Sun, Aug 1, 2010 at 2:06 PM, Peter Spam  wrote:
>> Thanks for the pointer, Lance!  Is there an example of this somewhere?
>> 
>> 
>> -Peter
>> 
>> On Jul 31, 2010, at 3:13 PM, Lance Norskog wrote:
>> 
>>> Ah! You're not just highlighting, you're snippetizing. This makes it easier.
>>> 
>>> Highlighting does not stream- it pulls the entire stored contents into
>>> one string and then pulls out the snippet.  If you want this to be
>>> fast, you have to split up the text into small pieces and only
>>> snippetize from the most relevant text. So, separate documents with a
>>> common group id for the document it came from. You might have to do 2
>>> queries to achieve what you want, but the second query for the same
>>> query will be blindingly fast. Often <1ms.
>>> 
>>> Good luck!
>>> 
>>> Lance
>>> 
>>> On Sat, Jul 31, 2010 at 1:12 PM, Peter Spam  wrote:
 However, I do need to search the entire document, or else the highlighting 
 will sometimes be blank :-(
 Thanks!
 
 - Peter
 
 ps. sorry for the many responses - I'm rushing around trying to get this 
 working.
 
 On Jul 31, 2010, at 1:11 PM, Peter Spam wrote:
 
> Correction - it went from 17 seconds to 10 seconds - I was changing the 
> hl.regex.maxAnalyzedChars the first time.
> Thanks!
> 
> -Peter
> 
> On Jul 31, 2010, at 1:06 PM, Peter Spam wrote:
> 
>> On Jul 30, 2010, at 1:16 PM, Peter Karich wrote:
>> 
>>> did you already try other values for hl.maxAnalyzedChars=2147483647
>> 
>> Yes, I tried dropping it down to 21, but it didn't have much of an 
>> impact (one search I just tried went from 17 seconds to 15.8 seconds, 
>> and this is an 8-core Mac Pro with 6GB RAM - 4GB for java).
>> 
>>> ? Also regular expression highlighting is more expensive, I think.
>>> What does the 'fuzzy' variable mean? If you use this to query via
>>> "~someTerm" instead "someTerm"
>>> then you should try the trunk of solr which is a lot faster for fuzzy or
>>> other wildcard search.
>> 
>> "fuzzy" could be set to "*" but isn't right now.
>> 
>> Thanks for the tips, Peter - this has been very frustrating!
>> 
>> 
>> - Peter
>> 
>>> Regards,
>>> Peter.
>>> 
 Data set: About 4,000 log files (will eventually grow to millions).  
 Average log file is 850k.  Largest log file (so far) is about 70MB.
 
 Problem: When I search for common terms, the query time goes from 
 under 2-3 seconds to about 60 seconds.  TermVectors etc are enabled.  
 When I disable highlighting, performance improves a lot, but is still 
 slow for some queries (7 seconds).  Thanks in advance for any ideas!
 
 
 -Peter
 
 
 -
 
 4GB RAM server
 % java -Xms2048M -Xmx3072M -jar start.jar
 
 -
 
 schema.xml changes:
 
  

  

>>> generateWordParts="0" generateNumberParts="0" catenateWords="0" 
 catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>

  
 
 ...
 
 >>> multiValued="false" termVectors="true" termPositions="true" 
 termOffsets="true" />
  >>> default="NOW" multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 >>> multiValued="false"/>
 
 ...
 
 
 body
 
 
 -

Re: Stress Test Solr

2010-08-02 Thread mbohlig

Tomas,

This seems like a really useful tool. 

We're holding a Solr/Lucene conference in Boston, 7-8 October 2010 - and this 
would be an interesting lightning talk subject. 


Let me know if you need more info on the event.

- Mike Bohlig, Lucid Imagination




- Original Message 
From: Tomas 
To: solr-user@lucene.apache.org
Sent: Mon, August 2, 2010 8:33:46 AM
Subject: Stress Test Solr

Hi All, we've been building an open source tool for load tests on Solr 
Installations. Thetool is called SolrMeter. It's on google code 
at http://code.google.com/p/solrmeter/. Here is some information about it:

SolrMeter is an stress testing / performance benchmarking tool for Apache Solr 
installations.  It is licensed under ASL and developed using JavaSE and Swing 
components, connected with Solr using SolrJ.

What can youdowith SolrMeter?
The main goal of this open source project is bring to the Apache Solr user 
community a tool for dealing with Solr specific issues regarding performance 
and 

stress testing like firing queries and adding documents to make sure that your 
Solr instalation will support real world's load and demands. With SolrMeter you 
can simulate a work load over the Apache Solr instalation and to obtain useful 
visual performance statistics and metrics.
Relevant Features:
* Execute queries against a Solr installation
* Execute dummy updates/inserts to the Solr installation, it can be the 
same 

server as the queries or a different one.
* Configure number of queries to fire in a time period interval
* Configure the number of updates/inserts in a time period.
* Configure commits frequency during adds
* Monitor error counts when adding and commiting documents.
* Perform and monitor index optimization
* Monitor query times online and visually
* Add filter queries into the test queries
* Add facet abilities into the test queries
* Import/Export test configuration
* Query time execution histogram chart
* Query times distribution chart
* Online error log and browsing capabilities
* Individual query graphical log and statistics
* and much more

Whatdo you need for use SolrMeter?
This is one of the most interesting points about SolrMeter, the requirements 
are 

minimal. It is simple to install and use.
* JRE versión 1.6
* The Solr Server you want to test.

Who can use SolrMeter?
Everyone who needs to assess the solrmeter server performance. To run the tool 
you only need to know about SOLR.



Try it and tell us what you think . . . . .  

Solrmeter Group
mailto:solrme...@googlegroups.com

What's next?
We are now building version 0.2.0, the objetive of this new version is to 
evolve 

SolrMeter into a pluggable architecture to allow deeper customizations like 
adding custom statistics, extractors or executors.
We are also adding some usability improvements.

On future versions we want to add a better interaction with Solr request 
handlers, for example, showing cache statistics online and graphically on some 
chart would be a great tool.
We also want to add more usability features to make of solrmeter a complete 
tool 

for testing a Solr instalation.
For more details on what's next che the Issues page on the google code site.

Register now for Surge 2010

2010-08-02 Thread Jason Dixon

Registration for Surge Scalability Conference 2010 is open for all
attendees!  We have an awesome lineup of leaders from across the various
communities that support highly scalable architectures, as well as the
companies that implement them.  Here's a small sampling from our list of
speakers:

John Allspaw, Etsy
Theo Schlossnagle, OmniTI
Rasmus Lerdorf, creator of PHP
Tom Cook, Facebook
Benjamin Black, fast_ip
Artur Bergman, Wikia
Christopher Brown, Opscode
Bryan Cantrill, Joyent
Baron Schwartz, Percona
Paul Querna, Cloudkick

Surge 2010 focuses on real case studies from production environments;
the lessons learned from failure and how to re-engineer your way to a
successful, highly scalable Internet architecture.  The conference takes
place at the Tremont Grand Historic Venue on Sept 30 and Oct 1, 2010 in
Baltimore, MD.  Register now to enjoy the Early Bird discount and
guarantee your seat to this year's event!

http://omniti.com/surge/2010/register

Thanks,

-- 
Jason Dixon
OmniTI Computer Consulting, Inc.
jdi...@omniti.com
443.325.1357 x.241

Re: enhancing auto complete

2010-08-02 Thread scrapy

Ok i'm still interested of the design
 

 


 

 

-Original Message-
From: Avlesh Singh 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 5:20 pm
Subject: Re: enhancing auto complete


Hahaha ... sorry its not. And there is no readymade code that I can give you

either. But yes, if you liked it, I can share the design of this feature

(solr, backend and frontend).



Cheers

Avlesh

@avlesh  | http://webklipper.com



On Mon, Aug 2, 2010 at 8:47 PM,  wrote:



>

>  Hi, I'm also interested of this feature... is it open source?

>

>

>

>

>

>

>

>

> -Original Message-

> From: Avlesh Singh 

> To: solr-user@lucene.apache.org

> Sent: Mon, Aug 2, 2010 5:09 pm

> Subject: Re: enhancing auto complete

>

>

> From whatever I could read in your broken table of sample use cases, I

> think

>

> you are looking for something similar to what has been done here -

>

> http://askme.in; if this is what you are looking do let me know.

>

>

>

> Cheers

>

> Avlesh

>

> @avlesh  | http://webklipper.com

>

>

>

> On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar <

>

> bhavnik.gaj...@gatewaynintec.com> wrote:

>

>

>

> > Hi,

>

> >

>

> > I'm looking for a solution related to auto complete feature for one

>

> > application.

>

> >

>

> > Below is a list of texts from which auto complete results would be

>

> > populated.

>

> >

>

> > Lorem ipsum dolor sit amet

>

> > tincidunt ut laoreet

>

> > dolore eu feugiat nulla facilisis at vero eros et

>

> > te feugait nulla facilisi

>

> > Claritas est etiam processus

>

> > anteposuerit litterarum formas humanitatis

>

> > fiant sollemnes in futurum

>

> > Hieyed ddi lorem ipsum dolor

>

> > test lorem ipsume

>

> > test xyz lorem ipslili

>

> >

>

> > Consider below table. First column describes user entered value and

>

> > second column describes expected result (list of auto complete terms

>

> > that should be populated from Solr)

>

> >

>

> > lorem

>

> >*Lorem* ipsum dolor sit amet

>

> > Hieyed ddi *lorem* ipsum dolor

>

> > test *lorem *ipsume

>

> > test xyz *lorem *ipslili

>

> > lorem ip

>

> >*Lorem ip*sum dolor sit amet

>

> > Hieyed ddi *lorem ip*sum dolor

>

> > test *lorem ip*sume

>

> > test xyz *lorem ip*slili

>

> > lorem ipsl

>

> >test xyz *lorem ipsl*ili

>

> >

>

> >

>

> >

>

> > Can anyone share ideas of how this can be achieved with Solr? Already

>

> > tried with various tokenizers and filter factories like,

>

> > WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,

>

> > ShingleFilterFactory etc. but no luck so far..

>

> >

>

> > Note that, It would be excellent if terms populated from Solr can be

>

> > highlighted by using Highlighting or any other component/mechanism of

> Solr.

>

> >

>

> > *Note :* Standard autocomplete (like,

>

> > facet.field=AutoComplete&f.AutoComplete.facet.prefix=

> > term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already

>

> > working fine with the application. but, nowadays, looking for enhancing

>

> > the existing auto complete stuff with the above requirement.

>

> >

>

> > Any thoughts?

>

> >

>

> > Thanks in advance

>

> >

>

> >

>

> >

>

> >

>

> > The contents of this eMail including the contents of attachment(s) are

>

> > privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL)

> and

>

> > should not be disclosed to, used by or copied in any manner by anyone

> other

>

> > than the intended addressee(s). If this eMail has been received by error,

>

> > please advise the sender immediately and delete it from your system. The

>

> > views expressed in this eMail message are those of the individual sender,

>

> > except where the sender expressly, and with authority, states them to be

> the

>

> > views of GNPL. Any unauthorized review, use, disclosure, dissemination,

>

> > forwarding, printing or copying of this eMail or any action taken in

>

> > reliance on this eMail is strictly prohibited and may be unlawful. This

>

> > eMail may contain viruses. GNPL has taken every reasonable precaution to

>

> > minimize this risk, but is not liable for any damage you may sustain as a

>

> > result of any virus in this eMail. You should carry out your own virus

>

> > checks before opening the eMail or attachment(s). GNPL is neither liable

> for

>

> > the proper and complete transmission of the information contained in this

>

> > communication nor for any delay in its receipt. GNPL reserves the right

> to

>

> > monitor and review the content of all messages sent to or from this eMail

>

> > address and may be stored on the GNPL eMail system. In case this eMail

> has

>

> > reached you in error, and you  would no longer like to receive eMails

> from

>

> > us, then please send an eMail to d...@gatewaynintec.com

>

> >

>

>

>

>

RE: Stress Test Solr

2010-08-02 Thread Markus Jelsma

Very interersting. Could you add some information and link to the relevant wiki 
page [1] ?

 

[1]: http://wiki.apache.org/solr/BenchmarkingSolr
 
-Original message-
From: Tomas 
Sent: Mon 02-08-2010 17:34
To: solr-user@lucene.apache.org; 
Subject: Stress Test Solr

Hi All, we've been building an open source tool for load tests on Solr 
Installations. Thetool is called SolrMeter. It's on google code 
at http://code.google.com/p/solrmeter/. Here is some information about it:

SolrMeter is an stress testing / performance benchmarking tool for Apache Solr 
installations.  It is licensed under ASL and developed using JavaSE and Swing 
components, connected with Solr using SolrJ.

What can youdowith SolrMeter?
The main goal of this open source project is bring to the Apache Solr user 
community a tool for dealing with Solr specific issues regarding performance 
and 
stress testing like firing queries and adding documents to make sure that your 
Solr instalation will support real world's load and demands. With SolrMeter you 
can simulate a work load over the Apache Solr instalation and to obtain useful 
visual performance statistics and metrics.
Relevant Features:
* Execute queries against a Solr installation
* Execute dummy updates/inserts to the Solr installation, it can be the same 
server as the queries or a different one.
* Configure number of queries to fire in a time period interval
* Configure the number of updates/inserts in a time period.
* Configure commits frequency during adds
* Monitor error counts when adding and commiting documents.
* Perform and monitor index optimization
* Monitor query times online and visually
* Add filter queries into the test queries
* Add facet abilities into the test queries
* Import/Export test configuration
* Query time execution histogram chart
* Query times distribution chart
* Online error log and browsing capabilities
* Individual query graphical log and statistics
* and much more

Whatdo you need for use SolrMeter?
This is one of the most interesting points about SolrMeter, the requirements 
are 
minimal. It is simple to install and use.
* JRE versión 1.6
* The Solr Server you want to test.

Who can use SolrMeter?
Everyone who needs to assess the solrmeter server performance. To run the tool 
you only need to know about SOLR.



Try it and tell us what you think . . . . .  

   Solrmeter Group
   mailto:solrme...@googlegroups.com

What's next?
We are now building version 0.2.0, the objetive of this new version is to 
evolve 
SolrMeter into a pluggable architecture to allow deeper customizations like 
adding custom statistics, extractors or executors.
We are also adding some usability improvements.

On future versions we want to add a better interaction with Solr request 
handlers, for example, showing cache statistics online and graphically on some 
chart would be a great tool.
We also want to add more usability features to make of solrmeter a complete 
tool 
for testing a Solr instalation.
For more details on what's next che the Issues page on the google code site.

Stress Test Solr

2010-08-02 Thread Tomas

Hi All, we've been building an open source tool for load tests on Solr 
Installations. Thetool is called SolrMeter. It's on google code 
at http://code.google.com/p/solrmeter/. Here is some information about it:

SolrMeter is an stress testing / performance benchmarking tool for Apache Solr 
installations.  It is licensed under ASL and developed using JavaSE and Swing 
components, connected with Solr using SolrJ.
 
What can youdowith SolrMeter?
The main goal of this open source project is bring to the Apache Solr user 
community a tool for dealing with Solr specific issues regarding performance 
and 
stress testing like firing queries and adding documents to make sure that your 
Solr instalation will support real world's load and demands. With SolrMeter you 
can simulate a work load over the Apache Solr instalation and to obtain useful 
visual performance statistics and metrics.
Relevant Features:
* Execute queries against a Solr installation
* Execute dummy updates/inserts to the Solr installation, it can be the 
same 
server as the queries or a different one.
* Configure number of queries to fire in a time period interval
* Configure the number of updates/inserts in a time period.
* Configure commits frequency during adds
* Monitor error counts when adding and commiting documents.
* Perform and monitor index optimization
* Monitor query times online and visually
* Add filter queries into the test queries
* Add facet abilities into the test queries
* Import/Export test configuration
* Query time execution histogram chart
* Query times distribution chart
* Online error log and browsing capabilities
* Individual query graphical log and statistics
* and much more
 
Whatdo you need for use SolrMeter?
This is one of the most interesting points about SolrMeter, the requirements 
are 
minimal. It is simple to install and use.
* JRE versión 1.6
* The Solr Server you want to test.
 
Who can use SolrMeter?
Everyone who needs to assess the solrmeter server performance. To run the tool 
you only need to know about SOLR.



Try it and tell us what you think . . . . .  

Solrmeter Group
mailto:solrme...@googlegroups.com

What's next?
We are now building version 0.2.0, the objetive of this new version is to 
evolve 
SolrMeter into a pluggable architecture to allow deeper customizations like 
adding custom statistics, extractors or executors.
We are also adding some usability improvements.

On future versions we want to add a better interaction with Solr request 
handlers, for example, showing cache statistics online and graphically on some 
chart would be a great tool.
We also want to add more usability features to make of solrmeter a complete 
tool 
for testing a Solr instalation.
For more details on what's next che the Issues page on the google code site.

Index compatibility 1.4 Vs 3.1 Trunk

2010-08-02 Thread Ravi Kiran

Hello,
I am trying to use the solr code from '
https://svn.apache.org/repos/asf/lucene/dev/trunk' as my design warrants use
of PolyType fields. My understanding is that the indexes are incompatible,
am I right ?. I have about a million docs in my index (indexed via solr
1.4). Is re-indexing my only option or is there a tool of some sort to
convert the 1.4 index to 3.1 format ?

Thanks,

Ravi Kiran

Re: enhancing auto complete

2010-08-02 Thread Avlesh Singh

Hahaha ... sorry its not. And there is no readymade code that I can give you
either. But yes, if you liked it, I can share the design of this feature
(solr, backend and frontend).

Cheers
Avlesh
@avlesh  | http://webklipper.com

On Mon, Aug 2, 2010 at 8:47 PM,  wrote:

>
>  Hi, I'm also interested of this feature... is it open source?
>
>
>
>
>
>
>
>
> -Original Message-
> From: Avlesh Singh 
> To: solr-user@lucene.apache.org
> Sent: Mon, Aug 2, 2010 5:09 pm
> Subject: Re: enhancing auto complete
>
>
> From whatever I could read in your broken table of sample use cases, I
> think
>
> you are looking for something similar to what has been done here -
>
> http://askme.in; if this is what you are looking do let me know.
>
>
>
> Cheers
>
> Avlesh
>
> @avlesh  | http://webklipper.com
>
>
>
> On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar <
>
> bhavnik.gaj...@gatewaynintec.com> wrote:
>
>
>
> > Hi,
>
> >
>
> > I'm looking for a solution related to auto complete feature for one
>
> > application.
>
> >
>
> > Below is a list of texts from which auto complete results would be
>
> > populated.
>
> >
>
> > Lorem ipsum dolor sit amet
>
> > tincidunt ut laoreet
>
> > dolore eu feugiat nulla facilisis at vero eros et
>
> > te feugait nulla facilisi
>
> > Claritas est etiam processus
>
> > anteposuerit litterarum formas humanitatis
>
> > fiant sollemnes in futurum
>
> > Hieyed ddi lorem ipsum dolor
>
> > test lorem ipsume
>
> > test xyz lorem ipslili
>
> >
>
> > Consider below table. First column describes user entered value and
>
> > second column describes expected result (list of auto complete terms
>
> > that should be populated from Solr)
>
> >
>
> > lorem
>
> >*Lorem* ipsum dolor sit amet
>
> > Hieyed ddi *lorem* ipsum dolor
>
> > test *lorem *ipsume
>
> > test xyz *lorem *ipslili
>
> > lorem ip
>
> >*Lorem ip*sum dolor sit amet
>
> > Hieyed ddi *lorem ip*sum dolor
>
> > test *lorem ip*sume
>
> > test xyz *lorem ip*slili
>
> > lorem ipsl
>
> >test xyz *lorem ipsl*ili
>
> >
>
> >
>
> >
>
> > Can anyone share ideas of how this can be achieved with Solr? Already
>
> > tried with various tokenizers and filter factories like,
>
> > WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,
>
> > ShingleFilterFactory etc. but no luck so far..
>
> >
>
> > Note that, It would be excellent if terms populated from Solr can be
>
> > highlighted by using Highlighting or any other component/mechanism of
> Solr.
>
> >
>
> > *Note :* Standard autocomplete (like,
>
> > facet.field=AutoComplete&f.AutoComplete.facet.prefix=
> > term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already
>
> > working fine with the application. but, nowadays, looking for enhancing
>
> > the existing auto complete stuff with the above requirement.
>
> >
>
> > Any thoughts?
>
> >
>
> > Thanks in advance
>
> >
>
> >
>
> >
>
> >
>
> > The contents of this eMail including the contents of attachment(s) are
>
> > privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL)
> and
>
> > should not be disclosed to, used by or copied in any manner by anyone
> other
>
> > than the intended addressee(s). If this eMail has been received by error,
>
> > please advise the sender immediately and delete it from your system. The
>
> > views expressed in this eMail message are those of the individual sender,
>
> > except where the sender expressly, and with authority, states them to be
> the
>
> > views of GNPL. Any unauthorized review, use, disclosure, dissemination,
>
> > forwarding, printing or copying of this eMail or any action taken in
>
> > reliance on this eMail is strictly prohibited and may be unlawful. This
>
> > eMail may contain viruses. GNPL has taken every reasonable precaution to
>
> > minimize this risk, but is not liable for any damage you may sustain as a
>
> > result of any virus in this eMail. You should carry out your own virus
>
> > checks before opening the eMail or attachment(s). GNPL is neither liable
> for
>
> > the proper and complete transmission of the information contained in this
>
> > communication nor for any delay in its receipt. GNPL reserves the right
> to
>
> > monitor and review the content of all messages sent to or from this eMail
>
> > address and may be stored on the GNPL eMail system. In case this eMail
> has
>
> > reached you in error, and you  would no longer like to receive eMails
> from
>
> > us, then please send an eMail to d...@gatewaynintec.com
>
> >
>
>
>
>

Re: enhancing auto complete

2010-08-02 Thread scrapy


 Hi, I'm also interested of this feature... is it open source?

 


 

 

-Original Message-
From: Avlesh Singh 
To: solr-user@lucene.apache.org
Sent: Mon, Aug 2, 2010 5:09 pm
Subject: Re: enhancing auto complete


>From whatever I could read in your broken table of sample use cases, I think

you are looking for something similar to what has been done here -

http://askme.in; if this is what you are looking do let me know.



Cheers

Avlesh

@avlesh  | http://webklipper.com



On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar <

bhavnik.gaj...@gatewaynintec.com> wrote:



> Hi,

>

> I'm looking for a solution related to auto complete feature for one

> application.

>

> Below is a list of texts from which auto complete results would be

> populated.

>

> Lorem ipsum dolor sit amet

> tincidunt ut laoreet

> dolore eu feugiat nulla facilisis at vero eros et

> te feugait nulla facilisi

> Claritas est etiam processus

> anteposuerit litterarum formas humanitatis

> fiant sollemnes in futurum

> Hieyed ddi lorem ipsum dolor

> test lorem ipsume

> test xyz lorem ipslili

>

> Consider below table. First column describes user entered value and

> second column describes expected result (list of auto complete terms

> that should be populated from Solr)

>

> lorem

>*Lorem* ipsum dolor sit amet

> Hieyed ddi *lorem* ipsum dolor

> test *lorem *ipsume

> test xyz *lorem *ipslili

> lorem ip

>*Lorem ip*sum dolor sit amet

> Hieyed ddi *lorem ip*sum dolor

> test *lorem ip*sume

> test xyz *lorem ip*slili

> lorem ipsl

>test xyz *lorem ipsl*ili

>

>

>

> Can anyone share ideas of how this can be achieved with Solr? Already

> tried with various tokenizers and filter factories like,

> WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,

> ShingleFilterFactory etc. but no luck so far..

>

> Note that, It would be excellent if terms populated from Solr can be

> highlighted by using Highlighting or any other component/mechanism of Solr.

>

> *Note :* Standard autocomplete (like,

> facet.field=AutoComplete&f.AutoComplete.facet.prefix= term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already

> working fine with the application. but, nowadays, looking for enhancing

> the existing auto complete stuff with the above requirement.

>

> Any thoughts?

>

> Thanks in advance

>

>

>

>

> The contents of this eMail including the contents of attachment(s) are

> privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL) and

> should not be disclosed to, used by or copied in any manner by anyone other

> than the intended addressee(s). If this eMail has been received by error,

> please advise the sender immediately and delete it from your system. The

> views expressed in this eMail message are those of the individual sender,

> except where the sender expressly, and with authority, states them to be the

> views of GNPL. Any unauthorized review, use, disclosure, dissemination,

> forwarding, printing or copying of this eMail or any action taken in

> reliance on this eMail is strictly prohibited and may be unlawful. This

> eMail may contain viruses. GNPL has taken every reasonable precaution to

> minimize this risk, but is not liable for any damage you may sustain as a

> result of any virus in this eMail. You should carry out your own virus

> checks before opening the eMail or attachment(s). GNPL is neither liable for

> the proper and complete transmission of the information contained in this

> communication nor for any delay in its receipt. GNPL reserves the right to

> monitor and review the content of all messages sent to or from this eMail

> address and may be stored on the GNPL eMail system. In case this eMail has

> reached you in error, and you  would no longer like to receive eMails from

> us, then please send an eMail to d...@gatewaynintec.com

>

Re: enhancing auto complete

2010-08-02 Thread Avlesh Singh

>From whatever I could read in your broken table of sample use cases, I think
you are looking for something similar to what has been done here -
http://askme.in; if this is what you are looking do let me know.

Cheers
Avlesh
@avlesh  | http://webklipper.com

On Mon, Aug 2, 2010 at 8:09 PM, Bhavnik Gajjar <
bhavnik.gaj...@gatewaynintec.com> wrote:

> Hi,
>
> I'm looking for a solution related to auto complete feature for one
> application.
>
> Below is a list of texts from which auto complete results would be
> populated.
>
> Lorem ipsum dolor sit amet
> tincidunt ut laoreet
> dolore eu feugiat nulla facilisis at vero eros et
> te feugait nulla facilisi
> Claritas est etiam processus
> anteposuerit litterarum formas humanitatis
> fiant sollemnes in futurum
> Hieyed ddi lorem ipsum dolor
> test lorem ipsume
> test xyz lorem ipslili
>
> Consider below table. First column describes user entered value and
> second column describes expected result (list of auto complete terms
> that should be populated from Solr)
>
> lorem
>*Lorem* ipsum dolor sit amet
> Hieyed ddi *lorem* ipsum dolor
> test *lorem *ipsume
> test xyz *lorem *ipslili
> lorem ip
>*Lorem ip*sum dolor sit amet
> Hieyed ddi *lorem ip*sum dolor
> test *lorem ip*sume
> test xyz *lorem ip*slili
> lorem ipsl
>test xyz *lorem ipsl*ili
>
>
>
> Can anyone share ideas of how this can be achieved with Solr? Already
> tried with various tokenizers and filter factories like,
> WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,
> ShingleFilterFactory etc. but no luck so far..
>
> Note that, It would be excellent if terms populated from Solr can be
> highlighted by using Highlighting or any other component/mechanism of Solr.
>
> *Note :* Standard autocomplete (like,
> facet.field=AutoComplete&f.AutoComplete.facet.prefix= term>&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already
> working fine with the application. but, nowadays, looking for enhancing
> the existing auto complete stuff with the above requirement.
>
> Any thoughts?
>
> Thanks in advance
>
>
>
>
> The contents of this eMail including the contents of attachment(s) are
> privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL) and
> should not be disclosed to, used by or copied in any manner by anyone other
> than the intended addressee(s). If this eMail has been received by error,
> please advise the sender immediately and delete it from your system. The
> views expressed in this eMail message are those of the individual sender,
> except where the sender expressly, and with authority, states them to be the
> views of GNPL. Any unauthorized review, use, disclosure, dissemination,
> forwarding, printing or copying of this eMail or any action taken in
> reliance on this eMail is strictly prohibited and may be unlawful. This
> eMail may contain viruses. GNPL has taken every reasonable precaution to
> minimize this risk, but is not liable for any damage you may sustain as a
> result of any virus in this eMail. You should carry out your own virus
> checks before opening the eMail or attachment(s). GNPL is neither liable for
> the proper and complete transmission of the information contained in this
> communication nor for any delay in its receipt. GNPL reserves the right to
> monitor and review the content of all messages sent to or from this eMail
> address and may be stored on the GNPL eMail system. In case this eMail has
> reached you in error, and you  would no longer like to receive eMails from
> us, then please send an eMail to d...@gatewaynintec.com
>

enhancing auto complete

2010-08-02 Thread Bhavnik Gajjar

Hi,

I'm looking for a solution related to auto complete feature for one
application.

Below is a list of texts from which auto complete results would be
populated.

Lorem ipsum dolor sit amet
tincidunt ut laoreet
dolore eu feugiat nulla facilisis at vero eros et
te feugait nulla facilisi
Claritas est etiam processus
anteposuerit litterarum formas humanitatis
fiant sollemnes in futurum
Hieyed ddi lorem ipsum dolor
test lorem ipsume
test xyz lorem ipslili

Consider below table. First column describes user entered value and
second column describes expected result (list of auto complete terms
that should be populated from Solr)

lorem
*Lorem* ipsum dolor sit amet
Hieyed ddi *lorem* ipsum dolor
test *lorem *ipsume
test xyz *lorem *ipslili
lorem ip
*Lorem ip*sum dolor sit amet
Hieyed ddi *lorem ip*sum dolor
test *lorem ip*sume
test xyz *lorem ip*slili
lorem ipsl
test xyz *lorem ipsl*ili

Can anyone share ideas of how this can be achieved with Solr? Already
tried with various tokenizers and filter factories like,
WhiteSpaceTokenizer, KeywordTokenizer, EdgeNGramFilterFactory,
ShingleFilterFactory etc. but no luck so far..

Note that, It would be excellent if terms populated from Solr can be
highlighted by using Highlighting or any other component/mechanism of Solr.

*Note :* Standard autocomplete (like,
facet.field=AutoComplete&f.AutoComplete.facet.prefix=&f.AutoComplete.facet.limit=10&facet.sort&rows=0) are already
working fine with the application. but, nowadays, looking for enhancing
the existing auto complete stuff with the above requirement.

Any thoughts?

Thanks in advance

The contents of this eMail including the contents of attachment(s) are
privileged and confidential material of Gateway NINtec Pvt. Ltd. (GNPL) and
should not be disclosed to, used by or copied in any manner by anyone other
than the intended addressee(s). If this eMail has been received by error,
please advise the sender immediately and delete it from your system. The views
expressed in this eMail message are those of the individual sender, except
where the sender expressly, and with authority, states them to be the views of
GNPL. Any unauthorized review, use, disclosure, dissemination, forwarding,
printing or copying of this eMail or any action taken in reliance on this eMail
is strictly prohibited and may be unlawful. This eMail may contain viruses.
GNPL has taken every reasonable precaution to minimize this risk, but is not
liable for any damage you may sustain as a result of any virus in this eMail.
You should carry out your own virus checks before opening the eMail or
attachment(s). GNPL is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt. GNPL reserves the right to monitor and review the content of all
messages sent to or from this eMail address and may be stored on the GNPL eMail
system. In case this eMail has reached you in error, and you would no longer
like to receive eMails from us, then please send an eMail to
d...@gatewaynintec.com

Re: Quering the database

2010-08-02 Thread Geert-Jan Brits

you should (as per the example) define the field as text in your solr-schema
not in your RDB.
something like:  

then search like: q=field_1:monitors

the example schema illustrates a lot of the possibilities on how you to
define fields and what is all means.
Moreover have a look at:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters

Geert-Jan

2010/8/2 Hando420 

>
> Thank you for your reply. Still the the problem persists even i tested with
> a
> simple example by defining a column of type text as varchar in database and
> in schema.xml used the default id which is set to string. Row is fetched
> and
> document created but searching doesn't give any results of the content in
> the column.
>
> Best Regards,
> Hando
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p1015890.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

RE: Quering the database


Thank you for your reply. Still the the problem persists even i tested with a
simple example by defining a column of type text as varchar in database and
in schema.xml used the default id which is set to string. Row is fetched and
document created but searching doesn't give any results of the content in
the column.

Best Regards,
Hando
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p1015890.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Quering the database

2010-08-02 Thread Fornoville, Tom

This question has come up several times over the past weeks.

The cause is probably all your fields being of type "string".
This is only good for exact matches like id's etc.
Try using "text" or another type that tokenizes.

-Original Message-
From: Hando420 [mailto:hando...@gmail.com] 
Sent: maandag 2 augustus 2010 13:42
To: solr-user@lucene.apache.org
Subject: Re: Quering the database

Thank you for your reply.

Below is my data-config details.

Below are the defined schema fields for the data-config

 field_1

 text
-- 
View this message in context:
http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p101567
1.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Rows fetch OK, Total Documents Failed??

2010-08-02 Thread Michael Griffiths

Check your schema.xml; one of the fields is probable "Required," and it's not 
matching to a field extracted from DIH. Keep in mind that schema.xml is 
case-sensitive for names.

-Original Message-
From: scr...@asia.com [mailto:scr...@asia.com] 
Sent: Saturday, July 31, 2010 3:30 AM
To: solr-user@lucene.apache.org
Subject: DIH: Rows fetch OK, Total Documents Failed??

 Hi,

I'm a bit lost with this, i'm trying to import a new XML via DIH, all row are 
fetched but no ducument are indexed? I don't find any log or error?

Any ideas?

Here is the STATUS:

status
idle

1 7554 0 2010-07-31 10:14:33 0 7554 0:0:4.720 

My xml file looks like this:

Moniteur VG1930wm 19 LCD Viewsonic

http://x.com/abc?a(12073231)p(2822679)prod(89042332277)ttid(5)url(http%3A%2F%2Fwww.ffdsssd.com%2Fproductinformation%2F%7E66297%7E%2Fproduct.htm%26sender%3D2003)
Moniteur VG1930wm 19  LCD Viewsonic VG1930WM
247.57
Ecrans

Re: Quering the database


Thank you for your reply.

Below is my data-config details.


  
  
  




  


Below are the defined schema fields for the data-config

   
  
   
 
 
 field_1

 
 text
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p1015671.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Quering the database

2010-08-02 Thread Rafal Bluszcz Zawadzki

On Mon, Aug 2, 2010 at 1:28 PM, Hando420  wrote:

>
> Hi,
>
> I am new to solr. I am indexing data from data using solr. Everything is
> working fine the documents are indexed and using *:* as a query keyword i
> am
> able to get all the results from the table. My problem is now if i want to
> query using a specific keyword such as monitors, it doesn't return any
> fields when searched for but the data does exist in the database. What am i
> missing here? Highly appreciate for any hints.
>
>
Can you send you schema?  And example what is not working?

Quering the database


Hi,

I am new to solr. I am indexing data from data using solr. Everything is
working fine the documents are indexed and using *:* as a query keyword i am
able to get all the results from the table. My problem is now if i want to
query using a specific keyword such as monitors, it doesn't return any
fields when searched for but the data does exist in the database. What am i
missing here? Highly appreciate for any hints.

Regards,
Hando
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Quering-the-database-tp1015636p1015636.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting DisMax queries with !boost component

2010-08-02 Thread MitchK



Jonathan Rochkind wrote:
> 
>> qf needs to have spaces in it, unfortunately the local query parser can
>> not
>> deal with that, as Erik Hatcher mentioned some months ago.
> 
> By "local query parser", you mean what I call the LocalParams stuff (for
> lack of being sure of the proper term)?  
> 
Yes, that was what I meant.

Kind regards,
- Mitch
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Boosting-DisMax-queries-with-boost-component-tp1011294p1015619.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrJ Response + JSON

2010-08-02 Thread MitchK


Hi,

as I promised, I want to give a feedback for transforming SolrJ's output 
into JSON with the package from json.org (the package was the json.org's 
one):


I need to make a small modification to the package, since they store the 
JSON-key-value-pairs in a HashMap, I changed this to a LinkedHashMap to 
make sure that the order of the retrived values is the same order as 
they were inserted in the map.


The result looks very, very pretty.

It was very easy to transform the SolrJ's output into the desired 
JSON-format and I can add now whatever I want to the response.


Kind regards,
- Mitch

Re: Solr Indexing slows down

2010-08-02 Thread Peter Karich

Thanks Otis, for this clarification!

That means I will have to descrease the commit frequency to speed up
indexing.
How could I do this if I don't want to introduce an artificial delay
time? ... via increasing the batch size?

Today I have read in another thread[1] that one should univert the
field? What is it and how can I do this?

Regards,
Peter.

[1]
http://www.mail-archive.com/solr-user@lucene.apache.org/msg36113.html


> As you make changes to your index, you probably want to see the new/modified 
> documents in your search results.  In order to do that, the new searcher 
> needs 
> to be reopened, and this happens on commit.
> Otis
> 
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
>
> - Original Message 
>   
>> From: Peter Karich 
>> To: solr-user@lucene.apache.org
>> Sent: Fri, July 30, 2010 6:19:03 PM
>> Subject: Re: Solr Indexing slows down
>>
>> Hi Otis,
>>
>> does it mean that a new searcher is opened after I commit?
>> I  thought only on startup...(?)
>>
>> Regards,
>> Peter.
>>
>> 
>>> Peter, there  are events in solrconfig where you define warm up queries 
>>> when 
>>>   
>> a 
>>
>> 
>>> new  searcher is opened.
>>>
>>> There are also cache settings that play a  role here.
>>>
>>> 30-60 seconds is pretty frequent for  Solr.
>>>
>>> Otis
>>> 
>>> Sematext :: http://sematext.com/ :: Solr -  Lucene - Nutch
>>> Lucene ecosystem search :: http://search-lucene.com/
>>>
>>>
>>>
>>> - Original  Message 
>>>  
>>>   
 From: Peter Karich 
 To: solr-user@lucene.apache.org
  Sent: Fri, July 30, 2010 4:06:48 PM
 Subject: Re: Solr Indexing  slows down

 Hi Erick!

 thanks for  the response!
 I will answer your questions   ;-)


 
> How often are you  making changes to your index?
>  
>   
  Every  30-60 seconds. Too heavy?


 
 
> Do you have autocommit  on?
>   
>   
 No.



 
> Do you commit when updating each   document?
>  
>   
 No. I commit after a  batch update of 200  documents


 
 
> Committing too often and consequently firing off   warmup queries is the 
>   
>> first 
>>
>> 
>  
>   
  place I'd look.

 Why is commiting firing  warmup  queries? Is there any documentation about
 this subject?
  How can I  be sure that the previous commit has done its  magic?


 
> there are   several config values that influence the commit frequency
>   
>   
 I now know  the autowarm and the  mergeFactor config. What else? Is this
 documentation   complete:
 http://wiki.apache.org/lucene-java/ImproveIndexingSpeed ?

 Regards,
  Peter.


 
> See the subject  about 1500 threads. The  first place I'd look is how
> often  you're committing. If you're  committing before the warmup  queries
> from the previous commit have done  their magic,  you might be getting
> into a death  spiral.
>
>  HTH
>  Erick
>
> On Thu, Jul 29, 2010 at 7:02 AM, Peter  Karich 
>   
> wrote:
>   
>  
>   
>   
>> Hi,
>>
>> I  am  indexing a solr 1.4.0 core and commiting gets slower and  slower.
>>  Starting from 3-5 seconds for ~200 documents  and ending with over 60
>>  seconds after 800 commits.  Then, if I reloaded the index, it is as  
>> 
> fast
>   
>> as  before! And today I have read a similar thread [1] and  indeed: if  I
>> set autowarming for the caches to 0 the slowdown   disappears.
>>
>> BUT at the same time I would  like to offer  searching on that core, 
>> 
> which
>   
>> would be  dramatically slowed down (due  to no  autowarming).
>>
>> Does someone know a better  solution  to avoid index-slow-down?
>>
>>  Regards,
>>   Peter.
>>
>> [1] http://www.mail-archive.com/solr-user@lucene.apache.org/msg20785.html
>>
>>
>>

Indexing data on MSSQL failed: Caused by: org.apache.solr.common.SolrException: Error loading class 'com.micros oft.sqlserver.jdbc.SQLServerDriver'