Re: Solr Schema Question

2010-04-18 Thread Lance Norskog
Highlighting is a complex topic. A field has to be stored to be highlight. It does not have to be indexed. But, if it is not, highlighting analyzes it just like if it was indexed in order to highlight it. http://www.lucidimagination.com/search/document/CDRG_ch07_7.9?q=highlighting http://www.luci

Re: Solr Schema Question

2010-04-18 Thread Serdar Sahin
Thanks everyone, It works! I have successfully indexed them. Thanks again! I have couple of more questions regarding with solr, if you don't mind. 1-) As I said before, the text files are quite large, between 100kb-10mb, but I need to store them as well for highlighting, including with their titl

Re: Solr Schema Question

2010-04-17 Thread Lance Norskog
Man you people are fast! There is a bug in Solr/Lucene. It keeps memory around from previous fields, so giant text files might run out of memory when they should not. This bug is fixed in the trunk. On 4/17/10, Lance Norskog wrote: > The DataImportHandler can let you fetch the file name from the

Re: Solr Schema Question

2010-04-17 Thread Lance Norskog
The DataImportHandler can let you fetch the file name from the database record, and then load the file as a field and process the text with Tika. It will not be easy :) but it is possible. http://wiki.apache.org/solr/DataImportHandler On 4/17/10, Serdar Sahin wrote: > Hi, > > I am rather new to

Re: Solr Schema Question

2010-04-17 Thread Ahmet Arslan
> I am rather new to Solr and have a question. > > We have around 200.000 txt files which are placed into the > file cloud. > The file path is something similar to this: > > file/97/8f/840/fa4-1.txt > file/a6/9d/ab0/ca2-2.txt etc. > > and we also store the metadata (like title, description, > ta

Re: Solr Schema Question

2010-04-17 Thread Sascha Szott
Hi Serdar, take a look at Solr's DataImportHandler: http://wiki.apache.org/solr/DataImportHandler Best, Sascha Serdar Sahin wrote: Hi, I am rather new to Solr and have a question. We have around 200.000 txt files which are placed into the file cloud. The file path is something similar to th

Solr Schema Question

2010-04-17 Thread Serdar Sahin
Hi, I am rather new to Solr and have a question. We have around 200.000 txt files which are placed into the file cloud. The file path is something similar to this: file/97/8f/840/fa4-1.txt file/a6/9d/ab0/ca2-2.txt etc. and we also store the metadata (like title, description, tags etc) about the