Re: Analyzer creates PhraseQuery

Separate the Analyzer into a index time analyzer with NgramFilter Factory and Query time analyzer without the N-gram Filter Factory Since your query is analyzed by this analyzer and produces more than one tokens for the given keyoword and hence the result is phrase query. Regards Sujatha On T

Re: Analyzer creates PhraseQuery

I guess 'to' may be listed in 'stopWords' . On 28 June 2011 08:27, entdeveloper wrote: > I have an analyzer setup in my schema like so: > > > > > maxGramSize="2"/> > > > What's happening is if I index a term like "toys and dolls", if I search > for > "to", I get no matches. The d

Replication without configs

I have replicated a solr instance without configs as the slave has it's own config. The replication has failed. My plan was to use replication to remove the indexes I no longer wish to use which is why the slave has a different schema.xml file. Does anyone know why the replication has failed? Th

Analyzer creates PhraseQuery

I have an analyzer setup in my schema like so: What's happening is if I index a term like "toys and dolls", if I search for "to", I get no matches. The debug output in solr gives me: to to PhraseQuery(autocomplete:"t o to") autocomplete:"t o to" Which means it looks like th

Re: Understanding query explain information

: Simply trying to understand why these strings generated such scores, and as : far as I can understand, the only difference between them is the field : norms, as all the other results maintain themselves. ... : Well, if this is true, the field norm for my first document should be 0.5 : (1/

Master/Dormant Master and NFS. SimpleFSLockFactory?

I am trying to run through a few failure scenarios using a dual master approach using NFS as a shared storage solution to hold the Master's indexes. My goal is to be able to bring up a secondary master in the case that the primary master fails. I have several slaves using replication to pull ind

Custom Query Processing

I have a need to take an incoming solr query and apply some additional constraints to it on the Solr end. Our previous implementation used a QueryWrapperFilter along with some custom code to build a new Filter from the query provided. How can we plug this filter into Solr?

Re: Advice wanted on approach/architecture

Hi Jan, The regex exclude is system-wide. I might have to do the regex filtering when loading the data. The problem is that there will be a process querying solr for matching records which will be "alerts". I have to prevent that the matching entries are loaded but not indexed. Thanks for the

TermVectors and custom queries

I have a field named content with the following definition I'm now trying to execute a query against content and get back the term vectors for the pieces that matched my query, but I must be messing something up. My query is as follows: http://localhost:8983/solr/select/?qt=tvrh&q=content:t

Re: Server Restart Required for Schema Changes After Document Delete All?

I'm not having any issues. I was just asking to see if any backward incompatible changes exist that would require a server restart. Thanks. 2011/6/27 Tomás Fernández Löbbe > This should work with dynamic fields too. Are you having any problems with > it? > > > On Thu, Jun 23, 2011 at 3:14 PM, Br

Re: Server Restart Required for Schema Changes After Document Delete All?

One more thing, it's not necessary to restart the server, just to reload the core: http://wiki.apache.org/solr/CoreAdmin#RELOAD 2011/6/27 Tomás Fernández Löbbe > This should work with dynamic fields too. Are you having any problems with > it? > > > On Thu, Jun 23, 2011 at 3:14 PM, Brandon Fish w

Re: Server Restart Required for Schema Changes After Document Delete All?

This should work with dynamic fields too. Are you having any problems with it? On Thu, Jun 23, 2011 at 3:14 PM, Brandon Fish wrote: > Are there any schema changes that would cause problems with the following > procedure from the > FAQ< > http://wiki.apache.org/solr/FAQ#How_can_I_rebuild_my_index

Re: intersecting map extent with solr spatial documents

Steve, I like your geospatial boosting algorithm; it makes sense. FYI I'm collaborating with a couple other Lucene/Solr committers on a new geospatial module here: http://code.google.com/p/lucene-spatial-playground/ It is very much in-progress. There is variable-points per document support (ba

Re: Using RAMDirectoryFactory in Master/Slave setup

Thanks for the pointer to MMapDirectoryFactory. Not having replication with RAMDirectoryFactory is a deal killer. We dont want to index on the machines that serve queries. >From what I can gather from reading, MMapDirectory + SSD could be a happy medium. I'll try to evaluate these a bit more form

After the query component has the results, can I do more filtering on them?

So I made a custom search component which runs right after the query component and this custom component will update the score of each based on some things (and no, I definitely can't use existing components). I didn't see any easy way to just update the score so what I currently do is something l

Re: Broken attachment link on Wiki

> On the SolrJetty page > > http://wiki.apache.org/solr/SolrJetty > > there's a link to a tar ball > > http://wiki.apache.org/solr/SolrJetty?action=AttachFile&do=view&target=DEMO_multiple_webapps_jetty_6.1.3.tgz > > which fails with the error > > You are not allowed to do AttachFile on this p

Broken attachment link on Wiki

On the SolrJetty page http://wiki.apache.org/solr/SolrJetty there's a link to a tar ball http://wiki.apache.org/solr/SolrJetty?action=AttachFile&do=view&target=DEMO_multiple_webapps_jetty_6.1.3.tgz which fails with the error You are not allowed to do AttachFile on this page. Can someone fix

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Of course, i can't print the system bell and stuff like that in XML. I'll improve the method to get rid of non-printable control characters as well. On Monday 27 June 2011 18:16:08 Mike Sokolov wrote: > Markus - if you want to make sure not to offend XML parsers, you should > strip all characters

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Markus - if you want to make sure not to offend XML parsers, you should strip all characters not in this list: http://en.wikipedia.org/wiki/XML#Valid_characters You'll see that article talks about XML 1.1, which accepts a wider range of characters than XML 1.0, and I believe the Woodstox parse

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Of course it doesn't work like this: use AND instead of OR! On Monday 27 June 2011 17:50:01 Markus Jelsma wrote: > Hi all, thanks for your comments. I seem to have fixed it by now by simply > stripping away all non-character codepoints [1] by iterating over the > individual chars and checking them

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Hi all, thanks for your comments. I seem to have fixed it by now by simply stripping away all non-character codepoints [1] by iterating over the individual chars and checking them against: if (ch % 0x1 != 0x || ch % 0x1 != 0xfffe || (ch <= 0xfdd0 && ch >= 0xfdef)) { pass; } Comment

Unique document count from index?

Hi all- I have a problem that I'm not sure how it can be (if it can be) solved in Solr. I am using Solr 3.2 with patch 2524 installed to provide grouping. I need to return the count of unique records that match a particular query. For an example of what I'm talking about, imagine I have an inde

Re: Default schema - 'keywords' not multivalued

Hi Tod, A list of keywords would be fine in a non multi valued field: keywords : "xxx yyy sss aaa " multi value field would allow you to repeat the field when indexing keywords: "xxx" keywords: "yyy" keywords: "sss" etc On 27 June 2011 16:13, Tod wrote: > This was a little curious to me a

Default schema - 'keywords' not multivalued

This was a little curious to me and I wondered what the thought process was behind it before I decide to change it. Thanks - Tod

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

I don't think this is a BOM - that would be 0xfeff. Anyway the problem we usually see w/processing XML with BOMs is in UTF8 (which really doesn't need a BOM since it's a byte stream anyway), in which if you transform the stream (bytes) into a reader (chars) before the xml parser can see it, th

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

On Monday 27 June 2011 16:33:16 lee carroll wrote: > Hi Markus > > I've seen similar issue before (but not with solr) when processing files as > xml. In our case the problem was due to processing a utf16 file with a > byte order mark. This presents itself as > 0x to the xml parser which is n

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Hi Markus I've seen similar issue before (but not with solr) when processing files as xml. In our case the problem was due to processing a utf16 file with a byte order mark. This presents itself as 0x to the xml parser which is not used by utf8 (the bom unicode would be represented as efbfbf i

Re: no answer with q empty q parameter

thanks! 2011/6/27 Ahmet Arslan > > > When I send a request to Solr with fq parameter defined and > > an empty q > > parameter, I get no answer. > > > > I use edismax with mm = 4<-1 6<-2. > > > > Any clues? > > You can use &q.alt=*:* > > http://wiki.apache.org/solr/DisMaxQParserPlugin#q.alt >

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

hı Its the same error I mentioned here http://lucene.472066.n3.nabble.com/strange-utf-8-problem-td3094473.html. Also if you use solr 1.4.1 there is no problem like that. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-1-indexing-error-Invalid-UTF-8-character-0x

Re: Exact phrase highlighting

has this bug been fixed? i'm using solr 3.1 and it still seems to be an issue. if i do a search for "bird house" i still get bird house returned instead of bird house, which is the desired result. -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-phrase-highlighting-tp48

Re: Inconsistent search results

Am 27.06.2011 15:56, schrieb Jihed Amine Maaref: > - normalizedContents:(EDOUAR* AND une) doesn't return anything This was discussed few days ago: http://lucene.472066.n3.nabble.com/Conflict-in-wildcard-query-and-spellchecker-in-solr-search-tt3095198.html > - normalizedContents:(edouar* AND un)

Inconsistent search results

Hi! I'm new to Apache Solr and trying to make a query using search terms against a field called "normalizedContents" and of type "text". All of the search terms must exist in the field. Problem is, I'm getting inconsistent results. For example, the solr index has only one document with normalizedCo

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Hello, Am 27.06.2011 um 12:40 schrieb Markus Jelsma: > Hi, > > I came across the indexing error below. It happened in a huge batch update > from Nutch with SolrJ 3.1. Since the crawl was huge it is very hard to trace > the error back to a specific document. So i try my luck here: anyone seen

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

OK - re-reading your message it seems maybe that is what you were trying to say too, Robert. FWIW I agree with you that XML is rigid, sometimes for purely arbitrary reasons. But nobody has really helped Markus here - unfortunately, there is no easy way out of this mess. What I do to handle i

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Actually - you are both wrong! It is true that 0x is a valid UTF8 character, and not a valid UTF8 byte sequence. But the parser is reporting (or trying to) that 0x is an invalid XML character. And Robert - if the wording offends you, you might want to send a note to Tatu (http://ji

Re: Solr PhraseSearch and ExactMatch

> I can use 'String' instead of 'Text' > for exact match, but I need ExactMatch > only on PhraseSearch. I got your point. Your best bet can be crate an extra field field populated via copyField declaration. Type of this new field won't have stemming, stop-word removal etc. (according to your exa

Re: no answer with q empty q parameter

> When I send a request to Solr with fq parameter defined and > an empty q > parameter, I get no answer. > > I use edismax with mm = 4<-1 6<-2. > > Any clues? You can use &q.alt=*:* http://wiki.apache.org/solr/DisMaxQParserPlugin#q.alt

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Am 27.06.2011 14:48, schrieb Robert Muir: On Mon, Jun 27, 2011 at 8:47 AM, Bernd Fehling wrote: correct!!! but what i said, is totally different than what you said. you are still wrong. http://www.unicode.org/faq//utf_bom.html see Q: What is a UTF?

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

On Mon, Jun 27, 2011 at 8:47 AM, Bernd Fehling wrote: > > correct!!! > but what i said, is totally different than what you said. you are still wrong.

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Am 27.06.2011 14:35, schrieb Robert Muir: On Mon, Jun 27, 2011 at 8:30 AM, Bernd Fehling wrote: Unicode U+ ist UTF-8 byte sequence "ef bf bf" that is right. But I was saying that UTF-8 0x (which is byte sequence "ff ff") is illegal and that's what the java.io.CharConversionException

Re: multicore file creation order

Jérôme, the complete directory structure, including required files, has to be created first - manually. the admin/cores will only "activate" the core for solr, that's it :) Regards Stefan Am 27.06.2011 12:20, schrieb Jérôme Étévé: Hi, When one issues a command admin/core&action=CREATE&core

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

On Mon, Jun 27, 2011 at 8:30 AM, Bernd Fehling wrote: > Unicode U+ ist UTF-8 byte sequence "ef bf bf" that is right. > > But I was saying that UTF-8 0x (which is byte sequence "ff ff") is > illegal > and that's what the java.io.CharConversionException is complaining about. > "Invalid UTF-

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Am 27.06.2011 14:02, schrieb Robert Muir: On Mon, Jun 27, 2011 at 7:11 AM, Bernd Fehling wrote: So there is no UTF-8 0x. It is illegal. you are wrong: it is legally encoded as a three byte sequence: ef bf bf Unicode U+ ist UTF-8 byte sequence "ef bf bf" that is right. But I wa

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

On Mon, Jun 27, 2011 at 7:11 AM, Bernd Fehling wrote: > > So there is no UTF-8 0x. It is illegal. > you are wrong: it is legally encoded as a three byte sequence: ef bf bf

Re: Solr PhraseSearch and ExactMatch

I can use 'String' instead of 'Text' for exact match, but I need ExactMatch only on PhraseSearch. On 27 June 2011 16:29, Gora Mohanty wrote: > On Mon, Jun 27, 2011 at 3:42 PM, Mohammad Shariq > wrote: > > Hello, > > I am using solr1.4 on ubuntu 10.10. > > Currently I got the requirement to do t

Re: Using RAMDirectoryFactory in Master/Slave setup

Your best bet is MMapDirectoryFactory, you can come very close to the performance of the RAMDirectory. Unfortunatelly this setup with Master_on_disk->Slaves_in_ram type of setup is not possible using solr. We are moving our architecture to solr at the moment, and this is one of "missings" we have

Re: Solr 3.1 indexing error Invalid UTF-8 character 0xffff

I suggest avoid illegal UTF-8 characters by pre-filtering your contentstream before loading. Unicode UTF-8(hex) U+07FFdf bf U+0800e0 a0 80 So there is no UTF-8 0x. It is illegal. Regards Am 27.06.2011 12:40, schrieb Markus Jelsma: Hi, I came across the indexing error below. It

Re: Solr PhraseSearch and ExactMatch

On Mon, Jun 27, 2011 at 3:42 PM, Mohammad Shariq wrote: > Hello, > I am using solr1.4 on ubuntu 10.10. > Currently I got the requirement to do the ExactMatch  for PhraseSearch. > I tried googling but I did'nt got the exact solution. > > I am doing the search on 'text' field. > if I give the search

Solr 3.1 indexing error Invalid UTF-8 character 0xffff

Hi, I came across the indexing error below. It happened in a huge batch update from Nutch with SolrJ 3.1. Since the crawl was huge it is very hard to trace the error back to a specific document. So i try my luck here: anyone seen this before with SolrJ 3.1? Anything else on the Nutch part i sho

multicore file creation order

Hi, When one issues a command admin/core&action=CREATE&core=blabla&instanceDir=...&dataDir=../../foobar , what gets created first on disk? Is it the new solr.xml or the new data directory? Cheers, Jerome. -- Jerome Eteve. http://sigstp.blogspot.com/ http://twitter.com/jeteve

Solr PhraseSearch and ExactMatch

Hello, I am using solr1.4 on ubuntu 10.10. Currently I got the requirement to do the ExactMatch for PhraseSearch. I tried googling but I did'nt got the exact solution. I am doing the search on 'text' field. if I give the search query : http://localhost:8983/solr/select/?q="the search agency"

Re: how to index data in solr form database automatically

On 27/06/11 11:01, Romi wrote: Can i not use it for full-import ?? yes!!! You must only change urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false' in urlCmd='http://localhost:8080/solr/mycore/dataimport?command=full-import&clean=false' or urlCmd='http://l

Re: how to index data in solr form database automatically

Can i not use it for full-import ?? - Thanks & Regards Romi -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-index-data-in-solr-form-database-automatically-tp3102893p3113002.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Using RAMDirectoryFactory in Master/Slave setup

On Mon, Jun 27, 2011 at 12:49 PM, nipunb wrote: > I found a similar post - > http://lucene.472066.n3.nabble.com/Problems-with-RAMDirectory-in-Solr-td1575223.html > It mentions that Java based replication might work (This is what I have > used, but didn't work for me) Solr Replication does not wor

Re: how to index data in solr form database automatically

Start from this point: http://wiki.apache.org/solr/DataImportHandler So, you can try to use this script: #!/bin/bash wgetProgram=wget urlCmd='http://localhost:8080/solr/mycore/dataimport?command=delta-import&clean=false' statusCmd='http://localhost:8080/solr/mycore/dataimport?command=status' ou

no answer with q empty q parameter

Hello, When I send a request to Solr with fq parameter defined and an empty q parameter, I get no answer. I use edismax with mm = 4<-1 6<-2. Any clues? Thanks, Elisabeth

Re: Using RAMDirectoryFactory in Master/Slave setup

I found a similar post - http://lucene.472066.n3.nabble.com/Problems-with-RAMDirectory-in-Solr-td1575223.html It mentions that Java based replication might work (This is what I have used, but didn't work for me) More interestingly it points out that OS's file system cache maybe able to do this job

Re: multicore and replication cause OOM

On Sun, Jun 26, 2011 at 5:37 AM, Esteban Donato wrote: > thanks Shalin.  One more question:  is there any way to avoid multiple > cores replicating at the same time?  Like synchronizing the > ReplicationHandler somehow? > Yes, just specify different poll intervals for each core. The ReplicationHa

Re: how to index data in solr form database automatically

Thanks for answering but i don't know how to write script for fetching data from database and index it in to solr and how to setup this script as cron-job to run automatically at certain interval. Please suggest . - Thanks & Regards Romi -- View this message in context: http://lucene.4720