Re: OR-FilterQuery

2012-02-14 Thread Em
Hi Mikhail, > it's just how org.apache.lucene.search.CachingWrapperFilter works. The > first out-of-the box stuff which I've found. Thanks for your explanation and snippets - I thought this was configurable. Regards, Em Am 15.02.2012 06:16, schrieb Mikhail Khludnev: > On Tue, Feb 14, 2012 at 11:

Re: OR-FilterQuery

2012-02-14 Thread Mikhail Khludnev
On Tue, Feb 14, 2012 at 11:13 PM, Em wrote: > Hi Mikhail, > > > it will use per segment bitset at contrast to Solr's fq which caches for > > top level reader. > Could you explain why this bitset would be per-segment based, please? I don't see a reason why this *have* to be so. > it's just how or

feeding mahout cluster output back to solr

2012-02-14 Thread abhayd
hi at present we use carrot2 for clustering and doing analysis on customer feedback data. Since its in memory and search time we are having issues with performance and cluster size. I was reading about generating clusters using mahout from solr index data. But can we feed segmentation generated

Re: SolrCloud Replication Question

2012-02-14 Thread Jamie Johnson
All of the nodes now show as being Active. When starting the replicas I did receive the following message though. Not sure if this is expected or not. INFO: Attempting to replicate from http://JamiesMac.local:8501/solr/slice2_shard2/ Feb 14, 2012 10:53:34 PM org.apache.solr.common.SolrException

Re: SolrCloud Replication Question

2012-02-14 Thread Jamie Johnson
Doing so now, will let you know if I continue to see the same issues On Tue, Feb 14, 2012 at 4:59 PM, Mark Miller wrote: > Doh - looks like I was just seeing a test issue. Do you mind updating and > trying the latest rev? At the least there should be some better logging > around the recovery. >

Re: Can I rebuild an index and remove some fields?

2012-02-14 Thread Li Li
I have roughly read the codes of 4.0 trunk. maybe it's feasible. SegmentMerger.add(IndexReader) will add to be merged Readers merge() will call mergeTerms(segmentWriteState); mergePerDoc(segmentWriteState); mergeTerms() will construct fields from IndexReaders for(int rea

Re: Solr soft commit feature

2012-02-14 Thread Mark Miller
This has not been ported back to the 3.X line yet - mostly because it involved some rather large and invasive changes that I wanted to bake on trunk for some time first. Even still, the back port is not trivial, so I don't know that it's something I'd personally be able to get to in the short t

Re: OR-FilterQuery

2012-02-14 Thread Erick Erickson
Ah, OK, I misread your post apparently. And yes, what you suggest would result in some efficiencies, but at present I don't think there's any syntax that allows one to combine filter queries as you suggest. There was some discussion about it in the JIRA I referenced, but no action that I could see.

Solr soft commit feature

2012-02-14 Thread Dipti Srivastava
Hi All, Is there a way to soft commit in the current released version of solr 3.5? Regards, Dipti Srivastava This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system.

payload and exact match

2012-02-14 Thread leonardo2
Is there the possibility of perform 'exact search' in a payload field? I'have to index text with auxiliary info for each word. In particular at each word is associated the bounding box containing it in the original pdf page (it is used for highligthing the search terms in the pdf). I used the payl

Re: Semantic autocomplete with Solr

2012-02-14 Thread Roman Chyla
done something along these lines: https://svnweb.cern.ch/trac/rcarepo/wiki/InspireAutoSuggest#Autosuggestautocompletefunctionality but you would need MontySolr for that - https://github.com/romanchyla/montysolr roman On Tue, Feb 14, 2012 at 11:10 PM, Octavian Covalschi wrote: > Hey guys, > > H

Re: Semantic autocomplete with Solr

2012-02-14 Thread Octavian Covalschi
Hm... I used it for some basic group by feature, but haven't thought of it for autocomplete. I'll give it a shot. Thanks! On Tue, Feb 14, 2012 at 4:19 PM, Paul Libbrecht wrote: > facetting? > > paul > > > Le 14 févr. 2012 à 23:10, Octavian Covalschi a écrit : > > > Hey guys, > > > > Has anyone

Re: Can I rebuild an index and remove some fields?

2012-02-14 Thread Robert Stewart
I was thinking if I make a wrapper class that aggregates another IndexReader and filter out terms I don't want anymore it might work. And then pass that wrapper into SegmentMerger. I think if I filter out terms on GetFieldNames(...) and Terms(...) it might work. Something like: HashSet igno

Re: Semantic autocomplete with Solr

2012-02-14 Thread Paul Libbrecht
facetting? paul Le 14 févr. 2012 à 23:10, Octavian Covalschi a écrit : > Hey guys, > > Has anyone done any kind of "smart" autocomplete? Let's say we have a web > store, and we'd like to autocomplete user's searches. So if I'll type in > "jacket" next word that will be suggested should be some

Re: Need help with graphing function (MATH)

2012-02-14 Thread Kent Fitch
agreeing with wunder - I don't know the application, but I think almost always, a set of linear approximations over a few ranges would be ok (and you could increase the number of ranges until it was), and will be faster. And if you need just one equation, a sigmoid function will do the trick, such

Re: SolrCloud Replication Question

2012-02-14 Thread Mark Miller
Doh - looks like I was just seeing a test issue. Do you mind updating and trying the latest rev? At the least there should be some better logging around the recovery. I'll keep working on tests in the meantime. - Mark On Feb 14, 2012, at 3:15 PM, Jamie Johnson wrote: > Sounds good, if I pull

Re: OR-FilterQuery

2012-02-14 Thread Em
Hi Erick, > Whoa! > > fq=id(1 OR 2) > is not the same thing at all as > fq=id:1&fq=id:2 Ahm, who said they would be the same? :) I mean, you are completely right in what you are saying but it seems to me that we are talking about two different things. I was talking about caching each filter-crite

Re: OR-FilterQuery

2012-02-14 Thread Erick Erickson
BTW, you're not the first person who would like this capability, see: https://issues.apache.org/jira/browse/SOLR-1223 But the fact that this JIRA was originally opened in in June of 2009 and hasn't been implemented yet indicates that it's not super-high priority. Best Erick On Tue, Feb 14, 2012

Re: OR-FilterQuery

2012-02-14 Thread Erick Erickson
Whoa! fq=id(1 OR 2) is not the same thing at all as fq=id:1&fq=id:2 Assuming that any document had one and only one ID, the second clause would return exactly 0 documents, each and every time. Multiple fq clauses are essentially set intersections. So the first query is the set of all documents

Re: SolrCloud Replication Question

2012-02-14 Thread Jamie Johnson
Sounds good, if I pull the latest from trunk and rerun will that be useful or were you able to duplicate my issue now? On Tue, Feb 14, 2012 at 3:00 PM, Mark Miller wrote: > Okay Jamie, I think I have a handle on this. It looks like an issue with what > config files are being used by cores create

Re: SolrCloud Replication Question

2012-02-14 Thread Mark Miller
Okay Jamie, I think I have a handle on this. It looks like an issue with what config files are being used by cores created with the admin core handler - I think it's just picking up default config and not the correct config for the collection. This means they end up using config that has no Upda

Re: Solr 3.5 not starting on CentOS 6 or RHEL 5

2012-02-14 Thread Yonik Seeley
Perhaps this is some kind of vufind specific issue? The server (/example) bundled with solr unpacks the war in /example/work and not /tmp -Yonik lucidimagination.com On Mon, Feb 13, 2012 at 7:06 PM, Bernhardt, Russell (CIV) wrote: > A software package we use recently upgraded to Solr 3.5 (from 1

Re: Need help with graphing function (MATH)

2012-02-14 Thread Walter Underwood
In practice, I expect a linear piecewise function (with sharp corners) would be indistinguishable from the smoothed function. It is also much easier to read, test, and debug. It might even be faster. Try the sharp corners one first. wunder On Feb 14, 2012, at 10:56 AM, Ted Dunning wrote: > In

Re: Need help with graphing function (MATH)

2012-02-14 Thread Em
Hi Mark, did you already had a look at http://wiki.apache.org/solr/FunctionQuery ? Regards, Em Am 14.02.2012 20:09, schrieb Mark: > Or better yet an example in solr would be best :) > > Thanks! > > On 2/14/12 11:05 AM, Mark wrote: >> Would you mind throwing out an example of these types of fun

Re: OR-FilterQuery

2012-02-14 Thread Em
Hi Mikhail, thanks for kicking in some brainstorming-code! The given thread is almost a year old and I was working with Solr in my freetime to see where it fails to behave/perform as I expect/wish. I found out that if you got a lot of different access-patterns for a filter-query, you might end up

Re: Need help with graphing function (MATH)

2012-02-14 Thread Mark
Or better yet an example in solr would be best :) Thanks! On 2/14/12 11:05 AM, Mark wrote: Would you mind throwing out an example of these types of functions. Looking at Wikipedia (http://en.wikipedia.org/wiki/Probit) its seems like the Probit function is very similar to what I want. Thanks

Re: Need help with graphing function (MATH)

2012-02-14 Thread Mark
Would you mind throwing out an example of these types of functions. Looking at Wikipedia (http://en.wikipedia.org/wiki/Probit) its seems like the Probit function is very similar to what I want. Thanks On 2/14/12 10:56 AM, Ted Dunning wrote: In general this kind of function is very easy to con

Re: Need help with graphing function (MATH)

2012-02-14 Thread Ted Dunning
In general this kind of function is very easy to construct using sums of basic sigmoidal functions. The logistic and probit functions are commonly used for this. Sent from my iPhone On Feb 14, 2012, at 10:05, Mark wrote: > Thanks I'll have a look at this. I should have mentioned that the act

Re: OR-FilterQuery

2012-02-14 Thread Mikhail Khludnev
Hi Em, I briefly read the thread. Are you talking about combing of cached clauses of BooleanQuery, instead of evaluating whole BQ as a filter? I found something like that in API (but only in API) http://lucene.apache.org/solr/api/org/apache/solr/search/ExtendedQuery.html#setCacheSep(boolean) Am

Re: Re: Solr 3.5 not starting on CentOS 6 or RHEL 5

2012-02-14 Thread Bernhardt, Russell (CIV)
Nope, I don't have a custom /tmp mount in fstab, I just have a basic CentOS 6 install for development and testing... Full everyone read/write permissions are in place on /tmp too. > Is /tmp a separate file system? There are problems with people > mounting /tmp with 'noexec' as a security precau

Re: Need help with graphing function (MATH)

2012-02-14 Thread Gora Mohanty
On 14 February 2012 23:35, Mark wrote: > Thanks I'll have a look at this. I should have mentioned that the actual > values on the graph aren't important rather I was showing an example of how > the function should behave. [...] >> either y = (100/(e^((x -50)/75)^2)) + 50 [...] In general, the ex

Re: OR-FilterQuery

2012-02-14 Thread Erick Erickson
bq: Is the Filter Cache used for the OR'ed fq? The filter cache is actually pretty simple conceptually. It's just a map where the key is the fq and the value is the set of documents that satisfy that fq (we'll skip the implementation here, just think of it as the list of all the docs that the fq

Re: Need help with graphing function (MATH)

2012-02-14 Thread Mark
Thanks I'll have a look at this. I should have mentioned that the actual values on the graph aren't important rather I was showing an example of how the function should behave. On 2/13/12 6:25 PM, Kent Fitch wrote: Hi, assuming you have x and want to generate y, then maybe - if x < 50, y = 15

Re: OR-FilterQuery

2012-02-14 Thread Mikhail Khludnev
On Mon, Feb 13, 2012 at 11:17 PM, wrote: > Hi, > > how efficent is such an query: > > q=some text > fq=id:(1 OR 2 OR 3...) > > Should I better use q:some text AND id:(1 OR 2 OR 3...)? > 1. These two opts have the different scoring. 2. if you hit same fq=id:(1 OR 2 OR 3...) many times you have a b

Re: SolrCloud Replication Question

2012-02-14 Thread Jamie Johnson
Thanks Mark, not a huge rush, just me trying to get to use the latest stuff on our project. On Tue, Feb 14, 2012 at 10:53 AM, Mark Miller wrote: > Sorry, have not gotten it yet, but will be back trying later today - monday, > tuesday tend to be slow for me (meetings and crap). > > - Mark > > On

Re: SolrJ + SolrCloud

2012-02-14 Thread Mark Miller
No hard plans around that that at the moment, but when I free up some time I plan on looking at the JIRA issue I pointed to. Looks like a lot of the work may already be done. - mark On Feb 12, 2012, at 8:14 AM, Darren Govoni wrote: > Thanks Mark. Is there any plan to make all the Solr search h

Re: SolrCloud Replication Question

2012-02-14 Thread Mark Miller
Sorry, have not gotten it yet, but will be back trying later today - monday, tuesday tend to be slow for me (meetings and crap). - Mark On Feb 14, 2012, at 9:10 AM, Jamie Johnson wrote: > Has there been any success in replicating this? I'm wondering if it > could be something with my setup tha

Re: Highlighting stopwords

2012-02-14 Thread O. Klein
Koji Sekiguchi wrote > > Uh, what you tried was that you changed the field between q and hl.q, > that I've not expected use case when I proposed hl.q. > > Do you think that hl.text meats your needs? > > https://issues.apache.org/jira/browse/SOLR-1926?focusedCommentId=12871234&page=com.atlassian

Re: Stemming and accents (HunspellStemFilterFactory)

2012-02-14 Thread Chantal Ackermann
Hi Bráulio, I don't know about HunspellStemFilterFactory especially but concerning accents: There are several accent filter that will remove accents from your tokens. If the Hunspell filter factory requires the accents, then simply add the accent filters after Hunspell in your index and query fil

Re: Highlighting stopwords

2012-02-14 Thread Koji Sekiguchi
(12/02/14 22:25), O. Klein wrote: I have not been able to find any logic in the behavior of hl.q and how it analyses the query. Could you explain how it is supposed to work? Nothing special on hl.q. If you use hl.q, the value of it will be used for highlighting rather than the value of q. There

Re: Improving performance for SOLR geo queries?

2012-02-14 Thread Bill Bell
Can we get this back ported to 3x? Bill Bell Sent from mobile On Feb 14, 2012, at 3:45 AM, Matthias Käppler wrote: > hey thanks all for the suggestions, didn't have time to look into them > yet as we're feature-sprinting for MWC, but will report back with some > feedback over the next weeks (w

Mmap

2012-02-14 Thread Bill Bell
Does someone have an example of using unmap in 3.5 and chunksize? I am using Solr 3.5. I noticed in solrconfig.xml: I don't see this parameter taking.. When I set -Dsolr.directoryFactory=solr.MMapDirectoryFactory How do I see the setting in the log or in stats.jsp ? I cannot find a place t

Re: Solr binary response for C#?

2012-02-14 Thread Erick Erickson
It's not as compact as binary format, but would just using something like JSON help enough? This is really simple, just specify &wt=json (there's a method to set this on the server, at least in Java). Otherwise, you might get a more knowledgeable response on the C# java list, I'm frankly clueless.

Debugging on 3,5

2012-02-14 Thread Bill Bell
I did find a solution, but the output is horrible. Why does explain look so badly? 6.351252 = (MATCH) boost(*:*,query(specialties_ids: #1;#0;#0;#0;#0;#0;#0;#0;#0; ,def=0.0)), product of: 1.0 = (MATCH) MatchAllDocsQuery, product of: 1.0 = queryNorm 6.351252 = query(specialties_ids: #1;#

Re: SolrCloud Replication Question

2012-02-14 Thread Jamie Johnson
Has there been any success in replicating this? I'm wondering if it could be something with my setup that is causing the issue... On Mon, Feb 13, 2012 at 8:55 AM, Jamie Johnson wrote: > Yes, I have the following layout on the FS > > ./bootstrap.sh > ./example (standard example directory from di

Stemming and accents (HunspellStemFilterFactory)

2012-02-14 Thread Bráulio Bhavamitra
Hello all, I'm evaluating the HunspellStemFilterFactory I found it works with a pt_PT dictionary. For example, if I search for 'fóruns' it stems it to 'fórum' and then find 'fórum' references. But if I search for 'foruns' (without accent), then HunspellStemFilterFactory cannot stem word, as it d

'foruns' don't match 'forum' with NGramFilterFactory (or EdgeNGramFilterFactory)

2012-02-14 Thread Bráulio Bhavamitra
Hello all, I'm experimenting with NGramFilterFactory and EgdeNGramFilterFactory. Both of them shows a match in my solr admin analysis, but when I query 'foruns' doesn't find any 'forum'. analysis http://bhakta.casadomato.org:8982/solr/admin/analysis.jsp?nt=type&name=text&verbose=on&highlight=on&v

Re: Highlighting stopwords

2012-02-14 Thread O. Klein
O. Klein wrote > > > O. Klein wrote >> >> Hmm, now the synonyms aren't highlighted anymore. >> >> OK back to basic (im using trunk and FVH). >> >> What is the way to go about if I want to search on a field without >> stopwords, but still want to highlight the stopwords? (and still >> highligh

Re: sort my results alphabetically on facetnames

2012-02-14 Thread Michael Kuhlmann
Hi! On 14.02.2012 13:09, PeterKerk wrote: I want to sort my results on the facetnames (not by their number of results). From the example you gave, I'd assume you don't want to sort by facet names but by facet values. Simply add facet.sort=index to your request; see http://wiki.apache.org/so

Re: Re:how to monitor solr in newrelic

2012-02-14 Thread roySolr
Try this when you start SOLR java -javaagent:/NEWRELICPATH/newrelic.jar -jar start.jar Normally you will see your SOLR installation on your newrelic dashboard in 2 minutes. -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-monitor-solr-in-newrelic-tp3739567p3743488.ht

sort my results alphabetically on facetnames

2012-02-14 Thread PeterKerk
I want to sort my results on the facetnames (not by their number of results). So now I have this (ordered by number of results): Instelling voor auditief gehandicapten (16) Audiologisch centrum (13) Huisartsenpraktijk (13) Instelling voor lichamelijk gehandicapten (13) Ambulancezorg (12) Beroepsor

Re: Language specific tokenizer for purpose of multilingual search in single-core solr,

2012-02-14 Thread Paul Libbrecht
only one field element? There should be two or? One for each language. paul Le 14 févr. 2012 à 07:34, bing a écrit : > > Hi, all, > > I want to do multilingual search in single-core solr. That requires to > define language specific tokenizers in scheme.xml. Say for example, I have > two toke

Re: Improving performance for SOLR geo queries?

2012-02-14 Thread Matthias Käppler
hey thanks all for the suggestions, didn't have time to look into them yet as we're feature-sprinting for MWC, but will report back with some feedback over the next weeks (we will have a few more performance sprints in March) Best, Matthias On Mon, Feb 13, 2012 at 2:32 AM, Yonik Seeley wrote: >

Re: Highlighting stopwords

2012-02-14 Thread O. Klein
O. Klein wrote > > Hmm, now the synonyms aren't highlighted anymore. > > OK back to basic (im using trunk and FVH). > > What is the way to go about if I want to search on a field without > stopwords, but still want to highlight the stopwords? (and still highlight > synonyms and stemmed words)?