Re: understanding phonetic matching

2016-03-22 Thread Alexandre Rafalovitch
I'd start by putting LowerCaseFF before the PhoneticFilter. But then, you say you were using Analysis screen and what? Do you get the matches when you put your sample text and the query text in the two boxes in the UI? I am not sure what "look at my solr data" means in this particular context. Re

understanding phonetic matching

2016-03-22 Thread Jay Potharaju
Hi, I am trying to do name matching using the phonetic filter factory. As part of that I was analyzing the data using analysis screen in solr UI. If i search for john, any documents containing john or jon should be found. Following is my definition of the custom field that I use for indexing the d

Re: Seasonal searches in SOLR 5.x

2016-03-22 Thread David Smiley
Hi, I suggest having a "season" field (or whatever you might want to call it) using DateRangeField but simply use a nominal year value. So basically all durations would be within this nominal year. For some docs that span new-years, this might mean 2 durations and that's okay. Also it's okay if

Re: Delete by query using JSON?

2016-03-22 Thread Jack Krupansky
See the correct syntax example here: https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-SendingJSONUpdateCommands Your query is fine. -- Jack Krupansky On Tue, Mar 22, 2016 at 3:07 PM, Paul Hoffman wrote: > I've been struggling to

Re: Delete by query using JSON?

2016-03-22 Thread Walter Underwood
“Why do you care?” might not be the best way to say it, but it is essential to understand the difference between selection (filtering) and ranking. As Solr params: * q is ranking and filtering * fq is filtering only * bq is ranking only When deleting documents, ordering does not matter, which i

Re: Delete by query using JSON?

2016-03-22 Thread Robert Brown
"why do you care? just do this ..." I see this a lot on mailing lists these days, it's usually a learning curve/task/question. I know I fall into these types of questions/tasks regularly. Which usually leads to "don't tell me my approach is wrong, just explain what's going on, and why", or

Re: Delete by query using JSON?

2016-03-22 Thread Alexandre Rafalovitch
Why do you care? The difference between Q and FQ are the scoring. For delete, you delete all of them regardless of scoring and there is no difference. Just chuck them all into Q. Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On

Delete by query using JSON?

2016-03-22 Thread Paul Hoffman
I've been struggling to find the right syntax for deleting by query using JSON, where the query includes an fq parameter. I know how to delete *all* documents, but how would I delete only documents with field doctype = "cres"? I have tried the following along with a number of variations, all t

RE: PERFORMANCE WARNING: Overlapping onDeckSearchers=2

2016-03-22 Thread Aswath Srinivasan (TMS)
>> Since you've already reproduced it on a small scale, we'll need your entire >> Solr logfile. The mailing list eats attachments, so you'll need to place it >> somewhere and provide a URL. Sites like gist and dropbox are excellent for >> sharing large text content. Sure. I will try and sent

DIH cant index adresses web

2016-03-22 Thread kostali hassan
I try to index rich data (msword and pdf) but when a content of document have multiple liens (web adress) i get an ERROR in log . what i have to add in my tika-config.xml to index web path .

Solr 5.3: anything similar to ChildDocTransformerFactory that does not flatten the hierarchical structure?

2016-03-22 Thread Alisa Z .
Hi all, Following the example from https://dzone.com/articles/using-solr-49-new , let's say we are given multiple-level nested structure: 1 I am the parent PARENT 1.1 I am the 1st child CHILD

Re: PERFORMANCE WARNING: Overlapping onDeckSearchers=2

2016-03-22 Thread Shawn Heisey
On 3/22/2016 11:32 AM, Aswath Srinivasan (TMS) wrote: > Thank you Shawn for taking time and responding. > > Unfortunately, this is not the case. My heap is not even going past 50% and I > have a heap of 10 GB on a instance that I just installed as a standalone > version and was only trying out th

Cant access new docs without restarting Solr (java.nio.file.NoSuchFileException)

2016-03-22 Thread Victor D'agostino
Hi I've setup a Solr Cloud 5.5.0 ensemble with ZooKeeper. If I post a few docs with curl it seems ok : [root@LXLYOSOL30 ~]# curl --noproxy '*' http://lxlyosol30:8983/solr/db/update --data-binary @/data/conf-cpm3/test.txt -H 'Content-type:application/xml' 0name="QTime">18 But when I go t

RE: PERFORMANCE WARNING: Overlapping onDeckSearchers=2

2016-03-22 Thread Aswath Srinivasan (TMS)
>> If you're not actually hitting OutOfMemoryError, then my best guess about >> what's happening is that you are running >>right at the edge of the >> available Java heap memory, so your JVM is constantly running full garbage >> collections to free up >>enough memory for normal operation. In th

Re: Creating new cluster with existing config in zookeeper

2016-03-22 Thread Robert Brown
Thanks Erick and Shawn, a "collection" is indeed what I meant. I was under the impression the entire Tree view in the admin GUI was showing everything in ZK, including things like "collections/name/state.json", not just the /configs directory. The solr.xml file is too isn't it? (I added it to

Re: java.lang.NullPointerException in json facet hll function

2016-03-22 Thread Yago Riveiro
Nop. A normal query with wt=json the q parameter is *:* The unique particular thing with this index is that some docs has the field visitor__visitor_id as dynamic type long and others has the field as type string. (our indexer tool didn't resolve the type right as result of a bug, th

Re: java.lang.NullPointerException in json facet hll function

2016-03-22 Thread Yonik Seeley
Hmmm, looks like the "hll" value is missing for some reason. It's not clear why that would happen... are you running any custom code? -Yonik On Tue, Mar 22, 2016 at 12:54 PM, Yago Riveiro wrote: > Solr version: 5.3.1 > > With this query: > > group: > { > type:terms, >

Re: Creating new cluster with existing config in zookeeper

2016-03-22 Thread Shawn Heisey
On 3/22/2016 6:38 AM, Robert Brown wrote: Is it safe to create a new cluster but use an existing config set that's in zookeeper? Or does that config set contain the cluster status too? I want to (re)-build a cluster from scratch, with a different amount of shards, but not using shard-splitti

java.lang.NullPointerException in json facet hll function

2016-03-22 Thread Yago Riveiro
Solr version: 5.3.1 With this query: group: { type:terms, limit:-1, field:group, sort:{index:asc}, numBuckets:true, facet:{ col_1_unique_visitors:'hll(visitor__visitor_i

Re: Creating new cluster with existing config in zookeeper

2016-03-22 Thread Erick Erickson
The whole _point_ of configsets is to re-use them in multiple collections, so please do! Best, Erick On Tue, Mar 22, 2016 at 5:38 AM, Robert Brown wrote: > Hi, > > Is it safe to create a new cluster but use an existing config set that's in > zookeeper? Or does that config set contain the cluste

Re: Next Solr Release - 5.5.1 or 6.0 ?

2016-03-22 Thread Erick Erickson
No real plans have been made that I know of for a 5.5.1 release. What happens is that 5.5 was cut as, potentially, the last 5x release. Some small fixes are still back-ported "just in case" there's a 5.5.1 release. As for 6.0, that's something of a moving target currently, we're still flushing out

Re: Solr 5.5.0: JVM args warning in console logfile.

2016-03-22 Thread Bram Van Dam
On 22/03/16 15:16, Shawn Heisey wrote: > This message is not coming from Solr. It's coming from Jetty. Solr > uses Jetty, but uses it completely unchanged. Ah you're right. Here's the offending code: https://github.com/eclipse/jetty.project/blob/ac24196b0d341534793308d585161381d5bca4ac/jetty-st

Re: Explain score is different from score

2016-03-22 Thread Ahmet Arslan
Hi all, May be it is better to move the discussion into a jira ticket. I created SOLR-8884 for this. aHmet On Tuesday, March 22, 2016 1:59 PM, Alessandro Benedetti wrote: I got this problem re-ranking. But in my short experience I was not able to reproduce nor fix the bug. Can I ask you

Next Solr Release - 5.5.1 or 6.0 ?

2016-03-22 Thread Alessandro Benedetti
Hi gents, I am planning a version upgrade, is it possible to know the next upcoming version ? >From the Solr news I see the next one will be Solr 6.0 in 4 weeks approximately. But from Jira I see also the 5.5.1 with 8 Jira issues in it. Is it possible to have an estimation of the release dates ?

Re: Paging and cursorMark

2016-03-22 Thread Steve Rowe
Hi Tom, There is an outstanding JIRA issue to directly support what you want (with a patch even!) but no work on it recently: . If you’re so inclined, please pitch in: bring the patch up-to-date, test it, contribute improvements, etc. -- Steve

Re: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Yonik Seeley
Much of the merging / client code in Solr (not just the JSON Facets) uses things like ((Number)count).longValue() to handle either int or long values. -Yonik On Tue, Mar 22, 2016 at 4:46 AM, Markus Jelsma wrote: > Hello, > > Using SolrJ i built a method that consumes output produced by JSON fac

Re: Issue with Auto Suggester Component

2016-03-22 Thread Alessandro Benedetti
Let's try to keep it simple , please provide a query and the expected ranking of the results. I find quite difficult to read and intepret in this way :) Then we can sort a proper autocomplete out :) Cheers On Wed, Mar 16, 2016 at 10:52 AM, Manohar Sripada wrote: > Thanks for the response. > If

RE: Save Number of words in field

2016-03-22 Thread G, Rajesh
It works.Thanks Jack. Corporate Executive Board India Private Limited. Registration No: U741040HR2004PTC035324. Registered office: 6th Floor, Tower B, DLF Building No.10 DLF Cyber City, Gurgaon, Haryana-122002, India. This e-mail and/or its attachments are intended only for the use of the ad

NPE when executing clustering query search

2016-03-22 Thread Tim Hearn
Hi everyone, I am trying to execute a clustering query to my single-core master-slave solr setup and it is returning a NullPointerException. I checked the line in the source code where it is being thrown, and it looks like the null object is some sort of 'filt' object, which doesn't make sense.

Paging and cursorMark

2016-03-22 Thread Tom Evans
Hi all With Solr 5.5.0, we're trying to improve our paging performance. When we are delivering results using infinite scrolling, cursorMark is perfectly fine - one page is followed by the next. However, we also offer traditional paging of results, and this is where it gets a little tricky. Say we

Re: Solr 5.5.0: JVM args warning in console logfile.

2016-03-22 Thread Shawn Heisey
On 3/22/2016 6:57 AM, Bram Van Dam wrote: > Hey folks, > > When I start 5.5.0 (on RHEL), the following entry is added to > server/logs/solr-8983-console.log: > > WARNING: System properties and/or JVM args set. Consider using > --dry-run or --exec > > I can't quite figure out what's causing this. A

Antw: Re: Boosting of Join Results

2016-03-22 Thread Alena Dengler
Mikhail, Thanks a lot for the suggestion. We now implemented the query as follows: q=(+geschichte +rom) OR _query_:{!boost b=0.01}{!join from=expandtype fromIndex=pages to=id score=avg v='pageno_content:(+geschichte +rom)'}) With the factor of 0.01 it seems to work well with our data. Best Re

Solr 5.5.0: JVM args warning in console logfile.

2016-03-22 Thread Bram Van Dam
Hey folks, When I start 5.5.0 (on RHEL), the following entry is added to server/logs/solr-8983-console.log: WARNING: System properties and/or JVM args set. Consider using --dry-run or --exec I can't quite figure out what's causing this. Any clues on how to get rid of it? Thanks, - Bram

Creating new cluster with existing config in zookeeper

2016-03-22 Thread Robert Brown
Hi, Is it safe to create a new cluster but use an existing config set that's in zookeeper? Or does that config set contain the cluster status too? I want to (re)-build a cluster from scratch, with a different amount of shards, but not using shard-splitting. Thanks, Rob

Re: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Yago Riveiro
I have a felling that this is related with the number of nodes of the cluster. My dev runs in cloud mode but only has one node, production has 12, and the version is the same. \-- /Yago Riveiro > On Mar 22 2016, at 9:13 am, Markus Jelsma wrote: > > I'

Re: Explain score is different from score

2016-03-22 Thread Alessandro Benedetti
I got this problem re-ranking. But in my short experience I was not able to reproduce nor fix the bug. Can I ask you the query aprser used and all the components involved in the query ? Cheers On Mon, Mar 21, 2016 at 8:40 PM, Rick Sullivan wrote: > I haven't checked this thread since Friday, b

Re: Boosting of Join Results

2016-03-22 Thread Mikhail Khludnev
what is you nest join into boost eg q=+foo {!boost ..}{!join ... v=...} see https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BoostQueryParser if it works, you may vote for https://issues.apache.org/jira/browse/SOLR-7814 On Tue, Mar 22, 2016 at 12:39 PM, Alena Dengler <

Boosting of Join Results

2016-03-22 Thread Alena Dengler
Hello, we are currently developing a combined index for book metadata and fulltexts. Our primary core contains metadata of ~12Mio. books. ~0.5Mio. of them have fulltexts; those fulltexts are indexed in a secondary core. This secondary core has one index document per fulltext page. We are joining

RE: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Markus Jelsma
I'm now using instanceof as ugly work around but i'd prefer a decent solution. M -Original message- > From:Yago Riveiro > Sent: Tuesday 22nd March 2016 9:52 > To: solr-user ; solr-user@lucene.apache.org > Subject: Re: JSON facets, count a long or an integer in cloud and non-cloud > m

Re: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Yago Riveiro
I have the same problem with a custom response writer. In production works but in my dev doesn't and are the same version 5.3.1 -- Yago Riveiro On 22 Mar 2016 08:47 +, Markus Jelsma, wrote: > Hello, > > Using SolrJ i built a method that consumes output produced by JSON facets, it > also ch

JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Markus Jelsma
Hello, Using SolrJ i built a method that consumes output produced by JSON facets, it also checks the count before further processing the output: 49 This is the code reading the count value via SolrJ: QueryResponse response = sourceClient.query(query); NamedList json

Custom shard key

2016-03-22 Thread Anil
HI, i am using explicit shading by creating custom shard key for my application using hbase util mumurhash (added snippet below). int hash = MurmurHash.getInstance().hash(sharekey.getBytes()); hash = Math.abs(hash); int routingValue = hash % shards; I noticed only 5 out of 8 shards used and 3 sh