Date number to text

2014-06-06 Thread O. Klein
Is there a way to tokenize 12-6-2014 as 12 june 2014 and the other way around as well? Making it possible for people to search on text and numbers for dates? -- View this message in context: http://lucene.472066.n3.nabble.com/Date-number-to-text-tp4140323.html Sent from the Solr - User

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Vineet Mishra
Hi Shawn, Thanks for your response, wanted to clarify a few things. *Does that mean for querying smoothly we need to have memory atleast equal or greater to the size of index? As in my case the index size will be very heavy(~2TB) and practically speaking that amount of memory is not possible.

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Vineet Mishra
Hey Jack, Well I have indexed around some 10 Million documents consuming 20 GB index size. Each Document is consisting of nearly 100 String Fields with data upto 10 characters per field. For my case each document containing number of fields can expand much widely (from current 100 to 500 or ever

Analysis browser not working in solr 4.8.1

2014-06-06 Thread Aman Tandon
Hi, I created a custom filter for my field named text_reversed, i tried my custom filter in solr 4.7.1 and i was able to analyse the result, it works fine but in solr 4.8.1 it gaves me error of : *Missing required parameter: analysis.fieldvalue. *It is also not working with any field*, *here is

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Aman Tandon
The url forming is *http://localhost:8984/solr/#/IM-Search/analysis?analysis.query=juteanalysis.fieldname=titleverbose_output=1 http://localhost:8984/solr/#/IM-Search/analysis?analysis.query=juteanalysis.fieldname=titleverbose_output=1* Here is the screenshot: http://picpaste.com/HrW26A8d.png

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Toke Eskildsen
On Fri, 2014-06-06 at 12:32 +0200, Vineet Mishra wrote: *Does that mean for querying smoothly we need to have memory atleast equal or greater to the size of index? If you absolutely, positively have to reduce latency as much as possible, then yes. With an estimated index size of 2TB, I would

Re: Tika: url issue

2014-06-06 Thread harshrossi
Thanks for the help.. But anyway I solved it using RegexTransformer In the db_link entity I used RegexTransformer and set the link field as: field column=link regex=^([^#]+) sourceColName=LINK/ and in tika-doc entity I set the Url value as: ${db_link.link} -- View this message in context:

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Vineet Mishra
Hi Toke, That was Spectacular, really great to hear that you have already indexed 2.7TB+ data to your server and still the query response time is under ms or a few seconds for such a huge dataset. Could you state what indexing mechanism are you using, as I started with EmbeddedSolrServer but it

Re: Integrate solr with openNLP

2014-06-06 Thread Aman Tandon
Hi Vikek, As everybody in the mail list mentioned to use UIMA you should go for it, as opennlp issues are not tracking properly, it can make stuck your development in near future if any issue comes, so its better to start investigate with uima. With Regards Aman Tandon On Fri, Jun 6, 2014 at

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Toke Eskildsen
On Fri, 2014-06-06 at 14:05 +0200, Vineet Mishra wrote: Could you state what indexing mechanism are you using, as I started with EmbeddedSolrServer but it was pretty slow after a few GB(~30+) of indexing. I suspect that is due to too-frequent commits, too small heap or something third,

Re: Solr-Distributed search

2014-06-06 Thread Aman Tandon
Hi, Does this *shards* parameter will also work in near future with solr 5? With Regards Aman Tandon On Thu, Jun 5, 2014 at 2:59 PM, Mahmoud Almokadem prog.mahm...@gmail.com wrote: Hi, you can search using this sample Url

span query with SHOUD semantic instead of MUST HAVE

2014-06-06 Thread ??????
hi, I have two docs, a) aa bb cc and, b) aa cc bb. The query is aa bb. What I expected is the doc a comes first with a higher score than doc b because the term distance in query and that in doc a are more similar. After google for a while I get it down with the span query q: aa

Re: Solr maximum Optimal Index Size per Shard

2014-06-06 Thread Vineet Mishra
Earlier I used to index with HtttpPost Mechanism only, making each post size specific to 2Mb to 20Mb that was going fine, but we had a suspect that instead of indexing through network call(which ofcourse results in latency due to network delays and http protocol) if we can index Offline by just

Re: SolrCloud: AliasAPI-Maximum number of collections

2014-06-06 Thread Shalin Shekhar Mangar
No, there's no theoretical limit. On Fri, Jun 6, 2014 at 11:20 AM, ku3ia dem...@gmail.com wrote: Hi all! The question is how many collections I can put to one alias, using SolrCloud alias collection API https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api4

Re: SolrCloud: AliasAPI-Maximum number of collections

2014-06-06 Thread Mark Miller
The main limit is the 1mb zk node limit. But even that can be raised. - Mark On Jun 6, 2014, at 6:21 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: No, there's no theoretical limit. On Fri, Jun 6, 2014 at 11:20 AM, ku3ia dem...@gmail.com wrote: Hi all! The question is

Deepy nested structure

2014-06-06 Thread harikrishna
we need to have the nested structure for the index, and the requirement is as follows we have application at root, then customer location, and then we have some entities data Add doc applicaton cust location Doc order id doc products /doc /Doc DoC /add i want to index the data in the above

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Stefan Matheis
I’m not sure that’s a bug in the UI .. in case the underlying service is barking with an exception we can’t do anything else than showing to you. are you sure the custom filter works as expected? like, verified with a unit-test or something along the lines? i can still work with the examples

Documents Added Not Available After Commit (Both Soft and Hard)

2014-06-06 Thread Justin Sweeney
Hi, An application I am working on indexes documents to a Solr index. This Solr index is setup as a single node, without any replication. This index is running Solr 4.5.0. We have noticed an issue lately that is causing some problems for our application. The problem is that we add/update a

Re: Date number to text

2014-06-06 Thread Shawn Heisey
On 6/6/2014 2:32 AM, O. Klein wrote: Is there a way to tokenize 12-6-2014 as 12 june 2014 and the other way around as well? Making it possible for people to search on text and numbers for dates? Dates in Solr (if you use an actual date field, and not a text field) follow strict interpretation

Re: Solr-Distributed search

2014-06-06 Thread Shawn Heisey
On 6/6/2014 6:25 AM, Aman Tandon wrote: Does this *shards* parameter will also work in near future with solr 5? I am not aware of any plan to deprecate or remove the shards parameter. My personal experience is with versions from 1.4.0 through 4.7.2. It works in all of those versions. Without

Re: Date number to text

2014-06-06 Thread Jack Krupansky
The parse date update processor can be used to parse dates in a number of formats (configurable) and convert to a Java date value: http://lucene.apache.org/solr/4_8_1/solr-core/org/apache/solr/update/processor/ParseDateFieldUpdateProcessorFactory.html You could copy the field to another field

Re: Solr-Distributed search

2014-06-06 Thread Aman Tandon
Thanks shawn. In my organisation we also want to implement the solrcloud, but the problem is that, we are using the master-slave architecture and on master we do all indexing, architecture of master is lower than the slaves. So if we implement the solrcloud in a fashion that master will be the

Re: Solr-Distributed search

2014-06-06 Thread Aman Tandon
Thanks shawn. In my organisation we also want to implement the solrcloud, but the problem is that, we are using the master-slave architecture and on master we do all indexing, architecture of master is lower than the slaves. So if we implement the solrcloud in a fashion that master will be the

Re: Solr-Distributed search

2014-06-06 Thread Shawn Heisey
On 6/6/2014 8:31 AM, Aman Tandon wrote: In my organisation we also want to implement the solrcloud, but the problem is that, we are using the master-slave architecture and on master we do all indexing, architecture of master is lower than the slaves. So if we implement the solrcloud in a

Re: Solr-Distributed search

2014-06-06 Thread Aman Tandon
Thanks shawn i will try to think in that way too :) With Regards Aman Tandon On Fri, Jun 6, 2014 at 8:19 PM, Shawn Heisey s...@elyograg.org wrote: On 6/6/2014 8:31 AM, Aman Tandon wrote: In my organisation we also want to implement the solrcloud, but the problem is that, we are using the

Can the elevation component work with synonyms?

2014-06-06 Thread eShard
Good morning Solr compatriots, I'm using Solr4.0Final and I have synonyms.txt in my schema (only at query time) like so: fieldType name=text_general class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Aman Tandon
It is not working with either custom filter/field as well as with normal text field, it only working fine if i pass the value/text in the both in the index time input as well as in query time input. Only query time analysis is impossible unless i wrote that extra parameter

Re: Can the elevation component work with synonyms?

2014-06-06 Thread Shawn Heisey
On 6/6/2014 9:08 AM, eShard wrote: However, when I try to call my /elevate handler; the synonyms are factored in but none of the results say [elevated]=true I'm assuming this is because the elevation must be an exact match and the synonyms are expanding it beyond that so elevation is thwarted.

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Shalin Shekhar Mangar
I think I created this problem in SOLR-6023. I went with the javadoc documentation which says that analysis.value is a required parameter and added that check. It seems that the either analysis.value or analysis.q (or q) can be specified to make it work. I'll fix in 4.9 On Fri, Jun 6, 2014 at

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Aman Tandon
Okay thanks shalin, i am fine with explicit till 4.9 :) So i don't need to open the issue right? With Regards Aman Tandon On Fri, Jun 6, 2014 at 9:20 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: I think I created this problem in SOLR-6023. I went with the javadoc documentation

timeout when create alias

2014-06-06 Thread lansing
Hi, We are running Solr cloud 4.8.0 on tomcat 7, java 7 update 51, with 5 shards and 3 replicas. When I try to create alias via collection admin api http://10.0.5.227:8201/solr/admin/collections?action=CREATEALIASname=Currentcollections=Current2 It timeout after 180 s, 181344574

Re: Analysis browser not working in solr 4.8.1

2014-06-06 Thread Shalin Shekhar Mangar
Yes, please do open an issue. I'll fix it there. On Fri, Jun 6, 2014 at 9:33 PM, Aman Tandon amantandon...@gmail.com wrote: Okay thanks shalin, i am fine with explicit till 4.9 :) So i don't need to open the issue right? With Regards Aman Tandon On Fri, Jun 6, 2014 at 9:20 PM, Shalin

RE: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

2014-06-06 Thread Joel Bernstein
Reposting this from jira ticket to users list: I'm noticing a very weird bug using the CollapsingQParserPlugin. We tried to use this plugin when we realized that faceting on the groups would take a ridiculous amount of time. To its credit, it works very quickly, however the facet counts that it

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

2014-06-06 Thread Joel Bernstein
The CollapsingQParserPlugin should give you the same facet counts as group.truncate. You're using group.facets, which the CollapsingQParserplugin doesn't yet support. I think this would be an excellent feature, so we could make a jira ticket to add this feature. Joel Bernstein Search Engineer at

Re: timeout when create alias

2014-06-06 Thread Shalin Shekhar Mangar
Yes, a collection API can timeout particularly when a long running collection operation is running. The collection API tasks are executed serially right now so new tasks get stuck behind an old one until it completes. This is going to change in the next release (4.9) with SOLR-5681 which makes the

Solr Scale Toolkit Access Denied Error

2014-06-06 Thread Mark Gershman
I've been attempting to experiment with the recently updated Solr Scale Tool Kit mentioned here: http://searchhub.org/2014/06/03/introducing-the-solr-scale-toolkit/ After making the very well documented configuration changes at AWS and installing Python, I was able to use the toolkit to connect

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

2014-06-06 Thread david.w.smi...@gmail.com
I may be misunderstanding the problem, but if it’s what I think it is, then users can work-around this now quite easily by using Solr faceting’s ability to exclude a named/tagged filter query: q=classIDs:12 fl=PrSKU fq={!collapse tag=collapse field=PrSKU} facet=true facet.field={!

Re: span query with SHOUD semantic instead of MUST HAVE

2014-06-06 Thread Ahmet Arslan
Hi, How come query aa bb returns document b? On Friday, June 6, 2014 3:48 PM, 郑华斌 huabin.zh...@qq.com wrote: hi, I have two docs,     a) aa bb cc and,     b) aa cc bb. The query is aa bb. What I expected is the doc a comes first with a higher score than doc b because the term distance in

Re: Multivalue wild card search

2014-06-06 Thread Ethan
Bumping the thread to see if anyone has a solution. On Thu, Jun 5, 2014 at 9:52 AM, Ethan eh198...@gmail.com wrote: Wildcard search do work on multiValued field. I was able to pull up records for following multiValued field - Code : [ 12344, 4534, 674 ] q=Code:45* fetched the correct

Re: Multivalue wild card search

2014-06-06 Thread Ahmet Arslan
Hi Ethan, It is hard to understand your example. Can you re-write it? Using xml?  On Friday, June 6, 2014 9:07 PM, Ethan eh198...@gmail.com wrote: Bumping the thread to see if anyone has a solution. On Thu, Jun 5, 2014 at 9:52 AM, Ethan eh198...@gmail.com wrote: Wildcard search do work

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

2014-06-06 Thread Joel Bernstein
With tag and exclude you can get the facets counts for the collapsed set and expanded set in the same query. But group.facets is a different count then either of those. group.facets counts each facet bucket once per group. Joel Bernstein Search Engineer at Heliosearch On Fri, Jun 6, 2014 at

Re: Delta Import Functionality

2014-06-06 Thread Ahmet Arslan
Hi, I think you need to select * in deltaImportQuery. You are just selecting one field for both delta*Query SQL sentences. On Thursday, June 5, 2014 3:34 PM, ajay59 ajay.gup...@wipro.com wrote: Hi, We are using the SOLR 4.6 version and trying to implement Delta import functionality .On

Re: Strange Behavior with Solr in Tomcat.

2014-06-06 Thread S.L
Anyone folks? On Wed, Jun 4, 2014 at 10:25 AM, S.L simpleliving...@gmail.com wrote: Hi Folks, I recently started using the spellchecker in my solrconfig.xml. I am able to build up an index in Solr. But,if I ever shutdown tomcat I am not able to restart it.The server never spits out the

RE: Strange Behavior with Solr in Tomcat.

2014-06-06 Thread Jean-Sebastien Vachon
I would try a thread dump and check the output to see what`s going on. You could also strace the process if you`re running on Unix or changed the log level in Solr to get more information logged -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: June-06-14 2:33 PM

Re: Strange Behavior with Solr in Tomcat.

2014-06-06 Thread Meraj A. Khan
This looks distinctly related to https://issues.apache.org/jira/browse/SOLR-4408 , try coldSearcher = true as being suggested in JIRA and let us know . On Fri, Jun 6, 2014 at 2:39 PM, Jean-Sebastien Vachon jean-sebastien.vac...@wantedanalytics.com wrote: I would try a thread dump and check

Solr and Kerberos

2014-06-06 Thread Tom Chen
Hi, I wonder how to secure Solr with Kerberos. We can Kerberos secure Solr by configuring the AuthenticationFilter from the hadoop-auth.jar that is packaged in solr.war. But after we do that, 1) How does a SolrJ client connect to the secured Solr server? 2) In SolrCloud environment, how one

Solr Realtime Get RemoteSolrException: Expected mime type application/xml but got text/html

2014-06-06 Thread Songtao Zheng
Solr version on remote server: solr-4.3.1 I am trying to use Solr Realtime Get http://wiki.apache.org/solr/RealTimeGet to retrieve document before commit. My code class Test3 { static main(args) { def test = new Test3() test.run() } private run() { String url =

RE: SolrCloud: facet range option f.field.facet.mincount=1 omits buckets on response

2014-06-06 Thread Chris Hostetter
Ronald: I'm having a little trouble understading the steps o reproduce that you are describing -- in particular Step 1 f ii because i'm not really sure i understand what exactly you are putting in mem2.xml Also: Since you don't appera to be using implicit routing, i'm not clear on why you

Mapping a field name before queryParser

2014-06-06 Thread Antoine LE FLOC'H
Hello, I have a query like the following where brand is a field in my schema: select?rows=1start=0sort=price+ascq=brand:sonyqt=for-searchwt=xml But I want to do this instead: select?rows=1start=0sort=price+ascq=brand_name:sonyqt=for-searchwt=xml and define something like brand_name:brand in

Re: Multivalue wild card search

2014-06-06 Thread Jack Krupansky
Wildcard, fuzzy, and regex query do work on multivalued fields, but on only a single value at a time. Your match cannot occur across values. -- Jack Krupansky -Original Message- From: Ethan Sent: Thursday, June 5, 2014 12:52 PM To: solr-user Subject: Re: Multivalue wild card search

Re: Mapping a field name before queryParser

2014-06-06 Thread Ahmet Arslan
Field aliasing exists when retrieving fields via fl parameter. But not when querying. Can you elaborate more your use case? On Saturday, June 7, 2014 12:57 AM, Antoine LE FLOC'H lefl...@gmail.com wrote: Hello, I have a query like the following where brand is a field in my schema:

Re: Mapping a field name before queryParser

2014-06-06 Thread Jack Krupansky
Edismax has field aliasing: http://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2F_renaming f.my_alias.qf=actual_field f.brand.qf=brand_name -- Jack Krupansky -Original Message- From: Antoine LE FLOC'H Sent: Friday, June 6, 2014 5:56 PM To: solr-user@lucene.apache.org

Re: Mapping a field name before queryParser

2014-06-06 Thread Antoine LE FLOC'H
OK thank you. I wanted to see what's possible if we want to change our Solr query interface. It is used by client applications. I wanted to see if we can be backward compatible. On Sat, Jun 7, 2014 at 12:10 AM, Jack Krupansky j...@basetechnology.com wrote: Edismax has field aliasing:

Re: Mapping a field name before queryParser

2014-06-06 Thread Jack Krupansky
Oops... for your example it would be: f.brand_name.qf=brand -- Jack Krupansky -Original Message- From: Antoine LE FLOC'H Sent: Friday, June 6, 2014 6:54 PM To: solr-user@lucene.apache.org Subject: Re: Mapping a field name before queryParser OK thank you. I wanted to see what's

Re: how to apply multiplcative Boost in multivalued field

2014-06-06 Thread Chris Hostetter
You need to explain what you mean here -- you're using hte term multiplicitive boost but the context in which you are asking the question suggests that isn't really applicable. what does that expression mean to you? Or to step back and rephrase: What is your usecase? What does your data look

Re: Solr Scale Toolkit Access Denied Error

2014-06-06 Thread Chris Hostetter
: My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not : accessible by my AWS credentials. Is this an AMI permissioning issue or is : it a problem with my particular account or how it is configured at AWS. I : did not experience this specific problem when working with the

Re: Solr Scale Toolkit Access Denied Error

2014-06-06 Thread Mark Gershman
Thanks, Hoss. I did substitute the previous AMI ID from the mid-May release of the toolkit and the build process does proceed further; however, it appears the the AMI changed enough that it is not compatible with the new toolkit release. In doing a little more research, I'm inclined to believe

Re: wildcard matches in EnumField - what do I need to change in code to enable wildcard matches?

2014-06-06 Thread Chris Hostetter
What you are asking for is comparable to saying: In my index I have a TrieIntField called severity when i search for severity:8765432 I get results, but when i search for severity:8* i get no results -- what do i need to change so that this type of query matches 8765432 and 8978 and 84356

Error when using URLDataSource to index RSS items

2014-06-06 Thread ienjreny
Hello, I am using the following script to index RSS items dataSource type=URLDataSource encoding=UTF-8 / document entity name=slashdot pk=link url=http://www.alarabiya.net/.mrss/ar.xml; processor=XPathEntityProcessor forEach=/rss/channel/item