Re: Any way to view lucene files

[Aman Tandon]

My solr version is 4.8.1 and luke is 3.5

With Regards
Aman Tandon


On Sat, Jun 7, 2014 at 12:21 PM, Chris Collins  wrote:

> What version of Solr / Lucene are you using?  You have to match the Luke
> version to the same version of Lucene.
>
> C
> On Jun 6, 2014, at 11:42 PM, Aman Tandon  wrote:
>
> > Yes  tried, but it not working at all every time i choose my index
> > directory it shows me EOF past
> >
> > With Regards
> > Aman Tandon
> >
> >
> > On Sat, Jun 7, 2014 at 12:01 PM, Chris Collins 
> wrote:
> >
> >> Have you tried:
> >>
> >> https://code.google.com/p/luke/
> >>
> >> Best
> >>
> >> Chris
> >> On Jun 6, 2014, at 11:24 PM, Aman Tandon 
> wrote:
> >>
> >>> Hi,
> >>>
> >>> Is there any way so that i can view what information and which is there
> >> in
> >>> my _e.fnm, etc files. may be with the help of any application or any
> >> viewer
> >>> tool.
> >>>
> >>> With Regards
> >>> Aman Tandon
> >>
> >>
>
>

Re: Any way to view lucene files

[Chris Collins]

What version of Solr / Lucene are you using?  You have to match the Luke 
version to the same version of Lucene.  

C
On Jun 6, 2014, at 11:42 PM, Aman Tandon  wrote:

> Yes  tried, but it not working at all every time i choose my index
> directory it shows me EOF past
> 
> With Regards
> Aman Tandon
> 
> 
> On Sat, Jun 7, 2014 at 12:01 PM, Chris Collins  wrote:
> 
>> Have you tried:
>> 
>> https://code.google.com/p/luke/
>> 
>> Best
>> 
>> Chris
>> On Jun 6, 2014, at 11:24 PM, Aman Tandon  wrote:
>> 
>>> Hi,
>>> 
>>> Is there any way so that i can view what information and which is there
>> in
>>> my _e.fnm, etc files. may be with the help of any application or any
>> viewer
>>> tool.
>>> 
>>> With Regards
>>> Aman Tandon
>> 
>> 

Re: Any way to view lucene files

[Aman Tandon]

Yes  tried, but it not working at all every time i choose my index
directory it shows me EOF past

With Regards
Aman Tandon


On Sat, Jun 7, 2014 at 12:01 PM, Chris Collins  wrote:

> Have you tried:
>
> https://code.google.com/p/luke/
>
> Best
>
> Chris
> On Jun 6, 2014, at 11:24 PM, Aman Tandon  wrote:
>
> > Hi,
> >
> > Is there any way so that i can view what information and which is there
> in
> > my _e.fnm, etc files. may be with the help of any application or any
> viewer
> > tool.
> >
> > With Regards
> > Aman Tandon
>
>

Re: Any way to view lucene files

[Chris Collins]

Have you tried:

https://code.google.com/p/luke/

Best

Chris
On Jun 6, 2014, at 11:24 PM, Aman Tandon  wrote:

> Hi,
> 
> Is there any way so that i can view what information and which is there in
> my _e.fnm, etc files. may be with the help of any application or any viewer
> tool.
> 
> With Regards
> Aman Tandon

Any way to view lucene files

[Aman Tandon]

Hi,

Is there any way so that i can view what information and which is there in
my _e.fnm, etc files. may be with the help of any application or any viewer
tool.

With Regards
Aman Tandon

Re: Analysis browser not working in solr 4.8.1

[Aman Tandon]

Okay i opened the issue.

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 9:43 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> Yes, please do open an issue. I'll fix it there.
>
>
> On Fri, Jun 6, 2014 at 9:33 PM, Aman Tandon 
> wrote:
>
> > Okay thanks shalin, i am fine with explicit till 4.9  :)
> >
> > So i don't need to open the issue right?
> >
> > With Regards
> > Aman Tandon
> >
> >
> > On Fri, Jun 6, 2014 at 9:20 PM, Shalin Shekhar Mangar <
> > shalinman...@gmail.com> wrote:
> >
> > > I think I created this problem in SOLR-6023.
> > >
> > > I went with the javadoc documentation which says that analysis.value
> is a
> > > required parameter and added that check. It seems that the either
> > > analysis.value or analysis.q (or q) can be specified to make it work.
> > I'll
> > > fix in 4.9
> > >
> > >
> > > On Fri, Jun 6, 2014 at 8:37 PM, Aman Tandon 
> > > wrote:
> > >
> > > > It is not working with either custom filter/field  as well as with
> > normal
> > > > text field, it only working fine if i pass the value/text in the both
> > in
> > > > the index time input as well as in query time input.
> > > > Only query time analysis is impossible unless i wrote that extra
> > > parameter
> > > > (analysis.fieldvalue=jute) explicitly in request(url).
> > > >
> > > > With Regards
> > > > Aman Tandon
> > > >
> > > >
> > > > On Fri, Jun 6, 2014 at 7:11 PM, Stefan Matheis <
> > matheis.ste...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I’m not sure that’s a bug in the UI .. in case the underlying
> service
> > > is
> > > > > barking with an exception we can’t do anything else than showing to
> > > you.
> > > > >
> > > > > are you sure the custom filter works as expected? like, verified
> > with a
> > > > > unit-test or something along the lines? i can still work with the
> > > > examples
> > > > > provided in the tutorial, so in general .. it works, looks like the
> > > only
> > > > > thing that doesn’t work is related to your custom components
> > > > >
> > > > > -Stefan
> > > > >
> > > > >
> > > > > On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I created a custom filter for my field named text_reversed, i
> tried
> > > my
> > > > > > custom filter in solr 4.7.1 and i was able to analyse the result,
> > it
> > > > > works
> > > > > > fine but in solr 4.8.1 it gaves me error of : *Missing required
> > > > > parameter:
> > > > > > analysis.fieldvalue. *It is also not working with any field*,
> *here
> > > is
> > > > > the
> > > > > > logs of the error
> > > > > >
> > > > > > 2090419 [http-bio-8984-exec-8] ERROR
> org.apache.solr.core.SolrCore
> > –
> > > > > > org.apache.solr.common.SolrException: Missing required parameter:
> > > > > > analysis.fieldvalue
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> > > > > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.catalina.valve

Error when using URLDataSource to index RSS items

[ienjreny]

Hello,

I am using the following script to index RSS items


  
http://www.alarabiya.net/.mrss/ar.xml";
processor="XPathEntityProcessor"
forEach="/rss/channel/item">

  
  


  

But I am facing the following error

Caused by: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag
; expected .

Can any body help?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Error-when-using-URLDataSource-to-index-RSS-items-tp4140548.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: wildcard matches in EnumField - what do I need to change in code to enable wildcard matches?

[Chris Hostetter]

What you are asking for is comparable to saying:

In my index I have a TrieIntField called severity when i search for 
severity:8765432 I get results, but when i search for severity:8* i get no 
results -- what do i need to change so that this type of query matches 
8765432 and 8978 and 84356 etc...


The bottom line is, but using the EnumField (or the TrieIntField, etc...) 
you are taking advantage of features to make specific usecases 
fast/efficient -- if those efficiencies don't server your usecase, you 
need to pick a differnet field type.  example: just use a String field.



: Date: Thu, 29 May 2014 06:50:32 +
: From: Elran Dvir 
: Reply-To: solr-user@lucene.apache.org
: To: "solr-user@lucene.apache.org" 
: Subject: wildcard matches in EnumField - what do I need to change in code to
: enable wildcard matches?
: 
: Hi all,
: 
: In my index, I have an EnumField called severity. This is its configuration 
in enumsConfig.xml:
: 
:  
:   Not Available 
:   Low
:Medium
:High
:Critical
:  
: 
: My index contains documents with these values.
: When I search for severity:High, I get results. But when I search for 
severity:H* , I get no results.
: What do  I need to change in Solr code to enable wildcard matches in 
EnumField  (or any other field)?
: 
: Thanks.
: 

-Hoss
http://www.lucidworks.com/

Re: Solr Scale Toolkit Access Denied Error

[Mark Gershman]

Thanks, Hoss.

I did substitute the previous AMI ID from the mid-May release of the
toolkit and the build process does proceed further; however, it appears the
the AMI changed enough that it is not compatible with the new toolkit
release.  In doing a little more research, I'm inclined to believe that the
permissions on the AMI may be the source of the problem and will post to
the issue tracker per your suggestion.


Mark Gershman


On Fri, Jun 6, 2014 at 7:41 PM, Chris Hostetter 
wrote:

>
> : My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not
> : accessible by my AWS credentials.  Is this an AMI permissioning issue or
> is
> : it a problem with my particular account or how it is configured at AWS.
>  I
> : did not experience this specific problem when working with the previous
> : iteration of the Solr Scale Toolkit back toward the latter part of May.
>  It
> : appears that the AMI was updated from ami-96779efe to ami-1e6b9d76 with
> the
> : newest version of the toolkit.
>
> I'm not much of an AWS expert, but i seem to recall that if you don't
> have your AWS security group setup properly this type of error can
> happen? is it possible that when you were trying out solr-scale-tk before
> you had this setup, but now you don't?
>
> https://github.com/LucidWorks/solr-scale-tk
>
> > You'll need to setup a security group named solr-scale-tk (or update the
> > fabfile.py to change the name).
> >
> > At a minimum you should allow TCP traffic to ports: 8983, 8984-8989,
> > SSH, and 2181 (ZooKeeper). However, it is your responsibility to review
> > the security configuration of your cluster and lock it down
> appropriately.
> >
> > You'll also need to create an keypair (using the Amazon console) named
> > solr-scale-tk (you can rename the key used by the framework, see:
> > AWS_KEY_NAME). After downloading the keypair file (solr-scale-tk.pem),
> > save it to ~/.ssh/ and change permissions: chmod 600
> > ~/.ssh/solr-scale-tk.pem
>
> ...if I'm wrong, and there really is a problem with the security on the
> AMI, the best place to report that would be in the project's issue
> tracker...
>
> https://github.com/LucidWorks/solr-scale-tk/issues
>
>
>
> -Hoss
> http://www.lucidworks.com/
>

Re: Solr Scale Toolkit Access Denied Error

[Chris Hostetter]

: My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not
: accessible by my AWS credentials.  Is this an AMI permissioning issue or is
: it a problem with my particular account or how it is configured at AWS.  I
: did not experience this specific problem when working with the previous
: iteration of the Solr Scale Toolkit back toward the latter part of May.  It
: appears that the AMI was updated from ami-96779efe to ami-1e6b9d76 with the
: newest version of the toolkit.

I'm not much of an AWS expert, but i seem to recall that if you don't 
have your AWS security group setup properly this type of error can 
happen? is it possible that when you were trying out solr-scale-tk before 
you had this setup, but now you don't?

https://github.com/LucidWorks/solr-scale-tk

> You'll need to setup a security group named solr-scale-tk (or update the 
> fabfile.py to change the name).
> 
> At a minimum you should allow TCP traffic to ports: 8983, 8984-8989, 
> SSH, and 2181 (ZooKeeper). However, it is your responsibility to review 
> the security configuration of your cluster and lock it down appropriately.
> 
> You'll also need to create an keypair (using the Amazon console) named 
> solr-scale-tk (you can rename the key used by the framework, see: 
> AWS_KEY_NAME). After downloading the keypair file (solr-scale-tk.pem), 
> save it to ~/.ssh/ and change permissions: chmod 600 
> ~/.ssh/solr-scale-tk.pem

...if I'm wrong, and there really is a problem with the security on the 
AMI, the best place to report that would be in the project's issue 
tracker...

https://github.com/LucidWorks/solr-scale-tk/issues



-Hoss
http://www.lucidworks.com/

Re: how to apply multiplcative Boost in multivalued field

[Chris Hostetter]

You need to explain what you mean here -- you're using hte term 
"multiplicitive boost" but the context in which you are asking the 
question suggests that isn't really applicable.

what does that expression mean to you?

Or to step back and rephrase: What is your usecase? What does your data 
look like and what types of results do you wnat ot get?  

https://wiki.apache.org/solr/UsingMailingLists

https://people.apache.org/~hossman/#xyproblem
XY Problem

Your question appears to be an "XY Problem" ... that is: you are dealing
with "X", you are assuming "Y" will help you, and you are asking about "Y"
without giving more details about the "X" so that we can understand the
full issue.  Perhaps the best solution doesn't involve "Y" at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341





: Date: Mon, 26 May 2014 17:13:10 +0530
: From: Aman Tandon 
: Reply-To: solr-user@lucene.apache.org
: To: "solr-user@lucene.apache.org" 
: Subject: how to apply multiplcative Boost in multivalued field
: 
: HI,
: 
: I am confused to how to apply the multiplicative boost on multivalued field.
: 
: 
: 
: 
: Suppose in plid the value goes like 111,1234,2345,4567,2335,9876,67
: 
: I am applying the filters on the plid like *..&fq=plid:(111 1234 2345 4567
: 2335 9876 67)*
: 
: Now i need to apply the boost on the first three plid as well, which is a
: multivalued field, so help me out here.
: 
: With Regards
: Aman Tandon
: 

-Hoss
http://www.lucidworks.com/

Re: Mapping a field name before queryParser

[Jack Krupansky]

Oops... for your example it would be:

f.brand_name.qf=brand

-- Jack Krupansky

-Original Message- 
From: Antoine LE FLOC'H

Sent: Friday, June 6, 2014 6:54 PM
To: solr-user@lucene.apache.org
Subject: Re: Mapping a field name before queryParser

OK thank you.
I wanted to see what's possible if we want to change our Solr query
interface. It is used by client applications. I wanted to see if we can be
backward compatible.



On Sat, Jun 7, 2014 at 12:10 AM, Jack Krupansky 
wrote:


Edismax has field aliasing:
http://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2F_renaming

f.my_alias.qf=actual_field

f.brand.qf=brand_name

-- Jack Krupansky

-Original Message- From: Antoine LE FLOC'H Sent: Friday, June 6,
2014 5:56 PM To: solr-user@lucene.apache.org Subject: Mapping a field
name before queryParser
Hello,

I have a query like the following where "brand" is a field in my schema:

select?rows=1&start=0&sort=price+asc&q=brand:sony&qt=for-search&wt=xml


But I want to do this instead:

select?rows=1&start=0&sort=price+asc&q=brand_name:sony&
qt=for-search&wt=xml

and define something like "brand_name:brand" in my Solr config to change
the field before or during the QueryParsing. Is there a way to do that ?


Ideally I would not want to do a copyField since it would grow my index 
and

would require re-indexing.


Thank you

Re: Mapping a field name before queryParser

[Antoine LE FLOC'H]

OK thank you.
I wanted to see what's possible if we want to change our Solr query
interface. It is used by client applications. I wanted to see if we can be
backward compatible.



On Sat, Jun 7, 2014 at 12:10 AM, Jack Krupansky 
wrote:

> Edismax has field aliasing:
> http://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2F_renaming
>
> f.my_alias.qf=actual_field
>
> f.brand.qf=brand_name
>
> -- Jack Krupansky
>
> -Original Message- From: Antoine LE FLOC'H Sent: Friday, June 6,
> 2014 5:56 PM To: solr-user@lucene.apache.org Subject: Mapping a field
> name before queryParser
> Hello,
>
> I have a query like the following where "brand" is a field in my schema:
>
> select?rows=1&start=0&sort=price+asc&q=brand:sony&qt=for-search&wt=xml
>
>
> But I want to do this instead:
>
> select?rows=1&start=0&sort=price+asc&q=brand_name:sony&
> qt=for-search&wt=xml
>
> and define something like "brand_name:brand" in my Solr config to change
> the field before or during the QueryParsing. Is there a way to do that ?
>
>
> Ideally I would not want to do a copyField since it would grow my index and
> would require re-indexing.
>
>
> Thank you
>

Re: Mapping a field name before queryParser

[Jack Krupansky]

Edismax has field aliasing:
http://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2F_renaming

f.my_alias.qf=actual_field

f.brand.qf=brand_name

-- Jack Krupansky

-Original Message- 
From: Antoine LE FLOC'H 
Sent: Friday, June 6, 2014 5:56 PM 
To: solr-user@lucene.apache.org 
Subject: Mapping a field name before queryParser 


Hello,

I have a query like the following where "brand" is a field in my schema:

select?rows=1&start=0&sort=price+asc&q=brand:sony&qt=for-search&wt=xml


But I want to do this instead:

select?rows=1&start=0&sort=price+asc&q=brand_name:sony&qt=for-search&wt=xml

and define something like "brand_name:brand" in my Solr config to change
the field before or during the QueryParsing. Is there a way to do that ?


Ideally I would not want to do a copyField since it would grow my index and
would require re-indexing.


Thank you

Re: Mapping a field name before queryParser

[Ahmet Arslan]

Field aliasing exists when retrieving fields via fl parameter. But not when 
querying.
Can you elaborate more your use case?





On Saturday, June 7, 2014 12:57 AM, Antoine LE FLOC'H  wrote:
Hello,

I have a query like the following where "brand" is a field in my schema:

select?rows=1&start=0&sort=price+asc&q=brand:sony&qt=for-search&wt=xml


But I want to do this instead:

select?rows=1&start=0&sort=price+asc&q=brand_name:sony&qt=for-search&wt=xml

and define something like "brand_name:brand" in my Solr config to change
the field before or during the QueryParsing. Is there a way to do that ?


Ideally I would not want to do a copyField since it would grow my index and
would require re-indexing.


Thank you

Re: Multivalue wild card search

[Jack Krupansky]

Wildcard, fuzzy, and regex query do work on multivalued fields, but on only 
a single value at a time. Your match cannot occur across values.


-- Jack Krupansky

-Original Message- 
From: Ethan

Sent: Thursday, June 5, 2014 12:52 PM
To: solr-user
Subject: Re: Multivalue wild card search

Wildcard search do work on multiValued field.  I was able to pull up
records for following multiValued field -

Code : [
"12344",
"4534",
"674"
]

q=Code:45* fetched the correct document.  It doesn't work in
quotes(q="Code:45*"), however.  Is there a workaround?


On Thu, Jun 5, 2014 at 9:34 AM, Ethan  wrote:


Are you implying there is not way to lookup on a multiValued field with a
substring?  If so, then how is it usually handled?


On Wed, Jun 4, 2014 at 4:44 PM, Jack Krupansky 
wrote:


Wildcard, fuzzy, and regex query operate on a single term of a single
tokenized field value or a single string field value.

-- Jack Krupansky

-Original Message- From: Ethan
Sent: Wednesday, June 4, 2014 6:59 PM
To: solr-user
Subject: Multivalue wild card search


I can't seem to find a solution to do wild card search on a multiValued
field.

For Eg consider a multiValued field called "Name" with 3 values -

"Name" : [
"[[\"Ethan\", \"G\", \"\"],[\"Steve\", \"Wonder\", \"\"]]",
"[]",
"[[\"hifte\", \"Grop\", \"\"]]"
]

For a multiValued like above, I want search like-

q="***[\"Steve\", \"Wonder\", \"\"]"


But I do not get back any results back. Any ideas on to create such
query?

Mapping a field name before queryParser

[Antoine LE FLOC'H]

Hello,

I have a query like the following where "brand" is a field in my schema:

select?rows=1&start=0&sort=price+asc&q=brand:sony&qt=for-search&wt=xml


But I want to do this instead:

select?rows=1&start=0&sort=price+asc&q=brand_name:sony&qt=for-search&wt=xml

and define something like "brand_name:brand" in my Solr config to change
the field before or during the QueryParsing. Is there a way to do that ?


Ideally I would not want to do a copyField since it would grow my index and
would require re-indexing.


Thank you

RE: SolrCloud: facet range option f..facet.mincount=1 omits buckets on response

[Chris Hostetter]

Ronald: I'm having a little trouble understading the  steps o reproduce 
that you are describing -- in particular Step "1 f ii" because i'm not 
really sure i understand what exactly you are putting in "mem2.xml"

Also: Since you don't appera to be using implicit routing, i'm not clear 
on why you are explicitly sending differnet updates to different ports in 
Step "1 f i" -- does that affect the results of your test?


If you can reliably reproduce using modified data from the example, could 
you please open a Jira outline these steps and atached the modified data 
to index directly to that issue?  (FWIW: If it doesn't matter what port 
you use to send which documents, then you should be able to create a single 
unified "data.xml" file containing all the docs to index in a single 
command)



: Date: Thu, 29 May 2014 18:06:38 +
: From: Ronald Matamoros 
: Reply-To: solr-user@lucene.apache.org
: To: "solr-user@lucene.apache.org" 
: Subject: RE: SolrCloud: facet range option f..facet.mincount=1 omits
: buckets on response
: 
: Hi all,
: 
: At the moment I am reviewing the code to determine if this is a legitimate 
bug that needs to be set as a JIRA ticket.
: Any insight or recommendation is appreciated.
: 
: Including the replication steps as text:
: 
: -
: Solr versions where issue was replicated.
:   * 4.5.1 (Linux)
:   * 4.8.1 (Windows + Cygwin)
: 
: Replicating
: 
:   1. Created two-shard environment - no replication 
:  
https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud
: 
:  a. Download Solr distribution from 
http://lucene.apache.org/solr/downloads.html 
:  b. Unzipped solr-4.8.1.zip to a temporary location:  
:  c. Ran once so the SolrCloud jars get unpacked: java -jar start.jar
:  d. Create nodes
:   i. cd 
:   ii. Via Windows Explorer copied example to node1
:   iii. Via Windows Explorer copied example to node2
: 
:  e. Start Nodes 
:   i. Start node 1
: 
:cd node1
:java -DzkRun -DnumShards=2 
-Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -jar 
start.jar
: 
:   ii. Start node 2
: 
:cd node2
:java -Djetty.port=7574 -DzkHost=localhost:9983 -jar start.jar
: 
:  f. Fed sample documents
:   i. Out of the box
: 
:curl http://localhost:8983/solr/update?commit=true -H 
"Content-Type: text/xml" -d "@mem.xml"
:curl http://localhost:7574/solr/update?commit=true -H 
"Content-Type: text/xml" -d "@monitor2.xml"
: 
:   ii. Create a copy of mem.xml to mem2.xml; modified identifiers, 
names, prices and fed
: 
:curl http://localhost:8983/solr/update?commit=true -H 
"Content-Type: text/xml" -d "@mem2.xml"
: 
:
:  
:COMPANY1
:COMPANY1 Device
:COMPANY1 Device Mfg
:.
:190
:.
:  
:  
:COMPANY2
:COMPANY2 flatscreen
:COMPANY2 Device Mfg.
:.
:200.00
:.
:  
:  
:COMPANY3
:COMPANY3 Laptop
:COMPANY3 Device Mfg.
:.
:800.00
:.
:  
:  
:  
: 
:   2. Query **without** f.price.facet.mincount=1, counts and buckets are OK
: 
:  
http://localhost:8983/solr/collection1/select?q=*:*&fl=id,price&sort=id+asc&facet=true&facet.range=price&f.price.facet.range.start=0&f.price.facet.range.end=1000&f.price.facet.range.gap=50&f.price.facet.range.other=all&f.price.facet.range.include=upper&spellcheck=false&hl=false
:  
:  Only six documents have prices
:  
:   
: 
:   
: 0
: 1
: 0
: 3
: 0
: 1
: 0
: 0
: 0
: 0
: 0
: 0
: 0
: 0
: 0
: 1
: 0
: 0
: 0
: 0
:   
:   50.0
:   0.0
:   1000.0
:   0
:   0
:   2
: 
:   
: 
:   Note: the value in  changes with every other 
refresh of the query. 
: 
:   3.Use of &f.price.facet.mincount=1, missing bucket  1
: 
:  
http://localhost:8983/solr/collection1/select?q=*:*&fl=id,price&sort=id+asc&facet=true&facet.range=price&f.price.facet.range.start=0&f.price.facet.range.end=1000&f.price.facet.range.gap=50&f.price.facet.range.other=

Solr Realtime Get RemoteSolrException: Expected mime type application/xml but got text/html

[Songtao Zheng]

Solr version on remote server: solr-4.3.1 I am trying to use Solr Realtime
Get  to retrieve document before
commit. My code

class Test3
{
  static main(args)
  {
def test = new Test3()
test.run()
  }

  private run()
  {
String url = "DEV_SERVER:8983/solr/emr"

HttpSolrServer solr = new HttpSolrServer(url)

SolrQuery q = new SolrQuery();
q.setRequestHandler("/get");
q.set("rid",
"6662c0f2.ee6a64fe.588j6qohe.9kd087u.0r00dg.6kr5pc2as0qu9m4ibr7f7");

QueryRequest req = new QueryRequest(q);
req.setResponseParser(new BinraryResponseParser());

println "=="
rsp = req.process(solr);// ERROR
  }
}

*The error stacktrace is:*
Caught:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Expected mime type application/octet-stream but got
text/html. 


Error 404 Not Found

HTTP ERROR 404
Problem accessing /solr/emr/get. Reason:
Not FoundPowered by
Jetty://























org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Expected mime type application/octet-stream but got text/htm
l. 


Error 404 Not Found

HTTP ERROR 404
Problem accessing /solr/emr/get. Reason:
Not FoundPowered by
Jetty://























at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:459)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at
org.apache.solr.client.solrj.request.QueryRequest$process.call(Unknown
Source)
at com.att.songtao.test.Test3.run(Test3.groovy:48)
at com.att.songtao.test.Test3.this$2$run(Test3.groovy)
at com.att.songtao.test.Test3$this$2$run.call(Unknown Source)
at com.att.songtao.test.Test3.main(Test3.groovy:14)


I am following Realtime Get document and added updateLog to updateHandler
in solrconfig.xml. My localhost "localhost:8983/solr/emr" (version
solr-4.7.2) Realtime Get works perfect, but having it on remote server
throws out the above error.

Anyone could provide the insight?

Thanks,

Songtao

Solr and Kerberos

[Tom Chen]

Hi,

I wonder how to secure Solr with Kerberos.

We can Kerberos secure Solr by configuring the AuthenticationFilter from
the hadoop-auth.jar that is packaged in solr.war.

But after we do that,

1) How does a SolrJ client connect to the secured Solr server?
2) In SolrCloud environment, how one Solr node connect to other secured
Solr node?

Regards

Re: Strange Behavior with Solr in Tomcat.

[Meraj A. Khan]

This looks distinctly related to
https://issues.apache.org/jira/browse/SOLR-4408 , try coldSearcher = true
as being suggested in JIRA and let us know .


On Fri, Jun 6, 2014 at 2:39 PM, Jean-Sebastien Vachon <
jean-sebastien.vac...@wantedanalytics.com> wrote:

> I would try a thread dump and check the output to see what`s going on.
> You could also strace the process if you`re running on Unix or changed the
> log level in Solr to get more information logged
>
> > -Original Message-
> > From: S.L [mailto:simpleliving...@gmail.com]
> > Sent: June-06-14 2:33 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Strange Behavior with Solr in Tomcat.
> >
> > Anyone folks?
> >
> >
> > On Wed, Jun 4, 2014 at 10:25 AM, S.L  wrote:
> >
> > >  Hi Folks,
> > >
> > > I recently started using the spellchecker in my solrconfig.xml. I am
> > > able to build up an index in Solr.
> > >
> > > But,if I ever shutdown tomcat I am not able to restart it.The server
> > > never spits out the server startup time in seconds in the logs,nor
> > > does it print any error messages in the catalina.out file.
> > >
> > > The only way for me to get around this is by delete the data directory
> > > of the index and then start the server,obviously this makes me loose my
> > index.
> > >
> > > Just wondering if anyone faced a similar issue and if they were able
> > > to solve this.
> > >
> > > Thanks.
> > >
> > >
> >
> > -
> > Aucun virus trouvé dans ce message.
> > Analyse effectuée par AVG - www.avg.fr
> > Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date:
> > 27/05/2014 La Base de données des virus a expiré.
>

RE: Strange Behavior with Solr in Tomcat.

[Jean-Sebastien Vachon]

I would try a thread dump and check the output to see what`s going on. 
You could also strace the process if you`re running on Unix or changed the log 
level in Solr to get more information logged

> -Original Message-
> From: S.L [mailto:simpleliving...@gmail.com]
> Sent: June-06-14 2:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Strange Behavior with Solr in Tomcat.
> 
> Anyone folks?
> 
> 
> On Wed, Jun 4, 2014 at 10:25 AM, S.L  wrote:
> 
> >  Hi Folks,
> >
> > I recently started using the spellchecker in my solrconfig.xml. I am
> > able to build up an index in Solr.
> >
> > But,if I ever shutdown tomcat I am not able to restart it.The server
> > never spits out the server startup time in seconds in the logs,nor
> > does it print any error messages in the catalina.out file.
> >
> > The only way for me to get around this is by delete the data directory
> > of the index and then start the server,obviously this makes me loose my
> index.
> >
> > Just wondering if anyone faced a similar issue and if they were able
> > to solve this.
> >
> > Thanks.
> >
> >
> 
> -
> Aucun virus trouvé dans ce message.
> Analyse effectuée par AVG - www.avg.fr
> Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date:
> 27/05/2014 La Base de données des virus a expiré.

Re: Strange Behavior with Solr in Tomcat.

[S.L]

Anyone folks?


On Wed, Jun 4, 2014 at 10:25 AM, S.L  wrote:

>  Hi Folks,
>
> I recently started using the spellchecker in my solrconfig.xml. I am able
> to build up an index in Solr.
>
> But,if I ever shutdown tomcat I am not able to restart it.The server never
> spits out the server startup time in seconds in the logs,nor does it print
> any error messages in the catalina.out file.
>
> The only way for me to get around this is by delete the data directory of
> the index and then start the server,obviously this makes me loose my index.
>
> Just wondering if anyone faced a similar issue and if they were able to
> solve this.
>
> Thanks.
>
>

Re: Delta Import Functionality

[Ahmet Arslan]

Hi,

I think you need to select * in deltaImportQuery. You are just selecting one 
field for both delta*Query SQL sentences.



On Thursday, June 5, 2014 3:34 PM, ajay59  wrote:
Hi,

We are using the SOLR 4.6 version and trying to implement Delta import
functionality .On implementing the delta import , the indexing is gettting
completed but not fetching the updated or newly inserted rows from the table
while executing the query.

Please suggest.

Below is the data config.xml:


    
       
                      
          
                        
               
            
        
            

                    
            

    
        
          





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Delta-Import-Functionality-tp4140063.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

[Joel Bernstein]

With tag and exclude you can get the facets counts for the collapsed set
and expanded set in the same query. But group.facets is a different count
then either of those.

group.facets counts each facet bucket once per group.

Joel Bernstein
Search Engineer at Heliosearch


On Fri, Jun 6, 2014 at 1:53 PM, david.w.smi...@gmail.com <
david.w.smi...@gmail.com> wrote:

> I may be misunderstanding the problem, but if it’s what I think it is, then
> users can work-around this now quite easily by using Solr faceting’s
> ability to exclude a named/tagged filter query:
>
> &q=classIDs:12
> &fl=PrSKU
> &fq={!collapse tag=collapse field=PrSKU}
> &facet=true
> &facet.field={! ex=collapse}at_12_wood_tone
> &fq=at_12_wood_tone:”Light Wood”
>
>
> ~ David Smiley
> Freelance Apache Lucene/Solr Search Consultant/Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Jun 6, 2014 at 1:09 PM, Joel Bernstein  wrote:
>
> > The CollapsingQParserPlugin should give you the same facet counts as
> > group.truncate.
> >
> > You're using group.facets, which the CollapsingQParserplugin doesn't yet
> > support. I think this would be an excellent feature, so we could make a
> > jira ticket to add this feature.
> >
> > Joel Bernstein
> > Search Engineer at Heliosearch
> >
> >
> > On Fri, Jun 6, 2014 at 1:07 PM, Joel Bernstein 
> wrote:
> >
> > > Reposting this from jira ticket to users list:
> > >
> > > I'm noticing a very weird bug using the CollapsingQParserPlugin. We
> tried
> > > to use this plugin when we realized that faceting on the groups would
> > take
> > > a ridiculous amount of time. To its credit, it works very quickly,
> > however
> > > the facet counts that it gives are incorrect.
> > >
> > > We have a smallish index of about 200k documents with about with about
> > 50k
> > > distinct groups within it.
> > >
> > > When we use the group implementation
> > > (&group=true&group.field=PrSKU&group.facet=true) which I believe this
> > > attempts to emulate, the facet counts are totally correct.
> > >
> > > When we use the field collapsing implementation, it will show an
> > incorrect
> > > count for the non-filtered query, but when we go to the filtered query,
> > the
> > > facet count corrects itself and matches the document count.
> > >
> > > Here are some SOLR responses:
> > >
> > > solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq=
> > > {!collapse%20field=PrSKU}&facet=true&facet.field=at_12_wood_tone
> > >
> > > The facet field will return
> > >
> > > 867
> > > 441
> > > 253
> > >
> > > When I actually apply a filter query like so:
> > >
> > >
> > >
> >
> solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq={!collapse%20field=PrSKU}
> > >
> > >
> > >
> >
> &facet=true&facet.field=at_12_wood_tone&fq=at_12_wood_tone:%22Light%20Wood%22
> > >
> > > I actually pull back 270 results and the facet updates itself with the
> > > correct number at the bottom
> > >
> > > 270
> > > 68
> > > 66
> > >
> > > If this were the same number pre and post filter query I would assume
> > that
> > > it was simply my data that was bad, however I've pored over this for
> the
> > > better part of a day and I'm pretty sure it's the plugin. For
> reference,
> > > this field that I'm faceting on is a multiValued field, however I have
> > > noticed the exact same behavior on non multiValued fields (such as
> > price).
> > >
> > > I can provide any other details you might need
> > >
> >
>

Re: Multivalue wild card search

[Ahmet Arslan]

Hi Ethan,


It is hard to understand your example. Can you re-write it? Using xml? 



On Friday, June 6, 2014 9:07 PM, Ethan  wrote:
Bumping the thread to see if anyone has a solution.





On Thu, Jun 5, 2014 at 9:52 AM, Ethan  wrote:

> Wildcard search do work on multiValued field.  I was able to pull up
> records for following multiValued field -
>
> Code : [
> "12344",
> "4534",
> "674"
> ]
>
> q=Code:45* fetched the correct document.  It doesn't work in
> quotes(q="Code:45*"), however.  Is there a workaround?
>
>
> On Thu, Jun 5, 2014 at 9:34 AM, Ethan  wrote:
>
>> Are you implying there is not way to lookup on a multiValued field with a
>> substring?  If so, then how is it usually handled?
>>
>>
>> On Wed, Jun 4, 2014 at 4:44 PM, Jack Krupansky 
>> wrote:
>>
>>> Wildcard, fuzzy, and regex query operate on a single term of a single
>>> tokenized field value or a single string field value.
>>>
>>> -- Jack Krupansky
>>>
>>> -Original Message- From: Ethan
>>> Sent: Wednesday, June 4, 2014 6:59 PM
>>> To: solr-user
>>> Subject: Multivalue wild card search
>>>
>>>
>>> I can't seem to find a solution to do wild card search on a multiValued
>>> field.
>>>
>>> For Eg consider a multiValued field called "Name" with 3 values -
>>>
>>> "Name" : [
>>> "[[\"Ethan\", \"G\", \"\"],[\"Steve\", \"Wonder\", \"\"]]",
>>> "[]",
>>> "[[\"hifte\", \"Grop\", \"\"]]"
>>> ]
>>>
>>> For a multiValued like above, I want search like-
>>>
>>> q="***[\"Steve\", \"Wonder\", \"\"]"
>>>
>>>
>>> But I do not get back any results back. Any ideas on to create such
>>> query?
>>>
>>
>>
>

Re: Multivalue wild card search

[Ethan]

Bumping the thread to see if anyone has a solution.


On Thu, Jun 5, 2014 at 9:52 AM, Ethan  wrote:

> Wildcard search do work on multiValued field.  I was able to pull up
> records for following multiValued field -
>
> Code : [
> "12344",
> "4534",
> "674"
> ]
>
> q=Code:45* fetched the correct document.  It doesn't work in
> quotes(q="Code:45*"), however.  Is there a workaround?
>
>
> On Thu, Jun 5, 2014 at 9:34 AM, Ethan  wrote:
>
>> Are you implying there is not way to lookup on a multiValued field with a
>> substring?  If so, then how is it usually handled?
>>
>>
>> On Wed, Jun 4, 2014 at 4:44 PM, Jack Krupansky 
>> wrote:
>>
>>> Wildcard, fuzzy, and regex query operate on a single term of a single
>>> tokenized field value or a single string field value.
>>>
>>> -- Jack Krupansky
>>>
>>> -Original Message- From: Ethan
>>> Sent: Wednesday, June 4, 2014 6:59 PM
>>> To: solr-user
>>> Subject: Multivalue wild card search
>>>
>>>
>>> I can't seem to find a solution to do wild card search on a multiValued
>>> field.
>>>
>>> For Eg consider a multiValued field called "Name" with 3 values -
>>>
>>> "Name" : [
>>> "[[\"Ethan\", \"G\", \"\"],[\"Steve\", \"Wonder\", \"\"]]",
>>> "[]",
>>> "[[\"hifte\", \"Grop\", \"\"]]"
>>> ]
>>>
>>> For a multiValued like above, I want search like-
>>>
>>> q="***[\"Steve\", \"Wonder\", \"\"]"
>>>
>>>
>>> But I do not get back any results back. Any ideas on to create such
>>> query?
>>>
>>
>>
>

Re: span query with SHOUD semantic instead of MUST HAVE

[Ahmet Arslan]

Hi,

How come query "aa bb" returns document b?



On Friday, June 6, 2014 3:48 PM, 郑华斌  wrote:
hi,


I have two docs, 
    a) "aa bb cc" and,
    b) "aa cc bb". 
The query is "aa bb". What I expected is the doc a comes first with a higher 
score than doc b because the term distance in query and that in doc a are more 
similar. 
After google for a while I get it down with the span query q: "aa bb"~10. 
However, when I change my query into "aa bb dd"~10, the span query return 
nothing 
hits becuase dd can not be found in any doc. So what's a solution to this 
problem? 


Thanks.

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

[david.w.smi...@gmail.com]

I may be misunderstanding the problem, but if it’s what I think it is, then
users can work-around this now quite easily by using Solr faceting’s
ability to exclude a named/tagged filter query:

&q=classIDs:12
&fl=PrSKU
&fq={!collapse tag=collapse field=PrSKU}
&facet=true
&facet.field={! ex=collapse}at_12_wood_tone
&fq=at_12_wood_tone:”Light Wood”


~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Jun 6, 2014 at 1:09 PM, Joel Bernstein  wrote:

> The CollapsingQParserPlugin should give you the same facet counts as
> group.truncate.
>
> You're using group.facets, which the CollapsingQParserplugin doesn't yet
> support. I think this would be an excellent feature, so we could make a
> jira ticket to add this feature.
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Fri, Jun 6, 2014 at 1:07 PM, Joel Bernstein  wrote:
>
> > Reposting this from jira ticket to users list:
> >
> > I'm noticing a very weird bug using the CollapsingQParserPlugin. We tried
> > to use this plugin when we realized that faceting on the groups would
> take
> > a ridiculous amount of time. To its credit, it works very quickly,
> however
> > the facet counts that it gives are incorrect.
> >
> > We have a smallish index of about 200k documents with about with about
> 50k
> > distinct groups within it.
> >
> > When we use the group implementation
> > (&group=true&group.field=PrSKU&group.facet=true) which I believe this
> > attempts to emulate, the facet counts are totally correct.
> >
> > When we use the field collapsing implementation, it will show an
> incorrect
> > count for the non-filtered query, but when we go to the filtered query,
> the
> > facet count corrects itself and matches the document count.
> >
> > Here are some SOLR responses:
> >
> > solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq=
> > {!collapse%20field=PrSKU}&facet=true&facet.field=at_12_wood_tone
> >
> > The facet field will return
> >
> > 867
> > 441
> > 253
> >
> > When I actually apply a filter query like so:
> >
> >
> >
> solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq={!collapse%20field=PrSKU}
> >
> >
> >
> &facet=true&facet.field=at_12_wood_tone&fq=at_12_wood_tone:%22Light%20Wood%22
> >
> > I actually pull back 270 results and the facet updates itself with the
> > correct number at the bottom
> >
> > 270
> > 68
> > 66
> >
> > If this were the same number pre and post filter query I would assume
> that
> > it was simply my data that was bad, however I've pored over this for the
> > better part of a day and I'm pretty sure it's the plugin. For reference,
> > this field that I'm faceting on is a multiValued field, however I have
> > noticed the exact same behavior on non multiValued fields (such as
> price).
> >
> > I can provide any other details you might need
> >
>

Solr Scale Toolkit Access Denied Error

[Mark Gershman]

I've been attempting to experiment with the recently updated  Solr Scale
Tool Kit mentioned here:

http://searchhub.org/2014/06/03/introducing-the-solr-scale-toolkit/

After making the very well documented configuration changes at AWS and
installing Python, I was able to use the toolkit to connect to AWS;
however, when the demo command (fab demo:demo1,n=1) is executed, the
following access denied error message is returned:

C:\Temp\sk\solr-scale-tk>fab demo:demo1,n=1
←[33mGoing to launch 1 new EC2 m3.medium instances using AMI
ami-1e6b9d76←[0m
←[32mSetup Instance store BlockDeviceMapping: /dev/sdb -> ephemeral0←[0m
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\fabric\main.py", line 743, in main
*args, **kwargs
  File "c:\python27\lib\site-packages\fabric\tasks.py", line 405, in execute
results[''] = task.run(*args, **new_kwargs)
  File "c:\python27\lib\site-packages\fabric\tasks.py", line 171, in run
return self.wrapped(*args, **kwargs)
  File "C:\Temp\sk\solr-scale-tk\fabfile.py", line 1314, in demo
ec2hosts = new_ec2_instances(cluster=demoCluster, n=n,
instance_type=instance_type)
  File "C:\Temp\sk\solr-scale-tk\fabfile.py", line 776, in new_ec2_instances
placement_group=placement_group)
  File "c:\python27\lib\site-packages\boto\ec2\connection.py", line 943, in
run_instances
verb='POST')
  File "c:\python27\lib\site-packages\boto\connection.py", line 1177, in
get_object
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

AuthFailureNot authorized
for images:
[ami-1e6b9d76]cbdcb582-3d21-4fab-a7ea
-fe68ba696e23

My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not
accessible by my AWS credentials.  Is this an AMI permissioning issue or is
it a problem with my particular account or how it is configured at AWS.  I
did not experience this specific problem when working with the previous
iteration of the Solr Scale Toolkit back toward the latter part of May.  It
appears that the AMI was updated from ami-96779efe to ami-1e6b9d76 with the
newest version of the toolkit.

Many thanks to Timothy Potter to assembling this excellent toolkit for
deploying and managing SolrCloud at AWS (and hopefully other cloud
providers in the future).

Thanks.


Mark Gershman

Re: timeout when create alias

[Shalin Shekhar Mangar]

Yes, a collection API can timeout particularly when a long running
collection operation is running. The collection API tasks are executed
serially right now so new tasks get stuck behind an old one until it
completes. This is going to change in the next release (4.9) with SOLR-5681
which makes the task processing multi-threaded.


On Fri, Jun 6, 2014 at 9:38 PM, lansing 
wrote:

> Hi,
> We are running Solr cloud 4.8.0 on tomcat 7, java 7 update 51, with 5
> shards
> and 3 replicas.
>
> When I try to create alias via collection admin api
>
> http://10.0.5.227:8201/solr/admin/collections?action=CREATEALIAS&name=Current&collections=Current2
>
> It timeout after 180 s,
>
> 181344574 [http-bio-8201-exec-1237] INFO
> org.apache.solr.servlet.SolrDispatchFilter  – [admin] webapp=null path=/ad
> min/collections
> params={name=Current&action=CREATEALIAS&async=true&collections=Current2}
> status=500 QTime=180003
> 181344574 [http-bio-8201-exec-1237] ERROR
> org.apache.solr.servlet.SolrDispatchFilter  – null:org.apache.solr.common.
> SolrException: createalias the collection time out:180s
> at
>
> org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:357)
> at
>
> org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:309)
> at
>
> org.apache.solr.handler.admin.CollectionsHandler.handleCreateAliasAction(CollectionsHandler.java:416)
> at
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:163)
> at
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:267)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
>
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
>
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/timeout-when-create-alias-tp4140437.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

[Joel Bernstein]

The CollapsingQParserPlugin should give you the same facet counts as
group.truncate.

You're using group.facets, which the CollapsingQParserplugin doesn't yet
support. I think this would be an excellent feature, so we could make a
jira ticket to add this feature.

Joel Bernstein
Search Engineer at Heliosearch


On Fri, Jun 6, 2014 at 1:07 PM, Joel Bernstein  wrote:

> Reposting this from jira ticket to users list:
>
> I'm noticing a very weird bug using the CollapsingQParserPlugin. We tried
> to use this plugin when we realized that faceting on the groups would take
> a ridiculous amount of time. To its credit, it works very quickly, however
> the facet counts that it gives are incorrect.
>
> We have a smallish index of about 200k documents with about with about 50k
> distinct groups within it.
>
> When we use the group implementation
> (&group=true&group.field=PrSKU&group.facet=true) which I believe this
> attempts to emulate, the facet counts are totally correct.
>
> When we use the field collapsing implementation, it will show an incorrect
> count for the non-filtered query, but when we go to the filtered query, the
> facet count corrects itself and matches the document count.
>
> Here are some SOLR responses:
>
> solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq=
> {!collapse%20field=PrSKU}&facet=true&facet.field=at_12_wood_tone
>
> The facet field will return
>
> 867
> 441
> 253
>
> When I actually apply a filter query like so:
>
>
> solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq={!collapse%20field=PrSKU}
>
>
> &facet=true&facet.field=at_12_wood_tone&fq=at_12_wood_tone:%22Light%20Wood%22
>
> I actually pull back 270 results and the facet updates itself with the
> correct number at the bottom
>
> 270
> 68
> 66
>
> If this were the same number pre and post filter query I would assume that
> it was simply my data that was bad, however I've pored over this for the
> better part of a day and I'm pretty sure it's the plugin. For reference,
> this field that I'm faceting on is a multiValued field, however I have
> noticed the exact same behavior on non multiValued fields (such as price).
>
> I can provide any other details you might need
>

RE: SOLR-6143 Bad facet counts from CollapsingQParserPlugin

[Joel Bernstein]

Reposting this from jira ticket to users list:

I'm noticing a very weird bug using the CollapsingQParserPlugin. We tried
to use this plugin when we realized that faceting on the groups would take
a ridiculous amount of time. To its credit, it works very quickly, however
the facet counts that it gives are incorrect.

We have a smallish index of about 200k documents with about with about 50k
distinct groups within it.

When we use the group implementation
(&group=true&group.field=PrSKU&group.facet=true) which I believe this
attempts to emulate, the facet counts are totally correct.

When we use the field collapsing implementation, it will show an incorrect
count for the non-filtered query, but when we go to the filtered query, the
facet count corrects itself and matches the document count.

Here are some SOLR responses:

solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq=
{!collapse%20field=PrSKU}&facet=true&facet.field=at_12_wood_tone

The facet field will return

867
441
253

When I actually apply a filter query like so:

solrslave01:8983/index/select?q=classIDs:12&fl=PrSKU&fq={!collapse%20field=PrSKU}

&facet=true&facet.field=at_12_wood_tone&fq=at_12_wood_tone:%22Light%20Wood%22

I actually pull back 270 results and the facet updates itself with the
correct number at the bottom

270
68
66

If this were the same number pre and post filter query I would assume that
it was simply my data that was bad, however I've pored over this for the
better part of a day and I'm pretty sure it's the plugin. For reference,
this field that I'm faceting on is a multiValued field, however I have
noticed the exact same behavior on non multiValued fields (such as price).

I can provide any other details you might need

Re: Analysis browser not working in solr 4.8.1

[Shalin Shekhar Mangar]

Yes, please do open an issue. I'll fix it there.


On Fri, Jun 6, 2014 at 9:33 PM, Aman Tandon  wrote:

> Okay thanks shalin, i am fine with explicit till 4.9  :)
>
> So i don't need to open the issue right?
>
> With Regards
> Aman Tandon
>
>
> On Fri, Jun 6, 2014 at 9:20 PM, Shalin Shekhar Mangar <
> shalinman...@gmail.com> wrote:
>
> > I think I created this problem in SOLR-6023.
> >
> > I went with the javadoc documentation which says that analysis.value is a
> > required parameter and added that check. It seems that the either
> > analysis.value or analysis.q (or q) can be specified to make it work.
> I'll
> > fix in 4.9
> >
> >
> > On Fri, Jun 6, 2014 at 8:37 PM, Aman Tandon 
> > wrote:
> >
> > > It is not working with either custom filter/field  as well as with
> normal
> > > text field, it only working fine if i pass the value/text in the both
> in
> > > the index time input as well as in query time input.
> > > Only query time analysis is impossible unless i wrote that extra
> > parameter
> > > (analysis.fieldvalue=jute) explicitly in request(url).
> > >
> > > With Regards
> > > Aman Tandon
> > >
> > >
> > > On Fri, Jun 6, 2014 at 7:11 PM, Stefan Matheis <
> matheis.ste...@gmail.com
> > >
> > > wrote:
> > >
> > > > I’m not sure that’s a bug in the UI .. in case the underlying service
> > is
> > > > barking with an exception we can’t do anything else than showing to
> > you.
> > > >
> > > > are you sure the custom filter works as expected? like, verified
> with a
> > > > unit-test or something along the lines? i can still work with the
> > > examples
> > > > provided in the tutorial, so in general .. it works, looks like the
> > only
> > > > thing that doesn’t work is related to your custom components
> > > >
> > > > -Stefan
> > > >
> > > >
> > > > On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I created a custom filter for my field named text_reversed, i tried
> > my
> > > > > custom filter in solr 4.7.1 and i was able to analyse the result,
> it
> > > > works
> > > > > fine but in solr 4.8.1 it gaves me error of : *Missing required
> > > > parameter:
> > > > > analysis.fieldvalue. *It is also not working with any field*, *here
> > is
> > > > the
> > > > > logs of the error
> > > > >
> > > > > 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore
> –
> > > > > org.apache.solr.common.SolrException: Missing required parameter:
> > > > > analysis.fieldvalue
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> > > > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> > > > > at
> > > > >
> > >
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.j

timeout when create alias

[lansing]

Hi,
We are running Solr cloud 4.8.0 on tomcat 7, java 7 update 51, with 5 shards
and 3 replicas.
 
When I try to create alias via collection admin api
http://10.0.5.227:8201/solr/admin/collections?action=CREATEALIAS&name=Current&collections=Current2

It timeout after 180 s, 

181344574 [http-bio-8201-exec-1237] INFO 
org.apache.solr.servlet.SolrDispatchFilter  – [admin] webapp=null path=/ad
min/collections
params={name=Current&action=CREATEALIAS&async=true&collections=Current2}
status=500 QTime=180003 
181344574 [http-bio-8201-exec-1237] ERROR
org.apache.solr.servlet.SolrDispatchFilter  – null:org.apache.solr.common.
SolrException: createalias the collection time out:180s
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:357)
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:309)
at
org.apache.solr.handler.admin.CollectionsHandler.handleCreateAliasAction(CollectionsHandler.java:416)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:163)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:267)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)




--
View this message in context: 
http://lucene.472066.n3.nabble.com/timeout-when-create-alias-tp4140437.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Analysis browser not working in solr 4.8.1

[Aman Tandon]

Okay thanks shalin, i am fine with explicit till 4.9  :)

So i don't need to open the issue right?

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 9:20 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> I think I created this problem in SOLR-6023.
>
> I went with the javadoc documentation which says that analysis.value is a
> required parameter and added that check. It seems that the either
> analysis.value or analysis.q (or q) can be specified to make it work. I'll
> fix in 4.9
>
>
> On Fri, Jun 6, 2014 at 8:37 PM, Aman Tandon 
> wrote:
>
> > It is not working with either custom filter/field  as well as with normal
> > text field, it only working fine if i pass the value/text in the both in
> > the index time input as well as in query time input.
> > Only query time analysis is impossible unless i wrote that extra
> parameter
> > (analysis.fieldvalue=jute) explicitly in request(url).
> >
> > With Regards
> > Aman Tandon
> >
> >
> > On Fri, Jun 6, 2014 at 7:11 PM, Stefan Matheis  >
> > wrote:
> >
> > > I’m not sure that’s a bug in the UI .. in case the underlying service
> is
> > > barking with an exception we can’t do anything else than showing to
> you.
> > >
> > > are you sure the custom filter works as expected? like, verified with a
> > > unit-test or something along the lines? i can still work with the
> > examples
> > > provided in the tutorial, so in general .. it works, looks like the
> only
> > > thing that doesn’t work is related to your custom components
> > >
> > > -Stefan
> > >
> > >
> > > On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:
> > >
> > > > Hi,
> > > >
> > > > I created a custom filter for my field named text_reversed, i tried
> my
> > > > custom filter in solr 4.7.1 and i was able to analyse the result, it
> > > works
> > > > fine but in solr 4.8.1 it gaves me error of : *Missing required
> > > parameter:
> > > > analysis.fieldvalue. *It is also not working with any field*, *here
> is
> > > the
> > > > logs of the error
> > > >
> > > > 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore –
> > > > org.apache.solr.common.SolrException: Missing required parameter:
> > > > analysis.fieldvalue
> > > > at
> > > >
> > >
> >
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> > > > at
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> > > > at
> > > >
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> > > > at
> > > >
> > >
> >
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> > > > at
> > > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > > at
> > > >
> > >
> >
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> > > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> > > > at
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> > > > at
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> > > > at
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> > > > at
> > > >
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> > > > at
> > > >
> > >
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > > > at
> > > >
> > >
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> > > > at
> > > >
> > >
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> > > > at
> > > >
> > >
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> > > > at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > at java.lang.Thread.run(Thre

Re: Analysis browser not working in solr 4.8.1

[Shalin Shekhar Mangar]

I think I created this problem in SOLR-6023.

I went with the javadoc documentation which says that analysis.value is a
required parameter and added that check. It seems that the either
analysis.value or analysis.q (or q) can be specified to make it work. I'll
fix in 4.9


On Fri, Jun 6, 2014 at 8:37 PM, Aman Tandon  wrote:

> It is not working with either custom filter/field  as well as with normal
> text field, it only working fine if i pass the value/text in the both in
> the index time input as well as in query time input.
> Only query time analysis is impossible unless i wrote that extra parameter
> (analysis.fieldvalue=jute) explicitly in request(url).
>
> With Regards
> Aman Tandon
>
>
> On Fri, Jun 6, 2014 at 7:11 PM, Stefan Matheis 
> wrote:
>
> > I’m not sure that’s a bug in the UI .. in case the underlying service is
> > barking with an exception we can’t do anything else than showing to you.
> >
> > are you sure the custom filter works as expected? like, verified with a
> > unit-test or something along the lines? i can still work with the
> examples
> > provided in the tutorial, so in general .. it works, looks like the only
> > thing that doesn’t work is related to your custom components
> >
> > -Stefan
> >
> >
> > On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:
> >
> > > Hi,
> > >
> > > I created a custom filter for my field named text_reversed, i tried my
> > > custom filter in solr 4.7.1 and i was able to analyse the result, it
> > works
> > > fine but in solr 4.8.1 it gaves me error of : *Missing required
> > parameter:
> > > analysis.fieldvalue. *It is also not working with any field*, *here is
> > the
> > > logs of the error
> > >
> > > 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore –
> > > org.apache.solr.common.SolrException: Missing required parameter:
> > > analysis.fieldvalue
> > > at
> > >
> >
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> > > at
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> > > at
> > >
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> > > at
> > >
> >
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> > > at
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > at
> > >
> >
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> > > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> > > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> > > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > > at
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > > at
> > >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > > at
> > >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > > at
> > >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > > at
> > >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > > at
> > >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> > > at
> > >
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> > > at
> > >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> > > at
> > >
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > > at
> > >
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> > > at
> > >
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> > > at
> > >
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > at java.lang.Thread.run(Thread.java:745)
> > >
> > >
> > >
> > > With Regards
> > > Aman Tandon
> > >
> > >
> >
> >
> >
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: Can the elevation component work with synonyms?

[Shawn Heisey]

On 6/6/2014 9:08 AM, eShard wrote:
> However, when I try to call my /elevate handler; the synonyms are factored
> in but none of the results say [elevated]=true
> I'm assuming this is because the elevation must be an exact match and the
> synonyms are expanding it beyond that so elevation is thwarted.
> For example, if I have TV elevated and TV is also in synonyms.txt then the
> query gets expanded to text:TV text:television.
>
> Is there any way to get the elevation to work correctly with synonyms?

I see two choices here.  The first, which is the only one that I can
reasonably be sure will work, is to do synonym expansion only at index
time.  The other is to put the fully expanded query into the elevate
config.  I do not know if this will actually work -- the situation may
involve more complexity.

Often synonyms are only done for one analysis chain, but if that's the
case, they are usually done for the query side, not the index side. 
Therefore, if the elevate config will do it, the latter option above
would be preferred.  If you change your synonyms, you might need to also
change your elevate config.

Thanks,
Shawn

Re: Analysis browser not working in solr 4.8.1

[Aman Tandon]

It is not working with either custom filter/field  as well as with normal
text field, it only working fine if i pass the value/text in the both in
the index time input as well as in query time input.
Only query time analysis is impossible unless i wrote that extra parameter
(analysis.fieldvalue=jute) explicitly in request(url).

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 7:11 PM, Stefan Matheis 
wrote:

> I’m not sure that’s a bug in the UI .. in case the underlying service is
> barking with an exception we can’t do anything else than showing to you.
>
> are you sure the custom filter works as expected? like, verified with a
> unit-test or something along the lines? i can still work with the examples
> provided in the tutorial, so in general .. it works, looks like the only
> thing that doesn’t work is related to your custom components
>
> -Stefan
>
>
> On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:
>
> > Hi,
> >
> > I created a custom filter for my field named text_reversed, i tried my
> > custom filter in solr 4.7.1 and i was able to analyse the result, it
> works
> > fine but in solr 4.8.1 it gaves me error of : *Missing required
> parameter:
> > analysis.fieldvalue. *It is also not working with any field*, *here is
> the
> > logs of the error
> >
> > 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore –
> > org.apache.solr.common.SolrException: Missing required parameter:
> > analysis.fieldvalue
> > at
> >
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> > at
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> > at
> >
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> > at
> >
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > at
> >
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > at
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> > at
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> > at
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> > With Regards
> > Aman Tandon
> >
> >
>
>
>

Can the elevation component work with synonyms?

[eShard]

Good morning Solr compatriots,
I'm using Solr4.0Final and I have synonyms.txt in my schema (only at query
time) like so:

  







  
  






  
  
  






  
  


However, when I try to call my /elevate handler; the synonyms are factored
in but none of the results say [elevated]=true
I'm assuming this is because the elevation must be an exact match and the
synonyms are expanding it beyond that so elevation is thwarted.
For example, if I have TV elevated and TV is also in synonyms.txt then the
query gets expanded to text:TV text:television.

Is there any way to get the elevation to work correctly with synonyms?

BTW
(I did find a custom synonym handler that works but this will require
significant changes to the front end and I'm not sure it will break if and
when we finally upgrade solr)
Here's the custom synonym filter (I had to drop the code in and rebuild
solr.war to get it to work):
https://github.com/healthonnet/hon-lucene-synonyms 




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Can-the-elevation-component-work-with-synonyms-tp4140423.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr-Distributed search

[Aman Tandon]

Thanks shawn i will try to think in that way too :)

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 8:19 PM, Shawn Heisey  wrote:

> On 6/6/2014 8:31 AM, Aman Tandon wrote:
> > In my organisation we also want to implement the solrcloud, but the
> problem
> > is that, we are using the master-slave architecture and on master we do
> all
> > indexing, architecture of master is lower than the slaves.
> >
> > So if we implement the solrcloud in a fashion that master will be the
> > leader, and slaves will be the replicas then in that case, in the case of
> > high load leader can bear it,  I guess every query firstly goes to leader
> > then it distributes the request as i noticed from the logs and blogs :)
> >
> > As well as master is in NY and slaves are in Dallas, which also might
> cause
> > latency issue and it will instead fail our purpose of faster query
> response.
> >
> > So i thought to use this shards parameter so that we query only from the
> > replicas not to the leader so that leader just work fine. But we were not
> > sure about this shards parameter, what do you think? what should we do
> with
> > latency issue and shards parameter.
>
> SolrCloud does not yet have any way to prefer one set of replicas over
> the others, so if you just send it requests, they would be sent to both
> Dallas and New York, affecting search latency.  Local replica preference
> is a desperately needed feature.
>
> Old-style distributed search with the shards parameter, combined with
> master/slave replication, is an effective way to be absolutely sure
> which servers you are querying.
>
> I would actually recommend that you get rid of replication and have your
> index updating software update each copy of the index independently.
> This is how I do my Solr install.  It opens up a whole new set of
> possibilities -- you can change the schema and/or config on one set of
> servers, or upgrade any component -- Solr, Java, etc., without affecting
> the other set of servers at all.
>
> One note: in order for the indexing paradigm I've outlined to be
> actually effective, you must separately track which
> inserts/updates/deletes have been done for each server set.  If you
> don't do that, they can get out of sync when you restart a server.
> Also, if you don't do this, having a server is down for an extended
> period of time might cause all indexing activity to stop on BOTH server
> sets.
>
> Thanks,
> Shawn
>
>

Re: Solr-Distributed search

[Shawn Heisey]

On 6/6/2014 8:31 AM, Aman Tandon wrote:
> In my organisation we also want to implement the solrcloud, but the problem
> is that, we are using the master-slave architecture and on master we do all
> indexing, architecture of master is lower than the slaves.
>
> So if we implement the solrcloud in a fashion that master will be the
> leader, and slaves will be the replicas then in that case, in the case of
> high load leader can bear it,  I guess every query firstly goes to leader
> then it distributes the request as i noticed from the logs and blogs :)
>
> As well as master is in NY and slaves are in Dallas, which also might cause
> latency issue and it will instead fail our purpose of faster query response.
>
> So i thought to use this shards parameter so that we query only from the
> replicas not to the leader so that leader just work fine. But we were not
> sure about this shards parameter, what do you think? what should we do with
> latency issue and shards parameter.

SolrCloud does not yet have any way to prefer one set of replicas over
the others, so if you just send it requests, they would be sent to both
Dallas and New York, affecting search latency.  Local replica preference
is a desperately needed feature.

Old-style distributed search with the shards parameter, combined with
master/slave replication, is an effective way to be absolutely sure
which servers you are querying.

I would actually recommend that you get rid of replication and have your
index updating software update each copy of the index independently. 
This is how I do my Solr install.  It opens up a whole new set of
possibilities -- you can change the schema and/or config on one set of
servers, or upgrade any component -- Solr, Java, etc., without affecting
the other set of servers at all.

One note: in order for the indexing paradigm I've outlined to be
actually effective, you must separately track which
inserts/updates/deletes have been done for each server set.  If you
don't do that, they can get out of sync when you restart a server. 
Also, if you don't do this, having a server is down for an extended
period of time might cause all indexing activity to stop on BOTH server
sets.

Thanks,
Shawn

Re: Solr-Distributed search

[Aman Tandon]

Thanks shawn.

In my organisation we also want to implement the solrcloud, but the problem
is that, we are using the master-slave architecture and on master we do all
indexing, architecture of master is lower than the slaves.

So if we implement the solrcloud in a fashion that master will be the
leader, and slaves will be the replicas then in that case, in the case of
high load leader can bear it,  I guess every query firstly goes to leader
then it distributes the request as i noticed from the logs and blogs :)

As well as master is in NY and slaves are in Dallas, which also might cause
latency issue and it will instead fail our purpose of faster query response.

So i thought to use this shards parameter so that we query only from the
replicas not to the leader so that leader just work fine. But we were not
sure about this shards parameter, what do you think? what should we do with
latency issue and shards parameter.

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 7:24 PM, Shawn Heisey  wrote:

> On 6/6/2014 6:25 AM, Aman Tandon wrote:
> >  Does this *shards* parameter will also work in near future with solr 5?
>
> I am not aware of any plan to deprecate or remove the shards parameter.
> My personal experience is with versions from 1.4.0 through 4.7.2.  It
> works in all of those versions.  Without SolrCloud, the shards parameter
> is the only way you can do a distributed search.
>
> Thanks,
> Shawn
>
>

Re: Solr-Distributed search

[Aman Tandon]

Thanks shawn.

In my organisation we also want to implement the solrcloud, but the problem
is that, we are using the master-slave architecture and on master we do all
indexing, architecture of master is lower than the slaves.

So if we implement the solrcloud in a fashion that master will be the
leader, and slaves will be the replicas then in that case, in the case of
high load leader can bear it,  I guess every query firstly goes to leader
then it distributes the request as i noticed from the logs and blogs :)

As well as master is in NY and slaves are in Dallas, which also might cause
latency issue and it will instead fail our purpose of faster query response.

So i thought to use this shards parameter so that we query only from the
replicas not to the leader so that leader just work fine. But we were not
sure about this shards parameter, what do you think? what should we do with
latency issue and shards parameter.

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 7:24 PM, Shawn Heisey  wrote:

> On 6/6/2014 6:25 AM, Aman Tandon wrote:
> >  Does this *shards* parameter will also work in near future with solr 5?
>
> I am not aware of any plan to deprecate or remove the shards parameter.
> My personal experience is with versions from 1.4.0 through 4.7.2.  It
> works in all of those versions.  Without SolrCloud, the shards parameter
> is the only way you can do a distributed search.
>
> Thanks,
> Shawn
>
>

Re: Date number to text

[Jack Krupansky]

The "parse date" update processor can be used to parse dates in a number of 
formats (configurable) and convert to a Java date value:

http://lucene.apache.org/solr/4_8_1/solr-core/org/apache/solr/update/processor/ParseDateFieldUpdateProcessorFactory.html

You could copy the field to another field to keep it as text.

You can also use your own update processor to convert to some other text 
format. See the stateless script update processor to write a simple 
JavaScript script.


-- Jack Krupansky

-Original Message- 
From: O. Klein

Sent: Friday, June 6, 2014 4:32 AM
To: solr-user@lucene.apache.org
Subject: Date number to text

Is there a way to tokenize 12-6-2014 as 12 june 2014 and the other way 
around

as well?

Making it possible for people to search on text and numbers for dates?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Date-number-to-text-tp4140323.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: Solr-Distributed search

[Shawn Heisey]

On 6/6/2014 6:25 AM, Aman Tandon wrote:
>  Does this *shards* parameter will also work in near future with solr 5?

I am not aware of any plan to deprecate or remove the shards parameter. 
My personal experience is with versions from 1.4.0 through 4.7.2.  It
works in all of those versions.  Without SolrCloud, the shards parameter
is the only way you can do a distributed search.

Thanks,
Shawn

Re: Date number to text

[Shawn Heisey]

On 6/6/2014 2:32 AM, O. Klein wrote:
> Is there a way to tokenize 12-6-2014 as 12 june 2014 and the other way around
> as well?
>
> Making it possible for people to search on text and numbers for dates?

Dates in Solr (if you use an actual date field, and not a text field)
follow strict interpretation of ISO 8601.  Here is an example of a
date/time in this format:

2014-06-06T03:07:29Z

Solr only displays and accepts time in UTC.  Although the ISO standard
does have an additional form that includes the timezone offset, this
extension won't work with Solr.

If you want human readable dates as output or input, your code must
convert them to/from the format that Solr expects.

Thanks,
Shawn

Documents Added Not Available After Commit (Both Soft and Hard)

[Justin Sweeney]

Hi,

An application I am working on indexes documents to a Solr index. This Solr
index is setup as a single node, without any replication. This index is
running Solr 4.5.0.

We have noticed an issue lately that is causing some problems for our
application. The problem is that we add/update a number of documents in the
Solr index and we have the index setup to autoCommit (hard) once every 30
minutes. In the Solr logs, I am able to see the add command to Solr and I
can also see Solr start the hard commit. When this hard commit occurs, we
see the following message:
INFO  - 2014-06-04 20:13:55.135;
org.apache.solr.update.DirectUpdateHandler2; No uncommitted changes.
Skipping IW.commit.

This only happens sometimes, but Solr will go hours (we have seen 6-12
hours of this behavior) before it does a hard commit where it find changes.
After the hard commit where the changes are found, we are then able to
search for and find the documents that were added hours ago, but up until
that point the documents are not searchable.

We tried enabling autoSoftCommit every 5 minutes in the hope that this
would help, but we are seeing the same behavior.

Here is a sampling of the logs showing this occurring (I've trimmed it down
to just show what is happening):

INFO  - 2014-06-05 20:00:41.300;
>> org.apache.solr.update.processor.LogUpdateProcessor; [zoomCollection]
>> webapp=/solr path=/update params={wt=javabin&version=2} {add=[359453225]} 0
>> 0
>
> INFO  - 2014-06-05 20:00:41.376;
>> org.apache.solr.update.processor.LogUpdateProcessor; [zoomCollection]
>> webapp=/solr path=/update params={wt=javabin&version=2} {add=[347170717]} 0
>> 1
>
> INFO  - 2014-06-05 20:00:51.527;
>> org.apache.solr.update.DirectUpdateHandler2; start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> INFO  - 2014-06-05 20:00:51.533; org.apache.solr.search.SolrIndexSearcher;
>> Opening Searcher@257c43d main
>
> INFO  - 2014-06-05 20:00:51.533;
>> org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
>
> INFO  - 2014-06-05 20:00:51.545; org.apache.solr.core.QuerySenderListener;
>> QuerySenderListener sending requests to Searcher@257c43d
>> main{StandardDirectoryReader(segments_acl:1367002775953
>> _2f28(4.5):C13583563/4081507 _2gl6(4.5):C2754573/193533
>> _2g21(4.5):C1046256/296354 _2ge2(4.5):C835858/206139
>> _2gqd(4.5):C383500/31051 _2gmu(4.5):C125197/32491 _2grl(4.5):C46906/1255
>> _2gpj(4.5):C66480/16562 _2gra(4.5):C364/22 _2gr1(4.5):C36064/2556
>> _2gqg(4.5):C42504/21515 _2gqm(4.5):C26821/12659 _2gqu(4.5):C24172/10240
>> _2gqy(4.5):C697/215 _2gr2(4.5):C878/352 _2gr7(4.5):C28135/11775
>> _2gr9(4.5):C3276/1341 _2grb(4.5):C5/1 _2grc(4.5):C3247/1219 _2grd(4.5):C6/1
>> _2grf(4.5):C5/2 _2grg(4.5):C23659/10967 _2grh(4.5):C1 _2grj(4.5):C1
>> _2grk(4.5):C5160/1482 _2grm(4.5):C1210/351 _2grn(4.5):C3957/1372
>> _2gro(4.5):C7734/2207 _2grp(4.5):C220/36)}
>
> INFO  - 2014-06-05 20:00:51.546; org.apache.solr.core.SolrCore;
>> [zoomCollection] webapp=null path=null
>> params={event=newSearcher&q=d_name:ibm&distrib=false} hits=38 status=0
>> QTime=0
>
> INFO  - 2014-06-05 20:00:51.546; org.apache.solr.core.QuerySenderListener;
>> QuerySenderListener done.
>
> INFO  - 2014-06-05 20:00:51.547; org.apache.solr.core.SolrCore;
>> [zoomCollection] Registered new searcher Searcher@257c43d
>> main{StandardDirectoryReader(segments_acl:1367002775953
>> _2f28(4.5):C13583563/4081507 _2gl6(4.5):C2754573/193533
>> _2g21(4.5):C1046256/296354 _2ge2(4.5):C835858/206139
>> _2gqd(4.5):C383500/31051 _2gmu(4.5):C125197/32491 _2grl(4.5):C46906/1255
>> _2gpj(4.5):C66480/16562 _2gra(4.5):C364/22 _2gr1(4.5):C36064/2556
>> _2gqg(4.5):C42504/21515 _2gqm(4.5):C26821/12659 _2gqu(4.5):C24172/10240
>> _2gqy(4.5):C697/215 _2gr2(4.5):C878/352 _2gr7(4.5):C28135/11775
>> _2gr9(4.5):C3276/1341 _2grb(4.5):C5/1 _2grc(4.5):C3247/1219 _2grd(4.5):C6/1
>> _2grf(4.5):C5/2 _2grg(4.5):C23659/10967 _2grh(4.5):C1 _2grj(4.5):C1
>> _2grk(4.5):C5160/1482 _2grm(4.5):C1210/351 _2grn(4.5):C3957/1372
>> _2gro(4.5):C7734/2207 _2grp(4.5):C220/36)}
>
> INFO  - 2014-06-05 20:01:10.557;
>> org.apache.solr.update.DirectUpdateHandler2; start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> INFO  - 2014-06-05 20:01:10.559; org.apache.solr.core.SolrCore;
>> [zoomCollection] webapp=/solr path=/select
>> params={fl=d_ticker,d_location,d_id,d_source_count,d_xml_domain,d_cik,d_keyword_count,d_xml_name,d_xml_contact,d_main_domain,d_location_code&start=0&q=d_domain:("
>> www.northwestcollege.edu")&wt=javabin&version=2&rows=99} hits=4
>> status=0 QTime=40
>
> INFO  - 2014-06-05 20:01:10.563; org.apache.solr.search.SolrIndexSearcher;
>> Opening Searcher@69f90ad1 main
>
> INFO  - 2014-06-05 20:01:10.564;
>> org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
>
> INFO  - 2014-06-05 20:01:10.579; org.apache.solr.core.QuerySenderListener;
>> QuerySenderListener sending 

Re: Analysis browser not working in solr 4.8.1

[Stefan Matheis]

I’m not sure that’s a bug in the UI .. in case the underlying service is 
barking with an exception we can’t do anything else than showing to you.  

are you sure the custom filter works as expected? like, verified with a 
unit-test or something along the lines? i can still work with the examples 
provided in the tutorial, so in general .. it works, looks like the only thing 
that doesn’t work is related to your custom components

-Stefan  


On Friday, June 6, 2014 at 1:25 PM, Aman Tandon wrote:

> Hi,
>  
> I created a custom filter for my field named text_reversed, i tried my
> custom filter in solr 4.7.1 and i was able to analyse the result, it works
> fine but in solr 4.8.1 it gaves me error of : *Missing required parameter:
> analysis.fieldvalue. *It is also not working with any field*, *here is the
> logs of the error
>  
> 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore –
> org.apache.solr.common.SolrException: Missing required parameter:
> analysis.fieldvalue
> at
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> at
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> at
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> at
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>  
>  
>  
> With Regards
> Aman Tandon
>  
>  

Deepy nested structure

[harikrishna]

we need to have the nested structure for the index, and the requirement is as
follows

we have application at root, then customer location, and then we have some
entities data




 








i want to index the data in the above formate, and wanted to retrieve in the
same way. 
please help on this



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Deepy-nested-structure-tp4140397.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud: AliasAPI-Maximum number of collections

[Mark Miller]

The main limit is the 1mb zk node limit. But even that can be raised. 

- Mark

> On Jun 6, 2014, at 6:21 AM, Shalin Shekhar Mangar  
> wrote:
> 
> No, there's no theoretical limit.
> 
> 
>> On Fri, Jun 6, 2014 at 11:20 AM, ku3ia  wrote:
>> 
>> Hi all!
>> The question is how many collections I can put to one alias, using
>> SolrCloud
>> alias collection API
>> 
>> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api4
>> to process distributed requests? Is it limited?
>> 
>> Thanks.
>> 
>> 
>> 
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/SolrCloud-AliasAPI-Maximum-number-of-collections-tp4140305.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
> 
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.

Re: SolrCloud: AliasAPI-Maximum number of collections

[Shalin Shekhar Mangar]

No, there's no theoretical limit.


On Fri, Jun 6, 2014 at 11:20 AM, ku3ia  wrote:

> Hi all!
> The question is how many collections I can put to one alias, using
> SolrCloud
> alias collection API
>
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api4
> to process distributed requests? Is it limited?
>
> Thanks.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-AliasAPI-Maximum-number-of-collections-tp4140305.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: Solr maximum Optimal Index Size per Shard

[Vineet Mishra]

Earlier I used to index with HtttpPost Mechanism only, making each post
size specific to 2Mb to 20Mb that was going fine, but we had a suspect that
instead of indexing through network call(which ofcourse results in latency
due to network delays and http protocol) if we can index Offline by just
writing the index and dumping it to Shards it would be much better.

Although I am doing commit with a batch of 25K docs which I will try to
replace with CommitWithin(seems it works faster) or probably have a look at
this Binary Prot.

Thanks!




On Fri, Jun 6, 2014 at 5:55 PM, Toke Eskildsen 
wrote:

> On Fri, 2014-06-06 at 14:05 +0200, Vineet Mishra wrote:
>
> > Could you state what indexing mechanism are you using, as I started
> > with EmbeddedSolrServer but it was pretty slow after a few GB(~30+) of
> > indexing.
>
> I suspect that is due to too-frequent commits, too small heap or
> something third, unrelated to EmbeddedSolrServer itself. Underneath the
> surface it is just the same as a standalone Solr.
>
> We're building our ~1TB indexes individually, using standalone workers
> for the heavy part of the analysis (Tika). The delivery from the workers
> to the Solr server is over the network, using the Solr binary protocol.
> My colleague Thomas Egense just created a small write-up at
> https://github.com/netarchivesuite/netsearch
>
> >  I started indexing 1 week back and still its 37GB, although I assume
> > HttpPost mechanism will perform lethargic slow due to network latency
> > and for the response await.
>
> Maybe if you send the documents one at a time, but if you bundle them in
> larger updates, the post-method should be fine.
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>

span query with SHOUD semantic instead of MUST HAVE

[??????]

hi,


I have two docs, 
a) "aa bb cc" and,
b) "aa cc bb". 
The query is "aa bb". What I expected is the doc a comes first with a higher 
score than doc b because the term distance in query and that in doc a are more 
similar. 
After google for a while I get it down with the span query q: "aa bb"~10. 
However, when I change my query into "aa bb dd"~10, the span query return 
nothing 
hits becuase dd can not be found in any doc. So what's a solution to this 
problem? 


Thanks.

Re: Solr-Distributed search

[Aman Tandon]

Hi,

 Does this *shards* parameter will also work in near future with solr 5?

With Regards
Aman Tandon


On Thu, Jun 5, 2014 at 2:59 PM, Mahmoud Almokadem 
wrote:

> Hi, you can search using this sample Url
>
>
> http://localhost:8080/solr/core1/select?q=*:*&shards=localhost:8080/solr/core1,localhost:8080/solr/core2,localhost:8080/solr/core3
>
> Mahmoud Almokadem
>
>
> On Thu, Jun 5, 2014 at 8:13 AM, Anurag Verma 
> wrote:
>
> > Hi,
> > Can you please help me solr distribued search in multicore? i
> would
> > be very happy as i am stuck here.
> >
> > In java code how do i implement distributed search?
> > --
> > Thanks & Regards
> > Anurag Verma
> >
>

Re: Solr maximum Optimal Index Size per Shard

[Toke Eskildsen]

On Fri, 2014-06-06 at 14:05 +0200, Vineet Mishra wrote:

> Could you state what indexing mechanism are you using, as I started
> with EmbeddedSolrServer but it was pretty slow after a few GB(~30+) of
> indexing.

I suspect that is due to too-frequent commits, too small heap or
something third, unrelated to EmbeddedSolrServer itself. Underneath the
surface it is just the same as a standalone Solr.

We're building our ~1TB indexes individually, using standalone workers
for the heavy part of the analysis (Tika). The delivery from the workers
to the Solr server is over the network, using the Solr binary protocol.
My colleague Thomas Egense just created a small write-up at
https://github.com/netarchivesuite/netsearch

>  I started indexing 1 week back and still its 37GB, although I assume
> HttpPost mechanism will perform lethargic slow due to network latency
> and for the response await.

Maybe if you send the documents one at a time, but if you bundle them in
larger updates, the post-method should be fine.

- Toke Eskildsen, State and University Library, Denmark

Re: Integrate solr with openNLP

[Aman Tandon]

Hi Vikek,

As everybody in the mail list mentioned to use UIMA you should go for it,
as opennlp issues are not tracking properly, it can make stuck your
development in near future if any issue comes, so its better to start
investigate with uima.


With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 11:00 AM, Vivekanand Ittigi 
wrote:

> Can anyone pleas reply..?
>
> Thanks,
> Vivek
>
> -- Forwarded message --
> From: Vivekanand Ittigi 
> Date: Wed, Jun 4, 2014 at 4:38 PM
> Subject: Re: Integrate solr with openNLP
> To: Tommaso Teofili 
> Cc: "solr-user@lucene.apache.org" , Ahmet
> Arslan 
>
>
> Hi Tommaso,
>
> Yes, you are right. 4.4 version will work.. I'm able to compile now. I'm
> trying to apply named recognition(person name) token but im not seeing any
> change. my schema.xml looks like this:
>
>  multiValued="true"/>
>
>  positionIncrementGap="100">
>   
>tokenizerModel="opennlp/en-token.bin"
> />
>nerTaggerModels="opennlp/en-ner-person.bin"
> />
> 
>   
>
> 
>
> Please guide..?
>
> Thanks,
> Vivek
>
>
> On Wed, Jun 4, 2014 at 1:27 PM, Tommaso Teofili  >
> wrote:
>
> > Hi all,
> >
> > Ahment was suggesting to eventually use UIMA integration because OpenNLP
> > has already an integration with Apache UIMA and so you would just have to
> > use that [1].
> > And that's one of the main reason UIMA integration was done: it's a
> > framework that you can easily hook into in order to plug your NLP
> algorithm.
> >
> > If you want to just use OpenNLP then it's up to you if either write your
> > own UpdateRequestProcessor plugin [2] to add metadata extracted by
> OpenNLP
> > to your documents or either you can write a dedicated analyzer /
> tokenizer
> > / token filter.
> >
> > For the OpenNLP integration (LUCENE-2899), the patch is not up to date
> > with the latest APIs in trunk, however you should be able to apply it to
> > (if I recall correctly) to 4.4 version or so, and also adapting it to the
> > latest API shouldn't be too hard.
> >
> > Regards,
> > Tommaso
> >
> > [1] :
> >
> http://opennlp.apache.org/documentation/1.5.3/manual/opennlp.html#org.apche.opennlp.uima
> > [2] : http://wiki.apache.org/solr/UpdateRequestProcessor
> >
> >
> >
> > 2014-06-03 15:34 GMT+02:00 Ahmet Arslan :
> >
> > Can you extract names, locations etc using OpenNLP in plain/straight java
> >> program?
> >>
> >> If yes, here are two seperate options :
> >>
> >> 1) Use http://searchhub.org/2012/02/14/indexing-with-solrj/ as an
> >> example to integrate your NER code into it and write your own indexing
> >> code. You have the full power here. No solr-plugins are involved.
> >>
> >> 2) Use 'Implementing a conditional copyField' given here :
> >> http://wiki.apache.org/solr/UpdateRequestProcessor
> >> as an example and integrate your NER code into it.
> >>
> >>
> >> Please note that these are separate ways to enrich your incoming
> >> documents, choose either (1) or (2).
> >>
> >>
> >>
> >> On Tuesday, June 3, 2014 3:30 PM, Vivekanand Ittigi <
> >> vi...@biginfolabs.com> wrote:
> >> Okay, but i dint understand what you said. Can you please elaborate.
> >>
> >> Thanks,
> >> Vivek
> >>
> >>
> >>
> >>
> >>
> >> On Tue, Jun 3, 2014 at 5:36 PM, Ahmet Arslan  wrote:
> >>
> >> > Hi Vivekanand,
> >> >
> >> > I have never use UIMA+Solr before.
> >> >
> >> > Personally I think it takes more time to learn how to configure/use
> >> these
> >> > uima stuff.
> >> >
> >> >
> >> > If you are familiar with java, write a class that extends
> >> > UpdateRequestProcessor(Factory). Use OpenNLP for NER, add these new
> >> fields
> >> > (organisation, city, person name, etc, to your document. This phase is
> >> > usually called 'enrichment'.
> >> >
> >> > Does that makes sense?
> >> >
> >> >
> >> >
> >> > On Tuesday, June 3, 2014 2:57 PM, Vivekanand Ittigi <
> >> vi...@biginfolabs.com>
> >> > wrote:
> >> > Hi Ahmet,
> >> >
> >> > I followed what you said
> >> > https://cwiki.apache.org/confluence/display/solr/UIMA+Integration.
> But
> >> how
> >> > can i achieve my goal? i mean extracting only name of the organization
> >> or
> >> > person from the content field.
> >> >
> >> > I guess i'm almost there but something is missing? please guide me
> >> >
> >> > Thanks,
> >> > Vivek
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, Jun 3, 2014 at 2:50 PM, Vivekanand Ittigi <
> >> vi...@biginfolabs.com>
> >> > wrote:
> >> >
> >> > > Entire goal cant be said but one of those tasks can be like this..
> we
> >> > have
> >> > > big document(can be website or pdf etc) indexed to the solr.
> >> > > Lets say  will sore store the contents of
> >> document.
> >> > > All i want to do is pick name of persons,places from it using
> openNLP
> >> or
> >> > > some other means.
> >> > >
> >> > > Those names should be reflected in solr itself.
> >> > >
> >> > > Thanks,
> >> > > Vivek
> >> > >
> >> > >
> >> > > On Tue, Jun 3, 2014 at 1:33 PM, Ahmet Arslan 
> >> wrote:
> >> > >
> >> > >> Hi,
>

Re: Solr maximum Optimal Index Size per Shard

[Vineet Mishra]

Hi Toke,

That was Spectacular, really great to hear that you have already indexed
2.7TB+ data to your server and still the query response time is under ms or
a few seconds for such a huge dataset.
Could you state what indexing mechanism are you using, as I started with
EmbeddedSolrServer but it was pretty slow after a few GB(~30+) of indexing.
I started indexing 1 week back and still its 37GB, although I assume
HttpPost mechanism will perform lethargic slow due to network latency and
for the response await. Furthermore I started with CloudSolrServer but
facing some weird exception saying "ClassCastException Cannot cast to
Exception" while adding the SolrInputDocument to the Server.

CloudSolrServer server1 = new
CloudSolrServer("zkHost:port1,zkHost:port2,zkHost:port3",false);
server1.setDefaultCollection("mycollection");
SolrInputDocument doc = new SolrInputDocument();
doc.addField( "ID", "123");
doc.addField( "A0_s", "282628854");

server1.add(doc); //Error at this line
server1.commit();

Thanks again Toke for sharing that Stats.


On Fri, Jun 6, 2014 at 5:04 PM, Toke Eskildsen 
wrote:

> On Fri, 2014-06-06 at 12:32 +0200, Vineet Mishra wrote:
> > *Does that mean for querying smoothly we need to have memory atleast
> equal
> > or greater to the size of index?
>
> If you absolutely, positively have to reduce latency as much as
> possible, then yes. With an estimated index size of 2TB, I would guess
> that 10-20 machines with powerful CPUs (1 per shard per expected
> concurrent request) would also be advisable. While you're at it, do make
> sure that you're using high-speed memory.
>
> That was not a serious suggestion, should you be in doubt. Very few
> people need the best latency possible. Most just need the individual
> searches to be "fast enough" and want to scale throughput instead.
>
> > As in my case the index size will be very heavy(~2TB) and practically
> > speaking that amount of memory is not possible. Even If it goes to
> > multiple shards, say around 10 Shards then also 200GB of RAM will not
> > be an feasible option.
>
> We're building a projected 24TB index collection and are currently at
> 2.7TB+, growing with about 1TB/10 days. Our current plan is to use a
> single machine with 256GB of RAM, but we will of course adjust along the
> way if it proves to be too small.
>
> Requirements differ with the corpus and the needs, but for us, SSDs as
> storage seems to provide quite enough of a punch. I did a little testing
> yesterday: https://plus.google.com/u/0/+TokeEskildsen/posts/4yPvzrQo8A7
>
> tl;dr: for small result sets (< 1M hits) on unwarmed searches with
> simple queries, response time is below 100ms. If we enable faceting with
> plain Solr, this jumps to about 1 second.
>
> I did a top on the machine and it says that 50GB is currently used for
> caching, so an 80GB (and probably less) machine would work fine for our
> 2.7TB index.
>
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>

Re: Tika: url issue

[harshrossi]

Thanks for the help.. But anyway I solved it using RegexTransformer

In the db_link entity I used RegexTransformer and set the link field as:


and in tika-doc entity I set the Url value as:
${db_link.link}



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Tika-url-issue-tp4139781p4140376.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr maximum Optimal Index Size per Shard

[Toke Eskildsen]

On Fri, 2014-06-06 at 12:32 +0200, Vineet Mishra wrote:
> *Does that mean for querying smoothly we need to have memory atleast equal
> or greater to the size of index?

If you absolutely, positively have to reduce latency as much as
possible, then yes. With an estimated index size of 2TB, I would guess
that 10-20 machines with powerful CPUs (1 per shard per expected
concurrent request) would also be advisable. While you're at it, do make
sure that you're using high-speed memory.

That was not a serious suggestion, should you be in doubt. Very few
people need the best latency possible. Most just need the individual
searches to be "fast enough" and want to scale throughput instead.

> As in my case the index size will be very heavy(~2TB) and practically
> speaking that amount of memory is not possible. Even If it goes to
> multiple shards, say around 10 Shards then also 200GB of RAM will not
> be an feasible option.

We're building a projected 24TB index collection and are currently at
2.7TB+, growing with about 1TB/10 days. Our current plan is to use a
single machine with 256GB of RAM, but we will of course adjust along the
way if it proves to be too small.

Requirements differ with the corpus and the needs, but for us, SSDs as
storage seems to provide quite enough of a punch. I did a little testing
yesterday: https://plus.google.com/u/0/+TokeEskildsen/posts/4yPvzrQo8A7

tl;dr: for small result sets (< 1M hits) on unwarmed searches with
simple queries, response time is below 100ms. If we enable faceting with
plain Solr, this jumps to about 1 second.

I did a top on the machine and it says that 50GB is currently used for
caching, so an 80GB (and probably less) machine would work fine for our
2.7TB index.


- Toke Eskildsen, State and University Library, Denmark

Re: Analysis browser not working in solr 4.8.1

[Aman Tandon]

The url forming is
*http://localhost:8984/solr/#/IM-Search/analysis?analysis.query=jute&analysis.fieldname=title&verbose_output=1
*

Here is the screenshot: http://picpaste.com/HrW26A8d.png

With Regards
Aman Tandon


On Fri, Jun 6, 2014 at 4:55 PM, Aman Tandon  wrote:

> Hi,
>
> I created a custom filter for my field named text_reversed, i tried my
> custom filter in solr 4.7.1 and i was able to analyse the result, it works
> fine but in solr 4.8.1 it gaves me error of : *Missing required
> parameter: analysis.fieldvalue. *It is also not working with any field*, *here
> is the logs of the error
>
> 2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore  –
> org.apache.solr.common.SolrException: Missing required parameter:
> analysis.fieldvalue
> at
> org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
> at
> org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
> at
> org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
> at
> org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> With Regards
> Aman Tandon
>

Analysis browser not working in solr 4.8.1

[Aman Tandon]

Hi,

I created a custom filter for my field named text_reversed, i tried my
custom filter in solr 4.7.1 and i was able to analyse the result, it works
fine but in solr 4.8.1 it gaves me error of : *Missing required parameter:
analysis.fieldvalue. *It is also not working with any field*, *here is the
logs of the error

2090419 [http-bio-8984-exec-8] ERROR org.apache.solr.core.SolrCore  –
org.apache.solr.common.SolrException: Missing required parameter:
analysis.fieldvalue
at
org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)
at
org.apache.solr.handler.FieldAnalysisRequestHandler.resolveAnalysisRequest(FieldAnalysisRequestHandler.java:142)
at
org.apache.solr.handler.FieldAnalysisRequestHandler.doAnalysis(FieldAnalysisRequestHandler.java:99)
at
org.apache.solr.handler.AnalysisRequestHandlerBase.handleRequestBody(AnalysisRequestHandlerBase.java:60)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)



With Regards
Aman Tandon

Re: Solr maximum Optimal Index Size per Shard

[Vineet Mishra]

Hey Jack,

Well I have indexed around some 10 Million documents consuming 20 GB index
size.
Each Document is consisting of nearly 100 String Fields with data upto 10
characters per field.
For my case each document containing number of fields can expand much
widely (from current 100 to 500 or ever more).

As for the typical exceptional case I was more interested for a way to
evenly maintain the right ratio of index vs shard.

Thanks!


On Wed, Jun 4, 2014 at 7:47 PM, Jack Krupansky 
wrote:

> How many documents was in that 20GB index?
>
> I'm skeptical that a 1 billion document shard "won't be a problem." I mean
> technically it is possible, but as you are already experiencing, it may
> take a long time and a very powerful machine to do so. 100 million (or 250
> million max) would be a more realistic goal. Even then, it depends on your
> doc size and machine size.
>
> The main point from the previous discussion is that although the technical
> hard limit for a Solr shard is 2G docs, from a practical perspective it is
> very difficult to get to that limit, not that indexing 1 billion docs on a
> single shard is "just fine"!
>
> As a general rule, if you want fast queries for high volume, strive to
> assure that your per-shard index fits entirely into the system memory
> available for OS caching of file system pages.
>
> In any case, a proof of concept implementation will tell you everything
> you need to know.
>
>
> -- Jack Krupansky
>
> -Original Message- From: Vineet Mishra
> Sent: Wednesday, June 4, 2014 2:45 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr maximum Optimal Index Size per Shard
>
>
> Thanks all for your response.
> I presume this conversation concludes that indexing around 1Billion
> documents per shard won't be a problem, as I have 10 Billion docs to index,
> so approx 10 shards with 1 Billion each should be fine with it and how
> about Memory, what size of RAM should be fine for this amount of data?
> Moreover what should be the indexing technique for this huge data set, as
> currently I am indexing with EmbeddedSolrServer but its going pathetically
> slow after some 20Gb of indexing. Comparatively SolrHttpPost was slow due
> to network delays and response but after this long running the indexing
> with EmbeddedSolrServer I am getting a different notion.
> Any good indexing technique for this huge dataset would be highly
> appreciated.
>
> Thanks again!
>
>
> On Wed, Jun 4, 2014 at 6:40 AM, rulinma  wrote:
>
>  mark.
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Solr-maximum-
>> Optimal-Index-Size-per-Shard-tp4139565p4139698.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>

Re: Solr maximum Optimal Index Size per Shard

[Vineet Mishra]

Hi Shawn,

Thanks for your response, wanted to clarify a few things.

*Does that mean for querying smoothly we need to have memory atleast equal
or greater to the size of index? As in my case the index size will be very
heavy(~2TB) and practically speaking that amount of memory is not possible.
Even If it goes to multiple shards, say around 10 Shards then also 200GB of
RAM will not be an feasible option.

*With CloudSolrServer can we specify which Shard the particular index
should go and reside, which I can do with EmbeddedSolrServer by indexing in
different directories and moving them to appropriate shard directories.

Thanks!



On Wed, Jun 4, 2014 at 12:43 PM, Shawn Heisey  wrote:

> On 6/4/2014 12:45 AM, Vineet Mishra wrote:
> > Thanks all for your response.
> > I presume this conversation concludes that indexing around 1Billion
> > documents per shard won't be a problem, as I have 10 Billion docs to
> index,
> > so approx 10 shards with 1 Billion each should be fine with it and how
> > about Memory, what size of RAM should be fine for this amount of data?
>
> Figure out the heap requirements of the operating system and every
> program on the machine (Solr especially).  Then you would add that
> number to the total size of the index data on the machine.  That is the
> ideal minimum RAM.
>
> http://wiki.apache.org/solr/SolrPerformanceProblems
>
> Unfortunately, if you are dealing with a huge index with billions of
> documents, it is likely to be prohibitively expensive to buy that much
> RAM.  If you are running Solr on Amazon's cloud, the cost for that much
> RAM would be astronomical.
>
> Exactly how much RAM would actually be required is very difficult to
> predict.  If you had only 25% of the ideal, your index might have
> perfectly acceptable performance, or it might not.  It might do fine
> under a light query load, but if you increase to 50 queries per second,
> performance may drop significantly ... or it might be good.  It's
> generally not possible to know how your hardware will perform until you
> actually build and use your index.
>
>
> http://searchhub.org/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
>
> A general rule of thumb for RAM that I have found to be useful is that
> if you've got less than half of the ideal memory size, you might have
> performance problems.
>
> > Moreover what should be the indexing technique for this huge data set, as
> > currently I am indexing with EmbeddedSolrServer but its going
> pathetically
> > slow after some 20Gb of indexing. Comparatively SolrHttpPost was slow due
> > to network delays and response but after this long running the indexing
> > with EmbeddedSolrServer I am getting a different notion.
> > Any good indexing technique for this huge dataset would be highly
> > appreciated.
>
> EmbeddedSolrServer is not recommended.  Run Solr in the traditional way
> with HTTP connectivity.  HTTP overhead on a LAN is usually quite small.
>  Solr is fully thread-safe, so you can have several indexing threads all
> going at the same time.
>
> Indexes at this scale should normally be built with SolrCloud, with
> enough servers so that each machine is only handling one shard replica.
>  The ideal indexing program would be written in Java, using
> CloudSolrServer.
>
> Thanks,
> Shawn
>
>

Date number to text

[O. Klein]

Is there a way to tokenize 12-6-2014 as 12 june 2014 and the other way around
as well?

Making it possible for people to search on text and numbers for dates?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Date-number-to-text-tp4140323.html
Sent from the Solr - User mailing list archive at Nabble.com.