Re: solr-map-reduce:jar:6.6.0

[Hrishikesh Gadre]

>>If we do need that functionality somehow, what is the recommended
approach?  Was it replaced by something?

AFAIK it was not replaced by anything on the Solr side. Kite sdk provides
identical functionality and it looks like it supports Solr 6.6. You may
want to try that out

https://github.com/kite-sdk/kite

Thanks
Hrishikesh


On Tue, Aug 29, 2017 at 3:28 PM, John Bickerstaff 
wrote:

> If we do need that functionality somehow, what is the recommended
> approach?  Was it replaced by something?
>
> On Tue, Aug 29, 2017 at 4:27 PM, John Bickerstaff <
> j...@johnbickerstaff.com>
> wrote:
>
> > A quick search of the code's imports suggests we aren't actually using
> it.
> > However I see it explicitly mentioned in the POM.
> >
> > I'll have to do more digging to be sure...
> >
> > On Tue, Aug 29, 2017 at 3:44 PM, Hrishikesh Gadre 
> > wrote:
> >
> >> Hi John,
> >>
> >> The map-reduce contrib module is removed as part of SOLR-9221
> >> . Are you using this
> >> contrib module?
> >>
> >> Thanks
> >> Hrishikesh
> >>
> >>
> >> On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff <
> >> j...@johnbickerstaff.com>
> >> wrote:
> >>
> >> > All,
> >> >
> >> > I'm looking at upgrading some java projects to solr and lucene 6.6.0.
> >> >
> >> > I get an error when trying to "mvn package" one of them - it
> complained
> >> > about the solr-map-reduce:jar:6.6.0 not being available.
> >> >
> >> > I went here:
> >> > https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
> >> >
> >> > And it looks like this hasn't been updated to 6.6...
> >> >
> >> > Is it going to be updated soon?  Am I misunderstanding something?
> >> >
> >>
> >
> >
>

Re: Spatial search with arbitrary rectangle?

[David Smiley]

Hi,

The "rectangular area" refers to a hypothetical map UI.  In this scenario,
the UI ought to communicate the lat-lon of each corner.  The geofilt and
bbox query parsers don't handle that; they only take a point and distance.

RE projections: You may or may not need to care depending on exactly what
you're doing.  Most people by far don't need to care, I've found.
Basically:  If geo="true" on the spatial field (the default), then you work
in decimal degrees latitude,longitude.  Point-distance queries (i.e.
circles) use spherical geometry.  When geo="false", the units are whatever
you want them to be (there is no transformation; it's up to you to
transform them if needed), and a point-distance (circle) query is on the 2D
plane.  Other shapes (rectangles, line strings, polygons) use 2D Euclidean
geometry no matter if geo=true or false.

BTW sorry for my delayed response; I was on vacation.

~ David

On Wed, Aug 23, 2017 at 11:21 AM Paweł Kordek 
wrote:

> Hi All
>
>
> I've been skimming through the spatial search docs and came across this
> section:
>
>
>
> https://lucene.apache.org/solr/guide/6_6/spatial-search.html#SpatialSearch-Filteringbyanarbitraryrectangle
>
>
> "Sometimes the spatial search requirement calls for finding everything in
> a rectangular area, such as the area covered by a map the user is looking
> at. For this case, geofilt and bbox won’t cut it. "
>
>
> I can't understand what is meant here by the "rectangular area". What is
> the coordinate system of this rectangle? If we talk about the map, don't we
> have to consider what is the projection? Any help will be much appreciated.
>
>
> Best regards
>
> Paweł
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Re: Download Sunplot for SQL/Streaming expr

[Joel Bernstein]

This is the github site https://github.com/sunplot/sunplot. You can ask
Michael about how to download.

Joel Bernstein
http://joelsolr.blogspot.com/

On Wed, Aug 23, 2017 at 2:43 PM, Susheel Kumar 
wrote:

> Hello,
>
> From where we can download Sunplot/setup to use SQL and streaming
> expressions?
>
> Thanks,
> Susheel
>

Re: solr 6.6.1: Lock held by this virtual machine

[Erick Erickson]

I need to look at this a bit.

If either of you have the time/inclination, I have some anecdotal
evidence that if you specify

5000

in solrconfig.xml you won't see this problem. This is not a proper fix
of the root issue, but if you test this and don't see the error it'd
be a great help in me tracking this down.

Erick

On Tue, Aug 29, 2017 at 3:37 PM, Shawn Heisey  wrote:
> On 8/26/2017 9:53 AM, Erick Erickson wrote:
>> Setting loadOnStartup=false won't work for you in the long run,
>> although it does provide something of a hint. Setting this to false
>> means the core at that location simply has its coreDescriptor read and
>> stashed away in memory. The first time you _use_ that core an attempt
>> will be made to load it and that should fail with the write.lock
>> problem.
>>
>> There is extensive locking of core loading to prevent two threads from
>> trying to open the same core at the same time, if it were
>> fundamentally broken you wouldn't be the only person seeing this error
>> I'd guess.
>
> I had originally thought that i had loadonStartup enabled, but on second
> glance, turns out that it was disabled on all my cores.
>
> I set it to true and restarted again, hoping that would get rid of the
> issue and we would have some concrete information about triggering it.
> It didn't help -- the same problem still happens.
>
> The cores named "s1live" and "spark5live" have the "error opening new
> searcher" message in the admin UI for this run.  I see these lines in
> the log for s1live:
>
> 2017-08-29 21:58:22.467 INFO  (coreLoadExecutor-6-thread-2) [   ]
> o.a.s.c.CoreContainer Creating SolrCore 's1live' using configuration
> from instancedir /index/solr6/data/cores/s1_0, trusted=true
> 2017-08-29 21:58:23.863 INFO  (qtp1394336709-212) [   x:s1live]
> o.a.s.c.CoreContainer Creating SolrCore 's1live' using configuration
> from instancedir /index/solr6/data/cores/s1_0, trusted=true
>
> The first one is the coreLoadExecutor thread, no real surprise there.
> The second one starts with qtp, which I think makes it a query thread.
>
> Through several restarts, I have never seen a "build" core have this
> problem, it's always live cores.  I have some aggregation cores that
> have shards parameters in the request handlers.  Only live cores are
> mentioned there, and all queries (including the every-five-seconds
> health check ping queries used by haproxy) utilize those aggregation
> cores.  No requests are typically sent to "build" cores unless a full
> index rebuild is underway, which is fairly rare.
>
> My best guess for what's gone wrong is that there is some kind of race
> condition between the time when a loading core creates its searcher and
> the time when the core is actually fully loaded, and if requests come in
> for that core during that time, Solr will try to initialize another new
> searcher, instead of returning the "still loading" message that I also
> commonly see during Solr startup.  It is possible that this race
> condition only happens with distributed queries, but I'm not sure about
> that part.
>
> This idea also accounts for the fact that it is different cores with the
> problem every time -- restart timing versus query timing will rarely
> ever match up perfectly.
>
> Here is the full startup log from Solr 6.6 for the most recent run,
> which contains the two log lines I quoted above:
>
> https://www.dropbox.com/s/k1b6g0ldp9vces2/solr6_6-startup.log?dl=0
>
> With confirmation that another user is having the same problem, I've
> opened an issue.
>
> https://issues.apache.org/jira/browse/SOLR-11297
>
> Thanks,
> Shawn
>

Re: solr-map-reduce:jar:6.6.0

[Erick Erickson]

This is now apparently maintained at http://kitesdk.org and was living
in both places.

unless you're using the MapReduceIndexerTool (a bulk indexing option
when running over HDFS) these contribs aren't needed.

Best,
Erick

On Tue, Aug 29, 2017 at 3:28 PM, John Bickerstaff
 wrote:
> If we do need that functionality somehow, what is the recommended
> approach?  Was it replaced by something?
>
> On Tue, Aug 29, 2017 at 4:27 PM, John Bickerstaff 
> wrote:
>
>> A quick search of the code's imports suggests we aren't actually using it.
>> However I see it explicitly mentioned in the POM.
>>
>> I'll have to do more digging to be sure...
>>
>> On Tue, Aug 29, 2017 at 3:44 PM, Hrishikesh Gadre 
>> wrote:
>>
>>> Hi John,
>>>
>>> The map-reduce contrib module is removed as part of SOLR-9221
>>> . Are you using this
>>> contrib module?
>>>
>>> Thanks
>>> Hrishikesh
>>>
>>>
>>> On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff <
>>> j...@johnbickerstaff.com>
>>> wrote:
>>>
>>> > All,
>>> >
>>> > I'm looking at upgrading some java projects to solr and lucene 6.6.0.
>>> >
>>> > I get an error when trying to "mvn package" one of them - it complained
>>> > about the solr-map-reduce:jar:6.6.0 not being available.
>>> >
>>> > I went here:
>>> > https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
>>> >
>>> > And it looks like this hasn't been updated to 6.6...
>>> >
>>> > Is it going to be updated soon?  Am I misunderstanding something?
>>> >
>>>
>>
>>

Re: solr 6.6.1: Lock held by this virtual machine

[Shawn Heisey]

On 8/26/2017 9:53 AM, Erick Erickson wrote:
> Setting loadOnStartup=false won't work for you in the long run,
> although it does provide something of a hint. Setting this to false
> means the core at that location simply has its coreDescriptor read and
> stashed away in memory. The first time you _use_ that core an attempt
> will be made to load it and that should fail with the write.lock
> problem.
>
> There is extensive locking of core loading to prevent two threads from
> trying to open the same core at the same time, if it were
> fundamentally broken you wouldn't be the only person seeing this error
> I'd guess.

I had originally thought that i had loadonStartup enabled, but on second
glance, turns out that it was disabled on all my cores.

I set it to true and restarted again, hoping that would get rid of the
issue and we would have some concrete information about triggering it. 
It didn't help -- the same problem still happens.

The cores named "s1live" and "spark5live" have the "error opening new
searcher" message in the admin UI for this run.  I see these lines in
the log for s1live:

2017-08-29 21:58:22.467 INFO  (coreLoadExecutor-6-thread-2) [   ]
o.a.s.c.CoreContainer Creating SolrCore 's1live' using configuration
from instancedir /index/solr6/data/cores/s1_0, trusted=true
2017-08-29 21:58:23.863 INFO  (qtp1394336709-212) [   x:s1live]
o.a.s.c.CoreContainer Creating SolrCore 's1live' using configuration
from instancedir /index/solr6/data/cores/s1_0, trusted=true

The first one is the coreLoadExecutor thread, no real surprise there. 
The second one starts with qtp, which I think makes it a query thread.

Through several restarts, I have never seen a "build" core have this
problem, it's always live cores.  I have some aggregation cores that
have shards parameters in the request handlers.  Only live cores are
mentioned there, and all queries (including the every-five-seconds
health check ping queries used by haproxy) utilize those aggregation
cores.  No requests are typically sent to "build" cores unless a full
index rebuild is underway, which is fairly rare.

My best guess for what's gone wrong is that there is some kind of race
condition between the time when a loading core creates its searcher and
the time when the core is actually fully loaded, and if requests come in
for that core during that time, Solr will try to initialize another new
searcher, instead of returning the "still loading" message that I also
commonly see during Solr startup.  It is possible that this race
condition only happens with distributed queries, but I'm not sure about
that part.

This idea also accounts for the fact that it is different cores with the
problem every time -- restart timing versus query timing will rarely
ever match up perfectly.

Here is the full startup log from Solr 6.6 for the most recent run,
which contains the two log lines I quoted above:

https://www.dropbox.com/s/k1b6g0ldp9vces2/solr6_6-startup.log?dl=0

With confirmation that another user is having the same problem, I've
opened an issue.

https://issues.apache.org/jira/browse/SOLR-11297

Thanks,
Shawn

Re: solr-map-reduce:jar:6.6.0

[John Bickerstaff]

If we do need that functionality somehow, what is the recommended
approach?  Was it replaced by something?

On Tue, Aug 29, 2017 at 4:27 PM, John Bickerstaff 
wrote:

> A quick search of the code's imports suggests we aren't actually using it.
> However I see it explicitly mentioned in the POM.
>
> I'll have to do more digging to be sure...
>
> On Tue, Aug 29, 2017 at 3:44 PM, Hrishikesh Gadre 
> wrote:
>
>> Hi John,
>>
>> The map-reduce contrib module is removed as part of SOLR-9221
>> . Are you using this
>> contrib module?
>>
>> Thanks
>> Hrishikesh
>>
>>
>> On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff <
>> j...@johnbickerstaff.com>
>> wrote:
>>
>> > All,
>> >
>> > I'm looking at upgrading some java projects to solr and lucene 6.6.0.
>> >
>> > I get an error when trying to "mvn package" one of them - it complained
>> > about the solr-map-reduce:jar:6.6.0 not being available.
>> >
>> > I went here:
>> > https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
>> >
>> > And it looks like this hasn't been updated to 6.6...
>> >
>> > Is it going to be updated soon?  Am I misunderstanding something?
>> >
>>
>
>

Re: solr-map-reduce:jar:6.6.0

[John Bickerstaff]

A quick search of the code's imports suggests we aren't actually using it.
However I see it explicitly mentioned in the POM.

I'll have to do more digging to be sure...

On Tue, Aug 29, 2017 at 3:44 PM, Hrishikesh Gadre 
wrote:

> Hi John,
>
> The map-reduce contrib module is removed as part of SOLR-9221
> . Are you using this
> contrib module?
>
> Thanks
> Hrishikesh
>
>
> On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff <
> j...@johnbickerstaff.com>
> wrote:
>
> > All,
> >
> > I'm looking at upgrading some java projects to solr and lucene 6.6.0.
> >
> > I get an error when trying to "mvn package" one of them - it complained
> > about the solr-map-reduce:jar:6.6.0 not being available.
> >
> > I went here:
> > https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
> >
> > And it looks like this hasn't been updated to 6.6...
> >
> > Is it going to be updated soon?  Am I misunderstanding something?
> >
>

Re: solr-map-reduce:jar:6.6.0

[Hrishikesh Gadre]

Hi John,

The map-reduce contrib module is removed as part of SOLR-9221
. Are you using this
contrib module?

Thanks
Hrishikesh


On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff 
wrote:

> All,
>
> I'm looking at upgrading some java projects to solr and lucene 6.6.0.
>
> I get an error when trying to "mvn package" one of them - it complained
> about the solr-map-reduce:jar:6.6.0 not being available.
>
> I went here:
> https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
>
> And it looks like this hasn't been updated to 6.6...
>
> Is it going to be updated soon?  Am I misunderstanding something?
>

solr-map-reduce:jar:6.6.0

[John Bickerstaff]

All,

I'm looking at upgrading some java projects to solr and lucene 6.6.0.

I get an error when trying to "mvn package" one of them - it complained
about the solr-map-reduce:jar:6.6.0 not being available.

I went here:
https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce

And it looks like this hasn't been updated to 6.6...

Is it going to be updated soon?  Am I misunderstanding something?

Re: solr 6.6.1: Lock held by this virtual machine

[Karthik Ramachandran]

I am also seeing 2 threads loading the cores, I am using Solr 6.6.0.

On Sat, Aug 26, 2017 at 11:53 AM, Erick Erickson 
wrote:

> Setting loadOnStartup=false won't work for you in the long run,
> although it does provide something of a hint. Setting this to false
> means the core at that location simply has its coreDescriptor read and
> stashed away in memory. The first time you _use_ that core an attempt
> will be made to load it and that should fail with the write.lock
> problem.
>
> There is extensive locking of core loading to prevent two threads from
> trying to open the same core at the same time, if it were
> fundamentally broken you wouldn't be the only person seeing this error
> I'd guess.
>
> I see several errors in the stack trace "... is not multivalued and
> destination for multiple copyFields". I do wonder if there's something
> weird there with the error checking, that's a long shot. If you change
> the fields identified to mulitValued=true does this still occur? I
> absolutely expect it to, grasping at straws here.
>
> I do see that corePropertiesLocator only finds a single
> core.properties file so that's as I'd expect.
>
> So this is puzzling, all the usual suspects seem to be eliminated. I
> cant get over the fact that you copied things around though, that
> usually means there's something we're not seeing that changed at the
> same time although it's not obvious what that would be.
>
> Personally I'd blow the core away, start Solr and work back up from there.
>
> It looks like both times the core is being opened it's from
> "coreLoadExecutor-6-thread-1" so it looks like it's the same thread.
> So I'm pretty clueless.
>
> Best,
> Erick
>
> On Fri, Aug 25, 2017 at 11:22 PM, Nawab Zada Asad Iqbal
>  wrote:
> > Hi Erick,
> >
> > I spent some more time on this and found that if I modify
> 'core.properties'
> > to contain the following values (my core.propreties file is empty
> otherwise
> > and only being used for shard discovery), then the solr server works
> fine.
> >
> > loadOnStartup=false
> > transient=false
> >
> > The fact is that shards are being loaded more than one time at the time
> of
> > startup. There is one possible cause (which I couldn't confirm), that if
> > some ping request or query arrives while the shard is loading and the
> > transient cache hasn't been initialized yet, will solr try to load the
> > core? What if the shard is already being loaded (due to loadOnStartup)
> but
> > not in the cache yet? Can that cause the problem which I am seeing. My
> test
> > machine, constantly gets ping traffic from an haproxy (which i don't have
> > control of), so I cannot test this hypothesis.
> >
> > However, on another machine with identical setup (except without haproxy
> > traffic), I was able to make start and use (index 100s GB and queries for
> > many hours) this solr version (6.6.1) without any problem.
> >
> >
> >
> > Thanks
> > Nawab
> >
> >
> >
> >
> > On Fri, Aug 25, 2017 at 3:38 PM, Nawab Zada Asad Iqbal  >
> > wrote:
> >
> >> Thanks Erik
> >> I expected that but it is really not the case . I have only one core per
> >> solr installation . Though i run 3 solr processes on each host.
> >>
> >> If you see the thread names they are :coreLoadExecutor and
> qtp761960786-31
> >> . If it was the case of two core pointing to one index (though it does
> not
> >> look like one based on my verification), then I expect to see two
> threads
> >> of
> >> coreLoadExecutor trying to load the core twice.
> >> Does the thread name prefix give any hint ?
> >>
> >>
> >> Nawab
> >>
> >>
> >> On Fri, Aug 25, 2017 at 1:55 PM Erick Erickson  >
> >> wrote:
> >>
> >>> In that case you probably have two different cores pointing to the
> >>> _same_ data directory. Examine your core.properties files and see if
> >>> any dataDir variables are set
> >>>
> >>> Best,
> >>> Erick
> >>>
> >>> On Fri, Aug 25, 2017 at 1:12 PM, Nawab Zada Asad Iqbal <
> khi...@gmail.com>
> >>> wrote:
> >>> > Ok, after looking at the logs for some more time, i found that there
> are
> >>> > more than one threads trying to load the core at startup time. This
> >>> doesn't
> >>> > make sense to me, is it configurable? Is there any reason why this is
> >>> even
> >>> > an option?
> >>> >
> >>> >
> >>> > Aug 25, 2017 12:04:37 PM INFO  (main) [   ] o.e.j.s.Server
> >>> > jetty-9.3.14.v20161028
> >>> > Aug 25, 2017 12:04:38 PM INFO  (main) [   ]
> o.a.s.s.SolrDispatchFilter
> >>> > ___  _   Welcome to Apache Solr™ version 6.6.1-SNAPSHOT
> >>> > 1a390a91b5b658150478e6fc3c43381bedd3c6d3 - niqbal - 2017-08-09
> 10:31:27
> >>> > Aug 25, 2017 12:04:38 PM INFO  (main) [   ]
> o.a.s.s.SolrDispatchFilter /
> >>> > __| ___| |_ _   Starting in standalone mode on port 8984
> >>> > Aug 25, 2017 12:04:38 PM INFO  (main) [   ]
> o.a.s.s.SolrDispatchFilter
> >>> \__
> >>> > \/ _ \ | '_|  Install dir: /local/bin/solr6/latest_solr
> >>> > Aug 25, 

Re: Solr learning to rank features question

[Brian Yee]

Thank you Diego! This works with some small changes. My feature ended up
looking like this:

{
  "name" : "FeatureA",
  "store" : "commonFeatureStore",
  "class" : "org.apache.solr.ltr.feature.SolrFeature",
  "params" : {
"q" : "{!func}if(gt(ms(CutoffDate,NOW),0),exists(query({!v=
PreZones:${zone}})),exists(query({!v=PostZones:${zone}})))"
  }
},

On Tue, Aug 29, 2017 at 12:32 PM, Diego Ceccarelli <
diego.ceccare...@gmail.com> wrote:

> Hi Brian,
>
> The plugin doesn't allow you to express multiple function queries in the
> same feature. Maybe in this case you can express both the tw queries in one
> unique function query, using the if function.
>
> Something like:
>
> "fq":"if(gt(ms(NOW,mydatefield),0,query(PreCutOffZones:${zone}), query(
> PostCutOffZones:${zone}))"
>
>
> (Function query documentation here:
> https://cwiki.apache.org/confluence/display/solr/Function+Queries)
>
> Let me know if it works.
>
> Cheers,
> Diego
>
>
>
>
>
>
>
> On Tue, Aug 29, 2017 at 5:47 PM, Brian Yee  wrote:
>
> > Hello,
> >
> > I have a somewhat complex solr filter query that I am hoping to turn
> into a
> > feature for learning to rank, but I am not sure if it's possible. I would
> > like to OR two fq together for one of my features. I have a cutoff date
> and
> > I need to check one field if NOW is before the cutoff, and a different
> > field if NOW is after the cutoff. Is this possible?
> >
> > Something to the effect of this:
> > {
> >   "name" : "FeatureA",
> >   "store" : "commonFeatureStore",
> >   "class" : "org.apache.solr.ltr.feature.SolrFeature",
> >   "params" : {
> > "fq" : [
> >   "{!frange l=NOW}CutOffDate",
> >   "{!term f=PreCutOffZones}${zone}"
> > ]
> > OR
> > "fq" : [
> >   "{!frange u=NOW}CutOffDate",
> >   "{!term f=PostCutOffZones}${zone}"
> > ]
> >   }
> > },
> >
> > Thank you!
> > Brian Yee
> >
>

Re: Indexed=false for a field,but still able to search on field.

[AshB]

Hi,

Thanks ,got this issue is happening because of docValues=true.

Please elaborate on "full table scan search"

Regards
Ashish



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-tp4352338p4352599.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr cloud in kubernetes

[Björn Häuser]

Hi Lars,

sorry, external traffic is a wrong name. 

Basically all traffic to Solr goes through a k8s service which uses all solr 
pods as endpoints. 
Additionally we use another service for intra cluster communication.

We do not use pod affinity.

Feel free to ask more question if something is unclear.

Regards
Björn

> On 28. Aug 2017, at 22:28, Lars Karlsson  
> wrote:
> 
> Thanks Björn for the detailed information, just wanted to understand:
> 
> When you say separate service for external traffic, does this mean a home
> brewed one that proxy solr queries?
> 
> And what is the difference between the above and "solr discovery"?
> 
> Do you specify pod anti affinity for solr hosts?
> 
> Regards
> Lars
> 
> On Sat, 26 Aug 2017 at 13:19, Björn Häuser  wrote:
> 
>> Hi Lars,
>> 
>> we are running Solr in kubernetes and after some initial problems we are
>> running quite stable now.
>> 
>> Here is the setup we choose for solr:
>> 
>> - separate service for external traffic to solr (called “solr”)
>> - statefulset for solr with 3 replicas with another service (called
>> “solr-discovery”)
>> 
>> We set the SOLR_HOST (which is used for intra cluster communication) to
>> the pod inside the statefulset
>> (solr-0.solr-discovery.default.svc.cluster.local. This ensures that on solr
>> pod restart the intra cluster communication still continues to work. In the
>> beginning we used the IP address of the pod, this caused problems when
>> restarting pods, they tried to talk with the old ip addresses.
>> 
>> Zookeeper inside kubernetes is a different story. Use the latest version
>> of kubernetes, because old versions never reresolved dns names. For
>> connecting to zookeeper we use the same approach, one service-ip for all
>> pods. The statefulset works again with a different service name.
>> 
>> The problems we are currently facing:
>> 
>> - Client timeouts whenever a solr pod stops and starts again, we currently
>> try to solve this with better readiness probes, no success yet
>> - Sometimes solr collections do not recover completely after a pod restart
>> and we manually have to force recovery, still not investigated fully
>> 
>> Hope this helps you!
>> 
>> Thanks
>> Björn
>> 
>>> On 26. Aug 2017, at 12:08, Lars Karlsson 
>> wrote:
>>> 
>>> Hi, I wanted to hear if anyone successfully got solr cloud running on
>>> kubernetes and can share challenges and limitations.
>>> 
>>> Can't find much uptodate github projects, would be great if you can point
>>> out blogposts or other useful links.
>>> 
>>> Thanks in advance.
>> 
>> 

Re: Searching With UTF-8

[Diego Ceccarelli]

Hello Lawrence,
Which type did you use in the solr schema for your fields?

Cheers,
Diego


On Tue, Aug 29, 2017 at 5:34 PM, Elitzer, Lawrence <
lelit...@lgsinnovations.com> wrote:

> Hello!
>
>
>
> It seems I can correctly import (with DIH) UTF-8 characters such as J but
> I am unable to search on the fields containing the UTF-8 data. I have tried
> from the Solr admin backend to send just a J and even URL encode it in
> the q parameter I am specifying. How would I go about searching UTF-8 data
> using Solr?
>
>
>
> Thanks!
>
> Lawrence
>
>
>
> *Lawrence Elitzer | Associate Software Engineer*
> *LGS Innovations*
> *lelit...@lgsinnovations.com *
> *www.lgsinnovations.com* 
>
> [image: cid:image004.png@01D16265.B8955B70]
>
>
>

Re: Solr learning to rank features question

[Diego Ceccarelli]

Hi Brian,

The plugin doesn't allow you to express multiple function queries in the
same feature. Maybe in this case you can express both the tw queries in one
unique function query, using the if function.

Something like:

"fq":"if(gt(ms(NOW,mydatefield),0,query(PreCutOffZones:${zone}), query(
PostCutOffZones:${zone}))"


(Function query documentation here:
https://cwiki.apache.org/confluence/display/solr/Function+Queries)

Let me know if it works.

Cheers,
Diego







On Tue, Aug 29, 2017 at 5:47 PM, Brian Yee  wrote:

> Hello,
>
> I have a somewhat complex solr filter query that I am hoping to turn into a
> feature for learning to rank, but I am not sure if it's possible. I would
> like to OR two fq together for one of my features. I have a cutoff date and
> I need to check one field if NOW is before the cutoff, and a different
> field if NOW is after the cutoff. Is this possible?
>
> Something to the effect of this:
> {
>   "name" : "FeatureA",
>   "store" : "commonFeatureStore",
>   "class" : "org.apache.solr.ltr.feature.SolrFeature",
>   "params" : {
> "fq" : [
>   "{!frange l=NOW}CutOffDate",
>   "{!term f=PreCutOffZones}${zone}"
> ]
> OR
> "fq" : [
>   "{!frange u=NOW}CutOffDate",
>   "{!term f=PostCutOffZones}${zone}"
> ]
>   }
> },
>
> Thank you!
> Brian Yee
>

Searching With UTF-8

[Elitzer, Lawrence]

Hello!

 

It seems I can correctly import (with DIH) UTF-8 characters such as J but I
am unable to search on the fields containing the UTF-8 data. I have tried
from the Solr admin backend to send just a J and even URL encode it in the q
parameter I am specifying. How would I go about searching UTF-8 data using
Solr?

 

Thanks!

Lawrence

 

Lawrence Elitzer | Associate Software Engineer
LGS Innovations
  lelit...@lgsinnovations.com
  www.lgsinnovations.com

cid:image004.png@01D16265.B8955B70

 



smime.p7s
Description: S/MIME cryptographic signature

Solr learning to rank features question

[Brian Yee]

Hello,

I have a somewhat complex solr filter query that I am hoping to turn into a
feature for learning to rank, but I am not sure if it's possible. I would
like to OR two fq together for one of my features. I have a cutoff date and
I need to check one field if NOW is before the cutoff, and a different
field if NOW is after the cutoff. Is this possible?

Something to the effect of this:
{
  "name" : "FeatureA",
  "store" : "commonFeatureStore",
  "class" : "org.apache.solr.ltr.feature.SolrFeature",
  "params" : {
"fq" : [
  "{!frange l=NOW}CutOffDate",
  "{!term f=PreCutOffZones}${zone}"
]
OR
"fq" : [
  "{!frange u=NOW}CutOffDate",
  "{!term f=PostCutOffZones}${zone}"
]
  }
},

Thank you!
Brian Yee

Fwd: Solr learning to rank features question

[Brian Yee]

Hello,

I have a somewhat complex solr filter query that I am hoping to turn into a
feature for learning to rank, but I am not sure if it's possible. I would
like to OR two fq together for one of my features. I have a cutoff date and
I need to check one field if NOW is before the cutoff, and a different
field if NOW is after the cutoff. Is this possible?

Something to the effect of this:
{
  "name" : "FeatureA",
  "store" : "commonFeatureStore",
  "class" : "org.apache.solr.ltr.feature.SolrFeature",
  "params" : {
"fq" : [
  "{!frange l=NOW}CutOffDate",
  "{!term f=PreCutOffZones}${zone}"
]
OR
"fq" : [
  "{!frange u=NOW}CutOffDate",
  "{!term f=PostCutOffZones}${zone}"
]
  }
},

Thank you!
Brian Yee

Re: ClassCastException when using HighFrequencyDictionaryFactory with SearchComponent

[Craig Smiles]

Woops. Thanks Rick, that was it. I'm so used to typing "str" that it became
white noise.

On 29 August 2017 at 16:03, Rick Leir  wrote:

> Craig,
> Just a guess, but your threshold is in a string field. Cheers -- Rick
>
> On August 29, 2017 7:37:56 AM MDT, Craig Smiles 
> wrote:
> >Hi,
> >
> >I've been trying to get a suggester working with the
> >HighFrequencyDictionaryFactory but when setting the threshold I'm
> >getting a
> >ClassCastException when the dictionary is built.
> >
> >Could someone please advise?
> >
> >To reproduce:
> >
> >1. Download solr 6.6
> >
> >2. Replace the SuggestComponent within the techproducts' solrconfig.xml
> >with:
> >  
> >
> >  mySuggester
> >  FuzzyLookupFactory
> >  HighFrequencyDictionaryFactory
> > 0.005
> >  cat
> >  string
> >  false
> >
> >  
> >
> >3. Run the techproducts example: "solr start -e techproducts"
> >
> >4. Visit
> >http://localhost:8983/solr/techproducts/suggest?suggest=
> true=true=mySuggester=elec
> >
> >I then see the following response:
> >
> >
> >
> >500
> >5
> >
> >
> >java.lang.String cannot be cast to
> >java.lang.Float
> >
> >java.lang.ClassCastException: java.lang.String cannot be cast to
> >java.lang.Float at
> >org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory.create(
> HighFrequencyDictionaryFactory.java:47)
> >at
> >org.apache.solr.spelling.suggest.SolrSuggester.build(
> SolrSuggester.java:179)
> >at
> >org.apache.solr.handler.component.SuggestComponent.
> prepare(SuggestComponent.java:179)
> >at
> >org.apache.solr.handler.component.SearchHandler.handleRequestBody(
> SearchHandler.java:270)
> >at
> >org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:173)
> >at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at
> >org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at
> >org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at
> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:361)
> >at
> >org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:305)
> >at
> >org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1691)
> >at
> >org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:582)
> >at
> >org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> >at
> >org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)
> >at
> >org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:226)
> >at
> >org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1180)
> >at
> >org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> >at
> >org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> >at
> >org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1112)
> >at
> >org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> >at
> >org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:213)
> >at
> >org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:119)
> >at
> >org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)
> >at
> >org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> RewriteHandler.java:335)
> >at
> >org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)
> >at org.eclipse.jetty.server.Server.handle(Server.java:534) at
> >org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at
> >org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:251)
> >at
> >org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> AbstractConnection.java:273)
> >at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at
> >org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> SelectChannelEndPoint.java:93)
> >at
> >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> executeProduceConsume(ExecuteProduceConsume.java:303)
> >at
> >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> produceConsume(ExecuteProduceConsume.java:148)
> >at
> >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
> ExecuteProduceConsume.java:136)
> >at
> >org.eclipse.jetty.util.thread.QueuedThreadPool.
> runJob(QueuedThreadPool.java:671)
> >at
> >org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:589)
> >at java.lang.Thread.run(Thread.java:745)
> >
> >500
> >
> >
> >
> >Thanks,
> >Craig
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com

Re: ClassCastException when using HighFrequencyDictionaryFactory with SearchComponent

[Rick Leir]

Craig,
Just a guess, but your threshold is in a string field. Cheers -- Rick

On August 29, 2017 7:37:56 AM MDT, Craig Smiles  wrote:
>Hi,
>
>I've been trying to get a suggester working with the
>HighFrequencyDictionaryFactory but when setting the threshold I'm
>getting a
>ClassCastException when the dictionary is built.
>
>Could someone please advise?
>
>To reproduce:
>
>1. Download solr 6.6
>
>2. Replace the SuggestComponent within the techproducts' solrconfig.xml
>with:
>  
>
>  mySuggester
>  FuzzyLookupFactory
>  HighFrequencyDictionaryFactory
> 0.005
>  cat
>  string
>  false
>
>  
>
>3. Run the techproducts example: "solr start -e techproducts"
>
>4. Visit
>http://localhost:8983/solr/techproducts/suggest?suggest=true=true=mySuggester=elec
>
>I then see the following response:
>
>
>
>500
>5
>
>
>java.lang.String cannot be cast to
>java.lang.Float
>
>java.lang.ClassCastException: java.lang.String cannot be cast to
>java.lang.Float at
>org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory.create(HighFrequencyDictionaryFactory.java:47)
>at
>org.apache.solr.spelling.suggest.SolrSuggester.build(SolrSuggester.java:179)
>at
>org.apache.solr.handler.component.SuggestComponent.prepare(SuggestComponent.java:179)
>at
>org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:270)
>at
>org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
>at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at
>org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at
>org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at
>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
>at
>org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
>at
>org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
>at
>org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
>at
>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>at
>org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>at
>org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>at
>org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>at
>org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>at
>org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>at
>org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
>at
>org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>at
>org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>at
>org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>at
>org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>at
>org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>at
>org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>at org.eclipse.jetty.server.Server.handle(Server.java:534) at
>org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at
>org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
>at
>org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at
>org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>at
>org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
>at
>org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
>at
>org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
>at
>org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
>at
>org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
>at java.lang.Thread.run(Thread.java:745)
>
>500
>
>
>
>Thanks,
>Craig

-- 
Sorry for being brief. Alternate email is rickleir at yahoo dot com 

Re: Search by similarity?

[Josh Lincoln]

I reviewed the dismax docs and it doesn't support the fieldname:term
portion of the lucene syntax.
To restrict a search to a field and use mm you can either
A) use edismax exactly as you're currently trying to use dismax
B) use dismax, with the following changes
* remove the title: portion of the query and just pass
q="title-123123123-end"
* set qf=title

On Tue, Aug 29, 2017 at 10:25 AM Josh Lincoln 
wrote:

> Darko,
> Can you use edismax instead?
>
> When using dismax, solr is parsing the title field as if it's a query
> term. E.g. the query seems to be interpreted as
> title "title-123123123-end"
> (note the lack of a colon)...which results in querying all your qf fields
> for both "title" and "title-123123123-end"
> I haven't used dismax in a very long time, so I don't know if this is
> intentional, but it's not what I expected.
>
> I'm able to reproduce the issue in 6.4.2 using the default techproducts
> Notice that in the below the parsedquery expands to both text:title and
> text:name (df=text)
> http://localhost:8983/solr/techproducts/select?indent=on=title
> :"name"=json=true=dismax
> rawquerystring: "title:"name"",
> querystring: "title:"name"",
> parsedquery: "(+(DisjunctionMaxQuery(((text:title)^1.0))
> DisjunctionMaxQuery(((text:name)^1.0))) ())/no_coord",
> parsedquery_toString: "+(((text:title)^1.0) ((text:name)^1.0)) ()"
>
> But it's not an issue if you use edismax
> http://localhost:8983/solr/techproducts/select?indent=on=title
> :"name"=json=true=edismax
> rawquerystring: "title:"name"",
> querystring: "title:"name"",
> parsedquery: "(+title:name)/no_coord",
> parsedquery_toString: "+title:name",
>
>
>
> On Tue, Aug 29, 2017 at 8:44 AM Darko Todoric  wrote:
>
>> Hi Erick,
>>
>> "debug":{ "rawquerystring":"title:\"title-123123123-end\"",
>> "querystring":"title:\"title-123123123-end\"",
>> "parsedquery":"(+(DisjunctionMaxQuery(((author_full:title)^7.0 |
>> (abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 |
>> (authors:title)^4.0 | (doi:title:)^1.0))
>> DisjunctionMaxQuery(((author_full:\"title 123123123 end\"~1)^7.0 |
>> (abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl 123123123
>> end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 |
>> (authors:\"title 123123123 end\"~1)^4.0 |
>> (doi:title-123123123-end)^1.0)))~1 ())/no_coord",
>> "parsedquery_toString":"+author_full:title)^7.0 |
>> (abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 |
>> (authors:title)^4.0 | (doi:title:)^1.0) ((author_full:\"title 123123123
>> end\"~1)^7.0 | (abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl
>> 123123123 end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 |
>> (authors:\"title 123123123 end\"~1)^4.0 |
>> (doi:title-123123123-end)^1.0))~1) ()", "explain":{ "23251":"\n16.848969
>> = sum of:\n 16.848969 = sum of:\n 16.848969 = max of:\n 16.848969 =
>> weight(abstract:titl in 23194) [], result of:\n 16.848969 =
>> score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 2.0 = boost\n
>> 5.503748 = idf(docFreq=74, docCount=18297)\n 1.5306814 = tfNorm,
>> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
>> parameter b\n 186.49593 = avgFieldLength\n 28.45 = fieldLength\n
>> 3.816711E-5 = weight(title:titl in 23194) [], result of:\n 3.816711E-5 =
>> score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
>> 1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm,
>> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
>> parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n",
>> "20495":"\n16.169483 = sum of:\n 16.169483 = sum of:\n 16.169483 = max
>> of:\n 16.169483 = weight(abstract:titl in 20489) [], result of:\n
>> 16.169483 = score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n
>> 2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.468952 =
>> tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75
>> = parameter b\n 186.49593 = avgFieldLength\n 40.96 = fieldLength\n
>> 3.816711E-5 = weight(title:titl in 20489) [], result of:\n 3.816711E-5 =
>> score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
>> 1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm,
>> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
>> parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n",
>> "28227":"\n15.670726 = sum of:\n 15.670726 = sum of:\n 15.670726 = max
>> of:\n 15.670726 = weight(abstract:titl in 28156) [], result of:\n
>> 15.670726 = score(doc=28156,freq=2.0 = termFreq=2.0\n), product of:\n
>> 2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.4236413 =
>> tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 = parameter k1\n 0.75
>> = parameter b\n 186.49593 = avgFieldLength\n 163.84 = fieldLength\n
>> 3.816711E-5 = weight(title:titl in 28156) [], result of:\n 3.816711E-5 =
>> score(doc=28156,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
>> 1.4457239E-5 = idf(docFreq=34584, 

Re: Search by similarity?

[Josh Lincoln]

Darko,
Can you use edismax instead?

When using dismax, solr is parsing the title field as if it's a query term.
E.g. the query seems to be interpreted as
title "title-123123123-end"
(note the lack of a colon)...which results in querying all your qf fields
for both "title" and "title-123123123-end"
I haven't used dismax in a very long time, so I don't know if this is
intentional, but it's not what I expected.

I'm able to reproduce the issue in 6.4.2 using the default techproducts
Notice that in the below the parsedquery expands to both text:title and
text:name (df=text)
http://localhost:8983/solr/techproducts/select?indent=on=title
:"name"=json=true=dismax
rawquerystring: "title:"name"",
querystring: "title:"name"",
parsedquery: "(+(DisjunctionMaxQuery(((text:title)^1.0))
DisjunctionMaxQuery(((text:name)^1.0))) ())/no_coord",
parsedquery_toString: "+(((text:title)^1.0) ((text:name)^1.0)) ()"

But it's not an issue if you use edismax
http://localhost:8983/solr/techproducts/select?indent=on=title
:"name"=json=true=edismax
rawquerystring: "title:"name"",
querystring: "title:"name"",
parsedquery: "(+title:name)/no_coord",
parsedquery_toString: "+title:name",



On Tue, Aug 29, 2017 at 8:44 AM Darko Todoric  wrote:

> Hi Erick,
>
> "debug":{ "rawquerystring":"title:\"title-123123123-end\"",
> "querystring":"title:\"title-123123123-end\"",
> "parsedquery":"(+(DisjunctionMaxQuery(((author_full:title)^7.0 |
> (abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 |
> (authors:title)^4.0 | (doi:title:)^1.0))
> DisjunctionMaxQuery(((author_full:\"title 123123123 end\"~1)^7.0 |
> (abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl 123123123
> end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 |
> (authors:\"title 123123123 end\"~1)^4.0 |
> (doi:title-123123123-end)^1.0)))~1 ())/no_coord",
> "parsedquery_toString":"+author_full:title)^7.0 |
> (abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 |
> (authors:title)^4.0 | (doi:title:)^1.0) ((author_full:\"title 123123123
> end\"~1)^7.0 | (abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl
> 123123123 end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 |
> (authors:\"title 123123123 end\"~1)^4.0 |
> (doi:title-123123123-end)^1.0))~1) ()", "explain":{ "23251":"\n16.848969
> = sum of:\n 16.848969 = sum of:\n 16.848969 = max of:\n 16.848969 =
> weight(abstract:titl in 23194) [], result of:\n 16.848969 =
> score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 2.0 = boost\n
> 5.503748 = idf(docFreq=74, docCount=18297)\n 1.5306814 = tfNorm,
> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
> parameter b\n 186.49593 = avgFieldLength\n 28.45 = fieldLength\n
> 3.816711E-5 = weight(title:titl in 23194) [], result of:\n 3.816711E-5 =
> score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
> 1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm,
> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
> parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n",
> "20495":"\n16.169483 = sum of:\n 16.169483 = sum of:\n 16.169483 = max
> of:\n 16.169483 = weight(abstract:titl in 20489) [], result of:\n
> 16.169483 = score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n
> 2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.468952 =
> tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75
> = parameter b\n 186.49593 = avgFieldLength\n 40.96 = fieldLength\n
> 3.816711E-5 = weight(title:titl in 20489) [], result of:\n 3.816711E-5 =
> score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
> 1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm,
> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
> parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n",
> "28227":"\n15.670726 = sum of:\n 15.670726 = sum of:\n 15.670726 = max
> of:\n 15.670726 = weight(abstract:titl in 28156) [], result of:\n
> 15.670726 = score(doc=28156,freq=2.0 = termFreq=2.0\n), product of:\n
> 2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.4236413 =
> tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 = parameter k1\n 0.75
> = parameter b\n 186.49593 = avgFieldLength\n 163.84 = fieldLength\n
> 3.816711E-5 = weight(title:titl in 28156) [], result of:\n 3.816711E-5 =
> score(doc=28156,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n
> 1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm,
> computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 =
> parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n",
> "20375":"\n15.052014 = sum of:\n 15.052014 = sum of:\n 15.052014 = max
> of:\n 15.052014 = weight(abstract:titl in 20369) [], result of:\n
> 15.052014 = score(doc=20369,freq=1.0 = termFreq=1.0\n), product of:\n
> 2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.3674331 =
> tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75
> = parameter b\n 186.49593 = 

ClassCastException when using HighFrequencyDictionaryFactory with SearchComponent

[Craig Smiles]

Hi,

I've been trying to get a suggester working with the
HighFrequencyDictionaryFactory but when setting the threshold I'm getting a
ClassCastException when the dictionary is built.

Could someone please advise?

To reproduce:

1. Download solr 6.6

2. Replace the SuggestComponent within the techproducts' solrconfig.xml
with:
  

  mySuggester
  FuzzyLookupFactory
  HighFrequencyDictionaryFactory
 0.005
  cat
  string
  false

  

3. Run the techproducts example: "solr start -e techproducts"

4. Visit
http://localhost:8983/solr/techproducts/suggest?suggest=true=true=mySuggester=elec

I then see the following response:



500
5


java.lang.String cannot be cast to java.lang.Float

java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Float at
org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory.create(HighFrequencyDictionaryFactory.java:47)
at
org.apache.solr.spelling.suggest.SolrSuggester.build(SolrSuggester.java:179)
at
org.apache.solr.handler.component.SuggestComponent.prepare(SuggestComponent.java:179)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:270)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534) at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)

500



Thanks,
Craig

Re: Solr client

[Susheel Kumar]

Looks good, Ganesh.  I do not see SolrJ explicitly mentioned in solr-client
page.

On Tue, Aug 29, 2017 at 2:28 AM, Aditya 
wrote:

> Hi
>
> I am aggregating open source solr client libraries across all languages.
> Below are the links. Very few projects are currently active. Most of them
> are last updated few years back. Please provide me pointers, if i missed
> any solr client library.
>
> http://www.findbestopensource.com/tagged/solr-client
> http://www.findbestopensource.com/tagged/solr-gui
>
>
> Regards
> Ganesh
>
> PS: The website http://www.findbestopensource.com search is powered by
> Solr.
>

Re: Search by similarity?

[Darko Todoric]

Hi Erick,

"debug":{ "rawquerystring":"title:\"title-123123123-end\"", 
"querystring":"title:\"title-123123123-end\"", 
"parsedquery":"(+(DisjunctionMaxQuery(((author_full:title)^7.0 | 
(abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 | 
(authors:title)^4.0 | (doi:title:)^1.0)) 
DisjunctionMaxQuery(((author_full:\"title 123123123 end\"~1)^7.0 | 
(abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl 123123123 
end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 | 
(authors:\"title 123123123 end\"~1)^4.0 | 
(doi:title-123123123-end)^1.0)))~1 ())/no_coord", 
"parsedquery_toString":"+author_full:title)^7.0 | 
(abstract:titl)^2.0 | (title:titl)^3.0 | (keywords:titl)^5.0 | 
(authors:title)^4.0 | (doi:title:)^1.0) ((author_full:\"title 123123123 
end\"~1)^7.0 | (abstract:\"titl 123123123 end\"~1)^2.0 | (title:\"titl 
123123123 end\"~1)^3.0 | (keywords:\"titl 123123123 end\"~1)^5.0 | 
(authors:\"title 123123123 end\"~1)^4.0 | 
(doi:title-123123123-end)^1.0))~1) ()", "explain":{ "23251":"\n16.848969 
= sum of:\n 16.848969 = sum of:\n 16.848969 = max of:\n 16.848969 = 
weight(abstract:titl in 23194) [], result of:\n 16.848969 = 
score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 2.0 = boost\n 
5.503748 = idf(docFreq=74, docCount=18297)\n 1.5306814 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 186.49593 = avgFieldLength\n 28.45 = fieldLength\n 
3.816711E-5 = weight(title:titl in 23194) [], result of:\n 3.816711E-5 = 
score(doc=23194,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n 
1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n", 
"20495":"\n16.169483 = sum of:\n 16.169483 = sum of:\n 16.169483 = max 
of:\n 16.169483 = weight(abstract:titl in 20489) [], result of:\n 
16.169483 = score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n 
2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.468952 = 
tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 
= parameter b\n 186.49593 = avgFieldLength\n 40.96 = fieldLength\n 
3.816711E-5 = weight(title:titl in 20489) [], result of:\n 3.816711E-5 = 
score(doc=20489,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n 
1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n", 
"28227":"\n15.670726 = sum of:\n 15.670726 = sum of:\n 15.670726 = max 
of:\n 15.670726 = weight(abstract:titl in 28156) [], result of:\n 
15.670726 = score(doc=28156,freq=2.0 = termFreq=2.0\n), product of:\n 
2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.4236413 = 
tfNorm, computed from:\n 2.0 = termFreq=2.0\n 1.2 = parameter k1\n 0.75 
= parameter b\n 186.49593 = avgFieldLength\n 163.84 = fieldLength\n 
3.816711E-5 = weight(title:titl in 28156) [], result of:\n 3.816711E-5 = 
score(doc=28156,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n 
1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n", 
"20375":"\n15.052014 = sum of:\n 15.052014 = sum of:\n 15.052014 = max 
of:\n 15.052014 = weight(abstract:titl in 20369) [], result of:\n 
15.052014 = score(doc=20369,freq=1.0 = termFreq=1.0\n), product of:\n 
2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.3674331 = 
tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 
= parameter b\n 186.49593 = avgFieldLength\n 64.0 = fieldLength\n 
3.816711E-5 = weight(title:titl in 20369) [], result of:\n 3.816711E-5 = 
score(doc=20369,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n 
1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n", 
"20381":"\n15.052014 = sum of:\n 15.052014 = sum of:\n 15.052014 = max 
of:\n 15.052014 = weight(abstract:titl in 20375) [], result of:\n 
15.052014 = score(doc=20375,freq=1.0 = termFreq=1.0\n), product of:\n 
2.0 = boost\n 5.503748 = idf(docFreq=74, docCount=18297)\n 1.3674331 = 
tfNorm, computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 
= parameter b\n 186.49593 = avgFieldLength\n 64.0 = fieldLength\n 
3.816711E-5 = weight(title:titl in 20375) [], result of:\n 3.816711E-5 = 
score(doc=20375,freq=1.0 = termFreq=1.0\n), product of:\n 3.0 = boost\n 
1.4457239E-5 = idf(docFreq=34584, docCount=34584)\n 0.88 = tfNorm, 
computed from:\n 1.0 = termFreq=1.0\n 1.2 = parameter k1\n 0.75 = 
parameter b\n 3.0 = avgFieldLength\n 4.0 = fieldLength\n", 
"29030":"\n13.699375 = sum of:\n 13.699375 = sum of:\n 13.699375 = max 
of:\n 13.699375 = weight(abstract:titl in 28959) [], result of:\n 
13.699375 = 

Re: Indexed=false for a field,but still able to search on field.

[Susheel Kumar]

#1
 If you will apply seach on this field in the Solr Admin Panel, no
result found. But you can see this field on doc there.

>> Yes, because this field is stored, you can see in the search result.

#2 . Its searchable and can do faceting also.
>> Yes, correct but if you want to use it for searching, put indexed="true"
otherwise it will be full table scan search and will be slow.

Thnx

On Tue, Aug 29, 2017 at 2:48 AM, Renuka Srishti 
wrote:

> Hii,
>
> I have tried two scanarios:
>
>1. I have tried  stored="true"/> and docValues is not set anything.
>2. I have tried  stored="true"/> and docValues is set true.
>
> #1. You can not search directly that field, but when you apply search in
> any other field of that doc, it will show you that field in the result.
>
>   You can not do faceting on this field as well.
>
>If you will apply seach on this field in the Solr Admin Panel, no
> result found. But you can see this field on doc there.
>
> #2. Its searchable and can do faceting also.
>
>
> Please correct me, if I am going wrong.
>
>
> Thanks
>
> Renuka Srishti
>
>
>
> On Tue, Aug 29, 2017 at 1:06 AM, AshB  wrote:
>
> > Hi,
> >
> > Yes docValues is true for fieldType
> >
> >  > docValues="true"/>
> >
> >
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.
> > nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-
> > tp4352338p4352442.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>

What do field(..) on fl parameter ?

[Barbet Alain]

Hi !

I was previously using field(..) to escape some parameter on field name.
Like this:

fl=field(my:data)

Not good to use it on binary data as you will get "Can't initialize
DocTermsIndex to generate (function) FunctionValues for field:
my:data"

So now I use

fl=my*data

And it work's like a charm.

But I have another field:

  

Example of data:

20170716122807

If I create a base from scratch, it's work:

./bin/solr create -c test_long
curl http://localhost:8983/solr/test_long/schema -X POST -H
'Content-type:application/json' --data-binary '{
"add-field" : {
"name":"name",
"type":"text_general",
"multiValued":false,
"stored":true
},
"add-field" : {
"name":"mydate",
"type":"long",
"multiValued":false,
"docValues":false,
"stored":true
}
}'

./bin/post -c test_long -type "application/json" -d '
[
  {
"name": "test name",
"mydate" : "20170716122807"
  }
]
'
curl "http://localhost:8983/solr/test_long/select?q=*:*=name,mydate;


02*:*name,mydatetest name20170716122807


But if I use my old base (migrated from lucene 2.9 => 3 => 4 =>5),
it's fail with:

select?q=*:*=created
java.lang.NullPointerException
at 
org.apache.lucene.util.LegacyNumericUtils.prefixCodedToLong(LegacyNumericUtils.java:189)

But if use field(...) it's work ...
select?q=*:*=field(created)"


00*:*field(created)20170716122747



Can someone explain me this behavior ? How avoid the
java.lang.NullPointerException ? What's do field(..) ?

Thank you for your help

Re: Fetch a binary field

[Barbet Alain]

Mmmmh ... my problem come from another side.
My field name has ":" in it. To avoid this, i use this in request:
fl=(field(name:point) but it's look like it isn't just to escape the
field name, it's do stuff on it.
If I use
fl=name*point
no problem I get my binary data

2017-08-18 6:01 GMT+02:00 GW :
> Had the same issue with long base64_encoded images. Binary & string failed.
> Set my field type to field type ignored. Doesn't seem right (or wrong) but
> it worked.
>
> On 17 August 2017 at 03:58, Rick Leir  wrote:
>
>> On 2017-08-12 04:19 AM, Barbet Alain wrote:
>>
>>> Hi !
>>>
>>> Because this field containt a zipped xml that is bigger than all
>>> others fields & I don't need it for searching, just for display. Yes
>>> it would be better if this field is outside the Lucene base, but as I
>>> have hundred of bases like that, with millions of documents for each,
>>> no I can't change this & reindex the stuff ...
>>>
>>> Any other idea ?
>>>
>> Alain,
>> Since nobody else said it, after a long while...
>> Your zipped xml could be opened before indexing. You should just index the
>> data from the xml which will be needed for display. Hopefully that will
>> consume less index space than the whole xml.
>> cheers -- Rick
>>

Re: Indexed=false for a field,but still able to search on field.

[Renuka Srishti]

Hii,

I have tried two scanarios:

   1. I have tried   and docValues is not set anything.
   2. I have tried   and docValues is set true.

#1. You can not search directly that field, but when you apply search in
any other field of that doc, it will show you that field in the result.

  You can not do faceting on this field as well.

   If you will apply seach on this field in the Solr Admin Panel, no
result found. But you can see this field on doc there.

#2. Its searchable and can do faceting also.


Please correct me, if I am going wrong.


Thanks

Renuka Srishti



On Tue, Aug 29, 2017 at 1:06 AM, AshB  wrote:

> Hi,
>
> Yes docValues is true for fieldType
>
>  docValues="true"/>
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Indexed-false-for-a-field-but-still-able-to-search-on-field-
> tp4352338p4352442.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Solr client

[Aditya]

Hi

I am aggregating open source solr client libraries across all languages.
Below are the links. Very few projects are currently active. Most of them
are last updated few years back. Please provide me pointers, if i missed
any solr client library.

http://www.findbestopensource.com/tagged/solr-client
http://www.findbestopensource.com/tagged/solr-gui


Regards
Ganesh

PS: The website http://www.findbestopensource.com search is powered by Solr.