Graph query from A to X[n] when number of hops is not known

[Sravani Kambhampati]

Hi,

How to graph query from A to X where number of hops is not known, but when 
graph query for each hop remains same.


For example:
If my graph looks like this,
id:A -> pk:A1 -> tgt:A2
id:B -> pk:B1 -> tgt:B2
...
id:X

To get from A to B,

  1.  We query A to A2 using (id->pk) + (pk -> tgt) {!graph from=tgt 
to=pk}{!graph from=pk to=id}id:A
  2.  Then from A2 to B using (tgt -> id) {!graph from=id to=tgt}


To get from A to C, steps 1 and 2 will be repeated:
{!graph from=id to=tgt}{!graph from=tgt to=pk}{!graph from=pk to=id}{!graph 
from=id to=tgt}{!graph from=tgt to=pk}{!graph from=pk to=id}id:A

Likewise, given a start node A, is it possible to query for X when number of 
hops is unknown, but when query is same for every hop?

Thanks,
Sravani

Re: wordpress anyone?

[Gora Mohanty]

On Thu, 4 Mar 2021 at 01:50, dmitri maziuk  wrote:

> Hi all,
>
> does anyone use Solr with WP? It seems there is one for-pay-only
> offering and a few defunct projects from a decade ago... a great web
> search engine is particularly useful if it can actually be used in a
> client.
>
> So has anyone heard about any active WP integration projects other than
> wpsolr.com?
>

Haven't had occasion to use Wordpress, and Solr with it, for a while. Since
nobody else has replied, there does seem to be another plugin that is
open-source,and hosted on Github: https://wordpress.org/plugins/solr-power/
. Cannot comment as to how well it works. Alternatively, one could use a
PHP client library like Solarium.

Regards,
Gora

wordpress anyone?

[dmitri maziuk]

Hi all,

does anyone use Solr with WP? It seems there is one for-pay-only 
offering and a few defunct projects from a decade ago... a great web 
search engine is particularly useful if it can actually be used in a client.


So has anyone heard about any active WP integration projects other than 
wpsolr.com?


Dima

Solr NRT Replicas Out of Sync

[Anshuman Singh]

Hi,

In our Solr 7.4 cluster, we have noticed that some replicas of some of our
Collections are out of sync, the slave replica has more number of records
than the leader.
This is resulting in different number of records on subsequent queries on
the same Collection. Commit is also not helping in this case.

I'm able to replicate the issue using the steps given below:

   1. Create a collection with 1 shard and 2 rf
   2. Ingest 10k records in the collection
   3. Turn down node with replica 2
   4. Ingest 10k records in the collection
   5. Turn down replica 1
   6. Turn up replica 2, wait till it become leader
   7. Ingest 20k records on replica 2
   8. Turn down replica 2
   9. Turn up replica 1, wait till it become leader or use FORCELEADER
   action of Collections API
   10. Turn up replica 2
   11. Now replica 2 has 30k records and replica 1 has 20k records and they
   never sync

I tried the same steps with TLOG replicas and in that case both replicas
had 20k records in the end and were in sync but 10k records were lost.

Is there any way to sync the replicas? I am looking for a lightweight
solution that doesn't require re-creating the index.

Regards,
Anshuman

Parallel SQL Interface and 'qt'

[Jostein Elvaker Haande]

Hi,

I've just started to look into the Parallel SQL interface available in
SOLR. I've done some tests across a few collections, and it works fairly
well.

However I've run into an issue with a few collections where the SQL
interface does not return any data. Now according to the documentation, it
seems like it relies on using the default /select handler to lookup and
return data. However the collections that do not return data in the SQL
interface, are all using /select handlers that have some logic that
requires additional parameters to return data.

I know that if you define 'handleSelect' to true in the /select handler,
you can pass the 'qt' parameter to define which handler to use on the fly.
The handler in question is configured to use 'handleSelect', however when I
pass the 'qt' parameter to the /sql handler, it does not seem to work.

I've gone through the documentation, however I can't find any information
in this regard. Is it possible to define which handler to use when using
the /sql handler?

-- 
Yours sincerely Jostein Elvaker Haande
"A free society is a society where it is safe to be unpopular"
- Adlai Stevenson

https://tolecnal.net  -- tolecnal at tolecnal dot net

RE: Programmatic Basic Auth on CloudSolrClient

[Subhajit Das]

Thanks. This would be very helpful.

From: Tomás Fernández Löbbe
Sent: 04 March 2021 12:32 AM
To: solr-user@lucene.apache.org
Subject: Re: Programmatic Basic Auth on CloudSolrClient

Maybe something like this (I omitted a lot of things you'll have to do,
like passing zk or the list of hosts):

static class CustomCloudSolrClient extends CloudSolrClient {

  protected CustomCloudSolrClient(CustomCloudSolrClientBuilder builder) {
super(builder);
  }

  @Override
  public NamedList request(SolrRequest request, String
collection) throws SolrServerException, IOException {
// your logic here to figure out which credentials to use...
String user = "user";
String pass = "pass";
request.setBasicAuthCredentials(user, pass);
return super.request(request, collection);
  }
}

static class CustomCloudSolrClientBuilder extends CloudSolrClient.Builder {

  @Override
  public CloudSolrClient build() {
return new CustomCloudSolrClient(this);
  }
}

public static void main(String[] args) {
  CloudSolrClient c = new CustomCloudSolrClientBuilder().build();
  ...
}

Do consider that "request" method is called per request, make sure whatever
logic you have there is not super expensive.

On Wed, Mar 3, 2021 at 10:48 AM Subhajit Das 
wrote:

> Hi Thomas,
>
> Thanks. Can you please also share a sample of code to configure the client
> with your workaround?
>
> From: Tomás Fernández Löbbe
> Sent: 04 March 2021 12:05 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Programmatic Basic Auth on CloudSolrClient
>
> As far as I know the current OOTB options are system properties or
> per-request (which would allow you to use different per collection, but
> probably not ideal if you do different types of requests from different
> parts of your code). A workaround (which I've used in the past) is to have
> a custom client that overrides and sets the credentials in the "request"
> method (you can put whatever logic there to identify which credentials to
> use). I recently created https://issues.apache.org/jira/browse/SOLR-15154
> and https://issues.apache.org/jira/browse/SOLR-15155 to try to address
> this
> issue in future releases.
>
> On Wed, Mar 3, 2021 at 5:42 AM Subhajit Das 
> wrote:
>
> >
> > Hi There,
> >
> > Is there any way to programmatically set basic authentication credential
> > on CloudSolrClient?
> >
> > The only documentation available is to use system property. This is not
> > useful if two collection required two separate set of credentials and
> they
> > are parallelly accessed.
> > Thanks in advance.
> >
>
>

Re: Programmatic Basic Auth on CloudSolrClient

[Tomás Fernández Löbbe]

Maybe something like this (I omitted a lot of things you'll have to do,
like passing zk or the list of hosts):

static class CustomCloudSolrClient extends CloudSolrClient {

  protected CustomCloudSolrClient(CustomCloudSolrClientBuilder builder) {
super(builder);
  }

  @Override
  public NamedList request(SolrRequest request, String
collection) throws SolrServerException, IOException {
// your logic here to figure out which credentials to use...
String user = "user";
String pass = "pass";
request.setBasicAuthCredentials(user, pass);
return super.request(request, collection);
  }
}

static class CustomCloudSolrClientBuilder extends CloudSolrClient.Builder {

  @Override
  public CloudSolrClient build() {
return new CustomCloudSolrClient(this);
  }
}

public static void main(String[] args) {
  CloudSolrClient c = new CustomCloudSolrClientBuilder().build();
  ...
}

Do consider that "request" method is called per request, make sure whatever
logic you have there is not super expensive.

On Wed, Mar 3, 2021 at 10:48 AM Subhajit Das 
wrote:

> Hi Thomas,
>
> Thanks. Can you please also share a sample of code to configure the client
> with your workaround?
>
> From: Tomás Fernández Löbbe
> Sent: 04 March 2021 12:05 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Programmatic Basic Auth on CloudSolrClient
>
> As far as I know the current OOTB options are system properties or
> per-request (which would allow you to use different per collection, but
> probably not ideal if you do different types of requests from different
> parts of your code). A workaround (which I've used in the past) is to have
> a custom client that overrides and sets the credentials in the "request"
> method (you can put whatever logic there to identify which credentials to
> use). I recently created https://issues.apache.org/jira/browse/SOLR-15154
> and https://issues.apache.org/jira/browse/SOLR-15155 to try to address
> this
> issue in future releases.
>
> On Wed, Mar 3, 2021 at 5:42 AM Subhajit Das 
> wrote:
>
> >
> > Hi There,
> >
> > Is there any way to programmatically set basic authentication credential
> > on CloudSolrClient?
> >
> > The only documentation available is to use system property. This is not
> > useful if two collection required two separate set of credentials and
> they
> > are parallelly accessed.
> > Thanks in advance.
> >
>
>

RE: Programmatic Basic Auth on CloudSolrClient

[Subhajit Das]

Hi Thomas,

Thanks. Can you please also share a sample of code to configure the client with 
your workaround?

From: Tomás Fernández Löbbe
Sent: 04 March 2021 12:05 AM
To: solr-user@lucene.apache.org
Subject: Re: Programmatic Basic Auth on CloudSolrClient

As far as I know the current OOTB options are system properties or
per-request (which would allow you to use different per collection, but
probably not ideal if you do different types of requests from different
parts of your code). A workaround (which I've used in the past) is to have
a custom client that overrides and sets the credentials in the "request"
method (you can put whatever logic there to identify which credentials to
use). I recently created https://issues.apache.org/jira/browse/SOLR-15154
and https://issues.apache.org/jira/browse/SOLR-15155 to try to address this
issue in future releases.

On Wed, Mar 3, 2021 at 5:42 AM Subhajit Das  wrote:

>
> Hi There,
>
> Is there any way to programmatically set basic authentication credential
> on CloudSolrClient?
>
> The only documentation available is to use system property. This is not
> useful if two collection required two separate set of credentials and they
> are parallelly accessed.
> Thanks in advance.
>

Re: NPE in QueryComponent.mergeIds when using timeAllowed and sorting SOLR 8.7

[Tomás Fernández Löbbe]

Patch looks good to me. Since it's a bugfix it can be committed to 8_8
branch and released on the next bugfix release, though I don't think it
should trigger one. In the meantime, if you can patch your environment and
confirm that it fixes your problem, that's a good comment to leave in
SOLR-14758. 

On Mon, Mar 1, 2021 at 3:12 PM Phill Campbell 
wrote:

> Anyone?
>
> > On Feb 24, 2021, at 7:47 AM, Phill Campbell
>  wrote:
> >
> > Last week I switched to Solr 8.7 from a “special” build of Solr 6.6
> >
> > The system has a timeout set for querying. I am now seeing this bug.
> >
> > https://issues.apache.org/jira/browse/SOLR-14758 <
> https://issues.apache.org/jira/browse/SOLR-14758>
> >
> > Max Query Time goes from 1.6 seconds to 20 seconds and affects the
> entire system for about 2 minutes as reported in New Relic.
> >
> > null:java.lang.NullPointerException
> >   at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:935)
> >   at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626)
> >   at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605)
> >   at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:486)
> >   at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214)
> >   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627)
> >
> >
> > Can this be fixed in a patch for Solr 8.8? I do not want to have to go
> back to Solr 6 and reindex the system, that takes 2 days using 180 EMR
> instances.
> >
> > Pease advise. Thank you.
>
>

Amazon Sponsor Product(Ads) Search team is looking for talents with expertise in Solr/Lucene at all levels

[pxiong]

Hi folks,

Are you interested in Amazon’s Advertising (that employs state of art 
solutions involving Search, Digital Advertising, AWS technologies etc) 
but wondering which team can best leverage your expertise in 
Solr/Lucene? Please continue to read:


My team is Sponsored Search Delivery (SSD) where we directly use Solr to 
efficiently render relevant Ads for shoppers. We deliver billions of ad 
impressions and millions of clicks daily and are breaking fresh ground 
to create world-class products. Powered with Solr, our systems scan 
billions of records in milliseconds to return relevant ads. The 
architectural excellence is evident from the fact that despite we’ve 
such high-throughput-low-tps Tier-1 system, our team enjoys an 
exceptionally low operations burden. Our solutions employ latest 
‘information retrieval(IR)’ algorithms and we try to leverage latest 
research in IR to ensure superior shopper experience. It’s a great mix 
of ‘engineering’ and ‘applied-science’ to render relevant ads within 
milliseconds.


My team is making tremendous investment in Solr research, development 
and application this year and the future. For example, we plan to apply 
vector search in Solr (SOLR-12890) to extend power of current 
keyword-based search to improve ads recall. We are experimenting 
approximate nearest vector search (LUCENE-9004) to achieve accuracy at a 
reasonable cost. We also plan to leverage rich features of Solr to 
return ads based on 'themes' (viz. brand-similar, economical, 4-star 
above etc.). We also plan to share the experience and lessons to 
Solr/Lucene community and find ways to contribute back to the community.


With a broad mandate to experiment and innovate, we are growing at an 
unprecedented rate together with Amazon Ads business. There is a 
seemingly endless range of new opportunities ahead of us. We’re looking 
for amazing minds at various positions in team for New York Location 
(Primary goal is to expand the team in NYC, However, Location 
preferences of Boulder, Seattle, Toronto: can also be considered on case 
by case basis). Prior experience in Solr/Lucene would be a great value add.
SDM: www.amazon.jobs/jobs/1427445?no_int_redir=1 

SDE2: www.amazon.jobs/jobs/1451662?no_int_redir=1 

SDE3:www.amazon.jobs/jobs/1357591?no_int_redir=1 

ASII: www.amazon.jobs/jobs/1427080?no_int_redir=1 

ASIII: www.amazon.jobs/jobs/1379670?no_int_redir=1 

TPM: www.amazon.jobs/jobs/1353298?no_int_redir=1 



If you are interested, please reply this email directly or reach out to 
the hiring manager Vikas Dhaka (dhavi...@amazon.com 
) Thanks!


Best

Pengcheng Xiong

(Apache Hive PMC member and committer working at Amazon)

Re: Programmatic Basic Auth on CloudSolrClient

[Tomás Fernández Löbbe]

As far as I know the current OOTB options are system properties or
per-request (which would allow you to use different per collection, but
probably not ideal if you do different types of requests from different
parts of your code). A workaround (which I've used in the past) is to have
a custom client that overrides and sets the credentials in the "request"
method (you can put whatever logic there to identify which credentials to
use). I recently created https://issues.apache.org/jira/browse/SOLR-15154
and https://issues.apache.org/jira/browse/SOLR-15155 to try to address this
issue in future releases.

On Wed, Mar 3, 2021 at 5:42 AM Subhajit Das  wrote:

>
> Hi There,
>
> Is there any way to programmatically set basic authentication credential
> on CloudSolrClient?
>
> The only documentation available is to use system property. This is not
> useful if two collection required two separate set of credentials and they
> are parallelly accessed.
> Thanks in advance.
>

Programmatic Basic Auth on CloudSolrClient

[Subhajit Das]

Hi There,

Is there any way to programmatically set basic authentication credential on 
CloudSolrClient?

The only documentation available is to use system property. This is not useful 
if two collection required two separate set of credentials and they are 
parallelly accessed.
Thanks in advance.

Solr backup no longer writes to a UNC path

[Gell-Holleron, Daniel]

Hi there,

We've upgraded from Solr 7.7.1 to Solr 8.8.1 (running on Windows Operating 
System) and I've noticed that when running a Solr backup, it will no longer 
allow me to write to a UNC path. Is this something that has been purposely 
changed?

I've noticed a new system property called SOLR_OPTS="%SOLR_OPTS% 
-Dsolr.allowPath=S:\" which I've enabled. Is there a way to point this to a 
remote server, rather than a different drive on the local server?

Thanks,

Daniel

Increase in response time in case of collapse queries.

[Parshant Kumar]

Hi all,

We have implemented collapse queries in place of grouped queries on our
production solr. As mentioned in solr documentation collapse queries are
recommended in place of grouped queries in terms of performance . But after
switching to collapsed queries from grouped queries response time of
queries have increased. This is unexpected behaviour, the response time
should have been improved but results are opposites.
Please someone help why response time is increased for collapsed queries.

Thanks
Parshant Kumar

-- 

How can I get dynamicField in Solr 6.1.0.

[vishal patel]

I am using Solr 6.1.0. We have 2 shards and each has one replica.

My schema field is below in one collection



My data like

FORM1065678510875540
2021-03-03T23:59:59Z
false
false

£


No





fdsfsfsfsf


Yes


2021-03-03


yes





2021-03-01T06:55:29


12:00 PM


no


12:00 PM

-1
-1


I want to get only fields 
/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Start_Date
, /myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/TenderEndDatePassed and 
/myFields/FORM_CUSTOM_FIELDS/Bid_Opportunity/Tender_Review_Time. How can I get 
only above fields? What need to pass in FL?

Regards,
Vishal

FW: Graph traversal when nodes are indirectly connected with references

[Sravani Kambhampati]

I have a graph with disjoint sets of nodes connected indirectly with a 
reference as shown below. Given an id is it possible to get the leaf node when 
the depth is unknown?

[
{ id: A, child: { ref: B } },
{ id: B, child: { ref: C } },
{ id: C, child: { ref: D } },
.
.
{ id: Y, child: { ref: Z } }
]
[cid:image001.png@01D71031.E9C961D0]
Thanks,
Sravani