Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Thank you for the input, Aroop.

It is probably a red herring.  I will have to pick the configuration apart 
piece by piece.  Sigh.

It's probably not a node down issue, since I'm only setting up one node.

(Reporting an unrelated error message should probably be considered a bug 
anyways.)

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Aroop Ganguly 
Envoyé : 14 juin 2020 17:37
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Isabele sometime 401’s are a red herring for other issues un related to auth.
We have had issues on 7.7 where an underlying transient replica recovery and/or 
leader down situation where the only message we got back from Solr was a 401.
Please see if u have any down replicas or other issues where certain nodes may 
have trouble getting more current information from zookeeper.


> On Jun 14, 2020, at 2:13 PM, Isabelle Giguere  <mailto:igigu...@opentext.com.INVALID>> wrote:
>
> I have created 
> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/SOLR-14569__;!!Obbck6kTJA!PBs90R0pHCmvm6hGqjUeowZNMwhTEibIfLyr8_szdm0Jh-s9okdbuGya_nBlsjED$
>   
> <https://urldefense.com/v3/__https://issues.apache.org/jira/browse/SOLR-14569__;!!Obbck6kTJA!PBs90R0pHCmvm6hGqjUeowZNMwhTEibIfLyr8_szdm0Jh-s9okdbuGya_nBlsjED$
>  >
> It includes a patch with the unit test to reproduce the issue, and a 
> simplification of our product-specific configuration, with instructions.
>
> Let's catch up on Jira.
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Jan Høydahl mailto:jan@cominvent.com>>
> Envoyé : 13 juin 2020 17:50
> À : solr-user  <mailto:solr-user@lucene.apache.org>>
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> I did not manage to reproduce. Feel free to open the JIRA and attach the 
> failing test. In the issue description, it is great if you manage to describe 
> the reproduction steps in a clean way, so anyone can reproduce with a minimal 
> neccessary config.
>
> Jan
>
>> 13. jun. 2020 kl. 00:41 skrev Isabelle Giguere 
>> mailto:igigu...@opentext.com.INVALID>>:
>>
>> Hello again;
>>
>> I have managed to reproduce the issue in a unit test.  I should probably add 
>> a Jira ticket with a patch for the unit test On Solr 8.5.0, not master.
>>
>> Meanwhile, for your suggested queries:
>>
>> 1.  Query on the collection:
>>
>> curl -i -u admin:admin 
>> https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$
>>  
>> <https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$>
>> HTTP/1.1 200 OK
>> Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
>> 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
>> 'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 
>> 'self'; worker-src 'self';
>> X-Content-Type-Options: nosniff
>> X-Frame-Options: SAMEORIGIN
>> X-XSS-Protection: 1; mode=block
>> Content-Type: application/xml; charset=UTF-8
>> Content-Length: 8214
>>
>> 
>> 
>>
>> 
>> true
>> 0
>> 2
>> 
>>   *:*
>> 
>> 
>> 
>> Response contains the Solr document, of course
>>
>>
>> 2. Query on the alias
>>
>> curl -i -u admin:admin 
>> https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7PZyiHWo$
>>  
>> <https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7PZyiHWo$><https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$
>>  
>> <https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$>
>>  >
>> HTTP/1.1 401 Unauthorized
>> Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
>> 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
>> 'self'; media-src 'self'; style-src 'se

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

I have created https://issues.apache.org/jira/browse/SOLR-14569
It includes a patch with the unit test to reproduce the issue, and a 
simplification of our product-specific configuration, with instructions.

Let's catch up on Jira.

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Jan Høydahl 
Envoyé : 13 juin 2020 17:50
À : solr-user 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

I did not manage to reproduce. Feel free to open the JIRA and attach the 
failing test. In the issue description, it is great if you manage to describe 
the reproduction steps in a clean way, so anyone can reproduce with a minimal 
neccessary config.

Jan

> 13. jun. 2020 kl. 00:41 skrev Isabelle Giguere 
> :
>
> Hello again;
>
> I have managed to reproduce the issue in a unit test.  I should probably add 
> a Jira ticket with a patch for the unit test On Solr 8.5.0, not master.
>
> Meanwhile, for your suggested queries:
>
>  1.  Query on the collection:
>
> curl -i -u admin:admin 
> https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$
> HTTP/1.1 200 OK
> Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
> 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
> 'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 
> 'self'; worker-src 'self';
> X-Content-Type-Options: nosniff
> X-Frame-Options: SAMEORIGIN
> X-XSS-Protection: 1; mode=block
> Content-Type: application/xml; charset=UTF-8
> Content-Length: 8214
>
> 
> 
>
> 
>  true
>  0
>  2
>  
>*:*
>  
> 
> 
> Response contains the Solr document, of course
>
>
> 2. Query on the alias
>
> curl -i -u admin:admin 
> https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7PZyiHWo$
>  
> <https://urldefense.com/v3/__http://10.5.106.115:8985/solr/test1/select?q=*:*=xml__;Kio!!Obbck6kTJA!LvZRdkAwPGTDqWqS-BYMmyuuwAp9coGzkDzz5BG7hTCLmCSV2bOZBM9A7JzikWgk$
>  >
> HTTP/1.1 401 Unauthorized
> Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
> 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
> 'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 
> 'self'; worker-src 'self';
> X-Content-Type-Options: nosniff
> X-Frame-Options: SAMEORIGIN
> X-XSS-Protection: 1; mode=block
> Cache-Control: no-cache, no-store
> Pragma: no-cache
> Expires: Sat, 01 Jan 2000 01:00:00 GMT
> Last-Modified: Fri, 12 Jun 2020 22:30:20 GMT
> ETag: "172aaa7c1eb"
> Content-Type: application/xml; charset=UTF-8
> Content-Length: 1332
>
> 
> 
>
> 
>  true
>  401
>  16
>  
>*:*
>  
> 
> 
> Error contains the full html HTTP 401 message (with escaped characters, of 
> course)
> Gist of it : HTTP ERROR 401 require authentication
>
> Thanks;
>
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Jan Høydahl 
> Envoyé : 12 juin 2020 17:30
> À : solr-user@lucene.apache.org 
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> I’d say, try the query with curl and enable http headers
>
> curl -i —user admin:admin http://localhost:8983/solr/mycollection/select?q=*:*
> curl -i —user admin:admin http://localhost:8983/solr/myalias/select?q=*:*
>
> Are you saying that you see a difference between the two? What are the 
> headers?
>
> Jan
>
>> 12. jun. 2020 kl. 20:06 skrev Isabelle Giguere 
>> :
>>
>> Hi Jan
>>
>> Thank you for your time on this.
>>
>> If I send a /select request directly on the alias (/solr/test/select), the 
>> browser asks for credentials, but the Solr response returns status=401 and 
>> an html error message with "HTTP ERROR 401 require authentication"
>>
>> Obviously, my expectation was that some query results would be returned.
>>
>> Since you can't reproduce the issue, I have to assume it's a configuration 
>> issue.
>>
>> So, if I may, let me provide as much details as I can about my setup.
>>
>> Can anyone see something wrong here, some incompatibility ?
>>
>> Solr 8.5.0
>>
>> solrconfig.xml
>> 7.1.0
>> 
>> 
>> 
>> 
>>   
>>   5
>>   5
>> 

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hello again;

I have managed to reproduce the issue in a unit test.  I should probably add a 
Jira ticket with a patch for the unit test On Solr 8.5.0, not master.

Meanwhile, for your suggested queries:

  1.  Query on the collection:

curl -i -u admin:admin http://10.5.106.115:8985/solr/test1/select?q=*:*=xml
 HTTP/1.1 200 OK
Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; 
worker-src 'self';
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Type: application/xml; charset=UTF-8
Content-Length: 8214





  true
  0
  2
  
*:*
  


Response contains the Solr document, of course


2. Query on the alias

curl -i -u admin:admin 
http://10.5.106.115:8985/solr/test/select?q=*:*=xml<http://10.5.106.115:8985/solr/test1/select?q=*:*=xml>
 HTTP/1.1 401 Unauthorized
Content-Security-Policy: default-src 'none'; base-uri 'none'; connect-src 
'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 
'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; 
worker-src 'self';
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store
Pragma: no-cache
Expires: Sat, 01 Jan 2000 01:00:00 GMT
Last-Modified: Fri, 12 Jun 2020 22:30:20 GMT
ETag: "172aaa7c1eb"
Content-Type: application/xml; charset=UTF-8
Content-Length: 1332





  true
  401
  16
  
*:*
  


Error contains the full html HTTP 401 message (with escaped characters, of 
course)
Gist of it : HTTP ERROR 401 require authentication

Thanks;


Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Jan Høydahl 
Envoyé : 12 juin 2020 17:30
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

I’d say, try the query with curl and enable http headers

curl -i —user admin:admin http://localhost:8983/solr/mycollection/select?q=*:*
curl -i —user admin:admin http://localhost:8983/solr/myalias/select?q=*:*

Are you saying that you see a difference between the two? What are the headers?

Jan

> 12. jun. 2020 kl. 20:06 skrev Isabelle Giguere 
> :
>
> Hi Jan
>
> Thank you for your time on this.
>
> If I send a /select request directly on the alias (/solr/test/select), the 
> browser asks for credentials, but the Solr response returns status=401 and an 
> html error message with "HTTP ERROR 401 require authentication"
>
> Obviously, my expectation was that some query results would be returned.
>
> Since you can't reproduce the issue, I have to assume it's a configuration 
> issue.
>
> So, if I may, let me provide as much details as I can about my setup.
>
> Can anyone see something wrong here, some incompatibility ?
>
> Solr 8.5.0
>
> solrconfig.xml
> 7.1.0
> 
> 
> 
> 
>
>5
>5
>5
>
>
> schema.xml
> version=1.6
> Some warnings on start-up about Trie* fields and deprecated filters (we 
> should fix that)
>
> security.json in Zookeeper, at the Solr ZK root (provided on this thread)
> blockUnknown : (true|false) = no change in behavior for me, for this issue
> forwardCredentials : (true|false) = no change in behavior for me, for this 
> issue
>
> No SSL
>
> solr.in.sh
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=admin:admin"
>
> start command params:
> solr start -force -c -m 4g -h  -p  -z 
> :/
>
>
> Am I missing anything ?
>
> Thank you.
>
> 
>
> My investigation so far:
>
> I have set logging levels to TRACE for anything related to HTTP, HTTP2, 
> Authorization, Authentication...
>
> Judging by a comment in 
> org.apache.solr.core.CoreContainer.setupHttpClientForAuthPlugin(Object), I 
> should see some logging from PKIAuthenticationPlugin, no matter what plugin 
> is actually used, and regardless if forwardCredentials is true or false:
> Comment:
> // Always register PKI auth interceptor, which will then delegate the 
> decision of who should secure
> // each request to the configured authentication plugin.
>
> Expected log message from 
> org.apache.solr.security.PKIAuthenticationPlugin.setup(Http2SolrClient) 
> and/or from 
> org.apache.solr.security.PKIAuthenticationPlugin.HttpHeaderClientInterceptor.process(HttpRequest,
>  HttpContext)
>
> When running a request on an alias, I only see the expected log message from 
> /admin requests, never for /select requests.
>
> Of course, if my configura

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hi Jan

Thank you for your time on this.

If I send a /select request directly on the alias (/solr/test/select), the 
browser asks for credentials, but the Solr response returns status=401 and an 
html error message with "HTTP ERROR 401 require authentication"

Obviously, my expectation was that some query results would be returned.

Since you can't reproduce the issue, I have to assume it's a configuration 
issue.

So, if I may, let me provide as much details as I can about my setup.

Can anyone see something wrong here, some incompatibility ?

Solr 8.5.0

solrconfig.xml
7.1.0





5
5
5


schema.xml
version=1.6
Some warnings on start-up about Trie* fields and deprecated filters (we should 
fix that)

security.json in Zookeeper, at the Solr ZK root (provided on this thread)
blockUnknown : (true|false) = no change in behavior for me, for this issue
forwardCredentials : (true|false) = no change in behavior for me, for this issue

No SSL

solr.in.sh
SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=admin:admin"

start command params:
solr start -force -c -m 4g -h  -p  -z 
:/


Am I missing anything ?

Thank you.



My investigation so far:

I have set logging levels to TRACE for anything related to HTTP, HTTP2, 
Authorization, Authentication...

Judging by a comment in 
org.apache.solr.core.CoreContainer.setupHttpClientForAuthPlugin(Object), I 
should see some logging from PKIAuthenticationPlugin, no matter what plugin is 
actually used, and regardless if forwardCredentials is true or false:
Comment:
// Always register PKI auth interceptor, which will then delegate the decision 
of who should secure
// each request to the configured authentication plugin.

Expected log message from 
org.apache.solr.security.PKIAuthenticationPlugin.setup(Http2SolrClient) and/or 
from 
org.apache.solr.security.PKIAuthenticationPlugin.HttpHeaderClientInterceptor.process(HttpRequest,
 HttpContext)

When running a request on an alias, I only see the expected log message from 
/admin requests, never for /select requests.

Of course, if my configuration is wrong, then my code and log analysis is 
useless.

**


Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Jan Høydahl 
Envoyé : 12 juin 2020 06:55
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Hi

I tried to reproduce, but I can successfully search both the collection and the 
alias. Both collection and alias promt for password, and when giving the 
password the search succeeds.

What was your expectation?

Jan

> 11. jun. 2020 kl. 16:53 skrev Isabelle Giguere 
> :
>
> Some extra info:
> Collections have 1 shard, 1 replica.  Only 1 Solr node running.
>
> The HTTP 401 is not intermittent, as reported in SOLR-13421 and SOLR-13510.
>
> Any request to the alias fails.
>
> Thanks;
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Isabelle Giguere 
> Envoyé : 10 juin 2020 16:11
> À : solr-user@lucene.apache.org 
> Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> Hi Jan;
>
> Thank you for your reply.
>
> This is security.json as seen in Zookeeper.  Credentials are admin / admin
>
> {
>  "authentication":{
>"blockUnknown":false,
>"realm":"MTM Solr",
>"forwardCredentials":true,
>"class":"solr.BasicAuthPlugin",
>"credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
> +tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
>"":{"v":2}},
>  "authorization":{
>"class":"solr.RuleBasedAuthorizationPlugin",
>"permissions":[{
>"name":"all",
>"role":"admin"}],
>"user-role":{"admin":"admin"},
>"":{"v":8}}}
>
> Thanks for feedback
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Jan Høydahl 
> Envoyé : 10 juin 2020 16:01
> À : solr-user@lucene.apache.org 
> Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr
>
> Please share your security.json file
>
> Jan Høydahl
>
>> 10. jun. 2020 kl. 21:53 skrev Isabelle Giguere 
>> :
>>
>> Hi;
>>
>> I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can 
&g

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Some extra info:
Collections have 1 shard, 1 replica.  Only 1 Solr node running.

The HTTP 401 is not intermittent, as reported in SOLR-13421 and SOLR-13510.

Any request to the alias fails.

Thanks;

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java


____
De : Isabelle Giguere 
Envoyé : 10 juin 2020 16:11
À : solr-user@lucene.apache.org 
Objet : Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Hi Jan;

Thank you for your reply.

This is security.json as seen in Zookeeper.  Credentials are admin / admin

{
  "authentication":{
"blockUnknown":false,
"realm":"MTM Solr",
"forwardCredentials":true,
"class":"solr.BasicAuthPlugin",
"credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
+tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
"":{"v":2}},
  "authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{
"name":"all",
"role":"admin"}],
"user-role":{"admin":"admin"},
"":{"v":8}}}

Thanks for feedback

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java


____
De : Jan Høydahl 
Envoyé : 10 juin 2020 16:01
À : solr-user@lucene.apache.org 
Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Please share your security.json file

Jan Høydahl

> 10. jun. 2020 kl. 21:53 skrev Isabelle Giguere 
> :
>
> Hi;
>
> I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can log 
> in the Solr Admin UI.  I can create collections and aliases, and I can index 
> documents in Solr.
>
> Collections : test1, test2
> Alias: test (combines test1, test2)
>
> Indexed document "solr-word.pdf" in collection test1
>
> Searching on a collection works:
> http://localhost:8983/solr/test1/select?q=*:*=xml
> 
>
> But searching on an alias results in HTTP 401
> http://localhost:8983/solr/test/select?q=*:*=xml
>
> Error from server at null: Expected mime type application/octet-stream but 
> got text/html.content="text/html;charset=utf-8"/> Error 401 Authentication failed, 
> Response code: 401  HTTP ERROR 401 Authentication 
> failed, Response code: 401  
> URI:/solr/test1_shard1_replica_n1/select 
> STATUS:401 MESSAGE:Authentication 
> failed, Response code: 401 
> SERVLET:default   
>
> Even if 
> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/SOLR-13510__;!!Obbck6kTJA!P6ugA-rw1I80PaH0U_GVasNqn8EXwmVQ33lwcPOU-cvNgTJK6-3zAf8ukzvv3ynJ$
>   is fixed in Solr 8.5.0, I did try to start Solr with -Dsolr.http1=true, and 
> I set "forwardCredentials":true in security.json.
>
> Nothing works.  I just cannot use aliases when Solr is secured.
>
> Can anyone confirm if this may be a configuration issue, or if this could 
> possibly be a bug ?
>
> Thank you;
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>

Re: [EXTERNAL] - SolR OOM error due to query injection

[Isabelle Giguere]

Hi Guilherme;

The only thing I can think of right now is the number of non-alphanumeric 
characters.

In the first 'q' in your examples, after resolving the character escapes, 1/3 
of characters are non-alphanumeric (* / = , etc).

Maybe filter-out queries that contain too many non-alphanumeric characters 
before sending the request to Solr ?  Whatever "too many" could be.

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java



De : Guilherme Viteri 
Envoyé : 10 juin 2020 16:57
À : solr-user@lucene.apache.org 
Objet : [EXTERNAL] - SolR OOM error due to query injection

Hi,

Environment: SolR 6.6.2, with org.apache.solr.solr-core:6.1.0. This setup has 
been running for at least 4 years without having OutOfMemory error. (it is 
never too late for an OOM…)

This week, our search tool has been attacked via ‘sql injection’ like, and that 
led to an OOM. These requests weren’t aggressive that stressed the server with 
an excessive number of hits, however 5 to 10 request of this nature was enough 
to crash the server.

I’ve come across a this link 
https://urldefense.com/v3/__https://stackoverflow.com/questions/26862474/prevent-from-solr-query-injections-when-using-solrj__;!!Obbck6kTJA!IdbT_RQCp3jXO5KJxMkWNJIRlNU9Hu1hnJsWqCWT_QS3zpZSAxYeFPM_hGWNwp3y$
  
, however, that’s not what I am after. In our case we do allow lucene query 
and field search like title:Title or our ids have dash and if it get escaped, 
then the search won’t work properly.

Does anyone have an idea ?

Cheers
G

Here are some of the requests that appeared in the logs in relation to the 
attack (see below: sorry it is messy)
query?q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28SELECT%2F%2A%2A%2F2%2A%28IF%28%28SELECT%2F%2A%2A%2F%2A%2F%2A%2A%2FFROM%2F%2A%2A%2F%28SELECT%2F%2A%2A%2FCONCAT%280x717a707871%2C%28SELECT%2F%2A%2A%2F%28ELT%283235%3D3235%2C1%29%29%29%2C0x717a626271%2C0x78%29%29s%29%2C%2F%2A%2A%2F8446744073709551610%2C%2F%2A%2A%2F8446744073709551610%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22YBXk%22%2F%2A%2A%2FLIKE%2F%2A%2A%2F%22YBXk=Homo%20sapiens=Reaction=Pathway=true

q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28SELECT%2F%2A%2A%2F2%2A%28IF%28%28SELECT%2F%2A%2A%2F%2A%2F%2A%2A%2FFROM%2F%2A%2A%2F%28SELECT%2F%2A%2A%2FCONCAT%280x717a707871%2C%28SELECT%2F%2A%2A%2F%28ELT%283235%3D3235%2C1%29%29%29%2C0x717a626271%2C0x78%29%29s%29%2C%2F%2A%2A%2F8446744073709551610%2C%2F%2A%2A%2F8446744073709551610%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22rDmG%22%3D%22rDmG=Homo%20sapiens=Reaction=Pathway=true

q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28SELECT%2F%2A%2A%2F3641%2F%2A%2A%2FFROM%28SELECT%2F%2A%2A%2FCOUNT%28%2A%29%2CCONCAT%280x717a707871%2C%28SELECT%2F%2A%2A%2F%28ELT%283641%3D3641%2C1%29%29%29%2C0x717a626271%2CFLOOR%28RAND%280%29%2A2%29%29x%2F%2A%2A%2FFROM%2F%2A%2A%2FINFORMATION_SCHEMA.PLUGINS%2F%2A%2A%2FGROUP%2F%2A%2A%2FBY%2F%2A%2A%2Fx%29a%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22dfkM%22%2F%2A%2A%2FLIKE%2F%2A%2A%2F%22dfkM=Homo%20sapiens=Reaction=Pathway=true

q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28SELECT%2F%2A%2A%2F3641%2F%2A%2A%2FFROM%28SELECT%2F%2A%2A%2FCOUNT%28%2A%29%2CCONCAT%280x717a707871%2C%28SELECT%2F%2A%2A%2F%28ELT%283641%3D3641%2C1%29%29%29%2C0x717a626271%2CFLOOR%28RAND%280%29%2A2%29%29x%2F%2A%2A%2FFROM%2F%2A%2A%2FINFORMATION_SCHEMA.PLUGINS%2F%2A%2A%2FGROUP%2F%2A%2A%2FBY%2F%2A%2A%2Fx%29a%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22yBhx%22%3D%22yBhx=Homo%20sapiens=Reaction=Pathway=true

q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F1695%3DCTXSYS.DRITHSX.SN%281695%2C%28CHR%28113%29%7C%7CCHR%28122%29%7C%7CCHR%28112%29%7C%7CCHR%28120%29%7C%7CCHR%28113%29%7C%7C%28SELECT%2F%2A%2A%2F%28CASE%2F%2A%2A%2FWHEN%2F%2A%2A%2F%281695%3D1695%29%2F%2A%2A%2FTHEN%2F%2A%2A%2F1%2F%2A%2A%2FELSE%2F%2A%2A%2F0%2F%2A%2A%2FEND%29%2F%2A%2A%2FFROM%2F%2A%2A%2FDUAL%29%7C%7CCHR%28113%29%7C%7CCHR%28122%29%7C%7CCHR%2898%29%7C%7CCHR%2898%29%7C%7CCHR%28113%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22eEdc%22%2F%2A%2A%2FLIKE%2F%2A%2A%2F%22eEdc=Homo%20sapiens=Reaction=Pathway=true

q=IPP%22%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F1695%3DCTXSYS.DRITHSX.SN%281695%2C%28CHR%28113%29%7C%7CCHR%28122%29%7C%7CCHR%28112%29%7C%7CCHR%28120%29%7C%7CCHR%28113%29%7C%7C%28SELECT%2F%2A%2A%2F%28CASE%2F%2A%2A%2FWHEN%2F%2A%2A%2F%281695%3D1695%29%2F%2A%2A%2FTHEN%2F%2A%2A%2F1%2F%2A%2A%2FELSE%2F%2A%2A%2F0%2F%2A%2A%2FEND%29%2F%2A%2A%2FFROM%2F%2A%2A%2FDUAL%29%7C%7CCHR%28113%29%7C%7CCHR%28122%29%7C%7CCHR%2898%29%7C%7CCHR%2898%29%7C%7CCHR%28113%29%29%29%2F%2A%2A%2FAND%2F%2A%2A%2F%28%28%28%22zAUD%22%3D%22zAUD=Homo%20sapiens=Reaction=Pathway=true

Re: [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hi Jan;

Thank you for your reply.

This is security.json as seen in Zookeeper.  Credentials are admin / admin

{
  "authentication":{
"blockUnknown":false,
"realm":"MTM Solr",
"forwardCredentials":true,
"class":"solr.BasicAuthPlugin",
"credentials":{"admin":"0rTOgObKYwzSyPoYuj2su2/90eQCfysF1aasxTx+wrc= 
+tCMmpawYYtTsp3JfkG9avb8bKZlm/IGTZirsufYvns="},
"":{"v":2}},
  "authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{
"name":"all",
"role":"admin"}],
"user-role":{"admin":"admin"},
"":{"v":8}}}

Thanks for feedback

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java


____
De : Jan Høydahl 
Envoyé : 10 juin 2020 16:01
À : solr-user@lucene.apache.org 
Objet : [EXTERNAL] - Re: HTTP 401 when searching on alias in secured Solr

Please share your security.json file

Jan Høydahl

> 10. jun. 2020 kl. 21:53 skrev Isabelle Giguere 
> :
>
> Hi;
>
> I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can log 
> in the Solr Admin UI.  I can create collections and aliases, and I can index 
> documents in Solr.
>
> Collections : test1, test2
> Alias: test (combines test1, test2)
>
> Indexed document "solr-word.pdf" in collection test1
>
> Searching on a collection works:
> http://localhost:8983/solr/test1/select?q=*:*=xml
> 
>
> But searching on an alias results in HTTP 401
> http://localhost:8983/solr/test/select?q=*:*=xml
>
> Error from server at null: Expected mime type application/octet-stream but 
> got text/html.content="text/html;charset=utf-8"/> Error 401 Authentication failed, 
> Response code: 401  HTTP ERROR 401 Authentication 
> failed, Response code: 401  
> URI:/solr/test1_shard1_replica_n1/select 
> STATUS:401 MESSAGE:Authentication 
> failed, Response code: 401 
> SERVLET:default   
>
> Even if 
> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/SOLR-13510__;!!Obbck6kTJA!P6ugA-rw1I80PaH0U_GVasNqn8EXwmVQ33lwcPOU-cvNgTJK6-3zAf8ukzvv3ynJ$
>   is fixed in Solr 8.5.0, I did try to start Solr with -Dsolr.http1=true, and 
> I set "forwardCredentials":true in security.json.
>
> Nothing works.  I just cannot use aliases when Solr is secured.
>
> Can anyone confirm if this may be a configuration issue, or if this could 
> possibly be a bug ?
>
> Thank you;
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>

HTTP 401 when searching on alias in secured Solr

[Isabelle Giguere]

Hi;

I'm using Solr 8.5.0.  I have uploaded security.json to Zookeeper.  I can log 
in the Solr Admin UI.  I can create collections and aliases, and I can index 
documents in Solr.

Collections : test1, test2
Alias: test (combines test1, test2)

Indexed document "solr-word.pdf" in collection test1

Searching on a collection works:
http://localhost:8983/solr/test1/select?q=*:*=xml


But searching on an alias results in HTTP 401
http://localhost:8983/solr/test/select?q=*:*=xml

Error from server at null: Expected mime type application/octet-stream but got 
text/html.Error 401 Authentication failed, 
Response code: 401  HTTP ERROR 401 Authentication 
failed, Response code: 401  
URI:/solr/test1_shard1_replica_n1/select 
STATUS:401 MESSAGE:Authentication 
failed, Response code: 401 SERVLET:default 
  

Even if https://issues.apache.org/jira/browse/SOLR-13510 is fixed in Solr 
8.5.0, I did try to start Solr with -Dsolr.http1=true, and I set 
"forwardCredentials":true in security.json.

Nothing works.  I just cannot use aliases when Solr is secured.

Can anyone confirm if this may be a configuration issue, or if this could 
possibly be a bug ?

Thank you;

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java

Rule-Based Auth - update not working

[Isabelle Giguere]

Hi;

I'm using Solr 8.5.0.

I'm having trouble setting up some permissions using the rule-based 
authorization plugin: 
https://lucene.apache.org/solr/guide/8_5/rule-based-authorization-plugin.html

I have 3 users: "admin", "search", and "indexer".

I have set permissions and user roles:
"permissions": [  {  "name": "all", "role": "admin", "index": 1  },
  { "name": "admin-luke", "collection": "*", "role": "luke", "index": 2, 
"path": "/admin/luke"  },
  { "name": "read", "role": "searching", "index": 3  },
  {  "name": "update", "role": "indexing", "index": 4 }],
"user-role": {  "admin": "admin",
  "search": ["searching","luke"],
  "indexer": "indexing"   }  }
Attached: full output of GET /admin/authorization

So why can't user "indexer" add anything in a collection ?  I always get HTTP 
403 Forbidden.
Using Postman, I click the checkbox to show the password, so I'm sure I typed 
the right one.

Note that user "search" can't use the /select handler either, as should be the 
case with permission to "read".   This user can, however, use the Luke handler, 
as the custom permission allows.

User "admin" can use any API.  So at least the predefined permission "all" does 
work.

Note that the collections were created before enabling authentication and 
authorization.  Could that be the cause of the permission issues ?

Thanks;

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java




permissions.json
Description: permissions.json

SecureRandom algorithm 'NativePRNG'

[Isabelle Giguere]

Hi;

Interagtion test run using Jenkins is failing with this error message:

SecureRandom algorithm 'NativePRNG' is in use by your JVM, which is a 
potentially blocking algorithm on some environments. Please report the details 
of this failure (and your JVM vendor/version) to solr-user@lucene.apache.org. 
You can try to run your tests with -Djava.security.egd=file:/dev/./urandom or 
bypass this check using -Dtest.solr.allowed.securerandom=NativePRNG as a JVM 
option when running tests.

Relevant system information:
java.runtime.version : 1.8.0_161-b12
java.vm.name : Java HotSpot(TM) 64-Bit Server VM
java.vm.vendor : Oracle Corporation
java.vm.version : 25.161-b12
os.arch : amd64
os.name : Linux
os.version : 3.13.0-110-generic

Complete stack trace attached.

Thank you;

Isabelle Giguère
Computational Linguist and Java Developer  |  Semantic Technologies R
Linguiste informaticienne et développeur Java  |  Semantic Technologies R
Phone: 514-908-5406 #225
Website:  www.opentext.com

[http://mimage.opentext.com/alt_content/binary/images/emailsupport-logo-opentext.gif]

This email message is confidential, may be privileged, and is intended for the 
exclusive use of the addressee. Any other person is strictly prohibited from 
disclosing or reproducing it. If the addressee cannot be reached or is unknown 
to you, please inform the sender by return email and delete this email message 
and all copies immediately.
---
Ce message s'adresse uniquement à la personne ou à l'entité indiquée et peut 
contenir des renseignements confidentiels ou privilégiés. Toute consultation, 
retransmission, diffusion ou tout autre usage de l'information contenue dans ce 
message ainsi que toute adoption de mesures fondée sur celle-ci, par des 
personnes ou des entités autres que le destinataire indiqué est interdit. Si ce 
message vous a été transmis par erreur, veuillez en informer l'envoyeur en le 
lui retournant sur-le-champ et supprimer ensuite immédiatement le message, 
ainsi que toutes les pièces jointes, sans le copier, le diffuser ou le 
divulguer.

Error Message

SecureRandom algorithm 'NativePRNG' is in use by your JVM, which is a 
potentially blocking algorithm on some environments. Please report the details 
of this failure (and your JVM vendor/version) to solr-user@lucene.apache.org. 
You can try to run your tests with -Djava.security.egd=file:/dev/./urandom or 
bypass this check using -Dtest.solr.allowed.securerandom=NativePRNG as a JVM 
option when running tests.

Stacktrace

java.lang.AssertionError: SecureRandom algorithm 'NativePRNG' is in use by your 
JVM, which is a potentially blocking algorithm on some environments. Please 
report the details of this failure (and your JVM vendor/version) to 
solr-user@lucene.apache.org. You can try to run your tests with 
-Djava.security.egd=file:/dev/./urandom or bypass this check using 
-Dtest.solr.allowed.securerandom=NativePRNG as a JVM option when running tests.
at __randomizedtesting.SeedInfo.seed([138DC08512C1C206]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertFalse(Assert.java:64)
at 
org.apache.solr.SolrTestCaseJ4.assertNonBlockingRandomGeneratorAvailable(SolrTestCaseJ4.java:2682)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at