from:"Hrishikesh Gadre"


[ 
https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462922#comment-16462922
 ] 

Hrishikesh Gadre commented on SOLR-7344:


[~erickerickson]
{quote}If this much work is going on, is it time to consider ditching Jetty and 
replacing with Netty or whatever?
{quote}
 
No. The synchronous RPCs is the root cause of the distributed deadlock issue 
and no matter what technology we use (Jetty, Netty, Tomcat) it will not go away 
until either we fix it or work around it. 

> Allow Jetty thread pool limits while still avoiding distributed deadlock.
> -
>
> Key: SOLR-7344
> URL: https://issues.apache.org/jira/browse/SOLR-7344
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Priority: Major
> Attachments: SOLR-7344.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7344) Allow Jetty thread pool limits while still avoiding distributed deadlock.


[ 
https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462840#comment-16462840
 ] 

Hrishikesh Gadre edited comment on SOLR-7344 at 5/3/18 5:45 PM:


[~markrmil...@gmail.com]
{quote}Is this deadlock even an issue anymore?

We are Jetty 9 now and it only offers NIO connectors (so long thread per 
request). AFAIK that means requests waiting on IO don't hold a thread.
{quote}
In order to fully utilize NIO connector capability, the application needs to 
use asynchronous servlet APIs (provided as part of Servlet 3 spec). Here is a 
good tutorial that you can take a look: 
https://docs.oracle.com/javaee/7/tutorial/servlets012.htm

Is it possible for us to use this feature for SOLR? Sure, but it will take a 
major rewrite of core parts of SOLR cloud (e.g. distributed querying, 
replication, remote queries etc.) as these components synchronously wait for 
the results of RPC calls. The servlet-request scheduler proposed in this Jira 
([https://github.com/hgadre/servletrequest-scheduler)] internally uses servlet 
3 async API to queue up the requests overflowing the thread-pool capacity, 
ensuring that distributed deadlocks are avoided without requiring *any* change 
in the SOLR cloud functionality.

 


was (Author: hgadre):
[~markrmil...@gmail.com]
{quote}Is this deadlock even an issue anymore?

We are Jetty 9 now and it only offers NIO connectors (so long thread per 
request). AFAIK that means requests waiting on IO don't hold a thread.
{quote}
In order to fully utilize NIO connector capability, the application needs to 
use asynchronous servlet APIs (provided as part of Servlet 3 spec). Here is a 
good tutorial that you can take a look: 
[https://www.javacodegeeks.com/2013/08/async-servlet-feature-of-servlet-3.html]

Is it possible for us to use this feature for SOLR? Sure, but it will take a 
major rewrite of core parts of SOLR cloud (e.g. distributed querying, 
replication, remote queries etc.) as these components synchronously wait for 
the results of RPC calls. The servlet-request scheduler proposed in this Jira 
([https://github.com/hgadre/servletrequest-scheduler)] internally uses servlet 
3 async API to queue up the requests overflowing the thread-pool capacity, 
ensuring that distributed deadlocks are avoided without requiring *any* change 
in the SOLR cloud functionality.

 

> Allow Jetty thread pool limits while still avoiding distributed deadlock.
> -
>
> Key: SOLR-7344
> URL: https://issues.apache.org/jira/browse/SOLR-7344
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Priority: Major
> Attachments: SOLR-7344.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7344) Allow Jetty thread pool limits while still avoiding distributed deadlock.


[ 
https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462840#comment-16462840
 ] 

Hrishikesh Gadre edited comment on SOLR-7344 at 5/3/18 5:43 PM:


[~markrmil...@gmail.com]
{quote}Is this deadlock even an issue anymore?

We are Jetty 9 now and it only offers NIO connectors (so long thread per 
request). AFAIK that means requests waiting on IO don't hold a thread.
{quote}
In order to fully utilize NIO connector capability, the application needs to 
use asynchronous servlet APIs (provided as part of Servlet 3 spec). Here is a 
good tutorial that you can take a look: 
[https://www.javacodegeeks.com/2013/08/async-servlet-feature-of-servlet-3.html]

Is it possible for us to use this feature for SOLR? Sure, but it will take a 
major rewrite of core parts of SOLR cloud (e.g. distributed querying, 
replication, remote queries etc.) as these components synchronously wait for 
the results of RPC calls. The servlet-request scheduler proposed in this Jira 
([https://github.com/hgadre/servletrequest-scheduler)] internally uses servlet 
3 async API to queue up the requests overflowing the thread-pool capacity, 
ensuring that distributed deadlocks are avoided without requiring *any* change 
in the SOLR cloud functionality.

 


was (Author: hgadre):
[~markrmil...@gmail.com]
{quote}Is this deadlock even an issue anymore?

We are Jetty 9 now and it only offers NIO connectors (so long thread per 
request). AFAIK that means requests waiting on IO don't hold a thread.
{quote}
In order to fully utilize NIO connector capability, the application needs to 
use asynchronous servlet APIs (provided as part of Servlet 3 spec). Here is a 
good tutorial that you can take a look: 
[https://plumbr.io/blog/java/how-to-use-asynchronous-servlets-to-improve-performance]

Is it possible for us to use this feature for SOLR? Sure, but it will take a 
major rewrite of core parts of SOLR cloud (e.g. distributed querying, 
replication, remote queries etc.) as these components synchronously wait for 
the results of RPC calls. The servlet-request scheduler proposed in this Jira 
([https://github.com/hgadre/servletrequest-scheduler)] internally uses servlet 
3 async API to queue up the requests overflowing the thread-pool capacity, 
ensuring that distributed deadlocks are avoided without requiring *any* change 
in the SOLR cloud functionality.

 

> Allow Jetty thread pool limits while still avoiding distributed deadlock.
> -
>
> Key: SOLR-7344
> URL: https://issues.apache.org/jira/browse/SOLR-7344
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Priority: Major
> Attachments: SOLR-7344.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7344) Allow Jetty thread pool limits while still avoiding distributed deadlock.


[ 
https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462840#comment-16462840
 ] 

Hrishikesh Gadre commented on SOLR-7344:


[~markrmil...@gmail.com]
{quote}Is this deadlock even an issue anymore?

We are Jetty 9 now and it only offers NIO connectors (so long thread per 
request). AFAIK that means requests waiting on IO don't hold a thread.
{quote}
In order to fully utilize NIO connector capability, the application needs to 
use asynchronous servlet APIs (provided as part of Servlet 3 spec). Here is a 
good tutorial that you can take a look: 
[https://plumbr.io/blog/java/how-to-use-asynchronous-servlets-to-improve-performance]

Is it possible for us to use this feature for SOLR? Sure, but it will take a 
major rewrite of core parts of SOLR cloud (e.g. distributed querying, 
replication, remote queries etc.) as these components synchronously wait for 
the results of RPC calls. The servlet-request scheduler proposed in this Jira 
([https://github.com/hgadre/servletrequest-scheduler)] internally uses servlet 
3 async API to queue up the requests overflowing the thread-pool capacity, 
ensuring that distributed deadlocks are avoided without requiring *any* change 
in the SOLR cloud functionality.

 

> Allow Jetty thread pool limits while still avoiding distributed deadlock.
> -
>
> Key: SOLR-7344
> URL: https://issues.apache.org/jira/browse/SOLR-7344
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Priority: Major
> Attachments: SOLR-7344.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11229) Add a configuration upgrade tool for Solr

2018-04-26 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455561#comment-16455561
 ] 

Hrishikesh Gadre commented on SOLR-11229:
-

[~janhoy] Yes, although the new development for this tool is happening in the 
internal cloudera repo. It includes the upgrade rules for Solr 6 -> 7 upgrade 
as well. I will sync the public github repository ASAP. Eventually we would 
like to incorporate this tool as part of the SOLR distribution if there is 
enough interest from the community.

> Add a configuration upgrade tool for Solr
> -
>
> Key: SOLR-11229
> URL: https://issues.apache.org/jira/browse/SOLR-11229
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
>Priority: Major
>
> Despite widespread enterprise adoption, Solr lacks automated upgrade tooling. 
> It has long been a challenge for users to understand the implications of a 
> Solr upgrade. Users must manually review the Solr release notes to identify 
> configuration changes either to fix backwards incompatibilities or to utilize 
> latest features in the new version. Additionally, users must identify a way 
> to migrate existing index data to the new version (either via an index 
> upgrade or re-indexing the raw data).
> Solr config upgrade tool aims to simplify the upgrade process by providing 
> upgrade instructions tailored to your configuration. These instructions can 
> help you to answer following questions
> - Does my Solr configuration have any backwards incompatible sections? If yes 
> which ones?
> - For each of the incompatibility - what do I need to do to fix this 
> incompatibility? Where can I get more information about why this 
> incompatibility was introduced (e.g. references to Lucene/Solr jiras)?
> - Are there any changes in Lucene/Solr which would require me to do a full 
> reindexing OR can I get away with an index upgrade?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12204) Upgrade commons-fileupload to address CVE-2016-1000031

2018-04-09 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-12204:
---

 Summary: Upgrade commons-fileupload to address CVE-2016-131
 Key: SOLR-12204
 URL: https://issues.apache.org/jira/browse/SOLR-12204
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.2
Reporter: Hrishikesh Gadre
Assignee: Hrishikesh Gadre


Currently SOLR is using 1.3.2 version of commons-fileupload library which is 
susceptible to  CVE-2016-131. We should upgrade the this library to the 
latest version (1.3.3 at the time of writing) to mitigate the security risk.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12120) New plugin type AuditLoggerPlugin

2018-03-29 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419523#comment-16419523
 ] 

Hrishikesh Gadre edited comment on SOLR-12120 at 3/29/18 6:35 PM:
--

{quote} * 
 ** If the latter, i.e. on all nodes, what to use as the "search id" to be able 
to correlate the events from each replica as belonging to the same end-user 
search?{quote}
Not sure if audit log plugin needs to worry about correlation. Typically 
tracing frameworks (e.g. HTrace) provide such functionality.

 

 


was (Author: hgadre):
{quote} * 
 ** If the latter, i.e. on all nodes, what to use as the "search id" to be able 
to correlate the events from each replica as belonging to the same end-user 
search?{quote}
Not sure if audit log plugin needs to do worry about correlation. Typically 
tracing frameworks (e.g. HTrace) provide such functionality.

 

 

> New plugin type AuditLoggerPlugin
> -
>
> Key: SOLR-12120
> URL: https://issues.apache.org/jira/browse/SOLR-12120
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Solr needs a well defined plugin point to implement audit logging 
> functionality, which is independent from whatever {{AuthenticationPlugin}} or 
> {{AuthorizationPlugin}} are in use at the time.
> It seems reasonable to introduce a new plugin type {{AuditLoggerPlugin}}. It 
> could be configured in solr.xml or it could be a third type of plugin defined 
> in {{security.json}}, i.e.
> {code:java}
> {
>   "authentication" : { "class" : ... },
>   "authorization" : { "class" : ... },
>   "auditlogging" : { "class" : "x.y.MyAuditLogger", ... }
> }
> {code}
> We could then instrument SolrDispatchFilter to the audit plugin with an 
> AuditEvent at important points such as successful authentication:
> {code:java}
> auditLoggerPlugin.audit(new SolrAuditEvent(EventType.AUTHENTICATED, 
> request)); 
> {code}
>  We will mark the impl as {{@lucene.experimental}} in the first release to 
> let it settle as people write their own plugin implementations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12120) New plugin type AuditLoggerPlugin

2018-03-29 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419523#comment-16419523
 ] 

Hrishikesh Gadre commented on SOLR-12120:
-

{quote} * 
 ** If the latter, i.e. on all nodes, what to use as the "search id" to be able 
to correlate the events from each replica as belonging to the same end-user 
search?{quote}
Not sure if audit log plugin needs to do worry about correlation. Typically 
tracing frameworks (e.g. HTrace) provide such functionality.

 

 

> New plugin type AuditLoggerPlugin
> -
>
> Key: SOLR-12120
> URL: https://issues.apache.org/jira/browse/SOLR-12120
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Solr needs a well defined plugin point to implement audit logging 
> functionality, which is independent from whatever {{AuthenticationPlugin}} or 
> {{AuthorizationPlugin}} are in use at the time.
> It seems reasonable to introduce a new plugin type {{AuditLoggerPlugin}}. It 
> could be configured in solr.xml or it could be a third type of plugin defined 
> in {{security.json}}, i.e.
> {code:java}
> {
>   "authentication" : { "class" : ... },
>   "authorization" : { "class" : ... },
>   "auditlogging" : { "class" : "x.y.MyAuditLogger", ... }
> }
> {code}
> We could then instrument SolrDispatchFilter to the audit plugin with an 
> AuditEvent at important points such as successful authentication:
> {code:java}
> auditLoggerPlugin.audit(new SolrAuditEvent(EventType.AUTHENTICATED, 
> request)); 
> {code}
>  We will mark the impl as {{@lucene.experimental}} in the first release to 
> let it settle as people write their own plugin implementations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12120) New plugin type AuditLoggerPlugin

2018-03-29 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419518#comment-16419518
 ] 

Hrishikesh Gadre commented on SOLR-12120:
-

[~janhoy] Sorry for late reply.
{quote}Should we strive to have only *one* audit log event per Solr request, or 
is it common to have multiple as currently done in this patch, i.e. one for 
successful authentication and another for authorization?
{quote}
I think there is no need to log authentication success events when 
authorization is configured. So in that case we can just track authentication 
failures. It may also be a good idea to support suppressing some of these 
events (e.g. a user may only care about actions performed by authenticated 
users. So we may not want to generate authentication failure events in that 
case).
{quote} * Should we log internal requests, i.e. overseer actions, or requests 
stemming from auto-scaling triggers etc?{quote}
I don't think audit log plugin needs to care about internal vs. external 
requests. It should just log every incoming request. At least this is how I 
have implemented audit logs for Solr in Sentry.
{quote}For distributed requests, should we log only on the first node, or on 
every replica that the request is distributed to?
{quote}
Same as above. Just by logging every incoming request, we can avoid all these 
complications.

 

> New plugin type AuditLoggerPlugin
> -
>
> Key: SOLR-12120
> URL: https://issues.apache.org/jira/browse/SOLR-12120
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Solr needs a well defined plugin point to implement audit logging 
> functionality, which is independent from whatever {{AuthenticationPlugin}} or 
> {{AuthorizationPlugin}} are in use at the time.
> It seems reasonable to introduce a new plugin type {{AuditLoggerPlugin}}. It 
> could be configured in solr.xml or it could be a third type of plugin defined 
> in {{security.json}}, i.e.
> {code:java}
> {
>   "authentication" : { "class" : ... },
>   "authorization" : { "class" : ... },
>   "auditlogging" : { "class" : "x.y.MyAuditLogger", ... }
> }
> {code}
> We could then instrument SolrDispatchFilter to the audit plugin with an 
> AuditEvent at important points such as successful authentication:
> {code:java}
> auditLoggerPlugin.audit(new SolrAuditEvent(EventType.AUTHENTICATED, 
> request)); 
> {code}
>  We will mark the impl as {{@lucene.experimental}} in the first release to 
> let it settle as people write their own plugin implementations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12120) New plugin type AuditLoggerPlugin

2018-03-23 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411877#comment-16411877
 ] 

Hrishikesh Gadre edited comment on SOLR-12120 at 3/23/18 6:44 PM:
--

[~janhoy] Thanks for this great feature!
{quote}can you have a look and consider whether your Audit logger plugin would 
be possible to realise with this API?
{quote}
I took a cursory look and we should be integrate this with the Sentry 
authorization framework. I will do a more thorough review next couple of days 
and will let you know. 

 
{quote}I made it asynchronous so you can call 
{{auditLoggerPlugin.auditAsync(event);}} and continue the request immediately, 
while the audit framework processes the events on queue with a background 
thread. Very simple but powerful concept.
{quote}
While I get the value of async processing, I am not sure why the audit logging 
plugin needs to expose this concept. This can be implemented inside the 
concrete implementation of AuditLoggerPlugin (e.g. log4j does support [async 
appender|https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/AsyncAppender.html]).
 My concern is that HttpSolrCall.java is using this method directly. This makes 
async behavior mandatory for all (?) audit logging implementations. What do you 
think?

 


was (Author: hgadre):
[~janhoy] Thanks for this great feature!
{quote}can you have a look and consider whether your Audit logger plugin would 
be possible to realise with this API?
{quote}
I took a cursory look and we should be integrate this with the Sentry 
authorization framework. I will do a more thorough review next couple of days 
and will let you know. 

 
{quote}I made it asynchronous so you can call 
{{auditLoggerPlugin.auditAsync(event);}} and continue the request immediately, 
while the audit framework processes the events on queue with a background 
thread. Very simple but powerful concept.
{quote}
While I get the value of async processing, I am not sure why the audit logging 
plugin needs to expose this concept. This can be implemented inside the 
concrete implementation of AuditLoggerPlugin (e.g. log4j does support [async 
appender|https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/AsyncAppender.html]).
 My concern is that HttpSolrCall.java is using this method directly. This makes 
async behavior is mandatory for all (?) audit logging implementations. What do 
you think?

 

> New plugin type AuditLoggerPlugin
> -
>
> Key: SOLR-12120
> URL: https://issues.apache.org/jira/browse/SOLR-12120
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Solr needs a well defined plugin point to implement audit logging 
> functionality, which is independent from whatever {{AuthenticationPlugin}} or 
> {{AuthorizationPlugin}} are in use at the time.
> It seems reasonable to introduce a new plugin type {{AuditLoggerPlugin}}. It 
> could be configured in solr.xml or it could be a third type of plugin defined 
> in {{security.json}}, i.e.
> {code:java}
> {
>   "authentication" : { "class" : ... },
>   "authorization" : { "class" : ... },
>   "auditlogging" : { "class" : "x.y.MyAuditLogger", ... }
> }
> {code}
> We could then instrument SolrDispatchFilter to the audit plugin with an 
> AuditEvent at important points such as successful authentication:
> {code:java}
> auditLoggerPlugin.audit(new SolrAuditEvent(EventType.AUTHENTICATED, 
> request)); 
> {code}
>  We will mark the impl as {{@lucene.experimental}} in the first release to 
> let it settle as people write their own plugin implementations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12120) New plugin type AuditLoggerPlugin

2018-03-23 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411877#comment-16411877
 ] 

Hrishikesh Gadre commented on SOLR-12120:
-

[~janhoy] Thanks for this great feature!
{quote}can you have a look and consider whether your Audit logger plugin would 
be possible to realise with this API?
{quote}
I took a cursory look and we should be integrate this with the Sentry 
authorization framework. I will do a more thorough review next couple of days 
and will let you know. 

 
{quote}I made it asynchronous so you can call 
{{auditLoggerPlugin.auditAsync(event);}} and continue the request immediately, 
while the audit framework processes the events on queue with a background 
thread. Very simple but powerful concept.
{quote}
While I get the value of async processing, I am not sure why the audit logging 
plugin needs to expose this concept. This can be implemented inside the 
concrete implementation of AuditLoggerPlugin (e.g. log4j does support [async 
appender|https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/AsyncAppender.html]).
 My concern is that HttpSolrCall.java is using this method directly. This makes 
async behavior is mandatory for all (?) audit logging implementations. What do 
you think?

 

> New plugin type AuditLoggerPlugin
> -
>
> Key: SOLR-12120
> URL: https://issues.apache.org/jira/browse/SOLR-12120
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Solr needs a well defined plugin point to implement audit logging 
> functionality, which is independent from whatever {{AuthenticationPlugin}} or 
> {{AuthorizationPlugin}} are in use at the time.
> It seems reasonable to introduce a new plugin type {{AuditLoggerPlugin}}. It 
> could be configured in solr.xml or it could be a third type of plugin defined 
> in {{security.json}}, i.e.
> {code:java}
> {
>   "authentication" : { "class" : ... },
>   "authorization" : { "class" : ... },
>   "auditlogging" : { "class" : "x.y.MyAuditLogger", ... }
> }
> {code}
> We could then instrument SolrDispatchFilter to the audit plugin with an 
> AuditEvent at important points such as successful authentication:
> {code:java}
> auditLoggerPlugin.audit(new SolrAuditEvent(EventType.AUTHENTICATED, 
> request)); 
> {code}
>  We will mark the impl as {{@lucene.experimental}} in the first release to 
> let it settle as people write their own plugin implementations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11781) Pass impersonator info to the authorization plugin

2018-03-16 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402083#comment-16402083
 ] 

Hrishikesh Gadre edited comment on SOLR-11781 at 3/16/18 4:04 PM:
--

[~janhoy] Typically audit logging is closely related to authorization as we 
want to identify which "authenticated" user tried to perform an operation that 
was not authorized. I enhanced AuthorizationContext to explicitly pass the 
impersonator username (please find attached patch) and implemented audit 
logging inside the authorization plugin.
{quote}Is there any method to pass information (except for the user principle) 
from Authentication to authorization? Can Auth plugin fill information in 
AuthorizationContext?
{quote}
While authentication plugin can pass any arbitrary information via 
HttpServletRequest object (e.g. using custom attributes), authorization context 
does not provide access to raw HttpServletRequest object. In my case, 
KerberosPlugin is already passing impersonator user name. I just had to add 
another method in AuthorizationContext to forward this info to the 
Authorization plugin. I wonder if it would make more sense to expose 
HttpServletRequest object directly to authorization plugin? This way 
authentication and authorization plugins can pass any information via 
HttpServletRequest object. I am not sure if the original design did not support 
it intentionally. What do you think?


was (Author: hgadre):
[~janhoy] Typically audit logging is closely related to authorization as we 
want to identify which "authenticated" user tried to perform an operation that 
was not authorized. I enhanced AuthorizationContext to explicitly pass the 
impersonator username (please find attached patch) and implemented audit 
logging inside the authorization plugin.
{quote}Is there any method to pass information (except for the user principle) 
from Authentication to authorization? Can Auth plugin fill information in 
AuthorizationContext?
{quote}
While authentication plugin can pass any arbitrary information via 
HttpServletRequest object (e.g. using custom attributes), authorization context 
does not provide access to raw HttpServletRequest object. In my case, 
KerberosPlugin is already passing impersonator user name. I just had to add 
another method in AuthorizationContext to forward this info to the 
Authorization plugin. I wonder if it would make more sense to expose 
HttpServletRequest object directly to authorization plugin? This way 
authentication and authorization plugins can pass any information via 
HttpServletRequest object. I am not sure if the original design did not support 
it intentionally. What do you think?

 

 

 

 

> Pass impersonator info to the authorization plugin
> --
>
> Key: SOLR-11781
> URL: https://issues.apache.org/jira/browse/SOLR-11781
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Affects Versions: 7.0
>Reporter: Hrishikesh Gadre
>Priority: Minor
> Attachments: SOLR-11781-00.patch
>
>
> SENTRY-1475 implemented Solr authorization plugin based on Sentry. This also 
> includes the audit log functionality in Sentry. Currently authorization 
> context is not providing the impersonator information which is required for 
> the audit logs. We should improve Solr authorization framework to pass this 
> extra information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11781) Pass impersonator info to the authorization plugin

2018-03-16 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402083#comment-16402083
 ] 

Hrishikesh Gadre commented on SOLR-11781:
-

[~janhoy] Typically audit logging is closely related to authorization as we 
want to identify which "authenticated" user tried to perform an operation that 
was not authorized. I enhanced AuthorizationContext to explicitly pass the 
impersonator username (please find attached patch) and implemented audit 
logging inside the authorization plugin.
{quote}Is there any method to pass information (except for the user principle) 
from Authentication to authorization? Can Auth plugin fill information in 
AuthorizationContext?
{quote}
While authentication plugin can pass any arbitrary information via 
HttpServletRequest object (e.g. using custom attributes), authorization context 
does not provide access to raw HttpServletRequest object. In my case, 
KerberosPlugin is already passing impersonator user name. I just had to add 
another method in AuthorizationContext to forward this info to the 
Authorization plugin. I wonder if it would make more sense to expose 
HttpServletRequest object directly to authorization plugin? This way 
authentication and authorization plugins can pass any information via 
HttpServletRequest object. I am not sure if the original design did not support 
it intentionally. What do you think?

 

 

 

 

> Pass impersonator info to the authorization plugin
> --
>
> Key: SOLR-11781
> URL: https://issues.apache.org/jira/browse/SOLR-11781
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.0
>Reporter: Hrishikesh Gadre
>Priority: Minor
> Attachments: SOLR-11781-00.patch
>
>
> SENTRY-1475 implemented Solr authorization plugin based on Sentry. This also 
> includes the audit log functionality in Sentry. Currently authorization 
> context is not providing the impersonator information which is required for 
> the audit logs. We should improve Solr authorization framework to pass this 
> extra information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11781) Pass impersonator info to the authorization plugin

2018-03-16 Thread Hrishikesh Gadre (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-11781:

Attachment: SOLR-11781-00.patch

> Pass impersonator info to the authorization plugin
> --
>
> Key: SOLR-11781
> URL: https://issues.apache.org/jira/browse/SOLR-11781
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.0
>    Reporter: Hrishikesh Gadre
>Priority: Minor
> Attachments: SOLR-11781-00.patch
>
>
> SENTRY-1475 implemented Solr authorization plugin based on Sentry. This also 
> includes the audit log functionality in Sentry. Currently authorization 
> context is not providing the impersonator information which is required for 
> the audit logs. We should improve Solr authorization framework to pass this 
> extra information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11781) Pass impersonator info to the authorization plugin

2017-12-19 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-11781:
---

 Summary: Pass impersonator info to the authorization plugin
 Key: SOLR-11781
 URL: https://issues.apache.org/jira/browse/SOLR-11781
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.0
Reporter: Hrishikesh Gadre
Priority: Minor


SENTRY-1475 implemented Solr authorization plugin based on Sentry. This also 
includes the audit log functionality in Sentry. Currently authorization context 
is not providing the impersonator information which is required for the audit 
logs. We should improve Solr authorization framework to pass this extra 
information.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11623) Every request handler in Solr should implement PermissionNameProvider interface

2017-11-08 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-11623:
---

 Summary: Every request handler in Solr should implement 
PermissionNameProvider interface
 Key: SOLR-11623
 URL: https://issues.apache.org/jira/browse/SOLR-11623
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.1
Reporter: Hrishikesh Gadre


Solr authorization framework expects request handler to implement 
PermissionNameProvider interface so that the type of the permission for the 
request can be extracted. Currently not all request handlers implement 
PermissionNameProvider, requiring authorization plugin implementation to check 
this case explicitly and return OK. During code review of SENTRY-1475, this 
issue was discussed. Since  PermissionNameProvider.Name enum provides "ALL" 
permission type, it should be possible to have every request handler to 
implement PermissionNameProvider interface and provide "ALL" permission type if 
no authorization checks are necessary.

The secondary benefit of this work would be that we can review all the request 
handlers and ensure that we aren't missing authorization support for any 
request handlers which provide sensitive information.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9120) o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- NoSuchFileException

2017-11-07 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242577#comment-16242577
 ] 

Hrishikesh Gadre commented on SOLR-9120:


[~varunthacker]

>>Essentially we need to take the same fix and apply it here

It may be a good idea to actually figure out what is the root cause of this 
behavior. Seems quite weird to me. There may be some fundamental issue at the 
lucene level ?

> o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- 
> NoSuchFileException
> 
>
> Key: SOLR-9120
> URL: https://issues.apache.org/jira/browse/SOLR-9120
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Markus Jelsma
> Attachments: SOLR-9120.patch, SOLR-9120.patch
>
>
> On Solr 6.0, we frequently see the following errors popping up:
> {code}
> java.nio.file.NoSuchFileException: 
> /var/lib/solr/logs_shard2_replica1/data/index/segments_2c5
>   at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>   at 
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>   at 
> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
>   at 
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
>   at java.nio.file.Files.readAttributes(Files.java:1737)
>   at java.nio.file.Files.size(Files.java:2332)
>   at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
>   at 
> org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:131)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:597)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:585)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:137)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
>   at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:518)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>   at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>   at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>   at 
> org.eclipse.jetty.ut

[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.

2017-10-26 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221582#comment-16221582
 ] 

Hrishikesh Gadre commented on SOLR-11556:
-

Looks good. Core admin APIs may have similar issue?

> Backup/Restore with multiple BackupRepository objects defined results in the 
> wrong repo being used.
> ---
>
> Key: SOLR-11556
> URL: https://issues.apache.org/jira/browse/SOLR-11556
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore
>Affects Versions: 6.3
>Reporter: Timothy Potter
>Assignee: Timothy Potter
> Attachments: SOLR-11556.patch
>
>
> I defined two repos for backup/restore, one local and one remote on GCS, e.g.
> {code}
> 
>  class="org.apache.solr.core.backup.repository.HdfsBackupRepository" 
> default="false">
>  ...
> 
>  class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" 
> default="false">
>   /tmp/solr-backups
> 
>  
> {code}
> Since the CollectionHandler does not pass the "repository" param along, once 
> the BackupCmd gets the ZkNodeProps, it selects the wrong repo! 
> The error I'm seeing is:
> {code}
> 2017-10-26 17:07:27.326 ERROR 
> (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [   ] 
> o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: 
> backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not 
> installed
> at java.nio.file.Paths.get(Paths.java:147)
> at 
> org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82)
> at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99)
> at 
> org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
> at 
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Notice the Local backup repo is being selected in the BackupCmd even though I 
> passed repository=hdfs in my backup command, e.g.
> {code}
> curl 
> "http://localhost:8983/solr/admin/collections?action=BACKUP=foo=foo=gs://tjp-solr-test/backups=hdfs;
> {code} 
> I think the fix here is to include the repository param, see patch. I'll fix 
> for the next 7.x release and those on 6 can just apply the patch here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11554) Support handling OPTIONS request for Hadoop authentication filter

2017-10-26 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-11554:
---

 Summary: Support handling OPTIONS request for Hadoop 
authentication filter
 Key: SOLR-11554
 URL: https://issues.apache.org/jira/browse/SOLR-11554
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.4
Reporter: Hrishikesh Gadre
Priority: Minor


As part of SOLR-9513 we added a Solr authentication plugin which uses Hadoop 
security framework. The HTTP client interface provided by Hadoop framework does 
not send the authentication information preemptively. Instead it sends an 
OPTIONS request first. If the server responds with 401 error, it resends the 
request with the proper authentication information. This jira is to handle the 
OPTIONS request as part of the Solr authentication plugin for Hadoop.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11238) Solr authorization plugin is not able to pass additional params downstream


 [ 
https://issues.apache.org/jira/browse/SOLR-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre resolved SOLR-11238.
-
Resolution: Won't Fix

> Solr authorization plugin is not able to pass additional params downstream
> --
>
> Key: SOLR-11238
> URL: https://issues.apache.org/jira/browse/SOLR-11238
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-11238.patch
>
>
> Authorization checks in Solr are implemented by invoking configured 
> authorization plugin with AuthorizationContext object. The plugin is expected 
> to return an AuthorizationResponse object which provides the result (which 
> can be OK/FORBIDDEN/PROMPT).
> In some cases (e.g. document level security implemented in Apache Sentry), it 
> is useful for the authorization plugin to add (or override) the request 
> parameters sent by the user (which are represented as SolrParams in 
> [AuthorizationContext| 
> https://github.com/apache/lucene-solr/blob/3cbbecca026eb2a9491fa4a24ecc2c43c26e58bd/solr/core/src/java/org/apache/solr/security/AuthorizationContext.java#L38]).
>  This jira is to introduce an ability to customize the parameters by the 
> authorization plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11238) Solr authorization plugin is not able to pass additional params downstream


[ 
https://issues.apache.org/jira/browse/SOLR-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217752#comment-16217752
 ] 

Hrishikesh Gadre commented on SOLR-11238:
-

Update - 2 months after posting this patch, I found an alternative to extract 
username and associated roles in the search component without requiring this 
fix.

https://github.com/hgadre/sentry/blob/a4ecc83d3e92c81e61aa5441102a9bcd6e90d421/sentry-solr/solr-sentry-handlers/src/main/java/org/apache/solr/handler/component/QueryDocAuthorizationComponent.java

I think we should close this jira as "Won't fix" since currently there is no 
use-case which requires this functionality. 

> Solr authorization plugin is not able to pass additional params downstream
> --
>
> Key: SOLR-11238
> URL: https://issues.apache.org/jira/browse/SOLR-11238
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>Reporter: Hrishikesh Gadre
> Attachments: SOLR-11238.patch
>
>
> Authorization checks in Solr are implemented by invoking configured 
> authorization plugin with AuthorizationContext object. The plugin is expected 
> to return an AuthorizationResponse object which provides the result (which 
> can be OK/FORBIDDEN/PROMPT).
> In some cases (e.g. document level security implemented in Apache Sentry), it 
> is useful for the authorization plugin to add (or override) the request 
> parameters sent by the user (which are represented as SolrParams in 
> [AuthorizationContext| 
> https://github.com/apache/lucene-solr/blob/3cbbecca026eb2a9491fa4a24ecc2c43c26e58bd/solr/core/src/java/org/apache/solr/security/AuthorizationContext.java#L38]).
>  This jira is to introduce an ability to customize the parameters by the 
> authorization plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr test framework - Locale randomization

2017-10-24 Thread Hrishikesh Gadre

I filed LUCENE-8009  to
track this work. I will submit a patch soon.

Thanks
Hrishikesh


On Mon, Oct 23, 2017 at 9:46 AM, Chris Hostetter 
wrote:

>
> : SuppressLocaleRandomization(locale="en-US", reason="Derby doesn't work
> : with complex locales, see issue ...")?
>
> that would be cool ... with 'reason' & 'locale' as optional, both
> defaulting to "" (whch for Locale should result in Locale.ROOT IIUC?)
>
> For consistency with existing annotations a non-optional bugUrl would be
> good.
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Created] (LUCENE-8009) Support disabling Locale randomization as part of Lucene test framework

Hrishikesh Gadre created LUCENE-8009:


 Summary: Support disabling Locale randomization as part of Lucene 
test framework
 Key: LUCENE-8009
 URL: https://issues.apache.org/jira/browse/LUCENE-8009
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/test-framework
Affects Versions: 7.1
Reporter: Hrishikesh Gadre
Priority: Minor


The Lucene test framework randomizes the Locale configuration to test the 
software in different locale settings.
https://github.com/apache/lucene-solr/blob/e2521b2a8baabdaf43b92192588f51e042d21e97/lucene/test-framework/src/java/org/apache/lucene/util/TestRuleSetupAndRestoreClassEnv.java#L206-L209

While this is a very good practice from engineering perspective, it causes 
issues when the Lucene/Solr test framework is used with third-party components 
which may have issues working with a subset of locale settings. e.g. for 
Solr/Sentry integration (SENTRY-1475), we are using Solr test framework to test 
the sentry authorization plugin for Solr. For unit-testing, it uses Apache 
Derby. We have found multiple cases when derby fail to initialize for a locale 
configured by Solr test framework. This causes tests to fail and create a 
confusion with respect to the quality of the integration source-code. Since the 
Derby failures are not related to Solr/Sentry integration, we would like to 
avoid such nasty surprises by suppressing the locale randomization. This is 
similar to the way we suppress Solr SSL configuration 
(@SolrTestCaseJ4.SuppressSSL).

Please refer to discussion on dev mailing list for more context,
http://lucene.472066.n3.nabble.com/Solr-test-framework-Locale-randomization-td4359671.html




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11543) Support fine grained authorization for multi-tenancy usecases


[ 
https://issues.apache.org/jira/browse/SOLR-11543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217706#comment-16217706
 ] 

Hrishikesh Gadre commented on SOLR-11543:
-

[~ichattopadhyaya] [~noble.paul] let me know your thoughts on this. As part of 
Sentry integration, I have added logic to parse the parameters passed as part 
of the admin operation and implement necessary permissions check. But this is 
brittle to maintain as we upgrade solr versions. Also other authorization 
plugins may have similar requirements. So it would be better to implement this 
logic cleanly as part of solr authorization framework.

https://github.com/hgadre/sentry/blob/a4ecc83d3e92c81e61aa5441102a9bcd6e90d421/sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/authz/SolrAuthzUtil.java#L101
https://github.com/hgadre/sentry/blob/a4ecc83d3e92c81e61aa5441102a9bcd6e90d421/sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/authz/SentrySolrPluginImpl.java#L188

> Support fine grained authorization for multi-tenancy usecases
> -
>
> Key: SOLR-11543
> URL: https://issues.apache.org/jira/browse/SOLR-11543
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.2
>    Reporter: Hrishikesh Gadre
>
> As part of SOLR-7275, we implemented a pluggable authorization framework for 
> Solr. But the current design (specifically wrt to admin apis) is not quite 
> suitable to support multi-tenant Solr cloud  in a secure environment. In such 
> environment, there are two types of system admins (a) admins belonging to the 
> service provider and (b) admins belonging to an individual tenant. 
> Type (a) admins are responsible to setup solr cluster e.g. setting up solr 
> servers, managing security etc.
> Type (b) admins on the other hand are responsible to perform collection-level 
> admin operations e.g. creating/deleting/configuring/reloading collections 
> belonging to the tenant. From security perspective it is important to ensure 
> that such admin is able to operate only on the collections belonging to the 
> respective tenant. 
> The current design of authorization framework has following limitations
> (a) it does not provide the resource names associated with the operation as 
> part of the authorization context (Ref: 
> https://github.com/apache/lucene-solr/blob/ef7b525123fc32b0703f1579c2422957d1b0a2ab/solr/core/src/java/org/apache/solr/security/PermissionNameProvider.java).
>  Here the collection names are hard-coded to either null or "*" value in the 
> parameter for Name instance. This results in providing global admin 
> privileges to the user. In a multi-tenant environment, this is not acceptable.
> (b) The authorization framework assumes that only a single permission is 
> necessary for any given operation (Ref: 
> https://github.com/apache/lucene-solr/blob/ef7b525123fc32b0703f1579c2422957d1b0a2ab/solr/core/src/java/org/apache/solr/security/PermissionNameProvider.java#L77).
>  For some of the admin operations (e.g. MIGRATE collections admin API) we 
> need to check additional permissions for multi-tenancy. Specifically in case 
> of MIGRATE API we need to check following permissions
>  - collections admin privilege (Update)
>  - collection privilege for the source collection (Read)
>  - collection privilege for the destination collection (Update)
> Hence ideally PermissionNameProvider needs to return a list of permission 
> names instead of a single permission (in most cases the list will contain a 
> single element). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11543) Support fine grained authorization for multi-tenancy usecases

Hrishikesh Gadre created SOLR-11543:
---

Summary: Support fine grained authorization for multi-tenancy
usecases
Key: SOLR-11543
URL: https://issues.apache.org/jira/browse/SOLR-11543
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 5.2
Reporter: Hrishikesh Gadre

As part of SOLR-7275, we implemented a pluggable authorization framework for
Solr. But the current design (specifically wrt to admin apis) is not quite
suitable to support multi-tenant Solr cloud in a secure environment. In such
environment, there are two types of system admins (a) admins belonging to the
service provider and (b) admins belonging to an individual tenant.

Type (a) admins are responsible to setup solr cluster e.g. setting up solr
servers, managing security etc.
Type (b) admins on the other hand are responsible to perform collection-level
admin operations e.g. creating/deleting/configuring/reloading collections
belonging to the tenant. From security perspective it is important to ensure
that such admin is able to operate only on the collections belonging to the
respective tenant.

The current design of authorization framework has following limitations
(a) it does not provide the resource names associated with the operation as
part of the authorization context (Ref:
https://github.com/apache/lucene-solr/blob/ef7b525123fc32b0703f1579c2422957d1b0a2ab/solr/core/src/java/org/apache/solr/security/PermissionNameProvider.java).
Here the collection names are hard-coded to either null or "*" value in the
parameter for Name instance. This results in providing global admin privileges
to the user. In a multi-tenant environment, this is not acceptable.

(b) The authorization framework assumes that only a single permission is
necessary for any given operation (Ref:
https://github.com/apache/lucene-solr/blob/ef7b525123fc32b0703f1579c2422957d1b0a2ab/solr/core/src/java/org/apache/solr/security/PermissionNameProvider.java#L77).
For some of the admin operations (e.g. MIGRATE collections admin API) we need
to check additional permissions for multi-tenancy. Specifically in case of
MIGRATE API we need to check following permissions
- collections admin privilege (Update)
- collection privilege for the source collection (Read)
- collection privilege for the destination collection (Update)

Hence ideally PermissionNameProvider needs to return a list of permission names
instead of a single permission (in most cases the list will contain a single
element).

--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr test framework - Locale randomization

2017-10-20 Thread Hrishikesh Gadre

>>I think you're going to be stuck with doing something like
https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/
apache/solr/cloud/KerberosTestServices.java#L55 where there is a hard-coded
list of locales that we know are broken.

@Mike - yes that's what I am doing currently.

>>See the documentation for the java Locale class for what I mean, it is
very thorough. https://docs.oracle.com/javase/8/docs/api/java/util/
Locale.html

Thanks Robert. I will review the javadoc for better understanding.


>>Yes but these are not bugs in what lucene is doing.
>>These are bugs in third party software that completely break under
certain Locales, because they don't know how to handle some newer
locales (broken parsing and so on).

Yes that seems to be case. But in this case, we are using Derby for unit
testing only. In production
deployments we would be using a standard RDBMS (e.g. mysql, postgres etc.).
So I am wondering
if we should provide a knob to suppress the randomization in such cases
(just to avoid such issues with third-party software).
e.g. we already have an annotation to suppress SSL
(@SolrTestCaseJ4.SuppressSSL).  May be we could do something similar
for the Locale setting?

To ensure that such setting is not abused inside lucene-solr repo, we can
add it to the list of forbidden APIs.

If this is acceptable, I will go ahead and create a jira.

Thanks
Hrishikesh


On Fri, Oct 20, 2017 at 1:50 PM, Robert Muir <rcm...@gmail.com> wrote:

> Yes but these are not bugs in what lucene is doing. It is just setting
> a random locale.
>
> These are bugs in third party software that completely break under
> certain Locales, because they don't know how to handle some newer
> locales (broken parsing and so on). Such code does not handle the fact
> that Locales can be more than language + country + variant. Since Java
> 1.7 they can also have script (seen in your example) and extensions
> and support BCP 47.
>
> See the documentation for the java Locale class for what I mean, it is
> very thorough. https://docs.oracle.com/javase/8/docs/api/java/util/
> Locale.html
>
>
> On Fri, Oct 20, 2017 at 4:40 PM, Mike Drob <md...@apache.org> wrote:
> > I think you're going to be stuck with doing something like
> > https://github.com/apache/lucene-solr/blob/master/solr/
> core/src/test/org/apache/solr/cloud/KerberosTestServices.java#L55
> > where there is a hard-coded list of locales that we know are broken.
> >
> > On Fri, Oct 20, 2017 at 3:31 PM, Hrishikesh Gadre <gadre.s...@gmail.com>
> > wrote:
> >>
> >> Hi Dawid,
> >>
> >> Thanks for the feedback.
> >>
> >> Here is one failure scenario,
> >>
> >> Locale configured (via -Dtests.locale) -> sr-Latn
> >>
> >> The error message,
> >>
> >> ERROR XBM0X: Supplied locale description 'sr__#Latn' is invalid,
> expecting
> >> ln[_CO[_variant]]
> >>
> >> ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter
> >> ISO-3166 country codes, see java.util.Locale.
> >>
> >> Note that if I use "sr-Latn-BA" instead, the test passes. My gut feeling
> >> is that "sr-Latn" is not a valid locale string as it is not listed here,
> >> http://www.oracle.com/technetwork/java/javase/java8locales-2095355.html
> >>
> >>
> >> Another failure is
> >>
> >> Locale configured (via -Dtests.locale) -> und
> >>
> >> The error message is
> >>
> >> Supplied locale description '' is invalid, expecting ln[_CO[_variant]]
> >>
> >> ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter
> >> ISO-3166 country codes, see java.util.Locale.
> >>
> >> For the time being, I am hard-coding these failure causing locales in
> the
> >> junit assume(...) so that I can skip the execution. But this is not
> >> full-proof since there may be more locale configurations which may not
> work
> >> with Derby. So I wonder if there is any way to suppress this locale
> >> randomization altogether?
> >>
> >> Thanks
> >> Hrishikesh
> >>
> >>
> >>
> >> On Fri, Oct 20, 2017 at 12:39 PM, Dawid Weiss <dawid.we...@gmail.com>
> >> wrote:
> >>>
> >>> Only valid locales (for your Java) are selected, so this has to be an
> >>> error. What failures do you see? Perhaps they should be reported to
> >>> Derby?
> >>>
> >>> Dawid
> >>>
> >>> On Fri, Oct 20, 2017 at 8:14 PM, Hrishikesh Gadre <
> gadre.s...@gmail.com>
> >&

Re: Solr test framework - Locale randomization

2017-10-20 Thread Hrishikesh Gadre

Hi Dawid,

Thanks for the feedback.

Here is one failure scenario,

Locale configured (via -Dtests.locale) -> sr-Latn

The error message,

ERROR XBM0X: Supplied locale description 'sr__#Latn' is invalid, expecting
ln[_CO[_variant]]

ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter
ISO-3166 country codes, see java.util.Locale.
Note that if I use "sr-Latn-BA" instead, the test passes. My gut feeling is
that "sr-Latn" is not a valid locale string as it is not listed here,
http://www.oracle.com/technetwork/java/javase/java8locales-2095355.html

Another failure is

Locale configured (via -Dtests.locale) -> und

The error message is

Supplied locale description '' is invalid, expecting ln[_CO[_variant]]

ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter
ISO-3166 country codes, see java.util.Locale.

For the time being, I am hard-coding these failure causing locales in the
junit assume(...) so that I can skip the execution. But this is not
full-proof since there may be more locale configurations which may not work
with Derby. So I wonder if there is any way to suppress this locale
randomization altogether?
Thanks
Hrishikesh

On Fri, Oct 20, 2017 at 12:39 PM, Dawid Weiss <dawid.we...@gmail.com> wrote:

> Only valid locales (for your Java) are selected, so this has to be an
> error. What failures do you see? Perhaps they should be reported to
> Derby?
>
> Dawid
>
> On Fri, Oct 20, 2017 at 8:14 PM, Hrishikesh Gadre <gadre.s...@gmail.com>
> wrote:
> > Hi,
> >
> > I am currently implementing solr authorization plugin backed by Apache
> > Sentry. For the unit tests, I am using Solr test framework (specifically
> > SolrCloudTestCase class). Occasionally I see unit test failures since the
> > sentry tests use Derby in-memory database and it doesn't work properly
> for
> > some of the Locale(s) configured by the Solr test framework.
> >
> > Couple of questions
> >
> > (a) Does the Solr test framework generates only valid Locale(s) or a mix
> of
> > valid/invalid Locale(s) ? The reason I am asking is that I have a test
> > failure with Locale as "sr-Latn". But it is not included in the list of
> > valid Locales supported by Java 8
> > (http://www.oracle.com/technetwork/java/javase/java8locales-2095355.html
> ).
> >
> > (b) Is there a way to turn off this Locale randomization?
> >
> >
> > Thanks
> > Hrishikesh
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Solr test framework - Locale randomization

2017-10-20 Thread Hrishikesh Gadre

Hi,

I am currently implementing solr authorization plugin backed by Apache
Sentry. For the unit tests, I am using Solr test framework (specifically
SolrCloudTestCase class). Occasionally I see unit test failures since the
sentry tests use Derby in-memory database and it doesn't work properly for
some of the Locale(s) configured by the Solr test framework.

Couple of questions

(a) Does the Solr test framework generates only valid Locale(s) or a mix of
valid/invalid Locale(s) ? The reason I am asking is that I have a test
failure with Locale as "sr-Latn". But it is not included in the list of
valid Locales supported by Java 8 (
http://www.oracle.com/technetwork/java/javase/java8locales-2095355.html).

(b) Is there a way to turn off this Locale randomization?


Thanks
Hrishikesh

[jira] [Commented] (SOLR-11209) Upgrade HttpClient to 4.5.3

2017-08-25 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141912#comment-16141912
 ] 

Hrishikesh Gadre commented on SOLR-11209:
-

[~risdenk] OK thanks. BTW all the unit tests are passing with this change. We 
are also using this patch in our internal environment and have not seen any 
issues with it. 

[~elyograg] [~shalinmangar] [~markrmil...@gmail.com] can you please review the 
patch?

> Upgrade HttpClient to 4.5.3
> ---
>
> Key: SOLR-11209
> URL: https://issues.apache.org/jira/browse/SOLR-11209
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Reporter: Hrishikesh Gadre
>Priority: Minor
>
> We have not upgraded HttpClient version for long time (since SOLR-6865 was 
> committed). It may be a good idea to upgrade to the latest stable version 
> (which is 4.5.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-08-25 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141918#comment-16141918
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul] ping :)

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-10814.patch
>
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11238) Solr authorization plugin is not able to pass additional params downstream

2017-08-21 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135690#comment-16135690
 ] 

Hrishikesh Gadre commented on SOLR-11238:
-

[~ichattopadhyaya] ping :) Any thoughts on the proposal?

> Solr authorization plugin is not able to pass additional params downstream
> --
>
> Key: SOLR-11238
> URL: https://issues.apache.org/jira/browse/SOLR-11238
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-11238.patch
>
>
> Authorization checks in Solr are implemented by invoking configured 
> authorization plugin with AuthorizationContext object. The plugin is expected 
> to return an AuthorizationResponse object which provides the result (which 
> can be OK/FORBIDDEN/PROMPT).
> In some cases (e.g. document level security implemented in Apache Sentry), it 
> is useful for the authorization plugin to add (or override) the request 
> parameters sent by the user (which are represented as SolrParams in 
> [AuthorizationContext| 
> https://github.com/apache/lucene-solr/blob/3cbbecca026eb2a9491fa4a24ecc2c43c26e58bd/solr/core/src/java/org/apache/solr/security/AuthorizationContext.java#L38]).
>  This jira is to introduce an ability to customize the parameters by the 
> authorization plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11229) Add a configuration upgrade tool for Solr

2017-08-18 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133497#comment-16133497
 ] 

Hrishikesh Gadre commented on SOLR-11229:
-

[~gerlowskija] Thanks for your feedback.

bq. I noticed that the TODO list in the README.md doesn't mention anything 
about Windows support. Is a Windows version of this script in the scope of this 
JIRA? Or is the idea that it'd be added as a part of a separate JIRA later on?

Sure - we can add windows support as part of this jira itself. I didn't think 
about Windows support earlier.

bq. I notice that ConfigUpgradeTool mimics but doesnt technically implement the 
Solr Tool interface. Is the idea that as this makes its way into Solr, it would 
actually implement that interface? Or is it coincidence that those classes look 
similar. This is not a suggestion or a critique; just trying to understand the 
code a little better.

Sure I think it would be a good idea to use Solr Tool interface for this tool 
as well. 

bq. I'm curious about the choice of XSLT here.

One of the important requirement is the ability to accept a Solr configuration 
file (e.g. solrconfig.xml) of version X and emit a identical configuration 
compatible with version Y (e.g. rewriting some of the configuration sections). 
As per my understanding XSLT is designed for such use-cases (while XPath is 
more useful towards querying the XML document). Obviously we can cobble 
together some java/bash code along with XPath to make it work. But I felt using 
a single technology to handle both querying and transformation would simplify 
the tool as well as implementation of upgrade rules.

Note -  I personally found XSLT to be adequate to implement all the upgrade 
rules from Solr 4 -> Solr 6. It took me about an 1 1/2 to read the Upgrade 
section in the release notes and implement the upgrade rules for one release 
(e.g. from Solr 4 -> Solr 5). And now we have some reference available, new 
rules can easily be added by referring to these definitions.


> Add a configuration upgrade tool for Solr
> -
>
> Key: SOLR-11229
> URL: https://issues.apache.org/jira/browse/SOLR-11229
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
>
> Despite widespread enterprise adoption, Solr lacks automated upgrade tooling. 
> It has long been a challenge for users to understand the implications of a 
> Solr upgrade. Users must manually review the Solr release notes to identify 
> configuration changes either to fix backwards incompatibilities or to utilize 
> latest features in the new version. Additionally, users must identify a way 
> to migrate existing index data to the new version (either via an index 
> upgrade or re-indexing the raw data).
> Solr config upgrade tool aims to simplify the upgrade process by providing 
> upgrade instructions tailored to your configuration. These instructions can 
> help you to answer following questions
> - Does my Solr configuration have any backwards incompatible sections? If yes 
> which ones?
> - For each of the incompatibility - what do I need to do to fix this 
> incompatibility? Where can I get more information about why this 
> incompatibility was introduced (e.g. references to Lucene/Solr jiras)?
> - Are there any changes in Lucene/Solr which would require me to do a full 
> reindexing OR can I get away with an index upgrade?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11209) Upgrade HttpClient to 4.5.3

2017-08-18 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133471#comment-16133471
 ] 

Hrishikesh Gadre commented on SOLR-11209:
-

[~risdenk] could you please review the pull request?

> Upgrade HttpClient to 4.5.3
> ---
>
> Key: SOLR-11209
> URL: https://issues.apache.org/jira/browse/SOLR-11209
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Reporter: Hrishikesh Gadre
>Priority: Minor
>
> We have not upgraded HttpClient version for long time (since SOLR-6865 was 
> committed). It may be a good idea to upgrade to the latest stable version 
> (which is 4.5.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-08-18 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133465#comment-16133465
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul] could you please review the attached patch?

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-10814.patch
>
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11238) Solr authorization plugin is not able to pass additional params downstream

2017-08-16 Thread Hrishikesh Gadre (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-11238:

Attachment: SOLR-11238.patch

[~ichattopadhyaya] Can you please take a look? The unit test that I have added 
is very similar to the Sentry based document level security logic.

> Solr authorization plugin is not able to pass additional params downstream
> --
>
> Key: SOLR-11238
> URL: https://issues.apache.org/jira/browse/SOLR-11238
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-11238.patch
>
>
> Authorization checks in Solr are implemented by invoking configured 
> authorization plugin with AuthorizationContext object. The plugin is expected 
> to return an AuthorizationResponse object which provides the result (which 
> can be OK/FORBIDDEN/PROMPT).
> In some cases (e.g. document level security implemented in Apache Sentry), it 
> is useful for the authorization plugin to add (or override) the request 
> parameters sent by the user (which are represented as SolrParams in 
> [AuthorizationContext| 
> https://github.com/apache/lucene-solr/blob/3cbbecca026eb2a9491fa4a24ecc2c43c26e58bd/solr/core/src/java/org/apache/solr/security/AuthorizationContext.java#L38]).
>  This jira is to introduce an ability to customize the parameters by the 
> authorization plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5129) If zookeeper is down, SolrCloud nodes will not start correctly, even if zookeeper is started later

2017-08-15 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128284#comment-16128284
 ] 

Hrishikesh Gadre commented on SOLR-5129:


[~caomanhdat] Couple of nits in your patch,

Instead of 
{code}
int startUpZkTimeOut = Integer.parseInt(System.getProperty("startUpZkTimeOut", 
"60"));
{code}

How about
{code}
int startUpZkTimeOut = Integer. getInteger("startUpZkTimeOut", 60);
{code}

Also ideally this parameter should be exposed via solr.xml (e.g. under 
solrcloud section). What do you think?



> If zookeeper is down, SolrCloud nodes will not start correctly, even if 
> zookeeper is started later
> --
>
> Key: SOLR-5129
> URL: https://issues.apache.org/jira/browse/SOLR-5129
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 4.4
>Reporter: Shawn Heisey
>Priority: Minor
> Fix For: 4.9, 6.0
>
> Attachments: SOLR-5129.patch
>
>
> Summary of report from user on mailing list:
> If zookeeper is down or doesn't have quorum when you start Solr nodes, they 
> will not function correctly, even if you later start zookeeper.  While 
> zookeeper is down, the log shows connection failures as expected.  When 
> zookeeper comes back, the log shows:
> INFO  - 2013-08-09 15:48:41.528; 
> org.apache.solr.common.cloud.ConnectionManager; Client->ZooKeeper status 
> change trigger but we are already closed
> At that point, Solr (admin UI and all other functions) does not work, and 
> won't work until it is restarted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11238) Solr authorization plugin is not able to pass additional params downstream

2017-08-14 Thread Hrishikesh Gadre (JIRA)

Hrishikesh Gadre created SOLR-11238:
---

 Summary: Solr authorization plugin is not able to pass additional 
params downstream
 Key: SOLR-11238
 URL: https://issues.apache.org/jira/browse/SOLR-11238
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.6
Reporter: Hrishikesh Gadre


Authorization checks in Solr are implemented by invoking configured 
authorization plugin with AuthorizationContext object. The plugin is expected 
to return an AuthorizationResponse object which provides the result (which can 
be OK/FORBIDDEN/PROMPT).

In some cases (e.g. document level security implemented in Apache Sentry), it 
is useful for the authorization plugin to add (or override) the request 
parameters sent by the user (which are represented as SolrParams in 
[AuthorizationContext| 
https://github.com/apache/lucene-solr/blob/3cbbecca026eb2a9491fa4a24ecc2c43c26e58bd/solr/core/src/java/org/apache/solr/security/AuthorizationContext.java#L38]).
 This jira is to introduce an ability to customize the parameters by the 
authorization plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-08-14 Thread Hrishikesh Gadre (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-10814:

Attachment: SOLR-10814.patch

[~noble.paul] I had opened a github pull request. Anyway I am uploading a patch 
as well. Please take a look and let me have your feedback.

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
> Attachments: SOLR-10814.patch
>
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124178#comment-16124178
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul] OK I filed SOLR-11230 for introducing the HTTP APIs for 
configuring global properties in security.json. If you get a chance, can you 
please review (and commit) the patch for this jira?

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11230) Support configuring global properties in security.json via HTTP API

Hrishikesh Gadre created SOLR-11230:
---

 Summary: Support configuring global properties in security.json 
via HTTP API
 Key: SOLR-11230
 URL: https://issues.apache.org/jira/browse/SOLR-11230
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 5.0
Reporter: Hrishikesh Gadre


Currently SecurityConfHandler in Solr does not support configuring top-level 
elements of the security.json file (e.g. authentication or authorization 
section). Users are expected to manually upload the security.json file 
containing initial configuration to Zookeeper. Once this is done, the 
individual authentication/authorization plugins can edit their own 
configuration via Solr HTTP APIs.

This jira is to allow these global properties to be configured via Solr HTTP 
APIs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11229) Add a configuration upgrade tool for Solr


[ 
https://issues.apache.org/jira/browse/SOLR-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123975#comment-16123975
 ] 

Hrishikesh Gadre commented on SOLR-11229:
-

The initial prototype of this tool is available @ 
https://github.com/hgadre/solr-upgrade-tool

> Add a configuration upgrade tool for Solr
> -
>
> Key: SOLR-11229
> URL: https://issues.apache.org/jira/browse/SOLR-11229
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>    Reporter: Hrishikesh Gadre
>
> Despite widespread enterprise adoption, Solr lacks automated upgrade tooling. 
> It has long been a challenge for users to understand the implications of a 
> Solr upgrade. Users must manually review the Solr release notes to identify 
> configuration changes either to fix backwards incompatibilities or to utilize 
> latest features in the new version. Additionally, users must identify a way 
> to migrate existing index data to the new version (either via an index 
> upgrade or re-indexing the raw data).
> Solr config upgrade tool aims to simplify the upgrade process by providing 
> upgrade instructions tailored to your configuration. These instructions can 
> help you to answer following questions
> - Does my Solr configuration have any backwards incompatible sections? If yes 
> which ones?
> - For each of the incompatibility - what do I need to do to fix this 
> incompatibility? Where can I get more information about why this 
> incompatibility was introduced (e.g. references to Lucene/Solr jiras)?
> - Are there any changes in Lucene/Solr which would require me to do a full 
> reindexing OR can I get away with an index upgrade?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Solr config upgrade tool

2017-08-11 Thread Hrishikesh Gadre

Hi,

I am currently working on a tool to identify (and in some cases fix) the
incompatibilities in the Solr configuration (e.g. schema.xml,
solrconfig.xml etc.) between major versions of Solr.

My goal is to simplify the Solr upgrade process by providing upgrade
instructions tailored to your configuration. These instuctions can help you
to answer following questions

- Does my Solr configuration have any backwards incompatible sections? If
yes which ones?
- For each of the incompatibility - what do I need to do to fix this
incompatibility? Where can I get more information about why this
incompatibility was introduced (e.g. references to Lucene/Solr jira)?
- Are there any changes in Lucene/Solr which would require me to do a full
reindexing OR can I get away with an index upgrade?

The initial prototype of this tool is available @
https://github.com/hgadre/solr-upgrade-tool

For comments (or suggestions) please refer to
https://issues.apache.org/jira/browse/SOLR-11229

I have implemented upgrade rules for Solr 4 -> Solr 5 and Solr 5 -> Solr 6
by referring to upgrade instructions in the Solr release notes. But during
testing I found that many of the configuration changes (e.g. plugin
deprecations) are not listed in the upgrade section. I am working on
identifying such configuration changes and adding upgrade rules
accordingly. I would love any community participation in improving this
tool.

I will be presenting a talk titled "Apache Solr - Upgrading your upgrade
experience" @ Lucene revolution this year. Please do attend to know more
about simplifying Solr upgrade process.

Regards

Hrishikesh

[jira] [Created] (SOLR-11229) Add a configuration upgrade tool for Solr

Hrishikesh Gadre created SOLR-11229:
---

 Summary: Add a configuration upgrade tool for Solr
 Key: SOLR-11229
 URL: https://issues.apache.org/jira/browse/SOLR-11229
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.6
Reporter: Hrishikesh Gadre


Despite widespread enterprise adoption, Solr lacks automated upgrade tooling. 
It has long been a challenge for users to understand the implications of a Solr 
upgrade. Users must manually review the Solr release notes to identify 
configuration changes either to fix backwards incompatibilities or to utilize 
latest features in the new version. Additionally, users must identify a way to 
migrate existing index data to the new version (either via an index upgrade or 
re-indexing the raw data).

Solr config upgrade tool aims to simplify the upgrade process by providing 
upgrade instructions tailored to your configuration. These instructions can 
help you to answer following questions

- Does my Solr configuration have any backwards incompatible sections? If yes 
which ones?
- For each of the incompatibility - what do I need to do to fix this 
incompatibility? Where can I get more information about why this 
incompatibility was introduced (e.g. references to Lucene/Solr jiras)?
- Are there any changes in Lucene/Solr which would require me to do a full 
reindexing OR can I get away with an index upgrade?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-08-08 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118958#comment-16118958
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

Thanks [~noble.paul] and [~bosco] for your feedback!

I have updated the pull request based on our discussion: 
https://github.com/apache/lucene-solr/pull/210

BTW I found that SecurityConfHandler in Solr does not support configuring 
top-level elements of the security.json file. Users are expected to upload the 
security.json file manually to Zookeeper. Once this is done, the individual 
authentication/authorization plugins can edit their own configuration via Solr 
HTTP APIs. Hence I think the ability to update this flag (as well as initial 
configuration of plugins) via HTTP API should be handled as part of a separate 
jira. Does that make sense?

[~noble.paul] [~anshumg] Please include this fix in the Solr 7.0 branch as it 
will simplify the Solr/Sentry integration. 



> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-08-08 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118434#comment-16118434
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul]

bq. Tell me , how will the new plugin implementation know whether to use a 
short name or a long name?

We will just use the short name all the time (and ignore the long name). 
Obviously this means that the plugin can only work for Solr releases which 
contain this fix. But I think that is ok.

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11209) Upgrade HttpClient to 4.5.3


[ 
https://issues.apache.org/jira/browse/SOLR-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117388#comment-16117388
 ] 

Hrishikesh Gadre commented on SOLR-11209:
-

[~risdenk] Thanks for pointing out. Submitting a patch for review shortly.

> Upgrade HttpClient to 4.5.3
> ---
>
> Key: SOLR-11209
> URL: https://issues.apache.org/jira/browse/SOLR-11209
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Reporter: Hrishikesh Gadre
>Priority: Minor
>
> We have not upgraded HttpClient version for long time (since SOLR-6865 was 
> committed). It may be a good idea to upgrade to the latest stable version 
> (which is 4.5.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11209) Upgrade HttpClient to 4.5.3


 [ 
https://issues.apache.org/jira/browse/SOLR-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-11209:

Description: We have not upgraded HttpClient version for long time (since 
SOLR-6865 was committed). It may be a good idea to upgrade to the latest stable 
version (which is 4.5.3).  (was: We have upgraded HttpClient version for long 
time (since SOLR-6865 was committed). It may be a good idea to upgrade to the 
latest stable version (which is 4.5.3).)

> Upgrade HttpClient to 4.5.3
> ---
>
> Key: SOLR-11209
> URL: https://issues.apache.org/jira/browse/SOLR-11209
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Reporter: Hrishikesh Gadre
>Priority: Minor
>
> We have not upgraded HttpClient version for long time (since SOLR-6865 was 
> committed). It may be a good idea to upgrade to the latest stable version 
> (which is 4.5.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11209) Upgrade HttpClient to 4.5.3

Hrishikesh Gadre created SOLR-11209:
---

 Summary: Upgrade HttpClient to 4.5.3
 Key: SOLR-11209
 URL: https://issues.apache.org/jira/browse/SOLR-11209
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Hrishikesh Gadre
Priority: Minor


We have upgraded HttpClient version for long time (since SOLR-6865 was 
committed). It may be a good idea to upgrade to the latest stable version 
(which is 4.5.3).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117059#comment-16117059
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

bq. What I meant was, all existing code will continue using getPrincipal(). And 
for anyone writing new authorization plugin, they can use either of the two 
methods. Those who want to keep to play it safe, can use getShortName() and not 
worry about the underlying authentication mode. And those who want to do 
additional processing then can use getPrincipal().

[~bosco] Thanks for your feedback. I think it make sense. I would prefer to use 
short userName for Sentry plugin (instead of requiring some special 
configuration from user).

[~noble.paul] it looks like Apache Ranger and Sentry plugins would not need 
special flag if the short username is exposed via AuthorizationContext. But as 
you said RuleBasedAuthorizationPlugin (and other third-party implementations) 
may benefit from a global flag. After thinking about it, I am not sure if we 
can have one solution which would benefit all plugins.

So I suggest following approach,
* Expose short userName via AuthorizationContext. This will allow new plugin 
implementations to work without any special configuration.
* Add a parameter in security.json which can define the result of the 
AuthorizationContext#getPrincipal() API (i.e. a fully qualified principal name 
vs short userName). This will allow RuleBasedAuthorization plugin as well as 
other third party implementations to work without any changes. (Note - user 
will need to set this parameter for that though).

Does that make sense?


> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11207) Add OWASP dependency checker to detect security vulnerabilities in third party libraries

Hrishikesh Gadre created SOLR-11207:
---

 Summary: Add OWASP dependency checker to detect security 
vulnerabilities in third party libraries
 Key: SOLR-11207
 URL: https://issues.apache.org/jira/browse/SOLR-11207
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.0
Reporter: Hrishikesh Gadre


Lucene/Solr project depends on number of third party libraries. Some of those 
libraries contain security vulnerabilities. Upgrading to versions of those 
libraries that have fixes for those vulnerabilities is a simple, critical step 
we can take to improve the security of the system. But for that we need a tool 
which can scan the Lucene/Solr dependencies and look up the security database 
for known vulnerabilities.

I found that [OWASP 
dependency-checker|https://jeremylong.github.io/DependencyCheck/dependency-check-ant/]
 can be used for this purpose. It provides a ant task which we can include in 
the Lucene/Solr build. We also need to figure out how (and when) to invoke this 
dependency-checker. But this can be figured out once we complete the first step 
of integrating this tool with the Lucene/Solr build system.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Limiting the number of queries/updates to Solr

2017-08-02 Thread Hrishikesh Gadre

At one point I was working on SOLR-7344
 (but it fell off the
radar due to various reasons). Specifically I built a servlet request
filter which implements a customizable queuing mechanism using asynchronous
servlet API (Servlet 3 spec). This way you can define how many concurrent
requests of a specific type (e.g. query, indexing etc.) you want to
process. This can also be extended at a core (or collection) level.

https://github.com/hgadre/servletrequest-scheduler


If this is something interesting and useful for the community, I would be
more than happy to help moving this forward. Otherwise I would like to get
any feedback for possible improvements (or drawbacks) etc.

Thanks
Hrishikesh




On Wed, Aug 2, 2017 at 9:45 PM, Walter Underwood 
wrote:

>
> > On Aug 2, 2017, at 8:33 PM, Shawn Heisey  wrote:
> >
> > IMHO, intentionally causing connections to fail when a limit is exceeded
> > would not be a very good idea.  When the rate gets too high, the first
> > thing that happens is all the requests slow down.  The slowdown could be
> > dramatic.  As the rate continues to increase, some of the requests
> > probably would begin to fail.
>
> No, this is a very good idea. It is called “load shedding” or “fail fast”.
> Gracefully dealing with overload is an essential part of system design.
>
> At Netflix, with a pre-Jetty Solr (war file running under Tomcat), we took
> down 40 front end servers with slow response times from the Solr server
> farm. We tied up all the front end threads waiting on responses from the
> Solr servers. That left no front end threads available to respond to
> incoming HTTP requests. It was not a fun evening.
>
> To fix this, we configured the Citrix load balancer to overflow to a
> different server when the outstanding back-end requests hit a limit. The
> overflow server was a virtual server that immediately returned a 503. That
> would free up front end connections and threads in an overload condition.
> The users would get a “search unavailable” page, but the rest of the site
> would continue to work.
>
> Unfortunately, the AWS load balancers don’t offer anything like this, ten
> years later.
>
> The worst case version of this is a stable congested state. It is pretty
> easy to put requests into a queue (connection/server) that are guaranteed
> to time out before they are serviced. If you have 35 requests in the queue,
> a 1 second service time, and a 30 second timeout, those requests are
> already dead when you put them on the queue.
>
> I learned about this when I worked with John Nagle at Ford Aerospace. I
> recommend his note “On Packet Switches with Infinite Storage” (1985) for
> the full story. It is only eight pages long, but packed with goodness.
>
> https://tools.ietf.org/html/rfc970
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
>

[jira] [Comment Edited] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100832#comment-16100832
 ] 

Hrishikesh Gadre edited comment on SOLR-10814 at 7/25/17 9:42 PM:
--

[~noble.paul]

bq. If there is a Ranger Authorization plugin, you want them to implement this 
flag as well?

AFAIK Ranger plugin does not need this flag (or this fix in general) since they 
have already implemented logic to parse Kerberos principal -> short userName. 
https://github.com/apache/ranger/blame/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java#L560

The concern here is that it "assumes" kerberos authentication. I don't know if 
this works for any authentication type other than Kerberos (e.g. basic auth).

bq.  The framework is supposed to take care of these things.

I agree. In the hindsight, it was a mistake for Solr authorization framework to 
provide java.security.Principal as part of AuthorizationContext (instead of 
short userName) since the Principal representation depends upon the type of the 
authentication scheme. 

bq. All authentication plugins should extend the AuthenticationPlugin class. 
So, it should decide what details it should return. This has nothing to do with 
what underlying authentication is used. 

How would we do that? By overriding getUserPrincipal() method in the 
HttpServletRequest class ? I would be concerned with that. Since  
getUserPrincipal() is a public method and can be used by downstream components 
(e.g. custom request handlers, plugins etc.), this would be a backwards 
incompatible change.

A safer alternative may be to let authorization framework handle this global 
flag and return short username (or user principal) accordingly.
https://github.com/apache/lucene-solr/blob/10875143b2eb4c6cd72c7a93e65783398b66/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L1034

{code:theme=FadeToGrey|linenumbers=true|language=java|firstline=0001|collapse=true}
  @Override
  public Principal getUserPrincipal() {
if (Boolean.getBoolean("solr.authorization.useShortName") {
  return new BasicUserPrincipal(getReq().getRemoteUser());
}
return getReq().getUserPrincipal();
  }
{code}








was (Author: hgadre):
[~noble.paul]

bq. If there is a Ranger Authorization plugin, you want them to implement this 
flag as well?

AFAIK Ranger plugin does not need this flag (or this fix in general) since they 
have already implemented logic to parse Kerberos principal -> short userName. 
https://github.com/apache/ranger/blame/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java#L560

The concern here is that it "assumes" kerberos authentication. I don't know if 
this works for any authentication type other than Kerberos (e.g. basic auth).

bq.  The framework is supposed to take care of these things.

I agree. In the hindsight, it was a mistake for Solr authorization framework to 
provide java.security.Principal as part of AuthorizationContext (instead of 
short userName) since the Principal representation depends upon the type of the 
authentication scheme. 

bq. All authentication plugins should extend the AuthenticationPlugin class. 
So, it should decide what details it should return. This has nothing to do with 
what underlying authentication is used. 

How would we do that? By overriding getUserPrincipal() method in the 
HttpServletRequest class ? I would be concerned with that. Since this is a 
public method and can be used by downstream components (e.g. custom request 
handlers, plugins etc.), this would be a backwards incompatible change.

A safer alternative may be to let authorization framework handle this global 
flag and return short username (or user principal) accordingly.
https://github.com/apache/lucene-solr/blob/10875143b2eb4c6cd72c7a93e65783398b66/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L1034

{code:theme=FadeToGrey|linenumbers=true|language=java|firstline=0001|collapse=true}
  @Override
  public Principal getUserPrincipal() {
if (Boolean.getBoolean("solr.authorization.useShortName") {
  return new BasicUserPrincipal(getReq().getRemoteUser());
}
return getReq().getUserPrincipal();
  }
{code}







> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>Reporter: Hrishikesh Gadre
>
> Solr allows configuring r

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100832#comment-16100832
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul]

bq. If there is a Ranger Authorization plugin, you want them to implement this 
flag as well?

AFAIK Ranger plugin does not need this flag (or this fix in general) since they 
have already implemented logic to parse Kerberos principal -> short userName. 
https://github.com/apache/ranger/blame/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java#L560

The concern here is that it "assumes" kerberos authentication. I don't know if 
this works for any authentication type other than Kerberos (e.g. basic auth).

bq.  The framework is supposed to take care of these things.

I agree. In the hindsight, it was a mistake for Solr authorization framework to 
provide java.security.Principal as part of AuthorizationContext (instead of 
short userName) since the Principal representation depends upon the type of the 
authentication scheme. 

bq. All authentication plugins should extend the AuthenticationPlugin class. 
So, it should decide what details it should return. This has nothing to do with 
what underlying authentication is used. 

How would we do that? By overriding getUserPrincipal() method in the 
HttpServletRequest class ? I would be concerned with that. Since this is a 
public method and can be used by downstream components (e.g. custom request 
handlers, plugins etc.), this would be a backwards incompatible change.

A safer alternative may be to let authorization framework handle this global 
flag and return short username (or user principal) accordingly.
https://github.com/apache/lucene-solr/blob/10875143b2eb4c6cd72c7a93e65783398b66/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L1034

{code:theme=FadeToGrey|linenumbers=true|language=java|firstline=0001|collapse=true}
  @Override
  public Principal getUserPrincipal() {
if (Boolean.getBoolean("solr.authorization.useShortName") {
  return new BasicUserPrincipal(getReq().getRemoteUser());
}
return getReq().getUserPrincipal();
  }
{code}







> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100260#comment-16100260
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~bosco]

bq. and it helps us to use the same auth to local rules from Hadoop UGI.

In my patch, we are using the short username provided by Hadoop authentication 
layer itself. So we would be using the same auth to local rules from the Hadoop 
UGI.

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100249#comment-16100249
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul]

Just to clarify - even though I have worded this jira as related 
RuleBasedAuthorizationPlugin + Kerberos, it can be a problem for any 
authentication mechanism which provides a Principal string other than the short 
user name.  

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100222#comment-16100222
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul]

bq. can you tell us which is the latest pull request

I have only one pull request. (Updated the same with the review feedback). 
https://github.com/apache/lucene-solr/pull/210

bq. KerBerosPlugin will emit a KerberoPrincipal which has 2 extra methods 
getRealm() and getFullName() .

BTW KerberosPlugin is not the only option for using kerberos with Solr. We have 
recently added HadoopAuthPlugin which allows configuring any authentication 
mechanism provided by underlying Hadoop framework (e.g. LDAP, OAuth etc.). Your 
suggestion will not work in that case (without adding hacks to identify 
Kerberos auth type).

What are your thoughts on adding getUserName() method to AuthorizationContext ? 
Since we are keeping getPrincipal() method as well, the latest patch is 
perfectly backwards compatible. Please review the pull request and let me know 
what you think.


> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-07-17 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090805#comment-16090805
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~anshumg]

bq. Also, can both of you weigh in on it's potential impact on existing code? I 
ask because I know that the Kerberos tests weren't the best ones, for multiple 
reasons

With my latest patch, there is no impact on the existing code. I have added a 
unit test verifying RuleBasedAuthorizationPlugin with Kerberos. Also all the 
unit tests are passing locally.

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-07-14 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087788#comment-16087788
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~ichattopadhyaya] I have updated the pull request. Note that

TestRuleBasedAuthorizationPlugin -> Uses the getUserPrincipal() API (For 
testing backwards compatibility)
TestRuleBasedAuthorizationWithKerberos -> Uses the getUserName() API 

[~anshumg] This is the jira that I mentioned yesterday during the Solr meetup. 
Assuming that [~ichattopadhyaya] is happy with the latest patch, can we please 
consider back-porting it to 7.0 branch? This is very important for Solr/Sentry 
integration that I am working on (SENTRY-1475) 

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-07-14 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087737#comment-16087737
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~noble.paul] I don't think any changes in the kerberos plugin are required. 
The representation of the user principal is a property of KERBEROS protocol and 
the KerberosPlugin is correctly reflecting it. The approach suggested by 
[~ichattopadhyaya] is safer from backwards compatibility perspective. Let me 
update the patch...

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-06-30 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070799#comment-16070799
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~ichattopadhyaya] [~noble.paul] Thanks for your feedback. 

Note - I have not yet updated the patch as I was wanted to ensure that we all 
agree on the approach. Here is the summary of my thinking so far,
* we need to be backwards compatible with respect to 
RuleBasedAuthorizationPlugin. Hence we should make the switch from principal 
name to user_name configurable.
*  We should use short user name instead of principal as most of the role based 
authorization solutions are using short user names. Here we have two options,

*option (a)* Expose both short user-name and principal

Advantages
(a) No need to worry about backwards incompatibility

Disadvantages
(a) Confusing for the third party integrators (Since similar information is 
available from both these APIs, which one should be used?) Clearly using 
getPrincipal() will require updating authorization metadata every time 
authentication mechanism is changed from/to kerberos. Hence most likely 
everyone will end up using getUserName() and will leave getPrincipal() unused 
for most part.

*option (b)* Expose short user-name and deprecate getPrincipal() method

Advantages
(a) No confusion with respect to which APIs to use.
(b) The API result will be consistent across authentication mechanisms.

Disadvantages
(a) May have backwards incompatibility concerns further down the line (e.g. 
during Solr 8.0 release).

Let me know your thoughts and I will update the patch accordingly.

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-06-29 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068447#comment-16068447
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

ping...

[~anshumg] [~noble.paul] Any thoughts?

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9578) Implement document level security in Solr

2017-06-26 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063401#comment-16063401
 ] 

Hrishikesh Gadre commented on SOLR-9578:


[~joel.bernstein] That's great. Please let me know if I can help out in the 
design/implementation of this feature in any way.

> Implement document level security in Solr
> -
>
> Key: SOLR-9578
> URL: https://issues.apache.org/jira/browse/SOLR-9578
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.2
>    Reporter: Hrishikesh Gadre
>
> SOLR-7275 introduced pluggable authorization framework. Currently it is not 
> supporting document level security (or authorization). This is required for 
> integrating Apache Sentry with this authorization framework (SENTRY-1475).
> Currently Sentry relies on custom Solr components to enforce this 
> authorization. Please refer to 
> http://blog.cloudera.com/blog/2014/07/new-in-cdh-5-1-document-level-security-for-cloudera-search/
> Please refer to this discussion as well,
> https://issues.apache.org/jira/browse/SOLR-7275?focusedCommentId=14490733=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14490733



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-06-26 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063398#comment-16063398
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~ichattopadhyaya] any thoughts? This bugfix is needed for Solr/Sentry 
integration. Hence any feedback would be great...

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-06-07 Thread Hrishikesh Gadre (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041860#comment-16041860
]

Hrishikesh Gadre edited comment on SOLR-10814 at 6/7/17 11:23 PM:
--

[~ichattopadhyaya]

bq. Also, this is a significant backcompat break for kerberos+authorization
setups that would prevent them from upgrading.
bq. How about using the getUserPrincipal or getUserName based on some
configuration / property? That is, if someone wants to setup Solr for a variety
of authentication mechanisms, he could enable some configuration property after
which the getUserName() would be used instead of getPrincipal (or other way
around)?

I agree. We can make this behavior *configurable* for
RuleBasedAuthorizationPlugin

bq. Also, I don't think dropping the realm is a good idea for an organization
that has setup cross realm authorization. Imagine that ishan@REALM1 and
ishan@REALM2 maybe completely different users.

I disagree. We are not dropping the RELM from the principal name. The kerberos
authentication plugin applies a set of "kerberos name rules" to the
authenticated principal and uses the result as the "short user name". Hence it
is possible for the system administrators to tweak the output of this process
by modifying the kerberos name rules parameter. Here are few pointers for more
details,

https://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.1/doc/krb5-admin/realms--krb5.conf-.html
https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_kerbprin_to_sn.html

bq. However, I don't think we should deprecate getUserPrincipal() in
AuthorizationContext, since many admins who only want kerberos would prefer to
keep the realm in their authorization configurations.

I disagree. This is not just restricted to RuleBasedAuthorizationPlugin, but it
applies to all future implementations of AuthorizationPlugin (e.g.
SENTRY-1475). Specifically - if AuthorizationContext provides both
user_principal and user_name, which of these should be used by the plugin
implementation?

I know that Apache Sentry *always* uses short user name for configuring
permissions (even when kerberos is used).

I found that Apache Ranger is converting the kerberos principal to the short
username as part of the authorization plugin implementation,
https://github.com/apache/ranger/blob/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java#L170

The problem I have with this implementation in Apache Ranger is that the
details of authentication mechanism are spilling into the authorization logic.
e.g. take a look at following comment,
https://github.com/apache/ranger/blame/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java#L560

Ideally I would expect the authorization plugin to deal with just short user
names and be completely agnostic of underlying authentication mechanism.

So to summarize - here is my proposal,
* I agree that we need to be backwards compatible with respect to
RuleBasedAuthorizationPlugin. Hence we should make the switch from principal
name to user_name configurable.
* I don't agree with the rationale of not deprecating getUserPrincipal(). Most
of the role based authorization solutions are using short user names. Hence if
we deprecate getUserPrincipal() for Solr 7.0 (and/or 6.x), we can wait until
8.0 to clean this up. That will provide sufficient time for the administrators
to migrate their authorization settings to user short user-names.

Please let me know your thoughts.

was (Author: hgadre):
[~ichattopadhyaya]

I agree. We can make this behavior *configurable* for
RuleBasedAuthorizationPlugin

bq. Also, I don't think dropping the realm is a good idea for an organization
that has setup cross realm authorization. Imagine that ishan@REALM1 and
ishan@REALM2 maybe completely different users.

https://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.1/doc/

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

2017-06-07 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041860#comment-16041860
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~ichattopadhyaya]

bq. Also, this is a significant backcompat break for kerberos+authorization 
setups that would prevent them from upgrading.
bq. How about using the getUserPrincipal or getUserName based on some 
configuration / property? That is, if someone wants to setup Solr for a variety 
of authentication mechanisms, he could enable some configuration property after 
which the getUserName() would be used instead of getPrincipal (or other way 
around)?

I agree. We can make this behavior *configurable* for 
RuleBasedAuthorizationPlugin

bq. Also, I don't think dropping the realm is a good idea for an organization 
that has setup cross realm authorization. Imagine that ishan@REALM1 and 
ishan@REALM2 maybe completely different users.

I disagree. We are not dropping the RELM from the principal name. The kerberos 
authentication plugin applies a set of "kerberos name rules" to the 
authenticated principal and uses the result as the "short user name". Hence it 
is possible for the system administrators to tweak the output of this process 
by modifying the kerberos name rules parameter. Here are few pointers for more 
details,

https://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.1/doc/krb5-admin/realms--krb5.conf-.html
https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_kerbprin_to_sn.html

bq. However, I don't think we should deprecate getUserPrincipal() in 
AuthorizationContext, since many admins who only want kerberos would prefer to 
keep the realm in their authorization configurations.

I disagree. This is not just restricted to RuleBasedAuthorizationPlugin, but it 
applies to all future implementations of AuthorizationPlugin (e.g. 
SENTRY-1475). Specifically - if AuthorizationContext provides both 
user_principal and user_name, which of these should be used by the plugin 
implementation? 

I know that Apache Sentry *always* users short user name for configuring 
permissions (even when kerberos is used). 

I found that Apache Ranger is converting the kerberos principal to the short 
username as part of the authorization plugin implementation,
https://github.com/apache/ranger/blob/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java#L170

The problem I have with this implementation in Apache Ranger is that the 
details of authentication mechanism are spilling into the authorization logic. 
e.g. take a look at following comment,
https://github.com/apache/ranger/blame/16f3dd3c350ac9ea85b157b81ceddfcdb761e128/agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java#L560

Ideally I would expect the authorization plugin to deal with just short user 
names and be completely agnostic of underlying authentication mechanism. 

So to summarize - here is my proposal,
* I agree that we need to be backwards compatible with respect to 
RuleBasedAuthorizationPlugin. Hence we should make the switch from principal 
name to user_name configurable.
* I don't agree with the rationale of not deprecating getUserPrincipal(). Most 
of the role based authorization solutions are using short user names. Hence if 
we deprecate getUserPrincipal() for Solr 7.0 (and/or 6.x), we can wait until 
8.0 to clean this up. That will provide sufficient time for the administrators 
to migrate their authorization settings to user short user-names.

Please let me know your thoughts.




> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/secur

[jira] [Commented] (SOLR-9578) Implement document level security in Solr

2017-06-06 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039734#comment-16039734
 ] 

Hrishikesh Gadre commented on SOLR-9578:


The genesis of this feature is the "document level security" feature 
implemented as part of Solr/Sentry integration. In this case, Sentry is 
implementing a custom SearchComponent which is responsible to add necessary 
query filters so that users can see only the data they are authorized for.

Interestingly the newer components in Solr (e.g. SQLHandler, GraphHandler, 
StreamHandler etc.) are not based on SearchHandler. So I am wondering how to 
integrate "document level security" with these newer components? 

[~joel.bernstein] Any thoughts?


> Implement document level security in Solr
> -
>
> Key: SOLR-9578
> URL: https://issues.apache.org/jira/browse/SOLR-9578
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Affects Versions: 5.2
>Reporter: Hrishikesh Gadre
>
> SOLR-7275 introduced pluggable authorization framework. Currently it is not 
> supporting document level security (or authorization). This is required for 
> integrating Apache Sentry with this authorization framework (SENTRY-1475).
> Currently Sentry relies on custom Solr components to enforce this 
> authorization. Please refer to 
> http://blog.cloudera.com/blog/2014/07/new-in-cdh-5-1-document-level-security-for-cloudera-search/
> Please refer to this discussion as well,
> https://issues.apache.org/jira/browse/SOLR-7275?focusedCommentId=14490733=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14490733



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037675#comment-16037675
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

[~ichattopadhyaya] can you please take a look?

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


 [ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-10814:

Description: 
Solr allows configuring roles to control user access to the system. This is 
accomplished through rule-based permission definitions which are assigned to 
users.

The authorization framework in Solr passes the information about the request 
(to be authorized) using an instance of AuthorizationContext class. Currently 
the only way to extract authenticated user is via getUserPrincipal() method 
which returns an instance of java.security.Principal class. The 
RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
Principal instance to fetch the list of associated roles.
https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156

In case of basic authentication mechanism, the principal is the userName. Hence 
it works fine. But in case of kerberos authentication, the user principal also 
contains the RELM information e.g. instead of foo, it would return 
f...@example.com. This means if the user changes the authentication mechanism, 
he would also need to change the user-role mapping in authorization section to 
use f...@example.com instead of foo. This is not good from usability 
perspective.   

  was:
Solr allows configuring roles to control user access to the system. This is 
accomplished through rule-based permission definitions which are assigned to 
users.

The authorization framework in Solr passes the information about the request 
(to be authorized) using an instance of AuthorizationContext class. Currently 
the only way to extract authenticated user is via getUserPrincipal() method 
which returns an instance of java.security.Principal class. The 
RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
Principal instance to fetch the list of associated roles.
https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156

In case of basic authentication mechanism, the principal is the userName. Hence 
it works fine. But in case of kerberos authentication, the user principal also 
contains the RELM information e.g. instead of foo, it would return 
f...@example.com. This means if the user changes the authentication mechanism, 
he would also need to changed the user-role mapping in authorization section to 
use f...@example.com instead of foo. This is not good from usability 
perspective.   


> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to change the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication


[ 
https://issues.apache.org/jira/browse/SOLR-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037127#comment-16037127
 ] 

Hrishikesh Gadre commented on SOLR-10814:
-

Upon investigation, I found that we can utilize getRemoteUser() method in 
HttpServletRequest class to fetch the name of the authenticated user.
http://docs.oracle.com/javaee/6/api/javax/servlet/http/HttpServletRequest.html#getRemoteUser()

Currently Hadoop authentication framework returns appropriate information for 
this method. So we should add an API in AuthorizationContext class to return 
this information as well (along with relevant changes in 
RuleBasedAuthorizationPlugin).

> Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos 
> authentication
> ---
>
> Key: SOLR-10814
> URL: https://issues.apache.org/jira/browse/SOLR-10814
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>    Reporter: Hrishikesh Gadre
>
> Solr allows configuring roles to control user access to the system. This is 
> accomplished through rule-based permission definitions which are assigned to 
> users.
> The authorization framework in Solr passes the information about the request 
> (to be authorized) using an instance of AuthorizationContext class. Currently 
> the only way to extract authenticated user is via getUserPrincipal() method 
> which returns an instance of java.security.Principal class. The 
> RuleBasedAuthorizationPlugin implementation invokes getName() method on the 
> Principal instance to fetch the list of associated roles.
> https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156
> In case of basic authentication mechanism, the principal is the userName. 
> Hence it works fine. But in case of kerberos authentication, the user 
> principal also contains the RELM information e.g. instead of foo, it would 
> return f...@example.com. This means if the user changes the authentication 
> mechanism, he would also need to changed the user-role mapping in 
> authorization section to use f...@example.com instead of foo. This is not 
> good from usability perspective.   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10814) Solr RuleBasedAuthorization config doesn't work seamlessly with kerberos authentication

Hrishikesh Gadre created SOLR-10814:
---

Summary: Solr RuleBasedAuthorization config doesn't work
seamlessly with kerberos authentication
Key: SOLR-10814
URL: https://issues.apache.org/jira/browse/SOLR-10814
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.2
Reporter: Hrishikesh Gadre

Solr allows configuring roles to control user access to the system. This is
accomplished through rule-based permission definitions which are assigned to
users.

The authorization framework in Solr passes the information about the request
(to be authorized) using an instance of AuthorizationContext class. Currently
the only way to extract authenticated user is via getUserPrincipal() method
which returns an instance of java.security.Principal class. The
RuleBasedAuthorizationPlugin implementation invokes getName() method on the
Principal instance to fetch the list of associated roles.
https://github.com/apache/lucene-solr/blob/2271e73e763b17f971731f6f69d6ffe46c40b944/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L156

In case of basic authentication mechanism, the principal is the userName. Hence
it works fine. But in case of kerberos authentication, the user principal also
contains the RELM information e.g. instead of foo, it would return
f...@example.com. This means if the user changes the authentication mechanism,
he would also need to changed the user-role mapping in authorization section to
use f...@example.com instead of foo. This is not good from usability
perspective.

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10137) Configsets created via API should always be mutable

2017-05-31 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031536#comment-16031536
 ] 

Hrishikesh Gadre commented on SOLR-10137:
-

[~markrmil...@gmail.com] Can you please post the parameters of failed tests so 
that I can take a look?

> Configsets created via API should always be mutable
> ---
>
> Key: SOLR-10137
> URL: https://issues.apache.org/jira/browse/SOLR-10137
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>    Reporter: Hrishikesh Gadre
>
> Please refer to this discussion for details,
> https://marc.info/?l=solr-dev=148679049516375=4



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10777) Replication Backup creation fails with NPE, while deleting the old backups

2017-05-30 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030660#comment-16030660
 ] 

Hrishikesh Gadre commented on SOLR-10777:
-

Actually SOLR-9536 is fixed in 6.3 release. So it is quite possible that you 
are hitting it in 6.2.1. But I am a bit surprised to see that happening in 
6.5.1 as well.

> Replication Backup creation fails with NPE, while deleting the old backups
> --
>
> Key: SOLR-10777
> URL: https://issues.apache.org/jira/browse/SOLR-10777
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, replication (java)
>Affects Versions: 6.2.1, 6.5.1
>Reporter: Nilesh Singh
>Priority: Minor
> Attachments: Screen Shot 2017-05-30 at 7.53.50 PM.png, Screen Shot 
> 2017-05-30 at 7.54.18 PM.png, SOLR-10777.patch
>
>   Original Estimate: 20m
>  Remaining Estimate: 20m
>
> in Solr back up SnapShooter will try to delete the old backups saved on the 
> disk automatically, but deletion fails as the files may not be the pattern 
> type expected by the OldBackUpDirectory. 
> ```
>   private static final Pattern dirNamePattern = 
> Pattern.compile("^snapshot[.](.*)$");
> ```
> In this case the following code throws NPE.
> ```
> if (obd.getTimestamp().isPresent()) {
>   dirs.add(obd);
> }
> ```
> also in OldBackUpDirectory`s pattern match should haver matcher.matches(), to 
> fetch the group(1) value.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10777) Replication Backup creation fails with NPE, while deleting the old backups

2017-05-30 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030652#comment-16030652
 ] 

Hrishikesh Gadre commented on SOLR-10777:
-

bq. Since the regular expression has explicitly defined the start (^) and end 
($), I think find() and matches() should return identical results 

Note - this is applicable only in this case since we are invoking find() only 
once on a given Matcher instance. 

> Replication Backup creation fails with NPE, while deleting the old backups
> --
>
> Key: SOLR-10777
> URL: https://issues.apache.org/jira/browse/SOLR-10777
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, replication (java)
>Affects Versions: 6.2.1, 6.5.1
>Reporter: Nilesh Singh
>Priority: Minor
> Attachments: Screen Shot 2017-05-30 at 7.53.50 PM.png, Screen Shot 
> 2017-05-30 at 7.54.18 PM.png, SOLR-10777.patch
>
>   Original Estimate: 20m
>  Remaining Estimate: 20m
>
> in Solr back up SnapShooter will try to delete the old backups saved on the 
> disk automatically, but deletion fails as the files may not be the pattern 
> type expected by the OldBackUpDirectory. 
> ```
>   private static final Pattern dirNamePattern = 
> Pattern.compile("^snapshot[.](.*)$");
> ```
> In this case the following code throws NPE.
> ```
> if (obd.getTimestamp().isPresent()) {
>   dirs.add(obd);
> }
> ```
> also in OldBackUpDirectory`s pattern match should haver matcher.matches(), to 
> fetch the group(1) value.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10777) Replication Backup creation fails with NPE, while deleting the old backups

2017-05-30 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030648#comment-16030648
 ] 

Hrishikesh Gadre commented on SOLR-10777:
-

[~singh.nilesh]

bq. also in OldBackUpDirectory`s pattern match should haver matcher.matches(), 
to fetch the group(1) value.

Since the regular expression has explicitly defined the start (^) and end ($), 
I think find() and matches() should return identical results (which I verified 
by writing a small program). Also refer to 
https://stackoverflow.com/questions/4450045/difference-between-matches-and-find-in-java-regex

BTW can you post the actual stack trace of the error you found? I am a bit 
surprised since we fixed NPE in the same codebase few months ago. Refer to 
SOLR-9536

> Replication Backup creation fails with NPE, while deleting the old backups
> --
>
> Key: SOLR-10777
> URL: https://issues.apache.org/jira/browse/SOLR-10777
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, replication (java)
>Affects Versions: 6.2.1, 6.5.1
>Reporter: Nilesh Singh
>Priority: Minor
> Attachments: Screen Shot 2017-05-30 at 7.53.50 PM.png, Screen Shot 
> 2017-05-30 at 7.54.18 PM.png, SOLR-10777.patch
>
>   Original Estimate: 20m
>  Remaining Estimate: 20m
>
> in Solr back up SnapShooter will try to delete the old backups saved on the 
> disk automatically, but deletion fails as the files may not be the pattern 
> type expected by the OldBackUpDirectory. 
> ```
>   private static final Pattern dirNamePattern = 
> Pattern.compile("^snapshot[.](.*)$");
> ```
> In this case the following code throws NPE.
> ```
> if (obd.getTimestamp().isPresent()) {
>   dirs.add(obd);
> }
> ```
> also in OldBackUpDirectory`s pattern match should haver matcher.matches(), to 
> fetch the group(1) value.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10718) Configuring Basic auth prevents adding a collection


[ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028513#comment-16028513
 ] 

Hrishikesh Gadre edited comment on SOLR-10718 at 5/29/17 4:56 PM:
--

[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the credentials but not 
the PreemptiveAuth request interceptor. Hence when this HTTP client is used 
subsequently, we observe IllegalArgumentException since PreemptiveAuth request 
interceptor *requires* non-null credentials.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.


was (Author: hgadre):
[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the credentials but not 
the PreemptiveAuth request interceptor. Hence when this HTTP client is used 
subsequently, we observe NPE since PreemptiveAuth request interceptor 
*requires* non-null credentials.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.

> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Critical
> Fix For: 6.6
>
> Attachments: repro.sh, SOLR-10718.patch, SOLR-10718.patch
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10718) Configuring Basic auth prevents adding a collection


[ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028513#comment-16028513
 ] 

Hrishikesh Gadre edited comment on SOLR-10718 at 5/29/17 4:52 PM:
--

[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the credentials but not 
the PreemptiveAuth request interceptor. Hence when this HTTP client is used 
subsequently, we observe NPE since PreemptiveAuth request interceptor 
*requires* non-null credentials.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.


was (Author: hgadre):
[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the credentials but not 
the PreemptiveAuth request interceptor. This result in the observed 
NullPointerException since PreemptiveAuth *requires* non-null credentials.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.

> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Critical
> Fix For: 6.6
>
> Attachments: repro.sh, SOLR-10718.patch, SOLR-10718.patch
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10718) Configuring Basic auth prevents adding a collection


[ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028513#comment-16028513
 ] 

Hrishikesh Gadre edited comment on SOLR-10718 at 5/29/17 4:50 PM:
--

[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the credentials but not 
the PreemptiveAuth request interceptor. This result in the observed 
NullPointerException since PreemptiveAuth *requires* non-null credentials.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.


was (Author: hgadre):
[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the CredentialProvider 
instance but not the PreemptiveAuth request interceptor. This result in the 
observed NullPointerException since PreemptiveAuth *requires* non-null 
CredentialProvider.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.

> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Critical
> Fix For: 6.6
>
> Attachments: repro.sh, SOLR-10718.patch, SOLR-10718.patch
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10718) Configuring Basic auth prevents adding a collection


 [ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-10718:

Attachment: SOLR-10718.patch

[~janhoy] [~ichattopadhyaya] I found the issue and the fix is attached.

Here is the summary of the problem,
- Since we are using embedded ZK, the security.json needs to be uploaded after 
starting Solr server.
- But since the basic authentication is configured during the server startup 
(via SOLR_AUTH_TYPE env variable), the default HTTP client in HttpShardHandler 
is configured with PreemptiveAuth request interceptor.
- When we upload security.json file, we invoke 
HttpShardHandlerFactory#reconfigureHttpClient(...) API to configure PKI 
authentication scheme. In this process, HttpClientUtil#setBasicAuth(...) API is 
invoked.
- In the setBasicAuth(...) method we are cleaning only the CredentialProvider 
instance but not the PreemptiveAuth request interceptor. This result in the 
observed NullPointerException since PreemptiveAuth *requires* non-null 
CredentialProvider.

So the fix in this case is to remove PreemptiveAuth request interceptor when 
basic auth is not to be used.

> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Critical
> Fix For: 6.6
>
> Attachments: repro.sh, SOLR-10718.patch, SOLR-10718.patch
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10718) Configuring Basic auth prevents adding a collection

2017-05-27 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027664#comment-16027664
 ] 

Hrishikesh Gadre commented on SOLR-10718:
-

[~janhoy] While I don't (yet) know the exact implementation issue, the 
root-cause is setting these two parameters while starting the Solr server.

{noformat}
grep -q "^SOLR_AUTH_TYPE=\"basic\"" bin/solr.in.sh
if [ $? != 0 ]; then
echo 'SOLR_AUTH_TYPE="basic"' >> bin/solr.in.sh
echo 'SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"' >> 
bin/solr.in.sh
fi
{noformat}

Note that these parameters are required only on the client side. If you don't 
add these parameters to solr.in.sh before server startup, then it works as 
expected. For testing purpose, I just exported these environment variables on 
the command-line (instead of adding them to solr.in.sh).

May be we should just ensure that these environment variables are not 
considered during Solr server startup?


> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Minor
> Attachments: repro.sh
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10718) Configuring Basic auth prevents adding a collection

2017-05-27 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027654#comment-16027654
 ] 

Hrishikesh Gadre commented on SOLR-10718:
-

[~janhoy] I contributed the relevant changes recently (SOLR-9997). Let me take 
a look and get back to you.

> Configuring Basic auth prevents adding a collection
> ---
>
> Key: SOLR-10718
> URL: https://issues.apache.org/jira/browse/SOLR-10718
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.5, 6.5.1
>Reporter: Shawn Feldman
>Priority: Minor
> Attachments: repro.sh
>
>
> Configure Basic auth according to documentation 
> Add basic auth params 
> SOLR_AUTH_TYPE="basic"
> SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
> Try to add a collection 
> Receive a timeout and error in the logs 
> {code}
> java.lang.IllegalArgumentException: Credentials may not be null
> at org.apache.http.util.Args.notNull(Args.java:54)
> at org.apache.http.auth.AuthState.update(AuthState.java:113)
> at 
> org.apache.solr.client.solrj.impl.PreemptiveAuth.process(PreemptiveAuth.java:56)
> at 
> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
> at 
> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:166)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:485)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9153) Update beanutils version to 1.9.2

2017-05-23 Thread Hrishikesh Gadre (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021751#comment-16021751
 ] 

Hrishikesh Gadre commented on SOLR-9153:


[~mdrob] What's your take on this? Should we commit this change?

> Update beanutils version to 1.9.2
> -
>
> Key: SOLR-9153
> URL: https://issues.apache.org/jira/browse/SOLR-9153
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - Velocity
>Affects Versions: 6.0
>Reporter: Mike Drob
>Priority: Minor
> Attachments: SOLR-9153.patch
>
>
> See CVE-2014-0114 -- 
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0114
> {quote}
> Apache Commons BeanUtils, as distributed in lib/commons-beanutils-1.8.0.jar 
> in Apache Struts 1.x through 1.3.10 and in other products requiring 
> commons-beanutils through 1.9.2, does not suppress the class property, which 
> allows remote attackers to "manipulate" the ClassLoader and execute arbitrary 
> code via the class parameter, as demonstrated by the passing of this 
> parameter to the getClass method of the ActionForm object in Struts 1. 
> {quote}
> We transitively depend on BeanUtils through Velocity, but it doesn't look 
> like there is much movement on the project there. See BEANUTILS-463 and 
> VELTOOLS-170
> Also, this might have impact on SOLR-3736 but that issue also looks largely 
> abandoned.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: JIRA Status - Finding Patch Submissions

2017-05-12 Thread Hrishikesh Gadre

>>For current patches, I think we could really benefit from a Patch
Available JIRA state. It would be a bright big flag for committers, instead
of making contributors have to hound us periodically to look. Additionally,
we could use that to start running precommit checks on Jenkins whenever a
new patch is put up. See Apache Yetus for how this might work.

+1. We should also consider running unit tests as part of the process.


On Thu, May 11, 2017 at 1:54 PM, Mike Drob  wrote:

> Wanted to follow up on this, since I've been steadily working away at old
> JIRA issues when I have some time for them. I think read through 100s of
> issues and closed about 20 as either duplicates or already committed, which
> is a tiny drop in the ocean of what we have open. In an attempt to cut the
> task to a more manageable size, I only looked at Solr issues.
>
> I'd like to adjust my earlier statement that most of the attachments are
> patches. Most of the really old attachments are patches, the mid-age ones
> are bug reports (indices, screen captures, logs) and the recent ones being
> actively worked are patches again. So, the situation isn't as bad as I
> imagined it at first. For a lot of these old issues, there's not much to be
> done going forward. I don't expect to have much traction asking
> contributors to rebase their patches from 1.x, 3.x or even 4.x onto the
> current code line, and without that many of them are just unworkable.
>
> For current patches, I think we could really benefit from a Patch
> Available JIRA state. It would be a bright big flag for committers, instead
> of making contributors have to hound us periodically to look. Additionally,
> we could use that to start running precommit checks on Jenkins whenever a
> new patch is put up. See Apache Yetus for how this might work.
>
> Is there interest in the community for this path? I'm personally a big fan
> of enabling static analysis and always like to explore ways to improve in
> that area.
>
> Mike
>
> On Fri, Apr 28, 2017 at 3:33 PM, Shawn Heisey  wrote:
>
>> On 4/28/2017 9:42 AM, Mike Drob wrote:
>> > Thanks for this hint, Alex.
>> >
>> > I ran the following JQL to get some idea of our current status:
>> > project in (lucene, solr) and "Attachment count" > 0 and status =
>> Open
>> >
>> > There were 1500 results.
>> >
>> > 1500. I couldn't believe it. This is a huge number of patches that are
>> > out there.
>> >
>> > I did a spot check, thinking that a lot of these might be bug reports
>> > with error logs or screen shots attached, but nope. These are mostly
>> > patches. I'm going to try starting with the oldest ones to see if they
>> > can be rebase, have already been committed, or generally try to triage
>> > them. Would appreciate any volunteers that want to help.
>>
>> This doesn't surprise me at all.  Many users submit patches for issues
>> they encounter, but for one reason or another, no committer action ever
>> happens.  There are many possible reasons.
>>
>> 1) The patch has bugs or some other problem that makes it unacceptable.
>> 2) When the issue/patch is reviewed, one of these situations exists:
>>  a) Committers don't think it's worth pursuing.
>>  b) The code is behaving as designed.
>>  c) The committer cannot reproduce the problem.
>>  d) The committer doesn't understand the problem.
>>  e) The committer doesn't think it's actually a problem.
>>  f) A workaround exists that is just as effective as the patch.
>> 3) Nobody has had time to review the issue/patch.
>>
>> In some of these situations, the reviewing committer should probably
>> close the issue with an appropriate reason ... but issue triage is a
>> difficult and unrewarding job.  Sometimes it just doesn't happen.
>>
>> Thanks,
>> Shawn
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth


[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001585#comment-16001585
 ] 

Hrishikesh Gadre commented on SOLR-8440:


[~ichattopadhyaya] and [~janhoy]

bq. Hrishikesh, I'll add support for both a prompt and ability to pass 
credentials to the script. We can document both options.

bq. For better security, I guess this should rather have been
{noformat}
-rw-r- 1 root solr 5968 Feb 15 14:55 /etc/default/solr.in.sh
{noformat}

+1 to both the suggestions. Let's not sacrifice security for the sake of 
usability. Let users decide what works for them depending on their security 
requirements.




> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Attachments: SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth


[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001322#comment-16001322
 ] 

Hrishikesh Gadre commented on SOLR-8440:


[~ichattopadhyaya]

bq. In both options, the actual credentials will stay in clear text (either in 
solr.in.sh or a separate file). I don't see how this improves either security 
or ease of use.

The main difference is that the second option allows admins to customize the 
file permissions upfront such that the config file will be readable only to a 
set of trusted users on the system. Specifying password on the command-line has 
number of [security related 
issues|https://unix.stackexchange.com/questions/78734/why-shouldnt-someone-use-passwords-in-the-command-line]
 

BTW what are the default file-permissions for the solr.in.sh ? Is it world 
readable?

bq.  It will also help ignorant users, who might inadvertently copy the 
redacted line to the solr.in.sh and nothing will work for him.

The second option also helps in this case. Since it just provides a file-system 
path, it is quite safe to be printed on the command-line. If a malicious user 
attempt to read this configuration file, he would get file permissions error 
from the operating system (assuming permissions are setup appropriately).


> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Attachments: SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth


[ 
https://issues.apache.org/jira/browse/SOLR-8440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000956#comment-16000956
 ] 

Hrishikesh Gadre commented on SOLR-8440:


[~ichattopadhyaya] [~janhoy] Thanks for the comments. I haven't reviewed this 
patch in detail, but the basic auth mechanism supports specifying a 
configuration file (which would contain the password). User can set permissions 
for this config file such that only solr admin can read/modify/delete the file.

https://github.com/apache/lucene-solr/blob/07cc043664c4b82c208237b111f698ab2ae0ae07/solr/solrj/src/java/org/apache/solr/client/solrj/impl/PreemptiveBasicAuthClientBuilderFactory.java#L51

Does this address your concerns? If yes, then it may be a good idea to expose 
this mechanism via the solr shell command instead of exposing passwords on 
command-line. 




 

> Script support for enabling basic auth
> --
>
> Key: SOLR-8440
> URL: https://issues.apache.org/jira/browse/SOLR-8440
> Project: Solr
>  Issue Type: New Feature
>  Components: scripts and tools
>Reporter: Jan Høydahl
>Assignee: Ishan Chattopadhyaya
>  Labels: authentication, security
> Attachments: SOLR-8440.patch, SOLR-8440.patch, SOLR-8440.patch, 
> SOLR-8440.patch, SOLR-8440.patch
>
>
> Now that BasicAuthPlugin will be able to work without an AuthorizationPlugin 
> (SOLR-8429), it would be sweet to provide a super simple way to "Password 
> protect Solr"™ right from the command line:
> {noformat}
> bin/solr basicAuth -adduser -user solr -pass SolrRocks
> {noformat}
> It would take the mystery out of enabling one single password across the 
> board. The command would do something like this
> # Check if HTTPS is enabled, and if not, print a friendly warning
> # Check if {{/security.json}} already exists
> ## NO => create one with only plugin class defined
> ## YES => Abort if exists but plugin is not {{BasicAuthPlugin}}
> # Using security REST API, add the new user



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8440) Script support for enabling basic auth