[jira] [Commented] (KNOX-736) Support NIS Schema in Demo LDAP
[ https://issues.apache.org/jira/browse/KNOX-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226850#comment-16226850 ] Sandeep More commented on KNOX-736: --- Hello [~hkropp] sorry for the delay in getting to this patch. It turns out as a part of KNOX-1088 refactoring Posix schema was enabled [see|https://github.com/apache/knox/blob/master/gateway-demo-ldap/src/main/java/org/apache/hadoop/gateway/security/ldap/SimpleLdapDirectoryServer.java#L93] Let me know if you think this still needs to be fixed. > Support NIS Schema in Demo LDAP > --- > > Key: KNOX-736 > URL: https://issues.apache.org/jira/browse/KNOX-736 > Project: Apache Knox > Issue Type: Improvement >Reporter: Henning Kropp >Assignee: Henning Kropp >Priority: Minor > Fix For: 0.14.0 > > Attachments: KNOX-736.diff > > > With support of the NIS schema users in the directory could be created with > POSIX attributes. With POSIX attributes LDAP users could be mapped to the OS > level for authentication with PAM. This could support testing and > demonstrating the PAM authentication in Knox (KNOX-537) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KNOX-736) Support NIS Schema in Demo LDAP
[ https://issues.apache.org/jira/browse/KNOX-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226850#comment-16226850 ] Sandeep More edited comment on KNOX-736 at 10/31/17 2:13 PM: - Hello [~hkropp] sorry for the delay in getting to this patch. It turns out as a part of KNOX-1088 refactoring, posix schema was enabled [see|https://github.com/apache/knox/blob/master/gateway-demo-ldap/src/main/java/org/apache/hadoop/gateway/security/ldap/SimpleLdapDirectoryServer.java#L93] Let me know if you think this still needs to be fixed. was (Author: smore): Hello [~hkropp] sorry for the delay in getting to this patch. It turns out as a part of KNOX-1088 refactoring, Posix schema was enabled [see|https://github.com/apache/knox/blob/master/gateway-demo-ldap/src/main/java/org/apache/hadoop/gateway/security/ldap/SimpleLdapDirectoryServer.java#L93] Let me know if you think this still needs to be fixed. > Support NIS Schema in Demo LDAP > --- > > Key: KNOX-736 > URL: https://issues.apache.org/jira/browse/KNOX-736 > Project: Apache Knox > Issue Type: Improvement >Reporter: Henning Kropp >Assignee: Henning Kropp >Priority: Minor > Fix For: 0.14.0 > > Attachments: KNOX-736.diff > > > With support of the NIS schema users in the directory could be created with > POSIX attributes. With POSIX attributes LDAP users could be mapped to the OS > level for authentication with PAM. This could support testing and > demonstrating the PAM authentication in Knox (KNOX-537) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KNOX-736) Support NIS Schema in Demo LDAP
[ https://issues.apache.org/jira/browse/KNOX-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226850#comment-16226850 ] Sandeep More edited comment on KNOX-736 at 10/31/17 2:13 PM: - Hello [~hkropp] sorry for the delay in getting to this patch. It turns out as a part of KNOX-1088 refactoring, Posix schema was enabled [see|https://github.com/apache/knox/blob/master/gateway-demo-ldap/src/main/java/org/apache/hadoop/gateway/security/ldap/SimpleLdapDirectoryServer.java#L93] Let me know if you think this still needs to be fixed. was (Author: smore): Hello [~hkropp] sorry for the delay in getting to this patch. It turns out as a part of KNOX-1088 refactoring Posix schema was enabled [see|https://github.com/apache/knox/blob/master/gateway-demo-ldap/src/main/java/org/apache/hadoop/gateway/security/ldap/SimpleLdapDirectoryServer.java#L93] Let me know if you think this still needs to be fixed. > Support NIS Schema in Demo LDAP > --- > > Key: KNOX-736 > URL: https://issues.apache.org/jira/browse/KNOX-736 > Project: Apache Knox > Issue Type: Improvement >Reporter: Henning Kropp >Assignee: Henning Kropp >Priority: Minor > Fix For: 0.14.0 > > Attachments: KNOX-736.diff > > > With support of the NIS schema users in the directory could be created with > POSIX attributes. With POSIX attributes LDAP users could be mapped to the OS > level for authentication with PAM. This could support testing and > demonstrating the PAM authentication in Knox (KNOX-537) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-719) Knox support for Yarn Resource Manager HA
[ https://issues.apache.org/jira/browse/KNOX-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227169#comment-16227169 ] Nixon Rodrigues commented on KNOX-719: -- [~jeffreyr97], Can you provide documentation to configure the Yarn Resource Manager in knox topology for accessing YARN RM UI ? Thanks Nixon > Knox support for Yarn Resource Manager HA > - > > Key: KNOX-719 > URL: https://issues.apache.org/jira/browse/KNOX-719 > Project: Apache Knox > Issue Type: Bug >Affects Versions: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0 >Reporter: Jeffrey E Rodriguez >Assignee: Jeffrey E Rodriguez > Fix For: 0.12.0 > > Attachments: KNOX-719-4.patch, KNOX_719_5.patch > > > This would support both REST/UI YARN Resource Manager HA. Based on other HA > providers added in Knox. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KNOX-1091) Knox Audit Logging - duplicate correlation ids
Sandeep More created KNOX-1091:
--
Summary: Knox Audit Logging - duplicate correlation ids
Key: KNOX-1091
URL: https://issues.apache.org/jira/browse/KNOX-1091
Project: Apache Knox
Issue Type: Bug
Components: Server
Reporter: Kevin Risden
Fix For: 0.15.0
>From the Knox User list thread: "Multiple topology audit logging", it came to
>my attention that Knox seems to be logging duplicate correlation ids.
>Separating out the topic specifically here to dig a bit deeper.
While looking at our Knox audit logs (Knox 0.9 on HDP 2.5) the "correlation id"
doesn't seem to be unique across requests. Is this to be expected? Here is a
snippet (anonymized):
grep 7557c91b-2a48-4e09-aefc-44e9892372da /var/knox/gateway-audit.log
{code}
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEaccess|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE1/ID1//|unavailable|Request
method: GET
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Groups:
[]
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|unavailable|Request
method: GET
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|success|Response
status: 200
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||access|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Response
status: 200
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEauthentication|principal|USER2|failure|LDAP
authentication failed.
17/10/10 12:50:09
||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEaccess|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE2/ID1//|success|Response
status: 401
{code}
The things to highlight here for the same correlation id:
* different topologies are being used
* different uris are being used
* different users are being used
Some of the things that we have configured that could impact results:
* authentication caching
* multiple Knox servers
* load balancer in front of Knox
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KNOX-1092) There is a carriage return in the hiveserver2 ha documentation
David Villarreal created KNOX-1092: -- Summary: There is a carriage return in the hiveserver2 ha documentation Key: KNOX-1092 URL: https://issues.apache.org/jira/browse/KNOX-1092 Project: Apache Knox Issue Type: Bug Affects Versions: 0.13.0 Reporter: David Villarreal There is a carriage return in the document before zookeeperNamespace in the hiveserver2 ha docs. ha HaProvider true HIVE maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181; zookeeperNamespace=hiveserver2 with this carriage return zookeeperNamespace becomes null in knox causing the stacktrace. Caused by: java.lang.NullPointerException at java.net.URI$Parser.parse(URI.java:3042) at java.net.URI.(URI.java:588) at java.net.URI.create(URI.java:850) at org.apache.hadoop.gateway.ha.provider.impl.DefaultURLManager.markFailed(DefaultURLManager.java:87) at org.apache.hadoop.gateway.ha.provider.impl.DefaultHaProvider.markFailedURL(DefaultHaProvider.java:91) at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.failoverRequest(DefaultHaDispatch.java:107) at org.apache.hadoop.gateway.ha.dispatch.DefaultHaDispatch.executeRequest(DefaultHaDispatch.java:94) Please correct the documentation so that the zookeeper line is all together without the carriage return. http://knox.apache.org/books/knox-0-13-0/user-guide.html#HiveServer2+HA -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KNOX-1093) KNOX Not Handling safemode state of one of the NameNode In HA state
Rajesh Chandramohan created KNOX-1093:
-
Summary: KNOX Not Handling safemode state of one of the NameNode
In HA state
Key: KNOX-1093
URL: https://issues.apache.org/jira/browse/KNOX-1093
Project: Apache Knox
Issue Type: Bug
Components: Server
Affects Versions: 0.10.0
Reporter: Rajesh Chandramohan
per your code WebHdfsHaDispatch.java , When Safemode exception happened it
calls the retryRequest() method. which also calls executeRequest() method as
like failover request but the namenode info is not changing for the thread for
all of its iteration until maxRetryAttempts=300
and retrySleep=1000 ( 1 sec )
After Max 5 minutes , client retries should pick the right namenode atleast in
next attempt.
But in this case if we need to copy a set of files in stipulated time there is
X% os connections falls into these namenode and fails. Can we candle that better
{code:java}
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest,
outboundResponse, inboundResponse);
} catch (StandbyException e) {
LOG.errorReceivedFromStandbyNode(e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (SafeModeException e) {
LOG.errorReceivedFromSafeModeNode(e);
retryRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (IOException e) {
LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
}
}
{code}
Need to change the logic in SafeModeexception state in KNOX HADispatch code to
flag the namenode which is stuck in safemode and maintain don't try queue and
redirect all further connection only to healthy active namenode . This way X5
of failures we can handle. What do we think
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KNOX-1093) KNOX Not Handling safemode state of one of the NameNode In HA state
[
https://issues.apache.org/jira/browse/KNOX-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Chandramohan updated KNOX-1093:
--
Description:
per your code WebHdfsHaDispatch.java , When Safemode exception happened it
calls the retryRequest() method. which also calls executeRequest() method as
like failover request but the namenode info is not changing for the thread for
all of its iteration until maxRetryAttempts=300
and retrySleep=1000 ( 1 sec )
After Max 5 minutes , client retries should pick the right namenode atleast in
next attempt.
But in this case if we need to copy a set of files in stipulated time there is
X% of connections falls into these namenode and fails. Can we handle that better
{code:java}
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest,
outboundResponse, inboundResponse);
} catch (StandbyException e) {
LOG.errorReceivedFromStandbyNode(e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (SafeModeException e) {
LOG.errorReceivedFromSafeModeNode(e);
retryRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (IOException e) {
LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
}
}
{code}
Need to change the logic in SafeModeexception state in KNOX HADispatch code to
flag the namenode which is stuck in safemode and maintain don't try queue and
redirect all further connection only to healthy active namenode . This way X5
of failures we can handle. What do we think
was:
per your code WebHdfsHaDispatch.java , When Safemode exception happened it
calls the retryRequest() method. which also calls executeRequest() method as
like failover request but the namenode info is not changing for the thread for
all of its iteration until maxRetryAttempts=300
and retrySleep=1000 ( 1 sec )
After Max 5 minutes , client retries should pick the right namenode atleast in
next attempt.
But in this case if we need to copy a set of files in stipulated time there is
X% os connections falls into these namenode and fails. Can we candle that better
{code:java}
try {
inboundResponse = executeOutboundRequest(outboundRequest);
writeOutboundResponse(outboundRequest, inboundRequest,
outboundResponse, inboundResponse);
} catch (StandbyException e) {
LOG.errorReceivedFromStandbyNode(e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (SafeModeException e) {
LOG.errorReceivedFromSafeModeNode(e);
retryRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
} catch (IOException e) {
LOG.errorConnectingToServer(outboundRequest.getURI().toString(), e);
failoverRequest(outboundRequest, inboundRequest, outboundResponse,
inboundResponse, e);
}
}
{code}
Need to change the logic in SafeModeexception state in KNOX HADispatch code to
flag the namenode which is stuck in safemode and maintain don't try queue and
redirect all further connection only to healthy active namenode . This way X5
of failures we can handle. What do we think
> KNOX Not Handling safemode state of one of the NameNode In HA state
>
>
> Key: KNOX-1093
> URL: https://issues.apache.org/jira/browse/KNOX-1093
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Affects Versions: 0.10.0
>Reporter: Rajesh Chandramohan
>
> per your code WebHdfsHaDispatch.java , When Safemode exception happened it
> calls the retryRequest() method. which also calls executeRequest() method as
> like failover request but the namenode info is not changing for the thread
> for all of its iteration until maxRetryAttempts=300
> and retrySleep=1000 ( 1 sec )
> After Max 5 minutes , client retries should pick the right namenode atleast
> in next attempt.
> But in this case if we need to copy a set of files in stipulated time there
> is X% of connections falls into these namenode and fails. Can we handle that
> better
> {code:java}
> try {
> inboundResponse = executeOutboundRequest(outboundRequest);
> writeOutboundResponse(outboundRequest, inboundRequest,
> outboundResponse, inboundResponse);
> } catch (StandbyException e) {
> LOG.errorReceivedFromStandbyNode(e);
> failoverRequest(outboundRequest, inboundRequest, outboundResponse,
> inboundResponse, e);
> } catch (SafeModeException e) {
> LOG.errorReceivedFromSafeModeNode(e);
> retr
[jira] [Commented] (KNOX-1091) Knox Audit Logging - duplicate correlation ids
[
https://issues.apache.org/jira/browse/KNOX-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233525#comment-16233525
]
Kevin Risden commented on KNOX-1091:
More context is available in the email thread:
https://mail-archives.apache.org/mod_mbox/knox-user/201710.mbox/%3CCAJU9nmifQR_D%3D9yVwbXVJ62VKqczZX8a4BedK6Dznwkk%3D1%2BnMw%40mail.gmail.com%3E
> Knox Audit Logging - duplicate correlation ids
> --
>
> Key: KNOX-1091
> URL: https://issues.apache.org/jira/browse/KNOX-1091
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
>Reporter: Kevin Risden
>Priority: Major
> Fix For: 0.15.0
>
>
> From the Knox User list thread: "Multiple topology audit logging", it came to
> my attention that Knox seems to be logging duplicate correlation ids.
> Separating out the topic specifically here to dig a bit deeper.
> While looking at our Knox audit logs (Knox 0.9 on HDP 2.5) the "correlation
> id" doesn't seem to be unique across requests. Is this to be expected? Here
> is a snippet (anonymized):
> grep 7557c91b-2a48-4e09-aefc-44e9892372da /var/knox/gateway-audit.log
> {code}
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEaccess|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE1/ID1//|unavailable|Request
> method: GET
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Groups:
> []
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|unavailable|Request
> method: GET
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|success|Response
> status: 200
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||access|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Response
> status: 200
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEauthentication|principal|USER2|failure|LDAP
> authentication failed.
> 17/10/10 12:50:09
> ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASEaccess|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE2/ID1//|success|Response
> status: 401
> {code}
> The things to highlight here for the same correlation id:
> * different topologies are being used
> * different uris are being used
> * different users are being used
> Some of the things that we have configured that could impact results:
> * authentication caching
> * multiple Knox servers
> * load balancer in front of Knox
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KNOX-1094) Knox loses inner exception in IllegalArgumentException issues during AD authentications
Pravin Bhagade created KNOX-1094:
Summary: Knox loses inner exception in IllegalArgumentException
issues during AD authentications
Key: KNOX-1094
URL: https://issues.apache.org/jira/browse/KNOX-1094
Project: Apache Knox
Issue Type: Bug
Components: Server
Affects Versions: 0.12.0
Reporter: Pravin Bhagade
Priority: Normal
Knox to use their Active Directory and noted that when IllegalArgumentException
exceptions are raised from a specific point in the code, the inner exception is
lost and make it difficult to diagnose the issue.
{code:java}
The exception is the one at line 733 of
https://github.com/hortonworks/knox-release/blob/HDP-2.6.2.17-tag/gateway-provider-security-shiro/src/main/java/org/apache/hadoop/gateway/shirorealm/KnoxLdapRealm.java
} catch (NamingException e) {
throw new IllegalArgumentException("Hit NamingException: " + e.getMessage());
{code}
Is it possible to change the code to preserve the inner exception ( set the
Throwable argument )?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
