[jira] [Updated] (RANGER-1289) Error occured in Ranger KMS function
[ https://issues.apache.org/jira/browse/RANGER-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1289: Fix Version/s: 1.0.0 > Error occured in Ranger KMS function > > > Key: RANGER-1289 > URL: https://issues.apache.org/jira/browse/RANGER-1289 > Project: Ranger > Issue Type: Bug > Components: kms >Affects Versions: 0.7.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > > > Steps: > 1.Start ranger-kms service > 2.Configure KMS Client and restart hdfs > 3.Create a key named key0 in the Ranger Web UI > 4.Execute the following command in hadoop environment,create an encrypted zone > {code:java} > hdfs dfs -mkdir /keyZone > hdfs crypto -createZone -keyName key0 -path /keyZone > {code} > Error message poped out as below: > ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log > {code:java} > 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for > servlet [webservices-driver] in context with path [/kms] threw exception > java.lang.NullPointerException > at > org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) > at > org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) > at > org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) > at > org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > {code} > I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there > is a bug, as follows: > org/apache/http/client/utils/URLEncodedUtils.java > {code:java} > public static List parse(String s, Charset charset) { > CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, new char[]{'&', ';'}); > } > {code} > When the parameter 's' is null, it will pop out NullPointException. > And in httpclient version 4.5.3, there is no problem. the new code is as > follows: > {code:java} > public static List parse(final String s, final Charset > charset) { > if (s == null) { > return Collections.emptyList(); > } > final CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, QP_SEP_A, QP_SEP_S); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1289) Error occured in Ranger KMS function
[ https://issues.apache.org/jira/browse/RANGER-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-1289: Attachment: (was: 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch) > Error occured in Ranger KMS function > > > Key: RANGER-1289 > URL: https://issues.apache.org/jira/browse/RANGER-1289 > Project: Ranger > Issue Type: Bug > Components: kms >Affects Versions: 0.7.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: > 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > > > Steps: > 1.Start ranger-kms service > 2.Configure KMS Client and restart hdfs > 3.Create a key named key0 in the Ranger Web UI > 4.Execute the following command in hadoop environment,create an encrypted zone > {code:java} > hdfs dfs -mkdir /keyZone > hdfs crypto -createZone -keyName key0 -path /keyZone > {code} > Error message poped out as below: > ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log > {code:java} > 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for > servlet [webservices-driver] in context with path [/kms] threw exception > java.lang.NullPointerException > at > org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) > at > org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) > at > org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) > at > org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > {code} > I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there > is a bug, as follows: > org/apache/http/client/utils/URLEncodedUtils.java > {code:java} > public static List parse(String s, Charset charset) { > CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, new char[]{'&', ';'}); > } > {code} > When the parameter 's' is null, it will pop out NullPointException. > And in httpclient version 4.5.3, there is no problem. the new code is as > follows: > {code:java} > public static List parse(final String s, final Charset > charset) { > if (s == null) { > return Collections.emptyList(); > } > final CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, QP_SEP_A, QP_SEP_S); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1289) Error occured in Ranger KMS function
[ https://issues.apache.org/jira/browse/RANGER-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-1289: Attachment: 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > Error occured in Ranger KMS function > > > Key: RANGER-1289 > URL: https://issues.apache.org/jira/browse/RANGER-1289 > Project: Ranger > Issue Type: Bug > Components: kms >Affects Versions: 0.7.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: > 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > > > Steps: > 1.Start ranger-kms service > 2.Configure KMS Client and restart hdfs > 3.Create a key named key0 in the Ranger Web UI > 4.Execute the following command in hadoop environment,create an encrypted zone > {code:java} > hdfs dfs -mkdir /keyZone > hdfs crypto -createZone -keyName key0 -path /keyZone > {code} > Error message poped out as below: > ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log > {code:java} > 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for > servlet [webservices-driver] in context with path [/kms] threw exception > java.lang.NullPointerException > at > org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) > at > org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) > at > org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) > at > org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > {code} > I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there > is a bug, as follows: > org/apache/http/client/utils/URLEncodedUtils.java > {code:java} > public static List parse(String s, Charset charset) { > CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, new char[]{'&', ';'}); > } > {code} > When the parameter 's' is null, it will pop out NullPointException. > And in httpclient version 4.5.3, there is no problem. the new code is as > follows: > {code:java} > public static List parse(final String s, final Charset > charset) { > if (s == null) { > return Collections.emptyList(); > } > final CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, QP_SEP_A, QP_SEP_S); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1289) Error occured in Ranger KMS function
[ https://issues.apache.org/jira/browse/RANGER-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-1289: Attachment: 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > Error occured in Ranger KMS function > > > Key: RANGER-1289 > URL: https://issues.apache.org/jira/browse/RANGER-1289 > Project: Ranger > Issue Type: Bug > Components: kms >Affects Versions: 0.7.0 >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > Attachments: > 0001-RANGER-1289-Error-occured-in-Ranger-KMS-function.patch > > > Steps: > 1.Start ranger-kms service > 2.Configure KMS Client and restart hdfs > 3.Create a key named key0 in the Ranger Web UI > 4.Execute the following command in hadoop environment,create an encrypted zone > {code:java} > hdfs dfs -mkdir /keyZone > hdfs crypto -createZone -keyName key0 -path /keyZone > {code} > Error message poped out as below: > ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log > {code:java} > 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for > servlet [webservices-driver] in context with path [/kms] threw exception > java.lang.NullPointerException > at > org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) > at > org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) > at > org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) > at > org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) > at > org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > {code} > I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there > is a bug, as follows: > org/apache/http/client/utils/URLEncodedUtils.java > {code:java} > public static List parse(String s, Charset charset) { > CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, new char[]{'&', ';'}); > } > {code} > When the parameter 's' is null, it will pop out NullPointException. > And in httpclient version 4.5.3, there is no problem. the new code is as > follows: > {code:java} > public static List parse(final String s, final Charset > charset) { > if (s == null) { > return Collections.emptyList(); > } > final CharArrayBuffer buffer = new CharArrayBuffer(s.length()); > buffer.append(s); > return parse(buffer, charset, QP_SEP_A, QP_SEP_S); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (RANGER-1289) Error occured in Ranger KMS function
[ https://issues.apache.org/jira/browse/RANGER-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Zhang updated RANGER-1289: Description: Steps: 1.Start ranger-kms service 2.Configure KMS Client and restart hdfs 3.Create a key named key0 in the Ranger Web UI 4.Execute the following command in hadoop environment,create an encrypted zone {code:java} hdfs dfs -mkdir /keyZone hdfs crypto -createZone -keyName key0 -path /keyZone {code} Error message poped out as below: ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log {code:java} 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception java.lang.NullPointerException at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) {code} I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows: org/apache/http/client/utils/URLEncodedUtils.java {code:java} public static List parse(String s, Charset charset) { CharArrayBuffer buffer = new CharArrayBuffer(s.length()); buffer.append(s); return parse(buffer, charset, new char[]{'&', ';'}); } {code} When the parameter 's' is null, it will pop out NullPointException. And in httpclient version 4.5.3, there is no problem. the new code is as follows: {code:java} public static List parse(final String s, final Charset charset) { if (s == null) { return Collections.emptyList(); } final CharArrayBuffer buffer = new CharArrayBuffer(s.length()); buffer.append(s); return parse(buffer, charset, QP_SEP_A, QP_SEP_S); } {code} was: Steps: 1.Start ranger-kms service 2.Configure KMS Client and restart hdfs 3.Create a key named key0 in the Ranger Web UI 4.Execute the following command in hadoop environment,create an encrypted zone {code:java} hdfs dfs -mkdir /keyZone hdfs crypto -createZone -keyName key0 -path /keyZone {code} Error message poped out as below: ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log {code:java} 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception java.lang.NullPointerException at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235) at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514) at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java