date:20170512

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Larry McCay (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009046#comment-16009046
 ] 

Larry McCay commented on KNOX-940:
--

You're right, [~risdenk] - we can close KNOX-783 as a duplicate.
Thanks for pointing it out!

> Support REST access exposing metrics
> 
>
> Key: KNOX-940
> URL: https://issues.apache.org/jira/browse/KNOX-940
> Project: Apache Knox
>  Issue Type: Task
>Affects Versions: 0.11.0
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> KNOX-643 added the support to publish metrics. However, the metrics are not 
> accessible through REST. This JIRA is to add REST support.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Kevin Risden (JIRA)


 [ 
https://issues.apache.org/jira/browse/KNOX-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated KNOX-940:
--
Affects Version/s: 0.11.0

> Support REST access exposing metrics
> 
>
> Key: KNOX-940
> URL: https://issues.apache.org/jira/browse/KNOX-940
> Project: Apache Knox
>  Issue Type: Task
>Affects Versions: 0.11.0
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> KNOX-643 added the support to publish metrics. However, the metrics are not 
> accessible through REST. This JIRA is to add REST support.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (KNOX-934) Support UI HA for resource manager

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)


 [ 
https://issues.apache.org/jira/browse/KNOX-934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mohammad Kamrul Islam resolved KNOX-934.

Resolution: Implemented

> Support UI HA for resource manager 
> ---
>
> Key: KNOX-934
> URL: https://issues.apache.org/jira/browse/KNOX-934
> Project: Apache Knox
>  Issue Type: New Feature
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> Looks like, Knox doesn't support YARN UI HA. For example, if there is two RM 
> instances running, any access to RM UI redirects ( code 307)  to the active 
> RM UI.
> This redirect doesn't work when the actual UI is running behind a firewall 
> (because primary UI is not directly accessible).
> However, if KNOX hits the active UI directly - there will be no forwarding 
> and UI is displayed properly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-934) Support UI HA for resource manager

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009013#comment-16009013
 ] 

Mohammad Kamrul Islam commented on KNOX-934:


Thanks [~jeffreyr97]  and [~lmc...@apache.org] for the response.


I was trying with Knox-0.9.1. 
I think KNOX-719 addressed my ask.
Closing the JIRA.




> Support UI HA for resource manager 
> ---
>
> Key: KNOX-934
> URL: https://issues.apache.org/jira/browse/KNOX-934
> Project: Apache Knox
>  Issue Type: New Feature
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> Looks like, Knox doesn't support YARN UI HA. For example, if there is two RM 
> instances running, any access to RM UI redirects ( code 307)  to the active 
> RM UI.
> This redirect doesn't work when the actual UI is running behind a firewall 
> (because primary UI is not directly accessible).
> However, if KNOX hits the active UI directly - there will be no forwarding 
> and UI is displayed properly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009009#comment-16009009
 ] 

Mohammad Kamrul Islam commented on KNOX-940:


Thanks [~lmccay].
I will look into knoxtoken  service.

I will follow the patch process. The diff in previous comment is to share my 
POC not a review candidate.


> Support REST access exposing metrics
> 
>
> Key: KNOX-940
> URL: https://issues.apache.org/jira/browse/KNOX-940
> Project: Apache Knox
>  Issue Type: Task
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> KNOX-643 added the support to publish metrics. However, the metrics are not 
> accessible through REST. This JIRA is to add REST support.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Larry McCay (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008868#comment-16008868
 ] 

Larry McCay commented on KNOX-940:
--

Hi [~kamrul] - I see you are taking a different approach in terms of library 
than what was suggested on the dev@ list discussion.
That is fine.

However, I do think that we will want to develop this API in the normal Jersey 
mechanism that we use for other Knox hosted REST APIs.
Take a look at the knoxtoken service as an example. This approach allows for 
the use of the rest of the provider infrastructure as well.

When you have a patch, please attach the file and click the submit patch button 
rather than copying the diff into a comment.
Much easier to review and we have a jenkins job that tries to validate 
submitted patches.


> Support REST access exposing metrics
> 
>
> Key: KNOX-940
> URL: https://issues.apache.org/jira/browse/KNOX-940
> Project: Apache Knox
>  Issue Type: Task
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> KNOX-643 added the support to publish metrics. However, the metrics are not 
> accessible through REST. This JIRA is to add REST support.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-934) Support UI HA for resource manager

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008839#comment-16008839
 ] 

Mohammad Kamrul Islam commented on KNOX-934:


Can someone please help me what is the best way to address this? In addition, 
is there any similar  case?



> Support UI HA for resource manager 
> ---
>
> Key: KNOX-934
> URL: https://issues.apache.org/jira/browse/KNOX-934
> Project: Apache Knox
>  Issue Type: New Feature
>Reporter: Mohammad Kamrul Islam
>Assignee: Mohammad Kamrul Islam
>
> Looks like, Knox doesn't support YARN UI HA. For example, if there is two RM 
> instances running, any access to RM UI redirects ( code 307)  to the active 
> RM UI.
> This redirect doesn't work when the actual UI is running behind a firewall 
> (because primary UI is not directly accessible).
> However, if KNOX hits the active UI directly - there will be no forwarding 
> and UI is displayed properly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008836#comment-16008836
 ] 

Mohammad Kamrul Islam commented on KNOX-940:


I need recommendations for the implementation.

Knox currently utilizes dropwizard to support metrics collection and reporting.
Dropwizard also supports  framework to publish the metrics through REST , in 
addition of  other important endpoints.

Key part is to add 
[AdminServlet|http://metrics.dropwizard.io/3.1.0/manual/servlets/#adminservlet] 
into gateway service.
In addition, we need to add couple of ServletListeners.

My question is : What is the recommended way of adding new servlet in Knox? Any 
example or details will make things easier for me.

I did a POC by creating the following diff and I accessed through these cURL 
commands. This can show what we have to do ultimately. But I'm not sure : where 
this code fits the best.
 
{quote}
curl -i -k https://localhost:8443/gateway/sandbox//metrics/metrics
curl -i -k https://localhost:8443/gateway/sandbox//metrics/ping
curl -i -k 
https://localhost:8443/gateway/sandbox//metrics/healthcheck?pretty=true
{quote}

{quote}
diff --git a/gateway-server/pom.xml b/gateway-server/pom.xml
index 652e5f3b..755e3a0e 100644
--- a/gateway-server/pom.xml
+++ b/gateway-server/pom.xml
@@ -328,6 +328,11 @@
 xmltool
 test
 
+
+io.dropwizard.metrics
+metrics-servlets
+3.1.2
+
 
 
 
diff --git 
a/gateway-server/src/main/java/org/apache/hadoop/gateway/deploy/DeploymentFactory.java
 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/deploy/DeploymentFactory.java
index d834e740..07eaed69 100644
--- 
a/gateway-server/src/main/java/org/apache/hadoop/gateway/deploy/DeploymentFactory.java
+++ 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/deploy/DeploymentFactory.java
@@ -37,6 +37,7 @@ import javax.xml.bind.JAXBContext;
 import javax.xml.bind.JAXBException;
 import javax.xml.bind.Marshaller;
 
+import com.codahale.metrics.servlets.AdminServlet;
 import org.apache.hadoop.gateway.GatewayMessages;
 import org.apache.hadoop.gateway.GatewayServlet;
 import org.apache.hadoop.gateway.config.GatewayConfig;
@@ -356,6 +357,10 @@ public abstract class DeploymentFactory {
   wad.createFilter().filterName( filterName ).filterClass( 
GatewayServlet.class.getName() );
   wad.createFilterMapping().filterName( filterName ).urlPattern( "/*" );
 }
+String servletName = "metrics-admin";
+System.out.println("registering AdminServlet");
+wad.createServlet().servletName( servletName ).servletClass( 
AdminServlet.class.getName() );
+wad.createServletMapping().servletName( servletName ).urlPattern( 
"/metrics/*" );
 if (gatewayServices != null) {
   gatewayServices.initializeContribution(context);
 } else {
diff --git 
a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/DefaultGatewayServices.java
 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/services/DefaultGatewayServices.java
index 2cf043eb..8cd672e5 100644
--- 
a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/DefaultGatewayServices.java
+++ 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/services/DefaultGatewayServices.java
@@ -218,6 +218,8 @@ public class DefaultGatewayServices implements 
GatewayServices {
   @Override
   public void finalizeContribution(DeploymentContext context) {
 // Tell the provider the location of the descriptor.
-context.getWebAppDescriptor().createListener().listenerClass( 
GatewayServicesContextListener.class.getName() );
+
context.getWebAppDescriptor().createListener().listenerClass(GatewayServicesContextListener.class.getName());
+
context.getWebAppDescriptor().createListener().listenerClass(GatewayMetricsServletContextListener.class.getName());
+
context.getWebAppDescriptor().createListener().listenerClass(GatewayHealthCheckServletContextListener.class.getName());
   }
 }
diff --git 
a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/GatewayHealthCheckServletContextListener.java
 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/services/GatewayHealthCheckServletContextListener.java
new file mode 100644
index ..1b0f3ef5
--- /dev/null
+++ 
b/gateway-server/src/main/java/org/apache/hadoop/gateway/services/GatewayHealthCheckServletContextListener.java
@@ -0,0 +1,19 @@
+package org.apache.hadoop.gateway.services;
+
+import com.codahale.metrics.health.HealthCheckRegistry;
+import com.codahale.metrics.servlets.HealthCheckServlet;
+
+/**
+ * Created by mislam on 5/9/17.
+ */
+public class GatewayHealthCheckServletContextListener extends 
HealthCheckServlet.ContextListener {
+
+  public static final HealthCheckRegistry HEALTH_CHECK_REGISTRY = new 
HealthCheckRegistry();
+
+  @Override
+  protected HealthCheckRegistry getHealthCheckRegistry()

[jira] [Created] (KNOX-940) Support REST access exposing metrics

2017-05-12 Thread Mohammad Kamrul Islam (JIRA)

Mohammad Kamrul Islam created KNOX-940:
--

 Summary: Support REST access exposing metrics
 Key: KNOX-940
 URL: https://issues.apache.org/jira/browse/KNOX-940
 Project: Apache Knox
  Issue Type: Task
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam


KNOX-643 added the support to publish metrics. However, the metrics are not 
accessible through REST. This JIRA is to add REST support.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-939) Missing Licenses Element in POM files

2017-05-12 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008285#comment-16008285
 ] 

ASF subversion and git services commented on KNOX-939:
--

Commit fb2e31348a25619e244ff6cf1e634c01001e2f18 in knox's branch 
refs/heads/master from [~lmccay]
[ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=fb2e313 ]

KNOX-939 - Missing Licenses Element in POM files

> Missing Licenses Element in POM files
> -
>
> Key: KNOX-939
> URL: https://issues.apache.org/jira/browse/KNOX-939
> Project: Apache Knox
>  Issue Type: Bug
>  Components: Server
>Reporter: Larry McCay
>Assignee: Larry McCay
> Fix For: 0.13.0
>
>
> Both of the following POM files need license elements to add clarity and 
> consistency across the modules.
> https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxtoken/0.12.0/gateway-service-knoxtoken-0.12.0.pom
> https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxssout/0.11.0/gateway-service-knoxssout-0.11.0.pom



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (KNOX-939) Missing Licenses Element in POM files

2017-05-12 Thread Larry McCay (JIRA)


 [ 
https://issues.apache.org/jira/browse/KNOX-939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Larry McCay resolved KNOX-939.
--
Resolution: Fixed

> Missing Licenses Element in POM files
> -
>
> Key: KNOX-939
> URL: https://issues.apache.org/jira/browse/KNOX-939
> Project: Apache Knox
>  Issue Type: Bug
>  Components: Server
>Reporter: Larry McCay
>Assignee: Larry McCay
> Fix For: 0.13.0
>
>
> Both of the following POM files need license elements to add clarity and 
> consistency across the modules.
> https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxtoken/0.12.0/gateway-service-knoxtoken-0.12.0.pom
> https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxssout/0.11.0/gateway-service-knoxssout-0.11.0.pom



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (KNOX-939) Missing Licenses Element in POM files

2017-05-12 Thread Larry McCay (JIRA)

Larry McCay created KNOX-939:


 Summary: Missing Licenses Element in POM files
 Key: KNOX-939
 URL: https://issues.apache.org/jira/browse/KNOX-939
 Project: Apache Knox
  Issue Type: Bug
  Components: Server
Reporter: Larry McCay
Assignee: Larry McCay
 Fix For: 0.13.0


Both of the following POM files need license elements to add clarity and 
consistency across the modules.

https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxtoken/0.12.0/gateway-service-knoxtoken-0.12.0.pom

https://search.maven.org/remotecontent?filepath=org/apache/knox/gateway-service-knoxssout/0.11.0/gateway-service-knoxssout-0.11.0.pom




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-928) Topology Port Mapping

2017-05-12 Thread Larry McCay (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008218#comment-16008218
 ] 

Larry McCay commented on KNOX-928:
--

No worries at all!

> Topology Port Mapping
> -
>
> Key: KNOX-928
> URL: https://issues.apache.org/jira/browse/KNOX-928
> Project: Apache Knox
>  Issue Type: New Feature
>  Components: Server
>Reporter: Sandeep More
>Assignee: Sandeep More
>  Labels: KIP-6
> Fix For: 0.13.0
>
> Attachments: KNOX-928.001.patch, KNOX-928.002.patch
>
>
> Instead of having a single "default topology" we can map topologies to
> specific ports and have a dedicated URL without the gateway specific app
> context.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-928) Topology Port Mapping

2017-05-12 Thread Sandeep More (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008176#comment-16008176
 ] 

Sandeep More commented on KNOX-928:
---

Sorry about that ! looks like I messed up the rebase :(  
I will try to start from scratch - from the first patch on the master and 
upload the new patch.
Again, apologies and thanks for the review !


> Topology Port Mapping
> -
>
> Key: KNOX-928
> URL: https://issues.apache.org/jira/browse/KNOX-928
> Project: Apache Knox
>  Issue Type: New Feature
>  Components: Server
>Reporter: Sandeep More
>Assignee: Sandeep More
>  Labels: KIP-6
> Fix For: 0.13.0
>
> Attachments: KNOX-928.001.patch, KNOX-928.002.patch
>
>
> Instead of having a single "default topology" we can map topologies to
> specific ports and have a dedicated URL without the gateway specific app
> context.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KNOX-937) Remove redundant type information from Collections

2017-05-12 Thread Colm O hEigeartaigh (JIRA)


[ 
https://issues.apache.org/jira/browse/KNOX-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007817#comment-16007817
 ] 

Colm O hEigeartaigh commented on KNOX-937:
--

Hi [~lmccay],

I was testing out my sed skills :-)

find . -name "*.java" -exec sed -i 's/new TreeMap<.\+>/new TreeMap<>/' {} \;

etc.

Colm.

> Remove redundant type information from Collections
> --
>
> Key: KNOX-937
> URL: https://issues.apache.org/jira/browse/KNOX-937
> Project: Apache Knox
>  Issue Type: Improvement
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Trivial
> Fix For: 0.13.0
>
> Attachments: 
> 0001-KNOX-937-Remove-redundant-type-information-from-Coll.patch
>
>
> As we are using Java 7, there is no need to specify redundant type 
> information when creating Collections.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Jenkins build is back to normal : Knox-master-patch-scan #45040

2017-05-12 Thread Apache Jenkins Server

See

Build failed in Jenkins: Knox-master-patch-scan #45039

2017-05-12 Thread Apache Jenkins Server

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Hadoop) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/knox.git # timeout=10
Fetching upstream changes from https://github.com/apache/knox.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/knox.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision c32cd10f2cc10b053d2bda47a76bcbb51d2e802e 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f c32cd10f2cc10b053d2bda47a76bcbb51d2e802e
 > git rev-list c32cd10f2cc10b053d2bda47a76bcbb51d2e802e # timeout=10
[Knox-master-patch-scan] $ /bin/bash /tmp/hudson7594023675424172820.sh
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
curl: (35) Unknown SSL protocol error in connection to issues.apache.org:443 
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

[jira] [Updated] (KNOX-940) Support REST access exposing metrics

[jira] [Resolved] (KNOX-934) Support UI HA for resource manager

[jira] [Commented] (KNOX-934) Support UI HA for resource manager

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

[jira] [Commented] (KNOX-934) Support UI HA for resource manager

[jira] [Commented] (KNOX-940) Support REST access exposing metrics

[jira] [Created] (KNOX-940) Support REST access exposing metrics

[jira] [Commented] (KNOX-939) Missing Licenses Element in POM files

[jira] [Resolved] (KNOX-939) Missing Licenses Element in POM files

[jira] [Created] (KNOX-939) Missing Licenses Element in POM files

[jira] [Commented] (KNOX-928) Topology Port Mapping

[jira] [Commented] (KNOX-928) Topology Port Mapping

[jira] [Commented] (KNOX-937) Remove redundant type information from Collections

Jenkins build is back to normal : Knox-master-patch-scan #45040

Build failed in Jenkins: Knox-master-patch-scan #45039

17 matches

Site Navigation

Mail list logo

Footer information