[
https://issues.apache.org/jira/browse/KNOX-2284?focusedWorklogId=400764&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-400764
]
ASF GitHub Bot logged work on KNOX-2284:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Mar/20 15:20
Start Date: 10/Mar/20 15:20
Worklog Time Spent: 10m
Work Description: smolnar82 commented on pull request #285: KNOX-2284 -
Handling CM descriptors after Knox shared-provider/descriptor/topology monitors
are started
URL: https://github.com/apache/knox/pull/285
## What changes were proposed in this pull request?
Handling CM descriptors after Knox shared-provider/descriptor/topology
monitors are started
## How was this patch tested?
Tested manually:
1. redeployed Knox with my changes in place; brand new installation, only
the OOTB topologies existed
2. produced a CM descriptor (cm-topologies.cm) with the following content:
```
<configuration>
<property>
<name>topology1</name>
<value>
providerConfigRef=default-providers;
app:knoxauth:param1.name=param1.value;
HIVE:url=http://localhost:456;
HIVE:version=1.0;
HIVE:httpclient.connectionTimeout=5m;
HIVE:httpclient.socketTimeout=100m
</value>
</property>
<property>
<name>topology2</name>
<value>
providerConfigRef=default-providers;
HDFS:url=http://localhost:456;
HDFS:httpclient.connectionTimeout=5m;
HDFS:httpclient.socketTimeout=100m
</value>
</property>
</configuration>
```
3. started Knox and confirmed (from logs and using the Admin UI):
- `$KNOX_HOME/conf/descriptors/topology1.json` and
`$KNOX_HOME/conf/descriptors/topology2.json` got persisted
- Knox picked them up and generated
`$KNOX_HOME/data/deployments/topology1.topo.170c48f42c0` and
`$KNOX_HOME/data/deployments/topology2.topo.170c48f42c0`
Relevant logs:
```
2020-03-10 14:07:50,348 INFO knox.gateway
(DefaultTopologyService.java:init(648)) - Monitoring simple descriptors in
directory: /Users/smolnar/test/knoxGateway/conf/descriptors
2020-03-10 14:07:50,577 INFO knox.gateway
(DefaultTopologyService.java:init(653)) - Monitoring shared provider
configurations in directory:
/Users/smolnar/test/knoxGateway/conf/shared-providers
...
2020-03-10 14:08:02,899 INFO knox.gateway (GatewayServer.java:start(700)) -
Monitoring topologies in directory:
/Users/smolnar/test/knoxGateway/conf/topologies
2020-03-10 14:08:02,920 INFO discovery.advanced
(AdvancedServiceDiscoveryConfigurationMonitor.java:setupMonitor(73)) -
Monitoring
/Users/smolnar/test/knoxGateway/conf/auto-discovery-advanced-configuration-*
for advanced service discovery configuration changes.
2020-03-10 14:08:02,925 INFO knox.gateway
(ClouderaManagerDescriptorMonitor.java:setupMonitor(66)) - Monitoring Cloudera
Manager descriptors in /Users/smolnar/test/knoxGateway/conf/descriptors ...
2020-03-10 14:08:02,925 INFO knox.gateway
(ClouderaManagerDescriptorParser.java:parse(79)) - Parsing Cloudera Manager
descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm.
Looking up all topologies...
2020-03-10 14:08:02,925 INFO knox.gateway
(GatewayServer.java:startGateway(392)) - Started gateway on port 8,443.
2020-03-10 14:08:02,936 INFO knox.gateway
(ClouderaManagerDescriptorParser.java:parse(84)) - Found Knox descriptors
topology1, topology2 in
/Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm
2020-03-10 14:08:03,125 INFO knox.gateway
(ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor
/Users/smolnar/test/knoxGateway/conf/descriptors/topology1.json
2020-03-10 14:08:03,125 INFO knox.gateway
(ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor
/Users/smolnar/test/knoxGateway/conf/descriptors/topology2.json
2020-03-10 14:08:08,011 INFO knox.gateway
(DefaultTopologyService.java:onFileChange(872)) - Generated topology
topology1.xml because the associated descriptor topology1.json changed.
...
2020-03-10 14:08:14,278 INFO knox.gateway
(GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology
topology1 to
/Users/smolnar/test/knoxGateway/data/deployments/topology1.topo.170c48f42c0
2020-03-10 14:08:14,278 INFO knox.gateway
(GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology
topology1
2020-03-10 14:08:14,450 INFO knox.gateway
(DefaultGatewayServices.java:initializeContribution(161)) - Creating credential
store for the cluster: topology1
2020-03-10 14:08:14,753 INFO knox.gateway
(GatewayServer.java:internalActivateTopology(898)) - Activating topology
topology1
2020-03-10 14:08:14,753 INFO knox.gateway
(GatewayServer.java:internalActivateArchive(908)) - Activating topology
topology1 archive %2Fknoxauth
2020-03-10 14:08:14,833 INFO knox.gateway
(GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology
topology2 to
/Users/smolnar/test/knoxGateway/data/deployments/topology2.topo.170c48f42c0
2020-03-10 14:08:14,833 INFO knox.gateway
(GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology
topology2
2020-03-10 14:08:14,940 INFO knox.gateway
(GatewayServer.java:internalActivateTopology(898)) - Activating topology
topology2
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 400764)
Remaining Estimate: 0h
Time Spent: 10m
> CM descriptors do not get picked up at Gateway start
> ----------------------------------------------------
>
> Key: KNOX-2284
> URL: https://issues.apache.org/jira/browse/KNOX-2284
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.4.0
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Major
> Fix For: 1.4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When there is a CM descriptor in the KNOX_DESCRIPTORS_DIR at startup time and
> the KNOX descriptors found in this CM descriptor did not exist before then
> Knox does not pick up them.
> The reason is: the shared-provider/descriptor/topology monitor starts after
> the CM descriptor is being parsed and the Knox descriptors are persisted.
> Recommended solution: move the CM descriptor handling after the monitors are
> started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
