Tamas Payer created AMBARI-25569:
------------------------------------
Summary: Reassess Ambari Metrics data migration
Key: AMBARI-25569
URL: https://issues.apache.org/jira/browse/AMBARI-25569
Project: Ambari
Issue Type: Task
Components: ambari-metrics
Affects Versions: 2.7.3, 2.7.4, 2.7.5
Reporter: Tamas Payer
Assignee: Tamas Payer
Fix For: 2.7.6
The data migration process of Ambari Metrics as described at
[https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-upgrade-major/content/upgrading_HDP_post_upgrade_tasks.html]
is causing issues, like not migrating data that would be expected by the user.
(e.g. Yarn Queue metrics other than the root queue's.)
The data migration is usually called by the
{code:java}
/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/
upgrade_start /etc/ambari-metrics-collector/conf/metrics_whitelist
{code}
command where the whitelist is specified.
The migration code only looks for the metrics that are present in the whitelist
file. This is true even in the case when the AMS Whitelisting is not enabled.
The user will only have those metrics migrated that are present in the
whitelist file, which is usually not all that are required.
I suggest the following change:
- If whitelist file parameter i*s provided* then
- migrate only the metrics that are in the whitelist file
- if whitelist file parameter is *not provided* then
- if whitelisting is *enabled* then
- discover the whitelist file configured in AMS and
migrate only the metrics that are in the whitelist file
- if whitelisting is *disabled* then
- migrate *all the metrics* present in the database
Furthermore, the migration process frequently dies silently while saving the
metadata. The log message "Saving metadata to store..." is present in the logs
but the "Metadata was saved." is mostly never there, but there are no other
error messages.
I suggest revising the current solution where the saving of the metadata is
triggered in a Shutdown hook.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
