----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58802/#review173221 -----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java Lines 563 (patched) <https://reviews.apache.org/r/58802/#comment246271> Default in the entity is false (and non-null), so not sure you need Objects.equals() here. ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql Lines 77-78 (original), 77-79 (patched) <https://reviews.apache.org/r/58802/#comment246272> we usually use some type of SMALLINT for booleans instead of VARCHAR(1)'s See host_role_command.is_background - Nate Cole On April 27, 2017, 12:16 p.m., Dmitro Lisnichenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58802/ > ----------------------------------------------------------- > > (Updated April 27, 2017, 12:16 p.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-20875 > https://issues.apache.org/jira/browse/AMBARI-20875 > > > Repository: ambari > > > Description > ------- > > When removing a service, the configurations for that service are kept for > historical purposes, but their various associations in the database are > removed (specifically, the {{serviceconfigmapping}} relationships). > > After removing a service, the orphaned configurations now cause a warning to > be displayed on Ambari Server startup: > > {noformat} > 2017-04-06 17:15:24,003 WARN - You have config(s): > ranger-storm-policymgr-ssl-version1467149286586,atlas-env-version1471883877194,falcon-env-version1467044148480,storm-site-version1467149286586,storm-site-version1474944944095,ranger-storm-plugin-properties-version1467149286586,hana_hadoop-env-version1476989318735,hana_hadoop-env-version1468951412523,hanaes-site-version1475773173499,hanaes-site-version1477639131416,atlas-env-version1471880496396,falcon-startup.properties-version1474944962583,ranger-storm-security-version1467149286586,falcon-env-version1474944962517,application-properties-version1471883877194,hanaes-site-version1468951412523,application-properties-version1471992143777,application-properties-version1471880496396,hana_hadoop-env-version1475790068354,hana_hadoop-env-version1477639131416,falcon-runtime.properties-version1467044148480,atlas-env-version1471992143777,hana_hadoop-env-version1475773173499,storm-env-version1467149286586,hanaes-site-version1475790068354,hanaes -site-version1476902714170,atlas-env-version1471883827584,hana_hadoop-env-version1477695406433,hanaes-site-version1476989583427,falcon-log4j-version1,falcon-env-version1474944962457,hanaes-site-version1468959251565,falcon-client.properties-version1,atlas-env-version1471993347065,falcon-startup.properties-version1467044148480,storm-cluster-log4j-version1467149286586,hanaes-site-version1472285532383,hana_hadoop-env-version1477695089738,hana_hadoop-env-version1468959251565,hana_hadoop-env-version1476989821279,atlas-log4j-version1,storm-site-version1467612840864,storm-worker-log4j-version1467149286586,ranger-storm-audit-version1467149286586,application-properties-version1471993347065,application-properties-version1471883827584,hana_hadoop-env-version1477695579450 that is(are) not mapped (in serviceconfigmapping table) to any service! > {noformat} > > These orphaned configurations have entries in both {{clusterconfig}} and > {{clusterconfigmapping}} but are otherwise not references anywhere. They > don't hurt anything, but do trigger this warning since we can't determine if > they _should_ have mappings in {{serviceconfigmapping}}. > > A few options: > - When removing a service, remove configurations as well, leaving no orphans. > Some would argue that this is a bad move since re-adding the service later > would allow you to see the old configurations. I do not believe this is true > since the old configurations are never associated with the new service's > {{serviceconfig}} or {{serviceconfigmapping}}. > > - Make the warning smarter somehow to ignore these since it's expected they > are orphaned. > -- Somehow determine the service which should own the config and see if it > exists in the cluster? > -- Add a new column to {{clusterconfig}} to mark it as deleted? > > > To clean these warnings, we had to: > {code} > CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS > (SELECT > cc.config_id, > cc.type_name, > cc.version_tag > FROM clusterconfig cc, clusterconfigmapping ccm > WHERE cc.config_id NOT IN (SELECT > scm.config_id > FROM serviceconfigmapping scm) > AND cc.type_name != 'cluster-env' > AND cc.type_name = ccm.type_name > AND cc.version_tag = ccm.version_tag); > > DELETE FROM clusterconfigmapping > WHERE EXISTS > (SELECT 1 FROM orphaned_configs > WHERE clusterconfigmapping.type_name = orphaned_configs.type_name AND > clusterconfigmapping.version_tag = orphaned_configs.version_tag); > > DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config_id > FROM orphaned_configs); > > SELECT * FROM orphaned_configs; > > DROP TABLE orphaned_configs; > {code} > > I've considered advanced heuristics based on service metainfo with config > dependencies, and service config mappings. But this approach may be > unreliable when given inaccurate service metainfo between stack upgrades. > Also, this approach is likely to delay server start on clusters with > thousands of configs. > So the solution Add a new column to clusterconfig to mark it as deleted. So > warning will not be generated for such cluster configs. > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > c33c4e3815 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ClusterDAO.java > b21aeff946 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java > f96db609b7 > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java > e533850e14 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java > 1b13a84417 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java > PRE-CREATION > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 43bf1def55 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 6def989c7f > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 1a0ab877b7 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 5c6668cb78 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > 37c69b2d11 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b78e2a4fc6 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58802/diff/1/ > > > Testing > ------- > > mvn clean test > Also manual check on Derby > > > Thanks, > > Dmitro Lisnichenko > >
