> On March 2, 2016, 4:56 a.m., Matt wrote: > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, > > line 1 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275503#file1275503line1> > > > > If I were to add HAWQ 2.0.0's metainfo.xml under HDP 2.3 stack, and say > > that it extends from common-services, will this service_advisor be honored?
I had some difficulty with HAWQ and PXF because they are not actually in the stack and can be added by installing a separate RPM. In order for HAWQ and PXF to inherit the service_advisor.py files from common-services, they would need to implement mostly empty service advisors under the <stack>/<version>/services/<service> folder that explicitly inherit from the ones in common-services. > On March 2, 2016, 4:56 a.m., Matt wrote: > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py, > > line 1 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275515#file1275515line1> > > > > If the service_advisor is under HDP 2.0.6's GANGLIA service folder, it > > is not required to repeat the stack name, the version and the service on > > the filename, for every combination of stack, version and service. > > > > This would be good enough: > > HDP/2.0.6/services/GANGLIA/service_advisor.py I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following: SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/') sys.path.append(PARENT_DIR) from service_advisor_HDP22HDFS import * If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class. The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names). > On March 2, 2016, 4:56 a.m., Matt wrote: > > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py, line > > 22 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275586#file1275586line22> > > > > If all the stack_advisor.py files are being refactored into service > > level advisors, the unit tests should be at service level as well. > > > > This would bring better modularity for introducing a new service: > > - Add a new service to the stack > > - Implement the service advisor for that service > > - Implement unit test for the service advisor for that service (For the > > current patch, the tests are still with stack advisor and not service > > advisor) I agree but hopefully that can be a future enhancement? I'd rather get this included and then work on refactoring the test code. > On March 2, 2016, 4:56 a.m., Matt wrote: > > ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py, line 1 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275592#file1275592line1> > > > > Why are the old stack_advisors retained under test folder? They are retained because there was some test code that wanted to run a test against the old stack advisor and the new one. It is a performance test, I guess making sure the new stack advisor doesn't slow things down. - Tim ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review121576 ----------------------------------------------------------- On March 1, 2016, 4:53 p.m., Tim Thorpe wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > ----------------------------------------------------------- > > (Updated March 1, 2016, 4:53 p.m.) > > > Review request for Ambari, Sumit Mohanty and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > ------- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > ----- > > ambari-agent/pom.xml c2c993f > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py > 53591cd > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > f6f8cde > > ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py > 3a552b8 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py > bffbc26 > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py > 985c101 > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 9dd5cc7 > ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py > af269de > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py > b99c484 > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py > 3631c88 > ambari-server/src/main/resources/stacks/default_stack_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25 > ambari-server/src/test/python/TestStackAdvisor.py 149ae1d > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > e15582e > ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py > 7ede5a3 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > d95e6c7 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py > 5b8c30c > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 8932bde > > ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py > PRE-CREATION > ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py > ba140bb > > ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py > PRE-CREATION > ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py > 74a4b31 > ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > ------- > > Ran all the non java unit tests. > > Total run:921 > Total errors:1 > Total failures:0 > > The only test which failed also failed on a fresh trunk test run: test_start > (test_metrics_grafana.TestMetricsGrafana) > > > Thanks, > > Tim Thorpe > >
