Hi Matt, That explains it! I had to bump the build due to various packages sitting behind the hortonworks paywall.
I’ll certainly sign up for the Dev mailing list, thanks for the reminder of it. I love the bleeding edge, and I’ll certainly take a look at the documentation. Thanks for the information, I appreciate it! Kind Regards, Nathan Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Matt Andruff <[email protected]> Sent: Wednesday, October 6, 2021 4:07 pm To: [email protected] Subject: Re: Fw: Bigtop Ambari Mpack Issue You don't often get email from [email protected]. Learn why this is important<http://aka.ms/LearnAboutSenderIdentification> Hey Nathan, This Bigtop MPAC was only tested on the 2.6.2 of Ambari. So you are likely in new territory! Welcome to the bleeding edge! Ambari MPack Documentation.<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FManagement%2BPacks&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604250711%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vHkbB58WJsGRCD%2BZzWiZh5EYMSgIbvPpJQSlGOpqXUI%3D&reserved=0> You can try the following paths Watch out for BigTop/Ambari gotchas. (Big Top supports OS's that aren't supported by Ambari. Make sure you are on a supported platform for both BIg Top and Ambari) Look at the Stack for 2.7.5. You can borrow from it to find what's missing/needed to help update the MPack from 2.6.2 to work with 2.7.5 You may be able to also ask for help on the Ambari Dev forum. ([email protected]<mailto:[email protected]> is likely not your target audience.) I haven't tried it but you might get lucky. On Wed, Oct 6, 2021 at 7:04 AM <[email protected]<mailto:[email protected]>> wrote: Hi There, I'm not sure which community fits the issue I'm having best, so apologies if it seems like I am duplicating an email across Bigtop and Ambari. As you can see in the below email, I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error. I've have retried the deployment again on a fresh cluster, but still getting the same error. When I look at the file '/var/lib/ambari-agent/data/command-13.json' it doesn't have a stack packages object in the cluster-env object. I have also attached a copy of the file. Stack Packages is also not present in the cluster-env.xml on the server. I'm a bit confused as to where I should look next, so if anyone has any suggestions, I would really appreciate it. Kind Regards, Nathan ________________________________ From: English,N,Nathan,VIR R Sent: 06 October 2021 10:13 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Bigtop Ambari Mpack Issue Hi There, I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error. Looking at stack_select.py[1], It looks like it's searching for the 'stack_packages' in the cluster-env.xml (Path: "/configurations/cluster-env/stack_packages") however when I look at the cluster-env.xml[2] in the bigtop mpack I don't see anything called stack_packages? I could be completely wrong at what it's looking for though! Does anyone have any ideas or suggestions? Kind Regards, Nathan stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 39, in <module> AfterInstallHook().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 32, in hook setup_stack_symlinks(self.stroutfile) File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", line 53, in setup_stack_symlinks stack_packages = stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL) File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 185, in get_packages raise Fail("The stack packages are not defined on the command. Unable to load packages for the stack-select tool") resource_management.core.exceptions.Fail: The stack packages are not defined on the command. Unable to load packages for the stack-select tool stdout: 2021-10-06 08:18:24,404 - Stack Feature Version Info: Cluster Stack=1.0, Command Stack=None, Command Version=None -> 1.0 2021-10-06 08:18:24,411 - Skipping creation of User and Group as host is sys prepped or ignore_groupsusers_create flag is on 2021-10-06 08:18:24,412 - Skipping setting dfs cluster admin and tez view acls as host is sys prepped 2021-10-06 08:18:24,412 - FS Type: HDFS 2021-10-06 08:18:24,412 - Directory['/etc/hadoop'] {'mode': 0755} 2021-10-06 08:18:24,464 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2021-10-06 08:18:24,466 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777} 2021-10-06 08:18:24,487 - Repository['BGTP-1.0-repo-4'] {'base_url': 'http://c7905.ambari.apache.org/<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fc7905.ambari.apache.org%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604260706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PhXohJ1GgI%2FQMqg4IkkybyE273nz8Suv87PjNpQ%2FD3A%3D&reserved=0>', 'action': ['prepare'], 'components': [u'BGTP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-bgtp-4', 'mirror_list': None} 2021-10-06 08:18:24,501 - Repository[None] {'action': ['create']} 2021-10-06 08:18:24,503 - File['/tmp/tmpSnZLdC'] {'content': '[BGTP-1.0-repo-4]\nname=BGTP-1.0-repo-4\nbaseurl=http://c7905.ambari.apache.org/\n\npath=/\nenabled=1\ngpgcheck=0<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fc7905.ambari.apache.org%2F%255Cn%255Cnpath%3D%2F%255Cnenabled%3D1%255Cngpgcheck%3D0&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604260706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PCe6hzAAiOY%2FxPfJe5T0kF%2FE7%2FwWJlRcWdfBsSar6KI%3D&reserved=0>'} 2021-10-06 08:18:24,504 - Writing File['/tmp/tmpSnZLdC'] because contents don't match 2021-10-06 08:18:24,505 - Rewriting /etc/yum.repos.d/ambari-bgtp-4.repo since it has changed. 2021-10-06 08:18:24,505 - File['/etc/yum.repos.d/ambari-bgtp-4.repo'] {'content': StaticFile('/tmp/tmpSnZLdC')} 2021-10-06 08:18:24,507 - Writing File['/etc/yum.repos.d/ambari-bgtp-4.repo'] because it doesn't exist 2021-10-06 08:18:24,508 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2021-10-06 08:18:24,689 - Skipping installation of existing package unzip 2021-10-06 08:18:24,689 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2021-10-06 08:18:24,712 - Skipping installation of existing package curl 2021-10-06 08:18:25,093 - Package['hadoop-yarn'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2021-10-06 08:18:25,271 - Skipping installation of existing package hadoop-yarn 2021-10-06 08:18:25,273 - Package['hadoop-hdfs'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2021-10-06 08:18:25,297 - Skipping installation of existing package hadoop-hdfs 2021-10-06 08:18:25,298 - Package['hadoop-mapreduce'] {'retry_on_repo_unavailability': False, 'retry_count': 5} 2021-10-06 08:18:25,321 - Skipping installation of existing package hadoop-mapreduce 1. https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604270702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0IDfdIoKBzV6y8e7Kw%2BLz%2Bz1fIPn0CrP%2BqjYm8qHDiE%3D&reserved=0> 2. https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604270702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=z6eHQfV0ecZ4zXi1%2BlzR%2FwDDGkBA9NZn7x2EmKT69CU%3D&reserved=0> [https://opengraph.githubassets.com/8810b73a2c2c4b7cab46ecfe0397fc64e5b171d1a2679752c656e7c8f03060e5/apache/bigtop]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604280695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mckvbaF49bJdtL6vpCxyYeM9YBdGb3%2BPe%2BGDOi2KBQ4%3D&reserved=0> bigtop/cluster-env.xml at release-1.5.0 · apache/bigtop<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604290689%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AUf2GzVJ8rimYCkSPZjzq3dl0kn6kAzYyP1Fg83Fi2Y%3D&reserved=0> Mirror of Apache Bigtop. Contribute to apache/bigtop development by creating an account on GitHub. github.com<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604290689%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yONUuRhzW72HQQuWKt1GMLQQ1%2FHZI3w0Gmu5JqiOCNs%3D&reserved=0> [https://opengraph.githubassets.com/9302541dd14e06f69f6b955eb7f20a4e495240e95027b33d65cf7da9995ea5e3/apache/ambari]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604300686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RtSM7PuheRmWL5%2FdQO5DXKVjwq0tv2YyX4RevnqbqKw%3D&reserved=0> ambari/stack_select.py at release-2.7.5 · apache/ambari<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604300686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RtSM7PuheRmWL5%2FdQO5DXKVjwq0tv2YyX4RevnqbqKw%3D&reserved=0> Mirror of Apache Ambari. Contribute to apache/ambari development by creating an account on GitHub. github.com<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604310678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iZ2Y%2FCdqbQnwVriXma%2Fd%2FDm4cFP0eJ5nJ515QiiwBDk%3D&reserved=0> 3. Command failed after 1 tries -- Live life Laugh Often
