[
https://issues.apache.org/jira/browse/METRON-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Leet updated METRON-2081:
--------------------------------
Fix Version/s: (was: 0.4.1)
> Is Metron compatible with newer version of HDP?
> ------------------------------------------------
>
> Key: METRON-2081
> URL: https://issues.apache.org/jira/browse/METRON-2081
> Project: Metron
> Issue Type: Improvement
> Affects Versions: 0.4.1
> Reporter: Abhishek Sinha
> Priority: Major
>
> I am unable to deploy HDP 2.5.0 as it gives repo error, can anyone help me
> with the latest HDP version with which it is compatible with? The error that
> I receive is as follows:
>
> 2019-04-16 15:39:58,855 - Could not find stack selector for stack: HDP
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
> line 155, in <module>
> ApplicationTimelineServer().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 285, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
> line 39, in install
> self.install_packages(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 576, in install_packages
> retry_count=agent_stack_retry_count)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
> line 155, in __init__
> self.env.run()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 160, in run
> self.run_action(resource, action)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 124, in run_action
> provider_action()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
> line 54, in action_install
> self.install_package(package_name, self.resource.use_repos,
> self.resource.skip_repos)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py",
> line 51, in install_package
> self.checked_call_with_retries(cmd, sudo=True,
> logoutput=self.get_logoutput())
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
> line 86, in checked_call_with_retries
> return self._call_with_retries(cmd, is_checked=True, **kwargs)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
> line 98, in _call_with_retries
> code, out = func(cmd, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 293, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install hadoop_2_5_0_0_1245-yarn' returned 1.
> Error: Nothing to do
> *stdout:* */var/lib/ambari-agent/data/output-185.txt*
> 2019-04-16 15:38:35,068 - Using hadoop conf dir:
> /usr/hdp/current/hadoop-client/conf
> 2019-04-16 15:38:35,070 - Group['hadoop'] {}
> 2019-04-16 15:38:35,071 - Group['users'] {}
> 2019-04-16 15:38:35,071 - Group['zeppelin'] {}
> 2019-04-16 15:38:35,071 - Group['metron'] {}
> 2019-04-16 15:38:35,072 - Group['kibana'] {}
> 2019-04-16 15:38:35,072 - Group['spark'] {}
> 2019-04-16 15:38:35,072 - Group['livy'] {}
> 2019-04-16 15:38:35,072 - Group['elasticsearch'] {}
> 2019-04-16 15:38:35,072 - User['hive'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,073 - User['zeppelin'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,074 - User['ambari-qa'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> 2019-04-16 15:38:35,074 - User['flume'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,075 - User['metron'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,076 - User['hdfs'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,076 - User['storm'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,077 - User['spark'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,077 - User['kibana'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,078 - User['livy'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,079 - User['mapred'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,079 - User['hbase'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,080 - User['tez'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'users']}
> 2019-04-16 15:38:35,080 - User['zookeeper'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,081 - User['kafka'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,082 - User['yarn'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,082 - User['hcat'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,083 - User['ams'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,083 - User['elasticsearch'] \{'gid': 'hadoop',
> 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
> 2019-04-16 15:38:35,084 - File['/var/lib/ambari-agent/tmp/changeUid.sh']
> \{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> 2019-04-16 15:38:35,086 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh
> ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
> \{'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
> 2019-04-16 15:38:35,105 - Skipping
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa
> /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
> due to not_if
> 2019-04-16 15:38:35,106 - Directory['/tmp/hbase-hbase'] \{'owner': 'hbase',
> 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
> 2019-04-16 15:38:35,107 - File['/var/lib/ambari-agent/tmp/changeUid.sh']
> \{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> 2019-04-16 15:38:35,109 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh
> hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
> \{'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
> 2019-04-16 15:38:35,129 - Skipping
> Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase
> /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due
> to not_if
> 2019-04-16 15:38:35,129 - Group['hdfs'] {}
> 2019-04-16 15:38:35,130 - User['hdfs'] \{'fetch_nonlocal_groups': True,
> 'groups': [u'hadoop', u'hdfs']}
> 2019-04-16 15:38:35,131 - FS Type:
> 2019-04-16 15:38:35,131 - Directory['/etc/hadoop'] \{'mode': 0755}
> 2019-04-16 15:38:35,132 -
> Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] \{'owner':
> 'hdfs', 'group': 'hadoop', 'mode': 01777}
> 2019-04-16 15:38:35,153 - Initializing 6 repositories
> 2019-04-16 15:38:35,155 - Repository['HDP-2.5'] \{'base_url':
> 'http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0',
> 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template':
> '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', 'mirror_list':
> None}
> 2019-04-16 15:38:35,165 - File['/etc/yum.repos.d/HDP.repo'] \{'content':
> '[HDP-2.5]\nname=HDP-2.5\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,165 - Repository['HDP-UTILS-1.1.0.21'] \{'base_url':
> 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7',
> 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template':
> '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS',
> 'mirror_list': None}
> 2019-04-16 15:38:35,169 - File['/etc/yum.repos.d/HDP-UTILS.repo']
> \{'content':
> '[HDP-UTILS-1.1.0.21]\nname=HDP-UTILS-1.1.0.21\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,169 - Repository['ES-Curator-4.x'] \{'base_url':
> 'http://packages.elastic.co/curator/4/centos/7', 'action': ['create'],
> 'components': [u'CURATOR', 'main'], 'repo_template':
> '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'CURATOR',
> 'mirror_list': None}
> 2019-04-16 15:38:35,172 - File['/etc/yum.repos.d/CURATOR.repo'] \{'content':
> '[ES-Curator-4.x]\nname=ES-Curator-4.x\nbaseurl=http://packages.elastic.co/curator/4/centos/7\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,172 - Repository['elasticsearch-2.x'] \{'base_url':
> 'https://packages.elastic.co/elasticsearch/2.x/centos', 'action': ['create'],
> 'components': [u'ELASTICSEARCH', 'main'], 'repo_template':
> '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ELASTICSEARCH',
> 'mirror_list': None}
> 2019-04-16 15:38:35,175 - File['/etc/yum.repos.d/ELASTICSEARCH.repo']
> \{'content':
> '[elasticsearch-2.x]\nname=elasticsearch-2.x\nbaseurl=https://packages.elastic.co/elasticsearch/2.x/centos\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,176 - Repository['kibana-4.x'] \{'base_url':
> 'http://packages.elastic.co/kibana/4.5/centos', 'action': ['create'],
> 'components': [u'KIBANA', 'main'], 'repo_template':
> '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'KIBANA',
> 'mirror_list': None}
> 2019-04-16 15:38:35,179 - File['/etc/yum.repos.d/KIBANA.repo'] \{'content':
> '[kibana-4.x]\nname=kibana-4.x\nbaseurl=http://packages.elastic.co/kibana/4.5/centos\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,179 - Repository['METRON-0.4.1'] \{'base_url':
> 'file:///localrepo', 'action': ['create'], 'components': [u'METRON', 'main'],
> 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list
> %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'METRON',
> 'mirror_list': None}
> 2019-04-16 15:38:35,182 - File['/etc/yum.repos.d/METRON.repo'] \{'content':
> '[METRON-0.4.1]\nname=METRON-0.4.1\nbaseurl=file:///localrepo\n\npath=/\nenabled=1\ngpgcheck=0'}
> 2019-04-16 15:38:35,183 - Package['unzip'] \{'retry_on_repo_unavailability':
> False, 'retry_count': 5}
> 2019-04-16 15:38:35,280 - Skipping installation of existing package unzip
> 2019-04-16 15:38:35,280 - Package['curl'] \{'retry_on_repo_unavailability':
> False, 'retry_count': 5}
> 2019-04-16 15:38:35,290 - Skipping installation of existing package curl
> 2019-04-16 15:38:35,290 - Package['hdp-select']
> \{'retry_on_repo_unavailability': False, 'retry_count': 5}
> 2019-04-16 15:38:35,300 - Skipping installation of existing package hdp-select
> 2019-04-16 15:38:35,538 - checked_call['rpm -q --queryformat
> '%\{version}-%\{release}' hdp-select | sed -e 's/\.el[0-9]//g''] \{'stderr':
> -1}
> 2019-04-16 15:38:35,590 - checked_call returned (0, '2.5.0.0-1245', '')
> 2019-04-16 15:38:35,591 - Package['hadoop_2_5_0_0_1245-yarn']
> \{'retry_on_repo_unavailability': False, 'retry_count': 5}
> 2019-04-16 15:38:35,680 - Installing package hadoop_2_5_0_0_1245-yarn
> ('/usr/bin/yum -d 0 -e 0 -y install hadoop_2_5_0_0_1245-yarn')
> 2019-04-16 15:38:40,403 - Execution of '/usr/bin/yum -d 0 -e 0 -y install
> hadoop_2_5_0_0_1245-yarn' returned 1. Error: Nothing to do
> 2019-04-16 15:38:40,403 - Failed to install package hadoop_2_5_0_0_1245-yarn.
> Executing '/usr/bin/yum clean metadata'
> 2019-04-16 15:38:40,672 - Retrying to install package
> hadoop_2_5_0_0_1245-yarn after 30 seconds
> 2019-04-16 15:39:58,855 - Could not find stack selector for stack: HDP
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
