[jira] [Commented] (METRON-2250) Missing services in HDP 3.1 metron mpack and installer stuck
[
https://issues.apache.org/jira/browse/METRON-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091393#comment-17091393
]
Zhao Yi Ming commented on METRON-2250:
--
[~mmiklavcic] I think this is not the ambari issue, because when you do the
*_getServiceComponentLayoutValidations_* __ There is not host selected for the
METRON service, I give a solution for this, use _*colocateService*_ method to
set the host for the service. Submit the pr:
[https://github.com/apache/metron/pull/1578]
*_ !屏幕快照 2020-04-24 下午5.24.31.png!_*
> Missing services in HDP 3.1 metron mpack and installer stuck
>
>
> Key: METRON-2250
> URL: https://issues.apache.org/jira/browse/METRON-2250
> Project: Metron
> Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Mohan Venkateshaiah
>Priority: Major
> Attachments: 屏幕快照 2020-04-24 下午5.24.31.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> We missed this during our full dev deployment because we use blueprints,
> which seems to bypass the service_advisor mechanism in Ambari.
>
> If you uninstall Metron and attempt to reinstall the service (or do any fresh
> install for that matter), the service recommendations will fail due to
> missing hostnames in the services list. The following stack traces appear in
> the Ambari server logs.
>
> {code:java}
> Error occured in stack advisor.
> Error details: list index out of range
> 2019-09-03 20:42:10,911 INFO [ambari-client-thread-34] StackAdvisorRunner:167
> - Advisor script stderr: Traceback (most recent call last):
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 190,
> in
> main(sys.argv)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 122,
> in main
> result = stackAdvisor.validateComponentLayout(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py",
> line 1072, in validateComponentLayout
> validationItems = self.getComponentLayoutValidations(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py",
> line 1095, in getComponentLayoutValidations
> items.extend(serviceAdvisor.getServiceComponentLayoutValidations(services,
> hosts))
> File
> "/var/lib/ambari-server/resources/common-services/METRON/0.7.2/service_advisor.py",
> line 46, in getServiceComponentLayoutValidations
> metronParsersHost = self.getHosts(componentsList, "METRON_PARSERS")[0]
> IndexError: list index out of range
> 2019-09-03 20:42:10,912 WARN [ambari-client-thread-34]
> ValidationResourceProvider:132 - Error occurred during validation
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorException:
> Stack Advisor reported an error. Exit Code: 2. Error: IndexError: list index
> out of range
> StdOut file: /var/run/ambari-server/stack-recommendations/7/stackadvisor.out
> StdErr file: /var/run/ambari-server/stack-recommendations/7/stackadvisor.err
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorRunner.processLogs(StackAdvisorRunner.java:149)
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorRunner.runScript(StackAdvisorRunner.java:89)
> at
> org.apache.ambari.server.api.services.stackadvisor.commands.StackAdvisorCommand.invoke(StackAdvisorCommand.java:314)
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorHelper.validate(StackAdvisorHelper.java:94)
> at
> org.apache.ambari.server.controller.internal.ValidationResourceProvider.createResources(ValidationResourceProvider.java:127)
> at
> org.apache.ambari.server.controller.internal.ClusterControllerImpl.createResources(ClusterControllerImpl.java:296)
> at
> org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.create(PersistenceManagerImpl.java:97)
> at
> org.apache.ambari.server.api.handlers.CreateHandler.persist(CreateHandler.java:50)
> at
> org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:68)
> at
> org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)
> at
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:163)
> at
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:127)
> at
> org.apache.ambari.server.api.services.ValidationService.getValidation(ValidationService.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
[jira] [Updated] (METRON-2250) Missing services in HDP 3.1 metron mpack and installer stuck
[
https://issues.apache.org/jira/browse/METRON-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhao Yi Ming updated METRON-2250:
-
Attachment: 屏幕快照 2020-04-24 下午5.24.31.png
> Missing services in HDP 3.1 metron mpack and installer stuck
>
>
> Key: METRON-2250
> URL: https://issues.apache.org/jira/browse/METRON-2250
> Project: Metron
> Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Mohan Venkateshaiah
>Priority: Major
> Attachments: 屏幕快照 2020-04-24 下午5.24.31.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> We missed this during our full dev deployment because we use blueprints,
> which seems to bypass the service_advisor mechanism in Ambari.
>
> If you uninstall Metron and attempt to reinstall the service (or do any fresh
> install for that matter), the service recommendations will fail due to
> missing hostnames in the services list. The following stack traces appear in
> the Ambari server logs.
>
> {code:java}
> Error occured in stack advisor.
> Error details: list index out of range
> 2019-09-03 20:42:10,911 INFO [ambari-client-thread-34] StackAdvisorRunner:167
> - Advisor script stderr: Traceback (most recent call last):
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 190,
> in
> main(sys.argv)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 122,
> in main
> result = stackAdvisor.validateComponentLayout(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py",
> line 1072, in validateComponentLayout
> validationItems = self.getComponentLayoutValidations(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py",
> line 1095, in getComponentLayoutValidations
> items.extend(serviceAdvisor.getServiceComponentLayoutValidations(services,
> hosts))
> File
> "/var/lib/ambari-server/resources/common-services/METRON/0.7.2/service_advisor.py",
> line 46, in getServiceComponentLayoutValidations
> metronParsersHost = self.getHosts(componentsList, "METRON_PARSERS")[0]
> IndexError: list index out of range
> 2019-09-03 20:42:10,912 WARN [ambari-client-thread-34]
> ValidationResourceProvider:132 - Error occurred during validation
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorException:
> Stack Advisor reported an error. Exit Code: 2. Error: IndexError: list index
> out of range
> StdOut file: /var/run/ambari-server/stack-recommendations/7/stackadvisor.out
> StdErr file: /var/run/ambari-server/stack-recommendations/7/stackadvisor.err
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorRunner.processLogs(StackAdvisorRunner.java:149)
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorRunner.runScript(StackAdvisorRunner.java:89)
> at
> org.apache.ambari.server.api.services.stackadvisor.commands.StackAdvisorCommand.invoke(StackAdvisorCommand.java:314)
> at
> org.apache.ambari.server.api.services.stackadvisor.StackAdvisorHelper.validate(StackAdvisorHelper.java:94)
> at
> org.apache.ambari.server.controller.internal.ValidationResourceProvider.createResources(ValidationResourceProvider.java:127)
> at
> org.apache.ambari.server.controller.internal.ClusterControllerImpl.createResources(ClusterControllerImpl.java:296)
> at
> org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.create(PersistenceManagerImpl.java:97)
> at
> org.apache.ambari.server.api.handlers.CreateHandler.persist(CreateHandler.java:50)
> at
> org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:68)
> at
> org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)
> at
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:163)
> at
> org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:127)
> at
> org.apache.ambari.server.api.services.ValidationService.getValidation(ValidationService.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
>
[GitHub] [metron] zhaoyim opened a new pull request #1578: METRON-2250 Missing services in HDP 3.1 metron mpack and installer stuck
zhaoyim opened a new pull request #1578:
URL: https://github.com/apache/metron/pull/1578
## Contributor Comments
[Please place any comments here. A description of the problem/enhancement,
how to reproduce the issue, your testing methodology, etc.]
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to be
created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON- where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [ ] Have you written or updated unit tests and or integration tests to
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and
running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered by building and verifying the site-book? If not then run the
following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
- [ ] Have you ensured that any documentation diagrams have been updated,
along with their source files, using [draw.io](https://www.draw.io/)? See
[Metron Development
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Development+Guidelines)
for instructions.
Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for
your personal repository such that your branches are built there before
submitting a pull request.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
