[ https://issues.apache.org/jira/browse/METRON-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504993#comment-16504993 ]
ASF GitHub Bot commented on METRON-1609: ---------------------------------------- GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/1056 METRON-1609 Elasticsearch settings in Ambari should not be required if Solr is the Indexer NOTE: This PR has been submitted against the Solr feature branch; not master. ## Description When a user deploys Metron with Solr, the Mpack requires that the user populate the Index Settings > Elasticsearch Hosts field to continue, even when deploying a Metron cluster that indexes to Solr and does not use Elasticsearch. * The Elasticsearch related properties under the Ambari 'Index settings' tab should only be required if Elasticsearch is chosen as the indexer. * The Solr related properties under the 'Index Settings' tab should only be required if Solr is chosen as the indexer. ## Changes * Removed the static validation checks for each of the parameters on the 'Index Settings' tab. Static validation no longer works because validation depends on which Indexer the user has chosen. * Added a function `check_indexing_parameters` that validates the parameters based on the Indexer that has been chosen by the user on the "Indexing" tab. * The parameters are checked before any of the following actions: * Start indexing * Stop indexing * Status check indexing * Service checking indexing * Install Elasticsearch Template * Delete Elasticsearch Template * Install Kibana Dashboard * Update global properties * When updating the global properties, we only update either Solr or Elasticsearch, not both as before. * It seems logical to move the 'Indexing Settings' tab parameters to the `Index Settings` tab, especially when it on this page where the user selects the Indexer; Elasticsearch or Solr. I am not sure if there are other implications in doing this, so I did not do that as part of this PR. ## Testing 1. Start the development environment. 1. Validate that the Alerts UI is receiving alerts and that the Ambari 'Service Check' passes. **Missing Elasticsearch parameters should be caught** 1. Stop the Indexing topology in Ambari. 1. Using Ambari, delete (define a blank value for) the `metron-env/es_binary_port` parameter in Metron > Index Settings. 1. Attempt to start the Indexing topology in Ambari. This should fail with the following error message. ``` resource_management.core.exceptions.Fail: Missing required indexing parameters(s): indexer=Elasticsearch, missing=['metron-env/es_binary_port'] ``` 1. Attempt the Elasticsearch Template Install. This should fail with the same error message. 1. Attempt the Elasticsearch Template Delete. This should fail with the same error message. 1. Attempt the Kibana Dashboard Install. This should fail with the same error message. **Missing Elasticsearch parameters should not impact unrelated functions** 1. Restart another unrelated topology like the Profiler or Enrichment. These should restart successfully as they are not impacted by the missing Indexing parameters. **Missing Elasticsearch parameters should not impact Solr** 1. Go to Metron > Indexing and change "Random Access Search Engine" to "Solr". 1. Attempt to start the Indexing topology in Ambari. This should succeed even though we are missing Elasticsearch parameters. **Missing Solr parameters should be caught** 1. Stop the Indexing topology in Ambari. 1. Using Ambari, delete (define a blank value for) the `metron-env/solr_zookeeper_url` parameter in Metron > Index Settings. 1. Attempt to start the Indexing topology in Ambari. This should fail with the following error message. ``` resource_management.core.exceptions.Fail: Missing required indexing parameters(s): indexer=Solr, missing=['metron-env/solr_zookeeper_url'] ``` ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1609 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1056.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1056 ---- commit 11acc9fa0be95ddb6b42cafe8b0a39cadc7d496e Author: Nick Allen <nick@...> Date: 2018-06-07T15:59:27Z METRON-1609 Elasticsearch settings in Ambari should not be required if Solr is the indexer ---- > Elasticsearch settings in Ambari should not be required if Solr is the indexer > ------------------------------------------------------------------------------ > > Key: METRON-1609 > URL: https://issues.apache.org/jira/browse/METRON-1609 > Project: Metron > Issue Type: Improvement > Reporter: Nick Allen > Assignee: Nick Allen > Priority: Major > Attachments: Screen Shot 2018-06-06 at 12.34.13 PM.png, Screen Shot > 2018-06-06 at 12.37.08 PM.png > > > When a user deploys Metron with Solr, the Mpack requires that the user > populate the Index Settings > Elasticsearch Hosts field to continue, even > when deploying a Metron cluster that indexes to Solr. > The other Elasticsearch specific fields on the Index Settings page are also > required, although defaults are provided and thus the user is not required to > enter anything here. If you remove the provided defaults, you can see that > each of the fields are required, even when Elasticsearch will not be used. > * The Elasticsearch related properties under the Ambari 'Index settings' tab > should only be required if Elasticsearch is chosen as the indexer. > * The Solr related properties under the 'Index Settings' tab should only be > required if Solr is chosen as the indexer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)