swagle commented on a change in pull request #1602: URL: https://github.com/apache/ozone/pull/1602#discussion_r526600832
########## File path: hadoop-hdds/docs/content/concept/Recon.md ########## @@ -0,0 +1,166 @@ +--- +title: "Recon" +date: "2020-10-27" +weight: 8 +menu: + main: + parent: Architecture +summary: Recon serves as a management and monitoring console for Ozone. +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Recon serves as a management and monitoring console for Ozone. It gives a +bird's-eye view of Ozone and helps users troubleshoot any issues by presenting +the current state of the cluster through REST based APIs and rich web UI. + + +## High Level Design + +{{<figure src="/concept/ReconHighLevelDesign.png" width="800px">}} + +<br/> + +On a high level, Recon collects and aggregates metadata from Ozone Manager (OM), +Storage Container Manager (SCM) and Datanodes (DN) and acts as a central +management and monitoring console. Ozone administrators can use Recon to query +the current state of the system without overloading OM or SCM. + +Recon maintains multiple databases to enable batch processing, faster querying +and to persist aggregate information. It maintains a local copy of OM db and +SCM db along with a SQL database for persisting aggregate information. + +Recon also integrates with Prometheus to provide a HTTP endpoint to query Prometheus +for Ozone metrics and also to display a few crucial point in time metrics in +the web UI. + +## Recon and Ozone Manager + +{{<figure src="/concept/ReconOmDesign.png" width="800px">}} + +<br/> + +Recon gets a full snapshot of OM rocks db initially from the leader OM's HTTP +endpoint, untars the file and initializes RocksDB for querying locally. The +database is kept in sync by periodically requesting delta updates from the leader +OM via RPC calls from the last applied sequence id. If for any reason, the delta +updates could not be retrieved or applied to the local db, a full snapshot is +requested again to keep the local db in sync with OM db. Due to this, Recon can +show stale information since the local db will not always be in sync. + +The db updates retrieved from OM is then converted into a batch of events for +further processing by OM db tasks via [Recon Tasks Framework](#tasks-framework). + + +## Recon and Storage Container Manager + +{{<figure src="/concept/ReconScmDesign.png" width="800px">}} + +<br/> + +Recon also acts as a passive SCM for datanodes. When Recon is configured in the +cluster, all the datanodes register with Recon and send heartbeats, container +reports, incremental container reports etc. to Recon similar to SCM. Recon uses +all the information it gets from datanodes to construct its own copy of SCM rocks db +locally. Recon never sends any command to datanodes in response and just acts as +a passive SCM for faster querying of SCM database. + +The data in local SCM db is consumed by all the SCM db tasks triggered at Review comment: Duplication below. Task framework will use all the dbs, from this statement I get the impression that only SCM will be. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
