Gargi-jais11 commented on code in PR #513:
URL: https://github.com/apache/ozone-site/pull/513#discussion_r3671101309
##########
docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/08-insights.mdx:
##########
@@ -0,0 +1,118 @@
+---
+sidebar_label: Insights
+---
+
+# Recon UI — Insights Page
+
+
+
+## 1. Page Overview
+
+The **Insights** page visualizes how data is distributed across the cluster by
+size. It shows two charts: a **File Size Frequency Distribution** (how many
+files fall into each size range) and a **Container Size Distribution** (how
+containers are spread across size ranges). The file chart can be narrowed to
+specific volumes and buckets.
+
+It is a read-only analytics page for understanding data-size patterns — for
+example, whether the cluster is dominated by many small files or a few large
+ones.
+
+> Note: this is the **Insights** page. A separate **OM DB Insights** page
+> covers open keys, keys pending deletion, and related OM database details.
+
+## 2. When to Use This Page
+
+- To understand the file-size profile of the cluster or of a specific bucket.
+- To detect a "small files problem" (a very high count of tiny files).
+- To see how containers are distributed by size.
+- To compare data-size patterns between volumes or buckets.
+
+## 3. How to Access the Page
+
+Open the **Insights** entry in the left navigation menu, or go to the
+`/Insights` route directly.
+
+## 4. Information Displayed
+
+The page header shows the title. While data loads, a **"Charts are being
+loaded"** message is shown. Once loaded, two cards appear side by side.
+
+### File Size Frequency Distribution (bar chart)
+
+- A bar chart where the **x-axis** is the file-size range (for example
+ `512 KiB - 1 MiB`) and the **y-axis** is the **Number of Files** in that
+ range.
+- Ranges are power-of-two size buckets; hovering a bar shows the exact size
+ range and count.
+- Two filters above the chart:
+ - **Volumes** — multi-select (with search and select-all) to include specific
+ volumes.
+ - **Buckets** — multi-select, enabled only when exactly one volume is
+ selected; it lists that volume's buckets.
+
+### Container Size Distribution (pie chart)
+
+- A pie chart of containers grouped into size ranges; each slice is a range and
+ its value is the number of containers in that range.
+- Hovering a slice shows the size range and container count.
+- This chart has no volume/bucket filter (containers are not tied to a single
+ bucket).
+
+## 5. Available Actions
+
+- **Volumes filter** (file chart) — choose which volumes to include; supports
+ search and select-all.
+- **Buckets filter** (file chart) — choose buckets within a single selected
+ volume. It is disabled when more than one volume is selected (unless the
+ cluster has only one volume).
+- **Chart tooltips** — hover any bar or slice for the exact size range and
count.
+
+There is no auto-refresh panel, search box, table, or download on this page.
+
+## 6. How to Interpret the Information
+
+- **Tall bars at small size ranges:** many small files. A very large count of
+ tiny files can hurt performance and metadata scaling (the "small files
+ problem").
+- **Bars concentrated at large size ranges:** the cluster is dominated by large
+ files.
+- **Container pie skewed to small ranges:** many small or partially filled
+ containers; skewed to large ranges means containers are well filled.
+- **Filtering the file chart:** select a single volume, then its buckets, to
see
+ the size profile for just that bucket — useful for comparing workloads.
+- **"No Data" result:** the corresponding metric has no data yet (for example a
+ new cluster with no files). This is expected, not an error.
+- **Error banner inside a chart** ("No data available. …"): the underlying data
+ could not be fetched.
+
+## 7. Common Use Cases
+
+1. **Diagnose a small-files problem.** Open the file chart for the whole
cluster
+ and check whether the smallest size ranges dominate the file count.
+2. **Profile one bucket.** Select a single volume and one of its buckets to see
+ that bucket's file-size distribution in isolation.
+3. **Assess container fill.** Use the container pie chart to judge whether
+ containers are mostly small (poorly filled) or large (well filled).
+
+## 8. Important Notes and Limitations
+
+- **Data source and freshness.** Both charts are built from counts Recon
+ maintains in its own database (file-count-by-size and
container-count-by-size),
+ populated by Recon background tasks that process its copy of the Ozone
Manager
+ (OM) data and container information. The charts reflect the last time those
+ tasks ran, not real time.
+- **Bucket filter needs a single volume.** The Buckets filter is only active
when
+ exactly one volume is selected; with multiple volumes it is disabled.
+- **Size ranges are power-of-two buckets**, so exact per-file sizes are not
+ shown — only how many files fall into each range.
+- The page has **no manual refresh control**; reload the page to re-fetch.
+- On no data, each chart shows a "No Data" message independently.
+
+## 9. Related Pages
+
+- **OM DB Insights** — open keys, keys pending deletion, and OM database
+ details.
+- **Namespace Usage** — size usage explored by path rather than by size range.
+- **Containers** — container health and per-state detail.
+- **Overview** — cluster-wide totals.
Review Comment:
It's better to link pages as well so that it is easier to navigate.
##########
docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/09-om-db-insights.mdx:
##########
@@ -0,0 +1,166 @@
+---
+sidebar_label: OM DB Insights
+---
+
+# Recon UI — OM DB Insights Page
+
+
+
+## 1. Page Overview
+
+The **OM DB Insights** page exposes detail from the Ozone Manager (OM) database
+and how it lines up with SCM's container view. It is organized into five tabs:
+container mismatches between OM and SCM, open (uncommitted) keys, keys pending
+deletion, keys mapped to deleted containers, and directories pending deletion.
+
+It is the drill-down destination for the Open Keys and Delete Pending Keys
+summaries on the Overview page, and the main place to investigate deletion
+backlogs and OM/SCM inconsistencies.
+
+> Note: this is the **OM DB Insights** page. The separately named **Insights**
+> page covers file-size and container-size charts.
+
+## 2. When to Use This Page
+
+- To investigate a growing backlog of keys or directories pending deletion.
+- To find open keys that were never committed (possible leaked/abandoned
writes).
+- To find containers that exist in OM but not SCM (or vice versa).
+- To see which keys map to containers SCM has marked deleted.
+- To list the keys inside a specific container.
+
+## 3. How to Access the Page
+
+Open the **OM DB Insights** entry in the left navigation menu, or go to the
+`/Om` route directly. From the **Overview** page, the **Open Keys Summary**
card
+opens this page on the Open Keys tab, and the **Delete Pending Keys Summary**
+card opens it on the Keys Pending for Deletion tab.
+
+## 4. Information Displayed
+
+The page header shows the title. Below it are five tabs. Each tab has a
**Limit**
+selector and a table; some tables allow row expansion to list the affected
keys.
+
+### Tab 1 — Container Mismatch Info
+
+Containers present in OM but missing in SCM, or the reverse. Columns:
+
+- **Container ID**
+- **Count Of Keys** — number of keys in the container.
+- **Pipelines** — the pipelines involved.
+- **Exists at** — where the container exists (OM or SCM).
+
+A toggle switches between containers **missing in OM** and **missing in SCM**.
+Rows expand to show the keys in the container.
+
+### Tab 2 — Open Keys
+
+Keys that are being written but not yet committed. Columns:
+
+- **Key Name**
+- **Size**
+- **Path**
+- **In state since** — how long the key has been open.
+- **Replication Type** and **Replication Factor**
+- **Type** — FSO or Non-FSO.
+
+A toggle switches between **FSO** and **Non-FSO** keys.
+
+### Tab 3 — Keys Pending for Deletion
+
+Keys awaiting deletion. Columns:
+
+- **Key Name**
+- **Path**
+- **Total Data Size**
+- **Total Key Count**
+
+Rows expand to show the individual pending-delete key entries.
+
+### Tab 4 — Deleted Container Keys
+
+Keys mapped to containers that SCM has marked as **DELETED**. Columns:
+
+- **Container ID**
+- **Count Of Keys**
+- **Pipelines**
+
+Rows expand to show the keys in the container.
+
+### Tab 5 — Directories Pending for Deletion
+
+Directories awaiting deletion. Columns:
+
+- **Directory Name**
+- **In state since**
+- **Path**
+- **Size**
+
+### Expanded key table (shared)
+
+When a container row is expanded, the keys are listed with **Volume**,
+**Bucket**, **Key**, **Size**, **Date Created**, and **Date Modified**.
+
+## 5. Available Actions
+
+- **Tabs** — switch between the five views.
+- **Limit** selector — how many rows to fetch (1000, 5000, 10000, 20000;
default
+ 1000). The selection is shared across tabs.
+- **Toggles:**
+ - Container Mismatch: **missing in OM** vs **missing in SCM**.
+ - Open Keys: **FSO** vs **Non-FSO**.
+- **Search** — filter the loaded rows within a tab.
+- **Row expand** — on the container-based tabs, expand a row to load and view
+ its keys.
+- **Pagination** — page through rows; the footer shows the range and total.
+
+There is no auto-refresh panel on this page.
+
+## 6. How to Interpret the Information
+
+- **Container Mismatch entries:** a healthy cluster generally has none. Entries
+ can appear transiently while OM and SCM converge; persistent mismatches
warrant
+ investigation. The **Exists at** column tells you which service still has the
+ container.
+- **Open Keys that are old (large "In state since"):** may be abandoned or
+ leaked writes holding space; a large open-keys total also shows on the
Overview
+ Open Keys summary.
+- **Keys / Directories Pending for Deletion growing over time:** the background
+ deletion services may be falling behind; a steadily rising backlog is the
+ signal to investigate.
+- **Deleted Container Keys:** keys still referencing containers SCM considers
+ deleted — useful when tracing why space is not being reclaimed.
+- **FSO vs Non-FSO:** file-system-optimized buckets vs object-store/legacy
+ buckets; the toggle lets you look at each layout's open keys separately.
+
+## 7. Common Use Cases
+
+1. **Diagnose a deletion backlog.** From the Overview Delete Pending Keys card,
+ land on the Keys Pending for Deletion tab, raise the Limit, and review total
+ size and counts to gauge the backlog.
+2. **Find leaked open keys.** On the Open Keys tab, sort/scan by "In state
since"
+ to find keys that have been open a long time and were likely never
committed.
+3. **Investigate OM/SCM drift.** On the Container Mismatch tab, toggle between
+ missing-in-OM and missing-in-SCM and expand rows to see exactly which keys
are
+ affected.
+
+## 8. Important Notes and Limitations
+
+- **Data source and freshness.** All tabs read from Recon's own copy of the OM
+ database (and its comparison against SCM's container view for the mismatch
+ tabs), updated by periodic sync. Values reflect the last sync, not real time,
+ and there is no auto-refresh control on this page — reload to re-fetch.
+- **Limit caps the result set.** Only up to the selected limit of rows is
+ fetched per tab; raise the limit on large clusters, keeping in mind larger
+ limits take longer to load.
+- **Search applies to the loaded rows only.**
+- **Mismatches can be transient** during normal OM/SCM synchronization and do
not
+ always indicate a problem.
+- On an error, the page shows a data-fetch error and the affected table stays
+ empty.
+
+## 9. Related Pages
+
+- **Overview** — the Open Keys and Delete Pending Keys summary cards link here.
+- **Containers** — container health and unhealthy-state detail.
+- **Insights** — file-size and container-size distribution charts.
+- **Namespace Usage** — size usage explored by path.
Review Comment:
It's better to link pages as well so that it is easier to navigate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
