deepakpanda93 opened a new pull request, #19572: URL: https://github.com/apache/hudi/pull/19572
### Describe the issue this Pull Request addresses Closes #15779. (JIRA: HUDI-5757.) Log compaction shipped in 0.13.0 but has no user-facing prose anywhere in the docs. Today it appears only as: | Where | What | |---|---| | `timeline.md` | a one-line LOGCOMPACTION action definition | | `hudi_stack.md`, `storage_layouts.md`, `metadata.md` | passing mentions | | `configurations.md` | three auto-generated config rows | | `learn/tech-specs.md` | a spec-level paragraph | Nothing tells a user what it is, when to enable it, or how. In particular **`compaction.md` — where someone looking for compaction actually lands — never mentions it**, so there is no path to discover that a minor compaction exists. I checked this rather than assuming: across all eleven versioned copies, `compaction.md` and `write_operations.md` return **zero** hits for `log compaction`, `logcompaction`, `minor compaction` or `hoodie.log.compaction.*`. This is net-new content on both pages, not an extension of something already there. ### Summary and Changelog Added a `## Log Compaction` section to `compaction.md` covering what it does, that it applies to Merge-on-Read tables, the `logcompaction` timeline action, the three configs, and a link to RFC-48. Config names, defaults and `sinceVersion` values were read from `HoodieCompactionConfig` at `release-1.2.0`: | Config | Default | Since | |---|---|---| | `hoodie.log.compaction.inline` | `false` | 0.13.0 | | `hoodie.log.compaction.blocks.threshold` | `5` | 0.13.0 | | `hoodie.log.compaction.enable` | `false` | 0.14.0 | ### A note on placement The issue asks for this on the **write operations** docs, and I have not put the body of it there. `write_operations.md` documents `hoodie.datasource.write.operation` values — UPSERT, INSERT, BULK_INSERT, DELETE, BOOTSTRAP, INSERT_OVERWRITE, INSERT_OVERWRITE_TABLE, DELETE_PARTITION. Log compaction is a **table service**, not one of those, and plain compaction is not listed on that page either. A `### LOG_COMPACTION` heading beside UPSERT would misfile it and break the page's organising principle. So the substance lives in `compaction.md`, and the **Write path** step in `write_operations.md` — which already links to `compaction.md` — gains a pointer: > 9. [Compaction](compaction.md): If you are using MOR tables, compaction will either run inline, or be scheduled > asynchronously. **[Log compaction](compaction.md#log-compaction) may also run, stitching small log blocks together > without rewriting the base file.** That way the page named in the issue does cover it, without putting a table service in the write-operation taxonomy. Happy to move it if the reporter intended something different — the issue body is only a JIRA link and the JIRA has no description, so the title was the whole specification. ### Two things deliberately not claimed **`hoodie.log.compaction.enable`.** The upstream config documentation says only *"By enabling log compaction through this config, log compaction will also get enabled for the metadata table"*, which is ambiguous about the effect on the data table. The table row stays as narrow as that rather than inventing a broader meaning. Happy to widen it if someone can confirm the intended semantics. **No procedure, CLI or utility.** Unlike compaction, log compaction has no SQL procedure, Hudi CLI command, or standalone utility — confirmed against `procedures.md`, `cli.md`, and the `release-1.2.0` file tree. A note records this so nobody goes hunting for a `run_log_compaction`. ### Version scope Applied to `next` and every 1.x versioned copy — 1.2.0, 1.1.1, 1.0.2, 1.0.1, 1.0.0 — since log compaction exists in all of them and the three configs are present in each version's `configurations.md` (verified per version). `compaction.md` differs in every version, so the section was placed against its own surroundings per file rather than by blind substitution. The added text is byte-identical in five of the six; 1.0.0 has no `## Related Resources` heading, so the section is appended at the end there, which differs only in surrounding blank lines. The `write_operations.md` hunk is identical in all six. The 0.14.x and 0.15.x copies are left alone as end-of-life. ### Site verification `npm run build` passes. The previous baseline predated the current `asf-site` head, so I rebuilt a fresh baseline at the same base commit as this branch; the warning set with the change is byte-identical to it — no new warnings and no new broken anchors. The new cross-page link is version-scoped, so it was checked on every affected version under `npm run serve`: | Page | `#log-compaction` anchor | link from `write_operations` | |---|---|---| | `/docs/compaction` | present | `/docs/compaction#log-compaction` | | `/docs/next/compaction` | present | `/docs/next/compaction#log-compaction` | | `/docs/1.1.1/compaction` | present | `/docs/1.1.1/compaction#log-compaction` | | `/docs/1.0.2/compaction` | present | `/docs/1.0.2/compaction#log-compaction` | | `/docs/1.0.1/compaction` | present | `/docs/1.0.1/compaction#log-compaction` | | `/docs/1.0.0/compaction` | present | `/docs/1.0.0/compaction#log-compaction` | | `/docs/0.15.1/compaction` | absent, as intended | no link | Also confirmed the section renders with its TOC entry, a real 3-row table, the note as an admonition, and a working RFC-48 link. ### Impact Documentation only. No code, config, or behaviour change. ### Risk Level none ### Documentation Update This PR is the documentation update — the Compaction page (`/docs/compaction`, `/docs/next/compaction`) and the Write Operations page (`/docs/write_operations`, `/docs/next/write_operations`). ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
