[
https://issues.apache.org/jira/browse/NUTCH-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on NUTCH-3195 started by Lewis John McGibbney.
---------------------------------------------------
> Make Apache Yetus a trustworthy precommit signal for Nutch
> ----------------------------------------------------------
>
> Key: NUTCH-3195
> URL: https://issues.apache.org/jira/browse/NUTCH-3195
> Project: Nutch
> Issue Type: Task
> Components: ci/cd
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.24
>
>
> NUTCH-3163 put Apache Yetus into GitHub Actions so pull requests get
> automated precommit feedback (spelling, whitespace, shell/Dockerfile lint,
> secret scanning, markdown, and related checks). On pushes to master, Yetus
> also scans the *entire* tree. That full-tree run currently fails hard:
> thousands of findings, many of them historical noise or intentional test
> content, mixed with a smaller set of real problems worth fixing.
> As long as master stays red for the wrong reasons, Yetus cannot be a reliable
> signal. Reviewers learn to ignore it; contributors get drowned in unrelated
> output; we cannot honestly make the check required. This parent tracks the
> work to separate noise from signal, fix what is actually wrong, pay down
> style debt in reviewable slices, and only then treat Yetus as a gate we trust.
> *What went wrong on master?*
> A recent full-tree run (Yetus 0.15.1) voted -1 with large counts in
> codespell, blanks/tabs, markdownlint, @author, shellcheck, detsecrets,
> hadolint, shelldocs, and jshint. Much of that is expected for a long-lived
> codebase the first time a
> whole-tree linter is applied (for example, language-identifier test corpora
> are full of non-English words that codespell treats as typos; auth test
> fixtures
> trip detect-secrets). A meaningful remainder is real: misspellings in product
> code, fragile patterns in bin/crawl and bin/nutch, Dockerfile structure,
> missing shelldocs annotations, and one jshint nit.
> PR patch mode already focuses on changed lines. The initiative is about
> making master full-tree results honest and eventually green—without forcing a
> single unreviewable mega-PR.
> *How we will attack it?*
> Work is split into linked child issues so each PR has one job and can be
> reviewed thoroughly. Early children are allowed to leave master Yetus red;
> getting the diff right matters more than painting the build green too soon.
> Yetus should remain informational until the cleanup is done; only then should
> we consider making it a required PR check.
> * Child A — Teach Yetus what “not a bug” looks like (excludes, codespell
> ignores, reviewed detsecrets hash baseline). No behavior changes.
> * Child B — Fix defects the gate is right about (product typos, shellcheck
> {*}errors{*}, Dockerfile structure, shelldocs, jshint). Behavior-preserving
> where scripts change; manual smoke checklist for bin tools.
> * Child C — Pay down historical style debt (@author removal, blanks/tabs,
> markdownlint) in small PRs; hybrid markdown rule tuning so we do not rewrite
> every README for list-marker fashion.
> * Child D — Finish shellcheck on bin/crawl and bin/nutch (warnings/notes)
> after the error-level fixes in B; can proceed in parallel with C.
> Related earlier work: NUTCH-3163, NUTCH-3192.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
