[
https://issues.apache.org/jira/browse/RANGER-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramachandran Krishnan resolved RANGER-5744.
-------------------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
> Harden plugins-docker-build CI against flaky Ozone datanode Kerberos/KDC
> startup race
> -------------------------------------------------------------------------------------
>
> Key: RANGER-5744
> URL: https://issues.apache.org/jira/browse/RANGER-5744
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Reporter: Ramachandran Krishnan
> Assignee: Ramachandran Krishnan
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Summary
> The {{plugins-docker-build}} GitHub Actions job intermittently fails when
> {{ozone-datanode}} exits during Kerberos login to the Ranger KDC. Re-running
> the failed job usually passes. This is an infrastructure timing issue in the
> Docker Compose CI stack, not a regression in Ranger plugin code.
> Example failure: [GitHub Actions run
> 31185432016|https://github.com/apache/ranger/actions/runs/31185432016/job/93444713551?pr=1142]
> (PR #1142).
> h3. Symptoms
> java.lang.RuntimeException: Can't start the HDDS datanode plugin
> Caused by: org.apache.hadoop.security.KerberosAuthException: failure to
> login: for principal: dn/[email protected]
> Caused by: java.net.PortUnreachableException
>
> {{ozone-datanode}} fails within ~1 second of start; other containers are
> unaffected. A job re-run typically succeeds.
> h3. Root cause
> Three conditions combine to create a race:
> # Weak startup ordering — Ozone datanode/scm use {{{}depends_on: ranger-kdc:
> service_started{}}}, so they start when the KDC container exists, not when
> Kerberos is ready. {{ranger-zk}} already uses {{service_healthy}} for KDC.
> # Weak KDC healthcheck — {{Dockerfile.ranger-kdc}} probes TCP port 88 with
> {{{}nc{}}}, but Java Kerberos clients try UDP first by default.
> # No Ozone startup wait — Unlike Hadoop ({{{}wait_for_keytab.sh{}}}) and OM
> ({{{}ranger-ozone-setup.sh{}}}), datanode/scm launch {{ozone datanode}} /
> {{ozone scm}} immediately with no keytab or KDC readiness wait.
> h3. Impact
> * Intermittent {{plugins-docker-build}} CI failures on unrelated PRs
> * False-negative signal; developers must re-run jobs manually
> * Noise in PR reviews (e.g. RANGER-5736, PR #1142)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
