[
https://issues.apache.org/jira/browse/KNOX-3474?focusedWorklogId=1042453&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-1042453
]
ASF GitHub Bot logged work on KNOX-3474:
----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Sep/26 19:29
Start Date: 17/Sep/26 19:29
Worklog Time Spent: 10m
Work Description: samuhasavak5 opened a new pull request, #1413:
URL: https://github.com/apache/knox/pull/1413
KNOX-3474 - Add spark4historyui Knox service definition
Depends on #1411, #1412
## What changes were proposed in this pull request?
This PR adds upstream Knox support for the dedicated Spark 4 History Server
Web UI (role: SPARK4HISTORYUI).
Problem: Knox has no service definition for Spark 4 History Server.
Deployments running a separate Spark 4 History Server cannot be proxied or
discovered through Knox, YARN UI links to Spark 4 history jobs do not route
correctly, and the Knox Homepage cannot show a dedicated Spark 4 History tile.
Solution: Introduce spark4historyui by mirroring the spark3historyui service
definition with Spark 4 naming, CM discovery, admin UI registration, homepage
logo, YARN rewrite rules, and unit tests.
### Changes
| File | Change |
|------|--------|
| gateway-service-definitions/.../spark4historyui/3.0.0/service.xml | New
service definition (metadata, routes) |
| gateway-service-definitions/.../spark4historyui/3.0.0/rewrite.xml |
Inbound/outbound rewrite rules for REST API and UI |
| gateway-discovery-cm/.../Spark4HistoryUIServiceModelGenerator.java | CM
discovery for SPARK4HISTORYUI |
| gateway-discovery-cm/.../Spark4HistoryUIServiceModelGeneratorTest.java |
Unit tests for model generator |
| gateway-discovery-cm/.../ServiceModelGenerator (SPI) | Register
Spark4HistoryUIServiceModelGenerator |
| ClouderaManagerServiceDiscoveryTest.java | Add
testSpark4HistoryUIDiscovery and testSpark4HistoryUIDiscoverySSL |
| gateway-service-definitions/.../yarnui/2.7.0/rewrite.xml | Route
?spark4=true history links to /spark4history/ |
| gateway-service-definitions/.../yarnuiv2/3.0.0/rewrite.xml | Same YARN
rewrite rule |
| gateway-admin-ui/.../new-desc-wizard.component.ts | Add SPARK4HISTORYUI to
descriptor wizard |
| knox-homepage-ui/home/assets/service-logos/spark4historyui.png | Homepage
logo for service role SPARK4HISTORYUI |
### Service details
| Item | Value |
|------|-------|
| Service role | SPARK4HISTORYUI |
| Service name / path | spark4history |
| CM service type | SPARK4_ON_YARN |
| CM role type | SPARK4_YARN_HISTORY_SERVER |
| Metadata type | UI |
| Homepage context | /spark4history/ |
### Design notes
* Mirrors spark3historyui with Spark 4 identifiers only
* Additive only — no changes to spark3historyui or sparkhistoryui service
definitions
* YARN rewrite — spark4=true query param routes to spark4history (parallel
to CDH spark3=true → spark3history pattern)
* Homepage logo — spark4historyui.png (same Spark branding as Spark 3
History)
### Compatibility
* No changes to existing sparkhistoryui or spark3historyui behavior
* New service is opt-in via topology / CM discovery configuration
* Backward compatible for clusters that do not deploy Spark 4 History Server
JIRA: https://issues.apache.org/jira/browse/KNOX-3474
## How was this patch tested?
### Build
mvn -s mvn-settings-central.xml -U -pl
gateway-service-definitions,gateway-discovery-cm -am test -DskipITs
Result: BUILD SUCCESS
### Unit tests
| Test | Coverage |
|------|----------|
| Spark4HistoryUIServiceModelGeneratorTest | CM model generator metadata and
URL generation |
| ClouderaManagerServiceDiscoveryTest.testSpark4HistoryUIDiscovery | HTTP
discovery for SPARK4HISTORYUI |
| ClouderaManagerServiceDiscoveryTest.testSpark4HistoryUIDiscoverySSL |
HTTPS discovery for SPARK4HISTORYUI |
| Full reactor (gateway-service-definitions, gateway-discovery-cm +
dependencies) | 351 tests passed, 0 failures |
### Manual verification
* Knox Homepage on spark4knoxui cluster shows "Spark 4 History Server Web
UI" tile with logo
* Tile click opens Spark 4 History UI through Knox gateway path
/spark4history/
* YARN UI history links with spark4=true route to /spark4history/history/
## Integration Tests
No new integration test workflow added. Coverage is provided by existing
unit tests in gateway-discovery-cm and gateway-service-definitions.
## UI changes
Yes — Knox Homepage shows a new Spark 4 History Server tile with logo when
SPARK4HISTORYUI is configured in the topology.
## Was this patch authored or co-authored using generative AI tooling?
Yes, this was co-authored using Cursor to help port the service definition
from spark3historyui, discovery generator, tests, YARN rewrite rules, and PR
description.
Issue Time Tracking
-------------------
Worklog Id: (was: 1042453)
Remaining Estimate: 0h
Time Spent: 10m
> Add Knox service definition for Spark 4 History Server UI (spark4historyui)
> ---------------------------------------------------------------------------
>
> Key: KNOX-3474
> URL: https://issues.apache.org/jira/browse/KNOX-3474
> Project: Apache Knox
> Issue Type: Task
> Reporter: Arun Kumar M
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Summary*
> Add upstream Knox support for the Spark 4 History Server Web UI (role:
> SPARK4HISTORYUI).
>
> *Motivation*
> Knox already supports spark3historyui (SPARK3HISTORYUI). Spark 4 deployments
> need an equivalent service definition so the History Server UI is reachable
> through Knox and linked correctly from YARN ResourceManager UI.
>
> *Implementation*
> Base on existing spark3historyui/3.0.0/ pattern:
>
> | Item | Value |
> |------|-------|
> | Service role | SPARK4HISTORYUI |
> | Service name / URL path | spark4history |
> | Metadata type | UI |
> | shortDesc | Spark 4 History Server Web UI |
> | CM discovery | Spark4HistoryUIServiceModelGenerator (extend
> SparkHistoryUIServiceModelGenerator) |
> | CM service type | SPARK4_ON_YARN (align with CM Spark 4 service type) |
> | CM role type | SPARK4_YARN_HISTORY_SERVER (align with CM Spark 4 role type)
> |
>
> *Scope*
> *1. Service definition*
> - gateway-service-definitions/services/spark4historyui/3.0.0/service.xml
> - gateway-service-definitions/services/spark4historyui/3.0.0/rewrite.xml
> - Inbound/outbound rewrite rules for /spark4history/, /history/, jobs pages,
> SSO header filters
>
> *2. Cloudera Manager discovery*
> - Spark4HistoryUIServiceModelGenerator.java + unit tests
> - META-INF/services ServiceModelGenerator registration
> - ClouderaManagerServiceDiscoveryTest (HTTP and SSL port variants)
>
> *3. YARN UI cross-links (required for RM → Spark 4 History navigation)*
> - gateway-service-definitions/services/yarnui/2.7.0/rewrite.xml
> - Add spark4history redirect rule parallel to existing spark3history rule
> - Pattern: *://*:*/history/\{**}?spark4=true?\{**} →
> \{$frontend[url]}/spark4history/history/\{**}?\{**}
> - gateway-service-definitions/services/yarnuiv2/3.0.0/rewrite.xml
> - Same spark4history rule as above
>
> *4. Admin / configuration*
> - gateway-admin-ui: add SPARK4HISTORYUI to service role list
> - Homepage icon for Spark 4 History Server (if icon metadata support is
> included)
>
> *Out of scope*
> - Spark History Server SPNEGO / authentication configuration
> - Spark 4 parcel installation issues
>
> *Acceptance criteria*
> - /spark4history/** routes to Spark 4 History Server backend through Knox
> - CM discovery resolves SPARK4HISTORYUI URLs (HTTP and HTTPS)
> - YARN RM UI links with ?spark4=true route to Knox spark4history path
> - Service tile appears on Knox Homepage
> - Unit tests pass
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
