samuhasavak5 opened a new pull request, #1412: URL: https://github.com/apache/knox/pull/1412
KNOX-3473 - Add livy_for_spark4 Knox service definition Depends on #1411 ## What changes were proposed in this pull request? This PR adds upstream Knox support for the dedicated Livy for Spark 4 service (role: LIVY_FOR_SPARK4). Problem: Knox has no service definition for Livy for Spark 4. Deployments running a separate Livy for Spark 4 instance cannot be proxied or discovered through Knox, and the Knox Homepage cannot show a dedicated Livy for Spark 4 tile with the correct logo. Solution: Introduce livy_for_spark4 by mirroring the livy_for_spark3 service definition (KNOX-3472 / #1411) with Spark 4 naming, CM discovery, admin UI registration, homepage logo, and unit tests. Routing reuses org.apache.knox.gateway.livy.LivyDispatch. ### Changes File | Change -- | -- gateway-service-definitions/.../livy_for_spark4/0.6.0/service.xml | New service definition (metadata, routes, LivyDispatch) gateway-service-definitions/.../livy_for_spark4/0.6.0/rewrite.xml | Inbound/outbound rewrite rules for REST API, UI, YARN proxy, and container logs gateway-discovery-cm/.../LivyForSpark4ServiceModelGenerator.java | CM discovery for LIVY_FOR_SPARK4 gateway-discovery-cm/.../LivyForSpark4ServiceModelGeneratorTest.java | Unit tests for model generator gateway-discovery-cm/.../ServiceModelGenerator (SPI) | Register LivyForSpark4ServiceModelGenerator ClouderaManagerServiceDiscoveryTest.java | Add testLivyForSpark4Discovery and testLivyForSpark4DiscoverySSL gateway-admin-ui/.../new-desc-wizard.component.ts | Add LIVY_FOR_SPARK4 to descriptor wizard gateway-release/home/conf/gateway-site.xml | Add LIVY_FOR_SPARK4 to gateway.xforwarded.header.context.append.servicename knox-homepage-ui/home/assets/service-logos/livy_for_spark4.png | Homepage logo for service role LIVY_FOR_SPARK4 ### Service details Item | Value -- | -- Service role | LIVY_FOR_SPARK4 Service name / path | livy_for_spark4 CM service type | LIVY_FOR_SPARK4 CM role type | LIVY_SERVER_FOR_SPARK4 Metadata type | API_AND_UI Homepage context | /livy_for_spark4/ Dispatch | org.apache.knox.gateway.livy.LivyDispatch ### Design notes - Mirrors livy_for_spark3 (#1411) with Spark 4 identifiers only - Additive only — no changes to livy or livy_for_spark3 service definitions - Reuses existing Livy dispatch — no new dispatch class - Homepage logo — livy_for_spark4.png (same Livy branding as Spark 3) - Out of scope — per-instance HA routing (not included) ### Compatibility - No changes to existing livy or livy_for_spark3 behavior - New service is opt-in via topology / CM discovery configuration - Backward compatible for clusters that do not deploy Livy for Spark 4 JIRA: https://issues.apache.org/jira/browse/KNOX-3473 ## How was this patch tested? ### Build mvn -s mvn-settings-central.xml -U -pl gateway-service-definitions,gateway-discovery-cm -am test -DskipITs Result: BUILD SUCCESS ### Unit tests mvn -s mvn-settings-central.xml -U -pl gateway-service-definitions,gateway-discovery-cm -am test -DskipITs Result: 345/345 tests passed, 0 failures Test | Coverage -- | -- LivyForSpark4ServiceModelGeneratorTest | CM model generator metadata and URL generation ClouderaManagerServiceDiscoveryTest.testLivyForSpark4Discovery | HTTP discovery for LIVY_FOR_SPARK4 ClouderaManagerServiceDiscoveryTest.testLivyForSpark4DiscoverySSL | HTTPS discovery for LIVY_FOR_SPARK4 Full reactor (gateway-service-definitions, gateway-discovery-cm + dependencies) | No regressions in existing modules ### Manual verification - Knox Homepage on spark4knoxui cluster shows "Livy for Spark 4" tile with Livy logo - Tile click opens Livy UI through Knox gateway path /livy_for_spark4/ ## Integration Tests No new integration test workflow added. Coverage is provided by existing unit tests in gateway-discovery-cm and gateway-service-definitions. ## UI changes Yes — Knox Homepage shows a new Livy for Spark 4 service tile with logo when LIVY_FOR_SPARK4 is configured in the topology. Screenshot: Knox Homepage on spark4knoxui cluster showing Livy for Spark 4 tile with logo (attach screenshot to PR). ## Was this patch authored or co-authored using generative AI tooling? Yes, this was co-authored using Cursor to help port the service definition from livy_for_spark3, discovery generator, tests, and PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
