Arun Kumar M created KNOX-3479:
----------------------------------
Summary: Add per-instance HA routing for Livy for Spark 3
(livy_for_spark3)
Key: KNOX-3479
URL: https://issues.apache.org/jira/browse/KNOX-3479
Project: Apache Knox
Issue Type: Task
Reporter: Arun Kumar M
Summary
Add per-instance high-availability routing for the dedicated Livy for Spark 3
Knox service (role: LIVY_FOR_SPARK3), as a follow-up to KNOX-3472 /
apache/knox#1411.
Motivation
KNOX-3472 introduced the livy_for_spark3 service definition, CM discovery, and
homepage logo with a single default HA path (LivyDispatch only). Deployments
with multiple Livy for Spark 3 instances need:
- Separate Knox Homepage tiles per backend URL
- Routing to the instance selected via ?host= on the tile link
- Sticky affinity for subsequent Livy UI requests (Referer / cookie)
- Fallback to standard HA when no per-instance hint is present
Relationship to KNOX-3472
KNOX-3472 explicitly excluded (see that ticket "Out of scope"):
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
This ticket implements those items.
Scope
gateway-provider-ha
- LivyHaDispatch.java — resolve backend: ?host= → Referer (host=) →
KNOX_LIVY_BACKEND cookie → default HA active URL
- LivyHaDispatchTest.java — unit tests (example.com hostnames only)
- pom.xml — test-scoped mockito-core for dispatch tests
gateway-service-definitions
- livy_for_spark3/0.6.0/service.xml
- metadata context: /livy_for_spark3/ui?host=\{{BACKEND_HOST}}
- iconFileName: livy_for_spark3.png (asset from KNOX-3472)
- dispatch ha-classname: org.apache.knox.gateway.ha.dispatch.LivyHaDispatch
knox-homepage-ui
- home/index.html — append short hostname to tile labels when multiple
instances share the same shortDesc
Out of scope
- Changes to generic livy (LIVYSERVER) service
- Cloudera Manager discovery changes (covered by KNOX-3472)
- livy_for_spark4 / spark4historyui (separate tickets)
Acceptance criteria
- Multiple LIVY_FOR_SPARK3 URLs in topology produce distinguishable homepage
tiles (hostname in label)
- Tile click routes to correct backend via ?host=
- Livy UI AJAX/session traffic stays on selected backend when cookie/Referer
path applies
- LivyHaDispatchTest passes; gateway-provider-ha module tests pass
- No vendor-specific hostnames or internal ticket references in ASF-licensed
sources
Implementation / PR
GitHub: https://github.com/apache/knox/pull/1420
Branch: samuhasavak5:KNOX-livy-for-spark3-per-instance-ha
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
