[
https://issues.apache.org/jira/browse/KNOX-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Kumar M updated KNOX-3479:
-------------------------------
Description:
*Summary*
Add per-instance high-availability routing for the dedicated Livy for Spark 3
Knox service (role: LIVY_FOR_SPARK3), as a follow-up to KNOX-3472 /
apache/knox#1411.
*Motivation*
KNOX-3472 introduced the livy_for_spark3 service definition, CM discovery, and
homepage logo with a single default HA path (LivyDispatch only). Deployments
with multiple Livy for Spark 3 instances need:
- Separate Knox Homepage tiles per backend URL
- Routing to the instance selected via ?host= on the tile link
- Sticky affinity for subsequent Livy UI requests (Referer / cookie)
- Fallback to standard HA when no per-instance hint is present
Relationship to KNOX-3472
KNOX-3472 explicitly excluded (see that ticket "Out of scope"):
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
This ticket implements those items.
*Scope*
gateway-provider-ha
- LivyHaDispatch.java — resolve backend: ?host= → Referer (host=) →
KNOX_LIVY_BACKEND cookie → default HA active URL
- LivyHaDispatchTest.java — unit tests (example.com hostnames only)
- pom.xml — test-scoped mockito-core for dispatch tests
gateway-service-definitions
- livy_for_spark3/0.6.0/service.xml
- metadata context: /livy_for_spark3/ui?host={{{}BACKEND_HOST{}}}
- iconFileName: livy_for_spark3.png (asset from KNOX-3472)
- dispatch ha-classname: org.apache.knox.gateway.ha.dispatch.LivyHaDispatch
knox-homepage-ui
- home/index.html — append short hostname to tile labels when multiple
instances share the same shortDesc
Out of scope
- Changes to generic livy (LIVYSERVER) service
- Cloudera Manager discovery changes (covered by KNOX-3472)
- livy_for_spark4 / spark4historyui (separate tickets)
Acceptance criteria
- Multiple LIVY_FOR_SPARK3 URLs in topology produce distinguishable homepage
tiles (hostname in label)
- Tile click routes to correct backend via ?host=
- Livy UI AJAX/session traffic stays on selected backend when cookie/Referer
path applies
- LivyHaDispatchTest passes; gateway-provider-ha module tests pass
- No vendor-specific hostnames or internal ticket references in ASF-licensed
sources
Implementation / PR
GitHub: [https://github.com/apache/knox/pull/1420]
Branch: samuhasavak5:KNOX-livy-for-spark3-per-instance-ha
was:
Summary
Add per-instance high-availability routing for the dedicated Livy for Spark 3
Knox service (role: LIVY_FOR_SPARK3), as a follow-up to KNOX-3472 /
apache/knox#1411.
Motivation
KNOX-3472 introduced the livy_for_spark3 service definition, CM discovery, and
homepage logo with a single default HA path (LivyDispatch only). Deployments
with multiple Livy for Spark 3 instances need:
- Separate Knox Homepage tiles per backend URL
- Routing to the instance selected via ?host= on the tile link
- Sticky affinity for subsequent Livy UI requests (Referer / cookie)
- Fallback to standard HA when no per-instance hint is present
Relationship to KNOX-3472
KNOX-3472 explicitly excluded (see that ticket "Out of scope"):
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
This ticket implements those items.
Scope
gateway-provider-ha
- LivyHaDispatch.java — resolve backend: ?host= → Referer (host=) →
KNOX_LIVY_BACKEND cookie → default HA active URL
- LivyHaDispatchTest.java — unit tests (example.com hostnames only)
- pom.xml — test-scoped mockito-core for dispatch tests
gateway-service-definitions
- livy_for_spark3/0.6.0/service.xml
- metadata context: /livy_for_spark3/ui?host=\{{BACKEND_HOST}}
- iconFileName: livy_for_spark3.png (asset from KNOX-3472)
- dispatch ha-classname: org.apache.knox.gateway.ha.dispatch.LivyHaDispatch
knox-homepage-ui
- home/index.html — append short hostname to tile labels when multiple
instances share the same shortDesc
Out of scope
- Changes to generic livy (LIVYSERVER) service
- Cloudera Manager discovery changes (covered by KNOX-3472)
- livy_for_spark4 / spark4historyui (separate tickets)
Acceptance criteria
- Multiple LIVY_FOR_SPARK3 URLs in topology produce distinguishable homepage
tiles (hostname in label)
- Tile click routes to correct backend via ?host=
- Livy UI AJAX/session traffic stays on selected backend when cookie/Referer
path applies
- LivyHaDispatchTest passes; gateway-provider-ha module tests pass
- No vendor-specific hostnames or internal ticket references in ASF-licensed
sources
Implementation / PR
GitHub: https://github.com/apache/knox/pull/1420
Branch: samuhasavak5:KNOX-livy-for-spark3-per-instance-ha
> Add per-instance HA routing for Livy for Spark 3 (livy_for_spark3)
> ------------------------------------------------------------------
>
> Key: KNOX-3479
> URL: https://issues.apache.org/jira/browse/KNOX-3479
> Project: Apache Knox
> Issue Type: Task
> Reporter: Arun Kumar M
> Priority: Major
> Labels: livy_for_spark3, service-definition
>
> *Summary*
> Add per-instance high-availability routing for the dedicated Livy for Spark 3
> Knox service (role: LIVY_FOR_SPARK3), as a follow-up to KNOX-3472 /
> apache/knox#1411.
> *Motivation*
> KNOX-3472 introduced the livy_for_spark3 service definition, CM discovery,
> and homepage logo with a single default HA path (LivyDispatch only).
> Deployments with multiple Livy for Spark 3 instances need:
> - Separate Knox Homepage tiles per backend URL
> - Routing to the instance selected via ?host= on the tile link
> - Sticky affinity for subsequent Livy UI requests (Referer / cookie)
> - Fallback to standard HA when no per-instance hint is present
> Relationship to KNOX-3472
> KNOX-3472 explicitly excluded (see that ticket "Out of scope"):
> - Per-instance HA routing (custom ha-classname, ?host= query param, sticky
> cookies)
> - Knox Homepage JavaScript tile-label patching
> This ticket implements those items.
> *Scope*
> gateway-provider-ha
> - LivyHaDispatch.java — resolve backend: ?host= → Referer (host=) →
> KNOX_LIVY_BACKEND cookie → default HA active URL
> - LivyHaDispatchTest.java — unit tests (example.com hostnames only)
> - pom.xml — test-scoped mockito-core for dispatch tests
> gateway-service-definitions
> - livy_for_spark3/0.6.0/service.xml
> - metadata context: /livy_for_spark3/ui?host={{{}BACKEND_HOST{}}}
> - iconFileName: livy_for_spark3.png (asset from KNOX-3472)
> - dispatch ha-classname: org.apache.knox.gateway.ha.dispatch.LivyHaDispatch
> knox-homepage-ui
> - home/index.html — append short hostname to tile labels when multiple
> instances share the same shortDesc
> Out of scope
> - Changes to generic livy (LIVYSERVER) service
> - Cloudera Manager discovery changes (covered by KNOX-3472)
> - livy_for_spark4 / spark4historyui (separate tickets)
> Acceptance criteria
> - Multiple LIVY_FOR_SPARK3 URLs in topology produce distinguishable homepage
> tiles (hostname in label)
> - Tile click routes to correct backend via ?host=
> - Livy UI AJAX/session traffic stays on selected backend when cookie/Referer
> path applies
> - LivyHaDispatchTest passes; gateway-provider-ha module tests pass
> - No vendor-specific hostnames or internal ticket references in ASF-licensed
> sources
> Implementation / PR
> GitHub: [https://github.com/apache/knox/pull/1420]
> Branch: samuhasavak5:KNOX-livy-for-spark3-per-instance-ha
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
