[
https://issues.apache.org/jira/browse/HADOOP-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jose Luis López updated HADOOP-19972:
-------------------------------------
Description:
Hadoop's web server is Jetty 9.4, which is end of life and gets no security
fixes. Jetty 10 and later require the {{jakarta.servlet}} namespace, which
would break every project that embeds Hadoop's web stack. Jetty 12's ee8
environment still serves {{{}javax.servlet{}}}, so this moves to a supported
Jetty without that break.
Jetty 9.4.58 to 12.1.12 on ee8. No public signature changes, and downstream
projects need not rebuild. {*}Requires Java 17{*}, so it cannot go to
branch-3.3 or branch-3.4. The YARN application catalog webapp stays on Jetty
9.4.58, which Solr 8 needs.
h2. ee8 is a staging post
ee8 is a stage, not the destination. The point of landing it is that the port
to Jetty 12's API happens here and is not repeated: afterwards Hadoop is on a
supported Jetty, and what remains is the namespace change on its own — ee10 or
ee11 and {{{}jakarta.servlet{}}}, under HADOOP-19395 — instead of a Jetty
upgrade and an API break having to be taken together. ee8 does carry a
compatibility layer that adapts every request, so it is a place to stand for a
release or two rather than indefinitely; unlike Jetty 10 and 11 it has a
supported vendor behind it, which is what makes standing there reasonable.
h2. Incompatible changes
This is the release note. The reason phrase alone makes this incompatible.
# *No custom HTTP reason phrase is sent.* Refusals from the authentication,
KMS and CSRF filters and from image transfer carry their message in the
response body instead, PUT and DELETE included, and Hadoop's clients read it
there. An older client sees the standard phrase: "Unauthorized" rather than
"Authentication required".
# *Stricter URI parsing.* {{{}%2F{}}}, {{%2E}} segments and malformed
encodings are refused with 400. {{hadoop.http.uri.compliance.violations}}
controls what is allowed through.
# *TLS renegotiation is refused.* A client's TLS 1.2 renegotiation closes the
connection. {{hadoop.http.ssl.renegotiation.allowed=true}} restores 9.4's
behaviour.
# *The five async HTTP metrics read 0.* The names and the metric set are
unchanged.
# *Jetty's own output differs:* the error-page markup, charset and {{Vary}} on
static files, a {{Date}} header on error responses, the default acceptor count,
and a context path without its trailing slash redirecting 301 with a relative
{{Location}} where 9.4 sent 302 with an absolute one.
Everything else keeps 9.4's behaviour: {{/static}} lists no directory,
{{/logs}} keeps its admin check when the log directory is missing, {{web.xml}}
error pages work, the HTTP server metrics stay in milliseconds under their
existing names, a stopped listener can be reopened, the NodeManager's web
socket upgrade works, and a redirect's {{Location}} stays absolute.
h2. New settings
||Setting||Default||
|{{hadoop.http.uri.compliance.violations}}|{{AMBIGUOUS_EMPTY_SEGMENT,AMBIGUOUS_PATH_ENCODING,SUSPICIOUS_PATH_CHARACTERS}}|
|{{hadoop.http.ssl.renegotiation.allowed}}|{{false}}|
Adding {{AMBIGUOUS_PATH_SEPARATOR}} to the first accepts {{%2F}} again.
h2. Split out
To keep this issue to the Jetty port itself, the parts that also work on Jetty
9.4 land first, each on its own:
* HADOOP-20003: clients read a refusal's reason from the body
* HADOOP-20004: filters send the reason in the body for every HTTP method
* HADOOP-20005: refused fsimage transfers carry their reason in the body
* HADOOP-20006: {{JettyUtils#clearContentType}}
* HADOOP-20007: test for the downstream javax.servlet contract
* HADOOP-20008: the app catalog's own Jetty version
This issue is blocked by all six. The incompatible changes listed above still
apply; after the split, the only cause of the reason-phrase change is Jetty 12
itself. The app catalog stays on Jetty 9.4 through {{solr.jetty.version}} until
HADOOP-19998.
h2. Not in scope
No move to {{{}jakarta.servlet{}}}, no Jersey upgrade, and no ee9, ee10 or
ee11. HADOOP-19395 lands the namespace change, in a major release;
HADOOP-19912, the umbrella for this move to Jetty 12 ee8, keeps
{{javax.servlet}}. This only stops that decision from blocking the security fix.
was:
Hadoop's web server is Jetty 9.4, which is end of life and gets no security
fixes. Jetty 10 and later require the {{jakarta.servlet}} namespace, which
would break every project that embeds Hadoop's web stack. Jetty 12's ee8
environment still serves {{{}javax.servlet{}}}, so this moves to a supported
Jetty without that break.
Jetty 9.4.58 to 12.1.12 on ee8. No public signature changes, and downstream
projects need not rebuild. {*}Requires Java 17{*}, so it cannot go to
branch-3.3 or branch-3.4. The YARN application catalog webapp stays on Jetty
9.4.58, which Solr 8 needs.
h2. ee8 is a staging post
ee8 is a stage, not the destination. The point of landing it is that the port
to Jetty 12's API happens here and is not repeated: afterwards Hadoop is on a
supported Jetty, and what remains is the namespace change on its own — ee10 or
ee11 and {{{}jakarta.servlet{}}}, under HADOOP-19395 — instead of a Jetty
upgrade and an API break having to be taken together. ee8 does carry a
compatibility layer that adapts every request, so it is a place to stand for a
release or two rather than indefinitely; unlike Jetty 10 and 11 it has a
supported vendor behind it, which is what makes standing there reasonable.
h2. Incompatible changes
This is the release note. The reason phrase alone makes this incompatible.
# *No custom HTTP reason phrase is sent.* Refusals from the authentication,
KMS and CSRF filters and from image transfer carry their message in the
response body instead, PUT and DELETE included, and Hadoop's clients read it
there. An older client sees the standard phrase: "Unauthorized" rather than
"Authentication required".
# *Stricter URI parsing.* {{{}%2F{}}}, {{%2E}} segments and malformed
encodings are refused with 400. {{hadoop.http.uri.compliance.violations}}
controls what is allowed through.
# *TLS renegotiation is refused.* A client's TLS 1.2 renegotiation closes the
connection. {{hadoop.http.ssl.renegotiation.allowed=true}} restores 9.4's
behaviour.
# *The five async HTTP metrics read 0.* The names and the metric set are
unchanged.
# *Jetty's own output differs:* the error-page markup, charset and {{Vary}} on
static files, a {{Date}} header on error responses, the default acceptor count,
and a context path without its trailing slash redirecting 301 with a relative
{{Location}} where 9.4 sent 302 with an absolute one.
Everything else keeps 9.4's behaviour: {{/static}} lists no directory,
{{/logs}} keeps its admin check when the log directory is missing, {{web.xml}}
error pages work, the HTTP server metrics stay in milliseconds under their
existing names, a stopped listener can be reopened, the NodeManager's web
socket upgrade works, and a redirect's {{Location}} stays absolute.
h2. New settings
||Setting||Default||
|{{hadoop.http.uri.compliance.violations}}|{{AMBIGUOUS_EMPTY_SEGMENT,AMBIGUOUS_PATH_ENCODING,SUSPICIOUS_PATH_CHARACTERS}}|
|{{hadoop.http.ssl.renegotiation.allowed}}|{{false}}|
Adding {{AMBIGUOUS_PATH_SEPARATOR}} to the first accepts {{%2F}} again.
h2. Not in scope
No move to {{{}jakarta.servlet{}}}, no Jersey upgrade, and no ee9, ee10 or
ee11. HADOOP-19395 lands the namespace change, in a major release;
HADOOP-19912, the umbrella for this move to Jetty 12 ee8, keeps
{{javax.servlet}}. This only stops that decision from blocking the security fix.
> Upgrade to Jetty 12 without changing the servlet namespace
> ----------------------------------------------------------
>
> Key: HADOOP-19972
> URL: https://issues.apache.org/jira/browse/HADOOP-19972
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, common
> Reporter: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> Hadoop's web server is Jetty 9.4, which is end of life and gets no security
> fixes. Jetty 10 and later require the {{jakarta.servlet}} namespace, which
> would break every project that embeds Hadoop's web stack. Jetty 12's ee8
> environment still serves {{{}javax.servlet{}}}, so this moves to a supported
> Jetty without that break.
> Jetty 9.4.58 to 12.1.12 on ee8. No public signature changes, and downstream
> projects need not rebuild. {*}Requires Java 17{*}, so it cannot go to
> branch-3.3 or branch-3.4. The YARN application catalog webapp stays on Jetty
> 9.4.58, which Solr 8 needs.
> h2. ee8 is a staging post
> ee8 is a stage, not the destination. The point of landing it is that the port
> to Jetty 12's API happens here and is not repeated: afterwards Hadoop is on a
> supported Jetty, and what remains is the namespace change on its own — ee10
> or ee11 and {{{}jakarta.servlet{}}}, under HADOOP-19395 — instead of a Jetty
> upgrade and an API break having to be taken together. ee8 does carry a
> compatibility layer that adapts every request, so it is a place to stand for
> a release or two rather than indefinitely; unlike Jetty 10 and 11 it has a
> supported vendor behind it, which is what makes standing there reasonable.
> h2. Incompatible changes
> This is the release note. The reason phrase alone makes this incompatible.
> # *No custom HTTP reason phrase is sent.* Refusals from the authentication,
> KMS and CSRF filters and from image transfer carry their message in the
> response body instead, PUT and DELETE included, and Hadoop's clients read it
> there. An older client sees the standard phrase: "Unauthorized" rather than
> "Authentication required".
> # *Stricter URI parsing.* {{{}%2F{}}}, {{%2E}} segments and malformed
> encodings are refused with 400. {{hadoop.http.uri.compliance.violations}}
> controls what is allowed through.
> # *TLS renegotiation is refused.* A client's TLS 1.2 renegotiation closes
> the connection. {{hadoop.http.ssl.renegotiation.allowed=true}} restores 9.4's
> behaviour.
> # *The five async HTTP metrics read 0.* The names and the metric set are
> unchanged.
> # *Jetty's own output differs:* the error-page markup, charset and {{Vary}}
> on static files, a {{Date}} header on error responses, the default acceptor
> count, and a context path without its trailing slash redirecting 301 with a
> relative {{Location}} where 9.4 sent 302 with an absolute one.
> Everything else keeps 9.4's behaviour: {{/static}} lists no directory,
> {{/logs}} keeps its admin check when the log directory is missing,
> {{web.xml}} error pages work, the HTTP server metrics stay in milliseconds
> under their existing names, a stopped listener can be reopened, the
> NodeManager's web socket upgrade works, and a redirect's {{Location}} stays
> absolute.
> h2. New settings
> ||Setting||Default||
> |{{hadoop.http.uri.compliance.violations}}|{{AMBIGUOUS_EMPTY_SEGMENT,AMBIGUOUS_PATH_ENCODING,SUSPICIOUS_PATH_CHARACTERS}}|
> |{{hadoop.http.ssl.renegotiation.allowed}}|{{false}}|
> Adding {{AMBIGUOUS_PATH_SEPARATOR}} to the first accepts {{%2F}} again.
> h2. Split out
> To keep this issue to the Jetty port itself, the parts that also work on
> Jetty 9.4 land first, each on its own:
> * HADOOP-20003: clients read a refusal's reason from the body
> * HADOOP-20004: filters send the reason in the body for every HTTP method
> * HADOOP-20005: refused fsimage transfers carry their reason in the body
> * HADOOP-20006: {{JettyUtils#clearContentType}}
> * HADOOP-20007: test for the downstream javax.servlet contract
> * HADOOP-20008: the app catalog's own Jetty version
> This issue is blocked by all six. The incompatible changes listed above still
> apply; after the split, the only cause of the reason-phrase change is Jetty
> 12 itself. The app catalog stays on Jetty 9.4 through {{solr.jetty.version}}
> until HADOOP-19998.
> h2. Not in scope
> No move to {{{}jakarta.servlet{}}}, no Jersey upgrade, and no ee9, ee10 or
> ee11. HADOOP-19395 lands the namespace change, in a major release;
> HADOOP-19912, the umbrella for this move to Jetty 12 ee8, keeps
> {{javax.servlet}}. This only stops that decision from blocking the security
> fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
