SEZ9 opened a new pull request, #12072:
URL: https://github.com/apache/seatunnel/pull/12072
Close #12068
### Purpose of this pull request
`/submit-job/upload` registers its multipart handling with `new
MultipartConfigElement("")`. That single-argument form leaves both
`maxFileSize` and `maxRequestSize` at `-1`, which Jetty reads as unlimited. The
uploaded part is then buffered to the temp directory by Jetty *and* read into a
`String` in full by the servlet, so one oversized body is enough to fill the
master's disk and exhaust its heap — a plain `curl --form` against an endpoint
that only ever receives job config files.
This adds `upload-max-file-size-mb` and `upload-max-request-size-mb` under
`seatunnel.engine.http`, both defaulting to 10 MB, which is orders of magnitude
above any real job config. A value of `0` or below maps to Jetty's unlimited,
so anyone who relies on the current behaviour can keep it.
### Does this PR introduce _any_ user-facing change?
Yes, two ways.
1. Two new options, documented in `docs/{en,zh}/engines/zeta/rest-api-v2.md`:
```yaml
seatunnel:
engine:
http:
enable-http: true
port: 8080
upload-max-file-size-mb: 10
upload-max-request-size-mb: 10
```
2. A previously-accepted upload larger than 10 MB is now rejected by Jetty
before the config is parsed. This is a behaviour change relative to dev and to
released versions, which is why the default is set well above any plausible
config file and why `0` restores the old behaviour.
### How was this patch tested?
No new test. Asserting on the limit means standing up Jetty and pushing a
>10 MB body through it — `MultipartConfigElement` exposes no state on the
`ServletHolder` registration to check otherwise — and an IT of that weight is
not worth it for a two-field constructor change. The existing
`/submit-job/upload` coverage exercises the normal path, since a real config
file is far below the default.
To reproduce the behaviour change on a running cluster:
```bash
dd if=/dev/zero of=/tmp/big.conf bs=1M count=64
# before: the master's temp dir grows by 64 MB, then OOM in IOUtils.toString
# after: Jetty rejects the request before the config is parsed
curl --location 'http://127.0.0.1:8080/submit-job/upload' --form
'config_file=@"/tmp/big.conf"'
# a normal config upload is unaffected
curl --location 'http://127.0.0.1:8080/submit-job/upload' --form
'config_file=@"/tmp/fake_to_console.conf"'
```
Both keys are wired through
`YamlSeaTunnelDomConfigProcessor.parseHttpConfig`, so they are parsed rather
than falling through to `Unrecognized element` — worth reviewing, since a
missing branch there fails silently.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
