[
https://issues.apache.org/jira/browse/HDDS-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-17333:
-------------------------------------
Assignee: Andrey Yarovoy
> Add a secure Ozone + YARN compose environment that can run YARN jobs
> ---------------------------------------------------------------------
>
> Key: HDDS-17333
> URL: https://issues.apache.org/jira/browse/HDDS-17333
> Project: Apache Ozone
> Issue Type: Bug
> Components: docker
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> *Problem*
>
> There's no docker images that can be used to reproduce MapReduce-on-Ozone
> behavior in dev environment. The
> existing `ozonesecure` + `common/hadoop-secure.yaml` setup runs MapReduce
> smoke tests (PI, WordCount), but
> differs from a production NodeManager in ways that hide real costs:
> - It uses `DefaultContainerExecutor`, which localizes job files inside the
> long-lived NodeManager JVM. A secure
> NodeManager uses `LinuxContainerExecutor`, which starts a separate
> `ContainerLocalizer` JVM, with a
> fresh Ozone client, for each container. In our tests that cost about 2 s for
> the AppMaster and 2–3 s for each map
> container on another node. With `DefaultContainerExecutor` it was 0 s.
> - It has a single NodeManager, so maps never run on a node that still has to
> localize the job's files.
> - Its 4 GB map and reduce containers, Recon, S3 Gateway and HttpFS don't fit
> next to YARN in a devs Docker VM.
> Also, a pipeline limit of 1 leaves no RATIS/THREE pipeline when one datanode
> goes stale.
> - The `apache/hadoop` image ships `container-executor` without setuid, and on
> 3.4.3 (Ubuntu 22.04) it is linked
> against OpenSSL 1.1, which the image doesn't have. So
> `LinuxContainerExecutor` can't run as shipped.
> ### Proposed change
>
> A new compose environment, eg. `compose/ozonesecure-mr`, reusing
> `ozonesecure` and `common/hadoop-secure.yaml`:
> - **NodeManagers:** N of them (`nm`, `nm2`, …) with their own hostnames and
> the shared `nm/nm` principal. Each
> offers 2 GB of container memory, so a job's containers spread across nodes.
> - **`LinuxContainerExecutor`:** a NodeManager start script installs
> `container-executor` setuid root under a
> root-owned prefix, with its own `container-executor.cfg`. Where needed it
> adds `libssl1.1`. A small derived
> NodeManager image, or a fixed `apache/hadoop` image, would be cleaner than
> installing at start time.
> - **Small cluster:** only `kdc kms scm om datanode rm jhs nm*`, with
> `ozone.datanode.pipeline.limit=2` and
> 512–768 MB MapReduce containers.
> - **`test-dfsio.sh`:** runs TestDFSIO write/read on `ofs://` with the MR
> staging dir on Ozone. It collects the
> per-container logs and the NodeManager logs and summarizes per-job phases:
> AppMaster and map localization from
> the NodeManager's container state changes, AppMaster and task start-up from
> the container logs, and time outside
> TestDFSIO's own timer.
> - **Not part of the default acceptance run,** because it's resource-heavy. A
> 1-NodeManager, 1-job variant could
> serve as a smoke test.
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
