GitHub user aljoscha opened a pull request:
https://github.com/apache/flink/pull/6377
[FLINK-8981] Add end-to-end test for running on YARN with Kerberos
This adds a complete Docker container setup and Docker Compose file for
starting a kerberized Hadoop cluster on Docker.
The test script does the following:
* package "build-target" Flink dist into a tarball
* build docker container
* start cluster using docker compose
* upload tarball and unpack
* modify flink-conf.yaml to use Kerberos keytab for hadoop-user
* Run Streaming WordCount Job
* verify results
We set an exit trap before to ensure that we shut down the docker
compose cluster at the end.
As a prerequisite, this also fixes how we resolve directories in the
end-to-end scripts.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/flink
jira-8981-kerberos-end-to-end-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6377.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6377
----
commit 5aec051a76089f623ebc21418ec5751f9fcad780
Author: Aljoscha Krettek <aljoscha.krettek@...>
Date: 2018-07-18T09:51:27Z
[hotfix] Resolve symbolic links in test scripts
commit 634426b096a36147c3180f9c732efef51155e5bb
Author: Aljoscha Krettek <aljoscha.krettek@...>
Date: 2018-07-18T11:46:29Z
[FLINK-8981] Add end-to-end test for running on YARN with Kerberos
This adds a complete Docker container setup and Docker Compose file for
starting a kerberized Hadoop cluster on Docker.
The test script does the following:
* package "build-target" Flink dist into a tarball
* build docker container
* start cluster using docker compose
* upload tarball and unpack
* modify flink-conf.yaml to use Kerberos keytab for hadoop-user
* Run Streaming WordCount Job
* verify results
We set an exit trap before to ensure that we shut down the docker
compose cluster at the end.
----
---
