[
https://issues.apache.org/jira/browse/KAFKA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357273#comment-17357273
]
Abhijit Mane commented on KAFKA-12847:
--------------------------------------
Thanks [~chia7712]
I guess my detailed explanation was not clear enough. System tests failed to
start when it failed to build *ducker-ak-openjdk-8* image while trying on
rhel / ubuntu on x86 due to bash UID conflict. It's *+not+* about numerical
value of UID but string literal "UID" itself whose value can't be altered in
bash.
I also tried on RHEL/Ubuntu on x86 and IBM Power (ppc64le) systems with bash
shell.
"echo $UID" will resolve to user id of logged in user. For ex:
--------------------------------------------------------------------------------
# useradd testuser && id testuser
uid=1005(testuser) gid=1005(testuser) groups=1005(testuser)
# echo $UID
1005
# UID=390
-bash: UID: readonly variable
# UID_DUCKER=390 // accepted as no conflict with bash built-in values
--------------------------------------------------------------------------------
A simple name change in Dockerfile from UID to UID_DUCKER fixes the issue so
there is no conflict.
Are you able to run sysTests as is after cloning and without making any changes
as the README states?
(https://github.com/apache/kafka/tree/trunk/tests#readme)
Let me know.
> Dockerfile needed for kafka system tests needs changes
> ------------------------------------------------------
>
> Key: KAFKA-12847
> URL: https://issues.apache.org/jira/browse/KAFKA-12847
> Project: Kafka
> Issue Type: Bug
> Components: system tests
> Affects Versions: 2.8.0, 2.7.1
> Environment: Issue tested in environments below but is independent of
> h/w arch. or Linux flavor: -
> 1.) RHEL-8.3 on x86_64
> 2.) RHEL-8.3 on IBM Power (ppc64le)
> 3.) apache/kafka branch tested: trunk (master)
> Reporter: Abhijit Mane
> Assignee: Abhijit Mane
> Priority: Major
> Labels: easyfix
> Attachments: Dockerfile.upstream
>
>
> Hello,
> I tried apache/kafka system tests as per documentation: -
> ([https://github.com/apache/kafka/tree/trunk/tests#readme|https://github.com/apache/kafka/tree/trunk/tests#readme_])
> =========================================================
> PROBLEM
> ~~~~~~
> 1.) As root user, clone kafka github repo and start "kafka system tests"
> # git clone [https://github.com/apache/kafka.git]
> # cd kafka
> # ./gradlew clean systemTestLibs
> # bash tests/docker/run_tests.sh
> 2.) Dockerfile issue -
> [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]
> This file has an *UID* entry as shown below: -
> -----------
> ARG *UID*="1000"
> RUN useradd -u $*UID* ducker
> // {color:#de350b}*Error during docker build*{color} => useradd: UID 0 is not
> unique, root user id is 0
> -----------
> I ran everything as root which means the built-in bash environment variable
> 'UID' always
> resolves to 0 and can't be changed. Hence, the docker build fails. The issue
> should be seen even if run as non-root.
> 3.) Next, as root, as per README, I ran: -
> server:/kafka> *bash tests/docker/run_tests.sh*
> The ducker tool builds the container images & switches to user '*ducker*'
> inside the container
> & maps kafka root dir ('kafka') from host to '/opt/kafka-dev' in the
> container.
> Ref:
> [https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak|https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak]
> Ex: docker run -d *-v "${kafka_dir}:/opt/kafka-dev"* <img_name>
> This fails as the 'ducker' user has *no write permissions* to create files
> under 'kafka' root dir. Hence, it needs to be made writeable.
> // *chmod -R a+w kafka*
> – needed as container is run as 'ducker' and needs write access since kafka
> root volume from host is mapped to container as "/opt/kafka-dev" where the
> 'ducker' user writes logs
> =========================================================
> =========================================================
> *FIXES needed*
> ~~~~~~~~~
> 1.) Dockerfile -
> [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]
> Change 'UID' to '*UID_DUCKER*'.
> This won't conflict with built in bash env. var UID and the docker image
> build should succeed.
> -----------
> ARG *UID_DUCKER*="1000"
> RUN useradd -u $*UID_DUCKER* ducker
> // *{color:#57d9a3}No Error{color}* => No conflict with built-in UID
> -----------
> 2.) README needs an update where we must ensure the kafka root dir from where
> the tests
> are launched is writeable to allow the 'ducker' user to create results/logs.
> # chmod -R a+w kafka
> With this, I was able to get the docker images built and system tests started
> successfully.
> =========================================================
> Also, I wonder whether or not upstream Dockerfile & System tests are part of
> CI/CD and get tested for every PR. If so, this issue should have been caught.
>
> *Question to kafka SME*
> -------------------------
> Do you believe this is a valid problem with the Dockerfile and the fix is
> acceptable?
> Please let me know and I am happy to submit a PR with this fix.
> Thanks,
> Abhijit
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
