smengcl commented on pull request #10:
URL:
https://github.com/apache/ozone-docker-runner/pull/10#issuecomment-1030783159
With the patch above, there is seemingly only one issue remaining (all
acceptance test suites up to `compose/upgrade` test have passed locally on my
Intel Mac) that might need to be fixed in ozone repo:
Can be reproduced by running `compose/xcompat/test.sh` solely:
```
==============================================================================
xcompat-cluster-1.1.0-client-1.0.0-read-1.0.0 :: Read Compatibility
==============================================================================
Key Can Be Read | PASS
|
------------------------------------------------------------------------------
Dir Can Be Listed | PASS
|
------------------------------------------------------------------------------
File Can Be Get | PASS
|
------------------------------------------------------------------------------
xcompat-cluster-1.1.0-client-1.0.0-read-1.0.0 :: Read Compatibility | PASS
|
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==============================================================================
Output:
/tmp/smoketest/xcompat/result/robot-xcompat-xcompat-write-new_client-3.xml
==============================================================================
xcompat-cluster-1.1.0-client-1.0.0-read-1.1.0 :: Read Compatibility
==============================================================================
Key Can Be Read | FAIL
|
'False' should be true.
------------------------------------------------------------------------------
Dir Can Be Listed | PASS
|
------------------------------------------------------------------------------
File Can Be Get | PASS
|
------------------------------------------------------------------------------
xcompat-cluster-1.1.0-client-1.0.0-read-1.1.0 :: Read Compatibility | FAIL
|
3 critical tests, 2 passed, 1 failed
3 tests total, 2 passed, 1 failed
==============================================================================
Output:
/tmp/smoketest/xcompat/result/robot-xcompat-xcompat-write-new_client-4.xml
Log:
/Users/smeng/repo/ozone/hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/xcompat/result/log.html
Report:
/Users/smeng/repo/ozone/hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/xcompat/result/report.html
```
Note that I have modded `compose/testlib.sh` to generate report even when
some test fails. And as a (good) side effect the cluster stays up so I can poke
around. Simply add this line:
```diff
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh
b/hadoop-ozone/dist/src/main/compose/testlib.sh
index 9c3d6c49c1..abad8c03dd 100755
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -204,6 +204,7 @@ execute_robot_test(){
set -e
if [[ ${rc} -gt 0 ]]; then
+ generate_report
stop_docker_env
fi
```
Back to the problem analysis, the test [`Key Can Be
Read`](https://github.com/apache/ozone/blob/2c53c821c25/hadoop-ozone/dist/src/main/smoketest/compatibility/read.robot#L25-L26)
seemingly assumed that the file `/etc/passwd` in container ozone-1.0.0 doesn't
change with switching to the new client version, which is no longer the case
(centos 8.4 based runner image has a different `/etc/passwd` than the previous
centos 7.6 based one):
New client (centos 8.4):
```
$ docker exec -it xcompat_new_client_1 bash
bash-4.4$ ozone fs -ls ofs://om/vol1/bucket1/
Found 5 items
drwxrwxrwx - hadoop hadoop 0 2022-02-06 07:52
ofs://om/vol1/bucket1/dir-1.0.0
drwxrwxrwx - hadoop hadoop 0 2022-02-06 07:51
ofs://om/vol1/bucket1/dir-1.1.0
-rw-rw-rw- 3 hadoop hadoop 671 2022-02-06 07:52
ofs://om/vol1/bucket1/key-1.0.0
-rw-rw-rw- 3 hadoop hadoop 744 2022-02-06 07:51
ofs://om/vol1/bucket1/key-1.1.0
drwxrwxrwx - hadoop hadoop 0 2022-02-06 07:51
ofs://om/vol1/bucket1/warmup
bash-4.4$ ls -l /etc/passwd
-rw-r--r-- 1 root root 744 Feb 6 03:22 /etc/passwd
bash-4.4$ cat /etc/*release
CentOS Linux release 8.4.2105
...
```
```
bash-4.4$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
systemd-coredump:x:999:997:systemd Core Dumper:/:/sbin/nologin
systemd-resolve:x:193:193:systemd Resolver:/:/sbin/nologin
hadoop:x:1000:100::/opt/hadoop:/bin/bash
```
Old client 1.0.0 (centos 7.6):
```
$ docker exec -it xcompat_old_client_1_0_0_1 bash
bash-4.2$ ls -l /etc/passwd
-rw-r--r-- 1 root root 671 Nov 7 2019 /etc/passwd
bash-4.2$ cat /etc/*release
CentOS Linux release 7.6.1810 (Core)
...
```
```
bash-4.2$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
hadoop:x:1000:100::/opt/hadoop:/bin/bash
```
Let's just say the test case assumption (that `/etc/passwd` doesn't change
between old and new ozone client images) is not ideal.
One hacky solution (without touching the ozone repo) might be to
intentionally overwrite the /etc/passwd in the new image. And I'm not sure
centos 8.4 image would work with
Proper solution:
1. Rebuild ozone-1.0.0 client image and push a centos 8.4 based one to
Docker Hub, which should be done at some point as well; or
2. Use another file rather than `/etc/passwd` for the testing (e.g. `echo
testing > /tmp/testkey` then upload this instead) -- need to be done in ozone
repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
