[Sorry, I accidentally e-mailed only alice yesterday. Here's the corrected
and extended version.]
Thank you all for taking care!
Today I found out that the problem lies probably in my server docker
runtime or underlying system than inside the container:
In attached out.txt:
I tested the command line on alpine containers 3.17 down to 3.12 and you
can see, the bash worked correctly for me (READABLE-INTERNAL-BASH) only in
3.13 and 3.12. There was no bash version change between alpine 3.13.12 and
3.14, when "test -r" went bad for me. So... some rogue patch to bash may
have caused this?
But there was a busybox version change at that time. There's no other
change on my side that I'm aware of.
Regular user always executes docker run.
Dockerfile and Makefile are attached.
make image is used to build with varied alpine version in FROM,
make sh to enter the container and run commands shown in out.txt.
The scripts in scripts/ are not involved in the testing, so not attached.
The underlying filesystem (/var/tmp/done/project-job-dir-1095) is ext4
(Debian 10 running on metal):
/var/tmp/done/project-job-dir-1095# find output/ -ls
38537360 4 drwxr-x--- 5 2001 2000 4096 úno 11 20:27 output/
38537362 4 drwxrwx--- 2 2001 2000 4096 úno 11 20:27 output/files
38537366 24 -rw-r--r-- 1 2002 2000 20512 úno 11 20:27
output/files/cnb_rate_gbp_czk.csv
38537365 24 -rw-r--r-- 1 2002 2000 20512 úno 11 20:27
output/files/cnb_rate_eur_czk.csv
38537364 24 -rw-r--r-- 1 2002 2000 20512 úno 11 20:27
output/files/cnb_rate_usd_czk.csv
lsattr shows "--------------e----" for all these items.
# uname -a
Linux ... 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
GNU/Linux
# dpkg -l docker.io | grep ii
ii docker.io 18.09.1+dfsg1-7.1+deb10u3 amd64 Linux container
runtime
BUT, i carefully transferred the job directory to my home Ubuntu system:
# uname -a
Linux Inspiron 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023
x86_64 x86_64 x86_64 GNU/Linux
# dpkg -l docker.io | grep ii
ii docker.io 20.10.12-0ubuntu4 amd64 Linux container runtime
then the new bash in alpine 3.17 in container works CORRECTLY:
e7a19212db0f:/task/output/files$ echo $BASH_VERSION ; cat
/etc/alpine-release ; busybox | head -1 ; cd /task/output/files/ ; [ -r
cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-BASH ; /usr/bin/[ -r
cnb_rate_eur_czk.csv ] && echo READABLE-EXTERNAL ; /bin/sh -c 'echo
$BB_ASH_VERSION ; [ -r cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-SH'
5.2.15(1)-release
3.17.2
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH
Damn... :-(
On 13. 02. 23 16:34, alice wrote:
On Mon Feb 13, 2023 at 3:24 PM CET, Vladimír Macek (Scripteo) wrote:
Thanks everyone for important pointers!
The problem seems to be brought by bash between alpine official container
3.13 and any later versions.
3.13 has the same bash version as 3.16- 3.17 has bash 5.2.
so, i assume it's new in 3.17 only.
The external command /usr/bin/[ and sh/ash from busybox both work
correctly, see:
5513a6a8232d:/task/output/files$ cat /etc/alpine-release
3.17.1
5513a6a8232d:/task/output/files$ busybox | head -1
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
5513a6a8232d:/task/output/files$ echo $BASH_VERSION
5.2.15(1)-release
5513a6a8232d:/task/output/files$ [ -r cnb_rate_eur_czk.csv ] && echo READABLE
5513a6a8232d:/task/output/files$ [[ -r cnb_rate_eur_czk.csv ]] && echo READABLE
5513a6a8232d:/task/output/files$ /usr/bin/[ -r cnb_rate_eur_czk.csv ] &&
echo READABLE
READABLE
5513a6a8232d:/task/output/files$ /usr/bin/[[ -r cnb_rate_eur_czk.csv ]] &&
echo READABLE
READABLE
5513a6a8232d:/task/output/files$ ls -la /bin/*sh
lrwxrwxrwx 1 root root 12 Jan 9 12:46 /bin/ash ->
/bin/busybox
-rwxr-xr-x 1 root root 789064 Dec 14 02:37 /bin/bash
lrwxrwxrwx 1 root root 12 Jan 9 12:46 /bin/fdflush ->
/bin/busybox
lrwxrwxrwx 1 root root 12 Jan 9 12:46 /bin/sh ->
/bin/busybox
5513a6a8232d:/task/output/files$ /bin/sh
/task/output/files $ set| head -1
BB_ASH_VERSION='1.35.0'
/task/output/files $ [ -r cnb_rate_eur_czk.csv ] && echo READABLE
READABLE
There was a bash-5.1.16-r0 in alpine:3.13, which was the latest version,
where test -r worked correctly for me.
So sorry for bugging here. I need to find out now what to do next to pursue
this.
you can probably open an issue, after checking if someone else hasn't already:
https://savannah.gnu.org/support/?group=bash
(i assume that's the right place.)
if you get it fixed and a patch merged, i'll happily backport it to 3.17.
Thanks again,
Vlada
$ echo $BASH_VERSION ; cat /etc/alpine-release ; busybox | head -1 ; cd
/task/output/files/ ; [ -r cnb_rate_eur_czk.csv ] && echo
READABLE-INTERNAL-BASH ; /usr/bin/[ -r cnb_rate_eur_czk.csv ] && echo
READABLE-EXTERNAL ; /bin/sh -c 'echo $BB_ASH_VERSION ; [ -r
cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-SH'
5.2.15(1)-release
3.17.1
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH
5.1.16(1)-release
3.16.4
BusyBox v1.35.0 (2022-08-01 15:14:44 UTC) multi-call binary.
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH
5.1.16(1)-release
3.15.7
BusyBox v1.34.1 (2022-07-19 20:11:24 UTC) multi-call binary.
READABLE-EXTERNAL
1.34.1
READABLE-INTERNAL-SH
5.1.16(1)-release
3.14.9
BusyBox v1.33.1 () multi-call binary.
READABLE-EXTERNAL
1.33.1
READABLE-INTERNAL-SH
5.1.16(1)-release
3.14.0
BusyBox v1.33.1 () multi-call binary.
READABLE-EXTERNAL
1.33.1
READABLE-SH-INTERNAL
5.1.16(1)-release
3.13.12
BusyBox v1.32.1 () multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL
READABLE-INTERNAL-SH
5.0.17(1)-release
3.12.12
BusyBox v1.31.1 () multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL
READABLE-INTERNAL-SH
FROM alpine:3.14.0
LABEL maintainer="d...@mixulo.com"
RUN apk add --no-cache \
curl \
bash \
tar \
tzdata \
&& addgroup --gid 2000 mixulo \
&& adduser --uid 2001 --home / --no-create-home --ingroup mixulo --gecos
"Project Setup-Teardown" --disabled-password setup \
&& adduser --uid 2002 --home /home/task --ingroup mixulo --gecos "Project
Task" --disabled-password task
USER setup
COPY ./scripts/*.sh /scripts/
# To test this container, use `-u task` parameter for `docker run` when using
this.
CMD /scripts/dummy.sh
MODULE := setup-teardown
REGISTRY ?= project
IMAGE_NAME := $(REGISTRY)/$(MODULE)
.PHONY: image push prod push push-prod run-setup run run-teardown sh
BUILD_HOST := $(shell hostname --fqdn)
BUILD_LOG :=
build-logs/docker-build-$(REGISTRY)-$(MODULE)-$(BUILD_HOST)-$(shell date
+%Y%m%d-%H%M%S-%N-%Z).log
BUILD_HEADER = "Image $(IMAGE_NAME) built $(shell date +"%FT%T%z") on
$(BUILD_HOST) from $(shell git describe --abbrev=0 --always --dirty)"
image:
echo "$(BUILD_HEADER)" >>$(BUILD_LOG)
docker build -t $(IMAGE_NAME) . 2>&1 | tee -a $(BUILD_LOG)
clean-image:
echo "$(BUILD_HEADER) (clean target)" >>$(BUILD_LOG)
docker build --pull --no-cache -t $(IMAGE_NAME) . 2>&1 | tee -a
$(BUILD_LOG)
push: image
docker push $(IMAGE_NAME)
prod: image
{ docker tag $(IMAGE_NAME) $(IMAGE_NAME):prod 2>&1 ; echo Tagged
$(IMAGE_NAME):prod ; } | tee -a $(BUILD_LOG)
push-prod: prod
docker push $(IMAGE_NAME):prod
# Development targets
RUN_FLAGS = \
--rm \
-e JOB_SPEC_inputBatchUrl \
-e JOB_SPEC_outputFilePostUrl \
-e JOB_SPEC_statusUrl \
-e TZ \
--security-opt=no-new-privileges \
-v /var/tmp/done/project-job-dir-1095:/task
run-setup:
docker run $(RUN_FLAGS) --network host --user setup $(RUN_ARGS)
$(IMAGE_NAME) /scripts/setup.sh
run:
docker run $(RUN_FLAGS) --network none -e DUMMY_SLEEP=1 $(RUN_ARGS)
$(IMAGE_NAME)
run-teardown:
docker run $(RUN_FLAGS) --network host $(IMAGE_NAME) $(RUN_ARGS)
/scripts/teardown.sh
sh:
docker run -ti $(RUN_FLAGS) $(RUN_ARGS) $(IMAGE_NAME) /bin/bash
_______________________________________________
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
