Hi, did you play with the buffering (man stdbuf) ? I suspect the kernel has a more aggressive buffering these days. In one project i had to add fflush() to make sure that i really get all data. I never found a good explanation. ________________________________________ Von: busybox <busybox-boun...@busybox.net> im Auftrag von John Thomson <li...@johnthomson.fastmail.com.au> Gesendet: Sonntag, 29. November 2020 14:21:00 An: busybox@busybox.net Betreff: [bug] dd piped input truncates ~1% of values to %4096 on multi-core systems
Hi Busybox, I may have run into a bug where dd with piped input will truncate a small number of results (~1%) to a modulo 4096 value on multi-core systems. The test script does not produce errors running under bash. This issue is not seen if dd reads the file with if= I have not looked into the cause. Example: # generate test file dd if=/dev/urandom of=/tmp/wlan_data_0 bs=1M count=1 # ~1% pipe error cat /tmp/wlan_data_0 | \ dd iflag=skip_bytes bs=$((0x2f20)) skip=0 count=1 2>/dev/null | wc -c First noticed on arm with busybox 1.31.1 Reproduced on x86_64 busybox g6a55b4e40 Reproduced on mipsel 4 core Not reproduced on mips single core cat data | dd used for Openwrt wifi radio calibration data extraction from sysfs: https://github.com/openwrt/openwrt/commit/7557e7f267e845db5a403139c49f2637f9021992 Scripted example: test_caldata_extract() { local counter local count_good local count_bad local result local result_latest local dd_bs dd_bs=$((0x2f20)) dd if=/dev/urandom of=/tmp/wlan_data_0 bs=1M count=1 result=$(cat /tmp/wlan_data_0 | \ dd iflag=skip_bytes bs=$dd_bs skip=0 count=1 2>/dev/null | wc -c) while [ ! $result -eq $dd_bs ]; do echo "bad initial result" result=$(cat /tmp/wlan_data_0 | \ dd iflag=skip_bytes bs=$dd_bs skip=0 count=1 2>/dev/null | wc -c) done counter=0 count_good=0 count_bad=0 echo "initial run $counter $result" while true; do counter=$(( counter + 1 )) result_latest=$(cat /tmp/wlan_data_0 | \ dd iflag=skip_bytes bs=$((0x2f20)) skip=0 count=1 2>/dev/null | wc -c) if [ "$result_latest" = "$result" ]; then count_good=$(( count_good + 1 )) #echo "run $counter okay" else count_bad=$(( count_bad + 1 )) echo "run $counter bad $result_latest ($count_bad / $counter)" #break fi done } test_caldata_extract Cheers -- John Thomson _______________________________________________ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox _______________________________________________ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox