Hi Niels, On Wed, Feb 07, 2018 at 01:13:32PM +0100, Niels Möller wrote:
> >> What's the host triplet? > > > > armv7veb-hardfloat-linux-gnueabi > ^^ > And the "eb" is for big-endian? Only the b actually. ve stands for virtualization extensions: http://gcc.gnu.org/ml/gcc-patches/2013-12/msg01783.html. But that's just my fancy. More common triples would most likely use armv7b or armv7eb and the above should perhaps have been armv7veeb. :) > > # define WORDS_BIGENDIAN 1 > Can you check if it's detected correctly also when cross-compiling? # ./configure --host=armv7veb-hardfloat-linux-gnueabi checking build system type... x86_64-unknown-linux-gnu checking host system type... armv7veb-hardfloat-linux-gnueabi [...] configure: summary of build options: Version: nettle 3.4 Host type: armv7veb-hardfloat-linux-gnueabi ABI: standard Assembly files: arm/v6 arm Install prefix: /usr/local Library directory: ${exec_prefix}/lib Compiler: armv7veb-hardfloat-linux-gnueabi-gcc Static libraries: yes Shared libraries: yes Public key crypto: no Using mini-gmp: no Documentation: yes # grep WORDS_BIG config.h /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most # define WORDS_BIGENDIAN 1 # ifndef WORDS_BIGENDIAN # define WORDS_BIGENDIAN 1 Seems fine. > > FAIL: memxor > This also does some tricks with word reads and rotate. (The C code does > that too, but with conditions on WORDS_BIGENDIAN). I think I got memxor, sha1 and sha256 sorted. Patch below. > > FAIL: chacha > The chacha code doesn't look endian-dependent to me. I'd guess it's a > consequence of incorrect memxor (below). This one is still failing, even though memxor and sha are fixed. I've been looking at the code and can't find any apparent reason. In chacha-core-internal.c I see the following bit of code that does seem to do endianness handling: dst[i] = LE_SWAP32 (t); Would this apply to chacha-core-internal.asm, too? > > FAIL: umac > Similar problem, I would guess. But this time, loading 64 bits at a time > into neon registers. I'm drawing a bit of a blank on this one. It fails on the very first test case of umac32 where only umac-nh is used and all the input is zeroes. So there does seem to be another endianness dependency in the actual computation code. Have I understood correctly, that vld1.8 reads a byte stream and should be endianness-neutral anyway and the keys are in host endianness? > If you feel like, > v6/aes-*.asm could also use better code for aligned reading of input > data. Huh, getting existing code to work again is one thing. But actual better code is certainly beyond me. :-/ > Aarch64 assembly (for both endian flavors) would be nice, but it's a > separate project. I haven't yet looked into aarch64-assembly. I made an > attempt to build nettle under termux on my android phone a while ago, > but it failed because it didn't provide /bin/sh at the expected place. Sorry, I think I had confused nettle with an other library I came across during debugging which had armv8 code. Again, I think I should leave producing actually working and efficient assembler code to someone who knows what they're doing. :) > >> Before attempting to support big-endian arm, I'd need some idea on how > >> to test it. > > > > Any halfway current ARM cross toolchain should be able to also output > > big-endian arm binaries (-mbig-endian). Then you could test those with > > qemu-user-armeb, which is very light-weight in that it doesn't need a > > kernel or emulated system and allows to run binaries directly. > Sounds good. I hope the needed tools are packaged in debian, I'll have > to check that. I was wrong: While the compiler is able to output big-endian objects with -mbig-endian, it needs matching libs as well (e.g. libgcc_s). Debian doesn't have anything precompiled for armeb. They refer you to Linaro's toolchains or rebootstrap for building from scratch instead (I do something similar with crossdev on Gentoo). This Linaro toolchain works for me: https://releases.linaro.org/components/toolchain/binaries/latest/armeb-linux-gnueabihf/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf.tar.xz michael@debian:~/nettle$ PATH=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/bin:$PATH ./configure --host=armeb-linux-gnueabihf michael@debian:~/nettle$ make [...] michael@debian:~/nettle$ file libnettle.so libnettle.so: ELF 32-bit MSB shared object, ARM, EABI5 BE8 version 1 (SYSV), dynamically linked, BuildID[sha1]=1a8daa9c1d3e61b9d99d34f462337d02c47c9d74, with debug_info, not stripped michael@debian:~/nettle$ make testsuite/sha1-test Now qemu can be installed, which automatically registers with binfmt so that arm binaries can just be executed: michael@debian:~/nettle$ sudo apt-get install qemu-user-static michael@debian:~/nettle$ file testsuite/sha1-test testsuite/sha1-test: ELF 32-bit MSB executable, ARM, EABI5 BE8 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=ec39b7153f4c09d11cac92d34c8e509bb1f4d0a0, with debug_info, not stripped michael@debian:~/nettle$ testsuite/sha1-test /lib/ld-linux-armhf.so.3: No such file or directory michael@debian:~/nettle$ QEMU_LD_PREFIX=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/armeb-linux-gnueabihf/libc testsuite/sha1-test qemu: uncaught target signal 11 (Segmentation fault) - core dumped Segmentation fault This segfaults because of a bug in qemu where it tries to use the host's /etc/ld.so.cache. Deleting it "solves" that. Alternatively, it could be run in a chroot to avoid the segfault but would require some fiddling with the compiler's sysroot. michael@debian:~/nettle$ sudo rm /etc/ld.so.cache michael@debian:~/nettle$ QEMU_LD_PREFIX=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/armeb-linux-gnueabihf/libc testsuite/sha1-test testsuite/sha1-test: error while loading shared libraries: libnettle.so.6: cannot open shared object file: No such file or directory michael@debian:~/nettle$ ln -sfn libnettle.so libnettle.so.6 michael@debian:~/nettle$ QEMU_LD_PREFIX=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/armeb-linux-gnueabihf/libc LD_LIBRARY_PATH=. testsuite/sha1-test This worked because configure detected only generic arm support: Assembly files: arm So plain arm assembly seems to be BE-safe. :) After hacking configure to also enable arm/v6 with this triple I get: michael@debian:~/nettle$ QEMU_LD_PREFIX=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/armeb-linux-gnueabihf/libc LD_LIBRARY_PATH=. testsuite/sha1-test Got: 9844f81e1408f6ec b932137d33bed7cf dcf518a3 Expected: da39a3ee5e6b4b0d 3255bfef95601890 afd80709 qemu: uncaught target signal 6 (Aborted) - core dumped Aborted Which seems about right. With the patch that goes away: michael@debian:~/nettle$ git am 0001-Support-big-endian-arm-in-sha1-armv6-assembly-code.patch Applying: Support big-endian arm in sha1 armv6 assembly code [make && make check] michael@debian:~/nettle$ QEMU_LD_PREFIX=$HOME/gcc-linaro-7.2.1-2017.11-x86_64_armeb-linux-gnueabihf/armeb-linux-gnueabihf/libc LD_LIBRARY_PATH=. testsuite/sha1-test michael@debian:~/nettle$ I also tried rebootstrap but this quickly got really involved. > > --- a/configure.ac > > +++ b/configure.ac > > @@ -691,6 +691,7 @@ ASM_TYPE_FUNCTION='@function' > > ASM_TYPE_PROGBITS='@progbits' > > ASM_MARK_NOEXEC_STACK='' > > ASM_ALIGN_LOG='' > > +ASM_WORDS_BIGENDIAN="$ac_cv_c_bigendian" > If you have the time, it would be good to file an autoconf bug report, > asking them to document (and support) that AC_C_BIGENDIAN sets the shell > variable ac_cv_c_bigendian. Instead I augmented the default action (which is documented and shouldn't change) by setting ASM_WORDS_BIGENDIAN directly. Also this should make the explicit value checking in IF_BE redundant because we now know for sure configure will never emit anything other than yes and no. Documentation says that AC_C_BIGENDIAN will abort if endianness can't be determined. From db70ecccdc65a97c103f3900b4f45d8370c1dd62 Mon Sep 17 00:00:00 2001 From: Michael Weiser <michael.wei...@gmx.de> Date: Wed, 7 Feb 2018 00:11:24 +0100 Subject: [PATCH] Support big-endian arm in assembly code Introduce m4 macros to conditionally handle differences of little- and big-endian arm in assembler code. Adjust sha1-compress, sha256-compress and memxor for arm to work in big-endian mode. --- arm/memxor.asm | 21 +++++++++++++++----- arm/memxor3.asm | 49 ++++++++++++++++++++++++++++++---------------- arm/v6/sha1-compress.asm | 8 ++++++-- arm/v6/sha256-compress.asm | 14 ++++++++----- asm.m4 | 3 +++ config.m4.in | 1 + configure.ac | 5 ++++- 7 files changed, 71 insertions(+), 30 deletions(-) diff --git a/arm/memxor.asm b/arm/memxor.asm index a50e91bc..239a4034 100644 --- a/arm/memxor.asm +++ b/arm/memxor.asm @@ -44,6 +44,11 @@ define(<N>, <r2>) define(<CNT>, <r6>) define(<TNC>, <r12>) +C little-endian and big-endian need to shift in different directions for +C alignment correction +define(<S0ADJ>, IF_LE(<lsr>, <lsl>)) +define(<S1ADJ>, IF_LE(<lsl>, <lsr>)) + .syntax unified .file "memxor.asm" @@ -99,6 +104,8 @@ PROLOGUE(nettle_memxor) C C With little-endian, we need to do C DST[i] ^= (SRC[i] >> CNT) ^ (SRC[i+1] << TNC) + C With big-endian, we need to do + C DST[i] ^= (SRC[i] << CNT) ^ (SRC[i+1] >> TNC) push {r4,r5,r6} @@ -117,14 +124,14 @@ PROLOGUE(nettle_memxor) .Lmemxor_word_loop: ldr r5, [SRC], #+4 ldr r3, [DST] - eor r3, r3, r4, lsr CNT - eor r3, r3, r5, lsl TNC + eor r3, r3, r4, S0ADJ CNT + eor r3, r3, r5, S1ADJ TNC str r3, [DST], #+4 .Lmemxor_odd: ldr r4, [SRC], #+4 ldr r3, [DST] - eor r3, r3, r5, lsr CNT - eor r3, r3, r4, lsl TNC + eor r3, r3, r5, S0ADJ CNT + eor r3, r3, r4, S1ADJ TNC str r3, [DST], #+4 subs N, #8 bcs .Lmemxor_word_loop @@ -132,10 +139,14 @@ PROLOGUE(nettle_memxor) beq .Lmemxor_odd_done C We have TNC/8 left-over bytes in r4, high end - lsr r4, CNT + S0ADJ r4, CNT ldr r3, [DST] eor r3, r4 + C memxor_leftover does an LSB store + C so we need to reverse if actually BE +IF_BE(< rev r3, r3>) + pop {r4,r5,r6} C Store bytes, one by one. diff --git a/arm/memxor3.asm b/arm/memxor3.asm index 139fd208..69598e1c 100644 --- a/arm/memxor3.asm +++ b/arm/memxor3.asm @@ -49,6 +49,11 @@ define(<ATNC>, <r10>) define(<BCNT>, <r11>) define(<BTNC>, <r12>) +C little-endian and big-endian need to shift in different directions for +C alignment correction +define(<S0ADJ>, IF_LE(<lsr>, <lsl>)) +define(<S1ADJ>, IF_LE(<lsl>, <lsr>)) + .syntax unified .file "memxor3.asm" @@ -124,6 +129,8 @@ PROLOGUE(nettle_memxor3) C C With little-endian, we need to do C DST[i-i] ^= (SRC[i-i] >> CNT) ^ (SRC[i] << TNC) + C With big-endian, we need to do + C DST[i-i] ^= (SRC[i-i] << CNT) ^ (SRC[i] >> TNC) rsb ATNC, ACNT, #32 bic BP, #3 @@ -138,14 +145,14 @@ PROLOGUE(nettle_memxor3) .Lmemxor3_au_loop: ldr r5, [BP, #-4]! ldr r6, [AP, #-4]! - eor r6, r6, r4, lsl ATNC - eor r6, r6, r5, lsr ACNT + eor r6, r6, r4, S1ADJ ATNC + eor r6, r6, r5, S0ADJ ACNT str r6, [DST, #-4]! .Lmemxor3_au_odd: ldr r4, [BP, #-4]! ldr r6, [AP, #-4]! - eor r6, r6, r5, lsl ATNC - eor r6, r6, r4, lsr ACNT + eor r6, r6, r5, S1ADJ ATNC + eor r6, r6, r4, S0ADJ ACNT str r6, [DST, #-4]! subs N, #8 bcs .Lmemxor3_au_loop @@ -154,7 +161,11 @@ PROLOGUE(nettle_memxor3) C Leftover bytes in r4, low end ldr r5, [AP, #-4] - eor r4, r5, r4, lsl ATNC + eor r4, r5, r4, S1ADJ ATNC + + C leftover does an LSB store + C so we need to reverse if actually BE +IF_BE(< rev r4, r4>) .Lmemxor3_au_leftover: C Store a byte at a time @@ -247,21 +258,25 @@ PROLOGUE(nettle_memxor3) ldr r5, [AP, #-4]! ldr r6, [BP, #-4]! eor r5, r6 - lsl r4, ATNC - eor r4, r4, r5, lsr ACNT + S1ADJ r4, ATNC + eor r4, r4, r5, S0ADJ ACNT str r4, [DST, #-4]! .Lmemxor3_uu_odd: ldr r4, [AP, #-4]! ldr r6, [BP, #-4]! eor r4, r6 - lsl r5, ATNC - eor r5, r5, r4, lsr ACNT + S1ADJ r5, ATNC + eor r5, r5, r4, S0ADJ ACNT str r5, [DST, #-4]! subs N, #8 bcs .Lmemxor3_uu_loop adds N, #8 beq .Lmemxor3_done + C leftover does an LSB store + C so we need to reverse if actually BE +IF_BE(< rev r4, r4>) + C Leftover bytes in a4, low end ror r4, ACNT .Lmemxor3_uu_leftover: @@ -290,18 +305,18 @@ PROLOGUE(nettle_memxor3) .Lmemxor3_uud_loop: ldr r5, [AP, #-4]! ldr r7, [BP, #-4]! - lsl r4, ATNC - eor r4, r4, r6, lsl BTNC - eor r4, r4, r5, lsr ACNT - eor r4, r4, r7, lsr BCNT + S1ADJ r4, ATNC + eor r4, r4, r6, S1ADJ BTNC + eor r4, r4, r5, S0ADJ ACNT + eor r4, r4, r7, S0ADJ BCNT str r4, [DST, #-4]! .Lmemxor3_uud_odd: ldr r4, [AP, #-4]! ldr r6, [BP, #-4]! - lsl r5, ATNC - eor r5, r5, r7, lsl BTNC - eor r5, r5, r4, lsr ACNT - eor r5, r5, r6, lsr BCNT + S1ADJ r5, ATNC + eor r5, r5, r7, S1ADJ BTNC + eor r5, r5, r4, S0ADJ ACNT + eor r5, r5, r6, S0ADJ BCNT str r5, [DST, #-4]! subs N, #8 bcs .Lmemxor3_uud_loop diff --git a/arm/v6/sha1-compress.asm b/arm/v6/sha1-compress.asm index 59d6297e..52739b69 100644 --- a/arm/v6/sha1-compress.asm +++ b/arm/v6/sha1-compress.asm @@ -52,7 +52,7 @@ define(<LOAD>, < sel W, WPREV, T0 ror W, W, SHIFT mov WPREV, T0 - rev W, W +IF_LE(< rev W, W>) str W, [SP,#eval(4*$1)] >) define(<EXPN>, < @@ -127,8 +127,12 @@ PROLOGUE(_nettle_sha1_compress) lsl SHIFT, SHIFT, #3 mov T0, #0 movne T0, #-1 - lsl W, T0, SHIFT +IF_LE(< lsl W, T0, SHIFT>) +IF_BE(< lsr W, T0, SHIFT>) uadd8 T0, T0, W C Sets APSR.GE bits + C on BE rotate right by 32-SHIFT bits + C because there is no rotate left +IF_BE(< rsb SHIFT, SHIFT, #32>) ldr K, .LK1 ldm STATE, {SA,SB,SC,SD,SE} diff --git a/arm/v6/sha256-compress.asm b/arm/v6/sha256-compress.asm index e6f4e1e9..324730c7 100644 --- a/arm/v6/sha256-compress.asm +++ b/arm/v6/sha256-compress.asm @@ -137,8 +137,12 @@ PROLOGUE(_nettle_sha256_compress) lsl SHIFT, SHIFT, #3 mov T0, #0 movne T0, #-1 - lsl I1, T0, SHIFT +IF_LE(< lsl I1, T0, SHIFT>) +IF_BE(< lsr I1, T0, SHIFT>) uadd8 T0, T0, I1 C Sets APSR.GE bits + C on BE rotate right by 32-SHIFT bits + C because there is no rotate left +IF_BE(< rsb SHIFT, SHIFT, #32>) mov DST, sp mov ILEFT, #4 @@ -146,16 +150,16 @@ PROLOGUE(_nettle_sha256_compress) ldm INPUT!, {I1,I2,I3,I4} sel I0, I0, I1 ror I0, I0, SHIFT - rev I0, I0 +IF_LE(< rev I0, I0>) sel I1, I1, I2 ror I1, I1, SHIFT - rev I1, I1 +IF_LE(< rev I1, I1>) sel I2, I2, I3 ror I2, I2, SHIFT - rev I2, I2 +IF_LE(< rev I2, I2>) sel I3, I3, I4 ror I3, I3, SHIFT - rev I3, I3 +IF_LE(< rev I3, I3>) subs ILEFT, ILEFT, #1 stm DST!, {I0,I1,I2,I3} mov I0, I4 diff --git a/asm.m4 b/asm.m4 index 4018c235..343a55fc 100644 --- a/asm.m4 +++ b/asm.m4 @@ -51,6 +51,9 @@ define(<ALIGN>, <.align ifelse(ALIGN_LOG,yes,<m4_log2($1)>,$1) >) +define(<IF_BE>, <ifelse(WORDS_BIGENDIAN,yes,<$1>,<$2>)>) +define(<IF_LE>, <IF_BE(<$2>, <$1>)>) + dnl Struct defining macros dnl STRUCTURE(prefix) diff --git a/config.m4.in b/config.m4.in index e39c880c..11f90a40 100644 --- a/config.m4.in +++ b/config.m4.in @@ -7,6 +7,7 @@ define(<TYPE_PROGBITS>, <@ASM_TYPE_PROGBITS@>)dnl define(<ALIGN_LOG>, <@ASM_ALIGN_LOG@>)dnl define(<W64_ABI>, <@W64_ABI@>)dnl define(<RODATA>, <@ASM_RODATA@>)dnl +define(<WORDS_BIGENDIAN>, <@ASM_WORDS_BIGENDIAN@>)dnl divert(1) @ASM_MARK_NOEXEC_STACK@ divert diff --git a/configure.ac b/configure.ac index 41bf0998..21eba3b5 100644 --- a/configure.ac +++ b/configure.ac @@ -201,7 +201,9 @@ LSH_FUNC_STRERROR # getenv_secure is used for fat overrides, # getline is used in the testsuite AC_CHECK_FUNCS(secure_getenv getline) -AC_C_BIGENDIAN +AC_C_BIGENDIAN([AC_DEFINE([WORDS_BIGENDIAN], 1) + [ASM_WORDS_BIGENDIAN=yes]], + [ASM_WORDS_BIGENDIAN=no]) AC_CACHE_CHECK([for __builtin_bswap64], nettle_cv_c_builtin_bswap64, @@ -811,6 +813,7 @@ AC_SUBST(ASM_TYPE_PROGBITS) AC_SUBST(ASM_MARK_NOEXEC_STACK) AC_SUBST(ASM_ALIGN_LOG) AC_SUBST(W64_ABI) +AC_SUBST(ASM_WORDS_BIGENDIAN) AC_SUBST(EMULATOR) AC_SUBST(LIBNETTLE_MAJOR) -- 2.16.1 -- bye, Micha _______________________________________________ nettle-bugs mailing list nettle-bugs@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/nettle-bugs
