Hello Mamone,

On Mon, Jan 11, 2021 at 11:39:43PM +0200, Maamoun TK wrote:

I have tuned the ghash patch to support big-endian mode but I'm really having difficulties testing it out through emulating, I'll attach the patch here so you can test it but I'm not sure how I can fix the bugs on big-endian system if any, you can feel free to send debugging info or setup a remote ssh connection so we can get it work properly.

Out of curiosity as I can't seem to find the beginning of the discussion: Is there anyone but me with an actual use-case for big-endian arm64 here? If not, I'd hate to cause a lot of effort for you and would certainly put in the effort to get this going myself.

The patch is built on top of the master branch.

First it failed to compile gcm-hash.o with error "No rule to make target" which turned out to be caused by a missing arm64/machine.m4. After I added an empty file there it compiled fine on aarch64 and the testsuite succeeded on the actual hardware as well as under qemu-aarch64 user mode emulation (both LE).

On aarch64_be it fails to compile with the following error message:

gcm-hash.s:113: Error: unknown mnemonic `zip' -- `zip v23.2d,v2.2d,v22.2d' gcm-hash.s:119: Error: unknown mnemonic `zip' -- `zip v25.2d,v3.2d,v22.2d' gcm-hash.s:129: Error: unknown mnemonic `zip' -- `zip v27.2d,v4.2d,v22.2d' gcm-hash.s:137: Error: unknown mnemonic `zip' -- `zip v29.2d,v5.2d,v22.2d'

This happens with gcc 10.2.0 on my hardware board as well as cross gcc 9.3.0 of Buildroot 2020.11.1 in a container.

I did a search of the aarch64 instruction set and saw that there's zip1 and zip2 instructions. So as a first test I just changed zip to zip1 which made it compile. As was to be expected, the testsuite failed though.

Before you try and get me up to speed on what the routine is supposed to be doing there's also an option for you to get a cross toolchain and emulator for your own tests without too much effort. Here's how I cross-compile nettle and run the testsuite using rootless podman (docker should do just as well) on my x86_64 box:

cd ~/Downloads mkdir nettle cd nettle git clone https://git.lysator.liu.se/nettle/nettle cd nettle git apply ~/arm64_ghash.patch ./.bootstrap podman run -it -v ~/Downloads/nettle:/nettle michaelweisernettleci/buildroot:2020.11.1-aarch64_be-glibc-gdb cd /nettle/ mkdir build-aarch64_be cd build-aarch64_be/ ../nettle/configure --host=$(cat /buildroot/triple) --enable-armv8-a-crypto make -j4 make -j4 check EMULATOR=/buildroot/qemu

Unfortunately, because in this case qemu-aarch64_be is running the testsuite binaries under emulation and doesn't support the ptrace syscall (and containers usually don't either), you can't just run it under an aarch64_be native gdb to see what it's executing.

One option would be to boot a full BE system image with kernel in qemu-system-aarch64 including a native gdb. But that's a bit of a hassle (building a rootfs and kernel e.g. using buildroot, getting it to boot in qemu, accessing it via console or network, ...)

qemu-user can however serve as a gdb server similar to qemu-system[1]. [1] https://qemu.readthedocs.io/en/latest/system/gdb.html

As luck would have it, above container image contains an x86_64-native gdb targeting aarch64_be. So you can start the testsuite test under qemu with the -g option and a port to listen on for the gdb remote debugging connection and then fire up gdb and connect there. After that you can debug as usual, single-step and look at register values:

root@6c85515d3939:/nettle/build-aarch64_be/testsuite# /buildroot/qemu -E LD_LIBRARY_PATH=../.lib -g 9000 ./gcm-test & [1] 4205 root@6c85515d3939:/nettle/build-aarch64_be/testsuite# aarch64_be-buildroot-linux-gnu-gdb ./gcm-test GNU gdb (GDB) 8.3.1 [...] Reading symbols from ./gcm-test... (gdb) break main Breakpoint 1 at 0x4037b0: file ../../nettle/testsuite/testutils.c, line 123. (gdb) target remote localhost:9000 Remote debugging using localhost:9000 warning: remote target does not support file transfer, attempting to access files from local filesystem. warning: Unable to find dynamic linker breakpoint function. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. 0x0000004000802040 in ?? () (gdb) c Continuing. warning: Could not load shared library symbols for 3 libraries, e.g. /usr/lib64/libgmp.so.10. Use the "info sharedlibrary" command to see the complete listing. Do you need "set solib-search-path" or "set sysroot"?

Breakpoint 1, main (argc=1, argv=0x4000800d58) at ../../nettle/testsuite/testutils.c:123 123 if (argc > 1) (gdb) b _nettle_gcm_init_key Breakpoint 2 at 0x40008b69f4: file gcm-hash.s, line 93. (gdb) c Continuing.

Breakpoint 2, _nettle_gcm_init_key () at gcm-hash.s:93 93 ldr q2,[x0,#16*128] (gdb) s 94 dup v0.16b,v2.b[0] (gdb) 96 mov x1,#0xC200000000000000 (gdb) 97 mov x2,#1 (gdb) 98 mov v6.d[0],x1 (gdb) 99 mov v6.d[1],x2 (gdb) 100 sshr v0.16b,v0.16b,#7 (gdb) 101 and v0.16b,v0.16b,v6.16b (gdb) 102 ushr v1.2d,v2.2d,#63 (gdb) 103 and v1.16b,v1.16b,v6.16b (gdb) 104 ext v1.16b,v1.16b,v1.16b,#8 (gdb) 105 shl v2.2d,v2.2d,#1 (gdb) 106 orr v2.16b,v2.16b,v1.16b (gdb) 107 eor v2.16b,v2.16b,v0.16b (gdb) 109 dup v6.2d,v6.d[0] (gdb) 113 PMUL_PARAM v2,v23,v24 ^--- doesn't seem to expand the macro here (gdb) 115 PMUL v2,v23,v24 (gdb) 117 REDUCTION v3 (gdb) i r x0 0x423390 4338576 x1 0xc200000000000000 -4467570830351532032 [...] x30 0x406c44 4222020 sp 0x4000800ad0 0x4000800ad0 pc 0x40008b6a5c 0x40008b6a5c <_nettle_gcm_init_key+104> cpsr 0x80000000 -2147483648 fpsr 0x0 0 fpcr 0x0 0

The trick to see and single-step the individual instructions of the macro seems to be disp/i $pc combined with stepi:

(gdb) disp/i $pc 1: x/i $pc => 0x40008b6a30 <_nettle_gcm_init_key+60>: pmull2 v20.1q, v2.2d, v6.2d (gdb) stepi 0x00000040008b6a34 113 PMUL_PARAM v2,v23,v24 1: x/i $pc => 0x40008b6a34 <_nettle_gcm_init_key+64>: ext v22.16b, v2.16b, v2.16b, #8 (gdb) 0x00000040008b6a38 113 PMUL_PARAM v2,v23,v24 1: x/i $pc => 0x40008b6a38 <_nettle_gcm_init_key+68>: eor v22.16b, v22.16b, v20.16b (gdb) 0x00000040008b6a3c 113 PMUL_PARAM v2,v23,v24 1: x/i $pc => 0x40008b6a3c <_nettle_gcm_init_key+72>: zip1 v23.2d, v2.2d, v22.2d

From here I would now continue to compare register contents after each

instruction on LE and BE to see where it's going wrong.

How would you like to proceed? Shall I dig into it or do you want to? :)

BTW: In case you want to build the image yourself, the diff to the Dockerfile.aarch64[3] is this:

diff --git a/Dockerfile.aarch64 b/Dockerfile.aarch64 index 36af2c5..5b51c17 100644 --- a/Dockerfile.aarch64 +++ b/Dockerfile.aarch64 @@ -41,6 +41,7 @@ RUN br_libc="${BR_LIBC}" ; \ echo "BR2_TOOLCHAIN_BUILDROOT_${libcopt}=y" ; \ echo 'BR2_KERNEL_HEADERS_4_19=y' ; \ echo 'BR2_PACKAGE_GMP=y' ; \ + echo 'BR2_PACKAGE_HOST_GDB=y' ; \ echo 'BR2_PER_PACKAGE_DIRECTORIES=y' ; \ ) > .config && \ make olddefconfig && \ @@ -75,7 +76,7 @@ MAINTAINER Michael Weiser michael.weiser@gmx.de RUN apt-get update -qq -y && \ apt-get dist-upgrade -y && \ apt-get autoremove -y && \ - apt-get install -y autoconf dash g++ make qemu-user && \ + apt-get install -y autoconf dash g++ libncurses6 libexpat1 make qemu-user && \ apt-get clean all && \ rm -rf /var/lib/apt/lists/*

[3] https://github.com/michaelweiser-nettle-ci/docker-buildroot/blob/master/Dock...

The command to build the image is:

podman build -f Dockerfile.aarch64 --build-arg BR_LIBC=glibc -t buildroot:2020.11.1-aarch64_be-glibc-gdb .

Hello Mamone,

On Mon, Jan 18, 2021 at 06:27:40PM +0200, Maamoun TK wrote:

It would be nice to get the implementation of the enhanced algorithm working for both endian modes as it yields a good performance boost. Also, there is no much effort here, the only thing I'm struggling with is to get the binary built for Aarch64_be, I'm using Ubuntu on x86_64 as host and it seems there is no official package to cross compile for Aarch64_be.

Yes, there are no packages for aarch64_be in any mainstream distribution I'm aware of. Buildroot and Gentoo are the ones I know that can target it, Yocto likely as well. All are compile-yourself-distributions and not for the faint of heart. Also, I've just learned that Buildroot has made a concious decision not to produce native toolchains for the target. So you can only ever cross-compile nettle to it, run it on an actual board or under qemu and then go back to the cross-compiler on the host.

...

I did a search of the aarch64 instruction set and saw that there's zip1 and zip2 instructions. So as a first test I just changed zip to zip1 which made it compile. As was to be expected, the testsuite failed though.

You are on the right track so far.

I've poked at the code a bit more and seemingly made the key init function work by eliminiating all the BE specific macros and instead adjusting the load from memory to produce the same register content. At least register values and the final output to memory look the same in an x/64xb $x0-64 and x64/xb $x0 for the first test cases in gcm-test (which they did not before).

137 PMUL_PARAM v5,v29,v30 (gdb) 139 st1 {v27.16b,v28.16b,v29.16b,v30.16b},[x0] (gdb) 141 ret (gdb) x/64xb $x0-64 0xaaaaaaac5390: 0x77 0x58 0x14 0xdf 0xa9 0x97 0xd2 0xcd [.. all the same on BE and LE ...] 0xaaaaaaac53c8: 0x0d 0x12 0x63 0x69 0x37 0x20 0xd3 0xfe (gdb) x/64xb $x0 0xaaaaaaac53d0: 0xf9 0xfa 0x22 0xc3 0x02 0xe7 0x95 0x86 [.. all the same on BE and LE ...] 0xaaaaaaac5408: 0x45 0x91 0xbd 0x48 0x73 0xd9 0x8b 0x5c (gdb)

The problem here once more seems to be that after a 128bit LE load which is later used as two 64bit operands, not only the bytes of the operands are reversed (which you already counter by rev64'ing them, I gather) but the operands (doublewords) also end up transposed in the register. This is something the rest of the routine expects but is only true on LE. So I adjusted for it on BE in a very pedestrian way:

diff --git a/arm64/v8/gcm-hash.asm b/arm64/v8/gcm-hash.asm index 1c14db54..74cd656a 100644 --- a/arm64/v8/gcm-hash.asm +++ b/arm64/v8/gcm-hash.asm @@ -55,17 +55,10 @@ C common macros: .endm

.macro REDUCTION out -IF_BE(` - pmull T.1q,F.1d,POLY.1d - ext \out().16b,F.16b,F.16b,#8 - eor R.16b,R.16b,T.16b - eor \out().16b,\out().16b,R.16b -',` pmull T.1q,F.1d,POLY.1d eor R.16b,R.16b,T.16b ext R.16b,R.16b,R.16b,#8 eor \out().16b,F.16b,R.16b -') .endm

C void gcm_init_key (union gcm_block *table) @@ -108,19 +101,11 @@ define(`H4M', `v29') define(`H4L', `v30')

.macro PMUL_PARAM in, param1, param2 -IF_BE(` - pmull2 Hp.1q,\in().2d,POLY.2d - ext Hm.16b,\in().16b,\in().16b,#8 - eor Hm.16b,Hm.16b,Hp.16b - zip \param1().2d,\in().2d,Hm.2d - zip2 \param2().2d,\in().2d,Hm.2d -',` pmull2 Hp.1q,\in().2d,POLY.2d eor Hm.16b,\in().16b,Hp.16b ext \param1().16b,Hm.16b,\in().16b,#8 ext \param2().16b,\in().16b,Hm.16b,#8 ext \param1().16b,\param1().16b,\param1().16b,#8 -') .endm

PROLOGUE(_nettle_gcm_init_key) @@ -128,6 +113,10 @@ PROLOGUE(_nettle_gcm_init_key) dup EMSB.16b,H.b[0] IF_LE(` rev64 H.16b,H.16b +',` + mov x1,H.d[0] + mov H.d[0],H.d[1] + mov H.d[1],x1 ') mov x1,#0xC200000000000000 mov x2,#1

If my understanding is correct, we could avoid the doubleword swap for both LE and BE if we were to load using ld1 to {H.b16} instead (with a precalculation of the offset because ld1 won't take an immediate offset that high, correct?). But then the rest of the routine would need to change its expectation what H.d[0] and H.d[1] contain, respectively, because they will no longer be transposed by neither the load on LE nor an explicit swap on BE.

Somehow I have a feeling, I'm terribly missing the actual point here, though. Are the zip instructions likely to give even further speedup beyond the LE version? Could this be exploited for LE as well by adjusting the loading scheme even more?

Also, it's not fully working yet. Before digging deeper I wanted to give a bit of an update and get guidance as to how to proceed.

...

podman run -it -v ~/Downloads/nettle:/nettle

I tried that but I'm having difficulty getting it work, it seems there is a problem in my system configuration that prevents podman establishing a socket for connection, I spend some time looking for alternative solutions with no chance. Do you have any other solutions? all what I can think of is either setup ssh connection or work together to get it work if you are into it!

I mulled this over from all directions. Access to the actual board is somewhat complicated by the limits of my available Internet connections (CGNAT being one, missing DMZ functionality on the routers another). It can certainly be done, I just would need some time to set it up.

But I have made the cross-compiling and -debugging setup of the container available on a vserver on the Net. Send me a mail directly with an SSH ID public key if you'd like to try this out and I'll send you instructions for login and use. We could meet up there in a tmux/screen session and work on it together as well.

I have also tried to extract the buildroot toolchain from the image and run it on my Gentoo box as well as Debian. It even seems relocatable, so you can just put it anywhere and add it to PATH and it'll work. If you want, I can put a tarball with the toolchain and qemu wrappers up on a web server somewhere for you to grab. (I just thought, a container image would be the easier delivery method nowadays. :)

Otherwise, what's your error message from podman? It's got no deamon, so it shouldn't need a socket to connect to it like docker does. Out to the Internet for image download it's also a standard client and respects environment variables for proxies as usual.

rootless podman (running as your standard user instead of root) can take a bit of tweaking before it stops throwing error messages but once that's done it works nicely. I've never actually run podman as root by luck of late birth with regards to containers.

Here's my command sequence on a Ubuntu 20.04 VM that's never seen rootless podman before as per https://www.vultr.com/docs/how-to-install-and-use-podman-on-ubuntu-20-04 (literally the first hit on search, can't vouch for the packages from opensuse though):

michael@demo:~$ podman

Command 'podman' not found, did you mean:

command 'pod2man' from deb perl (5.30.0-9ubuntu0.2)

Try: sudo apt install <deb name>

michael@demo:~$ source /etc/os-release michael@demo:~$ sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stabl... /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" michael@demo:~$ wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/... -O- | sudo apt-key add - 2021-01-19 21:13:19 URL:https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stab... [1093/1093] -> "-" [1] OK michael@demo:~$ sudo apt-get update -qq michael@demo:~$ sudo apt-get -qq --yes install podman fuse-overlayfs slirp4netns [...] michael@demo:~$ podman run -it michaelweisernettleci/buildroot:2020.11.1-aarch64_be-glibc-gdb Completed short name "michaelweisernettleci/buildroot" with unqualified-search registries (origin: /etc/containers/registries.conf) Trying to pull docker.io/michaelweisernettleci/buildroot:2020.11.1-aarch64_be-glibc-gdb... Getting image source signatures Copying blob 6c33745f49b4 done Copying blob ff35d554f2d5 done Copying blob 3927b287d6b9 done Copying blob 6bbc022f227c done Copying config 21663e44fe done Writing manifest to image destination Storing signatures root@06e70f1e12e4:/# aarch64_be-buildroot-linux-gnu-gcc -v Using built-in specs. COLLECT_GCC=/buildroot/output/host/bin/aarch64_be-buildroot-linux-gnu-gcc.br_real COLLECT_LTO_WRAPPER=/buildroot/output/host/bin/../libexec/gcc/aarch64_be-buildroot-linux-gnu/9.3.0/lto-wrapper Target: aarch64_be-buildroot-linux-gnu Configured with: ./configure --prefix=/buildroot/output/per-package/host-gcc-final/host [...] --enable-shared --disable-libgomp --silent Thread model: posix gcc version 9.3.0 (Buildroot 2020.11.1) root@06e70f1e12e4:/# git clone https://git.lysator.liu.se/nettle/nettle bash: git: command not found root@06e70f1e12e4:/# apt-get update Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB] Get:2 http://deb.debian.org/debian buster InRelease [121 kB] [...] root@06e70f1e12e4:/# apt-get install git Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: ca-certificates git-man krb5-locales less libbsd0 libcurl3-gnutls [...] root@06e70f1e12e4:/# git clone https://git.lysator.liu.se/nettle/nettle Cloning into 'nettle'... warning: redirecting to https://git.lysator.liu.se/nettle/nettle.git/ remote: Enumerating objects: 721, done. remote: Counting objects: 100% (721/721), done. remote: Compressing objects: 100% (349/349), done. remote: Total 21095 (delta 479), reused 593 (delta 372), pack-reused 20374 Receiving objects: 100% (21095/21095), 5.90 MiB | 3.47 MiB/s, done. Resolving deltas: 100% (15748/15748), done. root@06e70f1e12e4:/#

That was a lot easier than even I expected. Necessary stuff like entries in /etc/subuid are automatically added by useradd as standard nowadays without podman even being installed:

michael@demo:~$ cat /etc/subuid michael:100000:65536

Hope that helps.

If all else fails and it's not too trying for your patience I'm up for making it work iteratively by trial, error and discussion as above. ;)

Hello Mamone,

On Wed, Jan 20, 2021 at 10:25:19PM +0200, Maamoun TK wrote:

I'm trying to install Gentoo on VMware by walking through this receip https://medium.com/@steensply/vmware-installation-of-gentoo-linux-from-scrat... I'm in the middle of receip now but there a lot of instruction there so I'm gonna get the os working in the end.

As far as I can tell that recipe only encompasses basic installation. You'd additionally need to run crossdev to create a cross-toolchain and then install qemu as well. Gentoo has a very steep learning curve. There's no benefit compared to buildroot for our use-case here, IMO.

Here how I get the vector instruction operate on registers in LE mode, i'll take this instruction as example: pmull v0.1q,v1.1d,v2.1d Input represented as indexes v1: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 v2: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 the instruction byte-reverse each of 64-bit parts of register so the instruction consider the register as follow v1: 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 v2: 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 so what I did in LE mode is reverse the 64-bit parts before execute the doublework operation using rev64 instruction, according to that the pmull output will be 128-bit byte-reversed Output represented as indexes v0: 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

What I'm assuming in BE mode is operations are performed in normal way in registers side so no need to reverse the inputs in addition to get normal output hence the macros "REDUCTION" and "PMUL_PARAM" have differences in their structure, it's not matter of zip instruction perform better but how to handle the weird situation in LE mode.

I've tried for a number of hours to make this work today. Always when I added correct handling of the transposed doublewords to one macro, another broke down. To me the problem comes down to this: ldr HQ,[TABLE...] and st1.16b are fighting each other and can't be brought together without a lot of additional instructions (at least not by me).

Longer story: ldr does a 128bit load. This loads bytes in exactly reverse order into the register on LE and BE. As you describe above, the macros for LE expect a state which is neither of those: The bytes transposed as if BE but the doublewords as loaded on LE. For BE this poses the oppositve problem: It natively loads bytes in the order LE has to reproduce using rev64 but then needs to reproduce the doubleword order of LE for the LE routines to work or basically have native BE routines.

That's what my last pedestrian change did. After today I'd perhaps write it like this (untested):

@@ -125,10 +135,12 @@ IF_BE(`

PROLOGUE(_nettle_gcm_init_key) ldr HQ,[TABLE,#16*H_Idx] - dup EMSB.16b,H.b[0] IF_LE(` rev64 H.16b,H.16b +',` + ext H.16b,H.16b,H.16b,#8 ') + dup EMSB.16b,H.b[7] mov x1,#0xC200000000000000 mov x2,#1 mov POLY.d[0],x1

When trying to cater to the current layout on LE, all the other vectors which are later used in conjunction with H to be reversed as well. That leads to this diff to your initial patch:

@@ -125,14 +135,21 @@ IF_BE(`

PROLOGUE(_nettle_gcm_init_key) ldr HQ,[TABLE,#16*H_Idx] - dup EMSB.16b,H.b[0] IF_LE(` + dup EMSB.16b,H.b[0] rev64 H.16b,H.16b +',` + dup EMSB.16b,H.b[15] ') mov x1,#0xC200000000000000 mov x2,#1 +IF_LE(` mov POLY.d[0],x1 mov POLY.d[1],x2 +',` + mov POLY.d[1],x1 + mov POLY.d[0],x2 +') sshr EMSB.16b,EMSB.16b,#7 and EMSB.16b,EMSB.16b,POLY.16b ushr B.2d,H.2d,#63 @@ -142,7 +159,11 @@ IF_LE(` orr H.16b,H.16b,B.16b eor H.16b,H.16b,EMSB.16b

+IF_LE(` dup POLY.2d,POLY.d[0] +',` + dup POLY.2d,POLY.d[1] +')

C --- calculate H^2 = H*H ---

The difference in index in dup EMSB nicely shows the doubleword transposition compared to LE. If on LE the dup was done after the rev64, it'd be H.b[7] vs. H.b[15].

With this layout PMUL_PARAM can work on H and POLY but then needs to use pmull instead of pmull2 because the relevant data is in the other doublewords compared to LE. On the other hand, since the output of PMUL_PARAM is to be stored using st1.16b it must not have the doublewords transposed ("load-inverted" I termed it in the comments ;). That leads to the following adjustment and makes the first 16bytes of TABLE identical to LE:

@@ -109,11 +118,12 @@ define(`H4L', `v30')

.macro PMUL_PARAM in, param1, param2 IF_BE(` - pmull2 Hp.1q,\in().2d,POLY.2d + pmull Hp.1q,\in().1d,POLY.1d ext Hm.16b,\in().16b,\in().16b,#8 eor Hm.16b,Hm.16b,Hp.16b - zip \param1().2d,\in().2d,Hm.2d - zip2 \param2().2d,\in().2d,Hm.2d + C output must be in native register order (not load-inverted) for st1.16b to work + zip2 \param1().2d,\in().2d,Hm.2d + zip1 \param2().2d,\in().2d,Hm.2d ',` pmull2 Hp.1q,\in().2d,POLY.2d eor Hm.16b,\in().16b,Hp.16b

In PMUL is where it breaks down, at least for my brain: Its first call is handed H (which has doublewords "transposed" from the initial ldr) and H1M and H1L (which must not have doublewords transposed so st1.16b writes them to memory in correct order). It wants to pmull/pmull2 them which requires corresponding doublewords at the same index. So we'd need to temporarily transpose \in for that:

@@ -46,25 +46,34 @@ define(`R1', `v19')

C common macros: .macro PMUL in, param1, param2 - pmull F.1q,\param2().1d,\in().1d - pmull2 F1.1q,\param2().2d,\in().2d - pmull R.1q,\param1().1d,\in().1d - pmull2 R1.1q,\param1().2d,\in().2d + C PMUL_PARAM left us with \param1 and \param2 in native register order but + C \in is load-inverted from initial load of H using ldr, something must give +IF_BE(` + ext T.16b,\in().16b,\in().16b,#8 +',` + mov T.16b,\in().16b +') + pmull F.1q,\param2().1d,T.1d + pmull2 F1.1q,\param2().2d,T.2d + pmull R.1q,\param1().1d,T.1d + pmull2 R1.1q,\param1().2d,T.2d eor F.16b,F.16b,F1.16b eor R.16b,R.16b,R1.16b .endm

If we finally artificially restore the doubleword transposition in REDUCE for H2 and H3 we're all set for the next calls:

.macro REDUCTION out IF_BE(` - pmull T.1q,F.1d,POLY.1d ext \out().16b,F.16b,F.16b,#8 - eor R.16b,R.16b,T.16b - eor \out().16b,\out().16b,R.16b + pmull2 T.1q,\out().2d,POLY.2d ',` pmull T.1q,F.1d,POLY.1d +') eor R.16b,R.16b,T.16b ext R.16b,R.16b,R.16b,#8 eor \out().16b,F.16b,R.16b +C artificially restore load inversion for PMUL_PARAM :-( +IF_BE(` + ext \out().16b,\out().16b,\out().16b,#8 ') .endm

So all we're doing is catering to the quirk of the very first ldr operation. The easiest solution seems to me to align all types of load and store operations with each other or counteract their quirks right after or before executing them. That way we end up with identical register contents on LE and BE and don't have to maintain separate implementations.

That'd be in line with what we ended up with on arm32 NEON as well. memxor3.asm does do the dance of working with different register content but there it's only bitwise operations and the load and store operations have identical behaviour.

The advantage of the current implementation with transposed doublewords and only the LE routines seems to me that overhead on LE and BE would be about the same.

Do you think it makes sense to try and adjust the code to work with the BE layout natively and have a full 128bit reverse after ldr-like loads on LE instead (considering that 99,999% of aarch64 users will run LE)?

...

Otherwise, what's your error message from podman? It's got no deamon, so it shouldn't need a socket to connect to it like docker does. Out to the Internet for image download it's also a standard client and respects environment variables for proxies as usual.

I got Error: error creating network namespace for container. I think I can fix it by tracing the problem but let's try the other methods first as I think it's gonna be simpler for me..

I found this error on the Net in conjunction with a Debian/Ubuntu security-related custom kernel knob for disabling unprivileged user namespaces that was enabled by default once. I tested that with Ubuntu 18.04, 20.04 and 20.10 yesterday and it's disabled (i.e. namespaces for unprivileged users enabled) on all of them. You can still have a look at your setting in /proc/sys/kernel/unprivileged_userns_clone or with sysctl kernel.unprivileged_userns_clone. It needs to be set to 1 for rootless podman to work.

You're not by any chance running the Windows Subsystem for Linux (WSL)? https://github.com/containers/podman/issues/3288#issuecomment-501356136 :)

Or inside another container at a hosting service? https://github.com/containers/podman/issues/4056

Otherwise I have no idea what could be causing that and have never seen that error.

Hello Mamone,

On Fri, Jan 22, 2021 at 10:14:36PM +0200, Maamoun TK wrote:

...

The difference in index in dup EMSB nicely shows the doubleword transposition compared to LE. If on LE the dup was done after the rev64, it'd be H.b[7] vs. H.b[15].

I see what you did here, but I'm confused about ld1 and st1 instructions so let me clarify one thing before going on, how do ld1 and st1 load and store from/into memory in BE mode? If they perform in a normal way then there is no point of using ldr at all, I just used it because it handles imm offset. so to replace this line "ldr HQ,[TABLE,#16*H_Idx]" we can just add the offset to the register that hold the address "add x1,TABLE,#16*H_Idx" then

I've just retested and reread some ARM documents. Here's a patch that uses ld1.16b and thus eliminates almost all special BE treatment but subsequently has to leave in all the rev64s as well. This has the testsuite passing on BE and (still) LE. My take at an explanation below.

diff --git a/arm64/v8/gcm-hash.asm b/arm64/v8/gcm-hash.asm index 1c14db54..8c8a370e 100644 --- a/arm64/v8/gcm-hash.asm +++ b/arm64/v8/gcm-hash.asm @@ -55,17 +55,10 @@ C common macros: .endm

.macro REDUCTION out -IF_BE(` - pmull T.1q,F.1d,POLY.1d - ext \out().16b,F.16b,F.16b,#8 - eor R.16b,R.16b,T.16b - eor \out().16b,\out().16b,R.16b -',` pmull T.1q,F.1d,POLY.1d eor R.16b,R.16b,T.16b ext R.16b,R.16b,R.16b,#8 eor \out().16b,F.16b,R.16b -') .endm

C void gcm_init_key (union gcm_block *table) @@ -108,27 +101,20 @@ define(`H4M', `v29') define(`H4L', `v30')

.macro PMUL_PARAM in, param1, param2 -IF_BE(` - pmull2 Hp.1q,\in().2d,POLY.2d - ext Hm.16b,\in().16b,\in().16b,#8 - eor Hm.16b,Hm.16b,Hp.16b - zip \param1().2d,\in().2d,Hm.2d - zip2 \param2().2d,\in().2d,Hm.2d -',` pmull2 Hp.1q,\in().2d,POLY.2d eor Hm.16b,\in().16b,Hp.16b ext \param1().16b,Hm.16b,\in().16b,#8 ext \param2().16b,\in().16b,Hm.16b,#8 ext \param1().16b,\param1().16b,\param1().16b,#8 -') .endm

PROLOGUE(_nettle_gcm_init_key) - ldr HQ,[TABLE,#16*H_Idx] + C LSB vector load: x1+0 into H.b[0] and x1+15 into H.b[15] + add x1,TABLE,#16*H_Idx + ld1 {H.16b},[x1] dup EMSB.16b,H.b[0] -IF_LE(` + C treat H as two MSB doublewords rev64 H.16b,H.16b -') mov x1,#0xC200000000000000 mov x2,#1 mov POLY.d[0],x1 @@ -221,9 +207,7 @@ PROLOGUE(_nettle_gcm_hash) mov POLY.d[0],x4

ld1 {D.16b},[X] -IF_LE(` rev64 D.16b,D.16b -')

ands x4,LENGTH,#-64 b.eq L2x @@ -234,12 +218,10 @@ IF_LE(`

L4x_loop: ld1 {C0.16b,C1.16b,C2.16b,C3.16b},[DATA],#64 -IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b rev64 C2.16b,C2.16b rev64 C3.16b,C3.16b -')

eor C0.16b,C0.16b,D.16b

@@ -262,10 +244,8 @@ L2x: ld1 {H1M.16b,H1L.16b,H2M.16b,H2L.16b},[TABLE]

ld1 {C0.16b,C1.16b},[DATA],#32 -IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b -')

eor C0.16b,C0.16b,D.16b

@@ -283,9 +263,7 @@ L1x: ld1 {H1M.16b,H1L.16b},[TABLE]

ld1 {C0.16b},[DATA],#16 -IF_LE(` rev64 C0.16b,C0.16b -')

eor C0.16b,C0.16b,D.16b

@@ -335,9 +313,7 @@ Lmod_8_done: REDUCTION D

Ldone: -IF_LE(` rev64 D.16b,D.16b -') st1 {D.16b},[X] ret EPILOGUE(_nettle_gcm_hash)

My understanding is that ld1 and st1 are "single-element structure" operations. (Identical to vld1 in arm32 NEON we discussed recently for chacha and salsa2 asm.) That means they load a number of elements of a given type from consecutive memory locations into the corresponding vector register indices.

ld1 {v0.4s},[x0] would load four 32bit words from consecutive memory locations and put them into v0.s[0] through v0.s[3]. So x0+0..3 (bytes) would go into v0.s[0], x0+4..7 would to into v0.s[1] and so on. Endianness would apply to the internal byte order of the elements, so each word would be loaded MSB-first in BE-mode and LSB-first in LE-mode.

So, given memory content such as:

x0 + 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 byte 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

We should get on BE:

MSB LSB v0.s[0]: 0 1 2 3 v0.s[1]: 4 5 6 7 v0.s[2]: 8 9 10 11 v0.s[3]: 12 13 14 15

Or looked at as byte-vectors:

|v0.s[0]|v0.s[1]| v0.s[2] | v0.s[3] | v0.b[0] v0.b[15] v0.16b: 3 2 1 0 7 6 5 4 11 10 9 8 15 14 13 12

On LE we should get:

MSB LSB v0.d[0]: 3 2 1 0 v0.d[1]: 7 6 5 4 v0.d[2]: 11 10 9 8 v0.d[3]: 15 14 13 12

v0.b[0] v0.b[15] v0.16b: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

This was just meant as intro. I've not actually tested this. I hope I got it right and not just added to everyone's confusion (mine included). :/

Back to ld1.16b: This now loads a vector of 16 bytes consecutively. Since bytes have no endianness there will be no changes in order on either LE and BE modes. The register content will look the same on both:

x0 + 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 byte: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 v0.b[0] v0.b[15] v0.16b: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

So larger datatypes loaded that way should be stored little-endian in memory to make sense as e.g. .d[0] after such a load. Or we need to rev64 them.

load the H value by using ld1 "ld1 {H.16b},[x1]" in this way we can still have to deal with LE as transposed doublewords and with BE in normal way (not transposed doublewords or transposed quadword).

After sending my last email I realised that the doublewords aren't actually transposed with BE as such. They're just transposed compared to the original LE routine because the ldr instruction loads in completely reversed order in each mode and the LE routine does convert the internal byte order of the doublewords to BE but not the overall order of the 128bit quadword because it doesn't need to and regards them as a vector of two doublewords anyway.

ld1.16b doesn't change that at all. It just behaves the same on LE and BE. So we'll always load vectors of bytes. And it'll always be an LSB load. And if we want to treat them as big-endian doublewords we have to adjust them accordingly. That's why we now also need all the rev64s on BE above.

That opens another topic: As you may have noticed I haven't got the slightest idea of what the code is actually doing. Assembly also isn't my first language either. I'm only mechanically trying to get BE mode to produce the same results as LE.

This made me realise that I haven't the faintest idea what we're getting as input and producing as output either. :/ So are we working on blocks of bytes and producing blocks of bytes and just treating them as big-endian 64bit doublewords internally to exploit availability of instructions that can work on these types or could we actually declare the elements of TABLE to be quadwords in host endianness? Then we could actually throw ld1.2d at them and eliminate all the rev64s.

Duh, I think we can regardless, at least for BE:

diff --git a/arm64/v8/gcm-hash.asm b/arm64/v8/gcm-hash.asm index 1c14db54..642e3840 100644 --- a/arm64/v8/gcm-hash.asm +++ b/arm64/v8/gcm-hash.asm @@ -55,17 +55,10 @@ C common macros: .endm

.macro REDUCTION out -IF_BE(` - pmull T.1q,F.1d,POLY.1d - ext \out().16b,F.16b,F.16b,#8 - eor R.16b,R.16b,T.16b - eor \out().16b,\out().16b,R.16b -',` pmull T.1q,F.1d,POLY.1d eor R.16b,R.16b,T.16b ext R.16b,R.16b,R.16b,#8 eor \out().16b,F.16b,R.16b -') .endm

C void gcm_init_key (union gcm_block *table) @@ -108,27 +101,20 @@ define(`H4M', `v29') define(`H4L', `v30')

.macro PMUL_PARAM in, param1, param2 -IF_BE(` - pmull2 Hp.1q,\in().2d,POLY.2d - ext Hm.16b,\in().16b,\in().16b,#8 - eor Hm.16b,Hm.16b,Hp.16b - zip \param1().2d,\in().2d,Hm.2d - zip2 \param2().2d,\in().2d,Hm.2d -',` pmull2 Hp.1q,\in().2d,POLY.2d eor Hm.16b,\in().16b,Hp.16b ext \param1().16b,Hm.16b,\in().16b,#8 ext \param2().16b,\in().16b,Hm.16b,#8 ext \param1().16b,\param1().16b,\param1().16b,#8 -') .endm

PROLOGUE(_nettle_gcm_init_key) - ldr HQ,[TABLE,#16*H_Idx] - dup EMSB.16b,H.b[0] + add x1,TABLE,#16*H_Idx + ld1 {H.2d},[x1] IF_LE(` rev64 H.16b,H.16b ') + dup EMSB.16b,H.b[7] mov x1,#0xC200000000000000 mov x2,#1 mov POLY.d[0],x1 @@ -220,7 +206,7 @@ PROLOGUE(_nettle_gcm_hash) mov x4,#0xC200000000000000 mov POLY.d[0],x4

- ld1 {D.16b},[X] + ld1 {D.2d},[X] IF_LE(` rev64 D.16b,D.16b ') @@ -233,7 +219,7 @@ IF_LE(` ld1 {H3M.16b,H3L.16b,H4M.16b,H4L.16b},[x5]

L4x_loop: - ld1 {C0.16b,C1.16b,C2.16b,C3.16b},[DATA],#64 + ld1 {C0.2d,C1.2d,C2.2d,C3.2d},[DATA],#64 IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b @@ -261,7 +247,7 @@ L2x:

ld1 {H1M.16b,H1L.16b,H2M.16b,H2L.16b},[TABLE]

- ld1 {C0.16b,C1.16b},[DATA],#32 + ld1 {C0.2d,C1.2d},[DATA],#32 IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b @@ -282,7 +268,7 @@ L1x:

ld1 {H1M.16b,H1L.16b},[TABLE]

- ld1 {C0.16b},[DATA],#16 + ld1 {C0.2d},[DATA],#16 IF_LE(` rev64 C0.16b,C0.16b ') @@ -335,9 +321,7 @@ Lmod_8_done: REDUCTION D

Ldone: -IF_LE(` rev64 D.16b,D.16b -') st1 {D.16b},[X] ret EPILOGUE(_nettle_gcm_hash)

Please excuse my laboured and longwinded thinking. ;) I really have to start thinking in vectors also.

This also works for the whole TABLE and gives host-endianness storage there (where ld1.16b should have caused it to be little-endian before, if that's at all relevant):

diff --git a/arm64/v8/gcm-hash.asm b/arm64/v8/gcm-hash.asm index 1c14db54..bd6820b3 100644 --- a/arm64/v8/gcm-hash.asm +++ b/arm64/v8/gcm-hash.asm @@ -55,17 +55,10 @@ C common macros: .endm

.macro REDUCTION out -IF_BE(` - pmull T.1q,F.1d,POLY.1d - ext \out().16b,F.16b,F.16b,#8 - eor R.16b,R.16b,T.16b - eor \out().16b,\out().16b,R.16b -',` pmull T.1q,F.1d,POLY.1d eor R.16b,R.16b,T.16b ext R.16b,R.16b,R.16b,#8 eor \out().16b,F.16b,R.16b -') .endm

C void gcm_init_key (union gcm_block *table) @@ -108,27 +101,20 @@ define(`H4M', `v29') define(`H4L', `v30')

.macro PMUL_PARAM in, param1, param2 -IF_BE(` - pmull2 Hp.1q,\in().2d,POLY.2d - ext Hm.16b,\in().16b,\in().16b,#8 - eor Hm.16b,Hm.16b,Hp.16b - zip \param1().2d,\in().2d,Hm.2d - zip2 \param2().2d,\in().2d,Hm.2d -',` pmull2 Hp.1q,\in().2d,POLY.2d eor Hm.16b,\in().16b,Hp.16b ext \param1().16b,Hm.16b,\in().16b,#8 ext \param2().16b,\in().16b,Hm.16b,#8 ext \param1().16b,\param1().16b,\param1().16b,#8 -') .endm

PROLOGUE(_nettle_gcm_init_key) - ldr HQ,[TABLE,#16*H_Idx] - dup EMSB.16b,H.b[0] + add x1,TABLE,#16*H_Idx + ld1 {H.2d},[x1] IF_LE(` rev64 H.16b,H.16b ') + dup EMSB.16b,H.b[7] mov x1,#0xC200000000000000 mov x2,#1 mov POLY.d[0],x1 @@ -154,7 +140,7 @@ IF_LE(`

PMUL_PARAM H2,H2M,H2L

- st1 {H1M.16b,H1L.16b,H2M.16b,H2L.16b},[TABLE],#64 + st1 {H1M.2d,H1L.2d,H2M.2d,H2L.2d},[TABLE],#64

C --- calculate H^3 = H^1*H^2 ---

@@ -172,7 +158,7 @@ IF_LE(`

PMUL_PARAM H4,H4M,H4L

- st1 {H3M.16b,H3L.16b,H4M.16b,H4L.16b},[TABLE] + st1 {H3M.2d,H3L.2d,H4M.2d,H4L.2d},[TABLE]

ret EPILOGUE(_nettle_gcm_init_key) @@ -220,7 +206,7 @@ PROLOGUE(_nettle_gcm_hash) mov x4,#0xC200000000000000 mov POLY.d[0],x4

- ld1 {D.16b},[X] + ld1 {D.2d},[X] IF_LE(` rev64 D.16b,D.16b ') @@ -229,11 +215,11 @@ IF_LE(` b.eq L2x

add x5,TABLE,#64 - ld1 {H1M.16b,H1L.16b,H2M.16b,H2L.16b},[TABLE] - ld1 {H3M.16b,H3L.16b,H4M.16b,H4L.16b},[x5] + ld1 {H1M.2d,H1L.2d,H2M.2d,H2L.2d},[TABLE] + ld1 {H3M.2d,H3L.2d,H4M.2d,H4L.2d},[x5]

L4x_loop: - ld1 {C0.16b,C1.16b,C2.16b,C3.16b},[DATA],#64 + ld1 {C0.2d,C1.2d,C2.2d,C3.2d},[DATA],#64 IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b @@ -259,9 +245,9 @@ L2x: tst LENGTH,#-32 b.eq L1x

- ld1 {H1M.16b,H1L.16b,H2M.16b,H2L.16b},[TABLE] + ld1 {H1M.2d,H1L.2d,H2M.2d,H2L.2d},[TABLE]

- ld1 {C0.16b,C1.16b},[DATA],#32 + ld1 {C0.2d,C1.2d},[DATA],#32 IF_LE(` rev64 C0.16b,C0.16b rev64 C1.16b,C1.16b @@ -280,9 +266,9 @@ L1x: tst LENGTH,#-16 b.eq Lmod

- ld1 {H1M.16b,H1L.16b},[TABLE] + ld1 {H1M.2d,H1L.2d},[TABLE]

- ld1 {C0.16b},[DATA],#16 + ld1 {C0.2d},[DATA],#16 IF_LE(` rev64 C0.16b,C0.16b ') @@ -297,7 +283,7 @@ Lmod: tst LENGTH,#15 b.eq Ldone

- ld1 {H1M.16b,H1L.16b},[TABLE] + ld1 {H1M.2d,H1L.2d},[TABLE]

tbz LENGTH,3,Lmod_8 ldr C0D,[DATA],#8 @@ -338,6 +324,6 @@ Ldone: IF_LE(` rev64 D.16b,D.16b ') - st1 {D.16b},[X] + st1 {D.2d},[X] ret EPILOGUE(_nettle_gcm_hash)

And as always after all this guesswork I have found a likely very relevant comment in gcm.c:

/* Shift uses big-endian representation. */ #if WORDS_BIGENDIAN reduce = shift_table[x->u64[1] & 0xff];

Is that it? Or is TABLE just internal to the routine and we can store there however we please? (Apart from H at TABLE[128] initialised for us by gcm_set_key and stored BE?)

Hello Mamone,

On Sat, Jan 23, 2021 at 08:52:30PM +0200, Maamoun TK wrote:

...

@@ -280,9 +266,9 @@ L1x: tst LENGTH,#-16 b.eq Lmod

• ld1 {H1M.16b,H1L.16b},[TABLE]

• ld1 {H1M.2d,H1L.2d},[TABLE]

• ld1 {C0.16b},[DATA],#16

• ld1 {C0.2d},[DATA],#16

IF_LE(` rev64 C0.16b,C0.16b ')

behavior hence we can't get this patch working on BE mode. The core of

First off: All three patches from my previous mail had the test gcm-hash passing on LE and BE. I just reconfirmed the last patch with the whole testsuite on LE and BE. So they should be working and cause no regression.

I have one question here, do operations on doublewords transpose both doubleword parts in BE mode? for example pmull instruction transpose doublewords on LE mode when operated, in BE I don't expect the same behavior hence we can't get this patch working on BE mode. The core of pmull instruction is shift and xor operations so we can't perform pmull instruction on byte-reversed doublewords as it's gonna produce wrong results.

I think this directly corresponds to your next question:

Dealing with vector registers in aarch64 is really challenging, both x86_64 and PowerPC don't drag the endianness issues to vector registers, it's only applied to memory and once the data loaded from memory into vector register all endianness concerns are ended. Although PowerPC supports both endianness modes, AltiVec instructions operate the same on vector registers on both modes. It's a weird decision made by the Arm side.

I think there might be a misunderstanding here (possibly caused by my attemps at explaining what ldr does, sorry):

On arm(32) and aarch64, endianness is also exclusively handled on load and store operations. Register layout and operation behaviour is identical in both modes. I think ARM also speaks of "memory endianness" for just that reason. There is no adjustable "CPU endianness". It's always "CPU-native".

So pmull will behave exactly the same in BE and LE mode. We just have to make sure our load operations put the operands in the correct (i.e. CPU-native) representation into the correct vector register indices upon load.

So as an example:

pmull2 v0.1q,v1.2d,v2.2d

will always work on d[2] of v1 and v2 and put the result into all of v0. And it expects its operands there in exactly one format, i.e. the least significant bit at one end and the most-significant bit at the other (and it's the same ends/bits in both memory-endianness modes :). And it will also store to v0 in exactly the same representation in LE and BE mode. Nothing changes with an endianness mode switch.

That's where load and store come in:

ld1 {v1.2d,v2.2d},[x0]

will load v1 and v2 with one-dimensional vectors from memory. So v1.d[0] will be read from x0+0, v1.d[1] from x0+8 (bytes) and v2.d[0] from x0+16 and v2.d[1] from x0+24. That'll also be the same in LE and BE mode because that's the structure of the vector prescribed by the load operation we choose. Endianness will be applied to the individual doublewords but the order in which they're loaded from memory and in which they're put into d[0] and d[1] won't change, because they're vectors.

So if you've actually stored a vector from CPU registers using st1 {v1.2d, v2.2d},[x0] and then load them back using ld1 {v1.2d, v2.2d},[x0] there's nothing else that needs to be done. The individual bytes of the doublewords will be stored LE in memory in LE mode and BE in BE mode but you won't notice. And the order of the doublewords in memory will be the same in both modes.

If you're loading something that isn't stored LE or has no endianness at all, e.g. just a sequence of data bytes (as in DATA in our code) or something that was explicitly stored BE even on an LE CPU (as in TABLE[128] in our code, I gather) but want to treat it as a larger datatype, then you have to define endianness and need to apply correction. That's why we need to rev64 in one mode (e.g. LE) to get the same register-content in both endianness modes if what's loaded isn't actually stored in that endianness in memory.

Another way is to explicitly load a vector of bytes using ld1 {v1.16b, v2.16b},[x0]. Then you can be sure what you get as register content, no matter what memory endianness the CPU is using. If it's really just a sequence of data bytes stored in their correct and necessary order in memory and we only want to apply shifts and logical operations to each of them, we'd be all set.

Here we can also exploit but need to be careful to understand the different views on the register, so the fact that b[0] through b[7] is mapped to d[0] and that b[0] will be the least significant byte in d[0] and b[7] will be MSB. This layout is cpu-native, i.e. also the same in both endianness modes. It's just that an ld1 {v1.16b} will always load a vector of bytes with eight elements as consecutive bytes from memory into b[0] through b[7], so it'll always be an LSB-first load when interpreted as a larger data type. If we then look at that data trough d[0] it will appear reversed if it isn't really a doubleword that was stored little-endian.

That's why an ld1 {v1.b16,v2.b16},[x0] will produce incorrect results with a pmull2 v0.1q,v1.2d,v2.2d in at least one endianness because we're telling one operation that it's dealing with a byte-vector and the other expects us to provide a vector of doublewords. If what we're loading is actually something that was stored as doublewords in current memory endianness, then ld1 {v1.2d,v2.2d} is the correct load operation. If it's data bytes we want to *treat* as a big-endian doubleword, we can use either ld1 {v1.16b,v2.16b} or {v1.2d,v2.2d} but in both cases need to rev64 the register content if memory endianness is LE.

Now what *ldr* does is load a single 128bit quadword. And this will indeed transpose the doublewords in BE mode when looked at through d[0] and d[1]. Because as a big-endian load it will of course load the byte at x0 into the most significant byte of e.g. v2, i.e. v2.d[1], i.e. v2.b[15] and not v2.d[0], i.e. v2.b[7] (as with ld1.2d) or v2.b[0] (as with ld1.16b). Similarly, x0+15 will go into v2.b[0] in BE and v2.b[15] in LE mode. So this will only make sense if what we're loading was actually stored using str as a 128bit quadword in current memory endianness. If it's a sequence of bytes (st1.16b) we want to treat as a vector of doublewords, not only will the bytes appear inverted when looked at through d[0] and d[1] but also what's in d[0] will be in d[1] in the other endianness mode and vice-versa. If it's a vector of doublewords in memory endianness (st1.2d), byte order in the register will be correct in both modes (because it's different in memory) but d[0] and d[1] will still be transposed.

That's where all my rambling about doubleword transposition came from. Does that make sense?

I just found this document from the LLVM guys with pictures! :) https://llvm.org/docs/BigEndianNEON.html

BTW: ARM even goes as far as always storing *instructions* themselves, so the actual opcodes the CPU decodes and executes, little-endian, even in BE binaries. So the instruction fetch and decode stage always operates little-endian. When the instruction is executed it's then just an additional flag that tells load and store instructions how to behave when executed and accessing memory. (I'm actually extrapolation from what I know to be true for classic arm32 but it makes sense for that to be true for aarch64 as well.)

...

Please excuse my laboured and longwinded thinking. ;) I really have to start thinking in vectors also.

Actually, I'm impressed how you get and handle all these ideas in your mind and turn around quickly once you get a new one.

Uh, thanks, FWIW. :)

I think to gather you (same as me) prefer to think in big-endian representation. As for arm and aarch64, little-endian is the default, do you think, the routine could be changed to move the special endianness treatment using rev64 to BE mode, i.e. avoid them in the standard LE case? It's certainly beyond me but it might give some additional speedup.

Or would it be irrelevant compared to the speedup already given by using pmull in the first place?

...

@@ -335,9 +321,7 @@ Lmod_8_done: REDUCTION D

Ldone: -IF_LE(` rev64 D.16b,D.16b -') st1 {D.16b},[X] ret EPILOGUE(_nettle_gcm_hash)

I like your ideas so far as you're shrinking the gap between both endianness code but if my previous concern is right we still can't get this patch works too.

As said, the testsuite is passing with all three diffs from my previous mail.

[...] PASS: symbols PASS: dlopen ==================== All 110 tests passed ==================== make[1]: Leaving directory '/home/michael/build-aarch64_be/testsuite' Making check in examples make[1]: Entering directory '/home/michael/build-aarch64_be/examples' TEST_SHLIB_DIR="/home/michael/build-aarch64_be/.lib" \ srcdir="../../nettle/examples" EMULATOR="" EXEEXT="" \ "../../nettle"/run-tests rsa-sign-test rsa-verify-test rsa-encrypt-test xxxxxx xxxxxx PASS: rsa-sign PASS: rsa-verify PASS: rsa-encrypt ================== All 3 tests passed ================== make[1]: Leaving directory '/home/michael/build-aarch64_be/examples' [michael@aarch64-be:~/build-aarch64_be]

[...] PASS: symbols PASS: dlopen ==================== All 110 tests passed ==================== make[1]: Leaving directory '/home/michael/build-aarch64/testsuite' Making check in examples make[1]: Entering directory '/home/michael/build-aarch64/examples' TEST_SHLIB_DIR="/home/michael/build-aarch64/.lib" \ srcdir="../../nettle/examples" EMULATOR="" EXEEXT="" \ "../../nettle"/run-tests rsa-sign-test rsa-verify-test rsa-encrypt-test xxxxxx xxxxxx ee PASS: rsa-sign PASS: rsa-verify PASS: rsa-encrypt ================== All 3 tests passed ================== make[1]: Leaving directory '/home/michael/build-aarch64/examples' [michael@aarch64:~/build-aarch64]

...

And as always after all this guesswork I have found a likely very relevant comment in gcm.c:

/* Shift uses big-endian representation. */ #if WORDS_BIGENDIAN reduce = shift_table[x->u64[1] & 0xff];

Is that it? Or is TABLE just internal to the routine and we can store there however we please? (Apart from H at TABLE[128] initialised for us by gcm_set_key and stored BE?)

The assembly implementation of GHASH has a whole different scheme from C table-lookup implementation, you don't have to worry about any of that.

Perfect. So whether we use ld1/st1.16b or ld1/st1.2d for TABLE doesn't matter. I wouldn't expect it but we could benchmark whether one is faster than the other though!?

For clarification: How is H, i.e. TABLE[128] defined an interface to gcm_set_key? I see that gcm_set_key calls a cipher function to fill it. So I guess it provides the routine with a sequence of bytes (similar to DATA), i.e. the key, which will be the same on LE and BE and we *treat* it as a big-endian doubleword for the sake of using pmull on it. Correct?