[Bug target/103571] ABI: V2HF, V4HF and V8HFmode argument passing issues

[ubizjak at gmail dot com via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103571

--- Comment #7 from Uroš Bizjak <ubizjak at gmail dot com> ---
Created attachment 51941
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51941&action=edit
Proposed patch

The patch moves put V2HF+V4HF+V8HF/V16HF/V32HF TO
VALID_SSE2/AVX256/AVX512F_REG_MODE.

Also, introduces VALID_AVX512FP16_SCALAR_MODE to simplify some code.

(Probably we need to add V2HFmode to VALID_INT_MODE_P, but nevertheless the
patch fixes all the issues from the description):

64-bit targets:

foo:
        movl    %esi, %eax
        ret

bar:
        movaps  %xmm1, %xmm0
        ret

baz:
        movdqa  %xmm1, %xmm0
        ret

and for 32-bit targets:

foo:
        movl    8(%esp), %eax
        ret

bar:
        movq    %mm1, %mm0
        ret

baz:
        movdqa  %xmm1, %xmm0
        ret

The patch "regresses" 32bit testsuite:

FAIL: gcc.target/i386/pr102812.c scan-assembler movdqa

but only due to better generated code:

        pxor    %xmm0, %xmm0
        pinsrw  $0, 4(%esp), %xmm0
        ret

vs. the above demonstrated mess.