RE: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Passed the x86 bootstrap and regression tests. Pan -Original Message- From: Juzhe-Zhong Sent: Tuesday, October 31, 2023 5:59 PM To: gcc-patches@gcc.gnu.org Cc: rguent...@suse.de; jeffreya...@gmail.com; richard.sandif...@arm.com; rdapp@gmail.com; Juzhe-Zhong Subject: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN As previous Richard's suggested, we should support strided load/store in loop vectorizer instead hacking RISC-V backend. This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but with changing vector offset into scalar stride. We don't have strided_load/strided_store and mask_strided_load/mask_strided_store since it't unlikely RVV will have such optabs and we can't add the patterns that we can't test them. gcc/ChangeLog: * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. * internal-fn.cc (internal_load_fn_p): Ditto. (internal_strided_fn_p): Ditto. (internal_fn_len_index): Ditto. (internal_fn_mask_index): Ditto. (internal_fn_stored_value_index): Ditto. (internal_strided_fn_supported_p): Ditto. * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. (MASK_LEN_STRIDED_STORE): Ditto. * internal-fn.h (internal_strided_fn_p): Ditto. (internal_strided_fn_supported_p): Ditto. * optabs.def (OPTAB_CD): Ditto. --- gcc/doc/md.texi | 51 + gcc/internal-fn.cc | 44 ++ gcc/internal-fn.def | 4 gcc/internal-fn.h | 2 ++ gcc/optabs.def | 2 ++ 5 files changed, 103 insertions(+) diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index fab2513105a..5bac713a0dd 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -5094,6 +5094,32 @@ Bit @var{i} of the mask is set if element @var{i} of the result should be loaded from memory and clear if element @var{i} of the result should be undefined. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_load@var{m}@var{n}} instruction pattern +@item @samp{mask_len_strided_load@var{m}@var{n}} +Load several separate memory locations into a destination vector of mode @var{m}. +Operand 0 is a destination vector of mode @var{m}. +Operand 1 is a scalar base address and operand 2 is a scalar stride of mode @var{n}. +The instruction can be seen as a special case of @code{mask_len_gather_load@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 2 as step. +For each element index i: + +@itemize @bullet +@item +extend the stride to address width, using zero +extension if operand 3 is 1 and sign extension if operand 3 is zero; +@item +multiply the extended stride by operand 4; +@item +add the result to the base; and +@item +load the value at that address (operand 1 + @var{i} * multiplied and extended stride) into element @var{i} of operand 0. +@end itemize + +Similar to mask_len_load, the instruction loads at most (operand 6 + operand 7) elements from memory. +Bit @var{i} of the mask is set if element @var{i} of the result should +be loaded from memory and clear if element @var{i} of the result should be undefined. +Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. + @cindex @code{scatter_store@var{m}@var{n}} instruction pattern @item @samp{scatter_store@var{m}@var{n}} Store a vector of mode @var{m} into several distinct memory locations. @@ -5131,6 +5157,31 @@ at most (operand 6 + operand 7) elements of (operand 4) to memory. Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be stored. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_store@var{m}@var{n}} instruction pattern +@item @samp{mask_len_strided_store@var{m}@var{n}} +Store a vector of mode m into several distinct memory locations. +Operand 0 is a scalar base address and operand 1 is scalar stride of mode @var{n}. +Operand 2 is the vector of values that should be stored, which is of mode @var{m}. +The instruction can be seen as a special case of @code{mask_len_scatter_store@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 2 as step. +For each element index i: + +@itemize @bullet +@item +extend the stride to address width, using zero +extension if operand 2 is 1 and sign extension if operand 2 is zero; +@item +multiply the extended stride by operand 3; +@item +add the result to the base; and +@item +store element @var{i} of operand 4 to that address (operand 1 + @var{i} * multiplied and extended stride). +@end itemize + +Similar to mask_len_store, the instruction stores at most (operand 6 + operand 7) elements of (operand 4) to memory. +Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be stored. +Mask e
Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Tue, 31 Oct 2023, Juzhe-Zhong wrote: > As previous Richard's suggested, we should support strided load/store in > loop vectorizer instead hacking RISC-V backend. > > This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. > > The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but with > changing vector offset into scalar stride. I see that it follows gather/scatter. I'll note that when introducing those we failed to add a specifier for TBAA and alignment info for the data access. That means we have to use alias-set zero for the accesses (I see existing targets use UNSPECs with some not elaborate MEM anyway, but TBAA info might have been the "easy" and obvious property to preserve). For alignment we either have to assume unaligned or reject vectorization of accesses that do not have their original scalar accesses naturally aligned (aligned according to their mode). We don't seem to check that though. It might be fine to go forward with this since gather/scatter are broken in a similar way. Do we really need to have two modes for the optab though or could we simply require the target to support arbitrary offset modes (give it is implicitly constrained to ptr_mode for the base already)? Or properly extend/truncate the offset at expansion time, say to ptr_mode or to the mode of sizetype. Thanks, Richard. > We don't have strided_load/strided_store and > mask_strided_load/mask_strided_store since > it't unlikely RVV will have such optabs and we can't add the patterns that we > can't test them. > > > gcc/ChangeLog: > > * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. > * internal-fn.cc (internal_load_fn_p): Ditto. > (internal_strided_fn_p): Ditto. > (internal_fn_len_index): Ditto. > (internal_fn_mask_index): Ditto. > (internal_fn_stored_value_index): Ditto. > (internal_strided_fn_supported_p): Ditto. > * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. > (MASK_LEN_STRIDED_STORE): Ditto. > * internal-fn.h (internal_strided_fn_p): Ditto. > (internal_strided_fn_supported_p): Ditto. > * optabs.def (OPTAB_CD): Ditto. > > --- > gcc/doc/md.texi | 51 + > gcc/internal-fn.cc | 44 ++ > gcc/internal-fn.def | 4 > gcc/internal-fn.h | 2 ++ > gcc/optabs.def | 2 ++ > 5 files changed, 103 insertions(+) > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..5bac713a0dd 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,32 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}@var{n}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}@var{n}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of mode > @var{n}. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i: > + > +@itemize @bullet > +@item > +extend the stride to address width, using zero > +extension if operand 3 is 1 and sign extension if operand 3 is zero; > +@item > +multiply the extended stride by operand 4; > +@item > +add the result to the base; and > +@item > +load the value at that address (operand 1 + @var{i} * multiplied and > extended stride) into element @var{i} of operand 0. > +@end itemize > + > +Similar to mask_len_load, the instruction loads at most (operand 6 + operand > 7) elements from memory. > +Bit @var{i} of the mask is set if element @var{i} of the result should > +be loaded from memory and clear if element @var{i} of the result should be > undefined. > +Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > + > @cindex @code{scatter_store@var{m}@var{n}} instruction pattern > @item @samp{scatter_store@var{m}@var{n}} > Store a vector of mode @var{m} into several distinct memory locations. > @@ -5131,6 +5157,31 @@ at most (operand 6 + operand 7) elements of (operand > 4) to memory. > Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be > stored. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_store@var{m}@var{n}} instruction pattern > +@item @samp{mask_len_strided_store@var{m}@var{n}} > +Store a vector of mode m into several distinct memory locations. > +Operand 0 is a scalar base address and operand 1 is scalar stride of mode > @var{n}. > +Operand 2 is the vector of values that should b
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Ok. So drop 'scale' and keep signed/unsigned argument, is that right? And I wonder I should create the stride_type using size_type_node or ptrdiff_type_node ? Which is preferrable ? Thanks. juzhe.zh...@rivai.ai From: Richard Biener Date: 2023-11-02 22:27 To: 钟居哲 CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN On Thu, 2 Nov 2023, ??? wrote: > Hi, Richi. > > >> Do we really need to have two modes for the optab though or could we > >> simply require the target to support arbitrary offset modes (give it > >> is implicitly constrained to ptr_mode for the base already)? Or > >> properly extend/truncate the offset at expansion time, say to ptr_mode > >> or to the mode of sizetype. > > For RVV, it's ok by default set stride type as ptr_mode/size_type by default. > Is it ok that I define strided load/store as single mode optab and default > Pmode as stride operand? > How about scale and signed/unsigned operand ? > It seems scale operand can be removed ? Since we can pass DR_STEP directly to > the stride arguments. > But I think we can't remove signed/unsigned operand since for strided mode = > SI mode, the unsigned > maximum stride = 2^31 wheras signed is 2 ^ 30. On the GIMPLE side I think we want to have a sizetype operand and indeed drop 'scale', the sizetype operand should be readily available. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 21:52 > To: Juzhe-Zhong > CC: gcc-patches; jeffreyalaw; richard.sandiford; rdapp.gcc > Subject: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Tue, 31 Oct 2023, Juzhe-Zhong wrote: > > > As previous Richard's suggested, we should support strided load/store in > > loop vectorizer instead hacking RISC-V backend. > > > > This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. > > > > The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but > > with > > changing vector offset into scalar stride. > > I see that it follows gather/scatter. I'll note that when introducing > those we failed to add a specifier for TBAA and alignment info for the > data access. That means we have to use alias-set zero for the accesses > (I see existing targets use UNSPECs with some not elaborate MEM anyway, > but TBAA info might have been the "easy" and obvious property to > preserve). For alignment we either have to assume unaligned or reject > vectorization of accesses that do not have their original scalar accesses > naturally aligned (aligned according to their mode). We don't seem > to check that though. > > It might be fine to go forward with this since gather/scatter are broken > in a similar way. > > Do we really need to have two modes for the optab though or could we > simply require the target to support arbitrary offset modes (give it > is implicitly constrained to ptr_mode for the base already)? Or > properly extend/truncate the offset at expansion time, say to ptr_mode > or to the mode of sizetype. > > Thanks, > Richard. > > We don't have strided_load/strided_store and > > mask_strided_load/mask_strided_store since > > it't unlikely RVV will have such optabs and we can't add the patterns that > > we can't test them. > > > > > > gcc/ChangeLog: > > > > * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. > > * internal-fn.cc (internal_load_fn_p): Ditto. > > (internal_strided_fn_p): Ditto. > > (internal_fn_len_index): Ditto. > > (internal_fn_mask_index): Ditto. > > (internal_fn_stored_value_index): Ditto. > > (internal_strided_fn_supported_p): Ditto. > > * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. > > (MASK_LEN_STRIDED_STORE): Ditto. > > * internal-fn.h (internal_strided_fn_p): Ditto. > > (internal_strided_fn_supported_p): Ditto. > > * optabs.def (OPTAB_CD): Ditto. > > > > --- > > gcc/doc/md.texi | 51 + > > gcc/internal-fn.cc | 44 ++ > > gcc/internal-fn.def | 4 > > gcc/internal-fn.h | 2 ++ > > gcc/optabs.def | 2 ++ > > 5 files changed, 103 insertions(+) > > > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > > index fab2513105a..5bac713a0dd 100644 > > --- a/gcc/doc/md.texi > > +++ b/gcc/doc/md.texi > > @@ -5094,6 +5094,32 @@ Bit @var{i} of the mask is set if element @var{i} of > > the result should > > be l
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Hi, Richi. >> Do we really need to have two modes for the optab though or could we >> simply require the target to support arbitrary offset modes (give it >> is implicitly constrained to ptr_mode for the base already)? Or >> properly extend/truncate the offset at expansion time, say to ptr_mode >> or to the mode of sizetype. For RVV, it's ok by default set stride type as ptr_mode/size_type by default. Is it ok that I define strided load/store as single mode optab and default Pmode as stride operand? How about scale and signed/unsigned operand ? It seems scale operand can be removed ? Since we can pass DR_STEP directly to the stride arguments. But I think we can't remove signed/unsigned operand since for strided mode = SI mode, the unsigned maximum stride = 2^31 wheras signed is 2 ^ 30. juzhe.zh...@rivai.ai From: Richard Biener Date: 2023-11-02 21:52 To: Juzhe-Zhong CC: gcc-patches; jeffreyalaw; richard.sandiford; rdapp.gcc Subject: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN On Tue, 31 Oct 2023, Juzhe-Zhong wrote: > As previous Richard's suggested, we should support strided load/store in > loop vectorizer instead hacking RISC-V backend. > > This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. > > The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but with > changing vector offset into scalar stride. I see that it follows gather/scatter. I'll note that when introducing those we failed to add a specifier for TBAA and alignment info for the data access. That means we have to use alias-set zero for the accesses (I see existing targets use UNSPECs with some not elaborate MEM anyway, but TBAA info might have been the "easy" and obvious property to preserve). For alignment we either have to assume unaligned or reject vectorization of accesses that do not have their original scalar accesses naturally aligned (aligned according to their mode). We don't seem to check that though. It might be fine to go forward with this since gather/scatter are broken in a similar way. Do we really need to have two modes for the optab though or could we simply require the target to support arbitrary offset modes (give it is implicitly constrained to ptr_mode for the base already)? Or properly extend/truncate the offset at expansion time, say to ptr_mode or to the mode of sizetype. Thanks, Richard. > We don't have strided_load/strided_store and > mask_strided_load/mask_strided_store since > it't unlikely RVV will have such optabs and we can't add the patterns that we > can't test them. > > > gcc/ChangeLog: > > * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. > * internal-fn.cc (internal_load_fn_p): Ditto. > (internal_strided_fn_p): Ditto. > (internal_fn_len_index): Ditto. > (internal_fn_mask_index): Ditto. > (internal_fn_stored_value_index): Ditto. > (internal_strided_fn_supported_p): Ditto. > * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. > (MASK_LEN_STRIDED_STORE): Ditto. > * internal-fn.h (internal_strided_fn_p): Ditto. > (internal_strided_fn_supported_p): Ditto. > * optabs.def (OPTAB_CD): Ditto. > > --- > gcc/doc/md.texi | 51 + > gcc/internal-fn.cc | 44 ++ > gcc/internal-fn.def | 4 > gcc/internal-fn.h | 2 ++ > gcc/optabs.def | 2 ++ > 5 files changed, 103 insertions(+) > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..5bac713a0dd 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,32 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}@var{n}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}@var{n}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of mode > @var{n}. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i: > + > +@itemize @bullet > +@item > +extend the stride to address width, using zero > +extension if operand 3 is 1 and sign extension if operand 3 is zero; > +@item > +multiply the extended stride by operand 4; > +@item > +add the result to the base; and > +@item > +load the
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Thu, 2 Nov 2023, ??? wrote: > Ok. So drop 'scale' and keep signed/unsigned argument, is that right? I don't think we need signed/unsigned. RTL expansion has the signedness of the offset argument there and can just extend to the appropriate mode to offset a pointer. > And I wonder I should create the stride_type using size_type_node or > ptrdiff_type_node ? > Which is preferrable ? 'sizetype' - that's the type we require to be used for the POINTER_PLUS_EXPR offset operand. > Thanks. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 22:27 > To: ??? > CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Thu, 2 Nov 2023, ??? wrote: > > > Hi, Richi. > > > > >> Do we really need to have two modes for the optab though or could we > > >> simply require the target to support arbitrary offset modes (give it > > >> is implicitly constrained to ptr_mode for the base already)? Or > > >> properly extend/truncate the offset at expansion time, say to ptr_mode > > >> or to the mode of sizetype. > > > > For RVV, it's ok by default set stride type as ptr_mode/size_type by > > default. > > Is it ok that I define strided load/store as single mode optab and default > > Pmode as stride operand? > > How about scale and signed/unsigned operand ? > > It seems scale operand can be removed ? Since we can pass DR_STEP directly > > to the stride arguments. > > But I think we can't remove signed/unsigned operand since for strided mode > > = SI mode, the unsigned > > maximum stride = 2^31 wheras signed is 2 ^ 30. > > On the GIMPLE side I think we want to have a sizetype operand and > indeed drop 'scale', the sizetype operand should be readily available. > > > > > > > > > juzhe.zh...@rivai.ai > > > > From: Richard Biener > > Date: 2023-11-02 21:52 > > To: Juzhe-Zhong > > CC: gcc-patches; jeffreyalaw; richard.sandiford; rdapp.gcc > > Subject: Re: [PATCH V2] OPTABS/IFN: Add > > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > > On Tue, 31 Oct 2023, Juzhe-Zhong wrote: > > > > > As previous Richard's suggested, we should support strided load/store in > > > loop vectorizer instead hacking RISC-V backend. > > > > > > This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. > > > > > > The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but > > > with > > > changing vector offset into scalar stride. > > > > I see that it follows gather/scatter. I'll note that when introducing > > those we failed to add a specifier for TBAA and alignment info for the > > data access. That means we have to use alias-set zero for the accesses > > (I see existing targets use UNSPECs with some not elaborate MEM anyway, > > but TBAA info might have been the "easy" and obvious property to > > preserve). For alignment we either have to assume unaligned or reject > > vectorization of accesses that do not have their original scalar accesses > > naturally aligned (aligned according to their mode). We don't seem > > to check that though. > > > > It might be fine to go forward with this since gather/scatter are broken > > in a similar way. > > > > Do we really need to have two modes for the optab though or could we > > simply require the target to support arbitrary offset modes (give it > > is implicitly constrained to ptr_mode for the base already)? Or > > properly extend/truncate the offset at expansion time, say to ptr_mode > > or to the mode of sizetype. > > > > Thanks, > > Richard. > > > We don't have strided_load/strided_store and > > > mask_strided_load/mask_strided_store since > > > it't unlikely RVV will have such optabs and we can't add the patterns > > > that we can't test them. > > > > > > > > > gcc/ChangeLog: > > > > > > * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. > > > * internal-fn.cc (internal_load_fn_p): Ditto. > > > (internal_strided_fn_p): Ditto. > > > (internal_fn_len_index): Ditto. > > > (internal_fn_mask_index): Ditto. > > > (internal_fn_stored_value_index): Ditto. > > > (internal_strided_fn_supported_p): Ditto. > > > * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. > > > (MASK_LEN_STRIDED_ST
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Thu, 2 Nov 2023, ??? wrote: > Hi, Richi. > > >> Do we really need to have two modes for the optab though or could we > >> simply require the target to support arbitrary offset modes (give it > >> is implicitly constrained to ptr_mode for the base already)? Or > >> properly extend/truncate the offset at expansion time, say to ptr_mode > >> or to the mode of sizetype. > > For RVV, it's ok by default set stride type as ptr_mode/size_type by default. > Is it ok that I define strided load/store as single mode optab and default > Pmode as stride operand? > How about scale and signed/unsigned operand ? > It seems scale operand can be removed ? Since we can pass DR_STEP directly to > the stride arguments. > But I think we can't remove signed/unsigned operand since for strided mode = > SI mode, the unsigned > maximum stride = 2^31 wheras signed is 2 ^ 30. On the GIMPLE side I think we want to have a sizetype operand and indeed drop 'scale', the sizetype operand should be readily available. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 21:52 > To: Juzhe-Zhong > CC: gcc-patches; jeffreyalaw; richard.sandiford; rdapp.gcc > Subject: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Tue, 31 Oct 2023, Juzhe-Zhong wrote: > > > As previous Richard's suggested, we should support strided load/store in > > loop vectorizer instead hacking RISC-V backend. > > > > This patch adds MASK_LEN_STRIDED LOAD/STORE OPTABS/IFN. > > > > The GIMPLE IR is same as mask_len_gather_load/mask_len_scatter_store but > > with > > changing vector offset into scalar stride. > > I see that it follows gather/scatter. I'll note that when introducing > those we failed to add a specifier for TBAA and alignment info for the > data access. That means we have to use alias-set zero for the accesses > (I see existing targets use UNSPECs with some not elaborate MEM anyway, > but TBAA info might have been the "easy" and obvious property to > preserve). For alignment we either have to assume unaligned or reject > vectorization of accesses that do not have their original scalar accesses > naturally aligned (aligned according to their mode). We don't seem > to check that though. > > It might be fine to go forward with this since gather/scatter are broken > in a similar way. > > Do we really need to have two modes for the optab though or could we > simply require the target to support arbitrary offset modes (give it > is implicitly constrained to ptr_mode for the base already)? Or > properly extend/truncate the offset at expansion time, say to ptr_mode > or to the mode of sizetype. > > Thanks, > Richard. > > We don't have strided_load/strided_store and > > mask_strided_load/mask_strided_store since > > it't unlikely RVV will have such optabs and we can't add the patterns that > > we can't test them. > > > > > > gcc/ChangeLog: > > > > * doc/md.texi: Add mask_len_strided_load/mask_len_strided_store. > > * internal-fn.cc (internal_load_fn_p): Ditto. > > (internal_strided_fn_p): Ditto. > > (internal_fn_len_index): Ditto. > > (internal_fn_mask_index): Ditto. > > (internal_fn_stored_value_index): Ditto. > > (internal_strided_fn_supported_p): Ditto. > > * internal-fn.def (MASK_LEN_STRIDED_LOAD): Ditto. > > (MASK_LEN_STRIDED_STORE): Ditto. > > * internal-fn.h (internal_strided_fn_p): Ditto. > > (internal_strided_fn_supported_p): Ditto. > > * optabs.def (OPTAB_CD): Ditto. > > > > --- > > gcc/doc/md.texi | 51 + > > gcc/internal-fn.cc | 44 ++ > > gcc/internal-fn.def | 4 > > gcc/internal-fn.h | 2 ++ > > gcc/optabs.def | 2 ++ > > 5 files changed, 103 insertions(+) > > > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > > index fab2513105a..5bac713a0dd 100644 > > --- a/gcc/doc/md.texi > > +++ b/gcc/doc/md.texi > > @@ -5094,6 +5094,32 @@ Bit @var{i} of the mask is set if element @var{i} of > > the result should > > be loaded from memory and clear if element @var{i} of the result should be > > undefined. > > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > > > +@cindex @code{mask_len_strided_load@var{m}@var{n}} instruction pattern > > +@item @samp{mask_len_strided_load@var{m}@var{n}} > > +Load several separate memory locations into a destination vector of mode > > @var{m}. >
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Hi, Richi. The following is strided load/store doc: diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index fab2513105a..4f0821a291d 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of the result should be loaded from memory and clear if element @var{i} of the result should be undefined. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_load@var{m}} instruction pattern +@item @samp{mask_len_strided_load@var{m}} +Load several separate memory locations into a destination vector of mode @var{m}. +Operand 0 is a destination vector of mode @var{m}. +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. +operand 3 indicates stride operand is signed or unsigned number and operand 4 is mask operand. +operand 5 is length operand and operand 6 is bias operand. +The instruction can be seen as a special case of @code{mask_len_gather_load@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 2 as step. +For each element index i load address is operand 1 + @var{i} * operand 2. +operand 2 can be negative stride if operand 3 is 0. +Similar to mask_len_load, the instruction loads at most (operand 5 + operand 6) elements from memory. +Bit @var{i} of the mask (operand 4) is set if element @var{i} of the result should +be loaded from memory and clear if element @var{i} of the result should be undefined. +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. + @cindex @code{scatter_store@var{m}@var{n}} instruction pattern @item @samp{scatter_store@var{m}@var{n}} Store a vector of mode @var{m} into several distinct memory locations. @@ -5131,6 +5147,21 @@ at most (operand 6 + operand 7) elements of (operand 4) to memory. Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be stored. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_store@var{m}} instruction pattern +@item @samp{mask_len_strided_store@var{m}} +Store a vector of mode m into several distinct memory locations. +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. +operand 2 indicates stride operand is signed or unsigned number. +Operand 3 is the vector of values that should be stored, which is of mode @var{m}. +operand 4 is mask operand, operand 5 is length operand and operand 6 is bias operand. +The instruction can be seen as a special case of @code{mask_len_scatter_store@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 1 as step. +For each element index i store address is operand 0 + @var{i} * operand 1. +operand 1 can be negative stride if operand 2 is 0. +Similar to mask_len_store, the instruction stores at most (operand 5 + operand 6) elements of mask (operand 4) to memory. +Bit @var{i} of the mask is set if element @var{i} of (operand 3) should be stored. +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. + @cindex @code{vec_set@var{m}} instruction pattern @item @samp{vec_set@var{m}} Set given field in the vector value. Operand 0 is the vector to modify, Does it look reasonable ? Thanks. juzhe.zh...@rivai.ai From: Richard Biener Date: 2023-11-02 22:34 To: 钟居哲 CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN On Thu, 2 Nov 2023, ??? wrote: > Ok. So drop 'scale' and keep signed/unsigned argument, is that right? I don't think we need signed/unsigned. RTL expansion has the signedness of the offset argument there and can just extend to the appropriate mode to offset a pointer. > And I wonder I should create the stride_type using size_type_node or > ptrdiff_type_node ? > Which is preferrable ? 'sizetype' - that's the type we require to be used for the POINTER_PLUS_EXPR offset operand. > Thanks. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 22:27 > To: ??? > CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Thu, 2 Nov 2023, ??? wrote: > > > Hi, Richi. > > > > >> Do we really need to have two modes for the optab though or could we > > >> simply require the target to support arbitrary offset modes (give it > > >> is implicitly constrained to ptr_mode for the base already)? Or > > >> properly extend/truncate the offset at expansion time, say to ptr_mode > > >> or to the mode of sizetype. > > > > For RVV, it's ok by default set stride type as ptr_mode/size_type by > > default. > >
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > Hi, Richi. > > The following is strided load/store doc: > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..4f0821a291d 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. > +operand 3 indicates stride operand is signed or unsigned number and operand > 4 is mask operand. we don't need operand 3 > +operand 5 is length operand and operand 6 is bias operand. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i load address is operand 1 + @var{i} * operand 2. > +operand 2 can be negative stride if operand 3 is 0. > +Similar to mask_len_load, the instruction loads at most (operand 5 + operand > 6) elements from memory. > +Bit @var{i} of the mask (operand 4) is set if element @var{i} of the result > should Element @var{i}. I think we don't want to document 'undefined' but rather match mask_gather_load and say the result should be zero. Similar adjustments below. > +be loaded from memory and clear if element @var{i} of the result should be > undefined. > +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. > + > @cindex @code{scatter_store@var{m}@var{n}} instruction pattern > @item @samp{scatter_store@var{m}@var{n}} > Store a vector of mode @var{m} into several distinct memory locations. > @@ -5131,6 +5147,21 @@ at most (operand 6 + operand 7) elements of (operand > 4) to memory. > Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be > stored. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_store@var{m}} instruction pattern > +@item @samp{mask_len_strided_store@var{m}} > +Store a vector of mode m into several distinct memory locations. > +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. > +operand 2 indicates stride operand is signed or unsigned number. > +Operand 3 is the vector of values that should be stored, which is of mode > @var{m}. > +operand 4 is mask operand, operand 5 is length operand and operand 6 is bias > operand. > +The instruction can be seen as a special case of > @code{mask_len_scatter_store@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 1 as step. > +For each element index i store address is operand 0 + @var{i} * operand 1. > +operand 1 can be negative stride if operand 2 is 0. > +Similar to mask_len_store, the instruction stores at most (operand 5 + > operand 6) elements of mask (operand 4) to memory. > +Bit @var{i} of the mask is set if element @var{i} of (operand 3) should be > stored. > +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. > + > @cindex @code{vec_set@var{m}} instruction pattern > @item @samp{vec_set@var{m}} > Set given field in the vector value. Operand 0 is the vector to modify, > > Does it look reasonable ? Thanks. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 22:34 > To: ??? > CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Thu, 2 Nov 2023, ??? wrote: > > > Ok. So drop 'scale' and keep signed/unsigned argument, is that right? > > I don't think we need signed/unsigned. RTL expansion has the signedness > of the offset argument there and can just extend to the appropriate mode > to offset a pointer. > > > And I wonder I should create the stride_type using size_type_node or > > ptrdiff_type_node ? > > Which is preferrable ? > > 'sizetype' - that's the type we require to be used for > the POINTER_PLUS_EXPR offset operand. > > > > Thanks. > > > > > > > > juzhe.zh...@rivai.ai > > > > From: Richard Biener > > Date:
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Hi,Richi. >> Element @var{i}. I think we don't want to document 'undefined' but >> rather match mask_gather_load and say the result should be zero. I am confused here, will vectorizer depend on zero inactive value ? Since RVV load/store doesn't touch the inactive(tail or unmasked) elements, I am wondering whether it will cause issues. juzhe.zh...@rivai.ai From: Richard Biener Date: 2023-11-03 15:40 To: juzhe.zh...@rivai.ai CC: gcc-patches; jeffreyalaw; richard.sandiford; Robin Dapp Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > Hi, Richi. > > The following is strided load/store doc: > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..4f0821a291d 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. > +operand 3 indicates stride operand is signed or unsigned number and operand > 4 is mask operand. we don't need operand 3 > +operand 5 is length operand and operand 6 is bias operand. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i load address is operand 1 + @var{i} * operand 2. > +operand 2 can be negative stride if operand 3 is 0. > +Similar to mask_len_load, the instruction loads at most (operand 5 + operand > 6) elements from memory. > +Bit @var{i} of the mask (operand 4) is set if element @var{i} of the result > should Element @var{i}. I think we don't want to document 'undefined' but rather match mask_gather_load and say the result should be zero. Similar adjustments below. > +be loaded from memory and clear if element @var{i} of the result should be > undefined. > +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. > + > @cindex @code{scatter_store@var{m}@var{n}} instruction pattern > @item @samp{scatter_store@var{m}@var{n}} > Store a vector of mode @var{m} into several distinct memory locations. > @@ -5131,6 +5147,21 @@ at most (operand 6 + operand 7) elements of (operand > 4) to memory. > Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be > stored. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_store@var{m}} instruction pattern > +@item @samp{mask_len_strided_store@var{m}} > +Store a vector of mode m into several distinct memory locations. > +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. > +operand 2 indicates stride operand is signed or unsigned number. > +Operand 3 is the vector of values that should be stored, which is of mode > @var{m}. > +operand 4 is mask operand, operand 5 is length operand and operand 6 is bias > operand. > +The instruction can be seen as a special case of > @code{mask_len_scatter_store@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 1 as step. > +For each element index i store address is operand 0 + @var{i} * operand 1. > +operand 1 can be negative stride if operand 2 is 0. > +Similar to mask_len_store, the instruction stores at most (operand 5 + > operand 6) elements of mask (operand 4) to memory. > +Bit @var{i} of the mask is set if element @var{i} of (operand 3) should be > stored. > +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. > + > @cindex @code{vec_set@var{m}} instruction pattern > @item @samp{vec_set@var{m}} > Set given field in the vector value. Operand 0 is the vector to modify, > > Does it look reasonable ? Thanks. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-02 22:34 > To: ??? > CC: gcc-patches; Jeff Law; richard.sandiford; rdapp.gcc > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Thu, 2 Nov 2023, ??? wrote: > > > Ok. So drop 'scale' and keep signed/unsigned argument, is that rig
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
Hi, Richi. Is this following reasonable ? diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index fab2513105a..8d1cdad1607 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -5094,6 +5094,20 @@ Bit @var{i} of the mask is set if element @var{i} of the result should be loaded from memory and clear if element @var{i} of the result should be undefined. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_load@var{m}} instruction pattern +@item @samp{mask_len_strided_load@var{m}} +Load several separate memory locations into a destination vector of mode @var{m}. +Operand 0 is a destination vector of mode @var{m}. +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. +operand 3 is mask operand, operand 4 is length operand and operand 5 is bias operand. +The instruction can be seen as a special case of @code{mask_len_gather_load@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 2 as step. +For each element index i load address is operand 1 + @var{i} * operand 2. +Similar to mask_len_load, the instruction loads at most (operand 4 + operand 5) elements from memory. +Bit @var{i} of the mask (operand 3) is set if element @var{i} of the result should +be loaded from memory and clear if element @var{i} of the result should be zero. +Mask elements @var{i} with @var{i} > (operand 4 + operand 5) are ignored. + @cindex @code{scatter_store@var{m}@var{n}} instruction pattern @item @samp{scatter_store@var{m}@var{n}} Store a vector of mode @var{m} into several distinct memory locations. @@ -5131,6 +5145,19 @@ at most (operand 6 + operand 7) elements of (operand 4) to memory. Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be stored. Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. +@cindex @code{mask_len_strided_store@var{m}} instruction pattern +@item @samp{mask_len_strided_store@var{m}} +Store a vector of mode m into several distinct memory locations. +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. +Operand 2 is the vector of values that should be stored, which is of mode @var{m}. +operand 3 is mask operand, operand 4 is length operand and operand 5 is bias operand. +The instruction can be seen as a special case of @code{mask_len_scatter_store@var{m}@var{n}} +with an offset vector that is a @code{vec_series} with operand 1 as base and operand 1 as step. +For each element index i store address is operand 0 + @var{i} * operand 1. +Similar to mask_len_store, the instruction stores at most (operand 4 + operand 5) elements of mask (operand 3) to memory. +Bit @var{i} of the mask is set if element @var{i} of (operand 3) should be stored. +Mask elements @var{i} with @var{i} > (operand 4 + operand 5) are ignored. + @cindex @code{vec_set@var{m}} instruction pattern @item @samp{vec_set@var{m}} Set given field in the vector value. Operand 0 is the vector to modify, Thanks. juzhe.zh...@rivai.ai From: Richard Biener Date: 2023-11-03 15:40 To: juzhe.zh...@rivai.ai CC: gcc-patches; jeffreyalaw; richard.sandiford; Robin Dapp Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > Hi, Richi. > > The following is strided load/store doc: > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..4f0821a291d 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. > +operand 3 indicates stride operand is signed or unsigned number and operand > 4 is mask operand. we don't need operand 3 > +operand 5 is length operand and operand 6 is bias operand. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i load address is operand 1 + @var{i} * operand 2. > +operand 2 can be negative stride if operand 3 is 0. > +Similar to mask_len_load, the instruction loads at most (operand 5 + operand > 6) elements from memory. > +Bit @var{i} of the mask (operand 4) is set if element @var{i} of the result > should Element @var
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > Hi?Richi. > > >> Element @var{i}. I think we don't want to document 'undefined' but > >> rather match mask_gather_load and say the result should be zero. > > I am confused here, will vectorizer depend on zero inactive value ? I don't think it does, but why be inconsistent with gather and at the same time document it behaves like gather? > Since RVV load/store doesn't touch the inactive(tail or unmasked) elements, I > am wondering whether it will cause issues. For load not zeroing will cause a false dependence on the previous value of the register, so I doubt this is optimal behavior. For stores the inactive elements are of course not stored. > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-03 15:40 > To: juzhe.zh...@rivai.ai > CC: gcc-patches; jeffreyalaw; richard.sandiford; Robin Dapp > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > > > Hi, Richi. > > > > The following is strided load/store doc: > > > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > > index fab2513105a..4f0821a291d 100644 > > --- a/gcc/doc/md.texi > > +++ b/gcc/doc/md.texi > > @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of > > the result should > > be loaded from memory and clear if element @var{i} of the result should be > > undefined. > > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > > +@item @samp{mask_len_strided_load@var{m}} > > +Load several separate memory locations into a destination vector of mode > > @var{m}. > > +Operand 0 is a destination vector of mode @var{m}. > > +Operand 1 is a scalar base address and operand 2 is a scalar stride of > > Pmode. > > +operand 3 indicates stride operand is signed or unsigned number and > > operand 4 is mask operand. > > we don't need operand 3 > > > +operand 5 is length operand and operand 6 is bias operand. > > +The instruction can be seen as a special case of > > @code{mask_len_gather_load@var{m}@var{n}} > > +with an offset vector that is a @code{vec_series} with operand 1 as base > > and operand 2 as step. > > +For each element index i load address is operand 1 + @var{i} * operand 2. > > +operand 2 can be negative stride if operand 3 is 0. > > +Similar to mask_len_load, the instruction loads at most (operand 5 + > > operand 6) elements from memory. > > +Bit @var{i} of the mask (operand 4) is set if element @var{i} of the > > result should > > Element @var{i}. I think we don't want to document 'undefined' but > rather match mask_gather_load and say the result should be zero. > > Similar adjustments below. > > > +be loaded from memory and clear if element @var{i} of the result should be > > undefined. > > +Mask elements @var{i} with @var{i} > (operand 5 + operand 6) are ignored. > > + > > @cindex @code{scatter_store@var{m}@var{n}} instruction pattern > > @item @samp{scatter_store@var{m}@var{n}} > > Store a vector of mode @var{m} into several distinct memory locations. > > @@ -5131,6 +5147,21 @@ at most (operand 6 + operand 7) elements of (operand > > 4) to memory. > > Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be > > stored. > > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > > > +@cindex @code{mask_len_strided_store@var{m}} instruction pattern > > +@item @samp{mask_len_strided_store@var{m}} > > +Store a vector of mode m into several distinct memory locations. > > +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. > > +operand 2 indicates stride operand is signed or unsigned number. > > +Operand 3 is the vector of values that should be stored, which is of mode > > @var{m}. > > +operand 4 is mask operand, operand 5 is length operand and operand 6 is > > bias operand. > > +The instruction can be seen as a special case of > > @code{mask_len_scatter_store@var{m}@var{n}} > > +with an offset vector that is a @code{vec_series} with operand 1 as base > > and operand 1 as step. > > +For each element index i store address is operand 0 + @var{i} * operand 1. > > +operand 1 can be negative stride if operand 2 is 0. > > +Similar to mask_len_store, the instruction stores at most (operand 5 + > > operand 6) elements of mask (operand 4) to
Re: Re: [PATCH V2] OPTABS/IFN: Add mask_len_strided_load/mask_len_strided_store OPTABS/IFN
On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > Hi, Richi. > > Is this following reasonable ? > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > index fab2513105a..8d1cdad1607 100644 > --- a/gcc/doc/md.texi > +++ b/gcc/doc/md.texi > @@ -5094,6 +5094,20 @@ Bit @var{i} of the mask is set if element @var{i} of > the result should > be loaded from memory and clear if element @var{i} of the result should be > undefined. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > +@item @samp{mask_len_strided_load@var{m}} > +Load several separate memory locations into a destination vector of mode > @var{m}. > +Operand 0 is a destination vector of mode @var{m}. > +Operand 1 is a scalar base address and operand 2 is a scalar stride of Pmode. > +operand 3 is mask operand, operand 4 is length operand and operand 5 is bias > operand. > +The instruction can be seen as a special case of > @code{mask_len_gather_load@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 2 as step. > +For each element index i load address is operand 1 + @var{i} * operand 2. > +Similar to mask_len_load, the instruction loads at most (operand 4 + operand > 5) elements from memory. > +Bit @var{i} of the mask (operand 3) is set if element @var{i} of the result > should Element @var{i} of the mask > +be loaded from memory and clear if element @var{i} of the result should be > zero. > +Mask elements @var{i} with @var{i} > (operand 4 + operand 5) are ignored. > + > @cindex @code{scatter_store@var{m}@var{n}} instruction pattern > @item @samp{scatter_store@var{m}@var{n}} > Store a vector of mode @var{m} into several distinct memory locations. > @@ -5131,6 +5145,19 @@ at most (operand 6 + operand 7) elements of (operand > 4) to memory. > Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be > stored. > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > +@cindex @code{mask_len_strided_store@var{m}} instruction pattern > +@item @samp{mask_len_strided_store@var{m}} > +Store a vector of mode m into several distinct memory locations. > +Operand 0 is a scalar base address and operand 1 is scalar stride of Pmode. > +Operand 2 is the vector of values that should be stored, which is of mode > @var{m}. > +operand 3 is mask operand, operand 4 is length operand and operand 5 is bias > operand. > +The instruction can be seen as a special case of > @code{mask_len_scatter_store@var{m}@var{n}} > +with an offset vector that is a @code{vec_series} with operand 1 as base and > operand 1 as step. > +For each element index i store address is operand 0 + @var{i} * operand 1. > +Similar to mask_len_store, the instruction stores at most (operand 4 + > operand 5) elements of mask (operand 3) to memory. > +Bit @var{i} of the mask is set if element @var{i} of (operand 3) should be > stored. Element @var{i} Otherwise looks OK > +Mask elements @var{i} with @var{i} > (operand 4 + operand 5) are ignored. > + > @cindex @code{vec_set@var{m}} instruction pattern > @item @samp{vec_set@var{m}} > Set given field in the vector value. Operand 0 is the vector to modify, > > Thanks. > > > > juzhe.zh...@rivai.ai > > From: Richard Biener > Date: 2023-11-03 15:40 > To: juzhe.zh...@rivai.ai > CC: gcc-patches; jeffreyalaw; richard.sandiford; Robin Dapp > Subject: Re: Re: [PATCH V2] OPTABS/IFN: Add > mask_len_strided_load/mask_len_strided_store OPTABS/IFN > On Fri, 3 Nov 2023, juzhe.zh...@rivai.ai wrote: > > > Hi, Richi. > > > > The following is strided load/store doc: > > > > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi > > index fab2513105a..4f0821a291d 100644 > > --- a/gcc/doc/md.texi > > +++ b/gcc/doc/md.texi > > @@ -5094,6 +5094,22 @@ Bit @var{i} of the mask is set if element @var{i} of > > the result should > > be loaded from memory and clear if element @var{i} of the result should be > > undefined. > > Mask elements @var{i} with @var{i} > (operand 6 + operand 7) are ignored. > > > > +@cindex @code{mask_len_strided_load@var{m}} instruction pattern > > +@item @samp{mask_len_strided_load@var{m}} > > +Load several separate memory locations into a destination vector of mode > > @var{m}. > > +Operand 0 is a destination vector of mode @var{m}. > > +Operand 1 is a scalar base address and operand 2 is a scalar stride of > > Pmode. > > +operand 3 indicates stride operand is signed or unsigned number and > > operand 4 is mask operand. > > we don't ne