Re: [PATCH] Add loongarch native checksum implementation.

[Amit Kapila]

On Thu, Aug 10, 2023 at 5:07 PM John Naylor
 wrote:
>
> On Thu, Aug 10, 2023 at 5:54 PM Michael Paquier  wrote:
> >
> > On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> > > In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> > > facing the below error:
> > > Generating configuration headers...
> > > undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> > > 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
> > >
> > > Am I missing something or did the commit miss something?
> >
> > Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
> > Solution.pm.  If you want to be consistent with pg_config.h.in, you
> > could add it just after USE_LLVM, for instance.
>
> Oops, fixing now...
>

It is fixed now. Thanks!

-- 
With Regards,
Amit Kapila.

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Thu, Aug 10, 2023 at 5:54 PM Michael Paquier  wrote:
>
> On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> > In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> > facing the below error:
> > Generating configuration headers...
> > undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> > 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
> >
> > Am I missing something or did the commit miss something?
>
> Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
> Solution.pm.  If you want to be consistent with pg_config.h.in, you
> could add it just after USE_LLVM, for instance.

Oops, fixing now...

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[Michael Paquier]

On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> facing the below error:
> Generating configuration headers...
> undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
> 
> Am I missing something or did the commit miss something?

Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
Solution.pm.  If you want to be consistent with pg_config.h.in, you
could add it just after USE_LLVM, for instance.
--
Michael


signature.asc
Description: PGP signature

Re: [PATCH] Add loongarch native checksum implementation.

[Amit Kapila]

On Thu, Aug 10, 2023 at 10:35 AM John Naylor
 wrote:
>
> On Tue, Aug 8, 2023 at 2:22 PM YANG Xudong  wrote:
> >
> > On 2023/8/8 14:38, John Naylor wrote:
> >
> > > v4 0001 is the same as v3, but with a draft commit message. I will
> > > squash and commit this week, unless there is additional feedback.
> >
> > Looks good to me. Thanks for the additional patch.
>
> I pushed this with another small comment change. Unfortunately, I didn't 
> glance at the buildfarm beforehand -- it seems many members are failing an 
> isolation check added by commit fa2e87494, including both loongarch64 
> members. I'll check back periodically.
>
> https://buildfarm.postgresql.org/cgi-bin/show_status.pl
>

In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
facing the below error:
Generating configuration headers...
undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.

Am I missing something or did the commit miss something?

--
With Regards,
Amit Kapila.

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Tue, Aug 8, 2023 at 2:22 PM YANG Xudong  wrote:
>
> On 2023/8/8 14:38, John Naylor wrote:
>
> > v4 0001 is the same as v3, but with a draft commit message. I will
> > squash and commit this week, unless there is additional feedback.
>
> Looks good to me. Thanks for the additional patch.

I pushed this with another small comment change. Unfortunately, I didn't
glance at the buildfarm beforehand -- it seems many members are failing an
isolation check added by commit fa2e87494, including both loongarch64
members. I'll check back periodically.

https://buildfarm.postgresql.org/cgi-bin/show_status.pl

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

On 2023/8/8 14:38, John Naylor wrote:


It seems that platforms capable of running Postgres 
only support 64 bit.


I think so.



 > So I guess using aligned memory access is necessary and I have updated
 > the comment in the code.

Okay, so it's not "necessary" in the sense that it's illegal, so I'm 
thinking we can just re-use the Arm comment language, as in 0002.


Yes. I think it is similar to Arm.


v4 0001 is the same as v3, but with a draft commit message. I will 
squash and commit this week, unless there is additional feedback.


Looks good to me. Thanks for the additional patch.

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Tue, Aug 8, 2023 at 10:07 AM YANG Xudong  wrote:

> On 2023/8/7 19:01, John Naylor wrote:

> > The compilation test is found in c-compiler.m4, which still has all
> > logic for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can
> > this also be simplified?
>
> Fixed the function in c-compiler.m4 by removing the function argument
> and the logic of handling CFLAGS and CFLAGS_CRC.

Looks good to me. It seems that platforms capable of running Postgres only
support 64 bit. If that ever changes, the compiler intrinsic test (with 8
byte CRC input) should still gate that well enough in autoconf, I believe,
so in v4 I added a comment to clarify this. The Meson build checks hostcpu
first for all platforms, and the patch is consistent with surrounding code.
In the attached 0002 addendum, I change a comment in configure.ac to
clarify "override" is referring to the runtime check for x86 and Arm, and
that LoongArch doesn't need one.

> > Can you confirm the alignment requirement -- it's not clear what the
> > intention is since "doesn't require" wasn't carried over. Is there any
> > documentation (or even a report in some other context) about aligned vs
> > unaligned memory access performance?
>
> It is in the official document that the alignment is not required.
>
>
https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#74-unaligned-memory-access-support
>
>
> However, I found this patch in LKML that shows great performance gain
> when using aligned memory access similar to this patch.
>
> https://lore.kernel.org/lkml/20230410115734.93365-1-wang...@loongson.cn/
>
> So I guess using aligned memory access is necessary and I have updated
> the comment in the code.

Okay, so it's not "necessary" in the sense that it's illegal, so I'm
thinking we can just re-use the Arm comment language, as in 0002.

v4 0001 is the same as v3, but with a draft commit message. I will squash
and commit this week, unless there is additional feedback.

--
John Naylor
EDB: http://www.enterprisedb.com
From 561893beb4e3e008196b3e571685503e25a243f1 Mon Sep 17 00:00:00 2001
From: John Naylor 
Date: Tue, 8 Aug 2023 12:58:07 +0700
Subject: [PATCH v4 2/2] Some minor adjustemts to be squashed

---
 config/c-compiler.m4   | 4 
 configure  | 7 +--
 configure.ac   | 7 +--
 src/port/pg_crc32c_loongarch.c | 6 +++---
 4 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index ad6e90..bd3e6d6623 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -669,6 +669,10 @@ undefine([Ac_cachevar])dnl
 # __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
 # intrinsic functions.
 #
+# We test for the 8-byte variant since platforms capable of running
+# Postgres are 64-bit only (as of PG17), so we know CRC instructions
+# are available without a runtime check.
+#
 # If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
 AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
 [define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
diff --git a/configure b/configure
index fe0b02aa80..6a80e374f1 100755
--- a/configure
+++ b/configure
@@ -18119,8 +18119,11 @@ fi
 # we're not targeting such a processor, but can nevertheless produce code that
 # uses the CRC instructions, compile both, and select at runtime.
 #
-# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
+# You can skip the runtime check by setting the appropriate USE_*_CRC32 flag to 1
 # in the template or configure command line.
+#
+# If we are targeting a LoongArch processor, CRC instructions are
+# always available (at least on 64 bit), so no runtime check is needed.
 if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
   # Use Intel SSE 4.2 if available.
   if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
@@ -18139,8 +18142,8 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
 if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
   USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
 else
+  # LoongArch CRCC instructions.
   if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
-# LoongArch CRCC instructions.
 USE_LOONGARCH_CRC32C=1
   else
 # fall back to slicing-by-8 algorithm, which doesn't require any
diff --git a/configure.ac b/configure.ac
index 57f0f836c7..6105af6996 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2130,8 +2130,11 @@ AC_SUBST(CFLAGS_CRC)
 # we're not targeting such a processor, but can nevertheless produce code that
 # uses the CRC instructions, compile both, and select at runtime.
 #
-# Y

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Thanks for the comment. I have updated the patch to v3. Please have a look.


On 2023/8/7 19:01, John Naylor wrote:


On Fri, Jun 16, 2023 at 8:28 AM YANG Xudong > wrote:
 > > +# If the intrinsics are supported, sets 
pgac_loongarch_crc32c_intrinsics,

 > > +# and CFLAGS_CRC.
 > >
 > > +# Check if __builtin_loongarch_crcc_* intrinsics can be used
 > > +# with the default compiler flags.
 > > +# CFLAGS_CRC is set if the extra flag is required.
 > >
 > > Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
 > > confirm?
 > >
 >
 > We don't need to set CFLAGS_CRC as commented. I have updated the
 > configure script to make it align with the logic in meson build script.

(Looking again at v2)

The compilation test is found in c-compiler.m4, which still has all 
logic for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can 
this also be simplified?


Fixed the function in c-compiler.m4 by removing the function argument 
and the logic of handling CFLAGS and CFLAGS_CRC.





I diff'd pg_crc32c_loongarch.c with the current other files, and found 
it is structurally the same as the Arm implementation. That's logical if 
memory alignment is important.


   /*
- * ARMv8 doesn't require alignment, but aligned memory access is
- * significantly faster. Process leading bytes so that the loop below
- * starts with a pointer aligned to eight bytes.
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer 
aligned to eight bytes.


Can you confirm the alignment requirement -- it's not clear what the 
intention is since "doesn't require" wasn't carried over. Is there any 
documentation (or even a report in some other context) about aligned vs 
unaligned memory access performance?


It is in the official document that the alignment is not required.

https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#74-unaligned-memory-access-support


However, I found this patch in LKML that shows great performance gain 
when using aligned memory access similar to this patch.


https://lore.kernel.org/lkml/20230410115734.93365-1-wang...@loongson.cn/

So I guess using aligned memory access is necessary and I have updated 
the comment in the code.





--
John Naylor
EDB: http://www.enterprisedb.com From ced3f65d7445bdcca0628c4f5073b5657a81cd28 Mon Sep 17 00:00:00 2001
From: YANG Xudong 
Date: Tue, 8 Aug 2023 10:41:58 +0800
Subject: [PATCH v3] Add loongarch native checksum implementation.

---
 config/c-compiler.m4   | 29 ++
 configure  | 69 
 configure.ac   | 33 +++
 meson.build| 24 +++
 src/include/pg_config.h.in |  3 ++
 src/include/port/pg_crc32c.h   |  9 +
 src/port/meson.build   |  3 ++
 src/port/pg_crc32c_loongarch.c | 73 ++
 8 files changed, 228 insertions(+), 15 deletions(-)
 create mode 100644 src/port/pg_crc32c_loongarch.c

diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..ad6e90 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,32 @@ if test x"$Ac_cachevar" = x"yes"; then
 fi
 undefine([Ac_cachevar])dnl
 ])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
+AC_CACHE_CHECK(
+  [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, 
__builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w],
+  [Ac_cachevar],
+[AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+  [unsigned int crc = 0;
+   crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+   /* return computed value, to prevent the above being optimized away */
+   return crc == 0;])],
+  [Ac_cachevar=yes],
+  [Ac_cachevar=no])])
+if test x"$Ac_cachevar" = x"yes"; then
+  pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 963fbbcf1e..c5eb2814f1 100755
--- a/configure
+++ b/configure
@@ -18047,6 +18047,47 @@ fi
 
 fi
 
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for 
__builtin_loon

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Fri, Jun 16, 2023 at 8:28 AM YANG Xudong  wrote:
> > +# If the intrinsics are supported, sets
pgac_loongarch_crc32c_intrinsics,
> > +# and CFLAGS_CRC.
> >
> > +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> > +# with the default compiler flags.
> > +# CFLAGS_CRC is set if the extra flag is required.
> >
> > Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
> > confirm?
> >
>
> We don't need to set CFLAGS_CRC as commented. I have updated the
> configure script to make it align with the logic in meson build script.

(Looking again at v2)

The compilation test is found in c-compiler.m4, which still has all logic
for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can this
also be simplified?

I diff'd pg_crc32c_loongarch.c with the current other files, and found it
is structurally the same as the Arm implementation. That's logical if
memory alignment is important.

  /*
- * ARMv8 doesn't require alignment, but aligned memory access is
- * significantly faster. Process leading bytes so that the loop below
- * starts with a pointer aligned to eight bytes.
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer
aligned to eight bytes.

Can you confirm the alignment requirement -- it's not clear what the
intention is since "doesn't require" wasn't carried over. Is there any
documentation (or even a report in some other context) about aligned vs
unaligned memory access performance?

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Wed, Jul 26, 2023 at 8:25 AM YANG Xudong  wrote:
>
> Many thanks to huchangqi. Now we have loongarch64 support for both old
> world ABI and new world ABI on the buildfarm!

Glad to hear it!

On Wed, Jul 26, 2023 at 10:16 AM Michael Paquier 
wrote:
>
> The performance numbers presented upthread for the CRC computations
> are kind of nice in this environment, but honestly I have no idea how
> much this architecture is used.  Perhaps that's only something in
> China?  I am not seeing much activity around that in Japan, for
> instance, and that's really close.

That was my impression as well. My thinking was, we can give the same
treatment that we gave Arm a number of years ago (which is now quite
mainstream).

> Anyway, based on today's state of the buildfarm, we have a buildfarm
> member named cisticola that should be able to test this new CRC
> implementation, so I see no problem in applying this stuff now if you
> think it is in good shape.

I believe there was just a comment that needed updating, so I'll do that
and push within a few days.

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

On 2023/7/26 11:16, Michael Paquier wrote> The performance numbers 
presented upthread for the CRC computations

are kind of nice in this environment, but honestly I have no idea how
much this architecture is used.  Perhaps that's only something in
China?  I am not seeing much activity around that in Japan, for
instance, and that's really close.


The architecture is pretty new (to open source ecosystem). The support 
of it in Linux kernel and GCC were released last year.


Here is a site about the status of major open source project support of 
it. (Chinese only)


https://areweloongyet.com/

Re: [PATCH] Add loongarch native checksum implementation.

[Nathan Bossart]

On Wed, Jul 26, 2023 at 12:16:28PM +0900, Michael Paquier wrote:
> On Wed, Jul 05, 2023 at 02:11:02PM +0700, John Naylor wrote:
>> Before I look at this again: Are there any objections to another CRC
>> implementation for the reason of having no buildfarm member?
>
> [ ... ] 
> 
> Anyway, based on today's state of the buildfarm, we have a buildfarm
> member named cisticola that should be able to test this new CRC
> implementation, so I see no problem in applying this stuff now if you
> think it is in good shape.

IMHO we should strive to maintain buildfarm coverage for all the
instrinsics used within Postgres, if for no other reason than to ensure
future changes do not break those platforms.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Add loongarch native checksum implementation.

[Michael Paquier]

On Wed, Jul 05, 2023 at 02:11:02PM +0700, John Naylor wrote:
> Also, please don't top-post (which means: quoting an entire message, with
> new text at the top) -- it clutters our archives.
> 
> Before I look at this again: Are there any objections to another CRC
> implementation for the reason of having no buildfarm member?

The performance numbers presented upthread for the CRC computations
are kind of nice in this environment, but honestly I have no idea how
much this architecture is used.  Perhaps that's only something in
China?  I am not seeing much activity around that in Japan, for
instance, and that's really close.

Anyway, based on today's state of the buildfarm, we have a buildfarm
member named cisticola that should be able to test this new CRC
implementation, so I see no problem in applying this stuff now if you
think it is in good shape.
--
Michael


signature.asc
Description: PGP signature

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Many thanks to huchangqi. Now we have loongarch64 support for both old 
world ABI and new world ABI on the buildfarm!


 Forwarded Message 
Subject: Re: [PATCH] Add loongarch native checksum implementation.
Date: Tue, 25 Jul 2023 15:51:43 +0800
From: huchangqi 
To: YANG Xudong 

Both cisticola and nuthatch are on the buildfarm now。

cisticola is "old world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=cisticola&br=HEAD

nuthatch is "new world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=nuthatch&br=HEAD

--
Best regards,
huchangqi


On 2023/7/5 15:11, John Naylor wrote:
>
> Before I look at this again: Are there any objections to another CRC
> implementation for the reason of having no buildfarm member?

It is possible to try this patch on buildfarm now, I guess?

Re: [PATCH] Add loongarch native checksum implementation.

[huchangqi]

 Start of forwarded message 
From: huchangqi 
To: YANG Xudong 
Subject: Re: [PATCH] Add loongarch native checksum implementation.
Date: Tue, 25 Jul 2023 15:51:43 +0800

Both cisticola and nuthatch are on the buildfarm now。

cisticola is "old world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=cisticola&br=HEAD

nuthatch is "new world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=nuthatch&br=HEAD

--
Best regards,
huchangqi

YANG Xudong  writes:

> On 2023/7/6 15:14, huchangqi wrote:
>> Hi, i have a loongarch machine runing Loongnix-server (based on 
>> redhat/centos, it has gcc-8.3 on it), i am trying to test buildfarm on it, 
>> when i edit build-farm.conf it seems to need animal and secret to connect 
>> the buildfarm server.
>> then i go to  https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and 
>> registered the buildfarm  a week ago, but didn't receive any response. so 
>> what else i need to do next.
>> 
>> 
>
> Is it possible to provide a build farm instance for new world ABI of 
> loongarch also by loongson? It will be really appreciated.
>
> Thanks!
>
>> 
>>> -Original Messages-
>>> From: "YANG Xudong" 
>>> Send time:Wednesday, 07/05/2023 10:15:51
>>> To: "John Naylor" 
>>> Cc: pgsql-hackers@lists.postgresql.org, wengyanq...@ymatrix.cn, 
>>> wang...@ymatrix.cn
>>> Subject: Re: [PATCH] Add loongarch native checksum implementation.
>>>
>>> Is there any other comment?
>>>
>>> If the patch looks OK, I would like to update its status to ready for
>>> committer in the commitfest.
>>>
>>> Thanks!
>>>
>>> On 2023/6/16 09:28, YANG Xudong wrote:
>>>> Updated the patch based on the comments.
>>>>
>>>> On 2023/6/15 18:30, John Naylor wrote:
>>>>>
>>>>> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong >>>> <mailto:yangxud...@ymatrix.cn>> wrote:
>>>>>   >
>>>>>   > Attached a new patch with fixes based on the comment below.
>>>>>
>>>>> Note: It's helpful to pass "-v" to git format-patch, to have different
>>>>> versions.
>>>>>
>>>>
>>>> Added v2
>>>>
>>>>>   > > For x86 and Arm, if it fails to link without an -march flag, we
>>>>> allow
>>>>>   > > for a runtime check. The flags "-march=armv8-a+crc" and
>>>>> "-msse4.2" are
>>>>>   > > for instructions not found on all platforms. The patch also
>>>>> checks both
>>>>>   > > ways, and each one results in "Use LoongArch CRC instruction
>>>>>   > > unconditionally". The -march flag here is general, not specific. In
>>>>>   > > other words, if this only runs inside "+elif host_cpu ==
>>>>> 'loongarch64'",
>>>>>   > > why do we need both with -march and without?
>>>>>   > >
>>>>>   >
>>>>>   > Removed the elif branch.
>>>>>
>>>>> Okay, since we've confirmed that no arch flag is necessary, some other
>>>>> places can be simplified:
>>>>>
>>>>> --- a/src/port/Makefile
>>>>> +++ b/src/port/Makefile
>>>>> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
>>>>>    pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>>>>>    pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>>>>
>>>>> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
>>>>> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
>>>>> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>>>>> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>>>>
>>>>> This was copy-and-pasted from platforms that use a runtime check, so
>>>>> should be unnecessary.
>>>>>
>>>>
>>>> Removed these lines.
>>>>
>>>>> +# If the intrinsics are supported, sets
>>>>> pgac_loongarch_crc32c_intrinsics,
>>>>> +# and CFLAGS_CRC.
>>>>>
>>>>> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
>>>>> +# with the default compiler flags.
>>>>> +# CFLAGS_CRC is set if the extra flag is required.
>>

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

On 2023/7/6 15:14, huchangqi wrote:

Hi, i have a loongarch machine runing Loongnix-server (based on redhat/centos, 
it has gcc-8.3 on it), i am trying to test buildfarm on it, when i edit 
build-farm.conf it seems to need animal and secret to connect the buildfarm 
server.
then i go to  https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and 
registered the buildfarm  a week ago, but didn't receive any response. so what 
else i need to do next.




Is it possible to provide a build farm instance for new world ABI of 
loongarch also by loongson? It will be really appreciated.


Thanks!




-Original Messages-
From: "YANG Xudong" 
Send time:Wednesday, 07/05/2023 10:15:51
To: "John Naylor" 
Cc: pgsql-hackers@lists.postgresql.org, wengyanq...@ymatrix.cn, 
wang...@ymatrix.cn
Subject: Re: [PATCH] Add loongarch native checksum implementation.

Is there any other comment?

If the patch looks OK, I would like to update its status to ready for
committer in the commitfest.

Thanks!

On 2023/6/16 09:28, YANG Xudong wrote:

Updated the patch based on the comments.

On 2023/6/15 18:30, John Naylor wrote:


On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong mailto:yangxud...@ymatrix.cn>> wrote:
  >
  > Attached a new patch with fixes based on the comment below.

Note: It's helpful to pass "-v" to git format-patch, to have different
versions.



Added v2


  > > For x86 and Arm, if it fails to link without an -march flag, we
allow
  > > for a runtime check. The flags "-march=armv8-a+crc" and
"-msse4.2" are
  > > for instructions not found on all platforms. The patch also
checks both
  > > ways, and each one results in "Use LoongArch CRC instruction
  > > unconditionally". The -march flag here is general, not specific. In
  > > other words, if this only runs inside "+elif host_cpu ==
'loongarch64'",
  > > why do we need both with -march and without?
  > >
  >
  > Removed the elif branch.

Okay, since we've confirmed that no arch flag is necessary, some other
places can be simplified:

--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
   pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
   pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)

+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)

This was copy-and-pasted from platforms that use a runtime check, so
should be unnecessary.



Removed these lines.


+# If the intrinsics are supported, sets
pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.

+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.

Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
confirm?



We don't need to set CFLAGS_CRC as commented. I have updated the
configure script to make it align with the logic in meson build script.


  > > Also, I don't have a Loongarch machine for testing. Could you
show that
  > > the instructions are found in the binary, maybe using objdump and
grep?
  > > Or a performance test?
  > >
  >
  > The output of the objdump command `objdump -dS
  > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep
-B 30
  > -A 10 crcc` is attached.

Thanks for confirming.

--
John Naylor
EDB: http://www.enterprisedb.com <http://www.enterprisedb.com>





本邮件及其附件含有龙芯中科的商业秘密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制或散发）本邮件及其附件中的信息。如果您错收本邮件，请您立即电话或邮件通知发件人并删除本邮件。
This email and its attachments contain confidential information from Loongson 
Technology , which is intended only for the person or entity whose address is 
listed above. Any use of the information contained herein in any way 
(including, but not limited to, total or partial disclosure, reproduction or 
dissemination) by persons other than the intended recipient(s) is prohibited. 
If you receive this email in error, please notify the sender by phone or email 
immediately and delete it.

Re: [PATCH] Add loongarch native checksum implementation.

[Daniel Gustafsson]

> On 6 Jul 2023, at 09:14, huchangqi  wrote:
> 
> Hi, i have a loongarch machine runing Loongnix-server (based on 
> redhat/centos, it has gcc-8.3 on it), i am trying to test buildfarm on it, 
> when i edit build-farm.conf it seems to need animal and secret to connect the 
> buildfarm server.
> then i go to  https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and 
> registered the buildfarm  a week ago, but didn't receive any response. so 
> what else i need to do next.

Thanks for volunteering a buildfarm animal!  The registration is probably just
pending due to it being summer and things slow down, but I've added Andrew
Dunstan who is the Buildfarm expert on CC:.

--
Daniel Gustafsson

Re: Re: [PATCH] Add loongarch native checksum implementation.

[huchangqi]

Hi, i have a loongarch machine runing Loongnix-server (based on redhat/centos, 
it has gcc-8.3 on it), i am trying to test buildfarm on it, when i edit 
build-farm.conf it seems to need animal and secret to connect the buildfarm 
server.
then i go to  https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and 
registered the buildfarm  a week ago, but didn't receive any response. so what 
else i need to do next.



> -Original Messages-
> From: "YANG Xudong" 
> Send time:Wednesday, 07/05/2023 10:15:51
> To: "John Naylor" 
> Cc: pgsql-hackers@lists.postgresql.org, wengyanq...@ymatrix.cn, 
> wang...@ymatrix.cn
> Subject: Re: [PATCH] Add loongarch native checksum implementation.
> 
> Is there any other comment?
> 
> If the patch looks OK, I would like to update its status to ready for 
> committer in the commitfest.
> 
> Thanks!
> 
> On 2023/6/16 09:28, YANG Xudong wrote:
> > Updated the patch based on the comments.
> > 
> > On 2023/6/15 18:30, John Naylor wrote:
> >>
> >> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong  >> <mailto:yangxud...@ymatrix.cn>> wrote:
> >>  >
> >>  > Attached a new patch with fixes based on the comment below.
> >>
> >> Note: It's helpful to pass "-v" to git format-patch, to have different 
> >> versions.
> >>
> > 
> > Added v2
> > 
> >>  > > For x86 and Arm, if it fails to link without an -march flag, we 
> >> allow
> >>  > > for a runtime check. The flags "-march=armv8-a+crc" and 
> >> "-msse4.2" are
> >>  > > for instructions not found on all platforms. The patch also 
> >> checks both
> >>  > > ways, and each one results in "Use LoongArch CRC instruction
> >>  > > unconditionally". The -march flag here is general, not specific. In
> >>  > > other words, if this only runs inside "+elif host_cpu == 
> >> 'loongarch64'",
> >>  > > why do we need both with -march and without?
> >>  > >
> >>  >
> >>  > Removed the elif branch.
> >>
> >> Okay, since we've confirmed that no arch flag is necessary, some other 
> >> places can be simplified:
> >>
> >> --- a/src/port/Makefile
> >> +++ b/src/port/Makefile
> >> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
> >>   pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> >>   pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
> >>
> >> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
> >> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
> >> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> >> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
> >>
> >> This was copy-and-pasted from platforms that use a runtime check, so 
> >> should be unnecessary.
> >>
> > 
> > Removed these lines.
> > 
> >> +# If the intrinsics are supported, sets 
> >> pgac_loongarch_crc32c_intrinsics,
> >> +# and CFLAGS_CRC.
> >>
> >> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> >> +# with the default compiler flags.
> >> +# CFLAGS_CRC is set if the extra flag is required.
> >>
> >> Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you 
> >> confirm?
> >>
> > 
> > We don't need to set CFLAGS_CRC as commented. I have updated the 
> > configure script to make it align with the logic in meson build script.
> > 
> >>  > > Also, I don't have a Loongarch machine for testing. Could you 
> >> show that
> >>  > > the instructions are found in the binary, maybe using objdump and 
> >> grep?
> >>  > > Or a performance test?
> >>  > >
> >>  >
> >>  > The output of the objdump command `objdump -dS
> >>  > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep 
> >> -B 30
> >>  > -A 10 crcc` is attached.
> >>
> >> Thanks for confirming.
> >>
> >> -- 
> >> John Naylor
> >> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com>
> 


本邮件及其附件含有龙芯中科的商业秘密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制或散发）本邮件及其附件中的信息。如果您错收本邮件，请您立即电话或邮件通知发件人并删除本邮件。
 
This email and its attachments contain confidential information from Loongson 
Technology , which is intended only for the person or entity whose address is 
listed above. Any use of the information contained herein in any way 
(including, but not limited to, total or partial disclosure, reproduction or 
dissemination) by persons other than the intended recipient(s) is prohibited. 
If you receive this email in error, please notify the sender by phone or email 
immediately and delete it. 

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Wed, Jul 5, 2023 at 9:16 AM YANG Xudong  wrote:
>
> Is there any other comment?

It's only been a few weeks since the last patch, and this is not an urgent
bugfix, so there is no reason to ping the thread. Feature freeze will
likely be in April of next year.

Also, please don't top-post (which means: quoting an entire message, with
new text at the top) -- it clutters our archives.

Before I look at this again: Are there any objections to another CRC
implementation for the reason of having no buildfarm member?

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Is there any other comment?

If the patch looks OK, I would like to update its status to ready for 
committer in the commitfest.


Thanks!

On 2023/6/16 09:28, YANG Xudong wrote:

Updated the patch based on the comments.

On 2023/6/15 18:30, John Naylor wrote:


On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong > wrote:

 >
 > Attached a new patch with fixes based on the comment below.

Note: It's helpful to pass "-v" to git format-patch, to have different 
versions.




Added v2

 > > For x86 and Arm, if it fails to link without an -march flag, we 
allow
 > > for a runtime check. The flags "-march=armv8-a+crc" and 
"-msse4.2" are
 > > for instructions not found on all platforms. The patch also 
checks both

 > > ways, and each one results in "Use LoongArch CRC instruction
 > > unconditionally". The -march flag here is general, not specific. In
 > > other words, if this only runs inside "+elif host_cpu == 
'loongarch64'",

 > > why do we need both with -march and without?
 > >
 >
 > Removed the elif branch.

Okay, since we've confirmed that no arch flag is necessary, some other 
places can be simplified:


--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
  pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
  pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)

+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)

This was copy-and-pasted from platforms that use a runtime check, so 
should be unnecessary.




Removed these lines.

+# If the intrinsics are supported, sets 
pgac_loongarch_crc32c_intrinsics,

+# and CFLAGS_CRC.

+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.

Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you 
confirm?




We don't need to set CFLAGS_CRC as commented. I have updated the 
configure script to make it align with the logic in meson build script.


 > > Also, I don't have a Loongarch machine for testing. Could you 
show that
 > > the instructions are found in the binary, maybe using objdump and 
grep?

 > > Or a performance test?
 > >
 >
 > The output of the objdump command `objdump -dS
 > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep 
-B 30

 > -A 10 crcc` is attached.

Thanks for confirming.

--
John Naylor
EDB: http://www.enterprisedb.com 

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Updated the patch based on the comments.

On 2023/6/15 18:30, John Naylor wrote:


On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong > wrote:

 >
 > Attached a new patch with fixes based on the comment below.

Note: It's helpful to pass "-v" to git format-patch, to have different 
versions.




Added v2


 > > For x86 and Arm, if it fails to link without an -march flag, we allow
 > > for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are
 > > for instructions not found on all platforms. The patch also checks both
 > > ways, and each one results in "Use LoongArch CRC instruction
 > > unconditionally". The -march flag here is general, not specific. In
 > > other words, if this only runs inside "+elif host_cpu == 
'loongarch64'",

 > > why do we need both with -march and without?
 > >
 >
 > Removed the elif branch.

Okay, since we've confirmed that no arch flag is necessary, some other 
places can be simplified:


--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
  pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
  pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)

+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)

This was copy-and-pasted from platforms that use a runtime check, so 
should be unnecessary.




Removed these lines.


+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.

+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.

Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you 
confirm?




We don't need to set CFLAGS_CRC as commented. I have updated the 
configure script to make it align with the logic in meson build script.



 > > Also, I don't have a Loongarch machine for testing. Could you show that
 > > the instructions are found in the binary, maybe using objdump and grep?
 > > Or a performance test?
 > >
 >
 > The output of the objdump command `objdump -dS
 > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep -B 30
 > -A 10 crcc` is attached.

Thanks for confirming.

--
John Naylor
EDB: http://www.enterprisedb.com From b2329478b5331e2aa9942c7ee4e23e3bfa871c1d Mon Sep 17 00:00:00 2001
From: YANG Xudong 
Date: Fri, 16 Jun 2023 09:22:08 +0800
Subject: [PATCH v2] Add loongarch native checksum implementation.

---
 config/c-compiler.m4   | 34 
 configure  | 73 ++
 configure.ac   | 33 +++
 meson.build| 24 +++
 src/include/pg_config.h.in |  3 ++
 src/include/port/pg_crc32c.h   |  9 +
 src/port/meson.build   |  3 ++
 src/port/pg_crc32c_loongarch.c | 72 +
 8 files changed, 236 insertions(+), 15 deletions(-)
 create mode 100644 src/port/pg_crc32c_loongarch.c

diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
 fi
 undefine([Ac_cachevar])dnl
 ])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+  [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, 
__builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with 
CFLAGS=$1],
+  [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+  [unsigned int crc = 0;
+   crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+   /* return computed value, to prevent the above being optimized away */
+   return crc == 0;])],
+  [Ac_cachevar=yes],
+  [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+  CFLAGS_CRC="$1"
+  pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..7c60a26c11 100755
--- a/configure
+++ b/configure
@@ -18114,6 +18114,51 @@ fi
 
 fi
 
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong  wrote:
>
> Attached a new patch with fixes based on the comment below.

Note: It's helpful to pass "-v" to git format-patch, to have different
versions.

> > For x86 and Arm, if it fails to link without an -march flag, we allow
> > for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are
> > for instructions not found on all platforms. The patch also checks both
> > ways, and each one results in "Use LoongArch CRC instruction
> > unconditionally". The -march flag here is general, not specific. In
> > other words, if this only runs inside "+elif host_cpu == 'loongarch64'",
> > why do we need both with -march and without?
> >
>
> Removed the elif branch.

Okay, since we've confirmed that no arch flag is necessary, some other
places can be simplified:

--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
 pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
 pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)

+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)

This was copy-and-pasted from platforms that use a runtime check, so should
be unnecessary.

+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.

+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.

Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
confirm?

> > Also, I don't have a Loongarch machine for testing. Could you show that
> > the instructions are found in the binary, maybe using objdump and grep?
> > Or a performance test?
> >
>
> The output of the objdump command `objdump -dS
> ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep -B 30
> -A 10 crcc` is attached.

Thanks for confirming.

--
John Naylor
EDB: http://www.enterprisedb.com

Re: [PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Attached a new patch with fixes based on the comment below.


On 2023/6/13 18:26, John Naylor wrote:


On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong <mailto:yangxud...@ymatrix.cn>> wrote:

 >
 > This patch tries to add loongarch native crc32 check with crcc.*
 > instructions to postgresql.
 >
 > The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux
 > and GCC 13.1.0 / clang 16.0.0 with
 >
 > - default ./configure
 > - default meson setup

I took a quick look at this, and it seems mostly in line with other 
architectures we support for CRC. I have a couple questions and comments:


configure.ac <http://configure.ac>:

+AC_SUBST(CFLAGS_CRC)
 > This seems to be an unnecessary copy-paste. I think we only need one,
after all checks have run.



Removed the extra line.



meson.build

+  if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, 
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and 
__builtin_loongarch_crcc_w_d_w without -march=loongarch64',

+      args: test_c_args)
+    # Use LoongArch CRC instruction unconditionally
+    cdata.set('USE_LOONGARCH_CRC32C', 1)
+    have_optimized_crc = true
+  elif cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, 
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and 
__builtin_loongarch_crcc_w_d_w with -march=loongarch64',

+      args: test_c_args + ['-march=loongarch64'])
+    # Use LoongArch CRC instruction unconditionally

For x86 and Arm, if it fails to link without an -march flag, we allow 
for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are 
for instructions not found on all platforms. The patch also checks both 
ways, and each one results in "Use LoongArch CRC instruction 
unconditionally". The -march flag here is general, not specific. In 
other words, if this only runs inside "+elif host_cpu == 'loongarch64'", 
why do we need both with -march and without?




Removed the elif branch.


Also, I don't have a Loongarch machine for testing. Could you show that 
the instructions are found in the binary, maybe using objdump and grep? 
Or a performance test?




The output of the objdump command `objdump -dS 
../postgres-build/tmp_install/usr/local/pgsql/bin/postgres  | grep -B 30 
-A 10 crcc` is attached.


Also the output of make check is attached.


I run a simple test program to compare the performance of 
pg_comp_crc32c_loongarch and pg_comp_crc32c_sb8 on my test machine. The 
result is that pg_comp_crc32c_loongarch is over 2x faster than 
pg_comp_crc32c_sb8.



In the future, you may also consider running the buildfarm client on a 
machine dedicated for testing. That will give us quick feedback if some 
future new code doesn't work on this platform. More information here:


https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto 
<https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto>




I will contact the loongson community 
(https://github.com/loongson-community) to see if they are able to 
provide some machine for buildfarm or not.




--
John Naylor
EDB: http://www.enterprisedb.com <http://www.enterprisedb.com>


--
YANG XudongFrom c6441c9e446d13ddf60b1f5432114a5289968403 Mon Sep 17 00:00:00 2001
From: YANG Xudong 
Date: Wed, 14 Jun 2023 09:49:49 +0800
Subject: [PATCH] Add loongarch native checksum implementation.

---
 config/c-compiler.m4   |  34 ++
 configure  | 116 +++--
 configure.ac   |  37 ---
 meson.build|  24 +++
 src/include/pg_config.h.in |   3 +
 src/include/port/pg_crc32c.h   |   9 +++
 src/port/Makefile  |   5 ++
 src/port/meson.build   |   3 +
 src/port/pg_crc32c_loongarch.c |  72 
 9 files changed, 288 insertions(+), 15 deletions(-)
 create mode 100644 src/port/pg_crc32c_loongarch.c

diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
 fi
 undefine([Ac_cachevar])dnl
 ])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+  [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, 
__builtin_loongarch_crcc_w_w_w and __builtin_loongarc

Re: [PATCH] Add loongarch native checksum implementation.

[John Naylor]

On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong  wrote:
>
> This patch tries to add loongarch native crc32 check with crcc.*
> instructions to postgresql.
>
> The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux
> and GCC 13.1.0 / clang 16.0.0 with
>
> - default ./configure
> - default meson setup

I took a quick look at this, and it seems mostly in line with other
architectures we support for CRC. I have a couple questions and comments:

configure.ac:

+AC_SUBST(CFLAGS_CRC)

This seems to be an unnecessary copy-paste. I think we only need one, after
all checks have run.

meson.build

+  if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
__builtin_loongarch_crcc_w_d_w without -march=loongarch64',
+  args: test_c_args)
+# Use LoongArch CRC instruction unconditionally
+cdata.set('USE_LOONGARCH_CRC32C', 1)
+have_optimized_crc = true
+  elif cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
__builtin_loongarch_crcc_w_d_w with -march=loongarch64',
+  args: test_c_args + ['-march=loongarch64'])
+# Use LoongArch CRC instruction unconditionally

For x86 and Arm, if it fails to link without an -march flag, we allow for a
runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are for
instructions not found on all platforms. The patch also checks both ways,
and each one results in "Use LoongArch CRC instruction unconditionally".
The -march flag here is general, not specific. In other words, if this only
runs inside "+elif host_cpu == 'loongarch64'", why do we need both with
-march and without?

Also, I don't have a Loongarch machine for testing. Could you show that the
instructions are found in the binary, maybe using objdump and grep? Or a
performance test?

In the future, you may also consider running the buildfarm client on a
machine dedicated for testing. That will give us quick feedback if some
future new code doesn't work on this platform. More information here:

https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto

--
John Naylor
EDB: http://www.enterprisedb.com

[PATCH] Add loongarch native checksum implementation.

[YANG Xudong]

Hi,

This patch tries to add loongarch native crc32 check with crcc.* 
instructions to postgresql.


The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux 
and GCC 13.1.0 / clang 16.0.0 with


- default ./configure
- default meson setup


See:

[1]: 
https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#crc-check-instructions
[2]: 
https://gcc.gnu.org/onlinedocs/gcc/LoongArch-Base-Built-in-Functions.html
[3]: 
https://github.com/llvm/llvm-project/blob/release/16.x/clang/include/clang/Basic/BuiltinsLoongArch.def#L36-L39From 6ccee980a42f1706094a51fa146c55edc3adfbb1 Mon Sep 17 00:00:00 2001
From: YANG Xudong 
Date: Thu, 8 Jun 2023 13:22:03 +0800
Subject: [PATCH] Add loongarch native checksum implementation.

---
 config/c-compiler.m4   |  34 ++
 configure  | 118 +++--
 configure.ac   |  39 ---
 meson.build|  29 
 src/include/pg_config.h.in |   3 +
 src/include/port/pg_crc32c.h   |   9 +++
 src/port/Makefile  |   5 ++
 src/port/meson.build   |   3 +
 src/port/pg_crc32c_loongarch.c |  72 
 9 files changed, 297 insertions(+), 15 deletions(-)
 create mode 100644 src/port/pg_crc32c_loongarch.c

diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
 fi
 undefine([Ac_cachevar])dnl
 ])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+  [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, 
__builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with 
CFLAGS=$1],
+  [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+  [unsigned int crc = 0;
+   crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+   /* return computed value, to prevent the above being optimized away */
+   return crc == 0;])],
+  [Ac_cachevar=yes],
+  [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+  CFLAGS_CRC="$1"
+  pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..c5d8c2e4f6 100755
--- a/configure
+++ b/configure
@@ -18116,6 +18116,96 @@ fi
 
 
 
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for 
__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, 
__builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=" 
>&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, 
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and 
__builtin_loongarch_crcc_w_d_w with CFLAGS=... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics_+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS "
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+int
+main ()
+{
+unsigned int crc = 0;
+   crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+   crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+   /* return computed value, to prevent the above being optimized away */
+   return crc == 0;
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  pgac_cv_loongarch_crc32c_intrinsics_=yes
+else
+  pgac_cv_loongarch_crc32c_intrinsics_=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: 
$pgac_cv_loongarch_crc32c_intrinsics_" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics_" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics_" = x"yes"; then