Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
. Regards, Qi -Original Message- From: Bruce Ashfield Sent: Wednesday, March 13, 2024 11:57 PM To: bruce.ashfi...@gmail.com Cc: Chen, Qi ; meta-virtualization@lists.yoctoproject.org Subject: Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures On Wed, Mar 13, 2024 at 10:33 AM Bruce Ashfield via lists.yoctoproject.org wrote: > > My build testing definitely got further with this applied! > > I'm now hitting a build issue with boost, but I'll double check that > my oe-core and other layers are up to date before digging in. > > Is this the boost problem you saw ? > > /include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: > multiple definition of `boost::phoenix::placeholders::uarg1'; > src/common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:/usr/include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: > first defined here > collect2: error: ld returned 1 exit status Indeed. I updated my oe-core and it has made it past this build problem. Bruce > > Bruce > > On Wed, Mar 13, 2024 at 9:13 AM Bruce Ashfield via > lists.yoctoproject.org > wrote: > > > > These questions are for my own understanding, they aren't issues I > > have with the patch! > > > > On Wed, Mar 13, 2024 at 7:08 AM Chen Qi via lists.yoctoproject.org > > wrote: > > > > > > From: Chen Qi > > > > > > 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because > > >ceph's cmake files are using it to construct PY_CC. Without it, files > > >such as stdlib.h cannot be found. > > > > Since this used to be cross compiling (but it has been a while since > > it updated, hence my stubbornness in needing to upgrade it before > > the release). I'm curious as to why this wasn't already done. The > > inability to link the test C program showed that this was the case > > .. but I wasn't able to locate this variable and set it. > > > > I would have thought that this is something the cmake bbclass should > > take care of. > > > > Is the variable used for this in Ceph custom ? > > > > > 2. re2 is added to deps. Seems no way to disable it. > > > > ok. I didn't make it far enough to see that this was needed. > > > > > 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's > > > useless > > >and problematic for new version. > > > > What broke with this in place ? > > > > > 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not > > > needed anymore. > > > > Agreed. I was going to drop it as well (in particular since we are > > python12 and it wasn't failing checks). > > > > On this point: Did you not run into Thrift or python not being > > detected properly ? > > I was constantly hitting those two errors. > > > > > 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe > > > caused by > > >boost version incompatibility. > > > > Looks good. I didn't get far enough to run into this one. > > > > > 6. Some cleanups. > > > 7. A few more options are set. > > > > I assume it is these ones (among a few others) > > > > > + -DARROW_GANDIVA=OFF \ > > > + -DARROW_WITH_RE2=OFF \ > > > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \ > > > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \ > > > > Were those options needed to fix build issues ? I assume so, since > > they are being turned off. > > > > I support turning off everything we can in Ceph :) > > > > > 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation > > > error. > > >This is a workaround and may need further visit in the future. > > > > Interesting. What is the sanity error ? Without having hit it that > > way, that variable indicates that we will be building the docs .. I > > assume the sanity check is looking to see that they are present > > and/or that the build of them has been enabled ? > > > > > 9. Delete the conditional check for /etc/debian_version which adds > > > '--install-layout deb' > > >and causes the following error. > > > error: option --install-layout not recognized > > >The patch is 0001-delete-install-layout-deb.patch. > > > > Sounds good. I didn't make it this far :) > > > > > 10. ceph-volume[-systemd] are now in sbin. > > > 11. cephfs-mirror units are added to avoid package QA issue. > > > > Also goo
Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
On Wed, Mar 13, 2024 at 10:33 AM Bruce Ashfield via lists.yoctoproject.org wrote: > > My build testing definitely got further with this applied! > > I'm now hitting a build issue with boost, but I'll double check > that my oe-core and other layers are up to date before > digging in. > > Is this the boost problem you saw ? > > /include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: > multiple definition of `boost::phoenix::placeholders::uarg1'; > src/common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:/usr/include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: > first defined here > collect2: error: ld returned 1 exit status Indeed. I updated my oe-core and it has made it past this build problem. Bruce > > Bruce > > On Wed, Mar 13, 2024 at 9:13 AM Bruce Ashfield via > lists.yoctoproject.org > wrote: > > > > These questions are for my own understanding, they aren't issues I > > have with the patch! > > > > On Wed, Mar 13, 2024 at 7:08 AM Chen Qi via lists.yoctoproject.org > > wrote: > > > > > > From: Chen Qi > > > > > > 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because > > >ceph's cmake files are using it to construct PY_CC. Without it, files > > >such as stdlib.h cannot be found. > > > > Since this used to be cross compiling (but it has been a while since > > it updated, hence > > my stubbornness in needing to upgrade it before the release). I'm > > curious as to why > > this wasn't already done. The inability to link the test C program > > showed that this was > > the case .. but I wasn't able to locate this variable and set it. > > > > I would have thought that this is something the cmake bbclass should > > take care of. > > > > Is the variable used for this in Ceph custom ? > > > > > 2. re2 is added to deps. Seems no way to disable it. > > > > ok. I didn't make it far enough to see that this was needed. > > > > > 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's > > > useless > > >and problematic for new version. > > > > What broke with this in place ? > > > > > 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not > > > needed anymore. > > > > Agreed. I was going to drop it as well (in particular since we are python12 > > and > > it wasn't failing checks). > > > > On this point: Did you not run into Thrift or python not being > > detected properly ? > > I was constantly hitting those two errors. > > > > > 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe > > > caused by > > >boost version incompatibility. > > > > Looks good. I didn't get far enough to run into this one. > > > > > 6. Some cleanups. > > > 7. A few more options are set. > > > > I assume it is these ones (among a few others) > > > > > + -DARROW_GANDIVA=OFF \ > > > + -DARROW_WITH_RE2=OFF \ > > > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \ > > > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \ > > > > Were those options needed to fix build issues ? I assume so, > > since they are being turned off. > > > > I support turning off everything we can in Ceph :) > > > > > 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation > > > error. > > >This is a workaround and may need further visit in the future. > > > > Interesting. What is the sanity error ? Without having hit it that way, that > > variable indicates that we will be building the docs .. I assume the sanity > > check is looking to see that they are present and/or that the build of them > > has been enabled ? > > > > > 9. Delete the conditional check for /etc/debian_version which adds > > > '--install-layout deb' > > >and causes the following error. > > > error: option --install-layout not recognized > > >The patch is 0001-delete-install-layout-deb.patch. > > > > Sounds good. I didn't make it this far :) > > > > > 10. ceph-volume[-systemd] are now in sbin. > > > 11. cephfs-mirror units are added to avoid package QA issue. > > > > Also good. > > > > As I mentioned, we'll ignore the TMPDIR QA issues, this is going to be an > > incremental process to bring ceph back into something that is updated. > > > > Bruce > > > > > > > > Signed-off-by: Chen Qi > > > --- > > > .../ceph/0001-avoid-to_string-error.patch | 73 +++ > > > ...h-fix-build-errors-for-cross-compile.patch | 189 -- > > > ...001-cmake-add-support-for-python3.11.patch | 31 --- > > > .../ceph/0001-delete-install-layout-deb.patch | 37 > > > recipes-extended/ceph/ceph_18.2.0.bb | 61 +++--- > > > 5 files changed, 136 insertions(+), 255 deletions(-) > > > create mode 100644 > > > recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > > > delete mode 100644 > > > recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch > > > delete mode 100644 > > > recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11.patch > > > create mode 100644 >
Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
My build testing definitely got further with this applied! I'm now hitting a build issue with boost, but I'll double check that my oe-core and other layers are up to date before digging in. Is this the boost problem you saw ? /include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: multiple definition of `boost::phoenix::placeholders::uarg1'; src/common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:/usr/include/boost/preprocessor/iteration/detail/limits/local_256.hpp:16: first defined here collect2: error: ld returned 1 exit status Bruce On Wed, Mar 13, 2024 at 9:13 AM Bruce Ashfield via lists.yoctoproject.org wrote: > > These questions are for my own understanding, they aren't issues I > have with the patch! > > On Wed, Mar 13, 2024 at 7:08 AM Chen Qi via lists.yoctoproject.org > wrote: > > > > From: Chen Qi > > > > 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because > >ceph's cmake files are using it to construct PY_CC. Without it, files > >such as stdlib.h cannot be found. > > Since this used to be cross compiling (but it has been a while since > it updated, hence > my stubbornness in needing to upgrade it before the release). I'm > curious as to why > this wasn't already done. The inability to link the test C program > showed that this was > the case .. but I wasn't able to locate this variable and set it. > > I would have thought that this is something the cmake bbclass should > take care of. > > Is the variable used for this in Ceph custom ? > > > 2. re2 is added to deps. Seems no way to disable it. > > ok. I didn't make it far enough to see that this was needed. > > > 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's > > useless > >and problematic for new version. > > What broke with this in place ? > > > 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not > > needed anymore. > > Agreed. I was going to drop it as well (in particular since we are python12 > and > it wasn't failing checks). > > On this point: Did you not run into Thrift or python not being > detected properly ? > I was constantly hitting those two errors. > > > 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe > > caused by > >boost version incompatibility. > > Looks good. I didn't get far enough to run into this one. > > > 6. Some cleanups. > > 7. A few more options are set. > > I assume it is these ones (among a few others) > > > + -DARROW_GANDIVA=OFF \ > > + -DARROW_WITH_RE2=OFF \ > > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \ > > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \ > > Were those options needed to fix build issues ? I assume so, > since they are being turned off. > > I support turning off everything we can in Ceph :) > > > 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation error. > >This is a workaround and may need further visit in the future. > > Interesting. What is the sanity error ? Without having hit it that way, that > variable indicates that we will be building the docs .. I assume the sanity > check is looking to see that they are present and/or that the build of them > has been enabled ? > > > 9. Delete the conditional check for /etc/debian_version which adds > > '--install-layout deb' > >and causes the following error. > > error: option --install-layout not recognized > >The patch is 0001-delete-install-layout-deb.patch. > > Sounds good. I didn't make it this far :) > > > 10. ceph-volume[-systemd] are now in sbin. > > 11. cephfs-mirror units are added to avoid package QA issue. > > Also good. > > As I mentioned, we'll ignore the TMPDIR QA issues, this is going to be an > incremental process to bring ceph back into something that is updated. > > Bruce > > > > > Signed-off-by: Chen Qi > > --- > > .../ceph/0001-avoid-to_string-error.patch | 73 +++ > > ...h-fix-build-errors-for-cross-compile.patch | 189 -- > > ...001-cmake-add-support-for-python3.11.patch | 31 --- > > .../ceph/0001-delete-install-layout-deb.patch | 37 > > recipes-extended/ceph/ceph_18.2.0.bb | 61 +++--- > > 5 files changed, 136 insertions(+), 255 deletions(-) > > create mode 100644 > > recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > > delete mode 100644 > > recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch > > delete mode 100644 > > recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11.patch > > create mode 100644 > > recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch > > > > diff --git a/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > > b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > > new file mode 100644 > > index ..0b4fc984 > > --- /dev/null > > +++ b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > > @@ -0,0 +1,73 @@ > > +From
Re: [meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
These questions are for my own understanding, they aren't issues I have with the patch! On Wed, Mar 13, 2024 at 7:08 AM Chen Qi via lists.yoctoproject.org wrote: > > From: Chen Qi > > 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because >ceph's cmake files are using it to construct PY_CC. Without it, files >such as stdlib.h cannot be found. Since this used to be cross compiling (but it has been a while since it updated, hence my stubbornness in needing to upgrade it before the release). I'm curious as to why this wasn't already done. The inability to link the test C program showed that this was the case .. but I wasn't able to locate this variable and set it. I would have thought that this is something the cmake bbclass should take care of. Is the variable used for this in Ceph custom ? > 2. re2 is added to deps. Seems no way to disable it. ok. I didn't make it far enough to see that this was needed. > 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's useless >and problematic for new version. What broke with this in place ? > 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not needed > anymore. Agreed. I was going to drop it as well (in particular since we are python12 and it wasn't failing checks). On this point: Did you not run into Thrift or python not being detected properly ? I was constantly hitting those two errors. > 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe caused > by >boost version incompatibility. Looks good. I didn't get far enough to run into this one. > 6. Some cleanups. > 7. A few more options are set. I assume it is these ones (among a few others) > + -DARROW_GANDIVA=OFF \ > + -DARROW_WITH_RE2=OFF \ > + -DWITH_RADOSGW_SELECT_PARQUET=OFF \ > + -DWITH_RADOSGW_ARROW_FLIGHT=OFF \ Were those options needed to fix build issues ? I assume so, since they are being turned off. I support turning off everything we can in Ceph :) > 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation error. >This is a workaround and may need further visit in the future. Interesting. What is the sanity error ? Without having hit it that way, that variable indicates that we will be building the docs .. I assume the sanity check is looking to see that they are present and/or that the build of them has been enabled ? > 9. Delete the conditional check for /etc/debian_version which adds > '--install-layout deb' >and causes the following error. > error: option --install-layout not recognized >The patch is 0001-delete-install-layout-deb.patch. Sounds good. I didn't make it this far :) > 10. ceph-volume[-systemd] are now in sbin. > 11. cephfs-mirror units are added to avoid package QA issue. Also good. As I mentioned, we'll ignore the TMPDIR QA issues, this is going to be an incremental process to bring ceph back into something that is updated. Bruce > > Signed-off-by: Chen Qi > --- > .../ceph/0001-avoid-to_string-error.patch | 73 +++ > ...h-fix-build-errors-for-cross-compile.patch | 189 -- > ...001-cmake-add-support-for-python3.11.patch | 31 --- > .../ceph/0001-delete-install-layout-deb.patch | 37 > recipes-extended/ceph/ceph_18.2.0.bb | 61 +++--- > 5 files changed, 136 insertions(+), 255 deletions(-) > create mode 100644 > recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > delete mode 100644 > recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch > delete mode 100644 > recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11.patch > create mode 100644 > recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch > > diff --git a/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > new file mode 100644 > index ..0b4fc984 > --- /dev/null > +++ b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch > @@ -0,0 +1,73 @@ > +From f807220d13adc0656c30d3207d11c70360b88d06 Mon Sep 17 00:00:00 2001 > +From: Chen Qi > +Date: Wed, 13 Mar 2024 03:14:55 -0700 > +Subject: [PATCH] avoid to_string error > + > +Upstream-Status: Pending > + > +Signed-off-by: Chen Qi > +--- > + src/rgw/rgw_asio_client.cc | 15 --- > + 1 file changed, 8 insertions(+), 7 deletions(-) > + > +diff --git a/src/rgw/rgw_asio_client.cc b/src/rgw/rgw_asio_client.cc > +index a0ec0bf5c..17880eda5 100644 > +--- a/src/rgw/rgw_asio_client.cc > b/src/rgw/rgw_asio_client.cc > +@@ -3,6 +3,7 @@ > + > + #include > + #include > ++#include > + > + #include "rgw_asio_client.h" > + #include "rgw_perf_counters.h" > +@@ -39,11 +40,11 @@ int ClientIO::init_env(CephContext *cct) > + const auto& value = header->value(); > + > + if (field == beast::http::field::content_length) { > +- env.set("CONTENT_LENGTH", value.to_string()); > ++
[meta-virtualization][master-next][PATCH] ceph: fix do_compile/do_install failures
From: Chen Qi 1. OECMAKE_C/CXX_COMPILER needs to have sysroot setting in it, because ceph's cmake files are using it to construct PY_CC. Without it, files such as stdlib.h cannot be found. 2. re2 is added to deps. Seems no way to disable it. 3. 0001-ceph-fix-build-errors-for-cross-compile.patch is dropped. It's useless and problematic for new version. 4. 0001-cmake-add-support-for-python3.11.patch is dropped as it's not needed anymore. 5. 0001-avoid-to_string-error.patch is added to fix build error, maybe caused by boost version incompatibility. 6. Some cleanups. 7. A few more options are set. 8. BUILD_DOC=1 is exported so that check_sanity() causes compilation error. This is a workaround and may need further visit in the future. 9. Delete the conditional check for /etc/debian_version which adds '--install-layout deb' and causes the following error. error: option --install-layout not recognized The patch is 0001-delete-install-layout-deb.patch. 10. ceph-volume[-systemd] are now in sbin. 11. cephfs-mirror units are added to avoid package QA issue. Signed-off-by: Chen Qi --- .../ceph/0001-avoid-to_string-error.patch | 73 +++ ...h-fix-build-errors-for-cross-compile.patch | 189 -- ...001-cmake-add-support-for-python3.11.patch | 31 --- .../ceph/0001-delete-install-layout-deb.patch | 37 recipes-extended/ceph/ceph_18.2.0.bb | 61 +++--- 5 files changed, 136 insertions(+), 255 deletions(-) create mode 100644 recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch delete mode 100644 recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch delete mode 100644 recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.11.patch create mode 100644 recipes-extended/ceph/ceph/0001-delete-install-layout-deb.patch diff --git a/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch new file mode 100644 index ..0b4fc984 --- /dev/null +++ b/recipes-extended/ceph/ceph/0001-avoid-to_string-error.patch @@ -0,0 +1,73 @@ +From f807220d13adc0656c30d3207d11c70360b88d06 Mon Sep 17 00:00:00 2001 +From: Chen Qi +Date: Wed, 13 Mar 2024 03:14:55 -0700 +Subject: [PATCH] avoid to_string error + +Upstream-Status: Pending + +Signed-off-by: Chen Qi +--- + src/rgw/rgw_asio_client.cc | 15 --- + 1 file changed, 8 insertions(+), 7 deletions(-) + +diff --git a/src/rgw/rgw_asio_client.cc b/src/rgw/rgw_asio_client.cc +index a0ec0bf5c..17880eda5 100644 +--- a/src/rgw/rgw_asio_client.cc b/src/rgw/rgw_asio_client.cc +@@ -3,6 +3,7 @@ + + #include + #include ++#include + + #include "rgw_asio_client.h" + #include "rgw_perf_counters.h" +@@ -39,11 +40,11 @@ int ClientIO::init_env(CephContext *cct) + const auto& value = header->value(); + + if (field == beast::http::field::content_length) { +- env.set("CONTENT_LENGTH", value.to_string()); ++ env.set("CONTENT_LENGTH", std::string(value)); + continue; + } + if (field == beast::http::field::content_type) { +- env.set("CONTENT_TYPE", value.to_string()); ++ env.set("CONTENT_TYPE", std::string(value)); + continue; + } + +@@ -62,26 +63,26 @@ int ClientIO::init_env(CephContext *cct) + } + *dest = '\0'; + +-env.set(buf, value.to_string()); ++env.set(buf, std::string(value)); + } + + int major = request.version() / 10; + int minor = request.version() % 10; + env.set("HTTP_VERSION", std::to_string(major) + '.' + std::to_string(minor)); + +- env.set("REQUEST_METHOD", request.method_string().to_string()); ++ env.set("REQUEST_METHOD", std::string(request.method_string())); + + // split uri from query + auto uri = request.target(); + auto pos = uri.find('?'); + if (pos != uri.npos) { + auto query = uri.substr(pos + 1); +-env.set("QUERY_STRING", query.to_string()); ++env.set("QUERY_STRING", std::string(query)); + uri = uri.substr(0, pos); + } +- env.set("SCRIPT_URI", uri.to_string()); ++ env.set("SCRIPT_URI", std::string(uri)); + +- env.set("REQUEST_URI", request.target().to_string()); ++ env.set("REQUEST_URI", std::string(request.target())); + + char port_buf[16]; + snprintf(port_buf, sizeof(port_buf), "%d", local_endpoint.port()); +-- +2.42.0 + diff --git a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch b/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch deleted file mode 100644 index 9686becb.. --- a/recipes-extended/ceph/ceph/0001-ceph-fix-build-errors-for-cross-compile.patch +++ /dev/null @@ -1,189 +0,0 @@ -From 4712fe18405ffea31405308357a8e7fca358bcce Mon Sep 17 00:00:00 2001 -From: Dengke Du -Date: Mon, 11 Mar 2019 09:14:09 +0800 -Subject: [PATCH] ceph: fix build errors for cross compile - -1. set the cross compile sysroot to find the rocksdb library -2. correct the install path for library in Distutils.cmake -