Hi, On Thu, 27 Apr 2023 19:59:38 +0530 Pirate Praveen <prav...@onenetbeyond.org> wrote: > With a newer libabsl-dev from > https://people.debian.org/~praveen/new/pool/main/a/abseil/, the error > changed to > > CMake Error at cmake/tests.cmake:97 (target_link_libraries): > Target "tests" links to: > > absl::scoped_mock_log > > but the target was not found. Possible reasons include:
Tl;dr: I targeted the version "3.23.4" of protobuf and could get it building taking[0] changes as reference. Patch attached. This seems to be an issue with abseil itself which is currently unfixed[1]. The workaround suggested is to add in "ABSL_BUILD_TESTING=ON" and "BUILD_TESTING=ON" options to cmake flags. Now, the release of abseil in experimental seems to be doing that, but only for the static library and not the shared library, because it needs googltest for the same, and googletest does not vendor any shared library because the SO lib is not maintained properly upstream[2] and hence trying to enable the above options for abseil for shared object compilation chokes with a dh_shlibdeps error shouting that it also needs a so lib for gtest. The maintainer of abseil has implemented the same for shared lib in this commit[3] and has explained with a solid reasoning, and you were involved (as a reporter) in a bug too, regarding the same. The reason why you get the above issue (as far as i understand) is because protobuf tries to link with abseil in a shared lib form, and I found it non-trivial to change that to fit all usecases. Since this was a test-only thing, the best way for now is to disable the tests for now until this is fixed properly in abseil. This was straightforward was to add "-Dprotobuf_BUILD_TESTS=OFF" to configure options. Now, I had to do multiple changes to get it working. I'm mentioning them incase that helps you/someone else who looks at it: * Refreshed the existing patches * Added in -Dprotobuf_BUILD_TESTS=OFF -DBUILD_SHARED_LIBS=ON to get shared lib building as well. * There's no need of any `-D` in front of dh_auto_build since this translates into a `make` command (which fails with -D), removed that. * Saw some bazel stuff failing, had to add a B-D on bazel-bootstrap and also set HOME to /tmp for the descriptior_proto bash script so it builds fine in sbuild. * The build proceeded forward but choked at python build. Turns out due to buildsystem change, binaries are no longer available at src/ but in obj-* dir. Added a patch to setup.py to link against that dir. * Now the build moved but choked at ruby's build. Turns out it isn't sensible to add entire ruby/ext/google/protobuf_c to d/clean. Adding 4 files is enough. * Now the java part of the build failed. Here, d/p/no_javax.patch has to be updated to remove more instances of javax and @Nullable annotation. * In addition to this change, the java build xml expected protoc binary to be in src/ but it changed to obj-$(DEB_HOST_MULTIARCH) so had to copy it to src/ before build and removed it after build was done (sort of a hack). * Now the python testsuite choked, has to change LD_LIBRARY_PATH to new location. Also needed a B-D on numpy. Had to exclude a generator_test here which looked syntactically incorrect to me. * After this, the install step failed, because no static libs are being built. I've removed them from install files for now, because fixing them would mean a lot of work in either d/rules or patching cmake. * Installed some .cmake files for libprotobuf. There is also a compiled lib for third_party/utf8_range but I suppose we don't need it. So overrided dh_missing to ignore that. After doing all of this, protobuf finally built. All in all, I had to look at multiple languages and buildsystems to get this working. **NB1:** The static libs are not present in the protobuf lib at the moment because this changed after the build system changed, and needs some work to get both shared and static libs enabled and it's 5 in the morning and I'm too tired and sleepy to work on it. For gitlab's usecase, static libs are *probably* not needed. **NB2:** bazel buildsystem probably downloads some stuff off the internet for this package (I did not check in depth) so the debian way of doing it would be to either package the stuff it needs or download copies beforehand and use that in build. PS: Sponsor me a party/dinner when? [0]: https://people.debian.org/~praveen/new/pool/main/p/protobuf/ [1]: https://github.com/abseil/abseil-cpp/issues/1407 [2]: https://github.com/google/googletest/issues/2184 [3]: https://salsa.debian.org/debian/abseil/-/commit/136c2572f4d6e6ab8ae02f74d691634874458184 [4]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034908#48 Best, Nilesh
commit d3abfec3189ca5188fafe09f3003138027774422
Author: Nilesh Patra <nil...@debian.org>
Date: Sat Jul 22 00:15:06 2023 +0530
New version update
diff --git a/debian/changelog b/debian/changelog
index 338446c..03edb4e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+protobuf (3.23.4-1) UNRELEASED; urgency=medium
+
+ * New upstream version 3.23.4
+
+ -- Nilesh Patra <nil...@debian.org> Sat, 22 Jul 2023 00:14:20 +0530
+
protobuf (3.22.3-1) experimental; urgency=medium
* New upstream release 3.22.3.
diff --git a/debian/clean b/debian/clean
index 43ff921..905c503 100644
--- a/debian/clean
+++ b/debian/clean
@@ -1,3 +1,8 @@
protoc.1
ruby/ext/google/protobuf_c/third_party/wyhash/wyhash.h
ruby/tests/multi_level_nesting_test_pb.rb
+ruby/ext/google/protobuf_c/naive.c
+ruby/ext/google/protobuf_c/range2-neon.c
+ruby/ext/google/protobuf_c/range2-sse.c
+ruby/ext/google/protobuf_c/utf8_range.h
+python/google/protobuf/internal/generator_test.py
diff --git a/debian/control b/debian/control
index 6bc74c8..cff61f0 100644
--- a/debian/control
+++ b/debian/control
@@ -10,7 +10,8 @@ Build-Depends:
, cmake
# C/C++
, zlib1g-dev
- , libabsl-dev
+ , bazel-bootstrap
+ , libabsl-dev (>= 20230125.3-1~)
, libgmock-dev <!nocheck>
, libgtest-dev <!nocheck>
# Python
@@ -19,6 +20,7 @@ Build-Depends:
, libpython3-all-dev <!nopython>
, python3-setuptools <!nopython>
, python3-six <!nopython>
+ , python3-numpy <!nopython>
# Manpage generator
, xmlto
# Tests
diff --git a/debian/libprotobuf-dev.install b/debian/libprotobuf-dev.install
index ad15927..e865f6d 100644
--- a/debian/libprotobuf-dev.install
+++ b/debian/libprotobuf-dev.install
@@ -1,6 +1,5 @@
-usr/lib/*/libprotobuf.a
-usr/lib/*/libprotobuf-lite.a
usr/lib/*/libprotobuf.so
usr/lib/*/libprotobuf-lite.so
usr/lib/*/pkgconfig/*
usr/include
+usr/lib/*/cmake/protobuf/*.cmake
diff --git a/debian/libprotoc-dev.install b/debian/libprotoc-dev.install
index c463a30..0a697a0 100644
--- a/debian/libprotoc-dev.install
+++ b/debian/libprotoc-dev.install
@@ -1,3 +1,2 @@
-usr/lib/*/libprotoc.a
usr/lib/*/libprotoc.so
usr/include/google/protobuf/compiler
diff --git a/debian/patches/append-lib-dirs.patch b/debian/patches/append-lib-dirs.patch
new file mode 100644
index 0000000..c103dad
--- /dev/null
+++ b/debian/patches/append-lib-dirs.patch
@@ -0,0 +1,21 @@
+--- a/python/setup.py
++++ b/python/setup.py
+@@ -344,6 +344,9 @@
+ else:
+ library_dirs = ['..']
+
++ if library_dirs is None:
++ library_dirs= []
++ library_dirs.append('../obj-' + os.getenv('DEB_HOST_MULTIARCH'))
+ TestConformanceCmd.target = ('//python:conformance_test_cpp '
+ '--define=use_fast_cpp_protos=true')
+
+@@ -412,7 +415,7 @@
+ Extension(
+ 'google.protobuf.pyext._message',
+ glob.glob('google/protobuf/pyext/*.cc'),
+- include_dirs=['.', '../src', '../third_party/abseil-cpp'],
++ include_dirs=['.', '../src'],
+ libraries=libraries,
+ extra_objects=extra_objects,
+ extra_link_args=message_extra_link_args,
diff --git a/debian/patches/fix_hppa_alignof_assert.patch b/debian/patches/fix_hppa_alignof_assert.patch
index fad5aa3..d4b703b 100644
--- a/debian/patches/fix_hppa_alignof_assert.patch
+++ b/debian/patches/fix_hppa_alignof_assert.patch
@@ -8,7 +8,7 @@ Last-Update: 2023-04-09
--- a/src/google/protobuf/descriptor.cc
+++ b/src/google/protobuf/descriptor.cc
-@@ -381,7 +381,9 @@
+@@ -384,7 +384,9 @@
ABSL_CHECK(!has_allocated());
if (std::is_trivially_destructible<U>::value) {
// Trivial types are aligned to 8 bytes.
@@ -20,7 +20,7 @@ Last-Update: 2023-04-09
// Since we can't use `if constexpr`, just make the expression compile
--- a/src/google/protobuf/map.h
+++ b/src/google/protobuf/map.h
-@@ -128,7 +128,9 @@
+@@ -130,7 +130,9 @@
// MapAllocator does not support alignments beyond 8. Technically we should
// support up to std::max_align_t, but this fails with ubsan and tcmalloc
// debug allocation logic which assume 8 as default alignment.
diff --git a/debian/patches/hurd.patch b/debian/patches/hurd.patch
index 613c8df..1cc8a7a 100644
--- a/debian/patches/hurd.patch
+++ b/debian/patches/hurd.patch
@@ -6,7 +6,7 @@ Last-Update: 2022-03-11
--- a/src/google/protobuf/compiler/command_line_interface.cc
+++ b/src/google/protobuf/compiler/command_line_interface.cc
-@@ -60,6 +60,9 @@
+@@ -62,6 +62,9 @@
#include <iostream>
#include <limits.h> // For PATH_MAX
diff --git a/debian/patches/no_errorprone.patch b/debian/patches/no_errorprone.patch
index c74d837..c041e72 100644
--- a/debian/patches/no_errorprone.patch
+++ b/debian/patches/no_errorprone.patch
@@ -176,7 +176,7 @@ Last-Update: 2022-03-11
import com.google.j2objc.annotations.J2ObjCIncompatible;
import com.google.protobuf.Duration;
import com.google.protobuf.Timestamp;
-@@ -168,7 +168,6 @@
+@@ -170,7 +170,6 @@
}
/** Throws an {@link IllegalArgumentException} if the given {@link Timestamp} is not valid. */
@@ -184,7 +184,7 @@ Last-Update: 2022-03-11
public static Timestamp checkValid(Timestamp timestamp) {
long seconds = timestamp.getSeconds();
int nanos = timestamp.getNanos();
-@@ -297,7 +296,7 @@
+@@ -300,7 +299,7 @@
* @return a {@link Timestamp} parsed from the string
* @throws IllegalArgumentException if parsing fails
*/
diff --git a/debian/patches/no_j2objc.patch b/debian/patches/no_j2objc.patch
index 2ee1f99..09ccc30 100644
--- a/debian/patches/no_j2objc.patch
+++ b/debian/patches/no_j2objc.patch
@@ -8,7 +8,7 @@ Last-Update: 2022-03-11
--- a/java/util/src/main/java/com/google/protobuf/util/Timestamps.java
+++ b/java/util/src/main/java/com/google/protobuf/util/Timestamps.java
-@@ -38,7 +38,7 @@ import static com.google.common.math.Lon
+@@ -38,7 +38,7 @@
//import com.google.errorprone.annotations.CanIgnoreReturnValue;
//import com.google.errorprone.annotations.CompileTimeConstant;
@@ -17,7 +17,7 @@ Last-Update: 2022-03-11
import com.google.protobuf.Duration;
import com.google.protobuf.Timestamp;
import java.io.Serializable;
-@@ -338,7 +338,6 @@ public final class Timestamps {
+@@ -374,7 +374,6 @@
* @throws IllegalArgumentException if the year is before 1 CE or after 9999 CE
*/
@SuppressWarnings("GoodTime") // this is a legacy conversion API
diff --git a/debian/patches/no_javax.patch b/debian/patches/no_javax.patch
index 21daa26..38fbbaf 100644
--- a/debian/patches/no_javax.patch
+++ b/debian/patches/no_javax.patch
@@ -75,3 +75,31 @@ Last-Update: 2022-03-11
private Object parseFieldValue(FieldDescriptor field, JsonElement json, Message.Builder builder)
throws InvalidProtocolBufferException {
if (json instanceof JsonNull) {
+--- a/java/util/src/main/java/com/google/protobuf/util/Timestamps.java
++++ b/java/util/src/main/java/com/google/protobuf/util/Timestamps.java
+@@ -50,7 +50,7 @@
+ import java.util.GregorianCalendar;
+ import java.util.Locale;
+ import java.util.TimeZone;
+-import javax.annotation.Nullable;
++// import javax.annotation.Nullable;
+
+ /**
+ * Utilities to help create/manipulate {@code protobuf/timestamp.proto}. All operations throw an
+@@ -311,13 +311,12 @@
+
+ // the following 3 constants contain references to java.time.Instant methods (if that class is
+ // available at runtime); otherwise, they are null.
+- @Nullable private static final Method INSTANT_NOW = instantMethod("now");
++ private static final Method INSTANT_NOW = instantMethod("now");
+
+- @Nullable private static final Method INSTANT_GET_EPOCH_SECOND = instantMethod("getEpochSecond");
++ private static final Method INSTANT_GET_EPOCH_SECOND = instantMethod("getEpochSecond");
+
+- @Nullable private static final Method INSTANT_GET_NANO = instantMethod("getNano");
++ private static final Method INSTANT_GET_NANO = instantMethod("getNano");
+
+- @Nullable
+ private static Method instantMethod(String methodName) {
+ try {
+ return Class.forName("java.time.Instant").getMethod(methodName);
diff --git a/debian/patches/no_thirdparty.patch b/debian/patches/no_thirdparty.patch
index 6ce19f3..6986833 100644
--- a/debian/patches/no_thirdparty.patch
+++ b/debian/patches/no_thirdparty.patch
@@ -13,4 +13,4 @@ Last-Update: 2023-04-24
+option(protobuf_USE_EXTERNAL_GTEST "Use external Google Test (i.e. not the one in third_party/googletest)" ON)
if (protobuf_USE_EXTERNAL_GTEST)
- find_package(GTest REQUIRED)
+ find_package(GTest REQUIRED CONFIG)
diff --git a/debian/patches/s390x.patch b/debian/patches/s390x.patch
index e4f58c9..03b249f 100644
--- a/debian/patches/s390x.patch
+++ b/debian/patches/s390x.patch
@@ -7,8 +7,7 @@ Description: Add support for s390x architecture.
--- a/src/google/protobuf/stubs/platform_macros.h
+++ b/src/google/protobuf/stubs/platform_macros.h
-@@ -82,8 +82,15 @@
- #define GOOGLE_PROTOBUF_ARCH_64_BIT 1
+@@ -83,6 +83,13 @@
#elif defined(__PPC__)
#define GOOGLE_PROTOBUF_ARCH_PPC 1
#define GOOGLE_PROTOBUF_ARCH_32_BIT 1
@@ -22,4 +21,3 @@ Description: Add support for s390x architecture.
#elif defined(__GNUC__)
# if (((__GNUC__ == 4) && (__GNUC_MINOR__ >= 7)) || (__GNUC__ > 4))
// We fallback to the generic Clang/GCC >= 4.7 implementation in atomicops.h
- # elif defined(__clang__)
diff --git a/debian/patches/series b/debian/patches/series
index d2e16d1..e0ecb8d 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -10,3 +10,4 @@ no_javax.patch
fix_hppa_alignof_assert.patch
fix_C++_32bit_tests.patch
fix_Python_32bit_tests.patch
+append-lib-dirs.patch
diff --git a/debian/rules b/debian/rules
index cd7d8bc..f86c82c 100755
--- a/debian/rules
+++ b/debian/rules
@@ -6,6 +6,7 @@
include /usr/share/dpkg/architecture.mk
include /usr/share/dpkg/buildtools.mk
+include /usr/share/dpkg/default.mk
export DH_RUBY = --gem-install
export DH_RUBY_USE_DH_AUTO_INSTALL_DESTDIR = debian/ruby-google-protobuf
@@ -42,15 +43,15 @@ PYTHON_CROSS_VARS += CXX=$(CXX)
endif
override_dh_auto_configure:
- dh_auto_configure -- -Dprotobuf_ABSL_PROVIDER=package
+ dh_auto_configure -- -Dprotobuf_ABSL_PROVIDER=package -Dprotobuf_BUILD_TESTS=OFF -DBUILD_SHARED_LIBS=ON
override_dh_auto_build-arch:
## Chicken<->Egg problem: protobuf requires self-generated .pb.go files to
## be built. First we build it normally; then "generate_descriptor_proto.sh"
## generates .pb.go files and replaces 'em if they are different as well as
## invokes "make protoc" which re-builds "src/protoc" if required.
- dh_auto_build --arch -- -Dprotobuf_ABSL_PROVIDER=package
- bash -x ./generate_descriptor_proto.sh
+ dh_auto_build --arch
+ HOME=/tmp bash -x ./generate_descriptor_proto.sh
ifneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH))
# save the native compiler
@@ -81,8 +82,10 @@ endif
override_dh_auto_build-indep:
dh_auto_build --indep
+ cp obj-*/protoc src/
# Java build
ant -f debian/java-build.xml jar jar-util
+ rm -f src/protoc
override_dh_clean:
$(RM) -rv gmock
@@ -112,7 +115,7 @@ override_dh_auto_test-arch:
ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS))$(filter nopython,$(DEB_BUILD_PROFILES)))
# Python3 test
set -e; \
- export LD_LIBRARY_PATH=$(CURDIR)/src/.libs; \
+ export LD_LIBRARY_PATH=$(CURDIR)/obj-$(DEB_HOST_MULTIARCH); \
cd python3 && for PYTHON in $(shell py3versions -r); do \
$$PYTHON setup.py test --cpp_implementation; \
done
@@ -186,5 +189,8 @@ override_dh_installdocs-indep:
dh_installdocs --indep
dh_installdocs -Xprotobuf-mode.el # in protobuf-mode-el
+override_dh_missing:
+ dh_missing --list-missing
+
override_dh_gencontrol:
dh_gencontrol -- -Vpb:API=${API_VERSION}
signature.asc
Description: PGP signature
