I am experiencing binary compatibility issues trying to get pycapnp serialization/deserialization working with C extensions. There appear to be ABI compatibility issues when passing C++ structs compiled in pycapnp into our C extensions that are compiled in a different environment.
When serializing an instance of a class that's implemented in NuPIC, we create a message builder via pycapnp and pass it to the corresponding instance's write method, which in turn invokes write methods of its own contained members. This works fine for members whose classes are implemented in python, but doesn't always work for those implemented in the nupic.bindings extension due to ABI issues. For example, when serializing the TemporalMemory class, we might employ the following sequence: from nupic.proto import TemporalMemoryProto_capnp builder = TemporalMemoryProto_capnp.TemporalMemoryProto.new_message() temporal_memory.write(builder) Inside TemporalMemory.write(builder), we have something along these lines: class TemporalMemory(object): def write(self, builder): builder.columnDimensions = list(self.columnDimensions) self.connections.write(builder.connections) # pure python *self._random.write(builder.random) # C++ Random class from extension* The Random class that's implemented inside the nupic.bindings extension needs to rely on our own build of capnproto that's linked into the extension, but this doesn't seem to be compatible with the object constructed in pycapnp. We learned the hard way, after much trial and error, that we can't simply pass the underlying message builders that were instantiated by pycapnp's capnp.so module to our own build of capnproto contained in the nupic.bindigns extension. This was particularly evident when working on the manylinux wheel for nupic.bindings, which needs to be compiled using the toolchain and c/c++ runtimes from CentOS-6. This resulted in ABI incompatibilities when the capnproto code compiled into the extension attempts to operate on a message builder that was constructed by pycapnp's build of capnp.so. The message builder instance created by pycapnp's capnp.so appears corrupted when operated upon by the capnproto code linked into the extension. Is there any recommendation for handling this dual Python/C-extension scenario that avoids the ABI compatibility problem with C++ objects? -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/capnproto.