Re: [zeromq-dev] ZeroMQ 4.2 release, planning
Sorry, I meant if we go with (1), not (2), we might bump the size as well, since we are already doing another ABI-breaking change. I agree on the solution as well. On Mon, 2016-08-29 at 17:12 +0200, Pieter Hintjens wrote: > I'm confused between the (1) and (2) choices, and can't see where > bumping the message size fits. > > Nonetheless, I think bumping the size, fixing the alignment issues, > and bumping the ABI version is the best solution here. > > On Fri, Aug 26, 2016 at 12:33 PM, Luca Boccassi > wrote: > > I've given some more thoughts and testing to the alignment issue. I can > > reproduce the problem by enabling alignment checks on x86 too. > > > > But most importantly, I think we cannot get away from bumping the ABI > > with this fix, however we rearrange it, simply because applications need > > to be rebuilt against the new header to be fixed. A simple rebuild of > > the libzmq.so is not enough. And the way to do this is to bump the ABI > > so that distros can schedule transitions and rebuilds and so on. > > > > So the choice list is now restricted to: > > > > 1) Bump ABI > > 2) Revert the fix and leave everything broken on sparc64 and some > > aarch64 (rpi3 seems not to be affected, must depend on the SoC flavour) > > > > If we go with 2, we might as well get 2 birds with one stone and bump > > the zmq_msg_t size to 128 as we have talked about in the past. > > > > Doron, this would help with the new UDP based socket types right? > > > > Pros of bumping msg size: > > > > - we can get rid of the malloc() in the lmsg type case as all the data > > will fit > > > > Cons: > > > > - for the vsm/cmsg type cases (for most architectures anyway) it won't > > fit anymore into a single cacheline > > > > Given all this, I'd say we should go for it. > > > > Opinions? > > > > On Sat, 2016-08-13 at 16:59 +0100, Luca Boccassi wrote: > >> Hello, > >> > >> Trying to give some thoughts again on the libzmq 4.2 release. It's > >> really long overdue! > >> > >> The main issue from my point of view is this change: > >> > >> https://github.com/zeromq/libzmq/commit/d9fb1d36ff2008966af538f722a1f4ab158dbf64 > >> > >> -typedef struct zmq_msg_t {unsigned char _ [64];} zmq_msg_t; > >> +/* union here ensures correct alignment on architectures that require > >> it, e.g. > >> + * SPARC > >> + */ > >> +typedef union zmq_msg_t {unsigned char _ [64]; void *p; } zmq_msg_t; > >> > >> > >> This is flagged by the common ABI checkers tools as an ABI breakage > >> (see: http://abi-laboratory.pro/tracker/timeline/zeromq/ ). And it makes > >> sense from this point of view: if some applications on some > >> architectures are broken due to wrong alignment, they would need to be > >> rebuilt, and the way to ensure that is to bump the ABI "current" digit > >> to make sure maintainers do a rebuild. > >> > >> On the other hand, signaling an ABI breakage is a pain, and a cause of > >> major churn for packagers and maintainers. It means for example a new > >> package has to be created (eg: libzmq5 -> libzmq6), and a transition has > >> to be started and all reverse dependencies need to be rebuilt. And if > >> this is pointless for all save a few corner cases (eg SPARC64 as for > >> above) it's all quite frustrating. > >> > >> So we have a choice to make before we release 4.2, four possibilities as > >> far as I can see: > >> > >> 1) Ignore the ABI checkers and get yelled at by maintainers and > >> packagers. Also the SPARC64 users will most likely NOT get their bug > >> fixed > >> 2) Bump ABI revision to 6 and get yelled at by maintainers and packagers > >> 3) Revert the above change and postpone it to when we have a more > >> generally useful reason to break ABI (bump zmq_msg_t from 64 to 128 > >> bytes for example, Doron?) > >> 4) Try to be clever and revert the above change and use something like > >> #pragma pack(8). This will fool the ABI checkers (I tried it), and given > >> that typedef is only used externally to allocate the right size it > >> shouldn't actually affect anything, apart from the users of SPARC64 > >> which should get the bugfix with this too. This is very sneaky :-) > >> > >> CC'ing Lazslo, the Debian maintainer, given what we choose to do might > >> result in a lot of work for him :-) > >> > >> Opinions? > >> > >> Kind regards, > >> Luca Boccassi > >> > >> On Tue, 2016-05-03 at 10:39 +0200, Pieter Hintjens wrote: > >> > Hi all, > >> > > >> > I'm just throwing some ideas on the table. We have a good package of > >> > work on master and it's probably time to make a 4.2 release. > >> > > >> > Luca has already back-ported the enable/disable draft design from > >> > zproject (CZMQ et al). Yay! So we can now release stable master > >> > safely, while continuing to refine and extend the draft API sections. > >> > > >> > I propose: > >> > > >> > - to end with the stable fork policy; this was needed years ago when > >> > we had massively unstable masters. It's no longer a problem. > >> > - to use the github release function
Re: [zeromq-dev] ZeroMQ 4.2 release, planning
I'm confused between the (1) and (2) choices, and can't see where bumping the message size fits. Nonetheless, I think bumping the size, fixing the alignment issues, and bumping the ABI version is the best solution here. On Fri, Aug 26, 2016 at 12:33 PM, Luca Boccassi wrote: > I've given some more thoughts and testing to the alignment issue. I can > reproduce the problem by enabling alignment checks on x86 too. > > But most importantly, I think we cannot get away from bumping the ABI > with this fix, however we rearrange it, simply because applications need > to be rebuilt against the new header to be fixed. A simple rebuild of > the libzmq.so is not enough. And the way to do this is to bump the ABI > so that distros can schedule transitions and rebuilds and so on. > > So the choice list is now restricted to: > > 1) Bump ABI > 2) Revert the fix and leave everything broken on sparc64 and some > aarch64 (rpi3 seems not to be affected, must depend on the SoC flavour) > > If we go with 2, we might as well get 2 birds with one stone and bump > the zmq_msg_t size to 128 as we have talked about in the past. > > Doron, this would help with the new UDP based socket types right? > > Pros of bumping msg size: > > - we can get rid of the malloc() in the lmsg type case as all the data > will fit > > Cons: > > - for the vsm/cmsg type cases (for most architectures anyway) it won't > fit anymore into a single cacheline > > Given all this, I'd say we should go for it. > > Opinions? > > On Sat, 2016-08-13 at 16:59 +0100, Luca Boccassi wrote: >> Hello, >> >> Trying to give some thoughts again on the libzmq 4.2 release. It's >> really long overdue! >> >> The main issue from my point of view is this change: >> >> https://github.com/zeromq/libzmq/commit/d9fb1d36ff2008966af538f722a1f4ab158dbf64 >> >> -typedef struct zmq_msg_t {unsigned char _ [64];} zmq_msg_t; >> +/* union here ensures correct alignment on architectures that require >> it, e.g. >> + * SPARC >> + */ >> +typedef union zmq_msg_t {unsigned char _ [64]; void *p; } zmq_msg_t; >> >> >> This is flagged by the common ABI checkers tools as an ABI breakage >> (see: http://abi-laboratory.pro/tracker/timeline/zeromq/ ). And it makes >> sense from this point of view: if some applications on some >> architectures are broken due to wrong alignment, they would need to be >> rebuilt, and the way to ensure that is to bump the ABI "current" digit >> to make sure maintainers do a rebuild. >> >> On the other hand, signaling an ABI breakage is a pain, and a cause of >> major churn for packagers and maintainers. It means for example a new >> package has to be created (eg: libzmq5 -> libzmq6), and a transition has >> to be started and all reverse dependencies need to be rebuilt. And if >> this is pointless for all save a few corner cases (eg SPARC64 as for >> above) it's all quite frustrating. >> >> So we have a choice to make before we release 4.2, four possibilities as >> far as I can see: >> >> 1) Ignore the ABI checkers and get yelled at by maintainers and >> packagers. Also the SPARC64 users will most likely NOT get their bug >> fixed >> 2) Bump ABI revision to 6 and get yelled at by maintainers and packagers >> 3) Revert the above change and postpone it to when we have a more >> generally useful reason to break ABI (bump zmq_msg_t from 64 to 128 >> bytes for example, Doron?) >> 4) Try to be clever and revert the above change and use something like >> #pragma pack(8). This will fool the ABI checkers (I tried it), and given >> that typedef is only used externally to allocate the right size it >> shouldn't actually affect anything, apart from the users of SPARC64 >> which should get the bugfix with this too. This is very sneaky :-) >> >> CC'ing Lazslo, the Debian maintainer, given what we choose to do might >> result in a lot of work for him :-) >> >> Opinions? >> >> Kind regards, >> Luca Boccassi >> >> On Tue, 2016-05-03 at 10:39 +0200, Pieter Hintjens wrote: >> > Hi all, >> > >> > I'm just throwing some ideas on the table. We have a good package of >> > work on master and it's probably time to make a 4.2 release. >> > >> > Luca has already back-ported the enable/disable draft design from >> > zproject (CZMQ et al). Yay! So we can now release stable master >> > safely, while continuing to refine and extend the draft API sections. >> > >> > I propose: >> > >> > - to end with the stable fork policy; this was needed years ago when >> > we had massively unstable masters. It's no longer a problem. >> > - to use the github release function for libzmq releases and deprecate >> > the separate delivery of tarballs. >> > - we aim to make a 4.2.0 rc asap, then fix any issues we get, with >> > patch releases as usual. >> > - we backport the release function to older maintained releases (4.1, >> > 3.2) so that their tarballs are provided by github instead of >> > downloads.zeromq.org. >> > >> > Problems: >> > >> > - this will break a few things that depend on downloads.zeromq.org. To >> >