Bug#961814: marked as pending in golang-google-protobuf
On Fri, Jan 08, 2021 at 11:30:21PM +0800, Shengjing Zhu wrote: On Fri, Jan 8, 2021 at 11:17 PM Alberto Bertogli wrote: On Thu, Jan 07, 2021 at 11:12:24PM +0800, Shengjing Zhu wrote: >On Thu, Jan 7, 2021 at 10:59 PM Alberto Bertogli > wrote: >> But those issues are not made worse by allowing golang-google-protobuf >> to go in, right? >> > >Let golang-google-protobuf go in is one thing, it's not difficult. >However without golang-goprotobuf 1.4.x it's not useful currently. But >it will be changed if upstream has switched to golang-google-protobuf >only. But IIUC that's what foka@'s lastest changes do - now the two packages are independent so golang-google-protobuf can go in? This would unblock: 1) Some upstream packages that have upgraded to the new library and are using pregenerated pb.go without the dependency. 2) Packages that have moved/want to move to the new library and generate pb.go as part of the build (without needing grpc). And no packages are forced into anything, since upstream needs to do the update explicitly anyway, the ones using golang-goprotobuf will continue to function just fine. I understand not all problems are fixed and some things remain, but it seems it'd be a step in the right direction since at least some packages will be able to move forward, without causing any new complications/regressions. Or am I missing something? You are right. The only concern from my side is the usefulness of golang-google-protobuf without upgrading golang-goprotobuf to 1.4. If some packages are already ready for using golang-google-protobuf solely, sure we should try. Yeah, the reason I'm personally interested in this is I have one package I'm upstream for (chasquid) in this situation. I've already done the required patching and is blocked on this package. I am also waiting on the change to move other projects forward for the same reason :) Let me know if there's anything I can do, or how can I help! Thanks again! Alberto
Bug#961814: marked as pending in golang-google-protobuf
On Fri, Jan 8, 2021 at 11:17 PM Alberto Bertogli wrote: > > On Thu, Jan 07, 2021 at 11:12:24PM +0800, Shengjing Zhu wrote: > >On Thu, Jan 7, 2021 at 10:59 PM Alberto Bertogli > > wrote: > >> But those issues are not made worse by allowing golang-google-protobuf > >> to go in, right? > >> > > > >Let golang-google-protobuf go in is one thing, it's not difficult. > >However without golang-goprotobuf 1.4.x it's not useful currently. But > >it will be changed if upstream has switched to golang-google-protobuf > >only. > > But IIUC that's what foka@'s lastest changes do - now the two packages > are independent so golang-google-protobuf can go in? > > This would unblock: > 1) Some upstream packages that have upgraded to the new library and are > using pregenerated pb.go without the dependency. > 2) Packages that have moved/want to move to the new library and generate > pb.go as part of the build (without needing grpc). > > And no packages are forced into anything, since upstream needs to do the > update explicitly anyway, the ones using golang-goprotobuf will continue > to function just fine. > > I understand not all problems are fixed and some things remain, but it > seems it'd be a step in the right direction since at least some packages > will be able to move forward, without causing any new > complications/regressions. > > Or am I missing something? > You are right. The only concern from my side is the usefulness of golang-google-protobuf without upgrading golang-goprotobuf to 1.4. If some packages are already ready for using golang-google-protobuf solely, sure we should try. -- Shengjing Zhu
Bug#961814: marked as pending in golang-google-protobuf
On Thu, Jan 07, 2021 at 11:12:24PM +0800, Shengjing Zhu wrote: On Thu, Jan 7, 2021 at 10:59 PM Alberto Bertogli wrote: But those issues are not made worse by allowing golang-google-protobuf to go in, right? Let golang-google-protobuf go in is one thing, it's not difficult. However without golang-goprotobuf 1.4.x it's not useful currently. But it will be changed if upstream has switched to golang-google-protobuf only. But IIUC that's what foka@'s lastest changes do - now the two packages are independent so golang-google-protobuf can go in? This would unblock: 1) Some upstream packages that have upgraded to the new library and are using pregenerated pb.go without the dependency. 2) Packages that have moved/want to move to the new library and generate pb.go as part of the build (without needing grpc). And no packages are forced into anything, since upstream needs to do the update explicitly anyway, the ones using golang-goprotobuf will continue to function just fine. I understand not all problems are fixed and some things remain, but it seems it'd be a step in the right direction since at least some packages will be able to move forward, without causing any new complications/regressions. Or am I missing something? Thanks! Alberto
Bug#961814: marked as pending in golang-google-protobuf
On Thu, Jan 7, 2021 at 11:12 PM Shengjing Zhu wrote: > > On Thu, Jan 7, 2021 at 10:59 PM Alberto Bertogli > wrote: > > But those issues are not made worse by allowing golang-google-protobuf > > to go in, right? > > > > Let golang-google-protobuf go in is one thing, it's not difficult. > However without golang-goprotobuf 1.4.x it's not useful currently. But > it will be changed if upstream has switched to golang-google-protobuf > only. I think one thing possible is that, some packages' pb.go(which doesn't use grpc) can be regenerated by protoc-gen-go/1.25.0+git20201208.160c747-1, which will use golang-google-protobuf-dev/1.25.0+git20201208.160c747-1 only. The downside is only that we are ahead of most packages' upstream. But it doesn't hurt much. -- Shengjing Zhu
Bug#961814: marked as pending in golang-google-protobuf
On Thu, Jan 7, 2021 at 10:59 PM Alberto Bertogli wrote: > But those issues are not made worse by allowing golang-google-protobuf > to go in, right? > Let golang-google-protobuf go in is one thing, it's not difficult. However without golang-goprotobuf 1.4.x it's not useful currently. But it will be changed if upstream has switched to golang-google-protobuf only. > What's blocking golang-google-protobuf from moving to testing now that > the dependency on golang-goprotobuf is no longer there? > > Is the issue that golang-google-protobuf source also builds a > protoc-gen-go? > protoc-gen-go has been splitted and built by golang-google-protobuf. But most upstream currently still use golang-goprotobuf to generate pb.go, and haven't switched to use protoc-gen-go provided by golang-google-protobuf, and protoc-gen-go-grpc provided by golang-google-grpc-dev(which is not built by Debian currently). > Why can't the latter be split, so upstreams that include pre-generated > .pb.go (as is the official recommendation) can be included in Debian > as-is using the corresponding dependency? > I doubt it will work. Most upstream pb.go which is generated by new protoc-gen-go(provided by golang-goprotobuf/1.4), depends on both golang-goprotobuf and golang-google-protobuf, and need golang-goprotobuf/1.4 -- Shengjing Zhu
Bug#961814: marked as pending in golang-google-protobuf
On Thu, Jan 07, 2021 at 04:43:43PM +0800, Shengjing Zhu wrote: On Thu, Jan 7, 2021 at 2:51 PM Anthony Fok wrote: Hi Shengjing, On Tue, Jan 5, 2021 at 8:42 AM Shengjing Zhu wrote: > > Hi Anthony, Thanks for writing to me! Sorry for the late reply. I was going to re-open this with the pseudo header "Control: reopen -1" to keep this new version out of testing (buster), but then I decided to study the issue further, I think we are safe to move forward, allowing golang-google-protobuf to enter testing, while keeping the old golang-goprotobuf 1.3.x if necessary. > The reason that I haven't uploaded new version of this package, is > that using golang-google-protobuf usually means using > golang-goprotobuf 1.4+ as well. Looks like that is not the case any more. While golang-google-protobuf and protoc-gen-go v1.25.0 would still pull in the old golang-goprotobuf 1.4+ package in the generated file, apparently for backward compatibility during the 6-month transition period that Joe Tsai @dsnet set: import ( proto "github.com/golang/protobuf/proto" ... ) Well, I think we have good news! 1.25.0+git20201208.160c747 doesn't do that any more. The import proto "github.com/golang/protobuf/proto" line is gone; the "const _ = proto.ProtoPackageIsVersion4" is also gone. The latest golang-google-protobuf makes a clean break from the past. It is now self-contained, and no longer pulls in the legacy golang-goprotobuf. It is just like what you predicted: once GitHub issue #1077 is fixed, we can move forward. Thank you for your packaging of golang-google-protobuf which also allows the clean separation with the old golang-goprotobuf ecosystem. The problem is currently all upstream still use golang-goprotobuf to generate pb.go, not the plugin from https://github.com/protocolbuffers/protobuf-go/tree/master/cmd/protoc-gen-go If we choose to use this plugin ahead of upstream, and leave golang-goprotobuf not updated(and not used in B-D), then things will be fine. But upstream says there are still missing features like grpc support, which are moved to grpc package itself. I haven't looked at this part. However it will become a difficult transition, which involves golang-google-grpc-dev, golang-google-genproto-dev, etc... But those issues are not made worse by allowing golang-google-protobuf to go in, right? What's blocking golang-google-protobuf from moving to testing now that the dependency on golang-goprotobuf is no longer there? Is the issue that golang-google-protobuf source also builds a protoc-gen-go? Why can't the latter be split, so upstreams that include pre-generated .pb.go (as is the official recommendation) can be included in Debian as-is using the corresponding dependency? Thanks! Alberto
Bug#961814: marked as pending in golang-google-protobuf
On Thu, Jan 7, 2021 at 2:51 PM Anthony Fok wrote: > > Hi Shengjing, > > On Tue, Jan 5, 2021 at 8:42 AM Shengjing Zhu wrote: > > > > Hi Anthony, > > Thanks for writing to me! Sorry for the late reply. > I was going to re-open this with the pseudo header "Control: reopen > -1" to keep this new version out of testing (buster), but then I > decided to study the issue further, I think we are safe to move > forward, allowing golang-google-protobuf to enter testing, while > keeping the old golang-goprotobuf 1.3.x if necessary. > > > The reason that I haven't uploaded new version of this package, is > > that using golang-google-protobuf usually means using > > golang-goprotobuf 1.4+ as well. > > Looks like that is not the case any more. > While golang-google-protobuf and protoc-gen-go v1.25.0 would still > pull in the old golang-goprotobuf 1.4+ package in the generated file, > apparently for backward compatibility during the 6-month transition > period that Joe Tsai @dsnet set: > > import ( > proto "github.com/golang/protobuf/proto" > ... > ) > > Well, I think we have good news! > > 1.25.0+git20201208.160c747 doesn't do that any more. > The import proto "github.com/golang/protobuf/proto" line is gone; > the "const _ = proto.ProtoPackageIsVersion4" is also gone. > > The latest golang-google-protobuf makes a clean break from the past. > It is now self-contained, and no longer pulls in the legacy golang-goprotobuf. > It is just like what you predicted: once GitHub issue #1077 is fixed, > we can move forward. > > Thank you for your packaging of golang-google-protobuf which also > allows the clean separation with the old golang-goprotobuf ecosystem. > The problem is currently all upstream still use golang-goprotobuf to generate pb.go, not the plugin from https://github.com/protocolbuffers/protobuf-go/tree/master/cmd/protoc-gen-go If we choose to use this plugin ahead of upstream, and leave golang-goprotobuf not updated(and not used in B-D), then things will be fine. But upstream says there are still missing features like grpc support, which are moved to grpc package itself. I haven't looked at this part. However it will become a difficult transition, which involves golang-google-grpc-dev, golang-google-genproto-dev, etc... > > However golang-goprotobuf 1.4+ breaks important packages like > > golang-gogottrpc and golang-gogoprotobuf. > > See that bugs on > > https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no=golang-goprotobuf > > Since golang-google-protobuf 1.25.0+git20201208.160c747 does not > require golang-goprotobuf 1.4+ any more, we can keep golang-goprotobuf > at 1.3.4-2 for buster to be safe. > > I did try running ratt with a local test build of golang-goprotobuf > 1.4.2, and was surprised that golang-gogottrpc built fine with it: > > 2021/01/05 18:14:30 PASSED: golang-gogottrpc > > while golang-gogoprotobuf is not touched at all. Am I missing something? If I read this comment correctly https://github.com/containerd/ttrpc/issues/62#issuecomment-686768932 When golang-goprotobuf is updated to 1.4.x, then golang-google-genproto-dev is rebuilt with golang-goprotobuf/1.14, golang-gogoprotobuf will panic, and then golang-gogottrpc panics too. -- Shengjing Zhu
Bug#961814: marked as pending in golang-google-protobuf
Hi Shengjing, On Tue, Jan 5, 2021 at 8:42 AM Shengjing Zhu wrote: > > Hi Anthony, Thanks for writing to me! Sorry for the late reply. I was going to re-open this with the pseudo header "Control: reopen -1" to keep this new version out of testing (buster), but then I decided to study the issue further, I think we are safe to move forward, allowing golang-google-protobuf to enter testing, while keeping the old golang-goprotobuf 1.3.x if necessary. > The reason that I haven't uploaded new version of this package, is > that using golang-google-protobuf usually means using > golang-goprotobuf 1.4+ as well. Looks like that is not the case any more. While golang-google-protobuf and protoc-gen-go v1.25.0 would still pull in the old golang-goprotobuf 1.4+ package in the generated file, apparently for backward compatibility during the 6-month transition period that Joe Tsai @dsnet set: import ( proto "github.com/golang/protobuf/proto" ... ) Well, I think we have good news! 1.25.0+git20201208.160c747 doesn't do that any more. The import proto "github.com/golang/protobuf/proto" line is gone; the "const _ = proto.ProtoPackageIsVersion4" is also gone. The latest golang-google-protobuf makes a clean break from the past. It is now self-contained, and no longer pulls in the legacy golang-goprotobuf. It is just like what you predicted: once GitHub issue #1077 is fixed, we can move forward. Thank you for your packaging of golang-google-protobuf which also allows the clean separation with the old golang-goprotobuf ecosystem. > However golang-goprotobuf 1.4+ breaks important packages like > golang-gogottrpc and golang-gogoprotobuf. > See that bugs on > https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no=golang-goprotobuf Since golang-google-protobuf 1.25.0+git20201208.160c747 does not require golang-goprotobuf 1.4+ any more, we can keep golang-goprotobuf at 1.3.4-2 for buster to be safe. I did try running ratt with a local test build of golang-goprotobuf 1.4.2, and was surprised that golang-gogottrpc built fine with it: 2021/01/05 18:14:30 PASSED: golang-gogottrpc while golang-gogoprotobuf is not touched at all. Am I missing something? But yes, I'm sure you are much more familiar with golang-goprotobuf, so if 1.4+ is indeed dangerous to other legacy packages, we'll just have to make sure no one in the Go Team uploads golang-goprotobuf 1.4+ for buster. Cheers, Anthony
Bug#961814: marked as pending in golang-google-protobuf
Hi Anthony, On Tue, Jan 5, 2021 at 11:27 PM Anthony Fok wrote: > > Control: tag -1 pending > > Hello, > > Bug #961814 in golang-google-protobuf reported by you has been fixed in the > Git repository and is awaiting an upload. You can see the commit > message below and you can check the diff of the fix at: > > https://salsa.debian.org/go-team/packages/golang-google-protobuf/-/commit/674e3bd39673f62ccce5786c1bf8a1094a86d718 > > > debian/watch: Fetch from Git HEAD temporarily > > to get fixes for issues #1077 and #1168, see > https://github.com/golang/protobuf/issues/1168 and > https://github.com/golang/protobuf/issues/1077 > > Closes: #961814 > > > (this message was generated automatically) > -- > Greetings > > https://bugs.debian.org/961814 The reason that I haven't uploaded new version of this package, is that using golang-google-protobuf usually means using golang-goprotobuf 1.4+ as well. However golang-goprotobuf 1.4+ breaks important packages like golang-gogottrpc and golang-gogoprotobuf. See that bugs on https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no=golang-goprotobuf Meanwhile packages which need golang-google-protobuf can be easily fixed by regenerating protobuf files, or changing import path to old one. -- Shengjing Zhu
