Since others have been very capably discussing the issues surrounding "installer packages", I won't add much to that conversation except to make the observation that Go follows an established model for building projects with other's code (eg, ruby gems, python pip, etc). There is nothing wrong with this and I have no problem with developers leveraging these tools for their own projects since it is something that they actively opt in to. The problem comes when an "installer package" that comes from the archive wraps all this up for the user and pulls down code that is isn't verified or maintained by the distribution. If that "installer package" is in main, who is responsible for the authenticity of the downloaded software or for its maintenance?
So, putting the installer package issue aside, juju-core is the first Go software to pursue main inclusion and those responsible for maintenance of the Ubuntu archive realize that we need to be careful with how we proceed to make sure that we set the proper precedents and go down a sustainable path that works for everyone. I'd like to give my perspective on Go in Ubuntu to try to avoid an impasse. Go is marketed as an open source programming language that makes it easy to build simple, reliable, and efficient software. People are excited about it and its clear that we want to support Go in Ubuntu. Interesting software is being written in Go, whether that is juju, scopes, click apps and more. The Go community wants to use golang-gc over golang-gccgo and I recognize that viewpoint. Conversations surrounding Go have been challenging because Go was not designed with traditional OS distribution methods in mind (it statically links its runtime[1], uses remote code imports and encourages embedding code copies), yet we are trying to leverage Go using traditional distibution methods (ie, including Go software in the Ubuntu archive with Canonical support). If we take a step back, I think we have a disconnect where the Go developers may not be fully considering the problems of the Go model with regard to Canonical support while at the same time the traditional OS developers (ie, the ones responsible for the archive and its support) may not fully appreciate the needs of the Go community. Go's model of statically compiling software works fine for developers and administrators who are responsible for supporting their software and I have no objections with providing the tools to make Go development great on Ubuntu. I believe the developer model works mostly ok with click packages because click is about empowering developers to deliver their software in a manner that is much less dependent on the OS. Go developers can develop using standard Go techniques then package as click and things are mostly fine. That said, I challenge the proponents of Go in Ubuntu to consider how we can have a better developer story for people developing on Ubuntu-- namely, if Ubuntu provides the Go runtime and compiler that developers then use to statically compile their apps, what can we do to alert developers that they (or someone) should recompile when we update our runtime for a security update. While we could probably get away with just saying that developers are solely responsible for tracking Ubuntu's security updates, I can't help but feel we are doing the Go developers on Ubuntu a disservice if we take this stance. Go's model of statically compiling software doesn't work well for packages we deliver using the Ubuntu archive for a number of reasons: 1. static linking means recompiling all applications that use the standard library when there is an update to it 2. Debian has developed a methodology[2] for delivering Go software in its archive and this is available in Ubuntu. Essentially you use modern dh techniques with dh_golang and Build-Depends on whatever golang-*-dev packages you need. The golang-*-dev packages are unpacked into /usr/share/gocode and GOPATH is set to /usr/share/gocode. The compile proceeds as normal, statically linking into a monolithic binary (that inclues the developer's compiled code, the Go runtime and whatever was needed from /usr/share/gocode). At the time of this writing, there are 36 'golang-*-dev' packages in the archive, 18 of which are specified as a Build-Depends in 9 unique source packages. 3. remote imports are possible with Go, but impossible on our buildds, which promotes the use of embedded code copies 4. embedded code copies are extremely problematic for tracking security vulnerabilities since it is impossible to programmatically ascertain all packages that embed a particular source or to know what version was embedded 5. assuming we were able to enumerate all the software that embedded a given Go library along with its version, embedding code copies means we have to patch everything that embedded the software where each may have to be individually patched for the different version Developers and administrators responsible for their own software will of course feel the burden of maintaining software using Go's model, but they are in a position to weigh the costs and benefits and also implement infrastructure surrounding their deployments (ie, they control their remote import trees, have automation for triggering builds, etc). Clearly, the juju team has done some of this already for their own work. I have no doubt that the juju team would support juju for the next 5 years and supporting a single source package and its embedded code copies is not particularly difficult. However, I want to be very careful of any precedents we set here and think about when we will have 50 or 500 Go source packages in Ubuntu that are officially supported. Thinking about the 5 support problems I listed (there could be more), I think there needs to be further discussion for how we want to support Go software in general in the archive. I've given this quite a bit more thought and have softened my stance; I'll list why here since a lot of eyes are on this bug (note, these thoughts are only for officially supported Go software in the archive. Go developers shipping software in universe, as click or some other method are not necessarily limited by this) 1. The static linking problem can be addressed in a couple ways: * We use gccgo since gccgo supports dynamic linking. I've been a supporter of this coming from the perspective that if gccgo works well and we package the libraries in Ubuntu, then supporting Go in Ubuntu is easy because we are really good with this model-- it fits well with our existing practices. I believe gccgo is the most correct approach since Ubuntu engineers are experts in maintaining software in this fashion and because of all the benefits of shared libraries (not least of which is download size, which is increasingly important for the phone). The problem is, Go developers don't use gccgo and as mentioned in this bug, it doesn't work as well as it should. Furthermore, if Ubuntu insists on using gccgo, Go developers will probably avoid Ubuntu. We could take a middleground approach wth gccgo and make golang-gc development really great on Ubuntu, but make it policy that officially supported Go software must use gccgo (and we devote resources to fixes gccgo's deficiencies). I fe ar this hybrid approach will cause trouble for engineering teams though, since people may have to wrestle with two different compilers and it might stifle community contributions from people who only use golang-gc. Using gccgo doesn't solve all the issues either (see below) * We figure out how to have a reasonable support story using golang-go. One idea is that we could consider automatically performing no-change uploads to -proposed with some bug automation if we update the runtime in an SRU/security update. All packages in main are supposed to have a team subscribed to them, so that team would be responsible for verifying the package in -proposed. This seems to be in the spirit of Go development-- teams choosing to use Go are responsible for recompiling and retesting and a team's choice of Go will have to consider this cost. 2. If using golang-gc, it should be policy for main inclusion that Go software follow Debian's GoPackaging[2] without exception. We should all work together and participate with Debian on refining GoPackaging so it addresses our concerns and works as well as it can. If using gccgo, developers should use dynamic linking and we should make sure it works well for developers 3. It should be policy that remote imports not be allowed. This is easy to enforce-- packages with remote imports won't build on the buildd anyway 4. It should be policy that embedding code copies for Go is not allowed and we should block until the embedded code copy is moved to a golang-*-dev package and the package use it. Embedded copies are actively discouraged now with non-Go software, but between the Go paradigm promoting embedding, lack of remote imports and lack of existing Go libraries in the archive, it will be tempting to just embed rather than package as golang-*-dev. Canonical engineers should package their own Go libaries so as a company we can support our own code. If we do this, then we are able to track security vulnerabilties. Eg the security team could add automation to see what is affected (ie needs to be recompiled) and have reasonable reporting. 5. If we do '4', then no-change uploads with golong-gc or dynamic libraries with gccgo solves much of the patch burden If we do this with golang-gc (gccgo would follow established update procedures), then right away if there is a security update or SRU in golang-foo-dev, we can do 'reverse-depends -b golang-foo-dev' to see what needs no change rebuilds. In the short term, these can be uploaded manually. Once the number of security updates and SRUs for Go packages and their corresponding no change upload rebuilds becomes unwieldy, we should look into automating the no change rebuilds. At first we could trigger on all rdepends, but long term, perhaps we could figure out how to declare what changed in the update and have the no change auto builds mechanism detect what needs to be built based on that. I realize that this was a long response, but I've personally come to the conclusion that while I still believe the gccgo approach is most correct, we need to support Go development in a way that is consistent with the wider Go community since that provides the most benefit. Developers may of course choose to use gccgo, but my current thinking is that based on various conversations with Go developers, efforts to improve gccgo might be better spent making golang-go supportable and this necessarily means stretching our existing policies and processes. I don't claim that my ideas fully address all concerns, but I do think they demonstrate there is a path forward and for both sides to reach common ground. If we are in agreement that we can move forward without gccgo, I suggest the various teams work together to define the policies and procedures for moving forward on the mailing list and UDS. [1] http://code.google.com/p/go/issues/detail?id=256 [2] https://wiki.debian.org/MichaelStapelberg/GoPackaging [3] http://golang.org/doc/install/gccgo -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to golang in Ubuntu. https://bugs.launchpad.net/bugs/1267393 Title: [MIR] juju-core, juju-mongodb, gccgo-go, gccgo-4.9, golang To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gccgo-go/+bug/1267393/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
