This is a good illustration of something I've been concerned about for a while: the fact that pulling in a package also pulls in all testing dependencies of that package recursively.
To take github.com/hashicorp/vault/api as an example, I made this little module: -- main.go -- package x import ( _ "github.com/hashicorp/vault/api" ) func main() { } -- go.mod -- package x I ran `go build` on this. It took about a minute and fetched 13 dependencies. `go list -m all` showed 17 modules. I then ran `go mod tidy`. It took about 10 minutes and fetched 258 dependencies. `go list -m all` showed 316 modules. Here's an example of why one of the problematic dependencies was incurred: % go mod why -m labix.org/v2/mgo # labix.org/v2/mgo x github.com/hashicorp/vault/api github.com/hashicorp/vault/api.test github.com/hashicorp/vault/helper/builtinplugins github.com/hashicorp/vault/builtin/logical/nomad github.com/hashicorp/nomad/api github.com/hashicorp/nomad/nomad/structs github.com/hashicorp/go-msgpack/codec github.com/hashicorp/go-msgpack/codec.test labix.org/v2/mgo/bson It's a third-order testing dependency. For the record, the *actual *set of module dependencies of the package being imported is this: github.com/hashicorp/hcl github.com/mitchellh/mapstructure golang.org/x/net golang.org/x/text golang.org/x/time I wonder if it would be possible to adjust the MVS algorithm to avoid taking into account testing dependencies of external modules. cheers, rog. On Thu, 17 Jan 2019 at 11:36, Francis Chuang <f21.gro...@gmail.com> wrote: > Hey everyone, > > I was wondering if I can get some ideas on how to solve this dependency > problem with Go modules. I am using Go 1.11.4 on Linux (Windows Subsystem > for Linux to be exact). > > The first part of the problem is that for one of my libraries, I am > importing github.com/hashicorp/vault/api, which is an api client to > access Vault servers. This package is pretty thin and does not contain a > lot of dependencies. However, the root project at > github.com/hashicorp/vault has quite a few dependencies. > If I delete the current go.mod and go.sum files in my library and start > from scratch: > - I run go mod init github.com/username/mylibrary > - I then run go mod tidy to add missing packages to my go.mod and generate > a go.sum > > Go mod imports the Vault repository at its root ( > github.com/hashicorp/vault) as expected. Unfortunately, this pulls in a > lot of subdependencies and I have a ton of indirect dependencies in my > go.mod and go.sum. This is not unexpected of course as this is how go > modules work. Unfortunately, this does slow down continuous integration > builds and is probably not great for Github and VCS hosts as each build > pulls in all of these unneeded dependencies. > > The second problem is that 2 of Vault's subdependencies (not sure how > deep) is labix.org/v2/mgo and launchpad.net/gocheck. These are bazaar > repos, so they necessitate the installation of the bzr tool. In my library, > I added the following to my go.mod to force it to pull from the Github > repos: > > replace labix.org/v2/mgo => github.com/go-mgo/mgo > v0.0.0-20160801194620-b6121c6199b7 > > replace launchpad.net/gocheck => github.com/go-check/check > v0.0.0-20180628173108-788fd7840127 > > This works great for running tests in the username/mylibrary repo as I no > longer need to install bzr when running my CI builds. However, if I create > another project such as github.com/username/some-project and import > username/mylibrary, go mod will attempt to download launchpad.net/gocheck > and labix.org/v2/mgo from the original bazaar repos when running go test > and other commands.These causes errors such as `go: > labix.org/v2/mgo@v0.0.0-20140701140051-000000000287: bzr branch > --use-existing-dir https://launchpad.net/mgo/v2 . in > /go/pkg/mod/cache/vcs/ca61c737a32b1e09a0919e15375f9c2b6aa09860cc097f1333b3c3d29e040ea8: > exec: "bzr": executable file not found in $PATH `. This is quite annoying > and I'd like to avoid having to add those `replace` statements in the > go.mod of projects that consume the mylibrary package. In addition, > consumers of my library would need to install bazaar (which most people > probably won't have installed). > > The 2 options I have thought of are: > 1. Ask the Vault team to extract the api package into a separate > repository. I am not sure how likely they would be interested in doing > this, but I am guessing it would be quite low. > 2. Make a copy of the api package and copy it directly into my library. > This is something I am hoping to avoid as much as possible, because I'd > like to use go modules to manage my dependencies. > > Are there any other options I've missed? > > Cheers, > Francis > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.