Based on the wiki [1] you have to pass a -mod=vendor flag to the go commands to tell them to search the top-level vendor directory for the module, but that flag says that it causes the information in go.mod to be ignored and that the vendor directory is assumed to include all the right dependencies.
So it seems like if we add some new dependencies managed with Go modules, we would still have to run `go mod vendor` to vendor those dependencies alongside our existing dependencies. However, the description of the `go mod vendor` command says "resets the main module's vendor directory" which makes me think it might delete dependencies that are not in go.mod yet but already exist in the vendor directory. I just tried this, and it tried to fetch the latest versions of all the dependencies already in the vendor directory and replace them. Based on that, it does seem like it's meant to be an all-or-nothing approach, so I'd probably reaffirm that we should continue to vendor as we do today until we've set up go modules properly for all of our current dependencies. However, we should still track the commit hash and/or SemVer release tags somewhere for any new dependencies so that we can easily manage them with go.mod later. - Rawlin [1] https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away On Thu, Jul 11, 2019 at 2:37 PM Dan Kirkwood <[email protected]> wrote: > > I'd hope that the go build tools would treat the vendor directories as they > do today -- one of the places to look for the dependencies you've > included. So changing one to a go module would not affect the rest, and so > we could add any new ones as modules.. It's not all-or-nothing, is it? > > Dan > > On Thu, Jul 11, 2019 at 1:50 PM Rawlin Peters <[email protected]> > wrote: > > > It might be possible to have some dependencies handled via the Go > > module system and some with vendoring, but I haven't been able to find > > any good information related to our specific situation (vendored > > dependencies but without a tool like go dep managing them). It seems > > like most of the information I've found assumes that if you're using > > vendor directories that you must be using something like go dep, and > > the go module system can "take over" your Gopkg.toml and track those > > dependencies as part of the Go module system. So, I don't really know > > what would happen if we tried to set up Go modules for new > > dependencies. It would need to know about our vendor directories for > > build purposes, but it wouldn't have any idea what version/commit hash > > our dependencies are. > > > > Until we set up Go modules properly, we should probably at least save > > the commit hash of any new dependencies we vendor into the repo and > > try to vendor the latest SemVer-tagged release if we can. That would > > make it easier to set up module versioning for those dependencies in > > the future. > > > > - Rawlin > > > > On Thu, Jul 11, 2019 at 11:02 AM Fieck, Brennan > > <[email protected]> wrote: > > > > > > Sure, no reason to change that right now. But with that in mind, would > > > it be a problem for new dependencies to use the Go module system? > > > ________________________________________ > > > From: Rawlin Peters <[email protected]> > > > Sent: Thursday, July 11, 2019 10:55 AM > > > To: [email protected] > > > Subject: Re: [EXTERNAL] Go dependency management > > > > > > Well, I wasn't planning on tackling this right away -- I just wanted > > > to bring it up and get some feedback for whenever I (or some other > > > brave contributor) get the chance to do it. I'd lean towards doing it > > > after the next release since I think we'd like to get another release > > > out pretty soon, and I don't really know how long it will take. I > > > think it's important that we do it for the sake of having good Go > > > dependency management, but I don't think it's very urgent. > > > > > > - Rawlin > > > > > > On Wed, Jul 10, 2019 at 11:13 AM Dave Neuman <[email protected]> wrote: > > > > > > > > This sounds like a good plan, but I wonder if this is something we > > want to > > > > try to do before we cut the next release, or something we want to do > > > > after? > > > > > > > > On Wed, Jul 10, 2019 at 10:31 AM Dan Kirkwood <[email protected]> > > wrote: > > > > > > > > > +1 > > > > > > > > > > On Wed, Jul 10, 2019 at 10:26 AM Fieck, Brennan < > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > +1 sounds fantastic to me > > > > > > ________________________________________ > > > > > > From: Rawlin Peters <[email protected]> > > > > > > Sent: Tuesday, July 9, 2019 1:55 PM > > > > > > To: [email protected] > > > > > > Subject: [EXTERNAL] Go dependency management > > > > > > > > > > > > Hey all, > > > > > > > > > > > > Now that we've upgraded all our Go components to use v1.11, I > > believe > > > > > > we can start using Go modules [1] to start managing our Go > > > > > > dependencies for the project instead of vendoring our Go > > dependencies > > > > > > directly into our repo. > > > > > > > > > > > > Does anyone have any experience using Go modules yet? After doing > > some > > > > > > quick research on Go modules, I think we would just have to > > identify > > > > > > the commit hash of each of our Go dependencies at the time they > > were > > > > > > cloned into our repo and pin the versions of our dependencies to > > those > > > > > > commit hashes in the go.mod file (since I believe the dependencies > > > > > > were just cloned from the head of master and might not necessarily > > > > > > line up with a SemVer release tag). > > > > > > > > > > > > I think these would be the steps (which could each be broken down > > into > > > > > > multiple PRs if necessary): > > > > > > Step 1: identify commit hashes of our current dependencies, > > implement > > > > > > Go modules to manage them at our existing vendored versions, remove > > > > > > the vendored code from the repo, and fix the build processes to use > > > > > > the new Go module tooling. By the end of this step, the > > dependencies > > > > > > we pull down for the build should match our existing vendored > > > > > > dependencies exactly. > > > > > > Step 2: reevaluate the current versions of our dependencies in > > order > > > > > > to update them to compatible SemVer release tags provided by the > > > > > > dependency (rather than a specific commit hash) if necessary. Once > > > > > > pinned to specific SemVer versions, it should be easy to always > > keep > > > > > > our Go dependencies up to date, although some dependencies might > > not > > > > > > have implemented the required SemVer release tagging yet. > > > > > > > > > > > > Does this sound like a reasonable plan to everyone? I don't have > > any > > > > > > experience using Go modules yet, so if anyone has experience with > > them > > > > > > and has better ideas I'd love to get their input on this. Unless > > there > > > > > > are any objections within the next few days, I'll assume everyone > > is > > > > > > generally on board with Go modules, and someone could start > > working on > > > > > > this after that time. > > > > > > > > > > > > Thanks, > > > > > > Rawlin > > > > > > > > > > > > [1] > > > > > > > > > > > > > https://protect2.fireeye.com/url?k=5da6d7fab5a4c93e.5da6f04e-17c163f89d22e415&u=https://github.com/golang/go/wiki/Modules > > > > > > > > > > > > >
