[go-nuts] Re: Go install speedup within docker
Hi Vlad, I am using go install and the pkg directory is being mapped into the container. docker run -ti --rm \ -v "$PWD":/usr/src/pjt \ # Mount the entire project to this directory -w /usr/src/pjt \ # use the project dir as the cwd -e "GOPATH=/usr/src/pjt" \ # set the GOPATH the base of the project dir -e "STATICBUILD=1" \ -e "PREBUILD_DEPS=$PREBUILD_DEPS" \ gobuilder:1.8-alpine \ ./build.sh Then in build.sh (naming is misleading and ambiguous as it does not use go build, but instead go install) does a couple of things: # set the go path to he current dir export GOPATH=`pwd` # install the project, give a suffix in the pkg directory for docker built files go install -installsuffix docker \ -v --tags netgo --ldflags '-extldflags "-lm -lstdc++ -static"' \ org/project Looks like I figured it out though: 1) "-installsuffix" effects packages in GOPATH and *GOROOT* - so all of the standard libs were being built newly and GOROOT was in the docker image only, so that was rebuilt every time - and I can only assume this caused everything else to rebuild due to dependencies. I removed this and cut it down from 1 1/2 minutes to 19 seconds. This is not needed in my case as it turns out. 2) The build time for no changes should be closed to no time at all, a couple seconds from my tests. This turned out to be "--tags netgo" which looks like it caused a rebuild of the "net" package and the whole tree on top of that. I was able to remove that and shave the rebuild time down to 3 seconds including docker spin-up. netgo is probably a needed item in other static builds. In the my case it wasn't necessary. Building from alpine docker and then making the resultant binary to run as it's own base image, e.g. dockerfile: FROM scratch COPY bin/pjt /bin/pjt ENTRYPOINT ["/bin/pjt"] Turning off netgo increased the build by 6 megs, but 43 over 37MB is worth the tradeoff for quick rebuilds. And for the use case where it's using docker - it still works. This procedure is touched on throughout the net, hopefully down the road it is more codified. Best, James On Monday, June 12, 2017 at 3:06:49 AM UTC-7, Vladimir Varankin wrote: > > I think the difference is that I use my project root as my GOPATH, so pkg > directory ($GOPATH/pkg), which stores the build cache, is mounted inside > the container every time I run the build. What if you try > > ``` > docker container run --rm -v $PWD:/usr/src/prj -v $PWD/_build:/usr/pkg ... > ``` > > Doing so, should keep pkg caches between builds. Not sure if it still > valid, but you might consider to switch `go install` (see > https://dave.cheney.net/2014/06/04/what-does-go-build-build) > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Go install speedup within docker
"-pkgdir" flag of "go build" / "go install" could also be useful, although I haven't tried it. On Monday, June 12, 2017 at 1:06:49 PM UTC+3, Vladimir Varankin wrote: > > I think the difference is that I use my project root as my GOPATH, so pkg > directory ($GOPATH/pkg), which stores the build cache, is mounted inside > the container every time I run the build. What if you try > > ``` > docker container run --rm -v $PWD:/usr/src/prj -v $PWD/_build:/usr/pkg ... > ``` > > Doing so, should keep pkg caches between builds. Not sure if it still > valid, but you might consider to switch `go install` (see > https://dave.cheney.net/2014/06/04/what-does-go-build-build) > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Go install speedup within docker
I think the difference is that I use my project root as my GOPATH, so pkg directory ($GOPATH/pkg), which stores the build cache, is mounted inside the container every time I run the build. What if you try ``` docker container run --rm -v $PWD:/usr/src/prj -v $PWD/_build:/usr/pkg ... ``` Doing so, should keep pkg caches between builds. Not sure if it still valid, but you might consider to switch `go install` (see https://dave.cheney.net/2014/06/04/what-does-go-build-build) -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[go-nuts] Re: Go install speedup within docker
Hey Vlad, Here's the docker command: docker run -ti --rm \ -v "$PWD":/usr/src/pjt \ -w /usr/src/pjt \ -e "GOPATH=/usr/src/pjt" -e "STATICBUILD=1" \ -e "PREBUILD_DEPS=$PREBUILD_DEPS" \ gobuilder:1.8-alpine \ ./build.sh Regarding my docker: docker-machine version 0.10.0, build 76ed2a6 Client: Version: 17.03.0-ce API version: 1.26 Go version: go1.7.5 Git commit: 60ccb22 Built:Thu Mar 2 01:11:00 2017 OS/Arch: darwin/amd64 Server: Version: 17.03.0-ce API version: 1.26 (minimum version 1.12) Go version: go1.7.5 Git commit: 3a232c8 Built:Tue Feb 28 07:52:04 2017 OS/Arch: linux/amd64 Experimental: false Running on macOS. Best, James On Sunday, June 11, 2017 at 7:00:18 AM UTC-7, Vladimir Varankin wrote: > > Hey James, > > Could you show the docker run command, which you invoke to enter a > container? > > I use docker to build my application as well, so I just do `docker > container run --rm -ti --volume $PWD:/gocode/src/app --workdir > /gocode/src/app go build`. Doing so with container > reusage or not, I haven't found any speed difference so far. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
