Re: [osv-dev] [PATCH] Manage Dependency with Go Modules and Update APIs to Latest
On Mon, Dec 9, 2019 at 3:53 PM kayhaw wrote: > As a Go project, Capstan uses godep as its dependency manager, > however, as of Go 1.11, Go's new dependency management system(Go > Modules) is introduced officially, more and more Go projects use Go > Modules to make dependency version information explicit and easier to > manage, this patch makes Capstan migrate to Go Modules and update > codes to use the latest APIs, here are details: > > - Delete Godeps directory, add go.mod and go.sum file, which are > generated by `go mod` command and declare Capstan project is managed > with Go Modules now > - Update packages import path, change urfave/cli from v1 to v2, change > cheggaaa/pb from v0 to v3, all packages are newest released version > - Update code to adapt new APIs according to the old/new API doc > - Update the installation.md, use Go 1.11+ to build/manage Capstan on > Module Mode > > See [Go Modules Blogs](https://blog.golang.org/using-go-modules) to > learn more > > Signed-off-by: kayhaw > Waldek, I did not review the details, but the overall approach makes sense: Acked-by: Pekka Enberg -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CAGUyND-QAOgw1xB%2BqWjR%3Dej%3DiThj2mGHYShe9bAATFL1P%3DWp4g%40mail.gmail.com.
[osv-dev] [PATCH] Capstan:Migrate to Go Modules and Update APIs
As a Go project, Capstan uses godep as its dependency manager,
however, as of Go 1.11, Go's new dependency management system(Go
Modules) is introduced officially, more and more Go projects use Go
Modules to make dependency version information explicit and easier to
manage, this patch makes Capstan migrate to Go Modules and update
codes to use the latest APIs, here are details:
- Delete Godeps directory, add go.mod and go.sum file, which are
generated by `go mod` command and declare Capstan project is managed
with Go Modules now
- Update packages import path, change urfave/cli from v1 to v2, change
cheggaaa/pb from v0 to v3, all packages are newest released version
- Update code to adapt new APIs according to the old/new API doc
- Update the installation.md, use Go 1.11+ to build/manage Capstan on
Module Mode
See [Go Modules Blogs](https://blog.golang.org/using-go-modules) to
learn more
Signed-off-by: kayhaw
---
Documentation/Installation.md| 25 ++-
Godeps/Godeps.json | 38 -
Godeps/Readme| 5 -
capstan.go | 238 +--
cmd/build.go | 2 +-
cmd/compose.go | 7 +-
cmd/config.go| 4 +-
cmd/stack.go | 6 +-
go.mod | 14 ++
go.sum | 59 +++
provider/openstack/openstack_auth.go | 16 +-
util/remote_repository.go| 4 +-
util/repository.go | 10 +-
13 files changed, 234 insertions(+), 194 deletions(-)
delete mode 100644 Godeps/Godeps.json
delete mode 100644 Godeps/Readme
create mode 100644 go.mod
create mode 100644 go.sum
diff --git a/Documentation/Installation.md b/Documentation/Installation.md
index 41c438b..d5bd990 100644
--- a/Documentation/Installation.md
+++ b/Documentation/Installation.md
@@ -46,14 +46,14 @@ There you go, happy unikernel creating!
## Install Capstan from source (advanced)
-### Install Go 1.7+
+### Install Go 1.11+
Capstan is a Go project and needs to be compiled first. But heads up,
compiling Go project is trivial,
as long as you have Go installed. Consult [official
documentation](https://golang.org/doc/install)
to learn how to install Go, or use this bash snippet to do it for you:
```bash
-curl https://storage.googleapis.com/golang/go1.12.6.linux-amd64.tar.gz | sudo
tar xz -C /usr/local
-sudo mv /usr/local/go /usr/local/go1.12
-sudo ln -s /usr/local/go1.12 /usr/local/go
+curl https://dl.google.com/go/go1.13.5.linux-amd64.tar.gz | sudo tar xz -C
/usr/local
+sudo mv /usr/local/go /usr/local/go1.13
+sudo ln -s /usr/local/go1.13 /usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$PATH
@@ -63,15 +63,24 @@ export PATH=/usr/local/go/bin:$PATH
### Compile Capstan
Since Capstan is hosted on GitHub, the compilation process is as simple as:
```
-go get github.com/cloudius-systems/capstan
-go install github.com/cloudius-systems/capstan
+git clone g...@github.com:cloudius-systems/capstan.git
+cd capstan
+go env -w GO111MODULE=on# Needed for Go 1.11, 1.12
+go install
```
-That's it, we have Capstan installed. You should be able to use Capstan
immediately because it was
-installed in `$GOPATH/bin` added to your `$PATH` above. To test that it works,
try:
+That's it, we have Capstan installed. You should be able to use Capstan
immediately because it was installed in `$GOPATH/bin` added to your `$PATH`
above. To test that it works, try:
```
capstan --help
```
+## Maintain Capstan
+Capstan is managed with Go Modules as
[recommended](https://blog.golang.org/using-go-modules), while developing on
Capstan, you should follow these steps:
+1. Update import path and corresponding code as you might do
+2. Test you changes with `go test ./...`
+3. Clear up `go.mod` and `go.sum` file with `go mod tidy`
+
+Then you are ready to tag and release new version. To learn more details about
maintaining Go project on Module Mode, see [Go Modules Serial
Blogs](https://blog.golang.org/using-go-modules)
+
## Configure Capstan (advanced)
Capstan uses optimized default values under the hood. But you are allowed to
override them with
your own values and this section describes how. Actually, there are three ways
to override them
diff --git a/Godeps/Godeps.json b/Godeps/Godeps.json
deleted file mode 100644
index e39f221..000
--- a/Godeps/Godeps.json
+++ /dev/null
@@ -1,38 +0,0 @@
-{
- "ImportPath": "github.com/cloudius-systems/capstan",
- "GoVersion": "go1.7",
- "Packages": [
- "./..."
- ],
- "Deps": [
- {
- "ImportPath": "github.com/cheggaaa/pb",
- "Rev": "a9a708f18ae43493d86f3fd5b0cd04df679b170b"
- },
- {
- "ImportPath": "github.com/olekukonko/ts",
- "Rev":
[osv-dev] Re: [PATCH] Fix a file-path typo in the primary README
My apologizes, not sure why this sent on this thread. On Monday, December 9, 2019 at 10:59:56 PM UTC-6, Mason Davis wrote: > > The path is slightly incorrect and can cause issues if someone where to > copy and paste this code. > > Signed-off-by: Mason Davis > > --- > README.md | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/README.md b/README.md > index d51c5e08..a4708c4e 100644 > --- a/README.md > +++ b/README.md > @@ -90,13 +90,13 @@ The build script can be used like so per the examples > below: > ./scripts/build JAVA_VERSION=10 > image=openjdk-zulu-9-and-above,spring-boot-example > > # Create image with 'ls' executable taken from the host > -./scripts/manifest_from_host.sh -w ls && ./script/build --append-manifest > +./scripts/manifest_from_host.sh -w ls && ./scripts/build > --append-manifest > > # Create test image and run all tests in it > -./script/build check > +./scripts/build check > > # Clean the build tree > -./script/build clean > +./scripts/build clean > ``` > > Command nproc will calculate the number of jobs/threads for make and > scripts/build automatically. > -- > 2.24.0 > > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/98a25a3d-09df-407d-92f1-5b35e80561c1%40googlegroups.com.
[osv-dev] [PATCH] Fix a file-path typo in the primary README
The path is slightly incorrect and can cause issues if someone where to copy and paste this code. Signed-off-by: Mason Davis --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d51c5e08..a4708c4e 100644 --- a/README.md +++ b/README.md @@ -90,13 +90,13 @@ The build script can be used like so per the examples below: ./scripts/build JAVA_VERSION=10 image=openjdk-zulu-9-and-above,spring-boot-example # Create image with 'ls' executable taken from the host -./scripts/manifest_from_host.sh -w ls && ./script/build --append-manifest +./scripts/manifest_from_host.sh -w ls && ./scripts/build --append-manifest # Create test image and run all tests in it -./script/build check +./scripts/build check # Clean the build tree -./script/build clean +./scripts/build clean ``` Command nproc will calculate the number of jobs/threads for make and scripts/build automatically. -- 2.24.0 -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/20191210045931.6646-2-mason.davis%40utexas.edu.
[osv-dev] [PATCH] Add bare .travis.yml for testing build
After this patch, subsequent commits to the osv GitHub Repo should trigger a build at https://travis-ci.org/cloudius-systems/osv. At the moment, the build consists of building the Ubuntu Docker image and building osv. This can easily be expanded upon by adding new entries to the .travis.yml. Addtionally, this could also be changed to not use the Docker image, if desired. Future expansion of this functionality is detailed here: https://github.com/cloudius-systems/osv/wiki/Automated-Testing-Framework The goal would be to build and test various apps, to ensure the change doesn't make any major regressions. Refs #1044 Signed-off-by: Mason Davis --- .travis.yml | 9 + 1 file changed, 9 insertions(+) create mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml new file mode 100644 index ..fcaf5bce --- /dev/null +++ b/.travis.yml @@ -0,0 +1,9 @@ +sudo: required +language: c +services: + - docker +before_install: + - cd docker && docker build -t osv/builder -f ./Dockerfile.Ubuntu . + - docker run -it --privileged -d --name build osv/builder +script: + - docker exec build ./scripts/build -- 2.24.0 -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/20191210045931.6646-1-mason.davis%40utexas.edu.
[osv-dev] [PATCH] Capstan:Migrate to Go Modules and Update APIs
As a Go project, Capstan uses godep as its dependency manager, however, as of Go 1.11, Go's new dependency management system(Go Modules) is introduced officially, more and more Go projects use Go Modules to make dependency version information explicit and easier to manage, this patch makes Capstan migrate to Go Modules and update codes to use the latest APIs, here are details: - Delete Godeps directory, add go.mod and go.sum file, which are generated by `go mod` command and declare Capstan project is managed with Go Modules now - Update packages import path, change urfave/cli from v1 to v2, change cheggaaa/pb from v0 to v3, all packages are newest released version - Update code to adapt new APIs according to the old/new API doc - Update the installation.md, use Go 1.11+ to build/manage Capstan on Module Mode See [Go Modules Blogs](https://blog.golang.org/using-go-modules) to learn more Signed-off-by: kayhaw --- go.mod | 14 ++ go.sum | 59 ++ 2 files changed, 73 insertions(+) create mode 100644 go.mod create mode 100644 go.sum diff --git a/go.mod b/go.mod new file mode 100644 index 000..6374e48 --- /dev/null +++ b/go.mod @@ -0,0 +1,14 @@ +module github.com/cloudius-systems/capstan + +go 1.13 + +require ( + github.com/cheggaaa/pb/v3 v3.0.3 + github.com/gophercloud/gophercloud v0.7.0 + github.com/kr/pretty v0.1.0 // indirect + github.com/mattn/go-colorable v0.1.4 // indirect + github.com/urfave/cli/v2 v2.0.0 + gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 + gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce + gopkg.in/yaml.v2 v2.2.7 +) diff --git a/go.sum b/go.sum new file mode 100644 index 000..1cc10fe --- /dev/null +++ b/go.sum @@ -0,0 +1,59 @@ +github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= +github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM= +github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmxzcbUokwA= +github.com/cheggaaa/pb/v3 v3.0.3 h1:8WApbyUmgMOz7WIxJVNK0IRDcRfAmTxcEdi0TuxjdP4= +github.com/cheggaaa/pb/v3 v3.0.3/go.mod h1:Pp35CDuiEpHa/ZLGCtBbM6CBwMstv1bJlG884V+73Yc= +github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d h1:U+s90UTSYgptZMwQh2aRr3LuazLJIa+Pg3Kc1ylSYVY= +github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys= +github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/gophercloud/gophercloud v0.7.0 h1:vhmQQEM2SbnGCg2/3EzQnQZ3V7+UCGy9s8exQCprNYg= +github.com/gophercloud/gophercloud v0.7.0/go.mod h1:gmC5oQqMDOMO1t1gq5DquX/yAU808e/4mzjjDA76+Ss= +github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= +github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA= +github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= +github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= +github.com/mattn/go-isatty v0.0.10 h1:qxFzApOv4WsAL965uUPIsXzAKCZxN2p9UqdhFS4ZW10= +github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84= +github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k= +github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q= +github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo= +github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= +github.com/urfave/cli/v2 v2.0.0 h1:+HU9SCbu8GnEUFtIBfuUNXN39ofWViIEJIp6SURMpCg= +github.com/urfave/cli/v2 v2.0.0/go.mod h1:SE9GqnLQmjVa0iPEY0f1w3ygNIYcIJ0OKPMoW2caLfQ= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20191202143827-86a70503ff7e/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/net
[osv-dev] [PATCH] tests: stop using boost from externals
This patch makes necessary changes to th tests module Makefile
to replace "antique" version of boost with its version in host.
It is also another step in the effort to eliminate usage of artifacts in
externals
as described by the issue #743.
Refs #743
Signed-off-by: Waldemar Kozaczuk
---
modules/tests/Makefile | 12 ++--
modules/tests/usr.manifest.skel | 3 ---
2 files changed, 6 insertions(+), 9 deletions(-)
diff --git a/modules/tests/Makefile b/modules/tests/Makefile
index e72b68c2..6fe414bd 100644
--- a/modules/tests/Makefile
+++ b/modules/tests/Makefile
@@ -162,6 +162,8 @@ $(out)/tests/tst-tls-pie.so: \
$(makedir)
$(call quiet, cd $(out); $(CXX) $(CXXFLAGS) -fuse-ld=bfd -pthread -pie
-o $@ $< tests/libtls.so, LD tests/tst-tls-pie.so)
+boost-lib-dir := $(dir $(shell $(CC) --print-file-name libboost_system.so))
+
boost-tests := tst-vfs.so tst-libc-locking.so misc-fs-stress.so \
misc-bdev-write.so misc-bdev-wlatency.so misc-bdev-rw.so \
tst-promise.so tst-dlfcn.so tst-stat.so tst-wait-for.so \
@@ -177,12 +179,9 @@ zfs-only-boost-tests := tst-rename.so
boost-tests += $($(fs_type)-only-boost-tests)
-BOOSTLIBS=$(src)/external/$(ARCH)/misc.bin/usr/lib64
$(boost-tests:%=$(out)/tests/%): LIBS += \
-$(BOOSTLIBS)/libboost_unit_test_framework.so \
-$(BOOSTLIBS)/libboost_filesystem.so
-$(boost-tests:%=$(out)/tests/%): CXXFLAGS += -D_GLIBCXX_USE_CXX11_ABI=0 \
- -isystem $(src)/external/$(ARCH)/misc.bin/usr/include
+ -lboost_unit_test_framework \
+ -lboost_filesystem
tests += $(boost-tests)
@@ -220,9 +219,10 @@ build_all_tests: $(all_tests:%=$(out)/%)
.PHONY: build_all_tests clean
# Automatically generate usr.manifest which includes all tests.
-usr.manifest: $(lastword $(MAKEFILE_LIST)) usr.manifest.skel FORCE
+usr.manifest: build_all_tests $(lastword $(MAKEFILE_LIST)) usr.manifest.skel
FORCE
@echo " generating modules/tests/usr.manifest"
@cat $@.skel > $@
+ @ldd $(addprefix $(out)/tests/,$(boost-tests)) | grep libboost | sed
's/ *[^ ] *\(.*\) => \(.*\) .*/\/usr\/lib\/\1: \2/' | sort | uniq >> $@
@echo $(all_tests) | tr ' ' '\n' | grep -v "tests/rofs/tst-.*.so" | awk
'{print "/" $$0 ": ./" $$0}' >> $@
@echo $(all_tests) | tr ' ' '\n' | grep "tests/rofs/tst-.*.so" | sed
's/\.so//' | awk 'BEGIN { FS = "/" } ; { print "/tests/" $$3 "-rofs.so:
./tests/" $$2 "/" $$3 ".so"}' >> $@
./create_static.sh $(out) $(fs_type)
diff --git a/modules/tests/usr.manifest.skel b/modules/tests/usr.manifest.skel
index 2de441f9..8f2c824c 100644
--- a/modules/tests/usr.manifest.skel
+++ b/modules/tests/usr.manifest.skel
@@ -1,4 +1 @@
-/usr/lib/&/libboost_unit_test_framework.so.1.55.0: %(miscbase)s/usr/lib64/&
-/usr/lib/&/libboost_filesystem.so.1.55.0: %(miscbase)s/usr/lib64/&
-/usr/lib/&/libboost_system.so.1.55.0: %(miscbase)s/usr/lib64/&
/testrunner.so: ./tests/testrunner.so
--
2.20.1
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/20191209231857.4906-1-jwkozaczuk%40gmail.com.
[osv-dev] how to find missing symbols if it doesn't print the name
Hey, I'm encountering this when I'm using some tensorflow functions: /lib/python3.6/tensorflow/python/_pywrap_tensorflow_internal.so: failed looking up symbol [backtrace] 0x403442a7 0x40397dce 0x4033eed4 0x40341d27 0x40345623 , std::allocator >, std::vector, std::allocator >, std::allocator, std::allocator > > >, std::vector, std::allocator > >&)+1459> 0x40345e70 , std::allocator >, std::vector, std::allocator >, std::allocator, std::allocator > > >, bool)+336> 0x40465fd8 0x10937228 <_PyImport_FindSharedFuncptr+376> 0x745f70617277796f It seems the name is not print out. If I'm using the check-libcfunc-avail.sh to check the _pywrap_tensorflow_internal.so, I get the following output: pthread_mutex_consistent not found pthread_mutexattr_setrobust not found fmaf not found fma not found mallinfo not found not found DF not found *UND* not found not found (GLIBC_2.2.5) not found mremap not found Originally, it also contains some output of missing complex number related functions. I include those from musl. The list reduces to the above. It would be great to know which symbol it's missing. I'm not sure whether it's looking for a symbol from libc or from other library. Best, Zhiting -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/d84f4991-69d5-45d8-98a2-403257c9e4b2%40googlegroups.com.
[osv-dev] Re: [PATCH 14/16] cloud-init: Added support for Network v1 and ConfigDrive data source
I don't see that this was applied? The reason I ask is that we're using
cloud-init to set the IP addresses on multiple interfaces inside OSv
(to separate database traffic from the protocols). I have a secondary
patch to turn off dhcp that we also use.
I have my own network-module.cc, as I missed this one on the list. I've
just fixed up the cloud-init code in my tree to deal with the
libboost_program_options fall out and *then* I discover this code.
So how to proceed - guidance please? Charles' code is more complete
than mine, so I could try and merge that and get it working on the
current codebase - would that be acceptable?
Rick
On Mon, 2018-08-13 at 12:32 +0300, Nadav Har'El wrote:
> All looks good. Thanks.
>
>
> --
> Nadav Har'El
> n...@scylladb.com
>
> On Tue, Aug 7, 2018 at 5:49 AM, Charles Myers <
> charles.my...@spirent.com> wrote:
> > https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html
> >
> > Currently only interface naming, static IP, routes and DNS are
> > supported.e
> >
> > Signed-off-by: Charles Myers
> > ---
> > modules/cloud-init/Makefile | 2 +-
> > modules/cloud-init/main.cc | 77 +---
> > modules/cloud-init/network-module.cc | 369
> > +++
> > modules/cloud-init/network-module.hh | 43
> > 4 files changed, 465 insertions(+), 26 deletions(-)
> > create mode 100644 modules/cloud-init/network-module.cc
> > create mode 100644 modules/cloud-init/network-module.hh
> >
> > diff --git a/modules/cloud-init/Makefile b/modules/cloud-
> > init/Makefile
> > index daee8c3..ae45b24 100644
> > --- a/modules/cloud-init/Makefile
> > +++ b/modules/cloud-init/Makefile
> > @@ -13,7 +13,7 @@ INCLUDES += -I$(HTTPSERVER_API_DIR)
> >
> > # the build target executable:
> > TARGET = cloud-init
> > -CPP_FILES := client.cc cloud-init.cc data-source.cc main.cc
> > template.cc cassandra-module.cc json.cc
> > +CPP_FILES := client.cc cloud-init.cc data-source.cc main.cc
> > template.cc network-module.cc cassandra-module.cc json.cc
> > OBJ_FILES := $(addprefix obj/,$(CPP_FILES:.cc=.o))
> > DEPS := $(OBJ_FILES:.o=.d)
> >
> > diff --git a/modules/cloud-init/main.cc b/modules/cloud-
> > init/main.cc
> > index 5af23f0..a6674ef 100644
> > --- a/modules/cloud-init/main.cc
> > +++ b/modules/cloud-init/main.cc
> > @@ -9,6 +9,7 @@
> > #include
> > #include
> > #include "cloud-init.hh"
> > +#include "network-module.hh"
> > #include "files-module.hh"
> > #include "server-module.hh"
> > #include "cassandra-module.hh"
> > @@ -24,59 +25,84 @@ using namespace std;
> > using namespace init;
> > namespace po = boost::program_options;
> >
> > -// config_disk() allows to use NoCloud VM configuration method -
> > see
> > +// config_disk() allows to use NoCloud and ConfigDrive VM
> > configuration method - see
> > //
> > http://cloudinit.readthedocs.io/en/0.7.9/topics/datasources/nocloud.html
> > .
> > +//
> > http://cloudinit.readthedocs.io/en/0.7.9/topics/datasources/configdrive.html
> > +//
> > // NoCloud method provides two files with cnfiguration data
> > (/user-data and
> > // /meta-data) on a disk. The disk is required to have label
> > "cidata".
> > // It can contain ISO9660 or FAT filesystem.
> > //
> > +// ConfigDrive (version 2) method uses an unpartitioned VFAT or
> > ISO9660 disk
> > +// with files.
> > +// openstack/
> > +// - 2012-08-10/ or latest/
> > +//- meta_data.json
> > +//- user_data (not mandatory)
> > +// - content/
> > +//- (referenced content files)
> > +//- 0001
> > +//-
> > +// ec2
> > +// - latest/
> > +//- meta-data.json (not mandatory)
> > +//- user-data
> > +//
> > // config_disk() checks whether we have a second disk (/dev/vblkX)
> > with
> > // ISO image, and if there is, it copies the configuration file
> > from
> > -// /user-data to the given file.
> > +// the user user-data file to the given file.
> > // config_disk() returns true if it has successfully read the
> > configuration
> > -// into the requested file. It triest to get configuratioe from
> > first few
> > +// into the requested file. It tries to get configuration from
> > first few
> > // vblk devices, namely vblk1 to vblk10.
> > //
> > // OSv implementation limitations:
> > // The /meta-data file is currently ignored.
> > // Only ISO9660 filesystem is supported.
> > -// The mandatory "cidata" volume label is not checked.
> > +// The mandatory "cidata" (NoCloud) and "config-2" (ConfigDrive)
> > volume labels are not checked.
> > //
> > // Example ISO image can be created by running
> > // cloud-localds cloud-init.img cloud-init.yaml
> > // The cloud-localds command is provided by cloud-utils package
> > (fedora).
> > static bool config_disk(const char* outfile) {
> > +const char * userdata_file_paths[] {
> > +"/user-data", // NoCloud
> > +"/openstack/latest/user_data", // ConfigDrive OpenStack
> > +
[osv-dev] [PATCH] Manage Dependency with Go Modules and Update APIs to Latest
As a Go project, Capstan uses godep as its dependency manager,
however, as of Go 1.11, Go's new dependency management system(Go
Modules) is introduced officially, more and more Go projects use Go
Modules to make dependency version information explicit and easier to
manage, this patch makes Capstan migrate to Go Modules and update
codes to use the latest APIs, here are details:
- Delete Godeps directory, add go.mod and go.sum file, which are
generated by `go mod` command and declare Capstan project is managed
with Go Modules now
- Update packages import path, change urfave/cli from v1 to v2, change
cheggaaa/pb from v0 to v3, all packages are newest released version
- Update code to adapt new APIs according to the old/new API doc
- Update the installation.md, use Go 1.11+ to build/manage Capstan on
Module Mode
See [Go Modules Blogs](https://blog.golang.org/using-go-modules) to
learn more
Signed-off-by: kayhaw
---
Documentation/Installation.md| 22 ++-
Godeps/Godeps.json | 38 -
Godeps/Readme| 5 -
capstan.go | 242 +--
cmd/build.go | 2 +-
cmd/compose.go | 8 +-
cmd/config.go| 4 +-
cmd/stack.go | 6 +-
provider/openstack/openstack_auth.go | 16 +-
util/remote_repository.go| 4 +-
util/repository.go | 16 +-
11 files changed, 166 insertions(+), 197 deletions(-)
delete mode 100644 Godeps/Godeps.json
delete mode 100644 Godeps/Readme
diff --git a/Documentation/Installation.md b/Documentation/Installation.md
index 41c438b..9807033 100644
--- a/Documentation/Installation.md
+++ b/Documentation/Installation.md
@@ -46,14 +46,14 @@ There you go, happy unikernel creating!
## Install Capstan from source (advanced)
-### Install Go 1.7+
+### Install Go 1.11+
Capstan is a Go project and needs to be compiled first. But heads up,
compiling Go project is trivial,
as long as you have Go installed. Consult [official
documentation](https://golang.org/doc/install)
to learn how to install Go, or use this bash snippet to do it for you:
```bash
-curl https://storage.googleapis.com/golang/go1.12.6.linux-amd64.tar.gz | sudo
tar xz -C /usr/local
-sudo mv /usr/local/go /usr/local/go1.12
-sudo ln -s /usr/local/go1.12 /usr/local/go
+curl https://dl.google.com/go/go1.13.5.linux-amd64.tar.gz | sudo tar xz -C
/usr/local
+sudo mv /usr/local/go /usr/local/go1.13
+sudo ln -s /usr/local/go1.13 /usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$PATH
@@ -63,8 +63,10 @@ export PATH=/usr/local/go/bin:$PATH
### Compile Capstan
Since Capstan is hosted on GitHub, the compilation process is as simple as:
```
-go get github.com/cloudius-systems/capstan
-go install github.com/cloudius-systems/capstan
+git clone g...@github.com:cloudius-systems/capstan.git
+cd capstan
+go env -w GO111MODULE=on# Needed for Go 1.11, 1.12
+go install
```
That's it, we have Capstan installed. You should be able to use Capstan
immediately because it was
installed in `$GOPATH/bin` added to your `$PATH` above. To test that it works,
try:
@@ -72,6 +74,14 @@ installed in `$GOPATH/bin` added to your `$PATH` above. To
test that it works, t
capstan --help
```
+## Maintain Capstan
+Capstan is managed with Go Moudles as
[recommended](https://blog.golang.org/using-go-modules), while developing on
Capstan, you should follow these steps:
+1. Update import path and corresponding code as you might do
+2. Test you changes with `go test ./...`
+3. Clear up `go.mod` and `go.sum` file with `go mod tidy`
+
+Now you are ready to tag and release new version. To learn more details about
maintaining your Go project on Module Mode, see [Go Modules Serial
Blogs](https://blog.golang.org/using-go-modules)
+
## Configure Capstan (advanced)
Capstan uses optimized default values under the hood. But you are allowed to
override them with
your own values and this section describes how. Actually, there are three ways
to override them
diff --git a/Godeps/Godeps.json b/Godeps/Godeps.json
deleted file mode 100644
index e39f221..000
--- a/Godeps/Godeps.json
+++ /dev/null
@@ -1,38 +0,0 @@
-{
- "ImportPath": "github.com/cloudius-systems/capstan",
- "GoVersion": "go1.7",
- "Packages": [
- "./..."
- ],
- "Deps": [
- {
- "ImportPath": "github.com/cheggaaa/pb",
- "Rev": "a9a708f18ae43493d86f3fd5b0cd04df679b170b"
- },
- {
- "ImportPath": "github.com/olekukonko/ts",
- "Rev": "ecf753e7c962639ab5a1fb46f7da627d4c0a04b8"
- },
- {
- "ImportPath": "github.com/mitchellh/mapstructure",
- "Rev": "d2dd0262208475919e1a362f675cfc0e7c10e905"
- },
-
[osv-dev] [COMMIT osv master] libtools: make gcc compiler messages quiet
From: Waldemar Kozaczuk Committer: Nadav Har'El Branch: master libtools: make gcc compiler messages quiet Signed-off-by: Waldemar Kozaczuk Message-Id: <20191209044400.26532-1-jwkozac...@gmail.com> --- diff --git a/modules/libtools/Makefile b/modules/libtools/Makefile --- a/modules/libtools/Makefile +++ b/modules/libtools/Makefile @@ -40,6 +40,9 @@ lsroute.so: lsroute.o libtools.so ifconfig.so: ifconfig.o libtools.so $(call quiet, $(CXX) $(CXXFLAGS) -shared -o $@ $^, LINK ifconfig.so) +%.o: %.cc + $(call quiet, $(CXX) $(CXXFLAGS) -c -o $@ $<, CXX $*.cc) + clean: -rm -f libtools.so network_interface.o route_info.o ifconfig.o \ ifconfig.so lsroute.o lsroute.so -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/5708cb0599437c5d%40google.com.
[osv-dev] [COMMIT osv master] tests: replace tmpnam() with mkdtemp() in implementation of TempDir
From: Waldemar Kozaczuk
Committer: Nadav Har'El
Branch: master
tests: replace tmpnam() with mkdtemp() in implementation of TempDir
This should be the last patch addressing compiler warnings
in unit tests as reported by the issue #976.
Refs #976
Signed-off-by: Waldemar Kozaczuk
Message-Id: <20191209042729.21316-1-jwkozac...@gmail.com>
---
diff --git a/tests/tst-fs.hh b/tests/tst-fs.hh
--- a/tests/tst-fs.hh
+++ b/tests/tst-fs.hh
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -22,8 +23,13 @@ typedef boost::format fmt;
class TempDir : public fs::path
{
public:
- TempDir() : fs::path(tmpnam(NULL)) {
- fs::create_directories(*this);
+ TempDir() : fs::path() {
+ char path[64] = "/tmp/dirXX";
+ auto dir_path = mkdtemp(path);
+ if (!dir_path) {
+throw std::system_error(std::error_code(errno,
std::system_category()), "error creating temp directory");
+ }
+ *this += dir_path;
}
virtual ~TempDir() {
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/a14a620599426ebe%40google.com.
[osv-dev] [COMMIT osv master] misc-fs-stress: replace mktemp with mkstemp to fix compiler warnings
From: Waldemar Kozaczuk
Committer: Nadav Har'El
Branch: master
misc-fs-stress: replace mktemp with mkstemp to fix compiler warnings
Signed-off-by: Waldemar Kozaczuk
Message-Id: <20191209030913.1842-1-jwkozac...@gmail.com>
---
diff --git a/tests/misc-fs-stress.cc b/tests/misc-fs-stress.cc
--- a/tests/misc-fs-stress.cc
+++ b/tests/misc-fs-stress.cc
@@ -51,7 +51,14 @@ int main(int argc, char const *argv[])
if (argc == 2) {
fname = argv[1];
} else {
-mktemp(default_fname);
+// Use mkstemp to capture name of a temporary file
+auto fd = mkstemp(default_fname);
+if (fd <= 0) {
+perror("mkstemp");
+return -1;
+}
+close(fd);
+unlink(default_fname);
fname = reinterpret_cast(default_fname);
}
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/c5dd6205994266e7%40google.com.
[osv-dev] [COMMIT osv master] tst-zfs-mount: replace mktemp with mkstemp to fix compiler warnings
From: Waldemar Kozaczuk
Committer: Nadav Har'El
Branch: master
tst-zfs-mount: replace mktemp with mkstemp to fix compiler warnings
Signed-off-by: Waldemar Kozaczuk
Message-Id: <20191209012233.30437-1-jwkozac...@gmail.com>
---
diff --git a/tests/tst-zfs-mount.cc b/tests/tst-zfs-mount.cc
--- a/tests/tst-zfs-mount.cc
+++ b/tests/tst-zfs-mount.cc
@@ -50,8 +50,14 @@ int check_zfs_refcnt_behavior(void)
}
old_mcount = mp->m_count;
+// Use mkstemp to capture path of a temporary file used later
snprintf(file, 64, "%sfileXX", mount_path);
-mktemp(file);
+fd = mkstemp(file);
+if (fd <= 0) {
+return -1;
+}
+close(fd);
+unlink(file);
/* Create hard links, and remove them afterwards to exercise the
refcount code */
for (i = 0; i < 10; i++) {
@@ -69,8 +75,7 @@ int check_zfs_refcnt_behavior(void)
}
close(fd);
-snprintf(newfile, 64, "%snewfileXX", mount_path);
-mktemp(newfile);
+snprintf(newfile, 64, "%snewfile", file);
/* Create a link to file into newfile */
ret = link(file, newfile);
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/8ba94c0599425e7d%40google.com.
[osv-dev] [COMMIT osv master] tst-fs-link: replace mktemp with mkstemp to fix compiler warnings
From: Waldemar Kozaczuk
Committer: Nadav Har'El
Branch: master
tst-fs-link: replace mktemp with mkstemp to fix compiler warnings
Signed-off-by: Waldemar Kozaczuk
Message-Id: <20191209010643.27127-1-jwkozac...@gmail.com>
---
diff --git a/tests/tst-fs-link.cc b/tests/tst-fs-link.cc
--- a/tests/tst-fs-link.cc
+++ b/tests/tst-fs-link.cc
@@ -33,15 +33,13 @@ static int check_vnode_duplicity(void)
{
int err = 0;
char oldpath[64] = "/tmp/tst-fs-linkXX";
-char newpath[64] = "/tmp/tst-fs-linkXX";
struct dentry *olddp, *newdp;
-mktemp(oldpath);
-mktemp(newpath);
-
-auto fd = open(oldpath, O_CREAT | O_TRUNC | O_RDWR, 0666);
+auto fd = mkstemp(oldpath);
close(fd);
+char newpath[64];
+sprintf(newpath, "%s_new", oldpath);
if (link(oldpath, newpath) != 0) {
unlink(oldpath);
perror("link");
@@ -85,30 +83,34 @@ int main(int argc, char *argv[])
struct stat st[2];
char buf[4] = {0};
char buf2[4] = {0};
+int fd;
if (argc > 2) {
oldpath = argv[1];
newpath = argv[2];
-} else {
+fd = open(oldpath, O_CREAT | O_TRUNC | O_RDWR, 0666);
+report(fd > 0, "create a file");
+} else {
+//
+// Create a temporary file that's used in testing.
strcpy(oldp, "/tmp/tst-fs-linkXX");
-mktemp(oldp);
+fd = mkstemp(oldp);
+report(fd > 0, "create a file");
-strcpy(newp, "/tmp/tst-fs-linkXX");
-mktemp(newp);
+sprintf(newp, "%s_new", oldp);
oldpath = oldp;
newpath = newp;
}
-report(link(oldpath, newpath) < 0 && errno == ENOENT, "link returns
ENOENT if source path does not exists");
+char missing_file_path[64];
+sprintf(missing_file_path, "%s_missing", oldpath);
+report(link(missing_file_path, newpath) < 0 && errno == ENOENT, "link
returns ENOENT if source path does not exists");
-// Create a temporary file that's used in testing.
-auto fd = open(oldpath, O_CREAT | O_TRUNC | O_RDWR, 0666);
write(fd, "test", 4);
lseek(fd, 0, SEEK_SET);
read(fd, buf, 4);
-report(fd > 0, "create a file");
report(close(fd) == 0, "close the file");
// Create a hard link
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/c41e2805994256dd%40google.com.
[osv-dev] [COMMIT osv master] elf: reverse unloading of objects in needed list
From: Waldemar Kozaczuk
Committer: Nadav Har'El
Branch: master
elf: reverse unloading of objects in needed list
New versions of boost like 1.69.0 have more complicated
dependendencies and cause issues with some of the unit tests.
```
tst-rcu-list.so
libboost_unit_test_framework.so.1.69.0
libboost_timer.so.1.69.0
libboost_chrono.so.1.69.0
libboost_system.so.1.69.0
libgcc_s.so.1
libboost_system.so.1.69.0
libgcc_s.so.1
libboost_system.so.1.69.0
libgcc_s.so.1
libboost_filesystem.so.1.69.0
libboost_system.so.1.69.0
libgcc_s.so.1
libgcc_s.so.1
```
More specifically unit tests like tst-rcu-list and tst-rcu-hashtable
crash when executing FINI functions while unloading boost libraries
with a stack trace like this:
```
#0 0x403a3cf2 in processor::cli_hlt () at arch/x64/processor.hh:247
#1 arch::halt_no_interrupts () at arch/x64/arch.hh:48
#2 osv::halt () at arch/x64/power.cc:26
#3 0x40239dae in abort (fmt=fmt@entry=0x4064155f "Aborted\n") at
runtime.cc:132
#4 0x402028ab in abort () at runtime.cc:98
#5 0x4021981e in osv::generate_signal (siginfo=...,
ef=0x8123c068) at libc/signal.cc:124
#6 0x40463f7f in osv::handle_mmap_fault (addr=,
sig=, ef=) at libc/signal.cc:139
#7 0x4033da62 in mmu::vm_fault (addr=17592187015168,
addr@entry=17592187015648, ef=ef@entry=0x8123c068) at
core/mmu.cc:1337
#8 0x4039dc30 in page_fault (ef=0x8123c068) at
arch/x64/mmu.cc:42
#9
#10 0x100ed1e0 in std::__cxx11::basic_stringstd::char_traits, std::allocator >::~basic_string() ()
#11 0x4023a357 in __cxxabiv1::__cxa_finalize (dso=)
at runtime.cc:183
#12 0x1041f5f7 in __do_global_dtors_aux ()
#13 0x201008f0 in ?? ()
#14 0x40352f34 in elf::object::run_fini_funcs
(this=0xa15d0a00) at core/elf.cc:1105
```
The order in which FINI functions are executed currently follows
the order in which its objects are unloaded. It is not exactly clear why,
but there seems to be some dependency between child
objects in the DT_NEEDED list of paricular object. For example,
if in tst-rcu-list.so the libboost_unit_test_framework.so.1.69.0
is unloaded first and libboost_filesystem.so.1.69.0 unloaded later,
the std string destructor called by libboost_filesystem.so.1.69.0 FINI
function causes page fault shown above.
The commit #4d24b90aa38d90bdcf5ebecf7a5badeca3b657eb from 6 years ago
made process of unloading of objects symmetrical to the loading one
but not quite exactly which seems to be a culprit.
This patch refines the `elf::unload_needed()` method by changing the order
in which the dependent object references are released.
First we release references held by the `_used_by_resolve_plt_got` set
mostly because they are added after DT_NEEDED is processed and `_needed`
populated and
we want this to be symmetrical.
Secondly we release references from the `_needed` vector
by iterating and removing elements starting at the end. This
also makes the unloading behavior symmetrical to the order
in which the objects from DT_NEEDED were loaded in first place.
Concluding with this patch all the unit tests now
pass when linked the boost 1.69.0 from Fedora 30 host.
Signed-off-by: Waldemar Kozaczuk
Message-Id: <20191208185437.21976-1-jwkozac...@gmail.com>
---
diff --git a/core/elf.cc b/core/elf.cc
--- a/core/elf.cc
+++ b/core/elf.cc
@@ -1000,8 +1000,10 @@ void
object::load_needed(std::vector>& loaded_objects)
void object::unload_needed()
{
-_needed.clear();
_used_by_resolve_plt_got.clear();
+while (!_needed.empty()) {
+_needed.pop_back();
+}
}
ulong object::get_tls_size()
--
You received this message because you are subscribed to the Google Groups "OSv
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/osv-dev/e450400599422eec%40google.com.
