Symbol lookup errors while building C++ source code

2017-04-23 Thread Niels Bergsma 
Hi all,

I've been trying to build simple C++ apps for the past few weeks on OSV. I 
frequently receive "failed looking up symbol" errors and I can't figure out 
how to solve them properly. Simple programs like HelloWorld or variants 
which include simple shared library work correctly, but more sophisticated 
libraries like leveldb, restbed report errors at runtime. Even when I 
manual compile those libraries and link the object files with my own code. 

The error I receive look like:
OSv v0.24
eth0: 192.168.122.15
/tools/example.so: failed looking up symbol 
_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_ 
(std::__cxx11::basic_string, 
std::allocator >::basic_string(char const*, std::allocator 
const&))

[backtrace]
0x003447cd 
0x00346148 
0x00346325 
0x00391e64 
0x004c12df 
0x1584cd54 
0x00421153 
0x00423220 
0x0020ce43 
0x00423588 
0x0044d465 
0x003f10c7 
0x003931b5 

My capstan file looks like is:
base: cloudius/osv-base
cmdline: /tools/example.so
build: make
files:
  /tools/example.so: example.so


What am I doing wrong?

Thanks!

Niels 

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Avi Kivity 
It could be the new ABI, see 
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html.



Try compiling your code with _GLIBCXX_USE_CXX11_ABI=0.


On 04/23/2017 01:41 PM, Niels Bergsma wrote:

Hi all,

I've been trying to build simple C++ apps for the past few weeks on 
OSV. I frequently receive "failed looking up symbol" errors and I 
can't figure out how to solve them properly. Simple programs like 
HelloWorld or variants which include simple shared library work 
correctly, but more sophisticated libraries like leveldb, restbed 
report errors at runtime. Even when I manual compile those libraries 
and link the object files with my own code.


The error I receive look like:
|
OSv v0.24
eth0: 192.168.122.15
/tools/example.so: failed looking up symbol 
_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_ 
(std::__cxx11::basic_string, 
std::allocator >::basic_string(char const*, std::allocator 
const&))


[backtrace]
0x003447cd 
0x00346148 
0x00346325 
0x00391e64 
0x004c12df 
0x1584cd54 
0x00421153 char**)+371>

0x00423220 
0x0020ce43 
0x00423588 
0x0044d465 
0x003f10c7 
0x003931b5 
|

My capstan file looks like is:
|
base: cloudius/osv-base
cmdline: /tools/example.so
build: make
files:
  /tools/example.so: example.so
|


What am I doing wrong?

Thanks!

Niels
--
You received this message because you are subscribed to the Google 
Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to osv-dev+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Nadav Har'El 
On Sun, Apr 23, 2017 at 2:06 PM, Avi Kivity  wrote:

> It could be the new ABI, see https://gcc.gnu.org/
> onlinedocs/libstdc++/manual/using_dual_abi.html.
>
>
> Try compiling your code with _GLIBCXX_USE_CXX11_ABI=0.
>

Indeed. Check out also commit 6a3bff38a281e65ee715bab4fadef63e0918f7d3
where we encountered this problem (and its fix) previously.

This problem is an example of what happens when you compile the OSv kernel
itself with one compiler and libraries (in this case, the C++ standard
library) and then try to run on it software compiled on a very different
version of the compiler and/or library. The general issue is tracked in
https://github.com/cloudius-systems/osv/issues/821, "Combining pre-compiled
OSv kernel with pre-compiled executable".

In this case, the specific problem is that you're taking a very old
compiled OSv kernel (Capstan takes an antique OSv kernel which we haven't
recompiled in a very long time, as a result was also compiled with old
compilers and libraries) and try to run on it code compiled on newer
compilers. The newer compiler assumes new C++ ABI (in particular the
implementation of std::string) which the old C++ library embedded in OSv
did not include. If you use the macro which Avi suggested
(-D_GLIBCXX_USE_CXX11_ABI=0), it will cause the C++ header files to use the
old ABI and work on the old) library. Of course, this will not help if
you're trying to run on OSv a shared library or an entire application which
uses C++ and you cannot recompile.

A different approach would be to compile OSv yourself with the same
compiler that built your application.

Pekka, another question we should ask ourselves is what is the plan with
Capstan, and more importantly, the binary OSv releases. The OSv v0.24
version he used is two years old (!). If we're not going to produce any
more pre-compiled releases (are we?) should we at least remove the antique
ones and force people to compile OSv on their own before using Capstan? Or
should we make another release? Or what?

Nadav.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[COMMIT osv master] command line: add makefile-like syntax to conditionally set environ var

2017-04-23 Thread Commit Bot 

From: Justin Cinkelj 
Committer: Nadav Har'El 
Branch: master

command line: add makefile-like syntax to conditionally set environ var

Add "KEY?=value" syntax to set environ variable only if not set yet.
This allows runscript to contain default value in case KEY
is not set yet. The default value can be overriden by setting value
via REST API before runscript is started.

Fixes #870

Signed-off-by: Justin Cinkelj 
Message-Id: <20170421105050.21970-1-justin.cink...@xlab.si>

---
diff --git a/core/commands.cc b/core/commands.cc
--- a/core/commands.cc
+++ b/core/commands.cc
@@ -166,8 +166,22 @@ static void  
runscript_process_options(std::vector >& re


 if (vars.count("env")) {
 for (auto t : vars["env"].as>()) {
-debug("Setting in environment: %s\n", t);
-putenv(strdup(t.c_str()));
+size_t pos = t.find("?=");
+if (std::string::npos == pos) {
+// the basic "KEY=value" syntax
+debug("Setting in environment: %s\n", t);
+putenv(strdup(t.c_str()));
+}
+else {
+// "KEY?=value", makefile-like syntax, set variable  
only if not yet set

+auto key = t.substr(0, pos);
+auto value = t.substr(pos+2);
+if (nullptr == getenv(key.c_str())) {
+debug("Setting in environment: %s=%s\n", key,  
value);

+setenv(key.c_str(), value.c_str(), 1);
+}
+}
+
 }
 }

diff --git a/tests/tst-commands.cc b/tests/tst-commands.cc
--- a/tests/tst-commands.cc
+++ b/tests/tst-commands.cc
@@ -10,6 +10,7 @@

 #include 
 #include 
+#include 

 static int tests = 0, fails = 0;

@@ -760,6 +761,120 @@ static bool test_runscript_with_env_in_script()
 return true;
 }

+static bool test_runscript_with_conditional_env_in_script(bool  
set_env_vars_before_evaluation)

+{
+std::ofstream of1("/myscript", std::ios::out | std::ios::binary);
+of1 << "--env=ASDF?=ttrt --env=PORT?=$THEPORT /$THEPROG pp1a $PRM1b  
pp1c $PRM1d\n";

+of1.close();
+
+std::vector > result;
+std::vector cmd = { "/prog1" };
+std::map expected_vars;
+size_t expected_size[] = {6};
+bool ok;
+
+// those two are set during command parsing
+if (NULL != getenv("ASDF")) {
+return false;
+}
+if (NULL != getenv("PORT")) {
+return false;
+}
+// those are required during command parsing
+if (0 != setenv("THEPORT", "4321", 1)) {
+return false;
+}
+if (0 != setenv("THEPROG", "prog1", 1)) {
+return false;
+}
+if (0 != setenv("PRM1b", "pp1b", 1)) {
+return false;
+}
+if (0 != setenv("PRM1d", "pp1d", 1)) {
+return false;
+}
+// run test with conditional variables set or clear
+if (set_env_vars_before_evaluation) {
+expected_vars["ASDF"] = "asdf-old";
+expected_vars["PORT"] = "port-old";
+if (0 != setenv("ASDF", expected_vars["ASDF"].c_str(), 1)) {
+return false;
+}
+if (0 != setenv("PORT", expected_vars["PORT"].c_str(), 1)) {
+return false;
+}
+}
+else {
+expected_vars["ASDF"] = "ttrt";
+expected_vars["PORT"] = "4321";
+}
+
+result = osv::parse_command_line(
+std::string("runscript \"/myscript\";  "),
+ok);
+
+if (!ok) {
+return false;
+}
+
+if (result.size() != 1) {
+return false;
+}
+
+for (size_t i = 0; i < result.size(); i++) {
+if (result[i].size() != expected_size[i]) {
+return false;
+}
+if (result[i][0] != cmd[i]) {
+return false;
+}
+}
+
+if (result[0][1] != std::string("pp1a")) {
+return false;
+}
+if (result[0][2] != std::string("pp1b")) {
+return false;
+}
+if (result[0][3] != std::string("pp1c")) {
+return false;
+}
+if (result[0][4] != std::string("pp1d")) {
+return false;
+}
+
+// environ variable with ? in name should not be created
+if (nullptr != getenv("ASDF?")) {
+return false;
+}
+if (nullptr != getenv("PORT?")) {
+return false;
+}
+// std::string(NULL) is undefined behavior, hence check getenv returns  
a valid string.

+if (nullptr == getenv("ASDF")) {
+return false;
+}
+if (nullptr == getenv("PORT")) {
+return false;
+}
+
+if (expected_vars["ASDF"] != getenv("ASDF")) {
+return false;
+}
+if (expected_vars["PORT"] != getenv("PORT")) {
+return false;
+}
+
+unsetenv("ASDF");
+unsetenv("PORT");
+unsetenv("THEPORT");
+unsetenv("THEPROG");
+unsetenv("PRM1b");
+unsetenv("PRM1d ");
+
+return true;
+}
+
 int main(int argc, char *argv[])
 {
 report(test_parse_simplest(), "simplest

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Rick Payne (Offshore) 

> On 23 Apr 2017, at 05:14, Nadav Har'El  wrote:
> 
> Pekka, another question we should ask ourselves is what is the plan with 
> Capstan, and more importantly, the binary OSv releases. The OSv v0.24 version 
> he used is two years old (!). If we're not going to produce any more 
> pre-compiled releases (are we?) should we at least remove the antique ones 
> and force people to compile OSv on their own before using Capstan? Or should 
> we make another release? Or what?

On this front, I have a rebar_osv plugin for the erlang ‘rebar’ build tool that 
will take an erlang release and build an OSv image. Its simpler to port an 
erlang OTP application to OSv that way than using the apps/erlang setup. It 
will also include a recent OSv base image (as I use the same method of CPIO-ing 
in the files). Hope to release that soon - are you ok with me including an OSv 
base image?

Cheers,
Rick

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Nadav Har'El 
On Sun, Apr 23, 2017 at 4:23 PM, Rick Payne (Offshore)  wrote:

>
> > On 23 Apr 2017, at 05:14, Nadav Har'El  wrote:
> >
> > Pekka, another question we should ask ourselves is what is the plan with
> Capstan, and more importantly, the binary OSv releases. The OSv v0.24
> version he used is two years old (!). If we're not going to produce any
> more pre-compiled releases (are we?) should we at least remove the antique
> ones and force people to compile OSv on their own before using Capstan? Or
> should we make another release? Or what?
>
> On this front, I have a rebar_osv plugin for the erlang ‘rebar’ build tool
> that will take an erlang release and build an OSv image. Its simpler to
> port an erlang OTP application to OSv that way than using the apps/erlang
> setup. It will also include a recent OSv base image (as I use the same
> method of CPIO-ing in the files). Hope to release that soon - are you ok
> with me including an OSv base image?
>

Yes.

However note that I think that Capstan's notion of "base image" is
misdirected. The Capstan "base image" is an image with a particular size
(e.g., 10 GB) plus the OSv kernel and some collection of "default" files.
If you want a 100 MB or 100 GB image instead, you're out of luck.

What I think is a better approach is to take separatey the OSv kernel with
just cpiod.so and mkfs.so (i.e., take build/release/loader.img) and the
files, and compose the image with the size you want and uploading the files
you want to it.  This is more-or-less what the Mikelangelo project's fork
of Capstan is doing. Maybe that's what you are also planning to do?

Nadav.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Rick Payne (Offshore) 

> What I think is a better approach is to take separatey the OSv kernel with 
> just cpiod.so and mkfs.so (i.e., take build/release/loader.img) and the 
> files, and compose the image with the size you want and uploading the files 
> you want to it.  This is more-or-less what the Mikelangelo project's fork of 
> Capstan is doing. Maybe that's what you are also planning to do?

That is indeed the intention. Right now, I’m building an Erlang/OTP with the 
patches from your modules/ directory and reference that from the rebar build 
(just as you would if you were cross-compiling an erlang/otp application). The 
‘base’ image I’m using is created image with libncurses, ssl etc. Once a few 
Erlang/OTP build issues are sorted, I may move to using loader.img, but it is 
convenient to have an image with cloud-init and the httpserver installed too...

Cheers,
Rick

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread Niels Bergsma 
Yes that did the trick. Awesome! I switched to the build scripts in the
meanwhile, to make sure I have the latest version.

2017-04-23 17:11 GMT+02:00 Rick Payne (Offshore) :

>
> > What I think is a better approach is to take separatey the OSv kernel
> with just cpiod.so and mkfs.so (i.e., take build/release/loader.img) and
> the files, and compose the image with the size you want and uploading the
> files you want to it.  This is more-or-less what the Mikelangelo project's
> fork of Capstan is doing. Maybe that's what you are also planning to do?
>
> That is indeed the intention. Right now, I’m building an Erlang/OTP with
> the patches from your modules/ directory and reference that from the rebar
> build (just as you would if you were cross-compiling an erlang/otp
> application). The ‘base’ image I’m using is created image with libncurses,
> ssl etc. Once a few Erlang/OTP build issues are sorted, I may move to using
> loader.img, but it is convenient to have an image with cloud-init and the
> httpserver installed too...
>
> Cheers,
> Rick
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Symbol lookup errors while building C++ source code

2017-04-23 Thread miha . plesko . xlab 
Hi,

as Nadav said it might be best to use Capstan from MIKELANGELO repository 
. The Capstan that is there 
is under active development and contains much more recent OSv base image 
(we've last compiled it 2 weeks ago or so).
The Capstan there works so that it always uses same basic OSv image (it 
downloads it from the repository) and uploads all the files from your 
current directory to it. The way you would use it in your case is:

   1. compile your C++ application and copy the result (one or more `.so` 
   files and possible other files) into a new folder "mypackage"
   2. add the two configuration files meta/package.yaml and meta/run.yaml 
   into "mypackage" folder (consult README for details, should be fairly 
   simple)
   3. use command `capstan package compose ` - a new 
   unikernel will be composed with your application files
   4. use command `capstan run ` - the unikernel will be run

For details and a step-by-step guide please see README at the Capstan 
GithubRepository.





Dne nedelja, 23. april 2017 12.41.51 UTC+2 je oseba Niels Bergsma napisala:
>
> Hi all,
>
> I've been trying to build simple C++ apps for the past few weeks on OSV. I 
> frequently receive "failed looking up symbol" errors and I can't figure out 
> how to solve them properly. Simple programs like HelloWorld or variants 
> which include simple shared library work correctly, but more sophisticated 
> libraries like leveldb, restbed report errors at runtime. Even when I 
> manual compile those libraries and link the object files with my own code. 
>
> The error I receive look like:
> OSv v0.24
> eth0: 192.168.122.15
> /tools/example.so: failed looking up symbol 
> _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_ 
> (std::__cxx11::basic_string, 
> std::allocator >::basic_string(char const*, std::allocator 
> const&))
>
> [backtrace]
> 0x003447cd 
> 0x00346148 
> 0x00346325 
> 0x00391e64 
> 0x004c12df 
> 0x1584cd54 
> 0x00421153  char**)+371>
> 0x00423220 
> 0x0020ce43 
> 0x00423588 
> 0x0044d465 
> 0x003f10c7 
> 0x003931b5 
>
> My capstan file looks like is:
> base: cloudius/osv-base
> cmdline: /tools/example.so
> build: make
> files:
>   /tools/example.so: example.so
>
>
> What am I doing wrong?
>
> Thanks!
>
> Niels 
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.