Hey Matthew,
Great investigating! Glad you were able to at least find the core of
the issue, which does look like at least on Fedora Core 20 x86_64 we
are doing something funky with configure/make. I'm not sure why
we're choosing lib64 there and lib on Ubuntu 14.04 x86_64, might have
to do with how the python is configured. I think that directory may
be coming from a helper library in python that determines the correct
location for build output and has to do with some Python finalizer
scripts during the build process. I won't be totally surprised if
this might just be broken on Fedora Core x86_64 regardless of cmake
or configure-based builds.
I'll do a configure/make build on Ubuntu 14.04 x86_64 later today to
see if I'm getting the same results on configure/make - there has
been some changes in python handling more recently, generally to
support Windows builds, but we may have unintentionally introduced
that via those changes. At least that way we can see if its
consistent with configure/make or if it might just be specific to
Fedora Core.
Thanks for identifying that!
-Todd
On Mon, Jul 21, 2014 at 6:51 AM, Matthew Gardiner <[email protected]
<mailto:[email protected]>> wrote:
Todd,
I've since discovered that the cmake/ninja build system also
results in a lib and lib64 directory, with the liblldb.so and
other .sos in the lib folder but the python2.7/site-packages/lldb
under lib64. (In fact with cmake build I see no python2.7 folder
at all under lib, it's all under lib64).
So Todd, focusing back on autoconf/make builds, after doing the
./configure && make in the sibling build directory, do you see for
your linux 64-bit builds the python2.7/site-packages/lldb under
the lib64 sub-folder and the lldb/llvm .sos under the lib64
sub-folder?
How do you get this to work:
tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import lldb
>>> dir(lldb)
Do you run another script or make target after doing the basic
llvm/lldb make? That is to move python2.7/site-packages/lldb
underneath lib. I think that's what I'd have to do on my system,
since after the basic make my lib and lib64 directories have the
following contents:
~/src/p4play/build/Debug+Asserts/lib/python2.7/site-packages
$ ls -al
total 32
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 14:40 .
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:40 ..
-rwxr-xr-x 1 mg11 mg11 21795 Jul 21 14:40 readline.so
$ cd ../../../lib64/python2.7/site-packages/
~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages
$ ls -al
total 12
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 .
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 ..
drwxrwxr-x 5 mg11 mg11 4096 Jul 21 14:42 lldb
Any ideas?
Matt
Matthew Gardiner wrote:
Hi Todd,
Thanks for taking the time to come up with these tips. First
off my distro is Fedora Core 20 64-bit. So to further my
investigation I blew away all the sources, and did svn co
llvm, (lldb clang into llvm/tools), made the sibling build
dir, then did:
~/src/p4play/build
$ ../llvm/configure --enable-targets=x86_64 --enable-shared
--enable-cxx11 --prefix=`pwd`/../install && make -j 8
After the successful build I do ldd. Basically lld finds all
dependencies for liblldb.so...
~/src/p4play/build/Debug+Asserts/lib
$ ldd liblldb.so
linux-vdso.so.1 => (0x00007fffa61fe000)
libLLVM-3.5svn.so <http://libLLVM-3.5svn.so> =>
/home/mg11/src/p4play/build/Debug+Asserts/lib/./libLLVM-3.5svn.so
<http://libLLVM-3.5svn.so> (0x00007f2e53405000)
libpthread.so.0 => /lib64/libpthread.so.0
(0x00007f2e531cc000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e52fc8000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f2e52dc5000)
libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0
(0x00007f2e529fe000)
librt.so.1 => /lib64/librt.so.1 (0x00007f2e527f6000)
libedit.so.0 => /lib64/libedit.so.0 (0x00007f2e525ba000)
libncurses.so.5 => /lib64/libncurses.so.5
(0x00007f2e52392000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f2e52168000)
libpanel.so.5 => /lib64/libpanel.so.5 (0x00007f2e51f64000)
libz.so.1 => /lib64/libz.so.1 (0x00007f2e51d4d000)
libm.so.6 => /lib64/libm.so.6 (0x00007f2e51a46000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e5173e000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e51527000)
libc.so.6 => /lib64/libc.so.6 (0x00007f2e51168000)
/lib64/ld-linux-x86-64.so.2 (0x0000003675a00000)
However, regarding python loading the lldb bindings I see my
build/Debug+Asserts has a lib and a lib64 folder.
~/src/p4play/build/Debug+Asserts
$ ls
bin lib lib64
If I try to import lldb when I have "lib" as my PYTHONPATH i.e.
~/src/main/build/Debug+Asserts/lib
$ PYTHONPATH=`pwd`/python2.7/site-packages/ python
>>> import lldb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named lldb
>>>
This is not surprising, since
~/src/main/build/Debug+Asserts/lib/python2.7/site-packages/
only contains readline.so.
So I cd to lib64. This looks more promising since there is a
lldb folder below site-packages. So now I do "import lldb"
with "lib64" instead:
~/src/main/build/Debug+Asserts/lib64
$ PYTHONPATH=`pwd`/lib64/python2.7/site-packages/ python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import lldb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
line 52, in <module>
_lldb = swig_import_helper()
File
"/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
line 44, in swig_import_helper
import _lldb
ImportError: No module named _lldb
>>>
So cd to
~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb,
and list:
~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb
$ ls -l
total 1196
-rw-rw-r-- 1 mg11 mg11 3905 Jul 21 08:25
embedded_interpreter.py
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 08:25 formatters
-rw-rw-r-- 1 mg11 mg11 495675 Jul 21 08:25 __init__.py
-rw-rw-r-- 1 mg11 mg11 708200 Jul 21 09:09 __init__.pyc
lrwxrwxrwx 1 mg11 mg11 19 Jul 21 08:25 _lldb.so ->
../../../liblldb.so !BROKEN SYMLINK!
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 runtime
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 utils
_lldb.so is broken above as the link above assumes liblldb.so
is just 3 dirs back. However, it isn't - because it's not in
lib64, *it's in lib*. So it looks like things are getting a
bit mashed up due to 32-bit/64-bit directory issues. If I copy
the .so files from lib to lib64, then it fixes my broken
sym-link issue. And I progress a little further.
~/src/p4play/build/Debug+Asserts/lib64
$ PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import lldb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
line 52, in <module>
_lldb = swig_import_helper()
File
"/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py",
line 48, in swig_import_helper
_mod = imp.load_module('_lldb', fp, pathname, description)
ImportError: libLLVM-3.5svn.so <http://libLLVM-3.5svn.so>:
cannot open shared object file: No such file or directory
>>>
So now I push the location of these .sos into my environment
with LD_LIBRARY_PATH. Finally lldb imports!
~/src/p4play/build/Debug+Asserts/lib64
$ LD_LIBRARY_PATH=. PYTHONPATH=`pwd`/python2.7/site-packages
python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import lldb
>>> dir(lldb)
['INT32_MAX', 'LLDB_ARCH_DEFAULT',
... lots more ...
>>>
So, I think the crux of the problem I'm seeing here is that on
a 64-bit linux build (using autoconf/make) we get the binaries
distributed around the lib and lib64 sub-folders without
adequate coordination with the test tools. I'll retry now with
cmake/ninja and see if my experience differs.
Matt
Todd Fiala wrote:
Hey Matt,
(Adding back the list since these are general
troubleshooting tips for getting tests running).
Sorry you’re having so much trouble with this!
Ok so a few things to troubleshoot:
Regardless of the build system used, go to the build
output’s lib dir. Do this command:
ldd liblldb.so
That will spit out all the shared libraries that your
liblldb.so is trying to link against. The python message
your seeing will happen if python can’t find the lldb
module (as the message suggests and you were tracing
down), /or/ if it can find it but fails to load it (which
is often the case - something cannot be found when the
liblldb.so tries to load as a consequence of the lldb
python module trying to load).
You should see something that looks roughly like this:
|tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
ldd liblldb.so
linux-vdso.so.1 => (0x00007fffe9dfe000)
libstdc++.so.6 =>
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f6357e71000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
(0x00007f6357b6b000)
libedit.so.2 =>
/usr/lib/x86_64-linux-gnu/libedit.so.2 (0x00007f635793a000)
libpanel.so.5 =>
/usr/lib/x86_64-linux-gnu/libpanel.so.5 (0x00007f6357736000)
libncurses.so.5 =>
/lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f6357513000)
libtinfo.so.5 =>
/lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f63572e9000)
libpython2.7.so.1.0 =>
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
(0x00007f6356d82000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
(0x00007f6356b7a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
(0x00007f6356975000)
libpthread.so.0 =>
/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6356757000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
(0x00007f635653e000)
libgcc_s.so.1 =>
/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6356327000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
(0x00007f6355f61000)
/lib64/ld-linux-x86-64.so.2 (0x00007f635d584000)
libutil.so.1 =>
/lib/x86_64-linux-gnu/libutil.so.1 (0x00007f6355d5d000)
tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$site-packages
|
The left side shows the short name that the .so is
referencing. The right side shows what the loader actually
mapped it to with the current environment. If there is a
problem loading liblldb.so, this will show as (I think)
question marks on the right side indicating that the
shared library linkage was not found. Let me know if you
see something like that.
If you get past this part and you don’t see any issues,
the next thing to try is:
PYTHONPATH=/path/to/your/lldb/lib/python2.7/site-packages
python
Then do an ‘import lldb’, then a ‘dir (lldb)’, like so:
|tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$
PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import lldb
>>> dir(lldb)
['INT32_MAX', 'LLDB_ARCH_DEFAULT',
'LLDB_ARCH_DEFAULT_32BIT', 'LLDB_ARCH_DEFAULT_64BIT',
'LLDB_DEFAULT_BREAK_SIZE', 'LLDB_DEFAULT_SHELL',
'LLDB_GENERIC_ERROR', 'LLDB_INVALID_ADDRESS',
'LLDB_INVALID_BREAK_ID', 'LLDB_INVALID_CPUTYPE',
'LLDB_INVALID_FRAME_ID', 'LLDB_INVALID_IMAGE_TOKEN',
'LLDB_INVALID_INDEX32', 'LLDB_INVALID_IVAR_OFFSET',
'LLDB_INVALID_LINE_NUMBER', 'LLDB_INVALID_MODULE_VERSION',
'LLDB_INVALID_OFFSET', 'LLDB_INVALID_PROCESS_ID',
'LLDB_INVALID_QUEUE_ID', 'LLDB_INVALID_REGNUM',
'LLDB_INVALID_SIGNAL_NUMBER', 'LLDB_INVALID_THREAD_ID',
'LLDB_INVALID_UID', 'LLDB_INVALID_WATCH_ID',
'LLDB_MAX_NUM_OPTION_SETS', 'LLDB_OPT_SET_1',
... (lots of stuff removed)
|
That will tell you if you can load the lldb module when
you specify exactly where it should be. If you can do
this, the tests basically should not have trouble loading
lldb. (Obviously somewhere we’ll find an issue here). If
this does work, then we need to trace why the python test
runner calls are mixing up the python path. In general you
don’t need to specify any options to the ninja/make
commands to get the tests to run from the build dir.
Can you refresh my memory on what Linux distro you’re
using and whether it’s 64 or 32 bit? (I don’t get over to
the 32-bit distros much these days).
I hope that gets us closer to finding out what’s up! Keep
me posted.
-Todd
On Fri, Jul 18, 2014 at 4:56 AM, Matthew Gardiner
<[email protected] <mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>> wrote:
Hi Todd,
Well I managed to get cmake and ninja working! I
needed (it would
appear) to download/build/install the latest versions
of both from
source. Then I do:
build $ cmake ../llvm/ -G Ninja
-DLLVM_TARGETS_TO_BUILD="X86"
-DBUILD_SHARED_LIBS=ON -DLLVM_ENABLE_CXX1Y=ON
...
build $ ninja
...
However "ninja check-lldb" still fails to find
lldb.py. It is does
get built, but the path which dotest.py uses to find
it is wrong
on my system. Anyway, I'm going to continue trying to
analyse it
myself for now. I'll probably post a new thread to
lldb-dev if I
continue to struggle.
Anyway, have a good weekend,
Matt
Member of the CSR plc group of companies. CSR plc
registered in
England and Wales, registered number 4187346,
registered office
Churchill House, Cambridge Business Park, Cowley Road,
Cambridge,
CB4 0WZ, United Kingdom
More information can be found at www.csr.com
<http://www.csr.com> <http://www.csr.com>.
Keep up to date with CSR on our technical blog,
www.csr.com/blog <http://www.csr.com/blog>
<http://www.csr.com/blog>, CSR people blog,
www.csr.com/people <http://www.csr.com/people>
<http://www.csr.com/people>, YouTube,
www.youtube.com/user/CSRplc
<http://www.youtube.com/user/CSRplc>
<http://www.youtube.com/user/CSRplc>, Facebook,
www.facebook.com/pages/CSR/191038434253534
<http://www.facebook.com/pages/CSR/191038434253534>
<http://www.facebook.com/pages/CSR/191038434253534>,
or follow us
on Twitter at www.twitter.com/CSR_plc
<http://www.twitter.com/CSR_plc>
<http://www.twitter.com/CSR_plc>.
New for 2014, you can now access the wide range of
products
powered by aptX at www.aptx.com <http://www.aptx.com>
<http://www.aptx.com>.
-- Todd Fiala | Software Engineer |
[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>> | 650-943-3180
<tel:650-943-3180>
To report this email as spam click here
<https://www.mailcontrol.com/sr/ik8jrXaVfBTGX2PQPOmvUj%21GOBh06pKK3+K9eZ%212Dx6F%21Jeeftdx9RBpik88jjuoRisi26jPZSzJaydNu2tBwg==>.
_______________________________________________
lldb-dev mailing list
[email protected] <mailto:[email protected]>
http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
--
Todd Fiala | Software Engineer | [email protected]
<mailto:[email protected]> | 650-943-3180
