Re: [Ltrace-devel] No output on Ubuntu 16.10

[Dima Kogan]

Hi. I just did this a different way: made a new yakkety chroot with
debootstrap:

sudo debootstrap --variant=minbase yakkety yakkety_amd64 
http://mirror.math.ucdavis.edu/ubuntu/

This creates a fresh system, and I can then chroot into it. ltrace works
OK here. The MD5 hash of the stock /bin/ls doesn't match yours. Where
did you get it? What does 'dpkg -S /bin/ls' and 'dpkg -l coreutils' say?

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] No output on Ubuntu 16.10

[Dima Kogan]

Stefano Cristalli  writes:

> I post to this list with the hope to solve the problem I posted
> yesterday on Stack Overflow:
> http://stackoverflow.com/questions/43213505/no-output-when-running-ltrace
>
> ltrace (installed either from package or from source) shows no output
> on both compiled binaries and system binaries (e.g. /bin/ls), while it
> works with some binaries compiled elsewhere. Since another user
> commented that he's able to reproduce the issue on his system, I
> thought that the problem might be of interest for this mailling list.

I just tried it on a fresh Ubuntu 16.10 install, and it works OK. What
do you think you're doing differently?

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] New ltrace release

[Dima Kogan]

Petr Machata  writes:

> The real reason seems to be lack of resources.  I didn't have time to
> touch ltrace for almost two years now.  If somebody else is willing to
> step in and finalize a release, and the only thing that they are
> missing is Alioth permissions, I'll be happy to grant those.  As
> things stand, this project is basically dormant.

I can't devote tons of time to this, but I'm happy to do a release at
least. I think that will be better than the current state. Alioth
username: dkogan

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] New ltrace release

[Dima Kogan]

Thierry fa...@linux.vnet.ibm.com <thie...@linux.vnet.ibm.com> writes:

> We all agree that new features like dwarf support should be officially
> integrated in a release.
> However today the few tests I have made show that dward test is
> providing not coherent results on x86_64 so I believe that time is
> missing to have a more complete testing release ...
> At least on my side I need a few days to fix ppc64 problems with dwarf
> test ppc64le being in good shape
> I would be glad to hear from ARM side...

Hi. Sorry it took so long to reply. Are the DWARF tests the main blocker
here? There're more features the new code has: backtraces for instance.

I just looked at the dwarf tests, and I see a number of failures that
have appeared because my libc was updated between when the tests were
written and now.

One issue was a crash, that is fixed by the attached patch. A number of
other libc changes cause test failures, but I haven't fixed those yet. I
will probably simply disable those test chunks.

Are you having trouble on your ppc64 box still? Is it worth it for me to
try to replicate that in emulation?

>From 192e0a678bd628ef2faef638c50e70129128a6fb Mon Sep 17 00:00:00 2001
From: Dima Kogan <d...@secretsauce.net>
Date: Fri, 8 Apr 2016 16:12:25 -0700
Subject: [PATCH] void struct members are now ignored

Previously the dwarf parser would crash when encountering such a thing
---
 dwarf_prototypes.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/dwarf_prototypes.c b/dwarf_prototypes.c
index bfac177..0d53b8b 100644
--- a/dwarf_prototypes.c
+++ b/dwarf_prototypes.c
@@ -616,6 +616,11 @@ static struct arg_type_info *get_structure(Dwarf_Die *parent,
 			complain(, "Couldn't parse type from DWARF data");
 			CLEANUP_AND_RETURN_ERROR(NULL);
 		}
+		if (member_type->type == ARGTYPE_VOID) {
+			complain(, "Struct member has void type. "
+ "Giving up on this structure");
+			CLEANUP_AND_RETURN_ERROR(NULL);
+		}
 		if (type_struct_add(result, member_type,
 newly_allocated_member_type) != 0) {
 			complain(, "Couldn't add type to struct");
-- 
2.1.4

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF test suite

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 I reworked the branch into a couple pieces that I feel belong together
 and fixed those minor issues.

Hi. I see one small discrepancy.

At the end of dwarf.exp I was looking for

 ft...@libc.so.6(0x[0-9a-z]*) * = -1

Note the ' * ' with 1 space on either side of *. In your tree you
changed it to '  * '. This is in two places at the end of that file.

Other than that, it looks fine. Thanks

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF test suite

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 If I removed the gnu++11 part from the ltrace.exp patch, and put
 that into dwarf.exp somehow, then we're good?

 Yes.

 In the new-style tests (e.g. parameters2.exp), we use file ending to
 determine what's inside the file.  Would it make sense to have a cc
 ending for C++98, c11 for C++11, etc.?

Hi. I just uploaded a patch that does the filename-language-choice as
you suggested:

.c11  is C 11
.cc11 is C++ 11
.cc, .cpp is C++ (whatever gcc has as the default)

This choice is still in ltrace.exp. The DWARF-specific piece simply
renames its .cc - .cc11

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

[Ltrace-devel] DWARF test suite

[Dima Kogan]

Hi.

Thank you for merging the aliased symbol branch, Petr. I added a simple
test for the DWARF code into the test suite, and it appears in the
master branch of

 g...@github.com:dkogan/ltrace.git

There are several other commits in that branch that do various useful
things. They didn't seem significant enough to warrant their own
branches and mailing list threads, so I'm bundling them with this email.

Let me know if something is amiss. This is my first time writing (and
reading, really) TCL, so maybe I messed something up, but it does appear
to work.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

[Ltrace-devel] Config files

[Dima Kogan]

Hi all.

In putting together the test suite, I hit a few issues with conf files.
Those are all described together in this email.

When running the test suite, I would think we want to be testing the
build of ltrace that appears in the checked-out source tree, and that it
should be independent of the user's local configuration or the overall
system. Thus I would expect

- ~/.ltrace to not be read
- /etc/ltrace to not be read either
- etc/ in the source tree to be read

currently I'm observing that

- ~/.ltrace is read
- /etc/ltrace is not read
- etc/ in the source tree is not read

This sounds wrong. Complaints against patching it in this way?



The DWARF parser interprets every structure as deeply as possible. This
is often undesirable. For instance when tracing something like ftell(),
ltrace wants to print all components of its FILE* argument. On my libc,
this eventually contains a void value, and ltrace does an assert(0)
trying to print it. We want to ship something like

 typedef FILE = addr;

in our config files. We DO have this in libc.so.conf, but this has 2
problems:

1. this file isn't read by the test suite, as mentioned above

2. On Debian, the DWARF data has ftell(_IO_FILE*), not ftell(FILE*). It
also has typedef FILE = _IO_FILE. Thus the typedef in the conf file has
no effect. We can add to the conf file

 typedef _IO_FILE = FILE

but this is an implementation detail of glibc, so this is unideal. We
can also do something complicated like realizing that FILE is defined in
the conf file, and then treat the typedef in the DWARF backwards.
Suggestions?

Thanks

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF prototypes: handling symbol aliases

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 We could actually store vectors in the hash table directly.

 The reason I did it this way was to reduce the memory inefficiency of
 the hash table. [...] Still want this change?

 That's a good point.  I'm not decided one way or another, so let's keep
 it the way you wrote it.

ok

 In any case, the patch seems to work, but I'd be more comfortable if the
 test suite told me it was good. I see the same test/suite failures
 before and after, some non-deterministic. Do you know which tests
 specifically are good to look at for a change like this?

 There are two tests in filters.exp that test specifically the -l stuff,
 i.e. this sort of export list handling thing.

OK. That test passes. I really should write a test for the DWARF code.
I'll do that when I finish tying the loose ends.


 Non-determinism in ltrace has always been a problem.  It used to be much
 worse, but there might still be latent bugs, or bugs that the kernels
 other than those that I test on expose.  What's you configuration, and
 what failures are you seeing?

I didn't take good notes, so this is incomplete, sorry. I was seeing the
trace-irelative.exp test get into an infinite loop. ltrace was running
until I killed it. The ltrace command was

 ./ltrace -L -x xyz /tmp/pie-yjtNlTxFte

Running that same command with that same executable later worked just
fine. There was some log file that was being generated that had 200MB of
the same error repeated on every line. Something about an illegal or
duplicated breakpoint; don't exactly remember, sorry. Re-running the
test suite didn't show this problem.

By the way, how much of the suite is supposed to pass normally? I always
see some failures and a segfault. The configuration is Debian/sid on
amd64; nothing noteworthy.


 OK, so I tried to merge and do a test build, and am now getting this:

 errors

I just pushed a patch. You were using an older gcc than me, and it's
more picky. Before this patch I successfully tested with gcc 4.9. After,
gcc 4.6 works too.


I also just discovered that this new symbol aliasing code works with
'ltrace -l', but not any of the other filtering options. I'll take a
look when i have time; hopefully by next week.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF prototypes: handling symbol aliases

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Sorry this took so long again.  I'm a fresh father and things have been
 hectic the last two weeks or so.

That's ok, course. Thank you for putting in the work you have been. I
was busy, so likewise sorry for the delay.

I addressed your points. The new tree is at

https://github.com/dkogan/ltrace/tree/libsym_aliases_inexportlist

Each point you raised is in a separate patch for easier review. Probably
would be good to squash them before merging. The benchmark tree
(described below) is here:

https://github.com/dkogan/ltrace/tree/libsym_aliases_inexportlist_benchmarking


  https://github.com/dkogan/ltrace/tree/libsym_aliases_inexportlist

 99fe555 (protolib_lookup() now uses a dict* instead of a function that
 returns it) changes meaning of the code and breaks a lot of test cases.
 The complexity is in fact necessary.

Ah yes. I rebased and killed that patch.


 Regarding 6db578849 (We now use known prototypes...):

 Names that start with underscore (_dtor_string, _clone_vect, _dtor_vect)
 are reserved, we shouldn't use them in ltrace itself.

Done


 Don't use logical operations in integer context (i.e. where int
 carries an actual value, not a boolean).

Done

 This:

 + struct vect **paliases = DICT_FIND_REF(names-addrs,
 +addr, struct vect*);

 ... technically maps an address to a pointer to a vector of names.  We
 could actually store vectors in the hash table directly.  If you would
 be willing to make this change, that would be great, but I'm not going
 to push it ;)

The reason I did it this way was to reduce the memory inefficiency of
the hash table. As implemented, sizeof(hash table cell)=sizeof(struct
vect*), which is much smaller than sizeof(struct vect). Your suggestion
has much better malloc() overhead and fragmentation, so maybe it's still
worthwhile. Still want this change?


 This:

 + result = vect_pushback(aliases, namedup);
 + if(result != 0)
 + return result;

 ... should free(namedup).

Done


 Iterators in ltrace are generally restartable

Done


 Other than that, there's a number of instances of if not being followed
 by a space.  Same holds for while and for, though I don't recall having
 seen any offenders.

Done


 2. The name is still exported_names.
 I don't really mind the name.

OK. Leaving it


 3. The data structure currently is not ideally efficient.

 Why don't we turn the logic around?  Go through the list of symbols, and
 for each of them, consult the vector of aliases that we found out
 earlier.

Ah yes. You already know this, but for my own clarity
activate_latent_in() looks like

for(export names in lib1) // hash table iteration
{
  for(symbol names in lib2) // list iteration
  {
if(names equal  libsym-latent)
{
  proc_activate_latent_symbol(proc, libsym)
}
  }
}

I turned it into

for(symbol names in lib2) // list iteration
{
  if(name in lib1 export names  libsym-latent) // hash table lookup
  {
proc_activate_latent_symbol(proc, libsym)
  }
}

It LOOKS much more efficient, but benchmarks don't show a huge
difference, so as you predicted it doesn't matter a lot. Benchmarks
before the relevant patch (only the slowest libraries shown):

library: libpthread.so.0 _activate_latent_in took 21023 ns
library: tstlib.so _activate_latent_in took 22419 ns
library: tst _activate_latent_in took 35270 ns

After the patch:

library: libpthread.so.0 _activate_latent_in took 13759 ns
library: tstlib.so _activate_latent_in took 20254 ns
library: tst _activate_latent_in took 31708 ns

In any case, the patch seems to work, but I'd be more comfortable if the
test suite told me it was good. I see the same test/suite failures
before and after, some non-deterministic. Do you know which tests
specifically are good to look at for a change like this?



 4. I discovered that in the libc on my machine (Debian/sid amd64) some
 symbols appear at multiple addresses:

 ltrace doesn't really handle versions at all, except it knows to
 ignore them (see e.g. populate_this_symtab in ltrace-elf.c). ... But
 right now your approach makes sense.

Leaving it alone for now.

Thanks again for looking.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Addition to the manpage

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 Hi. I added a section to the manpage giving an example of the effects of
 -l, -x and -e. It's here:

  https://github.com/dkogan/ltrace/tree/manpage

 This looks good.  I wonder about the effect of grep in there--that only
 filters the exit messages, doesn't it?  I think it would be better to
 have them included and not have the grep there, so that it doesn't look
 as if it's hiding something important.

The grep is just there to highlight what is being described. Without it
it looks like this:

=== without -Bsymbolic ===
$ ltrace -x 'func*' -L ./tst
func_f_main() = void
func_f_...@tstlib.so( unfinished ...
func_g_...@tstlib.so()= void
... func_f_lib resumed )= void
+++ exited (status 163) +++

$ ltrace -e 'func*' ./tst
tst-func_f_lib( unfinished ...
tstlib.so-func_g_lib()   = void
... func_f_lib resumed )= void
+++ exited (status 163) +++

$ ltrace -l tstlib.so ./tst
tst-func_f_lib( unfinished ...
tstlib.so-func_g_lib()   = void
... func_f_lib resumed )= void
+++ exited (status 163) +++


=== with -Bsymbolic ===
$ ltrace -x 'func*' -L ./tst
func_f_main() = void
func_f_...@tstlib.so( unfinished ...
func_g_...@tstlib.so()= void
... func_f_lib resumed )= void
+++ exited (status 163) +++

$ ltrace -e 'func*' ./tst
tst-func_f_lib() = void
+++ exited (status 163) +++

$ ltrace -l tstlib.so ./tst
tst-func_f_lib() = void
+++ exited (status 163) +++


Do you think this is better? I can go either way.



 I would also put the whole example in a section of its own, presumably
 named EXAMPLE, or maybe SYMBOL FILTERING EXAMPLE.  That should only
 come after the FILTERING EXPRESSIONS, as conceptually it builds on
 what's written there.  It would however be good to refer to this new
 section from -e, -x and -l.

Sure? I think the way it is now is better. Both putting the examples
first (all perl docs do that, and they tend to be very good) and putting
them into the same section (but different subsections). If you don't
have strong feelings here, I'd rather leave it.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF prototypes: handling symbol aliases

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 2. Find the aliases in 'exported_names'. This requires a fancier data
 structure for 'exported_names'.

 I have implemented #2, and it works. The code was a bit of a pain to
 write, and it needs cleanup if we're to talk about merging it.
 Implementing #1 would be a pain too, but a very different pain. Petr, do
 you have any preference here?

 I'll check out your patches hopefully later this week.  Your №2 seems
 reasonable, but I'll need to check out the code and think about it for a
 bit.

Hi. This code wasn't pushed anywhere, but if you want to look at it,
it's here:

 https://github.com/dkogan/ltrace/tree/libsym_aliases_inexportlist

Note that this code is extremely rough, and you don't need to even
bother reviewing it. I'm only posting it here so you could see the main
logic, and see that this does indeed work.

Notes:

1. This patch extends the exported_names (in struct library) data
structure to contain both an address and a name for each symbol, so that
it can be used to find aliases. The intent was to eventually make this
structure efficient, but currently it's a just dumb linear list search
for all access operations.

2. This data structure is populated in populate_this_symtab() as before.

3. Then, in library_get_prototype() we go through every prototype we have
(from DWARF or from the config files), we find aliased symbols, and we
add a prototype for the aliases. This is done in a sloppy way, so ltrace
crashes on exit(). Yeah.



The implementation of #1 and #2 above could use the symbols member of
struct library, as I said before. #3 is a bit separate. It'd be nice to
not duplicate the prototypes, to find the duplicates when we need them.

I can make all this reasonable, but I await comments before proceeding.


Oh. The test program I'm using is this:

 #include unistd.h
 #include time.h
 int main(void)
 {
 usleep(33);
 nanosleep((struct timespec){.tv_nsec = 44}, NULL);
 }

The test ltrace invocations are

 ./ltrace -x '*sleep' -L ./tst
 ./ltrace -l 'libc*' ./tst

Thanks.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] DWARF prototypes: handling symbol aliases

[Dima Kogan]

Dima Kogan li...@dima.secretsauce.net writes:

 As noted in the previous thread, there's an issue with parsing the
 prototypes from the DWARF data caused by multiple symbol names referring
 to the same address.

I looked at this a bit more, and have some working prototype code, and
I'd like some comments before I clean it up for merging.

The previous version would look for aliased names when parsing the
DWARF. This didn't work completely, as described in the last email.
Doing this when looking for exported symbols does work.

The current code parses the exported symbols into struct library{} in
library.h. This is

 struct library {
 
 struct library_symbol *symbols;
 struct library_exported_name *exported_names;
 
 }

The symbols structure contains names, addresses, etc. This is populated
only for symbols we're explicitly tracing (i.e. not with -l). Currently
breakpoints are set for all entries in this structure.

The exported_names are populated regardless of what we're tracing, and
contains just the export names. This is all done in
populate_this_symtab() in ltrace-elf.c. It's essentially

 populate_this_symtab()
 {
 for(symbols) {
 populate(exported_names);
 if (!filter_matches_symbol())
 continue;
 populate(symbols);
 }
 }

So to implement the alias-finding here the options are

1. Find the aliases in 'symbols'. This would require the

 if (!filter_matches_symbol())
continue;

   logic to be removed, and it would require extra logic to not set
   breakpoints for some 'symbols'

2. Find the aliases in 'exported_names'. This requires a fancier data
structure for 'exported_names'.

I have implemented #2, and it works. The code was a bit of a pain to
write, and it needs cleanup if we're to talk about merging it.
Implementing #1 would be a pain too, but a very different pain. Petr, do
you have any preference here?

Please let me know if anything is unclear.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Mark Wielaard m...@redhat.com writes:

 On Mon, 2014-05-12 at 16:36 +0200, Petr Machata wrote:
 I see it now.  So correct me if I'm wrong, but that means that after the
 first dwfl_report_elf, dwfl_nextcu would iterate over just that one
 module. However after second dwfl_report_elf, dwfl_nextcu would again
 iterate over the first module, as well as the second module.
 
 If that's correct, then I believe what we should store at struct library
 is Dwfl_Module, not Dwfl itself.

 Yes, that seems correct. dwfl_report_elf does report the Dwfl_Module (or
 NULL on failure). So you can use that and dwfl_module_nextcu.

 Dima, I think you should be able to verify whether the second
 dwfl_nextcu iterates over the first module's data as well.  While that's
 probably harmless, it's a work duplication, and a better design would be
 to store Dwfl_Module at struct library and use dwfl_module_nextcu as
 indicated above.

Hi. I'll confess to not fully grokking what libdwfl does, as that
library is not encumbered by any documentation. For testing I added a
bit of diagnosting output right after the dwfl_nextcu() call, so that
snippet looks like

  while ((die = dwfl_nextcu(dwfl, die, bias)) != NULL) {
  fprintf(stderr, Looking at cu %s at %x\n,
  dwarf_diename(die), dwarf_dieoffset(die));

I then did some ltrace runs, filtering the output through

 grep 'Looking at' | sort | uniq -d

in order to see the same compile unit being processed more than once. I
did this with the same simple test program as before that calls usleep()
and nanosleep(). I tried with both -x and -l, and

   no duplicate CUs are being processed in either case.

Is this a sufficient test? Do I need to test more complicated
conditions?

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Compilation error with dwarf code

[Dima Kogan]

Thierry Fauck ( thierry @ linux.vnet.ibm.com ) thie...@linux.vnet.ibm.com 
writes:

 I also got this message if I don't have the proper libraries

 In file included from proc.c:39:0:
 dwarf_prototypes.h:4:30: fatal error: elfutils/libdwfl.h: No such file
 or directory
  #include elfutils/libdwfl.h

Hi. I fixed this, and it's committed to my local tree:

https://github.com/dkogan/ltrace/tree/build

I'm not an automake expert, and there may be a better way to do this,
but it works.

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

New version pushed. Comments inline.


Petr Machata pmach...@redhat.com writes:

 You addressed the *-at-variable in function prototypes, but this should
 apply everywhere

 Please drop the extra spaces.

I thought I took care of those earlier. In any case, those should be
fixed now.


 + switch (byte_size) {
 + case sizeof(int):
 + *type = is_signed ? ARGTYPE_INT : ARGTYPE_UINT;
 + return true;
 +
 + case sizeof(long):
 + *type = is_signed ? ARGTYPE_LONG : ARGTYPE_ULONG;
 + return true;
 + }
 +}

 This won't work on 32-bit arches, where int and long are both 4 bytes
 wide

I thought I took care of this too; weird. Fixed also.


 + while ((die = dwfl_nextcu(dwfl, die, bias)) != NULL) {

 This doesn't seem right.  dwfl_nextcu should go through all CU's of the
 whole Dwfl--i.e. if you add two Dwarf-based libraries to a process, it
 would iterate the former one twice.  Or maybe I don't quite grasp how
 your code works.

I'm not trying to do anything noteworthy here; just iterating through
all the compile units. You're saying that if a process that uses two
libraries, this could would look through those libraries twice? I don't
understand the issue you're describing.


 Is there a reason not to just use uint64_t for encoding?

Nope. Typo.


 Just a tip: %#02x includes the initial 0x.

Good tip! I'm leaving the current code alone to reduce churn, but
definitely will use that in the future.



 + type_struct_add(result, member_type,
 + newly_allocated_member_type);

 This can fail.

 + dict_insert(type_dieoffset_hash, die_offset, result);

 This as well.

I added more error checking.

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Is there a reason not to just use uint64_t for encoding?

 Nope. Typo.

 Then this doesn't need the cast ;)

oh yeah.


 One last thing.  Can you please:

 $ git remote add debian git://git.debian.org/git/collab-maint/ltrace.git
 $ git fetch debian
 $ git rebase debian/master
 $ git push -f

 ... so that I can merge cleanly, without the merge commit?

Rebased. Thanks for caring about that.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 I looked into this, and it's caused by the thread. If you modify your
 main() in hle1.c to do pthread_create() and call jedna() from the
 thread, then the in-second-thread jedna() call is shown with the default
 prototype. This is with 'ltrace -f -l'.

 Ah, you're right, I've got it reproduced now.  The following fixlet
 takes care of this problem:

Ah, great. Did you have any objections to merging the branch in its
current state? I have more patches, questions, comments, but I'd rather
start a new mailing list thread and patch series, if possible. This
thread is getting a bit long.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 The command at this point is

  ./ltrace -f [DK: forgot '-f' in the original email] -l 'libjpeg.so*' 
 /usr/bin/geeqie /tmp/small.jpg

  
 Second, running this way reports libjpeg-libjpeg calls correctly, but
 not geeqie-ltrace [PM: presumably libjpeg?] ones. So some libjpeg
 functions still get the wrong prototypes. For instance I see this:
  geeqie-jpeg_read_header(0x7f23f62506c0, 1, 0, 1496 unfinished ...

 I looked into this a bit. import_DWARF_prototypes() was only called on
 libjpeg.so.8. jpeg_read_header() was indeed parsed correctly, the parsed
 data just wasn't used. Were we looking in the 'geeqie' plib instead of

 Hmm, that's strange, it seems to work for me in a simplified test case:

 -- hle1.c --
 int jedna (void);
 int dva (void);
 int main (int argc, char *argv[]) { jedna (); return dva (); }

 -- hle2.c --
 int dva (void) { return 2; }
 int jedna (void) { return dva (); }

 $ gcc hle1.c -g -L. -Wl,-rpath,. -lhle2
 $ gcc hle2.c -g -fpic -shared -o libhle2.so
 $ ~/tmp/ltrace/build/ltrace -llibhle* ./a.out -e''
 a.out-jedna( unfinished ...
 libhle2.so-dva()   = 2
 ... jedna resumed )   = 2
 a.out-dva()= 2
 +++ exited (status 2) +++

I looked into this, and it's caused by the thread. If you modify your
main() in hle1.c to do pthread_create() and call jedna() from the
thread, then the in-second-thread jedna() call is shown with the default
prototype. This is with 'ltrace -f -l'. I have a little test library I
use to run these tests, and I'm attaching it so that you can see this in
action. Here's what happens:

 dima@shorty:~/projects/ltrace$ cd ltracetests/

 dima@shorty:~/projects/ltrace/ltracetests$ make
 snip

 dima@shorty:~/projects/ltrace/ltracetests$ ../ltrace -f -l tstlib.so ./tst

 [pid 23546] tst-linkedlisttest({ 5, nil })= void
 [pid 23546] tst-treetest({ 1, { 2, nil, nil }, { 3, nil, { 4, nil, nil } } } 
unfinished ...
 [pid 23546] tstlib.so-treetest({ 2, nil, nil })   = nil
 [pid 23546] tstlib.so-treetest({ 3, nil, { 4, nil, nil } } unfinished ...
 [pid 23546] tstlib.so-treetest({ 4, nil, nil })   = nil
 [pid 23546] ... treetest resumed )   = { 5, 
nil, nil }
 [pid 23546] ... treetest resumed )   = { 2, 
{ 3, nil, nil }, { 4, nil, { 5, nil, nil } } }
 [pid 23546] tst-looptest({ { recurse^, 6 }, 5 })  = void
 [pid 23546] tst-enumtest(A, B)= 0
 [pid 23546] tst-arraytest({ [ 1.00, 2.00, 1.00, 2.00... ] }) 
= 1.00
 [pid 23546] tst-enum_cxx_test( unfinished ...
 [pid 23546] tstlib.so-_Z3f127Colors2(GREEN2)  = RED1
 [pid 23546] tstlib.so-_Z3f347Colors4(BLUE4)   = RED3
 [pid 23546] tstlib.so-_Z3f557Colors5(RED5)= RED5
 [pid 23546] ... enum_cxx_test resumed )  = void
 [pid 23546] tst-stringtest(test)= 4
 [pid 23546] tst-stringtest(nil)   = -1
 [pid 23546] tst-FILEtest(0x7f84e7bc42a0, 0x7f84e7bc4060)  = void
 [pid 23546] tst-voidfunc()= void
 in thread
 [pid 23547] tst-looptest(0x7f84e7001f00, 10, 0x7f84e7bc5790, -1)  = 
0x7f84e7001ef0
 [pid 23547] +++ exited (status 0) +++
 [pid 23546] +++ exited (status 0) +++

Note the looptest() call. The first one is in the main thread, and it's
reported correctly. The second is in a child thread, and it's
misreported.

dima
 


ltracetests.tar.gz
Description: Binary data
___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 By the way, looking into dwarf_prototypes.c, I see a number of style
 points that will need addressing before this is accepted:

Hi. I patched the sources to take your style suggestions. I also rebased
to remove the temporary void*-printing fix:

 https://github.com/dkogan/ltrace/commit/fef6728

There's still some out-standing work remaining, but it's all not a part
of the core functionality, I think. So I propose the current patch
series for merging.

The main thing I'm looking at now is how to trace things like
__nanosleep(). Sounds good?

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

I plugged all my memory leaks. Stuff still leaks, but nothing new that I
introduced. Some comments inline.


 Dima Kogan li...@dima.secretsauce.net writes:

 1. filter_matches_symbol() and dwlf is leaking. Every time I call these
 we leak a bit. The dwfl is out of our hands, but the
 filter_matches_symbol() maybe could be plugged:

I looked into this. Ltrace is definitely leaking it. The regex is
released when filter_destroy() calls filter_rule_destroy(), but those
are not called by anything.



 2. The prototypes my DWARF parser produces leak a bit. Those are stored
 with protolib_add_prototype(). Is that sufficient? I.e. do protolibs
 eventually clean out their prototypes?

plugged

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 -e traces PLT calls (i.e. inter-library calls), -x traces symbol entry
 points.  -l traces PLT calls done to a symbol defined by a library in
 -l.

 Can we add this to the manpage? I don't feel qualified to do this myself
 yet.

 How about this?

 diff --git a/ltrace.1 b/ltrace.1
 index f683844..93032f2 100644
 --- a/ltrace.1
 +++ b/ltrace.1
 @@ -122,11 +122,13 @@ describing which debug messages should be displayed.  
 Use the option
  \-Dh to see what can be used, but note that currently the only
  reliable debugmask is 77, which shows all debug messages.
  .IP \-e \fIfilter
 -A qualifying expression which modifies which library calls to trace.
 -The format of the filter expression is described in the section
 -\fBFILTER EXPRESSIONS\fR.  If more than one \-e option appears on the
 -command line, the library calls that match any of them are traced.  If
 -no \-e is given, \fB@MAIN\fR is assumed as a default.
 +A qualifying expression which modifies which library calls (i.e. calls
 +done through PLT slots, which are typically calls from the main binary
 +to a library, or inter-library calls) to trace.  The format of the
 +filter expression is described in the section \fBFILTER
 +EXPRESSIONS\fR.  If more than one \-e option appears on the command
 +line, the library calls that match any of them are traced.  If no \-e
 +is given, \fB@MAIN\fR is assumed as a default.
  .IP \-f
  Trace child processes as they are created by
  currently traced processes as a result of the fork(2)
 @@ -145,6 +147,9 @@ Print the instruction pointer at the time of the library 
 call.
  .IP \-l, \-\-library \fIlibrary_pattern
  Display only calls to functions implemented by libraries that match
  .I library_pattern.
 +This is as if you specified one \-e for every symbol implemented in a
 +library specified by
 +.I library_pattern.
  Multiple library patters can be specified with several instances of
  this option.  Syntax of library_pattern is described in section
  \fBFILTER EXPRESSIONS\fR.
 @@ -200,10 +205,12 @@ option enabled only if elfutils or libunwind support 
 was enabled at compile
  time.
  .IP \-x \fIfilter
  A qualifying expression which modifies which symbol table entry points
 -to trace.  The format of the filter expression is described in the
 -section \fBFILTER EXPRESSIONS\fR.  If more than one \-x option appears
 -on the command line, the symbols that match any of them are traced.
 -No entry points are traced if no \-x is given.
 +to trace (those are typically calls inside a library or main binary,
 +though PLT calls, traced by \-e, land on entry points as well).  The
 +format of the filter expression is described in the section \fBFILTER
 +EXPRESSIONS\fR.  If more than one \-x option appears on the command
 +line, the symbols that match any of them are traced.  No entry points
 +are traced if no \-x is given.
  .IP \-V, \-\-version
  Show the version number of ltrace and exit.

Sure, that's much better than what was there before. I think a little
example would be ideal here. I might add one when I'm done with the
DWARF stuff. Hopefully I'll get to that today...

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 - If I define the type in libwhatever.so.conf then it works. If I
   define the type in .ltrace.conf, it doesn't work (DWARF definition
   used). If I define it in both, the it STILL doesn't work. Not sure why
   yet

 Hmm, if I recall, .ltrace.conf works as an implicit import to all
 imported libraries (so that you can put there whatever and it just gets
 picked up by default).

 So this is the result of my advice to only look to immediate protolib
 when looking up prototypes.  It seems you need to look recursively after
 all.  (Which should just mean passing true to
 protolib_lookup_prototypes.)

Passing true to protolib_lookup_type() makes it work the way one would
expect. Passing true to protolib_lookup_prototypes() doesn't make it
work. This isn't surprising, I think. Should I commit this change
(passing true to both), or was this just a suggestion for testing? I'm a
bit concerned about this treating the type namespace as global, even
though it isn't. What would happen if two different DSOs define an
identically-named type that isn't actually the same? Can looking for
exports get confused by this?


 Does it by any change give you these Redefinition of typedef messages?

Yes.


 - With things like FILE*, I might want it to be printed as a pointer,
   but I don't see an obvious way to do that. I can define a lens
   'typedef FILE = hex(int)', but this says that FILE is an integer, not
   FILE*. Am I missing something? Thoughts about this?

 This should be either addr or void*.

I don't think I was clear (or I didn't fully grok your suggestion).
Ideally I want to be able to define a FILE as something that always has
its pointer printed. So for instance in my little tst library I have
this in the header:

 void FILEtest( FILE* a, const FILE* b );

A lens that does what I want is

 void FILEtest( void*, void* );

But I'd like to achieve this with some sort of lens typedef on FILE. Is
this possible? Or more to the point, what would be the most useful
ltrace output for a FILEtest() call, and how should it be implemented?

I think this all works sufficiently now for me to get the details ironed
out (memory, error handling), so I'm going to be looking at that next.

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Dima Kogan li...@dima.secretsauce.net writes:

 OK. My near-term todo list:

 - Try to get prototypes from both DWARF and conf files.

I ran some tests, and pushed some new patches to that same tree. Some
joint .conf/DWARF prototyping now works!

I can now prototype a function in a .conf file, and the DWARF parsing
would then ignore this function's DWARF definition (it might be good to
still read the DWARF to make sure things match up, but I'm not doing
that yet). So for instance if I have a function that takes a string:
void f(const char* s); then the DWARF would interpret this as a pointer
to a single char. But the .conf can say 'void f(string);', and the right
thing then happens.

Similary I can define a named type in a .conf and have it override a
similarly-named definition in the DWARF. This works, but has some
issues:

- If I define the type in libwhatever.so.conf then it works. If I
  define the type in .ltrace.conf, it doesn't work (DWARF definition
  used). If I define it in both, the it STILL doesn't work. Not sure why
  yet

- With things like FILE*, I might want it to be printed as a pointer,
  but I don't see an obvious way to do that. I can define a lens
  'typedef FILE = hex(int)', but this says that FILE is an integer, not
  FILE*. Am I missing something? Thoughts about this?

I still haven't cleaned up the error handling, memory deallocation, but
that's on the list. In case it may be useful, I'm attaching a small test
case library I'm using to run tests.

dima



ltracetests.tar.gz
Description: Binary data
___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 The integration with conf-based protolibs is kinda tricky.  What we
 currently do for somelib.so.1.2.3 is look for somelib.so.1.2.3.conf,
 then somelib.so.1.2.conf, then somelib.so.1.conf, and then
 somelib.so.conf.  I think it's only there that we should fall back on
 dwarf

Hi Petr. Thanks for the detailed review. I'll address your comments in a
bit, but I still have some high-level questions.

I think it would be most useful to read in a combination of DWARF and
.conf files, instead of just one of the two. For instance, the code I
have can parse FILE structures, so when ltrace decodes an fprintf()
call, you get the full dissection of the FILE argument. This is less
than useful. In cases like this I can imagine a .conf file can define a
lens that says how to display a FILE, and the DWARF parsing can then be
used to pull out the function prototype. I'm thinking that the .conf
files should be parsed first, and those should take precedence over any
DWARF data. Any thoughts on this? Implementation suggestions?

Currently I parse the DWARF from all our DSOs, and there's a lot of
overlapping DWARF data there. It's all imported into separate plibs.
This sounds like it's probably fine (although inefficient). Thoughts?

The code I have tries to consult the ltrace filters to parse the DWARF
only from functions and libraries the user asked about. I can't tell if
this works or not. The ltrace documentation is very unclear on the
difference between -e, -x and -l, so I don't yet know if that code is
working correctly. Does it look like it's correct? I'm talking about the
filter_matches_symbol() and filter_matches_library() calls.

I think that's it. Thanks again

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel

Re: [Ltrace-devel] Getting prototypes from debug information

[Dima Kogan]

Petr Machata pmach...@redhat.com writes:

 Dima Kogan li...@dima.secretsauce.net writes:

 Currently it appears that all function prototypes are read from the
 .conf files. Can these be read in from the debug information instead?

 They sure can.

 ltrace already uses libdw, so I'm assuming either this is not possible,
 or it's on somebody's short-term todo list. Any thoughts about it?

 It uses it optionally for unwinding, yes.  It could use it optionally
 for prototype retrieval as well.  This idea has crossed my mind in the
 past, and I wouldn't oppose patches that extend ltrace into accepting
 debuginfo if available.

OK, I'll take a look. By the way, what was the issue with libunwind?
Isn't unwinding the whole point of that library?

Thanks

dima

___
Ltrace-devel mailing list
Ltrace-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/ltrace-devel