Bug#627132: New "-custom" binary generation breaks stack backtraces
Le 16/11/2011 00:07, Stephen McCamant a écrit : > So my personal opinion is that the original bug isn't really "fixed", > but it's close enough that I think it would be within your > maintainer's discretion to declare it so. Or alternatively this feels > somewhat like a "help" or "wontfix" situation, though it's not a > perfect match for either of those either IMO. You're right. I was thinking about sub-bug #627761 when I wrote my mail yesterday. This one (#627132) still stands. The buglog from [1] should be more clear now :-) [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=627132 Cheers, -- Stéphane -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
SMcC> The "strip"ping problem that appeared to motivate the new SMcC> embedding process had never bothered us, so another suitable SMcC> workaround from our perspective would be if there was a way to SMcC> disable the new "-output-obj"-style behavior. SG> On 05/24/2011 11:41 AM, Stéphane Glondu wrote: SG> I am working on that... [...] SG> Actually, there is already one: just put "c" in an environment SG> variable called "OCAML_COMPAT" to get the upstream behaviour. SG> Is that enough for you? It's an effective workaround for the problem I mentioned in the original email, yes. IMO, the one way in which it feels like a workaround rather than a proper solution is that it's still a divergence from the upstream behavior, but once you know about the issue it's not hard to set the environment variable. What we actually chose to do as the default behavior for the project where this arose is to switch to just compiling a native code binary with debugging symbols. This makes sense for us because we mostly care just about backtraces and not running "ocamldebug", and skipping the bytecode makes the build process noticeably faster. So in that sense none of the issues fixed under this bug have been blocking us. Though maybe what you're really asking is whether you should close this bug? As I understand the history you forked off and then resolved two sub-bugs, which make possible two different solutions/workarounds to the problem I originally posed: #627756 Sys.executable_name is not set properly by caml_startup_code #627761 provide a way to use legacy custom linking Setting those aside, I guess this original bug stands for the proposition that certain ways of ways of generating an executable with "-custom", which work to produce debuggable executables on previous and upstream versions of ocaml, produce non-debuggable executables when one uses the same compilation command with the current Debian version. There are a number of workarounds for this issue, as the previous messages discuss. Because of that, I'd gotten the impression that you did not consider fixing the original issue a very high priority, but I still think doing so would be technically possible, and would have some value in making the Debian version of OCaml "just work" for more use cases. So my personal opinion is that the original bug isn't really "fixed", but it's close enough that I think it would be within your maintainer's discretion to declare it so. Or alternatively this feels somewhat like a "help" or "wontfix" situation, though it's not a perfect match for either of those either IMO. Thanks, -- Stephen -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
On 05/24/2011 11:41 AM, Stéphane Glondu wrote: >> The "strip"ping problem that appeared to motivate the new embedding >> process had never bothered us, so another suitable workaround from our >> perspective would be if there was a way to disable the >> new "-output-obj"-style behavior. > > I am working on that... Actually, there is already one: just put "c" in an environment variable called "OCAML_COMPAT" to get the upstream behaviour. Is that enough for you? Cheers, -- Stéphane -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
clone 627132 -1 retitle -1 provide a way to use legacy custom linking thanks Le 17/05/2011 23:33, Stephen McCamant a écrit : The "strip"ping problem that appeared to motivate the new embedding process had never bothered us, so another suitable workaround from our perspective would be if there was a way to disable the new "-output-obj"-style behavior. I am working on that... Cheers, -- Stéphane -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
clone 627132 -1 retitle -1 Sys.executable_name is not set properly by caml_startup_code severity -1 important thanks Le 20/05/2011 03:11, Stephen McCamant a écrit : I haven't checked the source code for this, but my first guess is that the program might be trying to find its own executable by looking in $PATH (/home/smcc/bin through /sbin) and the current directory. The reason it's currently failing is that it's using "" as the program name, which is of course incorrect. This maybe would work well enough if the program used argv[0] instead of "", but using /proc/self/exe would really be better, IMO. This is a bug. Sys.executable_name is not set properly by caml_startup_code, which is called by code generated by -output-obj. -- Stéphane -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
> "SG" == Stephane Glondu writes:
SG> Le 17/05/2011 23:33, Stephen McCamant a ecrit :
SMcC> [...] another suitable workaround from our perspective would be
SMcC> if there was a way to disable the new "-output-obj"-style
SMcC> behavior.
SG> The previous behaviour was compiling a runtime (as with
SG> -make-runtime), a pure bytecode using this runtime (as with
SG> -use-runtime), and concatenating both.
SG> Did you try this way to see if backtraces are still broken?
I hadn't tried anything like this before. Here are two possible
interpretations of your suggestion:
1. I first tried what seemed closest to what you were suggesting
there. With some tweaking this seems to work.
I previously had a command "ocamlc ... -o prog.dbg", and I
tried replacing it with two commands,
ocamlc -make-runtime -o prog.runtime -ccopt -Wl,--export-dynamic ...
and
ocamlc -use-runtime ./prog.runtime ... -o prog.dbg.cbc
This makes prog.dbg.cbc as a "#!..././prog.runtime" executable. So you
can either run prog.dbg.cbc directly, or equivalently "./prog.runtime
prog.dbg.cbc", and they both work including backtraces.
The one trick that wasn't immediately obvious is the "-ccopt
-Wl,--export-dynamic", which tells the linker to make the symbols from
the binary available to libraries subsequently loaded with
dlopen(). If you don't have that, you get an error:
Fatal error: cannot load shared library dllcamlstr
Reason: /usr/lib/ocaml/stublibs/dllcamlstr.so: undefined symbol:
caml_string_length
The function is indeed undefined in dllcamlstr.so, but it is defined
in the prog.runtime executable.
So this approach does seem to create two files that together are a
complete replacement from what we used to have in a single file.
Concatenating these two files (first the runtime, then the bytecode)
also works, but it's less convenient because the runtime doesn't know
it has to look at the end of itself for the bytecode: you have to pass
the concatenated program as its own first argument (another trick that
was not immediately obvious to me).
2. Another related thing I tried was taking the ELF executable
compiled with the new "-custom", and appending the bytecode file
generated with -use-runtime to the end. This seems closer to
reproducing a single concatenated file like came from the old
"-custom", with the downside that the executable has two copies of the
bytecode. This generates a program that runs OK, but backtraces still
don't quite work. With "strace", I see the following:
write(2, "Fatal error: exception Failure(\""..., 48Fatal error: exception
Failure("int_of_string")
) = 48
stat64("/home/smcc/bin/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/home/smcc/bin/i386/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/bin/", {st_mode=S_IFDIR|0755, st_size=3000, ...}) = 0
stat64("/usr/local/bin/", {st_mode=S_IFDIR|S_ISGID|0775, st_size=48, ...}) = 0
stat64("/usr/bin/", {st_mode=S_IFDIR|0755, st_size=156576, ...}) = 0
stat64("/usr/X11R6/bin/", 0xd3d0) = -1 ENOENT (No such file or directory)
stat64("/usr/sbin/", {st_mode=S_IFDIR|0755, st_size=13464, ...}) = 0
stat64("/sbin/", {st_mode=S_IFDIR|0755, st_size=5568, ...}) = 0
open("", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
write(2, "(Program not linked with -g, can"..., 59(Program not linked with -g,
cannot print stack backtrace)
) = 59
exit_group(2) = ?
I haven't checked the source code for this, but my first guess is that
the program might be trying to find its own executable by looking in
$PATH (/home/smcc/bin through /sbin) and the current directory. The
reason it's currently failing is that it's using "" as the program
name, which is of course incorrect. This maybe would work well enough
if the program used argv[0] instead of "", but using /proc/self/exe
would really be better, IMO.
So in summary, I think (1) is the best work-around so far if one
really wants to have bytecode (for ocamldebug, maybe?), while just
generating native code with -g (Eric Cooper's suggestion) gives you
the other advantages of native code plus backtraces if you're using
3.10 or later.
-- Stephen
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
Le 17/05/2011 23:33, Stephen McCamant a écrit : > [...] another suitable workaround from our > perspective would be if there was a way to disable the > new "-output-obj"-style behavior. The previous behaviour was compiling a runtime (as with -make-runtime), a pure bytecode using this runtime (as with -use-runtime), and concatenating both. Did you try this way to see if backtraces are still broken? Cheers, -- Stéphane -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
On Tue, May 17, 2011 at 02:33:26PM -0700, Stephen McCamant wrote: > We have a large OCaml application that links with a number of > native-code libraries; we use native-code compilation for most > purposes, but also use byte compilation specifically to support > debugging. In particular we run with OCAMLRUNPARAM=b to get stack > backtraces from uncaught OCaml exceptions. Backtraces are now supported from native code too. -- Eric Cooper e c c @ c m u . e d u -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#627132: New "-custom" binary generation breaks stack backtraces
Package: ocaml Version: 3.12.0-5 Severity: normal It appears that the new "-output-obj" style strategy for generating -custom bytecode executables (cf. patches/0011-Embed-bytecode-in-C-object-when-using-custom.patch) has broken the normal usage of stack backtraces in such executables. We have a large OCaml application that links with a number of native-code libraries; we use native-code compilation for most purposes, but also use byte compilation specifically to support debugging. In particular we run with OCAMLRUNPARAM=b to get stack backtraces from uncaught OCaml exceptions. When I upgraded to the latest Debian OCaml packages, this stopped working. Now when I try to run a ELF-wrapper-around-bytecode executable with the environment variable OCAMLRUNPARAM=b to debug an exception, I see the error message (Program not linked with -g, cannot print stack backtrace) even though I did, in fact, compile the program with "-g". It appears this is because read_debug_info() is searching for debugging information in the executable and not finding it. After looking through the OCaml source code, I found that a combination of undocumented features can be used to provide an inconvenient workaround to this problem. The relevant debugging information is being created at link time, but it appears that it is stored in a file with an unpredictable name in /tmp, with the extension .cds. Though the file names like "/tmp/camlobjfe90df.cds" sound like temporary files that would be deleted at the end of compilation, they are not deleted; I currently have more than 100 such files (taking more than 250MB) in /tmp. If I supply the undocumented CAML_DEBUG_FILE environment variable at runtime, pointing to the correct .cds file, then backtraces work again. However I don't see any proper way to find the correct filename; hacky things that work include running ocamlc under "strace -eopen" and looking for the most recent .cds file in /tmp. The "strip"ping problem that appeared to motivate the new embedding process had never bothered us, so another suitable workaround from our perspective would be if there was a way to disable the new "-output-obj"-style behavior. -- Stephen -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
