#25386: Link Rust Tests to C Dependencies in Tor (allow integration testing from Rust to C) -------------------------------------------------+------------------------- Reporter: Hello71 | Owner: Hello71 Type: defect | Status: | needs_revision Priority: High | Milestone: Tor: | 0.3.4.x-final Component: Core Tor/Tor | Version: Tor: | 0.3.3.1-alpha Severity: Normal | Resolution: Keywords: rust, tor-test, 033-backport, | Actual Points: review-group-34, 034-triage-20180328, | 034-included-20180401 | Parent ID: | Points: 3 Reviewer: isis | Sponsor: | SponsorQ -------------------------------------------------+-------------------------
Comment (by Hello71): My plan was originally to work on this instead of writing down the problems, but apparently I really don't want to work on it, so I'll write them down as I see them. This obsoletes most if not all of what I said above. If we want to use `cargo test` to test things, then the following must be resolved: 1. Cargo needs to learn to link C Tor libraries in when running Rust tests. It's fine if the solution to this problem actually links C Tor libraries always instead of only during tests, because static libraries do not carry dependency information on any modern platform. (I believe not even Windows, but correct me if I'm wrong.) This can be accomplished a few ways: a. set RUSTFLAGS and RUSTDOCFLAGS to "-L... -L... -L... -l... -l..." etc, either always or only when testing. the downside of this option is that it also links C Tor libraries when compiling Rust Tor dependencies, which sounds bad (but I suspect is actually fine if everything is linked together in "big bag of parts" manner at the end). always is better because cargo will save the value of these variables and recompile *all Rust code* (due to the previous problem). this isn't a huge issue now, but if we add a bunch of dependencies (like winapi, ugh) that could be an issue later. b. do the same thing but specify rustflags in cargo configuration c. use one or more build scripts that output all C Tor libraries as `cargo:rustc-link-lib` lines (make sure you add `cargo:rustc-link-search` too obviously). it is possible that using one in `external` only will work. I suspect we should rename and split `external` into `tor_c` and `tor_c-sys`, where the latter uses bindgen or manual curation for pure bindings only, and the former does high-level wrappers, but that can probably be done separately. I believe this is supposed to automatically work for rustdoc too. d. use "links" key and provide the libraries in cargo configuration. this is possibly less preferable because it forecloses other uses of build scripts, like bindgen. I am in favor of method c. It is possible that *none* of these work, however. This might be the case if Rust calls into C which calls back into the crate we are currently testing. If so, then a complicated build script might be necessary that excludes the current crate from the RUSTFLAGS. Additionally, note that "make a shared library" does *not* work, because then there will be multiple copies of global data structures. 2. sanitizers: if rustc is used to link tests (mandatory if using cargo test), then one of the following must be done: a. interpose the linker to add `-fsanitize` options. I suspect `-C link- arg` won't work because of the weird order dependency of linker arguments. I also suspect that this will not work properly, especially if we want Rust tests to be run with sanitizers enabled. b. pass -Z sanitizer options to Rust, either via exporting RUSTFLAGS or setting build.rustflags in cargo config. this requires that the Rust sanitizers are of a compatible version with the C sanitizers, otherwise there will be very strange errors, like multiple definition errors. usually, using clang to build C code is both necessary and sufficient. presently, gcc 8 should work with rustc 1.26, but probably not with newer Rust versions (gcc tends to trail behind on sanitizer versions). 3. a method should be devised if possible to test only one crate at a time. komlo asked for this somewhere above. probably a wrapper script is the best solution, but I'm not sure about the details. for example, should running it from the build directory be necessary? does it need to be autoconf generated, or can it be static? -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25386#comment:47> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs