Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:  closed
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  needs_review => closed
 * resolution:   => fixed


Comment:

 This looks good. This is now commit
 `5be41831b52795ee87c57cdbd18da68bf73fe1cc` in master.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  needs_revision => needs_review


Comment:

 Alright! How does `bug_25481_v5` (https://gitweb.torproject.org/user/gk
 /tor-browser-build.git/commit/?h=bug_25481_v5) look like?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  needs_review => needs_revision


Comment:

 One small thing: the `prev_version` variable in `projects/rust/config`
 should be named `var/prev_version` instead, as it is not a variable that
 has a specific meaning for rbm.

 Other than this the patch looks good to me.

 I tried a 32bit build of Tor, ran checksec on it, and have been able to
 bootstrap with it.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * keywords:  tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804
 => tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804R
 * status:  needs_revision => needs_review


Comment:

 Replying to [comment:9 boklm]:
 > Replying to [comment:6 gk]:
 > > So, I worked around that by building LLVM during the rust compilation
 (which make the whole process taking significantly more time) using that
 version in turn.
 >
 > I don't see LLVM being built in your patch. Is it done internally by the
 rust build system?

 Yes, it uses the LLVM code that is shipped with the Rust source.

 > The extracting of gcc and update of the PATH/LD_LIBRARY_PATH could be
 replaced by:
 > {{{
 > [% pc('gcc', 'var/setup', { compiler_tarfile =>
 c('input_files_by_name/gcc') }) %]
 > }}}
 > However it will also set hardened-cc links. Is it because the build
 doesn't work with hardened-cc that you didn't use it?

 No, I originally wanted to avoid using our own GCC as this adds complexity
 but piece by piece I realized that this is not realistic and added
 workaounds which resulted in the final patch, forgetting about this macro.
 This is fixed now.

 > For the `ln -s gcc cc` link, I am wondering if we should add it directly
 in `projects/gcc/build`, or if there are cases where we might not want
 this link.

 I can avoid this clumsy symlink by using a configure option which seems
 neat and which we probably need to use anyway when actually cross-
 compiling. So, I think we should go with that.

 > In `projects/tor/build`, I think the lines after the `IF c("var/linux")
 && c("var/nightly")` can be indented.

 Fixed.

 The updated patch is at `bug_25481_v4`
 (https://gitweb.torproject.org/user/gk/tor-browser-
 build.git/commit/?h=bug_25481_v4=fde93d07ca425ee884930cf01fd435b31ace48b8)
 in my public `tor-browser-build` repo. Note: I tested the x86_64 tor and
 it is running fine for me. I even checked whether tor built from the same
 revision twice is identical, and it is! Moreover, I verified that the
 hardening properties on the tor binary are still the ones we want to have
 (with the checksec script). So, we are good here. I guess the 32bit tor
 could get tested, though, which I did not do yet.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804  |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by gk):

 Replying to [comment:10 boklm]:
 > Replying to [comment:9 boklm]:
 > > For the `ln -s gcc cc` link, I am wondering if we should add it
 directly in `projects/gcc/build`, or if there are cases where we might not
 want this link.
 >
 > If we add the `cc -> gcc` link directly in `projects/gcc/build`, we can
 remove the same link created in the windows build in
 `projects/firefox/build`.

 I have not looked at the Windows part for the Firefox build but I have a
 better fix for the Rust build and think we should use that one.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804  |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by boklm):

 Replying to [comment:9 boklm]:
 > For the `ln -s gcc cc` link, I am wondering if we should add it directly
 in `projects/gcc/build`, or if there are cases where we might not want
 this link.

 If we add the `cc -> gcc` link directly in `projects/gcc/build`, we can
 remove the same link created in the windows build in
 `projects/firefox/build`.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804  |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  needs_review => needs_revision
 * keywords:  tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804R
 => tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804


Comment:

 Replying to [comment:6 gk]:
 > So, I worked around that by building LLVM during the rust compilation
 (which make the whole process taking significantly more time) using that
 version in turn.

 I don't see LLVM being built in your patch. Is it done internally by the
 rust build system?

 The extracting of gcc and update of the PATH/LD_LIBRARY_PATH could be
 replaced by:
 {{{
 [% pc('gcc', 'var/setup', { compiler_tarfile =>
 c('input_files_by_name/gcc') }) %]
 }}}
 However it will also set hardened-cc links. Is it because the build
 doesn't work with hardened-cc that you didn't use it?

 For the `ln -s gcc cc` link, I am wondering if we should add it directly
 in `projects/gcc/build`, or if there are cases where we might not want
 this link.

 In `projects/tor/build`, I think the lines after the `IF c("var/linux") &&
 c("var/nightly")` can be indented.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by cypherpunks):

 Maybe, it's time to get rid of Wheezy as its EOL is May 2018.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  new => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:  new
 Priority:  High |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201804,  |  Actual Points:
  boklm201804, TorBrowserTeam201804R |
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * keywords:  tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804
 => tbb-rbm, GeorgKoppen201804, boklm201804, TorBrowserTeam201804R


Comment:

 `bug_25481_v2` (https://gitweb.torproject.org/user/gk/tor-browser-
 build.git/commit/?h=bug_25481_v2=1f7c6f2ae24c9e79382bd1fdf380bbceac4e4d1c)
 has a patch for this bug up for review. Some notes might be in order,
 though:

 1) Rust needs CMake 3.4.3 at least + LLVM 3.9, and some compiler that
 supports C++-11. It turns out we want the first two anyway during the
 update of our macOS toolchain and g++4.7 is the minimum requirement
 outlined on the rust-lang website. So, we should be good, right? Alas, it
 turns our that's not the case. For one, not all parts we need to build are
 happy with g++ 4.7.2 which Wheezy ships, so we need our own compiler. But,
 two, even salvaging some of our LLVM build efforts are failing: using LLVM
 3.9 crashes the compiler during the cargo build step. See e.g.:
 {{{
   process didn't exit successfully:
 `/var/tmp/build/rustc-1.25.0-src/build/build/bootstrap/debug/rustc
 --crate-name lazycell vendor/lazycell/src/lib.rs --crate-type lib --emit
 =dep-info,link -C opt-level=2 -C metadata=1c971c2102b3dfa3 -C extra-
 filename=-1c971c2102b3dfa3 --out-dir
 /var/tmp/build/rustc-1.25.0-src/build/build/x86_64-unknown-linux-
 gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps --target x86_64
 -unknown-linux-gnu -L
 dependency=/var/tmp/build/rustc-1.25.0-src/build/build/x86_64-unknown-
 linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps -L
 dependency=/var/tmp/build/rustc-1.25.0-src/build/build/x86_64-unknown-
 linux-gnu/stage2-tools/release/deps --cap-lints allow` (signal: 11,
 SIGSEGV: invalid memory reference)
 }}}
 So, I worked around that by building LLVM during the rust compilation
 (which make the whole process taking significantly more time) using that
 version in turn.

 2) The current setup is building basically all the tools which come with
 the `rustc` source tarball. We might only need `rustc` and `cargo`, and
 one way to achieve that might be to split both and build them in separate
 projects. I am fine if we want to pursue this option later on after we got
 the basics working.

 3) There is a bunch of documentation that is getting built during normal
 compilation. I guess we can optimize that away later on and find other
 knobs to produce a smaller toolchain faster.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-
 |  team
 Type:  task | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, boklm201803,|  Actual Points:
  TorBrowserTeam201803, GeorgKoppen201803|
Parent ID:  #25220   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * keywords:  tbb-rbm, boklm201803, TorBrowserTeam201803. GeorgKoppen201803
 => tbb-rbm, boklm201803, TorBrowserTeam201803, GeorgKoppen201803


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #25481 [Applications/Tor Browser]: Ship tor in Tor Browser nightly builds on Linux with Rust enabled

[Tor Bug Tracker & Wiki]

#25481: Ship tor in Tor Browser nightly builds on Linux with Rust enabled
-+-
 Reporter:  gk   |  Owner:  tbb-team
 Type:  task | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor |Version:
  Browser|   Keywords:  tbb-rbm, boklm201803,
 Severity:  Normal   |  TorBrowserTeam201803.
 |  GeorgKoppen201803
Actual Points:   |  Parent ID:  #25220
   Points:   |   Reviewer:
  Sponsor:   |
-+-
 We should test tor with Rust enabled in our Linux nightly builds.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs