Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  needs_review => closed
 * resolution:   => fixed


Comment:

 Thanks. Merged to `master` (commit
 f7ae040da39ac0009776802371e088f15531b9e7) and `maint-8.5` (commit
 7e784c57f614b97a9b2f614bc0550fd5ade1e735).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * keywords:  tbb-rbm, TorBrowserTeam201904, tbb-8.5 => tbb-rbm,
 TorBrowserTeam201904R, tbb-8.5
 * status:  needs_revision => needs_review


Comment:

 Replying to [comment:27 gk]:
 > Maybe a note in the commit message would be enough to help us
 remembering in case we stumble over that again in the future?

 Yes, I think it's probably enough. I did that in branch `bug_29981_v9`:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v9&id=f7ae040da39ac0009776802371e088f15531b9e7

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904,   |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  needs_review => needs_revision
 * keywords:  tbb-rbm, TorBrowserTeam201904R, tbb-8.5 => tbb-rbm,
 TorBrowserTeam201904, tbb-8.5


Comment:

 Replying to [comment:25 boklm]:
 > So it seems yasm is not needed for the android-armv7 build, but still
 needed for the android-x86 one.
 >
 > In branch `bug_29981_v8` I added a new commit mentioning that in the
 README:
 > https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v8&id=1959d1c9dbe993b8fb4732b333892cb04f7ccfae

 The previous `apt-get install` comment does not talk about `android-armv7`
 specifically, thus starting now talking of `android-x86` seems a bit weird
 to me. I think *if* we go with just installing `yasm` everywhere to not
 create an extra container (I agree with that idea) then we should treat it
 as a first-class citizen here and just add it to the other packages.
 Otherwise we are starting to divert regarding installed packages between
 container/non-container builds (think of folks thinking "I only want to
 build `android-armv7` packages, thus no yasm for me`) which might fall
 onto our feet at some point.

 Maybe a note in the commit message would be enough to help us remembering
 in case we stumble over that again in the future? Otherwise I could think
 of "This can be done
 with the following command (note: yasm is only needed for android-x86
 builds):" to make our point clear while minimizing the risk for deviating
 from the packages installed in our container builds.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  reopened => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  reopened
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 So it seems yasm is not needed for the android-armv7 build, but still
 needed for the android-x86 one.

 In branch `bug_29981_v8` I added a new commit mentioning that in the
 README:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v8&id=1959d1c9dbe993b8fb4732b333892cb04f7ccfae

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  needs_review => closed
 * resolution:   => fixed


Comment:

 Thanks for the detailed analysis! The new branch looks good to me. I've
 merged it to `master` (commits 9072578261fc2e4198e39b81313683bf2674c5b4
 and 0fc15a39f2adcf029676810c2ec37c566886879e).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by boklm):

 Replying to [comment:22 gk]:
 > Thanks for the update. How did you get the list of packages to install?
 E.g. I did not need to install `bison` nor `yasm` to get the mobile
 browser built, so it seems those are not actually needed.

 I looked at the list of packages we list as dependencies for android in
 the various `projects/*/config` files, and `rbm.conf`.

 For `bison`, we have it in `rbm.conf`. We added it to the default list of
 packages for the other platforms with commit
 `78a5009fa4e0fc3e7e4ccd854df7dd3833b6b174`, as it seems it is needed to
 build `binutils`, and adding it to the default list of packages avoids
 creating and storing a specific container for the binutils build. Then we
 probably copied it to the android list of dependencies without checking
 that it was really required. We could probably remove it from the osx-
 x86_64 build too as we don't need to build binutils there. I opened #30325
 for that.

 For yasm, we have it in the list of packages needed to build firefox on
 all platforms, although it seems it's not needed for the android build. I
 opened #30326 to fix that.

 In branch `bug_29981_v7` I updated the README to remove bison and yasm
 from the list of packages to install:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v7&id=0fc15a39f2adcf029676810c2ec37c566886879e

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by gk):

 Thanks for the update. How did you get the list of packages to install?
 E.g. I did not need to install `bison` nor `yasm` to get the mobile
 browser built, so it seems those are not actually needed.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904R,  |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * keywords:  tbb-rbm, TorBrowserTeam201904, tbb-8.5 => tbb-rbm,
 TorBrowserTeam201904R, tbb-8.5
 * status:  needs_revision => needs_review


Comment:

 Replying to [comment:14 gk]:
 > While still testing the changes here come some preliminary comments:
 >
 > commit e38b5d94a19caff488382d92968dbd8cd85b18eb
 >
 > The change to the `README` says "Adding the no_containers target will
 disable the use of containers", but the first thing that happens is
 actually building a container (called "empty"). So, we might want to be a
 bit more verbose explaining what is happening here to avoid confusion.

 I added something about that in the README, and renamed the file `empty`
 to `containers_disabled`.

 >
 > "and install build dependencies for all the components that are built"
 We are listing build dependencies in the `README` file. I think we should
 list all the additional build dependencies needed for building without
 containers as well while we are talking about them. There should be no
 need for builders that want to build without containers to try figuring
 that out themselves.
 >
 > commit 069c880961e93fe2f5052ae6d428b2f9efdd1d4e
 >
 > "Add the extensions in sorted order" it's not only extensions it seems?
 So, please adjust the commit message accordingly.

 I pushed a new revision of the commits fixing that in branch
 `bug_29981_v6`:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v6&id=718d1ab83e497cc9492d3c8b1fc101095e984e81


 >
 > I wonder why we need `container/use_container` and `container/disable`
 as two separate options. If `use_container` is false then cleary I build
 without containers, no? In other words: why can't we just use/amend the
 already existing `use_container` option to avoid creating another one that
 seems to do the same thing but just (slightly) differently?

 `container/use_container` is defined in projects configs, which have
 priority over definitions in `rbm.conf`. So if we want to change the value
 of `container/use_container` for the `no_containers` target, we would have
 to do it in all `projects/*/config` files where it is defined. I have
 renamed `var/container/disable` to `var/container/global_disable` to try
 to make it less confusing.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904,   |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by gk):

 Replying to [comment:19 boklm]:
 > Replying to [comment:17 gk]:
 >
 > >
 > > Okay, installing openjdk-8-jdk does not make a difference. I attached
 the package list of the stretch machine I used. In fact, you have access
 to it, too, it's the TPO build machine. In case you want to make more
 elaborate queries to figure out what the important difference here is.
 >
 > I did a build without containers on build-sunet-a, with commit
 `069c880961e93fe2f5052ae6d428b2f9efdd1d4e`, after installing all the
 packages from comment 13, I have x86 and armv7 apk files matching the ones
 I built on an other machine with containers.
 >
 > Can you try to do an other build on this machine?

 Will do.

 > Also, did you do alpha or nightly builds? In my case I did alpha builds,
 as it makes it easier to be sure both machines are using the same commits.

 Nightlies. But that should not matter much here as the Firefox libs where
 different and I used the same Firefox commit for both builds. But I can
 try doing an alpha one.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904,   |  Actual Points:
  tbb-8.5|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by boklm):

 Replying to [comment:17 gk]:

 >
 > Okay, installing openjdk-8-jdk does not make a difference. I attached
 the package list of the stretch machine I used. In fact, you have access
 to it, too, it's the TPO build machine. In case you want to make more
 elaborate queries to figure out what the important difference here is.

 I did a build without containers on build-sunet-a, with commit
 `069c880961e93fe2f5052ae6d428b2f9efdd1d4e`, after installing all the
 packages from comment 13, I have x86 and armv7 apk files matching the ones
 I built on an other machine with containers.

 Can you try to do an other build on this machine?

 Also, did you do alpha or nightly builds? In my case I did alpha builds,
 as it makes it easier to be sure both machines are using the same commits.

 This is the apk files that I get:
 {{{
 e88fcd32929f86765e17ef0ccf1979c64a1fe9b2d6e2099590bac3b061f31487
 testbuild/tor-browser-8.5a11-android-armv7-multi-qa.apk
 df0823edbc7aeb3c19a33bada6edde6743958d4b71629a24e6fed8d999ff4814
 testbuild/tor-browser-8.5a11-android-x86-multi-qa.apk
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * Attachment "pkg_list_tpo" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by gk):

 Replying to [comment:16 boklm]:
 > Replying to [comment:15 gk]:
 > >
 > > FWIW: I built on the bare stretch machine without installing bison,
 yasm, and openjdk-8-jdk. (However, openjdk-8-jre is installed it seems)
 >
 > I'm wondering if the difference in the build could be caused by some
 different versions of packages installed, or maybe additional or missing
 packages.
 >
 > In my case, the build that I did without containers is matching the one
 done with containers. I attached the list of packages installed on this
 machine (the output from `dpkg -l`). Is there any difference with the list
 of packages installed on your build machine, other than bison, yasm and
 openjdk-8-jdk?

 Okay, installing openjdk-8-jdk does not make a difference. I attached the
 package list of the stretch machine I used. In fact, you have access to
 it, too, it's the TPO build machine. In case you want to make more
 elaborate queries to figure out what the important difference here is.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by boklm):

 Replying to [comment:15 gk]:
 >
 > FWIW: I built on the bare stretch machine without installing bison,
 yasm, and openjdk-8-jdk. (However, openjdk-8-jre is installed it seems)

 I'm wondering if the difference in the build could be caused by some
 different versions of packages installed, or maybe additional or missing
 packages.

 In my case, the build that I did without containers is matching the one
 done with containers. I attached the list of packages installed on this
 machine (the output from `dpkg -l`). Is there any difference with the list
 of packages installed on your build machine, other than bison, yasm and
 openjdk-8-jdk?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * Attachment "pkg_list.txt" added.

 List of packages installed on my build machine.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * Attachment "liblgpllibs_2.so" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * Attachment "liblgpllibs_1.so" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by gk):

 Alright, I started a build on two different machines, both on commit
 069c880961e93fe2f5052ae6d428b2f9efdd1d4e (boklm/bug_29981_v5). One machine
 is building with containers and one without. While this setup builds
 reproducibly on each machine the *.apks on one machine differ from those
 on the second one.

 The differing parts are essentially all the Firefox .so files in assets/
 or lib/. Both machines built from the same `tor-browser` commit
 (c722d57604db58695140d95565a78433989fe9ca).

 I attached two .so files showing the issue.

 FWIW: I built on the bare stretch machine without installing bison, yasm,
 and openjdk-8-jdk. (However, openjdk-8-jre is installed it seems)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * status:  needs_review => needs_revision
 * keywords:  tbb-rbm, tbb-8.5-must, TorBrowserTeam201904R => tbb-rbm,
 tbb-8.5-must, TorBrowserTeam201904


Comment:

 While still testing the changes here come some preliminary comments:

 commit e38b5d94a19caff488382d92968dbd8cd85b18eb

 The change to the `README` says "Adding the no_containers target will
 disable the use of containers", but the first thing that happens is
 actually building a container (called "empty"). So, we might want to be a
 bit more verbose explaining what is happening here to avoid confusion.

 "and install build dependencies for all the components that are built" We
 are listing build dependencies in the `README` file. I think we should
 list all the additional build dependencies needed for building without
 containers as well while we are talking about them. There should be no
 need for builders that want to build without containers to try figuring
 that out themselves.

 commit 069c880961e93fe2f5052ae6d428b2f9efdd1d4e

 "Add the extensions in sorted order" it's not only extensions it seems?
 So, please adjust the commit message accordingly.

 I wonder why we need `container/use_container` and `container/disable` as
 two separate options. If `use_container` is false then cleary I build
 without containers, no? In other words: why can't we just use/amend the
 already existing `use_container` option to avoid creating another one that
 seems to do the same thing but just (slightly) differently?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904R  |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * status:  new => needs_review
 * keywords:  tbb-rbm, tbb-8.5-must, TorBrowserTeam201904 => tbb-rbm,
 tbb-8.5-must, TorBrowserTeam201904R


Comment:

 There are two patches for review in branch `bug_29981_v5`:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v5&id=069c880961e93fe2f5052ae6d428b2f9efdd1d4e

 I have been able to get matching builds from this branch for the
 android-x86 and android-armv7 builds.

 The packages that I installed on a stretch machine to be able to do the
 build are:
 {{{
 build-essential python bison automake libtool zip unzip autoconf2.13 yasm
 openjdk-8-jdk gettext-base autotools-dev automake autoconf libtool
 autopoint libssl-dev pkg-config zlib1g-dev libparallel-forkmanager-perl
 libfile-slurp-perl bzip2 xz-utils apksigner
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-
 |  team
 Type:  task | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor Browser |Version:
 Severity:  Normal   | Resolution:
 Keywords:  tbb-rbm, tbb-8.5-must,   |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by gk):

 * keywords:  tbb-rbm, TorBrowserTeam201904 => tbb-rbm, tbb-8.5-must,
 TorBrowserTeam201904


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by eighthave):

 Wow, great progress!  That sort order issue is frustrating.  My experience
 is that doing post-processing of APKs with `zip` or other tools usually
 leads to random repro issues like that.  The Android tools have been
 getting better, but they still have repro quirks.  It seems that the `zip`
 commands are used for adding and removing static assets.  I can help move
 that to the pre-packaging phase of the Android build, if gradle is in use.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by boklm):

 Using the patch from branch `bug_29981_v3` I have been able to do a build
 without containers:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_29981_v3&id=ab3c5d96be66d09d4dc98be385e495244bef48af

 The apk I got does not match the one built using container, but the only
 difference is:
 {{{
 --- tor-browser-8.5a10-android-armv7-multi-qa.apk
 +++ ../../testbuild/tor-browser-8.5a10-android-armv7-multi-qa.apk
 ├── zipinfo {}
 │ @@ -1502,13 +1502,13 @@
 │  -rwxr-xr-x  2.0 unx 5596 b- defX 10-Jan-01 00:00 lib/armeabi-v7a
 /libplugin-container.so
 │  -rw-r--r--  2.0 unx  382 b- defX 10-Jan-01 00:00 application.ini
 │  -rw-r--r--  2.0 unx   32 b- defX 10-Jan-01 00:00 package-name.txt
 │  -rw-r--r--  2.0 unx  795 b- defX 10-Jan-01 00:00 ua-update.json
 │  -rw-r--r--  2.0 unx   48 b- defX 10-Jan-01 00:00 platform.ini
 │  -rw-r--r--  2.0 unx   97 b- defX 10-Jan-01 00:00 removed-files
 │  -rw-r--r--  2.0 unx 10641927 b- stor 10-Jan-01 00:00 assets/omni.ja
 │ --rw-r--r--  3.0 unx   502590 b- defN 00-Jan-01 00:00
 assets/distribution/extensions/{73a6fe31-595d-460b-a920-fcc0f8843232}.xpi
 │  -rw-r--r--  3.0 unx  1765652 b- defN 00-Jan-01 00:00
 assets/distribution/extensions/https-everywhere-...@eff.org.xpi
 │ +-rw-r--r--  3.0 unx   502590 b- defN 00-Jan-01 00:00
 assets/distribution/extensions/{73a6fe31-595d-460b-a920-fcc0f8843232}.xpi
 │  -rw 2.0 fat   144465 b- defN 10-Jan-01 00:00 META-
 INF/ANDROIDQ.SF
 │  -rw 2.0 fat 1194 b- defN 10-Jan-01 00:00 META-
 INF/ANDROIDQ.RSA
 │  -rw 2.0 fat   144360 b- defN 10-Jan-01 00:00 META-
 INF/MANIFEST.MF
 │  1512 files, 58292874 bytes uncompressed, 44899152 bytes compressed:
 23.0%
 }}}

 I think the cause of this is this line in `projects/tor-
 browser/build.android`:
 {{{
 zip -r -X $apk $ext_dir
 }}}

 Which is not adding the extensions to the .apk in a sorted order.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by eighthave):

 While you are at it, you should also drop ''faketime'' for signing.  The
 Android tools have been zeroing out the dates in the ZIP header for a
 while now, like 2+ years.  If TBA doesn't use a version of the Android
 tools that does that, it is not hard to do it with a tool or with Python
 or something as a post-build, pre-signing process.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by boklm):

 Replying to [comment:7 sisbell]:
 > Technically, you don't need to update to apksigner although there is
 nothing wrong with it.
 >
 > You can override the default manifest Oracle. First create the a
 manifest.txt to use to override entries
 >
 >
 > {{{
 > Created-By: Tor Build
 > }}}
 >
 > Then use the 'm' flag
 >
 >
 > {{{
 > jar -cvfm tor-browser.jar manifest.txt
 > }}}
 >
 > The generated manifest will be
 >
 >
 > {{{
 > Manifest-Version: 1.0
 > Created-By: Tor Build
 > }}}

 The main issue is not that some entries in the manifest are different, but
 that the signing is stalling when we do it with faketime on stretch, which
 is the reason we currently have to do it in a buster container.

 If apksigner does not stall when used with faketime on stretch (or doesn't
 need faketime to produce reproducible output), then that would solve the
 issue.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by sisbell):

 Technically, you don't need to update to apksigner although there is
 nothing wrong with it.

 You can override the default manifest Oracle. First create the a
 manifest.txt to use to override entries


 {{{
 Created-By: Tor Build
 }}}

 Then use the 'm' flag


 {{{
 jar -cvfm tor-browser.jar manifest.txt
 }}}

 The generated manifest will be


 {{{
 Manifest-Version: 1.0
 Created-By: Tor Build
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by boklm):

 I started working on a patch for that, and after comparing a build done
 without containers, and a normal build with containers, I get an almost
 matching apk. The only difference is:
 {{{
 Binary files 1/META-INF/ANDROIDQ.RSA and 2/META-INF/ANDROIDQ.RSA differ
 diff -r 1/META-INF/ANDROIDQ.SF 2/META-INF/ANDROIDQ.SF
 2,4c2,4
 < SHA1-Digest-Manifest-Main-Attributes: nJdsdwNyiHipRN3sUz498qUG+L0=
 < SHA1-Digest-Manifest: vF7lnm0ro+G4diQqON1e3X8+snc=
 < Created-By: 1.8.0_171 (Oracle Corporation)
 ---
 > SHA1-Digest-Manifest-Main-Attributes: E8IbsJM7v8R6d8dy6OqA+g4Q9EE=
 > SHA1-Digest-Manifest: cBYG++7ArsIP93o3h2mGE0OM4WI=
 > Created-By: 1.8.0_212 (Oracle Corporation)
 diff -r 1/META-INF/MANIFEST.MF 2/META-INF/MANIFEST.MF
 2c2
 < Created-By: 1.8.0_171 (Oracle Corporation)
 ---
 > Created-By: 1.8.0_212 (Oracle Corporation)
 }}}

 It seems to be because the `tor-browser` step is done in a buster
 container. The debug signature is also causing the build without
 containers to stall.

 So I think if we find an other workaround for the debug signature that
 doesn't require using buster, we should be able to get matching build
 without and without containers.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by boklm):

 Replying to [comment:1 gk]:
 > Replying to [ticket:29981 boklm]:
 > > By default the Tor Browser build is done inside containers, which we
 run using runc, which require root access.
 > >
 > > In some cases, such as F-Droid builds (#27539), this can be a problem.
 I think we should be able to add an option to do the builds without using
 containers.
 >
 > How would that help the F-Droid case given that F-Droid is taking just
 the signature for an app and applying that one to the built they made
 themselves? I mean the non-container build would then need to match our
 build done in a containerized setup so that F-Droid would get a bit-for-
 bit identical output. Maybe I am too pessimistic here but I'd be surprised
 if that worked out-of-the-box...

 If the F-Droid build VM is using Debian Strech too, then I think the main
 differences between our build environment and the F-Droid build
 environment would be:
 - the username of the build user. In the rbm build we use an `rbm` user,
 but I think we can add an option to change it if that is an issue.
 - the list of packages installed. In our case we build each component with
 only the minimum set of dependencies required to build this component,
 while in the F-Droid case they would always have the dependencies required
 to build all of the components. I think in most cases the additional
 packages should not change the output of the build. Maybe in some cases it
 will requires some changes to avoid the effect of some additional package.

 Unless I'm forgetting an other important difference, I think it should not
 be very difficult to get matching builds. Although I did not try it, so
 maybe I'm too optimistic.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
---+--
 Reporter:  boklm  |  Owner:  tbb-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Applications/Tor Browser   |Version:
 Severity:  Normal | Resolution:
 Keywords:  tbb-rbm, TorBrowserTeam201904  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by gk):

 Replying to [ticket:29981 boklm]:
 > By default the Tor Browser build is done inside containers, which we run
 using runc, which require root access.
 >
 > In some cases, such as F-Droid builds (#27539), this can be a problem. I
 think we should be able to add an option to do the builds without using
 containers.

 How would that help the F-Droid case given that F-Droid is taking just the
 signature for an app and applying that one to the built they made
 themselves? I mean the non-container build would then need to match our
 build done in a containerized setup so that F-Droid would get a bit-for-
 bit identical output. Maybe I am too pessimistic here but I'd be surprised
 if that worked out-of-the-box...

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #29981 [Applications/Tor Browser]: Add option to build without using containers

[Tor Bug Tracker & Wiki]

#29981: Add option to build without using containers
-+-
 Reporter:  boklm|  Owner:  tbb-team
 Type:  task | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Applications/Tor |Version:
  Browser|   Keywords:  tbb-rbm,
 Severity:  Normal   |  TorBrowserTeam201904
Actual Points:   |  Parent ID:
   Points:   |   Reviewer:
  Sponsor:   |
-+-
 By default the Tor Browser build is done inside containers, which we run
 using runc, which require root access.

 In some cases, such as F-Droid builds (#27539), this can be a problem. I
 think we should be able to add an option to do the builds without using
 containers.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs