Many thanks Anil, that's precious information: this is clearly another very OS-specific feature!
The question is whether we want to add these policies and tools - outside opam (pro: no changes to opam, cons: opam has no control) - or make opam aware of them (pro: opam gets control, cons: various changes to opam) For CI testing, the "outside" approach seems to be quite sufficient. For day to day use, though, I am not sure... On Tue, Feb 24, 2015 at 04:22:31PM +0000, Anil Madhavapeddy wrote: > Hi Roberto, Simon, > > Sandboxing mechanisms have come along quite a bit in the last few years. > It's important to separate the two threat models that we want, since their > use can be quite intrusive if made mandatory. I see two modes of operation: > > - A mandatory sandbox in CI testing, where we use it to check that the > package isn't violating obvious policies such as network downloads > from within the package, or writing outside of ~/.opam or /tmp. > > - Optional sandbox for day-to-day use by end users. This will catch > even malicious behaviours, but imposes a rather heavy support burden. > > In terms of how to sandbox applications, I know of: > > - OpenBSD: I use the built-in systrace system call permissions subsystem > already with OPAM. I've uploaded my local policy to GitHub at: > https://gist.github.com/avsm/8293aa52c6cee772a9cb > This policy is used by "systrace opam install foo" and pulls up an > interactive dialog if an application attempts to write outside of > either ~/.opam or /tmp. > See: http://www.openbsd.org/cgi-bin/man.cgi?query=systrace > or: http://www.citi.umich.edu/u/provos/systrace/ > paper: http://www.citi.umich.edu/u/provos/papers/systrace.pdf > > - FreeBSD: the latest versions have the Capsicum capability system > integrated, and the Casper system daemon that grants common operations > is also being worked on upstream (e.g. for name resolution) > See: https://www.cl.cam.ac.uk/research/security/capsicum/ > > - Linux: As always with Linux, there are a myriad of possible solutions. > I'd discourage the use of LD_PRELOAD based solutions since they don't > work in several situations reliably (most obviously with static binaries). > The fakeroot-ng project uses ptrace instead, which is more reliable. > > However, the primary thing we want is to only let the package only > write into ~/.opam, and so the mount namespace feature (see CLONE_NEWNS) > may be sufficient for our needs -- a lightweight filesystem container, > in effect. David Sheets has done some investigation into this for > another project we're working on. > > - Windows: various, Sandboxie is one solution, but the underlying > mechanism is the Windows Integrity Mechanism that was introduced in > Vista: https://msdn.microsoft.com/en-us/library/bb625964.aspx > This lets applications drop privileges, and is used by the Chrome > sandbox (in general, following what Chrome does for sandboxing is a > good idea, since their needs are a superset of ours). > > - MacOS X: The App Sandbox requires code signing, but does almost > exactly what we need: it gives each app/user a unique directory > into which they can write files. All we should need to do is to > set OPAMROOT to that directory, and everything should just work. > In practise, this requires some investigation into how the App > signing infrastructure works, since I've only seen it done for > bundles and not for CLI tools. > > This is a quick off-the-top-of-my-head survey, but I think it's viable > and useful for us to build an `opam sandbox` in the same style as > `opam depext` that attempts to invoke a relevant sandboxing mechanism > depending on the OS. In the longer term, this will really serve us > well as the package database grows. > > I'm less sure about the viability of recording installed files > strictly -- I view thatas an advisory rather than enforcement-based > mechanism. The reason I like the "make ~/.opam a git store" is that > its possible for applications to write directly into the store as they > do right now, but still let us track changes precisely. In fact, if > we forbid subshells from writing into `~/.opam/.git`, this would be > a production grade solution that also offers instant-rollback in case > of compilation errors (no more waiting for a full recompilation of > the original dependencies!). > > cheers, > Anil > > > On 21 Feb 2015, at 09:16, Roberto Di Cosmo <[email protected]> wrote: > > > > Anil, Simon, can you provide more details on the sandboxing mechanisms you > > know of? > > > > We looked into all this for Mancoosi years ago; the most complete tool > > out there was installwatch (now checkinstall) that hijacks filesystem > > modifying > > commands using the standard LD_PRELOAD trick and a wrapper for system calls. > > Checkinstall does not alter user priviledges, though, so one sometimes > > needed > > a combination of fakeroot (that only alter user priviledges) with it. > > > > The best approach I know of was described in a Master thesis from ... > > Cambridge > > :-) It was under the supervision of Peter Sewell, and used the ptrace > > mechanism > > instead of the LD_PRELOAD trick, because LD_PRELOAD is blind to statically > > compiled binaries that have system calls hardcoded, while ptrace gets them > > all. > > > > The dissertation is still available today here > > http://robot101.net/files/diss.ps.gz > > and contains a very nice discussion of the issues related to monitoring and > > rolling back file system changes performed by a command in the Linux system. > > The source code is also available here > > http://robot101.net/files/trace.tar.gz > > and one can get in touch with Robert Mcqueen that will be delighted to see > > his > > work being used. > > > > Since all this is almost 10 years old, I suppose many exciting new ideas, > > tools > > and approaches surfaced in the meantime, and I'd really like to know more. > > > > Cheers > > > > -- > > Roberto > > > > On Sat, Feb 21, 2015 at 09:37:07AM +0100, Simon Cruanes wrote: > >> Sandboxing the build would also be a big security improvement. I think > >> cabal > >> now does it, and signing packages doesn't protect against malicious or > >> buggy > >> packages (see: bumblebee's uninstall target). That also goes hand in hand > >> with > >> file tracking. I don't know how difficult it is, though. > >> > >> Cheers! > >> > >> Le 21 février 2015 05:01:56 UTC+01:00, Louis Gesbert > >> <[email protected]> a écrit : > >> > >> With 1.2.1 almost out of the door, time has come to review the roadmap > >> discussed back in December and choose where we'll be going for 1.3. > >> Original mail attached for reference. > >> > >> > >> The topic that is burning hot at the moment is, specially after the > >> Debian Haskell build host has been compromised, security: we have no > >> signing at all at the moment, and we need to improve on this before it > >> becomes a problem. TUF [1] has devised a sane amount of rules for > >> repository management and signing that should make it easier to get it > >> right in OPAM. Hannes has mentionned writing an OCaml implementation for > >> TUF, which could get very helpful. > >> > >> > >> Also of importance is Windows support. It should at least be > >> straighforward and documented to get a basic Cygwin setup working. That > >> goes with adding automated tests (appveyor can now be added in Github > >> alongside Travis). Related is cleaning up external command usage (even if > >> not really justified by a Windows > >> port only, as David Allsopp pointed out) -- replacing curl calls by > >> cohttp, use ocaml-fileutils... > >> > >> > >> These are the other main features, that'll probably take more time if > >> we are to focus eg. on security: > >> > >> * a plugin mechanism with plugins for example for OCaml (for better > >> agnosticity), external dependency handling [2], documentation generation... > >> > >> * a 'provides:' field in OPAM files [3]. This is a loose requirement if > >> we want to switch the repository to have OCaml itself in a package (which > >> is already possible, but the system compiler may yet be an issue). > >> > >> * More flexible switch handling (multi-switch packages, per-switch > >> remotes, layered switches for cross-compilation...) > >> > >> * Tracking of files installed by packages. While unrelated to repo > >> signing, this might have some security implications, so we might want to > >> bring it in. > >> > >> * With file tracking, generating a binary repo (with some limitations) > >> could be quite > >> straight-forward. > >> > >> > >> Which of these do you think is most important ? Have I forgotten > >> anything ? > >> > >> Cheers, > >> Louis > >> > >> > >> [1] http://theupdateframework.com/ > >> [2] https://github.com/ocaml/opam/blob/master/doc/design/depexts-plugins > >> [3] https://github.com/ocaml/opam/blob/master/doc/design/provides.md > >> > >> message suivi > >> > >> De : Louis Gesbert > >> À : [email protected] > >> Envoyé : Wed Dec 17 11:07:40 UTC+01:00 2014 > >> Objet : [opam-devel] OPAM Roadmap -- what next ? > >> > >> Hi all, > >> > >> > >> > >> with some lag after the 1.2 release, I'd like to open some space for a > >> collective discussion of the priorities for where the energies should go > >> next. We have mainly 3 directions for improvements: first, portability, > >> with the main goal of a Windows version. Second, agnosticity, with the > >> goal > >> to make OPAM more generic, and try and open it to users outside of the > >> OCaml community (wouldn't OPAM for Haskell be fun ?). Third, there are > >> always lots of features and improvements that have been asked for, and > >> would improve the experience of current users. > >> > >> > >> > >> So here is a summary of what I've gathered. Feel free to add your own. > >> > >> > >> > >> > >> > >> > >> > >> ## Portability > >> > >> > >> > >> - **Rewrite parallel command engine.** / done. > >> > >> > >> > >> - **Native system manipulation (cp, rm, curl...).** > >> > >> These are mostly done through calls to the shell or external programs. > >> It's > >> > >> not very pretty but quite pragmatic actually... until we want to run on > >> > >> Windows without Cygwin. Anyway, this wouldn't be the end of portability > >> > >> issues. > >> > >> > >> > >> - **Windows support.** > >> > >> for OPAM itself to begin with. This should be pretty close. > >> > >> > >> > >> - **Packages on Windows.** > >> > >> Locate common issues and attempt to find generic fixes. > >> > >> > >> > >> - Allow **direct use of more solvers** or solver servers. > >> > >> > >> > >> - **Support cross-compilation** > >> > >> This is quite an open issue at the moment. > >> > >> > >> > >> ## Agnosticity > >> > >> > >> > >> - **Isolate OCaml-specific stuff.** > >> > >> E.g. specific opam-file variables. See ocaml-specific.md > >> > >> > >> > >> - **Separate as plugins** > >> > >> To open the gate to OPAM without these plugins, or with other ones > >> > >> > >> > >> - **Compilers as packages.** > >> > >> This should mostly work now, but needs some tests and strengthening. The > >> main > >> > >> thing still to do is to handle the system compiler changes properly ; > >> that > >> > >> part should be made more generic anyway (see discussion on hooks) > >> > >> > >> > >> ## Features > >> > >> > >> > >> - A **provides** field. Generally useful, but particulary so with > >> > >> compilers-as-packages > >> > >> > >> > >> - Releasing the **"features" field** for easier package configuration > >> > >> > >> > >> - **Track installed files** > >> > >> > >> > >> - **Improve security**: just checking md5s is quite light ; package > >> scripts > >> can > >> > >> write anywhere > >> > >> > >> > >> - **OS-specific handling of dependencies** (eg dependencies on packages > >> from the > >> > >> OS) with plugins (#1519) > >> > >> > >> > >> - Specify and implement **hooks** > >> > >> > >> > >> - Find a nicer way to **share dev repos** / undoable "pinning sources" > >> > >> > >> > >> - **Per-switch remotes** > >> > >> > >> > >> - **Multi-switch packages** > >> > >> > >> > >> - Support for (automatic generation of) **binary packages** > >> > >> > >> > >> - Nicer **ocamlfind interaction** > >> > >> > >> > >> > >> > >> > >> > >> Cheers, > >> > >> Louis Gesbert > >> > >> > >> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > >> > >> opam-devel mailing list > >> [email protected] > >> http://lists.ocaml.org/listinfo/opam-devel > >> > >> > >> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > >> > >> opam-devel mailing list > >> [email protected] > >> http://lists.ocaml.org/listinfo/opam-devel > >> > >> > >> -- > >> Simon > > > >> _______________________________________________ > >> opam-devel mailing list > >> [email protected] > >> http://lists.ocaml.org/listinfo/opam-devel > > > > > > -- > > Roberto Di Cosmo > > > > ------------------------------------------------------------------ > > Professeur En delegation a l'INRIA > > PPS E-mail: [email protected] > > Universite Paris Diderot WWW : http://www.dicosmo.org > > Case 7014 Tel : ++33-(0)1-57 27 92 20 > > 5, Rue Thomas Mann > > F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo > > FRANCE. Twitter: http://twitter.com/rdicosmo > > ------------------------------------------------------------------ > > Attachments: > > MIME accepted, Word deprecated > > http://www.gnu.org/philosophy/no-word-attachments.html > > ------------------------------------------------------------------ > > Office location: > > > > Bureau 3020 (3rd floor) > > Batiment Sophie Germain > > Avenue de France > > Metro Bibliotheque Francois Mitterrand, ligne 14/RER C > > ----------------------------------------------------------------- > > GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3 > > > > > -- Roberto Di Cosmo ------------------------------------------------------------------ Professeur En delegation a l'INRIA PPS E-mail: [email protected] Universite Paris Diderot WWW : http://www.dicosmo.org Case 7014 Tel : ++33-(0)1-57 27 92 20 5, Rue Thomas Mann F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo FRANCE. Twitter: http://twitter.com/rdicosmo ------------------------------------------------------------------ Attachments: MIME accepted, Word deprecated http://www.gnu.org/philosophy/no-word-attachments.html ------------------------------------------------------------------ Office location: Bureau 3020 (3rd floor) Batiment Sophie Germain Avenue de France Metro Bibliotheque Francois Mitterrand, ligne 14/RER C ----------------------------------------------------------------- GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3 _______________________________________________ opam-devel mailing list [email protected] http://lists.ocaml.org/listinfo/opam-devel
