I fully share this point of view: having opam in control of the sandbox provides much finer grain. So, from a functional standpoint, I would not hesitate.
What makes me hesitate is that we would need to change the opam code for this, and take into account all possible different mechanisms according to os/platform. This is pretty well doable, if we set up a proper plugin mechanism, and we could share code with a plugin-based depext support, but it requires more effort :-) 2015-02-25 11:54 GMT+01:00 Anil Madhavapeddy <[email protected]>: > I'm also not sure how viable activating it for day-to-day testing except > by advanced users is. However, we can at least get the infrastructure in > place for CI, which is necessary before we need to decide that. > > I forgot one point in my previous mail: why this needs to be in OPAM > instead > of just an external `sandbox` binary. We don't actually want to sandbox > OPAM itself (which we trust!), but rather the packages it invokes. In my > OpenBSD sandbox, I am constraining OPAM and all its future subshells. > Instead, I would prefer to have OPAM invoke an external sandbox only when > it is doing a build, with sufficient arguments to let it know which dirs > are ok to write into. > > This would let a package only write into the specific subdirectory relevant > to the current switch. This actually bites me quite often with pinning, > since a stale OASIS setup.data can cause an installation to write into the > wrong switch if a full clean is not done ahead of time. It also lets us > sandbox network access of builds independently of OPAM, which is needed to > "unplug" an OPAM package build. > > cheers, > Anil > > > On 25 Feb 2015, at 10:48, Roberto Di Cosmo <[email protected]> wrote: > > > > Many thanks Anil, that's precious information: > > this is clearly another very OS-specific feature! > > > > The question is whether we want to add these policies and tools > > > > - outside opam (pro: no changes to opam, cons: opam has no control) > > - or make opam aware of them (pro: opam gets control, cons: various > changes to opam) > > > > For CI testing, the "outside" approach seems to be quite sufficient. > > > > For day to day use, though, I am not sure... > > > > On Tue, Feb 24, 2015 at 04:22:31PM +0000, Anil Madhavapeddy wrote: > >> Hi Roberto, Simon, > >> > >> Sandboxing mechanisms have come along quite a bit in the last few years. > >> It's important to separate the two threat models that we want, since > their > >> use can be quite intrusive if made mandatory. I see two modes of > operation: > >> > >> - A mandatory sandbox in CI testing, where we use it to check that the > >> package isn't violating obvious policies such as network downloads > >> from within the package, or writing outside of ~/.opam or /tmp. > >> > >> - Optional sandbox for day-to-day use by end users. This will catch > >> even malicious behaviours, but imposes a rather heavy support burden. > >> > >> In terms of how to sandbox applications, I know of: > >> > >> - OpenBSD: I use the built-in systrace system call permissions subsystem > >> already with OPAM. I've uploaded my local policy to GitHub at: > >> https://gist.github.com/avsm/8293aa52c6cee772a9cb > >> This policy is used by "systrace opam install foo" and pulls up an > >> interactive dialog if an application attempts to write outside of > >> either ~/.opam or /tmp. > >> See: http://www.openbsd.org/cgi-bin/man.cgi?query=systrace > >> or: http://www.citi.umich.edu/u/provos/systrace/ > >> paper: http://www.citi.umich.edu/u/provos/papers/systrace.pdf > >> > >> - FreeBSD: the latest versions have the Capsicum capability system > >> integrated, and the Casper system daemon that grants common operations > >> is also being worked on upstream (e.g. for name resolution) > >> See: https://www.cl.cam.ac.uk/research/security/capsicum/ > >> > >> - Linux: As always with Linux, there are a myriad of possible solutions. > >> I'd discourage the use of LD_PRELOAD based solutions since they don't > >> work in several situations reliably (most obviously with static > binaries). > >> The fakeroot-ng project uses ptrace instead, which is more reliable. > >> > >> However, the primary thing we want is to only let the package only > >> write into ~/.opam, and so the mount namespace feature (see > CLONE_NEWNS) > >> may be sufficient for our needs -- a lightweight filesystem container, > >> in effect. David Sheets has done some investigation into this for > >> another project we're working on. > >> > >> - Windows: various, Sandboxie is one solution, but the underlying > >> mechanism is the Windows Integrity Mechanism that was introduced in > >> Vista: https://msdn.microsoft.com/en-us/library/bb625964.aspx > >> This lets applications drop privileges, and is used by the Chrome > >> sandbox (in general, following what Chrome does for sandboxing is a > >> good idea, since their needs are a superset of ours). > >> > >> - MacOS X: The App Sandbox requires code signing, but does almost > >> exactly what we need: it gives each app/user a unique directory > >> into which they can write files. All we should need to do is to > >> set OPAMROOT to that directory, and everything should just work. > >> In practise, this requires some investigation into how the App > >> signing infrastructure works, since I've only seen it done for > >> bundles and not for CLI tools. > >> > >> This is a quick off-the-top-of-my-head survey, but I think it's viable > >> and useful for us to build an `opam sandbox` in the same style as > >> `opam depext` that attempts to invoke a relevant sandboxing mechanism > >> depending on the OS. In the longer term, this will really serve us > >> well as the package database grows. > >> > >> I'm less sure about the viability of recording installed files > >> strictly -- I view thatas an advisory rather than enforcement-based > >> mechanism. The reason I like the "make ~/.opam a git store" is that > >> its possible for applications to write directly into the store as they > >> do right now, but still let us track changes precisely. In fact, if > >> we forbid subshells from writing into `~/.opam/.git`, this would be > >> a production grade solution that also offers instant-rollback in case > >> of compilation errors (no more waiting for a full recompilation of > >> the original dependencies!). > >> > >> cheers, > >> Anil > >> > >>> On 21 Feb 2015, at 09:16, Roberto Di Cosmo <[email protected]> > wrote: > >>> > >>> Anil, Simon, can you provide more details on the sandboxing mechanisms > you know of? > >>> > >>> We looked into all this for Mancoosi years ago; the most complete tool > >>> out there was installwatch (now checkinstall) that hijacks filesystem > modifying > >>> commands using the standard LD_PRELOAD trick and a wrapper for system > calls. > >>> Checkinstall does not alter user priviledges, though, so one sometimes > needed > >>> a combination of fakeroot (that only alter user priviledges) with it. > >>> > >>> The best approach I know of was described in a Master thesis from ... > Cambridge > >>> :-) It was under the supervision of Peter Sewell, and used the ptrace > mechanism > >>> instead of the LD_PRELOAD trick, because LD_PRELOAD is blind to > statically > >>> compiled binaries that have system calls hardcoded, while ptrace gets > them all. > >>> > >>> The dissertation is still available today here > http://robot101.net/files/diss.ps.gz > >>> and contains a very nice discussion of the issues related to > monitoring and > >>> rolling back file system changes performed by a command in the Linux > system. > >>> The source code is also available here > http://robot101.net/files/trace.tar.gz > >>> and one can get in touch with Robert Mcqueen that will be delighted to > see his > >>> work being used. > >>> > >>> Since all this is almost 10 years old, I suppose many exciting new > ideas, tools > >>> and approaches surfaced in the meantime, and I'd really like to know > more. > >>> > >>> Cheers > >>> > >>> -- > >>> Roberto > >>> > >>> On Sat, Feb 21, 2015 at 09:37:07AM +0100, Simon Cruanes wrote: > >>>> Sandboxing the build would also be a big security improvement. I > think cabal > >>>> now does it, and signing packages doesn't protect against malicious > or buggy > >>>> packages (see: bumblebee's uninstall target). That also goes hand in > hand with > >>>> file tracking. I don't know how difficult it is, though. > >>>> > >>>> Cheers! > >>>> > >>>> Le 21 février 2015 05:01:56 UTC+01:00, Louis Gesbert > >>>> <[email protected]> a écrit : > >>>> > >>>> With 1.2.1 almost out of the door, time has come to review the > roadmap discussed back in December and choose where we'll be going for 1.3. > Original mail attached for reference. > >>>> > >>>> > >>>> The topic that is burning hot at the moment is, specially after the > Debian Haskell build host has been compromised, security: we have no > signing at all at the moment, and we need to improve on this before it > becomes a problem. TUF [1] has devised a sane amount of rules for > repository management and signing that should make it easier to get it > right in OPAM. Hannes has mentionned writing an OCaml implementation for > TUF, which could get very helpful. > >>>> > >>>> > >>>> Also of importance is Windows support. It should at least be > straighforward and documented to get a basic Cygwin setup working. That > goes with adding automated tests (appveyor can now be added in Github > alongside Travis). Related is cleaning up external command usage (even if > not really justified by a Windows > >>>> port only, as David Allsopp pointed out) -- replacing curl calls by > cohttp, use ocaml-fileutils... > >>>> > >>>> > >>>> These are the other main features, that'll probably take more time > if we are to focus eg. on security: > >>>> > >>>> * a plugin mechanism with plugins for example for OCaml (for better > agnosticity), external dependency handling [2], documentation generation... > >>>> > >>>> * a 'provides:' field in OPAM files [3]. This is a loose > requirement if we want to switch the repository to have OCaml itself in a > package (which is already possible, but the system compiler may yet be an > issue). > >>>> > >>>> * More flexible switch handling (multi-switch packages, per-switch > remotes, layered switches for cross-compilation...) > >>>> > >>>> * Tracking of files installed by packages. While unrelated to repo > signing, this might have some security implications, so we might want to > bring it in. > >>>> > >>>> * With file tracking, generating a binary repo (with some > limitations) could be quite > >>>> straight-forward. > >>>> > >>>> > >>>> Which of these do you think is most important ? Have I forgotten > anything ? > >>>> > >>>> Cheers, > >>>> Louis > >>>> > >>>> > >>>> [1] http://theupdateframework.com/ > >>>> [2] > https://github.com/ocaml/opam/blob/master/doc/design/depexts-plugins > >>>> [3] > https://github.com/ocaml/opam/blob/master/doc/design/provides.md > >>>> > >>>> message suivi > >>>> > >>>> De : Louis Gesbert > >>>> À : [email protected] > >>>> Envoyé : Wed Dec 17 11:07:40 UTC+01:00 2014 > >>>> Objet : [opam-devel] OPAM Roadmap -- what next ? > >>>> > >>>> Hi all, > >>>> > >>>> > >>>> > >>>> with some lag after the 1.2 release, I'd like to open some space > for a > >>>> collective discussion of the priorities for where the energies > should go > >>>> next. We have mainly 3 directions for improvements: first, > portability, > >>>> with the main goal of a Windows version. Second, agnosticity, with > the goal > >>>> to make OPAM more generic, and try and open it to users outside of > the > >>>> OCaml community (wouldn't OPAM for Haskell be fun ?). Third, there > are > >>>> always lots of features and improvements that have been asked for, > and > >>>> would improve the experience of current users. > >>>> > >>>> > >>>> > >>>> So here is a summary of what I've gathered. Feel free to add your > own. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> ## Portability > >>>> > >>>> > >>>> > >>>> - **Rewrite parallel command engine.** / done. > >>>> > >>>> > >>>> > >>>> - **Native system manipulation (cp, rm, curl...).** > >>>> > >>>> These are mostly done through calls to the shell or external > programs. It's > >>>> > >>>> not very pretty but quite pragmatic actually... until we want to > run on > >>>> > >>>> Windows without Cygwin. Anyway, this wouldn't be the end of > portability > >>>> > >>>> issues. > >>>> > >>>> > >>>> > >>>> - **Windows support.** > >>>> > >>>> for OPAM itself to begin with. This should be pretty close. > >>>> > >>>> > >>>> > >>>> - **Packages on Windows.** > >>>> > >>>> Locate common issues and attempt to find generic fixes. > >>>> > >>>> > >>>> > >>>> - Allow **direct use of more solvers** or solver servers. > >>>> > >>>> > >>>> > >>>> - **Support cross-compilation** > >>>> > >>>> This is quite an open issue at the moment. > >>>> > >>>> > >>>> > >>>> ## Agnosticity > >>>> > >>>> > >>>> > >>>> - **Isolate OCaml-specific stuff.** > >>>> > >>>> E.g. specific opam-file variables. See ocaml-specific.md > >>>> > >>>> > >>>> > >>>> - **Separate as plugins** > >>>> > >>>> To open the gate to OPAM without these plugins, or with other ones > >>>> > >>>> > >>>> > >>>> - **Compilers as packages.** > >>>> > >>>> This should mostly work now, but needs some tests and > strengthening. The > >>>> main > >>>> > >>>> thing still to do is to handle the system compiler changes properly > ; that > >>>> > >>>> part should be made more generic anyway (see discussion on hooks) > >>>> > >>>> > >>>> > >>>> ## Features > >>>> > >>>> > >>>> > >>>> - A **provides** field. Generally useful, but particulary so with > >>>> > >>>> compilers-as-packages > >>>> > >>>> > >>>> > >>>> - Releasing the **"features" field** for easier package > configuration > >>>> > >>>> > >>>> > >>>> - **Track installed files** > >>>> > >>>> > >>>> > >>>> - **Improve security**: just checking md5s is quite light ; package > scripts > >>>> can > >>>> > >>>> write anywhere > >>>> > >>>> > >>>> > >>>> - **OS-specific handling of dependencies** (eg dependencies on > packages > >>>> from the > >>>> > >>>> OS) with plugins (#1519) > >>>> > >>>> > >>>> > >>>> - Specify and implement **hooks** > >>>> > >>>> > >>>> > >>>> - Find a nicer way to **share dev repos** / undoable "pinning > sources" > >>>> > >>>> > >>>> > >>>> - **Per-switch remotes** > >>>> > >>>> > >>>> > >>>> - **Multi-switch packages** > >>>> > >>>> > >>>> > >>>> - Support for (automatic generation of) **binary packages** > >>>> > >>>> > >>>> > >>>> - Nicer **ocamlfind interaction** > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> Cheers, > >>>> > >>>> Louis Gesbert > >>>> > >>>> > ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > >>>> > >>>> opam-devel mailing list > >>>> [email protected] > >>>> http://lists.ocaml.org/listinfo/opam-devel > >>>> > >>>> > ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > >>>> > >>>> opam-devel mailing list > >>>> [email protected] > >>>> http://lists.ocaml.org/listinfo/opam-devel > >>>> > >>>> > >>>> -- > >>>> Simon > >>> > >>>> _______________________________________________ > >>>> opam-devel mailing list > >>>> [email protected] > >>>> http://lists.ocaml.org/listinfo/opam-devel > >>> > >>> > >>> -- > >>> Roberto Di Cosmo > >>> > >>> ------------------------------------------------------------------ > >>> Professeur En delegation a l'INRIA > >>> PPS E-mail: [email protected] > >>> Universite Paris Diderot WWW : http://www.dicosmo.org > >>> Case 7014 Tel : ++33-(0)1-57 27 92 20 > >>> 5, Rue Thomas Mann > >>> F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo > >>> FRANCE. Twitter: http://twitter.com/rdicosmo > >>> ------------------------------------------------------------------ > >>> Attachments: > >>> MIME accepted, Word deprecated > >>> http://www.gnu.org/philosophy/no-word-attachments.html > >>> ------------------------------------------------------------------ > >>> Office location: > >>> > >>> Bureau 3020 (3rd floor) > >>> Batiment Sophie Germain > >>> Avenue de France > >>> Metro Bibliotheque Francois Mitterrand, ligne 14/RER C > >>> ----------------------------------------------------------------- > >>> GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3 > >>> > >> > > > > -- > > Roberto Di Cosmo > > > > ------------------------------------------------------------------ > > Professeur En delegation a l'INRIA > > PPS E-mail: [email protected] > > Universite Paris Diderot WWW : http://www.dicosmo.org > > Case 7014 Tel : ++33-(0)1-57 27 92 20 > > 5, Rue Thomas Mann > > F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo > > FRANCE. Twitter: http://twitter.com/rdicosmo > > ------------------------------------------------------------------ > > Attachments: > > MIME accepted, Word deprecated > > http://www.gnu.org/philosophy/no-word-attachments.html > > ------------------------------------------------------------------ > > Office location: > > > > Bureau 3020 (3rd floor) > > Batiment Sophie Germain > > Avenue de France > > Metro Bibliotheque Francois Mitterrand, ligne 14/RER C > > ----------------------------------------------------------------- > > GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3 > > > > -- Roberto Di Cosmo ------------------------------------------------------------------ Professeur En delegation a l'INRIA PPS E-mail: [email protected] Universite Paris Diderot WWW : http://www.dicosmo.org Case 7014 Tel : ++33-(0)1-57 27 92 20 5, Rue Thomas Mann F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo FRANCE. Twitter: http://twitter.com/rdicosmo ------------------------------------------------------------------ Attachments: MIME accepted, Word deprecated http://www.gnu.org/philosophy/no-word-attachments.html ------------------------------------------------------------------ Office location: Bureau 320 (3rd floor) Batiment Sophie Germain Avenue de France Metro Bibliotheque Francois Mitterrand, ligne 14/RER C ----------------------------------------------------------------- GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3
_______________________________________________ opam-devel mailing list [email protected] http://lists.ocaml.org/listinfo/opam-devel
