I guess by now most people have seen lennarts latest post on how to build linux systems:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html I don't necessarily agree to the use of btrfs[1] for the images, but otherwise I think this is generally the right approach for desktop apps. However, in order to actually be useful for desktop apps there is a lot more work required. I've recently started looking at what Gnome needs to do here, and wanted to start a discussion about this. So, lets start with some definitons: A "host" is a linux-based operating system that you wish to run an app on. Hosts can be of many forms, but they are typically a normal linux desktop distributions that meet some minimal requirements. An "app" is a set of files (including binaries) + metadata, which are designed to be combined with a particular runtime when executed. In lennarts proposal this would be a btrfs volume, although for purposes of the discussion here it could be done any way that lets the kernel see the files. These files would be available somewhere in /opt when the app is running. The metadata of the app specifies the exact runtime version the app needs, and any requirements that are not in the runtime must be bundled with the app. A "runtime" is an environment that an app runs against. The runtime provides a set of files that will appear as /usr when the app runs, including things like libraries, executables and resources. Each runtime gives a well defined set of guarantees on the files in the runtime. For instance, a typical "stable" runtime would give very hard ABI guarantees for libraries it contains, including things like .so versions, build prefix, libexecdir, enabled configuration options, etc. Its ok for a runtime to get minor changes like fixes security issues or bugs, but in general it should be very conservative with updates. A runtime can also give guarantees about the behaviour of the host system. For instance, a runtime that ships a client library that expects to talk to a service imply that the corresponding service is available in the user session, accessed via some form of IPC. It would also guarante certain ABI stability of said IPC, keeping the service backwards compatible with the client library for as long as the host claims to be able to supply a given runtime. This is important as we want to be able to keep old apps/runtimes working while upgrading the host. So, what does Gnome need to provide here? 1. A platform definition This includes the set of libraries, supported IPC APIs, icons and other resources, etc. These would be the libraries that application developers can be allowed to rely on without bundling. We also need a very explicit definition of how to build them, so that anyone can rebuild the software and get a compatible build. For instance, if a distribution wants to ship its own build of the gnome platform (and they will probably want to) the platform definition should allow them to build and maintain a compatible version. 2. A reference implementation We need to ship a binary build of the platform that app developers can test against, and that anyone can download if their distribution doesn't already have it. 3. A SDK and other tooling There needs to be an easy way to build an application against the gnome runtime. This means we need to ship a SDK with all the development stuff for the runtime (headers, symbols, debug info), as well as tooling to make it easy to use those without risking any contamination from the host. We also want to have tooling to make it easier to bundle libraries, as we don't want every app author to have to package everything from scratch. 4. IPC stability guarantees In theory we can put any kind of library in the runtime, as we rely on the global version of the runtime itself to keep applications running if API or ABI change. (Although we should of course try to minimize such breaks.) However, any kind of IPC mechanism we include in the runtime must have very strict backwards compatible requirements. Historically we have not done very well here. For instance, the GVfs dbus interfaces have generally been tightly coupled with the client libraries, such that you have to update both in lockstep. With an "app" based setup we have to take backwards compatibility much more seriously, across the board. This is not a code delivery, but rather a change in mentality that we as a project have to accept. 5. Sandboxing APIs In a sandboxed environment app code doesn't have access to most of the host system. However, apps still need some ways to securely access various services (like users files, hw, host services, etc). We need to define these APIs, and whatever security layer protects against their misuse. In practice this will require the definition of dbus APIs for a lot of operations that are currently done in-process. It will require a lot of work figuring out what applications need and how to securely provide it in a way that doesn't get in the way of the user. Alan has been doing some design work here which is a good place to start: http://afaikblog.wordpress.com/2014/07/10/sandboxed-applications-for-gnome/ This is one of the core goals of this endevour, however it is not necessary the most important initially. Just having a way to package and run apps (insecurely) within a stable runtime is interesting in itself. This is particularly important as it lets us get towards the sandboxed model in steps, by building the current applications unchanged as "non-sandboxed apps" and then from there start making changes to the apps themselves to play well in a sandboxed environment. Also it lets us kickstart work until the requirements for sandboxing are more ready (things like kdbus and wayland are really needed for secure sandboxing). I've recently started looking at how 1-3 could be done. I've experimented with openembedded which is a very nice way to do cross compilation that also lets you create SDK. However, some of stuff, in particular gobject-introspection is very tricky to do right when cross compiling. I've now changed over to the mixed model that gnome-continuos is using where openembedded is used to build a base containing the build environment, and the rest is built in a chroot. Hopefully I will have some initial cut of a runtime based on gnome-continuos some time next week. This will give us a runtime, but also a lot of stuff that can be used to make the SDK. We need to start looking into exactly what the runtime should include though. Obviously it should contain glib, gtk+ and the gnome platform, but we also need to pick the lower level dependencies (glibc, X, etc) as well as some level of standard unix tools. For instance: * Do we ship a shell interpreter so that shell scripts work? * What standard unix tools do we ship? cp? ps? ship? awk? sed? * What about language runtimes? gjs? pyton2? python3? perl5? * Some of our development stack is of lower quality than the core platform. How much of it do we want to ship? On one hand we can put whatever in the runtime and have the runtime versioning protect against stability issues, on the other hand we'll may be giving application developers false hope wrt the quality and stability of some code, as well as setting us up for long-term maintainance and security fixes for all those libraries. * What about things like plugins like gstreamer? We'll ship some in the runtime, and if the app needs a particular new one it can bundle it. But it would also be nice if the user can side-load things into a specific runtime. Maybe we need extra local layers on top of the runtime? There are also lots of technical issues. For instance how do we ship mesa and DRI drivers which have dependencies on the kernel part of the driver? How do we support e.g. the nvidia drivers in this? Do we need some kind of overlay for the runtime image for local hardware drivers? Well, this was a lot of text from my side with nothing practicaly useful yet. I think its good to get people started thinking about these things though, and I'll keep people updated as I make progress actually building something. [1] I think using btrfs send/recieve to upgrade apps is a nice idea but not really all that important. You need all sorts of contractions to produce such a diff if the new version is built from scratch, and its unlikely that a simple "tar up the changed files" approach (which is much simpler) would generate a substantially larger diff. _______________________________________________ gnome-os-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/gnome-os-list
