That's right, only one task can run off of a given vm_cache, and hence only one boinc2docker task per host. This is definitely a limitation of how I'm building this at the moment, but unfortunately I see no other clean way aside from requiring the user to have a true Docker install on their system (which maybe is something BOINC wants to suggest in the future like it does now with VBox, but for now I think its outside the scope of what I'm doing).
I think I probably didn't explain the setup well enough though. The vm_cache.vdi delivered to a fresh host is just a completely empty VDI. This gets mounted and when the job gets run it will download some Docker images and store them here, so after the first run vm_cache is no longer empty. Subsequent runs then dont have to redownload these images. So there's no issue making sure "vm_cache is not modified". Hope that helps. If there are better ways though I'm very open to suggestions. Marius On Wed, Jul 1, 2015 at 2:58 PM, Rom Walton <[email protected]> wrote: > I’m not sure you would want to persist the differencing disk. If all > the disk activity for what vm_cache points too is read-only, the > differencing disk will be very small. In your environment I see it as an > insurance policy to make sure vm_cache is not modified. > > > > In the normal BOINC environment, it allows several tasks to run off the > same vm_cache image. If it is marked mutable, only one task would > successfully be able to mount the image. The other tasks would fail with a > file locking error of some sort. > > > > ----- Rom > > > > *From:* Marius Millea [mailto:[email protected]] > *Sent:* Wednesday, July 01, 2015 5:31 PM > > *To:* Rom Walton <[email protected]> > *Cc:* [email protected] > *Subject:* Re: [boinc_dev] is there a way to persist files on the host > > > > The only issue with this solution seems to be that now you have vm_cache > on the host different that the "empty" vm_cache on the server. On my > combination of server/client version I'm testing with this hasn't been a > problem, the empty one is never redownloaded or anything wiping out the > persistence. > > > > Marius > > > > > > On Wed, Jul 1, 2015 at 2:27 PM, Marius Millea <[email protected]> wrote: > > Hi Rom, > > > > So if I understand correctly, with vm_cache.vdi marked immutable, VBox > makes a differencing VDI. To persist this, we would need to pick this file > up and copy it to projects/, then on the next run load *it* up instead of > vm_cache? If so this seems unnecessarily complicated, why not just have > vm_cache.vdi be mutable and simply attach it to the VM straight from the > projects/ folder? In practice, it would be marked NOT <copy_file/> so we > get a softlink to it in slots, vboxwrapper follows this to projects/ and > attaches the VM, so it would seem in line with the logic you'd expect from > an app dealing with a softlinked file. > > > > Thanks, > > > > Marius > > > > > > On Mon, Jun 29, 2015 at 12:11 PM, Rom Walton <[email protected]> wrote: > > Vboxwrapper.cpp:673 would be the place you would want to start with any > local changes. > > > > I think we wanted to go one step further when removing the <copy_file/> > flag and also mark the VDI as immutable. That would cause VirtualBox to > create a differencing disk in the local slot directory and avoid the disk > image copy operation to the slot directory. > > > > Doing so will probably lead to some bigger changes though. > > > > ----- Rom > > > > *From:* Marius Millea [mailto:[email protected]] > *Sent:* Monday, June 29, 2015 1:30 AM > *To:* Rom Walton <[email protected]>; [email protected] > > > *Subject:* Re: [boinc_dev] is there a way to persist files on the host > > > > Hi Rom, > > > > I saw your commits regarding the scratch directory. Thanks for the > amazingly quick response! > > > > Unfortunately, I now realize its not going to work out as I initially > imaged it (see this discussion > <https://forums.docker.com/t/how-does-boot2docker-persistence-work/2077> > with one of the boot2docker guys). Basically, the docker directory can't be > on a VBox shared folder due to insufficiencies of vboxfs. > > > > I should have realized that the correct way to do this is via > boot2docker's normal way of persistence, which just uses a VDI disk. > Essentially a persistent VDI would live in the BOINC project/ directory, > and vboxwrapper would attach that same one each time to the VM (meaning you > can only have 1 of these kinds of tasks running at a time, granted that was > always the case without properly installing the docker client on each > host). This is in fact *almost* exactly what the wrapper currently does. > The only change needed would be to allow vm_cache.vdi to not have > <copy_file/> and to have vboxwrapper correctly follow this link (I don't > think it currently does). I'm wondering what you think about possibly > adding an option to do that? I feel bad having you take the time for this > other scratch directory feature which was not really needed by me (I think > it might still be useful to some anyway), plus I acknowledge I don't know > what other road blocks I might hit. If you wanted to just point me to where > I might make this change myself, I'm happy to do it. > > > > In any case, I had been hoping I could avoid changes to vboxwrapper so > that I could have the convenience of using the precompiled versions you > guys put up. Since that doesn't look like its going to be possible, is > there any help you could give me on cross compiling? What does your setup > look like? (I've never done this before) > > > > Thanks, > > > > Marius > > > > > > On Thu, Jun 25, 2015 at 4:40 PM, Marius Millea <[email protected]> > wrote: > > Thanks for the interest. Thus far just Linux, although a Windows machine > is waiting once any of this kind of works. The way I currently envisage it > (which could well change), there would be no forking vboxwrapper required. > Basically I modified the boot2docker ISO to act as "multi purpose app" (in > the language of http://boinc.berkeley.edu/trac/wiki/VboxApps) so on boot > it mounts shared/ and from there runs some boinc_app script. That script > then calls "docker run..." etc.. so its all happening inside the VM, the > host system never needs to interact directly with the docker daemon. It > could certainly be done other ways too though. > > > > And agreed, the boot image size is very enticing. There is also the size > of the docker image that gets run, but there are decent to good base images > in the 5-100Mb range, and with this persistence thing working it would be a > one time download (and thank's to Docker, if you're running different > images all based on the same base image, you only download the changes). > > > > Marius > > > > On Thu, Jun 25, 2015 at 4:18 PM, Rom Walton <[email protected]> wrote: > > Marius, > > What host platform are you testing against? > > ----- Rom > > -----Original Message----- > From: boinc_dev [mailto:[email protected]] On Behalf Of > Rom Walton > Sent: Thursday, June 25, 2015 5:43 PM > To: Marius Millea <[email protected]> > Cc: [email protected] > Subject: Re: [boinc_dev] is there a way to persist files on the host > > Well, that is an interesting idea. > > Ultimately I think there will need to be a fork of vboxwrapper that knows > how to handle the docker daemon as well as VirtualBox. > > A 27MB boot image is pretty enticing. > > ----- Rom > > From: Marius Millea [mailto:[email protected]] > Sent: Thursday, June 25, 2015 4:43 PM > To: Rom Walton <[email protected]> > Cc: [email protected] > Subject: Re: [boinc_dev] is there a way to persist files on the host > > Yea sorry ambiguous what I meant by host there. The idea is that BOINC > clients are running VBoxwrapper which loads up boot2docker inside of which > I run my Docker apps. > > Marius > > On Thu, Jun 25, 2015 at 1:38 PM, Rom Walton <[email protected]<mailto: > [email protected]>> wrote: > Hosts or guests? > > Using VirtualBox implies that your docker application would be running > within a Linux guest. > > ----- Rom > > From: Marius Millea [mailto:[email protected]<mailto:[email protected] > >] > Sent: Thursday, June 25, 2015 4:10 PM > To: Rom Walton <[email protected]<mailto:[email protected]>> > Cc: [email protected]<mailto:[email protected]> > > Subject: Re: [boinc_dev] is there a way to persist files on the host > > Hi Rom, > > Yea, I think that's exactly what I'd need. I had started looking at the > source for vboxwrapper, but if that's something you could add officially to > BOINC then that'd be amazing, thanks! > > My end goal btw is to have hosts running Docker, and this would allow > persisting images between tasks. We'll see if there's any other road > blocks... > > Marius > > > > > > On Thu, Jun 25, 2015 at 12:25 PM, Rom Walton <[email protected]<mailto: > [email protected]>> wrote: > I can add another shared directory that points to a scratch area in the > project's directory. > > Would that work for you? > > ----- Rom > > -----Original Message----- > From: boinc_dev [mailto:[email protected]<mailto: > [email protected]>] On Behalf Of Marius Millea > Sent: Thursday, June 25, 2015 2:27 PM > > To: [email protected]<mailto:[email protected]> > Subject: Re: [boinc_dev] is there a way to persist files on the host > > (sorry if this screws up the threading, I can't figure out how to reply to > a specific message given I receive only the digest?) > > > I see, that makes a lot of sense, thanks. I suppose a followup problem is > that this is a VBox app, which AFAIK only gets access to the mounted > shared/ folder. Is there any way around this? > > Marius > > > The easiest way is to not tell BOINC about the file - > > just create it in the project directory, and look for it there in > > subsequent jobs. > > > > The app can get the project dir from the APP_INIT_DATA: > > http://boinc.berkeley.edu/trac/wiki/StatusApi > > > > -- David > > > > > > On 25-Jun-2015 2:10 AM, Marius Millea wrote: > > > Suppose the result of a computation yields a large file which I > > > would > > like > > > > > > 1) to remain on host > > > 2) to be used in subsequent computations > > > 3) to not be uploaded > > > > > > I think I know how to do 1), if I have the file in the output > > > template I can add the <copy_file> flag. Is there actually any way > > > to do 2 or 3 though? > > > > > > Thanks, > > > > > > Marius > _______________________________________________ > boinc_dev mailing list > > [email protected]<mailto:[email protected]> > > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. > > > _______________________________________________ > boinc_dev mailing list > [email protected] > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. > > > > > > > > > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
