Vboxwrapper.cpp:673 would be the place you would want to start with any local changes.
I think we wanted to go one step further when removing the <copy_file/> flag and also mark the VDI as immutable. That would cause VirtualBox to create a differencing disk in the local slot directory and avoid the disk image copy operation to the slot directory. Doing so will probably lead to some bigger changes though. ----- Rom From: Marius Millea [mailto:[email protected]] Sent: Monday, June 29, 2015 1:30 AM To: Rom Walton <[email protected]>; [email protected] Subject: Re: [boinc_dev] is there a way to persist files on the host Hi Rom, I saw your commits regarding the scratch directory. Thanks for the amazingly quick response! Unfortunately, I now realize its not going to work out as I initially imaged it (see this discussion<https://forums.docker.com/t/how-does-boot2docker-persistence-work/2077> with one of the boot2docker guys). Basically, the docker directory can't be on a VBox shared folder due to insufficiencies of vboxfs. I should have realized that the correct way to do this is via boot2docker's normal way of persistence, which just uses a VDI disk. Essentially a persistent VDI would live in the BOINC project/ directory, and vboxwrapper would attach that same one each time to the VM (meaning you can only have 1 of these kinds of tasks running at a time, granted that was always the case without properly installing the docker client on each host). This is in fact almost exactly what the wrapper currently does. The only change needed would be to allow vm_cache.vdi to not have <copy_file/> and to have vboxwrapper correctly follow this link (I don't think it currently does). I'm wondering what you think about possibly adding an option to do that? I feel bad having you take the time for this other scratch directory feature which was not really needed by me (I think it might still be useful to some anyway), plus I acknowledge I don't know what other road blocks I might hit. If you wanted to just poi nt me to where I might make this change myself, I'm happy to do it. In any case, I had been hoping I could avoid changes to vboxwrapper so that I could have the convenience of using the precompiled versions you guys put up. Since that doesn't look like its going to be possible, is there any help you could give me on cross compiling? What does your setup look like? (I've never done this before) Thanks, Marius On Thu, Jun 25, 2015 at 4:40 PM, Marius Millea <[email protected]<mailto:[email protected]>> wrote: Thanks for the interest. Thus far just Linux, although a Windows machine is waiting once any of this kind of works. The way I currently envisage it (which could well change), there would be no forking vboxwrapper required. Basically I modified the boot2docker ISO to act as "multi purpose app" (in the language of http://boinc.berkeley.edu/trac/wiki/VboxApps) so on boot it mounts shared/ and from there runs some boinc_app script. That script then calls "docker run..." etc.. so its all happening inside the VM, the host system never needs to interact directly with the docker daemon. It could certainly be done other ways too though. And agreed, the boot image size is very enticing. There is also the size of the docker image that gets run, but there are decent to good base images in the 5-100Mb range, and with this persistence thing working it would be a one time download (and thank's to Docker, if you're running different images all based on the same base image, you only download the changes). Marius On Thu, Jun 25, 2015 at 4:18 PM, Rom Walton <[email protected]<mailto:[email protected]>> wrote: Marius, What host platform are you testing against? ----- Rom -----Original Message----- From: boinc_dev [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Rom Walton Sent: Thursday, June 25, 2015 5:43 PM To: Marius Millea <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [boinc_dev] is there a way to persist files on the host Well, that is an interesting idea. Ultimately I think there will need to be a fork of vboxwrapper that knows how to handle the docker daemon as well as VirtualBox. A 27MB boot image is pretty enticing. ----- Rom From: Marius Millea [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, June 25, 2015 4:43 PM To: Rom Walton <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [boinc_dev] is there a way to persist files on the host Yea sorry ambiguous what I meant by host there. The idea is that BOINC clients are running VBoxwrapper which loads up boot2docker inside of which I run my Docker apps. Marius On Thu, Jun 25, 2015 at 1:38 PM, Rom Walton <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: Hosts or guests? Using VirtualBox implies that your docker application would be running within a Linux guest. ----- Rom From: Marius Millea [mailto:[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>] Sent: Thursday, June 25, 2015 4:10 PM To: Rom Walton <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> Cc: [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> Subject: Re: [boinc_dev] is there a way to persist files on the host Hi Rom, Yea, I think that's exactly what I'd need. I had started looking at the source for vboxwrapper, but if that's something you could add officially to BOINC then that'd be amazing, thanks! My end goal btw is to have hosts running Docker, and this would allow persisting images between tasks. We'll see if there's any other road blocks... Marius On Thu, Jun 25, 2015 at 12:25 PM, Rom Walton <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: I can add another shared directory that points to a scratch area in the project's directory. Would that work for you? ----- Rom -----Original Message----- From: boinc_dev [mailto:[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>] On Behalf Of Marius Millea Sent: Thursday, June 25, 2015 2:27 PM To: [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> Subject: Re: [boinc_dev] is there a way to persist files on the host (sorry if this screws up the threading, I can't figure out how to reply to a specific message given I receive only the digest?) I see, that makes a lot of sense, thanks. I suppose a followup problem is that this is a VBox app, which AFAIK only gets access to the mounted shared/ folder. Is there any way around this? Marius The easiest way is to not tell BOINC about the file - > just create it in the project directory, and look for it there in > subsequent jobs. > > The app can get the project dir from the APP_INIT_DATA: > http://boinc.berkeley.edu/trac/wiki/StatusApi > > -- David > > > On 25-Jun-2015 2:10 AM, Marius Millea wrote: > > Suppose the result of a computation yields a large file which I > > would > like > > > > 1) to remain on host > > 2) to be used in subsequent computations > > 3) to not be uploaded > > > > I think I know how to do 1), if I have the file in the output > > template I can add the <copy_file> flag. Is there actually any way > > to do 2 or 3 though? > > > > Thanks, > > > > Marius _______________________________________________ boinc_dev mailing list [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address. _______________________________________________ boinc_dev mailing list [email protected]<mailto:[email protected]> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address. _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
