Thanks for the info. We are going to test this further and we'll let you know how it goes.
-chad On Mon, Jan 30, 2023 at 2:14 PM Valentyn Tymofieiev <valen...@google.com> wrote: > It applies to custom containers as well. You can find the container > manifest in the GCE VM metadata, and it should have an entry for privileged > mode. The reason for this was to enable GPU accelerator support, but agree > with Robert that it is not part of any contracts, so in theory this could > change or perhaps be more strictly limited to accelerator support. In fact, > originally, this was only enabled for pipelines using accelerators but for > purely internal implementation details I believe it is currently enabled > for all pipelines. > > So for prototyping purposes I think you could try it, but I can't make any > guarantees in this thread that privileged mode will continue to work. > > cc: @Aaron Li <aaronle...@google.com> FYI > > > On Mon, Jan 30, 2023 at 12:16 PM Robert Bradshaw <rober...@google.com> > wrote: > >> I'm also not sure it's part of the contract that the containerization >> technology we use will always have these capabilities. >> >> On Mon, Jan 30, 2023 at 10:53 AM Chad Dombrova <chad...@gmail.com> wrote: >> > >> > Hi Valentyn, >> > >> >> >> >> Beam SDK docker containers on Dataflow VMs are currently launched in >> privileged mode. >> > >> > >> > Does this only apply to stock sdk containers? I'm asking because we >> use a custom sdk container that we build. We've tried various ways of >> running mount from within our custom beam container in Dataflow and we >> could not get it to work, while the same thing succeeds in local tests and >> in our CI (gitlab). The assessment at the time (this was maybe a year ago) >> was that the container was not running in privileged mode, but if you think >> that's incorrect we can revisit this and report back with some error logs. >> > >> > -chad >> > >> >
