It applies to custom containers as well. You can find the container manifest in the GCE VM metadata, and it should have an entry for privileged mode. The reason for this was to enable GPU accelerator support, but agree with Robert that it is not part of any contracts, so in theory this could change or perhaps be more strictly limited to accelerator support. In fact, originally, this was only enabled for pipelines using accelerators but for purely internal implementation details I believe it is currently enabled for all pipelines.
So for prototyping purposes I think you could try it, but I can't make any guarantees in this thread that privileged mode will continue to work. cc: @Aaron Li <aaronle...@google.com> FYI On Mon, Jan 30, 2023 at 12:16 PM Robert Bradshaw <rober...@google.com> wrote: > I'm also not sure it's part of the contract that the containerization > technology we use will always have these capabilities. > > On Mon, Jan 30, 2023 at 10:53 AM Chad Dombrova <chad...@gmail.com> wrote: > > > > Hi Valentyn, > > > >> > >> Beam SDK docker containers on Dataflow VMs are currently launched in > privileged mode. > > > > > > Does this only apply to stock sdk containers? I'm asking because we use > a custom sdk container that we build. We've tried various ways of running > mount from within our custom beam container in Dataflow and we could not > get it to work, while the same thing succeeds in local tests and in our CI > (gitlab). The assessment at the time (this was maybe a year ago) was that > the container was not running in privileged mode, but if you think that's > incorrect we can revisit this and report back with some error logs. > > > > -chad > > >
