I agree with Rodric. In the past, our assumption was that the risk associated with blackbox invocations is higher than with managed invocations. That's why we used two separate invoker pools such that the possible problems with blackbox invocations won't affect any managed invokers and thus, managed invocations.
We need to re-evaluate whether we still see the same risk as in the past and whether we see ways to address possible problems. For those who are willing to accept certain risks, Rodric's proposal of providing flexibility via configuration settings may be the right approach. Apparently, the requirements depend on the type of OW environment you run. For a large production environment supporting a large number of different tenants requirements are different than for your local development environment. I think we have (at least) these risks: => Docker images consume too much disk space so that no more new images can be pulled / no new containers can be created. A limited set of Docker images is required to support managed invocations. The situation is different for blackbox invocations. Potentially each blackbox invocation may lead to a pull of an additional Docker image so that the disk volume storing images and containers may fill up. At the moment, there are no restrictions on the number and size of blackbox images on an invoker. So invokers running blackbox actions have a higher risk for outages. OW does not have any image cleanup strategy and a limited container cleanup strategy. Kubernetes has a garbage collection approach [1]. We may add a cleanup strategy to OW as well or users need their own cleanup approach outside of OW. => Managed runtime images have more control. All managed runtimes containers run OW code up to the point where the /init REST endpoint can be used to inject end user action code. So we can prepare the environment and accept the initial socket connection from the invoker. Our timeout concept bases these socket connections. Blackbox containers have more control and can decide to not even open a listening socket. It's harder for the invoker to deal with such a non-cooperating container and prevent the container from consuming resources. => Blackbox images can bring much more user code than managed actions. It's easier to hide malicious code or tools in a blackbox image. => Blackbox image pulls keep container infrastructure busy. As already mentioned, each blackbox invocation may lead to a pull of an additional Docker image. These pulls consume CPU and network resources impacting other invocations. The pull mechanism could even be used to attack Docker Hub. Any other thoughts or feedback on the risks I described? Regards, Sven. [1] https://kubernetes.io/docs/concepts/cluster-administration/kubelet-garbage-collection/ Rodric Rabbah <[email protected]> wrote on 2018/11/28 13:09:12: > From: Rodric Rabbah <[email protected]> > To: [email protected] > Date: 2018/11/28 13:09 > Subject: Re: Don't differentiate between blackbox and whitebox images > > I think it would be a mistake to blindly merge two container pools - > docker actions as run today are pulled as needed and can take a long > time, they’re subject to different kinds of attacks, and can affect > performance of other tenants (higher noisy neighbors). > > If you want to allow more overlap, it should be at least done as a > configuration parameter which allows different pools to continue to > exist (note that for small N as in local dev they’re allowed to > overlap today). Or, used for overflow only as a way of providing > more elasticity. > > -r > > > On Nov 28, 2018, at 5:56 AM, Christian Bickel <[email protected]> wrote: > > > > Hi, > > > > in the past we divided the invokers into blackbox and whitebox invokers. > > This has the effect that one of these two types could be overloaded while > > the other type still has free capacity. > > > > Does anyone see issues on using every invoker for everything? > > Or does anyone see any action items that need to be addressed before we > > invoke every action-type on every invoker? > > > > If not I'll open a PR to not differentiate between these types anymore. > > > > Greetings > > Christian Bickel
