Toivo, You are correct that any extension could consume any number of resources within the JVM, host disk/memory/cpu/network/etc.. The framework grants control to those extensions as soon as it gives it a thread. We should definitely evaluate options that allow us to feasibly cap those things as well as security aspects/trusted code/etc.. But that is indeed a complicated space. It is unclear to me how it might work for things within the same JVM but definitely worth exploring.
Now, the other side of this is to allow the developer of those extensions to communicate fundamental characteristics of that extension through the use of annotations. This way when selecting components to use a flow the dataflow manager can make an informed choice. Further we record and display how many bytes are being read and written. We can do more too to understand actual resource usage. Thanks Joe On Sun, Jul 19, 2015 at 2:16 PM, Toivo Adams <[email protected]> wrote: > All processors can run on all nodes. > Some processors may be very resource hungry and eat up all JVM available > memory. > As a result this will bring down whole node. > > Or use all available CPU and other processors don't have change to do work > at all. > Also this may be delay sending heartbeats to NCM and NCM may think node is > not available and disconnect node. > > Is this correct? > > I am sure everyone want to write processors which behave like a good > citizens. > But still people do make mistakes and everything can happen. > > Have you considered to introduce some kind of processors isolation? > Something for example Apache Yarn / CGroups does. > > Or at least is it possible to specify which node(s) should run potentially > harmful processors? > > Thanks > Toivo > > > > > -- > View this message in context: > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Processors-isolation-to-protect-other-processors-tp2200.html > Sent from the Apache NiFi (incubating) Developer List mailing list archive at > Nabble.com.
