We have developed integrations with Hashicorp Consul and Vault, partly to deal with this same use-case. I'm happy to open source these if there is an interest.
On Fri, 28 Aug 2020, at 17:25, oliver twix wrote: > I would be great I think. > > Le ven. 28 août 2020 à 17:20, Bryan Bende <bbe...@gmail.com> a écrit : >> The reason for requiring the FS permissions on the HDFS processors is >> because you can provide a core-site.xml with file:/// as the default FS and >> then use ListHDFS/FetchHDFS/PutHDFS to essentially do the same thing as >> ListFile/FetchFile/PutFile. >> >> A possible consideration would be to remove the requirement for the FS >> permissions, and then add validation logic that prevents HDFS processors >> from being used with a file:/// filesystem. >> >> On Fri, Aug 28, 2020 at 11:14 AM oliver twix <olivertwix.pe...@gmail.com> >> wrote: >>> Hello, thank you for your quick answers and sorry for my late reply. >>> >>> In the context of preparing platform security audit, I am trying to >>> identify possible threats over our NIFI clusters. >>> >>> @Mark, I didn't know about NIFI_ALLOW_EXPLICIT_KEYTAB parameter. Good to >>> know; it will limit a semi-friendly user to re-use easily a keytab from the >>> filesystem. >>> >>> My main concern now is the following : In my context, HDFS related >>> processors are required for our standard users. It means that any standard >>> user will require Filesystem access. It leads to the fact that if a >>> "standard" account is corrupted (common security audit use case) the whole >>> cluster can be easily corrupted gaining for instance admin privileges and >>> obviously keytab leaks and associated sensitive data leaks. >>> >>> Coming more on a design question, which constraint led to add a local FS >>> access permission to HDFS related processors? I would expect similar >>> requirements than for other equivalent distant storage processors (S3, >>> etc.). Is it linked to explicit keytab configuration, HDFS cluster >>> configuration files ? other deeper reason? Is there a hope at some point >>> that a change could get rid of this specific permission requirement? >>> >>> So far, if I understand well, giving HDFS related process access means >>> security threats on the cluster itself (reason why processors are >>> restricted). A workaround I could identify is to spawn on behalf of users >>> HDFS related processors but I am not sure it will be possible keeping a >>> good UX. >>> >>> >>> Many thanks for your help, >>> Olivier >>> >>> >>> >>> Le jeu. 30 juil. 2020 à 18:17, Joe Witt <joe.w...@gmail.com> a écrit : >>>> ....in short this case has been really deeply considered and it is a very >>>> common usage pattern. We offer a set of policies/controls that let it be >>>> well restricted and locked down. But if a user is given too many accesses >>>> then yes they can be malicious. >>>> >>>> Thanks >>>> >>>> On Thu, Jul 30, 2020 at 9:13 AM Andy LoPresto <alopre...@apache.org> wrote: >>>>> If your concern is the malicious insider using FetchHDFS to read the >>>>> keytab as data from the filesystem, the *HDFS processors are marked as >>>>> Restricted and require an additional explicit permission to be granted >>>>> for users to configure them. At a file system interaction level, the NiFi >>>>> Java processes are running as a single OS user, so all of the keytabs >>>>> will need to be readable by that OS user, and the OS can’t detect which >>>>> Java process is acting as which application user. >>>>> >>>>> >>>>> Andy LoPresto >>>>> alopre...@apache.org >>>>> *alopresto.apa...@gmail.com* >>>>> He/Him >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>>> >>>>>> On Jul 30, 2020, at 8:10 AM, Mark Payne <marka...@hotmail.com> wrote: >>>>>> >>>>>> Olivier, >>>>>> >>>>>> As Joe mentioned, it may help to further explain the exact scenario that >>>>>> you are concerned about. >>>>>> >>>>>> But what I *think* you are concerned about is the following scenario: >>>>>> - You have several different users developing flows in NiFi. >>>>>> - You want the ability to give User A (and only User A) access to Keytab >>>>>> A by creating a KeytabCredentialsService and giving User A READ Access >>>>>> to it. >>>>>> - You want the ability to give User B (and only User B) access to Keytab >>>>>> B by creating a KeytabCredentialsService and giving User B READ Access >>>>>> to it. >>>>>> - If you do the above, but User A happens to know that Keytab B is >>>>>> stored at /etc/keytabs/keytab-b, then all User A has to do is configure >>>>>> PutHDFS’s Keytab property to “/etc/keytabs/keytab-b” instead of using >>>>>> the KeytabCredentialsService. Then User A has access to User B’s keytab. >>>>>> >>>>>> Is that the scenario that you are concerned about? >>>>>> >>>>>> If yes, then the answer is to set the NIFI_ALLOW_EXPLICIT_KEYTAB >>>>>> Environment Variable to a value of “false”. If you do that, then PutHDFS >>>>>> and related processors that allow for either a KeytabCredentialsService >>>>>> or an explicit keytab will become invalid (and therefore not runnable) >>>>>> if an explicit keytab is used. This prevents User A from using Keytab A >>>>>> or Keytab B directly and instead forces them to use no Keytab (which >>>>>> presumably will result in authorization failure) or using the >>>>>> KeytabCrdentialsService, which you can control READ access to. >>>>>> >>>>>> Does this help? >>>>>> >>>>>> Thanks >>>>>> -Mark >>>>>> >>>>>>> On Jul 30, 2020, at 10:09 AM, Joe Witt <joe.w...@gmail.com> wrote: >>>>>>> >>>>>>> Hello >>>>>>> >>>>>>> Can you more fully explain the scenario you have in mind and what an >>>>>>> intentionally malicious user might do? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> On Thu, Jul 30, 2020 at 6:54 AM oliver twix >>>>>>> <olivertwix.pe...@gmail.com> wrote: >>>>>>>> Hello, >>>>>>>> Getting deeper on using nifi in multitenant use cases, I am facing a >>>>>>>> security question: our nifi users must be able to interact with hdfs >>>>>>>> not sharing their credentials (keytabs). >>>>>>>> >>>>>>>> From what understood, keytabCredentialsService enable a way to give a >>>>>>>> policy based control over keytabs access. >>>>>>>> Where I miss something is that for a user to use an hdfs processor, it >>>>>>>> requires read/write filesystem permissions. In this context, any hdfs >>>>>>>> user is able to read the keytabs of any other users. So in my >>>>>>>> understanding, it breaks the initial objective of >>>>>>>> keytabCredentialsService to control keytabs accesses. >>>>>>>> >>>>>>>> Am I missing something ? Do you have a mean to avoid giving access to >>>>>>>> all keytabs stored on local filesystem? >>>>>>>> >>>>>>>> Olivier >>>>>>>> >>>>>>>>
