Thanks Damian. I chatted with Ashish offline and he will check the possibility an API (similar to Hadoop's credential store), but warned that since it is a more complex change, he may not get to it immediately.
Gwen On Sat, Aug 27, 2016 at 6:59 AM, Damian Guy <damian....@gmail.com> wrote: > I'm in agreement with Gwen. An API would be a better solution. Running > executables from Kafka is dangerous. > On Sat, 27 Aug 2016 at 12:02, Ismael Juma <ism...@juma.me.uk> wrote: > >> Hi Matthias, >> >> Improve Kafka Streams Join Semantics is not mentioned on the KIP page and >> that is probably the source of confusion: >> >> https://cwiki.apache.org/confluence/display/KAFKA/ >> Kafka+Improvement+Proposals >> >> Ismael >> >> On Thu, Aug 25, 2016 at 10:44 PM, Matthias J. Sax <matth...@confluent.io> >> wrote: >> >> > I guess this should be KIP-77 ? >> > >> > KIP-76 is "Improve Kafka Streams Join Semantics" >> > >> > See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain >> > >> > -Matthias >> > >> > On 08/25/2016 10:13 PM, Ashish Singh wrote: >> > > Hey Gwen, >> > > >> > > You’re right that if someone can alter the executable then they can do >> > > things in the context of the thing running the script, like kafka. But >> if >> > > you were kafka admin user (or root), you could also do lots of things >> to >> > > lots of other different files owned by the user, so it’s not really >> that >> > > much different than the current state of things. >> > > >> > > You’re right to wonder about the real security gains here. In some >> sense, >> > > they aren’t many, because if you know where to look and what to do, you >> > can >> > > coax the password out of that executable. What this approach really >> does >> > is >> > > make it *nontrivial* for an attacker to get the password. And people >> tend >> > > to flip out when they see passwords sitting in the clear on a disk, >> > because >> > > we’ve all been rightly trained that cleartext passwords are bad. >> > > >> > > This approach when combined with some strong security practices, like >> the >> > > ones mentioned below makes the system reasonably secure. This approach >> is >> > > probably the simplest way for folks to strengthen their Kafka security. >> > > There are other more complicated ways, like Hadoop’s credential store, >> > > which depends on external systems. If the community feels that this >> does >> > > not help, we can definitely move towards more complicated mechanisms. >> > > However, this has sufficed for our needs so far and others have >> expressed >> > > their satisfaction on the JIRA. >> > > >> > > - Executable decrypts a file that stores encrypted passwords. >> > > - The secret to decrypt that file is passed in via environment, >> which >> > is >> > > generally a bit harder to find than files on disk. >> > > - The perms also protect the executable. >> > > - The file sits on an ephemeral disk that’s mounted to memory, so >> > > stealing a physical disk won’t result in getting even the encrypted >> > > password. >> > > >> > > On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <g...@confluent.io> >> wrote: >> > > >> > > Hi Ashish, >> > >> >> > >> I appreciate the need to integrate our authentication with other >> > >> systems that store passwords. >> > >> I am not sure that doing so by running a binary is the best solution. >> > >> >> > >> First, it does not add security: As you said, a file is just "sitting >> > >> there" the same way an executable is just "sitting there" - we still >> > >> rely on file system privileges for security. >> > >> Second, the idea that Kafka will run arbitrary filesystem executables >> > >> is pretty terrifying. Reading a string from a file is harmless, but an >> > >> incorrectly privileged executable can be replaced with "rm -rf /" or >> > >> anything really. Kafka sometimes runs from privileged account, so this >> > >> is a serious risk. >> > >> >> > >> I looked at the Hadoop credential store you helpfully linked to in the >> > >> KIP, and it seems like the Hadoop proposal includes a well thought out >> > >> API to integrate with external systems. Since we took this approach in >> > >> the past, I'm wondering why not follow the same and use an API to >> > >> integrate with credential stores rather than arbitrary executables. >> > >> >> > >> Gwen >> > >> >> > >> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <asi...@cloudera.com> >> > >> wrote: >> > >>> Hey Guys, >> > >>> >> > >>> I’ve just posted KIP-76: Enable getting password from executable >> rather >> > >>> than passing as plaintext in config files >> > >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En >> > >> able+getting+password+from+executable+rather+than+passing+ >> > >> as+plaintext+in+config+files> >> > >>> . >> > >>> >> > >>> The proposal is to enable getting passwords from executable. This is >> an >> > >> ask >> > >>> from very security conscious users. >> > >>> >> > >>> Full details are here: >> > >>> >> > >>> KIP: >> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena >> > >> ble+getting+password+from+executable+rather+than+passing+as+ >> > >> plaintext+in+config+files >> > >>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629 >> > >>> POC: https://github.com/apache/kafka/pull/1770 >> > >>> >> > >>> Thanks >> > >>> >> > >>> -- >> > >>> >> > >>> Regards, >> > >>> Ashish >> > >> >> > >> >> > >> >> > >> -- >> > >> Gwen Shapira >> > >> Product Manager | Confluent >> > >> 650.450.2760 | @gwenshap >> > >> Follow us: Twitter | blog >> > >> >> > > >> > > >> > >> > >> -- Gwen Shapira Product Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog
