This is an automated email from the ASF dual-hosted git repository.
suneet pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b51632b Update security overview with additional recommendations
(#11016)
b51632b is described below
commit b51632b0bf38d1ef396e62a4851d84433029713d
Author: Charles Smith <[email protected]>
AuthorDate: Wed Apr 14 08:58:17 2021 -0700
Update security overview with additional recommendations (#11016)
* updatee security overview with additional recommendations for improved
security
* address first set of review questions
* Update docs/operations/security-overview.md
* Update docs/operations/security-overview.md
* apply changes from review
* Update docs/operations/security-overview.md
Co-authored-by: Suneet Saldanha <[email protected]>
* Update docs/operations/security-overview.md
Co-authored-by: Suneet Saldanha <[email protected]>
* Update docs/operations/security-overview.md
Co-authored-by: Suneet Saldanha <[email protected]>
* Update security-overview.md
fix additional comments & typos cc: @suneet-s, @jihoonsoon
Co-authored-by: Suneet Saldanha <[email protected]>
---
docs/operations/security-overview.md | 186 +++++++++++++++++++++++++---------
docs/operations/security-user-auth.md | 2 +-
website/.spelling | 3 +-
3 files changed, 139 insertions(+), 52 deletions(-)
diff --git a/docs/operations/security-overview.md
b/docs/operations/security-overview.md
index d65e9f5..c51e7d2 100644
--- a/docs/operations/security-overview.md
+++ b/docs/operations/security-overview.md
@@ -1,6 +1,7 @@
---
id: security-overview
title: "Security overview"
+description: Overiew of Apache Druid security. Includes best practices,
configuration instructions, and a description of the security model.
---
<!--
@@ -23,66 +24,112 @@ title: "Security overview"
-->
-## Overview
+
+This document provides an overview of Apache Druid security features,
configuration instructions, and some best practices to secure Druid.
By default, security features in Druid are disabled, which simplifies the
initial deployment experience. However, security features must be configured in
a production deployment. These features include TLS, authentication, and
authorization.
-To implement Druid security, you configure authenticators and authorizers.
Authenticators control the way user identities are verified, while authorizers
map the authenticated users (via user roles) to the datasources they are
permitted to access. Consequently, implementing Druid security also involves
considering your datasource scheme, since that scheme represents the
granularity at which data access permissions are allocated.
-The following graphic depicts the course of request through the authentication
process:
+## Best practices
+The following recommendations apply to the Druid cluster setup:
+* Run Druid as an unprivileged Unix user. Do not run Druid as the root user.
+ > **WARNING!** \
+ Druid administrators have the same OS permissions as the Unix user account
running Druid. See [Authentication and authorization
model](security-user-auth.md#authentication-and-authorization-model). If the
Druid process is running under the OS root user account, then Druid
administrators can read or write all files that the root account has access to,
including sensitive files such as `/etc/passwd`.
+* Enable authentication to the Druid cluster for production environments and
other environments that can be accessed by untrusted networks.
+* Enable authorization and do not expose the Druid Console without
authorization enabled. If authorization is not enabled, any user that has
access to the web console has the same privileges as the operating system user
that runs the Druid Console process.
+* Grant users the minimum permissions necessary to perform their functions.
For instance, do not allow users who only need to query data to write to data
sources or view state.
+* Disable JavaScript, as noted in the [Security
section](https://druid.apache.org/docs/latest/development/javascript.html#security)
of the JavaScript guide.
-
+The following recommendations apply to the network where Druid runs:
+* Enable TLS to encrypt communication within the cluster.
+* Use an API gateway to:
+ - Restrict access from untrusted networks
+ - Create an allow list of specific APIs that your users need to access
+ - Implement account lockout and throttling features.
+* When possible, use firewall and other network layer filtering to only expose
Druid services and ports specifically required for your use case. For example,
only expose Broker ports to downstream applications that execute queries. You
can limit access to a specific IP address or IP range to further tighten and
enhance security.
+The following recommendation applies to Druids authorization and
authentication model:
+* Only grant `WRITE` permissions to any `DATASOURCE` to trusted users. Druid's
trust model assumes those users have the same privileges as the operating
system user that runs the Druid Console process.
+* Only grant `STATE READ`, `STATE WRITE`, `CONFIG WRITE`, and `DATASOURCE
WRITE` permissions to highly-trusted users. These permissions allow users to
access resources on behalf of the Druid server process regardless of the
datasource.
+* If your Druid client application allows less-trusted users to control the
input source or firehose of an ingestion task, validate the URLs from the
users. It is possible to point unchecked URLs to other locations and resources
within your network or local file system.
-This document gives you an overview of security features in Druid and how to
configure them, and some best practices for securing Druid.
+## Enable TLS
-## Best practices
+Enabling TLS encrypts the traffic between external clients and the Druid
cluster and traffic between services within the cluster.
-* Do not expose the Druid Console without authentication on untrusted
networks. Access to the console effectively confers access the file system on
the installation machine, via file browsers in the UI. You should use an API
gateway that restricts who can connect from untrusted networks, allow list the
specific APIs that your users need to access, and implements account lockout
and throttling features.
-* You should only grant `WRITE` permissions to a `DATASOURCE` to trusted
users. Druid assumes that these users have the same privileges as the operating
system user that runs the Druid process.
-* Grant users the minimum permissions necessary to perform their functions.
For instance, do not allow users who only need to query data to write to data
sources or view state.
-* Disable JavaScript, as noted in the [Security
section](https://druid.apache.org/docs/latest/development/javascript.html#security)
of the JavaScript guide.
-* Run Druid as an unprivileged Unix user on the installation machine (not
root).
- > This is an important point! Administrator users on Druid have the same
permission as the Unix user account it is running under. If the Druid process
is running under the root user account in the OS, then Administrator users on
Druid can read/write all files that the root account has access to, including
sensitive files such as `/etc/passwd`.
+### Generating keys
+Before you enable TLS in Druid, generate the KeyStore and truststore. When one
Druid process, e.g. Broker, contacts another Druid process , e.g. Historical,
the first service is a client for the second service, considered the server.
-You can configure authentication and authorization to control access to the
the Druid APIs. The first step is enabling TLS for the cluster nodes. Then
configure users, roles, and permissions, as described in the following
sections.
+The client uses a trustStore that contains certificates trusted by the client.
For example, the Broker.
-The configuration settings mentioned below are primarily located in the
`common.runtime.properties` file. Note that you need to make the configuration
changes on each Druid server in the cluster.
+The server uses a KeyStore that contains private keys and certificate chain
used to securely identify itself.
+The following example demonstrates how to use Java keytool to generate the
KeyStore for the server and then create a trustStore to trust the key for the
client:
-## Enable TLS
+1. Generate the KeyStore with the Java `keytool` command:
+```
+$> keytool -keystore keystore.jks -alias druid -genkey -keyalg RSA
+```
+2. Export a public certificate:
+```
+$> keytool -export -alias druid -keystore keystore.jks -rfc -file public.cert
+```
+3. Create the trustStore:
+```
+$> keytool -import -file public.cert -alias druid -keystore truststore.jks
+```
-The first step in securing Druid is enabling TLS. You can enable TLS to secure
external client connections to Druid as well as connections between cluster
nodes.
+Druid uses Jetty as its embedded web server. See [Configuring SSL/TLS KeyStores
+](https://www.eclipse.org/jetty/documentation/jetty-11/operations-guide/index.html#og-keystore)
from the Jetty documentation.
-The configuration steps are:
-1. Enable TLS by adding `druid.enableTlsPort=true` to
`common.runtime.properties` on each node in the Druid cluster.
-2. Disable the non-TLS port by setting `druid.enablePlaintextPort` to `false`.
-2. Follow the steps in [Understanding Certificates and
Keys](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#understanding-certificates-and-keys)
to generate or import a key and certificate.
-3. Configure the keystore and truststore settings in
`common.runtime.properties`. The file should look something like this:
- ```
- druid.enablePlaintextPort=false
- druid.enableTlsPort=true
-
- druid.server.https.keyStoreType=jks
- druid.server.https.keyStorePath=sample-keystore.jks
- druid.server.https.keyStorePassword=secret123 # replace with your own
password
- druid.server.https.certAlias=druid
-
- druid.client.https.protocol=TLSv1.2
- druid.client.https.trustStoreType=jks
- druid.client.https.trustStorePath=sample-truststore.jks
- druid.client.https.trustStorePassword=secret123 # replace with your own
password
-
- ```
-4. Add the `simple-client-sslcontext` extension to `druid.extensions.loadList`
in `common.runtime.properties`. This enables TLS for Druid nodes acting as
clients.
-5. Restart the cluster.
+ > WARNING: Do not use use self-signed certificates for production
environments. Instead, rely on your current public key infrastructure to
generate and distribute trusted keys.
+
+
+### Update Druid TLS configurations
+Edit `common.runtime.properties` for all Druid services on all nodes. Add or
update the following TLS options. Restart the cluster when you are finished.
+
+```
+# Turn on TLS globally
+druid.enableTlsPort=true
+
+# Disable non-TLS communicatoins
+druid.enablePlaintextPort=false
+
+# For Druid processes acting as a client
+# Load simple-client-sslcontext to enable client side TLS
+# Add the following to extension load list
+druid.extensions.loadList=[......., "simple-client-sslcontext"]
+
+# Setup client side TLS
+druid.client.https.protocol=TLSv1.2
+druid.client.https.trustStoreType=jks
+druid.client.https.trustStorePath=truststore.jks # replace with correct
turstStore file
+druid.client.https.trustStorePassword=secret123 # replace with your own
password
+
+# Setup server side TLS
+druid.server.https.keyStoreType=jks
+druid.server.https.keyStorePath=my-keystore.jks # replace with correct
keyStore file
+druid.server.https.keyStorePassword=secret123 # replace with your own password
+druid.server.https.certAlias=druid
+
+```
For more information, see [TLS support](tls-support.md) and [Simple SSLContext
Provider Module](../development/extensions-core/simple-client-sslcontext.md).
-Druid uses Jetty as its embedded web server. Therefore you refer to
[Understanding Certificates and
Keys](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html)
for complete instructions.
+
+## Authentication and authorization
+
+You can configure authentication and authorization to control access to the
the Druid APIs. Then configure users, roles, and permissions, as described in
the following sections. Make the configuration changes in the
`common.runtime.properties` file on all Druid servers in the cluster.
+
+Within Druid's operating context, authenticators control the way user
identities are verified. Authorizers employ user roles to relate authenticated
users to the datasources they are permitted to access. You can set the
finest-grained permissions on a per-datasource basis.
+
+The following graphic depicts the course of request through the authentication
process:
+
+
+
## Enable an authenticator
@@ -95,16 +142,26 @@ The following takes you through sample configuration steps
for enabling basic au
```
druid.extensions.loadList=["druid-basic-security", "druid-histogram",
"druid-datasketches", "druid-kafka-indexing-service"]
```
-2. Configure the basic Authenticator, Authorizer, and Escalator settings in
the same common.runtime.properties file. For example:
+2. Configure the basic Authenticator, Authorizer, and Escalator settings in
the same common.runtime.properties file. The Escalator defines how Druid
processes authenticate with one another.
+
+An example configuration:
```
# Druid basic security
druid.auth.authenticatorChain=["MyBasicMetadataAuthenticator"]
-
druid.auth.authenticator.MyBasicMetadataAuthenticator.type=basic
+
+ # Default password for 'admin' user, should be changed for production.
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialAdminPassword=password1
+
+ # Default password for internal 'druid_system' user, should be changed for
production.
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialInternalClientPassword=password2
+
+ # Uses the metadata store for storing users, you can use authentication API
to create new users and grant permissions
druid.auth.authenticator.MyBasicMetadataAuthenticator.credentialsValidator.type=metadata
+
+ # If true and the request credential doesn't exists in this credentials
store, the request will proceed to next Authenticator in the chain.
druid.auth.authenticator.MyBasicMetadataAuthenticator.skipOnFailure=false
+
druid.auth.authenticator.MyBasicMetadataAuthenticator.authorizerName=MyBasicMetadataAuthorizer
# Escalator
@@ -136,37 +193,37 @@ The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281
> for secured connections.
-1. Create a user by issuing a POST request to
`druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
replacing USERNAME with the new username. For example:
+1. Create a user by issuing a POST request to
`druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
replacing USERNAME with the *new* username you are trying to create. For
example:
```
- curl -u admin:password -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname
+ curl -u admin:password1 -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname
```
> If you have TLS enabled, be sure to adjust the curl command accordingly.
For example, if your Druid servers use self-signed certificates, you may choose
to include the `insecure` curl option to forgo certificate checking for the
curl command.
2. Add a credential for the user by issuing a POST to
`druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials`.
For example:
```
- curl -u admin:password -H'Content-Type: application/json' -XPOST
--data-binary @pass.json
https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname/credentials
+ curl -u admin:password1 -H'Content-Type: application/json' -XPOST
--data-binary @pass.json
https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname/credentials
```
The password is conveyed in the `pass.json` file in the following form:
```
{
- "password": "password"
+ "password": "myname_password"
}
```
2. For each authenticator user you create, create a corresponding authorizer
user by issuing a POST request to
`druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>`.
For example:
```
- curl -u admin:password -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname
+ curl -u admin:password1 -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname
```
3. Create authorizer roles to control permissions by issuing a POST request to
`druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>`.
For example:
```
- curl -u admin:password -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole
+ curl -u admin:password1 -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole
```
4. Assign roles to users by issuing a POST request to
`druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>/roles/<ROLENAME>`.
For example:
```
- curl -u admin:password -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname/roles/myrole
| jq
+ curl -u admin:password1 -XPOST
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname/roles/myrole
| jq
```
5. Finally, attach permissions to the roles to control how they can interact
with Druid at
`druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>/permissions`.
For example:
```
- curl -u admin:password -H'Content-Type: application/json' -XPOST
--data-binary @perms.json
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole/permissions
+ curl -u admin:password1 -H'Content-Type: application/json' -XPOST
--data-binary @perms.json
https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole/permissions
```
The payload of `perms.json` should be in the form:
```
@@ -187,11 +244,12 @@ The following steps walk through a sample setup procedure:
}
]
```
+ > Note: Druid treats the resource name as a regular expression (regex).
You can use a specific datasource name or regex to grant permissions for
multiple datasources at a time.
## Configuring an LDAP authenticator
-As an alternative to using the basic metadata authenticator, as shown in the
previous section, you can use LDAP to authenticate users. The following steps
provide an overview of the setup steps. For more information on these settings,
see [Properties for LDAP user
authentication](../development/extensions-core/druid-basic-security.md#properties-for-ldap-user-authentication).
+As an alternative to using the basic metadata authenticator, you can use LDAP
to authenticate users. The following steps provide an overview of the setup
procedure. For more information on these settings, see [Properties for LDAP
user
authentication](../development/extensions-core/druid-basic-security.md#properties-for-ldap-user-authentication).
1. In `common.runtime.properties`, add LDAP to the authenticator chain in the
order in which you want requests to be evaluated. For example:
```
@@ -264,3 +322,31 @@ As an alternative to using the basic metadata
authenticator, as shown in the pre
Congratulations, you have configured permissions for user-assigned roles in
Druid!
+
+
+## Druid security trust model
+Within Druid's trust model there users can have different authorization levels:
+- Users with resource write permissions are allowed to do anything that the
druid process can do.
+- Authenticated read only users can execute queries against resources to which
they have permissions.
+- An authenticated user without any permissions is allowed to execute queries
that don't require access to a resource.
+
+Additionally, Druid operates according to the following principles:
+
+From the inner most layer:
+1. Druid processes have the same access to the local files granted to the
specified system user running the process.
+2. The Druid ingestion system can create new processes to execute tasks. Those
tasks inherit the user of their parent process. This means that any user
authorized to submit an ingestion task can use the ingestion task permissions
to read or write any local files or external resources that the Druid process
has access to.
+
+> Note: Only grant the `DATASOURCE WRITE` to trusted users because they can
act as the Druid process.
+
+Within the cluster:
+1. Druid assumes it operates on an isolated, protected network where no
reachable IP within the network is under adversary control. When you implement
Druid, take care to setup firewalls and other security measures to secure both
inbound and outbound connections.
+Druid assumes network traffic within the cluster is encrypted, including API
calls and data transfers. The default encryption implementation uses TLS.
+3. Druid assumes auxiliary services such as the metadata store and ZooKeeper
nodes are not under adversary control.
+
+Cluster to deep storage:
+1. Druid does not make assumptions about the security for deep storage. It
follows the system's native security policies to authenticate and authorize
with deep storage.
+2. Druid does not encrypt files for deep storage. Instead, it relies on the
storage system's native encryption capabilities to ensure compatibility with
encryption schemes across all storage types.
+
+Cluster to client:
+1. Druid authenticates with the client based on the configured authenticator.
+2. Druid only performs actions when an authorizer grants permission. The
default configuration is `allowAll authorizer`.
diff --git a/docs/operations/security-user-auth.md
b/docs/operations/security-user-auth.md
index bb0ff51..d128bb7 100644
--- a/docs/operations/security-user-auth.md
+++ b/docs/operations/security-user-auth.md
@@ -44,7 +44,7 @@ There are two actions:
In practice, most deployments will only need to define two classes of users:
-* Admins, who have WRITE action permissions on all resource types. These users
will add datasources and administer the system.
+* Administrators, who have WRITE action permissions on all resource types.
These users will add datasources and administer the system.
* Data users, who only need READ access to DATASOURCE. These users should
access Query APIs only through an API gateway. Other APIs and permissions
include functionality that should be limited to server admins.
It is important to note that WRITE access to DATASOURCE grants a user broad
access. For instance, such users will have access to the Druid file system, S3
buckets, and credentials, among other things. As such, the ability to add and
manage datasources should be allocated selectively to administrators.
diff --git a/website/.spelling b/website/.spelling
index c0d5898..bd2ecfa 100644
--- a/website/.spelling
+++ b/website/.spelling
@@ -125,6 +125,7 @@ JsonProperty
KMS
Kerberized
Kerberos
+KeyStores
Kinesis
Kubernetes
LRU
@@ -283,7 +284,7 @@ javadoc
joinable
kerberos
keystore
-KeyStores
+keytool
keytab
kubernetes
laning
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
