New Revision...
This revision acknowledges the reality that we often have multiple phases
of feature lifecycle and that we need to account for each phase.
It has also been made more generic.
I have created a Tech Preview Security Audit list and a GA Readiness
Security Audit list.
I've also included suggested items into the GA Readiness list.
It has also been suggested that we publish the information as part of docs
so that the state of such features can be easily determined from these
pages. We can discuss this aspect as well.
Thoughts?
*Tech Preview Security Audit*
For features that are being merged without full security model coverage,
there need to be a base line of assurances that they do not introduce new
attack vectors in deployments that are from actual releases or even just
built from trunk.
*1. UIs*
1.1. Are there new UIs added with this merge?
1.2. Are they enabled/accessible by default?
1.3. Are they hosted in existing processes or as part of a new
process/server?
1.4. If new process/server, is it launched by default?
*2. APIs*
2.1. Are there new REST APIs added with this merge?
2.2. Are they enabled by default?
2.3. Are there RPC based APIs added with this merge?
2.4. Are they enabled by default?
*3. Secure Clusters*
3.1. Is this feature disabled completely in secure deployments?
3.2. If not, is there some justification as to why it should be available?
*4. CVEs*
4.1. Have all dependencies introduced by this merge been checked for known
issues?
--------------------------------------------------------------------------------------------------------------------------------------------------
*GA Readiness Security Audit*
At this point, we are merging full or partial security model
implementations.
Let's inventory what is covered by the model at this point and whether
there are future merges required to be full.
*1. UIs*
1.1. What sort of validation is being done on any accepted user input?
(pointers to code would be appreciated)
1.2. What explicit protections have been built in for (pointers to code
would be appreciated):
1.2.1. cross site scripting
1.2.2. cross site request forgery
1.2.3. click jacking (X-Frame-Options)
1.3. What sort of authentication is required for access to the UIs?
1.3.1. Kerberos
1.3.1.1. has TGT renewal been accounted for
1.3.1.2. SPNEGO support?
1.3.1.3. Delegation token?
1.3.2. Proxy User ACL?
1.4. What authorization is available for determining who can access what
capabilities of the UIs for either viewing, modifying data and/or related
processes?
1.5. Is there any input that will ultimately be persisted in configuration
for executing shell commands or processes?
1.6. Do the UIs support the trusted proxy pattern with doas impersonation?
1.7. Is there TLS/SSL support?
*2. REST APIs*
2.1. Do the REST APIs support the trusted proxy pattern with doas
impersonation capabilities?
2.2. What explicit protections have been built in for:
2.2.1. cross site scripting (XSS)
2.2.2. cross site request forgery (CSRF)
2.2.3. XML External Entity (XXE)
2.3. What is being used for authentication - Hadoop Auth Module?
2.4. Are there separate processes for the HTTP resources (UIs and REST
endpoints) or are they part of existing processes?
2.5. Is there TLS/SSL support?
2.6. Are there new CLI commands and/or clients for accessing the REST APIs?
2.7. What authorization enforcement points are there within the REST APIs?
*3. Encryption*
3.1. Is there any support for encryption of persisted data?
3.2. If so, is KMS and the hadoop key command used for key management?
3.3. KMS interaction with Proxy Users?
*4. Configuration*
4.1. Are there any passwords or secrets being added to configuration?
4.2. If so, are they accessed via Configuration.getPassword() to allow for
provisioning to credential providers?
4.3. Are there any settings that are used to launch docker containers or
shell out command execution, etc?
*5. HA*
5.1. Are there provisions for HA?
5.2. Are there any single point of failures?
*6. CVEs*
Dependencies need to have been checked for known issues before we merge.
We don't however want to list any CVEs that have been fixed but not
released yet.
6.1. All dependencies checked for CVEs?
On Sat, Oct 21, 2017 at 10:26 AM, larry mccay <lmc...@apache.org> wrote:
> Hi Marton -
>
> I don't think there is any denying that it would be great to have such
> documentation for all of those reasons.
> If it is a natural extension of getting the checklist information as an
> assertion of security state when merging then we can certainly include it.
>
> I think that backfilling all such information across the project is a
> different topic altogether and wouldn't want to expand the scope of this
> discussion in that direction.
>
> Thanks for the great thoughts on this!
>
> thanks,
>
> --larry
>
>
>
>
>
> On Sat, Oct 21, 2017 at 3:00 AM, Elek, Marton <h...@anzix.net> wrote:
>
>>
>>
>> On 10/21/2017 02:41 AM, larry mccay wrote:
>>
>>>
>>> "We might want to start a security section for Hadoop wiki for each of
>>>> the
>>>> services and components.
>>>> This helps to track what has been completed."
>>>>
>>>
>>> Do you mean to keep the audit checklist for each service and component
>>> there?
>>> Interesting idea, I wonder what sort of maintenance that implies and
>>> whether we want to take on that burden even though it would be great
>>> information to have for future reviewers.
>>>
>>
>> I think we should care about the maintenance of the documentation anyway.
>> We also need to maintain all the other documentations. I think it could be
>> even part of the generated docs and not the wiki.
>>
>> I also suggest to fill this list about the current trunk/3.0 as a first
>> step.
>>
>> 1. It would be a very usefull documentation for the end-users (some
>> answers could link the existing documentation, it exists, but I am not sure
>> if all the answers are in the current documentation.)
>>
>> 2. It would be a good example who the questions could be answered.
>>
>> 3. It would help to check, if something is missing from the list.
>>
>> 4. There are future branches where some of the components are not
>> touched. For example, no web ui or no REST service. A prefilled list could
>> help to check if the branch doesn't break any old security functionality on
>> trunk.
>>
>> 5. It helps to document the security features in one place. If we have a
>> list for the existing functionality in the same format, it would be easy to
>> merge the new documentation of the new features as they will be reported in
>> the same form. (So it won't be so hard to maintain the list...).
>>
>> Marton
>>
>
>
