On Wed, Jul 16, 2014 at 11:29 AM, Henry Saputra <henry.sapu...@gmail.com>
wrote:
> HI,
>
> Just curious if you could give reference to "Apache encourages
> disjoint teams to form independent projects, even when those projects
> overlap in scope" statement ?
>
Henry,
It generally comes from the "Community over Code," which is a phrase
discussing how the focus is on how the people working on a project work
together and not the code. The assumption is teams with healthy communities
will build good code.
In terms of documentation, it like most of Apache is learned by doing. In
Nick Burch's talk "The Apache Way" at ApacheCon this year (
http://events.linuxfoundation.org/sites/events/files/slides/TheApacheWay14.pdf)
there was a slide about this:
Don't pick winners, pick runners
• Board doesn't say “We want X”
• Developers say “X is cool”
• We enable developers to do cool stuff
• Apache developers are at the forefront of innovation
• Not interested in a single runner
• We want relay teams
• Community is critical to the Apache Way
• Apache is about supporting communities
.. Owen
> - Henry
>
> On Mon, Jul 14, 2014 at 7:16 PM, Selvamohan Neethiraj
> <sneethi...@hortonworks.com> wrote:
> > Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal)
> >
> > == Abstract ==
> >
> > Argus is a framework to enable, monitor and manage comprehensive data
> security across the Hadoop platform.
> >
> > The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in
> Greek mythology, endowed with a role to keep “an eye” open and be an
> effective watchman at all times.
> >
> > == Background ==
> >
> > The vision with Argus is to provide comprehensive security across the
> Apache Hadoop ecosystem. With the advent of Apache YARN, the Hadoop
> platform can now support a true data lake architecture. Enterprises can
> potentially run multiple workloads, in a multi tenant environment. Data
> security within Hadoop needs to evolve to support multiple use cases for
> data access, while also providing a framework for central administration of
> security policies and monitoring of user access.
> >
> > XA Secure, a Hadoop security focused startup, developed the initial
> technology behind Argus. XA Secure was acquired by Hortonworks, which now
> is contributing the technology to the open source community to extend and
> innovate.
> >
> > == Rationale ==
> >
> > Many of the projects in the Hadoop ecosystem have their own
> authentication, authorization, and auditing components. There are no
> central administration and auditing capabilities. We are looking to address
> these enterprises security needs of central administration and
> comprehensive security through the Argus project.
> > Our initial focus would be around authorization and auditing, the longer
> term vision would be to tie all aspects around data security within the
> Hadoop platform.
> >
> > == Proposal Details ==
> >
> > The vision of Argus is to enable comprehensive data security across the
> Hadoop platform. The goal is provide a single user interface or API to
> manage security policies, monitor user access and policy changes history.
> The framework would work with individual components in enforcing these
> policies and in capturing relevant audit information.
> > Initial Goals
> > 1. Donate the Argus source code and documentation to the
> Apache Software Foundation
> > 2. Setup and standardize the open governance of the Argus
> project
> > 3. Build a user and developer community
> > 4. Deeper Integration with Hadoop Platform
> > a. Enable integration with Apache Storm, Apache
> Knox and Apache Falcon for authorization and auditing
> > 5. Configurable centralized storage of audit data into HDFS
> > 6. Enable framework to be run in both Linux and Windows
> environments
> > 7. Rationalize install procedure, making it easier for
> enterprises to deploy
> >
> > == Longer Term Goals ==
> >
> > In longer term, Argus should provide a comprehensive security framework
> for Hadoop platform components, covering the following
> > 1. Centralized security administration to manage all
> security related tasks in a central UI
> > 2. Fine grained authorization to do a specific action
> and/or operation with Hadoop component/tool and managed through a central
> administration tool
> > a. Standardize authorization method across all
> Hadoop components
> > b. Enhanced support for different authorization
> methods - Role based access control, attribute based access control etc
> > c. Enable tag based global policies
> > 3. Centralize auditing of user access and administrative
> actions (security related) within all the components of Hadoop
> >
> > == Current Status ==
> >
> > Argus’ technology is currently being used by enterprises and is under
> active development.
> >
> > The key components of Argus are:
> > • Enterprise Security Administration Portal
> > ◦ A Java Web Application, designed for
> administration of security policies from a single location for the entire
> hadoop cluster (and even multiple hadoop clusters)
> > • Security Agents
> > ◦ A light-weight Java Agent, which will be
> embedded into the hadoop component (e.g. Hive, HBase and Hadoop) as an
> authorization provider to enforce the security policies and also collect
> access events/logs.
> > • User/Group Synchronizer Module
> > ◦ A standalone daemon which allows the user/group
> information to be synched from the enterprise user repositories like
> LDAP/AD to Argus local database. This user/group information in Argus local
> database will help the security policy administrators
> > ▪ to define security policies by
> selecting users/groups from a drop-down box (instead of typing their
> name/group in a text-box).
> > ▪ to delegate policy administration to
> other users/groups
> > ▪ to restrict view of reports based on the
> users/groups
> > • Centralized Audit Logs and Monitoring
> > ◦ Log events to central data storage/database
> > ◦ Interactive query of audit events
> > ◦ Audit administrator actions
> >
> >
> > The initial version provides ability to
> > 1. Define security policies using a central security
> administration UI.
> > 2. Fine grained access control for HDFS (file level) , Hive
> (column level) and HBase (column level)
> > 3. Framework to record access/operational events/logs as
> part of auditing and view using a central monitoring UI.
> > 4. Support delegated policy administration
> > 5. Centralized audit monitoring and query capabilities
> >
> > === Meritocracy ===
> >
> > We plan to invest in supporting a meritocracy. We will discuss the
> requirements in an open forum. Several companies have already expressed
> interest in this project, and we intend to invite additional developers to
> participate. We will encourage and monitor community participation so that
> privileges can be extended to those that contribute.
> >
> > === Community ===
> >
> > We are happy to report that there are existing Apache committers and
> corporate users who are closely involved in the project already. We hope to
> extend the user and developer base further in the future and build a solid
> open source community around Argus, growing the community and adding
> committers following the Apache meritocracy model.
> >
> > === Core Developers ===
> >
> > The initial technology within Argus was originally built by the team at
> XA Secure. XA Secure was founded and managed by experienced members with a
> wide background in enterprise security. Some of the XA’s core team have
> been proposed as core developers for this project. The developer list also
> include an Apache member and PMC members from several Apache projects
> (Hadoop, HBase, and Knox). A concern is that all of the core developers are
> employed by Hortonworks and thus an emphasis will be on increasing the
> diversity of the developer community.
> > Alignment
> > The initial committers strongly believe that a unified security portal
> for Apache Hadoop, Hive, and HBase will gain broad adoption as an open
> source, community driven project. Our hope is that the Apache Falcon,
> Apache Storm, Apache Knox, and other communities will find tremendous
> value in Argus and will adopt it en masse.
> >
> > == Known Risks ==
> >
> > === Orphaned Products ===
> >
> > The initial code behind Argus is under active development and is being
> actively used by several enterprises. It is not expected to be orphaned.
> >
> > === Inexperience with Open Source ===
> >
> > Many of the core developers have long-standing experience in open
> source, Dili Aramugam, Kevin Minder and Larry McCay are committers on the
> Apache Knox project. Sanjay Radia and Owen O’Malley are PMC members on
> several Apache projects. We have several mentors that will work with the
> inexperienced committers on building a thriving developer community.
> >
> > === Homogeneous Developers ===
> > The current core developers are all from Hortonworks. However, we expect
> to establish a thriving developer community that includes users of Argus
> and developers of other Hadoop components.
> >
> > === Reliance on Salaried Developers ===
> >
> > Currently, all of the developers are paid to work on Argus. A key goal
> for the incubation process will be to broaden the developer base.
> > Relationships with Other Apache Products
> > The biggest risk is fast rate of growth of new features within the
> Hadoop ecosystem and security standards not being applied during the
> initial development of these new products. We believe an active engagement
> from the Hadoop community would significantly aid adoption of common
> security framework across the ecosystem and will help in establishing cross
> component standards.
> >
> > As mentioned in the Alignment section, Argus is closely integrated with
> Hadoop, Hive and HBase in a numerous ways. We look forward to collaborating
> with those communities, as well as other Apache communities.
> >
> > There is some overlap between the goals of Argus and Apache Sentry.
> Apache encourages disjoint teams to form independent projects, even when
> those projects overlap in scope. Additionally, we feel that the distinct
> code bases, development teams, and different approaches to the problem
> should be represented by different projects. This will provide better
> choices for users to choose from.
> >
> > === An Excessive Fascination with the Apache Brand ===
> >
> > While we respect the reputation of the Apache brand and have no doubts
> that it will attract contributors and users, our interest is primarily to
> give Argus a solid home as an open source project with a broad developer
> base and to encourage adoption by the related ASF projects and foster
> innovation around security
> >
> > == Documentation ==
> >
> http://hortonworks.com/blog/hortonworks-acquires-xasecure-to-provide-comprehensive-security-for-enterprise-hadoop/
> >
> > == Initial Source ==
> >
> > We will make the initial source available as a patch.
> >
> > == Source and IP Submission Plan ==
> > 1. All source will be moved to Apache Infrastructure
> > 2. All outstanding issues in our in-house JIRA infrastructure will
> be replicated into the Apache JIRA system.
> > 3. We will be acquiring a twitter handle for project Argus (eg:
> @apacheargus )
> >
> > == External Dependencies ==
> >
> > Argus has no external dependencies except for some Java libraries that
> are considered ASF-compatible (JUnit, SLF4J, …) and Apache artifacts :
> Hadoop, Log4J and the transient dependencies of all these artifacts.
> >
> > == Cryptography ==
> >
> > Argus does not incorporate encryption currently.
> >
> > == Required Resources ==
> >
> > === Mailing Lists: ===
> > 1. argus-dev
> > 2. argus-commits
> > 3. argus-private
> >
> > === Infrastructure: ===
> > 1. Git repository
> > 2. JIRA Argus
> > 3. Gerrit for reviewing patches
> > The existing code includes local host integration tests, so we would
> like a Jenkins instance to run them whenever a new patch is submitted.
> >
> > == Initial Committers ==
> >
> > * Balaji Ganesan (bganesan at hortonworks.com)
> > * Dilli Arumugam (darumugam at hortonworks.com)
> > * Don Bosco Durai (bdurai at hortonworks.com)
> > * Kevin Minder (kminder at apache.org)
> > * Larry McCay (lmccay at apache.org)
> > * Madhanmohan Neethiraj (mneethiraj at hortonworks.com)
> > * Owen O’Malley (omalley at apache.org)
> > * Ramesh Mani (rmani at hortonworks.com)
> > * Sanjay Radia (sradia at apache.org)
> > * Selvamohan Neethiraj (sneethiraj at hortonworks.com)
> >
> > == Affiliations ==
> >
> > * Balaji Ganesan - Hortonworks
> > * Dilli Arumugam - Hortonworks
> > * Don Bosco Durai - Hortonworks
> > * Kevin Minder - Hortonworks
> > * Larry McCay - Hortonworks
> > * Madhanmohan Neethiraj - Hortonworks
> > * Owen O’Malley - Hortonworks
> > * Ramesh Mani - Hortonworks
> > * Sanjay Radia - Hortonworks
> > * Selvamohan Neethiraj - Hortonworks
> >
> > == Sponsors ==
> >
> > === Champion: ===
> >
> > * Owen O’Malley (omalley at apache.org) - Hortonworks
> >
> > === Nominated Mentors: ===
> >
> > * Alan Gates - Hortonworks
> > * Devaraj Das - Hortonworks
> > * Jakob Homan - LinkedIn
> > * Owen O’Malley - Hortonworks
> >
> > === Sponsoring Entity ===
> >
> > Incubator PMC
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > For additional commands, e-mail: general-h...@incubator.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>
