Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Konstantin Boudnik
If an objective of a podding is to build a diversified community then perhaps
it should be an explicitly defined graduation criteria? In fact, earlier I have
raised exactly the same question about Tez but I don't think it ever been
satisfactory answered.

As for chances of a project to stick with github if it was started their - I
respectfully disagree. I think only projects which feel comfortable within
the github will stay there. Others will move on: Apache Spark is a great
example of late. After 2 years of organic growth @github they came to the
Incubator and graduated very fast with flying colors. Without stacking the
PPMC IIRC.

So, I am fond of Henry's hope that the project will solicit a more diversified
set of mentors and initial committers.

--
  Cos

On Tue, Jul 15, 2014 at 02:38PM, Henry Saputra wrote:
 Hi Andrew, thanks for chiming in =)
 
 I like your and Owen's opinion about initial members of an incubator project.
 And also agree it is responsibility of the polling to achieve better
 diversity during its time under Apache incubator.
 
 For this particular proposal, however, the mentors mostly coming from
 same organization.
 Hopefully with this proposal announcement, the project could solicit
 mentors from different organizations to help add check and balance.
 
 - Henry
 
 On Tue, Jul 15, 2014 at 2:22 PM, Andrew Purtell apurt...@apache.org wrote:
  I had started typing up a response to Henry's mail but will discard the
  beginning of it to say I agree with Owen. A new project coming into the
  incubator quite naturally could have the initial set of committers entirely
  from one organization. An organization donating an existing code base, for
  example.
 
  However, there has been recent discussion elsewhere that the Incubator
  should more closely consider if an incubating project has succeeded to grow
  a community beyond the initially limited group before declaring a project
  ready for graduation. (Refer to the discussion on the graduation of Apache
  Tez.) This position seems reasonable, and should naturally apply here. If a
  project exits graduation with the same lack of PMC/committer diversity with
  which it entered, this is in effect Apache-washing, in my opinion. A
  stacked PMC is no more open a community then one controlled by a BDFL and
  hosted on GitHub.
 
 
 
  On Tue, Jul 15, 2014 at 2:14 PM, Owen O'Malley omal...@apache.org wrote:
 
  On Tue, Jul 15, 2014 at 1:59 PM, Henry Saputra henry.sapu...@gmail.com
  wrote:
 
   Maybe we should start asking incubator project to try to build some
   kind of momentum or community before going to ASF incubator.
  
 
  Apache incubator is a great place for new projects to grow their community
  and diversity. Once a project is established it is much harder to change
  the infrastructure and there projects started on github are more likely to
  stay on github. I think there is significant value in these projects being
  done in the Apache Way.
 
 
  
   All but one PPMC members for this proposal would be from Hortonworks.
  
 
  Would you like to volunteer to help mentor?
 
 
   Just from high level it seems like it has similar goal as Apache Knox
   [1], what is the differences between the 2?
  
 
  I think Knox and Argus complement each other. Knox provides perimeter
  security via proxy/gateway services, while Argus is providing fine grain
  integrated authorization and auditing.
 
  .. Owen
 
 
 
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. - Piet Hein
  (via Tom White)
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [DISCUSS] Incubator exit criteria

2014-07-16 Thread ant elder
On Wed, Jul 16, 2014 at 5:16 AM, Roman Shaposhnik ro...@shaposhnik.org
wrote:

 On Sat, Jul 12, 2014 at 4:15 AM, Christian Grobmeier
 grobme...@gmail.com wrote:
  On 12 Jul 2014, at 8:05, Roman Shaposhnik wrote:
  That's actually the part of the thread that I have a lot of interest in.
  Is there any reason not to use attic for hibernated podlings?
 
  I am not sure if there is a real reason, but maybe its because the attic
  currently contains only real ASF projects where all legal things were
  sorted out.
 
  With a podling coming in this might not be the case. We would need to
  check back with the Attic people if they can handle this.
 
  Otherwise I don't see a reason why we shouldn't use the attic, imho
  its the reason why its there.

 Quick question: who's in charge of saying 'yay' or 'nay' to Incubator
 projects coming to the Attic?

  Roman, would you mind to create an update patch and send it around
  here for some kind of discussing/voting? Then we would see how people
 feel.
  I think this thread is already to big to get the right attention.

 Will do shortly. Btw, I presume you're talking about the patch
 to the Incubator web pages documenting the policy, right?

 At least that's what I'm going to patch and send for a VOTE



What happpened to you agreeing to try it with a few of the old podlings
first?

   ...ant


Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Jean-Baptiste Onofré

Hi,

it looks interesting.

Do you have an idea about the interactions with other projects (Knox, 
Shiro, Syncope, whatever) ?


Regards
JB

On 07/15/2014 04:16 AM, Selvamohan Neethiraj wrote:

Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal)

== Abstract ==

Argus is a framework to enable, monitor and manage comprehensive data security 
across the Hadoop platform.

The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in Greek 
mythology, endowed with a role to keep “an eye” open and be an effective 
watchman at all times.

== Background ==

The vision with Argus is to provide comprehensive security across the Apache 
Hadoop ecosystem. With the advent of  Apache YARN, the Hadoop platform can now 
support a true data lake architecture. Enterprises can potentially run multiple 
workloads, in a multi tenant environment. Data security within Hadoop needs to 
evolve to support multiple use cases for data access, while also providing a 
framework for central administration of security policies and monitoring of 
user access.

XA Secure, a Hadoop security focused startup, developed the initial technology 
behind Argus. XA Secure was acquired by Hortonworks, which now is contributing 
the technology to the open source community to extend and innovate.

== Rationale ==

Many of the projects in the Hadoop ecosystem have their own authentication, 
authorization, and auditing components. There are no central administration and 
auditing capabilities. We are looking to address these enterprises security 
needs of central administration and comprehensive security through the Argus 
project.
Our initial focus would be around authorization and auditing, the longer term 
vision would be to tie all aspects around data security within the Hadoop 
platform.

== Proposal Details ==

The vision of Argus is to enable comprehensive data security across the Hadoop 
platform. The goal is provide a single user interface or API to manage security 
policies, monitor user access and policy changes history. The framework would 
work with individual components in enforcing these policies and in capturing 
relevant audit information.
Initial Goals
1.  Donate the Argus source code and documentation to the Apache 
Software Foundation
2.  Setup and standardize the open governance of the Argus project
3.  Build a user and developer community
4.  Deeper Integration with Hadoop Platform
a.  Enable integration with Apache Storm, Apache Knox and 
Apache Falcon for authorization and auditing
5.  Configurable centralized storage of audit data into HDFS
6.  Enable framework to be run in both Linux and Windows 
environments
7.  Rationalize install procedure, making it easier for enterprises 
to deploy

== Longer Term Goals ==

In longer term, Argus should provide a comprehensive security framework for 
Hadoop platform components, covering the following
1.  Centralized security administration to manage all security 
related tasks in a central UI
2.  Fine grained authorization to do a specific action and/or 
operation with Hadoop component/tool and managed through a central 
administration tool
a.  Standardize authorization method across all Hadoop 
components
b.  Enhanced support for different authorization methods - 
Role based access control, attribute based access control etc
c.  Enable tag based global policies
3.  Centralize auditing of user access and administrative actions 
(security related) within all the components of Hadoop

== Current Status ==

Argus’ technology is currently being used by enterprises and is under active 
development.

The key components of Argus are:
•   Enterprise Security Administration Portal
◦   A Java Web Application, designed for administration of 
security policies from a single location for the entire hadoop cluster (and 
even multiple hadoop clusters)
•   Security Agents
◦   A light-weight Java Agent, which will be embedded into 
the hadoop component (e.g. Hive, HBase and Hadoop) as an authorization provider 
to enforce the security policies and also collect access events/logs.
•   User/Group Synchronizer Module
◦   A standalone daemon which allows the user/group 
information to be synched from the enterprise user repositories like LDAP/AD to 
Argus local database. This user/group information in Argus local database will 
help the security policy administrators
▪   to define security policies by  selecting 
users/groups from a drop-down box (instead of typing their name/group in a 
text-box).
▪   to delegate policy administration to other 
users/groups
▪   to restrict view of reports based on the 

Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Chris Douglas
On Tue, Jul 15, 2014 at 11:30 PM, Konstantin Boudnik c...@apache.org wrote:
 If an objective of a podding is to build a diversified community then perhaps
 it should be an explicitly defined graduation criteria? In fact, earlier I 
 have
 raised exactly the same question about Tez but I don't think it ever been
 satisfactory answered.

The incubator is a curriculum. What would the Tez project learn from
continued incubation? To cynically add token contributors to ward
off criticism? We shouldn't create incentives that reward faking
community over effecting it, because the former is much easier.

 So, I am fond of Henry's hope that the project will solicit a more diversified
 set of mentors and initial committers.

The committer and PPMC list should be honest. If there is no
diversity, that fact must be visible for it to be tracked.

Initial committers are a dodge that demonstrates nothing about the
project's willingness to not only accept, but attract new ideas. The
viability of an OSS project is not inaccurately *defined* as its
ability to compete for that attention. Demanding pro forma adherence
to heuristics is a harmful lesson to podlings, unless the goal is to
teach them to evade accountability. -C

 On Tue, Jul 15, 2014 at 02:38PM, Henry Saputra wrote:
 Hi Andrew, thanks for chiming in =)

 I like your and Owen's opinion about initial members of an incubator project.
 And also agree it is responsibility of the polling to achieve better
 diversity during its time under Apache incubator.

 For this particular proposal, however, the mentors mostly coming from
 same organization.
 Hopefully with this proposal announcement, the project could solicit
 mentors from different organizations to help add check and balance.

 - Henry

 On Tue, Jul 15, 2014 at 2:22 PM, Andrew Purtell apurt...@apache.org wrote:
  I had started typing up a response to Henry's mail but will discard the
  beginning of it to say I agree with Owen. A new project coming into the
  incubator quite naturally could have the initial set of committers entirely
  from one organization. An organization donating an existing code base, for
  example.
 
  However, there has been recent discussion elsewhere that the Incubator
  should more closely consider if an incubating project has succeeded to grow
  a community beyond the initially limited group before declaring a project
  ready for graduation. (Refer to the discussion on the graduation of Apache
  Tez.) This position seems reasonable, and should naturally apply here. If a
  project exits graduation with the same lack of PMC/committer diversity with
  which it entered, this is in effect Apache-washing, in my opinion. A
  stacked PMC is no more open a community then one controlled by a BDFL and
  hosted on GitHub.
 
 
 
  On Tue, Jul 15, 2014 at 2:14 PM, Owen O'Malley omal...@apache.org wrote:
 
  On Tue, Jul 15, 2014 at 1:59 PM, Henry Saputra henry.sapu...@gmail.com
  wrote:
 
   Maybe we should start asking incubator project to try to build some
   kind of momentum or community before going to ASF incubator.
  
 
  Apache incubator is a great place for new projects to grow their community
  and diversity. Once a project is established it is much harder to change
  the infrastructure and there projects started on github are more likely to
  stay on github. I think there is significant value in these projects being
  done in the Apache Way.
 
 
  
   All but one PPMC members for this proposal would be from Hortonworks.
  
 
  Would you like to volunteer to help mentor?
 
 
   Just from high level it seems like it has similar goal as Apache Knox
   [1], what is the differences between the 2?
  
 
  I think Knox and Argus complement each other. Knox provides perimeter
  security via proxy/gateway services, while Argus is providing fine grain
  integrated authorization and auditing.
 
  .. Owen
 
 
 
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. - Piet Hein
  (via Tom White)

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Ted Dunning
On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org wrote:

  So, I am fond of Henry's hope that the project will solicit a more
 diversified
  set of mentors and initial committers.

 The committer and PPMC list should be honest. If there is no
 diversity, that fact must be visible for it to be tracked.

 Initial committers are a dodge that demonstrates nothing about the
 project's willingness to not only accept, but attract new ideas. The
 viability of an OSS project is not inaccurately *defined* as its
 ability to compete for that attention. Demanding pro forma adherence
 to heuristics is a harmful lesson to podlings, unless the goal is to
 teach them to evade accountability. -C


+1 to this.

Sham committers are more of a pain than lack of diversity, as much as I
dislike sock-puppet projects.


Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Henry Saputra
HI,

Just curious if you could give reference to Apache encourages
disjoint teams to form independent projects, even when those projects
overlap in scope statement ?

- Henry

On Mon, Jul 14, 2014 at 7:16 PM, Selvamohan Neethiraj
sneethi...@hortonworks.com wrote:
 Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal)

 == Abstract ==

 Argus is a framework to enable, monitor and manage comprehensive data 
 security across the Hadoop platform.

 The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in Greek 
 mythology, endowed with a role to keep “an eye” open and be an effective 
 watchman at all times.

 == Background ==

 The vision with Argus is to provide comprehensive security across the Apache 
 Hadoop ecosystem. With the advent of  Apache YARN, the Hadoop platform can 
 now support a true data lake architecture. Enterprises can potentially run 
 multiple workloads, in a multi tenant environment. Data security within 
 Hadoop needs to evolve to support multiple use cases for data access, while 
 also providing a framework for central administration of security policies 
 and monitoring of user access.

 XA Secure, a Hadoop security focused startup, developed the initial 
 technology behind Argus. XA Secure was acquired by Hortonworks, which now is 
 contributing the technology to the open source community to extend and 
 innovate.

 == Rationale ==

 Many of the projects in the Hadoop ecosystem have their own authentication, 
 authorization, and auditing components. There are no central administration 
 and auditing capabilities. We are looking to address these enterprises 
 security needs of central administration and comprehensive security through 
 the Argus project.
 Our initial focus would be around authorization and auditing, the longer term 
 vision would be to tie all aspects around data security within the Hadoop 
 platform.

 == Proposal Details ==

 The vision of Argus is to enable comprehensive data security across the 
 Hadoop platform. The goal is provide a single user interface or API to manage 
 security policies, monitor user access and policy changes history. The 
 framework would work with individual components in enforcing these policies 
 and in capturing relevant audit information.
 Initial Goals
 1.  Donate the Argus source code and documentation to the Apache 
 Software Foundation
 2.  Setup and standardize the open governance of the Argus project
 3.  Build a user and developer community
 4.  Deeper Integration with Hadoop Platform
 a.  Enable integration with Apache Storm, Apache Knox and 
 Apache Falcon for authorization and auditing
 5.  Configurable centralized storage of audit data into HDFS
 6.  Enable framework to be run in both Linux and Windows 
 environments
 7.  Rationalize install procedure, making it easier for 
 enterprises to deploy

 == Longer Term Goals ==

 In longer term, Argus should provide a comprehensive security framework for 
 Hadoop platform components, covering the following
 1.  Centralized security administration to manage all security 
 related tasks in a central UI
 2.  Fine grained authorization to do a specific action and/or 
 operation with Hadoop component/tool and managed through a central 
 administration tool
 a.  Standardize authorization method across all Hadoop 
 components
 b.  Enhanced support for different authorization methods 
 - Role based access control, attribute based access control etc
 c.  Enable tag based global policies
 3.  Centralize auditing of user access and administrative actions 
 (security related) within all the components of Hadoop

 == Current Status ==

 Argus’ technology is currently being used by enterprises and is under active 
 development.

 The key components of Argus are:
 •   Enterprise Security Administration Portal
 ◦   A Java Web Application, designed for administration 
 of security policies from a single location for the entire hadoop cluster 
 (and even multiple hadoop clusters)
 •   Security Agents
 ◦   A light-weight Java Agent, which will be embedded 
 into the hadoop component (e.g. Hive, HBase and Hadoop) as an authorization 
 provider to enforce the security policies and also collect access events/logs.
 •   User/Group Synchronizer Module
 ◦   A standalone daemon which allows the user/group 
 information to be synched from the enterprise user repositories like LDAP/AD 
 to Argus local database. This user/group information in Argus local database 
 will help the security policy administrators
 ▪   to define security policies by  selecting 
 users/groups from a drop-down box (instead of typing their name/group in a 
 text-box).
  

Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Henry Saputra
Definitely +1, active members, with lack diversity, are more useful.

My original comment was to suggest having mentors come from different
organizations so hopefully could help provide different perspectives
and inputs to the podling.
Never my intention to suggest false diversity to start an Apache
incubator project.

- Henry


On Wed, Jul 16, 2014 at 9:15 AM, Ted Dunning ted.dunn...@gmail.com wrote:
 On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org wrote:

  So, I am fond of Henry's hope that the project will solicit a more
 diversified
  set of mentors and initial committers.

 The committer and PPMC list should be honest. If there is no
 diversity, that fact must be visible for it to be tracked.

 Initial committers are a dodge that demonstrates nothing about the
 project's willingness to not only accept, but attract new ideas. The
 viability of an OSS project is not inaccurately *defined* as its
 ability to compete for that attention. Demanding pro forma adherence
 to heuristics is a harmful lesson to podlings, unless the goal is to
 teach them to evade accountability. -C


 +1 to this.

 Sham committers are more of a pain than lack of diversity, as much as I
 dislike sock-puppet projects.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Andreas Neumann
Is it worth a thought to require that new podlings find a champion who is
not employed by the same organization?


On Wed, Jul 16, 2014 at 11:53 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 Definitely +1, active members, with lack diversity, are more useful.

 My original comment was to suggest having mentors come from different
 organizations so hopefully could help provide different perspectives
 and inputs to the podling.
 Never my intention to suggest false diversity to start an Apache
 incubator project.

 - Henry


 On Wed, Jul 16, 2014 at 9:15 AM, Ted Dunning ted.dunn...@gmail.com
 wrote:
  On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org
 wrote:
 
   So, I am fond of Henry's hope that the project will solicit a more
  diversified
   set of mentors and initial committers.
 
  The committer and PPMC list should be honest. If there is no
  diversity, that fact must be visible for it to be tracked.
 
  Initial committers are a dodge that demonstrates nothing about the
  project's willingness to not only accept, but attract new ideas. The
  viability of an OSS project is not inaccurately *defined* as its
  ability to compete for that attention. Demanding pro forma adherence
  to heuristics is a harmful lesson to podlings, unless the goal is to
  teach them to evade accountability. -C
 
 
  +1 to this.
 
  Sham committers are more of a pain than lack of diversity, as much as I
  dislike sock-puppet projects.

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Alejandro Abdelnur
 There is some overlap between the goals of Argus and Apache
 Sentry. Apache encourages disjoint teams to form independent
 projects, even when those projects overlap in scope. Additionally,
 we feel that the distinct code bases, development teams, and
 different approaches to the problem should be represented by
 different projects. This will provide better choices for users to
 choose from.

Approaches may be different, though I wouldn't think they are
irreconcilable and we could assume that the end result would be the best of
both, thus being a better choice.

Have you considered reaching out to the Sentry community to see if they
would be interested in bringing Argus existing code base into Sentry?

If you are interested, I'd be happy to spend some cycles to see if this
would be possible.

Cheers.



On Wed, Jul 16, 2014 at 11:53 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 Definitely +1, active members, with lack diversity, are more useful.

 My original comment was to suggest having mentors come from different
 organizations so hopefully could help provide different perspectives
 and inputs to the podling.
 Never my intention to suggest false diversity to start an Apache
 incubator project.

 - Henry


 On Wed, Jul 16, 2014 at 9:15 AM, Ted Dunning ted.dunn...@gmail.com
 wrote:
  On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org
 wrote:
 
   So, I am fond of Henry's hope that the project will solicit a more
  diversified
   set of mentors and initial committers.
 
  The committer and PPMC list should be honest. If there is no
  diversity, that fact must be visible for it to be tracked.
 
  Initial committers are a dodge that demonstrates nothing about the
  project's willingness to not only accept, but attract new ideas. The
  viability of an OSS project is not inaccurately *defined* as its
  ability to compete for that attention. Demanding pro forma adherence
  to heuristics is a harmful lesson to podlings, unless the goal is to
  teach them to evade accountability. -C
 
 
  +1 to this.
 
  Sham committers are more of a pain than lack of diversity, as much as I
  dislike sock-puppet projects.

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




-- 
Alejandro


Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Owen O'Malley
On Wed, Jul 16, 2014 at 11:29 AM, Henry Saputra henry.sapu...@gmail.com
wrote:

 HI,

 Just curious if you could give reference to Apache encourages
 disjoint teams to form independent projects, even when those projects
 overlap in scope statement ?


Henry,
It generally comes from the Community over Code, which is a phrase
discussing how the focus is on how the people working on a project work
together and not the code. The assumption is teams with healthy communities
will build good code.

In terms of documentation, it like most of Apache is learned by doing. In
Nick Burch's talk The Apache Way at ApacheCon this year (
http://events.linuxfoundation.org/sites/events/files/slides/TheApacheWay14.pdf)
there was a slide about this:

Don't pick winners, pick runners
• Board doesn't say “We want X”
• Developers say “X is cool”
• We enable developers to do cool stuff
• Apache developers are at the forefront of innovation
• Not interested in a single runner
• We want relay teams
• Community is critical to the Apache Way
• Apache is about supporting communities

.. Owen


 - Henry

 On Mon, Jul 14, 2014 at 7:16 PM, Selvamohan Neethiraj
 sneethi...@hortonworks.com wrote:
  Apache Argus Proposal (http://wiki.apache.org/incubator/ArgusProposal)
 
  == Abstract ==
 
  Argus is a framework to enable, monitor and manage comprehensive data
 security across the Hadoop platform.
 
  The name “Argus” is derived from Argus Panoptes, a 100-eyed giant in
 Greek mythology, endowed with a role to keep “an eye” open and be an
 effective watchman at all times.
 
  == Background ==
 
  The vision with Argus is to provide comprehensive security across the
 Apache Hadoop ecosystem. With the advent of  Apache YARN, the Hadoop
 platform can now support a true data lake architecture. Enterprises can
 potentially run multiple workloads, in a multi tenant environment. Data
 security within Hadoop needs to evolve to support multiple use cases for
 data access, while also providing a framework for central administration of
 security policies and monitoring of user access.
 
  XA Secure, a Hadoop security focused startup, developed the initial
 technology behind Argus. XA Secure was acquired by Hortonworks, which now
 is contributing the technology to the open source community to extend and
 innovate.
 
  == Rationale ==
 
  Many of the projects in the Hadoop ecosystem have their own
 authentication, authorization, and auditing components. There are no
 central administration and auditing capabilities. We are looking to address
 these enterprises security needs of central administration and
 comprehensive security through the Argus project.
  Our initial focus would be around authorization and auditing, the longer
 term vision would be to tie all aspects around data security within the
 Hadoop platform.
 
  == Proposal Details ==
 
  The vision of Argus is to enable comprehensive data security across the
 Hadoop platform. The goal is provide a single user interface or API to
 manage security policies, monitor user access and policy changes history.
 The framework would work with individual components in enforcing these
 policies and in capturing relevant audit information.
  Initial Goals
  1.  Donate the Argus source code and documentation to the
 Apache Software Foundation
  2.  Setup and standardize the open governance of the Argus
 project
  3.  Build a user and developer community
  4.  Deeper Integration with Hadoop Platform
  a.  Enable integration with Apache Storm, Apache
 Knox and Apache Falcon for authorization and auditing
  5.  Configurable centralized storage of audit data into HDFS
  6.  Enable framework to be run in both Linux and Windows
 environments
  7.  Rationalize install procedure, making it easier for
 enterprises to deploy
 
  == Longer Term Goals ==
 
  In longer term, Argus should provide a comprehensive security framework
 for Hadoop platform components, covering the following
  1.  Centralized security administration to manage all
 security related tasks in a central UI
  2.  Fine grained authorization to do a specific action
 and/or operation with Hadoop component/tool and managed through a central
 administration tool
  a.  Standardize authorization method across all
 Hadoop components
  b.  Enhanced support for different authorization
 methods - Role based access control, attribute based access control etc
  c.  Enable tag based global policies
  3.  Centralize auditing of user access and administrative
 actions (security related) within all the components of Hadoop
 
  == Current Status ==
 
  Argus’ technology is currently being used by enterprises and is under
 active development.
 
  The key components of Argus are:
  •   Enterprise Security Administration Portal
  ◦   A 

Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Owen O'Malley
On Wed, Jul 16, 2014 at 1:20 AM, Chris Douglas cdoug...@apache.org wrote:

 The committer and PPMC list should be honest. If there is no
 diversity, that fact must be visible for it to be tracked.


a big +1

Chris, I know you are pretty overloaded at the moment with podlings, but
would
you be willing to be a mentor for Argus?

.. Owen



 Initial committers are a dodge that demonstrates nothing about the
 project's willingness to not only accept, but attract new ideas. The
 viability of an OSS project is not inaccurately *defined* as its
 ability to compete for that attention. Demanding pro forma adherence
 to heuristics is a harmful lesson to podlings, unless the goal is to
 teach them to evade accountability. -C

  On Tue, Jul 15, 2014 at 02:38PM, Henry Saputra wrote:
  Hi Andrew, thanks for chiming in =)
 
  I like your and Owen's opinion about initial members of an incubator
 project.
  And also agree it is responsibility of the polling to achieve better
  diversity during its time under Apache incubator.
 
  For this particular proposal, however, the mentors mostly coming from
  same organization.
  Hopefully with this proposal announcement, the project could solicit
  mentors from different organizations to help add check and balance.
 
  - Henry
 
  On Tue, Jul 15, 2014 at 2:22 PM, Andrew Purtell apurt...@apache.org
 wrote:
   I had started typing up a response to Henry's mail but will discard
 the
   beginning of it to say I agree with Owen. A new project coming into
 the
   incubator quite naturally could have the initial set of committers
 entirely
   from one organization. An organization donating an existing code
 base, for
   example.
  
   However, there has been recent discussion elsewhere that the Incubator
   should more closely consider if an incubating project has succeeded
 to grow
   a community beyond the initially limited group before declaring a
 project
   ready for graduation. (Refer to the discussion on the graduation of
 Apache
   Tez.) This position seems reasonable, and should naturally apply
 here. If a
   project exits graduation with the same lack of PMC/committer
 diversity with
   which it entered, this is in effect Apache-washing, in my opinion. A
   stacked PMC is no more open a community then one controlled by a BDFL
 and
   hosted on GitHub.
  
  
  
   On Tue, Jul 15, 2014 at 2:14 PM, Owen O'Malley omal...@apache.org
 wrote:
  
   On Tue, Jul 15, 2014 at 1:59 PM, Henry Saputra 
 henry.sapu...@gmail.com
   wrote:
  
Maybe we should start asking incubator project to try to build some
kind of momentum or community before going to ASF incubator.
   
  
   Apache incubator is a great place for new projects to grow their
 community
   and diversity. Once a project is established it is much harder to
 change
   the infrastructure and there projects started on github are more
 likely to
   stay on github. I think there is significant value in these projects
 being
   done in the Apache Way.
  
  
   
All but one PPMC members for this proposal would be from
 Hortonworks.
   
  
   Would you like to volunteer to help mentor?
  
  
Just from high level it seems like it has similar goal as Apache
 Knox
[1], what is the differences between the 2?
   
  
   I think Knox and Argus complement each other. Knox provides perimeter
   security via proxy/gateway services, while Argus is providing fine
 grain
   integrated authorization and auditing.
  
   .. Owen
  
  
  
  
   --
   Best regards,
  
  - Andy
  
   Problems worthy of attack prove their worth by hitting back. - Piet
 Hein
   (via Tom White)
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Owen O'Malley
On Wed, Jul 16, 2014 at 2:14 PM, Alejandro Abdelnur t...@cloudera.com
wrote:

 Have you considered reaching out to the Sentry community to see if they
 would be interested in bringing Argus existing code base into Sentry?


We did consider it, but decided that the challenge of integrating the two
distinct
code bases moving in different directions would likely create an umbrella
project.
Such umbrella projects are strongly discouraged by Apache and are
usually broken up into separate projects.

 .. Owen


Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Don Bosco Durai
Hi JB

We will be centralizing the administration and auditing for Knox. And we will 
be also standardizing the authentication for web applications for all 
components within Hadoop ecosystem, for which we might consider Shiro. I would 
like to understand more about Syncope and see how production ready it is...

The principle is to leverage existing security solutions where applicable. Even 
within Hadoop complex eco-system, each components have limited or no security 
controls. Instead of re-inventing everything, we will extend the core component 
security capabilities and add where needed. So the security is uniform, plug 
able and scalable.

Providing a layered security along with central administration and auditing 
capabilities will enhance the security, usability, enterprise integration, 
compliance, etc. which will lead to more adoption of Apache Hadoop and projects 
working within its eco system.

Regards

Bosco

`
On Jul 16, 2014, at 12:12 AM, Jean-Baptiste Onofré j...@nanthrax.net wrote:

 Hi,
 
 it looks interesting.
 
 Do you have an idea about the interactions with other projects (Knox, Shiro, 
 Syncope, whatever) ?
 
 Regards
 JB
 
 -- 
 Jean-Baptiste Onofré
 jbono...@apache.org
 http://blog.nanthrax.net
 Talend - http://www.talend.com
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Reviewing license / notice and bundled software

2014-07-16 Thread Justin Mclean
Hi,

Last few times I've reviewed LICENSE / NOTICE files in projects it ends up 
being quite difficult knowing what exactly has been bundled and exactly how 
those bits of included software are licensed. In particular some software (i.e. 
bootstap) have moved form an Apache license to an MIT one in recent times and 
it not always immediately clear which version has been bundled. 

So what the the best way for projects to indicate what versions of software 
(and what licences) have been bundled and to make reviewing LICENSE/NOTICE 
easier? IMO this helps both the incubator (more people vote/less issues get 
through) and incubating projects (less -1s due to LICENSE/NOTICE issues).

In particular bundled Apache licensed software is an issue. How do you easily 
tell the difference between a a missing entry to LICENSE (as the bundled 
software may be say BSD or MIT license) vs nothing required in LICENSE as the 
bundled software is Apache licence? In some cases searching for file headers 
can help but quite often they are missing and/or it not immediately obvious 
what terms an external projects is licensed under.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [PROPOSAL] Apache Argus Proposal

2014-07-16 Thread Andrew Purtell
This statement might not be quite right:

 Even within Hadoop complex eco-system, each components have limited or no
security controls.

How do you define the 'Hadoop complex eco-system'? If that definition
includes projects such as HBase, we have significant security controls, so
that wouldn't be a correct statement. Not sure those working on Accumulo,
or the combination of Hive+Sentry would agree with that statement either.

It's not necessary to survey the Hadoop ecosystem before incubating of
course, or even after, but it sounds like that might be a good idea.



On Wed, Jul 16, 2014 at 5:06 PM, Don Bosco Durai bdu...@hortonworks.com
wrote:

 Hi JB

 We will be centralizing the administration and auditing for Knox. And we
 will be also standardizing the authentication for web applications for all
 components within Hadoop ecosystem, for which we might consider Shiro. I
 would like to understand more about Syncope and see how production ready it
 is...

 The principle is to leverage existing security solutions where applicable.
 Even within Hadoop complex eco-system, each components have limited or no
 security controls. Instead of re-inventing everything, we will extend the
 core component security capabilities and add where needed. So the security
 is uniform, plug able and scalable.

 Providing a layered security along with central administration and
 auditing capabilities will enhance the security, usability, enterprise
 integration, compliance, etc. which will lead to more adoption of Apache
 Hadoop and projects working within its eco system.

 Regards

 Bosco

 `
 On Jul 16, 2014, at 12:12 AM, Jean-Baptiste Onofré j...@nanthrax.net
 wrote:

  Hi,
 
  it looks interesting.
 
  Do you have an idea about the interactions with other projects (Knox,
 Shiro, Syncope, whatever) ?
 
  Regards
  JB
 
  --
  Jean-Baptiste Onofré
  jbono...@apache.org
  http://blog.nanthrax.net
  Talend - http://www.talend.com
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 


 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)