[jira] [Updated] (AMBARI-15395) Enhance blueprint support for using references

2016-03-12 Thread Shantanu Mundkur (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-15395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shantanu Mundkur updated AMBARI-15395:
--
Description: 
An exported blueprint should provide ready portability i.e. an exported 
blueprint be usable without changes to deploy another cluster. Some elements 
that are masked or omitted use tokens or placeholders. These have been called 
references in previous Jiras. A reference follow some convention that indicates 
that it is a reference by using a keyword and a pattern e.g.
ReferenceName:configType:configVersion:propertyName

References would be a good indicators of properties that user could choose to 
customize before deploying the cluster. It could also indicate the need for a 
"global" default for that property in the cluster template. Examples:
- Passwords
- Hostnames 
- External databases

Currently Ambari has a concept of SECRET references. E.g.
SECRET:hive-site:2:hive.server2.keystore.password

These are used for masking the password when a blueprint is exported and the 
reference itself is not exported. It would be useful to have an reference 
exported as long as it is processed appropriately during deployment. 

Similar to the secret reference, for hostnames in a one could have,
HOST:kerberos-env:-1:kdc_host
and so forth.

For any reference, in the cluster template there would be a corresponding 
property that would be used for substituting a value for the reference during 
deployment if the registered blueprint had such references. Currently such 
behavior is used if a property of type password is not specified 
(default_password). Such references could be used to tag properties to flag 
them to be the ones that users must customize or include in the cluster 
template. It could also serve a way to annotate/comment parts of the blueprint 
JSON.

  was:
An exported blueprint should provide ready portability i.e. an exported 
blueprint be usable without changes to deploy another cluster; some elements 
that are masked or omitted could still use tokens or placeholders. These have 
been called references in previous Jiras. A reference follow some convention 
that indicates that it is a reference by using a keyword and a pattern e.g.
ReferenceName:configType:configVersion:propertyName

References would be a good indicators of properties that user could choose to 
customize before deploying the cluster. It could also indicate the need for a 
"global" default for that property in the cluster template. Examples:
Passwords
Hostnames 
External databases

Currently Ambari has a concept of SECRET references. E.g.
SECRET:hive-site:2:hive.server2.keystore.password

These are used for masking the password when a blueprint is exported and the 
reference itself is not exported. It would be useful to have an reference 
exported as long as it is processed appropriately during deployment. 

Similar to the secret reference, one could have,
HOST:kerberos-env:-1:kdc_host
and so forth.

For any reference, in the cluster template there would be a corresponding 
property that would be used for substituting a value for the reference during 
deployment if the registered blueprint had such references. Currently such 
behavior is used if a property of type password is not specified 
(default_password). Such references could be used to tag properties to flag 
them to be the ones that users must customize or include in the cluster 
template. It could also serve a way to annotate/comment parts of the blueprint 
JSON.


> Enhance blueprint support for using references
> --
>
> Key: AMBARI-15395
> URL: https://issues.apache.org/jira/browse/AMBARI-15395
> Project: Ambari
>  Issue Type: Story
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Shantanu Mundkur
>
> An exported blueprint should provide ready portability i.e. an exported 
> blueprint be usable without changes to deploy another cluster. Some elements 
> that are masked or omitted use tokens or placeholders. These have been called 
> references in previous Jiras. A reference follow some convention that 
> indicates that it is a reference by using a keyword and a pattern e.g.
> ReferenceName:configType:configVersion:propertyName
> References would be a good indicators of properties that user could choose to 
> customize before deploying the cluster. It could also indicate the need for a 
> "global" default for that property in the cluster template. Examples:
> - Passwords
> - Hostnames 
> - External databases
> Currently Ambari has a concept of SECRET references. E.g.
> SECRET:hive-site:2:hive.server2.keystore.password
> These are used for masking the password when a blueprint is exported and the 
> reference itself is not exported. It would be useful to have 

[jira] [Updated] (AMBARI-15395) Enhance blueprint support for using references

2016-03-12 Thread Shantanu Mundkur (JIRA)

 [ 
https://issues.apache.org/jira/browse/AMBARI-15395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shantanu Mundkur updated AMBARI-15395:
--
Description: 
An exported blueprint should provide ready portability i.e. an exported 
blueprint be usable without changes to deploy another cluster; some elements 
that are masked or omitted could still use tokens or placeholders. These have 
been called references in previous Jiras. A reference follow some convention 
that indicates that it is a reference by using a keyword and a pattern e.g.
ReferenceName:configType:configVersion:propertyName

References would be a good indicators of properties that user could choose to 
customize before deploying the cluster. It could also indicate the need for a 
"global" default for that property in the cluster template. Examples:
Passwords
Hostnames 
External databases

Currently Ambari has a concept of SECRET references. E.g.
SECRET:hive-site:2:hive.server2.keystore.password

These are used for masking the password when a blueprint is exported and the 
reference itself is not exported. It would be useful to have an reference 
exported as long as it is processed appropriately during deployment. 

Similar to the secret reference, one could have,
HOST:kerberos-env:-1:kdc_host
and so forth.

For any reference, in the cluster template there would be a corresponding 
property that would be used for substituting a value for the reference during 
deployment if the registered blueprint had such references. Currently such 
behavior is used if a property of type password is not specified 
(default_password). Such references could be used to tag properties to flag 
them to be the ones that users must customize or include in the cluster 
template. It could also serve a way to annotate/comment parts of the blueprint 
JSON.

  was:
An exported blueprint should provide ready portability i.e. an exported 
blueprint be usable without changes to deploy another cluster; some elements 
that are masked or omitted could still use tokens or placeholders. These have 
been called references in previous Jiras. A reference follow some convention 
that indicates that it is a reference by using a keyword and a pattern e.g.
ReferenceName:configType:configVersion:propertyName

References would be a good indicators of properties that user could choose to 
customize before deploying the cluster. It could also indicate the need for a 
"global" default for that property in the cluster template. Examples:
Passwords
Hostnames 
External databases

Currently Ambari has a concept of SECRET references. E.g.
SECRET:hive-site:2:hive.server2.keystore.password

These are used for masking the password when a blueprint is exported. However, 
it would be useful to have an entry exported but using a reference.

Similarly one could have,
HOST:kerberos-env:-1:kdc_host
and so forth.

For any reference, in the cluster template there would be a corresponding 
property that would be used for substituting a value for the reference during 
deployment if the registered blueprint had such references. Currently such 
behavior is used if a property of type password is not specified 
(default_password). Such references could be used to tag properties to flag 
them to be the ones that users must customize or include in the cluster 
template. It could also serve a way to annotate/comment parts of the blueprint 
JSON.


> Enhance blueprint support for using references
> --
>
> Key: AMBARI-15395
> URL: https://issues.apache.org/jira/browse/AMBARI-15395
> Project: Ambari
>  Issue Type: Story
>  Components: ambari-server
>Affects Versions: 2.4.0
>Reporter: Shantanu Mundkur
>
> An exported blueprint should provide ready portability i.e. an exported 
> blueprint be usable without changes to deploy another cluster; some elements 
> that are masked or omitted could still use tokens or placeholders. These have 
> been called references in previous Jiras. A reference follow some convention 
> that indicates that it is a reference by using a keyword and a pattern e.g.
> ReferenceName:configType:configVersion:propertyName
> References would be a good indicators of properties that user could choose to 
> customize before deploying the cluster. It could also indicate the need for a 
> "global" default for that property in the cluster template. Examples:
> Passwords
> Hostnames 
> External databases
> Currently Ambari has a concept of SECRET references. E.g.
> SECRET:hive-site:2:hive.server2.keystore.password
> These are used for masking the password when a blueprint is exported and the 
> reference itself is not exported. It would be useful to have an reference 
> exported as long as it is processed appropriately during deployment. 
> Similar to the secret reference, one could 

[jira] [Commented] (AMBARI-6432) FreeIPA Support in Ambari

2016-03-12 Thread Bolke de Bruin (JIRA)

[ 
https://issues.apache.org/jira/browse/AMBARI-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15191784#comment-15191784
 ] 

Bolke de Bruin commented on AMBARI-6432:


[~rlevas] [~u39kun] v6 patch should address all issues mentioned. I would like 
to consider it Final if it passes the tests (can someone kicks those off, it 
does not seem to run on uploading a patch?).

* FreeIPA 3 should work
* Marked experimental by 'enableIpa' at the experimental page
* Tested on 10+2 node system
* toLower Function implemented in VariableReplacementHelper and applied where 
needed

Note:
* There seems to be a bug in how Ambari handles adding and removing principals: 
it gets called multiple times for these operations. By turning on debugging 
output for the Kerberos classes (and use IPA as the other classes dont log 
enough) you can see this. I haven't been able to look at this. 



> FreeIPA Support in Ambari
> -
>
> Key: AMBARI-6432
> URL: https://issues.apache.org/jira/browse/AMBARI-6432
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: trunk
>Reporter: jay vyas
>Assignee: Bolke de Bruin
> Fix For: 2.4.0
>
> Attachments: AMBARI-6432-FreeIPA.patch, AMBARI-6432.patch, 
> AMBARI-6432.patch, AMBARI-6432.trunk.v1.patch, AMBARI-6432.trunk.v2.patch, 
> AMBARI-6432.trunk.v3.patch, AMBARI-6432.trunk.v4.patch, 
> AMBARI-6432.trunk.v5.patch, AMBARI-6432.trunk.v5.patch, 
> AMBARI-6432.trunk.v6.patch, ipa-patch-v0.5.patch
>
>
> FreeIPA Is a powerful tool for unifying identity, kerberos credentials, 
> across a cluster.
> A great value add for ambari would be to provide support for using FreeIPA to 
> kerberize services.  This would allow for 
> 1) better HCFS interoperability, because first class GID/UID is critical for 
> certain file systems (GlusterFS, Lustre, and any other file system which uses 
> kernel / FUSE apis for determining identity)
> 2) better enterprise interoperability.  Because of the fact that FreeIPA 
> makes it easy to interop with different identity solutions (like active 
> directory), it would make ambari easier to adopt for various enterprises.
> 3) broadens ambaris scope.  Now ambari could also allow people to setup the 
> users of their clusters, and at least some of the security features of their 
> clusters, all from one interface (no more manual handling of TGTs and such - 
> it could all be done quite easily via the ambari UI which could make calls to 
> underlying FreeIPA clients).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)