[jira] [Updated] (AMBARI-24606) Deleting a service should do a better job of cleaning up the Ambari DB
[
https://issues.apache.org/jira/browse/AMBARI-24606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-24606:
Description:
When a service is deleted, there are artifacts that are left behind in the
Ambari DB such as configuration versions.
These are no longer needed and can cause issues.
When restarting the ambari server, you will see the following error:
DB configs consistency check found warnings. See
/var/log/ambari-server/ambari-server-check-database.log for more details.
/var/log/ambari-server/ambari-server-check-database.log:
2018-09-06 11:12:49,455 WARN - You have config(s): ... that is(are) not
mapped (in serviceconfigmapping table) to any service!
You can use following command to cleanup the postgress database:
grep WARN /var/log/ambari-server/ambari-server-check-database.log | tail -n 1 |
awk '\{print $8}'| sed 's/,/\n/g' | sed 's/-version/ version/g' | awk '\{printf
"delete from clusterconfig WHERE type_name='\''%s'\'' AND
version_tag='\''%s'\'';\n", $1, $2}' | psql -U ambari ambari
There are no REST API calls that can be used to clean up the unwanted
configurations.
curl -u admin:admin -i -H 'X-Requested-By: ambari' -X DELETE
'http://localhost:8080/api/v1/clusters/cc/configurations/service_config_versions?service_name=HDFS
HTTP/1.1 405 Method Not Allowed'
curl -u admin:admin -i -H 'X-Requested-By: ambari' -X DELETE
'http://localhost:8080/api/v1/clusters/cc/configurations?type=hdfs-conf&tag=version1'
HTTP/1.1 405 Method Not Allowed'
There are many defects related to this which are still opened:
https://issues.apache.org/jira/browse/AMBARI-18358
https://issues.apache.org/jira/browse/AMBARI-14624
https://issues.apache.org/jira/browse/AMBARI-19990
https://issues.apache.org/jira/browse/AMBARI-21269
https://issues.apache.org/jira/browse/AMBARI-12573
https://issues.apache.org/jira/browse/AMBARI-11713
I haven't verified whether all of the issues described are still valid or not.
was:
When a service is deleted, there are artifacts that are left behind in the
Ambari DB such as configuration versions.
These are no longer needed and can cause issues.
When restarting the ambari server, you will see the following error:
DB configs consistency check found warnings. See
/var/log/ambari-server/ambari-server-check-database.log for more details.
/var/log/ambari-server/ambari-server-check-database.log:
2018-09-06 11:12:49,455 WARN - You have config(s): ... that is(are) not mapped
(in serviceconfigmapping table) to any service!
You can use following command to cleanup the postgress database:
grep WARN /var/log/ambari-server/ambari-server-check-database.log | tail -n 1 |
awk '\{print $8}'| sed 's/,/\n/g' | sed 's/-version/ version/g' | awk '\{printf
"delete from clusterconfig WHERE type_name='\''%s'\'' AND
version_tag='\''%s'\'';\n", $1, $2}' | psql -U ambari ambari
There are no REST API calls that can be used to clean up the unwanted
configurations.
There are many defects related to this which are still opened:
https://issues.apache.org/jira/browse/AMBARI-18358
https://issues.apache.org/jira/browse/AMBARI-14624
https://issues.apache.org/jira/browse/AMBARI-19990
https://issues.apache.org/jira/browse/AMBARI-21269
https://issues.apache.org/jira/browse/AMBARI-12573
https://issues.apache.org/jira/browse/AMBARI-11713
I haven't verified whether all of the issues described are still valid or not.
> Deleting a service should do a better job of cleaning up the Ambari DB
> --
>
> Key: AMBARI-24606
> URL: https://issues.apache.org/jira/browse/AMBARI-24606
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.7.0
>Reporter: Tim Thorpe
>Priority: Major
>
> When a service is deleted, there are artifacts that are left behind in the
> Ambari DB such as configuration versions.
> These are no longer needed and can cause issues.
> When restarting the ambari server, you will see the following error:
> DB configs consistency check found warnings. See
> /var/log/ambari-server/ambari-server-check-database.log for more details.
> /var/log/ambari-server/ambari-server-check-database.log:
> 2018-09-06 11:12:49,455 WARN - You have config(s): ... that is(are) not
> mapped (in serviceconfigmapping table) to any service!
> You can use following command to cleanup the postgress database:
> grep WARN /var/log/ambari-server/ambari-server-check-database.log | tail -n 1
> | awk '\{print $8}'| sed 's/,/\n/g' | sed 's/-version/ version/g' | awk
> '\{printf "delete from clusterconfig WHERE type_name='\''%s'\'' AND
> version_tag='\''%s'\'';\n", $1, $2}' | psql -U ambari ambari
> There are no REST API calls that can be used to clean up the unwanted
> configurations.
> curl -u admin:admin -i -H 'X-Requested
[jira] [Created] (AMBARI-24606) Deleting a service should do a better job of cleaning up the Ambari DB
Tim Thorpe created AMBARI-24606:
---
Summary: Deleting a service should do a better job of cleaning up
the Ambari DB
Key: AMBARI-24606
URL: https://issues.apache.org/jira/browse/AMBARI-24606
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.7.0
Reporter: Tim Thorpe
When a service is deleted, there are artifacts that are left behind in the
Ambari DB such as configuration versions.
These are no longer needed and can cause issues.
When restarting the ambari server, you will see the following error:
DB configs consistency check found warnings. See
/var/log/ambari-server/ambari-server-check-database.log for more details.
/var/log/ambari-server/ambari-server-check-database.log:
2018-09-06 11:12:49,455 WARN - You have config(s): ... that is(are) not mapped
(in serviceconfigmapping table) to any service!
You can use following command to cleanup the postgress database:
grep WARN /var/log/ambari-server/ambari-server-check-database.log | tail -n 1 |
awk '\{print $8}'| sed 's/,/\n/g' | sed 's/-version/ version/g' | awk '\{printf
"delete from clusterconfig WHERE type_name='\''%s'\'' AND
version_tag='\''%s'\'';\n", $1, $2}' | psql -U ambari ambari
There are no REST API calls that can be used to clean up the unwanted
configurations.
There are many defects related to this which are still opened:
https://issues.apache.org/jira/browse/AMBARI-18358
https://issues.apache.org/jira/browse/AMBARI-14624
https://issues.apache.org/jira/browse/AMBARI-19990
https://issues.apache.org/jira/browse/AMBARI-21269
https://issues.apache.org/jira/browse/AMBARI-12573
https://issues.apache.org/jira/browse/AMBARI-11713
I haven't verified whether all of the issues described are still valid or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-23591: Description: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a required service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. Some components may require either a slave component or a service client. This situation applies to components that can be installed either on slave nodes or edge nodes. Ex. with the IBM DB2 Big SQL service for a worker node, it can be installed either on a slave node with a DataNode or on an edge node with the HDFS client. was: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. Some components may require either a slave component or a service client. This situation applies to components that can be installed either on slave nodes or edge nodes. Ex. with the IBM DB2 Big SQL service for a worker node, it can be installed either on a slave node with a DataNode or on an edge node with the HDFS client. > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Jayush Luniya >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependencies can be defined > (some examples are listed below). We also need to make sure these > dependencies are enforced in the same manner when installing the se
[jira] [Updated] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-23591: Description: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. Some components may require either a slave component or a service client. This situation applies to components that can be installed either on slave nodes or edge nodes. Ex. with the IBM DB2 Big SQL service for a worker node, it can be installed either on a slave node with a DataNode or on an edge node with the HDFS client. was: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDFS DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. Some components may require either a slave component or a service client. This situation applies to components that can be installed either on slave nodes or edge nodes. Ex. with the IBM DB2 Big SQL service for a worker node, it can be installed either on a slave node with a DataNode or on an edge node with the HDFS client. > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Jayush Luniya >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependenc
[jira] [Updated] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-23591: Description: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDFS DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. Some components may require either a slave component or a service client. This situation applies to components that can be installed either on slave nodes or edge nodes. Ex. with the IBM DB2 Big SQL service for a worker node, it can be installed either on a slave node with a DataNode or on an edge node with the HDFS client. was: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDFS DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Jayush Luniya >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependencies can be defined > (some examples are listed below). We also need to make sure these > dependencies are enforced in the same manner when installing the service and > a
[jira] [Updated] (AMBARI-24497) Stack advisor services.json shows HBASE client as installed on all nodes
[ https://issues.apache.org/jira/browse/AMBARI-24497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-24497: Description: Only a few of the nodes in my cluster actually have the HBASE client installed but the services.json passed to the stack advisor shows it installed on all nodes. In a service advisor, we are unable to recommend placement of components based on the HBASE client or validate those placements or even recommend or validate configuration changes based on whether the HBASE clients are installed on certain nodes. This only impacts add service. Once the service is actually installed, then recommending and validating configuration changes work fine. was: Only a few of the nodes in my cluster actually have the HBASE client installed but the services.json passed to the stack advisor shows it installed on all nodes. In a service advisor, we are unable to recommend placement of components based on the HBASE client or validate those placements or even recommend or validate configuration changes based on whether the HBASE clients are installed on certain nodes. > Stack advisor services.json shows HBASE client as installed on all nodes > > > Key: AMBARI-24497 > URL: https://issues.apache.org/jira/browse/AMBARI-24497 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.6.2 > Environment: ambari-server.x86_64 2.6.2.0-155 > @ambari-2.6.2.0 >Reporter: Tim Thorpe >Priority: Major > > Only a few of the nodes in my cluster actually have the HBASE client > installed but the services.json passed to the stack advisor shows it > installed on all nodes. > In a service advisor, we are unable to recommend placement of components > based on the HBASE client or validate those placements or even recommend or > validate configuration changes based on whether the HBASE clients are > installed on certain nodes. > This only impacts add service. Once the service is actually installed, then > recommending and validating configuration changes work fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AMBARI-24497) Stack advisor services.json shows HBASE client as installed on all nodes
Tim Thorpe created AMBARI-24497: --- Summary: Stack advisor services.json shows HBASE client as installed on all nodes Key: AMBARI-24497 URL: https://issues.apache.org/jira/browse/AMBARI-24497 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.6.2 Environment: ambari-server.x86_64 2.6.2.0-155 @ambari-2.6.2.0 Reporter: Tim Thorpe Only a few of the nodes in my cluster actually have the HBASE client installed but the services.json passed to the stack advisor shows it installed on all nodes. In a service advisor, we are unable to recommend placement of components based on the HBASE client or validate those placements or even recommend or validate configuration changes based on whether the HBASE clients are installed on certain nodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-23827) Update extension link doesn't persist to the Ambari DB
[
https://issues.apache.org/jira/browse/AMBARI-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471943#comment-16471943
]
Tim Thorpe commented on AMBARI-23827:
-
Please note, you *do not* need to install the stack or the service(s) in the
extension version.
You need to have 2 different versions of the same extension and follow the
steps listed in the description above.
> Update extension link doesn't persist to the Ambari DB
> --
>
> Key: AMBARI-23827
> URL: https://issues.apache.org/jira/browse/AMBARI-23827
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.6.0, 2.7.0
>Reporter: Tim Thorpe
>Priority: Major
>
> Update extension link doesn't persist to the Ambari DB. The REST API will
> return the proper results but the DB won't be altered. This results in the
> extension link reverting to the previous extension after an Ambari server
> restart.
> Here are the reproduction steps:
> * Install ambari-server
> * Install 2 versions of an extension - just the extension definitions with
> their service definitions
> * Restart ambari-server
> * Link the 1st version of the extension
> * Update the extension link to the 2nd version of the extension
> * Verify the Ambari DB
> * Restart ambari-server
> * Verify the list of links via the REST API (post restart)
> *
> ## Link the 1st version*
> [root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X POST* -d
> '{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
> "extension_name": "IBM-Big_SQL", *"extension_version": "5.0.2.0"*}}'
> [http://node1.fyre.ibm.com:8080/api/v1/links/]
> *
> ## Verify the extension link created*
> [root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
> [http://node1.fyre.ibm.com:8080/api/v1/links/]
> {
> "href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
> "items" : [
>
> Unknown macro: \{ "href" }
> ]
> }
> *
> ## Update the link to the 2nd extension version*
> [root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X PUT* -d
> '{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
> "extension_name": "IBM-Big_SQL", *"extension_version": "5.0.3.0"*}}'
> [http://node1.fyre.ibm.com:8080/api/v1/links/1]
> *
> ## Verify the extension link is updated*
> [root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
> [http://node1.fyre.ibm.com:8080/api/v1/links/]
> {
> "href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
> "items" : [
>
> Unknown macro: \{ "href" }
> ]
> }
> # *Verify the Ambari DB*
> ambari=> select * from stack where stack_version = '2.5';
> stack_id | stack_name | stack_version
> --++-
> 6 | HDP | 2.5
> ambari=> select * from extension;
> extension_id | extension_name | extension_version
> --++---
> 1 | IBM-Big_SQL | 5.0.3.0
> *2 | IBM-Big_SQL | 5.0.2.0*
> *ambari=> select * from extensionlink;*
> *link_id | stack_id | {color:#ff}extension_id{color}*
> *-++*
> *1 | 6 | {color:#ff} 2
> >> Extension link is still listed as IBM-Big_SQL 5.0.2.0.{color}
> *
>
>
> *# Verify the extension link after ambari-server restart*
> [root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
> [http://node1.fyre.ibm.com:8080/api/v1/links/]
> {
> "href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
> "items" : [
> {
> "href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
> "ExtensionLink" : {
> *"extension_name" : "IBM-Big_SQL",*
> *"extension_version" : "5.0.2.0",*
> *"link_id" : 1,*
> "stack_name" : "HDP",
> "stack_version" : "2.5"
> }
> }
> ]
> }
> *>> Extension link is back to IBM-Big_SQL 5.0.2.0 via REST API.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-23827) Update extension link doesn't persist to the Ambari DB
[
https://issues.apache.org/jira/browse/AMBARI-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-23827:
Description:
Update extension link doesn't persist to the Ambari DB. The REST API will
return the proper results but the DB won't be altered. This results in the
extension link reverting to the previous extension after an Ambari server
restart.
Here are the reproduction steps:
* Install ambari-server
* Install 2 versions of an extension - just the extension definitions with
their service definitions
* Restart ambari-server
* Link the 1st version of the extension
* Update the extension link to the 2nd version of the extension
* Verify the Ambari DB
* Restart ambari-server
* Verify the list of links via the REST API (post restart)
*
## Link the 1st version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X POST* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-Big_SQL", *"extension_version": "5.0.2.0"*}}'
[http://node1.fyre.ibm.com:8080/api/v1/links/]
*
## Verify the extension link created*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
[http://node1.fyre.ibm.com:8080/api/v1/links/]
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
Unknown macro: \{ "href" }
]
}
*
## Update the link to the 2nd extension version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X PUT* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-Big_SQL", *"extension_version": "5.0.3.0"*}}'
[http://node1.fyre.ibm.com:8080/api/v1/links/1]
*
## Verify the extension link is updated*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
[http://node1.fyre.ibm.com:8080/api/v1/links/]
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
Unknown macro: \{ "href" }
]
}
# *Verify the Ambari DB*
ambari=> select * from stack where stack_version = '2.5';
stack_id | stack_name | stack_version
--++-
6 | HDP | 2.5
ambari=> select * from extension;
extension_id | extension_name | extension_version
--++---
1 | IBM-Big_SQL | 5.0.3.0
*2 | IBM-Big_SQL | 5.0.2.0*
*ambari=> select * from extensionlink;*
*link_id | stack_id | {color:#ff}extension_id{color}*
*-++*
*1 | 6 | {color:#ff} 2
>> Extension link is still listed as IBM-Big_SQL 5.0.2.0.{color}
*
*# Verify the extension link after ambari-server restart*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
[http://node1.fyre.ibm.com:8080/api/v1/links/]
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
"ExtensionLink" : {
*"extension_name" : "IBM-Big_SQL",*
*"extension_version" : "5.0.2.0",*
*"link_id" : 1,*
"stack_name" : "HDP",
"stack_version" : "2.5"
}
}
]
}
*>> Extension link is back to IBM-Big_SQL 5.0.2.0 via REST API.*
was:
Update extension link doesn't persist to the Ambari DB. The REST API will
return the proper results but the DB won't be altered. This results in the
extension link reverting to the previous extension after an Ambari server
restart.
Here are the reproduction steps:
* Install ambari-server
* Install 2 versions of an extension
* Restart ambari-server
* Link the 1st version of the extension
* Update the extension link to the 2nd version of the extension
* Verify the Ambari DB
* Restart ambari-server
* Verify the list of links via the REST API (post restart)
*# Link the 1st version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X POST* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-Big_SQL", *"extension_version": "5.0.2.0"*}}'
http://node1.fyre.ibm.com:8080/api/v1/links/
*# Verify the extension link created*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
http://node1.fyre.ibm.com:8080/api/v1/links/
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
"ExtensionLink" : {
*"extension_name" : "IBM-Big_SQL",*
*"extension_version" : "5.0.2.0",*
*"link_id" : 1,*
"stack_name" : "HDP",
"stack_version" : "2.5"
}
}
]
}
*# Update the link to the 2nd extension version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X PUT* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-
[jira] [Created] (AMBARI-23827) Update extension link doesn't persist to the Ambari DB
Tim Thorpe created AMBARI-23827:
---
Summary: Update extension link doesn't persist to the Ambari DB
Key: AMBARI-23827
URL: https://issues.apache.org/jira/browse/AMBARI-23827
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.6.0, 2.7.0
Reporter: Tim Thorpe
Update extension link doesn't persist to the Ambari DB. The REST API will
return the proper results but the DB won't be altered. This results in the
extension link reverting to the previous extension after an Ambari server
restart.
Here are the reproduction steps:
* Install ambari-server
* Install 2 versions of an extension
* Restart ambari-server
* Link the 1st version of the extension
* Update the extension link to the 2nd version of the extension
* Verify the Ambari DB
* Restart ambari-server
* Verify the list of links via the REST API (post restart)
*# Link the 1st version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X POST* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-Big_SQL", *"extension_version": "5.0.2.0"*}}'
http://node1.fyre.ibm.com:8080/api/v1/links/
*# Verify the extension link created*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
http://node1.fyre.ibm.com:8080/api/v1/links/
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
"ExtensionLink" : {
*"extension_name" : "IBM-Big_SQL",*
*"extension_version" : "5.0.2.0",*
*"link_id" : 1,*
"stack_name" : "HDP",
"stack_version" : "2.5"
}
}
]
}
*# Update the link to the 2nd extension version*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' *-X PUT* -d
'{"ExtensionLink": {"stack_name": "HDP", "stack_version": "2.5",
"extension_name": "IBM-Big_SQL", *"extension_version": "5.0.3.0"*}}'
http://node1.fyre.ibm.com:8080/api/v1/links/1
*# Verify the extension link is updated*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
http://node1.fyre.ibm.com:8080/api/v1/links/
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
"ExtensionLink" : {
*"extension_name" : "IBM-Big_SQL",*
*"extension_version" : "5.0.3.0",*
*"link_id" : 1,*
"stack_name" : "HDP",
"stack_version" : "2.5"
}
}
]
}
# *Verify the Ambari DB*
ambari=> select * from stack where stack_version = '2.5';
stack_id | stack_name | stack_version
--++-
6 | HDP | 2.5
ambari=> select * from extension;
extension_id | extension_name | extension_version
--++---
1 | IBM-Big_SQL | 5.0.3.0
*2 | IBM-Big_SQL | 5.0.2.0*
*ambari=> select * from extensionlink;*
*link_id | stack_id | {color:#FF}extension_id{color}*
*-+--+--*
*1 | 6 | {color:#FF} 2
{color:#33}>> Extension link is still listed as IBM-Big_SQL 5.0.2.0.{color}
{color}*
*# Verify the extension link after ambari-server restart*
[root@node1 ~]# curl -u admin:admin -H 'X-Requested-By: ambari' -X GET
http://node1.fyre.ibm.com:8080/api/v1/links/
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/";,
"items" : [
{
"href" : "http://node1.fyre.ibm.com:8080/api/v1/links/1";,
"ExtensionLink" : {
*"extension_name" : "IBM-Big_SQL",*
{color:#FF} *"extension_version" : "5.0.2.0",*{color}
*"link_id" : 1,*
"stack_name" : "HDP",
"stack_version" : "2.5"
}
}
]
}
*>> Extension link is back to IBM-Big_SQL 5.0.2.0 via REST API.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-23591: Description: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDFS DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. was: In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDSF DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Jayush Luniya >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependencies can be defined > (some examples are listed below). We also need to make sure these > dependencies are enforced in the same manner when installing the service and > adding a component to a node. Currently, when adding a service, the > component dependencies are only applied if the service is actually installed. > For example, the following dependency would only be applied if HDFS was > installed: > > > HDFS/HDFS_CLIENT > host > >
[jira] [Assigned] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe reassigned AMBARI-23591: --- Assignee: Jayush Luniya (was: Tim Thorpe) > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Jayush Luniya >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependencies can be defined > (some examples are listed below). We also need to make sure these > dependencies are enforced in the same manner when installing the service and > adding a component to a node. Currently, when adding a service, the > component dependencies are only applied if the service is actually installed. > For example, the following dependency would only be applied if HDFS was > installed: > > > HDFS/HDFS_CLIENT > host > > true > > > When adding a component to a different node, after the service has been > installed, the dependencies within the metainfo.xml are then enforced. This > means that even if you don't have HDFS listed as a require service, the HDFS > Client dependency will still be enforced even if HDFS is not installed in the > cluster. > The behavior should be consistent. Add service and add component should both > follow the same behavior in the Ambari UI. > *Dependency Examples* > Certain service dependencies are optional. Ex. Spark has a dependency on the > Hive service but Hive is only needed if Spark Thrift Server component is > installed. > > HDFS > YARN > *HIVE* > > Some services may be required but only if a certain setting is enabled. Ex. > HDFS requires Zookeeper but only if NameNode HA is enabled. > Some components may require a dependency between a choice of components. Ex. > IBM DB2 Big SQL worker node requires either an HDSF DataNode or an HDFS > Client. > Some components may require a dependency on a service client only if that > service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head > node requires the Atlas client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AMBARI-23591) Improve inter-service/component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe reassigned AMBARI-23591: --- Assignee: Tim Thorpe > Improve inter-service/component dependencies > > > Key: AMBARI-23591 > URL: https://issues.apache.org/jira/browse/AMBARI-23591 > Project: Ambari > Issue Type: Epic > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe >Priority: Major > > In order to better support service dependencies and third party or custom > services, Ambari should be more flexible in how dependencies can be defined > (some examples are listed below). We also need to make sure these > dependencies are enforced in the same manner when installing the service and > adding a component to a node. Currently, when adding a service, the > component dependencies are only applied if the service is actually installed. > For example, the following dependency would only be applied if HDFS was > installed: > > > HDFS/HDFS_CLIENT > host > > true > > > When adding a component to a different node, after the service has been > installed, the dependencies within the metainfo.xml are then enforced. This > means that even if you don't have HDFS listed as a require service, the HDFS > Client dependency will still be enforced even if HDFS is not installed in the > cluster. > The behavior should be consistent. Add service and add component should both > follow the same behavior in the Ambari UI. > *Dependency Examples* > Certain service dependencies are optional. Ex. Spark has a dependency on the > Hive service but Hive is only needed if Spark Thrift Server component is > installed. > > HDFS > YARN > *HIVE* > > Some services may be required but only if a certain setting is enabled. Ex. > HDFS requires Zookeeper but only if NameNode HA is enabled. > Some components may require a dependency between a choice of components. Ex. > IBM DB2 Big SQL worker node requires either an HDSF DataNode or an HDFS > Client. > Some components may require a dependency on a service client only if that > service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head > node requires the Atlas client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AMBARI-23591) Improve inter-service/component dependencies
Tim Thorpe created AMBARI-23591: --- Summary: Improve inter-service/component dependencies Key: AMBARI-23591 URL: https://issues.apache.org/jira/browse/AMBARI-23591 Project: Ambari Issue Type: Epic Components: ambari-server Affects Versions: 3.0.0 Reporter: Tim Thorpe In order to better support service dependencies and third party or custom services, Ambari should be more flexible in how dependencies can be defined (some examples are listed below). We also need to make sure these dependencies are enforced in the same manner when installing the service and adding a component to a node. Currently, when adding a service, the component dependencies are only applied if the service is actually installed. For example, the following dependency would only be applied if HDFS was installed: HDFS/HDFS_CLIENT host true When adding a component to a different node, after the service has been installed, the dependencies within the metainfo.xml are then enforced. This means that even if you don't have HDFS listed as a require service, the HDFS Client dependency will still be enforced even if HDFS is not installed in the cluster. The behavior should be consistent. Add service and add component should both follow the same behavior in the Ambari UI. *Dependency Examples* Certain service dependencies are optional. Ex. Spark has a dependency on the Hive service but Hive is only needed if Spark Thrift Server component is installed. HDFS YARN *HIVE* Some services may be required but only if a certain setting is enabled. Ex. HDFS requires Zookeeper but only if NameNode HA is enabled. Some components may require a dependency between a choice of components. Ex. IBM DB2 Big SQL worker node requires either an HDSF DataNode or an HDFS Client. Some components may require a dependency on a service client only if that service is installed. Ex. if Atlas is installed, then IBM DB2 Big SQL head node requires the Atlas client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMBARI-22253) Cluster creation using v2 blueprints
[ https://issues.apache.org/jira/browse/AMBARI-22253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318341#comment-16318341 ] Tim Thorpe commented on AMBARI-22253: - Just curious, why are we dropping support for this? Has it been determined that the v1 blueprint can handle the mpack scenario adequately? > Cluster creation using v2 blueprints > > > Key: AMBARI-22253 > URL: https://issues.apache.org/jira/browse/AMBARI-22253 > Project: Ambari > Issue Type: Task > Components: ambari-server >Reporter: Magyari Sandor Szilard >Assignee: Doroszlai, Attila > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22465) Post-Upgrade Tasks Use the Wrong Repository and Hooks Folders
[
https://issues.apache.org/jira/browse/AMBARI-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266802#comment-16266802
]
Tim Thorpe commented on AMBARI-22465:
-
[~dgrinenko] I'm not sure how Jonathan tested this but my test involved a stack
extension linked to the stack. When the upgrade kicked off, my extension
version was 5.0.1. The script version that was used for both pre and post
upgrade was from the 5.0.1 extension. During the pre-upgrade, I changed the
linked extension version to 5.0.2 (and then restarted Ambari server).
The restart action used the correct version from the 5.0.2 extension but the
post upgrade still used the scripts directory from the 5.0.1 extension. It
seems to me that the pre and post upgrade actions use a pre-calculated
directory.
I have been playing around with this and trying to get my extension upgrade to
work if I have the new extension version linked to the new stack version
through out the upgrade process. Part of what was complicating this is I was
doing an HDP 2.6.2 to 2.6.3 upgrade which uses the same HDP 2.6 stack version.
> Post-Upgrade Tasks Use the Wrong Repository and Hooks Folders
> -
>
> Key: AMBARI-22465
> URL: https://issues.apache.org/jira/browse/AMBARI-22465
> Project: Ambari
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jonathan Hurley
>Assignee: Dmytro Grinenko
>Priority: Critical
> Fix For: 2.6.1
>
>
> During an express upgrade, when performing an upgrade between stack versions,
> the wrong repository/stacks/hooks are sent down during the {{post-upgrade}}
> step:
> {code}
>
>
>
> scripts/upgrade.py
> bar
>
>
>
>
>
>
>
> scripts/upgrade.py
> baz
>
>
> {code}
> In both the {{pre-upgrade}} and {{post-upgrade}} commands, the scripts folder
> location refers to the old stack. The restart/start command, however, is
> correct.
> It appears as though the problem is found here:
> https://github.com/apache/ambari/blob/branch-2.6/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java#L1052-L1055
> {code:title=makeActionStage()}
> // the ru_execute_tasks invokes scripts - it needs information about where
> // the scripts live and for that it should always use the target
> repository
> // stack
> applyRepositoryAssociatedParameters(wrapper,
> effectiveRepositoryVersion.getStackId(), params);
> {code}
> It uses the effective stack ID which during an express upgrade is the source
> stack of the upgrade.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
[ https://issues.apache.org/jira/browse/AMBARI-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-0: Fix Version/s: 2.6.1 > Should be able to switch the extension version to which a stack version is > linked > -- > > Key: AMBARI-0 > URL: https://issues.apache.org/jira/browse/AMBARI-0 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk, 2.5.0, 2.6.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk, 2.6.1 > > Attachments: AMBARI-0.patch > > > Currently the only way to switch the extension version to which a stack > version is linked is to perform the following actions: > * Stop the extension services > * Reregister (delete) the extension services > * Unlink the old extension version > * Link the new extension version > * Add the extension services back > The REST API should allow us to do an update action on the extension link. > In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or > something similar) which both use the HDP 2.6 stack version then we could > perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Fix Version/s: 2.6.1 > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk, 2.6.1 > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22137) Different stack versions should be able to link to different extension versions
[ https://issues.apache.org/jira/browse/AMBARI-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-22137: Fix Version/s: 2.6.1 > Different stack versions should be able to link to different extension > versions > --- > > Key: AMBARI-22137 > URL: https://issues.apache.org/jira/browse/AMBARI-22137 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk, 2.6.1 > > Attachments: AMBARI-22137.patch > > > Currently if you have extension version 0.1 linked to HDP 2.5, you can't link > extension version 0.2 to HDP 2.6. > Attempting to make that link will fail with an exception stating that the > services included in extension version 0.2 are already in HDP 2.6 (inherited > from HDP 2.5 via the 0.1 version of the extension). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
[ https://issues.apache.org/jira/browse/AMBARI-0?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206037#comment-16206037 ] Tim Thorpe commented on AMBARI-0: - It is python tests which are failing. The java tests work fine. The failing tests are not related to my patch. > Should be able to switch the extension version to which a stack version is > linked > -- > > Key: AMBARI-0 > URL: https://issues.apache.org/jira/browse/AMBARI-0 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk, 2.5.0, 2.6.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-0.patch > > > Currently the only way to switch the extension version to which a stack > version is linked is to perform the following actions: > * Stop the extension services > * Reregister (delete) the extension services > * Unlink the old extension version > * Link the new extension version > * Add the extension services back > The REST API should allow us to do an update action on the extension link. > In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or > something similar) which both use the HDP 2.6 stack version then we could > perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
[ https://issues.apache.org/jira/browse/AMBARI-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-0: Resolution: Fixed Status: Resolved (was: Patch Available) > Should be able to switch the extension version to which a stack version is > linked > -- > > Key: AMBARI-0 > URL: https://issues.apache.org/jira/browse/AMBARI-0 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk, 2.5.0, 2.6.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-0.patch > > > Currently the only way to switch the extension version to which a stack > version is linked is to perform the following actions: > * Stop the extension services > * Reregister (delete) the extension services > * Unlink the old extension version > * Link the new extension version > * Add the extension services back > The REST API should allow us to do an update action on the extension link. > In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or > something similar) which both use the HDP 2.6 stack version then we could > perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
[ https://issues.apache.org/jira/browse/AMBARI-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-0: Attachment: AMBARI-0.patch > Should be able to switch the extension version to which a stack version is > linked > -- > > Key: AMBARI-0 > URL: https://issues.apache.org/jira/browse/AMBARI-0 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk, 2.5.0, 2.6.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-0.patch > > > Currently the only way to switch the extension version to which a stack > version is linked is to perform the following actions: > * Stop the extension services > * Reregister (delete) the extension services > * Unlink the old extension version > * Link the new extension version > * Add the extension services back > The REST API should allow us to do an update action on the extension link. > In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or > something similar) which both use the HDP 2.6 stack version then we could > perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
[ https://issues.apache.org/jira/browse/AMBARI-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-0: Status: Patch Available (was: Open) > Should be able to switch the extension version to which a stack version is > linked > -- > > Key: AMBARI-0 > URL: https://issues.apache.org/jira/browse/AMBARI-0 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk, 2.5.0, 2.6.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-0.patch > > > Currently the only way to switch the extension version to which a stack > version is linked is to perform the following actions: > * Stop the extension services > * Reregister (delete) the extension services > * Unlink the old extension version > * Link the new extension version > * Add the extension services back > The REST API should allow us to do an update action on the extension link. > In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or > something similar) which both use the HDP 2.6 stack version then we could > perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22220) Should be able to switch the extension version to which a stack version is linked
Tim Thorpe created AMBARI-0: --- Summary: Should be able to switch the extension version to which a stack version is linked Key: AMBARI-0 URL: https://issues.apache.org/jira/browse/AMBARI-0 Project: Ambari Issue Type: Improvement Components: ambari-server Affects Versions: trunk, 2.5.0, 2.6.0 Reporter: Tim Thorpe Assignee: Tim Thorpe Fix For: trunk Currently the only way to switch the extension version to which a stack version is linked is to perform the following actions: * Stop the extension services * Reregister (delete) the extension services * Unlink the old extension version * Link the new extension version * Add the extension services back The REST API should allow us to do an update action on the extension link. In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or something similar) which both use the HDP 2.6 stack version then we could perform the extension link switch as a manual step during the upgrade process. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22137) Different stack versions should be able to link to different extension versions
[ https://issues.apache.org/jira/browse/AMBARI-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-22137: Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to trunk. The issue brought up by hadoop qa was not related to this change. > Different stack versions should be able to link to different extension > versions > --- > > Key: AMBARI-22137 > URL: https://issues.apache.org/jira/browse/AMBARI-22137 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-22137.patch > > > Currently if you have extension version 0.1 linked to HDP 2.5, you can't link > extension version 0.2 to HDP 2.6. > Attempting to make that link will fail with an exception stating that the > services included in extension version 0.2 are already in HDP 2.6 (inherited > from HDP 2.5 via the 0.1 version of the extension). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22137) Different stack versions should be able to link to different extension versions
[ https://issues.apache.org/jira/browse/AMBARI-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-22137: Attachment: AMBARI-22137.patch > Different stack versions should be able to link to different extension > versions > --- > > Key: AMBARI-22137 > URL: https://issues.apache.org/jira/browse/AMBARI-22137 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-22137.patch > > > Currently if you have extension version 0.1 linked to HDP 2.5, you can't link > extension version 0.2 to HDP 2.6. > Attempting to make that link will fail with an exception stating that the > services included in extension version 0.2 are already in HDP 2.6 (inherited > from HDP 2.5 via the 0.1 version of the extension). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-22137) Different stack versions should be able to link to different extension versions
[ https://issues.apache.org/jira/browse/AMBARI-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-22137: Fix Version/s: trunk Status: Patch Available (was: Open) > Different stack versions should be able to link to different extension > versions > --- > > Key: AMBARI-22137 > URL: https://issues.apache.org/jira/browse/AMBARI-22137 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-22137.patch > > > Currently if you have extension version 0.1 linked to HDP 2.5, you can't link > extension version 0.2 to HDP 2.6. > Attempting to make that link will fail with an exception stating that the > services included in extension version 0.2 are already in HDP 2.6 (inherited > from HDP 2.5 via the 0.1 version of the extension). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-22137) Different stack versions should be able to link to different extension versions
Tim Thorpe created AMBARI-22137: --- Summary: Different stack versions should be able to link to different extension versions Key: AMBARI-22137 URL: https://issues.apache.org/jira/browse/AMBARI-22137 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: trunk, 2.5.0 Reporter: Tim Thorpe Assignee: Tim Thorpe Currently if you have extension version 0.1 linked to HDP 2.5, you can't link extension version 0.2 to HDP 2.6. Attempting to make that link will fail with an exception stating that the services included in extension version 0.2 are already in HDP 2.6 (inherited from HDP 2.5 via the 0.1 version of the extension). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe resolved AMBARI-21206. - Resolution: Fixed > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21554) Attempting to make a configuration change to a IOP-4.2.x cluster will result in a service advisor error
[
https://issues.apache.org/jira/browse/AMBARI-21554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-21554:
Summary: Attempting to make a configuration change to a IOP-4.2.x cluster
will result in a service advisor error (was: Attempting to make a
configuration change to a IOP-4.2.5 cluster will result in a service advisor
error)
> Attempting to make a configuration change to a IOP-4.2.x cluster will result
> in a service advisor error
> ---
>
> Key: AMBARI-21554
> URL: https://issues.apache.org/jira/browse/AMBARI-21554
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.5.2
>Reporter: Tim Thorpe
>
> Traceback (most recent call last):
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 166,
> in
> main(sys.argv)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 98,
> in main
> stackAdvisor = instantiateStackAdvisor(stackName, stackVersion,
> parentVersions)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 147,
> in instantiateStackAdvisor
> stack_advisor = imp.load_module('stack_advisor_impl', fp, path, ('.py',
> 'rb', imp.PY_SOURCE))
> File
> "/var/lib/ambari-server/resources/scripts/./../stacks/BigInsights/4.0/services/stack_advisor.py",
> line 20, in
> from stack_advisor_25 import *
> ImportError: No module named stack_advisor_25
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AMBARI-21554) Attempting to make a configuration change to a IOP-4.2.5 cluster will result in a service advisor error
Tim Thorpe created AMBARI-21554:
---
Summary: Attempting to make a configuration change to a IOP-4.2.5
cluster will result in a service advisor error
Key: AMBARI-21554
URL: https://issues.apache.org/jira/browse/AMBARI-21554
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.2
Reporter: Tim Thorpe
Traceback (most recent call last):
File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 166,
in
main(sys.argv)
File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 98, in
main
stackAdvisor = instantiateStackAdvisor(stackName, stackVersion,
parentVersions)
File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 147,
in instantiateStackAdvisor
stack_advisor = imp.load_module('stack_advisor_impl', fp, path, ('.py',
'rb', imp.PY_SOURCE))
File
"/var/lib/ambari-server/resources/scripts/./../stacks/BigInsights/4.0/services/stack_advisor.py",
line 20, in
from stack_advisor_25 import *
ImportError: No module named stack_advisor_25
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AMBARI-21539) Resource Manager fails to restart properly during an IOP to HDP upgrade
[ https://issues.apache.org/jira/browse/AMBARI-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095307#comment-16095307 ] Tim Thorpe commented on AMBARI-21539: - I have tried changing the yarn.resourcemanager.zk-state-store.parent-path to a new value and starting the ResourceManager, that still fails with the same error. > Resource Manager fails to restart properly during an IOP to HDP upgrade > --- > > Key: AMBARI-21539 > URL: https://issues.apache.org/jira/browse/AMBARI-21539 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.2 >Reporter: Tim Thorpe > > 2017-07-20 06:19:28,250 INFO service.AbstractService > (AbstractService.java:noteFailure(272)) - Service > org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in > state STARTED; cause: org.apache.zookeeper.KeeperException$NoAuthException: > KeeperErrorCode = NoAuth for /rmstore/ZKRMStateRoot > org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = > NoAuth for /rmstore/ZKRMStateRoot > at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.setACL(ZooKeeper.java:1399) > at > org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$3.run(ZKRMStateStore.java:372) > Workaround: > /usr/hdp/current/zookeeper-server/bin/zkCli.sh -server 127.0.0.1:2181 > rmr /rmstore > This workaround fails in a kerberized environment with this error: > Authentication is not valid : /rmstore/ZKRMStateRoot -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21539) Resource Manager fails to restart properly during an IOP to HDP upgrade
Tim Thorpe created AMBARI-21539: --- Summary: Resource Manager fails to restart properly during an IOP to HDP upgrade Key: AMBARI-21539 URL: https://issues.apache.org/jira/browse/AMBARI-21539 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.5.2 Reporter: Tim Thorpe 2017-07-20 06:19:28,250 INFO service.AbstractService (AbstractService.java:noteFailure(272)) - Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in state STARTED; cause: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /rmstore/ZKRMStateRoot org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /rmstore/ZKRMStateRoot at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.setACL(ZooKeeper.java:1399) at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$3.run(ZKRMStateStore.java:372) Workaround: /usr/hdp/current/zookeeper-server/bin/zkCli.sh -server 127.0.0.1:2181 rmr /rmstore This workaround fails in a kerberized environment with this error: Authentication is not valid : /rmstore/ZKRMStateRoot -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21509) BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
[ https://issues.apache.org/jira/browse/AMBARI-21509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21509: Resolution: Fixed Status: Resolved (was: Patch Available) > BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't > exist in 4.2's hbase_upgrade.py > --- > > Key: AMBARI-21509 > URL: https://issues.apache.org/jira/browse/AMBARI-21509 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.2 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-21509.patch > > > > > scripts/hbase_upgrade.py > *take_snapshot* > > > def *snapshot*(self, env): -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21509) BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
[ https://issues.apache.org/jira/browse/AMBARI-21509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21509: Attachment: AMBARI-21509.patch > BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't > exist in 4.2's hbase_upgrade.py > --- > > Key: AMBARI-21509 > URL: https://issues.apache.org/jira/browse/AMBARI-21509 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.2 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-21509.patch > > > > > scripts/hbase_upgrade.py > *take_snapshot* > > > def *snapshot*(self, env): -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21509) BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
[ https://issues.apache.org/jira/browse/AMBARI-21509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21509: Status: Patch Available (was: Open) > BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't > exist in 4.2's hbase_upgrade.py > --- > > Key: AMBARI-21509 > URL: https://issues.apache.org/jira/browse/AMBARI-21509 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.2 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-21509.patch > > > > > scripts/hbase_upgrade.py > *take_snapshot* > > > def *snapshot*(self, env): -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21509) BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
[ https://issues.apache.org/jira/browse/AMBARI-21509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21509: Description: scripts/hbase_upgrade.py *take_snapshot* def *snapshot*(self, env): > BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't > exist in 4.2's hbase_upgrade.py > --- > > Key: AMBARI-21509 > URL: https://issues.apache.org/jira/browse/AMBARI-21509 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.2 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > > > > scripts/hbase_upgrade.py > *take_snapshot* > > > def *snapshot*(self, env): -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AMBARI-21509) BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
Tim Thorpe created AMBARI-21509: --- Summary: BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py Key: AMBARI-21509 URL: https://issues.apache.org/jira/browse/AMBARI-21509 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.5.2 Reporter: Tim Thorpe Assignee: Tim Thorpe -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070320#comment-16070320 ] Tim Thorpe edited comment on AMBARI-21206 at 6/30/17 3:56 PM: -- [~ssath...@hortonworks.com] - I've been able to get the stage where I was about to deploy an HDP 3.0 cluster with HDFS and ZOOKEEPER. That would be past the point where you hit the error. Did you want to try a fresh build using my commit? I can't really see what else I can do because I don't see that error. was (Author: tim thorpe): [~ssath...@hortonworks.com] - I've been able to get the stage where I was about to deploy an HDP 3.0 cluster with HDFS and ZOOKEEPER. That would be past the point where you hit the error. Did you want to a fresh build using my commit? I can't really see what else I can do because I don't see that error. > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070320#comment-16070320 ] Tim Thorpe commented on AMBARI-21206: - [~ssath...@hortonworks.com] - I've been able to get the stage where I was about to deploy an HDP 3.0 cluster with HDFS and ZOOKEEPER. That would be past the point where you hit the error. Did you want to a fresh build using my commit? I can't really see what else I can do because I don't see that error. > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068527#comment-16068527 ] Tim Thorpe commented on AMBARI-21206: - [~ssath...@hortonworks.com] - can you explain to me when exactly this breaks? I have applied this patch again to a fresh trunk workspace, it builds fine, I managed to install ambari-server and the agents and got to the stage where I was about to deploy a cluster with HDFS and ZOOKEEPER when I hit this unrelated error: 29 Jun 2017 08:57:37,000 ERROR [ambari-client-thread-29] BaseManagementHandler:63 - Bad request received: The properties [ServiceInfo/desired_repository_version_id] specified in the request or predicate are not supported for the resource type Service. > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe reopened AMBARI-21206: - > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Resolution: Fixed Status: Resolved (was: Patch Available) > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AMBARI-20993) Stack advisor needs to enforce conditional component dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe resolved AMBARI-20993. - Resolution: Invalid During cluster deploy or even when adding a new service you need both the layout and the configurations to default to valid settings. In order to enforce conditional dependencies in the layout, you first need to know what the configuration settings will be. In order to know what default configuration settings to use, you first need the layout. > Stack advisor needs to enforce conditional component dependency for slaves > and masters > -- > > Key: AMBARI-20993 > URL: https://issues.apache.org/jira/browse/AMBARI-20993 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > > AMBARI-19685 started the work but is only enforcing non-conditional > dependencies for slaves and masters. This will add in the ability to > validate the layout and the configuration for conditional dependencies both > for "property exists" and "property equals" type of dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-20967) Remove Zookeeper as a required service from HDFS
[ https://issues.apache.org/jira/browse/AMBARI-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20967: Resolution: Fixed Status: Resolved (was: Patch Available) > Remove Zookeeper as a required service from HDFS > > > Key: AMBARI-20967 > URL: https://issues.apache.org/jira/browse/AMBARI-20967 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20967.patch > > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe resolved AMBARI-20122. - Resolution: Fixed > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > Attachments: AMBARI-20122.patch > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > Stack defined dependencies are service/component based and has a scope > CLUSTER|HOST. > During recommendation the services to install have already been selected. We > can't really utilize the cluster scope because either the dependent service > was selected or it was not. If it was not selected it will be caught during > validation. We can only recommend based on HOST scope. > This JIRA is also limited to only handling those which don't have conditional > dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
[
https://issues.apache.org/jira/browse/AMBARI-21202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049352#comment-16049352
]
Tim Thorpe commented on AMBARI-21202:
-
Pushed this in again as it had nothing to do with the failure. I have the
patch for AMBARI-20122 on review to fix the issue mentioned above.
commit 192162e05b7036c5dbbe120c447d7b5ac7ea5b13
> YARN service advisor has spelling error in yarn-site
>
>
> Key: AMBARI-21202
> URL: https://issues.apache.org/jira/browse/AMBARI-21202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-21202.patch
>
>
> self.validators = [("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP206),
> ("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP25),
> ("yarn-ste" ,
> self.validateYarnSiteConfigurationsFromHDP26),
> The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Status: Patch Available (was: Open) > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Attachment: AMBARI-21206.patch > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-21206.patch > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Issue Type: Improvement (was: New Feature) > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Issue Type: New Feature (was: Bug) > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-21206) Remove Zookeeper as a required service from YARN
[ https://issues.apache.org/jira/browse/AMBARI-21206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-21206: Description: YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an explicit dependency because HDFS had one. Enabling the following settings in YARN requires ZOOKEEPER: * yarn.resourcemanager.recovery.enabled * yarn.nodemanager.recovery.enabled * yarn.resourcemanager.ha.enabled * hadoop.registry.rm.enabled In addition, these properties are related to ZOOKEEPER and should be empty if ZOOKEEPER is not installed: * yarn.resourcemanager.zk-address * hadoop.registry.zk.quorum was: YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an explicit dependency because HDFS had one. Enabling the following settings in YARN requires ZOOKEEPER: yarn.resourcemanager.recovery.enabled yarn.nodemanager.recovery.enabled yarn.resourcemanager.ha.enabled hadoop.registry.rm.enabled In addition, these properties are related to ZOOKEEPER and should be empty if ZOOKEEPER is not installed: yarn.resourcemanager.zk-address hadoop.registry.zk.quorum > Remove Zookeeper as a required service from YARN > > > Key: AMBARI-21206 > URL: https://issues.apache.org/jira/browse/AMBARI-21206 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an > explicit dependency because HDFS had one. > Enabling the following settings in YARN requires ZOOKEEPER: > * yarn.resourcemanager.recovery.enabled > * yarn.nodemanager.recovery.enabled > * yarn.resourcemanager.ha.enabled > * hadoop.registry.rm.enabled > In addition, these properties are related to ZOOKEEPER and should be empty if > ZOOKEEPER is not installed: > * yarn.resourcemanager.zk-address > * hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-21206) Remove Zookeeper as a required service from YARN
Tim Thorpe created AMBARI-21206: --- Summary: Remove Zookeeper as a required service from YARN Key: AMBARI-21206 URL: https://issues.apache.org/jira/browse/AMBARI-21206 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: trunk Reporter: Tim Thorpe Assignee: Tim Thorpe Fix For: trunk YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an explicit dependency because HDFS had one. Enabling the following settings in YARN requires ZOOKEEPER: yarn.resourcemanager.recovery.enabled yarn.nodemanager.recovery.enabled yarn.resourcemanager.ha.enabled hadoop.registry.rm.enabled In addition, these properties are related to ZOOKEEPER and should be empty if ZOOKEEPER is not installed: yarn.resourcemanager.zk-address hadoop.registry.zk.quorum -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
[
https://issues.apache.org/jira/browse/AMBARI-21202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042975#comment-16042975
]
Tim Thorpe commented on AMBARI-21202:
-
The change didn't involve any java changes and did not impact upgrade. This
couldn't be responsible for the build error:
Test Result (1 failure / +1)
org.apache.ambari.server.state.UpgradeHelperTest.testServiceCheckUpgradeStages
> YARN service advisor has spelling error in yarn-site
>
>
> Key: AMBARI-21202
> URL: https://issues.apache.org/jira/browse/AMBARI-21202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-21202.patch
>
>
> self.validators = [("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP206),
> ("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP25),
> ("yarn-ste" ,
> self.validateYarnSiteConfigurationsFromHDP26),
> The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20122: Resolution: Fixed Status: Resolved (was: Patch Available) > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > Attachments: AMBARI-20122.patch > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > Stack defined dependencies are service/component based and has a scope > CLUSTER|HOST. > During recommendation the services to install have already been selected. We > can't really utilize the cluster scope because either the dependent service > was selected or it was not. If it was not selected it will be caught during > validation. We can only recommend based on HOST scope. > This JIRA is also limited to only handling those which don't have conditional > dependencies. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
[
https://issues.apache.org/jira/browse/AMBARI-21202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-21202:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> YARN service advisor has spelling error in yarn-site
>
>
> Key: AMBARI-21202
> URL: https://issues.apache.org/jira/browse/AMBARI-21202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-21202.patch
>
>
> self.validators = [("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP206),
> ("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP25),
> ("yarn-ste" ,
> self.validateYarnSiteConfigurationsFromHDP26),
> The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
[
https://issues.apache.org/jira/browse/AMBARI-21202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-21202:
Status: Patch Available (was: Open)
> YARN service advisor has spelling error in yarn-site
>
>
> Key: AMBARI-21202
> URL: https://issues.apache.org/jira/browse/AMBARI-21202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-21202.patch
>
>
> self.validators = [("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP206),
> ("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP25),
> ("yarn-ste" ,
> self.validateYarnSiteConfigurationsFromHDP26),
> The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
[
https://issues.apache.org/jira/browse/AMBARI-21202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-21202:
Attachment: AMBARI-21202.patch
> YARN service advisor has spelling error in yarn-site
>
>
> Key: AMBARI-21202
> URL: https://issues.apache.org/jira/browse/AMBARI-21202
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: trunk
>Reporter: Tim Thorpe
>Assignee: Tim Thorpe
> Fix For: trunk
>
> Attachments: AMBARI-21202.patch
>
>
> self.validators = [("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP206),
> ("yarn-site",
> self.validateYARNSiteConfigurationsFromHDP25),
> ("yarn-ste" ,
> self.validateYarnSiteConfigurationsFromHDP26),
> The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (AMBARI-21202) YARN service advisor has spelling error in yarn-site
Tim Thorpe created AMBARI-21202:
---
Summary: YARN service advisor has spelling error in yarn-site
Key: AMBARI-21202
URL: https://issues.apache.org/jira/browse/AMBARI-21202
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: trunk
Reporter: Tim Thorpe
Assignee: Tim Thorpe
Fix For: trunk
self.validators = [("yarn-site",
self.validateYARNSiteConfigurationsFromHDP206),
("yarn-site",
self.validateYARNSiteConfigurationsFromHDP25),
("yarn-ste" ,
self.validateYarnSiteConfigurationsFromHDP26),
The HDP26 validator should be for yarn-site not yarn-ste.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20122: Description: After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. Stack defined dependencies are service/component based and has a scope CLUSTER|HOST. During recommendation the services to install have already been selected. We can't really utilize the cluster scope because either the dependent service was selected or it was not. If it was not selected it will be caught during validation. We can only recommend based on HOST scope. This JIRA is also limited to only handling those which don't have conditional dependencies. was: After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. stack defined dependencies are servicecomponent based and has a scope CLUSTER|HOST > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > Attachments: AMBARI-20122.patch > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > Stack defined dependencies are service/component based and has a scope > CLUSTER|HOST. > During recommendation the services to install have already been selected. We > can't really utilize the cluster scope because either the dependent service > was selected or it was not. If it was not selected it will be caught during > validation. We can only recommend based on HOST scope. > This JIRA is also limited to only handling those which don't have conditional > dependencies. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20122: Status: Patch Available (was: In Progress) > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > Attachments: AMBARI-20122.patch > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > stack defined dependencies are servicecomponent based and has a scope > CLUSTER|HOST -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20122: Attachment: AMBARI-20122.patch > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > Attachments: AMBARI-20122.patch > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > stack defined dependencies are servicecomponent based and has a scope > CLUSTER|HOST -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (AMBARI-20122) Stack advisor needs to recommend dependency for slaves and masters
[ https://issues.apache.org/jira/browse/AMBARI-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe reassigned AMBARI-20122: --- Assignee: Tim Thorpe > Stack advisor needs to recommend dependency for slaves and masters > -- > > Key: AMBARI-20122 > URL: https://issues.apache.org/jira/browse/AMBARI-20122 > Project: Ambari > Issue Type: Task > Components: ambari-server >Affects Versions: 3.0.0 >Reporter: Jaimin Jetly >Assignee: Tim Thorpe > Fix For: 3.0.0 > > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > stack defined dependencies are servicecomponent based and has a scope > CLUSTER|HOST -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-19685) Stack advisor needs to enforce component dependency for slaves and masters
[
https://issues.apache.org/jira/browse/AMBARI-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-19685:
Component/s: (was: ambari-sever)
ambari-server
> Stack advisor needs to enforce component dependency for slaves and masters
> --
>
> Key: AMBARI-19685
> URL: https://issues.apache.org/jira/browse/AMBARI-19685
> Project: Ambari
> Issue Type: Task
> Components: ambari-server
>Affects Versions: 2.5.0
>Reporter: Jaimin Jetly
>Assignee: Jaimin Jetly
> Fix For: 3.0.0
>
> Attachments: AMBARI-19685.2.patch, AMBARI-19685.3.patch,
> AMBARI-19685.4.patch, AMBARI-19685.patch
>
>
> * Install Ambari cluster with latest trunk bits
> * Modify
> /var/lib/ambari-server/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
> to add the following:
> {noformat}
> Add to node of ZEPPELIN_MASTER:
>
> SPARK/LIVY_SERVER
> host
>
> Add to node:
>
> SPARK
>
> {noformat}
> * Install HDP 2.5.3 cluster with Spark
> * Once the cluster is installed, use Add Service Wizard to add Zeppelin
> * Assign Slaves and Clients step should appear (without the metainfo.xml
> change, it will be skipped)
> * The user should not be able to proceed beyond this page unless at least one
> Livy Server is selected <- this validation should be handled by the stack
> advisor
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006964#comment-16006964 ] Tim Thorpe commented on AMBARI-20992: - The failing test is not related to my change: Test Result (1 failure / ±0) org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSinkTest.testMetricsRetrievalExceptionTolerance > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. This > affects the metainfo.xml from common-services/HDFS both 2.1.0.2.0 and > 3.0.0.3.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Resolution: Fixed Fix Version/s: trunk Status: Resolved (was: Patch Available) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. This > affects the metainfo.xml from common-services/HDFS both 2.1.0.2.0 and > 3.0.0.3.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-20993) Stack advisor needs to enforce conditional component dependency for slaves and masters
Tim Thorpe created AMBARI-20993: --- Summary: Stack advisor needs to enforce conditional component dependency for slaves and masters Key: AMBARI-20993 URL: https://issues.apache.org/jira/browse/AMBARI-20993 Project: Ambari Issue Type: New Feature Components: ambari-server Affects Versions: trunk Reporter: Tim Thorpe Assignee: Tim Thorpe AMBARI-19685 started the work but is only enforcing non-conditional dependencies for slaves and masters. This will add in the ability to validate the layout and the configuration for conditional dependencies both for "property exists" and "property equals" type of dependencies. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Description: HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are listed with host. They should all have cluster scope. This affects the metainfo.xml from common-services/HDFS both 2.1.0.2.0 and 3.0.0.3.0. (was: HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are listed with host. They should all have cluster scope.) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. This > affects the metainfo.xml from common-services/HDFS both 2.1.0.2.0 and > 3.0.0.3.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Attachment: AMBARI-20992.patch > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Status: Patch Available (was: Open) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Status: Open (was: Patch Available) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Attachment: (was: AMBARI-20992.patch) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Attachment: AMBARI-20992.patch > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
[ https://issues.apache.org/jira/browse/AMBARI-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20992: Status: Patch Available (was: Open) > HDFS metainfo.xml has the wrong scope for component dependencies > > > Key: AMBARI-20992 > URL: https://issues.apache.org/jira/browse/AMBARI-20992 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20992.patch > > > HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are > listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-20992) HDFS metainfo.xml has the wrong scope for component dependencies
Tim Thorpe created AMBARI-20992: --- Summary: HDFS metainfo.xml has the wrong scope for component dependencies Key: AMBARI-20992 URL: https://issues.apache.org/jira/browse/AMBARI-20992 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: trunk Reporter: Tim Thorpe Assignee: Tim Thorpe HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are listed with host. They should all have cluster scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Resolution: Fixed Status: Resolved (was: Patch Available) > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20967) Remove Zookeeper as a required service from HDFS
[ https://issues.apache.org/jira/browse/AMBARI-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20967: Status: Patch Available (was: Open) > Remove Zookeeper as a required service from HDFS > > > Key: AMBARI-20967 > URL: https://issues.apache.org/jira/browse/AMBARI-20967 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20967.patch > > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20967) Remove Zookeeper as a required service from HDFS
[ https://issues.apache.org/jira/browse/AMBARI-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20967: Attachment: AMBARI-20967.patch > Remove Zookeeper as a required service from HDFS > > > Key: AMBARI-20967 > URL: https://issues.apache.org/jira/browse/AMBARI-20967 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Attachments: AMBARI-20967.patch > > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20967) Remove Zookeeper as a required service from HDFS
[ https://issues.apache.org/jira/browse/AMBARI-20967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20967: Description: HDFS has a static dependency on Zookeeper even though it is only needed when NameNode HA is enabled. The check for Zookeeper should be done before enabling HA rather than enforcing it at installation time. > Remove Zookeeper as a required service from HDFS > > > Key: AMBARI-20967 > URL: https://issues.apache.org/jira/browse/AMBARI-20967 > Project: Ambari > Issue Type: Improvement > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-20967) Remove Zookeeper as a required service from HDFS
Tim Thorpe created AMBARI-20967: --- Summary: Remove Zookeeper as a required service from HDFS Key: AMBARI-20967 URL: https://issues.apache.org/jira/browse/AMBARI-20967 Project: Ambari Issue Type: Improvement Components: ambari-server Affects Versions: trunk Reporter: Tim Thorpe Assignee: Tim Thorpe -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20433) Need the ability to have dynamic dependency between component/services
[ https://issues.apache.org/jira/browse/AMBARI-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20433: Issue Type: Epic (was: Improvement) > Need the ability to have dynamic dependency between component/services > -- > > Key: AMBARI-20433 > URL: https://issues.apache.org/jira/browse/AMBARI-20433 > Project: Ambari > Issue Type: Epic > Components: ambari-server > Environment: all >Reporter: Tuong Truong >Assignee: Tim Thorpe > > Currently, service interdependency are statically defined in the stack > regardless its needed or not. For instance, Spark has a static dependency on > Hive and will force Hive to be installed whenever Spark is install, but Hive > is only needed if Spark Thrift Server component is installed. > It would be nice if the stack has the ability to declare the dependency > conditionally and at component level and the dependency will only be in > effect when the component is chosen. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20433) Need the ability to have dynamic dependency between component/services
[ https://issues.apache.org/jira/browse/AMBARI-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20433: Description: Currently, service interdependency are statically defined in the stack regardless its needed or not. For instance, Spark has a static dependency on Hive and will force Hive to be installed whenever Spark is install, but Hive is only needed if Spark Thrift Server component is installed. It would be nice if the stack has the ability to declare the dependency conditionally and at component level and the dependency will only be in effect when the component is chosen. was: Currently, service interdependency are statically defined in the stack regardless its needed or not. For instance, Spark has a static dependency on Hive and will force Hive to be installed whenevern Spark is install, but Hive is only needed if Spark Thrift Server component is installed. It would be nice if the stack has to ability to declare the dependency conditionally and at component level and the dependency will only be in effect when the component is chosen. > Need the ability to have dynamic dependency between component/services > -- > > Key: AMBARI-20433 > URL: https://issues.apache.org/jira/browse/AMBARI-20433 > Project: Ambari > Issue Type: Improvement > Components: ambari-server > Environment: all >Reporter: Tuong Truong >Assignee: Tim Thorpe > > Currently, service interdependency are statically defined in the stack > regardless its needed or not. For instance, Spark has a static dependency on > Hive and will force Hive to be installed whenever Spark is install, but Hive > is only needed if Spark Thrift Server component is installed. > It would be nice if the stack has the ability to declare the dependency > conditionally and at component level and the dependency will only be in > effect when the component is chosen. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Status: Patch Available (was: Open) > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Attachment: AMBARI-20891.patch > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Attachment: (was: AMBARI-20891.patch) > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Status: Open (was: Patch Available) > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Attachment: AMBARI-20891.patch > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
[ https://issues.apache.org/jira/browse/AMBARI-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20891: Fix Version/s: trunk Status: Patch Available (was: Open) > Allow extensions to auto-link with supported stack versions > --- > > Key: AMBARI-20891 > URL: https://issues.apache.org/jira/browse/AMBARI-20891 > Project: Ambari > Issue Type: New Feature > Components: ambari-server >Affects Versions: trunk >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20891.patch > > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-20891) Allow extensions to auto-link with supported stack versions
Tim Thorpe created AMBARI-20891: --- Summary: Allow extensions to auto-link with supported stack versions Key: AMBARI-20891 URL: https://issues.apache.org/jira/browse/AMBARI-20891 Project: Ambari Issue Type: New Feature Components: ambari-server Affects Versions: trunk Reporter: Tim Thorpe Assignee: Tim Thorpe It would possible to link extensions to supported stack versions while parsing the stacks, extensions and common-services directories. This would allow extensions to avoid making rest API calls to set up the link. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AMBARI-20433) Need the ability to have dynamic dependency between component/services
[ https://issues.apache.org/jira/browse/AMBARI-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979177#comment-15979177 ] Tim Thorpe commented on AMBARI-20433: - It is possible to drop some of the service dependencies from the metainfo.xml. Instead the stack advisor or service advisor would need to be changed to provide better recommendations and validations of the component layout and configuration based on which of the services (which it *could* depend on) are installed. I'm not sure if the HDFS dependency on Zookeeper is the easiest example for this because enabling NameNode HA is triggered by a hard coded button on the Ambari UI. It may fail gracefully if Zookeeper is not installed (or not installed on enough nodes). I'll need to test that. The biggest problem with this is the scale of the changes. Ideally this would be an epic and we could proceed one service at a time. There are changes at the common services level. First to the metainfo.xml to remove the dependency. Next to the python scripts which handle install, configure, start and stop for the service. Currently those scripts are written to expect the services that are now declared as dependencies are actually installed. Those scripts need to handle the case where those services aren't installed. There would also be changes required at the stack advisor level (or service advisor level) for the recommendations and validations (mentioned above). > Need the ability to have dynamic dependency between component/services > -- > > Key: AMBARI-20433 > URL: https://issues.apache.org/jira/browse/AMBARI-20433 > Project: Ambari > Issue Type: Improvement > Components: ambari-server > Environment: all >Reporter: Tuong Truong >Assignee: Tim Thorpe > > Currently, service interdependency are statically defined in the stack > regardless its needed or not. For instance, Spark has a static dependency on > Hive and will force Hive to be installed whenevern Spark is install, but Hive > is only needed if Spark Thrift Server component is installed. > It would be nice if the stack has to ability to declare the dependency > conditionally and at component level and the dependency will only be in > effect when the component is chosen. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20780) Update HBase Configuration group is ignored in upgrade.xml
[ https://issues.apache.org/jira/browse/AMBARI-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20780: Resolution: Fixed Status: Resolved (was: Patch Available) > Update HBase Configuration group is ignored in upgrade.xml > -- > > Key: AMBARI-20780 > URL: https://issues.apache.org/jira/browse/AMBARI-20780 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20780.patch > > > The Update HBase Configuration group is ignored in upgrade.xml because it has > an invalid syntax. It should be of xsi:type="cluster" because it contains an > execute-stage. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20780) Update HBase Configuration group is ignored in upgrade.xml
[ https://issues.apache.org/jira/browse/AMBARI-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20780: Attachment: AMBARI-20780.patch > Update HBase Configuration group is ignored in upgrade.xml > -- > > Key: AMBARI-20780 > URL: https://issues.apache.org/jira/browse/AMBARI-20780 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20780.patch > > > The Update HBase Configuration group is ignored in upgrade.xml because it has > an invalid syntax. It should be of xsi:type="cluster" because it contains an > execute-stage. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20780) Update HBase Configuration group is ignored in upgrade.xml
[ https://issues.apache.org/jira/browse/AMBARI-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20780: Status: Patch Available (was: Open) > Update HBase Configuration group is ignored in upgrade.xml > -- > > Key: AMBARI-20780 > URL: https://issues.apache.org/jira/browse/AMBARI-20780 > Project: Ambari > Issue Type: Bug > Components: ambari-server >Affects Versions: 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20780.patch > > > The Update HBase Configuration group is ignored in upgrade.xml because it has > an invalid syntax. It should be of xsi:type="cluster" because it contains an > execute-stage. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AMBARI-20780) Update HBase Configuration group is ignored in upgrade.xml
Tim Thorpe created AMBARI-20780: --- Summary: Update HBase Configuration group is ignored in upgrade.xml Key: AMBARI-20780 URL: https://issues.apache.org/jira/browse/AMBARI-20780 Project: Ambari Issue Type: Bug Components: ambari-server Affects Versions: 2.5.0 Reporter: Tim Thorpe Assignee: Tim Thorpe Fix For: trunk The Update HBase Configuration group is ignored in upgrade.xml because it has an invalid syntax. It should be of xsi:type="cluster" because it contains an execute-stage. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20754) get_value_from_jmx constantly prints exception message in retry mechanism, which brings bad user experience
[
https://issues.apache.org/jira/browse/AMBARI-20754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Thorpe updated AMBARI-20754:
Fix Version/s: trunk
> get_value_from_jmx constantly prints exception message in retry mechanism,
> which brings bad user experience
> ---
>
> Key: AMBARI-20754
> URL: https://issues.apache.org/jira/browse/AMBARI-20754
> Project: Ambari
> Issue Type: Bug
>Reporter: Yuanbo Liu
>Assignee: Yuanbo Liu
> Fix For: trunk
>
> Attachments: AMBARI-20754.001.patch
>
>
> {{get_value_from_jmx}} of {{jmx.py}} is used in getting NameNode HA state. As
> we know, if the cluster is large, it takes a long time for Namenode to leave
> safe mode when restarting Namenode, thus we use retry mechanism to invoke
> {{get_value_from_jmx}} in case of getting wrong state. The problem is that,
> {{get_value_from_jmx}} will print several exception message into std_error
> during retrying, it confuses users because there're error messages in
> std_error, while all the services restart successfully. Here are the error
> messages:
> {quote}
> 2017-04-12 15:12:56,633 - Getting jmx metrics from NN failed. URL:
> http://:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem
> Traceback (most recent call last):
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py",
> line 38, in get_value_from_jmx
>_, data, _ = get_user_call_output(cmd, user=run_user, quiet=False)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py",
> line 61, in get_user_call_output
>raise ExecutionFailed(err_msg, code, files_output[0], files_output[1])
> ExecutionFailed: Execution of 'curl --negotiate -u : -s
> 'http://:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem'
> 1>/tmp/tmpWp05DF 2>/tmp/tmphm2dny' returned 7.
> 2017-04-12 15:12:58,562 - Getting jmx metrics from NN failed. URL:
> http://:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem
> Traceback (most recent call last):
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py",
> line 42, in get_value_from_jmx
>return data_dict["beans"][0][property]
> IndexError: list index out of range
> {quote}
> We should improve it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (AMBARI-20757) Selecting ignore service checks results in blank dialog box when hitting an error
[ https://issues.apache.org/jira/browse/AMBARI-20757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968134#comment-15968134 ] Tim Thorpe commented on AMBARI-20757: - The failure seen is in ambari-server code and can't be related to my UI change. java.lang.AssertionError: expected:<6> but was:<7> at org.apache.ambari.server.state.UpgradeHelperTest.testServiceCheckUpgradeStages(UpgradeHelperTest.java:1091) > Selecting ignore service checks results in blank dialog box when hitting an > error > - > > Key: AMBARI-20757 > URL: https://issues.apache.org/jira/browse/AMBARI-20757 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20757.patch > > > In order to see what has gone wrong, you need to drill down into the last > step until you find the service check that failed. Ideally there would be a > message above where they normally appear. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20757) Selecting ignore service checks results in blank dialog box when hitting an error
[ https://issues.apache.org/jira/browse/AMBARI-20757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20757: Resolution: Fixed Status: Resolved (was: Patch Available) > Selecting ignore service checks results in blank dialog box when hitting an > error > - > > Key: AMBARI-20757 > URL: https://issues.apache.org/jira/browse/AMBARI-20757 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20757.patch > > > In order to see what has gone wrong, you need to drill down into the last > step until you find the service check that failed. Ideally there would be a > message above where they normally appear. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20757) Selecting ignore service checks results in blank dialog box when hitting an error
[ https://issues.apache.org/jira/browse/AMBARI-20757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20757: Attachment: AMBARI-20757.patch > Selecting ignore service checks results in blank dialog box when hitting an > error > - > > Key: AMBARI-20757 > URL: https://issues.apache.org/jira/browse/AMBARI-20757 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20757.patch > > > In order to see what has gone wrong, you need to drill down into the last > step until you find the service check that failed. Ideally there would be a > message above where they normally appear. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20757) Selecting ignore service checks results in blank dialog box when hitting an error
[ https://issues.apache.org/jira/browse/AMBARI-20757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20757: Status: Patch Available (was: Open) > Selecting ignore service checks results in blank dialog box when hitting an > error > - > > Key: AMBARI-20757 > URL: https://issues.apache.org/jira/browse/AMBARI-20757 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > Attachments: AMBARI-20757.patch > > > In order to see what has gone wrong, you need to drill down into the last > step until you find the service check that failed. Ideally there would be a > message above where they normally appear. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AMBARI-20757) Selecting ignore service checks results in blank dialog box when hitting an error
[ https://issues.apache.org/jira/browse/AMBARI-20757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Thorpe updated AMBARI-20757: Attachment: (was: AMBARI-20757.patch) > Selecting ignore service checks results in blank dialog box when hitting an > error > - > > Key: AMBARI-20757 > URL: https://issues.apache.org/jira/browse/AMBARI-20757 > Project: Ambari > Issue Type: Bug > Components: ambari-web >Affects Versions: trunk, 2.5.0 >Reporter: Tim Thorpe >Assignee: Tim Thorpe > Fix For: trunk > > > In order to see what has gone wrong, you need to drill down into the last > step until you find the service check that failed. Ideally there would be a > message above where they normally appear. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
