Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

[Vishal Ghugare]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/
---

(Updated Oct. 13, 2016, 8:25 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.


Bugs: AMBARI-12263
https://issues.apache.org/jira/browse/AMBARI-12263


Repository: ambari


Description
---

Hello Robert,

How are you doing? 

We have been working on PAM support into Ambari and have something ready for 
review. Can you please take a look at the patch and documentation and provide 
your feedback.

Please let me know if you have any questions.

Note: I have added you as a reviewer as i see some authentication related 
commits under your name.

Thanks,
-Vishal


Diffs
-

  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
78a6884 
  ambari-agent/conf/unix/ambari-agent 30897dd 
  ambari-agent/conf/unix/ambari-agent.ini 1c39c24 
  ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
  ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
  ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py b476671 
  ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 6caf1d0 
  ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
  ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 32773b8 
  ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
a4bdc1c 
  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
66227c6 
  ambari-common/src/main/python/ambari_commons/inet_utils.py b5cea75 
  ambari-common/src/main/python/resource_management/core/exceptions.py a59611d 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 21de183 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
d095173 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 ea10a86 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 265c162 
  ambari-common/src/main/python/resource_management/core/shell.py f8f056a 
  
ambari-common/src/main/python/resource_management/libraries/functions/data_structure_utils.py
 59e01aa 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
 7b7acae 
  
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 f1aa3e1 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 133646a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 0d4deab 
  ambari-logsearch/pom.xml a1b6c9d 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 efa5cba 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
 e7f7cfd 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
 3d00270 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 36ec074 
  
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
 c1b684b 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 d81187e 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 3259c18 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/blacklisted_set.py
 dab54c0 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 890d3ce 
  ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
ba3f18e 
  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 fef1f24 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 ef73a0e 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 60c1427 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 802e57d 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 5a3eac1 
  ambari-metrics/ambari-metrics-timelineservice/pom.xml 6896c6b 
  ambari-metrics/pom.xml 

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

[Vishal Ghugare]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/
---

(Updated Oct. 13, 2016, 7:58 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.


Bugs: AMBARI-12263
https://issues.apache.org/jira/browse/AMBARI-12263


Repository: ambari


Description
---

Hello Robert,

How are you doing? 

We have been working on PAM support into Ambari and have something ready for 
review. Can you please take a look at the patch and documentation and provide 
your feedback.

Please let me know if you have any questions.

Note: I have added you as a reviewer as i see some authentication related 
commits under your name.

Thanks,
-Vishal


Diffs
-

  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
78a6884 
  ambari-agent/conf/unix/ambari-agent 30897dd 
  ambari-agent/conf/unix/ambari-agent.ini 1c39c24 
  ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
  ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
  ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py b476671 
  ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 6caf1d0 
  ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
  ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 32773b8 
  ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
a4bdc1c 
  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
66227c6 
  ambari-common/src/main/python/ambari_commons/inet_utils.py b5cea75 
  ambari-common/src/main/python/resource_management/core/exceptions.py a59611d 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 21de183 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
d095173 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 ea10a86 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 265c162 
  ambari-common/src/main/python/resource_management/core/shell.py f8f056a 
  
ambari-common/src/main/python/resource_management/libraries/functions/data_structure_utils.py
 59e01aa 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
 7b7acae 
  
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 f1aa3e1 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 133646a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 0d4deab 
  ambari-logsearch/pom.xml a1b6c9d 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 efa5cba 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
 e7f7cfd 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
 3d00270 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 36ec074 
  
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
 c1b684b 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 d81187e 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 3259c18 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/blacklisted_set.py
 dab54c0 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 890d3ce 
  ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
ba3f18e 
  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 fef1f24 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 ef73a0e 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 60c1427 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 802e57d 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 5a3eac1 
  ambari-metrics/ambari-metrics-timelineservice/pom.xml 6896c6b 
  ambari-metrics/pom.xml 

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

[Vishal Ghugare]

> On Oct. 7, 2016, 10:55 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java,
> >  line 817
> > 
> >
> > Since this is called each time Ambari starts up new groups can be 
> > _magically_ created each time Ambari is started.  This seems like a 
> > possible issue since it gives a non-Ambari-administrator the ability to 
> > create groups and assign roles to them. In many cases, the user that has 
> > write access to the ambari.properties file does not have admin access to 
> > Ambari. So being able to change something like this becomes a security 
> > hole. 
> > 
> > If we do find a way to do this securely, the solution should be more 
> > generic since it may not apply only to PAM.

we could possibly do the PAM group creation securely & in a generic way by 
invoking a rest api (a new api).


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review151829
---


On Oct. 13, 2016, 6:45 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 13, 2016, 6:45 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
> 78a6884 
>   ambari-agent/conf/unix/ambari-agent 30897dd 
>   ambari-agent/conf/unix/ambari-agent.ini 1c39c24 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
>   ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py 
> b476671 
>   ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 6caf1d0 
>   ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
>   ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 32773b8 
>   ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
> a4bdc1c 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c6 
>   ambari-common/src/main/python/ambari_commons/inet_utils.py b5cea75 
>   ambari-common/src/main/python/resource_management/core/exceptions.py 
> a59611d 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de183 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a86 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c162 
>   ambari-common/src/main/python/resource_management/core/shell.py f8f056a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/data_structure_utils.py
>  59e01aa 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
>  7b7acae 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  f1aa3e1 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
>  133646a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
>  0d4deab 
>   ambari-logsearch/pom.xml a1b6c9d 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
>  efa5cba 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
>  e7f7cfd 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
>  3d00270 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
>  36ec074 
>   
> 

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

[Vishal Ghugare]

> On Oct. 3, 2016, 1:17 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java,
> >  lines 955-960
> > 
> >
> > roles may be set today, but in the future these may be customizabled by 
> > the user.
> 
> Vishal Ghugare wrote:
> -is anyone already working on it? is there any JIRA to track this work? 
> -How is upgrade handled in this case (upgrade from version which has 
> predefined roles to version which has configurable roles)?
> 
> Robert Levas wrote:
> I am not sure how this will work yet, any there are no JIRA open, but I 
> would still avoid hard-coding this. That said, I am not sure why there is a 
> need to make this part of the configuration data.
> 
> Vishal Ghugare wrote:
> The hardcoded roles are to help customers make use of the predefined 
> roles in Ambari (Cluster Admin, Cluster Operator and so on) and bootstrap 
> their  PAM authorization. I understand that the predefined roles (and their 
> names) may be customizable by the user in the future, but  at the minimum, 
> the "Admin " role will still need to be predefined since the pre-created 
> "admin" user should be removed in the near future for PAM/LDAP use-case as it 
> creates a security hole.  When the change to make roles configurable happens, 
> the PAM related code in Users.java & setupSecurity.py will also need to be 
> changed accordingly (not sure how these customized roles will be defined 
> initially).  Having the choice (which is optional) to create custom groups 
> (with assigned roles) during PAM setup gives user an entry point to 
> boot-strap the authorization in Ambari.   Also these custom groups are part 
> of the pam setup (just like any other setup for example setup-ldap) and it 
> makes sense for the propert
 ies to be stored into configuration file.
> 
> Robert Levas wrote:
> It is not clear to me why we need to preload Ambari with certain groups 
> when PAM authentcation is enabled. We do not do this with other _remote_ 
> authentication facilities - like LDAP.  It seems like if this is needed, we 
> might be able to come up with a more generic way to handle it since it could 
> be useful elsewhere.

Thank you for reviewing the patch.

"It is not clear to me why we need to preload Ambari with certain groups when 
PAM authentcation is enabled."
-When there are no locally created users exist in ambari (currently admin local 
user is created by default), the preloaded groups will become entry point to 
get into Ambari.
-When PAM is enabled, the PAM user & all the groups (the user belongs to) are 
created into database only when a user logs into Ambari, so without any 
predefined users (local admin user) & groups (preloaded PAM groups) there is no 
way to login to Ambari.


"We do not do this with other remote authentication facilities - like LDAP."
-In my previous comment, i was referring to the properties 
(authentication.ldap.baseDn, authentication.ldap.primaryUrl ... all the 
properties that get added during ldap setup).

"It seems like if this is needed, we might be able to come up with a more 
generic way to handle it since it could be useful elsewhere."
-Please let us know if you any suggestions on doing it in more generic way. One 
that i could think of is to have a separate setup action for creating these 
groups.


If you have some time, i could setup a conference call to discuss this.


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review151223
---


On Oct. 13, 2016, 6:45 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 13, 2016, 6:45 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
> 78a6884 
>   ambari-agent/conf/unix/ambari-agent 30897dd 
>   ambari-agent/conf/unix/ambari-agent.ini 1c39c24 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
>   

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Nate Cole]

> On Oct. 13, 2016, 7:10 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  line 512
> > 
> >
> > I'm curious - why use the configuration property getDefaultValue()?  
> > Shouldn't there be a Configuration accessor that actually checks for the 
> > property being set?
> 
> Amarnath reddy pappu wrote:
> I am not sure if having a configurable value would be useful in this 
> case. 5mins default value would be good enough i feel.
> 
> do you think there may be customers who would want to enable the DEBUG 
> logs less than 5mins time? that is the only case i can think of.
> 
> please advise so that I can proceed accordingly.

Well, this requested feature didn't originate with me :)  It just seems odd to 
have a configuration that could never be used.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review152595
---


On Oct. 13, 2016, 2:01 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 13, 2016, 2:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

[Vishal Ghugare]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/
---

(Updated Oct. 13, 2016, 6:45 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.


Bugs: AMBARI-12263
https://issues.apache.org/jira/browse/AMBARI-12263


Repository: ambari


Description
---

Hello Robert,

How are you doing? 

We have been working on PAM support into Ambari and have something ready for 
review. Can you please take a look at the patch and documentation and provide 
your feedback.

Please let me know if you have any questions.

Note: I have added you as a reviewer as i see some authentication related 
commits under your name.

Thanks,
-Vishal


Diffs (updated)
-

  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
78a6884 
  ambari-agent/conf/unix/ambari-agent 30897dd 
  ambari-agent/conf/unix/ambari-agent.ini 1c39c24 
  ambari-agent/src/main/python/ambari_agent/ActionQueue.py c03ee4f 
  ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
  ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py b476671 
  ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 6caf1d0 
  ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
  ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 32773b8 
  ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
a4bdc1c 
  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
66227c6 
  ambari-common/src/main/python/ambari_commons/inet_utils.py b5cea75 
  ambari-common/src/main/python/resource_management/core/exceptions.py a59611d 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 21de183 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
d095173 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 ea10a86 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 265c162 
  ambari-common/src/main/python/resource_management/core/shell.py f8f056a 
  
ambari-common/src/main/python/resource_management/libraries/functions/data_structure_utils.py
 59e01aa 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
 7b7acae 
  
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 f1aa3e1 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 133646a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 0d4deab 
  ambari-logsearch/pom.xml a1b6c9d 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 efa5cba 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
 e7f7cfd 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
 3d00270 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 36ec074 
  
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
 c1b684b 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 d81187e 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 3259c18 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/blacklisted_set.py
 dab54c0 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 890d3ce 
  ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
ba3f18e 
  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 fef1f24 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 ef73a0e 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 60c1427 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 802e57d 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 5a3eac1 
  ambari-metrics/ambari-metrics-timelineservice/pom.xml 6896c6b 
  

Review Request 52851: rpm command broken on utility project

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52851/
---

Review request for Ambari, Dmytro Grinenko and Sumit Mohanty.


Repository: ambari


Description
---

The following command to build RPMs is failing because the utility project does 
not have a phase for it.

mvn -B -am clean package rpm:rpm -DskipTests -Dpython.ver="python >= 2.6" -pl 
ambari-admin,ambari-agent,utility,ambari-server,ambari-web,ambari-views


[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [1.034s]
[INFO] Apache Ambari Project POM . SUCCESS [0.064s]
[INFO] Ambari Web  SUCCESS [1:58.669s]
[INFO] Ambari Views .. SUCCESS [1.403s]
[INFO] Ambari Admin View . SUCCESS [1:13.669s]
[INFO] utility ... FAILURE [2.306s]
[INFO] ambari-metrics  SKIPPED
[INFO] Ambari Metrics Common . SKIPPED
[INFO] Ambari Server . SKIPPED
[INFO] Ambari Agent .. SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 3:17.814s
[INFO] Finished at: Wed Oct 12 12:48:37 UTC 2016
[INFO] Final Memory: 35M/563M
[INFO] 
[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.1.5:rpm 
(default-cli) on project utility: The parameters 'sourceEncoding', 'group' for 
goal org.codehaus.mojo:rpm-maven-plugin:2.1.5:rpm are missing or invalid -> 
[Help 1]


Diffs
-

  utility/pom.xml 9f3fe32 

Diff: https://reviews.apache.org/r/52851/diff/


Testing
---

Verified it works now.


Thanks,

Alejandro Fernandez

Review Request 52850: Icons don't display in Windows 10 Microsoft Edge or Internet Explorer 11

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52850/
---

Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, Richard Zang, Xi Wang, 
and Yusaku Sako.


Bugs: AMBARI-18595
https://issues.apache.org/jira/browse/AMBARI-18595


Repository: ambari


Description
---

Icons do not display in any page as shown in the screen-shot attached.


Diffs
-

  ambari-web/pom.xml 759b63e 

Diff: https://reviews.apache.org/r/52850/diff/


Testing
---

Verified Manually in the latest versions of Google Chrome Mac OS X, Google 
Chrome Windows 10, Microsoft Internet Explorer 11 and Microsoft Edge.
Ambari web unit tests pass.
30374 tests complete (30 seconds)
151 tests pending


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

> On Oct. 13, 2016, 6:20 p.m., Alejandro Fernandez wrote:
> > Ship It!

thank you


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review152549
---


On Oct. 13, 2016, 6:01 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 13, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

> On Oct. 13, 2016, 11:10 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  line 512
> > 
> >
> > I'm curious - why use the configuration property getDefaultValue()?  
> > Shouldn't there be a Configuration accessor that actually checks for the 
> > property being set?

I am not sure if having a configurable value would be useful in this case. 
5mins default value would be good enough i feel.

do you think there may be customers who would want to enable the DEBUG logs 
less than 5mins time? that is the only case i can think of.

please advise so that I can proceed accordingly.


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review152595
---


On Oct. 13, 2016, 6:01 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 13, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review152595
---




ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 (line 512)


I'm curious - why use the configuration property getDefaultValue()?  
Shouldn't there be a Configuration accessor that actually checks for the 
property being set?


- Nate Cole


On Oct. 13, 2016, 2:01 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 13, 2016, 2:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52837: Hive Rolling Upgrade Is No Longer Supported In Ambari

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52837/#review152594
---


Ship it!




Ship It!

- Nate Cole


On Oct. 13, 2016, 1:17 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52837/
> ---
> 
> (Updated Oct. 13, 2016, 1:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-18591
> https://issues.apache.org/jira/browse/AMBARI-18591
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HS2 RU expects all users to be using zookeeper based JDBC URLs. However, few 
> users are using these new Zookeeper discovery urls and they use the old 
> format, directly specifying the hostname and port number for the HS2 instance 
> it connects to. During RU, the HS2 port number changes and users using old 
> url have an outage.  Hive needs to be able to support RU without changing the 
> port.
> 
> Due to too many Hive Client user outages as a result of the RU workflow for 
> HIve, the decision is to just change RU to bounce the HS2 process, knowing 
> that there will be down time and causing all active queries to fail.  So the 
> runbook for the HS2 process will look like:
> 
> * We will no longer change the Hive port at all during either RU or EU.
> * During an RU, we will present a message if Hive is present that indicates 
> that Hive is not rolling and it will be taken down.
> * Run the Hive deregister command to remove the ZK entries for the Hive 
> Servers
> * Hive will be shutdown, without draining, during an RU and a new instance 
> started up on the same port.
> 
> In addition, we need to remove the warnings that were added as part of 
> AMBARI-17319.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  7fdd0ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveNotRollingWarning.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarning.java
>  3db31ea 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
>  614b2a9 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_upgrade.py
>  478c240 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml 
> 5957da4 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 6920b2c 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> 1b04237 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> 8c5f9aa 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 6105665 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 9b9848e 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 51fbcbf 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> cca4b14 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> db59f8c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> 17d7d9c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 921fc9c 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> 962d92e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 28b34a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveNotRollingWarningTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarningTest.java
>  cf76d12 
> 
> Diff: https://reviews.apache.org/r/52837/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 28:16 min
> [INFO] Finished at: 2016-10-13T12:18:24-04:00
> [INFO] Final Memory: 60M/720M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52843: BP format does not generate property oozie.service.JPAService.jdbc.url

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52843/#review152579
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 13, 2016, 8:58 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52843/
> ---
> 
> (Updated Oct. 13, 2016, 8:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18546
> https://issues.apache.org/jira/browse/AMBARI-18546
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> BlueprintConfigurationProcessorTest.testOozieConfigExported test fail
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  b49abe9 
> 
> Diff: https://reviews.apache.org/r/52843/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 52843: BP format does not generate property oozie.service.JPAService.jdbc.url

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52843/
---

Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18546
https://issues.apache.org/jira/browse/AMBARI-18546


Repository: ambari


Description
---

BlueprintConfigurationProcessorTest.testOozieConfigExported test fail


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 b49abe9 

Diff: https://reviews.apache.org/r/52843/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18593
https://issues.apache.org/jira/browse/AMBARI-18593


Repository: ambari


Description
---

HDFS exposes top user activity broken down by operations in jmx (nntop).
These metrics should be captured in AMS and exposed in Grafana's HDFS 
dashboards.

Downsampling should likely be a function like MIN, MAX, AVG, SUM of underlying 
timeseries specified from the client.


Diffs
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 ba7807b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
 c056d79 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
 118c695 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
 177e444 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
 f7060e0 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 c2aea8c 

Diff: https://reviews.apache.org/r/52842/diff/


Testing
---

Manually tested.
Unit tests added.
mvn clean test on ambari-metrics pending.


Thanks,

Aravindan Vijayan

Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

[Amarnath reddy pappu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sid Wagle.


Bugs: AMBARI-18592
https://issues.apache.org/jira/browse/AMBARI-18592


Repository: ambari


Description
---

Ability to enable the DEBUG logs in Ambari-server without server restart

1. PropertyConfigurator would check log4j.properties update every 5mins. if 
there is any change in the file/timestamp it will reload the log4j 
configurations.
2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
3. Have modified only for ambari-server
4.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 c05840e 

Diff: https://reviews.apache.org/r/52839/diff/


Testing
---

1. Change the log4j.properties from INFO to DEBUG
[Expected Result] server should start logging the DEBUG logs with in 5mins of 
time.
[Resutl] PASS

2. Change the log4j.properties from DEBUG to INFO
[Expected Result] server should reduce the logging to INFO mode with in 5mins 
of time.
[Resutl] PASS


Thanks,

Amarnath reddy pappu

Re: Review Request 52837: Hive Rolling Upgrade Is No Longer Supported In Ambari

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52837/#review152540
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 13, 2016, 5:17 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52837/
> ---
> 
> (Updated Oct. 13, 2016, 5:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-18591
> https://issues.apache.org/jira/browse/AMBARI-18591
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HS2 RU expects all users to be using zookeeper based JDBC URLs. However, few 
> users are using these new Zookeeper discovery urls and they use the old 
> format, directly specifying the hostname and port number for the HS2 instance 
> it connects to. During RU, the HS2 port number changes and users using old 
> url have an outage.  Hive needs to be able to support RU without changing the 
> port.
> 
> Due to too many Hive Client user outages as a result of the RU workflow for 
> HIve, the decision is to just change RU to bounce the HS2 process, knowing 
> that there will be down time and causing all active queries to fail.  So the 
> runbook for the HS2 process will look like:
> 
> * We will no longer change the Hive port at all during either RU or EU.
> * During an RU, we will present a message if Hive is present that indicates 
> that Hive is not rolling and it will be taken down.
> * Run the Hive deregister command to remove the ZK entries for the Hive 
> Servers
> * Hive will be shutdown, without draining, during an RU and a new instance 
> started up on the same port.
> 
> In addition, we need to remove the warnings that were added as part of 
> AMBARI-17319.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  7fdd0ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveNotRollingWarning.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarning.java
>  3db31ea 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
>  614b2a9 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_upgrade.py
>  478c240 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml 
> 5957da4 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 6920b2c 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> 1b04237 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> 8c5f9aa 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 6105665 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 9b9848e 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 51fbcbf 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> cca4b14 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> db59f8c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> 17d7d9c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 921fc9c 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> 962d92e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 28b34a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveNotRollingWarningTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarningTest.java
>  cf76d12 
> 
> Diff: https://reviews.apache.org/r/52837/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 28:16 min
> [INFO] Finished at: 2016-10-13T12:18:24-04:00
> [INFO] Final Memory: 60M/720M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52837: Hive Rolling Upgrade Is No Longer Supported In Ambari

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52837/#review152536
---




ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 (lines 293 - 298)


I'm open to new wording here if this isn't clear or needs more.


- Jonathan Hurley


On Oct. 13, 2016, 1:17 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52837/
> ---
> 
> (Updated Oct. 13, 2016, 1:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-18591
> https://issues.apache.org/jira/browse/AMBARI-18591
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HS2 RU expects all users to be using zookeeper based JDBC URLs. However, few 
> users are using these new Zookeeper discovery urls and they use the old 
> format, directly specifying the hostname and port number for the HS2 instance 
> it connects to. During RU, the HS2 port number changes and users using old 
> url have an outage.  Hive needs to be able to support RU without changing the 
> port.
> 
> Due to too many Hive Client user outages as a result of the RU workflow for 
> HIve, the decision is to just change RU to bounce the HS2 process, knowing 
> that there will be down time and causing all active queries to fail.  So the 
> runbook for the HS2 process will look like:
> 
> * We will no longer change the Hive port at all during either RU or EU.
> * During an RU, we will present a message if Hive is present that indicates 
> that Hive is not rolling and it will be taken down.
> * Run the Hive deregister command to remove the ZK entries for the Hive 
> Servers
> * Hive will be shutdown, without draining, during an RU and a new instance 
> started up on the same port.
> 
> In addition, we need to remove the warnings that were added as part of 
> AMBARI-17319.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  7fdd0ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveNotRollingWarning.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarning.java
>  3db31ea 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
>  614b2a9 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_upgrade.py
>  478c240 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml 
> 5957da4 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 6920b2c 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> 1b04237 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
> 8c5f9aa 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 6105665 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 9b9848e 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 51fbcbf 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> cca4b14 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> db59f8c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> 17d7d9c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 921fc9c 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> 962d92e 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 28b34a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveNotRollingWarningTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarningTest.java
>  cf76d12 
> 
> Diff: https://reviews.apache.org/r/52837/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 28:16 min
> [INFO] Finished at: 2016-10-13T12:18:24-04:00
> [INFO] Final Memory: 60M/720M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 52837: Hive Rolling Upgrade Is No Longer Supported In Ambari

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52837/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-18591
https://issues.apache.org/jira/browse/AMBARI-18591


Repository: ambari


Description
---

HS2 RU expects all users to be using zookeeper based JDBC URLs. However, few 
users are using these new Zookeeper discovery urls and they use the old format, 
directly specifying the hostname and port number for the HS2 instance it 
connects to. During RU, the HS2 port number changes and users using old url 
have an outage.  Hive needs to be able to support RU without changing the port.

Due to too many Hive Client user outages as a result of the RU workflow for 
HIve, the decision is to just change RU to bounce the HS2 process, knowing that 
there will be down time and causing all active queries to fail.  So the runbook 
for the HS2 process will look like:

* We will no longer change the Hive port at all during either RU or EU.
* During an RU, we will present a message if Hive is present that indicates 
that Hive is not rolling and it will be taken down.
* Run the Hive deregister command to remove the ZK entries for the Hive Servers
* Hive will be shutdown, without draining, during an RU and a new instance 
started up on the same port.

In addition, we need to remove the warnings that were added as part of 
AMBARI-17319.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 7fdd0ce 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/HiveNotRollingWarning.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarning.java
 3db31ea 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
 614b2a9 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_upgrade.py
 478c240 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml 
5957da4 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
6920b2c 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
1b04237 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
8c5f9aa 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
6105665 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
9b9848e 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
51fbcbf 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
cca4b14 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
db59f8c 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
17d7d9c 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
921fc9c 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
962d92e 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
28b34a2 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/HiveNotRollingWarningTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/HiveRollingPortChangeWarningTest.java
 cf76d12 

Diff: https://reviews.apache.org/r/52837/diff/


Testing
---

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 28:16 min
[INFO] Finished at: 2016-10-13T12:18:24-04:00
[INFO] Final Memory: 60M/720M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Alejandro Fernandez]

> On Oct. 13, 2016, 5:02 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py,
> >  line 64
> > 
> >
> > Let's decrease the sleep time to 10 secs.
> 
> Jonathan Hurley wrote:
> Is there a reason? I am hesistent against changing stuff like this; we've 
> been burned before where customer environments take much longer than we 
> think. HBase, especially, as it rebuilds regions.

Maybe decrease sleep by factor of 3 and increase attempts by same number.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/#review152531
---


On Oct. 13, 2016, 3:19 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52833/
> ---
> 
> (Updated Oct. 13, 2016, 3:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-18590
> https://issues.apache.org/jira/browse/AMBARI-18590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade, the upgrade orchestration must wait for each 
> RegionServer to register with the HBase master before moving onto the next RS 
> restart. This is a very asynchronous process which may occur several minutes 
> after the daemon has actually started. 
> 
> We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
> to determine if the host has registered by looking for the hostname. 
> 
> However, if reverse DNS is not enabled, then this could potentially be IP 
> addresses. As a result, the check would always fail during upgrades:
> 
> The HBase status command we use is {{status simple}}, which returns like so:
> 
> ```
> active master:  10.0.0.8:16000 1475801031124
> 2 backup masters
> 10.0.0.10:16000 1475801061290
> 10.0.0.13:16000 1475801046018
> 2 live servers
> 10.0.0.5:16020 1475798271407
> requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
> maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
> coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
> 10.0.0.7:16020 1475872741297
> requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
> maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
> coprocessors=[SecureBulkLoadEndpoint]
> 0 dead servers
> Aggregate load: 0, regions: 3
> ```
> 
> If this lookup fails for the hostname, we should also try by IP address.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  f1fa80c 
> 
> Diff: https://reviews.apache.org/r/52833/diff/
> 
> 
> Testing
> ---
> 
> Total run:1133
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Nate Cole]

> On Oct. 13, 2016, 1:02 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py,
> >  line 64
> > 
> >
> > Let's decrease the sleep time to 10 secs.
> 
> Jonathan Hurley wrote:
> Is there a reason? I am hesistent against changing stuff like this; we've 
> been burned before where customer environments take much longer than we 
> think. HBase, especially, as it rebuilds regions.
> 
> Alejandro Fernandez wrote:
> Maybe decrease sleep by factor of 3 and increase attempts by same number.

Doubly long for the environment which is requiring this change in the first 
place.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/#review152531
---


On Oct. 13, 2016, 11:19 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52833/
> ---
> 
> (Updated Oct. 13, 2016, 11:19 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-18590
> https://issues.apache.org/jira/browse/AMBARI-18590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade, the upgrade orchestration must wait for each 
> RegionServer to register with the HBase master before moving onto the next RS 
> restart. This is a very asynchronous process which may occur several minutes 
> after the daemon has actually started. 
> 
> We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
> to determine if the host has registered by looking for the hostname. 
> 
> However, if reverse DNS is not enabled, then this could potentially be IP 
> addresses. As a result, the check would always fail during upgrades:
> 
> The HBase status command we use is {{status simple}}, which returns like so:
> 
> ```
> active master:  10.0.0.8:16000 1475801031124
> 2 backup masters
> 10.0.0.10:16000 1475801061290
> 10.0.0.13:16000 1475801046018
> 2 live servers
> 10.0.0.5:16020 1475798271407
> requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
> maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
> coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
> 10.0.0.7:16020 1475872741297
> requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
> maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
> coprocessors=[SecureBulkLoadEndpoint]
> 0 dead servers
> Aggregate load: 0, regions: 3
> ```
> 
> If this lookup fails for the hostname, we should also try by IP address.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  f1fa80c 
> 
> Diff: https://reviews.apache.org/r/52833/diff/
> 
> 
> Testing
> ---
> 
> Total run:1133
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Jonathan Hurley]

> On Oct. 13, 2016, 1:02 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py,
> >  line 64
> > 
> >
> > Let's decrease the sleep time to 10 secs.

Is there a reason? I am hesistent against changing stuff like this; we've been 
burned before where customer environments take much longer than we think. 
HBase, especially, as it rebuilds regions.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/#review152531
---


On Oct. 13, 2016, 11:19 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52833/
> ---
> 
> (Updated Oct. 13, 2016, 11:19 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-18590
> https://issues.apache.org/jira/browse/AMBARI-18590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade, the upgrade orchestration must wait for each 
> RegionServer to register with the HBase master before moving onto the next RS 
> restart. This is a very asynchronous process which may occur several minutes 
> after the daemon has actually started. 
> 
> We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
> to determine if the host has registered by looking for the hostname. 
> 
> However, if reverse DNS is not enabled, then this could potentially be IP 
> addresses. As a result, the check would always fail during upgrades:
> 
> The HBase status command we use is {{status simple}}, which returns like so:
> 
> ```
> active master:  10.0.0.8:16000 1475801031124
> 2 backup masters
> 10.0.0.10:16000 1475801061290
> 10.0.0.13:16000 1475801046018
> 2 live servers
> 10.0.0.5:16020 1475798271407
> requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
> maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
> coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
> 10.0.0.7:16020 1475872741297
> requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
> maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
> coprocessors=[SecureBulkLoadEndpoint]
> 0 dead servers
> Aggregate load: 0, regions: 3
> ```
> 
> If this lookup fails for the hostname, we should also try by IP address.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  f1fa80c 
> 
> Diff: https://reviews.apache.org/r/52833/diff/
> 
> 
> Testing
> ---
> 
> Total run:1133
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/#review152531
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
 (line 62)


Let's decrease the sleep time to 10 secs.


- Alejandro Fernandez


On Oct. 13, 2016, 3:19 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52833/
> ---
> 
> (Updated Oct. 13, 2016, 3:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-18590
> https://issues.apache.org/jira/browse/AMBARI-18590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade, the upgrade orchestration must wait for each 
> RegionServer to register with the HBase master before moving onto the next RS 
> restart. This is a very asynchronous process which may occur several minutes 
> after the daemon has actually started. 
> 
> We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
> to determine if the host has registered by looking for the hostname. 
> 
> However, if reverse DNS is not enabled, then this could potentially be IP 
> addresses. As a result, the check would always fail during upgrades:
> 
> The HBase status command we use is {{status simple}}, which returns like so:
> 
> ```
> active master:  10.0.0.8:16000 1475801031124
> 2 backup masters
> 10.0.0.10:16000 1475801061290
> 10.0.0.13:16000 1475801046018
> 2 live servers
> 10.0.0.5:16020 1475798271407
> requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
> maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
> coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
> 10.0.0.7:16020 1475872741297
> requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
> maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
> coprocessors=[SecureBulkLoadEndpoint]
> 0 dead servers
> Aggregate load: 0, regions: 3
> ```
> 
> If this lookup fails for the hostname, we should also try by IP address.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  f1fa80c 
> 
> Diff: https://reviews.apache.org/r/52833/diff/
> 
> 
> Testing
> ---
> 
> Total run:1133
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/#review152529
---


Ship it!




Ship It!

- Nate Cole


On Oct. 13, 2016, 11:19 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52833/
> ---
> 
> (Updated Oct. 13, 2016, 11:19 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-18590
> https://issues.apache.org/jira/browse/AMBARI-18590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a rolling upgrade, the upgrade orchestration must wait for each 
> RegionServer to register with the HBase master before moving onto the next RS 
> restart. This is a very asynchronous process which may occur several minutes 
> after the daemon has actually started. 
> 
> We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
> to determine if the host has registered by looking for the hostname. 
> 
> However, if reverse DNS is not enabled, then this could potentially be IP 
> addresses. As a result, the check would always fail during upgrades:
> 
> The HBase status command we use is {{status simple}}, which returns like so:
> 
> ```
> active master:  10.0.0.8:16000 1475801031124
> 2 backup masters
> 10.0.0.10:16000 1475801061290
> 10.0.0.13:16000 1475801046018
> 2 live servers
> 10.0.0.5:16020 1475798271407
> requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
> maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
> coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
> 10.0.0.7:16020 1475872741297
> requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
> maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
> storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
> storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
> rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
> totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
> coprocessors=[SecureBulkLoadEndpoint]
> 0 dead servers
> Aggregate load: 0, regions: 3
> ```
> 
> If this lookup fails for the hostname, we should also try by IP address.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  f1fa80c 
> 
> Diff: https://reviews.apache.org/r/52833/diff/
> 
> 
> Testing
> ---
> 
> Total run:1133
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152527
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 13, 2016, 7:41 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 7:41 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml 0ed79ac 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/
---

(Updated Oct. 13, 2016, 4:41 p.m.)


Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18586
https://issues.apache.org/jira/browse/AMBARI-18586


Repository: ambari


Description
---

Increase forkCount for Ambari Server Unit Tests


Diffs (updated)
-

  ambari-project/pom.xml 0ed79ac 

Diff: https://reviews.apache.org/r/52826/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Jonathan Hurley]

> On Oct. 13, 2016, 9:40 a.m., Sumit Mohanty wrote:
> > ambari-project/pom.xml, line 516
> > 
> >
> > Why 1.5C?
> 
> Jonathan Hurley wrote:
> 1.5C means 1.5 x # of cores - so that lowered powered machines (like 
> build machines) don't spawn too many. We should test 1.5C on some of the 
> known developer configurations (I know that a lot of people use macbook pros 
> and a lot use linux desktops).
> 
> Andrew Onischuk wrote:
> On 4 core linux env this gives 5 processes and leverages all CPU cores.

On my MBP, it yields 12 forks! That's a lot :) ... Maybe we should make this 1C 
instead ...


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152494
---


On Oct. 13, 2016, 9:24 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 9:24 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52747: Migration to Bootstrap3 framework

[Yusaku Sako]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52747/#review152522
---


Ship it!




Ship It!

- Yusaku Sako


On Oct. 13, 2016, 1:48 p.m., Aleksandr Kovalenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52747/
> ---
> 
> (Updated Oct. 13, 2016, 1:48 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Zhe (Joe) Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18573
> https://issues.apache.org/jira/browse/AMBARI-18573
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Migrate from Bootstrap2 to Bootstrap3 framework.
> 
> 
> Diffs
> -
> 
>   LICENSE.txt 906c0590c2a749b910139dee2af70e4b4b17c949 
>   ambari-web/app/app.js 53394a055bb5997c3589b0e18413e2f984b6 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.eot PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.svg PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.ttf PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff2 PRE-CREATION 
>   ambari-web/app/config.js 4550eb8905d6064f44a194d31ee3727c55e9bfb5 
>   ambari-web/app/controllers/application.js 
> dca689f949dfa3583fac3e2d181e2d876716664d 
>   ambari-web/app/controllers/main/admin/service_auto_start.js 
> b425c785770ef05c4b7b519d50645be0f1b655ea 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> d00a652f8da42fba20fb86394e06dcedba18edfc 
>   ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
> f05d366057d75ca47418891c86504f18e8e3f1c4 
>   
> ambari-web/app/controllers/main/alerts/add_alert_definition/step1_controller.js
>  43e25c8c5338d9c264aa8346cce1b12d195f64f0 
>   
> ambari-web/app/controllers/main/alerts/alert_definitions_actions_controller.js
>  6ca3e786d65cfe515615e10aa141a3d5dca2fbd5 
>   ambari-web/app/controllers/main/alerts/alert_instances_controller.js 
> 5913070d3ba799b6d2415361381aea781b1f313d 
>   ambari-web/app/controllers/main/alerts/manage_alert_groups_controller.js 
> 6ba595738234aa840de8bdbca47884a5d699b843 
>   ambari-web/app/controllers/main/charts/heatmap.js 
> 73d18005202e956b2af4732abf3467a72a7d96f2 
>   ambari-web/app/controllers/main/service/info/summary.js 
> c16bb3fc728fe89743d308e0ab96b073e5a8845f 
>   ambari-web/app/controllers/main/service/manage_config_groups_controller.js 
> 539d03eb3d0c5b0e7ea8e5284b5a1c796d3af3cc 
>   ambari-web/app/controllers/wizard/step10_controller.js 
> 8c4b5dbe371e78fd5cb0bc7e87e0a7a965486f2d 
>   ambari-web/app/controllers/wizard/step1_controller.js 
> a1ebad83cbe9e5082c4cc56b391c7d1a25108e00 
>   ambari-web/app/controllers/wizard/step7/assign_master_controller.js 
> 399000eaac97dc4bcfa121b5b3906db7bf71013f 
>   ambari-web/app/data/host/categories.js 
> 7e010d5fb117bcb9e1c633484748b8b4aaf35f97 
>   ambari-web/app/messages.js 71500810a0993c5c02102bfd0ffa15028f78d842 
>   ambari-web/app/mixins/common/configs/configs_saver.js 
> 6592203d614d55c7940657e56e119d4a2f2809bd 
>   ambari-web/app/mixins/common/widgets/widget_mixin.js 
> 5ec4dd6cad9320220da375891894bc41ece71908 
>   ambari-web/app/mixins/main/dashboard/widgets/editable.js 
> c5d3bb4967d02f7193c32a0212550acf69ae8b5a 
>   ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 
> b10d386ebfb6624dad507f63904edc6d03665fe3 
>   ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js 
> cbb81b369b98baafe5c7962242812522f53ab823 
>   ambari-web/app/mixins/main/host/details/host_components/decommissionable.js 
> 38d5b1770a53704c650008afc6ab6a1bbda08d1d 
>   ambari-web/app/mixins/main/service/configs/config_overridable.js 
> a8bd273e2ebc25834559224cd0f9d2e78f91 
>   ambari-web/app/mixins/main/service/configs/widget_popover_support.js 
> 96a25781e27d2be364454aa0337ffec21b85aaf1 
>   ambari-web/app/mixins/wizard/wizardProgressPageView.js 
> 81970866b4e9dd158e0a6821c4f53c01df845855 
>   ambari-web/app/models/alerts/alert_config.js 
> efbfebff5840dcdafea7657ff44ba0c7206b05bb 
>   ambari-web/app/models/alerts/alert_definition.js 
> 4da7199320bc925977071b41d2a0ee1c0b03bcbe 
>   ambari-web/app/models/alerts/alert_instance.js 
> b366802acfe56eac234abbe130198c9ae6bc8e00 
>   ambari-web/app/models/authentication.js 
> 436c18410c79928edd33367c99fe9ec94d8454d8 
>   ambari-web/app/models/host_component.js 
> 11ea659089742d261ceb9f31aaecfd668b553bfa 
>   ambari-web/app/models/hosts.js 9275b3e320136d6932e91b18d9f832eeb7d93e4b 
>   ambari-web/app/models/repository.js 
> 0b7ee69d0f108132c9a6b694b3b8918864d784c4 
>   ambari-web/app/router.js 02707711b8c132d2815854a7bc902ac39a034aae 
>   

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

> On Oct. 11, 2016, 6:37 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  line 410
> > 
> >
> > Does this only affect fresh install and not other commands, especially 
> > during Rolling/Express upgrade?

Yes only START commands in fresh Blueprint based install and only if is enabled 
in ambari.properties.


> On Oct. 11, 2016, 6:37 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java,
> >  line 78
> > 
> >
> > MasterHostResolver.java already has a method called isNameNodeHA().
> > Not sure if you want to use it.

Yes I saw, would be good to put these checks somewhere else but may be would 
better to do in a separate patch, I don't want change too much existing 
functionality (I the normal way of execution of commands) with this patch.


- Sandor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review152173
---


On Oct. 13, 2016, 3:50 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 13, 2016, 3:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  ef4fc33 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  091c6d9 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 

Re: Review Request 52691: Provision actions to happen based only on specified dependencies

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/
---

(Updated Oct. 13, 2016, 3:50 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo Puskas, 
Nate Cole, Sumit Mohanty, and Sebastian Toader.


Changes
---

Change command filtering in ActionScheduler, add more comments.


Bugs: AMBARI-18553
https://issues.apache.org/jira/browse/AMBARI-18553


Repository: ambari


Description
---

Today, the START commands stored in role_command_orders table are bound to 
multiple stages created based on dependencies between components - a component 
in the second stage can only begin its START after the entire first stage is 
done as opposed to just its dependencies in the first stage. This eventually 
increases the overall blueprint deployment time.
The goal is to be able to configure a direct dependency based execution model 
of commands, for now only for Blueprint based deployment commands.

Implementation:
---
When creating stages we set the commandExecutionType to RoleGraph. In case 
commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
created. commandExecutionType is persisted into Stage object / entity as well, 
so ActionScheduler can decide based on commandExecutionType how to execute the 
stage. In case commandExecutionType is set to DEPENDENCY_ORDERED it will filter 
out commands having dependencies on other commands IN_PROGRESS. By default 
commandExecutionType is STAGE_BASED which works as before, creating one or more 
stages dependening on dependecies.
DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
initiated by Blueprint deployment and if Ambari property 
server.stage.command.execution_type = DEPENDENCY_ORDERED.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 8cbfb1e 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
f03d8ea 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 ef4fc33 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 5afaba8 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 bbdb808 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
 eaea913 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c9ab6f9 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
 625b168 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
 5ca4d88 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 091c6d9 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql c2c965d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql f28cdc9 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 bd23e00 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
 4f2580a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewURLResourceProviderTest.java
 e223d0f 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java
 a03f05b 
  
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleGraphTest.java
 8bead43 
  
ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java
 d26816c 
  

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Andrew Onischuk]

> On Oct. 13, 2016, 1:40 p.m., Sumit Mohanty wrote:
> > ambari-project/pom.xml, line 515
> > 
> >
> > Should this be true?

this part will be committed separately. This small change significantly 
decreases time on multy core CPUs.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152494
---


On Oct. 13, 2016, 1:24 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 1:24 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 52833: RegionServer Registration Checks Fail During Upgrade If rDNS is Not Enabled

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52833/
---

Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-18590
https://issues.apache.org/jira/browse/AMBARI-18590


Repository: ambari


Description
---

During a rolling upgrade, the upgrade orchestration must wait for each 
RegionServer to register with the HBase master before moving onto the next RS 
restart. This is a very asynchronous process which may occur several minutes 
after the daemon has actually started. 

We have a check now which uses {{hbase shell}} along with {{status 'simple'}} 
to determine if the host has registered by looking for the hostname. 

However, if reverse DNS is not enabled, then this could potentially be IP 
addresses. As a result, the check would always fail during upgrades:

The HBase status command we use is {{status simple}}, which returns like so:

```
active master:  10.0.0.8:16000 1475801031124
2 backup masters
10.0.0.10:16000 1475801061290
10.0.0.13:16000 1475801046018
2 live servers
10.0.0.5:16020 1475798271407
requestsPerSecond=0.0, numberOfOnlineRegions=2, usedHeapMB=159, 
maxHeapMB=7840, numberOfStores=3, numberOfStorefiles=1, 
storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
storefileIndexSizeMB=0, readRequestsCount=14, writeRequestsCount=1, 
rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
totalCompactingKVs=14, currentCompactedKVs=14, compactionProgressPct=1.0, 
coprocessors=[MultiRowMutationEndpoint, SecureBulkLoadEndpoint]
10.0.0.7:16020 1475872741297
requestsPerSecond=0.0, numberOfOnlineRegions=1, usedHeapMB=1002, 
maxHeapMB=7840, numberOfStores=1, numberOfStorefiles=1, 
storefileUncompressedSizeMB=0, storefileSizeMB=0, memstoreSizeMB=0, 
storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=0, 
rootIndexSizeKB=0, totalStaticIndexSizeKB=0, totalStaticBloomSizeKB=0, 
totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, 
coprocessors=[SecureBulkLoadEndpoint]
0 dead servers
Aggregate load: 0, regions: 3
```

If this lookup fails for the hostname, we should also try by IP address.


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
 f1fa80c 

Diff: https://reviews.apache.org/r/52833/diff/


Testing
---

Total run:1133
Total errors:0
Total failures:0
OK


Thanks,

Jonathan Hurley

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Andrew Onischuk]

> On Oct. 13, 2016, 1:40 p.m., Sumit Mohanty wrote:
> > ambari-project/pom.xml, line 516
> > 
> >
> > Why 1.5C?
> 
> Jonathan Hurley wrote:
> 1.5C means 1.5 x # of cores - so that lowered powered machines (like 
> build machines) don't spawn too many. We should test 1.5C on some of the 
> known developer configurations (I know that a lot of people use macbook pros 
> and a lot use linux desktops).

On 4 core linux env this gives 5 processes and leverages all CPU cores.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152494
---


On Oct. 13, 2016, 1:24 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 1:24 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52785: Fix problem on Suse for ambari-agent installation

[Zsombor Gegesy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52785/
---

(Updated Oct. 13, 2016, 2:52 p.m.)


Review request for Ambari and Sandor Magyari.


Bugs: AMBARI-18577
https://issues.apache.org/jira/browse/AMBARI-18577


Repository: ambari


Description
---

Add "--non-interactive" flag to zypper invocations in setupAgent


Diffs (updated)
-

  ambari-server/src/main/python/setupAgent.py 9a1cae9 

Diff: https://reviews.apache.org/r/52785/diff/


Testing (updated)
---

I've tested on a Suse 11p3, with manually editing the setupAgent.py file, to 
start the installation.
Unfortunately I couldn't build ambari locally, some npm/brunch compilation 
error blocks maven


Thanks,

Zsombor Gegesy

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Jonathan Hurley]

> On Oct. 13, 2016, 9:40 a.m., Sumit Mohanty wrote:
> > ambari-project/pom.xml, line 516
> > 
> >
> > Why 1.5C?

1.5C means 1.5 x # of cores - so that lowered powered machines (like build 
machines) don't spawn too many. We should test 1.5C on some of the known 
developer configurations (I know that a lot of people use macbook pros and a 
lot use linux desktops).


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152494
---


On Oct. 13, 2016, 9:24 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 9:24 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52829: HCat client install during Ambari install wizard

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52829/#review152502
---


Ship it!




Ship It!

- Dmytro Sen


On Окт. 13, 2016, 2:43 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52829/
> ---
> 
> (Updated Окт. 13, 2016, 2:43 п.п.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18589
> https://issues.apache.org/jira/browse/AMBARI-18589
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HCat client install during Ambari install wizard as yum install mysql-
> connector-java command failed.
> 
> Live cluster from nightly [May get killed within few hours](https://hortonwork
> s.jira.com/wiki/display/BUG/May+get+killed+within+few+hours):
> 
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py",
>  line 85, in 
> HCatClient().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py",
>  line 35, in install
> self.install_packages(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 567, in install_packages
> retry_count=agent_stack_retry_count)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
>  line 54, in action_install
> self.install_package(package_name, self.resource.use_repos, 
> self.resource.skip_repos)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py",
>  line 49, in install_package
> self.checked_call_with_retries(cmd, sudo=True, 
> logoutput=self.get_logoutput())
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
>  line 83, in checked_call_with_retries
> return self._call_with_retries(cmd, is_checked=True, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
>  line 91, in _call_with_retries
> code, out = func(cmd, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 71, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d 0 
> -e 0 -y install mysql-connector-java' returned 1. Error: failure: 
> repodata/12897763d92966a7a56d2d2f20c2251ebe89e068afde5185bcd62612a3577428-filelists.sqlite.bz2
>  from updates: [Errno 256] No more mirrors to try.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 1f2250d 
>   ambari-common/src/main/python/resource_management/core/exceptions.py 
> 25e7993 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  04da9b6 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  476e39b 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  0739f66 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  4681b49 
>   ambari-common/src/main/python/resource_management/core/shell.py 94933bd 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
>  4b11614 
> 
> Diff: https://reviews.apache.org/r/52829/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 52829: HCat client install during Ambari install wizard

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52829/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18589
https://issues.apache.org/jira/browse/AMBARI-18589


Repository: ambari


Description
---

HCat client install during Ambari install wizard as yum install mysql-
connector-java command failed.

Live cluster from nightly [May get killed within few hours](https://hortonwork
s.jira.com/wiki/display/BUG/May+get+killed+within+few+hours):





Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py",
 line 85, in 
HCatClient().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py",
 line 35, in install
self.install_packages(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 567, in install_packages
retry_count=agent_stack_retry_count)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
 line 54, in action_install
self.install_package(package_name, self.resource.use_repos, 
self.resource.skip_repos)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py",
 line 49, in install_package
self.checked_call_with_retries(cmd, sudo=True, 
logoutput=self.get_logoutput())
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
 line 83, in checked_call_with_retries
return self._call_with_retries(cmd, is_checked=True, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py",
 line 91, in _call_with_retries
code, out = func(cmd, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 71, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 93, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 141, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 294, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/yum -d 0 
-e 0 -y install mysql-connector-java' returned 1. Error: failure: 
repodata/12897763d92966a7a56d2d2f20c2251ebe89e068afde5185bcd62612a3577428-filelists.sqlite.bz2
 from updates: [Errno 256] No more mirrors to try.


Diffs
-

  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
1f2250d 
  ambari-common/src/main/python/resource_management/core/exceptions.py 25e7993 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 04da9b6 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
476e39b 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 0739f66 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 4681b49 
  ambari-common/src/main/python/resource_management/core/shell.py 94933bd 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
 4b11614 

Diff: https://reviews.apache.org/r/52829/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 52785: Fix problem on Suse for ambari-agent installation

[Zsombor Gegesy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52785/
---

(Updated Oct. 13, 2016, 2:24 p.m.)


Review request for Ambari and Sandor Magyari.


Bugs: AMBARI-18577
https://issues.apache.org/jira/browse/AMBARI-18577


Repository: ambari


Description
---

Add "--non-interactive" flag to zypper invocations in setupAgent


Diffs (updated)
-

  ambari-server/src/main/python/setupAgent.py 9a1cae9 

Diff: https://reviews.apache.org/r/52785/diff/


Testing
---


Thanks,

Zsombor Gegesy

Re: Review Request 52825: AMBARI-18585 Set _HOST for Hbase Master principal in Kerberos enabled Ranger Hbase Plugin

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52825/#review152500
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Oct. 13, 2016, 12:39 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52825/
> ---
> 
> (Updated Oct. 13, 2016, 12:39 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-18585
> https://issues.apache.org/jira/browse/AMBARI-18585
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Need to update hbase plugin repo config for master principal to have a _HOST 
> instead of a specific node.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  49e4575 
> 
> Diff: https://reviews.apache.org/r/52825/diff/
> 
> 
> Testing
> ---
> 
> Tested on centos 6 with Ranger Hbase plugin enabled.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 52747: Migration to Bootstrap3 framework

[Aleksandr Kovalenko]

> On Окт. 12, 2016, 8:35 п.п., Yusaku Sako wrote:
> > Thanks for moving the ambari bootstrap override as unminified, non-vendor 
> > style.
> > Can we move the Roboto font files back to vendor, though?

done


- Aleksandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52747/#review152395
---


On Окт. 13, 2016, 1:48 п.п., Aleksandr Kovalenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52747/
> ---
> 
> (Updated Окт. 13, 2016, 1:48 п.п.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Zhe (Joe) Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18573
> https://issues.apache.org/jira/browse/AMBARI-18573
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Migrate from Bootstrap2 to Bootstrap3 framework.
> 
> 
> Diffs
> -
> 
>   LICENSE.txt 906c0590c2a749b910139dee2af70e4b4b17c949 
>   ambari-web/app/app.js 53394a055bb5997c3589b0e18413e2f984b6 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.eot PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.svg PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.ttf PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff PRE-CREATION 
>   ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff2 PRE-CREATION 
>   ambari-web/app/config.js 4550eb8905d6064f44a194d31ee3727c55e9bfb5 
>   ambari-web/app/controllers/application.js 
> dca689f949dfa3583fac3e2d181e2d876716664d 
>   ambari-web/app/controllers/main/admin/service_auto_start.js 
> b425c785770ef05c4b7b519d50645be0f1b655ea 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> d00a652f8da42fba20fb86394e06dcedba18edfc 
>   ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
> f05d366057d75ca47418891c86504f18e8e3f1c4 
>   
> ambari-web/app/controllers/main/alerts/add_alert_definition/step1_controller.js
>  43e25c8c5338d9c264aa8346cce1b12d195f64f0 
>   
> ambari-web/app/controllers/main/alerts/alert_definitions_actions_controller.js
>  6ca3e786d65cfe515615e10aa141a3d5dca2fbd5 
>   ambari-web/app/controllers/main/alerts/alert_instances_controller.js 
> 5913070d3ba799b6d2415361381aea781b1f313d 
>   ambari-web/app/controllers/main/alerts/manage_alert_groups_controller.js 
> 6ba595738234aa840de8bdbca47884a5d699b843 
>   ambari-web/app/controllers/main/charts/heatmap.js 
> 73d18005202e956b2af4732abf3467a72a7d96f2 
>   ambari-web/app/controllers/main/service/info/summary.js 
> c16bb3fc728fe89743d308e0ab96b073e5a8845f 
>   ambari-web/app/controllers/main/service/manage_config_groups_controller.js 
> 539d03eb3d0c5b0e7ea8e5284b5a1c796d3af3cc 
>   ambari-web/app/controllers/wizard/step10_controller.js 
> 8c4b5dbe371e78fd5cb0bc7e87e0a7a965486f2d 
>   ambari-web/app/controllers/wizard/step1_controller.js 
> a1ebad83cbe9e5082c4cc56b391c7d1a25108e00 
>   ambari-web/app/controllers/wizard/step7/assign_master_controller.js 
> 399000eaac97dc4bcfa121b5b3906db7bf71013f 
>   ambari-web/app/data/host/categories.js 
> 7e010d5fb117bcb9e1c633484748b8b4aaf35f97 
>   ambari-web/app/messages.js 71500810a0993c5c02102bfd0ffa15028f78d842 
>   ambari-web/app/mixins/common/configs/configs_saver.js 
> 6592203d614d55c7940657e56e119d4a2f2809bd 
>   ambari-web/app/mixins/common/widgets/widget_mixin.js 
> 5ec4dd6cad9320220da375891894bc41ece71908 
>   ambari-web/app/mixins/main/dashboard/widgets/editable.js 
> c5d3bb4967d02f7193c32a0212550acf69ae8b5a 
>   ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 
> b10d386ebfb6624dad507f63904edc6d03665fe3 
>   ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js 
> cbb81b369b98baafe5c7962242812522f53ab823 
>   ambari-web/app/mixins/main/host/details/host_components/decommissionable.js 
> 38d5b1770a53704c650008afc6ab6a1bbda08d1d 
>   ambari-web/app/mixins/main/service/configs/config_overridable.js 
> a8bd273e2ebc25834559224cd0f9d2e78f91 
>   ambari-web/app/mixins/main/service/configs/widget_popover_support.js 
> 96a25781e27d2be364454aa0337ffec21b85aaf1 
>   ambari-web/app/mixins/wizard/wizardProgressPageView.js 
> 81970866b4e9dd158e0a6821c4f53c01df845855 
>   ambari-web/app/models/alerts/alert_config.js 
> efbfebff5840dcdafea7657ff44ba0c7206b05bb 
>   ambari-web/app/models/alerts/alert_definition.js 
> 4da7199320bc925977071b41d2a0ee1c0b03bcbe 
>   ambari-web/app/models/alerts/alert_instance.js 
> b366802acfe56eac234abbe130198c9ae6bc8e00 
>   ambari-web/app/models/authentication.js 
> 436c18410c79928edd33367c99fe9ec94d8454d8 
>   ambari-web/app/models/host_component.js 
> 11ea659089742d261ceb9f31aaecfd668b553bfa 
>   ambari-web/app/models/hosts.js 

Re: Review Request 52747: Migration to Bootstrap3 framework

[Aleksandr Kovalenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52747/
---

(Updated Окт. 13, 2016, 1:48 п.п.)


Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Zhe (Joe) Wang, 
and Yusaku Sako.


Changes
---

Updated patch to fix open issues.


Bugs: AMBARI-18573
https://issues.apache.org/jira/browse/AMBARI-18573


Repository: ambari


Description
---

Migrate from Bootstrap2 to Bootstrap3 framework.


Diffs (updated)
-

  LICENSE.txt 906c0590c2a749b910139dee2af70e4b4b17c949 
  ambari-web/app/app.js 53394a055bb5997c3589b0e18413e2f984b6 
  ambari-web/app/assets/fonts/glyphicons-halflings-regular.eot PRE-CREATION 
  ambari-web/app/assets/fonts/glyphicons-halflings-regular.svg PRE-CREATION 
  ambari-web/app/assets/fonts/glyphicons-halflings-regular.ttf PRE-CREATION 
  ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff PRE-CREATION 
  ambari-web/app/assets/fonts/glyphicons-halflings-regular.woff2 PRE-CREATION 
  ambari-web/app/config.js 4550eb8905d6064f44a194d31ee3727c55e9bfb5 
  ambari-web/app/controllers/application.js 
dca689f949dfa3583fac3e2d181e2d876716664d 
  ambari-web/app/controllers/main/admin/service_auto_start.js 
b425c785770ef05c4b7b519d50645be0f1b655ea 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
d00a652f8da42fba20fb86394e06dcedba18edfc 
  ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
f05d366057d75ca47418891c86504f18e8e3f1c4 
  
ambari-web/app/controllers/main/alerts/add_alert_definition/step1_controller.js 
43e25c8c5338d9c264aa8346cce1b12d195f64f0 
  
ambari-web/app/controllers/main/alerts/alert_definitions_actions_controller.js 
6ca3e786d65cfe515615e10aa141a3d5dca2fbd5 
  ambari-web/app/controllers/main/alerts/alert_instances_controller.js 
5913070d3ba799b6d2415361381aea781b1f313d 
  ambari-web/app/controllers/main/alerts/manage_alert_groups_controller.js 
6ba595738234aa840de8bdbca47884a5d699b843 
  ambari-web/app/controllers/main/charts/heatmap.js 
73d18005202e956b2af4732abf3467a72a7d96f2 
  ambari-web/app/controllers/main/service/info/summary.js 
c16bb3fc728fe89743d308e0ab96b073e5a8845f 
  ambari-web/app/controllers/main/service/manage_config_groups_controller.js 
539d03eb3d0c5b0e7ea8e5284b5a1c796d3af3cc 
  ambari-web/app/controllers/wizard/step10_controller.js 
8c4b5dbe371e78fd5cb0bc7e87e0a7a965486f2d 
  ambari-web/app/controllers/wizard/step1_controller.js 
a1ebad83cbe9e5082c4cc56b391c7d1a25108e00 
  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 
399000eaac97dc4bcfa121b5b3906db7bf71013f 
  ambari-web/app/data/host/categories.js 
7e010d5fb117bcb9e1c633484748b8b4aaf35f97 
  ambari-web/app/messages.js 71500810a0993c5c02102bfd0ffa15028f78d842 
  ambari-web/app/mixins/common/configs/configs_saver.js 
6592203d614d55c7940657e56e119d4a2f2809bd 
  ambari-web/app/mixins/common/widgets/widget_mixin.js 
5ec4dd6cad9320220da375891894bc41ece71908 
  ambari-web/app/mixins/main/dashboard/widgets/editable.js 
c5d3bb4967d02f7193c32a0212550acf69ae8b5a 
  ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 
b10d386ebfb6624dad507f63904edc6d03665fe3 
  ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js 
cbb81b369b98baafe5c7962242812522f53ab823 
  ambari-web/app/mixins/main/host/details/host_components/decommissionable.js 
38d5b1770a53704c650008afc6ab6a1bbda08d1d 
  ambari-web/app/mixins/main/service/configs/config_overridable.js 
a8bd273e2ebc25834559224cd0f9d2e78f91 
  ambari-web/app/mixins/main/service/configs/widget_popover_support.js 
96a25781e27d2be364454aa0337ffec21b85aaf1 
  ambari-web/app/mixins/wizard/wizardProgressPageView.js 
81970866b4e9dd158e0a6821c4f53c01df845855 
  ambari-web/app/models/alerts/alert_config.js 
efbfebff5840dcdafea7657ff44ba0c7206b05bb 
  ambari-web/app/models/alerts/alert_definition.js 
4da7199320bc925977071b41d2a0ee1c0b03bcbe 
  ambari-web/app/models/alerts/alert_instance.js 
b366802acfe56eac234abbe130198c9ae6bc8e00 
  ambari-web/app/models/authentication.js 
436c18410c79928edd33367c99fe9ec94d8454d8 
  ambari-web/app/models/host_component.js 
11ea659089742d261ceb9f31aaecfd668b553bfa 
  ambari-web/app/models/hosts.js 9275b3e320136d6932e91b18d9f832eeb7d93e4b 
  ambari-web/app/models/repository.js 0b7ee69d0f108132c9a6b694b3b8918864d784c4 
  ambari-web/app/router.js 02707711b8c132d2815854a7bc902ac39a034aae 
  ambari-web/app/routes/activate_hawq_standby_routes.js 
8ab19dba452a33d389f66b030626c87afa2f9489 
  ambari-web/app/routes/add_alert_definition_routes.js 
15d010394e81fae48b63964f3d5e18e02f2e6f03 
  ambari-web/app/routes/add_hawq_standby_routes.js 
d802808e91c99523e166639311b533fe16c87629 
  ambari-web/app/routes/add_host_routes.js 
c733a6665637cd9a8c749a844d531a530abe071b 
  ambari-web/app/routes/add_kerberos_routes.js 
1568577b3a2591c3ac1059c3625d6c64cf0f7913 
  

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152494
---




ambari-project/pom.xml (line 515)


Should this be true?



ambari-project/pom.xml (line 516)


Why 1.5C?


- Sumit Mohanty


On Oct. 13, 2016, 1:24 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Oct. 13, 2016, 1:24 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/#review152492
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Жов. 13, 2016, 1:24 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52826/
> ---
> 
> (Updated Жов. 13, 2016, 1:24 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18586
> https://issues.apache.org/jira/browse/AMBARI-18586
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increase forkCount for Ambari Server Unit Tests
> 
> 
> Diffs
> -
> 
>   ambari-project/pom.xml c0fd7bf 
> 
> Diff: https://reviews.apache.org/r/52826/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 52826: Increase forkCount for Ambari Server Unit Tests

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52826/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18586
https://issues.apache.org/jira/browse/AMBARI-18586


Repository: ambari


Description
---

Increase forkCount for Ambari Server Unit Tests


Diffs
-

  ambari-project/pom.xml c0fd7bf 

Diff: https://reviews.apache.org/r/52826/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/#review152489
---




ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
 (lines 27 - 29)


indent here should be two spaces



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
 (lines 51 - 53)


two spaces



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
 (lines 59 - 66)


two spaces



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
 (lines 64 - 70)


This is odd - it's like you're forcing an additional element to determine a 
class name where java XML parsing gives that to you for free.  See comment 
regarding schema.



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
 (lines 49 - 52)


two spaces



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
 (lines 73 - 75)


two spaces



ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java 
(lines 169 - 171)


We're not above using Apache commons:  
!CollectionUtils.isEmpty(dependencyConditions)



ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 (lines 294 - 296)


this looks like it will continue to check even if a prior dependency is not 
resolved.  Do you need to continue checking or can you 'break' after 
conditionsSatisfied is set to false?



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
(lines 43 - 47)


This is a bit of an odd syntax.  the  element should be a 
different class then you don't need the  element.  Now, these 
metainfo.xml files don't have schema for validation, but java XML is able to 
use correct classes based on the element name.  So for example, you would have 
two types such that:


  

  
  

  




- Nate Cole


On Oct. 12, 2016, 7:42 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51815/
> ---
> 
> (Updated Oct. 12, 2016, 7:42 p.m.)
> 
> 
> Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo  
> Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18355
> https://issues.apache.org/jira/browse/AMBARI-18355
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. Please refer to document attached to Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
>  e3db662 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  a5f33ff 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 65d166a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
>  ff9af17 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  b1de8ef 
> 
> Diff: https://reviews.apache.org/r/51815/diff/
> 
> 
> Testing
> ---
> 
> Written Junit test cases. Perfomred manual testing to check the Namenode HA 
> component dependency. Was able to proceed with the installation for valid 
> blueprint. and got validation error message while registering blueprint when 
> the Blueprint did not satisfy the conditional dependencies.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

[Nate Cole]

> On Oct. 7, 2016, 3:39 p.m., Nate Cole wrote:
> > This code is breaking tests, and has been reverted from trunk.  Please fix 
> > the tests, and also fix the formatting.  Ambari is standardized on two 
> > space tabs.  DependencyConditionInfo is a undocumented and unformatted 
> > class.  In addition, you shouldn't be making subclasses as internal classes 
> > of the parent abstract class.
> > 
> > Di Li, when you commit code on behalf of someone else, you must make the 
> > message like so: "AMBARI-X. Some message (Joe Smith via dili)"
> 
> Di Li wrote:
> Anything else you think I should put in as the comment, Nate? I did put 
> (Amruta Borkar via dili) at the end of the comment though...
> 
> commit f6124a056d2a8ed16bec917775b9d3554ab5d74d
> Author: Di Li 
> Date:   Fri Oct 7 13:34:54 2016 -0400
> 
> AMBARI-18355: Introduce conditional dependencies in stack defition to 
> handle blueprint validation gracefully (Amruta Borkar via dili)

No no, that was my mistake - my browser had cut off the comment so all I saw 
the author's name.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/#review151861
---


On Oct. 12, 2016, 7:42 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51815/
> ---
> 
> (Updated Oct. 12, 2016, 7:42 p.m.)
> 
> 
> Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo  
> Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18355
> https://issues.apache.org/jira/browse/AMBARI-18355
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. Please refer to document attached to Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
>  e3db662 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  a5f33ff 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 65d166a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
>  ff9af17 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  b1de8ef 
> 
> Diff: https://reviews.apache.org/r/51815/diff/
> 
> 
> Testing
> ---
> 
> Written Junit test cases. Perfomred manual testing to check the Namenode HA 
> component dependency. Was able to proceed with the installation for valid 
> blueprint. and got validation error message while registering blueprint when 
> the Blueprint did not satisfy the conditional dependencies.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Review Request 52825: AMBARI-18585 Set _HOST for Hbase Master principal in Kerberos enabled Ranger Hbase Plugin

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52825/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
and Velmurugan Periasamy.


Bugs: AMBARI-18585
https://issues.apache.org/jira/browse/AMBARI-18585


Repository: ambari


Description
---

Need to update hbase plugin repo config for master principal to have a _HOST 
instead of a specific node.


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 49e4575 

Diff: https://reviews.apache.org/r/52825/diff/


Testing
---

Tested on centos 6 with Ranger Hbase plugin enabled.


Thanks,

Mugdha Varadkar

Re: Review Request 52785: Fix problem on Suse for ambari-agent installation

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52785/#review152483
---


Ship it!





ambari-server/src/main/python/setupAgent.py (line 125)


The `-y` flag for `install` is an alias for `--non-interactive`, so this 
change is not strictly necessary.

https://en.opensuse.org/SDB:Zypper_manual_(plain)


- Attila Doroszlai


On Oct. 12, 2016, 4:36 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52785/
> ---
> 
> (Updated Oct. 12, 2016, 4:36 p.m.)
> 
> 
> Review request for Ambari and Sandor Magyari.
> 
> 
> Bugs: AMBARI-18577
> https://issues.apache.org/jira/browse/AMBARI-18577
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "--non-interactive" flag to zypper invocations in setupAgent
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/setupAgent.py 9a1cae9 
> 
> Diff: https://reviews.apache.org/r/52785/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 52791: Getting errors with max length 1000byte, when using Mysql db with charset UTF8

[Myroslav Papirkovskyy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52791/#review152481
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Жов. 12, 2016, 8:41 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52791/
> ---
> 
> (Updated Жов. 12, 2016, 8:41 після полудня)
> 
> 
> Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-18532
> https://issues.apache.org/jira/browse/AMBARI-18532
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add related code to UpgradeCatalog. And change all blueprint_name fields to 
> varchar(100).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
>  ebfb267 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog242.java
>  PRE-CREATION 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql f84129c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52791/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 50561: Hive View Upload table does not support line feed and carriage return in field delimiter

[Ashwin Rajeev]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50561/#review152475
---


Ship it!




Ship It!

- Ashwin Rajeev


On July 28, 2016, 11:22 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50561/
> ---
> 
> (Updated July 28, 2016, 11:22 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17943
> https://issues.apache.org/jira/browse/AMBARI-17943
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> removed the linefeed and carriage return and improved UX
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive-next/src/main/resources/ui/hive-web/app/controllers/upload-table.js
>  061a445 
>   
> contrib/views/hive-next/src/main/resources/ui/hive-web/app/templates/upload-table.hbs
>  d84dc4d 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/upload-table.js
>  061a445 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/templates/upload-table.hbs
>  d84dc4d 
>   
> contrib/views/hive/src/test/java/org/apache/ambari/view/hive/resources/upload/OpenCSVParserTest.java
>  2c890f5 
>   
> contrib/views/hive/src/test/java/org/apache/ambari/view/hive/resources/upload/OpenCSVTest.java
>  be5733f 
> 
> Diff: https://reviews.apache.org/r/50561/diff/
> 
> 
> Testing
> ---
> 
> manual testing done
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Re: Review Request 52822: Ambari Hive View 'Upload Table' does not support UTF8 files with BOM

[Ashwin Rajeev]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52822/#review152474
---


Ship it!




Ship It!

- Ashwin Rajeev


On Oct. 13, 2016, 8:56 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52822/
> ---
> 
> (Updated Oct. 13, 2016, 8:56 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-18583
> https://issues.apache.org/jira/browse/AMBARI-18583
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ealiear BOM characters in the file were creating problem in upload. Now they 
> are removed using apache commons io library.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/uploads/UploadService.java
>  9800c22 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/UploadService.java
>  7dccbd4 
> 
> Diff: https://reviews.apache.org/r/52822/diff/
> 
> 
> Testing
> ---
> 
> manual testing done to verify support for both Big-endian and little-endian 
> BOM.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Re: Review Request 52785: Fix problem on Suse for ambari-agent installation

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52785/#review152470
---


Ship it!




Ship It!

- Laszlo Puskas


On Oct. 12, 2016, 2:36 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52785/
> ---
> 
> (Updated Oct. 12, 2016, 2:36 p.m.)
> 
> 
> Review request for Ambari and Sandor Magyari.
> 
> 
> Bugs: AMBARI-18577
> https://issues.apache.org/jira/browse/AMBARI-18577
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "--non-interactive" flag to zypper invocations in setupAgent
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/setupAgent.py 9a1cae9 
> 
> Diff: https://reviews.apache.org/r/52785/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Review Request 52823: For Upload Table on hive view, upload fails when file extension is .json

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52823/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, 
Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-18584
https://issues.apache.org/jira/browse/AMBARI-18584


Repository: ambari


Description
---

Earlier GsonJsonProvider was trying to convert uploaded InputStream because the 
content type that was coming was application/json for files with extension .json
Now the GsonJsonProvider will ignore conversion of object of InputStream.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/GsonJsonProvider.java 
8ba9ff7 

Diff: https://reviews.apache.org/r/52823/diff/


Testing
---

Manual Testing Done.


Thanks,

Nitiraj Rathore

Review Request 52822: Ambari Hive View 'Upload Table' does not support UTF8 files with BOM

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52822/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, 
Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-18583
https://issues.apache.org/jira/browse/AMBARI-18583


Repository: ambari


Description
---

Ealiear BOM characters in the file were creating problem in upload. Now they 
are removed using apache commons io library.


Diffs
-

  
contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/uploads/UploadService.java
 9800c22 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/UploadService.java
 7dccbd4 

Diff: https://reviews.apache.org/r/52822/diff/


Testing
---

manual testing done to verify support for both Big-endian and little-endian BOM.


Thanks,

Nitiraj Rathore

Re: Review Request 52785: Fix problem on Suse for ambari-agent installation

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52785/#review152466
---


Ship it!




Could you please run 'mvn clean test  -DskipSurefireTests=true' to make sure 
pyhton unit tests are fine and add the result to 'Testing done' section. Also 
you can add there if you have made some manual testing etc.

- Sandor Magyari


On Oct. 12, 2016, 2:36 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52785/
> ---
> 
> (Updated Oct. 12, 2016, 2:36 p.m.)
> 
> 
> Review request for Ambari and Sandor Magyari.
> 
> 
> Bugs: AMBARI-18577
> https://issues.apache.org/jira/browse/AMBARI-18577
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add "--non-interactive" flag to zypper invocations in setupAgent
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/setupAgent.py 9a1cae9 
> 
> Diff: https://reviews.apache.org/r/52785/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 52792: AMBARI-18581 Oozie start script ignores sysprep_skip_copy_oozie_share_lib_to_hdfs flag

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52792/#review152462
---


Ship it!




Ship It!

- Sebastian Toader


On Oct. 12, 2016, 7:51 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52792/
> ---
> 
> (Updated Oct. 12, 2016, 7:51 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Laszlo 
> Puskas, Sandor Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18581
> https://issues.apache.org/jira/browse/AMBARI-18581
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added missing `/configurations` prefix.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py
>  e5d674a 
> 
> Diff: https://reviews.apache.org/r/52792/diff/
> 
> 
> Testing
> ---
> 
> Manual testing:
>  * create sysprepped cluster
>  * submit blueprint with WebHDFS disabled and 
> sysprep_skip_copy_oozie_share_lib_to_hdfs enabled
>  * submit START_ONLY cluster creation request
>  * verify Oozie is started (see 
> https://issues.apache.org/jira/browse/AMBARI-18567)
>  * verify Oozie skipped steps: "Skipping creation of oozie sharelib as host 
> is sys prepped"
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>