Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 d472b79 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas

Review Request 51708: While checking for component dependency code looks for incorrect component name

[Amruta Borkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51708/
---

Review request for Ambari, Laszlo Puskas and Robert Nettleton.


Bugs: AMBARI-18333
https://issues.apache.org/jira/browse/AMBARI-18333


Repository: ambari


Description
---

While validating host groups in blueprint for component dependency, code should 
check if a dependency component is present in the host group component list 
rather than the component on whose dependency list it is iterating.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 398772e 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
 ea6bee3 

Diff: https://reviews.apache.org/r/51708/diff/


Testing
---

Added Test cases, performed manual testing


Thanks,

Amruta Borkar

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/
---

(Updated Sept. 7, 2016, 9:14 p.m.)


Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18334
https://issues.apache.org/jira/browse/AMBARI-18334


Repository: ambari


Description
---

The configurations.json file loaded in the ambari-agent cache located at 
/var/lib/ambari-agent/cache/cluster_configuration contains password details in 
plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
etc.). The values are loaded both in the memory cache and file cache, the file 
seems to be used only for debugging purposes, so it would be a better approach 
to mask the passwords in the file.

Approach:

The password_config_type is included in the heartbeat response for alert 
definition command and execution command, for which the values are dumped into 
the json file. The password_config_type contains the information on which 
properties in the configurations has the propertyType password. Based on the 
response, the json is parsed and the password values are masked before dumping 
it into the configurations.json file.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py 72b87be 
  ambari-agent/src/test/python/ambari_agent/TestAlerts.py e114daa 
  ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py 
a418f6d 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
 0562c15 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
 4d2e048 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 29737ee 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
70c24f9 

Diff: https://reviews.apache.org/r/51705/diff/


Testing (updated)
---

Updated the test cases.
Ran mvn test.

Manually tested by setting up a cluster, the password fields in the 
configurations.json is masked. During testing, everytime the ambari agent is 
restarted, it registers with the server and the memory cache and file cache are 
updated, the alerts in turn uses the value from the memory cache.


Thanks,

Anita Jebaraj

Re: Review Request 51678: Code review request for AMBARI-18290 (Ambari does not support HBase on HTTPS mode)

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51678/#review148085
---


Ship it!




Ship It!

- Sid Wagle


On Sept. 7, 2016, 9:02 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51678/
> ---
> 
> (Updated Sept. 7, 2016, 9:02 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18290
> https://issues.apache.org/jira/browse/AMBARI-18290
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If HTTPS is enabled for HBase, Ambari still tries to communicate on HTTP port 
> to fetch JMX properties. because of that Ambari shows incorrect 
> Active/Standby node information.
> 
> Have made code changes to read the "hbase.http.policy" property and then 
> decide the protocol.
> 
> Note: I have not made any changes to Ambari Web. If HTTPS is enabled for 
> HBase then all quick links should be opened on HTTPS. Currently Ambari checks 
>  "dfs.http.policy" HDFS/Hadoop property and decide whether for all the quick 
> links.  it does not check HBase "hbase.http.policy" property to decide. am 
> leaving this functionality as it is.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  066d5c4 
> 
> Diff: https://reviews.apache.org/r/51678/diff/
> 
> 
> Testing
> ---
> 
> Test cases executed
> --
> 1. Install Ambari2.4, install HBase service, in summary it should show 
> "Active HBase Master"  properly
> 2. Install Ambari2.4, install HBase service and enable HTTPS, in summary it 
> should show "Active HBase Master"  properly
> 3. Install Ambari2.4, install HBase service and add one more HBase master. it 
> should show one node as active and other is standby
> 4. Install Ambari2.4, install HBase service and add one more HBase master and 
> enable HTTPS. it should show one node as active and other is standby
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 51678: Code review request for AMBARI-18290 (Ambari does not support HBase on HTTPS mode)

[Amarnath reddy pappu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51678/
---

(Updated Sept. 7, 2016, 9:02 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sid Wagle.


Changes
---

I have incorporated reviews comments and attached the new code diff again.


Bugs: AMBARI-18290
https://issues.apache.org/jira/browse/AMBARI-18290


Repository: ambari


Description
---

If HTTPS is enabled for HBase, Ambari still tries to communicate on HTTP port 
to fetch JMX properties. because of that Ambari shows incorrect Active/Standby 
node information.

Have made code changes to read the "hbase.http.policy" property and then decide 
the protocol.

Note: I have not made any changes to Ambari Web. If HTTPS is enabled for HBase 
then all quick links should be opened on HTTPS. Currently Ambari checks  
"dfs.http.policy" HDFS/Hadoop property and decide whether for all the quick 
links.  it does not check HBase "hbase.http.policy" property to decide. am 
leaving this functionality as it is.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 066d5c4 

Diff: https://reviews.apache.org/r/51678/diff/


Testing
---

Test cases executed
--
1. Install Ambari2.4, install HBase service, in summary it should show "Active 
HBase Master"  properly
2. Install Ambari2.4, install HBase service and enable HTTPS, in summary it 
should show "Active HBase Master"  properly
3. Install Ambari2.4, install HBase service and add one more HBase master. it 
should show one node as active and other is standby
4. Install Ambari2.4, install HBase service and add one more HBase master and 
enable HTTPS. it should show one node as active and other is standby


Thanks,

Amarnath reddy pappu

Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/
---

Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18334
https://issues.apache.org/jira/browse/AMBARI-18334


Repository: ambari


Description
---

The configurations.json file loaded in the ambari-agent cache located at 
/var/lib/ambari-agent/cache/cluster_configuration contains password details in 
plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
etc.). The values are loaded both in the memory cache and file cache, the file 
seems to be used only for debugging purposes, so it would be a better approach 
to mask the passwords in the file.

Approach:

The password_config_type is included in the heartbeat response for alert 
definition command and execution command, for which the values are dumped into 
the json file. The password_config_type contains the information on which 
properties in the configurations has the propertyType password. Based on the 
response, the json is parsed and the password values are masked before dumping 
it into the configurations.json file.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py 72b87be 
  ambari-agent/src/test/python/ambari_agent/TestAlerts.py e114daa 
  ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py 
a418f6d 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
 0562c15 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
 4d2e048 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 29737ee 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
70c24f9 

Diff: https://reviews.apache.org/r/51705/diff/


Testing
---

Updated the test cases.
Ran mvn test.

Manually tested by setting up a cluster, the password fields in the 
configurations.json is masked.


Thanks,

Anita Jebaraj

Re: Review Request 51632: Refactor logsearch portal code

[Oliver Szabo]

> On Sept. 7, 2016, 1:20 p.m., Miklos Gergely wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java,
> >  line 36
> > 
> >
> > The name of the property after the PropertiesFile patch will be applied 
> > is logsearch.auth.external_auth.login_url

https://reviews.apache.org/r/51704/ will contain these fixes


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51632/#review148006
---


On Sept. 5, 2016, 4:46 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51632/
> ---
> 
> (Updated Sept. 5, 2016, 4:46 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sandor Magyari, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18310
> https://issues.apache.org/jira/browse/AMBARI-18310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - rename/move auditLogRest/manager and serviceLogRest/manager (review board 
> shows it as new added code)
> - using @Inject instead of @Autowired
> - using objects instead of strings for Solr responses (later I want to use 
> spring data solr project to get rid of query generation custom code) ... for 
> that i will fill the Criteria objects with fields as well (now those are only 
> markers)
> - adding converters between requests and search criterias
> - rename and refactor a lot of Vxxx objects, and use them as response objects 
> on endpoints, because of this rest api documentations can show them as they 
> are (not as strings)
> - use javaConfig instead of xml for spring (now security part still in xml, 
> but i will change that later too)
> - move Configuration values into objects, trying to get rid of Property Helper
> 
> Because that is a pretty big change, I created that review, its not done yet 
> (there are still Vxxx objects and also i need to get rid of xml 
> configurations or web.xml as well), also after the documentation was created, 
> there are a lot of unused (or almost duplicated like: 
> componentName/component_name/component) query params there (from the UI), so 
> i will need some manual testing to make sure i can remove them.
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml b36c9bb 
>   ambari-logsearch/ambari-logsearch-portal/production/build.js afeea67 
>   ambari-logsearch/ambari-logsearch-portal/production/r.js 6e6ffd0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/managed-schema
>  f58652c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/managed-schema
>  3b4cf19 
>   ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/README.md 
> 3cc4915 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/reset_collections.sh
>  7c742c0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/restart_solr.sh
>  bd436bb 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/update_config.sh
>  93e3496 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  15355a1 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
>  edb4ffa 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  5235fab 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
>  142b29b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/Marker.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/SearchCriteria.java
>  6b74144 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApiDocConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApplicationConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
>  PRE-CREATION 
>   
> 

Re: Review Request 51698: Added database metrics source

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51698/#review148070
---




ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 (line 377)


Unncessary comments.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AbstractMetricsSource.java
 (line 50)


We should try configuration for supplied hostname, if not found then switch 
to InetAddress, which can be unreliable or not what user wants.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
 (line 66)


Please define and enum or EnumSet and put comments.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
 (line 137)


This is not a threadsafe check. Necessary to declare isInint as volatile. 
Best is to do the init outside this method call.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
 (line 34)


If this instance is created by guice, why the private constructor?



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
 (line 35)


Can you rename to DBMetricsSource, since interface is named MetricsSource 
(plural)



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
 (line 43)


Not the optimium impl of Singleton patter if you have have to use it, 
static reference instantiation is recommeneded.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
 (line 51)


Again improper use of Singleton. Why is guice not used ?


- Sid Wagle


On Sept. 7, 2016, 5:59 p.m., Li-Wei Tseng wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51698/
> ---
> 
> (Updated Sept. 7, 2016, 5:59 p.m.)
> 
> 
> Review request for Ambari and Aravindan Vijayan.
> 
> 
> Bugs: AMBARI-17589
> https://issues.apache.org/jira/browse/AMBARI-17589
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added Database metrics generated by Eclipselink Performance Profiler
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/metrics.properties 5f01e39 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  bd34c25 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/AmbariMetricSink.java
>  809176be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsService.java
>  23845c9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSink.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSource.java
>  cf10408 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AbstractMetricsSource.java
>  58e2045 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
>  4618618 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/Configuration.java
>  705971f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
>  a04ca43 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
>  1645ebf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/EclipseLinkSessionCustomizer.java
>  6717e01 
>   ambari-server/src/main/resources/META-INF/persistence.xml 2bc3282 
>   
> ambari-server/src/test/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImplTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51698/diff/
> 
> 
> Testing
> ---
> 
> Unit test for emit metrics to AMS
> 
> 
> Thanks,
> 
> Li-Wei Tseng
> 
>

Re: Review Request 51201: Generate REST API docs with Swagger for Log Search

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51201/#review148074
---


Ship it!




Ship It!

- Miklos Gergely


On Aug. 18, 2016, 10:37 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51201/
> ---
> 
> (Updated Aug. 18, 2016, 10:37 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert 
> Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18196
> https://issues.apache.org/jira/browse/AMBARI-18196
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - generate rest api doc with swagger
> - changed the root endpoint mapping form /service/* to /api/v1
> - currently I using implicit params for the endpoint, but the after i will 
> rework the rest API, i expect to use objects instead of that (if swagger 
> supports not primitive types for queries) ... there i will update the 
> endpoints that ambari uses as well
> - i created a lot of empty descriptions, we will fill that later (im not sure 
> about: what some query parameters do or do we need them anyway, etc.)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-it/pom.xml 075a22a 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 7a46fe5 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  819d3b9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
>  6d18b4a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
>  cb87f6f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/LogFileREST.java
>  ef1bb8f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/PublicREST.java
>  7977703 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/UserConfigREST.java
>  40b215c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/META-INF/applicationContext.xml
>  5e24d88 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/META-INF/security-applicationContext.xml
>  96852b5 
>   ambari-logsearch/ambari-logsearch-portal/src/main/webapp/WEB-INF/web.xml 
> dc8519a 
>   ambari-logsearch/ambari-logsearch-portal/src/main/webapp/login.html ff546ef 
>   ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/App.js 
> ef9b662 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/Globals.js
>  212846b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditAggregatedView.js
>  acca51c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/LogFileView.js
>  6932355 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/helpers/Helpers.js
>  2117aa3 
> 
> Diff: https://reviews.apache.org/r/51201/diff/
> 
> 
> Testing
> ---
> 
> testing done.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 51537: Use the PropertiesFile python command to create property files instead of jinja templates

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51537/#review148073
---


Ship it!




Ship It!

- Oliver Szabo


On Sept. 7, 2016, 5:45 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51537/
> ---
> 
> (Updated Sept. 7, 2016, 5:45 p.m.)
> 
> 
> Review request for Ambari, Hayat Behlim, Dharmesh Makwana, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18279
> https://issues.apache.org/jira/browse/AMBARI-18279
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Instead of using jinja templates use the PropetiesFile command in 
> resource_management.libraries.resources.properties_file to generate the 
> property files, thus prevent enumerating all the non-custom properties in 
> params.py
> 
> Also fixed a bug: the default values for the audit logs solr znodes and 
> quorum were mixed up.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
>  56ff2d1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  5c3954f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  d3851d3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  aa89609 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  86f0be4 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  1bcee2d 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 
> f4dbd63 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> 5a2e6e8 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json a016ce0 
>   ambari-web/app/data/HDP2/site_properties.js 1fad1d6 
> 
> Diff: https://reviews.apache.org/r/51537/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 51354: Clean up Log Feeder

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51354/#review148072
---


Ship it!




Ship It!

- Oliver Szabo


On Sept. 6, 2016, 10:51 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51354/
> ---
> 
> (Updated Sept. 6, 2016, 10:51 p.m.)
> 
> 
> Review request for Ambari, Hayat Behlim, Dharmesh Makwana, Oliver Szabo, and 
> Robert Nettleton.
> 
> 
> Bugs: AMBARI-18246
> https://issues.apache.org/jira/browse/AMBARI-18246
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - long methods were broken up to smaller independent parts
> - narrowed the scope of too widely defined variables
> - fixed visibilities of the methods/fields
> - converted constants / enums to use java conventions for names
> - maximum line length is 120 characters, fixed unnecessary short lines
> - rearranged the functions so that they follow the order of the lifecycle of 
> these components
> - refactored some overcomplicated parts
> - removed unused codes
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
>  373d743 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigBlock.java
>  287982f 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java
>  ab371f1 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java
>  372c208 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterJSON.java
>  2954106 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java
>  7adb468 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
>  5feb9c4 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java
>  c9f5ded 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputMarker.java
>  48a7f1d 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputMgr.java
>  b18c9b0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java
>  c9d28bd 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
>  5ba56a5 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/reader/GZIPReader.java
>  a2a9db2 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/reader/LogsearchReaderFactory.java
>  c57c028 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FetchConfigFromSolr.java
>  ae0cfc0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FilterLogData.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogConfigFetcher.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandler.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogFeederConstants.java
>  09673a0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogFeederFilter.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogFeederFilterWrapper.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogfeederScheduler.java
>  bc807193 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/filter/ApplyLogFilter.java
>  b5e4eb3 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/filter/DefaultDataFilter.java
>  04d2ca4 
>   
> 

Re: Review Request 51685: AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server log file

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51685/
---

(Updated Sept. 7, 2016, 12:48 p.m.)


Review request for Ambari, Robert Nettleton and Sumit Mohanty.


Changes
---

Added branch-2.5 to the list of branches.


Bugs: AMBARI-18328
https://issues.apache.org/jira/browse/AMBARI-18328


Repository: ambari


Description
---

AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server 
log file


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
 de998cea801dba4b554255b209d20fb61bca36cc 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java
 2f4f890ddbc922fc732334e450d45d30222b2216 

Diff: https://reviews.apache.org/r/51685/diff/


Testing
---

** Manual Testing **
Deployed a VM with the latest trunk build and patched ambari-server JAR with 
the one from the local build. Ran a blueprint deployed and verified that the 
*settings* section of the blueprint was logged in 
/var/log/ambari-server/ambari-server.log.

** mvn test -Dtest=Blueprint*Test -DskipPythonTests **

Results :

Tests run: 245, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 39.290s
[INFO] Finished at: Wed Sep 07 12:26:24 PDT 2016
[INFO] Final Memory: 54M/861M
[INFO] 


Thanks,

Nahappan Somasundaram

Re: Review Request 51685: AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server log file

[Robert Nettleton]

> On Sept. 7, 2016, 1:58 p.m., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java,
> >  line 499
> > 
> >
> > Why is this being logged? 
> > 
> > It seems to me that you should be able to just GET the Blueprint back 
> > from the Ambari REST API, so logging this here seems redundant, especially 
> > considering that we generally don't log pieces of the Blueprint during the 
> > Blueprint POST. 
> > 
> > Is this section returned properly via the Blueprint GET call?
> 
> Nahappan Somasundaram wrote:
> Looking at the ambari logs, we want to figure out the auto start 
> configuration, without having to query the cluster. I agree that we don't log 
> pieces of the blueprint. The *settings* section is typically very small so 
> logging it will not take up much space in the logs. 
> 
> https://issues.apache.org/jira/browse/AMBARI-18332 tracks the API fix.

Ok, thanks for clarifying this.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51685/#review148010
---


On Sept. 7, 2016, 7:29 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51685/
> ---
> 
> (Updated Sept. 7, 2016, 7:29 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18328
> https://issues.apache.org/jira/browse/AMBARI-18328
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server 
> log file
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
>  de998cea801dba4b554255b209d20fb61bca36cc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java
>  2f4f890ddbc922fc732334e450d45d30222b2216 
> 
> Diff: https://reviews.apache.org/r/51685/diff/
> 
> 
> Testing
> ---
> 
> ** Manual Testing **
> Deployed a VM with the latest trunk build and patched ambari-server JAR with 
> the one from the local build. Ran a blueprint deployed and verified that the 
> *settings* section of the blueprint was logged in 
> /var/log/ambari-server/ambari-server.log.
> 
> ** mvn test -Dtest=Blueprint*Test -DskipPythonTests **
> 
> Results :
> 
> Tests run: 245, Failures: 0, Errors: 0, Skipped: 0
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 39.290s
> [INFO] Finished at: Wed Sep 07 12:26:24 PDT 2016
> [INFO] Final Memory: 54M/861M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 51685: AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server log file

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51685/
---

(Updated Sept. 7, 2016, 12:29 p.m.)


Review request for Ambari, Robert Nettleton and Sumit Mohanty.


Changes
---

Fixed broken UTs for BlueprintResourceProviderTests.


Bugs: AMBARI-18328
https://issues.apache.org/jira/browse/AMBARI-18328


Repository: ambari


Description
---

AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server 
log file


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
 de998cea801dba4b554255b209d20fb61bca36cc 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java
 2f4f890ddbc922fc732334e450d45d30222b2216 

Diff: https://reviews.apache.org/r/51685/diff/


Testing (updated)
---

** Manual Testing **
Deployed a VM with the latest trunk build and patched ambari-server JAR with 
the one from the local build. Ran a blueprint deployed and verified that the 
*settings* section of the blueprint was logged in 
/var/log/ambari-server/ambari-server.log.

** mvn test -Dtest=Blueprint*Test -DskipPythonTests **

Results :

Tests run: 245, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 39.290s
[INFO] Finished at: Wed Sep 07 12:26:24 PDT 2016
[INFO] Final Memory: 54M/861M
[INFO] 


Thanks,

Nahappan Somasundaram

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148053
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 39 - 40)


Syntax.



ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 45 - 46)


Odd syntax.  Usually see:

@Nullable
private String repoDir;



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 166 - 169)


Objects.equals() for simplicity.



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 179 - 181)


Use Objects.hash(...) for easier going


- Nate Cole


On Sept. 5, 2016, 9:26 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 9:26 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> 

Re: Review Request 51698: Added database metrics source

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51698/#review148050
---




ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
 (line 66)


What are these values?



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
 (line 81)


Should this be case insensitive match?


- Alejandro Fernandez


On Sept. 7, 2016, 5:59 p.m., Li-Wei Tseng wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51698/
> ---
> 
> (Updated Sept. 7, 2016, 5:59 p.m.)
> 
> 
> Review request for Ambari and Aravindan Vijayan.
> 
> 
> Bugs: AMBARI-17589
> https://issues.apache.org/jira/browse/AMBARI-17589
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added Database metrics generated by Eclipselink Performance Profiler
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/metrics.properties 5f01e39 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  bd34c25 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/AmbariMetricSink.java
>  809176be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsService.java
>  23845c9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSink.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSource.java
>  cf10408 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AbstractMetricsSource.java
>  58e2045 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
>  4618618 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/Configuration.java
>  705971f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
>  a04ca43 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
>  1645ebf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/EclipseLinkSessionCustomizer.java
>  6717e01 
>   ambari-server/src/main/resources/META-INF/persistence.xml 2bc3282 
>   
> ambari-server/src/test/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImplTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51698/diff/
> 
> 
> Testing
> ---
> 
> Unit test for emit metrics to AMS
> 
> 
> Thanks,
> 
> Li-Wei Tseng
> 
>

Review Request 51698: Added database metrics source

[Li-Wei Tseng]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51698/
---

Review request for Ambari and Aravindan Vijayan.


Bugs: AMBARI-17589
https://issues.apache.org/jira/browse/AMBARI-17589


Repository: ambari


Description
---

Added Database metrics generated by Eclipselink Performance Profiler


Diffs
-

  ambari-server/conf/unix/metrics.properties 5f01e39 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 d20b1d7 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 bd34c25 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/AmbariMetricSink.java
 809176be 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsService.java
 23845c9 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSink.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSource.java
 cf10408 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AbstractMetricsSource.java
 58e2045 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
 4618618 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/Configuration.java
 705971f 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DBSessionEventListener.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/DbMetricSource.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
 a04ca43 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
 1645ebf 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/EclipseLinkSessionCustomizer.java
 6717e01 
  ambari-server/src/main/resources/META-INF/persistence.xml 2bc3282 
  
ambari-server/src/test/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImplTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51698/diff/


Testing
---

Unit test for emit metrics to AMS


Thanks,

Li-Wei Tseng

Re: Review Request 51537: Use the PropertiesFile python command to create property files instead of jinja templates

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51537/
---

(Updated Sept. 7, 2016, 5:45 p.m.)


Review request for Ambari, Hayat Behlim, Dharmesh Makwana, Oliver Szabo, Robert 
Nettleton, and Sumit Mohanty.


Changes
---

Fix property name in site_properties.js as well


Bugs: AMBARI-18279
https://issues.apache.org/jira/browse/AMBARI-18279


Repository: ambari


Description
---

Instead of using jinja templates use the PropetiesFile command in 
resource_management.libraries.resources.properties_file to generate the 
property files, thus prevent enumerating all the non-custom properties in 
params.py

Also fixed a bug: the default values for the audit logs solr znodes and quorum 
were mixed up.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 56ff2d1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 5c3954f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 d3851d3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 aa89609 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
 86f0be4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
 1bcee2d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py f4dbd63 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 5a2e6e8 
  ambari-server/src/test/python/stacks/2.4/configs/default.json a016ce0 
  ambari-web/app/data/HDP2/site_properties.js 1fad1d6 

Diff: https://reviews.apache.org/r/51537/diff/


Testing
---

Tested on local cluster.


Thanks,

Miklos Gergely

Re: Review Request 51629: Add check to DB conistency checker for duplicate hostcomponentstate

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51629/#review148045
---




ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 (line 332)


The variable name is "GET_HOST_COMPONENT_STATE_DUPLICATES_QUERY" so I 
interpreted this as any pairs that are dups of the tuple , in 
which the state is also different.
But you're right that the component and host should only appear once.


- Alejandro Fernandez


On Sept. 6, 2016, 10:29 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51629/
> ---
> 
> (Updated Sept. 6, 2016, 10:29 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18309
> https://issues.apache.org/jira/browse/AMBARI-18309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add db consistency check for hostcomponent tables, for each component we 
> should have only one state.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  f302b8b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
>  535d74f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
>  4663310 
> 
> Diff: https://reviews.apache.org/r/51629/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51382: AMBARI-18250

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51382/#review148040
---


Ship it!




Ship It!

- Nate Cole


On Aug. 24, 2016, 2:25 p.m., Jay SenSharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51382/
> ---
> 
> (Updated Aug. 24, 2016, 2:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18250
> https://issues.apache.org/jira/browse/AMBARI-18250
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pig View Caching issue causes File does not exist: 
> /user//pig/jobs/job_id/stdout and stderr intermittently
> 
> 
> Diffs
> -
> 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/initialize.js 38ee093 
> 
> Diff: https://reviews.apache.org/r/51382/diff/
> 
> 
> Testing
> ---
> 
> done. Issue is intermittent and can not be reproduced everytime.
> 
> 
> Thanks,
> 
> Jay SenSharma
> 
>

Re: Review Request 51691: JMX metric retrieval method may unnecessarily refresh metrics at a high rate

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51691/#review148036
---


Ship it!




Ship It!

- Nate Cole


On Sept. 7, 2016, 1:18 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51691/
> ---
> 
> (Updated Sept. 7, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-18331
> https://issues.apache.org/jira/browse/AMBARI-18331
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In AMBARI-16913 we revised the JMX retrieval method to maintain an internal 
> state of JMX metrics, with the retrieval taking place out of band of the 
> actual jetty query requiring the metric.  However, each query will still 
> generate a refresh request to the metric, regardless of it's current state.
> 
> Recommend setting a TTL on a given metric such as 5 seconds, and only 
> generate a new request for the metric if a TTL has expired, to avoid large 
> amounts of repeat metrics collections in short windows.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 18234d0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  db1b81e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
>  7665d7f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
>  cbe827a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
>  fa36905 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
>  80d7438 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProviderTest.java
>  304b42f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/services/MetricsRetrievalServiceTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51691/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4623, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:19 min
> [INFO] Finished at: 2016-09-07T11:45:03-04:00
> [INFO] Final Memory: 38M/635M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51691: JMX metric retrieval method may unnecessarily refresh metrics at a high rate

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51691/#review148019
---




ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
 (lines 88 - 98)


Just a nit, but this might be called MetricSourceType or something.  
"Metric type" could be "point in time" or "series" etc.


- Nate Cole


On Sept. 7, 2016, 1:18 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51691/
> ---
> 
> (Updated Sept. 7, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-18331
> https://issues.apache.org/jira/browse/AMBARI-18331
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In AMBARI-16913 we revised the JMX retrieval method to maintain an internal 
> state of JMX metrics, with the retrieval taking place out of band of the 
> actual jetty query requiring the metric.  However, each query will still 
> generate a refresh request to the metric, regardless of it's current state.
> 
> Recommend setting a TTL on a given metric such as 5 seconds, and only 
> generate a new request for the metric if a TTL has expired, to avoid large 
> amounts of repeat metrics collections in short windows.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 18234d0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  db1b81e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
>  7665d7f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
>  cbe827a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
>  fa36905 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
>  80d7438 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProviderTest.java
>  304b42f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/services/MetricsRetrievalServiceTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51691/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4623, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:19 min
> [INFO] Finished at: 2016-09-07T11:45:03-04:00
> [INFO] Final Memory: 38M/635M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51691: JMX metric retrieval method may unnecessarily refresh metrics at a high rate

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51691/
---

(Updated Sept. 7, 2016, 1:18 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-18331
https://issues.apache.org/jira/browse/AMBARI-18331


Repository: ambari


Description
---

In AMBARI-16913 we revised the JMX retrieval method to maintain an internal 
state of JMX metrics, with the retrieval taking place out of band of the actual 
jetty query requiring the metric.  However, each query will still generate a 
refresh request to the metric, regardless of it's current state.

Recommend setting a TTL on a given metric such as 5 seconds, and only generate 
a new request for the metric if a TTL has expired, to avoid large amounts of 
repeat metrics collections in short windows.


Diffs (updated)
-

  ambari-server/docs/configuration/index.md 18234d0 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 db1b81e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 7665d7f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
 cbe827a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
 fa36905 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
 80d7438 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProviderTest.java
 304b42f 
  
ambari-server/src/test/java/org/apache/ambari/server/state/services/MetricsRetrievalServiceTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51691/diff/


Testing (updated)
---

Tests run: 4623, Failures: 0, Errors: 0, Skipped: 34

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 37:19 min
[INFO] Finished at: 2016-09-07T11:45:03-04:00
[INFO] Final Memory: 38M/635M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 51578: AMBARI-18071: Ambari Files View needs to have ability to load security configurations

[Keta Patel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51578/
---

(Updated Sept. 7, 2016, 4:57 p.m.)


Review request for Ambari, Di Li, DIPAYAN BHOWMICK, and Pallav Kulshreshtha.


Changes
---

There is some issue with "update diff", as one of the files in the patch 
"ConfigurationBuilder" shows not patched cleanly.
This same patch was tested by Hadoop QA (+1 result) on the Jira.
For reference, I have also attached the same patch here as 
"AMBARI-18071-Sep6.patch".


Bugs: AMBARI-18071
https://issues.apache.org/jira/browse/AMBARI-18071


Repository: ambari


Description
---

When HDFS is configured with Encryption Zones, Files View to browse files will 
give "No KeyProvider" error.

Steps to reproduce this issue:
1. Configure an encrypted zone in HDFS (Transparent Data Encryption). I have 
used Hadoop's KMS (installed tar manually).
2. Create a Files View instance and provide a user/group with the privilege to 
use the instance.
3. Log into the Ambari console as the user with the Files View permission.
4. Open the Files View instance.
5. Go to the folder which is configured as an encrypted zone.
6. Try to open an existing file in this folder.
7. This throws an error - java.io.IOException: No KeyProvider is configured, 
cannot access an encrypted file. 
8. When trying through the shell, opening this file works.

This happens because Files View doesn't have enough configuration set to browse 
secured zone. Files view doesn't even provide an option to add this 
configuration. This is why we see errors "No KeyProvider is configured, cannot 
access an encrypted file". 
To work around this, you could download client configuration from HDFS service 
tab, and copy the core-site.xml and hdfs-site.xml files to 
/etc/ambari-server/conf, then restart ambari-server. After this, the user is 
able to open the file in the encrypted zone. Basically, the property 
"dfs.encryption.key.provider.uri" which provides details of the KeyProvider, is 
obtained from the hdfs-site.xml present in the /etc/ambari-server/conf folder.


Diffs
-

  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
 d505e9a 
  
contrib/views/utils/src/test/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilderTest.java
 ed3025b 

Diff: https://reviews.apache.org/r/51578/diff/


Testing
---

**CAUSE:**
==
The error of "No KeyProvider is configured" is seen only for those cases when 
the HDFS uses DistributedFileSystem for its communication. When HDFS uses 
WebHDFSFileSystem for communication, this error is not seen and the Ambari View 
instance is able to open the files in the encrypted zones.

Why Ambari Views use either Distributed or WebHDFS file systems is explained 
below:
Ambari views can be created using one of the 3 modes of configuration:
1. Local cluster
2. Remote cluster
3. Custom configuration (no cluster is associated here).

The HDFS works through abstraction. For Ambari Views, the actual file system 
used during execution depends on whether the view instance was created using a 
Local/Remote cluster or using Custom configuration. For instances created using 
Local/Remote cluster, HDFS uses Distributed File System and for instances 
created using Custom configuration, HDFS uses WebHDFSFileSystem.
WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of 
all the HDFS configuration. For this reason, when a KMS is configured in HDFS, 
WebHDFSFileSystem is aware of the KeyProvider and no special config mapping is 
needed. Thus, even the view instance created using Custom configuration doesn't 
need any special configuration and can talk to the Encryption Zones 
successfully.

However, for view instances created using Local/Remote cluster configuration, 
HDFS uses the Distributed FileSystem. This Distributed FileSystem works as an 
HDFS client and hence, is not fully aware of all the HDFS configuration. We 
need to explicitly provide HDFS properties like 
"dfs.encryption.key.provider.uri" to these ambari view instances to provide 
details of the KeyProvider. The proposed fix helps in providing this property 
value to the view as follows.

**FIX:**

The proposed fix (attached as "AMBARI-18071.patch") checks if the current view 
instance configuration has any cluster associated in its context. If there is 
an associated cluster then the instance has a Local/Remote cluster 
configuration and needs to be provided with the HDFS KeyProvider information. 
Otherwise, the WebHDFSFileSystem will take care of the KeyProvider if KMS is 
configured.

To provide the property information, the parseProperties() in 
ConfigurationBuilder.java looked best as we also set the defaultFS property 
here. If a cluster is associated with the context, and if the property 
"dfs.encryption.key.provider.uri" is not null, 

Re: Review Request 51597: Desired state of client component should not be changed in case configuration changes are applied through a "Restart"

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51597/#review148012
---


Ship it!




Ship It!

- Laszlo Puskas


On Sept. 7, 2016, 2:15 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51597/
> ---
> 
> (Updated Sept. 7, 2016, 2:15 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18302
> https://issues.apache.org/jira/browse/AMBARI-18302
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Configuration changes are propagated to host components by issuing a restart 
> of the affected components.
> In case of master components this action will start the affected components 
> (regardless it is in INSTALLED or STARTED state) and the desired state will 
> be set to STARTED.
> In Ambari client components are always expected to be in INSTALLED state. 
> However upon configuration changes, the desired state of these components are 
> also set to STARTED. This may be misleading for ambari API users that 
> determine component states by checking the desired state and actual state. 
> (The actual state in case of client components will always be INSTALLED)
> This issue addresses this problem by not updating the desired state when 
> client components are restarted.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  8c8ae10 
> 
> Diff: https://reviews.apache.org/r/51597/diff/
> 
> 
> Testing
> ---
> 
> Manually done.
> 
> commit 6d13be5b9ab6aa129bc1a71fc2eb57ccf7bbe6cd
> Author: Toader, Sebastian 
> Date:?? Wed Sep 7 15:18:25 2016 +0200
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 51597: Desired state of client component should not be changed in case configuration changes are applied through a "Restart"

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51597/
---

(Updated Sept. 7, 2016, 2:15 p.m.)


Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
Toader.


Changes
---

Added commit.


Bugs: AMBARI-18302
https://issues.apache.org/jira/browse/AMBARI-18302


Repository: ambari


Description
---

Configuration changes are propagated to host components by issuing a restart of 
the affected components.
In case of master components this action will start the affected components 
(regardless it is in INSTALLED or STARTED state) and the desired state will be 
set to STARTED.
In Ambari client components are always expected to be in INSTALLED state. 
However upon configuration changes, the desired state of these components are 
also set to STARTED. This may be misleading for ambari API users that determine 
component states by checking the desired state and actual state. (The actual 
state in case of client components will always be INSTALLED)
This issue addresses this problem by not updating the desired state when client 
components are restarted.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 8c8ae10 

Diff: https://reviews.apache.org/r/51597/diff/


Testing (updated)
---

Manually done.

commit 6d13be5b9ab6aa129bc1a71fc2eb57ccf7bbe6cd
Author: Toader, Sebastian 
Date:?? Wed Sep 7 15:18:25 2016 +0200


Thanks,

Laszlo Puskas

Re: Review Request 51597: Desired state of client component should not be changed in case configuration changes are applied through a "Restart"

[Laszlo Puskas]

> On Sept. 2, 2016, 2:22 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java,
> >  lines 473-478
> > 
> >
> > I'm curious if we should also try to catch this in the 
> > ServiceComponentHost; don't let it's state move to STARTED if it knows it's 
> > a client.
> 
> Laszlo Puskas wrote:
> Good point, i am a bit uncertain though that such thing goes into that 
> class. (I'd expect some kind of state machine or other specialized service 
> implementation to decide which are the valid state transitions for 
> hostcomponents.) As this change only affects custom commands i'd leave this 
> as it is; please let me know if this is acceptable.
> 
> Jonathan Hurley wrote:
> I don't think it should hold up this review as you've fixed the specific 
> issue with the command. I also agree that there should be a state machine to 
> govern this. The problem is that the low budget state machine we currently 
> use doesn't take into account the type of component which the state is being 
> added for. Catching it in the impl will ensure that another mistake can't be 
> made. Having a client in the STARTED state could really mess with other logic 
> in the system.

I am about to register this in a bugdb item.


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51597/#review147660
---


On Sept. 2, 2016, 12:11 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51597/
> ---
> 
> (Updated Sept. 2, 2016, 12:11 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18302
> https://issues.apache.org/jira/browse/AMBARI-18302
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Configuration changes are propagated to host components by issuing a restart 
> of the affected components.
> In case of master components this action will start the affected components 
> (regardless it is in INSTALLED or STARTED state) and the desired state will 
> be set to STARTED.
> In Ambari client components are always expected to be in INSTALLED state. 
> However upon configuration changes, the desired state of these components are 
> also set to STARTED. This may be misleading for ambari API users that 
> determine component states by checking the desired state and actual state. 
> (The actual state in case of client components will always be INSTALLED)
> This issue addresses this problem by not updating the desired state when 
> client components are restarted.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  8c8ae10 
> 
> Diff: https://reviews.apache.org/r/51597/diff/
> 
> 
> Testing
> ---
> 
> Unit tests running.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 51685: AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server log file

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51685/#review148010
---


Fix it, then Ship it!




The change looks fine to me, just need a clarification as to why this is being 
logged in addition to being available via the REST API. 

Thanks.


ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
 (line 499)


Why is this being logged? 

It seems to me that you should be able to just GET the Blueprint back from 
the Ambari REST API, so logging this here seems redundant, especially 
considering that we generally don't log pieces of the Blueprint during the 
Blueprint POST. 

Is this section returned properly via the Blueprint GET call?


- Robert Nettleton


On Sept. 7, 2016, 2:14 a.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51685/
> ---
> 
> (Updated Sept. 7, 2016, 2:14 a.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Sid Wagle.
> 
> 
> Bugs: AMBARI-18328
> https://issues.apache.org/jira/browse/AMBARI-18328
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18328: Blueprints: Log "setting" section of blueprint in ambari server 
> log file
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
>  de998cea801dba4b554255b209d20fb61bca36cc 
> 
> Diff: https://reviews.apache.org/r/51685/diff/
> 
> 
> Testing
> ---
> 
> ** Manual Testing **
> Deployed a VM with the latest trunk build and patched ambari-server JAR with 
> the one from the local build. Ran a blueprint deployed and verified that the 
> *settings* section of the blueprint was logged in 
> /var/log/ambari-server/ambari-server.log.
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 51537: Use the PropertiesFile python command to create property files instead of jinja templates

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51537/
---

(Updated Sept. 7, 2016, 1:24 p.m.)


Review request for Ambari, Hayat Behlim, Dharmesh Makwana, Oliver Szabo, Robert 
Nettleton, and Sumit Mohanty.


Changes
---

Updated external auth enable property name


Bugs: AMBARI-18279
https://issues.apache.org/jira/browse/AMBARI-18279


Repository: ambari


Description
---

Instead of using jinja templates use the PropetiesFile command in 
resource_management.libraries.resources.properties_file to generate the 
property files, thus prevent enumerating all the non-custom properties in 
params.py

Also fixed a bug: the default values for the audit logs solr znodes and quorum 
were mixed up.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 56ff2d1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 5c3954f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 d3851d3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 aa89609 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
 86f0be4 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
 1bcee2d 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py f4dbd63 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 5a2e6e8 
  ambari-server/src/test/python/stacks/2.4/configs/default.json a016ce0 
  ambari-web/app/data/HDP2/site_properties.js 1fad1d6 

Diff: https://reviews.apache.org/r/51537/diff/


Testing
---

Tested on local cluster.


Thanks,

Miklos Gergely

Re: Review Request 51632: Refactor logsearch portal code

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51632/#review148006
---




ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
 (line 33)


You are not using logsearch.auth.external_auth.enable



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
 (line 36)


The name of the property after the PropertiesFile patch will be applied is 
logsearch.auth.external_auth.login_url


- Miklos Gergely


On Sept. 5, 2016, 4:46 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51632/
> ---
> 
> (Updated Sept. 5, 2016, 4:46 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sandor Magyari, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18310
> https://issues.apache.org/jira/browse/AMBARI-18310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - rename/move auditLogRest/manager and serviceLogRest/manager (review board 
> shows it as new added code)
> - using @Inject instead of @Autowired
> - using objects instead of strings for Solr responses (later I want to use 
> spring data solr project to get rid of query generation custom code) ... for 
> that i will fill the Criteria objects with fields as well (now those are only 
> markers)
> - adding converters between requests and search criterias
> - rename and refactor a lot of Vxxx objects, and use them as response objects 
> on endpoints, because of this rest api documentations can show them as they 
> are (not as strings)
> - use javaConfig instead of xml for spring (now security part still in xml, 
> but i will change that later too)
> - move Configuration values into objects, trying to get rid of Property Helper
> 
> Because that is a pretty big change, I created that review, its not done yet 
> (there are still Vxxx objects and also i need to get rid of xml 
> configurations or web.xml as well), also after the documentation was created, 
> there are a lot of unused (or almost duplicated like: 
> componentName/component_name/component) query params there (from the UI), so 
> i will need some manual testing to make sure i can remove them.
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml b36c9bb 
>   ambari-logsearch/ambari-logsearch-portal/production/build.js afeea67 
>   ambari-logsearch/ambari-logsearch-portal/production/r.js 6e6ffd0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/managed-schema
>  f58652c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/managed-schema
>  3b4cf19 
>   ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/README.md 
> 3cc4915 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/reset_collections.sh
>  7c742c0 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/restart_solr.sh
>  bd436bb 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/dev/solrcloud/update_config.sh
>  93e3496 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  15355a1 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
>  edb4ffa 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  5235fab 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
>  142b29b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/Marker.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/SearchCriteria.java
>  6b74144 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApiDocConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApplicationConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
>  PRE-CREATION 
>   
> 

Review Request 51691: JMX metric retrieval method may unnecessarily refresh metrics at a high rate

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51691/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-18331
https://issues.apache.org/jira/browse/AMBARI-18331


Repository: ambari


Description
---

In AMBARI-16913 we revised the JMX retrieval method to maintain an internal 
state of JMX metrics, with the retrieval taking place out of band of the actual 
jetty query requiring the metric.  However, each query will still generate a 
refresh request to the metric, regardless of it's current state.

Recommend setting a TTL on a given metric such as 5 seconds, and only generate 
a new request for the metric if a TTL has expired, to avoid large amounts of 
repeat metrics collections in short windows.


Diffs
-

  ambari-server/docs/configuration/index.md 18234d0 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 db1b81e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 7665d7f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/RestMetricsPropertyProvider.java
 cbe827a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/services/MetricsRetrievalService.java
 fa36905 
  
ambari-server/src/test/java/org/apache/ambari/server/state/services/MetricsRetrievalServiceTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51691/diff/


Testing
---

mvn clean test


Thanks,

Jonathan Hurley

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148002
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 1129)


This won't include extension services.  You would need to loop through all 
the ServiceInfo objects and check to see if they have repos.


- Tim Thorpe


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51629: Add check to DB conistency checker for duplicate hostcomponentstate

[Vitalyi Brodetskyi]

> On Вер. 6, 2016, 11:55 після полудня, Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java,
> >  line 332
> > 
> >
> > This query is not correct.
> > 
> > try,
> > select component_name from (select component_name, count(*) as num from 
> > (select component_name, current_state from hostcomponentstate group by 
> > component_name, current_state) as agg group by component_name) as result 
> > where num > 1;
> 
> Sid Wagle wrote:
> Why is the query wrong?

Alejandro, my query search for duplicates of component - host pair. For each 
host we should have only one instance of hostcomponent. In your query you are 
using component - current_state pair, it's a different logic.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51629/#review147949
---


On Вер. 6, 2016, 10:29 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51629/
> ---
> 
> (Updated Вер. 6, 2016, 10:29 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18309
> https://issues.apache.org/jira/browse/AMBARI-18309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add db consistency check for hostcomponent tables, for each component we 
> should have only one state.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  f302b8b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
>  535d74f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
>  4663310 
> 
> Diff: https://reviews.apache.org/r/51629/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51530: Provide Notes On Service Config Changes During Ambari Upgrade

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51530/#review147997
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Вер. 7, 2016, 10:26 до полудня, Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51530/
> ---
> 
> (Updated Вер. 7, 2016, 10:26 до полудня)
> 
> 
> Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18278
> https://issues.apache.org/jira/browse/AMBARI-18278
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For every configuration change made, a meaningful note should be included so 
> that administrators understand why so many versions are being created. Any of 
> the following would be fine:
> 
> - Ambari Upgrade updated storm-site
> - Ambari Upgrade from 2.2.0 to 2.4.0 changed storm-site
> - The following configurations were changed as part of the Ambar Server 
> Upgrade: storm-site
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  57cce3f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java
>  e3b5d15 
> 
> Diff: https://reviews.apache.org/r/51530/diff/
> 
> 
> Testing
> ---
> 
> checked on live cluster (attached screenshot to jira)
> running unit tests
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 51530: Provide Notes On Service Config Changes During Ambari Upgrade

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51530/
---

(Updated Sept. 7, 2016, 1:26 p.m.)


Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Changes
---

fixed unit tests


Bugs: AMBARI-18278
https://issues.apache.org/jira/browse/AMBARI-18278


Repository: ambari


Description
---

For every configuration change made, a meaningful note should be included so 
that administrators understand why so many versions are being created. Any of 
the following would be fine:

- Ambari Upgrade updated storm-site
- Ambari Upgrade from 2.2.0 to 2.4.0 changed storm-site
- The following configurations were changed as part of the Ambar Server 
Upgrade: storm-site


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 57cce3f 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java
 e3b5d15 

Diff: https://reviews.apache.org/r/51530/diff/


Testing
---

checked on live cluster (attached screenshot to jira)
running unit tests


Thanks,

Dmitro Lisnichenko

Re: Review Request 51659: Ambari cannot install HDP from mirror list

[Dmitro Lisnichenko]

> On Sept. 6, 2016, 9:04 p.m., Alejandro Fernandez wrote:
> > What is the format of the mirror list?
> 
> Nate Cole wrote:
> +1 for this - I would prefer if mirrors is not just some CSV but was 
> structured (set of items, etc).

Mirror list looks like 
mirrorlist=http://mirrorlist.centos.org/?release=$releasever=$basearch=os
 or mirrorlist=file:///etc/yum.repos.d/local-base
That is just a url or local file that contains list of repository mirrors. So 
on Ambari side, it's just a string with URI


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51659/#review147878
---


On Sept. 6, 2016, 8:23 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51659/
> ---
> 
> (Updated Sept. 6, 2016, 8:23 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18325
> https://issues.apache.org/jira/browse/AMBARI-18325
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari should update the code and be able to use a mirror list instead of a 
> single url.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  6aa0853 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java
>  84929bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  8d6e6e2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackContext.java 
> da7f021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java
>  b1c643a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java
>  8c629ca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java
>  143f9af 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
>  538fa48 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  3ad1f1f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/StackUpgradeUtilTest.java
>  35650a1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
>  dcac986 
> 
> Diff: https://reviews.apache.org/r/51659/diff/
> 
> 
> Testing
> ---
> 
> preview of patch (works on live cluster)
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 51382: AMBARI-18250

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51382/#review147983
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On Aug. 24, 2016, 6:25 p.m., Jay SenSharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51382/
> ---
> 
> (Updated Aug. 24, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18250
> https://issues.apache.org/jira/browse/AMBARI-18250
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pig View Caching issue causes File does not exist: 
> /user//pig/jobs/job_id/stdout and stderr intermittently
> 
> 
> Diffs
> -
> 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/initialize.js 38ee093 
> 
> Diff: https://reviews.apache.org/r/51382/diff/
> 
> 
> Testing
> ---
> 
> done. Issue is intermittent and can not be reproduced everytime.
> 
> 
> Thanks,
> 
> Jay SenSharma
> 
>

Re: Review Request 51382: AMBARI-18250

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51382/#review147981
---


Ship it!




Ship It!

- Gaurav Nagar


On Aug. 24, 2016, 6:25 p.m., Jay SenSharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51382/
> ---
> 
> (Updated Aug. 24, 2016, 6:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18250
> https://issues.apache.org/jira/browse/AMBARI-18250
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pig View Caching issue causes File does not exist: 
> /user//pig/jobs/job_id/stdout and stderr intermittently
> 
> 
> Diffs
> -
> 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/initialize.js 38ee093 
> 
> Diff: https://reviews.apache.org/r/51382/diff/
> 
> 
> Testing
> ---
> 
> done. Issue is intermittent and can not be reproduced everytime.
> 
> 
> Thanks,
> 
> Jay SenSharma
> 
>

Re: Review Request 51599: AMBARI-18292 Support dispatching notifications of assigned alert states for script-based alert dispatcher

[yao lei]

> On 九月 3, 2016, 7:17 p.m., Jonathan Hurley wrote:
> > This is not the correct way to accomplish this. Instead, when you create 
> > the alert target, you can specify which alert states the alert target cares 
> > about:
> > 
> > ```
> > {
> >   "AlertTarget": {
> > "name": "Simple",
> > "description": "This target does not work and is only an exmaple of 
> > setting states",
> > "alert_states": ["OK", "WARNING"]
> > "notification_type": "FOO",
> > "groups": [1,2,3]
> >   }
> > }
> > ```
> > 
> > This is also accomplished through the Web Client when created/editing the 
> > target.
> 
> yao lei wrote:
> thanks for your explaination.
> Do your mean that we can creating or editing the  alert target on web ui?
> 
> Jonathan Hurley wrote:
> Yes you can.

Thanks your reply.
I think i need to upgrade my current ambari  2.1.1


- yao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51599/#review147727
---


On 九月 3, 2016, 2:36 p.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51599/
> ---
> 
> (Updated 九月 3, 2016, 2:36 p.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18292
> https://issues.apache.org/jira/browse/AMBARI-18292
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default, Ambari allow all notifications of all kinds of alert states 
> (OK,WARNING,CRITICAL,UNKNOWN) to dispatch in script-based alert dispatcher.
> However, sometimes we hope to filter notification of some alert states 
> This patch will resolve this by setting 
> 'notification.dispatch.alert.script.states' in ambari.properties as following:
> 
> #Only dispatch WARNING and CRITICAL state notification 
> notification.dispatch.alert.script.states=WARNING,CRITICAL
> 
> If you don't set this property that means all notifications will be 
> dispatched.
> 
> I think making the desired alerts notifications configurable may be useful 
> for Ambari users.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> Diff: https://reviews.apache.org/r/51599/diff/
> 
> 
> Testing
> ---
> 
> Tested
> 
> Running 
> org.apache.ambari.server.notifications.dispatchers.AlertScriptDispatcherTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.096 sec - in
> 
> 
> Thanks,
> 
> yao lei
> 
>