Re: Review Request 63020: jdbc(spark[2]) URLs not configured correctly for secure clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63020/ --- (Updated Oct. 16, 2017, 12:38 p.m.) Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, Rohit Choudhary, Sumit Mohanty, and venkat sairam. Bugs: AMBARI-3 https://issues.apache.org/jira/browse/AMBARI-3 Repository: ambari Description --- In case of JDBC interpreter auto-configuration, URLs being set are not accounting for security, transport mode and SSL upon zeppelin server restart via ambari. Diffs (updated) - ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py 6a84d7926b ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/params.py e69037cb10 Diff: https://reviews.apache.org/r/63020/diff/2/ Changes: https://reviews.apache.org/r/63020/diff/1-2/ Testing --- Manually Thanks, Prabhjyot Singh
Re: Review Request 62976: Zeppelin service check failed during EU from 2.5 to 2.6 as ZeppelinServer can not be instantiated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62976/#review188102 --- Ship it! Ship It! - venkat sairam On Oct. 13, 2017, 2:32 p.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62976/ > --- > > (Updated Oct. 13, 2017, 2:32 p.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-22233 > https://issues.apache.org/jira/browse/AMBARI-22233 > > > Repository: ambari > > > Description > --- > > There are two issuse > - one a file can exists with zero byte, it should be treated as file not in > file system > - premission of interpreter.json should be 644 instead of 640 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > 6a84d7926b > ambari-server/src/test/python/stacks/2.6/ZEPPELIN/test_zeppelin_070.py > 400350c4f0 > > > Diff: https://reviews.apache.org/r/62976/diff/1/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Re: Review Request 62976: Zeppelin service check failed during EU from 2.5 to 2.6 as ZeppelinServer can not be instantiated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62976/#review188103 --- Ship it! Ship It! - Pallav Kulshreshtha On Oct. 13, 2017, 2:32 p.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62976/ > --- > > (Updated Oct. 13, 2017, 2:32 p.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-22233 > https://issues.apache.org/jira/browse/AMBARI-22233 > > > Repository: ambari > > > Description > --- > > There are two issuse > - one a file can exists with zero byte, it should be treated as file not in > file system > - premission of interpreter.json should be 644 instead of 640 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > 6a84d7926b > ambari-server/src/test/python/stacks/2.6/ZEPPELIN/test_zeppelin_070.py > 400350c4f0 > > > Diff: https://reviews.apache.org/r/62976/diff/1/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Re: Review Request 63020: jdbc(spark[2]) URLs not configured correctly for secure clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63020/#review188101 --- Ship it! Ship It! - venkat sairam On Oct. 16, 2017, 7:08 a.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63020/ > --- > > (Updated Oct. 16, 2017, 7:08 a.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-3 > https://issues.apache.org/jira/browse/AMBARI-3 > > > Repository: ambari > > > Description > --- > > In case of JDBC interpreter auto-configuration, URLs being set are not > accounting for security, transport mode and SSL upon zeppelin server restart > via ambari. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > 6a84d7926b > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/params.py > e69037cb10 > > > Diff: https://reviews.apache.org/r/63020/diff/2/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Re: Review Request 62976: Zeppelin service check failed during EU from 2.5 to 2.6 as ZeppelinServer can not be instantiated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62976/#review188104 --- Ship it! Ship It! - Gaurav Nagar On Oct. 13, 2017, 2:32 p.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62976/ > --- > > (Updated Oct. 13, 2017, 2:32 p.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-22233 > https://issues.apache.org/jira/browse/AMBARI-22233 > > > Repository: ambari > > > Description > --- > > There are two issuse > - one a file can exists with zero byte, it should be treated as file not in > file system > - premission of interpreter.json should be 644 instead of 640 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > 6a84d7926b > ambari-server/src/test/python/stacks/2.6/ZEPPELIN/test_zeppelin_070.py > 400350c4f0 > > > Diff: https://reviews.apache.org/r/62976/diff/1/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Re: Review Request 63020: jdbc(spark[2]) URLs not configured correctly for secure clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63020/#review188105 --- Ship it! Ship It! - Gaurav Nagar On Oct. 16, 2017, 7:08 a.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63020/ > --- > > (Updated Oct. 16, 2017, 7:08 a.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-3 > https://issues.apache.org/jira/browse/AMBARI-3 > > > Repository: ambari > > > Description > --- > > In case of JDBC interpreter auto-configuration, URLs being set are not > accounting for security, transport mode and SSL upon zeppelin server restart > via ambari. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > 6a84d7926b > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/params.py > e69037cb10 > > > Diff: https://reviews.apache.org/r/63020/diff/2/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Review Request 62982: Expression parser support for JMXServerSide alerts
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62982/
---
Review request for Ambari, Jonathan Hurley, Nate Cole, and Sebastian Toader.
Bugs: AMBARI-22236
https://issues.apache.org/jira/browse/AMBARI-22236
Repository: ambari
Description
---
Many of the existing JMX alerts use expressions like this
"jmx": {
"property_list": [
"Hadoop:service=NameNode,name=FSNamesystemState/CapacityUsed",
"Hadoop:service=NameNode,name=FSNamesystemState/CapacityRemaining"
],
"value": "{0}/({0} + {1}) * 100.0"
}
JMXServerSide alert should be able to evaluate these expressions in a secure
and safe way.
Supported operators: +,-,*,/,(,)
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/alerts/JmxServerSideAlert.java
a4b86f8
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
665430d
ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java
d7283fe
ambari-server/src/main/java/org/apache/ambari/server/state/alert/Reporting.java
51d074e
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProviderTest.java
3ef2c48
ambari-server/src/test/java/org/apache/ambari/server/state/alert/JmxInfoTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/62982/diff/1/
Testing
---
tested with an JMX alert with value="{0}/({0} + {1}) * 100.0"
existing tests: pending
Thanks,
Attila Magyar
Re: Review Request 62982: Expression parser support for JMXServerSide alerts
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62982/#review188109
---
Ship it!
Ship It!
- Sebastian Toader
On Oct. 16, 2017, 10:27 a.m., Attila Magyar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62982/
> ---
>
> (Updated Oct. 16, 2017, 10:27 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sebastian Toader.
>
>
> Bugs: AMBARI-22236
> https://issues.apache.org/jira/browse/AMBARI-22236
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Many of the existing JMX alerts use expressions like this
>
> "jmx": {
> "property_list": [
> "Hadoop:service=NameNode,name=FSNamesystemState/CapacityUsed",
>
> "Hadoop:service=NameNode,name=FSNamesystemState/CapacityRemaining"
> ],
> "value": "{0}/({0} + {1}) * 100.0"
> }
>
>
> JMXServerSide alert should be able to evaluate these expressions in a secure
> and safe way.
>
> Supported operators: +,-,*,/,(,)
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/alerts/JmxServerSideAlert.java
> a4b86f8
>
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
> 665430d
>
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java
> d7283fe
>
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/Reporting.java
> 51d074e
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProviderTest.java
> 3ef2c48
>
> ambari-server/src/test/java/org/apache/ambari/server/state/alert/JmxInfoTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62982/diff/1/
>
>
> Testing
> ---
>
> tested with an JMX alert with value="{0}/({0} + {1}) * 100.0"
>
> existing tests: pending
>
>
> Thanks,
>
> Attila Magyar
>
>
Re: Review Request 62996: Restart of random service could fail during express downgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62996/#review188111 --- Ship it! Ship It! - Dmitro Lisnichenko On Oct. 16, 2017, 4:53 a.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62996/ > --- > > (Updated Oct. 16, 2017, 4:53 a.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Jonathan > Hurley. > > > Bugs: AMBARI-22240 > https://issues.apache.org/jira/browse/AMBARI-22240 > > > Repository: ambari > > > Description > --- > > Turns out when we did the import for PU, we neglected to bring over the > per-service config changes that are required to support it. This didn't show > during all our development because we were only checking on the same stack. > > This jira brings in those changes such that we are correctly creating and > reverting configs when crossing stack versions. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 91bfe09deb > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 18659d44a1 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigFactory.java > d6cd99786c > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > 987ff3818d > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java > 0a861d80c9 > > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java > 77e2d47d09 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 6bde42c10b > > ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java > 1a8b5b61f2 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > d7cd087001 > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 87721e4ccd > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java > 76f4bb30d9 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java > 088ec8ddd2 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalogTest.java > 96ccc57f39 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 2590d1c995 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java > e6dbb7c0cf > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > 23962b7d79 > > > Diff: https://reviews.apache.org/r/62996/diff/2/ > > > Testing > --- > > Manual IN PROGRESS: > RU Full through Finalize (DONE) > RU Full but Downgrade (DONE) > EU Full through Finalize (DONE) > EU Full but Downgrade (DONE) > > EU Patch but Downgrade (DONE) > EU Patch then Finalize (DONE) > EU Patch then Finalize then Revert (DONE) > > PENDING Unit tests: > Verify no tests are broken with this change: > > > Thanks, > > Nate Cole > >
Re: Review Request 63020: jdbc(spark[2]) URLs not configured correctly for secure clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63020/ --- (Updated Oct. 16, 2017, 2:46 p.m.) Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, Rohit Choudhary, Sumit Mohanty, and venkat sairam. Bugs: AMBARI-3 https://issues.apache.org/jira/browse/AMBARI-3 Repository: ambari Description --- In case of JDBC interpreter auto-configuration, URLs being set are not accounting for security, transport mode and SSL upon zeppelin server restart via ambari. Diffs (updated) - ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py 6a84d7926b ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/params.py e69037cb10 Diff: https://reviews.apache.org/r/63020/diff/3/ Changes: https://reviews.apache.org/r/63020/diff/2-3/ Testing --- Manually Thanks, Prabhjyot Singh
Re: Review Request 62996: Restart of random service could fail during express downgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62996/#review188112 --- Ship it! Ship It! - Dmytro Grinenko On Oct. 16, 2017, 1:53 a.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62996/ > --- > > (Updated Oct. 16, 2017, 1:53 a.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Jonathan > Hurley. > > > Bugs: AMBARI-22240 > https://issues.apache.org/jira/browse/AMBARI-22240 > > > Repository: ambari > > > Description > --- > > Turns out when we did the import for PU, we neglected to bring over the > per-service config changes that are required to support it. This didn't show > during all our development because we were only checking on the same stack. > > This jira brings in those changes such that we are correctly creating and > reverting configs when crossing stack versions. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 91bfe09deb > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 18659d44a1 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigFactory.java > d6cd99786c > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > 987ff3818d > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java > 0a861d80c9 > > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java > 77e2d47d09 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 6bde42c10b > > ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java > 1a8b5b61f2 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > d7cd087001 > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 87721e4ccd > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java > 76f4bb30d9 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java > 088ec8ddd2 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalogTest.java > 96ccc57f39 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 2590d1c995 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java > e6dbb7c0cf > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > 23962b7d79 > > > Diff: https://reviews.apache.org/r/62996/diff/2/ > > > Testing > --- > > Manual IN PROGRESS: > RU Full through Finalize (DONE) > RU Full but Downgrade (DONE) > EU Full through Finalize (DONE) > EU Full but Downgrade (DONE) > > EU Patch but Downgrade (DONE) > EU Patch then Finalize (DONE) > EU Patch then Finalize then Revert (DONE) > > PENDING Unit tests: > Verify no tests are broken with this change: > > > Thanks, > > Nate Cole > >
Re: Review Request 62976: Zeppelin service check failed during EU from 2.5 to 2.6 as ZeppelinServer can not be instantiated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62976/ --- (Updated Oct. 16, 2017, 4:23 p.m.) Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, Rohit Choudhary, Sumit Mohanty, and venkat sairam. Bugs: AMBARI-22233 https://issues.apache.org/jira/browse/AMBARI-22233 Repository: ambari Description --- There are two issuse - one a file can exists with zero byte, it should be treated as file not in file system - premission of interpreter.json should be 644 instead of 640 Diffs - ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py 6a84d7926b ambari-server/src/test/python/stacks/2.6/ZEPPELIN/test_zeppelin_070.py 400350c4f0 Diff: https://reviews.apache.org/r/62976/diff/1/ Testing --- Manually File Attachments (updated) AMBARI-22233_branch-2.6_v1.patch https://reviews.apache.org/media/uploaded/files/2017/10/16/a0a67ed4-f1d1-4e87-a76b-e599468b587d__AMBARI-22233_branch-2.6_v1.patch Thanks, Prabhjyot Singh
Re: Review Request 62996: Restart of random service could fail during express downgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62996/#review188125 --- Ship it! Ship It! - Jonathan Hurley On Oct. 15, 2017, 9:53 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62996/ > --- > > (Updated Oct. 15, 2017, 9:53 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Jonathan > Hurley. > > > Bugs: AMBARI-22240 > https://issues.apache.org/jira/browse/AMBARI-22240 > > > Repository: ambari > > > Description > --- > > Turns out when we did the import for PU, we neglected to bring over the > per-service config changes that are required to support it. This didn't show > during all our development because we were only checking on the same stack. > > This jira brings in those changes such that we are correctly creating and > reverting configs when crossing stack versions. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java > 91bfe09deb > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 18659d44a1 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigFactory.java > d6cd99786c > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > 987ff3818d > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java > 0a861d80c9 > > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java > 77e2d47d09 > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 6bde42c10b > > ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java > 1a8b5b61f2 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > d7cd087001 > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 87721e4ccd > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java > 76f4bb30d9 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java > 088ec8ddd2 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalogTest.java > 96ccc57f39 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java > 2590d1c995 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java > e6dbb7c0cf > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > 23962b7d79 > > > Diff: https://reviews.apache.org/r/62996/diff/2/ > > > Testing > --- > > Manual IN PROGRESS: > RU Full through Finalize (DONE) > RU Full but Downgrade (DONE) > EU Full through Finalize (DONE) > EU Full but Downgrade (DONE) > > EU Patch but Downgrade (DONE) > EU Patch then Finalize (DONE) > EU Patch then Finalize then Revert (DONE) > > PENDING Unit tests: > Verify no tests are broken with this change: > > > Thanks, > > Nate Cole > >
Review Request 63027: Allow wildcard for log directory folder in the path component of Logfeeder input
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63027/ --- Review request for Ambari, Miklos Gergely and Robert Nettleton. Bugs: AMBARI-21145 https://issues.apache.org/jira/browse/AMBARI-21145 Repository: ambari Description --- Allow wildcards for handling log directories. Using ant library for this. (on trunk, the solution will be JDK 8 based, so its a temporal hack for 2.6.1, for trunk some classes have cleaned up, so the implementation there will be different) The idea is if we have a folder pattern, we will craete a thread group, and there will be different threads for every matching input (now only for tail file) Also there is 2 new thread introduced: - one which checks is the monitored file too old, if it is, stop the thread for that (detach) - an another one which checks there is any new file available (like log files with date patterns in its name), and if it is, update the monitoring thread with that (log path upgrade) Introduced some new configuration as well for inputs: - path_update_interval_min: the period in minutes for checking new files (default: 5, based on detach values, its possible that a new input wont be monitored) - detach_interval_min: the period in minutes for checking which files are too old (default: 300) - detach_time_min: the period in minutes when we flag a file is too old (default: 2000) Also fix an issue which cause that we cannot use grok field names more deeply (use deep_extract param for this one in grok filter) and i updated the docker container as well to make it work with branch-2.6 (and use different version for container in order to keep that) Diffs - ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java 7e2da70 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java 9f54d8a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java 3737839 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/AbstractLogFileMonitor.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/LogFileDetachMonitor.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/LogFilePathUpdateMonitor.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandler.java 0ece637 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java ffd6cec ambari-logsearch/docker/Dockerfile dfa1462 ambari-logsearch/docker/logsearch-docker.sh a2df90f Diff: https://reviews.apache.org/r/63027/diff/1/ Testing --- done, no UTs here, FTs are in progress Thanks, Oliver Szabo
Review Request 63032: YARN Service Checks Fails Because of Old hadoop-client Classpath Entry
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63032/
---
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-22245
https://issues.apache.org/jira/browse/AMBARI-22245
Repository: ambari
Description
---
STR:
- Install basic YARN on HDP 2.6.0.0 (no Tez)
- Upgrade just YARN to HDP 2.6.0.2
Service checks will fail with the following:
```
17/10/12 17:55:20 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoSuchMethodError:
org.apache.hadoop.io.retry.RetryPolicies.retryForeverWithFixedSleep(JLjava/util/concurrent/TimeUnit;)Lorg/apache/hadoop/io/retry/RetryPolicy;
at
org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:280)
```
- All YARN daemons and clients are reporting 2.6.0.2
- All YARN daemons have loaded JARs for 2.6.0.2
What is happening here is that the applications being run on YARN are picking
up the older hadoop-common JAR file. The method {{retryForeverWithFixedSleep}}
did not exist in HDP 2.6.0.0.
It is picking up the older JARs for running applications because of the
{{yarn-site.xml}} property:
```
yarn-site.xml- yarn.application.classpath
yarn-site.xml:
/etc/hadoop/conf,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*,/usr/hdp/current/ext/hadoop/*
```
Ambari should be parameterizing the {{/usr/hdp/current/hadoop-client}} paths
here.
Diffs
-
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
f21719261a
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/status_params.py
c2e9d92a04
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
d0b4bb1f0f
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
5dbcf2da1b
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
b7548eab30
ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
d3a096a4fa
Diff: https://reviews.apache.org/r/63032/diff/1/
Testing
---
Manual
Thanks,
Jonathan Hurley
Review Request 63034: AMBARI-22244. Update service component API call to work for second service instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63034/ --- Review request for Ambari, Jayush Luniya, Madhuvanthi Radhakrishnan, and Swapan Shridhar. Bugs: AMBARI-22244 https://issues.apache.org/jira/browse/AMBARI-22244 Repository: ambari Description --- Replaced `serviceName` with `serviceType` in `getService` and `getComponent` calls where found to be necessary. Diffs - ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java bb298d5a5ceda4619de958c7a529101c6e4f2e3f ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 45cc97348e89cdf5bd73a5d4d6139f808bc23180 ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 2bb450bd0ac6a0da064d892e74bbdd05e2475dd6 ambari-server/src/main/java/org/apache/ambari/server/api/services/ComponentService.java 60230f6b9fe476deed40ed0f4083ec24bea742f5 ambari-server/src/main/java/org/apache/ambari/server/checks/ComponentsExistInRepoCheck.java 69492574f8756006c9bd7aab495efb269265cfb7 ambari-server/src/main/java/org/apache/ambari/server/checks/ServicesUpCheck.java 5ecce88a84bb8897788fe881f0059fcb553c776f ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java 0f6eb90803ca6b7a5fb9791294a269aaf4e45fb9 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java e12477eede7176a19b456294602f91d44a35ad61 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 28b5c28e848720191ddcdf142d52c862755eaab0 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java ca80961ab3634960e8f12cc56b876109afae33ec ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java ba5a4e78aebb49dbcc24d220c35abfd46fe3dbc0 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java f7559c6d3422f4902e28b4f205564bc0ac0aaefa ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java ba5fccc1dc1d6ace00ba0f9f36820b055a66ce3b ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 252d7726e392307e1a8a0688b8de4e95e4982e8e ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProvider.java 8c2198bc185da616200bf2701acd4cb8a31f7022 ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java 0cd55c31a90a17bcd3a8b17783156dc92b2a6a57 ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java cae330b3466982923deb388ac5323c67fffea02d ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/DatabaseChecker.java e3f4e1a005c358ef5a7801691b56c9b91c9aaa11 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/DefaultServiceCalculatedState.java 603e650894ab27d12cbe2b9c49daf299e5818d5f ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/HBaseServiceCalculatedState.java 765e6fcaaa75086d2bc37e058b6e69e1268be8c2 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/HDFSServiceCalculatedState.java 704b4b96fbe5b54a7c36c22280f27142c0a8cfe9 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/HiveServiceCalculatedState.java 5e23f0b592519746c111dd89afbda37a79c425b7 ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/OozieServiceCalculatedState.java 6c5eab4090a18dce7cea29d8d5038d5ad964fe1c ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/state/YARNServiceCalculatedState.java 2fd99e48c34605cd2e28ffdba0c2ea40fe6b49ab ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertMaintenanceModeListener.java f73d7c98ef6639fa7feaea4399b39b8b6b70a7e0 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertServiceStateListener.java 053fb5417ddd09a98a66a1bdf45ca0e569b5ed4e ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java 4846eb1d68049965615ab4366adf32c5e9b1f4be ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java e5298dc629262aca4472c58d7d7abd2e7a63720a ambari-server/src/main/java/org/apache/ambari/server/mpack/MpackManager.java 0fcc28565d26ee0e9459b8329a0b64fbc4c2bde1 ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java d6876d9881116711eafb95eaa0e1901a6b870
Review Request 63036: AMBARI-22234 Optimizing Ranger KMS imports
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63036/
---
Review request for Ambari, Gautam Borad and Jonathan Hurley.
Bugs: AMBARI-22234
https://issues.apache.org/jira/browse/AMBARI-22234
Repository: ambari
Description
---
Updating Ranger Kms script file and using proper methods for importing modules.
Diffs
-
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
fcf2478
ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
4c313c4
Diff: https://reviews.apache.org/r/63036/diff/1/
Testing
---
Running tests for stack:None service:RANGER_KMS
2017-10-13 17:44:18,232 - Stack Feature Version Info: Cluster Stack=2.5,
Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
2017-10-13 17:44:18,255 - Using hadoop conf dir:
/usr/hdp/2.5.0.0-777/hadoop/conf
test_db_flavor_0_5_0_2_3 (test_db_flavor_config.TestDbFlavorConfig) ...
2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
default name mysql-connector-java.jar in ranger kms lib dir. So, db connection
check can fail. Please run 'ambari-server setup --jdbc-db={db_name}
--jdbc-driver={path_to_jdbc} on server host.'
2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
default name mysql-connector-java.jar in ranger kms lib dir. So, db connection
check can fail. Please run 'ambari-server setup --jdbc-db={db_name}
--jdbc-driver={path_to_jdbc} on server host.'
ok
2017-10-13 17:44:18,275 - Stack Feature Version Info: Cluster Stack=2.5,
Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
2017-10-13 17:44:18,298 - Using hadoop conf dir:
/usr/hdp/2.5.0.0-777/hadoop/conf
test_db_flavor_1_0_0_3_0 (test_db_flavor_config.TestDbFlavorConfig) ...
2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
default name mysql-connector-java.jar in ranger kms lib dir. So, db connection
check can fail. Please run 'ambari-server setup --jdbc-db={db_name}
--jdbc-driver={path_to_jdbc} on server host.'
2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
default name mysql-connector-java.jar in ranger kms lib dir. So, db connection
check can fail. Please run 'ambari-server setup --jdbc-db={db_name}
--jdbc-driver={path_to_jdbc} on server host.'
ok
2017-10-13 17:44:18,314 - Stack Feature Version Info: Cluster Stack=2.5,
Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
2017-10-13 17:44:18,337 - Using hadoop conf dir:
/usr/hdp/2.5.0.0-777/hadoop/conf
test_unsupported_db_flavor_0_5_0_2_3 (test_db_flavor_config.TestDbFlavorConfig)
... ok
2017-10-13 17:44:18,344 - Stack Feature Version Info: Cluster Stack=2.5,
Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
2017-10-13 17:44:18,367 - Using hadoop conf dir:
/usr/hdp/2.5.0.0-777/hadoop/conf
test_unsupported_db_flavor_1_0_0_3_0 (test_db_flavor_config.TestDbFlavorConfig)
... ok
--
Ran 4 tests in 0.147s
OK
Thanks,
Mugdha Varadkar
Re: Review Request 63032: YARN Service Checks Fails Because of Old hadoop-client Classpath Entry
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63032/#review188151
---
Ship it!
Ship It!
- Dmitro Lisnichenko
On Oct. 16, 2017, 7:22 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63032/
> ---
>
> (Updated Oct. 16, 2017, 7:22 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22245
> https://issues.apache.org/jira/browse/AMBARI-22245
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> - Install basic YARN on HDP 2.6.0.0 (no Tez)
> - Upgrade just YARN to HDP 2.6.0.2
>
> Service checks will fail with the following:
> ```
> 17/10/12 17:55:20 FATAL distributedshell.ApplicationMaster: Error running
> ApplicationMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.io.retry.RetryPolicies.retryForeverWithFixedSleep(JLjava/util/concurrent/TimeUnit;)Lorg/apache/hadoop/io/retry/RetryPolicy;
> at
> org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:280)
> ```
>
> - All YARN daemons and clients are reporting 2.6.0.2
> - All YARN daemons have loaded JARs for 2.6.0.2
>
> What is happening here is that the applications being run on YARN are picking
> up the older hadoop-common JAR file. The method
> {{retryForeverWithFixedSleep}} did not exist in HDP 2.6.0.0.
>
> It is picking up the older JARs for running applications because of the
> {{yarn-site.xml}} property:
> ```
> yarn-site.xml- yarn.application.classpath
> yarn-site.xml:
> /etc/hadoop/conf,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*,/usr/hdp/current/ext/hadoop/*
> ```
>
> Ambari should be parameterizing the {{/usr/hdp/current/hadoop-client}} paths
> here.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> f21719261a
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/status_params.py
> c2e9d92a04
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
> d0b4bb1f0f
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 5dbcf2da1b
>
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
> b7548eab30
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
> d3a096a4fa
>
>
> Diff: https://reviews.apache.org/r/63032/diff/1/
>
>
> Testing
> ---
>
> Manual
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63032: YARN Service Checks Fails Because of Old hadoop-client Classpath Entry
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63032/#review188152
---
Ship it!
Ship It!
- Dmytro Grinenko
On Oct. 16, 2017, 4:22 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63032/
> ---
>
> (Updated Oct. 16, 2017, 4:22 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22245
> https://issues.apache.org/jira/browse/AMBARI-22245
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> - Install basic YARN on HDP 2.6.0.0 (no Tez)
> - Upgrade just YARN to HDP 2.6.0.2
>
> Service checks will fail with the following:
> ```
> 17/10/12 17:55:20 FATAL distributedshell.ApplicationMaster: Error running
> ApplicationMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.io.retry.RetryPolicies.retryForeverWithFixedSleep(JLjava/util/concurrent/TimeUnit;)Lorg/apache/hadoop/io/retry/RetryPolicy;
> at
> org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:280)
> ```
>
> - All YARN daemons and clients are reporting 2.6.0.2
> - All YARN daemons have loaded JARs for 2.6.0.2
>
> What is happening here is that the applications being run on YARN are picking
> up the older hadoop-common JAR file. The method
> {{retryForeverWithFixedSleep}} did not exist in HDP 2.6.0.0.
>
> It is picking up the older JARs for running applications because of the
> {{yarn-site.xml}} property:
> ```
> yarn-site.xml- yarn.application.classpath
> yarn-site.xml:
> /etc/hadoop/conf,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*,/usr/hdp/current/ext/hadoop/*
> ```
>
> Ambari should be parameterizing the {{/usr/hdp/current/hadoop-client}} paths
> here.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> f21719261a
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/status_params.py
> c2e9d92a04
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
> d0b4bb1f0f
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 5dbcf2da1b
>
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
> b7548eab30
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
> d3a096a4fa
>
>
> Diff: https://reviews.apache.org/r/63032/diff/1/
>
>
> Testing
> ---
>
> Manual
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63036: AMBARI-22234 Optimizing Ranger KMS imports
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63036/#review188153
---
Ship it!
Ship It!
- Jonathan Hurley
On Oct. 16, 2017, 12:54 p.m., Mugdha Varadkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63036/
> ---
>
> (Updated Oct. 16, 2017, 12:54 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Gautam Borad, and Jonathan
> Hurley.
>
>
> Bugs: AMBARI-22234
> https://issues.apache.org/jira/browse/AMBARI-22234
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Updating Ranger Kms script file and using proper methods for importing
> modules.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
> fcf2478
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
> 4c313c4
>
>
> Diff: https://reviews.apache.org/r/63036/diff/1/
>
>
> Testing
> ---
>
> Running tests for stack:None service:RANGER_KMS
> 2017-10-13 17:44:18,232 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,255 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_0_5_0_2_3 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,275 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,298 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_1_0_0_3_0 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,314 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,337 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_0_5_0_2_3
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
> 2017-10-13 17:44:18,344 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,367 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_1_0_0_3_0
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
>
> --
> Ran 4 tests in 0.147s
>
> OK
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 63036: AMBARI-22234 Optimizing Ranger KMS imports
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63036/#review188163
---
Ship it!
Ship It!
- Attila Doroszlai
On Oct. 16, 2017, 6:54 p.m., Mugdha Varadkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63036/
> ---
>
> (Updated Oct. 16, 2017, 6:54 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Gautam Borad, and Jonathan
> Hurley.
>
>
> Bugs: AMBARI-22234
> https://issues.apache.org/jira/browse/AMBARI-22234
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Updating Ranger Kms script file and using proper methods for importing
> modules.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
> fcf2478
>
> ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
> 4c313c4
>
>
> Diff: https://reviews.apache.org/r/63036/diff/1/
>
>
> Testing
> ---
>
> Running tests for stack:None service:RANGER_KMS
> 2017-10-13 17:44:18,232 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,255 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_0_5_0_2_3 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,262 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,275 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,298 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_db_flavor_1_0_0_3_0 (test_db_flavor_config.TestDbFlavorConfig) ...
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> 2017-10-13 17:44:18,305 - Error! Sorry, but we can't find jdbc driver with
> default name mysql-connector-java.jar in ranger kms lib dir. So, db
> connection check can fail. Please run 'ambari-server setup
> --jdbc-db={db_name} --jdbc-driver={path_to_jdbc} on server host.'
> ok
> 2017-10-13 17:44:18,314 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,337 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_0_5_0_2_3
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
> 2017-10-13 17:44:18,344 - Stack Feature Version Info: Cluster Stack=2.5,
> Command Stack=None, Command Version=2.5.0.0-777 -> 2.5.0.0-777
> 2017-10-13 17:44:18,367 - Using hadoop conf dir:
> /usr/hdp/2.5.0.0-777/hadoop/conf
> test_unsupported_db_flavor_1_0_0_3_0
> (test_db_flavor_config.TestDbFlavorConfig) ... ok
>
> --
> Ran 4 tests in 0.147s
>
> OK
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 63032: YARN Service Checks Fails Because of Old hadoop-client Classpath Entry
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63032/#review188175
---
Ship it!
Ship It!
- Nate Cole
On Oct. 16, 2017, 12:22 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63032/
> ---
>
> (Updated Oct. 16, 2017, 12:22 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22245
> https://issues.apache.org/jira/browse/AMBARI-22245
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> - Install basic YARN on HDP 2.6.0.0 (no Tez)
> - Upgrade just YARN to HDP 2.6.0.2
>
> Service checks will fail with the following:
> ```
> 17/10/12 17:55:20 FATAL distributedshell.ApplicationMaster: Error running
> ApplicationMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.io.retry.RetryPolicies.retryForeverWithFixedSleep(JLjava/util/concurrent/TimeUnit;)Lorg/apache/hadoop/io/retry/RetryPolicy;
> at
> org.apache.hadoop.yarn.client.RMProxy.createRetryPolicy(RMProxy.java:280)
> ```
>
> - All YARN daemons and clients are reporting 2.6.0.2
> - All YARN daemons have loaded JARs for 2.6.0.2
>
> What is happening here is that the applications being run on YARN are picking
> up the older hadoop-common JAR file. The method
> {{retryForeverWithFixedSleep}} did not exist in HDP 2.6.0.0.
>
> It is picking up the older JARs for running applications because of the
> {{yarn-site.xml}} property:
> ```
> yarn-site.xml- yarn.application.classpath
> yarn-site.xml:
> /etc/hadoop/conf,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*,/usr/hdp/current/ext/hadoop/*
> ```
>
> Ambari should be parameterizing the {{/usr/hdp/current/hadoop-client}} paths
> here.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> f21719261a
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/status_params.py
> c2e9d92a04
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
> d0b4bb1f0f
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 5dbcf2da1b
>
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
> b7548eab30
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
> d3a096a4fa
>
>
> Diff: https://reviews.apache.org/r/63032/diff/1/
>
>
> Testing
> ---
>
> Manual
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 63044: Update JWT Authentication process to work with improved user management facility
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63044/ --- Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Myroslav Papirkovskyy, Nate Cole, Robert Nettleton, and Sebastian Toader. Bugs: AMBARI-21217 https://issues.apache.org/jira/browse/AMBARI-21217 Repository: ambari Description --- Update JWT Authentication process to work with improved user management facility Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/AmbariErrorHandler.java a57effcd62 ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java 62e8b864f5 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java 0d24ef21e7 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationEventHandlerImpl.java 4cfce2a373 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilter.java b3bc4c31fe ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationProvider.java 3d20cb9ec2 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariBasicAuthenticationFilter.java 3667012ae4 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilter.java 3d35578524 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariLocalAuthenticationProvider.java dcdf4710d4 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/UserNotFoundException.java 0f2fbb6477 ambari-server/src/main/java/org/apache/ambari/server/security/authentication/jwt/AmbariJwtAuthenticationProvider.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/security/authentication/jwt/JwtAuthenticationToken.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationFilter.java 23fa1715b1 ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java 0823729905 ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariUserAuthentication.java 9445882b62 ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariUserAuthorizationFilter.java 8fbd81614a ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthentication.java 7b21ce667d ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilter.java f42df6cb24 ambari-server/src/main/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationProperties.java cb456fa6b4 ambari-server/src/main/java/org/apache/ambari/server/security/ldap/AmbariLdapDataPopulator.java 32dd6dc4b4 ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 6650f67c22 ambari-server/src/test/java/org/apache/ambari/server/security/SecurityHelperImplTest.java 4d6d5a9e73 ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AbstractAuthenticationProviderTest.java 96b4883fb2 ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariJWTAuthenticationFilterTest.java 14c103223e ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariLocalAuthenticationProviderTest.java d445c07e9a ambari-server/src/test/java/org/apache/ambari/server/security/authentication/jwt/AmbariJwtAuthenticationFilterTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java 4e080b1617 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AuthorizationHelperTest.java 41816aa5c8 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationFilterTest.java 47df030990 ambari-server/src/test/java/org/apache/ambari/server/security/authorization/jwt/JwtAuthenticationPropertiesTest.java 144d90a712 Diff: https://reviews.apache.org/r/63044/diff/1/ Testing --- Manually tested # Local test results ``` [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 28:50 min [INFO] Finished at: 2017-10-16T16:53:23-04:00 [INFO] Final Memory: 99M/1929M [INFO] ``` # Jenkins test result: PENDING Thanks, Robert Levas
Review Request 63051: Add service group dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63051/ --- Review request for Ambari, Jayush Luniya and Swapan Shridhar. Bugs: AMBARI-22249 https://issues.apache.org/jira/browse/AMBARI-22249 Repository: ambari Description --- Add table servicegroupdependencies Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java 1e2eaf4 ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupRequest.java 53c3d1e ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupResponse.java 147650c ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceGroupResourceProvider.java 2e935af ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntity.java 7f8facb ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 8fd878e ambari-server/src/main/java/org/apache/ambari/server/state/ServiceGroup.java 9850462 ambari-server/src/main/java/org/apache/ambari/server/state/ServiceGroupFactory.java 7146bdf ambari-server/src/main/java/org/apache/ambari/server/state/ServiceGroupImpl.java 735a946 ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 3bde889 ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f48bcd6 Diff: https://reviews.apache.org/r/63051/diff/1/ Testing --- Will fix tests after patch approve Thanks, Vitalyi Brodetskyi
