[jira] [Assigned] (AMBARI-12970) Fix Tez view to work with RM HA

[DIPAYAN BHOWMICK (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

DIPAYAN BHOWMICK reassigned AMBARI-12970:
-

Assignee: DIPAYAN BHOWMICK

> Fix Tez view to work with RM HA
> ---
>
> Key: AMBARI-12970
> URL: https://issues.apache.org/jira/browse/AMBARI-12970
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.0
>Reporter: DIPAYAN BHOWMICK
>Assignee: DIPAYAN BHOWMICK
> Fix For: 2.1.2
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Tez view fails when working with RM is configured in HA mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12970) Fix Tez view to work with RM HA

[DIPAYAN BHOWMICK (JIRA)]

DIPAYAN BHOWMICK created AMBARI-12970:
-

 Summary: Fix Tez view to work with RM HA
 Key: AMBARI-12970
 URL: https://issues.apache.org/jira/browse/AMBARI-12970
 Project: Ambari
  Issue Type: Bug
  Components: ambari-views
Affects Versions: 2.1.0
Reporter: DIPAYAN BHOWMICK
 Fix For: 2.1.2


Tez view fails when working with RM is configured in HA mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: AMBARI-12931.v2.patch

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, AMBARI-12931.v1.patch, 
> AMBARI-12931.v2.patch, ambari-hdfs-screenshot.png, hdfs-dfsadmin-report.log, 
> hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12971) Add Result Set Visualization

[Pallav Kulshreshtha (JIRA)]

Pallav Kulshreshtha created AMBARI-12971:


 Summary: Add Result Set Visualization
 Key: AMBARI-12971
 URL: https://issues.apache.org/jira/browse/AMBARI-12971
 Project: Ambari
  Issue Type: Improvement
  Components: ambari-views
Affects Versions: 2.1.0
Reporter: Pallav Kulshreshtha
Assignee: Pallav Kulshreshtha
 Fix For: 2.1.2


Add result set visualization to Hive View




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38047: Fix Tez view to work with RM HA

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38047/
---

Review request for Ambari, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.


Bugs: AMBARI-12970
https://issues.apache.org/jira/browse/AMBARI-12970


Repository: ambari


Description
---

The code to fetch the RM Url has been refactored to use the Service Class in 
ambari-views-utils module.
Services class modified to also return the ATS server Url the logic is mostly 
similar to fetching the RM Url excepting connecting to the RM server(ATS server 
in this case)


Diffs
-

  contrib/views/tez/pom.xml c7b9776 
  
contrib/views/tez/src/main/java/org/apache/ambari/view/tez/ViewControllerImpl.java
 df6b468 
  contrib/views/tez/src/main/resources/view.xml 91196fa 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/Services.java
 120e377 
  
contrib/views/utils/src/test/java/org/apache/ambari/view/utils/ambari/ServicesTest.java
 1950df8 

Diff: https://reviews.apache.org/r/38047/diff/


Testing
---

Test case added for the ATS url. Ran the unit test cases. Locally tested in VMs.


Thanks,

DIPAYAN BHOWMICK

[jira] [Updated] (AMBARI-12970) Fix Tez view to work with RM HA

[DIPAYAN BHOWMICK (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

DIPAYAN BHOWMICK updated AMBARI-12970:
--
Attachment: AMBARI-12970_branch-2.1.patch

> Fix Tez view to work with RM HA
> ---
>
> Key: AMBARI-12970
> URL: https://issues.apache.org/jira/browse/AMBARI-12970
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.0
>Reporter: DIPAYAN BHOWMICK
>Assignee: DIPAYAN BHOWMICK
> Fix For: 2.1.2
>
> Attachments: AMBARI-12970_branch-2.1.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Tez view fails when working with RM is configured in HA mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: (was: AMBARI-12931.v1.patch)

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, AMBARI-12931.v2.patch, 
> ambari-hdfs-screenshot.png, hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: (was: AMBARI-12931.patch)

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.v2.patch, ambari-hdfs-screenshot.png, 
> hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12972) Ambari uses port 8080 as default port for hbase.rest.port for knox topology

[Pradeep Bhadani (JIRA)]

Pradeep Bhadani created AMBARI-12972:


 Summary: Ambari uses port 8080 as default port for hbase.rest.port 
for knox topology
 Key: AMBARI-12972
 URL: https://issues.apache.org/jira/browse/AMBARI-12972
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Reporter: Pradeep Bhadani


By default , Ambari use the port 8080 for hbase.rest.port in knox topology XML 
file. But port 8080 is used by ambari itself (if ambari server and knox are on 
same machine) which leads to conflicts and user cannot access hbase REST via 
knox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38048: Add Result Set Visualization

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38048/
---

Review request for Ambari, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.


Bugs: AMBARI-12971
https://issues.apache.org/jira/browse/AMBARI-12971


Repository: ambari


Description
---

- Added code for including visualization feature for HIVE view.
- Implemented code for enabling the visualization tab only after select query 
gives result successfully.


Diffs
-

  ambari-web/app/views/main/views/details.js e90cb20 
  contrib/views/hive/pom.xml dc0d5f6 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/browser/HiveBrowserService.java
 a0d44f5 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/jobs/JobService.java
 526f13f 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/jobs/ResultsPaginationController.java
 18152ad 
  contrib/views/hive/src/main/resources/ui/hive-web/.jshintrc c1fe863 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/components/visualization-tabs-widget.js
 PRE-CREATION 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/query-tabs.js 
4f5176c 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/visualization-ui.js
 PRE-CREATION 
  contrib/views/hive/src/main/resources/ui/hive-web/app/initializers/i18n.js 
1ce3d9d 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/components/visualization-tabs-widget.hbs
 PRE-CREATION 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/query-tabs.hbs 
bea77ba 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/visualization-ui.hbs
 PRE-CREATION 
  contrib/views/hive/src/main/resources/ui/hive-web/app/utils/constants.js 
e4e445a 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/views/visualization-ui.js 
PRE-CREATION 
  contrib/views/hive/src/main/resources/ui/hive-web/bower.json 37ea901 

Diff: https://reviews.apache.org/r/38048/diff/


Testing
---

Tested manually with different datasets of different size. 
Tested switching to visualization tab with multiple SQL tabs having different 
queries. 
Tested switching between the two visualization tabs multiple times.
Tested with upto 3 rows in the resultset with all possible combinations of 
scale and chart types.
Tested with Google Chrome, Safari, Firefox and Internet Explorer 10.
Tested end to end with Ambari Web and Hive View.

Here is the build log:

---
 T E S T S
---
Running org.apache.ambari.view.hive.backgroundjobs.BackgroundJobControllerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.372 sec
Running org.apache.ambari.view.hive.PropertyValidatorTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 sec
Running org.apache.ambari.view.hive.resources.files.FileServiceTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.515 sec
Running org.apache.ambari.view.hive.resources.jobs.AggregatorTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.ambari.view.hive.resources.jobs.ATSParserTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.113 sec
Running org.apache.ambari.view.hive.resources.jobs.JobServiceTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.664 sec
Running org.apache.ambari.view.hive.resources.jobs.LogParserTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec
Running org.apache.ambari.view.hive.resources.resources.FileResourceServiceTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.44 sec
Running 
org.apache.ambari.view.hive.resources.savedQueries.SavedQueryResourceManagerTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 sec
Running org.apache.ambari.view.hive.resources.savedQueries.SavedQueryServiceTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.431 sec
Running org.apache.ambari.view.hive.resources.udfs.UDFServiceTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.453 sec

Results :

Tests run: 48, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- apache-rat-plugin:0.11:check (default) @ hive ---
[INFO] 51 implicit excludes (use -debug for more details).
[INFO] Exclude: .git/
[INFO] Exclude: **/.gitignore
[INFO] Exclude: **/.gitattributes
[INFO] Exclude: .idea/
[INFO] Exclude: pass.txt
[INFO] Exclude: .DS_Store
[INFO] Exclude: .iml/
[INFO] Exclude: .classpath
[INFO] Exclude: .project
[INFO] Exclude: .settings
[INFO] Exclude: **/target/**
[INFO] Exclude: **/.gitkeep
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: **/.editorconfig
[INFO] Exclude: **/.jshintrc
[INFO] Exclude: **/.tmp/**
[INFO] Exclude: **/tmp/**
[INFO] Exclude: **/*.json
[I

[jira] [Updated] (AMBARI-12971) Add Result Set Visualization

[Pallav Kulshreshtha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pallav Kulshreshtha updated AMBARI-12971:
-
Attachment: AMBARI-12971_branch-2.1.patch

> Add Result Set Visualization
> 
>
> Key: AMBARI-12971
> URL: https://issues.apache.org/jira/browse/AMBARI-12971
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-views
>Affects Versions: 2.1.0
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.2
>
> Attachments: AMBARI-12971_branch-2.1.patch
>
>
> Add result set visualization to Hive View



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12940) SQLA: add property validation check for BP oozie/hive, about SQLA should be available only for stack 2.3+

[Vitaly Brodetskyi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-12940:
---
Attachment: (was: AMBARI-12940.patch)

> SQLA: add property validation check for BP oozie/hive, about SQLA should be 
> available only for stack 2.3+
> -
>
> Key: AMBARI-12940
> URL: https://issues.apache.org/jira/browse/AMBARI-12940
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12940.patch
>
>
> .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37953: SQLA: add property validation check for BP oozie/hive, about SQLA should be available only for stack 2.3+

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37953/
---

(Updated Вер. 2, 2015, 10:43 до полудня)


Review request for Ambari, Jayush Luniya, Myroslav Papirkovskyy, and Sumit 
Mohanty.


Repository: ambari


Description
---

.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 70d1907 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
 0b1573b 

Diff: https://reviews.apache.org/r/37953/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

[jira] [Updated] (AMBARI-12940) SQLA: add property validation check for BP oozie/hive, about SQLA should be available only for stack 2.3+

[Vitaly Brodetskyi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-12940:
---
Attachment: AMBARI-12940.patch

> SQLA: add property validation check for BP oozie/hive, about SQLA should be 
> available only for stack 2.3+
> -
>
> Key: AMBARI-12940
> URL: https://issues.apache.org/jira/browse/AMBARI-12940
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12940.patch
>
>
> .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12959) FE: Server overloaded with POST calls after changing config group

[Andrii Tkach (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727177#comment-14727177
 ] 

Andrii Tkach commented on AMBARI-12959:
---

committed to trunk and branch-2.1

> FE: Server overloaded with POST calls after changing config group
> -
>
> Key: AMBARI-12959
> URL: https://issues.apache.org/jira/browse/AMBARI-12959
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.1
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12959.patch
>
>
> After changing config group, calls for recommendations sent, and number of 
> these calls equals to number of Config groups, so 100 Config groups generate 
> 100 POST calls for recommendations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12971) Add Result Set Visualization

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727201#comment-14727201
 ] 

Hadoop QA commented on AMBARI-12971:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12753737/AMBARI-12971_branch-2.1.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3698//console

This message is automatically generated.

> Add Result Set Visualization
> 
>
> Key: AMBARI-12971
> URL: https://issues.apache.org/jira/browse/AMBARI-12971
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-views
>Affects Versions: 2.1.0
>Reporter: Pallav Kulshreshtha
>Assignee: Pallav Kulshreshtha
> Fix For: 2.1.2
>
> Attachments: AMBARI-12971_branch-2.1.patch
>
>
> Add result set visualization to Hive View



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: (was: AMBARI-12931.v2.patch)

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, ambari-hdfs-screenshot.png, 
> hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: AMBARI-12931.patch

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, ambari-hdfs-screenshot.png, 
> hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Andrew Onischuk (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727224#comment-14727224
 ] 

Andrew Onischuk commented on AMBARI-12969:
--

[~sumitmohanty], are you sure that host_sys_prepped will be in params of all 
services that calls this function with the exact name?
I would suggest moving host_sys_prepped as a parameter of copy_to_hdfs function.

> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12940) SQLA: add property validation check for BP oozie/hive, about SQLA should be available only for stack 2.3+

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727228#comment-14727228
 ] 

Hudson commented on AMBARI-12940:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3369 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3369/])
AMBARI-12940. SQLA: add property validation check for BP oozie/hive, about SQLA 
should be available only for stack 2.3+.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ac06e6d839009d802357a5fcdf45dd493d782906)
* 
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
* 
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java


> SQLA: add property validation check for BP oozie/hive, about SQLA should be 
> available only for stack 2.3+
> -
>
> Key: AMBARI-12940
> URL: https://issues.apache.org/jira/browse/AMBARI-12940
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12940.patch
>
>
> .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12940) SQLA: add property validation check for BP oozie/hive, about SQLA should be available only for stack 2.3+

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727260#comment-14727260
 ] 

Hudson commented on AMBARI-12940:
-

FAILURE: Integrated in Ambari-branch-2.1 #462 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/462/])
AMBARI-12940. SQLA: add property validation check for BP oozie/hive, about SQLA 
should be available only for stack 2.3+.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0a624f66219160d86ee7bd30a117963c22c2bfc7)
* 
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
* 
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java


> SQLA: add property validation check for BP oozie/hive, about SQLA should be 
> available only for stack 2.3+
> -
>
> Key: AMBARI-12940
> URL: https://issues.apache.org/jira/browse/AMBARI-12940
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12940.patch
>
>
> .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12973) Web Client Should Warn On Host Version Mismatch For Maintenance Mode

[Andrii Babiichuk (JIRA)]

Andrii Babiichuk created AMBARI-12973:
-

 Summary: Web Client Should Warn On Host Version Mismatch For 
Maintenance Mode
 Key: AMBARI-12973
 URL: https://issues.apache.org/jira/browse/AMBARI-12973
 Project: Ambari
  Issue Type: Bug
  Components: ambari-web
Affects Versions: 2.1.2
Reporter: Andrii Babiichuk
Assignee: Andrii Babiichuk
Priority: Critical
 Fix For: 2.1.2


After a stack upgrade, if a host is brought out of maintenance mode it will 
begin to report versions for its components which don't match the stack . The 
current behavior today will mark this host as being "out of sync" with the 
current stack. When bringing a host of out maintenance mode, the web client 
should check versions of the components on that host. If they do not match the 
cluster's current stack, a warning should be presented to the administrator.
Once the host is brought out of maintenance mode, if its component versions are 
different, the stack will be marked out-of-sync as it is today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12973) Web Client Should Warn On Host Version Mismatch For Maintenance Mode

[Andrii Babiichuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrii Babiichuk updated AMBARI-12973:
--
Attachment: AMBARI-12973.patch

> Web Client Should Warn On Host Version Mismatch For Maintenance Mode
> 
>
> Key: AMBARI-12973
> URL: https://issues.apache.org/jira/browse/AMBARI-12973
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12973.patch
>
>
> After a stack upgrade, if a host is brought out of maintenance mode it will 
> begin to report versions for its components which don't match the stack . The 
> current behavior today will mark this host as being "out of sync" with the 
> current stack. When bringing a host of out maintenance mode, the web client 
> should check versions of the components on that host. If they do not match 
> the cluster's current stack, a warning should be presented to the 
> administrator.
> Once the host is brought out of maintenance mode, if its component versions 
> are different, the stack will be marked out-of-sync as it is today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12973) Web Client Should Warn On Host Version Mismatch For Maintenance Mode

[Andrii Tkach (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727269#comment-14727269
 ] 

Andrii Tkach commented on AMBARI-12973:
---

+1 for the patch

> Web Client Should Warn On Host Version Mismatch For Maintenance Mode
> 
>
> Key: AMBARI-12973
> URL: https://issues.apache.org/jira/browse/AMBARI-12973
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12973.patch
>
>
> After a stack upgrade, if a host is brought out of maintenance mode it will 
> begin to report versions for its components which don't match the stack . The 
> current behavior today will mark this host as being "out of sync" with the 
> current stack. When bringing a host of out maintenance mode, the web client 
> should check versions of the components on that host. If they do not match 
> the cluster's current stack, a warning should be presented to the 
> administrator.
> Once the host is brought out of maintenance mode, if its component versions 
> are different, the stack will be marked out-of-sync as it is today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12974) fast-hdfs-resource fails when sticky bit is used for chmod

[Nathan Falk (JIRA)]

Nathan Falk created AMBARI-12974:


 Summary: fast-hdfs-resource fails when sticky bit is used for chmod
 Key: AMBARI-12974
 URL: https://issues.apache.org/jira/browse/AMBARI-12974
 Project: Ambari
  Issue Type: Bug
  Components: contrib
Affects Versions: 2.1.0
 Environment: x86 or power, any OS
IBM Open Platform
Reporter: Nathan Falk


IBM Open Platform version 4.1 uses the permission 01777 for Spark's event log 
directory:

{code}
[root@compute000 ~]# grep spark_eventlog_dir_mode 
/var/lib/ambari-server/resources/stacks/BigInsights/4.1/services/SPARK/package/scripts/params.py
 
spark_eventlog_dir_mode = 01777
{code}

In our case, the error is that the Spark History Server fails to start with an 
IllegalArgumentException:

{code}
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/stacks/BigInsights/4.1/services/SPARK/package/scripts/job_history_server.py",
 line 167, in 
JobHistoryServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/stacks/BigInsights/4.1/services/SPARK/package/scripts/job_history_server.py",
 line 73, in start
self.create_historyServer_directory()
  File 
"/var/lib/ambari-agent/cache/stacks/BigInsights/4.1/services/SPARK/package/scripts/job_history_server.py",
 line 120, in create_historyServer_directory
params.HdfsResource(None, action="execute")
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 157, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 152, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 118, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 396, in action_execute
self.get_hdfs_resource_executor().action_execute(self)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 117, in action_execute
logoutput=logoutput,
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 157, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 152, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 118, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 258, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 70, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 92, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'hadoop --config 
/usr/iop/current/hadoop-client/conf jar 
/var/lib/ambari-agent/lib/fast-hdfs-resource.jar 
/var/lib/ambari-agent/data/hdfs_resources.json' returned 1. WARNING: Use "yarn 
jar" to launch YARN applications.
Using filesystem uri: hdfs://localhost:8020
Creating: Resource [source=null, target=/user/spark, type=directory, 
action=create, owner=spark, group=hadoop, mode=755, recursiveChown=false, 
recursiveChmod=false, changePermissionforParents=false]
Creating: Resource [source=null, 
target=hdfs://localhost:8020/iop/apps/4.1.0.0/spark/logs/history-server, 
type=directory, action=create, owner=spark, group=hadoop, mode=1777, 
recursiveChown=false, recursiveChmod=false, changePermissionforParents=false]
Exception in thread "main" java.lang.IllegalArgumentException: 1777
at 
org.apache.hadoop.fs.permission.PermissionParser.(PermissionParser.java:60)
at org.apache.hadoop.fs.permission.UmaskParser.(UmaskParser.java:42)
at 
org.apache.hadoop.fs.permission.FsPermission.(FsPermission.java:106)
at org.apache.ambari.fast_hdfs_resource.Resource.setMode(Resource.java:217)
at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.

[jira] [Created] (AMBARI-12975) HBase metrics on Heatmaps are not available from RegionServer component

[Vitaly Brodetskyi (JIRA)]

Vitaly Brodetskyi created AMBARI-12975:
--

 Summary: HBase metrics on Heatmaps are not available from 
RegionServer component
 Key: AMBARI-12975
 URL: https://issues.apache.org/jira/browse/AMBARI-12975
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.1
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
 Fix For: 2.1.2


All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
component from API. Not a regression.
Effected metrics:
metrics/hbase/regionserver/compactionQueueSize
metrics/hbase/regionserver/memstoreSize
metrics/hbase/regionserver/readRequestsCount
metrics/hbase/regionserver/writeRequestsCount
metrics/hbase/regionserver/regions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12975) HBase metrics on Heatmaps are not available from RegionServer component

[Vitaly Brodetskyi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-12975:
---
Attachment: AMBARI-12975.patch

> HBase metrics on Heatmaps are not available from RegionServer component
> ---
>
> Key: AMBARI-12975
> URL: https://issues.apache.org/jira/browse/AMBARI-12975
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12975.patch
>
>
> All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
> component from API. Not a regression.
> Effected metrics:
> metrics/hbase/regionserver/compactionQueueSize
> metrics/hbase/regionserver/memstoreSize
> metrics/hbase/regionserver/readRequestsCount
> metrics/hbase/regionserver/writeRequestsCount
> metrics/hbase/regionserver/regions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38052: HBase metrics on Heatmaps are not available from RegionServer component

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38052/
---

Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro Sen.


Bugs: AMBARI-12975
https://issues.apache.org/jira/browse/AMBARI-12975


Repository: ambari


Description
---

All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
component from API. Not a regression.
Effected metrics:
metrics/hbase/regionserver/compactionQueueSize
metrics/hbase/regionserver/memstoreSize
metrics/hbase/regionserver/readRequestsCount
metrics/hbase/regionserver/writeRequestsCount
metrics/hbase/regionserver/regions


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json 
7f9cc16 

Diff: https://reviews.apache.org/r/38052/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 38052: HBase metrics on Heatmaps are not available from RegionServer component

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38052/#review97439
---

Ship it!


Ship It!

- Dmitro Lisnichenko


On Sept. 2, 2015, 1:15 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38052/
> ---
> 
> (Updated Sept. 2, 2015, 1:15 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro 
> Sen.
> 
> 
> Bugs: AMBARI-12975
> https://issues.apache.org/jira/browse/AMBARI-12975
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
> component from API. Not a regression.
> Effected metrics:
> metrics/hbase/regionserver/compactionQueueSize
> metrics/hbase/regionserver/memstoreSize
> metrics/hbase/regionserver/readRequestsCount
> metrics/hbase/regionserver/writeRequestsCount
> metrics/hbase/regionserver/regions
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json 
> 7f9cc16 
> 
> Diff: https://reviews.apache.org/r/38052/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Created] (AMBARI-12976) DDL create script for SQLA is outdated

[Myroslav Papirkovskyy (JIRA)]

Myroslav Papirkovskyy created AMBARI-12976:
--

 Summary: DDL create script for SQLA is outdated
 Key: AMBARI-12976
 URL: https://issues.apache.org/jira/browse/AMBARI-12976
 Project: Ambari
  Issue Type: Bug
  Components: ambari-server
Affects Versions: 2.1.2
Reporter: Myroslav Papirkovskyy
Assignee: Myroslav Papirkovskyy
Priority: Blocker
 Fix For: 2.1.2


Database init script for SQL Anywhere should match last version of entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 36587: Ambari server deadlock causing cluster create to be stuck - CANCELLED PATCH PENDING FURTHER ANALYSIS

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36587/#review97440
---


Can this review be closed? Seems to have gone stale.

- Jonathan Hurley


On July 19, 2015, 5:59 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36587/
> ---
> 
> (Updated July 19, 2015, 5:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Mahadev 
> Konar, Myroslav Papirkovskyy, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-12453
> https://issues.apache.org/jira/browse/AMBARI-12453
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The high level picture seems to be: Requests made from the UI for host 
> metrics trying to figure out the actual metrics service and the code that 
> updates in-memory maps which hold information of where that service is and 
> what ports to use to connect to it etc. These property maps are update by 
> Observers on important events like Cluster/Service/Host CRUD by resetting a 
> volatile variable.
> 
> The contention occurs due the thread that actually enters the monitor 
> protecting the volatile var and asks for another lock on the cluster which is 
> held by some other thread which also needs a value from the in-memory maps 
> and waits on the object monitor that it cannot enter.
> 
> Note: Web based deployments get away because not a lot of CRUD ops happen in 
> parallel to Reads like the use case of monitoring the Blueprint deploy as the 
> cluster is being provisioned.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  380a0fe 
> 
> Diff: https://reviews.apache.org/r/36587/diff/
> 
> 
> Testing
> ---
> 
> All unit test passed.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: (was: AMBARI-12931.patch)

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: ambari-hdfs-screenshot.png, hdfs-dfsadmin-report.log, 
> hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: AMBARI-12931.patch

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, ambari-hdfs-screenshot.png, 
> hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37690: Adding host via blueprint fails on secure cluster

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37690/#review97442
---

Ship it!


Two minor changes, otherwise looks great.


ambari-server/src/main/java/org/apache/ambari/server/security/encryption/InMemoryCredentialStoreService.java
 (line 70)


As Ambari becomes more asynchronous, it's probably a good idea to created 
threads/thread pools with names that can be easiliy identified during a thread 
dump. I think you can pass in a threadfactory here so that this cache cleaning 
thread can be named.



ambari-server/src/main/java/org/apache/ambari/server/security/encryption/InMemoryCredentialStoreService.java
 (lines 75 - 76)


Instead of String.format(), you can just use {} here instead; it can 
prevent runtime formatting exceptions.


- Jonathan Hurley


On Sept. 1, 2015, 8:27 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37690/
> ---
> 
> (Updated Sept. 1, 2015, 8:27 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Larry McCay, Robert Nettleton, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-12772
> https://issues.apache.org/jira/browse/AMBARI-12772
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> #STR
> Install cluster via blueprints
> Enable Kerberos security
> Add host via blueprints
> 
> #Result
> Adding hosts freeze forever
> In ambari-server.log:
> ```
> The KDC administrator credentials must be set in session by updating the 
> relevant Cluster resource.This may be done by issuing a PUT to the 
> api/v1/clusters/(cluster name) API entry point with the following payload:
> {
>   "session_attributes" : {
> "kerberos_admin" : {"principal" : "(PRINCIPAL)", "password" : 
> "(PASSWORD)"}
>   }
> ```
> #Cause
> This is caused because the KDC administrative credentials are not available 
> when needed during the add host process.  If set in the HTTP session, the 
> credentials are not accessible since the Kerberos logic is executed outside 
> the scope of that HTTP session.  
> 
> #Solution
> Store the KDC credentials to a _more secure_ global credential store that is 
> accessible no matter what the context is.  This storage facility is in-memory 
> and has a retention period of 90 minutes.  This solution refactors the 
> current CredentialStoreService and MasterKeyService classes to allow for 
> file-based and in-memory implementations. It also paves the way for future 
> changes to allow for the KDC administrative credentials to be persisted 
> indefinitely.
> 
> *Note:* This patch is rather large due to refactoring the 
> CredentialStoreService and releated classes in an effort to make way for 
> future features related to storing sensitive data.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  6d98c01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  cb9e6ca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  708d267 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java
>  8351a99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreService.java
>  8ea7ca2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceImpl.java
>  d93faec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/FileBasedCredentialStoreService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/InMemoryCredentialStoreService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyServiceImpl.java
>  219c14b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosCredential.java
>  19997e7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java
>  425aa06 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
>  389f1b8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
>  d3e3fa4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  2a1ac3c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  5d84fbc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/encryption/CredentialProviderTest.java
>  5

[jira] [Resolved] (AMBARI-12976) DDL create script for SQLA is outdated

[Myroslav Papirkovskyy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Myroslav Papirkovskyy resolved AMBARI-12976.

Resolution: Fixed

Pushed to trunk and branch-2.1

> DDL create script for SQLA is outdated
> --
>
> Key: AMBARI-12976
> URL: https://issues.apache.org/jira/browse/AMBARI-12976
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Myroslav Papirkovskyy
>Assignee: Myroslav Papirkovskyy
>Priority: Blocker
> Fix For: 2.1.2
>
>
> Database init script for SQL Anywhere should match last version of entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37946: Stop-and-Start Upgrade: If using HDP 2.x, can only register repo for HDP 2.y since need an upgrade pack to support it

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37946/#review97443
---


I'm a bit confused as to why we need to verify that creation of a repository 
needs to have an upgrade pack. For things like patch upgrades, there won't be 
an upgrade pack, but a new repository will be created.


ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 (line 129)


Should not need this in this resource provider. (See comment below)



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 (lines 158 - 163)


Indentation fix.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 (lines 359 - 360)


StackStack?



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 (lines 423 - 425)


Why iterate over all clusters? You have the cluster name from the request.


- Jonathan Hurley


On Aug. 31, 2015, 9:54 a.m., Dmytro Grinenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37946/
> ---
> 
> (Updated Aug. 31, 2015, 9:54 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12934
> https://issues.apache.org/jira/browse/AMBARI-12934
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari 2.2.0, we'll need to support Stop-and-Start Upgrade from HDP 2.x -> 
> 2.y.
> This means that if the user is still on HDP 2.x, the only upgrade pack 
> possible will be Stop-and-Start Upgrade to HDP 2.y, so it doesn't make sense 
> to allow the user to register a repo for any version except HDP 2.y.
> During repo_version creation, we need to ensure that the version being 
> installed has an upgrade pack that will allow upgrading to it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
>  442bcb2 
> 
> Diff: https://reviews.apache.org/r/37946/diff/
> 
> 
> Testing
> ---
> 
> tests passed
> 
> 
> Thanks,
> 
> Dmytro Grinenko
> 
>

Re: Review Request 37984: Stop-and-Start Upgrade: DB Schema Changes

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37984/#review97437
---



ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 (line 1192)


This doesn't make sense - you can still keep a name/UP map.  Every time you 
use it in this patch you end up iterating it.  Cheaper to keep it in a map and 
fewer changes overall.



ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java
 (line 78)


We do this already, so no need for an extra method that gets called.  I 
prefer method 1, since changing which checks run would be an xml change, not 
requiring new code.



ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
 (line 246)


Is UpgradeType a new resource, or just some value passed in to the 
UpgradeResourceProvider?  Shouldn't need this change.


- Nate Cole


On Sept. 1, 2015, 11:54 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37984/
> ---
> 
> (Updated Sept. 1, 2015, 11:54 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12699
> https://issues.apache.org/jira/browse/AMBARI-12699
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Make required database schema changes such as moving the upgrade_pack column 
> from the repo_version to the upgrade table.
> Also, added upgrade_type column to the upgrade_table.
> 
> In the process, I changed the UpgradePack class so that it contains a name, 
> and changed several methods that expected Map to 
> Collection
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
>  ef93337 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java
>  493042f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/PrereqCheckRequest.java
>  f8c5316 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java
>  6344aa2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  c394498 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  1b208fb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RepositoryVersionDAO.java
>  4ac1314 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java 
> bc0652c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  0fb2f10 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  802ea03 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java
>  17ee22c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  63f015b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
>  77e2e93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  4eb7a80 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 265e42e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0053837 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 30b669d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4f7569c 
>   ambari-server/src/ma

[jira] [Updated] (AMBARI-12581) Host check/clean-up logic should clean up new packages and folder names

[Dmitry Lysnichenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-12581:

Attachment: AMBARI-12581.branch2.1.2.patch

> Host check/clean-up logic should clean up new packages and folder names
> ---
>
> Key: AMBARI-12581
> URL: https://issues.apache.org/jira/browse/AMBARI-12581
> Project: Ambari
>  Issue Type: Bug
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.1.2
>
> Attachments: AMBARI-12581.2.patch, AMBARI-12581.branch2.1.2.patch, 
> AMBARI-12581.patch
>
>
> Few HDP-2.3 installation ran into problem when install performed on VMs that 
> are not clean. One of the reported problem is:
> {code}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 38, in 
> AfterInstallHook().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 35, in hook
> link_configs(self.stroutfile)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 91, in link_configs
> _link_configs(k, json_version, v)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 156, in _link_configs
> conf_select.select("HDP", package, version)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
>  line 241, in select
> shell.checked_call(get_cmd("set-conf-dir", package, version), 
> logoutput=False, quiet=False, sudo=True)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'conf-select 
> set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
> 0' returned 1. ranger-kms not installed or incorrect package name
> {code}
> AMBARI-12515 is looking into making conf-select call more robust but 
> host-clean up script (along with host checkup) need to be modified to handle 
> the new folders introduced under "/etc" as well as "/usr/hdp/current" or 
> "/usr/hdp/version".
> Suggestion from [~ncole] regarding cleaning up conf dirs.
> {code}
> cd /etc
> remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
> spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
> ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
> {code}
> The host check script should be modified to include these folders in the 
> report so that the cleanup script can clean it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37984: Stop-and-Start Upgrade: DB Schema Changes

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37984/#review97446
---



ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
 (lines 83 - 90)


Why did you move away from maps here?



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java 
(line 56)


Named query instead?



ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java
 (line 34)


I'd prefer NON_ROLLING for readability


- Jonathan Hurley


On Sept. 1, 2015, 11:54 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37984/
> ---
> 
> (Updated Sept. 1, 2015, 11:54 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12699
> https://issues.apache.org/jira/browse/AMBARI-12699
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Make required database schema changes such as moving the upgrade_pack column 
> from the repo_version to the upgrade table.
> Also, added upgrade_type column to the upgrade_table.
> 
> In the process, I changed the UpgradePack class so that it contains a name, 
> and changed several methods that expected Map to 
> Collection
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
>  ef93337 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java
>  493042f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/PrereqCheckRequest.java
>  f8c5316 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java
>  6344aa2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  c394498 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  1b208fb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RepositoryVersionDAO.java
>  4ac1314 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java 
> bc0652c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  0fb2f10 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  802ea03 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java
>  17ee22c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  63f015b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
>  77e2e93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  4eb7a80 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 265e42e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0053837 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 30b669d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4f7569c 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 81d0e6f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
>  bf237c6 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 9b7848f 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
> 25df73a 
>   ambari-server/src/main/resourc

Review Request 38054: Host check/clean-up logic should clean up new packages and folder names

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38054/
---

Review request for Ambari and Andrew Onischuk.


Bugs: AMBARI-12581
https://issues.apache.org/jira/browse/AMBARI-12581


Repository: ambari


Description
---

Few HDP-2.3 installation ran into problem when install performed on VMs that 
are not clean. One of the reported problem is:
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
 line 38, in 
AfterInstallHook().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
 line 35, in hook
link_configs(self.stroutfile)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
 line 91, in link_configs
_link_configs(k, json_version, v)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
 line 156, in _link_configs
conf_select.select("HDP", package, version)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
 line 241, in select
shell.checked_call(get_cmd("set-conf-dir", package, version), 
logoutput=False, quiet=False, sudo=True)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 70, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 92, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'conf-select 
set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
0' returned 1. ranger-kms not installed or incorrect package name
AMBARI-12515 is looking into making conf-select call more robust but host-clean 
up script (along with host checkup) need to be modified to handle the new 
folders introduced under "/etc" as well as "/usr/hdp/current" or 
"/usr/hdp/version".
Suggestion from Nate Cole regarding cleaning up conf dirs.
cd /etc
remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
The host check script should be modified to include these folders in the report 
so that the cleanup script can clean it.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
a4b898d 
  ambari-agent/src/main/python/ambari_agent/HostInfo.py b979242 
  ambari-server/src/main/resources/custom_actions/scripts/check_host.py 1e54180 

Diff: https://reviews.apache.org/r/38054/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [2.008s]
[INFO] Ambari Metrics Common . SUCCESS [0.904s]
[INFO] Ambari Server . SUCCESS [51.725s]
[INFO] Ambari Agent .. SUCCESS [9.076s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:07.935s
[INFO] Finished at: Wed Sep 02 16:47:43 EEST 2015
[INFO] Final Memory: 78M/1002M
[INFO] 


Thanks,

Dmitro Lisnichenko

Re: Review Request 38054: Host check/clean-up logic should clean up new packages and folder names

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38054/
---

(Updated Sept. 2, 2015, 1:49 p.m.)


Review request for Ambari and Andrew Onischuk.


Bugs: AMBARI-12581
https://issues.apache.org/jira/browse/AMBARI-12581


Repository: ambari


Description
---

Few HDP-2.3 installation ran into problem when install performed on VMs that 
are not clean. One of the reported problem is:
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
 line 38, in 
AfterInstallHook().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
 line 35, in hook
link_configs(self.stroutfile)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
 line 91, in link_configs
_link_configs(k, json_version, v)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
 line 156, in _link_configs
conf_select.select("HDP", package, version)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
 line 241, in select
shell.checked_call(get_cmd("set-conf-dir", package, version), 
logoutput=False, quiet=False, sudo=True)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 70, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 92, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'conf-select 
set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
0' returned 1. ranger-kms not installed or incorrect package name
AMBARI-12515 is looking into making conf-select call more robust but host-clean 
up script (along with host checkup) need to be modified to handle the new 
folders introduced under "/etc" as well as "/usr/hdp/current" or 
"/usr/hdp/version".
Suggestion from Nate Cole regarding cleaning up conf dirs.
cd /etc
remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
The host check script should be modified to include these folders in the report 
so that the cleanup script can clean it.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
a4b898d 
  ambari-agent/src/main/python/ambari_agent/HostInfo.py b979242 
  ambari-server/src/main/resources/custom_actions/scripts/check_host.py 1e54180 

Diff: https://reviews.apache.org/r/38054/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Views .. SUCCESS [2.008s]
[INFO] Ambari Metrics Common . SUCCESS [0.904s]
[INFO] Ambari Server . SUCCESS [51.725s]
[INFO] Ambari Agent .. SUCCESS [9.076s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:07.935s
[INFO] Finished at: Wed Sep 02 16:47:43 EEST 2015
[INFO] Final Memory: 78M/1002M
[INFO] 


Thanks,

Dmitro Lisnichenko

Re: Review Request 37682: [Preview] Stop-and-Start Upgrade: Move Configs out of Upgrade Pack

[Jonathan Hurley]

> On Aug. 27, 2015, 1:40 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml, 
> > line 467
> > 
> >
> > But what about cases where there a configuration done in the  
> > section?
> 
> Dmitro Lisnichenko wrote:
> How about implementing a virtual "apply all config changes for service" 
> task type to handle this use case? When executing cross-stack upgrade (like 
> 2.2->2.4), we are not going to parse and execute intermediate upgrade packs, 
> but only apply config changes for installed services. So, when executing 
> upgrade pack for target stack, "apply all config changes for service" task 
> would apply all config changes for service defined at 
> `upgrade-config-changes-*.xml` files for intermediate stacks. 
> Does this approach sound good?

But we can still parse the configurations by an ID, right? The 2.2 to 2.4 
upgrade pack won't parse intermediate packs, but it can still have 
configurations by ID from 2.2, 2.3, and 2.4. If you used a special task to 
"apply all configs", how are you going to determine which configs apply? By 
looking through all of the upgrade packs for all of the configs they reference? 
What about if there is an order issue where 2.2 sets a value to X, 2.3 checks 
value for X and sets to Y. In this case, it would never be set.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37682/#review96707
---


On Aug. 27, 2015, 11:08 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37682/
> ---
> 
> (Updated Aug. 27, 2015, 11:08 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12700
> https://issues.apache.org/jira/browse/AMBARI-12700
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The configs need to move out of the Upgrade Packs and into their own file.
> This will make it easier to maintain, and clearer since there will not be any 
> dups.
> 
> Since it is going to be a massive change, it would be great to get early 
> feedback. Code is not complete (still full of TODOs and does not even build)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  c717582 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
>  8f81b5a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 4b88aff 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradeConfigChangesDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  8361ea6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  8361ea6 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 9b7848f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java
>  93e29b5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  f7898ee 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java
>  a73775f 
>   
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade-config-changes-2.2.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37682/diff/
> 
> 
> Testing
> ---
> 
> just published preview of changes
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

[jira] [Updated] (AMBARI-12977) ambari-metrics fails to package from .deb on ubuntu 14.04

[Canan Girgin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Canan Girgin updated AMBARI-12977:
--
Attachment: patchBuildDebian.txt

> ambari-metrics fails to package from .deb on ubuntu 14.04
> -
>
> Key: AMBARI-12977
> URL: https://issues.apache.org/jira/browse/AMBARI-12977
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-metrics
>Reporter: Canan Girgin
>Priority: Minor
> Attachments: patchBuildDebian.txt
>
>
> running "mvn -B clean install package jdeb:jdeb -DskipTests 
> -Dpython.ver="python >= 2.6" -Preplaceurl" command on ambari directory, shows 
> failure caused by problems during dep packaging. 
> if want to package only ambari-metrics and run command on ambari-metrics 
> directory everything is ok.
> [INFO] Ambari Main ... SUCCESS [7.888s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.069s]
> [INFO] Ambari Web  SUCCESS [15.178s]
> [INFO] Ambari Views .. SUCCESS [2.366s]
> [INFO] Ambari Admin View . SUCCESS [24.536s]
> [INFO] ambari-metrics  SUCCESS [1.602s]
> [INFO] Ambari Metrics Common . SUCCESS [1.215s]
> [INFO] Ambari Metrics Hadoop Sink  FAILURE [3.265s]
> [INFO] Ambari Metrics Flume Sink . SKIPPED
> [INFO] Ambari Metrics Kafka Sink . SKIPPED
> [INFO] Ambari Metrics Storm Sink . SKIPPED
> [INFO] Ambari Metrics Collector .. SKIPPED
> [INFO] Ambari Metrics Monitor  SKIPPED
> [INFO] Ambari Metrics Assembly ... SKIPPED
> [INFO] Ambari Server . SKIPPED
> [INFO] Ambari Agent .. SKIPPED
> [INFO] Ambari Client . SKIPPED
> [INFO] Ambari Python Client .. SKIPPED
> [INFO] Ambari Groovy Client .. SKIPPED
> [INFO] Ambari Shell .. SKIPPED
> [INFO] Ambari Python Shell ... SKIPPED
> [INFO] Ambari Groovy Shell ... SKIPPED
> [ERROR] Failed to create debian package 
> /home/a/repo/ambariRoot/ambari-metrics/ambari-metrics-hadoop-sink/target/ambari-metrics-hadoop-sink_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambariRoot/ambari-metrics/ambari-metrics-hadoop-sink/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)
> *
> [ERROR] Failed to create debian package 
> /home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/target/ambari-metrics-flume-sink_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)
> *
> [ERROR] Failed to create debian package 
> /home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/target/ambari-metrics-flume-sink_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)
> *
> [ERROR] Failed to create debian package 
> /home/a/repo/ambari/ambari-metrics/ambari-metrics-kafka-sink/target/ambari-metrics-kafka-sink_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambari/ambari-metrics/ambari-metrics-kafka-sink/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)
> *
> [ERROR] Failed to create debian package 
> /home/a/repo/ambari/ambari-metrics/ambari-metrics-timelineservice/target/ambari-metrics-timelineservice_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambari/ambari-metrics/ambari-metrics-timelineservice/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)
> *
> [ERROR] Failed to create debian package 
> /home/a/repo/ambari/ambari-metrics/ambari-metrics-host-monitoring/target/ambari-metrics-host-monitoring_2.1.0~SNAPSHOT_all.deb
> org.vafer.jdeb.PackagingException: 
> "/home/a/repo/ambari/ambari-metrics/ambari-metrics-host-monitoring/src/main/package/deb/control"
>  is not a valid 'control' directory)
> at org.vafer.jdeb.maven.DebMaker.mak

[jira] [Created] (AMBARI-12977) ambari-metrics fails to package from .deb on ubuntu 14.04

[Canan Girgin (JIRA)]

Canan Girgin created AMBARI-12977:
-

 Summary: ambari-metrics fails to package from .deb on ubuntu 14.04
 Key: AMBARI-12977
 URL: https://issues.apache.org/jira/browse/AMBARI-12977
 Project: Ambari
  Issue Type: Bug
  Components: ambari-metrics
Reporter: Canan Girgin
Priority: Minor
 Attachments: patchBuildDebian.txt

running "mvn -B clean install package jdeb:jdeb -DskipTests 
-Dpython.ver="python >= 2.6" -Preplaceurl" command on ambari directory, shows 
failure caused by problems during dep packaging. 

if want to package only ambari-metrics and run command on ambari-metrics 
directory everything is ok.

[INFO] Ambari Main ... SUCCESS [7.888s]
[INFO] Apache Ambari Project POM . SUCCESS [0.069s]
[INFO] Ambari Web  SUCCESS [15.178s]
[INFO] Ambari Views .. SUCCESS [2.366s]
[INFO] Ambari Admin View . SUCCESS [24.536s]
[INFO] ambari-metrics  SUCCESS [1.602s]
[INFO] Ambari Metrics Common . SUCCESS [1.215s]
[INFO] Ambari Metrics Hadoop Sink  FAILURE [3.265s]
[INFO] Ambari Metrics Flume Sink . SKIPPED
[INFO] Ambari Metrics Kafka Sink . SKIPPED
[INFO] Ambari Metrics Storm Sink . SKIPPED
[INFO] Ambari Metrics Collector .. SKIPPED
[INFO] Ambari Metrics Monitor  SKIPPED
[INFO] Ambari Metrics Assembly ... SKIPPED
[INFO] Ambari Server . SKIPPED
[INFO] Ambari Agent .. SKIPPED
[INFO] Ambari Client . SKIPPED
[INFO] Ambari Python Client .. SKIPPED
[INFO] Ambari Groovy Client .. SKIPPED
[INFO] Ambari Shell .. SKIPPED
[INFO] Ambari Python Shell ... SKIPPED
[INFO] Ambari Groovy Shell ... SKIPPED


[ERROR] Failed to create debian package 
/home/a/repo/ambariRoot/ambari-metrics/ambari-metrics-hadoop-sink/target/ambari-metrics-hadoop-sink_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambariRoot/ambari-metrics/ambari-metrics-hadoop-sink/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)

*
[ERROR] Failed to create debian package 
/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/target/ambari-metrics-flume-sink_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)

*
[ERROR] Failed to create debian package 
/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/target/ambari-metrics-flume-sink_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambari/ambari-metrics/ambari-metrics-flume-sink/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)

*
[ERROR] Failed to create debian package 
/home/a/repo/ambari/ambari-metrics/ambari-metrics-kafka-sink/target/ambari-metrics-kafka-sink_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambari/ambari-metrics/ambari-metrics-kafka-sink/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)

*
[ERROR] Failed to create debian package 
/home/a/repo/ambari/ambari-metrics/ambari-metrics-timelineservice/target/ambari-metrics-timelineservice_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambari/ambari-metrics/ambari-metrics-timelineservice/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)

*
[ERROR] Failed to create debian package 
/home/a/repo/ambari/ambari-metrics/ambari-metrics-host-monitoring/target/ambari-metrics-host-monitoring_2.1.0~SNAPSHOT_all.deb
org.vafer.jdeb.PackagingException: 
"/home/a/repo/ambari/ambari-metrics/ambari-metrics-host-monitoring/src/main/package/deb/control"
 is not a valid 'control' directory)
at org.vafer.jdeb.maven.DebMaker.makeDeb(DebMaker.java:186)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Sumit Mohanty (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727373#comment-14727373
 ] 

Sumit Mohanty commented on AMBARI-12969:


Good suggestion [~aonishuk]. At this point the parameter exists but it will not 
be obvious to custom stack developers.

> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12959) FE: Server overloaded with POST calls after changing config group

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727375#comment-14727375
 ] 

Hudson commented on AMBARI-12959:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3370 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3370/])
AMBARI-12959 FE: Server overloaded with POST calls after changing config group. 
(atkach) (atkach: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=f917a88f8a9f7f5a5918e51a0bd91ff1ef0679ea)
* ambari-web/app/views/common/configs/widgets/slider_config_widget_view.js
* ambari-web/app/controllers/main/service/info/configs.js


> FE: Server overloaded with POST calls after changing config group
> -
>
> Key: AMBARI-12959
> URL: https://issues.apache.org/jira/browse/AMBARI-12959
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.1
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12959.patch
>
>
> After changing config group, calls for recommendations sent, and number of 
> these calls equals to number of Config groups, so 100 Config groups generate 
> 100 POST calls for recommendations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-12969:
---
Attachment: AMBARI-12969-II.patch

> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969-II.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38054: Host check/clean-up logic should clean up new packages and folder names

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38054/#review97449
---

Ship it!


Ship It!

- Andrew Onischuk


On Sept. 2, 2015, 1:49 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38054/
> ---
> 
> (Updated Sept. 2, 2015, 1:49 p.m.)
> 
> 
> Review request for Ambari and Andrew Onischuk.
> 
> 
> Bugs: AMBARI-12581
> https://issues.apache.org/jira/browse/AMBARI-12581
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Few HDP-2.3 installation ran into problem when install performed on VMs that 
> are not clean. One of the reported problem is:
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 38, in 
> AfterInstallHook().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 35, in hook
> link_configs(self.stroutfile)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 91, in link_configs
> _link_configs(k, json_version, v)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 156, in _link_configs
> conf_select.select("HDP", package, version)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
>  line 241, in select
> shell.checked_call(get_cmd("set-conf-dir", package, version), 
> logoutput=False, quiet=False, sudo=True)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'conf-select 
> set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
> 0' returned 1. ranger-kms not installed or incorrect package name
> AMBARI-12515 is looking into making conf-select call more robust but 
> host-clean up script (along with host checkup) need to be modified to handle 
> the new folders introduced under "/etc" as well as "/usr/hdp/current" or 
> "/usr/hdp/version".
> Suggestion from Nate Cole regarding cleaning up conf dirs.
> cd /etc
> remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
> spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
> ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
> The host check script should be modified to include these folders in the 
> report so that the cleanup script can clean it.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
> a4b898d 
>   ambari-agent/src/main/python/ambari_agent/HostInfo.py b979242 
>   ambari-server/src/main/resources/custom_actions/scripts/check_host.py 
> 1e54180 
> 
> Diff: https://reviews.apache.org/r/38054/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Views .. SUCCESS [2.008s]
> [INFO] Ambari Metrics Common . SUCCESS [0.904s]
> [INFO] Ambari Server . SUCCESS [51.725s]
> [INFO] Ambari Agent .. SUCCESS [9.076s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:07.935s
> [INFO] Finished at: Wed Sep 02 16:47:43 EEST 2015
> [INFO] Final Memory: 78M/1002M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 38055: sys_prepped clusters should not have to copy tarballs again

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38055/
---

Review request for Ambari, Andrew Onischuk and Ivan Mitic.


Bugs: AMBARI-12969
https://issues.apache.org/jira/browse/AMBARI-12969


Repository: ambari


Description
---

When deploying on a sys-prepped cluster, the DFS already contains the required 
set of files. Ambari should not try to upload the fils again.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 ad4aadc 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 6081b39 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
 3ec1747 
  
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py
 2f8da76 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
 8cdafc4 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py
 8d758e3 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/pre_upgrade.py
 7731bc7 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py
 9bd366b 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
 16e34d4 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 7ca3d78 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 6cf0f88 
  ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py 
c4bfa8e 

Diff: https://reviews.apache.org/r/38055/diff/


Testing
---

Ran unit tests and fixed the failing ones.


Thanks,

Sumit Mohanty

Re: Review Request 38055: sys_prepped clusters should not have to copy tarballs again

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38055/#review97450
---

Ship it!


Ship It!

- Andrew Onischuk


On Sept. 2, 2015, 2:07 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38055/
> ---
> 
> (Updated Sept. 2, 2015, 2:07 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Ivan Mitic.
> 
> 
> Bugs: AMBARI-12969
> https://issues.apache.org/jira/browse/AMBARI-12969
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When deploying on a sys-prepped cluster, the DFS already contains the 
> required set of files. Ambari should not try to upload the fils again.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  ad4aadc 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6081b39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
>  3ec1747 
>   
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py
>  2f8da76 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
>  8cdafc4 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py
>  8d758e3 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/pre_upgrade.py
>  7731bc7 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py
>  9bd366b 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
>  16e34d4 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 7ca3d78 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 
> 6cf0f88 
>   ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py 
> c4bfa8e 
> 
> Diff: https://reviews.apache.org/r/38055/diff/
> 
> 
> Testing
> ---
> 
> Ran unit tests and fixed the failing ones.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

[jira] [Updated] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Sumit Mohanty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Mohanty updated AMBARI-12969:
---
Attachment: (was: AMBARI-12969.patch)

> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969-II.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727401#comment-14727401
 ] 

Hadoop QA commented on AMBARI-12969:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12753774/AMBARI-12969-II.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3701//console

This message is automatically generated.

> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969-II.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12970) Fix Tez view to work with RM HA

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727400#comment-14727400
 ] 

Hadoop QA commented on AMBARI-12970:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12753720/AMBARI-12970_branch-2.1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in 
contrib/views/tez 

Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3700//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3700//console

This message is automatically generated.

> Fix Tez view to work with RM HA
> ---
>
> Key: AMBARI-12970
> URL: https://issues.apache.org/jira/browse/AMBARI-12970
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-views
>Affects Versions: 2.1.0
>Reporter: DIPAYAN BHOWMICK
>Assignee: DIPAYAN BHOWMICK
> Fix For: 2.1.2
>
> Attachments: AMBARI-12970_branch-2.1.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Tez view fails when working with RM is configured in HA mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12699) Stop-and-Start Upgrade: DB Schema Changes

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727403#comment-14727403
 ] 

Hadoop QA commented on AMBARI-12699:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12753667/AMBARI-12699.patch
  against trunk revision .

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/3702//console

This message is automatically generated.

> Stop-and-Start Upgrade: DB Schema Changes
> -
>
> Key: AMBARI-12699
> URL: https://issues.apache.org/jira/browse/AMBARI-12699
> Project: Ambari
>  Issue Type: Story
>  Components: ambari-server
>Affects Versions: 2.2.0
>Reporter: Alejandro Fernandez
>Assignee: Alejandro Fernandez
>  Labels: branch:branch-dev-stop-all-upgrade
> Fix For: 2.2.0, 2.1.2
>
> Attachments: AMBARI-12699.patch
>
>
> Make required database schema changes such as moving the upgrade_pack column 
> from the repo_version to the upgrade table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12916) Make API server hostname for views configurable

[Greg Hill (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727430#comment-14727430
 ] 

Greg Hill commented on AMBARI-12916:


I looked for the tests, but apparently we don't have any tests for this module 
that involve overriding config values.  We just mock out the method that I 
modified entirely and make sure other dependent methods work as expected.  I'll 
try to find another example of how we mock out the config file elsewhere and 
add some tests.  Input appreciated if someone can point me to a good example.

> Make API server hostname for views configurable
> ---
>
> Key: AMBARI-12916
> URL: https://issues.apache.org/jira/browse/AMBARI-12916
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server, ambari-views
>Affects Versions: 2.1.0
>Reporter: Greg Hill
>Assignee: Greg Hill
>Priority: Minor
> Attachments: AMBARI-12916.patch
>
>
> The views that need to get data from the Ambari API use the system hostname 
> to connect to the Ambari API.  The problem is that the API could be 
> configured with an SSL cert for a domain name other than the system hostname, 
> and if you connect to it using the hostname, you will get SSL validation 
> errors.  Simply adding an optional hostname to the Ambari configs that will 
> be used here would make it much easier to work in this sort of setup.
> https://github.com/apache/ambari/blob/4c73ea906d02df2c79ef76f5cf6fd2b94ea78ca6/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java#L299



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37682: [Preview] Stop-and-Start Upgrade: Move Configs out of Upgrade Pack

[Dmitro Lisnichenko]

> On Aug. 27, 2015, 5:40 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml, 
> > line 467
> > 
> >
> > But what about cases where there a configuration done in the  
> > section?
> 
> Dmitro Lisnichenko wrote:
> How about implementing a virtual "apply all config changes for service" 
> task type to handle this use case? When executing cross-stack upgrade (like 
> 2.2->2.4), we are not going to parse and execute intermediate upgrade packs, 
> but only apply config changes for installed services. So, when executing 
> upgrade pack for target stack, "apply all config changes for service" task 
> would apply all config changes for service defined at 
> `upgrade-config-changes-*.xml` files for intermediate stacks. 
> Does this approach sound good?
> 
> Jonathan Hurley wrote:
> But we can still parse the configurations by an ID, right? The 2.2 to 2.4 
> upgrade pack won't parse intermediate packs, but it can still have 
> configurations by ID from 2.2, 2.3, and 2.4. If you used a special task to 
> "apply all configs", how are you going to determine which configs apply? By 
> looking through all of the upgrade packs for all of the configs they 
> reference? What about if there is an order issue where 2.2 sets a value to X, 
> 2.3 checks value for X and sets to Y. In this case, it would never be set.

Our current vision (as discussed with Alejandro in "Re: Move out configs from 
upgrade pack - few discussion points" discussion) is that all config changes 
are referenced by ID. IDs share the same namespace of all stacks. And upgrade 
pack for 2.2->2.4 stack has to explicitly reference all config changes to be 
applied. Probably I'll implement a restriction that upgrade pack for 2.2->2.4 
can only reference config changes for 2.3 and 2.4 stacks.


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37682/#review96707
---


On Aug. 27, 2015, 3:08 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37682/
> ---
> 
> (Updated Aug. 27, 2015, 3:08 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12700
> https://issues.apache.org/jira/browse/AMBARI-12700
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The configs need to move out of the Upgrade Packs and into their own file.
> This will make it easier to maintain, and clearer since there will not be any 
> dups.
> 
> Since it is going to be a massive change, it would be great to get early 
> feedback. Code is not complete (still full of TODOs and does not even build)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  c717582 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
>  8f81b5a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 4b88aff 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradeConfigChangesDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  8361ea6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java
>  8361ea6 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 
> 9b7848f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java
>  93e29b5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  f7898ee 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java
>  a73775f 
>   
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade-config-changes-2.2.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37682/diff/
> 
> 
> Testing
> ---
> 
> just published preview of changes
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

[jira] [Commented] (AMBARI-12959) FE: Server overloaded with POST calls after changing config group

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727445#comment-14727445
 ] 

Hudson commented on AMBARI-12959:
-

SUCCESS: Integrated in Ambari-branch-2.1 #463 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/463/])
AMBARI-12959 FE: Server overloaded with POST calls after changing config group. 
(atkach) (atkach: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=56885a5934be6f95b5789ecd6349be2abc8fb7f2)
* ambari-web/app/controllers/main/service/info/configs.js
* ambari-web/app/views/common/configs/widgets/slider_config_widget_view.js


> FE: Server overloaded with POST calls after changing config group
> -
>
> Key: AMBARI-12959
> URL: https://issues.apache.org/jira/browse/AMBARI-12959
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.1
>Reporter: Andrii Tkach
>Assignee: Andrii Tkach
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12959.patch
>
>
> After changing config group, calls for recommendations sent, and number of 
> these calls equals to number of Config groups, so 100 Config groups generate 
> 100 POST calls for recommendations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (AMBARI-12581) Host check/clean-up logic should clean up new packages and folder names

[Dmitry Lysnichenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko resolved AMBARI-12581.
-
Resolution: Fixed

Committed to branch-2.1 as well

> Host check/clean-up logic should clean up new packages and folder names
> ---
>
> Key: AMBARI-12581
> URL: https://issues.apache.org/jira/browse/AMBARI-12581
> Project: Ambari
>  Issue Type: Bug
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.1.2
>
> Attachments: AMBARI-12581.2.patch, AMBARI-12581.branch2.1.2.patch, 
> AMBARI-12581.patch
>
>
> Few HDP-2.3 installation ran into problem when install performed on VMs that 
> are not clean. One of the reported problem is:
> {code}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 38, in 
> AfterInstallHook().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 35, in hook
> link_configs(self.stroutfile)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 91, in link_configs
> _link_configs(k, json_version, v)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 156, in _link_configs
> conf_select.select("HDP", package, version)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
>  line 241, in select
> shell.checked_call(get_cmd("set-conf-dir", package, version), 
> logoutput=False, quiet=False, sudo=True)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'conf-select 
> set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
> 0' returned 1. ranger-kms not installed or incorrect package name
> {code}
> AMBARI-12515 is looking into making conf-select call more robust but 
> host-clean up script (along with host checkup) need to be modified to handle 
> the new folders introduced under "/etc" as well as "/usr/hdp/current" or 
> "/usr/hdp/version".
> Suggestion from [~ncole] regarding cleaning up conf dirs.
> {code}
> cd /etc
> remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
> spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
> ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
> {code}
> The host check script should be modified to include these folders in the 
> report so that the cleanup script can clean it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ambari pull request: [AMBARI-12972] Change default port number for...

[pradeepbhadani]

GitHub user pradeepbhadani opened a pull request:

https://github.com/apache/ambari/pull/23

[AMBARI-12972] Change default port number for hbase.rest.port in KNOX 
configuration

Currently, Knox set the default port for "hbase.rest.port" as 8080 which is 
same as Ambari server port which can cause issue if ambari-server and hbase 
REST server are on same node.

Changed this port number to "16080" as most of the Hbase ports are n 16000 
range.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pradeepbhadani/ambari AMBARI-12972

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ambari/pull/23.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #23


commit e93e290bf14c10fc033f6c286e98fdce388eaaac
Author: Pradeep Bhadani 
Date:   2015-09-02T14:38:09Z

Change default port number for hbase.rest.port




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (AMBARI-12972) Ambari uses port 8080 as default port for hbase.rest.port for knox topology

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727464#comment-14727464
 ] 

ASF GitHub Bot commented on AMBARI-12972:
-

GitHub user pradeepbhadani opened a pull request:

https://github.com/apache/ambari/pull/23

[AMBARI-12972] Change default port number for hbase.rest.port in KNOX 
configuration

Currently, Knox set the default port for "hbase.rest.port" as 8080 which is 
same as Ambari server port which can cause issue if ambari-server and hbase 
REST server are on same node.

Changed this port number to "16080" as most of the Hbase ports are n 16000 
range.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pradeepbhadani/ambari AMBARI-12972

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ambari/pull/23.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #23


commit e93e290bf14c10fc033f6c286e98fdce388eaaac
Author: Pradeep Bhadani 
Date:   2015-09-02T14:38:09Z

Change default port number for hbase.rest.port




> Ambari uses port 8080 as default port for hbase.rest.port for knox topology
> ---
>
> Key: AMBARI-12972
> URL: https://issues.apache.org/jira/browse/AMBARI-12972
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Pradeep Bhadani
>
> By default , Ambari use the port 8080 for hbase.rest.port in knox topology 
> XML file. But port 8080 is used by ambari itself (if ambari server and knox 
> are on same machine) which leads to conflicts and user cannot access hbase 
> REST via knox.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12978) some atlas service properties are not properly displayed by UI

[Jonathan Maron (JIRA)]

Jonathan Maron created AMBARI-12978:
---

 Summary: some atlas service properties are not properly displayed 
by UI
 Key: AMBARI-12978
 URL: https://issues.apache.org/jira/browse/AMBARI-12978
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.1.0
Reporter: Jonathan Maron
Assignee: Jonathan Maron


there are some property values in the atlas configuration that are modified by 
the service scripts but those values are not reflected in the UI.  
Specifically, the following properties should reflect their actual value in the 
UI:

'atlas.http.authentication.kerberos.name.rules'
'atlas.server.bind.address'

This can be achieved by moving their value setting logic to the stack advisor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12979) Devdeploy:Start Services fails at enabling security

[Andrew Onischuk (JIRA)]

Andrew Onischuk created AMBARI-12979:


 Summary: Devdeploy:Start Services fails at enabling security 
 Key: AMBARI-12979
 URL: https://issues.apache.org/jira/browse/AMBARI-12979
 Project: Ambari
  Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
 Fix For: 2.1.2


Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
have failed at start services step of the security wizard.  
At the repro cluster it has failed at Knox gateway start with the below error
:




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 267, in 
KnoxGateway().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 146, in start
self.configure(env)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 63, in configure
knox()
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
 line 125, in knox
not_if=master_secret_exist,
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 154, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 152, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 118, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 258, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 
'/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
[PROTECTED]' returned 1. Master secret is already present on disk. Please be 
aware that overwriting it will require updating other security artifacts.  Use 
--force to overwrite the existing master secret.
ERROR: Invalid Command
Unrecognized option:create-master
A fatal exception has occurred. Program will exit.


Repro cluster :   
Corresponding jenkins job  


Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
/qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
tests.installer.InstallHadoop/install/) to artifacts.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12976) DDL create script for SQLA is outdated

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727519#comment-14727519
 ] 

Hudson commented on AMBARI-12976:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3371 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3371/])
AMBARI-12976. DDL create script for SQLA is outdated. (mpapirkovskyy) 
(mpapyrkovskyy: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=32d485ae169da97272c555eea9a2b3c774bd0a51)
* ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql


> DDL create script for SQLA is outdated
> --
>
> Key: AMBARI-12976
> URL: https://issues.apache.org/jira/browse/AMBARI-12976
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Myroslav Papirkovskyy
>Assignee: Myroslav Papirkovskyy
>Priority: Blocker
> Fix For: 2.1.2
>
>
> Database init script for SQL Anywhere should match last version of entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38057: Devdeploy:Start Services fails at enabling security

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38057/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-12979
https://issues.apache.org/jira/browse/AMBARI-12979


Repository: ambari


Description
---

Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
have failed at start services step of the security wizard.  
At the repro cluster it has failed at Knox gateway start with the below error
:




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 267, in 
KnoxGateway().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 218, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 146, in start
self.configure(env)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
 line 63, in configure
knox()
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
 line 125, in knox
not_if=master_secret_exist,
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 154, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 152, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 118, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 258, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 
'/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
[PROTECTED]' returned 1. Master secret is already present on disk. Please be 
aware that overwriting it will require updating other security artifacts.  Use 
--force to overwrite the existing master secret.
ERROR: Invalid Command
Unrecognized option:create-master
A fatal exception has occurred. Program will exit.


Repro cluster :   
Corresponding jenkins job  


Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
/qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
tests.installer.InstallHadoop/install/) to artifacts.


Diffs
-

  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 5917cda 
  ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 11f476e 

Diff: https://reviews.apache.org/r/38057/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

[jira] [Commented] (AMBARI-12975) HBase metrics on Heatmaps are not available from RegionServer component

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727521#comment-14727521
 ] 

Hudson commented on AMBARI-12975:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3371 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3371/])
AMBARI-12975. HBase metrics on Heatmaps are not available from RegionServer 
component.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=133535e6efdba81d8c9c8f38fb8c5253b294901f)
* ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json


> HBase metrics on Heatmaps are not available from RegionServer component
> ---
>
> Key: AMBARI-12975
> URL: https://issues.apache.org/jira/browse/AMBARI-12975
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12975.patch
>
>
> All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
> component from API. Not a regression.
> Effected metrics:
> metrics/hbase/regionserver/compactionQueueSize
> metrics/hbase/regionserver/memstoreSize
> metrics/hbase/regionserver/readRequestsCount
> metrics/hbase/regionserver/writeRequestsCount
> metrics/hbase/regionserver/regions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12973) Web Client Should Warn On Host Version Mismatch For Maintenance Mode

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727520#comment-14727520
 ] 

Hudson commented on AMBARI-12973:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3371 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3371/])
AMBARI-12973 Web Client Should Warn On Host Version Mismatch For Maintenance 
Mode. (ababiichuk) (ababiichuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=c93d2b932e78bc24c12a615ed2037e6ad186a50a)
* ambari-web/app/views/main/host.js
* ambari-web/app/controllers/main/host/details.js
* ambari-web/app/messages.js
* ambari-web/app/models/stack_version/version.js


> Web Client Should Warn On Host Version Mismatch For Maintenance Mode
> 
>
> Key: AMBARI-12973
> URL: https://issues.apache.org/jira/browse/AMBARI-12973
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12973.patch
>
>
> After a stack upgrade, if a host is brought out of maintenance mode it will 
> begin to report versions for its components which don't match the stack . The 
> current behavior today will mark this host as being "out of sync" with the 
> current stack. When bringing a host of out maintenance mode, the web client 
> should check versions of the components on that host. If they do not match 
> the cluster's current stack, a warning should be presented to the 
> administrator.
> Once the host is brought out of maintenance mode, if its component versions 
> are different, the stack will be marked out-of-sync as it is today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38057: Devdeploy:Start Services fails at enabling security

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38057/#review97463
---

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Вер. 2, 2015, 3:39 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38057/
> ---
> 
> (Updated Вер. 2, 2015, 3:39 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-12979
> https://issues.apache.org/jira/browse/AMBARI-12979
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
> have failed at start services step of the security wizard.  
> At the repro cluster it has failed at Knox gateway start with the below error
> :
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 267, in 
> KnoxGateway().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 146, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 63, in configure
> knox()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
>  line 125, in knox
> not_if=master_secret_exist,
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 154, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 152, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 118, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
> [PROTECTED]' returned 1. Master secret is already present on disk. Please be 
> aware that overwriting it will require updating other security artifacts.  
> Use --force to overwrite the existing master secret.
> ERROR: Invalid Command
> Unrecognized option:create-master
> A fatal exception has occurred. Program will exit.
> 
> 
> Repro cluster :   
> Corresponding jenkins job  
>  Setup/59014/console>
> 
> Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
> /qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
> tests.installer.InstallHadoop/install/) to artifacts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  5917cda 
>   ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 11f476e 
> 
> Diff: https://reviews.apache.org/r/38057/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

[jira] [Resolved] (AMBARI-12979) Devdeploy:Start Services fails at enabling security

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk resolved AMBARI-12979.
--
Resolution: Fixed

Committed to trunk

> Devdeploy:Start Services fails at enabling security 
> 
>
> Key: AMBARI-12979
> URL: https://issues.apache.org/jira/browse/AMBARI-12979
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.1.2
>
>
> Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
> have failed at start services step of the security wizard.  
> At the repro cluster it has failed at Knox gateway start with the below error
> :
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 267, in 
> KnoxGateway().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 146, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 63, in configure
> knox()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
>  line 125, in knox
> not_if=master_secret_exist,
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 154, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 152, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 118, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
> [PROTECTED]' returned 1. Master secret is already present on disk. Please be 
> aware that overwriting it will require updating other security artifacts.  
> Use --force to overwrite the existing master secret.
> ERROR: Invalid Command
> Unrecognized option:create-master
> A fatal exception has occurred. Program will exit.
> 
> Repro cluster :   
> Corresponding jenkins job  
>  Setup/59014/console>
> Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
> /qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
> tests.installer.InstallHadoop/install/) to artifacts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12979) Devdeploy:Start Services fails at enabling security

[Andrew Onischuk (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727537#comment-14727537
 ] 

Andrew Onischuk commented on AMBARI-12979:
--

Committed to branch-2.1

> Devdeploy:Start Services fails at enabling security 
> 
>
> Key: AMBARI-12979
> URL: https://issues.apache.org/jira/browse/AMBARI-12979
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.1.2
>
>
> Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
> have failed at start services step of the security wizard.  
> At the repro cluster it has failed at Knox gateway start with the below error
> :
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 267, in 
> KnoxGateway().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 146, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 63, in configure
> knox()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
>  line 125, in knox
> not_if=master_secret_exist,
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 154, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 152, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 118, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
> [PROTECTED]' returned 1. Master secret is already present on disk. Please be 
> aware that overwriting it will require updating other security artifacts.  
> Use --force to overwrite the existing master secret.
> ERROR: Invalid Command
> Unrecognized option:create-master
> A fatal exception has occurred. Program will exit.
> 
> Repro cluster :   
> Corresponding jenkins job  
>  Setup/59014/console>
> Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
> /qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
> tests.installer.InstallHadoop/install/) to artifacts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12018) JCE distribution should be done as part of Kerberos service configure function instead from hooks (refactoring)

[Jayush Luniya (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727555#comment-14727555
 ] 

Jayush Luniya commented on AMBARI-12018:


Committed to branch-2.1
commit 113a0b602f59f03a05851ad4f4bfcec7d8f00d57
Author: Emil Anca 
Date:   Mon Jun 22 11:22:39 2015 -0400

AMBARI-12018. JCE distribution should be done as part of Kerberos service 
configure function instead from hooks (refactoring) (Emil Anca via rlevas)

> JCE distribution should be done as part of Kerberos service configure 
> function instead from hooks (refactoring)
> ---
>
> Key: AMBARI-12018
> URL: https://issues.apache.org/jira/browse/AMBARI-12018
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.0.0
>Reporter: Emil Anca
>Assignee: Emil Anca
> Fix For: 2.1.2
>
> Attachments: AMBARI-12018_01.patch
>
>
> Ambari distributes JCE policy to a host before any install/start task is 
> executed in a secure cluster. This is done via before-any hooks.
> Now since kerberos service is introduced in the stack, JCE policy should be 
> distributed as part of the configure function of Kerberos service stack 
> definition.
> This task should involve just ambari-agent code changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12018) JCE distribution should be done as part of Kerberos service configure function instead from hooks (refactoring)

[Jayush Luniya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jayush Luniya updated AMBARI-12018:
---
Fix Version/s: (was: 2.2.0)
   2.1.2

> JCE distribution should be done as part of Kerberos service configure 
> function instead from hooks (refactoring)
> ---
>
> Key: AMBARI-12018
> URL: https://issues.apache.org/jira/browse/AMBARI-12018
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.0.0
>Reporter: Emil Anca
>Assignee: Emil Anca
> Fix For: 2.1.2
>
> Attachments: AMBARI-12018_01.patch
>
>
> Ambari distributes JCE policy to a host before any install/start task is 
> executed in a secure cluster. This is done via before-any hooks.
> Now since kerberos service is introduced in the stack, JCE policy should be 
> distributed as part of the configure function of Kerberos service stack 
> definition.
> This task should involve just ambari-agent code changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37946: Stop-and-Start Upgrade: If using HDP 2.x, can only register repo for HDP 2.y since need an upgrade pack to support it

[Alejandro Fernandez]

> On Sept. 2, 2015, 1:37 p.m., Jonathan Hurley wrote:
> > I'm a bit confused as to why we need to verify that creation of a 
> > repository needs to have an upgrade pack. For things like patch upgrades, 
> > there won't be an upgrade pack, but a new repository will be created.

This was mainly for RU and Stop-the-World, in which it doesn't make sense to 
register a repo for a version if no upgrade pack supports its.
However, you can also argue this case. Assume that the customer is on HDP 2.2, 
and HDP 2.4 is already out, and we only support upgrade packs for HDP 2.2->2.3, 
and 2.3->2.4. In this case, the user should perhaps be allowed to register both 
repos, and then perform 2 RUs, first from 2.2->2.3, and then 2.3->2.4


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37946/#review97443
---


On Aug. 31, 2015, 1:54 p.m., Dmytro Grinenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37946/
> ---
> 
> (Updated Aug. 31, 2015, 1:54 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12934
> https://issues.apache.org/jira/browse/AMBARI-12934
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari 2.2.0, we'll need to support Stop-and-Start Upgrade from HDP 2.x -> 
> 2.y.
> This means that if the user is still on HDP 2.x, the only upgrade pack 
> possible will be Stop-and-Start Upgrade to HDP 2.y, so it doesn't make sense 
> to allow the user to register a repo for any version except HDP 2.y.
> During repo_version creation, we need to ensure that the version being 
> installed has an upgrade pack that will allow upgrading to it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
>  442bcb2 
> 
> Diff: https://reviews.apache.org/r/37946/diff/
> 
> 
> Testing
> ---
> 
> tests passed
> 
> 
> Thanks,
> 
> Dmytro Grinenko
> 
>

Re: Review Request 37946: Stop-and-Start Upgrade: If using HDP 2.x, can only register repo for HDP 2.y since need an upgrade pack to support it

[Alejandro Fernandez]

> On Sept. 2, 2015, 1:37 p.m., Jonathan Hurley wrote:
> > I'm a bit confused as to why we need to verify that creation of a 
> > repository needs to have an upgrade pack. For things like patch upgrades, 
> > there won't be an upgrade pack, but a new repository will be created.
> 
> Alejandro Fernandez wrote:
> This was mainly for RU and Stop-the-World, in which it doesn't make sense 
> to register a repo for a version if no upgrade pack supports its.
> However, you can also argue this case. Assume that the customer is on HDP 
> 2.2, and HDP 2.4 is already out, and we only support upgrade packs for HDP 
> 2.2->2.3, and 2.3->2.4. In this case, the user should perhaps be allowed to 
> register both repos, and then perform 2 RUs, first from 2.2->2.3, and then 
> 2.3->2.4

I'm ok with keeping D.G.'s logic for now, at least until patch upgrades comes 
out.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37946/#review97443
---


On Aug. 31, 2015, 1:54 p.m., Dmytro Grinenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37946/
> ---
> 
> (Updated Aug. 31, 2015, 1:54 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12934
> https://issues.apache.org/jira/browse/AMBARI-12934
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In Ambari 2.2.0, we'll need to support Stop-and-Start Upgrade from HDP 2.x -> 
> 2.y.
> This means that if the user is still on HDP 2.x, the only upgrade pack 
> possible will be Stop-and-Start Upgrade to HDP 2.y, so it doesn't make sense 
> to allow the user to register a repo for any version except HDP 2.y.
> During repo_version creation, we need to ensure that the version being 
> installed has an upgrade pack that will allow upgrading to it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
>  442bcb2 
> 
> Diff: https://reviews.apache.org/r/37946/diff/
> 
> 
> Testing
> ---
> 
> tests passed
> 
> 
> Thanks,
> 
> Dmytro Grinenko
> 
>

[jira] [Updated] (AMBARI-12772) Adding host via blueprint fails on secure cluster

[Robert Levas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Levas updated AMBARI-12772:
--
Attachment: AMBARI-12772_branch-2.1_03.patch

> Adding host via blueprint fails on secure cluster
> -
>
> Key: AMBARI-12772
> URL: https://issues.apache.org/jira/browse/AMBARI-12772
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.0.0
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Critical
>  Labels: blueprints, kerberos
> Fix For: 2.1.2
>
> Attachments: AMBARI-12772_branch-2.1_01.patch, 
> AMBARI-12772_branch-2.1_03.patch, AMBARI-12772_trunk_01.patch, 
> AMBARI-12772_trunk_02.patch
>
>
> *STR*
> Install cluster via blueprints
> Enable Kerberos security
> Add host via blueprints
> *Result*
> Adding hosts freeze forever
> In ambari-server.log:
> {code}
> The KDC administrator credentials must be set in session by updating the 
> relevant Cluster resource.This may be done by issuing a PUT to the 
> api/v1/clusters/(cluster name) API entry point with the following payload:
> {
>   "session_attributes" : {
> "kerberos_admin" : {"principal" : "(PRINCIPAL)", "password" : 
> "(PASSWORD)"}
>   }
> {code}
> *Cause*
> This is caused because the KDC administrative credentials are not available 
> when needed during the add host process.  If set in the HTTP session, the 
> credentials are not accessible since the Kerberos logic is executed outside 
> the scope of that HTTP session.  
> *Solution*
> Store the KDC credentials to a _more secure_ global credential store that is 
> accessible no matter what the context is.  This storage facility is in-memory 
> and has a retention period of 90 minutes.  This solution refactors the 
> current CredentialStoreService and MasterKeyService classes to allow for 
> file-based and in-memory implementations. It also paves the way for future 
> changes to allow for the KDC administrative credentials to be persisted 
> indefinitely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37984: Stop-and-Start Upgrade: DB Schema Changes

[Alejandro Fernandez]

> On Sept. 2, 2015, 1:48 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java,
> >  lines 83-90
> > 
> >
> > Why did you move away from maps here?

I'll revert back to a hashmap.


> On Sept. 2, 2015, 1:48 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java,
> >  line 56
> > 
> >
> > Named query instead?

Will switch this to named query.


> On Sept. 2, 2015, 1:48 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java,
> >  line 34
> > 
> >
> > I'd prefer NON_ROLLING for readability

Will do.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37984/#review97446
---


On Sept. 2, 2015, 3:54 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37984/
> ---
> 
> (Updated Sept. 2, 2015, 3:54 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12699
> https://issues.apache.org/jira/browse/AMBARI-12699
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Make required database schema changes such as moving the upgrade_pack column 
> from the repo_version to the upgrade table.
> Also, added upgrade_type column to the upgrade_table.
> 
> In the process, I changed the UpgradePack class so that it contains a name, 
> and changed several methods that expected Map to 
> Collection
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
>  ef93337 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java
>  493042f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/PrereqCheckRequest.java
>  f8c5316 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java
>  6344aa2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  c394498 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  1b208fb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RepositoryVersionDAO.java
>  4ac1314 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java 
> bc0652c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  0fb2f10 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  802ea03 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java
>  17ee22c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  63f015b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
>  77e2e93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  4eb7a80 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 265e42e 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0053837 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 30b669d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4f7569c 
>   ambari-server/src/main/resources/Amba

Re: Review Request 37984: Stop-and-Start Upgrade: DB Schema Changes

[Alejandro Fernandez]

> On Sept. 2, 2015, 1:29 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java,
> >  line 246
> > 
> >
> > Is UpgradeType a new resource, or just some value passed in to the 
> > UpgradeResourceProvider?  Shouldn't need this change.

It's a new field in the Upgrade resource.


> On Sept. 2, 2015, 1:29 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java,
> >  line 78
> > 
> >
> > We do this already, so no need for an extra method that gets called.  I 
> > prefer method 1, since changing which checks run would be an xml change, 
> > not requiring new code.

I prefer method 1. I can envision in the future an RU from 2.2->2.3 having 
different checks than an RU from 2.3->2.4.
So this will need to be in the upgrade pack itself.


> On Sept. 2, 2015, 1:29 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java,
> >  line 1192
> > 
> >
> > This doesn't make sense - you can still keep a name/UP map.  Every time 
> > you use it in this patch you end up iterating it.  Cheaper to keep it in a 
> > map and fewer changes overall.

Makes sense, will revert to hashmap.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37984/#review97437
---


On Sept. 2, 2015, 3:54 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37984/
> ---
> 
> (Updated Sept. 2, 2015, 3:54 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12699
> https://issues.apache.org/jira/browse/AMBARI-12699
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Make required database schema changes such as moving the upgrade_pack column 
> from the repo_version to the upgrade table.
> Also, added upgrade_type column to the upgrade_table.
> 
> In the process, I changed the UpgradePack class so that it contains a name, 
> and changed several methods that expected Map to 
> Collection
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4afa9b0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
>  ef93337 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheck.java
>  493042f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/PrereqCheckRequest.java
>  f8c5316 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CompatibleRepositoryVersionResourceProvider.java
>  6344aa2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  c394498 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  f1fa3bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fa743be 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  1b208fb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RepositoryVersionDAO.java
>  4ac1314 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UpgradeDAO.java 
> bc0652c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  0fb2f10 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  802ea03 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  89c10c6 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 2aa89cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 3e25d01 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  79b8eb5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  2e17cf4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/UpgradeType.java
>  17ee22c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  63f015b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
>  77e2e93 
>   
> a

[jira] [Updated] (AMBARI-12772) Adding host via blueprint fails on secure cluster

[Robert Levas (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Levas updated AMBARI-12772:
--
Attachment: AMBARI-12772_trunk_03.patch

> Adding host via blueprint fails on secure cluster
> -
>
> Key: AMBARI-12772
> URL: https://issues.apache.org/jira/browse/AMBARI-12772
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.0.0
>Reporter: Robert Levas
>Assignee: Robert Levas
>Priority: Critical
>  Labels: blueprints, kerberos
> Fix For: 2.1.2
>
> Attachments: AMBARI-12772_branch-2.1_01.patch, 
> AMBARI-12772_branch-2.1_03.patch, AMBARI-12772_trunk_01.patch, 
> AMBARI-12772_trunk_02.patch, AMBARI-12772_trunk_03.patch
>
>
> *STR*
> Install cluster via blueprints
> Enable Kerberos security
> Add host via blueprints
> *Result*
> Adding hosts freeze forever
> In ambari-server.log:
> {code}
> The KDC administrator credentials must be set in session by updating the 
> relevant Cluster resource.This may be done by issuing a PUT to the 
> api/v1/clusters/(cluster name) API entry point with the following payload:
> {
>   "session_attributes" : {
> "kerberos_admin" : {"principal" : "(PRINCIPAL)", "password" : 
> "(PASSWORD)"}
>   }
> {code}
> *Cause*
> This is caused because the KDC administrative credentials are not available 
> when needed during the add host process.  If set in the HTTP session, the 
> credentials are not accessible since the Kerberos logic is executed outside 
> the scope of that HTTP session.  
> *Solution*
> Store the KDC credentials to a _more secure_ global credential store that is 
> accessible no matter what the context is.  This storage facility is in-memory 
> and has a retention period of 90 minutes.  This solution refactors the 
> current CredentialStoreService and MasterKeyService classes to allow for 
> file-based and in-memory implementations. It also paves the way for future 
> changes to allow for the KDC administrative credentials to be persisted 
> indefinitely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: (was: AMBARI-12931.patch)

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: ambari-hdfs-screenshot.png, hdfs-dfsadmin-report.log, 
> hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37690: Adding host via blueprint fails on secure cluster

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37690/
---

(Updated Sept. 2, 2015, 12:55 p.m.)


Review request for Ambari, Jonathan Hurley, Larry McCay, Robert Nettleton, and 
Sid Wagle.


Changes
---

- Added named threads to the scheduled cleanup process in the 
InMemoryCredentialStoreService
- Used logger-specific string formatter rather than String.format in 
InMemoryCredentialStoreService


Bugs: AMBARI-12772
https://issues.apache.org/jira/browse/AMBARI-12772


Repository: ambari


Description
---

#STR
Install cluster via blueprints
Enable Kerberos security
Add host via blueprints

#Result
Adding hosts freeze forever
In ambari-server.log:
```
The KDC administrator credentials must be set in session by updating the 
relevant Cluster resource.This may be done by issuing a PUT to the 
api/v1/clusters/(cluster name) API entry point with the following payload:
{
  "session_attributes" : {
"kerberos_admin" : {"principal" : "(PRINCIPAL)", "password" : "(PASSWORD)"}
  }
```
#Cause
This is caused because the KDC administrative credentials are not available 
when needed during the add host process.  If set in the HTTP session, the 
credentials are not accessible since the Kerberos logic is executed outside the 
scope of that HTTP session.  

#Solution
Store the KDC credentials to a _more secure_ global credential store that is 
accessible no matter what the context is.  This storage facility is in-memory 
and has a retention period of 90 minutes.  This solution refactors the current 
CredentialStoreService and MasterKeyService classes to allow for file-based and 
in-memory implementations. It also paves the way for future changes to allow 
for the KDC administrative credentials to be persisted indefinitely.

*Note:* This patch is rather large due to refactoring the 
CredentialStoreService and releated classes in an effort to make way for future 
features related to storing sensitive data.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 6d98c01 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
 cb9e6ca 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 708d267 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialProvider.java
 8351a99 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreService.java
 8ea7ca2 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceImpl.java
 d93faec 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/FileBasedCredentialStoreService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/InMemoryCredentialStoreService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/encryption/MasterKeyServiceImpl.java
 219c14b 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosCredential.java
 19997e7 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java
 425aa06 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
 389f1b8 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
 d3e3fa4 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 2a1ac3c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 5d84fbc 
  
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/CredentialProviderTest.java
 51f2220 
  
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/CredentialStoreServiceTest.java
 0652a52 
  
ambari-server/src/test/java/org/apache/ambari/server/security/encryption/MasterKeyServiceTest.java
 993601b 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandlerTest.java
 9ad3da6 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosCredentialTest.java
 305b122 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandlerTest.java
 44a68ae 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerActionTest.java
 8fc5325 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
 8c096b0 

Diff: https://reviews.apache.org/r/37690/diff/


Testing
---

Manually tested the following on trunk and branch-2.1:  
- backwards compatibailiy with storing and retrieving the master key and key 
store data
- adding a host on a non-kerb

[jira] [Created] (AMBARI-12980) Alerts: HDFS alert based on the value of PendingDeletionBlocks

[Vitaly Brodetskyi (JIRA)]

Vitaly Brodetskyi created AMBARI-12980:
--

 Summary: Alerts: HDFS alert based on the value of 
PendingDeletionBlocks
 Key: AMBARI-12980
 URL: https://issues.apache.org/jira/browse/AMBARI-12980
 Project: Ambari
  Issue Type: Improvement
  Components: ambari-server
Affects Versions: 2.1.1
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
 Fix For: 2.1.2


Alert Type = METRIC
Lets add a new Alert for HDFS based on PendingDeletionBlocks. This is a JMX 
data point FSNamesystem --> PendingDeletionBlocks.
This alert could potentially be designed based on NameNode Blocks Health. 
Warning and Critical both can be issued when the value exceeds 100,000. These 
thresholds should be configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AMBARI-12980) Alerts: HDFS alert based on the value of PendingDeletionBlocks

[Vitaly Brodetskyi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaly Brodetskyi updated AMBARI-12980:
---
Attachment: AMBARI-12980.patch

> Alerts: HDFS alert based on the value of PendingDeletionBlocks
> --
>
> Key: AMBARI-12980
> URL: https://issues.apache.org/jira/browse/AMBARI-12980
> Project: Ambari
>  Issue Type: Improvement
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12980.patch
>
>
> Alert Type = METRIC
> Lets add a new Alert for HDFS based on PendingDeletionBlocks. This is a JMX 
> data point FSNamesystem --> PendingDeletionBlocks.
> This alert could potentially be designed based on NameNode Blocks Health. 
> Warning and Critical both can be issued when the value exceeds 100,000. These 
> thresholds should be configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38062: Alerts: HDFS alert based on the value of PendingDeletionBlocks

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38062/
---

Review request for Ambari, Jonathan Hurley and Sumit Mohanty.


Bugs: AMBARI-12980
https://issues.apache.org/jira/browse/AMBARI-12980


Repository: ambari


Description
---

Alert Type = METRIC
Lets add a new Alert for HDFS based on PendingDeletionBlocks. This is a JMX 
data point FSNamesystem --> PendingDeletionBlocks.
This alert could potentially be designed based on NameNode Blocks Health. 
Warning and Critical both can be issued when the value exceeds 100,000. These 
thresholds should be configurable.


Diffs
-

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
477fd95 

Diff: https://reviews.apache.org/r/38062/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

[jira] [Updated] (AMBARI-12931) Ambari shows "Corrupted Blocks" but it is "Corrupted replica"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivér Szabó updated AMBARI-12931:
--
Attachment: AMBARI-12931.patch

> Ambari shows "Corrupted Blocks" but it is "Corrupted replica"
> -
>
> Key: AMBARI-12931
> URL: https://issues.apache.org/jira/browse/AMBARI-12931
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.0
> Environment: CentOS-6 
> HDP 2.1.15
> Ambari 2.1.0
>Reporter: Olivér Szabó
>Assignee: Olivér Szabó
>Priority: Minor
> Attachments: AMBARI-12931.patch, ambari-hdfs-screenshot.png, 
> hdfs-dfsadmin-report.log, hdfs-fsck-02-Aug.log
>
>
> #hdfs dfsadmin -report shows "Blocks with corrupt replica : 20 "
> &
> #hdfs fsck / shows " Corrupt blocks : 0 "
> But on the UI "Corrupted block" shows as 20 from Ambari and Summary of HDFS 
> services shows " Block 20 corrupt / 0 missing / ... "



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12975) HBase metrics on Heatmaps are not available from RegionServer component

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727650#comment-14727650
 ] 

Hudson commented on AMBARI-12975:
-

FAILURE: Integrated in Ambari-branch-2.1 #464 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/464/])
AMBARI-12975. HBase metrics on Heatmaps are not available from RegionServer 
component.(vbrodetskyi) (vbrodetskyi: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=fdd3029e757c92552f5c6ea732c011e69473ec48)
* ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json


> HBase metrics on Heatmaps are not available from RegionServer component
> ---
>
> Key: AMBARI-12975
> URL: https://issues.apache.org/jira/browse/AMBARI-12975
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.1
>Reporter: Vitaly Brodetskyi
>Assignee: Vitaly Brodetskyi
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12975.patch
>
>
> All HBase heatmap metrics(HDP 2.3) not existed anymore on the RegionServer 
> component from API. Not a regression.
> Effected metrics:
> metrics/hbase/regionserver/compactionQueueSize
> metrics/hbase/regionserver/memstoreSize
> metrics/hbase/regionserver/readRequestsCount
> metrics/hbase/regionserver/writeRequestsCount
> metrics/hbase/regionserver/regions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12976) DDL create script for SQLA is outdated

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727648#comment-14727648
 ] 

Hudson commented on AMBARI-12976:
-

FAILURE: Integrated in Ambari-branch-2.1 #464 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/464/])
AMBARI-12976. DDL create script for SQLA is outdated. (mpapirkovskyy) 
(mpapyrkovskyy: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=5164055896a553caf03f16b92c6d9c4073be9511)
* ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql


> DDL create script for SQLA is outdated
> --
>
> Key: AMBARI-12976
> URL: https://issues.apache.org/jira/browse/AMBARI-12976
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.1.2
>Reporter: Myroslav Papirkovskyy
>Assignee: Myroslav Papirkovskyy
>Priority: Blocker
> Fix For: 2.1.2
>
>
> Database init script for SQL Anywhere should match last version of entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12973) Web Client Should Warn On Host Version Mismatch For Maintenance Mode

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727649#comment-14727649
 ] 

Hudson commented on AMBARI-12973:
-

FAILURE: Integrated in Ambari-branch-2.1 #464 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/464/])
AMBARI-12973 Web Client Should Warn On Host Version Mismatch For Maintenance 
Mode. (ababiichuk) (ababiichuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=bb1473df49174838b48331a601a1cfd4a1255407)
* ambari-web/app/models/stack_version/version.js
* ambari-web/app/controllers/main/host/details.js
* ambari-web/app/messages.js
* ambari-web/app/views/main/host.js


> Web Client Should Warn On Host Version Mismatch For Maintenance Mode
> 
>
> Key: AMBARI-12973
> URL: https://issues.apache.org/jira/browse/AMBARI-12973
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-web
>Affects Versions: 2.1.2
>Reporter: Andrii Babiichuk
>Assignee: Andrii Babiichuk
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12973.patch
>
>
> After a stack upgrade, if a host is brought out of maintenance mode it will 
> begin to report versions for its components which don't match the stack . The 
> current behavior today will mark this host as being "out of sync" with the 
> current stack. When bringing a host of out maintenance mode, the web client 
> should check versions of the components on that host. If they do not match 
> the cluster's current stack, a warning should be presented to the 
> administrator.
> Once the host is brought out of maintenance mode, if its component versions 
> are different, the stack will be marked out-of-sync as it is today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12979) Devdeploy:Start Services fails at enabling security

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727679#comment-14727679
 ] 

Hudson commented on AMBARI-12979:
-

SUCCESS: Integrated in Ambari-trunk-Commit #3372 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/3372/])
AMBARI-12979. Devdeploy:Start Services fails at enabling security  (aonishuk) 
(aonishuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d2e92bbdd6b99ba26a3e5da92359df6da85586c6)
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
* ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py


> Devdeploy:Start Services fails at enabling security 
> 
>
> Key: AMBARI-12979
> URL: https://issues.apache.org/jira/browse/AMBARI-12979
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.1.2
>
>
> Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
> have failed at start services step of the security wizard.  
> At the repro cluster it has failed at Knox gateway start with the below error
> :
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 267, in 
> KnoxGateway().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 146, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 63, in configure
> knox()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
>  line 125, in knox
> not_if=master_secret_exist,
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 154, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 152, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 118, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
> [PROTECTED]' returned 1. Master secret is already present on disk. Please be 
> aware that overwriting it will require updating other security artifacts.  
> Use --force to overwrite the existing master secret.
> ERROR: Invalid Command
> Unrecognized option:create-master
> A fatal exception has occurred. Program will exit.
> 
> Repro cluster :   
> Corresponding jenkins job  
>  Setup/59014/console>
> Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
> /qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
> tests.installer.InstallHadoop/install/) to artifacts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AMBARI-12981) designate atlas.cluster.name as a property requiring input

[Jonathan Maron (JIRA)]

Jonathan Maron created AMBARI-12981:
---

 Summary: designate atlas.cluster.name as a property requiring input
 Key: AMBARI-12981
 URL: https://issues.apache.org/jira/browse/AMBARI-12981
 Project: Ambari
  Issue Type: Bug
  Components: stacks
Affects Versions: 2.1.0
Reporter: Jonathan Maron
Assignee: Jonathan Maron


That atlas.cluster.name required by the hive hook (in hive-site) needs a value 
and should not necessarily use the ambari cluster name since it is changeable.  
It should therefore be designating as requiring an input so that users are 
prompted to provide a value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38062: Alerts: HDFS alert based on the value of PendingDeletionBlocks

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38062/#review97501
---

Ship it!


Let's make sure we want this to be for all stacks since it's at the 
common-services level.


ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
(line 216)


Most of our metric alerts are 5. I have no problem with this being 2, but I 
just want to ensure it's right.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
(line 238)


Maybe run this by Jeff Sposetti: did we want to make this more human 
readable? Something like

There are {0} block(s) pending deletion.


- Jonathan Hurley


On Sept. 2, 2015, 1:04 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38062/
> ---
> 
> (Updated Sept. 2, 2015, 1:04 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-12980
> https://issues.apache.org/jira/browse/AMBARI-12980
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Alert Type = METRIC
> Lets add a new Alert for HDFS based on PendingDeletionBlocks. This is a JMX 
> data point FSNamesystem --> PendingDeletionBlocks.
> This alert could potentially be designed based on NameNode Blocks Health. 
> Warning and Critical both can be issued when the value exceeds 100,000. These 
> thresholds should be configurable.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> 477fd95 
> 
> Diff: https://reviews.apache.org/r/38062/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

[jira] [Created] (AMBARI-12982) Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk (JIRA)]

Andrew Onischuk created AMBARI-12982:


 Summary: Ambari-server 'mvn clean test' fails when Python 2.6 is 
picked up
 Key: AMBARI-12982
 URL: https://issues.apache.org/jira/browse/AMBARI-12982
 Project: Ambari
  Issue Type: Bug
Reporter: Andrew Onischuk
Assignee: Andrew Onischuk
 Fix For: 2.1.2


On my OSX system I have Python2.6 and Python2.7. I do not have any
`/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
tests fail with below message



==
ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
--
Traceback (most recent call last):
  File 
"/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
 line 997, in test_validateNonRootFs
self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
'property1', hostInfo))
AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
'assertIsNone'


It looks like `assertIsNone()` is not defined in unittest.TestCase for
Python2.6.

I started hitting this issue after AMBARI-12912





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 38063: Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38063/
---

Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-12982
https://issues.apache.org/jira/browse/AMBARI-12982


Repository: ambari


Description
---

On my OSX system I have Python2.6 and Python2.7. I do not have any
`/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
tests fail with below message



==
ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
--
Traceback (most recent call last):
  File 
"/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
 line 997, in test_validateNonRootFs
self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
'property1', hostInfo))
AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
'assertIsNone'


It looks like `assertIsNone()` is not defined in unittest.TestCase for
Python2.6.

I started hitting this issue after AMBARI-12912


Diffs
-

  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 78d25bc 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 24ee07a 
  ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py 
1b518d4 

Diff: https://reviews.apache.org/r/38063/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

[jira] [Commented] (AMBARI-12018) JCE distribution should be done as part of Kerberos service configure function instead from hooks (refactoring)

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727800#comment-14727800
 ] 

Hudson commented on AMBARI-12018:
-

SUCCESS: Integrated in Ambari-branch-2.1 #465 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/465/])
AMBARI-12018. JCE distribution should be done as part of Kerberos service 
configure function instead from hooks (refactoring) (Emil Anca via rlevas) 
(jluniya: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=113a0b602f59f03a05851ad4f4bfcec7d8f00d57)
* ambari-server/src/test/python/stacks/2.2/KERBEROS/test_kerberos_client.py
* 
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py
* 
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py
* ambari-server/src/test/python/stacks/2.0.6/hooks/before-ANY/test_before_any.py
* 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py
* 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
* 
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py
* 
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py


> JCE distribution should be done as part of Kerberos service configure 
> function instead from hooks (refactoring)
> ---
>
> Key: AMBARI-12018
> URL: https://issues.apache.org/jira/browse/AMBARI-12018
> Project: Ambari
>  Issue Type: Task
>  Components: ambari-server
>Affects Versions: 2.0.0
>Reporter: Emil Anca
>Assignee: Emil Anca
> Fix For: 2.1.2
>
> Attachments: AMBARI-12018_01.patch
>
>
> Ambari distributes JCE policy to a host before any install/start task is 
> executed in a secure cluster. This is done via before-any hooks.
> Now since kerberos service is introduced in the stack, JCE policy should be 
> distributed as part of the configure function of Kerberos service stack 
> definition.
> This task should involve just ambari-agent code changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12969) sys_prepped clusters should not have to copy tarballs again

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727802#comment-14727802
 ] 

Hudson commented on AMBARI-12969:
-

SUCCESS: Integrated in Ambari-branch-2.1 #465 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/465/])
AMBARI-12969. sys_prepped clusters should not have to copy tarballs again 
(smohanty: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=31edfe3ba168673ef20d7edaa95c7026ac84)
* 
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
* ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py
* 
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py
* 
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py
* 
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
* 
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/pre_upgrade.py
* 
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
* 
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
* ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py
* ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py
* 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
* 
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py


> sys_prepped clusters should not have to copy tarballs again
> ---
>
> Key: AMBARI-12969
> URL: https://issues.apache.org/jira/browse/AMBARI-12969
> Project: Ambari
>  Issue Type: Bug
>  Components: stacks
>Affects Versions: 2.1.1
>Reporter: Sumit Mohanty
>Assignee: Sumit Mohanty
>Priority: Critical
> Fix For: 2.1.2
>
> Attachments: AMBARI-12969-II.patch
>
>
> Ambari should not re-upload tarballs when the host/clusters are sys prepped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12581) Host check/clean-up logic should clean up new packages and folder names

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727801#comment-14727801
 ] 

Hudson commented on AMBARI-12581:
-

SUCCESS: Integrated in Ambari-branch-2.1 #465 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/465/])
AMBARI-12581. Host check/clean-up logic should clean up new packages and folder 
names (dlysnichenko) (dlysnichenko: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3e9e182d6acdd48402e80529c384357125a3e406)
* ambari-server/src/main/resources/custom_actions/scripts/check_host.py
* ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py
* ambari-agent/src/main/python/ambari_agent/HostInfo.py


> Host check/clean-up logic should clean up new packages and folder names
> ---
>
> Key: AMBARI-12581
> URL: https://issues.apache.org/jira/browse/AMBARI-12581
> Project: Ambari
>  Issue Type: Bug
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.1.2
>
> Attachments: AMBARI-12581.2.patch, AMBARI-12581.branch2.1.2.patch, 
> AMBARI-12581.patch
>
>
> Few HDP-2.3 installation ran into problem when install performed on VMs that 
> are not clean. One of the reported problem is:
> {code}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 38, in 
> AfterInstallHook().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py",
>  line 35, in hook
> link_configs(self.stroutfile)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 91, in link_configs
> _link_configs(k, json_version, v)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py",
>  line 156, in _link_configs
> conf_select.select("HDP", package, version)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py",
>  line 241, in select
> shell.checked_call(get_cmd("set-conf-dir", package, version), 
> logoutput=False, quiet=False, sudo=True)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'conf-select 
> set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 
> 0' returned 1. ranger-kms not installed or incorrect package name
> {code}
> AMBARI-12515 is looking into making conf-select call more robust but 
> host-clean up script (along with host checkup) need to be modified to handle 
> the new folders introduced under "/etc" as well as "/usr/hdp/current" or 
> "/usr/hdp/version".
> Suggestion from [~ncole] regarding cleaning up conf dirs.
> {code}
> cd /etc
> remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger 
> spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, 
> ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
> {code}
> The host check script should be modified to include these folders in the 
> report so that the cleanup script can clean it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AMBARI-12979) Devdeploy:Start Services fails at enabling security

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/AMBARI-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727803#comment-14727803
 ] 

Hudson commented on AMBARI-12979:
-

SUCCESS: Integrated in Ambari-branch-2.1 #465 (See 
[https://builds.apache.org/job/Ambari-branch-2.1/465/])
AMBARI-12979. Devdeploy:Start Services fails at enabling security  (aonishuk) 
(aonishuk: 
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=189beebd34476f392c39a7e02e7df40c0f46ab6e)
* 
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
* ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py


> Devdeploy:Start Services fails at enabling security 
> 
>
> Key: AMBARI-12979
> URL: https://issues.apache.org/jira/browse/AMBARI-12979
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.1.2
>
>
> Upon checking the devdeploys on suse,centos6 and centos7, noticed that they
> have failed at start services step of the security wizard.  
> At the repro cluster it has failed at Knox gateway start with the below error
> :
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 267, in 
> KnoxGateway().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 218, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 146, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py",
>  line 63, in configure
> knox()
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py",
>  line 125, in knox
> not_if=master_secret_exist,
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 154, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 152, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 118, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master 
> [PROTECTED]' returned 1. Master secret is already present on disk. Please be 
> aware that overwriting it will require updating other security artifacts.  
> Use --force to overwrite the existing master secret.
> ERROR: Invalid Command
> Unrecognized option:create-master
> A fatal exception has occurred. Program will exit.
> 
> Repro cluster :   
> Corresponding jenkins job  
>  Setup/59014/console>
> Please find the [^link](http://linux-jenkins.qe.hortonworks.com/home/jenkins
> /qe-artifacts/myos-r7-qvibjs-devdeploy/artifacts/screenshots/com.hw.ambari.ui.
> tests.installer.InstallHadoop/install/) to artifacts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38063: Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38063/
---

(Updated Sept. 2, 2015, 6:34 p.m.)


Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-12982
https://issues.apache.org/jira/browse/AMBARI-12982


Repository: ambari


Description
---

On my OSX system I have Python2.6 and Python2.7. I do not have any
`/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
tests fail with below message



==
ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
--
Traceback (most recent call last):
  File 
"/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
 line 997, in test_validateNonRootFs
self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
'property1', hostInfo))
AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
'assertIsNone'


It looks like `assertIsNone()` is not defined in unittest.TestCase for
Python2.6.

I started hitting this issue after AMBARI-12912


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 78d25bc 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 24ee07a 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
86a34d9 

Diff: https://reviews.apache.org/r/38063/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 38063: Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38063/
---

(Updated Sept. 2, 2015, 6:35 p.m.)


Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-12982
https://issues.apache.org/jira/browse/AMBARI-12982


Repository: ambari


Description
---

On my OSX system I have Python2.6 and Python2.7. I do not have any
`/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
tests fail with below message



==
ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
--
Traceback (most recent call last):
  File 
"/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
 line 997, in test_validateNonRootFs
self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
'property1', hostInfo))
AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
'assertIsNone'


It looks like `assertIsNone()` is not defined in unittest.TestCase for
Python2.6.

I started hitting this issue after AMBARI-12912


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 78d25bc 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 24ee07a 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
86a34d9 

Diff: https://reviews.apache.org/r/38063/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 38064: Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38064/
---

Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-12982
https://issues.apache.org/jira/browse/AMBARI-12982


Repository: ambari


Description
---

On my OSX system I have Python2.6 and Python2.7. I do not have any
`/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
tests fail with below message



==
ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
--
Traceback (most recent call last):
  File 
"/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
 line 997, in test_validateNonRootFs
self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
'property1', hostInfo))
AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
'assertIsNone'


It looks like `assertIsNone()` is not defined in unittest.TestCase for
Python2.6.

I started hitting this issue after AMBARI-12912


Diffs
-

  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 78d25bc 
  ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 24ee07a 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
86a34d9 

Diff: https://reviews.apache.org/r/38064/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 38064: Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Myroslav Papirkovskyy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38064/#review97511
---

Ship it!


Ship It!

- Myroslav Papirkovskyy


On Вер. 2, 2015, 9:36 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38064/
> ---
> 
> (Updated Вер. 2, 2015, 9:36 після полудня)
> 
> 
> Review request for Ambari and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-12982
> https://issues.apache.org/jira/browse/AMBARI-12982
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On my OSX system I have Python2.6 and Python2.7. I do not have any
> `/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
> tests fail with below message
> 
> 
> 
> ==
> ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
> --
> Traceback (most recent call last):
>   File 
> "/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
>  line 997, in test_validateNonRootFs
> self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
> 'property1', hostInfo))
> AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
> 'assertIsNone'
> 
> 
> It looks like `assertIsNone()` is not defined in unittest.TestCase for
> Python2.6.
> 
> I started hitting this issue after AMBARI-12912
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
>  78d25bc 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 
> 24ee07a 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 86a34d9 
> 
> Diff: https://reviews.apache.org/r/38064/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

[jira] [Resolved] (AMBARI-12982) Ambari-server 'mvn clean test' fails when Python 2.6 is picked up

[Andrew Onischuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AMBARI-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Onischuk resolved AMBARI-12982.
--
Resolution: Fixed

Committed to branch-2.1

> Ambari-server 'mvn clean test' fails when Python 2.6 is picked up
> -
>
> Key: AMBARI-12982
> URL: https://issues.apache.org/jira/browse/AMBARI-12982
> Project: Ambari
>  Issue Type: Bug
>Reporter: Andrew Onischuk
>Assignee: Andrew Onischuk
> Fix For: 2.1.2
>
>
> On my OSX system I have Python2.6 and Python2.7. I do not have any
> `/usr/bin/python2`. When I run `mvn clean test` in ambari-server/, the python
> tests fail with below message
> 
> 
> ==
> ERROR: test_validateNonRootFs (test_stack_advisor.TestHDP206StackAdvisor)
> --
> Traceback (most recent call last):
>   File 
> "/Users/sgunturi/dev/git/ambari/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py",
>  line 997, in test_validateNonRootFs
> self.assertIsNone(self.stackAdvisor.validatorNotRootFs(properties, 
> 'property1', hostInfo))
> AttributeError: 'TestHDP206StackAdvisor' object has no attribute 
> 'assertIsNone'
> 
> It looks like `assertIsNone()` is not defined in unittest.TestCase for
> Python2.6.
> I started hitting this issue after AMBARI-12912



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)