[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-27 Thread Mahadev konar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated AMBARI-16121:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.2.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.2.2
>
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderF

[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-26 Thread Dmitry Lysnichenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-16121:

Fix Version/s: 2.2.2

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.2.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.2.2
>
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderFactory: <== 
> JVMShutdownHook.run()

[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-26 Thread Dmitry Lysnichenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-16121:

Affects Version/s: 2.2.2

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Affects Versions: 2.2.2
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Fix For: 2.2.2
>
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderFactory: <== 
> JVMShutdownHook.r

[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-26 Thread Dmitry Lysnichenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-16121:

Component/s: ambari-server

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderFactory: <== 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,507 INFO  [Thread-7] regionserv

[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-26 Thread Dmitry Lysnichenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-16121:

Attachment: AMBARI-16121.patch

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderFactory: <== 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,507 INFO  [Thread-7] region

[jira] [Updated] (AMBARI-16121) HBase RegionServers go down after Ambari upgrade due to ClassCastException

2016-04-26 Thread Dmitry Lysnichenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AMBARI-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Lysnichenko updated AMBARI-16121:

Status: Patch Available  (was: Open)

> HBase RegionServers go down after Ambari upgrade due to ClassCastException
> --
>
> Key: AMBARI-16121
> URL: https://issues.apache.org/jira/browse/AMBARI-16121
> Project: Ambari
>  Issue Type: Bug
>  Components: ambari-server
>Reporter: Dmitry Lysnichenko
>Assignee: Dmitry Lysnichenko
> Attachments: AMBARI-16121.patch
>
>
> ambari-server --hash
> 2b112376b631384852a6c8aaa2e102d8dd39a9f1
> ambari-server-2.2.2.0-456.x86_64
> *Steps*
> # Deploy HDP 2.3.4.0 cluster with Ambari 2.2.0.0 (HA, unsecure cluster and 
> Ranger enabled)
> # Upgrade Ambari to 2.2.2.0
> Observed that all HBase RS reported as down
> Logs show below error:
> {code}
> 016-04-26 09:47:23,575 ERROR [regionserver/host1/ip:16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,584 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: ABORTING region server host1,16020,1461664036377: 
> The coprocessor 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw 
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> java.lang.ClassCastException: 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$RegionServerEnvironment
>  cannot be cast to 
> org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment
> at 
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:131)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:411)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:253)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:156)
> at 
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:69)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:888)
> at java.lang.Thread.run(Thread.java:745)
> 2016-04-26 09:47:23,586 FATAL [regionserver/host1/ip:16020] 
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.token.TokenProvider]
> 2016-04-26 09:47:27,474 ERROR [main] regionserver.HRegionServerCommandLine: 
> Region server exiting
> java.lang.RuntimeException: HRegionServer Aborted
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at 
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2651)
> 2016-04-26 09:47:27,491 INFO  [Thread-129] provider.AuditProviderFactory: ==> 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: Stop 
> called. name=hbaseRegional.async
> 2016-04-26 09:47:27,492 INFO  [Thread-129] queue.AuditAsyncQueue: 
> Interrupting consumerThread. name=hbaseRegional.async, 
> consumer=hbaseRegional.async.summary
> 2016-04-26 09:47:27,496 INFO  [Thread-129] provider.AuditProviderFactory: <== 
> JVMShutdownHook.run()
> 2016-04-26 09:47:27,507 INFO  [Thread-7]