[jira] [Reopened] (HBASE-20472) InfoServer doesnot honour any acl set by the admin
[
https://issues.apache.org/jira/browse/HBASE-20472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain reopened HBASE-20472:
> InfoServer doesnot honour any acl set by the admin
> --
>
> Key: HBASE-20472
> URL: https://issues.apache.org/jira/browse/HBASE-20472
> Project: HBase
> Issue Type: Bug
> Components: security, UI
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-20472.master.001.patch
>
>
> The adminsAcl property can be used to restrict access to certain sections of
> the web UI only to a particular set of users/groups. But in hbase, adminAcl
> variable for InfoServer is always null, rendering it to not honour any acl
> set by the admin. In fact I could not find any property in hbase to specify
> acl list for web server.
> *Analysis*:
> * *InfoSever* object forgets(?) to set any *adminAcl* in the builder object
> for http server.
> {code:java}
> public InfoServer(String name, String bindAddress, int port, boolean findPort,
> final Configuration c) {
> .
> .
>
> HttpServer.Builder builder =
> new org.apache.hadoop.hbase.http.HttpServer.Builder();
> .
> .
> this.httpServer = builder.build();
> }{code}
> [See InfoServer
> constructor|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/InfoServer.java#L55]
> * http server retreives a null value and sets it as adminsAcl, which is
> passed to *createWebAppContext*() method
> {code:java}
> private HttpServer(final Builder b) throws IOException {
> .
> .
> .
> this.adminsAcl = b.adminsAcl;
> this.webAppContext = createWebAppContext(b.name, b.conf, adminsAcl,
> appDir);
>
> .
> .
> }{code}
> [See L527
> HttpServer.java|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L527]
> * This method next sets *ADMIN_ACL* attribute for the servlet context to
> *null*
> {code:java}
> private static WebAppContext createWebAppContext(String name,
> Configuration conf, AccessControlList adminsAcl, final String appDir) {
> WebAppContext ctx = new WebAppContext();
> .
> .
> ctx.getServletContext().setAttribute(ADMINS_ACL, adminsAcl);
> .
> .
> }
> {code}
> * Now any page having *HttpServer.hasAdministratorAccess*() will allow
> access to everyone, making this check useless.
> {code:java}
> @Override
> public void doGet(HttpServletRequest request, HttpServletResponse response
> ) throws ServletException, IOException {
> // Do the authorization
> if (!HttpServer.hasAdministratorAccess(getServletContext(), request,
> response)) {
> return;
> }
> .
> .
> }{code}
> [For example See L104
> LogLevel.java|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/log/LogLevel.java#L104]
> * *hasAdministratorAccess()* checks for the following and returns true, in
> any case as *ADMIN_ACL* is always *null*
> {code:java}
> public static boolean hasAdministratorAccess(
> ServletContext servletContext, HttpServletRequest request,
> HttpServletResponse response) throws IOException {
> .
> .
> if (servletContext.getAttribute(ADMINS_ACL) != null &&
> !userHasAdministratorAccess(servletContext, remoteUser)) {
> response.sendError(HttpServletResponse.SC_UNAUTHORIZED, "User "
> + remoteUser + " is unauthorized to access this page.");
>return false;
> }
> return true;
> }{code}
> [See line 1196 in
> HttpServer|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L1196]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20472) InfoServer doesnot honour any acl set by the admin
[
https://issues.apache.org/jira/browse/HBASE-20472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain resolved HBASE-20472.
Resolution: Duplicate
> InfoServer doesnot honour any acl set by the admin
> --
>
> Key: HBASE-20472
> URL: https://issues.apache.org/jira/browse/HBASE-20472
> Project: HBase
> Issue Type: Bug
> Components: security, UI
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-20472.master.001.patch
>
>
> The adminsAcl property can be used to restrict access to certain sections of
> the web UI only to a particular set of users/groups. But in hbase, adminAcl
> variable for InfoServer is always null, rendering it to not honour any acl
> set by the admin. In fact I could not find any property in hbase to specify
> acl list for web server.
> *Analysis*:
> * *InfoSever* object forgets(?) to set any *adminAcl* in the builder object
> for http server.
> {code:java}
> public InfoServer(String name, String bindAddress, int port, boolean findPort,
> final Configuration c) {
> .
> .
>
> HttpServer.Builder builder =
> new org.apache.hadoop.hbase.http.HttpServer.Builder();
> .
> .
> this.httpServer = builder.build();
> }{code}
> [See InfoServer
> constructor|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/InfoServer.java#L55]
> * http server retreives a null value and sets it as adminsAcl, which is
> passed to *createWebAppContext*() method
> {code:java}
> private HttpServer(final Builder b) throws IOException {
> .
> .
> .
> this.adminsAcl = b.adminsAcl;
> this.webAppContext = createWebAppContext(b.name, b.conf, adminsAcl,
> appDir);
>
> .
> .
> }{code}
> [See L527
> HttpServer.java|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L527]
> * This method next sets *ADMIN_ACL* attribute for the servlet context to
> *null*
> {code:java}
> private static WebAppContext createWebAppContext(String name,
> Configuration conf, AccessControlList adminsAcl, final String appDir) {
> WebAppContext ctx = new WebAppContext();
> .
> .
> ctx.getServletContext().setAttribute(ADMINS_ACL, adminsAcl);
> .
> .
> }
> {code}
> * Now any page having *HttpServer.hasAdministratorAccess*() will allow
> access to everyone, making this check useless.
> {code:java}
> @Override
> public void doGet(HttpServletRequest request, HttpServletResponse response
> ) throws ServletException, IOException {
> // Do the authorization
> if (!HttpServer.hasAdministratorAccess(getServletContext(), request,
> response)) {
> return;
> }
> .
> .
> }{code}
> [For example See L104
> LogLevel.java|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/log/LogLevel.java#L104]
> * *hasAdministratorAccess()* checks for the following and returns true, in
> any case as *ADMIN_ACL* is always *null*
> {code:java}
> public static boolean hasAdministratorAccess(
> ServletContext servletContext, HttpServletRequest request,
> HttpServletResponse response) throws IOException {
> .
> .
> if (servletContext.getAttribute(ADMINS_ACL) != null &&
> !userHasAdministratorAccess(servletContext, remoteUser)) {
> response.sendError(HttpServletResponse.SC_UNAUTHORIZED, "User "
> + remoteUser + " is unauthorized to access this page.");
>return false;
> }
> return true;
> }{code}
> [See line 1196 in
> HttpServer|https://github.com/apache/hbase/blob/46cb5dfa226892fd2580f26ce9ce77225bd7e67c/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L1196]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21222) [amv2] Closing region on a non-existent server creates STUCK regions
[
https://issues.apache.org/jira/browse/HBASE-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-21222.
---
Resolution: Duplicate
Resolving as duplicate of HBASE-21259. It goes into more depth on why we
suspend/get STUCK.
> [amv2] Closing region on a non-existent server creates STUCK regions
>
>
> Key: HBASE-21222
> URL: https://issues.apache.org/jira/browse/HBASE-21222
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Ran into this one where a Region had been on a server but after a bunch of
> crashing and meddling in Master Proc WALs, any attempt at unassign has the
> procedure fail (see below) and then report the region as STUCK.
> I broke the lock w/ new hbck2 tooling and then tried to offline again but
> same thing happened. Bug. Fix.
> {code}
> 2018-09-22 18:36:41,900 INFO
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
> pid=138650, ppid=121871, state=RUNNABLE:REGION_TRANSITION_DISPATCH,
> locked=true; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180614072614,
> region=51cdade76ca7217ec191f39e5f56c61c,
> server=vd0637.halxg.cloudera.com,22101,1537397969558; rit=CLOSING,
> location=vd0637.halxg.cloudera.com,22101,1537397969558
> 2018-09-22 18:36:41,899 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler:
> pid=138646, ppid=121871, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> UnassignProcedure table=IntegrationTestBigLinkedList_20180614072614,
> region=0780467efe4c5901887fb12bfa406fa7,
> server=vc1228.halxg.cloudera.com,22101,1537578279837 checking lock on
> 0780467efe4c5901887fb12bfa406fa7
> 2018-09-22 18:36:41,900 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Remote
> call failed vd0637.halxg.cloudera.com,22101,1537397969558; pid=138650,
> ppid=121871, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=IntegrationTestBigLinkedList_20180614072614,
> region=51cdade76ca7217ec191f39e5f56c61c,
> server=vd0637.halxg.cloudera.com,22101,1537397969558; rit=CLOSING,
> location=vd0637.halxg.cloudera.com,22101,1537397969558;
> exception=NoServerDispatchException
> org.apache.hadoop.hbase.procedure2.NoServerDispatchException:
> vd0637.halxg.cloudera.com,22101,1537397969558; pid=138650, ppid=121871,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180614072614,
> region=51cdade76ca7217ec191f39e5f56c61c,
> server=vd0637.halxg.cloudera.com,22101,1537397969558
> at
> org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.addOperationToNode(RemoteProcedureDispatcher.java:177)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.addToRemoteDispatcher(RegionTransitionProcedure.java:277)
> at
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:202)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:370)
> at
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:924)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1684)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1471)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:77)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983)
> 2018-09-22 18:36:41,903 WARN
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure: Expiring
> vd0637.halxg.cloudera.com,22101,1537397969558, pid=138650, ppid=121871,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
> table=IntegrationTestBigLinkedList_20180614072614,
> region=51cdade76ca7217ec191f39e5f56c61c,
> server=vd0637.halxg.cloudera.com,22101,1537397969558 rit=CLOSING,
> location=vd0637.halxg.cloudera.com,22101,1537397969558;
> exception=NoServerDispatchException
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21264) Namespace Quota : Split after exceeding maxregions- regions went to other regions state and never come back online
Chandra Sekhar created HBASE-21264:
--
Summary: Namespace Quota : Split after exceeding maxregions-
regions went to other regions state and never come back online
Key: HBASE-21264
URL: https://issues.apache.org/jira/browse/HBASE-21264
Project: HBase
Issue Type: Bug
Components: Region Assignment
Reporter: Chandra Sekhar
{{As my namespace quota (max regions = 5) exceeds, when i perform split region
more than 4 times, regions went to Other Regions state and never come back
online. It was showing 0 Online Regions}}
{{Scenario:}}
{noformat}
1. Create namespace with quota - 5 max regions
2. Create table
3. Split the table more than 5 split keys
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [VOTE] The first HBase 1.4.8 release candidate (RC0) is available
+1 - signatures, checksums: OK - rat check: OK - built from source (8u112): OK - unit tests (8u112): OK - ltt with 1M rows: OK - web UI: OK - shell and basic commands: OK Thanks, Balazs On Wed, Oct 3, 2018 at 2:58 AM Andrew Purtell wrote: > The first HBase 1.4.8 release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1233/ > > The git tag corresponding to the candidate is '1.4.8RC0' (91118ce5f1). > > A detailed source and binary compatibility report for this release is > available for your review at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/compat-check-report.html > . There are no reported compatibility issues. > > A list of the 33 issues resolved in this release can be found at > https://s.apache.org/xpxo . > > Please try out the candidate and vote +1/0/-1. > > The vote will be open for at least 72 hours. Unless objection I will try to > close it Monday October 8, 2018 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks: > > RAT check passes (7u80) > Unit test suite passes (7u80, 8u181) > Opened the UI in a browser, poked around > LTT load 1M rows with 100% verification and 20% updates (8u181) > ITBLL 500M rows with serverKilling monkey (8u181) > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
[jira] [Resolved] (HBASE-19444) RSGroups test units cannot be concurrently executed
[ https://issues.apache.org/jira/browse/HBASE-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-19444. Resolution: Duplicate Fix Version/s: (was: 1.4.9) (was: 2.2.0) (was: 1.5.0) (was: 3.0.0) Duping this out. Replacing with a task issue to break up TestRSGroups into smaller units. Current run time is ~240 seconds / 4 minutes and the test is only stable when run by itself. > RSGroups test units cannot be concurrently executed > --- > > Key: HBASE-19444 > URL: https://issues.apache.org/jira/browse/HBASE-19444 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Priority: Minor > > TestRSGroups and friends cannot be concurrently executed or they are very > likely to flake, failing with constraint exceptions. If executed serially all > units pass. Fix for concurrent execution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21265) Split up TestRSGroups
Andrew Purtell created HBASE-21265: -- Summary: Split up TestRSGroups Key: HBASE-21265 URL: https://issues.apache.org/jira/browse/HBASE-21265 Project: HBase Issue Type: Task Components: rsgroup, test Affects Versions: 1.4.8 Reporter: Andrew Purtell Fix For: 3.0.0, 1.5.0, 2.2.0 TestRSGroups is flaky. It is stable when run in isolation but when run as part of the suite with concurrent executors it can fail. The current running time of this unit on my dev box is ~240 seconds (4 minutes), which is far too much time. This unit should be broken up 5 to 8 ways, grouped by functionality under test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] The first HBase 1.4.8 release candidate (RC0) is available
RC errata: TestRSGroups will pass in isolation but may fail when run as part of the suite. There have been several JIRAs filed against this unit like HBASE-19444, HBASE-19461, HBASE-20137, and mentioned on HBASE-21187. It's running time is far too long. I have filed HBASE-21265 to split it up. Test stabilization work would be a part of that. I don't think this rises to the level of failing the RC vote because TestRSGroups will pass consistently, at least for me, when run in isolation. I do agree that without work to improve the test it doesn't offer the kind of functional assurance we'd like to derive from a unit test. On Tue, Oct 2, 2018 at 5:57 PM Andrew Purtell wrote: > The first HBase 1.4.8 release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1233/ > > The git tag corresponding to the candidate is '1.4.8RC0' (91118ce5f1). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/compat-check-report.html > . There are no reported compatibility issues. > > A list of the 33 issues resolved in this release can be found at > https://s.apache.org/xpxo . > > Please try out the candidate and vote +1/0/-1. > > The vote will be open for at least 72 hours. Unless objection I will try > to close it Monday October 8, 2018 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks: > > RAT check passes (7u80) > Unit test suite passes (7u80, 8u181) > Opened the UI in a browser, poked around > LTT load 1M rows with 100% verification and 20% updates (8u181) > ITBLL 500M rows with serverKilling monkey (8u181) > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-21266) Not running balancer because processing dead regionservers, but empty rs list, and state does not recover
Andrew Purtell created HBASE-21266: -- Summary: Not running balancer because processing dead regionservers, but empty rs list, and state does not recover Key: HBASE-21266 URL: https://issues.apache.org/jira/browse/HBASE-21266 Project: HBase Issue Type: Bug Affects Versions: 1.4.8 Reporter: Andrew Purtell Fix For: 1.5.0, 1.4.9 Found during ITBLL testing. AM in master gets into a state where manual attempts from the shell to run the balancer always return false and this is printed in the master log: 2018-10-03 19:17:14,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=21,queue=0,port=8100] master.HMaster: Not running balancer because processing dead regionserver(s): Note the empty list. This errant state did not recover without intervention by way of master restart, but the test environment was chaotic so needs investigation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] The first HBase 1.4.8 release candidate (RC0) is available
A 1 billion row ITBLL job with slowDeterministic chaos policy also completed successfully (8u181). A second 1 billion row ITBLL job on another test cluster also completed but I noticed HBASE-21266 while poking around. I couldn't reproduce this on the other cluster with some manual regionserver kills. Needs investigation I'd say but not a mark against this RC. On Tue, Oct 2, 2018 at 5:57 PM Andrew Purtell wrote: > The first HBase 1.4.8 release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1233/ > > The git tag corresponding to the candidate is '1.4.8RC0' (91118ce5f1). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/compat-check-report.html > . There are no reported compatibility issues. > > A list of the 33 issues resolved in this release can be found at > https://s.apache.org/xpxo . > > Please try out the candidate and vote +1/0/-1. > > The vote will be open for at least 72 hours. Unless objection I will try > to close it Monday October 8, 2018 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks: > > RAT check passes (7u80) > Unit test suite passes (7u80, 8u181) > Opened the UI in a browser, poked around > LTT load 1M rows with 100% verification and 20% updates (8u181) > ITBLL 500M rows with serverKilling monkey (8u181) > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-21267) Spark in java examples throws 'A master URL must be set in your configuration'
ramkrishna.s.vasudevan created HBASE-21267:
--
Summary: Spark in java examples throws 'A master URL must be set
in your configuration'
Key: HBASE-21267
URL: https://issues.apache.org/jira/browse/HBASE-21267
Project: HBase
Issue Type: Test
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Trying out some spark on java examples I found this
'A master URL must be set in your configuration'.
It comes from here
bq.SparkConf sparkConf = new
SparkConf().setAppName("JavaHBaseBulkPutExample " + tableName);
Adding the below line solved the problem,
bq. SparkConf sparkConf = new SparkConf().setAppName("JavaHBaseBulkPutExample "
+ tableName).setMaster("local[2]")
But am not sure if there is any better way.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
