[jira] [Updated] (HBASE-17997) jruby-complete-1.6.8.jar is in cached_classpath.txt
[ https://issues.apache.org/jira/browse/HBASE-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-17997: - Attachment: HBASE-17997.master.001.patch > jruby-complete-1.6.8.jar is in cached_classpath.txt > --- > > Key: HBASE-17997 > URL: https://issues.apache.org/jira/browse/HBASE-17997 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Xiang Li > Attachments: HBASE-17997.master.000.patch, > HBASE-17997.master.001.patch > > > HBASE-15199 moves jruby-complete-1.6.8.jar to lib/ruby directory. > However, jruby-complete-1.6.8.jar still appears in cached_classpath.txt > This means that user would see exception similar to the following when > starting hbase in standalone mode with s3a as rootdir : > {code} > 2017-05-04 16:41:32,854 WARN > [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=38659] > internal.S3MetadataResponseHandler: Unable to parse last modified date: Thu, > 04 May 2017 16:27:09 GMT > java.lang.IllegalStateException: Joda-time 2.2 or later version is required, > but found version: null > at com.amazonaws.util.DateUtils.handleException(DateUtils.java:149) > at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:195) > at > com.amazonaws.services.s3.internal.ServiceUtils.parseRfc822Date(ServiceUtils.java:78) > at > com.amazonaws.services.s3.internal.AbstractS3ResponseHandler.populateObjectMetadata(AbstractS3ResponseHandler.java:115) > at > com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:52) > at > com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:30) > at > com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1072) > at > com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:746) > at > com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489) > at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310) > at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785) > at > com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1191) > at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:148) > at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:281) > at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:364) > at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:75) > at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock.positionalReadWithExtra(HFileBlock.java:722) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1420) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1677) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1504) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:439) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:904) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:267) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:169) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:217) > at > org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2132) > at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2122) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5687) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2679) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2665) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2647) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6906) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6885) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2007) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17997) jruby-complete-1.6.8.jar is in cached_classpath.txt
[ https://issues.apache.org/jira/browse/HBASE-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-17997: - Status: Patch Available (was: Open) > jruby-complete-1.6.8.jar is in cached_classpath.txt > --- > > Key: HBASE-17997 > URL: https://issues.apache.org/jira/browse/HBASE-17997 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Xiang Li > Attachments: HBASE-17997.master.000.patch, > HBASE-17997.master.001.patch > > > HBASE-15199 moves jruby-complete-1.6.8.jar to lib/ruby directory. > However, jruby-complete-1.6.8.jar still appears in cached_classpath.txt > This means that user would see exception similar to the following when > starting hbase in standalone mode with s3a as rootdir : > {code} > 2017-05-04 16:41:32,854 WARN > [RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=38659] > internal.S3MetadataResponseHandler: Unable to parse last modified date: Thu, > 04 May 2017 16:27:09 GMT > java.lang.IllegalStateException: Joda-time 2.2 or later version is required, > but found version: null > at com.amazonaws.util.DateUtils.handleException(DateUtils.java:149) > at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:195) > at > com.amazonaws.services.s3.internal.ServiceUtils.parseRfc822Date(ServiceUtils.java:78) > at > com.amazonaws.services.s3.internal.AbstractS3ResponseHandler.populateObjectMetadata(AbstractS3ResponseHandler.java:115) > at > com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:52) > at > com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:30) > at > com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1072) > at > com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:746) > at > com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489) > at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310) > at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785) > at > com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1191) > at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:148) > at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:281) > at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:364) > at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:75) > at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock.positionalReadWithExtra(HFileBlock.java:722) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1420) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1677) > at > org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1504) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:439) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:904) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:267) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:169) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:217) > at > org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2132) > at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2122) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:5687) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2679) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2665) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2647) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6906) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6885) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2007) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022399#comment-16022399 ] Phil Yang commented on HBASE-15576: --- Exception will break the loop so users can not choose to continue the scanning? The previous design is using a new class Cursor but it may be more complicated. Another option is let Cursor extend Result so the code of users will not be changed so much like this: {code} while (r = scanner.next() && r != null) { if(r instanceof Cursor){ // scanning is not end, it is a cursor, save its row key and close scanner if you want, or // just continue the loop to call next(). } else { // just like before } } // scanning is end {code} > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setNeedCursorResult(true) to get the cursor row > key when there is a RPC response but client can not return any Result. In > this mode we will not block ResultScanner.next() longer than this timeout > setting. > {code} > while (r = scanner.next() && r != null) { > if(r.isCursor()){ > // scanning is not end, it is a cursor, save its row key and close scanner > if you want, or > // just continue the loop to call next(). > } else { > // just like before > } > } > // scanning is end > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18066) Get with closest_row_before on "hbase:meta" can return empty Cell during region merge/split
[ https://issues.apache.org/jira/browse/HBASE-18066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-18066: - Attachment: HBASE-18066.branch-1.v2.patch Upload patch v2: Introduce a buildScanForGetWithClosestRowBefore method to construct Scan object for Get with closestRowBefore. > Get with closest_row_before on "hbase:meta" can return empty Cell during > region merge/split > --- > > Key: HBASE-18066 > URL: https://issues.apache.org/jira/browse/HBASE-18066 > Project: HBase > Issue Type: Bug > Components: hbase, regionserver >Affects Versions: 1.3.1 > Environment: Linux (16.04.2), MacOS 10.11.6. > Standalone and distributed HBase setup. >Reporter: Andrey Elenskiy >Assignee: Zheng Hu > Attachments: HBASE-18066.branch-1.v1.patch, > HBASE-18066.branch-1.v2.patch, TestGetWithClosestRowBeforeWhenSplit.java > > > During region split/merge there's a brief period of time where doing a "Get" > with "closest_row_before=true" on "hbase:meta" may return empty > "GetResponse.result.cell" field even though parent, splitA and splitB regions > are all in "hbase:meta". Both gohbase (https://github.com/tsuna/gohbase) and > AsyncHBase (https://github.com/OpenTSDB/asynchbase) interprets this as > "TableDoesNotExist", which is returned to the client. > Here's a gist that reproduces this problem: > https://gist.github.com/Timoha/c7a236b768be9220e85e53e1ca53bf96. Note that > you have to use older HTable client (I used 1.2.4) as current versions ignore > `Get.setClosestRowBefore(bool)` option. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18084) Improve CleanerChore to clean from directory which consumes more disk space
[ https://issues.apache.org/jira/browse/HBASE-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-18084: -- Attachment: HBASE-18084.v3.patch Minor improvement: don't need to get space usage and sort when there's only one directory > Improve CleanerChore to clean from directory which consumes more disk space > --- > > Key: HBASE-18084 > URL: https://issues.apache.org/jira/browse/HBASE-18084 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-18084.patch, HBASE-18084.v2.patch, > HBASE-18084.v3.patch > > > Currently CleanerChore cleans the directory in dictionary order, rather than > from the directory with largest space usage. And when data abnormally > accumulated to some huge volume in archive directory, the cleaning speed > might not be enough. > This proposal is another improvement working together with HBASE-18083 to > resolve our online issue (archive dir consumed more than 1.8PB SSD space) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022376#comment-16022376 ] Duo Zhang commented on HBASE-15576: --- In the current patch, we use Result.isCursor to determine if this is a cursor, and then use Result.getRow to get the rowkey which means that the cursor can only be a rowkey? I would suggest that we return a data structure, or at least, a byte array with no meanings to user and introduce Scan.createFromCursor method to create a Scan object. This is better for us to change the content of the cursor in the future to include mvcc or make the cursor start at the middle of a row. And maybe we could wrapper the cursor with an exception when using ResultScanner to keep the API of Result clean? And for async client, we could change the RawScanResultConsumer to pass the cursor when calling onHeartbeat? Thanks. > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setNeedCursorResult(true) to get the cursor row > key when there is a RPC response but client can not return any Result. In > this mode we will not block ResultScanner.next() longer than this timeout > setting. > {code} > while (r = scanner.next() && r != null) { > if(r.isCursor()){ > // scanning is not end, it is a cursor, save its row key and close scanner > if you want, or > // just continue the loop to call next(). > } else { > // just like before > } > } > // scanning is end > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18085) Prevent parallel purge in ObjectPool
[ https://issues.apache.org/jira/browse/HBASE-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022369#comment-16022369 ] Yu Li commented on HBASE-18085: --- {{TestLockManager}} failure is irrelative with change here, and confirmed it could pass locally. Will commit soon if no objections. > Prevent parallel purge in ObjectPool > > > Key: HBASE-18085 > URL: https://issues.apache.org/jira/browse/HBASE-18085 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: e89l05465.st3.jstack, HBASE-18085.patch, > HBASE-18085.patch > > > Parallel purge in ObjectPool is meaningless and will cause contention issue > since {{ReferenceQueue#poll}} has synchronization (source code shown below) > {code} > public Reference poll() { > if (head == null) > return null; > synchronized (lock) { > return reallyPoll(); > } > } > {code} > We observed threads blocking on the purge method while using offheap bucket > cache, and we could easily reproduce this by testing the 100% cache hit case > in bucket cache with enough reading threads. > We propose to add a purgeLock and use tryLock to avoid parallel purge. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Issue Comment Deleted] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Comment: was deleted (was: Post a document at https://docs.google.com/document/d/1pGf2J7xt5V0E8hD1hd-zjsTvnPN8CC1OEyoaoozJQhU Comments are welcomed :)) > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setNeedCursorResult(true) to get the cursor row > key when there is a RPC response but client can not return any Result. In > this mode we will not block ResultScanner.next() longer than this timeout > setting. > {code} > while (r = scanner.next() && r != null) { > if(r.isCursor()){ > // scanning is not end, it is a cursor, save its row key and close scanner > if you want, or > // just continue the loop to call next(). > } else { > // just like before > } > } > // scanning is end > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Description: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in this scene. And maybe we can keep mvcc in cursor so we can get a consistent view? HBASE-13099 had some discussion, but it has no further progress by now. API: In Scan we need a new method setNeedCursorResult(true) to get the cursor row key when there is a RPC response but client can not return any Result. In this mode we will not block ResultScanner.next() longer than this timeout setting. {code} while (r = scanner.next() && r != null) { if(r.isCursor()){ // scanning is not end, it is a cursor, save its row key and close scanner if you want, or // just continue the loop to call next(). } else { // just like before } } // scanning is end {code} was: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in this scene. And maybe we can keep mvcc in cursor so we can get a consistent view? HBASE-13099 had some discussion, but it has no further progress by now. API: In Scan we need a new method setNeedCursorResult(true) to get the cursor row key when there is a RPC response but client can not return any Result. In this mode we will not block ResultScanner.next() longer than this timeout setting. {code} while (r = scanner.next() && r != null) { if(r.isCursor()){ // scanning is not end, it is a cursor, save its row key and close scanner if you want, or // just continue the loop to call next(). } else { // just like before } } // scanning is end {code} Implementation: We will have two options to sup
[jira] [Issue Comment Deleted] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Comment: was deleted (was: Post a document at https://docs.google.com/document/d/1pGf2J7xt5V0E8hD1hd-zjsTvnPN8CC1OEyoaoozJQhU Comments are welcomed :)) > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setNeedCursorResult(true) to get the cursor row > key when there is a RPC response but client can not return any Result. In > this mode we will not block ResultScanner.next() longer than this timeout > setting. > {code} > while (r = scanner.next() && r != null) { > if(r.isCursor()){ > // scanning is not end, it is a cursor, save its row key and close scanner > if you want, or > // just continue the loop to call next(). > } else { > // just like before > } > } > // scanning is end > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Description: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in this scene. And maybe we can keep mvcc in cursor so we can get a consistent view? HBASE-13099 had some discussion, but it has no further progress by now. API: In Scan we need a new method setNeedCursorResult(true) to get the cursor row key when there is a RPC response but client can not return any Result. In this mode we will not block ResultScanner.next() longer than this timeout setting. {code} while (r = scanner.next() && r != null) { if(r.isCursor()){ // scanning is not end, it is a cursor, save its row key and close scanner if you want, or // just continue the loop to call next(). } else { // just like before } } // scanning is end {code} Implementation: We will have two options to support stateless scanning: Only one rpc like small scanning, not supporting batch/partials and cursor is row level. It is simple to implementation. Support big scanning with several rpc requests, supporting batch/partials and cursor is cell level. It is a little complex because we need seek at server side and we should make sure total time of rpc requests not exceed timeout setting. Or we can make it by two phases, support one-shot first? Any thoughts? Thanks. was: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in this scene. And maybe we can keep mvcc in cursor so we can get a consistent view? HBASE-13099 had some discussion, but it has no further progress by now. API: In Scan we need a new method setStateles
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Description: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in this scene. And maybe we can keep mvcc in cursor so we can get a consistent view? HBASE-13099 had some discussion, but it has no further progress by now. API: In Scan we need a new method setStateless to make the scanning stateless and need another timeout setting for stateless scanning. In this mode we will not block ResultScanner.next() longer than this timeout setting. And we will return Results in next() as usual but the last Result (or only Result if we receive empty heartbeat) has a special flag to mark it a cursor. The cursor Result has only one Cell. Users can scan like this: {code} while (r = scanner.next() && r != null) { if(r.isCursor()){ // scanning is not end, it is a cursor, save its row key and close scanner if you want, or // just continue the loop to call next(). } else { // just like before } } // scanning is end {code} Implementation: We will have two options to support stateless scanning: Only one rpc like small scanning, not supporting batch/partials and cursor is row level. It is simple to implementation. Support big scanning with several rpc requests, supporting batch/partials and cursor is cell level. It is a little complex because we need seek at server side and we should make sure total time of rpc requests not exceed timeout setting. Or we can make it by two phases, support one-shot first? Any thoughts? Thanks. was: After 1.1.0 released, we have partial and heartbeat protocol in scanning to prevent responding large data or timeout. Now for ResultScanner.next(), we may block for longer time larger than timeout settings to get a Result if the row is very large, or filter is sparse, or there are too many delete markers in files. However, in some scenes, we don't want it to be blocked for too long. For example, a web service which handles requests from mobile devices whose network is not stable and we can not set timeout too long(eg. only 5 seconds) between mobile and web service. This service will scan rows from HBase and return it to mobile devices. In this scene, the simplest way is to make the web service stateless. Apps in mobile devices will send several requests one by one to get the data until enough just like paging a list. In each request it will carry a start position which depends on the last result from web service. Different requests can be sent to different web service server because it is stateless. Therefore, the stateless web service need a cursor from HBase telling where we have scanned in RegionScanner when HBase client receives an empty heartbeat. And the service will return the cursor to mobile device although the response has no data. In next request we can start at the position of cursor, without the cursor we have to scan from last returned result and we may timeout forever. And of course even if the heartbeat message is not empty we can still use cursor to prevent re-scan the same rows/cells which has beed skipped. Obviously, we will give up consistency for scanning because even HBase client is also stateless, but it is acceptable in th
[jira] [Commented] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022360#comment-16022360 ] Phil Yang commented on HBASE-15576: --- https://reviews.apache.org/r/59516/ > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setStateless to make the scanning stateless and > need another timeout setting for stateless scanning. In this mode we will not > block ResultScanner.next() longer than this timeout setting. And we will > return Results in next() as usual but the last Result (or only Result if we > receive empty heartbeat) has a special flag to mark it a cursor. The cursor > Result has only one Cell. Users can scan like this: > {code} > while( r = scanner.next() && r != null && !r.isCursor()){ > //just like before > } > if(r != null){ > // scanning is not end, it is a cursor > } else { > // scanning is end > } > scanner.close() > {code} > Implementation: > We will have two options to support stateless scanning: > Only one rpc like small scanning, not supporting batch/partials and cursor is > row level. It is simple to implementation. > Support big scanning with several rpc requests, supporting batch/partials and > cursor is cell level. It is a little complex because we need seek at server > side and we should make sure total time of rpc requests not exceed timeout > setting. > Or we can make it by two phases, support one-shot first? > Any thoughts? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Attachment: HBASE-15576.v01.patch Upload a patch to support row-level scan cursor > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15576.v01.patch > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setStateless to make the scanning stateless and > need another timeout setting for stateless scanning. In this mode we will not > block ResultScanner.next() longer than this timeout setting. And we will > return Results in next() as usual but the last Result (or only Result if we > receive empty heartbeat) has a special flag to mark it a cursor. The cursor > Result has only one Cell. Users can scan like this: > {code} > while( r = scanner.next() && r != null && !r.isCursor()){ > //just like before > } > if(r != null){ > // scanning is not end, it is a cursor > } else { > // scanning is end > } > scanner.close() > {code} > Implementation: > We will have two options to support stateless scanning: > Only one rpc like small scanning, not supporting batch/partials and cursor is > row level. It is simple to implementation. > Support big scanning with several rpc requests, supporting batch/partials and > cursor is cell level. It is a little complex because we need seek at server > side and we should make sure total time of rpc requests not exceed timeout > setting. > Or we can make it by two phases, support one-shot first? > Any thoughts? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Issue Type: New Feature (was: Sub-task) Parent: (was: HBASE-17143) > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setStateless to make the scanning stateless and > need another timeout setting for stateless scanning. In this mode we will not > block ResultScanner.next() longer than this timeout setting. And we will > return Results in next() as usual but the last Result (or only Result if we > receive empty heartbeat) has a special flag to mark it a cursor. The cursor > Result has only one Cell. Users can scan like this: > {code} > while( r = scanner.next() && r != null && !r.isCursor()){ > //just like before > } > if(r != null){ > // scanning is not end, it is a cursor > } else { > // scanning is end > } > scanner.close() > {code} > Implementation: > We will have two options to support stateless scanning: > Only one rpc like small scanning, not supporting batch/partials and cursor is > row level. It is simple to implementation. > Support big scanning with several rpc requests, supporting batch/partials and > cursor is cell level. It is a little complex because we need seek at server > side and we should make sure total time of rpc requests not exceed timeout > setting. > Or we can make it by two phases, support one-shot first? > Any thoughts? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Fix Version/s: 1.4.0 2.0.0 > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0 > > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setStateless to make the scanning stateless and > need another timeout setting for stateless scanning. In this mode we will not > block ResultScanner.next() longer than this timeout setting. And we will > return Results in next() as usual but the last Result (or only Result if we > receive empty heartbeat) has a special flag to mark it a cursor. The cursor > Result has only one Cell. Users can scan like this: > {code} > while( r = scanner.next() && r != null && !r.isCursor()){ > //just like before > } > if(r != null){ > // scanning is not end, it is a cursor > } else { > // scanning is end > } > scanner.close() > {code} > Implementation: > We will have two options to support stateless scanning: > Only one rpc like small scanning, not supporting batch/partials and cursor is > row level. It is simple to implementation. > Support big scanning with several rpc requests, supporting batch/partials and > cursor is cell level. It is a little complex because we need seek at server > side and we should make sure total time of rpc requests not exceed timeout > setting. > Or we can make it by two phases, support one-shot first? > Any thoughts? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15576) Scanning cursor to prevent blocking long time on ResultScanner.next()
[ https://issues.apache.org/jira/browse/HBASE-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15576: -- Summary: Scanning cursor to prevent blocking long time on ResultScanner.next() (was: Support stateless scanning and scanning cursor) > Scanning cursor to prevent blocking long time on ResultScanner.next() > - > > Key: HBASE-15576 > URL: https://issues.apache.org/jira/browse/HBASE-15576 > Project: HBase > Issue Type: Sub-task >Reporter: Phil Yang >Assignee: Phil Yang > > After 1.1.0 released, we have partial and heartbeat protocol in scanning to > prevent responding large data or timeout. Now for ResultScanner.next(), we > may block for longer time larger than timeout settings to get a Result if the > row is very large, or filter is sparse, or there are too many delete markers > in files. > However, in some scenes, we don't want it to be blocked for too long. For > example, a web service which handles requests from mobile devices whose > network is not stable and we can not set timeout too long(eg. only 5 seconds) > between mobile and web service. This service will scan rows from HBase and > return it to mobile devices. In this scene, the simplest way is to make the > web service stateless. Apps in mobile devices will send several requests one > by one to get the data until enough just like paging a list. In each request > it will carry a start position which depends on the last result from web > service. Different requests can be sent to different web service server > because it is stateless. > Therefore, the stateless web service need a cursor from HBase telling where > we have scanned in RegionScanner when HBase client receives an empty > heartbeat. And the service will return the cursor to mobile device although > the response has no data. In next request we can start at the position of > cursor, without the cursor we have to scan from last returned result and we > may timeout forever. And of course even if the heartbeat message is not empty > we can still use cursor to prevent re-scan the same rows/cells which has beed > skipped. > Obviously, we will give up consistency for scanning because even HBase client > is also stateless, but it is acceptable in this scene. And maybe we can keep > mvcc in cursor so we can get a consistent view? > HBASE-13099 had some discussion, but it has no further progress by now. > API: > In Scan we need a new method setStateless to make the scanning stateless and > need another timeout setting for stateless scanning. In this mode we will not > block ResultScanner.next() longer than this timeout setting. And we will > return Results in next() as usual but the last Result (or only Result if we > receive empty heartbeat) has a special flag to mark it a cursor. The cursor > Result has only one Cell. Users can scan like this: > {code} > while( r = scanner.next() && r != null && !r.isCursor()){ > //just like before > } > if(r != null){ > // scanning is not end, it is a cursor > } else { > // scanning is end > } > scanner.close() > {code} > Implementation: > We will have two options to support stateless scanning: > Only one rpc like small scanning, not supporting batch/partials and cursor is > row level. It is simple to implementation. > Support big scanning with several rpc requests, supporting batch/partials and > cursor is cell level. It is a little complex because we need seek at server > side and we should make sure total time of rpc requests not exceed timeout > setting. > Or we can make it by two phases, support one-shot first? > Any thoughts? Thanks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18042) Client Compatibility breaks between versions 1.2 and 1.3
[ https://issues.apache.org/jira/browse/HBASE-18042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022346#comment-16022346 ] Hadoop QA commented on HBASE-18042: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 45s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 18s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 56m 3s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 18s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 218m 59s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 55s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 302m 1s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.client.TestAsyncProcedureAdminApi | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.client.TestAsyncSnapshotAdminApi | | Timed out junit tests | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 | | | org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream | | | org.apache.hadoop.hbase.master.balancer.TestFavoredStochasticLoadBalancer | | | org.apache.hadoop.hbase.master.TestSplitLogManager | | | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869462/HBASE-18042.patch | | JIRA Issue | HBASE-18042 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9d61eef71410 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6908/artifact/patchprocess/patch-unit-hbas
[jira] [Commented] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022322#comment-16022322 ] Chia-Ping Tsai commented on HBASE-18096: +1 > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2 > > Attachments: HBASE-18096-BM-0001.patch > > > HFileUtil should be package private and should have > @InterfaceAudience.Private annotation. This class was introduced in > HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16488) Starting namespace and quota services in master startup asynchronizely
[ https://issues.apache.org/jira/browse/HBASE-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-16488: --- Attachment: HBASE-16488.v7-branch-1.patch > Starting namespace and quota services in master startup asynchronizely > -- > > Key: HBASE-16488 > URL: https://issues.apache.org/jira/browse/HBASE-16488 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.1.5, 1.2.2 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Attachments: HBASE-16488.v1-branch-1.patch, > HBASE-16488.v1-master.patch, HBASE-16488.v2-branch-1.patch, > HBASE-16488.v2-branch-1.patch, HBASE-16488.v3-branch-1.patch, > HBASE-16488.v3-branch-1.patch, HBASE-16488.v4-branch-1.patch, > HBASE-16488.v5-branch-1.patch, HBASE-16488.v6-branch-1.patch, > HBASE-16488.v7-branch-1.patch > > > From time to time, during internal IT test and from customer, we often see > master initialization failed due to namespace table region takes long time to > assign (eg. sometimes split log takes long time or hanging; or sometimes RS > is temporarily not available; sometimes due to some unknown assignment > issue). In the past, there was some proposal to improve this situation, eg. > HBASE-13556 / HBASE-14190 (Assign system tables ahead of user region > assignment) or HBASE-13557 (Special WAL handling for system tables) or > HBASE-14623 (Implement dedicated WAL for system tables). > This JIRA proposes another way to solve this master initialization fail > issue: namespace service is only used by a handful operations (eg. create > table / namespace DDL / get namespace API / some RS group DDL). Only quota > manager depends on it and quota management is off by default. Therefore, > namespace service is not really needed for master to be functional. So we > could start namespace service asynchronizely without blocking master startup. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022310#comment-16022310 ] stack commented on HBASE-6721: -- This issue is resolved [~javaman_chen] Open a new one or ask up on the dev mailing list? Thanks. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, balanceCluster Sequence > Diagram.svg, HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, > HBASE-6721_11.patch, HBASE-6721_12.patch, HBASE-6721_13.patch, > HBASE-6721_14.patch, HBASE-6721_15.patch, HBASE-6721_8.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, hbase-6721-v15-branch-1.1.patch, > hbase-6721-v16.patch, hbase-6721-v17.patch, hbase-6721-v18.patch, > hbase-6721-v19.patch, hbase-6721-v20.patch, hbase-6721-v21.patch, > hbase-6721-v22.patch, hbase-6721-v23.patch, hbase-6721-v25.patch, > hbase-6721-v26_draft1.patch, hbase-6721-v26.patch, hbase-6721-v27.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch.txt, hbase-6721-v28.patch, > hbase-6721-v28.patch, hbase-6721-v29.patch, immediateAssignments Sequence > Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment Sequence > Diagram.svg, roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022288#comment-16022288 ] chenxu commented on HBASE-6721: --- {code:title=RSGroupBasedLoadBalancer.roundRobinAssignment|borderStyle=solid} ... for(String groupKey : regionMap.keySet()) { if (regionMap.get(groupKey).size() > 0) { Map> result = this.internalBalancer.roundRobinAssignment( regionMap.get(groupKey), serverMap.get(groupKey)); if(result != null) { assignments.putAll(result); } } } ... {code} if two group both has BOGUS_SERVER_NAME, assignments.putAll may be not right {code:title=RSGroupBasedLoadBalancer.randomAssignment|borderStyle=solid} public ServerName randomAssignment(HRegionInfo region, List servers) throws HBaseIOException { ListMultimap regionMap = LinkedListMultimap.create(); ListMultimap serverMap = LinkedListMultimap.create(); generateGroupMaps(Lists.newArrayList(region), servers, regionMap, serverMap); List filteredServers = serverMap.get(regionMap.keySet().iterator().next()); return this.internalBalancer.randomAssignment(region, filteredServers); } {code} if internalBalancer.randomAssignment return BOGUS_SERVER_NAME, how does AM work? I think we shoud return null instead of BOGUS_SERVER_NAME > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, balanceCluster Sequence > Diagram.svg, HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, > HBASE-6721_11.patch, HBASE-6721_12.patch, HBASE-6721_13.patch, > HBASE-6721_14.patch, HBASE-6721_15.patch, HBASE-6721_8.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, hbase-6721-v15-branch-1.1.patch, > hbase-6721-v16.patch, hbase-6721-v17.patch, hbase-6721-v18.patch, > hbase-6721-v19.patch, hbase-6721-v20.patch, hbase-6721-v21.patch, > hbase-6721-v22.patch, hbase-6721-v23.patch, hbase-6721-v25.patch, > hbase-6721-v26_draft1.patch, hbase-6721-v26.patch, hbase-6721-v27.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch.txt, hbase-6721-v28.patch, > hbase-6721-v28.patch, hbase-6721-v29.patch, immediateAssignments Sequence > Diagram.svg, randomAssignment Sequence Diagram.svg, retainAssignment Sequence > Diagram.svg, roundRobinAssignment Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18066) Get with closest_row_before on "hbase:meta" can return empty Cell during region merge/split
[ https://issues.apache.org/jira/browse/HBASE-18066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-18066: - Attachment: HBASE-18066.branch-1.v1.patch uploaded initial version , and implemented the getClosestRowBefore method by reverse scan at server side. > Get with closest_row_before on "hbase:meta" can return empty Cell during > region merge/split > --- > > Key: HBASE-18066 > URL: https://issues.apache.org/jira/browse/HBASE-18066 > Project: HBase > Issue Type: Bug > Components: hbase, regionserver >Affects Versions: 1.3.1 > Environment: Linux (16.04.2), MacOS 10.11.6. > Standalone and distributed HBase setup. >Reporter: Andrey Elenskiy >Assignee: Zheng Hu > Attachments: HBASE-18066.branch-1.v1.patch, > TestGetWithClosestRowBeforeWhenSplit.java > > > During region split/merge there's a brief period of time where doing a "Get" > with "closest_row_before=true" on "hbase:meta" may return empty > "GetResponse.result.cell" field even though parent, splitA and splitB regions > are all in "hbase:meta". Both gohbase (https://github.com/tsuna/gohbase) and > AsyncHBase (https://github.com/OpenTSDB/asynchbase) interprets this as > "TableDoesNotExist", which is returned to the client. > Here's a gist that reproduces this problem: > https://gist.github.com/Timoha/c7a236b768be9220e85e53e1ca53bf96. Note that > you have to use older HTable client (I used 1.2.4) as current versions ignore > `Get.setClosestRowBefore(bool)` option. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022253#comment-16022253 ] Hadoop QA commented on HBASE-18091: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 59s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 16s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 19s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 52s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 27m 6s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 52s {color} | {color:green} hbase-procedure in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 112m 7s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 30s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 155m 44s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869555/HBASE-18091.master.002.patch | | JIRA Issue | HBASE-18091 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux c5806a813a58 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6907/testReport/ | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6907/console | | Powe
[jira] [Updated] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-16196: -- Attachment: HBASE-16196.v9.patch v9: * Update joni & jcodings versions We need to use a version compatible with what jruby ships with because our local versions will show up earlier on the classpath than the bundled jruby one. In particular, we need a jcodings with [isUTF8|https://github.com/jruby/jcodings/commit/6a23e09664e7bc48671ccda06aff0779cfde6d3c] which means a minimum of 1.0.15 (from our current 1.0.8). I didn't want to end up chasing these so went straight to the same version that jruby-complete ships with. It's a bit of a manual hack, maybe there's some way to make maven handle the resolution better or we can ask the jruby folks to relocate their deps. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch, HBASE-16196.v9.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-16196: -- Status: Patch Available (was: Open) > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch, HBASE-16196.v9.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18094) Display the return value of the command append
[ https://issues.apache.org/jira/browse/HBASE-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022200#comment-16022200 ] Guangxu Cheng commented on HBASE-18094: --- Thanks all for reviewing.[~yuzhih...@gmail.com] [~chia7712] [~ashish singhi] Thanks :) > Display the return value of the command append > -- > > Key: HBASE-18094 > URL: https://issues.apache.org/jira/browse/HBASE-18094 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.0.0, 1.3.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18094-branch-1.patch, HBASE-18094-master-v0.patch, > HBASE-18094-master-v1.patch, HBASE-18094-master-v2.patch > > > It is more convenient for users to display the return result of the command > append. > {code} > hbase(main):001:0> append 't1','r1','f1:name','Guangxu' > CURRENT VALUE = Guangxu > Took 0.8820 seconds > > > hbase(main):002:0> append 't1','r1','f1:name',' Cheng' > CURRENT VALUE = Guangxu Cheng > Took 0.0140 seconds > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18027) Replication should respect RPC size limits when batching edits
[ https://issues.apache.org/jira/browse/HBASE-18027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18027: --- Status: Open (was: Patch Available) > Replication should respect RPC size limits when batching edits > -- > > Key: HBASE-18027 > URL: https://issues.apache.org/jira/browse/HBASE-18027 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1, 2.0.0, 1.4.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 1.4.0, 1.3.2 > > Attachments: HBASE-18027.patch, HBASE-18027.patch, HBASE-18027.patch, > HBASE-18027.patch, HBASE-18027.patch > > > In HBaseInterClusterReplicationEndpoint#replicate we try to replicate in > batches. We create N lists. N is the minimum of configured replicator > threads, number of 100-waledit batches, or number of current sinks. Every > pending entry in the replication context is then placed in order by hash of > encoded region name into one of these N lists. Each of the N lists is then > sent all at once in one replication RPC. We do not test if the sum of data in > each N list will exceed RPC size limits. This code presumes each individual > edit is reasonably small. Not checking for aggregate size while assembling > the lists into RPCs is an oversight and can lead to replication failure when > that assumption is violated. > We can fix this by generating as many replication RPC calls as we need to > drain a list, keeping each RPC under limit, instead of assuming the whole > list will fit in one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18027) Replication should respect RPC size limits when batching edits
[ https://issues.apache.org/jira/browse/HBASE-18027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18027: --- Status: Patch Available (was: Open) > Replication should respect RPC size limits when batching edits > -- > > Key: HBASE-18027 > URL: https://issues.apache.org/jira/browse/HBASE-18027 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.1, 2.0.0, 1.4.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 1.4.0, 1.3.2 > > Attachments: HBASE-18027.patch, HBASE-18027.patch, HBASE-18027.patch, > HBASE-18027.patch, HBASE-18027.patch > > > In HBaseInterClusterReplicationEndpoint#replicate we try to replicate in > batches. We create N lists. N is the minimum of configured replicator > threads, number of 100-waledit batches, or number of current sinks. Every > pending entry in the replication context is then placed in order by hash of > encoded region name into one of these N lists. Each of the N lists is then > sent all at once in one replication RPC. We do not test if the sum of data in > each N list will exceed RPC size limits. This code presumes each individual > edit is reasonably small. Not checking for aggregate size while assembling > the lists into RPCs is an oversight and can lead to replication failure when > that assumption is violated. > We can fix this by generating as many replication RPC calls as we need to > drain a list, keeping each RPC under limit, instead of assuming the whole > list will fit in one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022197#comment-16022197 ] Allan Yang commented on HBASE-18095: As far as I know, in branch-1, replication information, like replication peers are stored in ZK, if user want to list or modify replication relationship using ReplicationAdmin, they need zookeeper client > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Purtell > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-16196: -- Status: Open (was: Patch Available) The failure is related to HBASE-18075, I think. Investigating further. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18060) Backport to branch-1 HBASE-9774 HBase native metrics and metric collection for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-18060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022188#comment-16022188 ] Andrew Purtell commented on HBASE-18060: While running the test suite for branch-1 locally I get the below at step maven-remote-resources-plugin:1.5:process (aggregate-licenses) @ hbase-assembly : {noformat} [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (aggregate-licenses) on project hbase-assembly: Error rendering velocity resource. Error invoking method 'get(java.lang.Integer)' in java.util.ArrayList at META-INF/LICENSE.vm[line 1678, column 8]: InvocationTargetException: Index: 0, Size: 0 -> [Help 1] {noformat} Did not investigate why. supplemental-models.xml on branch-1 needs adjustment? TestReplicasClient flaked. Not related to the change as far as I can tell. {noformat} Flaked tests: org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Run 1: TestReplicasClient.testCancelOfMultiGet:585 null Run 2: PASS {noformat} > Backport to branch-1 HBASE-9774 HBase native metrics and metric collection > for coprocessors > --- > > Key: HBASE-18060 > URL: https://issues.apache.org/jira/browse/HBASE-18060 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.4.0, 1.3.2, 1.5.0 >Reporter: Vincent Poon >Assignee: Vincent Poon > Attachments: HBASE-18060.branch-1.3.v1.patch, > HBASE-18060.branch-1.3.v2.patch, HBASE-18060.branch-1.3.v3.patch, > HBASE-18060.branch-1.3.v4.patch, HBASE-18060.branch-1.3.v5.patch, > HBASE-18060.branch-1.v1.patch, HBASE-18060.branch-1.v2.patch, > HBASE-18060.branch-1.v3.patch, HBASE-18060.branch-1.v4.patch, > HBASE-18060.branch-1.v5.patch > > > I'd like to explore backporting HBASE-9774 to branch-1, as the ability for > coprocessors to report custom metrics through HBase is useful for us, and if > we have coprocessors use the native API, a re-write won't be necessary after > an upgrade to 2.0. > The main issues I see so far are: > - the usage of Java 8 language features. Seems we can work around this as > most of it is syntactic sugar. Will need to find a backport for LongAdder > - dropwizard 3.1.2 in Master. branch-1 is still on yammer metrics 2.2. Not > sure if these can coexist just for this feature -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18097) Client can save 1 RPC call for CloseScannerRequest
Karan Mehta created HBASE-18097: --- Summary: Client can save 1 RPC call for CloseScannerRequest Key: HBASE-18097 URL: https://issues.apache.org/jira/browse/HBASE-18097 Project: HBase Issue Type: Improvement Reporter: Karan Mehta Starting version 1.3, HBase automatically closes scanner on server side whenever the results are exhausted and corresponding bits are set in the {{ScanResponse}} proto returned to the client. We can use that info to eliminate the closeScanRequest RPC call, thereby saving 1 RPC per region per scan. This can be particularly useful for tables with more regions. Also, currently the {{ScanResponse}} proto sends out 1 bit per {{Result}} that it has embeds inside the {{CellScanner}} to indicate if it is partial or not. {code} // In every RPC response there should be at most a single partial result. Furthermore, if // there is a partial result, it is guaranteed to be in the last position of the array. {code} According to client, only the last result can be partial, thus this repeated bool can be converted to a bool, thus reducing overhead of serialization and deserialization of the array. This will break wire compatibility therefore this is something to look for in upcoming versions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18038) Rename StoreFile to HStoreFile and add a StoreFile interface for CP
[ https://issues.apache.org/jira/browse/HBASE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022180#comment-16022180 ] Duo Zhang commented on HBASE-18038: --- Any comments [~stack] [~apurtell]? Thanks. > Rename StoreFile to HStoreFile and add a StoreFile interface for CP > --- > > Key: HBASE-18038 > URL: https://issues.apache.org/jira/browse/HBASE-18038 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18038.patch, HBASE-18038-v1.patch, > HBASE-18038-v1.patch, HBASE-18038-v2.patch, HBASE-18038-v3.patch, > HBASE-18038-v3.patch, HBASE-18038-v4.patch, HBASE-18038-v5.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18085) Prevent parallel purge in ObjectPool
[ https://issues.apache.org/jira/browse/HBASE-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022174#comment-16022174 ] Hadoop QA commented on HBASE-18085: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 14s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 20s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 43s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 28m 7s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s {color} | {color:green} hbase-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 40m 56s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 87m 14s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Timed out junit tests | org.apache.hadoop.hbase.master.locking.TestLockManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869551/HBASE-18085.patch | | JIRA Issue | HBASE-18085 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d92f3b4d91f3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6906/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https:
[jira] [Updated] (HBASE-18042) Client Compatibility breaks between versions 1.2 and 1.3
[ https://issues.apache.org/jira/browse/HBASE-18042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18042: -- Status: Patch Available (was: Open) > Client Compatibility breaks between versions 1.2 and 1.3 > > > Key: HBASE-18042 > URL: https://issues.apache.org/jira/browse/HBASE-18042 > Project: HBase > Issue Type: Bug > Components: regionserver, scan >Affects Versions: 1.3.1, 2.0.0, 1.4.0 >Reporter: Karan Mehta >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0, 1.4.0, 1.3.2 > > Attachments: HBASE-18042.patch > > > OpenTSDB uses AsyncHBase as its client, rather than using the traditional > HBase Client. From version 1.2 to 1.3, the {{ClientProtos}} have been > changed. Newer fields are added to {{ScanResponse}} proto. > For a typical Scan request in 1.2, would require caller to make an > OpenScanner Request, GetNextRows Request and a CloseScanner Request, based on > {{more_rows}} boolean field in the {{ScanResponse}} proto. > However, from 1.3, new parameter {{more_results_in_region}} was added, which > limits the results per region. Therefore the client has to now manage sending > all the requests for each region. Further more, if the results are exhausted > from a particular region, the {{ScanResponse}} will set > {{more_results_in_region}} to false, but {{more_results}} can still be true. > Whenever the former is set to false, the {{RegionScanner}} will also be > closed. > OpenTSDB makes an OpenScanner Request and receives all its results in the > first {{ScanResponse}} itself, thus creating a condition as described in > above paragraph. Since {{more_rows}} is true, it will proceed to send next > request at which point the {{RSRpcServices}} will throw > {{UnknownScannerException}}. The protobuf client compatibility is maintained > but expected behavior is modified. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022150#comment-16022150 ] Umesh Agashe commented on HBASE-18091: -- retry > Add API for who currently holds a lock on namespace/ table/ region and log > when state is LOCK_EVENT_WAIT > > > Key: HBASE-18091 > URL: https://issues.apache.org/jira/browse/HBASE-18091 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-18091.master.001.patch, > HBASE-18091.master.002.patch, HBASE-18091.master.002.patch > > > Add API for who currently holds a lock on namespace/ table/ region and log > message when state is LOCK_EVENT_WAIT -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18091: - Attachment: HBASE-18091.master.002.patch > Add API for who currently holds a lock on namespace/ table/ region and log > when state is LOCK_EVENT_WAIT > > > Key: HBASE-18091 > URL: https://issues.apache.org/jira/browse/HBASE-18091 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-18091.master.001.patch, > HBASE-18091.master.002.patch, HBASE-18091.master.002.patch > > > Add API for who currently holds a lock on namespace/ table/ region and log > message when state is LOCK_EVENT_WAIT -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022149#comment-16022149 ] Hadoop QA commented on HBASE-18091: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 19s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 38s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 40s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 47s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 52s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 55m 56s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 2s {color} | {color:green} hbase-procedure in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 205m 36s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m 53s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 295m 56s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.client.TestAsyncRegionAdminApi | | | hadoop.hbase.client.TestAsyncBalancerAdminApi | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.master.TestMasterBalanceThrottling | | Timed out junit tests | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 | | | org.apache.hadoop.hbase.master.balancer.TestFavoredStochasticLoadBalancer | | | org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures | | | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer | | | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd | | | org.apache.hadoop.hbase.master.procedure.TestSplitTableRegionProcedure | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869514/HBASE-18091.master.001.p
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022132#comment-16022132 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #134 (See [https://builds.apache.org/job/HBase-1.2-JDK8/134/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev e5e47fc99a557e7986fb39b2c035384f4137da90) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022113#comment-16022113 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #168 (See [https://builds.apache.org/job/HBase-1.3-JDK7/168/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev fbde5ed6bda13ed6261ec6fefcbcfc835da57fd6) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022114#comment-16022114 ] Hadoop QA commented on HBASE-18091: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 3s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 40s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 38s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 12s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 35s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 16s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 25s {color} | {color:green} hbase-procedure in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 106m 59s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m 38s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 164m 23s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Timed out junit tests | org.apache.hadoop.hbase.filter.TestFilterListOrOperatorWithBlkCnt | | | org.apache.hadoop.hbase.TestAcidGuarantees | | | org.apache.hadoop.hbase.mapred.TestTableMapReduce | | | org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover | | | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd | | | org.apache.hadoop.hbase.master.procedure.TestModifyColumnFamilyProcedure | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.10.1 Server=1.10.1 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869523/HBASE-18091.master.002.patch | | JIRA Issue | HBASE-18091 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux c84e8d76065e 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | mav
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022111#comment-16022111 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #139 (See [https://builds.apache.org/job/HBase-1.2-JDK7/139/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev e5e47fc99a557e7986fb39b2c035384f4137da90) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18085) Prevent parallel purge in ObjectPool
[ https://issues.apache.org/jira/browse/HBASE-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-18085: -- Attachment: HBASE-18085.patch Minor change to trigger UT on hbase-server > Prevent parallel purge in ObjectPool > > > Key: HBASE-18085 > URL: https://issues.apache.org/jira/browse/HBASE-18085 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: e89l05465.st3.jstack, HBASE-18085.patch, > HBASE-18085.patch > > > Parallel purge in ObjectPool is meaningless and will cause contention issue > since {{ReferenceQueue#poll}} has synchronization (source code shown below) > {code} > public Reference poll() { > if (head == null) > return null; > synchronized (lock) { > return reallyPoll(); > } > } > {code} > We observed threads blocking on the purge method while using offheap bucket > cache, and we could easily reproduce this by testing the 100% cache hit case > in bucket cache with enough reading threads. > We propose to add a purgeLock and use tryLock to avoid parallel purge. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022102#comment-16022102 ] Hudson commented on HBASE-18093: FAILURE: Integrated in Jenkins build HBase-1.4 #745 (See [https://builds.apache.org/job/HBase-1.4/745/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev 50708d9524c7575324bf277c8ca0d3d711eb46be) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022088#comment-16022088 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1874 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1874/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev c964b660df7321d23c7229010f717ba1bec5f1ef) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022076#comment-16022076 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1957 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1957/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev c964b660df7321d23c7229010f717ba1bec5f1ef) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15533) Add RSGroup Favored Balancer
[ https://issues.apache.org/jira/browse/HBASE-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022070#comment-16022070 ] Thiruvel Thirumoolan commented on HBASE-15533: -- Unit test failures are unrelated. > Add RSGroup Favored Balancer > > > Key: HBASE-15533 > URL: https://issues.apache.org/jira/browse/HBASE-15533 > Project: HBase > Issue Type: Sub-task > Components: FavoredNodes >Reporter: Francis Liu >Assignee: Thiruvel Thirumoolan > Fix For: 2.0.0 > > Attachments: HBASE-15533.master.001.patch, > HBASE-15533.master.002.patch, HBASE-15533.patch, HBASE-15533.rough.draft.patch > > > HBASE-16942 added favored stochastic load balancer so we can pick and choose > nodes to assign based on the favored nodes and load/locality. The intention > of this jira is to add a group based load balancer on top of the favored > stochastic balancer. This will ensure splits/merges will only use favored > nodes from that group and will inherit from the parents appropriately. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15533) Add RSGroup Favored Balancer
[ https://issues.apache.org/jira/browse/HBASE-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022063#comment-16022063 ] Hadoop QA commented on HBASE-15533: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 6 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 48s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 44s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 31s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 38s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 7s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 31s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 32m 2s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 123m 35s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m 13s {color} | {color:green} hbase-rsgroup in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 42s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 182m 1s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869518/HBASE-15533.master.002.patch | | JIRA Issue | HBASE-15533 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux da916a8028ad 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6904/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/6904/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | htt
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022061#comment-16022061 ] Hadoop QA commented on HBASE-16196: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 30s {color} | {color:blue} Docker mode activated. {color} | | {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 10s {color} | {color:blue} rubocop was not available. {color} | | {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 10s {color} | {color:blue} Ruby-lint was not available. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 4 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 59s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 5s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 22s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 40s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s {color} | {color:blue} Skipped patched modules with no Java source: hbase-resource-bundle . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 17s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 41s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 27m 25s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s {color} | {color:blue} Skipped patched modules with no Java source: hbase-resource-bundle . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 6s {color} | {color:green} hbase-resource-bundle in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 43s {color} | {color:red} hbase-shell in the patch failed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 143m 14s {color} | {color:red} root in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m 5s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 200m 12s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.client.TestShell | | | hadoop.hbase.client.TestShell | \\ \\ || Subsystem || Report/Notes || | Docker
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022032#comment-16022032 ] Hadoop QA commented on HBASE-14614: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 45s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 96 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 55s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 12s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 52s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 3s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 33s {color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 25s {color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 9s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 52s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 27s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 654 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 31m 27s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m 53s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 29s {color} | {color:red} hbase-procedure generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 28s {color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 44s {color} | {color:green} hbase-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s {color} | {color:green} hbase-protocol-shaded in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 48s {color} | {color:green} hbase-procedure in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s {color} | {color:green} hbase-hadoop-compat in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s {color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 27s {color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 136m 41s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s {color} | {color:green} hbase-rsgroup in the patch passed. {color} | | {color:gree
[jira] [Assigned] (HBASE-18092) Removing a peer does not properly clean up the ReplicationSourceManager state and metrics
[ https://issues.apache.org/jira/browse/HBASE-18092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri reassigned HBASE-18092: - Assignee: Ashu Pachauri > Removing a peer does not properly clean up the ReplicationSourceManager state > and metrics > - > > Key: HBASE-18092 > URL: https://issues.apache.org/jira/browse/HBASE-18092 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > Removing a peer does not clean up the associated metrics and state from > walsById map in the ReplicationSourceManager. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18092) Removing a peer does not properly clean up the ReplicationSourceManager state and metrics
[ https://issues.apache.org/jira/browse/HBASE-18092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021989#comment-16021989 ] Ashu Pachauri commented on HBASE-18092: --- Picking this up. > Removing a peer does not properly clean up the ReplicationSourceManager state > and metrics > - > > Key: HBASE-18092 > URL: https://issues.apache.org/jira/browse/HBASE-18092 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > > Removing a peer does not clean up the associated metrics and state from > walsById map in the ReplicationSourceManager. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021976#comment-16021976 ] Dima Spivak commented on HBASE-18041: - And to answer the question of what's needed to move forward, I'd say let's just get a run of the code base with this and see what it gives us. If it looks reasonable, I'll +1 it and we can handle cleanup later. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021975#comment-16021975 ] Dima Spivak commented on HBASE-18041: - Eh, it's probably fine. It's kinda hard to review a big 300 line config file that's full of commented out stuff since I can't tell what's different from the 'default' and what isn't. Personally, I'd have started from a clean {{pylintrc}} and then added stuff based on which errors we think are spurious, but I dunno if it's worth taking the time to go through this exercise if there's not much value to be gained. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18060) Backport to branch-1 HBASE-9774 HBase native metrics and metric collection for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-18060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021974#comment-16021974 ] Andrew Purtell commented on HBASE-18060: Thanks. Let me test this patch locally to see if I can confirm any of the precommit failures. > Backport to branch-1 HBASE-9774 HBase native metrics and metric collection > for coprocessors > --- > > Key: HBASE-18060 > URL: https://issues.apache.org/jira/browse/HBASE-18060 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.4.0, 1.3.2, 1.5.0 >Reporter: Vincent Poon >Assignee: Vincent Poon > Attachments: HBASE-18060.branch-1.3.v1.patch, > HBASE-18060.branch-1.3.v2.patch, HBASE-18060.branch-1.3.v3.patch, > HBASE-18060.branch-1.3.v4.patch, HBASE-18060.branch-1.3.v5.patch, > HBASE-18060.branch-1.v1.patch, HBASE-18060.branch-1.v2.patch, > HBASE-18060.branch-1.v3.patch, HBASE-18060.branch-1.v4.patch, > HBASE-18060.branch-1.v5.patch > > > I'd like to explore backporting HBASE-9774 to branch-1, as the ability for > coprocessors to report custom metrics through HBase is useful for us, and if > we have coprocessors use the native API, a re-write won't be necessary after > an upgrade to 2.0. > The main issues I see so far are: > - the usage of Java 8 language features. Seems we can work around this as > most of it is syntactic sugar. Will need to find a backport for LongAdder > - dropwizard 3.1.2 in Master. branch-1 is still on yammer metrics 2.2. Not > sure if these can coexist just for this feature -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18095: --- Description: Clients are required to connect to ZooKeeper to find the location of the regionserver hosting the meta table region. Site configuration provides the client a list of ZK quorum peers and the client uses an embedded ZK client to query meta location. Timeouts and retry behavior of this embedded ZK client are managed orthogonally to HBase layer settings and in some cases the ZK cannot manage what in theory the HBase client can, i.e. fail fast upon outage or network partition. We should consider new configuration settings that provide a list of well-known master and backup master locations, and with this information the client can contact any of the master processes directly. Any master in either active or passive state will track meta location and respond to requests for it with its cached last known location. If this location is stale, the client can ask again with a flag set that requests the master refresh its location cache and return the up-to-date location. Every client interaction with the cluster thus uses only HBase RPC as transport, with appropriate settings applied to the connection. The configuration toggle that enables this alternative meta location lookup should be false by default. This removes the requirement that HBase clients embed the ZK client and contact the ZK service directly at the beginning of the connection lifecycle. This has several benefits. ZK service need not be exposed to clients, and their potential abuse, yet no benefit ZK provides the HBase server cluster is compromised. Normalizing HBase client and ZK client timeout settings and retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer necessary. And, from [~ghelmling]: There is an additional complication here for token-based authentication. When a delegation token is used for SASL authentication, the client uses the cluster ID obtained from Zookeeper to select the token identifier to use. So there would also need to be some Zookeeper-less, unauthenticated way to obtain the cluster ID as well. was: Clients are required to connect to ZooKeeper to find the location of the regionserver hosting the meta table region. Site configuration provides the client a list of ZK quorum peers and the client uses an embedded ZK client to query meta location. Timeouts and retry behavior of this embedded ZK client are managed orthogonally to HBase layer settings and in some cases the ZK cannot manage what in theory the HBase client can, i.e. fail fast upon outage or network partition. We should consider new configuration settings that provide a list of well-known master and backup master locations, and with this information the client can contact any of the master processes directly. Any master in either active or passive state will track meta location and respond to requests for it with its cached last known location. If this location is stale, the client can ask again with a flag set that requests the master refresh its location cache and return the up-to-date location. Every client interaction with the cluster thus uses only HBase RPC as transport, with appropriate settings applied to the connection. The configuration toggle that enables this alternative meta location lookup should be false by default. This removes the requirement that HBase clients embed the ZK client and contact the ZK service directly at the beginning of the connection lifecycle. This has several benefits. ZK service need not be exposed to clients, and their potential abuse, yet no benefit ZK provides the HBase server cluster is compromised. Normalizing HBase client and ZK client timeout settings and retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer necessary. > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Purtell > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021928#comment-16021928 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #873 (See [https://builds.apache.org/job/HBase-1.2-IT/873/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev e5e47fc99a557e7986fb39b2c035384f4137da90) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021921#comment-16021921 ] Hadoop QA commented on HBASE-18096: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 35s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 33s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 21s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 28m 49s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 10s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 116m 3s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 174m 7s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869509/HBASE-18096-BM-0001.patch | | JIRA Issue | HBASE-18096 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ea791253fd48 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6900/testReport/ | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6900/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Balazs Mesz
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021916#comment-16021916 ] Hudson commented on HBASE-18093: FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #182 (See [https://builds.apache.org/job/HBase-1.3-JDK8/182/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev fbde5ed6bda13ed6261ec6fefcbcfc835da57fd6) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15903) Delete Object
[ https://issues.apache.org/jira/browse/HBASE-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15903: --- Attachment: 15903.v2.txt > Delete Object > - > > Key: HBASE-15903 > URL: https://issues.apache.org/jira/browse/HBASE-15903 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Ted Yu > Attachments: 15903.v2.txt, HBASE-15903.HBASE-14850.v1.patch > > > Patch for creating Delete objects. These Delete objects are used by the Table > implementation to delete rowkey from a table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021911#comment-16021911 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #50 (See [https://builds.apache.org/job/HBase-1.3-IT/50/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev fbde5ed6bda13ed6261ec6fefcbcfc835da57fd6) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021904#comment-16021904 ] Mike Drob commented on HBASE-16196: --- Ok, in that case, it's probably fine to go in. I don't know that it will apply cleanly, but it shouldn't be too bad. The license part is probably the worst of it. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18087) Fix unit tests for TestTableFavoredNodes
[ https://issues.apache.org/jira/browse/HBASE-18087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021891#comment-16021891 ] Hudson commented on HBASE-18087: FAILURE: Integrated in Jenkins build HBase-HBASE-14614 #247 (See [https://builds.apache.org/job/HBase-HBASE-14614/247/]) HBASE-18087 Fix unit tests in TestTableFavoredNodes (stack: rev 4143c0176ffd46030f89aad5730b72f8f7627ae6) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedureSyncWait.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionFileSystem.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentOnRSCrash.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterOperationsForRegionReplicas.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteColumnFamilyProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestIncrementTimeRange.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestAddColumnFamilyProcedure.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSplitThread.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureInMemoryChore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableNamespaceManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyColumnFamilyProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSplitOrMergeStatus.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java * (delete) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionStates.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineNamespaceProcedure.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestMergeTableRegionsProcedure.java * (edit) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManagerSourceImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckOneRS.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/NoopProcedureStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsckTwoRS.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureScheduler.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterDDLOperationHelper.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStateStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreadedWithBasicCompaction.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterWalManager.java * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon * (edit) hbase-server/src/test/java/org/apache/hadoop/
[jira] [Commented] (HBASE-18094) Display the return value of the command append
[ https://issues.apache.org/jira/browse/HBASE-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021890#comment-16021890 ] Hudson commented on HBASE-18094: FAILURE: Integrated in Jenkins build HBase-HBASE-14614 #247 (See [https://builds.apache.org/job/HBase-HBASE-14614/247/]) HBASE-18094 Display the return value of the command append (tedyu: rev ebe92c8fb3153367531aac3cf2b60d65f782083d) * (edit) hbase-shell/src/test/ruby/hbase/table_test.rb * (edit) hbase-shell/src/main/ruby/hbase/table.rb * (edit) hbase-shell/src/main/ruby/shell/commands/append.rb > Display the return value of the command append > -- > > Key: HBASE-18094 > URL: https://issues.apache.org/jira/browse/HBASE-18094 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.0.0, 1.3.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18094-branch-1.patch, HBASE-18094-master-v0.patch, > HBASE-18094-master-v1.patch, HBASE-18094-master-v2.patch > > > It is more convenient for users to display the return result of the command > append. > {code} > hbase(main):001:0> append 't1','r1','f1:name','Guangxu' > CURRENT VALUE = Guangxu > Took 0.8820 seconds > > > hbase(main):002:0> append 't1','r1','f1:name',' Cheng' > CURRENT VALUE = Guangxu Cheng > Took 0.0140 seconds > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18094) Display the return value of the command append
[ https://issues.apache.org/jira/browse/HBASE-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021874#comment-16021874 ] Hudson commented on HBASE-18094: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3064 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3064/]) HBASE-18094 Display the return value of the command append (tedyu: rev ebe92c8fb3153367531aac3cf2b60d65f782083d) * (edit) hbase-shell/src/test/ruby/hbase/table_test.rb * (edit) hbase-shell/src/main/ruby/hbase/table.rb * (edit) hbase-shell/src/main/ruby/shell/commands/append.rb > Display the return value of the command append > -- > > Key: HBASE-18094 > URL: https://issues.apache.org/jira/browse/HBASE-18094 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.0.0, 1.3.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18094-branch-1.patch, HBASE-18094-master-v0.patch, > HBASE-18094-master-v1.patch, HBASE-18094-master-v2.patch > > > It is more convenient for users to display the return result of the command > append. > {code} > hbase(main):001:0> append 't1','r1','f1:name','Guangxu' > CURRENT VALUE = Guangxu > Took 0.8820 seconds > > > hbase(main):002:0> append 't1','r1','f1:name',' Cheng' > CURRENT VALUE = Guangxu Cheng > Took 0.0140 seconds > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021873#comment-16021873 ] Hudson commented on HBASE-18077: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3064 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3064/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev 9e7b0c1a4f24feeecb55498d7926596af9fc284a) * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021872#comment-16021872 ] Hudson commented on HBASE-18093: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3064 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3064/]) HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to (syuanjiangdev: rev 1d0295f4e290ce9f0bcc30df9398cd81d75c4d50) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021855#comment-16021855 ] stack commented on HBASE-11544: --- [~karanmehta93] Inside the one return on a Scan? I presumed only the last row could be partial. Repeated in pb seems odd. Let me ping [~jonathan.lawlor] ... he is probably not watching this... > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > -- > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jonathan Lawlor >Priority: Critical > Fix For: 2.0.0, 1.1.0 > > Attachments: Allocation_Hot_Spots.html, gc.j.png, > HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch, > HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, > HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, > HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, > HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, > HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, hits.j.png, h.png, > mean.png, m.png, net.j.png, q (2).png > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18091: - Attachment: HBASE-18091.master.002.patch > Add API for who currently holds a lock on namespace/ table/ region and log > when state is LOCK_EVENT_WAIT > > > Key: HBASE-18091 > URL: https://issues.apache.org/jira/browse/HBASE-18091 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-18091.master.001.patch, > HBASE-18091.master.002.patch > > > Add API for who currently holds a lock on namespace/ table/ region and log > message when state is LOCK_EVENT_WAIT -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021849#comment-16021849 ] Sean Busbey commented on HBASE-16196: - I dunno. I used to hold that position, but then essentially [no one on users@hbase had feedback on changing from Ruby 1.8|https://s.apache.org/DXQK]. After that I had much less concern that folks were building on top of the shell. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020554#comment-16020554 ] Karan Mehta edited comment on HBASE-11544 at 5/23/17 8:42 PM: -- [~jonathan.lawlor] [~stack] Can the server return multiple partial rows? If yes, why does client side code assume that only the last result is partial in {{ClientScanner}} ? If not, why do we have a repeated boolean value for {{partial_flag_per_result}} in {{Client.protos}} ? was (Author: karanmehta93): [~jonathan.lawlor] Can the server return multiple partial rows? If yes, why does client side code assume that only the last result is partial in {{ClientScanner}} ? If not, why do we have a repeated boolean value for {{partial_flag_per_result}} in {{Client.protos}} ? > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > -- > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jonathan Lawlor >Priority: Critical > Fix For: 2.0.0, 1.1.0 > > Attachments: Allocation_Hot_Spots.html, gc.j.png, > HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch, > HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, > HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, > HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, > HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, > HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, hits.j.png, h.png, > mean.png, m.png, net.j.png, q (2).png > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021793#comment-16021793 ] Mike Drob commented on HBASE-16196: --- Downstream folks - Ruby 1.8 to Ruby 2.x is a large enough change that I'd be nervous springing it on somebody in a minor release. At least, I wouldn't want to do it without additional guarantees like HBASE-13583, but I think that one also gets easier to implement after we pick up a modern JRuby version since there were known bugs in an older jrubyc. Kind of a chicken and egg situation, where I think we want to get some burn in on new branches before taking it back to stable. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18094) Display the return value of the command append
[ https://issues.apache.org/jira/browse/HBASE-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021783#comment-16021783 ] Hudson commented on HBASE-18094: FAILURE: Integrated in Jenkins build HBase-1.4 #744 (See [https://builds.apache.org/job/HBase-1.4/744/]) HBASE-18094 Display the return value of the command append (tedyu: rev 8c313d5be46a02c212d38a1ee782cbff570f007f) * (edit) hbase-shell/src/test/ruby/hbase/table_test.rb * (edit) hbase-shell/src/main/ruby/shell/commands/append.rb * (edit) hbase-shell/src/main/ruby/hbase/table.rb > Display the return value of the command append > -- > > Key: HBASE-18094 > URL: https://issues.apache.org/jira/browse/HBASE-18094 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.0.0, 1.3.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18094-branch-1.patch, HBASE-18094-master-v0.patch, > HBASE-18094-master-v1.patch, HBASE-18094-master-v2.patch > > > It is more convenient for users to display the return result of the command > append. > {code} > hbase(main):001:0> append 't1','r1','f1:name','Guangxu' > CURRENT VALUE = Guangxu > Took 0.8820 seconds > > > hbase(main):002:0> append 't1','r1','f1:name',' Cheng' > CURRENT VALUE = Guangxu Cheng > Took 0.0140 seconds > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-18093: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.1.11 1.3.2 1.2.6 1.4.0 2.0.0 Status: Resolved (was: Patch Available) > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021761#comment-16021761 ] Sean Busbey commented on HBASE-16196: - bq. Just noticed that this is also set for 1.5.0, I don't think that is a great idea since the changes are pretty severe from a backend point of view. Is the concern here the difficulty in the changes in our code base? Or the impact on downstream folks? > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15533) Add RSGroup Favored Balancer
[ https://issues.apache.org/jira/browse/HBASE-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-15533: - Attachment: HBASE-15533.master.002.patch > Add RSGroup Favored Balancer > > > Key: HBASE-15533 > URL: https://issues.apache.org/jira/browse/HBASE-15533 > Project: HBase > Issue Type: Sub-task > Components: FavoredNodes >Reporter: Francis Liu >Assignee: Thiruvel Thirumoolan > Fix For: 2.0.0 > > Attachments: HBASE-15533.master.001.patch, > HBASE-15533.master.002.patch, HBASE-15533.patch, HBASE-15533.rough.draft.patch > > > HBASE-16942 added favored stochastic load balancer so we can pick and choose > nodes to assign based on the favored nodes and load/locality. The intention > of this jira is to add a group based load balancer on top of the favored > stochastic balancer. This will ensure splits/merges will only use favored > nodes from that group and will inherit from the parents appropriately. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18093) Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good
[ https://issues.apache.org/jira/browse/HBASE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021752#comment-16021752 ] Stephen Yuan Jiang commented on HBASE-18093: Test failures in branch-1 run are not related to the change. > Overloading the meaning of 'enabled' in Quota Manager to indicate either > quota disabled or quota manager not ready is not good > -- > > Key: HBASE-18093 > URL: https://issues.apache.org/jira/browse/HBASE-18093 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.1.10 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang >Priority: Minor > Attachments: HBASE-18093.v1-branch-1.patch, > HBASE-18093.v1-master.patch, HBASE-18093.v2-master.patch, > HBASE-18093.v3-master.patch > > > In MasterQuotaManager, a member 'enabled' is used to indicate either quota > feature is disabled or quota manager is not fully initialized. This would > create confusion whether caller should wait for quota manager to be > initialized or change configuration to enable quota. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-16196: -- Status: Patch Available (was: In Progress) > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021735#comment-16021735 ] Mike Drob commented on HBASE-16196: --- Just noticed that this is also set for 1.5.0, I don't think that is a great idea since the changes are pretty severe from a backend point of view. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-16196: -- Attachment: HBASE-16196.v8.patch v8: * Updates Ruby License to point at BSD-2 instead of GPL as the alternative. My earlier concern about NOTICE files is not an issue because most of the libraries that jruby bundles that are ASLv2 don't actually include notice files, so we don't need to propagate anything. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Mike Drob >Priority: Critical > Fix For: 2.0.0, 1.5.0 > > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch, hbase-16196.v2.branch-1.patch, > hbase-16196.v3.branch-1.patch, hbase-16196.v4.branch-1.patch, > HBASE-16196.v5.patch, HBASE-16196.v6.patch, HBASE-16196.v7.patch, > HBASE-16196.v8.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021725#comment-16021725 ] Hudson commented on HBASE-18077: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #181 (See [https://builds.apache.org/job/HBase-1.3-JDK8/181/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev d1b1eab1036c3f69d06d573f52028aea00cdac5b) * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18091: - Attachment: HBASE-18091.master.001.patch > Add API for who currently holds a lock on namespace/ table/ region and log > when state is LOCK_EVENT_WAIT > > > Key: HBASE-18091 > URL: https://issues.apache.org/jira/browse/HBASE-18091 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-18091.master.001.patch > > > Add API for who currently holds a lock on namespace/ table/ region and log > message when state is LOCK_EVENT_WAIT -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18091) Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT
[ https://issues.apache.org/jira/browse/HBASE-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18091: - Status: Patch Available (was: In Progress) > Add API for who currently holds a lock on namespace/ table/ region and log > when state is LOCK_EVENT_WAIT > > > Key: HBASE-18091 > URL: https://issues.apache.org/jira/browse/HBASE-18091 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe > Fix For: 2.0.0 > > Attachments: HBASE-18091.master.001.patch > > > Add API for who currently holds a lock on namespace/ table/ region and log > message when state is LOCK_EVENT_WAIT -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: (was: HBASE-14614.master.037.patch) > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 14614.040.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.013.patch, > HBASE-14614.master.014.patch, HBASE-14614.master.015.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.022.patch, HBASE-14614.master.023.patch, > HBASE-14614.master.024.patch, HBASE-14614.master.025.patch, > HBASE-14614.master.026.patch, HBASE-14614.master.027.patch, > HBASE-14614.master.028.patch, HBASE-14614.master.029.patch, > HBASE-14614.master.030.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: (was: HBASE-14614.master.036.patch) > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 14614.040.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.013.patch, > HBASE-14614.master.014.patch, HBASE-14614.master.015.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.022.patch, HBASE-14614.master.023.patch, > HBASE-14614.master.024.patch, HBASE-14614.master.025.patch, > HBASE-14614.master.026.patch, HBASE-14614.master.027.patch, > HBASE-14614.master.028.patch, HBASE-14614.master.029.patch, > HBASE-14614.master.030.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: (was: HBASE-14614.master.033.patch) > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 14614.040.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.013.patch, > HBASE-14614.master.014.patch, HBASE-14614.master.015.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.022.patch, HBASE-14614.master.023.patch, > HBASE-14614.master.024.patch, HBASE-14614.master.025.patch, > HBASE-14614.master.026.patch, HBASE-14614.master.027.patch, > HBASE-14614.master.028.patch, HBASE-14614.master.029.patch, > HBASE-14614.master.030.patch, HBASE-14614.master.036.patch, > HBASE-14614.master.037.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: (was: HBASE-14614.master.034.patch) > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 14614.040.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.013.patch, > HBASE-14614.master.014.patch, HBASE-14614.master.015.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.022.patch, HBASE-14614.master.023.patch, > HBASE-14614.master.024.patch, HBASE-14614.master.025.patch, > HBASE-14614.master.026.patch, HBASE-14614.master.027.patch, > HBASE-14614.master.028.patch, HBASE-14614.master.029.patch, > HBASE-14614.master.030.patch, HBASE-14614.master.036.patch, > HBASE-14614.master.037.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: (was: HBASE-14614.master.035.patch) > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: 14614.040.patch, HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.012.patch, HBASE-14614.master.013.patch, > HBASE-14614.master.014.patch, HBASE-14614.master.015.patch, > HBASE-14614.master.017.patch, HBASE-14614.master.018.patch, > HBASE-14614.master.019.patch, HBASE-14614.master.020.patch, > HBASE-14614.master.022.patch, HBASE-14614.master.023.patch, > HBASE-14614.master.024.patch, HBASE-14614.master.025.patch, > HBASE-14614.master.026.patch, HBASE-14614.master.027.patch, > HBASE-14614.master.028.patch, HBASE-14614.master.029.patch, > HBASE-14614.master.030.patch, HBASE-14614.master.036.patch, > HBASE-14614.master.037.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021695#comment-16021695 ] Hudson commented on HBASE-18077: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #138 (See [https://builds.apache.org/job/HBase-1.2-JDK7/138/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev 6ee7a4932ab0a24956168d6482c30712a247a17a) * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021693#comment-16021693 ] Hudson commented on HBASE-18077: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #167 (See [https://builds.apache.org/job/HBase-1.3-JDK7/167/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev d1b1eab1036c3f69d06d573f52028aea00cdac5b) * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021692#comment-16021692 ] Hudson commented on HBASE-18077: FAILURE: Integrated in Jenkins build HBase-1.2-JDK8 #133 (See [https://builds.apache.org/job/HBase-1.2-JDK8/133/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev 6ee7a4932ab0a24956168d6482c30712a247a17a) * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18005) read replica: handle the case that region server hosting both primary replica and meta region is down
[ https://issues.apache.org/jira/browse/HBASE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021672#comment-16021672 ] Hadoop QA commented on HBASE-18005: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 9s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 43s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 6s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 28m 41s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 6s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 31s {color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 114m 19s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 163m 25s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12869483/HBASE-18005-master-005.patch | | JIRA Issue | HBASE-18005 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a217079f21f6 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / ebe92c8 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6898/testReport/ | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6898/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > read replica: handle the case
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021661#comment-16021661 ] Gary Helmling commented on HBASE-18095: --- There is an additional complication here for token-based authentication. When a delegation token is used for SASL authentication, the client uses the cluster ID obtained from Zookeeper to select the token identifier to use. So there would also need to be some Zookeeper-less, unauthenticated way to obtain the cluster ID as well. > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Purtell > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-18096: Fix Version/s: 1.3.2 1.2.6 1.4.0 2.0.0 > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2 > > Attachments: HBASE-18096-BM-0001.patch > > > HFileUtil should be package private and should have > @InterfaceAudience.Private annotation. This class was introduced in > HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18077) Update JUnit license to EPL from CPL
[ https://issues.apache.org/jira/browse/HBASE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021637#comment-16021637 ] Hudson commented on HBASE-18077: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1873 (See [https://builds.apache.org/job/HBase-1.1-JDK7/1873/]) HBASE-18077 Update JUnit licensing to use EPL (busbey: rev dbd72f9b0c187dc1d9296d5f8b58c4a75a7fedb9) * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Update JUnit license to EPL from CPL > > > Key: HBASE-18077 > URL: https://issues.apache.org/jira/browse/HBASE-18077 > Project: HBase > Issue Type: Bug > Components: build, community >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2, 1.1.11 > > Attachments: HBASE-18077.patch > > > JUnit is listed as using the CPL, but it actually uses the EPL. We need to > update our LICENSE file in the shaded jars. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18096: -- Hadoop Flags: Reviewed Affects Version/s: 2.0.0 Status: Patch Available (was: Open) +1 > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros > Attachments: HBASE-18096-BM-0001.patch > > > HFileUtil should be package private and should have > @InterfaceAudience.Private annotation. This class was introduced in > HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18096: -- Priority: Trivial (was: Major) > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Trivial > Attachments: HBASE-18096-BM-0001.patch > > > HFileUtil should be package private and should have > @InterfaceAudience.Private annotation. This class was introduced in > HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
[ https://issues.apache.org/jira/browse/HBASE-18096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-18096: Attachment: HBASE-18096-BM-0001.patch > Limit HFileUtil visibility and add missing annotations > -- > > Key: HBASE-18096 > URL: https://issues.apache.org/jira/browse/HBASE-18096 > Project: HBase > Issue Type: Task >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros > Attachments: HBASE-18096-BM-0001.patch > > > HFileUtil should be package private and should have > @InterfaceAudience.Private annotation. This class was introduced in > HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18096) Limit HFileUtil visibility and add missing annotations
Balazs Meszaros created HBASE-18096: --- Summary: Limit HFileUtil visibility and add missing annotations Key: HBASE-18096 URL: https://issues.apache.org/jira/browse/HBASE-18096 Project: HBase Issue Type: Task Reporter: Balazs Meszaros Assignee: Balazs Meszaros HFileUtil should be package private and should have @InterfaceAudience.Private annotation. This class was introduced in HBASE-17501. -- This message was sent by Atlassian JIRA (v6.3.15#6346)