[jira] [Created] (HBASE-13394) Failed to recreate a table when quota is enabled
Y. SREENIVASULU REDDY created HBASE-13394:
-
Summary: Failed to recreate a table when quota is enabled
Key: HBASE-13394
URL: https://issues.apache.org/jira/browse/HBASE-13394
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashish Singhi
Fix For: 2.0.0
Steps to reproduce.
Enable quota by setting {{hbase.quota.enabled}} to true
Create a table say with name 't1', make sure the creation fails after adding
this table entry into namespace quota cache.
Now correct the failure and recreate the table 't1'. It fails with below
exception.
{noformat}
2015-04-02 14:23:53,729 | ERROR | FifoRpcScheduler.handler1-thread-23 |
Unexpected throwable object |
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2154)
java.lang.IllegalStateException: Table already in the cache t1
at
org.apache.hadoop.hbase.namespace.NamespaceTableAndRegionInfo.addTable(NamespaceTableAndRegionInfo.java:97)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.addTable(NamespaceStateManager.java:171)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceTableCount(NamespaceStateManager.java:147)
at
org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToCreateTable(NamespaceAuditor.java:76)
at
org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkNamespaceTableAndRegionQuota(MasterQuotaManager.java:344)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1781)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1818)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42273)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2116)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{noformat}
P.S: Line numbers may not be in sync.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Please welcome new HBase committer Jing Chen (Jerry) He
Kudos, Jerry! On Thu, Apr 2, 2015 at 10:49 PM, Mikhail Antonov olorinb...@gmail.com wrote: Congrats Jerry! -Mikhail On Thu, Apr 2, 2015 at 9:21 PM, Pankaj kr pankaj...@huawei.com wrote: Congrats Jerry..!! :) -Original Message- From: Andrew Purtell [mailto:apurt...@apache.org] Sent: 02 April 2015 01:53 To: dev@hbase.apache.org; u...@hbase.apache.org Subject: Please welcome new HBase committer Jing Chen (Jerry) He On behalf of the Apache HBase PMC, I am pleased to announce that Jerry He has accepted the PMC's invitation to become a committer on the project. We appreciate all of Jerry's hard work and generous contributions thus far, and look forward to his continued involvement. Congratulations and welcome, Jerry! -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Thanks, Michael Antonov
[jira] [Created] (HBASE-13395) Remove HTableInterface
Mikhail Antonov created HBASE-13395: --- Summary: Remove HTableInterface Key: HBASE-13395 URL: https://issues.apache.org/jira/browse/HBASE-13395 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Fix For: 2.0.0 This class is marked as deprecated, probably can remove it, and if any methods from this specific class are in active use, need to decide what to do on callers' side. Should be able to replace with just Table interface usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13396) Cleanup unclosed writers in later writer rolling
Liu Shaohui created HBASE-13396: --- Summary: Cleanup unclosed writers in later writer rolling Key: HBASE-13396 URL: https://issues.apache.org/jira/browse/HBASE-13396 Project: HBase Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Currently, the default value of hbase.regionserver.logroll.errors.tolerated is 2, which means regionserver can tolerate two continuous failures of closing writers at most. Temporary problems of network or namenode may cause those failures. After those failures, the hdfs clients in RS may continue to renew the lease of the hlog of the writer and the namenode will not help to recover the lease of this hlog. So the last block of this hlog will be RBW(replica being written) state until the regionserver is down. Blocks in this state will block the datanode decommission and other operations in HDFS. So I think we need a mechanism to clean up those unclosed writers afterwards. A simple solution is to record those unclosed writers and attempt to close these writers until success. Discussions and suggestions are welcomed~ Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13401) Snapshot export to S3 fails with jets3t error - The Content-MD5 you specified did not match what we received.
Joseph Reid created HBASE-13401:
---
Summary: Snapshot export to S3 fails with jets3t error - The
Content-MD5 you specified did not match what we received.
Key: HBASE-13401
URL: https://issues.apache.org/jira/browse/HBASE-13401
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 0.98.0
Reporter: Joseph Reid
We're running into issues exporting snapshots of large tables to Amazon S3.
The snapshot completes successfully, but the snapshot export job runs into
errors with jets3t when we attempt to export to S3.
Error snippet, from job log:
{code}
2015-04-03 16:59:16,425 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_08_1, Status : FAILED
Error: org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 Error Message. -- ResponseCode: 400,
ResponseStatus: Bad Request, XML Error Message: ?xml version=1.0
encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe Content-MD5 you
specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleS3ServiceException(Jets3tNativeFileSystemStore.java:405)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at org.apache.hadoop.fs.s3native.$Proxy19.storeFile(Unknown Source)
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.close(NativeS3FileSystem.java:221)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:103)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:200)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:140)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.jets3t.service.S3ServiceException: S3 Error Message. --
ResponseCode: 400, ResponseStatus: Bad Request, XML Error Message: ?xml
version=1.0 encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe
Content-MD5 you specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at org.jets3t.service.S3Service.putObject(S3Service.java:2267)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:113)
... 21 more
2015-04-03 17:03:50,613 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_10_1, Status : FAILED
AttemptID:attempt_1426532296228_55454_m_10_1 Timed out after 300 secs
{\code}
We've verified that exports to other clusters from these same snapshots work
fine. Thus the issue appears to lie within the snapshot export utility,
jets3t, and S3.
The Content-MD5 you specified did not match what we received seems to
indicate that the snapshot changed between when the upload started and the
error. Can that be?
Related to:
[Discussion on jets3t user
group,.|https://groups.google.com/forum/#!topic/jets3t-users/Bg2qh7OdE2U]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Disable HBase System tables
The acl table and label tables are currently guarded in the
AccessController or VisibilityController. As Srikanth mentioned, it is
related to HBASE-13336.
Should we avoid duplicate checking and make the logic/responsibility clear?
The problem probably also exist for the delete/alter/modify table
operations.
Jerry
On Fri, Apr 3, 2015 at 11:09 AM, Stephen Jiang syuanjiang...@gmail.com
wrote:
yes, my proposed change is that do the 'tableName.isSystemTable ()' check,
instead of 'tableName.equals(TableName.META_TABLE_NAME)' - this check would
include all tables inside the hbase namespace.
Thanks
Stephen
By the way, we should rename the 'hbase' namespace to 'system' namespace to
make it clearer :-). Now is too late :-(.
On Thu, Apr 2, 2015 at 5:33 PM, Sean Busbey bus...@cloudera.com wrote:
+1 disabling the visibility label table is going to be a bad time.
Maybe just disallow for the whole hbase namespace?
--
Sean
On Apr 2, 2015 5:54 PM, Stephen Jiang syuanjiang...@gmail.com wrote:
In disable table, we specifically check whether it is a META table; if
a
table is a META table, we disallow the table disable. However, I think
other system tables should have the same treatment (is it possible
that a
namespace table is disable and the system is still functional without
issue?).
if(tableName.equals(TableName.META_TABLE_NAME)) {
throw new ConstraintException(Cannot disable catalog table);
}
I want to extend the disable-not-allowed treatment to all system
tables,
please let me know if you disagree.
Thanks
Stephen
[jira] [Created] (HBASE-13398) Document HBase Quota
Ashish Singhi created HBASE-13398: - Summary: Document HBase Quota Key: HBASE-13398 URL: https://issues.apache.org/jira/browse/HBASE-13398 Project: HBase Issue Type: Bug Components: documentation Reporter: Ashish Singhi As part of this we should document HBASE-11598 and HBASE-8410 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Disable HBase System tables
yes, my proposed change is that do the 'tableName.isSystemTable ()' check,
instead of 'tableName.equals(TableName.META_TABLE_NAME)' - this check would
include all tables inside the hbase namespace.
Thanks
Stephen
By the way, we should rename the 'hbase' namespace to 'system' namespace to
make it clearer :-). Now is too late :-(.
On Thu, Apr 2, 2015 at 5:33 PM, Sean Busbey bus...@cloudera.com wrote:
+1 disabling the visibility label table is going to be a bad time.
Maybe just disallow for the whole hbase namespace?
--
Sean
On Apr 2, 2015 5:54 PM, Stephen Jiang syuanjiang...@gmail.com wrote:
In disable table, we specifically check whether it is a META table; if a
table is a META table, we disallow the table disable. However, I think
other system tables should have the same treatment (is it possible that a
namespace table is disable and the system is still functional without
issue?).
if(tableName.equals(TableName.META_TABLE_NAME)) {
throw new ConstraintException(Cannot disable catalog table);
}
I want to extend the disable-not-allowed treatment to all system tables,
please let me know if you disagree.
Thanks
Stephen
[jira] [Resolved] (HBASE-13399) HBase Snapshot export to S3 fails with Content-MD5 errors.
[
https://issues.apache.org/jira/browse/HBASE-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-13399.
Resolution: Duplicate
HBase Snapshot export to S3 fails with Content-MD5 errors.
--
Key: HBASE-13399
URL: https://issues.apache.org/jira/browse/HBASE-13399
Project: HBase
Issue Type: Bug
Components: Filesystem Integration, hadoop2
Affects Versions: 0.98.0
Environment: CentOS 6.5, Hortonworks Data Platform 2.1.2, Hadoop 2.4.0
Reporter: Joseph Reid
We're running into issues exporting snapshots of large tables to Amazon S3.
The snapshot completes successfully, but the snapshot export job runs into
errors with jets3t when we attempt to export to S3.
Error snippet, from job log:
{code}
2015-04-03 16:59:16,425 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_08_1, Status : FAILED
Error: org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 Error Message. -- ResponseCode:
400, ResponseStatus: Bad Request, XML Error Message: ?xml version=1.0
encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe Content-MD5 you
specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleS3ServiceException(Jets3tNativeFileSystemStore.java:405)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at org.apache.hadoop.fs.s3native.$Proxy19.storeFile(Unknown Source)
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.close(NativeS3FileSystem.java:221)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:103)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:200)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:140)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.jets3t.service.S3ServiceException: S3 Error Message. --
ResponseCode: 400, ResponseStatus: Bad Request, XML Error Message: ?xml
version=1.0 encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe
Content-MD5 you specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at org.jets3t.service.S3Service.putObject(S3Service.java:2267)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:113)
... 21 more
2015-04-03 17:03:50,613 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_10_1, Status : FAILED
AttemptID:attempt_1426532296228_55454_m_10_1 Timed out after 300 secs
{\code}
We've verified that exports to other clusters from these same snapshots work
fine. Thus the issue appears to lie within the snapshot export utility,
jets3t, and S3.
The Content-MD5 you specified did not match what we received seems to
indicate that the snapshot changed between when the upload started and the
error. Can that be?
Related to:
[Discussion on jets3t user
[jira] [Created] (HBASE-13399) HBase Snapshot export to S3 fails with Content-MD5 errors.
Joseph Reid created HBASE-13399:
---
Summary: HBase Snapshot export to S3 fails with Content-MD5 errors.
Key: HBASE-13399
URL: https://issues.apache.org/jira/browse/HBASE-13399
Project: HBase
Issue Type: Bug
Components: Filesystem Integration, hadoop2
Affects Versions: 0.98.0
Environment: CentOS 6.5, Hortonworks Data Platform 2.1.2, Hadoop 2.4.0
Reporter: Joseph Reid
We're running into issues exporting snapshots of large tables to Amazon S3.
The snapshot completes successfully, but the snapshot export job runs into
errors with jets3t when we attempt to export to S3.
Error snippet, from job log:
{code}
2015-04-03 16:59:16,425 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_08_1, Status : FAILED
Error: org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 Error Message. -- ResponseCode: 400,
ResponseStatus: Bad Request, XML Error Message: ?xml version=1.0
encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe Content-MD5 you
specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleS3ServiceException(Jets3tNativeFileSystemStore.java:405)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at org.apache.hadoop.fs.s3native.$Proxy19.storeFile(Unknown Source)
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.close(NativeS3FileSystem.java:221)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:103)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:200)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:140)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.jets3t.service.S3ServiceException: S3 Error Message. --
ResponseCode: 400, ResponseStatus: Bad Request, XML Error Message: ?xml
version=1.0 encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe
Content-MD5 you specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at org.jets3t.service.S3Service.putObject(S3Service.java:2267)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:113)
... 21 more
2015-04-03 17:03:50,613 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_10_1, Status : FAILED
AttemptID:attempt_1426532296228_55454_m_10_1 Timed out after 300 secs
{\code}
We've verified that exports to other clusters from these same snapshots work
fine. Thus the issue appears to lie within the snapshot export utility,
jets3t, and S3.
The Content-MD5 you specified did not match what we received seems to
indicate that the snapshot changed between when the upload started and the
error. Can that be?
Related to:
[Discussion on jets3t user
group,.|https://groups.google.com/forum/#!topic/jets3t-users/Bg2qh7OdE2U]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13400) HBase Snapshot export to S3 fails with Content-MD5 errors.
Joseph Reid created HBASE-13400:
---
Summary: HBase Snapshot export to S3 fails with Content-MD5 errors.
Key: HBASE-13400
URL: https://issues.apache.org/jira/browse/HBASE-13400
Project: HBase
Issue Type: Bug
Components: Filesystem Integration, hadoop2
Affects Versions: 0.98.0
Environment: CentOS 6.5, Hortonworks Data Platform 2.1.2, Hadoop 2.4.0
Reporter: Joseph Reid
We're running into issues exporting snapshots of large tables to Amazon S3.
The snapshot completes successfully, but the snapshot export job runs into
errors with jets3t when we attempt to export to S3.
Error snippet, from job log:
{code}
2015-04-03 16:59:16,425 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_08_1, Status : FAILED
Error: org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 Error Message. -- ResponseCode: 400,
ResponseStatus: Bad Request, XML Error Message: ?xml version=1.0
encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe Content-MD5 you
specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleS3ServiceException(Jets3tNativeFileSystemStore.java:405)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at org.apache.hadoop.fs.s3native.$Proxy19.storeFile(Unknown Source)
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.close(NativeS3FileSystem.java:221)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:103)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:200)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:140)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.jets3t.service.S3ServiceException: S3 Error Message. --
ResponseCode: 400, ResponseStatus: Bad Request, XML Error Message: ?xml
version=1.0 encoding=UTF-8?ErrorCodeBadDigest/CodeMessageThe
Content-MD5 you specified did not match what we
received./MessageExpectedDigestCWiSsgzVAJyzPy2oT8u4Ag==/ExpectedDigestCalculatedDigest2DIsv6jZJ8FuGtalOO8SPA==/CalculatedDigestRequestIdCA325C738970C313/RequestIdHostIdtnE+O1zPZovaQWMhCuM4lkX0h/wN9173FQ7omxZzLb6eH0OCHASyan+mb8WBJkNn/HostId/Error
at org.jets3t.service.S3Service.putObject(S3Service.java:2267)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.storeFile(Jets3tNativeFileSystemStore.java:113)
... 21 more
2015-04-03 17:03:50,613 INFO [main] mapreduce.Job: Task Id :
attempt_1426532296228_55454_m_10_1, Status : FAILED
AttemptID:attempt_1426532296228_55454_m_10_1 Timed out after 300 secs
{\code}
We've verified that exports to other clusters from these same snapshots work
fine. Thus the issue appears to lie within the snapshot export utility,
jets3t, and S3.
The Content-MD5 you specified did not match what we received seems to
indicate that the snapshot changed between when the upload started and the
error. Can that be?
Related to:
[Discussion on jets3t user
group,.|https://groups.google.com/forum/#!topic/jets3t-users/Bg2qh7OdE2U]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Removal of deprecated features
I am +1 , especially on tagging/marking the deprecations proper with a version when it will be removed. Right now we have deprecated functions (for good reason) that are still lingering around. We have JIRAs deprecating features, but no follow up process to really remove them. There are no scheduled JIRAs to track those. Having them marked with a drop version will make it easy to do a clean sweep JIRA before a major release. As for keeping cruft for the sake of slow changing user setups, I would also go with the doubling up of keeping them, as in deprecated in two major releases before removal. That should give plenty of time. No? Lars On Tue, Mar 31, 2015 at 8:36 AM, Lars Francke lars.fran...@gmail.com wrote: Sean, thanks for your comments. On Tue, Mar 31, 2015 at 2:09 AM, Sean Busbey bus...@cloudera.com wrote: I thought deprecated for a major version meant that to be removed in e.g. HBase 2.0.0 we had to have deprecated the API by HBase 1.0.0, that is to say I thought we claimed a stronger stance than semver required specificly for API compat. I can see the ambiguity though. Ah...I see...well that could be too. I happily defer to native speakers in interpreting that sentence. Or obviously to the original discussion/intent. If what you say is the original intent that's fine too. Maybe we could then clarify the example and only remove things in 2.0.0 that were already deprecated pre-1.0.0. Are we removing deprecated things just for code cleanup, of because keeping them around is limiting some other set of changes we want to make? I want to remove them for even more selfish reasons: Limit confusion on my end (hoping that there's more people like me out there). I only get to play with HBase every so often so have to relearn bits and pieces every time. And it's confusing to tell why certain things are deprecated but still in use etc. Currently I'm working with a client on HBase and reviewing the second edition of LarsG's book and we've stumbled across quite a few things that cause confusion in both projects because the intent is not clear or because long deprecated features still linger around. There's also things like this https://issues.apache.org/jira/browse/HBASE-13274 which get flushed out. So I'd say we should remove deprecated things mostly for the end user and for (new) contributors to the project. This ties back into discussions of how long we keep doing minor releases once a newer major release happens. If we expect major releases to have a good deal of overlap, then I'm less concerned about breaking old clients. But if we expect people to go through a major release upgrade e.g. every 2 years in order to keep seeing updates, then I'd rather err on the side of cruft if it makes downstream maintenance easier I see what you mean and maybe keeping deprecated things around an extra version is good for that. Keeping deprecated things for an unspecified time is more a disservice than removing them at clearly documented points in time in my opinion. My new proposal would thus be: * In the master branch (which will be released as 2.0.0 if I'm not mistaken) remove (or undeprecate if it turns out the functionality is actually still needed) all functionality that was marked deprecated prior to 1.0.0 * Clarify that all deprecations that were added in 1.x will be removed in 3.0.0 * Clarify that all deprecations that were added in 2.x will be removed in 4.0.0 * Clarify the SemVer documentation with a different example * All new deprecations could mention a version when they are going to be removed, according to SemVer this should be the next major version (e.g. This feature is scheduled to be removed in HBase 3.0.0[2]) -- Sean On Mar 30, 2015 6:53 PM, Lars Francke lars.fran...@gmail.com wrote: I know this was discussed briefly last year[1] but I'd like to bring it up again. I'd like to remove a bunch of deprecated features. Now with Semantic Versioning this should be a bit simpler. I propose the following: * In the master branch (which will be released as 2.0.0 if I'm not mistaken) remove (or undeprecate if it turns out the functionality is actually still needed) all functionality that was marked deprecated prior to 1.0.0 or in any 1.x release * All new deprecations could mention a version when they are going to be removed, according to SemVer this should be the next major version (e.g. This feature is scheduled to be removed in HBase 3.0.0[2]) Do you think that's reasonable? If so I'm happy to file JIRAs and go through the code to get started. I think this is also in line with what our docs state: An API needs to deprecated for a major version before we will change/remove it. Example: A user using a newly deprecated api does not need to modify application code with hbase api calls until the next major version. Cheers, Lars
[jira] [Created] (HBASE-13404) Add/Refactor vint and vlong related functions in Bytes
Apekshit Sharma created HBASE-13404: --- Summary: Add/Refactor vint and vlong related functions in Bytes Key: HBASE-13404 URL: https://issues.apache.org/jira/browse/HBASE-13404 Project: HBase Issue Type: Improvement Reporter: Apekshit Sharma Priority: Minor Bytes has these 4 functions for a lot of data types, might make sense to add missing ones/fix existing ones related to vint and vlong. 1) read X from byte[] 2) read X from offset in byte[] 3) put X in byte[] at a given offset 4) return byte[] for X Also add tests for these new/changed functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13403) Make waitOnSafeMode configurable in MasterFileSystem
Esteban Gutierrez created HBASE-13403: - Summary: Make waitOnSafeMode configurable in MasterFileSystem Key: HBASE-13403 URL: https://issues.apache.org/jira/browse/HBASE-13403 Project: HBase Issue Type: Bug Components: master Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Minor We currently wait whatever is the configured value of hbase.server.thread.wakefrequency or the default 10 seconds. We should have a configuration to control how long we wait until the HDFS is no longer in safe mode, since using the existing hbase.server.thread.wakefrequency property to tune that can have adverse side effects. My proposal is to add a new property called hbase.master.waitonsafemode and start with the current default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13405) TestHBaseFsck is flaky
Mikhail Antonov created HBASE-13405:
---
Summary: TestHBaseFsck is flaky
Key: HBASE-13405
URL: https://issues.apache.org/jira/browse/HBASE-13405
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0
Reporter: Mikhail Antonov
Once in a while I'm seeing the following, running #testContainedRegionOverlap
test in IDE after clean install (mac osx, hbase master):
{code}
regionserver.HRegionServer(1863): Post open deploy tasks for
tableContainedRegionOverlap,A,1428099123733.03a139b02119e99ef08149addd9a7996.
2015-04-03 15:12:11,695 INFO
[PostOpenDeployTasks:03a139b02119e99ef08149addd9a7996]
regionserver.HRegionServer(1956): Failed to report region transition, will retry
java.io.InterruptedIOException: Origin: InterruptedException
at
org.apache.hadoop.hbase.util.ExceptionUtil.asInterrupt(ExceptionUtil.java:65)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1955)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1882)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:241)
Caused by: java.lang.InterruptedException: callId: 158 methodName:
ReportRegionStateTransition param {TODO: class
org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$ReportRegionStateTransitionRequest}
at
io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:333)
at
io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:226)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at
org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.reportRegionStateTransition(RegionServerStatusProtos.java:9030)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.reportRegionStateTransition(HRegionServer.java:1946)
... 2 more
2015-04-03 15:12:11,696 INFO [B.defaultRpcServer.handler=1,queue=0,port=51217]
master.MasterRpcServices(237): Client=mantonov//10.1.4.219 set
balanceSwitch=false
2015-04-03 15:12:11,696 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(388): maste
{code}
and then:
{code}
015-04-03 15:12:11,796 INFO [Thread-3019] client.HBaseAdmin$10(981): Started
disable of tableContainedRegionOverlap
2015-04-03 15:12:21,641 INFO [B.defaultRpcServer.handler=1,queue=0,port=51217]
master.HMaster(1645): Client=mantonov//10.1.4.219 disable
tableContainedRegionOverlap
java.lang.AssertionError:
Expected :[]
Actual :[NOT_DEPLOYED, HOLE_IN_REGION_CHAIN]
Click to see difference
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertNoErrors(HbckTestingUtil.java:92)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testContainedRegionOverlap(TestHBaseFsck.java:941)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13402) [Backport 1.1] HBASE-12552: listSnapshots should list only owned snapshots for non-super user
Ashish Singhi created HBASE-13402: - Summary: [Backport 1.1] HBASE-12552: listSnapshots should list only owned snapshots for non-super user Key: HBASE-13402 URL: https://issues.apache.org/jira/browse/HBASE-13402 Project: HBase Issue Type: Bug Components: snapshots Affects Versions: 1.1.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 1.1.0 After HBASE-11869 pushed in branch-1. It's time to push HBASE-12552 also. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13407) Add a configurable jitter to MemStoreFlusher#FlushHandler in order to smooth write latency
Esteban Gutierrez created HBASE-13407: - Summary: Add a configurable jitter to MemStoreFlusher#FlushHandler in order to smooth write latency Key: HBASE-13407 URL: https://issues.apache.org/jira/browse/HBASE-13407 Project: HBase Issue Type: Improvement Reporter: Esteban Gutierrez There is a very interesting behavior that I can reproduce consistently with many workloads from HBase 0.98 to HBase 1.0 since hbase.hstore.flusher.count was set by default to 2: when writes are evenly distributed across regions, memstores grow and flush about the same rate causing spikes in IO and CPU. The side effect of those spikes is loss in throughput which in some cases can above 10% impacting write metrics. When the flushes get a out of sync the spikes lower and and throughput is very stable. Reverting hbase.hstore.flusher.count to 1 doesn't help too much with write heavy workloads since we end with a large flush queue that eventually can block writers. Adding a small configurable jitter hbase.server.thread.wakefrequency.jitter.pct (a percentage of the hbase.server.thread.wakefrequency frequency) can help to stagger the writes from FlushHandler to HDFS and smooth the write latencies when the memstores are flushed in multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-13373) Squash HFileReaderV3 together with HFileReaderV2 and AbstractHFileReader; ditto for Scanners and BlockReader, etc.
[ https://issues.apache.org/jira/browse/HBASE-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-13373: --- Reverted from branch-1. The migration tests are failing. Will look into why later. Squash HFileReaderV3 together with HFileReaderV2 and AbstractHFileReader; ditto for Scanners and BlockReader, etc. -- Key: HBASE-13373 URL: https://issues.apache.org/jira/browse/HBASE-13373 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 2.0.0, 1.1.0 Attachments: 0001-HBASE-13373-Squash-HFileReaderV3-together-with-HFile.patch, 13373.txt, 13373.v3.txt, 13373.v3.txt, 13373.v5.txt, 13373.v6.txt, 13373.v6.txt, 13373.v6.txt, 13373.v6.txt, 13373.v6.txt, 13373.wip.txt Profiling I actually ran into case where complaint that could not inline because: MaxInlineLevel maximum number of nested calls that are inlined 9 intx i.e. method was more than 9 levels deep. The HFileReaderV? with Abstracts is not needed anymore now we are into the clear with V3 enabled since hbase 1.0.0; we can have just an Interface and an implementation. If we need to support a new hfile type, can hopefully do it in a backward compatible way now we have Cell Interface, etc. Squashing all this stuff together actually makes it easier figuring what is going on when reading code. I can also get rid of a bunch of duplication too. Attached is a WIP. Doesn't fully compile yet but you get the idea. I'll keep on unless objection. Will try it against data written with old classes as soon as I have something working. I don't believe we write classnames into our data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
