[jira] [Updated] (HBASE-6980) Parallel Flushing Of Memstores
[ https://issues.apache.org/jira/browse/HBASE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Muthukkaruppan updated HBASE-6980: - Description: For write dominated workloads, single threaded memstore flushing is an unnecessary bottleneck. With a single flusher thread, we are basically not setup to take advantage of the aggregate throughput that multi-disk nodes provide. * For puts with WAL enabled, the bottleneck is more likely the "single" WAL per region server. So this particular fix may not buy as much unless we unlock that bottleneck with multiple commit logs per region server. (Topic for a separate JIRA-- HBASE-6981). * But for puts with WAL disabled (e.g., when using HBASE-5783 style fast bulk imports), we should be able to support much better ingest rates with parallel flushing of memstores. was: For write dominated workloads, single threaded memstore flushing is an unnecessary bottleneck. With a single flusher thread, we are basically not setup to take advantage of the aggregate throughput that multi-disk nodes provide. * For puts with WAL enabled, the bottleneck is more likely the "single" WAL per region server. So this particular fix may not buy as much unless we unlock that bottleneck with multiple commit logs per region server. (Topic for a separate JIRA-- HBASE-6980). * But for puts with WAL disabled (e.g., when using HBASE-5783 style fast bulk imports), we should be able to support much better ingest rates with parallel flushing of memstores. > Parallel Flushing Of Memstores > -- > > Key: HBASE-6980 > URL: https://issues.apache.org/jira/browse/HBASE-6980 > Project: HBase > Issue Type: New Feature >Reporter: Kannan Muthukkaruppan >Assignee: Kannan Muthukkaruppan > > For write dominated workloads, single threaded memstore flushing is an > unnecessary bottleneck. With a single flusher thread, we are basically not > setup to take advantage of the aggregate throughput that multi-disk nodes > provide. > * For puts with WAL enabled, the bottleneck is more likely the "single" WAL > per region server. So this particular fix may not buy as much unless we > unlock that bottleneck with multiple commit logs per region server. (Topic > for a separate JIRA-- HBASE-6981). > * But for puts with WAL disabled (e.g., when using HBASE-5783 style fast bulk > imports), we should be able to support much better ingest rates with parallel > flushing of memstores. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5956) Checksum Check
[ https://issues.apache.org/jira/browse/HBASE-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack closed HBASE-5956. > Checksum Check > -- > > Key: HBASE-5956 > URL: https://issues.apache.org/jira/browse/HBASE-5956 > Project: HBase > Issue Type: Brainstorming > Components: Replication >Affects Versions: 0.20.0 > Environment: RHEL 5.6 >Reporter: Vivek Jaiswal > Fix For: 0.20.0 > > > We are copying the files through Rsync from our Data Center to the DR site. > We want to verify the integrity of all the files which are transferred > through Rsync. Basically we want to verify if the checksum values at DR site > are of the same count in our Data Center -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-1024) Rearchitect regionserver I/O
[ https://issues.apache.org/jira/browse/HBASE-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack closed HBASE-1024. > Rearchitect regionserver I/O > > > Key: HBASE-1024 > URL: https://issues.apache.org/jira/browse/HBASE-1024 > Project: HBase > Issue Type: Improvement > Components: io, regionserver >Reporter: Andrew Purtell > Fix For: 0.20.0 > > > Consider supporting: > * 2GB store files > * 1TB per node (500 store files) > * Cell values up to ~100MB > * Typical use case of RS running with 1GB of heap only > Some ideas: > * Drop MapFile and make a custom store file format with (competing) design > goals: > ** heap efficiency > ** fast lookups > ** minimize I/O operations > ** optimize for typical DFS blocksizes (8MB, 64MB) > * MRU cache for filehandles and store file indexes > * Memory mapped store file indexes -- don't hold the indexes in heap; rely on > the OS blockcache for performance > * "Zero copy" I/O from IPC to store file and vice versa, like NIO buffers > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474827#comment-13474827 ] Lars Hofhansl commented on HBASE-5954: -- Hi Luke, you mean hflush in hadoop 2 less durable than hflush in hadoop 1. hsync is still better (even when it is not on the synchronous path, so there's a little gap where a client was told that everything is on disk when in fact it isn't). Filed HDFS-3979 (you know that :), just for the benefit of others reading here), which needs some testing to be committed. Since the hsync code is only in hadoop 2.1.0+ we'd need a new shim here for that version (or reflect the sh*t out of it). I'm still happy to commit this to 0.94.x. > Allow proper fsync support for HBase > > > Key: HBASE-5954 > URL: https://issues.apache.org/jira/browse/HBASE-5954 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.3, 0.96.0 > > Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, > 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, > 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6069) TableInputFormatBase#createRecordReader() doesn't initialize TableRecordReader which causes NPE
[ https://issues.apache.org/jira/browse/HBASE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474822#comment-13474822 ] Jie Huang commented on HBASE-6069: -- I am taking sick leave from Oct. 10 to Oct. 23. Any urgency, please call 18964958151. Sorry for any action delay during these days. Thanks. > TableInputFormatBase#createRecordReader() doesn't initialize > TableRecordReader which causes NPE > --- > > Key: HBASE-6069 > URL: https://issues.apache.org/jira/browse/HBASE-6069 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Jie Huang >Assignee: Jie Huang >Priority: Critical > Fix For: 0.94.1, 0.96.0 > > Attachments: 6069-v2.txt, HBASE-6069.patch > > > While running Hive(0.9.0) query over HBase(0.94.0) with hive-hbase-handler, > there always throws a Null Pointer Exception on Scanner object. Since the > TableInputFormatBase#createRecordReader() missed the initialization of > TableRecordReader object. The scanner will be null in that case. This issue > causes Hive query fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6040) Use block encoding and HBase handled checksum verification in bulk loading using HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6040. > Use block encoding and HBase handled checksum verification in bulk loading > using HFileOutputFormat > -- > > Key: HBASE-6040 > URL: https://issues.apache.org/jira/browse/HBASE-6040 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 0.94.0, 0.96.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.94.1 > > Attachments: HBASE-6040_94.patch, HBASE-6040_Trunk.patch > > > When the data is bulk loaded using HFileOutputFormat, we are not using the > block encoding and the HBase handled checksum features.. When the writer is > created for making the HFile, I am not seeing any such info passing to the > WriterBuilder. > In HFileOutputFormat.getNewWriter(byte[] family, Configuration conf), we dont > have these info and do not pass also to the writer... So those HFiles will > not have these optimizations.. > Later in LoadIncrementalHFiles.copyHFileHalf(), where we physically divide > one HFile(created by the MR) iff it can not belong to just one region, I can > see we pass the datablock encoding details and checksum details to the new > HFile writer. But this step wont happen normally I think.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6021) NullPointerException when running LoadTestTool without specifying compression type
[
https://issues.apache.org/jira/browse/HBASE-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6021.
> NullPointerException when running LoadTestTool without specifying compression
> type
> --
>
> Key: HBASE-6021
> URL: https://issues.apache.org/jira/browse/HBASE-6021
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.1, 0.96.0
> Environment: Hadoop 1.0.2, HBase 0.94.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 6021.patch
>
>
> If you don't specify a compression type on the LoadTestTool command line then
> this happens:
> {noformat}
> 12/05/16 18:41:23 ERROR util.AbstractHBaseTool: Error running command-line
> tool
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.HColumnDescriptor.setCompressionType(HColumnDescriptor.java:535)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createPreSplitLoadTestTable(HBaseTestingUtility.java:1885)
> at
> org.apache.hadoop.hbase.util.LoadTestTool.doWork(LoadTestTool.java:297)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:103)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.doStaticMain(AbstractHBaseTool.java:173)
> at org.apache.hadoop.hbase.util.LoadTestTool.main(LoadTestTool.java:341)
> {noformat}
> This should be handled better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6188) Remove the concept of table owner
[ https://issues.apache.org/jira/browse/HBASE-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6188. > Remove the concept of table owner > - > > Key: HBASE-6188 > URL: https://issues.apache.org/jira/browse/HBASE-6188 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.94.1, 0.96.0 >Reporter: Andrew Purtell >Assignee: Laxman > Labels: security > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6188.1.patch, HBASE-6188.2.patch, > HBASE-6188.3.patch, HBASE-6188.4.patch, HBASE-6188.patch, > org.apache.hadoop.hbase.security.access.TestAccessController.txt > > > The table owner concept was a design simplification in the initial drop. > First, the design changes under review means only a user with GLOBAL CREATE > permission can create a table, which will probably be an administrator. > Then, granting implicit permissions may lead to oversights and it adds > unnecessary conditionals to our code. So instead the administrator with > GLOBAL CREATE permission should make the appropriate grants at table create > time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6329) Stopping META regionserver when splitting region could cause daughter region to be assigned twice
[ https://issues.apache.org/jira/browse/HBASE-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6329. > Stopping META regionserver when splitting region could cause daughter region > to be assigned twice > - > > Key: HBASE-6329 > URL: https://issues.apache.org/jira/browse/HBASE-6329 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.94.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.94.1, 0.96.0 > > Attachments: 6329-0.94.txt, 6329v3.txt, HBASE-6329v1.patch, > HBASE-6329v2.patch > > > We found this issue in 0.94, first let me describe the case： > Stop META rs when split is in progress > 1.Stopping META rs(Server A). > 2.The main thread of rs close ZK and delete ephemeral node of the rs. > 3.SplitTransaction is retring MetaEditor.addDaughter > 4.Master's ServerShutdownHandler process the above dead META server > 5.Master fixup daughter and assign the daughter > 6.The daughter is opened on another server(Server B) > 7.Server A's splitTransaction successfully add the daughter to .META. with > serverName=Server A > 8.Now, in the .META., daughter's region location is Server A but it is > onlined on Server B > 9.Restart Master, and master will assign the daughter again. > Attaching the logs, daughter region 80f999ea84cb259e20e9a228546f6c8a > Master log: > 2012-07-04 13:45:56,493 INFO > org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Splitting logs > for dw93.kgb.sqa.cm4,60020,1341378224464 > 2012-07-04 13:45:58,983 INFO > org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Fixup; missing > daughter > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a. > > 2012-07-04 13:45:58,985 INFO org.apache.hadoop.hbase.catalog.MetaEditor: > Added daughter > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a., > serverName=null > 2012-07-04 13:45:58,988 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Assigning region > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a. > to dw88.kgb.sqa.cm4,60020,1341379188777 > 2012-07-04 13:46:00,201 INFO > org.apache.hadoop.hbase.master.AssignmentManager: The master has opened the > region > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a. > that was online on dw88.kgb.sqa.cm4,60020,1341379188777 > Master log after restart: > 2012-07-04 14:27:05,824 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: > master:6-0x136187d60e34644 Creating (or updating) unassigned node for > 80f999ea84cb259e20e9a228546f6c8a with OFFLINE state > 2012-07-04 14:27:05,851 INFO > org.apache.hadoop.hbase.master.AssignmentManager: Processing region > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a. > in state M_ZK_REGION_OFFLINE > 2012-07-04 14:27:05,854 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Assigning region > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a. > to dw93.kgb.sqa.cm4,60020,1341380812020 > 2012-07-04 14:27:06,051 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Handling > transition=RS_ZK_REGION_OPENED, server=dw93.kgb.sqa.cm4,60020,1341380812020, > region=80f999ea84cb259e20e9a228546f6c8a > Regionserver(META rs) log: > 2012-07-04 13:45:56,491 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server > dw93.kgb.sqa.cm4,60020,1341378224464; zookeeper connection c > losed. > 2012-07-04 13:46:11,951 INFO org.apache.hadoop.hbase.catalog.MetaEditor: > Added daughter > writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a., > serverName=dw93.kgb.sqa.cm4,60020,1341378224464 > 2012-07-04 13:46:11,952 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: Done with post open > deploy task for > region=writetest,JC\xCA\xC8\xCFOH\xCEV\xCC\xC2\xB5\xC2@\xD4,1341380730558.80f999ea84cb259e20e9a228546f6c8a., > daughter=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6394) verifyrep MR job map tasks throws NullPointerException
[
https://issues.apache.org/jira/browse/HBASE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6394.
> verifyrep MR job map tasks throws NullPointerException
> ---
>
> Key: HBASE-6394
> URL: https://issues.apache.org/jira/browse/HBASE-6394
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
>Priority: Minor
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 6394-trunk.patch, 6394-trunk_v2.patch
>
>
> {noformat}
> 2012-07-02 16:23:34,871 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 2012-07-02 16:23:34,876 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication$Verifier.cleanup(VerifyReplication.java:140)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:645)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at org.apache.hadoop.mapred.Child.main(Child.java:264)
> 2012-07-02 16:23:34,882 INFO org.apache.hadoop.mapred.Task: Runnning cleanup
> for the task
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6236. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.94.1, 0.96.0 > > Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, > HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6238) Grant on META not taking effect
[ https://issues.apache.org/jira/browse/HBASE-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6238. > Grant on META not taking effect > --- > > Key: HBASE-6238 > URL: https://issues.apache.org/jira/browse/HBASE-6238 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.94.1, 0.96.0 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6238.patch > > > User is not able to perform authorized operations on Meta. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-4720) Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
[ https://issues.apache.org/jira/browse/HBASE-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-4720. > Implement atomic update operations (checkAndPut, checkAndDelete) for REST > client/server > > > Key: HBASE-4720 > URL: https://issues.apache.org/jira/browse/HBASE-4720 > Project: HBase > Issue Type: Improvement >Reporter: Daniel Lord >Assignee: Mubarak Seyed > Fix For: 0.92.1, 0.94.1, 0.96.0 > > Attachments: 4720_trunk.patch, 4720_trunk_v2.patch, > 4720_trunk_v3.1.patch, 4720_trunk_v3.patch, 4720_trunk_v3.patch, > HBASE-4720.trunk.v1.patch, HBASE-4720.trunk.v2.patch, > HBASE-4720.trunk.v3.patch, HBASE-4720.trunk.v4.patch, > HBASE-4720.trunk.v5.patch, HBASE-4720.trunk.v6.patch, > HBASE-4720.trunk.v7.patch, HBASE-4720.v1.patch, HBASE-4720.v3.patch > > > I have several large application/HBase clusters where an application node > will occasionally need to talk to HBase from a different cluster. In order > to help ensure some of my consistency guarantees I have a sentinel table that > is updated atomically as users interact with the system. This works quite > well for the "regular" hbase client but the REST client does not implement > the checkAndPut and checkAndDelete operations. This exposes the application > to some race conditions that have to be worked around. It would be ideal if > the same checkAndPut/checkAndDelete operations could be supported by the REST > client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6195) Increment data will be lost when the memstore is flushed
[ https://issues.apache.org/jira/browse/HBASE-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6195. > Increment data will be lost when the memstore is flushed > > > Key: HBASE-6195 > URL: https://issues.apache.org/jira/browse/HBASE-6195 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: ShiXing >Assignee: ShiXing > Fix For: 0.94.1, 0.96.0 > > Attachments: 6195.addendum, 6195-trunk-V7.patch, > HBASE-6195-trunk.patch, HBASE-6195-trunk-V2.patch, HBASE-6195-trunk-V3.patch, > HBASE-6195-trunk-V4.patch, HBASE-6195-trunk-V5.patch, > HBASE-6195-trunk-V6.patch > > > There are two problems in increment() now: > First: > I see that the timestamp(the variable now) in HRegion's Increment() is > generated before got the rowLock, so when there are multi-thread increment > the same row, although it generate earlier, it may got the lock later. > Because increment just store one version, so till now, the result will still > be right. > When the region is flushing, these increment will read the kv from snapshot > and memstore with whose timestamp is larger, and write it back to memstore. > If the snapshot's timestamp larger than the memstore, the increment will got > the old data and then do the increment, it's wrong. > Secondly: > Also there is a risk in increment. Because it writes the memstore first and > then HLog, so if it writes HLog failed, the client will also read the > incremented value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6337) [MTTR] Remove renaming tmp log file in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6337. > [MTTR] Remove renaming tmp log file in SplitLogManager > --- > > Key: HBASE-6337 > URL: https://issues.apache.org/jira/browse/HBASE-6337 > Project: HBase > Issue Type: Bug >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.94.1, 0.96.0 > > Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, > HBASE-6337v3.patch, HBASE-6337v4.patch > > > As HBASE-6309 mentioned, we also encounter problem of > distributed-log-splitting take much more time than matser-local-log-splitting > because lots of SplitLogManager 's renaming operations when finishing task. > Could we try to remove renaming tmp log file in SplitLogManager through > splitting log to regions' recover.edits directory directly as the same as the > master-local-log-splitting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5802) Change the default metrics class to NullContextWithUpdateThread
[ https://issues.apache.org/jira/browse/HBASE-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5802. > Change the default metrics class to NullContextWithUpdateThread > --- > > Key: HBASE-5802 > URL: https://issues.apache.org/jira/browse/HBASE-5802 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Fix For: 0.94.1 > > Attachments: HBASE-5802-0.patch, HBASE-5802-1.patch, > HBASE-5802-2.patch > > > Since lots more metrics are being placed into the Dynamic metrics bucket > changing the default class to NullContextWithUpdateThread seems like it might > be worth it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6070) AM.nodeDeleted and SSH races creating problems for regions under SPLIT
[ https://issues.apache.org/jira/browse/HBASE-6070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6070. > AM.nodeDeleted and SSH races creating problems for regions under SPLIT > -- > > Key: HBASE-6070 > URL: https://issues.apache.org/jira/browse/HBASE-6070 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6070_0.92_1.patch, HBASE-6070_0.92.patch, > HBASE-6070_0.94_1.patch, HBASE-6070_0.94.patch, HBASE-6070_trunk_1.patch, > HBASE-6070_trunk.patch > > > We tried to address the problems in Master restart and RS restart while SPLIT > region is in progress as part of HBASE-5806. > While doing some more we found still there is one race condition. > -> Split has just started and the znode is in RS_SPLIT state. > -> RS goes down. > -> First call back for SSH comes. > -> As part of the fix for HBASE-5806 SSH knows that some region is in RIT. > -> But now nodeDeleted event comes for the SPLIt node and there we try to > delete the RIT. > -> After this we try to see in the SSH whether any node is in RIT. As we > dont find the region in RIT the region is never assigned. > When we fixed HBASE-5806 step 6 happened first and then step 5 happened. So > we missed it. Now we found that. Will come up with a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
[
https://issues.apache.org/jira/browse/HBASE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5722.
> NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
> down.
> --
>
> Key: HBASE-5722
> URL: https://issues.apache.org/jira/browse/HBASE-5722
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Affects Versions: 0.94.0, 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-5722.patch
>
>
> {code}
> List nodes =
> ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
> List newNodes = new ArrayList();
> for (String node: nodes) {
> String nodePath = ZKUtil.joinZNode(baseNode, node);
> byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
> newNodes.add(new NodeAndData(nodePath, data));
> }
> {code}
> The above code can throw NPE when listChildrenAndWatchForNewChildren returns
> null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6240) Race in HCM.getMaster stalls clients
[
https://issues.apache.org/jira/browse/HBASE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6240.
> Race in HCM.getMaster stalls clients
>
>
> Key: HBASE-6240
> URL: https://issues.apache.org/jira/browse/HBASE-6240
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-6240_1_0.94.patch, HBASE-6240.patch
>
>
> I found this issue trying to run YCSB on 0.94, I don't think it exists on any
> other branch. I believe that this was introduced in HBASE-5058 "Allow
> HBaseAdmin to use an existing connection".
> The issue is that in HCM.getMaster it does this recipe:
> # Check if the master is null and runs (if so, return)
> # Grab a lock on masterLock
> # nullify this.master
> # try to get a new master
> The issue happens at 3, it should re-run 1 since while you're waiting on the
> lock someone else could have already fixed it for you. What happens right now
> is that the threads are all able to set the master to null before others are
> able to get out of getMaster and it's a complete mess.
> Figuring it out took me some time because it doesn't manifest itself right
> away, silent retries are done in the background. Basically the first clue was
> this:
> {noformat}
> Error doing get: org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Failed after attempts=10, exceptions:
> Tue Jun 19 23:40:46 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:47 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:48 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:49 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:51 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:53 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:40:57 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:41:01 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:41:09 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> Tue Jun 19 23:41:25 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException:
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
> closed
> {noformat}
> This was caused by the little dance up in HBaseAdmin where it deletes "stale"
> connections... which are not stale at all.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6375) Master may be using a stale list of region servers for creating assignment plan during startup
[
https://issues.apache.org/jira/browse/HBASE-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6375.
> Master may be using a stale list of region servers for creating assignment
> plan during startup
> --
>
> Key: HBASE-6375
> URL: https://issues.apache.org/jira/browse/HBASE-6375
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0
> Environment: All
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-6375_94.patch, HBASE-6375_trunk.patch
>
>
> While investigating an Out of Memory issue, I had an interesting observation
> where the master tries to assign all regions to a single region server even
> though 7 other had already registered with it.
> As the cluster had MSLAB enabled, this resulted in OOM on the RS when it
> tired to open all of them.
> *From master's log (edited for brevity):*
> {quote}
> 55,468 Waiting on regionserver(s) to checkin
> 56,968 Waiting on regionserver(s) to checkin
> 58,468 Waiting on regionserver(s) to checkin
> 59,968 Waiting on regionserver(s) to checkin
> 01,242 Registering server=srv109.datacenter,60020,1338673920529,regionCount=0,userLoad=false
> 01,469 Waiting on regionserver(s) count to settle; currently=1
> 02,969 Finished waiting for regionserver count to settle; count=1,sleptFor=46500
> 02,969 Exiting wait on regionserver(s) to checkin; count=1, stopped=false,count of regions out on cluster=0
> 03,010 Processing region \-ROOT\-,,0.70236052 in state M_ZK_REGION_OFFLINE
> 03,220 \-ROOT\- assigned=0, rit=true, location=srv109.datacenter:60020
> 03,221 Processing region .META.,,1.1028785192 in state M_ZK_REGION_OFFLINE
> 03,336 Detected completed assignment of META, notifying catalog tracker
> 03,350 .META. assigned=0, rit=true, location=srv109.datacenter:60020
> 03,350 Master startup proceeding: cluster startup
> 04,006 Registering server=srv111.datacenter,60020,1338673923399,regionCount=0,userLoad=false
> 04,012 Registering server=srv113.datacenter,60020,1338673923532,regionCount=0,userLoad=false
> 04,269 Registering server=srv115.datacenter,60020,1338673923471,regionCount=0,userLoad=false
> 04,363 Registering server=srv117.datacenter,60020,1338673923928,regionCount=0,userLoad=false
> 04,599 Registering server=srv127.datacenter,60020,1338673924067,regionCount=0,userLoad=false
> 04,606 Registering server=srv119.datacenter,60020,1338673923953,regionCount=0,userLoad=false
> 04,804 Registering server=srv129.datacenter,60020,1338673924339,regionCount=0,userLoad=false
> 05,126 Bulk assigning 1252 region(s) across 1 server(s), retainAssignment=true
> 05,546 hd109.datacenter,60020,1338673920529 unassigned znodes=207 of
> {quote}
> *A peek at AssignmentManager code offer some explanation:*
> {code}
> public void assignAllUserRegions() throws IOException, InterruptedException
> {
> // Get all available servers
> List servers = serverManager.getOnlineServersList();
> // Scan META for all user regions, skipping any disabled tables
> Map allRegions =
> MetaReader.fullScan(catalogTracker, this.zkTable.getDisabledTables(),
> true);
> if (allRegions == null || allRegions.isEmpty()) return;
> // Determine what type of assignment to do on startup
> boolean retainAssignment = master.getConfiguration().
> getBoolean("hbase.master.startup.retainassign", true);
> Map> bulkPlan = null;
> if (retainAssignment) {
> // Reuse existing assignment info
> bulkPlan = LoadBalancer.retainAssignment(allRegions, servers);
> } else {
> // assign regions in round-robin fashion
> bulkPlan = LoadBalancer.roundRobinAssignment(new
> ArrayList(allRegions.keySet()), servers);
> }
> LOG.info("Bulk assigning " + allRegions.size() + " region(s) across " +
> servers.size() + " server(s), retainAssignment=" + retainAssignment);
> ...
> {code}
> In the function assignAllUserRegions(), listed above, AM fetches the server
> list from ServerManager long before it actually use it to create assignment
> plan.
> In between these, it performs a full scan of META to create an assignment map
> of regions. So even if additional RSes have registered in the meantime (as
> happened in this case), AM still has the old list of just one server.
> This code snippet is from 0.90.6 but the same issue exists in 0.92, 0.94 and
> trunk. Since MSLAB is enabled by default in 0.92 onwards, any large cluster
> can hit this issue upon cluster start-up when the following sequence holds
> true.
> # Master start long before the RSes (by default this long ~= 4.5 seconds)
> # All the RSes start togather but one wins the race of registering
[jira] [Closed] (HBASE-6363) HBaseConfiguration can carry a main method that dumps XML output for debug purposes
[ https://issues.apache.org/jira/browse/HBASE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6363. > HBaseConfiguration can carry a main method that dumps XML output for debug > purposes > --- > > Key: HBASE-6363 > URL: https://issues.apache.org/jira/browse/HBASE-6363 > Project: HBase > Issue Type: Improvement > Components: util >Affects Versions: 0.94.0 >Reporter: Harsh J >Priority: Trivial > Labels: conf, newbie, noob > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6363.2.patch, HBASE-6363.patch > > > Just like the Configuration class carries a main() method in it, that simply > loads itself and writes XML out to System.out, HBaseConfiguration can use the > same kinda method. > That way we can do "hbase org.apache.hadoop.….HBaseConfiguration" to get an > Xml dump of things HBaseConfiguration has properly loaded. Nifty in checking > app classpaths sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6214) Backport HBASE-5998 to 94.1
[ https://issues.apache.org/jira/browse/HBASE-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6214. > Backport HBASE-5998 to 94.1 > --- > > Key: HBASE-6214 > URL: https://issues.apache.org/jira/browse/HBASE-6214 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.0 >Reporter: Anoop Sam John >Assignee: rajeshbabu > Fix For: 0.94.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-4956) Control direct memory buffer consumption by HBaseClient
[ https://issues.apache.org/jira/browse/HBASE-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-4956. > Control direct memory buffer consumption by HBaseClient > --- > > Key: HBASE-4956 > URL: https://issues.apache.org/jira/browse/HBASE-4956 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu >Assignee: Bob Copeland > Fix For: 0.94.1, 0.96.0 > > Attachments: 4956.txt, thread_get.rb > > > As Jonathan explained here > https://groups.google.com/group/asynchbase/browse_thread/thread/c45bc7ba788b2357?pli=1 > , standard hbase client inadvertently consumes large amount of direct memory. > We should consider using netty for NIO-related tasks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5927) SSH and DisableTableHandler happening together does not clear the znode of the region and RIT map.
[
https://issues.apache.org/jira/browse/HBASE-5927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5927.
> SSH and DisableTableHandler happening together does not clear the znode of
> the region and RIT map.
> --
>
> Key: HBASE-5927
> URL: https://issues.apache.org/jira/browse/HBASE-5927
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.1, 0.96.0
>Reporter: Jieshan Bean
>Assignee: rajeshbabu
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-5927_94.patch, HBASE-5927_94_v2.patch,
> HBASE-5927_trunk_2.patch, HBASE-5927_trunk.patch, TestCaseForReProduce.txt
>
>
> A possible exception: If the related regionserver was just killed(But HMaster
> has not perceived that), then we will get a local exception "Connection reset
> by peer". If this region belongs to a disabling table. what will happen?
> ServerShutdownHandler will remove this region from AM#regions. So this region
> is still existing in RIT. TimeoutMonitor will take care of it after it got
> timeout. Then invoke unassign again. Since this region has been removed from
> AM#regions, it will return directly due to the below code:
> {code}
> synchronized (this.regions) {
> // Check if this region is currently assigned
> if (!regions.containsKey(region)) {
> LOG.debug("Attempted to unassign region " +
> region.getRegionNameAsString() + " but it is not " +
> "currently assigned anywhere");
> return;
> }
> }
> {code}
> Then it leads to an end-less loop.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5741) ImportTsv does not check for table existence
[ https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5741. > ImportTsv does not check for table existence > - > > Key: HBASE-5741 > URL: https://issues.apache.org/jira/browse/HBASE-5741 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.90.4 >Reporter: Clint Heath >Assignee: Himanshu Vashishtha > Fix For: 0.94.1, 0.96.0 > > Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741.patch, > HBase-5741-v2.patch > > > The usage statement for the "importtsv" command to hbase claims this: > "Note: if you do not use this option, then the target table must already > exist in HBase" (in reference to the "importtsv.bulk.output" command-line > option) > The truth is, the table must exist no matter what, importtsv cannot and will > not create it for you. > This is the case because the createSubmittableJob method of ImportTsv does > not even attempt to check if the table exists already, much less create it: > (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java) > 305 HTable table = new HTable(conf, tableName); > The HTable method signature in use there assumes the table exists and runs a > meta scan on it: > (From org.apache.hadoop.hbase.client.HTable.java) > 142 * Creates an object to access a HBase table. > ... > 151 public HTable(Configuration conf, final String tableName) > What we should do inside of createSubmittableJob is something similar to what > the "completebulkloads" command would do: > (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java) > 690 boolean tableExists = this.doesTableExist(tableName); > 691 if (!tableExists) this.createTable(tableName,dirPath); > Currently the docs are misleading, the table in fact must exist prior to > running importtsv. We should check if it exists rather than assume it's > already there and throw the below exception: > 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: > Encountered problems when prefetch META table: > org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for > table: myTable2, row=myTable2,,99 > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150) > ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6325) [replication] Race in ReplicationSourceManager.init can initiate a failover even if the node is alive
[
https://issues.apache.org/jira/browse/HBASE-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6325.
> [replication] Race in ReplicationSourceManager.init can initiate a failover
> even if the node is alive
> -
>
> Key: HBASE-6325
> URL: https://issues.apache.org/jira/browse/HBASE-6325
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.6, 0.92.1, 0.94.0
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-6325-0.92.patch, HBASE-6325-0.92-v2.patch
>
>
> Yet another bug found during the leap second madness, it's possible to miss
> the registration of new region servers so that in
> ReplicationSourceManager.init we start the failover of a live and replicating
> region server. I don't think there's data loss but the RS that's being failed
> over will die on:
> {noformat}
> 2012-07-01 06:25:15,604 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> sv4r23s48,10304,1341112194623: Writing replication status
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for
> /hbase/replication/rs/sv4r23s48,10304,1341112194623/4/sv4r23s48%2C10304%2C1341112194623.1341112195369
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1246)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.setData(RecoverableZooKeeper.java:372)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:655)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:697)
> at
> org.apache.hadoop.hbase.replication.ReplicationZookeeper.writeReplicationStatus(ReplicationZookeeper.java:470)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:154)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:607)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:368)
> {noformat}
> It seems to me that just refreshing {{otherRegionServers}} after getting the
> list of {{currentReplicators}} would be enough to fix this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6246) Admin.move without specifying destination does not go through AccessController
[
https://issues.apache.org/jira/browse/HBASE-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6246.
> Admin.move without specifying destination does not go through AccessController
> --
>
> Key: HBASE-6246
> URL: https://issues.apache.org/jira/browse/HBASE-6246
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, security
>Affects Versions: 0.94.0, 0.94.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
> Labels: coprocessors, security
> Fix For: 0.94.1
>
> Attachments: 6246.addendum, HBASE-6246_94-addendum.patch,
> HBASE-6246_94.patch
>
>
> {code}
> if (destServerName == null || destServerName.length == 0) {
> LOG.info("Passed destination servername is null/empty so " +
> "choosing a server at random");
> this.assignmentManager.clearRegionPlan(hri);
> // Unassign will reassign it elsewhere choosing random server.
> this.assignmentManager.unassign(hri);
> {code}
> I think we should go through security to see if there is sufficient
> permissions to do this operation?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class
[
https://issues.apache.org/jira/browse/HBASE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6185.
> Update javadoc for ConstantSizeRegionSplitPolicy class
> --
>
> Key: HBASE-6185
> URL: https://issues.apache.org/jira/browse/HBASE-6185
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.0
>Reporter: nneverwei
> Fix For: 0.94.1
>
> Attachments: HBASE-6185.patch, HBASE-6185.v2.patch,
> HBASE-6185.v3.patch
>
>
> When using hbase0.94.0 we met a strange problem.
> We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value to
> act as auto-split turn off).
> {code:xml}
>
> hbase.hregion.max.filesize
> 107374182400
>
> {code}
> Then we keep putting datas into a table.
> But when the data size far more less than 100Gb(about 500~600 uncompressed
> datas), the table auto splte to 2 regions...
> I change the log4j config to DEBUG, and saw logs below:
> {code}
> 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.0m/134221272, currentsize=1.5m/1617744 for
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in
> 3201ms, sequenceid=176387980, compaction requested=false
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,240 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
> compaction_queue=(0:0), split_queue=0
> 2012-06-07 10:30:52,265 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
> 2012-06-07 10:30:52,265 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Attempting to transition node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.:
> disabling compactions & flushes
> 2012-06-07 10:30:52,410 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> 2012-06-07 10:30:52,411 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> {code}
> {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info
> size=138657416, sizeToCheck=134217728{color}
> I did not config splitPolicy for hbase, so it means
> *IncreasingToUpperBoundRegionSplitPolicy is the default splitPolicy of 0.94.0*
> After add
> {code:xml}
>
> hbase.regionserver.region.split.policy
>
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
>
> {code}
> autosplit did not happen again and everything goes well.
> But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'This
> is the default split policy'. Or even in the
> http://hbase.apache.org/book/regions.arch.html 9.7.4.1. Custom Split
> Policies, 'default split policy: ConstantSizeRegionSplitPolicy.'.
> Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,
> than the auto-split can be almost shutdown.
> You may change those docs, and What more, in many scenerys, we actually need
> to control split manually（As you know when spliting the table are offline,
> reads and writes will fail）
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6265) Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed
[ https://issues.apache.org/jira/browse/HBASE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6265. > Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed > - > > Key: HBASE-6265 > URL: https://issues.apache.org/jira/browse/HBASE-6265 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0, 0.94.0, 0.96.0 >Reporter: Lars George >Assignee: Lars George >Priority: Critical > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6265-0.92.patch, HBASE-6265-0.94.patch, > HBASE-6265.patch, HBASE-6265-v2.patch, HBASE-6265-v3.patch > > > There is an issue when you call getTimestamp() on any KV handed into a > Coprocessor's prePut(). It initializes the internal "timestampCache" > variable. > When you then pass it to the normal processing, the region server sets the > time to the server time in case you have left it unset from the client side > (updateLatestStamp() call). > The TimeRangeTracker then calls getTimestamp() later on to see if it has to > include the KV, but instead of getting the proper time it sees the cached > timestamp from the prePut() call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6229) AM.assign() should not set table state to ENABLED directly.
[ https://issues.apache.org/jira/browse/HBASE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6229. > AM.assign() should not set table state to ENABLED directly. > --- > > Key: HBASE-6229 > URL: https://issues.apache.org/jira/browse/HBASE-6229 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.2, 0.94.1 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.94.1, 0.96.0 > > Attachments: 6229_trunk_2.patch, HBASE-6229_94_1.patch, > HBASE-6229_94_2.patch, HBASE-6229_94.patch, HBASE-6229_trunk_1.patch, > HBASE-6229_trunk.patch > > > In case of assign from EnableTableHandler table state is ENABLING. Any how > EnableTableHandler will set ENABLED after assigning all the the table > regions. If we try to set to ENABLED directly then client api may think > ENABLE table is completed. When we have a case like all the regions are added > directly into META and we call assignRegion then we need to make the table > ENABLED. Hence in such case the table will not be in ENABLING or ENABLED > state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5985) TestMetaMigrationRemovingHTD failed with HADOOP 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5985. > TestMetaMigrationRemovingHTD failed with HADOOP 2.0.0 > - > > Key: HBASE-5985 > URL: https://issues.apache.org/jira/browse/HBASE-5985 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.1, 0.96.0 > > Attachments: hbase-5985.patch > > > --- > Test set: org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD > --- > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.448 sec <<< > FAILURE! > org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD Time elapsed: 0 > sec <<< ERROR! > java.io.IOException: Failed put; errcode=1 > at > org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD.doFsCommand(TestMetaMigrationRemovingHTD.java:124) > at > org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD.setUpBeforeClass(TestMetaMigrationRemovingHTD.java:80) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at > org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6095) ActiveMasterManager NullPointerException
[
https://issues.apache.org/jira/browse/HBASE-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6095.
> ActiveMasterManager NullPointerException
>
>
> Key: HBASE-6095
> URL: https://issues.apache.org/jira/browse/HBASE-6095
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.94.1
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
>Priority: Minor
> Fix For: 0.94.1
>
> Attachments: hbase-6095.patch
>
>
> It is for 0.94 and 0.92. Trunk doesn't have the issue.
> {code}
> byte [] bytes =
> ZKUtil.getDataAndWatch(watcher, watcher.masterAddressZNode);
> // TODO: redo this to make it atomic (only added for tests)
> ServerName master = ServerName.parseVersionedServerName(bytes);
> {code}
> bytes could be null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6129) Backport of Add Increment Coalescing in thrift.
[ https://issues.apache.org/jira/browse/HBASE-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6129. > Backport of Add Increment Coalescing in thrift. > --- > > Key: HBASE-6129 > URL: https://issues.apache.org/jira/browse/HBASE-6129 > Project: HBase > Issue Type: Task >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.94.1 > > Attachments: HBASE-6129-0.patch > > > backport HBASE-6043 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6050) HLogSplitter renaming recovered.edits and CJ removing the parent directory race, making the HBCK think cluster is inconsistent.
[
https://issues.apache.org/jira/browse/HBASE-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6050.
> HLogSplitter renaming recovered.edits and CJ removing the parent directory
> race, making the HBCK think cluster is inconsistent.
> ---
>
> Key: HBASE-6050
> URL: https://issues.apache.org/jira/browse/HBASE-6050
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-6050.patch
>
>
> The scenario is like this
> -> A region is getting splitted.
> -> The master is still not processed the split .
> -> Region server goes down.
> -> Split log manager starts splitting the logs and creates the
> recovered.edits in the splitlog path.
> -> CJ starts and deletes the entry from META and also just completes the
> deletion of the region dir.
> -> in hlogSplitter on final step we rename the recovered.edits to come under
> the regiondir.
> There if the regiondir doesnot exist we tend to create and then add the
> recovered.edits.
> Because of this HBCK thinks it to be an orphan region because we have the
> regiondir but with no regioninfo.
> Ideally cluster is fine but we it is misleading.
> {code}
> } else {
> Path dstdir = dst.getParent();
> if (!fs.exists(dstdir)) {
> if (!fs.mkdirs(dstdir)) LOG.warn("mkdir failed on " + dstdir);
> }
> }
> fs.rename(src, dst);
> LOG.debug(" moved " + src + " => " + dst);
> } else {
> LOG.debug("Could not move recovered edits from " + src +
> " as it doesn't exist");
> }
> }
> archiveLogs(null, corruptedLogs, processedLogs,
> oldLogDir, fs, conf);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5886) Add new metric for possible data loss due to puts without WAL
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5886. > Add new metric for possible data loss due to puts without WAL > -- > > Key: HBASE-5886 > URL: https://issues.apache.org/jira/browse/HBASE-5886 > Project: HBase > Issue Type: New Feature > Components: metrics, regionserver >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: metrics > Fix For: 0.94.1, 0.96.0 > > Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, > HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, > HBASE-5886-v4.patch > > > Add a metrics to keep track of puts without WAL and possible data loss size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5659) TestAtomicOperation.testMultiRowMutationMultiThreads is still failing occasionally
[
https://issues.apache.org/jira/browse/HBASE-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5659.
> TestAtomicOperation.testMultiRowMutationMultiThreads is still failing
> occasionally
> --
>
> Key: HBASE-5659
> URL: https://issues.apache.org/jira/browse/HBASE-5659
> Project: HBase
> Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 5659-addendum.txt, 5659-addendum-v2.txt, 5659.txt,
> 5659-v2.txt
>
>
> See run here:
> https://builds.apache.org/job/PreCommit-HBASE-Build/1318//testReport/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/
> {quote}
> 2012-03-27 04:36:12,627 DEBUG [Thread-118] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/7202/Put/vlen=6/ts=7922,and after =
> rowB/colfamily11:qual1/7199/DeleteColumn/vlen=0/ts=0
> 2012-03-27 04:36:12,629 INFO [Thread-121] regionserver.HRegion(1558):
> Finished memstore flush of ~2.9k/2952, currentsize=1.6k/1640 for region
> testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81. in 14ms,
> sequenceid=7927, compaction requested=true
> 2012-03-27 04:36:12,629 DEBUG [Thread-126]
> regionserver.TestAtomicOperation$2(362): flushing
> 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1426):
> Started memstore flush for
> testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81., current region
> memstore size 1.9k
> 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1474):
> Finished snapshotting
> testtable,,1332822963417.7cd30e219714cfc5e91f69def66e7f81., commencing wait
> for mvcc, flushsize=1968
> 2012-03-27 04:36:12,630 DEBUG [Thread-126] regionserver.HRegion(1484):
> Finished snapshotting, commencing flushing stores
> 2012-03-27 04:36:12,630 DEBUG [Thread-126] util.FSUtils(153): Creating
> file=/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57
> with permission=rwxrwxrwx
> 2012-03-27 04:36:12,631 DEBUG [Thread-126] hfile.HFileWriterV2(143):
> Initialized with CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-03-27 04:36:12,631 INFO [Thread-126]
> regionserver.StoreFile$Writer(997): Delete Family Bloom filter type for
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57:
> CompoundBloomFilterWriter
> 2012-03-27 04:36:12,632 INFO [Thread-126]
> regionserver.StoreFile$Writer(1220): NO General Bloom and NO DeleteFamily was
> added to HFile
> (/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57)
>
> 2012-03-27 04:36:12,632 INFO [Thread-126] regionserver.Store(770): Flushed ,
> sequenceid=7934, memsize=1.9k, into tmp file
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57
> 2012-03-27 04:36:12,632 DEBUG [Thread-126] regionserver.Store(795): Renaming
> flushed file at
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/.tmp/61954619003e469baf1a34be5ff2ec57
> to
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/colfamily11/61954619003e469baf1a34be5ff2ec57
> 2012-03-27 04:36:12,634 INFO [Thread-126] regionserver.Store(818): Added
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/b9091c3c-961e-4035-850a-83ad14d517cc/TestAtomicOperationtestMultiRowMutationMultiThreads/testtable/7cd30e219714cfc5e91f69def66e7f81/colfamily11/61954619003e469baf1a34be5ff2ec57,
> entries=12, sequenceid=7934, filesize=1.3k
> 2012-03-27 04:36:12,642 DEBUG [Thread-118]
> regio
[jira] [Closed] (HBASE-6341) Publicly expose HConnectionKey
[ https://issues.apache.org/jira/browse/HBASE-6341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6341. > Publicly expose HConnectionKey > -- > > Key: HBASE-6341 > URL: https://issues.apache.org/jira/browse/HBASE-6341 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.92.0 >Reporter: Micah Whitacre >Priority: Trivial > Fix For: 0.92.2, 0.94.1 > > Attachments: hbase_6341.patch > > > HBASE-3777 introduced the concept of a HConnectionKey to quickly identify and > compare Configuration instances which lack equals/hashCode values. > We currently have use cases where being able to key Configuration objects in > a similar way would be helpful. An example of this would be maintain a cache > of HTablePool instances based on the HConnectionKey instead of the > Configuration instance. > I propose that HConnectionKey be made publicly available instead of its > current package scope. Or another possibility would be to move it from being > a static inner class to being part of the API. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5733. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-5733.patch, HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6382. > Upgrade Jersey to 1.8 to match Hadoop 1 and 2 > - > > Key: HBASE-6382 > URL: https://issues.apache.org/jira/browse/HBASE-6382 > Project: HBase > Issue Type: Improvement > Components: REST >Affects Versions: 0.90.7, 0.92.2, 0.94.2, 0.96.0 >Reporter: David S. Wang >Assignee: David S. Wang > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6382-trunk.patch > > > Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6013) Polish sharp edges from CopyTable
[ https://issues.apache.org/jira/browse/HBASE-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6013. > Polish sharp edges from CopyTable > - > > Key: HBASE-6013 > URL: https://issues.apache.org/jira/browse/HBASE-6013 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: hbase-6013-92.patch, hbase-6013.patch > > > CopyTable doesn't report errors when invalid arguments are specified. For > example, having a typo in --peer.adr (such as --peer.addr or -peer.adr) > silently uses the default cluster and does a same-cluster copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5916) RS restart just before master intialization we make the cluster non operative
[
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5916.
> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-5916_92.patch, HBASE-5916_94.patch,
> HBASE-5916_trunk_1.patch, HBASE-5916_trunk_1.patch, HBASE-5916_trunk_2.patch,
> HBASE-5916_trunk_3.patch, HBASE-5916_trunk_4.patch, HBASE-5916_trunk.patch,
> HBASE-5916_trunk_v5.patch, HBASE-5916_trunk_v6.patch,
> HBASE-5916_trunk_v7.patch, HBASE-5916_trunk_v8.patch,
> HBASE-5916_trunk_v9.patch, HBASE-5916v8.patch
>
>
> Consider a case where my master is getting restarted. RS that was alive when
> the master restart started, gets restarted before the master initializes the
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will
> try to expire the server but the server cannot be expired as still the
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
> if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
> existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
> }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6314) Fast fail behavior for unauthenticated user
[ https://issues.apache.org/jira/browse/HBASE-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6314. > Fast fail behavior for unauthenticated user > --- > > Key: HBASE-6314 > URL: https://issues.apache.org/jira/browse/HBASE-6314 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, security >Affects Versions: 0.92.0, 0.94.0, 0.96.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6314-v1.patch, 6314-v2.patch, HBASE-6314-0.94.patch, > HBASE-6314-trunk.patch > > > In case of unauthenticated users in secure hbase, hbase shell does a > connection retry at two levels: > a) HConnection: It retries hbase.client.retries.number times in the > getMaster() > b) HBaseAdmin: it again retries hbase.client.retries.number times in its ctr > So, hbase shell retries square number of times of the configured setting. We > can make it failfast (no retries) in case the user is not authenticated (no > valid kerberos credentials). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5757) TableInputFormat should handle as many errors as possible
[ https://issues.apache.org/jira/browse/HBASE-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5757. > TableInputFormat should handle as many errors as possible > - > > Key: HBASE-5757 > URL: https://issues.apache.org/jira/browse/HBASE-5757 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.90.6 >Reporter: Jan Lukavsky >Assignee: Jan Lukavsky > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: 5757-trunk-v2.txt, hbase-5757-92.patch, > HBASE-5757.patch, HBASE-5757.patch, HBASE-5757-trunk-r1341041.patch > > > Prior to HBASE-4196 there was different handling of IOExceptions thrown from > scanner in mapred and mapreduce API. The patch to HBASE-4196 unified this > handling so that if exception is caught a reconnect is attempted (without > bothering the mapred client). After that, HBASE-4269 changed this behavior > back, but in both mapred and mapreduce APIs. The question is, is there any > reason not to handle all errors that the input format can handle? In other > words, why not try to reissue the request after *any* IOException? I see the > following disadvantages of current approach > * the client may see exceptions like LeaseException and > ScannerTimeoutException if he fails to process all fetched data in timeout > * to avoid ScannerTimeoutException the client must raise > hbase.regionserver.lease.period > * timeouts for tasks is aready configured in mapred.task.timeout, so this > seems to me a bit redundant, because typically one needs to update both these > parameters > * I don't see any possibility to get rid of LeaseException (this is > configured on server side) > I think all of these issues would be gone, if the DoNotRetryIOException would > not be rethrown. -On the other hand, handling errors in InputFormat has > disadvantage, that it may hide from the user some inefficiency. Eg. if I have > very big scanner.caching, and I manage to process only a few rows in timeout, > I will end up with single row being fetched many times (and will not be > explicitly notified about this). Could we solve this problem by adding some > counter to the InputFormat?- -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6426) Add Hadoop 2.0.x profile to 0.92+
[ https://issues.apache.org/jira/browse/HBASE-6426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6426. > Add Hadoop 2.0.x profile to 0.92+ > - > > Key: HBASE-6426 > URL: https://issues.apache.org/jira/browse/HBASE-6426 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.1 > > Attachments: 6426.txt > > > 0.96 already has a Hadoop-2.0 build profile. Let's add this to 0.92 and 0.94 > as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5372) Table mutation operations should check table level rights, not global rights
[ https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5372. > Table mutation operations should check table level rights, not global rights > - > > Key: HBASE-5372 > URL: https://issues.apache.org/jira/browse/HBASE-5372 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.94.1, 0.96.0 >Reporter: Enis Soztutar >Assignee: Laxman > Labels: security > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-5372.patch > > > drop/modify/disable/enable etc table operations should not check for global > CREATE/ADMIN rights, but table CREATE/ADMIN rights. Since we check for global > permissions first for table permissions, configuring table access using > global permissions will continue to work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6157) Revoke of Global permission is not taking effect without restart.
[ https://issues.apache.org/jira/browse/HBASE-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6157. > Revoke of Global permission is not taking effect without restart. > - > > Key: HBASE-6157 > URL: https://issues.apache.org/jira/browse/HBASE-6157 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6157.patch > > > Revoke of Global permission is not taking effect without restart. > Revoke is updating the acl table but it's not updating the USER_CACHE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5955) Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it
[
https://issues.apache.org/jira/browse/HBASE-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5955.
> Guava 11 drops MapEvictionListener and Hadoop 2.0.0-alpha requires it
> -
>
> Key: HBASE-5955
> URL: https://issues.apache.org/jira/browse/HBASE-5955
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Andrew Purtell
>Assignee: Lars Hofhansl
> Fix For: 0.94.1
>
> Attachments: 5955.txt
>
>
> Hadoop 2.0.0-alpha depends on Guava 11.0.2. Updating HBase dependencies to
> match produces the following compilation errors:
> {code}
> [ERROR] SingleSizeCache.java:[41,32] cannot find symbol
> [ERROR] symbol : class MapEvictionListener
> [ERROR] location: package com.google.common.collect
> [ERROR]
> [ERROR] SingleSizeCache.java:[94,4] cannot find symbol
> [ERROR] symbol : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> [ERROR]
> [ERROR] SingleSizeCache.java:[94,69] cannot find symbol
> [ERROR] symbol : class MapEvictionListener
> [ERROR] location: class org.apache.hadoop.hbase.io.hfile.slab.SingleSizeCache
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6248) Jetty init may fail if directory name contains "master"
[ https://issues.apache.org/jira/browse/HBASE-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6248. > Jetty init may fail if directory name contains "master" > --- > > Key: HBASE-6248 > URL: https://issues.apache.org/jira/browse/HBASE-6248 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.94.0 >Reporter: Dave Revell >Assignee: Dave Revell >Priority: Minor > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6248-0.94.0-0.diff, HBASE-6248-0.94.0-1.diff > > > InfoServer.getWebAppsPath() unsafely assumes that any instance of the string > "master" in the full path to hbase-webapps can be truncated. This breaks in > the case where hbase is installed in a directory such as "/my/hbasemaster/". > The result is that Jetty initialization will fail since the master determines > an incorrect path to hbase-webapps. The master still runs but the web UI > returns 503. > I have a patch for this problem that I'll upload soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6181) TestStoreFile fails with jdk1.7
[ https://issues.apache.org/jira/browse/HBASE-6181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6181. > TestStoreFile fails with jdk1.7 > --- > > Key: HBASE-6181 > URL: https://issues.apache.org/jira/browse/HBASE-6181 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 > Environment: jdk1.7.0_04 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.2, 0.94.1 > > Attachments: hbase-6181.patch > > > Running org.apache.hadoop.hbase.regionserver.TestStoreFile > Tests run: 10, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 51.65 sec > <<< FAILURE! > Results : > Tests in error: > testBloomFilter(org.apache.hadoop.hbase.regionserver.TestStoreFile): [B > cannot be cast to java.lang.Comparable > testBloomTypes(org.apache.hadoop.hbase.regionserver.TestStoreFile): [B > cannot be cast to java.lang.Comparable > testBloomEdgeCases(org.apache.hadoop.hbase.regionserver.TestStoreFile): [B > cannot be cast to java.lang.Comparable > testMultipleTimestamps(org.apache.hadoop.hbase.regionserver.TestStoreFile) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6397) [hbck] print out bulk load commands for sidelined regions if necessary
[ https://issues.apache.org/jira/browse/HBASE-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6397. > [hbck] print out bulk load commands for sidelined regions if necessary > -- > > Key: HBASE-6397 > URL: https://issues.apache.org/jira/browse/HBASE-6397 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6397-trunk.patch > > > It's better to print out in the log the command line to bulk load back > sidelined regions, if any. > Separate it out from HBASE-6392 since it is a different issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5975) Failed suppression of fs shutdown hook with Hadoop 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5975. > Failed suppression of fs shutdown hook with Hadoop 2.0.0 > > > Key: HBASE-5975 > URL: https://issues.apache.org/jira/browse/HBASE-5975 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.1, 0.96.0 > > Attachments: 5975+5966.patch, hbase-5975.patch > > > Unit test failed with error: Failed suppression of fs shutdown hook > ShutdownHookManager.deleteShutdownHook failed to delete the hook since it > should be runnable instead of a thread for HADOOP 2.0.0. > For other HADOOP version, runnable should work too since thread implements > runnable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6029) HBCK doesn't recover Balance switch if exception occurs in onlineHbck()
[ https://issues.apache.org/jira/browse/HBASE-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6029. > HBCK doesn't recover Balance switch if exception occurs in onlineHbck() > --- > > Key: HBASE-6029 > URL: https://issues.apache.org/jira/browse/HBASE-6029 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 0.94.0 >Reporter: Maryann Xue >Assignee: Maryann Xue > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6029.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6244) [REST] Result generators do not need to query table schema
[
https://issues.apache.org/jira/browse/HBASE-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6244.
> [REST] Result generators do not need to query table schema
> --
>
> Key: HBASE-6244
> URL: https://issues.apache.org/jira/browse/HBASE-6244
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Affects Versions: 0.92.2, 0.94.1, 0.96.0
>Reporter: ShiXing
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 6244-0.94.patch, 6244.patch
>
>
> Now, the RowResultGenerator and the ScanerResultGenerator will fit the column
> family if the request doesn't contain any column info.
> {code}
> if (rowspec.hasColumns()) {
> //addColumn for get or scan
> } else {
> for (HColumnDescriptor family:
> table.getTableDescriptor().getFamilies()) {
> scan/get.addFamily(family.getName());
> }
> }
> {code}
> The table.getTableDescriptor() will cost 10+ milliseconds in our hbase
> cluster each request. We can remove these code because the server will auto
> add the columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5883) Backup master is going down due to connection refused exception
[
https://issues.apache.org/jira/browse/HBASE-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5883.
> Backup master is going down due to connection refused exception
> ---
>
> Key: HBASE-5883
> URL: https://issues.apache.org/jira/browse/HBASE-5883
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6, 0.92.1, 0.94.0
>Reporter: Gopinathan A
>Assignee: Jieshan Bean
> Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 90-addendum.patch, 92-addendum.patch, 94-addendum.patch,
> HBASE-5883-90.patch, HBASE-5883-92.patch, HBASE-5883-94.patch,
> HBASE-5883-trunk.patch, trunk-addendum.patch
>
>
> The active master node network was down for some time (This node contains
> Master,DN,ZK,RS). Here backup node got
> notification, and started to became active. Immedietly backup node got
> aborted with the below exception.
> {noformat}
> 2012-04-09 10:42:24,270 INFO org.apache.hadoop.hbase.master.SplitLogManager:
> finished splitting (more than or equal to) 861248320 bytes in 4 log files in
> [hdfs://192.168.47.205:9000/hbase/.logs/HOST-192-168-47-202,60020,1333715537172-splitting]
> in 26374ms
> 2012-04-09 10:42:24,316 FATAL org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are: []
> 2012-04-09 10:42:24,333 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unhandled exception. Starting shutdown.
> java.io.IOException: java.net.ConnectException: Connection refused
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:375)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> at $Proxy13.getProtocolVersion(Unknown Source)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1276)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1233)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1220)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:569)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getRootServerConnection(CatalogTracker.java:369)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForRootServerConnection(CatalogTracker.java:353)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.verifyRootRegionLocation(CatalogTracker.java:660)
> at
> org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:616)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:540)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:488)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
> ... 20 more
> 2012-04-09 10:42:24,336 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
> 2012-04-09 10:42:24,336 DEBUG org.apache.hadoop.hbase.master.HMaster:
> Stopping service threads
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5853. > java.lang.RuntimeException: readObject can't find class > org.apache.hadoop.hdfs.protocol.HdfsFileStatus > -- > > Key: HBASE-5853 > URL: https://issues.apache.org/jira/browse/HBASE-5853 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.1 > Environment: hadoop-0.23.1 hbase-0.92.1 >Reporter: jiafeng.zhang > Fix For: 0.92.1, 0.94.1 > > > 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error > reading responses on connection Thread[IPC Client (1260987126) connection to > server121/172.16.40.121:9000 from smp,5,main] > java.lang.RuntimeException: readObject can't find class > org.apache.hadoop.hdfs.protocol.HdfsFileStatus > at > org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) > at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) > at > org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) > at > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) > Caused by: java.lang.ClassNotFoundException: Class > org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) > at > org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) > ... 4 more > 2012-04-23 12:51:07,797 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > server124,60020,1335152900476: Replay of HLog required. Forcing server > shutdown > org.apache.hadoop.hbase.DroppedSnapshotException: region: > hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) > at > org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) > at > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.IOException: Failed on local exception: > java.io.IOException: Error reading responses; Host Details : local host is: > "server124/172.16.40.124"; destination host is: ""server121":9000; > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) > at org.apache.hadoop.ipc.Client.call(Client.java:1094) > at > org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) > at $Proxy10.getFileInfo(Unknown Source) > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) > at $Proxy10.getFileInfo(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) > at > org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) > at > org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) > at > org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) > at > org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) > at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) > at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) > at > org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) > ... 6 more > Caused by: java.io.IOException: Error reading responses > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) > Caused by: java.lang.RuntimeException: readObject can't find class > org.apache.hadoop.hdfs.protocol.HdfsFileStatus >
[jira] [Closed] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
[ https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5840. > Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing > the old status > -- > > Key: HBASE-5840 > URL: https://issues.apache.org/jira/browse/HBASE-5840 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: rajeshbabu > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-5840_0.94.patch, HBASE-5840.patch, > HBASE-5840_trunk.patch, HBASE-5840_v2.patch > > > TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will > keeps showing old status. > This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name
[ https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6158. > Data loss if the words 'merges' or 'splits' are used as Column Family name > -- > > Key: HBASE-6158 > URL: https://issues.apache.org/jira/browse/HBASE-6158 > Project: HBase > Issue Type: Bug > Components: master, regionserver >Affects Versions: 0.94.0 >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.92.2, 0.94.1 > > Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch > > > If a table is creates with either 'merges' or 'splits' as one of the Column > Family name it can never be flushed to the disk even though the table > creation (and data population) succeeds. > The reason for this is that these two are used as temporary directory names > inside the region folder or merge and splits respectively and hence conflicts > with the directories created for CF with same name. > A simple fix would be to uses ".merges' and ".splits" as the working folder > (patch attached). This will also be consistent with other work folder names. > An alternate fix would be to declare these words (and other similar) as > reserve words and throw exception when they are used. However, I do find the > alternate approach as unnecessarily restrictive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5918) Master will block forever at startup if root server dies between assigning root and assigning meta
[
https://issues.apache.org/jira/browse/HBASE-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5918.
> Master will block forever at startup if root server dies between assigning
> root and assigning meta
> --
>
> Key: HBASE-5918
> URL: https://issues.apache.org/jira/browse/HBASE-5918
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-5918_0.94, HBASE-5918.patch, HBASE-5918.patch,
> HBASE-5918V2.patch
>
>
> When master is initializing, if root server died between assign root and
> assign meta, master will block at
> HMaster#assignRootAndMeta:{code}assignmentManager.assignMeta();
> this.catalogTracker.waitForMeta();{code}
> because ServerShutdownHandler is disabled,
> So we should enable ServerShutdownHandler after called
> assignmentManager.assignMeta();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6065) Log for flush would append a non-sequential edit in the hlog, leading to possible data loss
[
https://issues.apache.org/jira/browse/HBASE-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6065.
> Log for flush would append a non-sequential edit in the hlog, leading to
> possible data loss
> ---
>
> Key: HBASE-6065
> URL: https://issues.apache.org/jira/browse/HBASE-6065
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-6065.patch, HBASE-6065v2.patch
>
>
> After completing flush region, we will append a log edit in the hlog file
> through HLog#completeCacheFlush.
> {code}
> public void completeCacheFlush(final byte [] encodedRegionName,
> final byte [] tableName, final long logSeqId, final boolean
> isMetaRegion)
> {
> ...
> HLogKey key = makeKey(encodedRegionName, tableName, logSeqId,
> System.currentTimeMillis(), HConstants.DEFAULT_CLUSTER_ID);
> ...
> }
> {code}
> when we make the hlog key, we use the seqId from the parameter, and it is
> generated by HLog#startCacheFlush,
> Here, we may append a lower seq id edit than the last edit in the hlog file.
> If it is the last edit log in the file, it may cause data loss.
> because
> {code}
> HRegion#replayRecoveredEditsIfAny{
> ...
> maxSeqId = Math.abs(Long.parseLong(fileName));
> if (maxSeqId <= minSeqId) {
> String msg = "Maximum sequenceid for this log is " + maxSeqId
> + " and minimum sequenceid for the region is " + minSeqId
> + ", skipped the whole file, path=" + edits;
> LOG.debug(msg);
> continue;
> }
> ...
> }
> {code}
> We may skip the splitted log file, because we use the lase edit's seq id as
> its file name, and consider this seqId as the max seq id in this log file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5546) Master assigns region in the original region server when opening region failed
[ https://issues.apache.org/jira/browse/HBASE-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5546. > Master assigns region in the original region server when opening region > failed > > > Key: HBASE-5546 > URL: https://issues.apache.org/jira/browse/HBASE-5546 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.92.0 >Reporter: gaojinchao >Assignee: Ashutosh Jindal >Priority: Minor > Fix For: 0.94.1, 0.96.0 > > Attachments: hbase-5546_1.patch, hbase-5546_2.patch, > hbase-5546_3.patch, hbase-5546_4.patch, hbase-5546.patch > > > Master assigns region in the original region server when > RS_ZK_REGION_FAILED_OPEN envent was coming. > Maybe we should choose other region server. > [2012-03-07 10:14:21,750] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:14:31,826] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:14:41,903] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:14:51,975] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:02,056] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:12,167] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:22,231] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:32,303] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:42,375] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:15:52,447] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:16:02,528] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:16:12,600] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 > [2012-03-07 10:16:22,676] [DEBUG] [main-EventThread] > [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling > transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, > region=c70e98bdca98a0657a56436741523053 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6160) META entries from daughters can be deleted before parent entries
[ https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6160. > META entries from daughters can be deleted before parent entries > > > Key: HBASE-6160 > URL: https://issues.apache.org/jira/browse/HBASE-6160 > Project: HBase > Issue Type: Bug > Components: Client, regionserver >Affects Versions: 0.92.2, 0.94.0, 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.92.2, 0.94.1 > > Attachments: HBASE-6160_v1.patch, HBASE-6160v2092.txt, > HBASE-6160_v2.patch, HBASE-6160_v2.patch > > > HBASE-5986 fixed and issue, where the client sees the META entry for the > parent, but not the children. However, after the fix, we have seen the > following issue in tests: > Region A is split to -> B, C > Region B is split to -> D, E > After some time, META entry for B is deleted since it is not needed anymore, > but META entry for Region A stays in META (C still refers it). In this case, > the client throws RegionOfflineException for B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6034) Upgrade Hadoop dependencies
[ https://issues.apache.org/jira/browse/HBASE-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6034. > Upgrade Hadoop dependencies > --- > > Key: HBASE-6034 > URL: https://issues.apache.org/jira/browse/HBASE-6034 > Project: HBase > Issue Type: Task >Affects Versions: 0.92.2 >Reporter: Andrew Purtell >Assignee: stack >Priority: Minor > Fix For: 0.92.2, 0.94.1 > > Attachments: 6034.092.txt, 6034.094.txt, 6034.txt > > > 0.92 branch currently depends on Hadoop 1.0.0, but this has been moved to the > archive. The earliest release on www.apache.org/dist/ is 1.0.1. Consider > moving up? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6092) Authorize flush, split, compact operations in AccessController
[ https://issues.apache.org/jira/browse/HBASE-6092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6092. > Authorize flush, split, compact operations in AccessController > -- > > Key: HBASE-6092 > URL: https://issues.apache.org/jira/browse/HBASE-6092 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.94.1, 0.96.0 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6092.1.patch, HBASE-6092.patch > > > Currently, flush, split and compaction are not checked for authorization in > AccessController. With the current implementation any unauthorized client can > trigger these operations on a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6068) Secure HBase cluster : Client not able to call some admin APIs
[ https://issues.apache.org/jira/browse/HBASE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6068. > Secure HBase cluster : Client not able to call some admin APIs > -- > > Key: HBASE-6068 > URL: https://issues.apache.org/jira/browse/HBASE-6068 > Project: HBase > Issue Type: Bug > Components: security >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: Anoop Sam John >Assignee: Matteo Bertozzi > Fix For: 0.92.2, 0.94.1 > > Attachments: HBASE-6068-0.92.patch, HBASE-6068-v0.patch, > HBASE-6068-v1.patch, HBASE-6068-v2.patch, HBASE-6068-v3.patch > > > In case of secure cluster, we allow the HBase clients to read the zk nodes by > providing the global read permissions to all for certain nodes. These nodes > are the master address znode, root server znode and the clusterId znode. In > ZKUtil.createACL() , we can see these node names are specially handled. > But there are some other client side admin APIs which makes a read call into > the zookeeper from the client. This include the isTableEnabled() call (May be > some other. I have seen this). Here the client directly reads a node in the > zookeeper ( node created for this table ) and the data is matched to know > whether this is enabled or not. > Now in secure cluster case any client can read zookeeper nodes which it needs > for its normal operation like the master address and root server address. > But what if the client calls this API? [isTableEnaled () ]. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6227) SSH and cluster startup causes data loss
[ https://issues.apache.org/jira/browse/HBASE-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6227. > SSH and cluster startup causes data loss > - > > Key: HBASE-6227 > URL: https://issues.apache.org/jira/browse/HBASE-6227 > Project: HBase > Issue Type: Bug > Components: master >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6227.patch, HBASE-6227v2-94.patch, > HBASE-6227v2.patch > > > In AssignmentManager#processDeadServersAndRegionsInTransition, if > servershutdownhandler is processing and master consider it cluster startup, > master will assign all user regions, however, servershutdownhandler has not > completed splitting logs. > Let me describe it in more detail. > Suppose there are two regionservers A1 and B1, A1 carried META and ROOT > 1.master restart and completed assignRootAndMeta > 2.A1 and B1 are both restarted, new regionservers are A2 and B2 > 3.SSH which processed for A1 completed assigning ROOT and META > 4.master do rebuilding user regions and no regions added to master's region > list > 5.master consider it as a cluster startup, and assign all user regions > 6.SSH which processed for B1 start to split B1's logs > 7.All regions' data carried by B1 would loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5986) Clients can see holes in the META table when regions are being split
[ https://issues.apache.org/jira/browse/HBASE-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5986. > Clients can see holes in the META table when regions are being split > > > Key: HBASE-5986 > URL: https://issues.apache.org/jira/browse/HBASE-5986 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.1, 0.96.0 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 5986-v2.txt, HBASE-5986-0.92.patch, > HBASE-5986-0.94.patch, HBASE-5986-test_v1.patch, HBASE-5986_v3.patch > > > We found this issue when running large scale ingestion tests for HBASE-5754. > The problem is that the .META. table updates are not atomic while splitting a > region. In SplitTransaction, there is a time lap between the marking the > parent offline, and adding of daughters to the META table. This can result in > clients using MetaScanner, of HTable.getStartEndKeys (used by the > TableInputFormat) missing regions which are made just offline, but the > daughters are not added yet. > This is also related to HBASE-4335. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6303) HCD.setCompressionType should use Enum support for storing compression types as strings
[ https://issues.apache.org/jira/browse/HBASE-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6303. > HCD.setCompressionType should use Enum support for storing compression types > as strings > --- > > Key: HBASE-6303 > URL: https://issues.apache.org/jira/browse/HBASE-6303 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.94.0, 0.96.0 >Reporter: Gopinathan A >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6303-0.94.patch, HBASE-6303-addendum-0.94.patch, > HBASE-6303-addendum-trunk.patch, HBASE-6303-trunk.patch > > > Let's not require an update to HCD every time the HFile compression enum is > changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]
[ https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-4379. > [hbck] Does not complain about tables with no end region [Z,] > - > > Key: HBASE-4379 > URL: https://issues.apache.org/jira/browse/HBASE-4379 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Anoop Sam John > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: > 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, > hbase-4379-90.patch, hbase-4379-92.patch, HBASE-4379_94.patch, > HBASE-4379_94_V2.patch, HBASE-4379_Trunk.patch, hbase-4379.v2.patch, > TestcaseForDisabledTableIssue.patch > > > hbck does not detect or have an error condition when the last region of a > table is missing (end key != ''). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6210) Backport HBASE-6197 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6210. > Backport HBASE-6197 to 0.94 > --- > > Key: HBASE-6210 > URL: https://issues.apache.org/jira/browse/HBASE-6210 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.1 > > Attachments: HBASE-6210_0.92.patch, HBASE-6210.patch > > > Backport HBASE-6197 'HRegion's append operation may lose data' and the > accompanying HBASE-6195 to 0.94 and 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5609) Add the ability to pass additional information for slow query logging
[ https://issues.apache.org/jira/browse/HBASE-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5609. > Add the ability to pass additional information for slow query logging > - > > Key: HBASE-5609 > URL: https://issues.apache.org/jira/browse/HBASE-5609 > Project: HBase > Issue Type: New Feature > Components: Client, IPC/RPC >Reporter: Michael Drzal >Assignee: Michael Drzal >Priority: Minor > Fix For: 0.94.1 > > Attachments: HBASE-5609.patch, HBASE-5609-v2.patch, > HBASE-5609-v3.patch, HBASE-5609-v4-094.txt, HBASE-5609-v4.patch > > > HBase-4117 added the ability to log information about queries that returned > too much data or ran for too long. There is some information written as a > fingerprint that can be used to tell what table/column families/... are > affected. I would like to extend this functionality to allow the client to > insert an identifier into the operation that gets output in the log. The > idea behind this would be that if there were N places in the client > application that touched a given table in a certain way, you could quickly > narrow things down by inserting a className:functionName or similar > identifier. I'm fully willing to go back on this if people think that it > isn't a problem in real life and it would just add complexity to the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6047) Put.has() can't determine result correctly
[
https://issues.apache.org/jira/browse/HBASE-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6047.
> Put.has() can't determine result correctly
> --
>
> Key: HBASE-6047
> URL: https://issues.apache.org/jira/browse/HBASE-6047
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.92.1
>Reporter: Wang Qiang
>Assignee: Alex Newman
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments:
> 0001-HBASE-6047.-Put.has-can-t-determine-result-correctly.patch,
> 0001-HBASE-6047.-Put.has-can-t-determine-result-correctly-v2.patch,
> 6047-92.txt, PutTest.java
>
>
> the public method 'has(byte [] family, byte [] qualifier)' internally invoked
> the private method 'has(byte [] family, byte [] qualifier, long ts, byte []
> value, boolean ignoreTS, boolean ignoreValue)' with 'value=new byte[0],
> ignoreTS=true, ignoreValue=true', but there's a logical error in the body,
> it'll enter the block
> {code}
> else if (ignoreValue) {
> for (KeyValue kv: list) {
> if (Arrays.equals(kv.getFamily(), family) &&
> Arrays.equals(kv.getQualifier(), qualifier)
> && kv.getTimestamp() == ts) {
> return true;
> }
> }
> }
> {code}
> the expression 'kv.getTimestamp() == ts' in the if conditions should only
> exist when 'ignoreTS=false', otherwise, the following code will return false!
> {code}
> Put put = new Put(Bytes.toBytes("row-01"));
> put.add(Bytes.toBytes("family-01"), Bytes.toBytes("qualifier-01"),
> 1234567L, Bytes.toBytes("value-01"));
> System.out.println(put.has(Bytes.toBytes("family-01"),
> Bytes.toBytes("qualifier-01")));
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6332) Improve POM for better integration with downstream ivy projects
[ https://issues.apache.org/jira/browse/HBASE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6332. > Improve POM for better integration with downstream ivy projects > --- > > Key: HBASE-6332 > URL: https://issues.apache.org/jira/browse/HBASE-6332 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.94.1, 0.96.0 >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6332.patch.txt > > > Currently there are 2 issues affecting the downstream ivy projects: ># no default value for slf4j.version ># dependency on a non-standard junit artifact > I suggest we correct both of these in order to ensure the smooth upgrade path > for things like Sqoop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6420) Gracefully shutdown logsyncer
[ https://issues.apache.org/jira/browse/HBASE-6420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6420. > Gracefully shutdown logsyncer > - > > Key: HBASE-6420 > URL: https://issues.apache.org/jira/browse/HBASE-6420 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.1, 0.96.0 > > Attachments: 6420-trunk.patch > > > Currently, in closing a HLog, logSyncerThread is interrupted. logSyncer could > be in the middle to sync the writer. We should avoid interrupting the sync. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6284) Introduce HRegion#doMiniBatchMutation()
[ https://issues.apache.org/jira/browse/HBASE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6284. > Introduce HRegion#doMiniBatchMutation() > --- > > Key: HBASE-6284 > URL: https://issues.apache.org/jira/browse/HBASE-6284 > Project: HBase > Issue Type: Bug > Components: Performance, regionserver >Reporter: Ted Yu >Assignee: Anoop Sam John > Fix For: 0.94.1, 0.96.0 > > Attachments: 6284_Trunk-Addendum.patch, 6284_Trunk-V3.patch, > HBASE-6284_94.patch, HBASE-6284_Trunk.patch, HBASE-6284_Trunk-V2.patch, > HBASE-6284_Trunk-V3.patch > > > From Anoop under thread 'Can there be a doMiniBatchDelete in HRegion': > The HTable#delete(List) groups the Deletes for the same RS and make > one n/w call only. But within the RS, there will be N number of delete calls > on the region one by one. This will include N number of HLog write and sync. > If this also can be grouped can we get better performance for the multi row > delete. > I have made the new miniBatchDelete () and made the > HTable#delete(List) to call this new batch delete. > Just tested initially with the one node cluster. In that itself I am getting > a performance boost which is very much promising. > Only one CF and qualifier. > 10K total rows delete with a batch of 100 deletes. Only deletes happening on > the table from one thread. > With the new way the net time taken is reduced by more than 1/10 > Will test in a 4 node cluster also. I think it will worth doing this change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6252) TABLE ADMIN should be allowed to relocate regions
[ https://issues.apache.org/jira/browse/HBASE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6252. > TABLE ADMIN should be allowed to relocate regions > - > > Key: HBASE-6252 > URL: https://issues.apache.org/jira/browse/HBASE-6252 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.94.1, 0.96.0 >Reporter: Laxman >Assignee: Laxman > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6252.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5894) Table deletion failed but HBaseAdmin#deletetable reports it as success
[
https://issues.apache.org/jira/browse/HBASE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5894.
> Table deletion failed but HBaseAdmin#deletetable reports it as success
> --
>
> Key: HBASE-5894
> URL: https://issues.apache.org/jira/browse/HBASE-5894
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.7, 0.92.2, 0.94.0
> Environment: all versions
>Reporter: xufeng
>Assignee: xufeng
>Priority: Minor
> Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-5894_90_patch_v1.patch,
> HBASE-5894_90_patch_v1_surefire-report.html, HBASE-5894_90_patch_v2.patch,
> HBASE-5894_92_patch_v1.patch, HBASE-5894_92_patch_v1_surefire-report.html,
> HBASE-5894_92_patch_v2.patch, HBASE-5894_94_patch_v1.patch,
> HBASE-5894_94_patch_v1_surefire-report.html, HBASE-5894_94_patch_v2.patch,
> HBASE-5894_trunk_patch_v1.patch,
> HBASE-5894_trunk_patch_v1_surefire-report.html,
> HBASE-5894_trunk_patch_v2.patch,
> HBASE-5894_trunk_patch_v2_surefire-report.html,
> HBASE-5894_trunk_patch_v3.patch
>
>
> Reproduce this issue by following steps:
> For reproduce it I add this code in DeleteTableHandler#handleTableOperation():
> {noformat}
> LOG.debug("Deleting region " + region.getRegionNameAsString() +
> " from META and FS");
> +if (true) {
> + throw new IOException("ERROR");
> +}
> // Remove region from META
> MetaEditor.deleteRegion(this.server.getCatalogTracker(), region);
> {noformat}
> step1:create a table and disable it.
> step2:delete it by HBaseAdmin#deleteTable() API.
> result:after lone time, The log say the Table has been deleted, but in fact
> if we do "list" in shell,the table also exists.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5385) Delete table/column should delete stored permissions on -acl- table
[ https://issues.apache.org/jira/browse/HBASE-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5385. > Delete table/column should delete stored permissions on -acl- table > - > > Key: HBASE-5385 > URL: https://issues.apache.org/jira/browse/HBASE-5385 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0 >Reporter: Enis Soztutar >Assignee: Matteo Bertozzi > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 5385-v3.patch, HBASE-5385-v0.patch, HBASE-5385-v1.patch, > HBASE-5385-v2.patch, HBASE-5385-v3.patch > > > Deleting the table or a column does not cascade to the stored permissions at > the -acl- table. We should also remove those permissions, otherwise, it can > be a security leak, where freshly created tables contain permissions from > previous same-named tables. We might also want to ensure, upon table > creation, that no entries are already stored at the -acl- table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6334) TestImprovement for TestHRegion.testWritesWhileGetting
[
https://issues.apache.org/jira/browse/HBASE-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6334.
> TestImprovement for TestHRegion.testWritesWhileGetting
> --
>
> Key: HBASE-6334
> URL: https://issues.apache.org/jira/browse/HBASE-6334
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.92.1, 0.94.0, 0.96.0
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Minor
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: HBASE-6334-92-v2.patch, HBASE-6334-94-v2.patch,
> HBASE-6334-trunk.patch, HBASE-6334-trunk-v2.patch
>
>
> Two improvements:
> 1. I was running testWritesWhileGetting on 0.90.6 (which lacks HBASE-4195)
> and I couldn't get the 20% failures mentioned in the JIRA. It was more like
> 1%. So I increased the flush frequency and it failed most of the time.
> 2. The check on failure is wrong:
> {code}
> assertEquals(previousKV.getValue(), thisValue);
> {code}
> Since this is a byte array, we should use Bytes.compareTo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6293) HMaster does not go down while splitting logs even if explicit shutdown is called.
[
https://issues.apache.org/jira/browse/HBASE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6293.
> HMaster does not go down while splitting logs even if explicit shutdown is
> called.
> --
>
> Key: HBASE-6293
> URL: https://issues.apache.org/jira/browse/HBASE-6293
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: rajeshbabu
>Assignee: Lars Hofhansl
> Fix For: 0.92.2, 0.94.1, 0.96.0
>
> Attachments: 6293.txt
>
>
> When master starts up and tries to do splitlog, in case of any error we try
> to do that infinitely in a loop until it succeeds.
> But now if we get a shutdown call, inside SplitLogManager
> {code}
> if (stopper.isStopped()) {
> LOG.warn("Stopped while waiting for log splits to be completed");
> return;
> }
> {code}
> Here we know that the master has stopped. As the task may not be completed
> now
> {code}
> if (batch.done != batch.installed) {
> batch.isDead = true;
> tot_mgr_log_split_batch_err.incrementAndGet();
> LOG.warn("error while splitting logs in " + logDirs +
> " installed = " + batch.installed + " but only " + batch.done + "
> done");
> throw new IOException("error or interrupt while splitting logs in "
> + logDirs + " Task = " + batch);
> }
> {code}
> we throw an exception. In MasterFileSystem.splitLogAfterStartup() we don't
> check if the master is stopped and we try continously.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6406) TestReplicationPeer.testResetZooKeeperSession and TestZooKeeper.testClientSessionExpired fail frequently
[ https://issues.apache.org/jira/browse/HBASE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6406. > TestReplicationPeer.testResetZooKeeperSession and > TestZooKeeper.testClientSessionExpired fail frequently > > > Key: HBASE-6406 > URL: https://issues.apache.org/jira/browse/HBASE-6406 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.1, 0.96.0 > > Attachments: 6406.txt, testReplication.jstack, testZooKeeper.jstack > > > Looking back through the 0.94 test runs these two tests accounted for 11 of > 34 failed tests. > They should be fixed or (temporarily) disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5913) Speed up the full scan of META
[ https://issues.apache.org/jira/browse/HBASE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5913. > Speed up the full scan of META > -- > > Key: HBASE-5913 > URL: https://issues.apache.org/jira/browse/HBASE-5913 > Project: HBase > Issue Type: Improvement >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.94.1, 0.96.0 > > Attachments: 5913-v2.txt, HBASE-5913.patch > > > In the master, we will do the full scan of META in some situations > for example, > 1.master start up > 2.CatalogJanitor do the full scan per 5 mins > 3.ServerShutdownHandler, getServerUserRegions for dead server. > For the online applications, we should try the best to reduce the process > time of ServerShutdownHandler in the situation 3. > However, we found MetaReader#getServerUserRegions take 14mins for 10w regions > in our production environment. > And it is caused by two reasons: > The first, we don't use cache and get one row per next() when fully scan > .META. > The second, "hbase.ipc.client.tcpnodelay" is false as default, and in our > environment it take 40ms for per next() (It is related to the length of row > in the .META. , if someone also found, could try to set it true) > For this issue, I think we could set the caching when do the full scan of META -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5922) HalfStoreFileReader seekBefore causes StackOverflowError
[ https://issues.apache.org/jira/browse/HBASE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5922. > HalfStoreFileReader seekBefore causes StackOverflowError > > > Key: HBASE-5922 > URL: https://issues.apache.org/jira/browse/HBASE-5922 > Project: HBase > Issue Type: Bug > Components: Client, io >Affects Versions: 0.90.0 > Environment: HBase 0.90.4 >Reporter: Nate Putnam >Assignee: Nate Putnam >Priority: Critical > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: 5922.092.txt, HBASE-5922.patch, HBASE-5922.patch, > HBASE-5922.v2.patch, HBASE-5922.v3.patch, HBASE-5922.v4.patch > > > Calling HRegionServer.getClosestRowBefore() can cause a stack overflow if the > underlying store file is a reference and the row key is in the bottom. > java.io.IOException: java.io.IOException: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:978) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1651) > at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:147) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5966) MapReduce based tests broken on Hadoop 2.0.0-alpha
[
https://issues.apache.org/jira/browse/HBASE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5966.
> MapReduce based tests broken on Hadoop 2.0.0-alpha
> --
>
> Key: HBASE-5966
> URL: https://issues.apache.org/jira/browse/HBASE-5966
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, test
>Affects Versions: 0.94.0, 0.96.0
> Environment: Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT,
> Ubuntu 12.04 LTS (GNU/Linux 3.2.0-24-generic x86_64)
>Reporter: Andrew Purtell
>Assignee: Jimmy Xiang
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-5966-1.patch, HBASE-5966-94.patch,
> hbase-5966.patch, HBASE-5966.patch
>
>
> Some fairly recent change in Hadoop 2.0.0-alpha has broken our MapReduce test
> rigging. Below is a representative error, can be easily reproduced with:
> {noformat}
> mvn -PlocalTests -Psecurity \
> -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
> clean test \
> -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> {noformat}
> And the result:
> {noformat}
> ---
> T E S T S
> ---
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
> <<< FAILURE!
> ---
> Test set: org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.292 sec
> <<< FAILURE!
> testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)
> Time elapsed: 21.935 sec <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:134)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:183)
> at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:216)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:339)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
> at
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.runTestOnTable(TestTableMapReduce.java:151)
> at
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce.testMultiRegionTable(TestTableMapReduce.java:129)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunA
[jira] [Closed] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6377. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.94.1, 0.96.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1, 0.96.0 > > Attachments: 6377-0.94.patch, > 6377-0.94-remove-get-put-delete-histograms.patch, 6377.patch, > 6377-trunk-remove-get-put-delete-histograms.patch, 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6023) Normalize security audit logging level with Hadoop
[ https://issues.apache.org/jira/browse/HBASE-6023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6023. > Normalize security audit logging level with Hadoop > -- > > Key: HBASE-6023 > URL: https://issues.apache.org/jira/browse/HBASE-6023 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, security >Affects Versions: 0.92.2, 0.94.1, 0.96.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6023.patch > > > A pretty trivial change, we log failed authentication attempts at WARN level, > as does Hadoop, but log successful authentication at TRACE level, where > Hadoop instead logs it at INFO level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6002) Possible chance of resource leak in HlogSplitter
[ https://issues.apache.org/jira/browse/HBASE-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6002. > Possible chance of resource leak in HlogSplitter > > > Key: HBASE-6002 > URL: https://issues.apache.org/jira/browse/HBASE-6002 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0, 0.96.0 >Reporter: Chinna Rao Lalam >Assignee: Chinna Rao Lalam > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6002_0.92_1.patch, HBASE-6002_0.94_1.patch, > HBASE-6002_0.94_1.patch, HBASE-6002.patch, HBASE-6002_trunk_1.patch, > HBASE-6002_trunk_2.patch, HBASE-6002_trunk.patch > > > In HLogSplitter.splitLogFileToTemp-Reader(in) is not closed and in finally > block in loop while closing the writers(wap.w) if any exception comes other > writers won't close. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6061) Fix ACL "Admin" Table inconsistent permission check
[ https://issues.apache.org/jira/browse/HBASE-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6061. > Fix ACL "Admin" Table inconsistent permission check > --- > > Key: HBASE-6061 > URL: https://issues.apache.org/jira/browse/HBASE-6061 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Labels: acl, security > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6061-0.92.patch, HBASE-6061-v0.patch, > HBASE-6061-v1.patch > > > the requirePermission() check for "admin" operation on a table is currently > inconsistent. > Table Owner with CREATE rights (that means, the owner has created that table) > can enable/disable and delete the table but needs ADMIN rights to > add/remove/modify a column. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6089) SSH and AM.joinCluster causes Concurrent Modification exception.
[ https://issues.apache.org/jira/browse/HBASE-6089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6089. > SSH and AM.joinCluster causes Concurrent Modification exception. > > > Key: HBASE-6089 > URL: https://issues.apache.org/jira/browse/HBASE-6089 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0 >Reporter: ramkrishna.s.vasudevan >Assignee: rajeshbabu > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: HBASE-6089_92.patch, HBASE-6089_94.patch, > HBASE-6089_trunk.patch > > > AM.regions map is parallely accessed in SSH and Master initialization leading > to ConcurrentModificationException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6067. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Ted Yu > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6067-addendum.txt, 6067.txt, 6067-v2.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6443) HLogSplitter should ignore 0 length files
[ https://issues.apache.org/jira/browse/HBASE-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6443. > HLogSplitter should ignore 0 length files > - > > Key: HBASE-6443 > URL: https://issues.apache.org/jira/browse/HBASE-6443 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.1, 0.96.0 > > > Somehow, some WAL files have size 0. Distributed log splitting can't handle > it. > HLogSplitter should ignore them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5963) ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
[
https://issues.apache.org/jira/browse/HBASE-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-5963.
> ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
> -
>
> Key: HBASE-5963
> URL: https://issues.apache.org/jira/browse/HBASE-5963
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
> Environment: Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop
> 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 5963-v2.txt, 5963-v3.txt, HBASE-5963.patch,
> HBASE-5963.patch
>
>
> {code}
> 12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed
> initialization
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to
> java.lang.Thread
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5876) TestImportExport has been failing against hadoop 0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5876. > TestImportExport has been failing against hadoop 0.23 profile > - > > Key: HBASE-5876 > URL: https://issues.apache.org/jira/browse/HBASE-5876 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: Ted Yu >Assignee: Jonathan Hsieh > Fix For: 0.94.1, 0.96.0 > > Attachments: hbase-5876-94.patch, hbase-5876-94-v3.patch, > hbase-5876.patch, hbase-5876-trunk-v3.patch, hbase-5876-v2.patch > > > TestImportExport has been failing against hadoop 0.23 profile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6281) Assignment need not be called for disabling table regions during clean cluster start up.
[ https://issues.apache.org/jira/browse/HBASE-6281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6281. > Assignment need not be called for disabling table regions during clean > cluster start up. > > > Key: HBASE-6281 > URL: https://issues.apache.org/jira/browse/HBASE-6281 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6281.addendum, 6281-trunk-v2.txt, HBASE-6281_92.patch, > HBASE-6281_94_2.patch, HBASE-6281_94.patch, HBASE-6281_trunk.patch > > > Currently during clean cluster start up if there are tables in DISABLING > state, we do bulk assignment through assignAllUserRegions() and after region > is OPENED in RS, master checks if the table is in DISBALING/DISABLED state > (in Am.regionOnline) and again calls unassign. This roundtrip can be avoided > even before calling assignment. > This JIRA is to address the above scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6355) Allow HBase to compile against JDK7
[ https://issues.apache.org/jira/browse/HBASE-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6355. > Allow HBase to compile against JDK7 > --- > > Key: HBASE-6355 > URL: https://issues.apache.org/jira/browse/HBASE-6355 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.94.1, 0.96.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.1, 0.96.0 > > Attachments: 6355-addendum.txt, 6355-trunk.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5630) hbck should disable the balancer using synchronousBalanceSwitch.
[ https://issues.apache.org/jira/browse/HBASE-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5630. > hbck should disable the balancer using synchronousBalanceSwitch. > > > Key: HBASE-5630 > URL: https://issues.apache.org/jira/browse/HBASE-5630 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.94.1, 0.96.0 > > Attachments: 5630-0.94-v3.txt, 5630-trunk-v3.patch, > HBASE-5630-94.patch, HBASE-5630-94-v2.patch, HBASE-5630.patch, > HBASE-5630-trunk-v2.patch > > > hbck disable the balancer using admin.balanceSwith(bool) when it would be > preferable to use the newer synchronusBalanceSwitch method found in 0.94 and > trunk branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6392) UnknownRegionException blocks hbck from sideline big overlap regions
[ https://issues.apache.org/jira/browse/HBASE-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6392. > UnknownRegionException blocks hbck from sideline big overlap regions > > > Key: HBASE-6392 > URL: https://issues.apache.org/jira/browse/HBASE-6392 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: 6392-0.90.patch, 6392_0.92.patch, 6392-trunk.patch, > 6392-trunk_v2.patch > > > Before sidelining a big overlap region, hbck tries to close it and offline it > at first. However, sometimes, it throws NotServingRegion or > UnknownRegionException. > It could be because the region is not open/assigned at all, or some other > issue. > We should figure out why and fix it. > By the way, it's better to print out in the log the command line to bulk load > back sidelined regions, if any. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-5838) Add an LZ4 compression option to HFile
[ https://issues.apache.org/jira/browse/HBASE-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-5838. > Add an LZ4 compression option to HFile > -- > > Key: HBASE-5838 > URL: https://issues.apache.org/jira/browse/HBASE-5838 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.2, 0.94.1, 0.96.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.94.1, 0.96.0 > > Attachments: 5838.patch > > > HADOOP-7657 adds support for LZ4 compression to Hadoop core. As with Snappy, > we should add reflection based support for this alternative to > HFile.Compression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6283) [region_mover.rb] Add option to exclude list of hosts on unload instead of just assuming the source node.
[ https://issues.apache.org/jira/browse/HBASE-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6283. > [region_mover.rb] Add option to exclude list of hosts on unload instead of > just assuming the source node. > - > > Key: HBASE-6283 > URL: https://issues.apache.org/jira/browse/HBASE-6283 > Project: HBase > Issue Type: Improvement >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Labels: jruby > Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 > > Attachments: hbase-6283.patch, hbase-6283-v2.patch > > > Currently, the region_mover.rb script excludes a single host, the host > offloading data, as a region move target. This essentially limits the number > of machines that can be shutdown at a time to one. For larger clusters, it > is manageable to have several nodes down at a time and desirable to get this > process done more quickly. > The proposed patch adds an exclude file option, that allows multiple hosts to > be excluded as targets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6237) Fix race on ACL table creation in TestTablePermissions
[ https://issues.apache.org/jira/browse/HBASE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl closed HBASE-6237. > Fix race on ACL table creation in TestTablePermissions > -- > > Key: HBASE-6237 > URL: https://issues.apache.org/jira/browse/HBASE-6237 > Project: HBase > Issue Type: Bug > Components: security, test >Affects Versions: 0.94.1, 0.96.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1, 0.96.0 > > Attachments: HBASE-6237.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6311) Data error after majorCompaction caused by keeping MVCC for opened scanners
[
https://issues.apache.org/jira/browse/HBASE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl closed HBASE-6311.
> Data error after majorCompaction caused by keeping MVCC for opened scanners
> ---
>
> Key: HBASE-6311
> URL: https://issues.apache.org/jira/browse/HBASE-6311
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Blocker
> Fix For: 0.94.1, 0.96.0
>
> Attachments: HBASE-6311-test.patch, HBASE-6311v1.patch,
> HBASE-6311v2.patch
>
>
> It is a big problem we found in 0.94, and you could reproduce the problem in
> Trunk using the test case I uploaded.
> When we do compaction, we will use region.getSmallestReadPoint() to keep MVCC
> for opened scanners;
> However,It will make data mistake after majorCompaction because we will skip
> delete type KV but keep the put type kv in the compacted storefile.
> The following is the reason from code:
> In StoreFileScanner, enforceMVCC is false when compaction, so we could read
> the delete type KV,
> However, we will skip this delete type KV in ScanQueryMatcher because
> following code
> {code}
> if (kv.isDelete())
> {
> ...
> if (includeDeleteMarker
> && kv.getMemstoreTS() <= maxReadPointToTrackVersions) {
> System.out.println("add deletes,maxReadPointToTrackVersions="
> + maxReadPointToTrackVersions);
> this.deletes.add(bytes, offset, qualLength, timestamp, type);
> }
> ...
> }
> {code}
> Here maxReadPointToTrackVersions = region.getSmallestReadPoint();
> and kv.getMemstoreTS() > maxReadPointToTrackVersions
> So we won't add this to DeleteTracker.
> Why test case passed if remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Because in the StoreFileScanner#skipKVsNewerThanReadpoint
> {code}
> if (cur.getMemstoreTS() <= readPoint) {
> cur.setMemstoreTS(0);
> }
> {code}
> So if we remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will
> add it to DeleteTracker in ScanQueryMatcher
> Solution:
> We use smallestReadPoint of region when compaction to keep MVCC for OPENED
> scanner, So we should retain delete type kv in output in the case(Already
> deleted KV is retained in output to make old opened scanner could read this
> KV) even if it is a majorcompaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
