[jira] [Resolved] (HBASE-12084) Remove deprecated APIs from Result

[Anoop Sam John (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John resolved HBASE-12084.

Resolution: Fixed

Pushed addendum to master. Thanks Ted.

> Remove deprecated APIs from Result
> --
>
> Key: HBASE-12084
> URL: https://issues.apache.org/jira/browse/HBASE-12084
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-12084.patch, HBASE-12084.patch, 
> HBASE-12084_shell_addendum.patch, HBASE-12084_shell_addendum.patch
>
>
> Remove KeyValue based APIs and keep only Cell based.
> These are deprecated since 0.96



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12100) Run tests for all modules in QA run

[Ted Yu (JIRA)]

Ted Yu created HBASE-12100:
--

 Summary: Run tests for all modules in QA run
 Key: HBASE-12100
 URL: https://issues.apache.org/jira/browse/HBASE-12100
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Priority: Minor


Some tests in hbase-server module are flaky.
Currently tests in subsequent modules such as hbase-shell may be skipped if one 
or more tests from hbase-server module fail.

For JIRAs whose patch has incompatible change, it is important to detect failed 
test(s) in all modules.

We can apply '--fail-at-end' option in runTests() method of test-patch.sh



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-12084) Remove deprecated APIs from Result

[Anoop Sam John (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John reopened HBASE-12084:


That is sad.
When we deprecate some API with a replacement, we can make sure within HBase 
code deprecated code is not used. In Shell we were using the deprecated method 
Result#list. :(

Thanks Ted for pointing it out. I will give addendum here.

> Remove deprecated APIs from Result
> --
>
> Key: HBASE-12084
> URL: https://issues.apache.org/jira/browse/HBASE-12084
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-12084.patch, HBASE-12084.patch
>
>
> Remove KeyValue based APIs and keep only Cell based.
> These are deprecated since 0.96



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: custom hbase commands

[Ted Yu]

Here is a recent thread on HBase shell:
http://search-hadoop.com/m/DHED4A0E6K/hbase+shell+sean&subj=Re+HBase+shell+compatibility+needs

FYI

On Thu, Sep 25, 2014 at 5:07 PM, Shaun Elliott 
wrote:

> Yup, that was the missing link. Thank you!
>
> Does anyone here know if there is there any plan for the near future to be
> able to add extended shell commands without having to modify the shell.rb
> file? Perhaps having an extensions directory of sorts?
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/custom-hbase-commands-tp4064391p4064394.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

[jira] [Created] (HBASE-12099) TestScannerModel fails if using jackson 1.9.13

[Esteban Gutierrez (JIRA)]

Esteban Gutierrez created HBASE-12099:
-

 Summary: TestScannerModel fails if using jackson 1.9.13
 Key: HBASE-12099
 URL: https://issues.apache.org/jira/browse/HBASE-12099
 Project: HBase
  Issue Type: Bug
  Components: REST
Affects Versions: 2.0.0, 0.98.7, 0.99.1
 Environment: hadoop-2.5.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez


TestScannerModel fails if jackson 1.9.13 is used. (Hadoop 2.5 now uses that 
version, see HADOOP-10104):

{code}
Failed tests:   
testToJSON(org.apache.hadoop.hbase.rest.model.TestScannerModel): 
expected:<{"batch":100,"caching":1000,"cacheBlocks":false,"endRow":"enp5eng=","endTime":1245393318192,"maxVersions":2147483647,"startRow":"YWJyYWNhZGFicmE=","startTime":1245219839331,"column":["Y29sdW1uMQ==","Y29sdW1uMjpmb28="],"labels":["private","public"]}>
 but 
was:<{"startRow":"YWJyYWNhZGFicmE=","endRow":"enp5eng=","batch":100,"startTime":1245219839331,"endTime":1245393318192,"maxVersions":2147483647,"caching":1000,"cacheBlocks":false,"column":["Y29sdW1uMQ==","Y29sdW1uMjpmb28="],"label":["private","public"]}>
{code}

The problem is the annotation used for the labels element which is 'label' 
instead of 'labels'.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: custom hbase commands

[Shaun Elliott]

Yup, that was the missing link. Thank you!

Does anyone here know if there is there any plan for the near future to be
able to add extended shell commands without having to modify the shell.rb
file? Perhaps having an extensions directory of sorts?



--
View this message in context: 
http://apache-hbase.679495.n3.nabble.com/custom-hbase-commands-tp4064391p4064394.html
Sent from the HBase Developer mailing list archive at Nabble.com.

Restoring HBase from backup

[Krishna]

Hi,

We are trying to restore HBase from a backup saved to AWS S3 but "hbase
shell" command is stuck at this step:

> DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
> sessionid:0x148af374d1f0031, packet:: clientPath:null serverPath:null
> finished:false header:: 8,4  replyHeader:: 8,262,-101  request::
> '/hbase/root-region-server,T  response::


Its not clear why "zookeeper.znode.parent" is not using hbase.rootdir.

hbase-site.xml:

>
> hbase.zookeeper.quorumip-xx-xx-xx-xx.compute.internal
>
> hbase.rootdirs3://bucketname/hbasebackup/
>
> hbase.cluster.distributedtrue
>
> hbase.tmp.dirs3://bucketname/hbase-tmp/


Here are the versions:

> HBase 0.94
> Hadoop:Amazon 2.4.0


Following files exist in: s3://bucketname/hbasebackup/

> -ROOT-
> .META.
> .logs
> hbase_table
> hbase.id
> hbase.version



Thanks

Re: custom hbase commands

[Ted Yu]

Take a look at hbase-shell/src/main/ruby/shell.rb, starting line 273
(master branch):

Shell.load_command_group(
  'dml',
  :full_name => 'DATA MANIPULATION COMMANDS',
  :commands => %w[
count
delete
deleteall
get

You can add your command following 'get'

Cheers

On Thu, Sep 25, 2014 at 3:49 PM, Shaun Elliott 
wrote:

> This is a potential answer to my question  here
> <
> http://apache-hbase.679495.n3.nabble.com/view-decoded-thrift-in-hbase-shell-td4064130.html
> >
> that I never got a response to. I assume it is the only way...
>
> Anyways... I would like to build my own hbase shell command. I have not
> found relevant documentation explaining this. (perhaps I've missed it). To
> do this, I have copied
>
> $HBASE_HOME/libexec/lib/ruby/shell/commands/get.rb
>
> To
>
> $HBASE_HOME/libexec/lib/ruby/shell/commands/foo.rb
>
> Additionally, I changed the class to: class Foo < Command as well as the
> methods:
>   def command(table, row, *args)
> foo(table(table), row, *args)
>   end
>
>   def foo(table, row, *args)
>
> and the addition to the shell:
> #add get command to table
> ::Hbase::Table.add_shell_command('foo')
>
> From the getting started samples, I run this:
> 1.8.7-p357 :003 > get 'test','row1'
> COLUMN CELL
>  cf:a  timestamp=1411441874803, value=value1
> 1 row(s) in 0.0210 seconds
>
> 1.8.7-p357 :004 > foo 'test','row1'
> NoMethodError: undefined method `foo' for #
>
> What else do I need to do?
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/custom-hbase-commands-tp4064391.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

custom hbase commands

[Shaun Elliott]

This is a potential answer to my question  here

  
that I never got a response to. I assume it is the only way...

Anyways... I would like to build my own hbase shell command. I have not
found relevant documentation explaining this. (perhaps I've missed it). To
do this, I have copied 

$HBASE_HOME/libexec/lib/ruby/shell/commands/get.rb

To 

$HBASE_HOME/libexec/lib/ruby/shell/commands/foo.rb

Additionally, I changed the class to: class Foo < Command as well as the
methods:
  def command(table, row, *args)
foo(table(table), row, *args)
  end

  def foo(table, row, *args)

and the addition to the shell:
#add get command to table
::Hbase::Table.add_shell_command('foo')

>From the getting started samples, I run this:
1.8.7-p357 :003 > get 'test','row1'
COLUMN CELL 
  
 cf:a  timestamp=1411441874803, value=value1
  
1 row(s) in 0.0210 seconds

1.8.7-p357 :004 > foo 'test','row1'
NoMethodError: undefined method `foo' for #

What else do I need to do?



--
View this message in context: 
http://apache-hbase.679495.n3.nabble.com/custom-hbase-commands-tp4064391.html
Sent from the HBase Developer mailing list archive at Nabble.com.

[jira] [Resolved] (HBASE-12097) Blocked threads on hbase, slowly increasing, appears to be updating metrics

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-12097.

Resolution: Invalid

Please write to u...@hbase.apache.org for troubleshooting assistance.

> Blocked threads on hbase, slowly increasing, appears to be updating metrics
> ---
>
> Key: HBASE-12097
> URL: https://issues.apache.org/jira/browse/HBASE-12097
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 0.94.6
> Environment: RHEL 6.2, CDH 4.3.0.0
>Reporter: Michael Smith
> Attachments: total_blocks_versus_threads-week.png
>
>
> Hbase shows an increasing number of IPC Threads in BLOCKED state
> Hundreds of these,more and more appearing over hours, performance degrading, 
> requiring regionserver restart to restore performance.
> Thread:
> Thread 421 (IPC Server handler 368 on 60201):
>   State: BLOCKED
>   Blocked count: 19314
>   Waited count: 322565
>   Blocked on org.apache.hadoop.metrics.util.MetricsIntValue@1ec5ca55
>   Blocked by 236 (IPC Server handler 183 on 60201)
>   Stack:
> 
> org.apache.hadoop.metrics.util.MetricsIntValue.set(MetricsIntValue.java:73)
> org.apache.hadoop.hbase.ipc.HBaseServer.updateCallQueueLenMetrics(HBaseServer.java:1360)
>org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1399)
> i dont actually know how to troubleshoot this much further... Happy to take 
> suggestions...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12098) User granted namespace table create permissions can't create a table

[Dima Spivak (JIRA)]

Dima Spivak created HBASE-12098:
---

 Summary: User granted namespace table create permissions can't 
create a table
 Key: HBASE-12098
 URL: https://issues.apache.org/jira/browse/HBASE-12098
 Project: HBase
  Issue Type: Bug
  Components: Client, security
Affects Versions: 0.98.6
Reporter: Dima Spivak
Priority: Critical


>From the HBase shell and Java API, I am seeing
{code}ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: 
Insufficient permissions for user 'dima' (global, action=CREATE){code}
when I try to create a table in a namespace to which I have been granted RWXCA 
permissions by a global admin. Interestingly enough, this only seems to extend 
to table creation; the same user is then allowed to disable and drop a table 
created by a global admin in that namespace.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-12041) AssertionError in HFilePerformanceEvaluation.UniformRandomReadBenchmark

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-12041.
---
   Resolution: Fixed
Fix Version/s: 2.0.0

Fixed by subtask patch.  Resolving.

> AssertionError in HFilePerformanceEvaluation.UniformRandomReadBenchmark
> ---
>
> Key: HBASE-12041
> URL: https://issues.apache.org/jira/browse/HBASE-12041
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.99.1
>Reporter: Jean-Marc Spaggiari
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
>
> {code}
> 2014-09-19 05:18:54,719 INFO  [0] hbase.HFilePerformanceEvaluation: Running 
> UniformRandomReadBenchmark for 100 rows.
> 2014-09-19 05:18:54,719 INFO  [0] hfile.CacheConfig: CacheConfig:disabled
> Exception in thread "0" java.lang.AssertionError: Expected 472128 but got 
> 472127
> at 
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50)
> at 
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45)
> at 
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295)
> at 
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
> at 
> org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
> at 
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87)
> at java.lang.Thread.run(Thread.java:744)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12097) Blocked threads on hbase, slowly increasing, appears to be updating metrics

[Michael Smith (JIRA)]

Michael Smith created HBASE-12097:
-

 Summary: Blocked threads on hbase, slowly increasing, appears to 
be updating metrics
 Key: HBASE-12097
 URL: https://issues.apache.org/jira/browse/HBASE-12097
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.6
 Environment: RHEL 6.2, CDH 4.3.0.0
Reporter: Michael Smith


Hbase shows an increasing number of IPC Threads in BLOCKED state

Hundreds of these,more and more appearing over hours, performance degrading, 
requiring regionserver restart to restore performance.

Thread:

Thread 421 (IPC Server handler 368 on 60201):
  State: BLOCKED
  Blocked count: 19314
  Waited count: 322565
  Blocked on org.apache.hadoop.metrics.util.MetricsIntValue@1ec5ca55
  Blocked by 236 (IPC Server handler 183 on 60201)
  Stack:
org.apache.hadoop.metrics.util.MetricsIntValue.set(MetricsIntValue.java:73)
org.apache.hadoop.hbase.ipc.HBaseServer.updateCallQueueLenMetrics(HBaseServer.java:1360)
   org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1399)

i dont actually know how to troubleshoot this much further... Happy to take 
suggestions...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-11405.

   Resolution: Fixed
Fix Version/s: (was: 0.94.25)
   0.94.24

Integrated to 0.94 as well.

Thanks Sean and bharath

> Multiple invocations of hbck in parallel disables balancer permanently 
> ---
>
> Key: HBASE-11405
> URL: https://issues.apache.org/jira/browse/HBASE-11405
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer, hbck
>Affects Versions: 0.99.0
>Reporter: bharath v
>Assignee: bharath v
> Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1
>
> Attachments: 11405-0.98-v2.patch, 11405-1.0.txt, 11405-v3.txt, 
> 11405-v5.patch, 11405-v6.patch, 11405v7.patch, HBASE-11405-0.94.patch.txt, 
> HBASE-11405-0.98.patch.txt, HBASE-11405-trunk-rebased.patch, 
> HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1, 
> hbase-11405.rebase.140911.patch
>
>
> This is because of the following piece of code in hbck
> {code:borderStyle=solid}
>   boolean oldBalancer = admin.setBalancerRunning(false, true);
> try {
>   onlineConsistencyRepair();
> }
> finally {
>   admin.setBalancerRunning(oldBalancer, false);
> }
> {code}
> Newer invocations set oldBalancer to false as it was disabled by previous 
> invocations and this disables balancer permanently unless its manually turned 
> on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2 
> different sessions and you can see that balancer is set to false in the 
> HMaster logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Master shuts down during log splitting on restart

[Ted Yu]

HBASE-12095 was logged by Ashish.

I provided patch on that issue.

FYI

On Thu, Sep 25, 2014 at 1:45 AM, Ted Yu  wrote:

> Can you try the patch below ?
>
> http://pastebin.com/SnYZQf7Y
>
> Cheers
>
> On Wed, Sep 24, 2014 at 10:36 PM, ashish singhi 
> wrote:
>
>> Hi All.
>>
>> I am using 0.98.6 HBase.
>>
>> I observed that when I have the following value set in my hbase-site.xml
>> file
>>
>> 
>> hbase.regionserver.wal.encryption
>> false
>> 
>> 
>> hbase.regionserver.hlog.reader.impl
>>
>> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader
>> 
>> 
>> hbase.regionserver.hlog.writer.impl
>>
>> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter
>> 
>>
>> And while log splitting on hbase service restart, master shutdown with
>> following exception.
>>
>> Exception in master log
>>
>> 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6]
>> master.HMaster: Master server abort: loaded coprocessors are:
>> [org.apache.hadoop.hbase.security.access.AccessController]
>> 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6]
>> master.HMaster: Unhandled exception. Starting shutdown.
>> java.io.IOException: error or interrupted while splitting logs in [hdfs://
>> 10.18.40.18:8020/tmp/hbase-ashish/hbase/WALs/host-10-18-40-18,60020,1411558717849-splitting]
>> Task = installed = 6 done = 0 error = 6
>> at
>> org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:378)
>> at
>> org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:415)
>> at
>> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:307)
>> at
>> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:298)
>> at
>> org.apache.hadoop.hbase.master.HMaster.splitMetaLogBeforeAssignment(HMaster.java:1071)
>> at
>> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:863)
>> at
>> org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:612)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Exception in region server log
>>
>> 2014-09-24 20:10:16,535 WARN
>> [RS_LOG_REPLAY_OPS-host-10-18-40-18:60020-1] regionserver.SplitLogWorker:
>> log splitting of
>> WALs/host-10-18-40-18,60020,1411558717849-splitting/host-10-18-40-18%2C60020%2C1411558717849.1411558724316.meta
>> failed, returning error
>> java.io.IOException: Cannot get log reader
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:161)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:89)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:660)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:569)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:282)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:225)
>> at
>> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:143)
>> at
>> org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:82)
>> at
>> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.UnsupportedOperationException: Unable to find
>> suitable constructor for class
>> org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec
>> at
>> org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:39)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.create(WALCellCodec.java:101)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.getCodec(ProtobufLogReader.java:242)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:247)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader.initAfterCompression(SecureProtobufLogReader.java:138)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:85)
>> at
>> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:113)
>> ... 11 more
>> Caused by: java.lang.NoSuchMethodException:
>> org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec.(org.apache.hadoop.conf.Configuration,
>> 

[jira] [Created] (HBASE-12096) In ZKSplitLog Coordination and AggregateImplementation replace enhaced for statements with basic for statement to avoid unnecessary object allocation

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-12096:
-

 Summary: In ZKSplitLog Coordination and AggregateImplementation 
replace enhaced for statements with basic for statement to avoid unnecessary 
object allocation
 Key: HBASE-12096
 URL: https://issues.apache.org/jira/browse/HBASE-12096
 Project: HBase
  Issue Type: Sub-task
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor


In ZKSplitLog Coordination classes and AggregateImplementation replace enhanced 
for statements with basic for statement to avoid unnecessary object allocation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12095) Master shuts down during log splitting on restart

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-12095:
-

 Summary: Master shuts down during log splitting on restart
 Key: HBASE-12095
 URL: https://issues.apache.org/jira/browse/HBASE-12095
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.6
Reporter: Ashish Singhi
 Fix For: 0.98.7


I observed that when I have the following value set in my hbase-site.xml file


hbase.regionserver.wal.encryption
false


hbase.regionserver.hlog.reader.impl
org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader


hbase.regionserver.hlog.writer.impl
org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter


And while log splitting on hbase service restart, master shutdown with 
following exception.

Exception in master log

2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6] master.HMaster: 
Master server abort: loaded coprocessors are: 
[org.apache.hadoop.hbase.security.access.AccessController]
2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6] master.HMaster: 
Unhandled exception. Starting shutdown.
java.io.IOException: error or interrupted while splitting logs in 
[hdfs://10.18.40.18:8020/tmp/hbase-ashish/hbase/WALs/host-10-18-40-18,60020,1411558717849-splitting]
 Task = installed = 6 done = 0 error = 6
at 
org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:378)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:415)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:307)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:298)
at 
org.apache.hadoop.hbase.master.HMaster.splitMetaLogBeforeAssignment(HMaster.java:1071)
at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:863)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:612)
at java.lang.Thread.run(Thread.java:745)

Exception in region server log

2014-09-24 20:10:16,535 WARN  [RS_LOG_REPLAY_OPS-host-10-18-40-18:60020-1] 
regionserver.SplitLogWorker: log splitting of 
WALs/host-10-18-40-18,60020,1411558717849-splitting/host-10-18-40-18%2C60020%2C1411558717849.1411558724316.meta
 failed, returning error
java.io.IOException: Cannot get log reader
at 
org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:161)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:89)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:660)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:569)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:282)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:225)
at 
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:143)
at 
org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:82)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsupportedOperationException: Unable to find suitable 
constructor for class 
org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec
at 
org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:39)
at 
org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.create(WALCellCodec.java:101)
at 
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.getCodec(ProtobufLogReader.java:242)
at 
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:247)
at 
org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader.initAfterCompression(SecureProtobufLogReader.java:138)
at 
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:85)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:113)
... 11 more
Caused by: java.lang.NoSuchMethodException: 
org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec.(org.apache.hadoop.conf.Configuration,
 org.apache.hadoop.hbase.regionserver.wal.CompressionContext)
at java.lang.Class.getConstructor0(Class.java:2849)
at java.lang.C

[jira] [Created] (HBASE-12094) Clean the code after adding IS_MOB and MOB_THRESHOLD to column family

[Jingcheng Du (JIRA)]

Jingcheng Du created HBASE-12094:


 Summary: Clean the code after adding IS_MOB and MOB_THRESHOLD to 
column family
 Key: HBASE-12094
 URL: https://issues.apache.org/jira/browse/HBASE-12094
 Project: HBase
  Issue Type: Sub-task
Reporter: Jingcheng Du
Assignee: Jingcheng Du


Previously we defined two attribute IS_MOB and MOB_THRESHOLD in  the 
MobConstants, and defined the method getMobThreshold() and isMobColumnFamily() 
in the MobUtils.
Now we have these attributes and methods in HColumnDescriptor. They're 
duplicated, we will remove the ones in MobConstants and MobUtils, and modify 
the code that reference them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Master shuts down during log splitting on restart

[Ted Yu]

Can you try the patch below ?

http://pastebin.com/SnYZQf7Y

Cheers

On Wed, Sep 24, 2014 at 10:36 PM, ashish singhi 
wrote:

> Hi All.
>
> I am using 0.98.6 HBase.
>
> I observed that when I have the following value set in my hbase-site.xml
> file
>
> 
> hbase.regionserver.wal.encryption
> false
> 
> 
> hbase.regionserver.hlog.reader.impl
>
> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader
> 
> 
> hbase.regionserver.hlog.writer.impl
>
> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter
> 
>
> And while log splitting on hbase service restart, master shutdown with
> following exception.
>
> Exception in master log
>
> 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.security.access.AccessController]
> 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6]
> master.HMaster: Unhandled exception. Starting shutdown.
> java.io.IOException: error or interrupted while splitting logs in [hdfs://
> 10.18.40.18:8020/tmp/hbase-ashish/hbase/WALs/host-10-18-40-18,60020,1411558717849-splitting]
> Task = installed = 6 done = 0 error = 6
> at
> org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:378)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:415)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:307)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:298)
> at
> org.apache.hadoop.hbase.master.HMaster.splitMetaLogBeforeAssignment(HMaster.java:1071)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:863)
> at
> org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:612)
> at java.lang.Thread.run(Thread.java:745)
>
> Exception in region server log
>
> 2014-09-24 20:10:16,535 WARN  [RS_LOG_REPLAY_OPS-host-10-18-40-18:60020-1]
> regionserver.SplitLogWorker: log splitting of
> WALs/host-10-18-40-18,60020,1411558717849-splitting/host-10-18-40-18%2C60020%2C1411558717849.1411558724316.meta
> failed, returning error
> java.io.IOException: Cannot get log reader
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:161)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:89)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:660)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:569)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:282)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:225)
> at
> org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:143)
> at
> org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:82)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.UnsupportedOperationException: Unable to find
> suitable constructor for class
> org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:39)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.create(WALCellCodec.java:101)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.getCodec(ProtobufLogReader.java:242)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:247)
> at
> org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader.initAfterCompression(SecureProtobufLogReader.java:138)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:85)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:113)
> ... 11 more
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec.(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.hbase.regionserver.wal.CompressionContext)
> at java.lang.Class.getConstructor0(Class.java:2849)
> at java.lang.Class.getDeclaredConstructor(Class.java:2053)
> at
> org.apach

[jira] [Created] (HBASE-12093) Support the mob attributes in hbase shell when create/alter table

[Jingcheng Du (JIRA)]

Jingcheng Du created HBASE-12093:


 Summary: Support the mob attributes in hbase shell when 
create/alter table
 Key: HBASE-12093
 URL: https://issues.apache.org/jira/browse/HBASE-12093
 Project: HBase
  Issue Type: Sub-task
  Components: shell
Reporter: Jingcheng Du
Assignee: Jingcheng Du






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12092) Use thrift2 c + + put data into hbase

[xuwenbo (JIRA)]

xuwenbo created HBASE-12092:
---

 Summary: Use thrift2 c + + put data into hbase
 Key: HBASE-12092
 URL: https://issues.apache.org/jira/browse/HBASE-12092
 Project: HBase
  Issue Type: Bug
  Components: regionserver, Thrift
Reporter: xuwenbo
 Fix For: 0.98.1


1.When a client to insert data(two table: 1.test-cdr; 2.test-cdr-index.) 
program after the region server log to print the following information, but the 
insert is correct.
---
2014-09-25 14:22:33,213 INFO 
org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 61 lease expired on 
region test-cdr,,1411625852310.bff14c946ca7533d2844c728e7e8cc78.
2014-09-25 14:22:36,336 INFO 
org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 72 lease expired on 
region test-cdr,,1411625852310.bff14c946ca7533d2844c728e7e8cc78.
...
-

2.When the watch I delete and submitted to the following error.
---
2014-09-25 14:22:42,903 INFO 
org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 3 lease expired on 
region test-cdr-index,,1411625852825.091745b9f0da2ad6db986095e3661298.
2014-09-25 14:22:42,903 ERROR 
org.apache.hadoop.hbase.regionserver.HRegionServer: Closing scanner for 
test-cdr-index,,1411625852825.091745b9f0da2ad6db986095e3661298.
org.apache.hadoop.hbase.NotServingRegionException: Region 
test-cdr-index,,1411625852825.091745b9f0da2ad6db986095e3661298. is not online 
on td-123,60020,1411625181947
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2673)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2650)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer$ScannerListener.leaseExpired(HRegionServer.java:2457)
at org.apache.hadoop.hbase.regionserver.Leases.run(Leases.java:122)
at java.lang.Thread.run(Thread.java:745)

---

Seen from the first log, it seems the scanner is not closed, but I didn't open 
the scanner.
Table is deleted, after the second log shows going Closing scanner, then can't 
find the region!

that is why?





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)