Re: Re: Re: question on "drain region servers"

[Ted Yu]

Please see HBASE-4298 where this feature was introduced.

On Tue, Apr 26, 2016 at 5:12 AM, WangYQ <wangyongqiang0...@163.com> wrote:

> yes,  there is a tool graceful_stop.sh to graceful stop regionserver, and
> can move the regions back to the rs after rs come back.
> but i can not find the relation with drain region servers...
>
>
> i think drain region servers function is good, but can not think up with a
> pracital use case
>
>
>
>
>
>
>
>
> At 2016-04-26 16:01:55, "Dejan Menges" <dejan.men...@gmail.com> wrote:
> >One of use cases we use it is graceful stop of regionserver - you unload
> >regions from the server before you restart it. Of course, after restart
> you
> >expect HBase to move regions back.
> >
> >Now I'm not really remembering correctly, but I kinda remember that one of
> >the features was at least that it will move back regions which were
> already
> >there, hence not destroy too much block locality.
> >
> >On Tue, Apr 26, 2016 at 8:15 AM WangYQ <wangyongqiang0...@163.com> wrote:
> >
> >> thanks
> >> in hbase 0.99.0,  I find the rb file: draining_servers.rb
> >>
> >>
> >> i have some suggestions on this tool:
> >> 1. if I add rs hs1 to draining_servers, when hs1 restart, the zk node
> >> still exists in zk, but hmaster will not treat hs1 as draining_servers
> >> i think when we add a hs to draining_servers, we do not need to
> store
> >> the start code in zk, just store the hostName and port
> >> 2.  we add hs1 to draining_servers, but if hs1 always restart, we will
> >> need to add hs1 several times
> >>   when we need to delete the draining_servers info of hs1, we  will
> need
> >> to delete hs1 several times
> >>
> >>
> >>
> >> finally, what is the original motivation of this tool, some scenario
> >> descriptions are good.
> >>
> >>
> >>
> >>
> >>
> >>
> >> At 2016-04-26 11:33:10, "Ted Yu" <yuzhih...@gmail.com> wrote:
> >> >Please take a look at:
> >> >bin/draining_servers.rb
> >> >
> >> >On Mon, Apr 25, 2016 at 8:12 PM, WangYQ <wangyongqiang0...@163.com>
> >> wrote:
> >> >
> >> >> in hbase,  I find there is a "drain regionServer" feature
> >> >>
> >> >>
> >> >> if a rs is added to drain regionServer in ZK, then regions will not
> be
> >> >> move to on these regionServers
> >> >>
> >> >>
> >> >> but, how can a rs be add to  drain regionServer,   we add it handly
> or
> >> rs
> >> >> will add itself automaticly
> >>
>

Re: question on "drain region servers"

[Ted Yu]

Please take a look at:
bin/draining_servers.rb

On Mon, Apr 25, 2016 at 8:12 PM, WangYQ  wrote:

> in hbase,  I find there is a "drain regionServer" feature
>
>
> if a rs is added to drain regionServer in ZK, then regions will not be
> move to on these regionServers
>
>
> but, how can a rs be add to  drain regionServer,   we add it handly or rs
> will add itself automaticly

Re: Slow sync cost

[Ted Yu]

w.r.t. the pipeline, please see this description:

http://itm-vm.shidler.hawaii.edu/HDFS/ArchDocUseCases.html

On Mon, Apr 25, 2016 at 12:18 PM, Saad Mufti  wrote:

> Hi,
>
> In our large HBase cluster based on CDH 5.5 in AWS, we're constantly seeing
> the following messages in the region server logs:
>
> 2016-04-25 14:02:55,178 INFO
> org.apache.hadoop.hbase.regionserver.wal.FSHLog: Slow sync cost: 258 ms,
> current pipeline:
> [DatanodeInfoWithStorage[10.99.182.165:50010
> ,DS-281d4c4f-23bd-4541-bedb-946e57a0f0fd,DISK],
> DatanodeInfoWithStorage[10.99.182.236:50010
> ,DS-f8e7e8c9-6fa0-446d-a6e5-122ab35b6f7c,DISK],
> DatanodeInfoWithStorage[10.99.182.195:50010
> ,DS-3beae344-5a4a-4759-ad79-a61beabcc09d,DISK]]
>
>
> These happen regularly while HBase appear to be operating normally with
> decent read and write performance. We do have occasional performance
> problems when regions are auto-splitting, and at first I thought this was
> related but now I se it happens all the time.
>
>
> Can someone explain what this means really and should we be concerned? I
> tracked down the source code that outputs it in
>
>
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
>
> but after going through the code I think I'd need to know much more about
> the code to glean anything from it or the associated JIRA ticket
> https://issues.apache.org/jira/browse/HBASE-11240.
>
> Also, what is this "pipeline" the ticket and code talks about?
>
> Thanks in advance for any information and/or clarification anyone can
> provide.
>
> 
>
> Saad
>

Re: current leaseholder is trying to recreate file error with ProcedureV2

[Ted Yu]

Can you pastebin more of the master log ?

Which version of hadoop are you using ?

Log snippet from namenode w.r.t. state-0073.log may also
provide some more clue.

Thanks

On Mon, Apr 25, 2016 at 12:56 PM, donmai  wrote:

> Hi all,
>
> I'm getting a strange error during table creation / disable in HBase 1.1.2:
>
>
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
> failed to create file /hbase/MasterProcWALs/state-0073.log
> for DFSClient_NONMAPREDUCE_87753856_1 for client 10.66.102.192 because
> current leaseholder is trying to recreate file.
>
> Looks somewhat related to HBASE-14234 - what's the root cause behind this
> error message?
>
> Full stack trace below:
>
> -
>
> 2016-04-25 15:24:01,356 WARN
>  [B.defaultRpcServer.handler=7,queue=7,port=6] wal.WALProcedureStore:
> failed to create log file with id=73
>
>
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
> failed to create file /hbase/MasterProcWALs/state-0073.log
> for DFSClient_NONMAPREDUCE_87753856_1 for client 10.66.102.192 because
> current leaseholder is trying to recreate file.
>
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2988)
>
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2737)
>
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2632)
>
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2519)
>
> at
>
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:566)
>
> at
>
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:394)
>
> at
>
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>
> at
>
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:635)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
>
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
>
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1468)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1399)
>
> at
>
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:241)
>
> at com.sun.proxy.$Proxy31.create(Unknown Source)
>
> at
>
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:295)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>
> at com.sun.proxy.$Proxy32.create(Unknown Source)
>
> at
>
> org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1738)
>
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1671)
>
> at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1596)
>
> at
>
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:397)
>
> at
>
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:393)
>
> at
>
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>
> at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:393)
>
> at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:337)
>
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:912)
>
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:893)
>
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:790)
>
> at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:706)
>
> at
>
> 

Re: Hbase shell script from java

[Ted Yu]

Please see

https://blog.art-of-coding.eu/executing-operating-system-commands-from-java/

> On Apr 23, 2016, at 10:18 PM, Saurabh Malviya (samalviy)  
> wrote:
> 
> Hi,
> 
> Is there any way to run hbase shell script from Java. Also mentioned this 
> question earlier in below url earlier.
> 
> As we are having bunch of scripts and need to change frequently for 
> performance tuning.
> 
> http://grokbase.com/p/hbase/user/161ezbnk11/run-hbase-shell-script-from-java
> 
> 
> -Saurabh

Re: Region server crashes after GC pause time

[Ted Yu]

Jayesh:
Is it possible for you to share the JVM parameters ?

Thanks

On Fri, Apr 22, 2016 at 7:48 AM, Thakrar, Jayesh <
jthak...@conversantmedia.com> wrote:

> Karthik,
>
> Yes, tuning can help - but the biggest help is to give "sufficient" memory
> to the regionserver.
> And "sufficient" is relative - e.g. we have a 75GB heap (our increases
> were like this - 8, 12, 16, 24, 32, 45, 60 and 75 GB)
> Note that we had started off with 45 GB RAM and are now running on 148 GB
> RAM servers.
>
> Now we are very stable at 75GB and with appropriate JVM tuning our GCs are
> well contained (sub-100 ms).
>
> Also, although I like G1, it did not fare well for us at 75 GB - with the
> best tuning we could get upto 20-25 second pauses (HBase 1.0).
> CMS seems to work best for us.
>
> Jayesh
>
> -Original Message-
> From: karthi keyan [mailto:karthi93.san...@gmail.com]
> Sent: Friday, April 22, 2016 7:04 AM
> To: user@hbase.apache.org; dingdongc...@baidu.com
> Subject: Re: Region server crashes after GC pause time
>
> Hi Ding,
>
> I have increased the Heap to 2G , still getting out of memory exception .
> Actually i had write the data to HBase at 40K writes/sec .
>  Is there any parameter to tune up , as my knowledge "-
> XX:CMSInitiatingOccupancyFraction=N " i tuned like this in HBase.
> Is there any other parameter required to resolve this issue???
>
> Best,
> Karthik
>
> On Thu, Apr 14, 2016 at 12:21 PM, Ding,Dongchao 
> wrote:
>
> > Dump the jvm heap，analysis the the heap and find which query(s) cost
> > so many memory？
> > In my ever bad case，the RS crashed for Long GC pauses because of a big
> > query on Batch Get operation.
> >
> >
> > In addition，I think you can increase the men of JVM, 512m is so small
> > for RS.
> >
> >
> >
> > 在 16/4/14 14:00， "karthi keyan"  写入:
> >
> > >Hi ,
> > >
> > >i got this issue in HBase while at peak time handling more requests .
> > >can any one pls guide me to resolve the Long GC pauses in hbase .
> > >
> > >JDK-7 , JVM heap 512m
> > >
> > >HBase 0.98.13
> > >
> > >
> > > INFO  [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM
> > >or host machine (eg GC): pause of approximately 1466ms GC pool
> > >'ConcurrentMarkSweep' had collection(s): count=1 time=1967ms  INFO
> > >[JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host
> > >machine (eg GC): pause of approximately 2304ms GC pool
> > >'ConcurrentMarkSweep' had collection(s): count=1 time=2775ms  INFO
> > >[JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host
> > >machine (eg GC): pause of approximately 2287ms GC pool
> > >'ConcurrentMarkSweep' had collection(s): count=1 time=2775ms
> > >
> > > INFO  [RS:0;0:0:0:0:0:0:0:0:44037-SendThread(:2181)]
> > >zookeeper.ClientCnxn: Client session timed out, have not heard from
> > >server in 6819ms for sessionid 0x1540ab48b280004, closing socket
> > >connection and attempting reconnect  INFO
> > >[SplitLogWorker-,44037,1460468489645-SendThread(:2181)]
> > >zookeeper.ClientCnxn: Client session timed out, have not heard from
> > >server in 6819ms for sessionid 0x1540ab48b280005, closing socket
> > >connection and attempting reconnect
> > >
> > >Once after this HBase Region Server moved to Dead state.
> > >
> > >Best,
> > >Karthik
> >
> >
>
>
>
>
> This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.
>

Re: Can not connect local java client to a remote Hbase

[Ted Yu]

Are you using hbase 1.0 or 1.1 ?

I assume you have verified that hbase master is running normally on
master-sigma.
Are you able to use hbase shell on that node ?

If you check master log, you would see which node hosts hbase:meta
On that node, do you see anything interesting in region server log ?

Cheers

On Thu, Apr 21, 2016 at 10:41 AM, SOUFIANI Mustapha | السفياني مصطفى <
s.mustaph...@gmail.com> wrote:

> Hi all,
> I'm trying to connect my local java client (pentaho) to a remote Hbase but
> every time I get a TimeOut error telleing me that this connection couldn't
> be established.
>
> herer is the full message error:
>
>
> ***
>
> java.io.IOException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=36, exceptions:
> Wed Apr 20 10:32:43 WEST 2016, null, java.net.SocketTimeoutException:
> callTimeout=6, callDuration=75181: row 'pentaho_mappings,,' on table
> 'hbase:meta' at region=hbase:meta,,1.1588230740,
> hostname=localhost,16020,1461071963695, seqNum=0
>
>
> at
>
> com.pentaho.big.data.bundles.impl.shim.hbase.table.HBaseTableImpl.exists(HBaseTableImpl.java:71)
>
> at
>
> org.pentaho.big.data.kettle.plugins.hbase.mapping.MappingAdmin.getMappedTables(MappingAdmin.java:502)
>
> at
>
> org.pentaho.big.data.kettle.plugins.hbase.output.HBaseOutputDialog.setupMappedTableNames(HBaseOutputDialog.java:818)
>
> at
>
> org.pentaho.big.data.kettle.plugins.hbase.output.HBaseOutputDialog.access$900(HBaseOutputDialog.java:88)
>
> at
>
> org.pentaho.big.data.kettle.plugins.hbase.output.HBaseOutputDialog$7.widgetSelected(HBaseOutputDialog.java:398)
>
> at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
>
> at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
>
> at
>
> org.pentaho.big.data.kettle.plugins.hbase.output.HBaseOutputDialog.open(HBaseOutputDialog.java:603)
>
> at
>
> org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:125)
>
> at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8783)
>
> at
> org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3072)
>
> at
>
> org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:755)
>
> at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
>
> at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
>
> at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
>
> at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1347)
>
> at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7989)
>
> at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9269)
>
> at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:662)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>
> at java.lang.reflect.Method.invoke(Unknown Source)
>
> at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
>
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
> after attempts=36, exceptions:
> Wed Apr 20 10:32:43 WEST 2016, null, java.net.SocketTimeoutException:
> callTimeout=6, callDuration=75181: row 'pentaho_mappings,,' on table
> 'hbase:meta' at region=hbase:meta,,1.1588230740,
> hostname=localhost,16020,1461071963695, seqNum=0
>
>
> at
>
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:270)
>
> at
>
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:225)
>
> at
>
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:63)
>
> at
>
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
>
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:314)
>
> at
>
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:289)
>
> at
>
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:161)
>
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:156)
>
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:888)
>
> at
>
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:601)
>
> at
>
> 

Re: ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper.

[Ted Yu]

or = 
> java.io.EOFException
>   at java.io.DataInputStream.readInt(DataInputStream.java:392)
>   at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
> 2016-04-21 11:07:45,876 [myid:3] - WARN  
> [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
> 2016-04-21 11:07:45,888 [myid:3] - WARN  
> [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting 
> for message on queue
> java.lang.InterruptedException
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
>   at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
>   at 
> java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
>   at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
>   at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
>   at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
> 2016-04-21 11:07:45,889 [myid:3] - WARN  
> [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
> 2016-04-21 11:07:58,881 [myid:3] - INFO  
> [/192.168.1.4:3889:QuorumCnxManager$Listener@541] - Received connection 
> request /192.168.1.2:51099
> 2016-04-21 11:07:58,915 [myid:3] - INFO  
> [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message 
> format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING 
> (n.state), 1 (n.sid), 0x2 (n.peerEpoch) LEADING (my state)
> 2016-04-21 11:07:59,150 [myid:3] - INFO  
> [LearnerHandler-/192.168.1.2:35263:LearnerHandler@329] - Follower sid: 1 : 
> info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@4a7baf7d
> 2016-04-21 11:07:59,155 [myid:3] - INFO  
> [LearnerHandler-/192.168.1.2:35263:LearnerHandler@384] - Synchronizing with 
> Follower sid: 1 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0
> 2016-04-21 11:07:59,155 [myid:3] - INFO  
> [LearnerHandler-/192.168.1.2:35263:LearnerHandler@458] - Sending SNAP
> 2016-04-21 11:07:59,156 [myid:3] - INFO  
> [LearnerHandler-/192.168.1.2:35263:LearnerHandler@482] - Sending snapshot 
> last zxid of peer is 0x0  zxid of leader is 0x2sent zxid of db as 
> 0x2
> 2016-04-21 11:07:59,212 [myid:3] - INFO  
> [LearnerHandler-/192.168.1.2:35263:LearnerHandler@518] - Received 
> NEWLEADER-ACK message from 1
> Eric Gao
> Keep on going never give up.
> Blog:
> http://gaoqiang.blog.chinaunix.net/
> http://gaoqiangdba.blog.163.com/
> 
> 
>  
> From: Ted Yu
> Date: 2016-04-20 22:31
> To: Eric Gao
> CC: user
> Subject: Re: Re: ERROR [main] 
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not in 
> ZooKeeper.
> bq. dataDir=/tmp/zookeeper
>  
> When machine restarts, you would lose the data, right ?
> Please change to other directory.
>  
> Was zookeeper.out from slave2 ?
>  
> Please check port  on 192.168.1.3 <http://192.168.1.3:/>
>  
> On Wed, Apr 20, 2016 at 6:22 AM, Eric Gao <gaoqiang...@163.com> wrote:
>  
> > Dear Ted,
> > Thank you for your kind attention.
> > I'm a complete novice at Hadoop. ^___^
> >
> >
> >
> > I have 3 host---1 master server and 2 data node,and zookeeper is installed 
> > on each server.
> > Zookeeper and Hbase are still unable to start.
> >
> > Each server has the same status:
> >
> > [root@master data]# /opt/zookeeper/bin/zkServer.sh start
> > ZooKeeper JMX enabled by default
> > Using config: /opt/zookeeper/bin/../conf/zoo.cfg
> > Starting zookeeper ... date
> > STARTED
> > [root@slave2 ~]# /opt/zookeeper/bin/zkServer.sh status
> > ZooKeeper JMX enabled by default
> > Using config: /opt/zookeeper/bin/../conf/zoo.cfg
> > Error contacting service. It is probably not running.
> >
> >
> > Here are some of my configuration infomation:
> >
> >
> > *hbase-site.xml:*
> > 
> > 
> > hbase.rootdir
> > hdfs://master:9000/hbase/data
> > 
> > 
> > hbase.cluster.distributed
> > true
> > 
> >
> > 
> > zookeeper.znode.parent
> > /hbase
> > Root ZNode for HBase in ZooKeeper. All of HBase's ZooKeeper
> > files that are configured with a relative path will go under this node.
> > By default, all of HBase's ZooKeeper file path are configured with a
> > relative path, so they will all go under this directory unless changed.
> > 
> > 
> >
> &g

Re: Re: ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper.

[Ted Yu]

onnect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
>
> at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
>
> at 
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:426)
>
> at 
> org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:843)
> at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:822)
>
>
>
>
> *core-site.xml:*
> 
> 
> hadoop.tmp.dir
> /opt/hadoop/tmp
> A base for other temporary directories.
> 
> 
> 
> fs.default.name
> hdfs://master:9000
> 
> 
>
> Please help to check the problem. Thanks a lot!
>
> --
> *Eric Gao*
> Keep on going never give up.
> *Blog:*
> http://gaoqiang.blog.chinaunix.net/
> http://gaoqiangdba.blog.163.com/
>
>
>
> *From:* Ted Yu <yuzhih...@gmail.com>
> *Date:* 2016-04-16 23:21
> *To:* user@hbase.apache.org
> *Subject:* Re: Re: ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper.
> Can you verify that hbase is running by logging onto master node and check
> the Java processes ?
>
> If master is running, can you do a listing of the zookeeper znode (using
> zkCli) and pastebin the result ?
>
> Thanks
>
> On Sat, Apr 16, 2016 at 8:14 AM, Eric Gao <gaoqiang...@163.com> wrote:
>
> > Yes,I have seen your reply.Thanks very much for your kindness.
> >
> > This is my hbase-site.xml:
> > 
> > 
> > hbase.rootdir
> > hdfs://master:9000/hbase/data
> > 
> > 
> > hbase.cluster.distributed
> > true
> > 
> >
> > 
> > zookeeper.znode.parent
> > /hbase
> > Root ZNode for HBase in ZooKeeper. All of HBase's
> > ZooKeeper
> >   files that are configured with a relative path will go under this
> > node.
> >   By default, all of HBase's ZooKeeper file path are configured with
> a
> >   relative path, so they will all go under this directory unless
> > changed.
> > 
> >   
> >
> >
> > 
> > hbase.zookeeper.quorum
> > master,slave1,slave2
> > Comma separated list of servers in the ZooKeeper Quorum. For
> > example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". By
> > default this is set to localhost for local and pseudo-distributed modes
> of
> > operation. For a fully-distributed setup, this should be set to a full
> list
> > of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
> > this is the list of servers which we will start/stop ZooKeeper on.
> > 
> > 
> > 
> > hbase.zookeeper.property.dataDir
> > /opt/zookeeper/data
> > Property from ZooKeeper's config zoo.cfg. The directory
> where
> > the snapshot is stored. 
> > 
> > 
> >
> > This is my hbase-env.sh:
> >
> >
> > [root@master ~]# cat /opt/hbase/conf/hbase-env.sh
> > #
> > #/**
> > # * Licensed to the Apache Software Foundation (ASF) under one
> > # * or more contributor license agreements.  See the NOTICE file
> > # * distributed with this work for additional information
> > # * regarding copyright ownership.  The ASF licenses this file
> > # * to you under the Apache License, Version 2.0 (the
> > # * "License"); you may not use this file except in compliance
> > # * with the License.  You may obtain a copy of the License at
> > # *
> > # * http://www.apache.org/licenses/LICENSE-2.0
> > # *
> > # * Unless required by applicable law or agreed to in writing, software
> > # * distributed under the License is distributed on an "AS IS" BASIS,
> > # * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied.
> > # * See the License for the specific language governing permissions and
> > # * limitations under the License.
> > # */
> >
> > # Set environment variables here.
> >
> > # This script sets variables multiple times over the course of starting
> an
> > hbase process,
> > # so try to keep things idempotent unless you want to take an even deeper
> > look
> > # into the startup scripts (bin/hbase, etc.)
> >
> > # The java implementation to use.  Java 1.7+ required.
> > export JAVA_HOME=/usr
> >
> > # Extra Java CLASSPATH elements.  Optional.
> >  export HBASE_CLASSPATH=/opt/hadoop
> >
>

Re: Re: single thread threadpool for master_table_operation

[Ted Yu]

bq. all cp of all tables

What does cp mean ? Do you mean column family ?

bq. will never create/enable tables in the future

Then what would the cluster be used for ?

On Wed, Apr 20, 2016 at 6:16 AM, WangYQ <wangyongqiang0...@163.com> wrote:

> i want to remove all cp of all tables in hbase
> so i disable tables concurrently, and then modify tables
> for hundreds of tables, remove all cp costs 30 miniutes,  is not so fast
>
>
> so, i want to speed the whoe process. and will never create/enable tables
> in the future
>
>
> after examine the code, i want ot increase the  pool size for
> master_table_operation
>
>
> but not sure if there are any problems.
>
>
> thanks..
>
> At 2016-04-20 21:07:06, "Ted Yu" <yuzhih...@gmail.com> wrote:
> >Adding subject.
> >
> >Adding back user@hbase
> >
> >But the master wouldn't know what next action admin is going to perform,
> >right ?
> >
> >On Wed, Apr 20, 2016 at 5:59 AM, WangYQ <wangyongqiang0...@163.com>
> wrote:
> >
> >> if i just disable tables in concurrently,
> >> and will never enable, modify, create table
> >>
> >> i think is ok, right?
> >>
> >>
> >>
> >>
> >> 在2016年04月20日 20:53 ，Ted Yu <yuzhih...@gmail.com>写道：
> >>
> >>
> >> Have you seen the comment above that line ?
> >>
> >>// We depend on there being only one instance of this executor
> running
> >>// at a time.  To do concurrency, would need fencing of
> enable/disable
> >> of
> >>// tables.
> >>// Any time changing this maxThreads to > 1, pls see the comment at
> >>// AccessController#postCreateTableHandler
> >>
> >> BTW in the future, please send queries to user@hbase
> >>
> >> On Wed, Apr 20, 2016 at 5:50 AM, WangYQ <wangyongqiang0...@163.com>
> wrote:
> >>
> >>> hbase 0.98.10
> >>>
> >>> in class hmaster
> >>> line 1298, the threadpool size for master_table_operation is 1, can not
> >>> be set
> >>>
> >>> are there any problems if i disable tables in concurrently
> >>>
> >>> thanks
> >>
> >>
> >>
> >>
> >>
>

Re: single thread threadpool for master_table_operation

[Ted Yu]

Adding subject.

Adding back user@hbase

But the master wouldn't know what next action admin is going to perform,
right ?

On Wed, Apr 20, 2016 at 5:59 AM, WangYQ <wangyongqiang0...@163.com> wrote:

> if i just disable tables in concurrently,
> and will never enable, modify, create table
>
> i think is ok, right?
>
>
>
>
> 在2016年04月20日 20:53 ，Ted Yu <yuzhih...@gmail.com>写道：
>
>
> Have you seen the comment above that line ?
>
>// We depend on there being only one instance of this executor running
>// at a time.  To do concurrency, would need fencing of enable/disable
> of
>// tables.
>// Any time changing this maxThreads to > 1, pls see the comment at
>// AccessController#postCreateTableHandler
>
> BTW in the future, please send queries to user@hbase
>
> On Wed, Apr 20, 2016 at 5:50 AM, WangYQ <wangyongqiang0...@163.com> wrote:
>
>> hbase 0.98.10
>>
>> in class hmaster
>> line 1298, the threadpool size for master_table_operation is 1, can not
>> be set
>>
>> are there any problems if i disable tables in concurrently
>>
>> thanks
>
>
>
>
>

Re: (无主题)

[Ted Yu]

Have you seen the comment above that line ?

   // We depend on there being only one instance of this executor running
   // at a time.  To do concurrency, would need fencing of enable/disable of
   // tables.
   // Any time changing this maxThreads to > 1, pls see the comment at
   // AccessController#postCreateTableHandler

BTW in the future, please send queries to user@hbase

On Wed, Apr 20, 2016 at 5:50 AM, WangYQ  wrote:

> hbase 0.98.10
>
> in class hmaster
> line 1298, the threadpool size for master_table_operation is 1, can not be
> set
>
> are there any problems if i disable tables in concurrently
>
> thanks

Re: Major Compaction Strategy

[Ted Yu]

Please use the following method of HBaseAdmin:

  public CompactionState getCompactionStateForRegion(final byte[]
regionName)

Cheers

On Tue, Apr 19, 2016 at 12:56 PM, Saad Mufti  wrote:

> Hi,
>
> We have a large HBase 1.x cluster in AWS and have disabled automatic major
> compaction as advised. We were running our own code for compaction daily
> around midnight which calls HBaseAdmin.majorCompactRegion(byte[]
> regionName) in a rolling fashion across all regions.
>
> But we missed the fact that this is an asynchronous operation, so in
> practice this causes major compaction to run across all regions, at least
> those not already major compacted (for example because previous minor
> compactions got upgraded to major ones).
>
> We don't really have a suitable low load period, so what is a suitable way
> to make major compaction run in a rolling fashion region by region? The API
> above provides no return value for us to be able to wait for one compaction
> to finish before moving to the next.
>
> Thanks.
>
> 
> Saad
>

Re: Processing rows in parallel with MapReduce jobs.

[Ted Yu]

>From the error, you need to provide an argumentless ctor for
MyTableInputFormat.

On Tue, Apr 19, 2016 at 12:12 AM, Ivan Cores gonzalez <ivan.co...@inria.fr>
wrote:

>
> Hi Ted,
>
> Sorry, I forgot to write the error. In runtime I have the next exception:
>
> Exception in thread "main" java.lang.RuntimeException:
> java.lang.NoSuchMethodException:
> simplerowcounter.SimpleRowCounter$MyTableInputFormat.()
>
> the program works fine if I don't use "MyTableInputFormat" modifying the
> call to initTableMapperJob:
>
> TableMapReduceUtil.initTableMapperJob(tableName, scan,
> RowCounterMapper.class,
> ImmutableBytesWritable.class, Result.class, job);   // -->
> works fine without MyTableInputFormat
>
> That's why I asked If you see any problem in the code. Because maybe I
> forgot override some method or something is missing.
>
> Best,
> Iván.
>
>
> - Mensaje original -
> > De: "Ted Yu" <yuzhih...@gmail.com>
> > Para: user@hbase.apache.org
> > Enviados: Martes, 19 de Abril 2016 0:22:05
> > Asunto: Re: Processing rows in parallel with MapReduce jobs.
> >
> > Did you see the "Message to log?" log ?
> >
> > Can you pastebin the error / exception you got ?
> >
> > On Mon, Apr 18, 2016 at 1:54 AM, Ivan Cores gonzalez <
> ivan.co...@inria.fr>
> > wrote:
> >
> > >
> > >
> > > Hi Ted,
> > > So, If I understand the behaviour of getSplits(), I can create
> "virtual"
> > > splits overriding the getSplits function.
> > > I was performing some tests, but my code crash in runtime and I cannot
> > > found the problem.
> > > Any help? I didn't find examples.
> > >
> > >
> > > public class SimpleRowCounter extends Configured implements Tool {
> > >
> > >   static class RowCounterMapper extends
> > > TableMapper<ImmutableBytesWritable, Result> {
> > > public static enum Counters { ROWS }
> > > @Override
> > > public void map(ImmutableBytesWritable row, Result value, Context
> > > context) {
> > >   context.getCounter(Counters.ROWS).increment(1);
> > > try {
> > > Thread.sleep(3000); //Simulates work
> > > } catch (InterruptedException name) { }
> > > }
> > >   }
> > >
> > >   public class MyTableInputFormat extends TableInputFormat {
> > > @Override
> > > public List getSplits(JobContext context) throws
> > > IOException {
> > > //Just to detect if this method is being called ...
> > > List splits = super.getSplits(context);
> > > System.out.printf("Message to log? \n" );
> > > return splits;
> > > }
> > >   }
> > >
> > >   @Override
> > >   public int run(String[] args) throws Exception {
> > > if (args.length != 1) {
> > >   System.err.println("Usage: SimpleRowCounter ");
> > >   return -1;
> > > }
> > > String tableName = args[0];
> > >
> > > Scan scan = new Scan();
> > > scan.setFilter(new FirstKeyOnlyFilter());
> > > scan.setCaching(500);
> > > scan.setCacheBlocks(false);
> > >
> > > Job job = new Job(getConf(), getClass().getSimpleName());
> > > job.setJarByClass(getClass());
> > >
> > > TableMapReduceUtil.initTableMapperJob(tableName, scan,
> > > RowCounterMapper.class,
> > > ImmutableBytesWritable.class, Result.class, job, true,
> > > MyTableInputFormat.class);
> > >
> > > job.setNumReduceTasks(0);
> > > job.setOutputFormatClass(NullOutputFormat.class);
> > > return job.waitForCompletion(true) ? 0 : 1;
> > >   }
> > >
> > >   public static void main(String[] args) throws Exception {
> > > int exitCode = ToolRunner.run(HBaseConfiguration.create(),
> > > new SimpleRowCounter(), args);
> > > System.exit(exitCode);
> > >   }
> > > }
> > >
> > > Thanks so much,
> > > Iván.
> > >
> > >
> > >
> > >
> > > - Mensaje original -
> > > > De: "Ted Yu" <yuzhih...@gmail.com>
> > > > Para: user@hbase.apache.org
> > > > Enviados: Martes, 12 de Abril 2016 17:29:52
> > > > Asunto: Re: 

Re: Processing rows in parallel with MapReduce jobs.

[Ted Yu]

Did you see the "Message to log?" log ?

Can you pastebin the error / exception you got ?

On Mon, Apr 18, 2016 at 1:54 AM, Ivan Cores gonzalez <ivan.co...@inria.fr>
wrote:

>
>
> Hi Ted,
> So, If I understand the behaviour of getSplits(), I can create "virtual"
> splits overriding the getSplits function.
> I was performing some tests, but my code crash in runtime and I cannot
> found the problem.
> Any help? I didn't find examples.
>
>
> public class SimpleRowCounter extends Configured implements Tool {
>
>   static class RowCounterMapper extends
> TableMapper<ImmutableBytesWritable, Result> {
> public static enum Counters { ROWS }
> @Override
> public void map(ImmutableBytesWritable row, Result value, Context
> context) {
>   context.getCounter(Counters.ROWS).increment(1);
> try {
> Thread.sleep(3000); //Simulates work
> } catch (InterruptedException name) { }
> }
>   }
>
>   public class MyTableInputFormat extends TableInputFormat {
> @Override
> public List getSplits(JobContext context) throws
> IOException {
> //Just to detect if this method is being called ...
> List splits = super.getSplits(context);
> System.out.printf("Message to log? \n" );
> return splits;
> }
>   }
>
>   @Override
>   public int run(String[] args) throws Exception {
> if (args.length != 1) {
>   System.err.println("Usage: SimpleRowCounter ");
>   return -1;
> }
> String tableName = args[0];
>
> Scan scan = new Scan();
> scan.setFilter(new FirstKeyOnlyFilter());
> scan.setCaching(500);
> scan.setCacheBlocks(false);
>
> Job job = new Job(getConf(), getClass().getSimpleName());
> job.setJarByClass(getClass());
>
> TableMapReduceUtil.initTableMapperJob(tableName, scan,
> RowCounterMapper.class,
> ImmutableBytesWritable.class, Result.class, job, true,
> MyTableInputFormat.class);
>
> job.setNumReduceTasks(0);
> job.setOutputFormatClass(NullOutputFormat.class);
> return job.waitForCompletion(true) ? 0 : 1;
>   }
>
>   public static void main(String[] args) throws Exception {
> int exitCode = ToolRunner.run(HBaseConfiguration.create(),
> new SimpleRowCounter(), args);
> System.exit(exitCode);
>   }
> }
>
> Thanks so much,
> Iván.
>
>
>
>
> - Mensaje original -
> > De: "Ted Yu" <yuzhih...@gmail.com>
> > Para: user@hbase.apache.org
> > Enviados: Martes, 12 de Abril 2016 17:29:52
> > Asunto: Re: Processing rows in parallel with MapReduce jobs.
> >
> > Please take a look at TableInputFormatBase#getSplits() :
> >
> >* Calculates the splits that will serve as input for the map tasks.
> The
> >
> >* number of splits matches the number of regions in a table.
> >
> > Each mapper would be reading one of the regions.
> >
> > On Tue, Apr 12, 2016 at 8:18 AM, Ivan Cores gonzalez <
> ivan.co...@inria.fr>
> > wrote:
> >
> > > Hi Ted,
> > > Yes, I mean same region.
> > >
> > > I wasn't using the getSplits() function. I'm trying to add it to my
> code
> > > but I'm not sure how I have to do it. Is there any example in the
> website?
> > > I can not find anything. (By the way, I'm using TableInputFormat, not
> > > InputFormat)
> > >
> > > But just to confirm, with the getSplits() function, Are mappers
> processing
> > > rows in the same region executed in parallel? (assuming that there are
> > > empty
> > > processors/cores)
> > >
> > > Thanks,
> > > Ivan.
> > >
> > >
> > > - Mensaje original -
> > > > De: "Ted Yu" <yuzhih...@gmail.com>
> > > > Para: user@hbase.apache.org
> > > > Enviados: Lunes, 11 de Abril 2016 15:10:29
> > > > Asunto: Re: Processing rows in parallel with MapReduce jobs.
> > > >
> > > > bq. if they are located in the same split?
> > > >
> > > > Probably you meant same region.
> > > >
> > > > Can you show the getSplits() for the InputFormat of your MapReduce
> job ?
> > > >
> > > > Thanks
> > > >
> > > > On Mon, Apr 11, 2016 at 5:48 AM, Ivan Cores gonzalez <
> > > ivan.co...@inria.fr>
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > 

Re: maven dependency resources for hbase .98 and apache-spark

[Ted Yu]

The referenced link is from a specific vendor.

Mind posting on the vendor's mailing list ?

On Mon, Apr 18, 2016 at 12:45 PM, Colin Kincaid Williams 
wrote:

> I would like to insert some data from Spark and or Spark streaming
> into Hbase, on v .98. I found this section of the book which shows
> examples of using the apis:
> https://archive.cloudera.com/cdh5/cdh/5/hbase/book.html#spark .
>
> However I'm unsure what dependency sections I need to add to my maven
> manifest. Does anybody have any references?
>

Re: To Store Large Number of Video and Image files

[Ted Yu]

There was HBASE-15370 for backport but it was decided not to backport the
feature.

FYI

On Sat, Apr 16, 2016 at 7:26 PM, Ascot Moss <ascot.m...@gmail.com> wrote:

> Hi,
>
> About HBase-11339,
> "The size of the MOB data could not be very large, it better to keep the
> MOB size within 100KB and 10MB. Since MOB cells are written into the
> memstore before flushing, large MOB cells stress the memory in region
> servers."
>
> Can this be resolved if we provide more RAM in region servers? for
> instances, the servers in the cluster, each has 768GB RAM + 14 x 6T HDD.
>
> regards
>
>
>
> On Sun, Apr 17, 2016 at 9:56 AM, Ascot Moss <ascot.m...@gmail.com> wrote:
>
> > Thanks Ted!
> >
> > Just visited HBASE-11339, its status is "resolved" however, it is for
> > "Fix Version : 2.0.0."
> > How to patch it to current HBase stable version (v1.1.4) ?
> >
> > About Fault Tolerance to DataCenter level, I am thinking HBase
> Replication
> > method to replicate HBase Tables to another cluster (backup one), is
> there
> > any real world reference about the replication performance, for instances
> > if the bandwidth is 100MB/s?
> >
> > Regards
> >
> >
> >
> >
> >
> >
> >-
> >
> >
> > On Sun, Apr 17, 2016 at 9:40 AM, Ted Yu <yuzhih...@gmail.com> wrote:
> >
> >> Have you taken a look at HBASE-11339 (HBase MOB) ?
> >>
> >> Note: this feature does not handle 10GB objects well. Consider store GB
> >> image on hdfs.
> >>
> >> Cheers
> >>
> >> On Sat, Apr 16, 2016 at 6:21 PM, Ascot Moss <ascot.m...@gmail.com>
> wrote:
> >>
> >> > Hi,
> >> >
> >> > I have a project that needs to store large number of image and video
> >> files,
> >> > the file size varies from 10MB to 10GB, the initial number of files
> >> will be
> >> > 0.1 billion and would grow over 1 billion, what will be the practical
> >> > recommendations to store and view these files?
> >> >
> >> >
> >> >
> >> > #1 One cluster, store the HDFS URL in HBase and store the actual file
> in
> >> > HDFS? (block_size as 128MB and replication factor as 3)
> >> >
> >> >
> >> > #2 One cluster, Store small files in HBase directly and use #1 for
> large
> >> > files? (block_size as 128MB and replication factor as 3)
> >> >
> >> >
> >> > #3 Multiple Hadoop/HBase clusters, each with different block_size
> >> settings?
> >> >
> >> >
> >> >  e.g. cluster 1 (small): block_size as 128MB and replication
> factor
> >> as
> >> > 3, store all files in HBase if their file size is smaller 128MB
> >> >
> >> > cluster 2 (large): bigger block_size, say 4GB, replication
> >> > factor as 3, store the HDFS URL in HBase and store the actual file in
> >> HDFS
> >> >
> >> >
> >> >
> >> > #4 Use Hadoop Federation for large number of files?
> >> >
> >> >
> >> > About Fault Tolerance, need to consider four types of failures:
> driver,
> >> > host, rack, and  datacenter failures.
> >> >
> >> >
> >> > Regards
> >> >
> >>
> >
> >
>

Re: To Store Large Number of Video and Image files

[Ted Yu]

Have you taken a look at HBASE-11339 (HBase MOB) ?

Note: this feature does not handle 10GB objects well. Consider store GB
image on hdfs.

Cheers

On Sat, Apr 16, 2016 at 6:21 PM, Ascot Moss  wrote:

> Hi,
>
> I have a project that needs to store large number of image and video files,
> the file size varies from 10MB to 10GB, the initial number of files will be
> 0.1 billion and would grow over 1 billion, what will be the practical
> recommendations to store and view these files?
>
>
>
> #1 One cluster, store the HDFS URL in HBase and store the actual file in
> HDFS? (block_size as 128MB and replication factor as 3)
>
>
> #2 One cluster, Store small files in HBase directly and use #1 for large
> files? (block_size as 128MB and replication factor as 3)
>
>
> #3 Multiple Hadoop/HBase clusters, each with different block_size settings?
>
>
>  e.g. cluster 1 (small): block_size as 128MB and replication factor as
> 3, store all files in HBase if their file size is smaller 128MB
>
> cluster 2 (large): bigger block_size, say 4GB, replication
> factor as 3, store the HDFS URL in HBase and store the actual file in HDFS
>
>
>
> #4 Use Hadoop Federation for large number of files?
>
>
> About Fault Tolerance, need to consider four types of failures: driver,
> host, rack, and  datacenter failures.
>
>
> Regards
>

Re: Re: ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper.

[Ted Yu]

 my env:
> [hadoop@master ~]$ env
> XDG_SESSION_ID=1
> HOSTNAME=master
> SHELL=/bin/bash
> TERM=xterm
> HADOOP_HOME=/opt/hadoop
> HISTSIZE=1000
> USER=hadoop
>
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
> MAIL=/var/spool/mail/hadoop
>
> PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/hadoop/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64/jre/bin:/usr/bin:/home/hadoop/.local/bin:/home/hadoop/bin
> PWD=/home/hadoop
> JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64
> LANG=en_US.UTF-8
> HISTCONTROL=ignoredups
> SHLVL=1
> HOME=/home/hadoop
> LOGNAME=hadoop
>
> CLASSPATH=.::/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64/lib:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64/jre/lib
> LESSOPEN=||/usr/bin/lesspipe.sh %s
> _=/bin/env
>
>
>
> Eric Gao
> Keep on going never give up.
> Blog:
> http://gaoqiang.blog.chinaunix.net/
> http://gaoqiangdba.blog.163.com/
>
>
>
> From: Ted Yu
> Date: 2016-04-16 22:59
> To: user@hbase.apache.org
> Subject: Re: ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper.
> Have you seen my reply ?
>
> http://search-hadoop.com/m/q3RTtJHewi1jOgc21
>
> The actual value for zookeeper.znode.parent could be /hbase-secure (just an
> example).
>
> Make sure the correct hbase-site,xml is in classpath for hbase shell.
>
> On Sat, Apr 16, 2016 at 7:53 AM, Eric Gao <gaoqiang...@163.com> wrote:
>
> > Dear expert,
> >   I have encountered a problem,when I run hbase cmd :status it shows:
> >
> > hbase(main):001:0> status
> > 2016-04-16 13:03:02,333 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:02,538 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:02,843 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:03,348 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:04,355 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:06,369 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> > 2016-04-16 13:03:10,414 ERROR [main]
> > client.ConnectionManager$HConnectionImplementation: The node /hbase is
> not
> > in ZooKeeper. It should have been written by the master. Check the value
> > configured in 'zookeeper.znode.parent'. There could be a mismatch with
> the
> > one configured in the master.
> >
> > How can I solve the problem?
> > Thanks very much
> >
> >
> >
> > Eric Gao
> > Keep on going never give up.
> > Blog:
> > http://gaoqiang.blog.chinaunix.net/
> > http://gaoqiangdba.blog.163.com/
> >
> >
> >
>

Re: ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper.

[Ted Yu]

Have you seen my reply ?

http://search-hadoop.com/m/q3RTtJHewi1jOgc21

The actual value for zookeeper.znode.parent could be /hbase-secure (just an
example).

Make sure the correct hbase-site,xml is in classpath for hbase shell.

On Sat, Apr 16, 2016 at 7:53 AM, Eric Gao  wrote:

> Dear expert,
>   I have encountered a problem,when I run hbase cmd :status it shows:
>
> hbase(main):001:0> status
> 2016-04-16 13:03:02,333 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:02,538 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:02,843 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:03,348 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:04,355 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:06,369 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
> 2016-04-16 13:03:10,414 ERROR [main]
> client.ConnectionManager$HConnectionImplementation: The node /hbase is not
> in ZooKeeper. It should have been written by the master. Check the value
> configured in 'zookeeper.znode.parent'. There could be a mismatch with the
> one configured in the master.
>
> How can I solve the problem?
> Thanks very much
>
>
>
> Eric Gao
> Keep on going never give up.
> Blog:
> http://gaoqiang.blog.chinaunix.net/
> http://gaoqiangdba.blog.163.com/
>
>
>

Re: Read HFile from Local file system for studying and testing

[Ted Yu]

Please take a look at:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java

There're various tests under hbase-server/src/test which demonstrate the
usage.

FYI

On Thu, Apr 14, 2016 at 8:03 PM, Bin Wang  wrote:

> Hi there,
>
> I have a HFile that I copied from HDFS down to my laptop.
>
> I am trying to use the HFile.createreader to read in that HFile and play
> with it. However, it was really hard to figure that out how to read it from
> local file system.
>
> public class MyHbase {
> public static void main(String[] args) throws IOException {
>
> Path p = new Path("/Desktop/testHFile");
> Configuration conf = new Configuration();
> //  FileSystem localfs = FileSystem.newInstanceLocal(conf);
> //  FileSystem localfs =
> FileSystem.getLocal(conf).getRawFileSystem();
> //  FileSystem localfs = p.getFileSystem(conf);
>  RawLocalFileSystem localfs = new RawLocalFileSystem();
> System.out.println(localfs.toString());
> HFile.Reader reader = HFile.createReader(localfs, p, new
> CacheConfig(conf), conf);
> reader.loadFileInfo();
>
> HFileScanner scanner = reader.getScanner(false, false);
> scanner.seekTo();
> int count = 0;
> do {
> count++;
> } while (scanner.next());
> System.out.println(count);
> reader.close();
> }
> }
>
> And it is giving the error message:
>
> LocalFS
> 16/04/13 14:54:25 INFO hfile.CacheConfig: Allocating LruBlockCache
> size=1.42 GB, blockSize=64 KB
> 16/04/13 14:54:25 INFO hfile.CacheConfig:
> blockCache=LruBlockCache{blockCount=0, currentSize=1567264,
> freeSize=1525578848, maxSize=1527146112, heapSize=1567264,
> minSize=1450788736, minFactor=0.95, multiSize=725394368,
> multiFactor=0.5, singleSize=362697184, singleFactor=0.25},
> cacheDataOnRead=true, cacheDataOnWrite=false,
> cacheIndexesOnWrite=false, cacheBloomsOnWrite=false,
> cacheEvictOnClose=false, cacheDataCompressed=false,
> prefetchOnOpen=false
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
> at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:98)
> at
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:79)
> at
> org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:519)
> at myhbase.MyHbase.main(MyHbase.java:23)
>
>
> I asked this question
> <
> http://stackoverflow.com/questions/36609428/trying-to-read-hfile-using-hfile-createreader-from-local-file-system
> >on
> Stackoverflow and seems like there are not that many hbase people there.
>
> Can anyone on this mailing-list help me on that?
>
> Best regards,
>
> Bin
>

Re: Append Visibility Labels?

[Ted Yu]

There is currently no API for appending Visibility Labels.

checkAndPut() only allows you to compare value, not labels.

On Wed, Apr 13, 2016 at 8:12 AM, 
wrote:

> We sell data. A product can be defined as a permission to access data (at
> a cell level). Visibility Labels look like a very good candidate for
> implementing this model.
>
> The implementation works well until we create a new product over old data.
> We can set the visibility label for the new product but, whoops, by
> applying it to the relevant cells we've overwritten all the existing labels
> on those cells, destroying the permissioning of our older products. What to
> do?
>
> One answer would be to append the new visibility label to the existing
> label expressions on the cells with an 'OR'. But I'm not sure that's
> possible .. yet?
>
> Thanks,
>
> Ben
>
> 
>
> This e-mail is for the sole use of the intended recipient and contains
> information that may be privileged and/or confidential. If you are not an
> intended recipient, please notify the sender by return e-mail and delete
> this e-mail and any attachments. Certain required legal entity disclosures
> can be accessed on our website.<
> http://site.thomsonreuters.com/site/disclosures/>
>

Re: Processing rows in parallel with MapReduce jobs.

[Ted Yu]

Please take a look at TableInputFormatBase#getSplits() :

   * Calculates the splits that will serve as input for the map tasks. The

   * number of splits matches the number of regions in a table.

Each mapper would be reading one of the regions.

On Tue, Apr 12, 2016 at 8:18 AM, Ivan Cores gonzalez <ivan.co...@inria.fr>
wrote:

> Hi Ted,
> Yes, I mean same region.
>
> I wasn't using the getSplits() function. I'm trying to add it to my code
> but I'm not sure how I have to do it. Is there any example in the website?
> I can not find anything. (By the way, I'm using TableInputFormat, not
> InputFormat)
>
> But just to confirm, with the getSplits() function, Are mappers processing
> rows in the same region executed in parallel? (assuming that there are
> empty
> processors/cores)
>
> Thanks,
> Ivan.
>
>
> - Mensaje original -
> > De: "Ted Yu" <yuzhih...@gmail.com>
> > Para: user@hbase.apache.org
> > Enviados: Lunes, 11 de Abril 2016 15:10:29
> > Asunto: Re: Processing rows in parallel with MapReduce jobs.
> >
> > bq. if they are located in the same split?
> >
> > Probably you meant same region.
> >
> > Can you show the getSplits() for the InputFormat of your MapReduce job ?
> >
> > Thanks
> >
> > On Mon, Apr 11, 2016 at 5:48 AM, Ivan Cores gonzalez <
> ivan.co...@inria.fr>
> > wrote:
> >
> > > Hi all,
> > >
> > > I have a small question regarding the MapReduce jobs behaviour with
> HBase.
> > >
> > > I have a HBase test table with only 8 rows. I splitted the table with
> the
> > > hbase shell
> > > split command into 2 splits. So now there are 4 rows in every split.
> > >
> > > I create a MapReduce job that only prints the row key in the log files.
> > > When I run the MapReduce job, every row is processed by 1 mapper. But
> the
> > > mappers
> > > in the same split are executed sequentially (inside the same
> container).
> > > That means,
> > > the first four rows are processed sequentially by 4 mappers. The system
> > > has cores
> > > that are free, so is it possible to process rows in parallel if they
> are
> > > located
> > > in the same split?
> > >
> > > The only way I found to have 8 mappers executed in parallel is split
> the
> > > table
> > > in 8 splits (1 split per row). But obviously this is not the best
> solution
> > > for
> > > big tables ...
> > >
> > > Thanks,
> > > Ivan.
> > >
> >
>

Re: unstable cluster

[Ted Yu]

>From region server log:

2016-04-11 03:11:51,589 WARN org.apache.zookeeper.ClientCnxnSocket:
Connected to an old server; r-o mode will be unavailable
2016-04-11 03:11:51,589 INFO org.apache.zookeeper.ClientCnxn: Unable to
reconnect to ZooKeeper service, session 0x52ee1452fec5ac has expired,
closing socket connection

>From zookeeper log:

2016-04-11 03:11:27,323 - INFO  [CommitProcessor:0:NIOServerCnxn@1435] -
Closed socket connection for client /172.20.67.19:58404 which had sessionid
0x52ee1452fec71f
2016-04-11 03:11:53,301 - INFO  [CommitProcessor:0:NIOServerCnxn@1435] -
Closed socket connection for client /172.20.67.13:32946 which had sessionid
0x52ee1452fec6ea

Note the 26 second gap.

What do you see in the logs of the other two zookeeper servers ?

Thanks

On Mon, Apr 11, 2016 at 5:08 PM, Ted Tuttle  wrote:

> Hello -
>
> We've started experiencing regular failures of our HBase cluster.  For the
> last week we've had nightly failures about 1hr after a heavy batch process
> starts.
>
> In the logs below we see the failure starting at 2016-04-11 03:11 in
> zookeeper, master and region server logs:
>
> zookeeper:  http://pastebin.com/kf7ja22K
>
> region server: http://pastebin.com/tduJgKqq
>
> master:  http://pastebin.com/0szhi0bJ
>
> The master log seems most interesting.  Here we see problems connecting to
> Zookeeper then a number of region servers dying in quick succession.  From
> the log evidence it appears Zookeeper is not responding rather than the
> more typical GC causing isolated RS to abort.
>
> Any insights on what may be happening here?
>
> Best,
> Ted
>

Re: How to serialize Result in HBase 1.0

[Ted Yu]

It seems your code didn't go through.

Please take a look at ResultSerialization and related classes.

Cheers

On Mon, Apr 11, 2016 at 5:29 PM, 乔彦克  wrote:

> Hi, all
> recently we upgrade our HBase  cluster from cdh-0.94 to cdh-1.0. In 0.94
> we use Result.java(implement Writable) as the map out value.
> [image: pasted1]
> but int the newer HBase version Result.java has changed, it can't be
> serialized any more. Is there  any alternative methods to use Result as
>  MapReduce input/output?
> Thanks for any response.
>

Re: Question about open table

[Ted Yu]

bq. I am using hbase 2.x

2.0 has not been released yet.
Probably you meant 1.x ?

On Mon, Apr 11, 2016 at 6:48 AM, Yi Jiang <yi.ji...@ubisoft.com> wrote:

> Thanks
> I am using hbase 2.x, so only once to create the connection in my project.
> According to Ted, the getTable is not expensive, then I am able to get and
> close table in each request.
> Jacky
>
> -Original Message-
> From: Yu Li [mailto:car...@gmail.com]
> Sent: Monday, April 11, 2016 12:05 AM
> To: user@hbase.apache.org
> Subject: Re: Question about open table
>
> If using HBase 1.x, please make sure to reuse the connection since
> creating connection is expensive. More specified, don't call
> {{ConnectionFactory.createConnection}} (the 1.x recommended way) multiple
> times but only once and share it among threads, while
> {{HConnectionManager#getConnection}} (the <=0.98 way) would just be ok. FYI.
>
> Best Regards,
> Yu
>
> On 10 April 2016 at 14:08, Yi Jiang <yi.ji...@ubisoft.com> wrote:
>
> > Hi, Ted
> > Thanks for help. The getConnection is just a simple getter
> > method.public Connection getConnection(){return connection;} I have
> > configured my connection in constructor.
> > Jacky
> >
> > -Original Message-
> > From: Ted Yu [mailto:yuzhih...@gmail.com]
> > Sent: Saturday, April 09, 2016 11:03 AM
> > To: user@hbase.apache.org
> > Subject: Re: Question about open table
> >
> > Can you show the body of getConnection() ?
> >
> > getTable() itself is not expensive - assuming the same underlying
> > Connection.
> >
> > Cheers
> >
> > On Sat, Apr 9, 2016 at 7:18 AM, Yi Jiang <yi.ji...@ubisoft.com> wrote:
> >
> > > Hi, Guys
> > > I just have a question, I am trying to save the data into table in
> > > HBase I am using
> > >
> > > Table table =
> > > getConnection().getTable(TableName.valueOf(tableName));
> > > ...
> > > ...
> > > Table.close
> > >
> > > My question is that, is the "getTable" expensive?
> > > Shall I use that get table and close table in each saving?
> > > Or I just get table at the beginning, saving all data and then close
> > > the table after all saving.
> > > Because from the aspect of code design, I will save data into the
> > > different table on the flying. So, if it is expensive, I would like
> > > to rewrite the design.
> > > Thank you
> > > Jacky
> > >
> >
>

Re: Processing rows in parallel with MapReduce jobs.

[Ted Yu]

bq. if they are located in the same split?

Probably you meant same region.

Can you show the getSplits() for the InputFormat of your MapReduce job ?

Thanks

On Mon, Apr 11, 2016 at 5:48 AM, Ivan Cores gonzalez 
wrote:

> Hi all,
>
> I have a small question regarding the MapReduce jobs behaviour with HBase.
>
> I have a HBase test table with only 8 rows. I splitted the table with the
> hbase shell
> split command into 2 splits. So now there are 4 rows in every split.
>
> I create a MapReduce job that only prints the row key in the log files.
> When I run the MapReduce job, every row is processed by 1 mapper. But the
> mappers
> in the same split are executed sequentially (inside the same container).
> That means,
> the first four rows are processed sequentially by 4 mappers. The system
> has cores
> that are free, so is it possible to process rows in parallel if they are
> located
> in the same split?
>
> The only way I found to have 8 mappers executed in parallel is split the
> table
> in 8 splits (1 split per row). But obviously this is not the best solution
> for
> big tables ...
>
> Thanks,
> Ivan.
>

Re: HBase TTL

[Ted Yu]

Have you looked at :

http://hbase.apache.org/book.html#ttl

Please describe your use case.

Thanks

On Mon, Apr 11, 2016 at 2:11 AM, hsdcl...@163.com  wrote:

> hi,
>
> I want to know the  principle of HBase TTL,I would like to use the same
> principle to develop Rowkey of TTL, HBase TTL now only for  Cell.
>
> thanks!
>
> hsdcl...@163.com
>

Re: HBase 1.2 master CPU usage spin

[Ted Yu]

Can you look at master log during this period to see what procedure was retried 
?

Turning on DEBUG logging if necessary and pastebin relevant portion of master 
log. 

Thanks

> On Apr 11, 2016, at 1:11 AM, Kevin Bowling  wrote:
> 
> Hi,
> 
> I'm running HBase 1.2.0 on FreeBSD via the ports system (
> http://www.freshports.org/databases/hbase/), and it is generally working
> well.  However, in an HA setup, the HBase master spins at 200% CPU usage
> when it is active and this follows the active master and disappears when
> standby.  Since this cluster is fairly idle, and an older production Linux
> one uses much less master CPU, I am curious what is going on.
> 
> Using visualvm, I can see that the ProcedureExecutor threads are quite
> busy.  Using dtrace, I can see that there is tons of native umtx activity.
> I am guessing that some procedure is failing, and continuously retrying as
> fast as the procedure dispatch allows.  Using dtrace, I can also see that a
> lot of time seems to be spent in the JVM's 'libzip.so' native library.  I'm
> wondering if it's a classloader run amok or something.
> 
> I need to rebuild the JVM with debugging to get more out of dtrace, but the
> JVM doesn't implement a dtrace ustack helper for FreeBSD like it does on
> Solarish.  Hopefully then I can get some idea of what is going on.
> 
> Does this speak to anyone for ideas to look into?  Other than noticing the
> CPU usage in top, the master seems to function fine and is responsive.
> 
> Here's a sample thread dump, this doesn't really jump out to me, but I
> don't know if any of my captures are at the right moment either:
> 
> "ProcedureExecutor-7" - Thread t@116
>   java.lang.Thread.State: WAITING
>at sun.misc.Unsafe.park(Native Method)
>- parking to wait for <4653d04c> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
>at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
>at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
>at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.poll(MasterProcedureScheduler.java:145)
>at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.poll(MasterProcedureScheduler.java:139)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:803)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:75)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:494)
> 
>   Locked ownable synchronizers:
>- None
> 
> "ProcedureExecutor-5" - Thread t@114
>   java.lang.Thread.State: WAITING
>at sun.misc.Unsafe.park(Native Method)
>- parking to wait for <4653d04c> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
>at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
>at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
>at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.poll(MasterProcedureScheduler.java:145)
>at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.poll(MasterProcedureScheduler.java:139)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:803)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:75)
>at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:494)
> 
>   Locked ownable synchronizers:
>- None
> 
> "ProcedureExecutor-4" - Thread t@113
>   java.lang.Thread.State: WAITING
>at sun.misc.Unsafe.park(Native Method)
>- parking to wait for <4653d04c> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
>at
> 

Re: choosing between hbase-spark / spark-hbase

[Ted Yu]

For 2.0.0-SNAPSHOT version, you should build trunk by yourself.

For 1.2.0-cdh5.7.0 , please contact cdh mailing list.

On Sun, Apr 10, 2016 at 7:09 PM, yeshwanth kumar <yeshwant...@gmail.com>
wrote:

> Thank you for the reply,
>
> i am having trouble in finding out the dependency in maven repository, the
> only dependency i can find is
>
>   
> org.apache.hbase
> hbase-spark
> 1.2.0-cdh5.7.0
> 
>
> from cloudera maven repository,
>
> dependency specified in this page, was not able to resolve
>
> http://hbase.apache.org/hbase-spark/dependency-info.html
>
> do i need to build it from the trunk?
>
> please let me know
>
> Thanks,
> Yeshwanth
>
> -Yeshwanth
> Can you Imagine what I would do if I could do all I can - Art of War
>
> On Tue, Apr 5, 2016 at 5:30 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > There are some outstanding bug fixes, e.g. HBASE-15333, for hbase-spark
> > module.
> >
> > FYI
> >
> > On Tue, Apr 5, 2016 at 2:36 PM, Nkechi Achara <nkach...@googlemail.com>
> > wrote:
> >
> > > So Hbase-spark is a continuation of the spark on hbase project, but
> > within
> > > the Hbase project.
> > > They are not any significant differences apart from the fact that Spark
> > on
> > > hbase is not updated.
> > > Dependent on the version you are using it would be more beneficial to
> use
> > > Hbase-Spark
> > >
> > > Kay
> > > On 5 Apr 2016 9:12 pm, "yeshwanth kumar" <yeshwant...@gmail.com>
> wrote:
> > >
> > > > i have cloudera cluster,
> > > > i am exploring spark with HBase,
> > > >
> > > > after going through this blog
> > > >
> > > >
> > > >
> > >
> >
> http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
> > > >
> > > > i found two options for using Spark with HBase,
> > > >
> > > > Cloudera's Spark on HBase or
> > > > Apache hbase-spark.
> > > >
> > > > do they have significance difference?
> > > > which one should i use,
> > > >
> > > > can someone please point me out to their API documentation.
> > > > i did searched for documentation, but couldn't find it.
> > > >
> > > >
> > > > Thanks,
> > > > -Yeshwanth
> > > > Can you Imagine what I would do if I could do all I can - Art of War
> > > >
> > >
> >
>

Re: Question about open table

[Ted Yu]

Can you show the body of getConnection() ?

getTable() itself is not expensive - assuming the same underlying
Connection.

Cheers

On Sat, Apr 9, 2016 at 7:18 AM, Yi Jiang  wrote:

> Hi, Guys
> I just have a question, I am trying to save the data into table in HBase
> I am using
>
> Table table = getConnection().getTable(TableName.valueOf(tableName));
> ...
> ...
> Table.close
>
> My question is that, is the "getTable" expensive?
> Shall I use that get table and close table in each saving?
> Or I just get table at the beginning, saving all data and then close the
> table after all saving.
> Because from the aspect of code design, I will save data into the
> different table on the flying. So, if it is expensive, I would like to
> rewrite the design.
> Thank you
> Jacky
>

Re: Column qualifier name in byte array in hbase

[Ted Yu]

I guess you have read:
https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-HiveMAPtoHBaseColumnFamily

where example for column family mapping is given.

If you need to map column qualifier, probably poll hive mailing list on
syntax.

FYI

On Sat, Apr 9, 2016 at 12:30 AM, Viswanathan J 
wrote:

> Hi,
>
> I'm trying to create external hive table to map hbase table and i'm facing
> issue in the column mapping because the column qualifier is in byte array
> as below, how to define/map this.
>
> eg.,
>
> column=cf:\x00\x00\x00\x00\x00\x00\x00\x00, timestamp=1455990558044
>
> --
> Regards,
> Viswa.J
>

Re: "Some problems after upgrade from 0.98.6 to 0.98.17"

[Ted Yu]

The 'No StoreFiles for' is logged at DEBUG level.

Was 9151f75eaa7d00a81e5001f4744b8b6a among the regions which didn't finish
split ?

Can you pastebin more of the master log during this period ?

Any other symptom that you observed ?

Cheers

On Thu, Apr 7, 2016 at 12:59 AM, Heng Chen  wrote:

> This is log about one region i grep from master log
>
> 2016-04-07 12:20:53,984 INFO  [AM.ZK.Worker-pool2-t145]
> master.RegionStates: Transition null to {9151f75eaa7d00a81e5001f4744b8b6a
> state=SPLITTING_NEW, ts=1460002853984,
> server=dx-ape-regionserver40-online,60020,1459998494013}
> 2016-04-07 12:20:55,326 INFO  [AM.ZK.Worker-pool2-t147]
> master.RegionStates: Transition {9151f75eaa7d00a81e5001f4744b8b6a
> state=SPLITTING_NEW, ts=1460002855326,
> server=dx-ape-regionserver40-online,60020,1459998494013} to
> {9151f75eaa7d00a81e5001f4744b8b6a state=OPEN, ts=1460002855326,
> server=dx-ape-regionserver40-online,60020,1459998494013}
> 2016-04-07 12:20:55,326 INFO  [AM.ZK.Worker-pool2-t147]
> master.RegionStates: Onlined 9151f75eaa7d00a81e5001f4744b8b6a on
> dx-ape-regionserver40-online,60020,1459998494013
> 2016-04-07 12:20:55,328 INFO  [AM.ZK.Worker-pool2-t147]
> master.AssignmentManager: Handled SPLIT event;
>
> parent=apolo_pdf,\x00\x00\x00\x00\x01\x94\xC0\xA8,1457693428562.3410ea47a97d0aefb12ec62e8e89b605.,
> daughter
>
> a=apolo_pdf,\x00\x00\x00\x00\x01\x94\xC0\xA8,1460002853961.9151f75eaa7d00a81e5001f4744b8b6a.,
> daughter
>
> b=apolo_pdf,\x00\x00\x00\x00\x01\xA2\x0D\x96,1460002853961.a7d6d735ccbf47e0a9d3016b8fef181a.,
> on dx-ape-regionserver40-online,60020,1459998494013
> 2016-04-07 12:21:44,083 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/16b7f857eb6741a5bcaaa5516034929f.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=16b7f857eb6741a5bcaaa5516034929f
> 2016-04-07 12:21:44,123 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/1a3f77a7588a4ad38d34ed97f6c095be.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=1a3f77a7588a4ad38d34ed97f6c095be
> 2016-04-07 12:21:44,132 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/544552889a4c4de99d52814b3c229c30.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=544552889a4c4de99d52814b3c229c30
> 2016-04-07 12:21:44,138 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/fcf658954ff84e998eb71ee6477c2ebe.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=fcf658954ff84e998eb71ee6477c2ebe
> 2016-04-07 12:21:44,138 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/16b7f857eb6741a5bcaaa5516034929f.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=16b7f857eb6741a5bcaaa5516034929f
> 2016-04-07 12:21:44,177 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/1a3f77a7588a4ad38d34ed97f6c095be.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=1a3f77a7588a4ad38d34ed97f6c095be
> 2016-04-07 12:21:44,179 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/544552889a4c4de99d52814b3c229c30.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=544552889a4c4de99d52814b3c229c30
> 2016-04-07 12:21:44,181 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.StoreFileInfo: reference
>
> 'hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/d/fcf658954ff84e998eb71ee6477c2ebe.3410ea47a97d0aefb12ec62e8e89b605'
> to region=3410ea47a97d0aefb12ec62e8e89b605
> hfile=fcf658954ff84e998eb71ee6477c2ebe
> 2016-04-07 12:21:44,184 DEBUG
> [dx-ape-hmaster1-online,6,1459998573658-ClusterStatusChore]
> regionserver.HRegionFileSystem: No StoreFiles for:
>
> hdfs://hdfs-master:8020/hbase/data/default/apolo_pdf/9151f75eaa7d00a81e5001f4744b8b6a/m
> 2016-04-07 12:23:54,259 DEBUG [region-location-4]
> 

Re: HBase region pre-split and store evenly

[Ted Yu]

Dropping user@hadoop.

Can you tell us more about your schema design ?

Was the behavior below happening even when you have pre-split your table ?

Cheers

On Thu, Apr 7, 2016 at 2:48 AM, Viswanathan J <jayamviswanat...@gmail.com>
wrote:

> Hi Ted,
>
> When we inserting data into hbase it's utilize only the single region
> server, not other region servers and we have 3 to 4 rs.
>
> On Thu, Apr 7, 2016 at 3:27 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>
>> Please take a look at:
>> http://hbase.apache.org/book.html#disable.splitting
>>
>> especially the section titled:
>> Determine the Optimal Number of Pre-Split Regions
>>
>> For writing data evenly across the cluster, can you tell us some more
>> about your use case(s) ?
>>
>> Thanks
>>
>> On Tue, Apr 5, 2016 at 11:48 PM, Viswanathan J <
>> jayamviswanat...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Please help on region pre-split and write data evenly in all the region
>>> servers.
>>>
>>> --
>>> Regards,
>>> Viswa.J
>>>
>>
>>
>
>
> --
> Regards,
> Viswa.J
>

Re: HBase region pre-split and store evenly

[Ted Yu]

Please take a look at:
http://hbase.apache.org/book.html#disable.splitting

especially the section titled:
Determine the Optimal Number of Pre-Split Regions

For writing data evenly across the cluster, can you tell us some more about
your use case(s) ?

Thanks

On Tue, Apr 5, 2016 at 11:48 PM, Viswanathan J 
wrote:

> Hi,
>
> Please help on region pre-split and write data evenly in all the region
> servers.
>
> --
> Regards,
> Viswa.J
>

Re: choosing between hbase-spark / spark-hbase

[Ted Yu]

There are some outstanding bug fixes, e.g. HBASE-15333, for hbase-spark
module.

FYI

On Tue, Apr 5, 2016 at 2:36 PM, Nkechi Achara 
wrote:

> So Hbase-spark is a continuation of the spark on hbase project, but within
> the Hbase project.
> They are not any significant differences apart from the fact that Spark on
> hbase is not updated.
> Dependent on the version you are using it would be more beneficial to use
> Hbase-Spark
>
> Kay
> On 5 Apr 2016 9:12 pm, "yeshwanth kumar"  wrote:
>
> > i have cloudera cluster,
> > i am exploring spark with HBase,
> >
> > after going through this blog
> >
> >
> >
> http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
> >
> > i found two options for using Spark with HBase,
> >
> > Cloudera's Spark on HBase or
> > Apache hbase-spark.
> >
> > do they have significance difference?
> > which one should i use,
> >
> > can someone please point me out to their API documentation.
> > i did searched for documentation, but couldn't find it.
> >
> >
> > Thanks,
> > -Yeshwanth
> > Can you Imagine what I would do if I could do all I can - Art of War
> >
>

Re: delete hbase data in the previous month

[Ted Yu]

Have you considered setting TTL ?

bq. HBase will not be easily removed

Can you clarify the above ?

Cheers

On Tue, Apr 5, 2016 at 12:03 AM, hsdcl...@163.com  wrote:

>
> If I want to delete hbase data in the previous month , how do? To avoid
> errors,
> when delete date ,  we required to  block   insertion operation. HBase can
> do it?
> See the document said,  the historical data , HBase will not be easily
> removed, then the thing that  I need  can achive?
>
>
> hsdcl...@163.com
>

Re: hbase custom scan

[Ted Yu]

How many regions does your table have ?

After sorting, is there a chance that the top N rows come from distinct
regions ?

On Mon, Apr 4, 2016 at 8:27 PM, Shushant Arora 
wrote:

> Hi
>
> I have a requirement to scan a hbase table based on insertion timestamp.
> I need to fetch the keys sorted by insertion timestamp not by key .
>
> I can't made timestamp as prefix of key to avoid hot spotting.
> Is there any efficient way possible for this requirement.
>
> Thanks!
>

Re: Compacting same table

[Ted Yu]

bq. I have been informed

Can you disclose the source of such information ?

For hbase.hstore.compaction.kv.max , hbase-default.xml has:

The maximum number of KeyValues to read and then write in a batch when
flushing or
  compacting. Set this lower if you have big KeyValues and problems
with Out Of Memory
  Exceptions Set this higher if you have wide, small rows.

Is the above description not clear ?

Thanks

On Sun, Apr 3, 2016 at 4:32 AM, Sumit Nigam 
wrote:

> Hi,
> I have been informed that compacting (manual) the same hbase table takes
> same amount of time even when done in quick succession. This seems
> counter-intuitive because an already compacted table should not take same
> amount of time.
> Also, what is the use of hbase.hstore.compaction.kv.max setting in
> compaction? I am unable to determine its implications on other compaction
> tuning  factors? The default of 10 seems too less.
> Thanks,Sumit

Re: build error

[Ted Yu]

In refguide, I don't see -Dsnappy mentioned.
I didn't find snappy in pom.xml either.

Have you tried building without this -D ?

On Fri, Apr 1, 2016 at 12:40 AM, Micha  wrote:

> Hi,
>
> this is my first maven build, thought this should just work :-)
>
> after calling:
>
> MAVEN_OPTS="-Xmx2g" mvn site install assembly:assembly -DskipTests
> -Dhadoop-two.version=2.7.2   -Dsnappy
>
>
> I get:
>
> Downloading:
>
> http://people.apache.org/~garyh/mvn/org/apache/hadoop/hadoop-snappy/0.0.1-SNAPSHOT/hadoop-snappy-0.0.1-SNAPSHOT.pom
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/hadoop/hadoop-snappy/0.0.1-SNAPSHOT/hadoop-snappy-0.0.1-SNAPSHOT.pom
> [WARNING] The POM for org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT
> is missing, no dependency information available
>
>
> this leads to:
>
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
> project hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal on
> project hbase-server: Could not resolve dependencies for project
> org.apache.hbase:hbase-server:jar:1.1.4: Could not find artifact
> org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT in apache release
> (https://repository.apache.org/content/repositories/releases/) -> [Help 1]
> [ERROR]
>
>
>
> how to fix this?
>
> thanks,
>  Michael
>
>

Re: difference between dus and df output

[Ted Yu]

Have you performed major compaction lately ?

Are there non-expired hbase snapshots ?

Cheers

On Thu, Mar 31, 2016 at 2:50 PM, Ted Tuttle <t...@mentacapital.com> wrote:

> This is very interesting, Ted. Thank you.
>
> We are only running HBase on hdfs.
>
> Does this mostly empty block appending behavior make sense for HBase-only
> usage?
>
> If this is, in fact, unused storage how do we get it back?
>
> Currently df shows 75% filled while du shows 25%.  The former is prompting
> us to consider more hardware.  If in fact we are 25% we don't need to.
>
> -Original Message-
> From: Ted Yu [mailto:yuzhih...@gmail.com]
> Sent: Thursday, March 31, 2016 1:13 PM
> To: user@hbase.apache.org
> Subject: difference between dus and df output
>
> Have you seen this thread ?
>
> http://search-hadoop.com/m/uOzYtatlmAcqgzM
>
> On Thu, Mar 31, 2016 at 11:58 AM, Ted Tuttle <t...@mentacapital.com> wrote:
>
> > Hello-
> >
> > We are running v0.94.9 cluster.
> >
> > I am seeing that 'fs -dus' reports 24TB used and 'fs -df' reports
> > 74.TB used.
> >
> > Does anyone know why these do not reconcile? Our replication factor is
> > 2 so that is not a likely explanation.
> >
> > Shown below are results from my cluster (doctored to TB for ease of
> > reading):
> >
> > bash-4.1$ hadoop fs -dus /hbase
> > hdfs://host/hbase  24.5TB
> >
> > bash-4.1$ hadoop fs -df /hbase
> > Filesystem  SizeUsedAvail   Use%
> > /hbase  103.8TB 74.2TB 24.3TB  71%
> >
>

difference between dus and df output

[Ted Yu]

Have you seen this thread ?

http://search-hadoop.com/m/uOzYtatlmAcqgzM

On Thu, Mar 31, 2016 at 11:58 AM, Ted Tuttle  wrote:

> Hello-
>
> We are running v0.94.9 cluster.
>
> I am seeing that 'fs -dus' reports 24TB used and 'fs -df' reports 74.TB
> used.
>
> Does anyone know why these do not reconcile? Our replication factor is 2
> so that is not a likely explanation.
>
> Shown below are results from my cluster (doctored to TB for ease of
> reading):
>
> bash-4.1$ hadoop fs -dus /hbase
> hdfs://host/hbase  24.5TB
>
> bash-4.1$ hadoop fs -df /hbase
> Filesystem  SizeUsedAvail   Use%
> /hbase  103.8TB 74.2TB 24.3TB  71%
>

Re: find size of each table in the cluster

[Ted Yu]

bq. COMPRESSION => 'LZ4',

The answer is given by above attribute :-)

On Thu, Mar 31, 2016 at 10:41 AM, marjana  wrote:

> Sure, here's describe of one table:
>
> Table RAWHITS_AURORA-COM is ENABLED
> RAWHITS_AURORA-COM
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'f1', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4',
> MIN_VERSIONS => '0', TTL => '216 SECONDS (2
> 5 DAYS)', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '262144', IN_MEMORY
> =>
> 'false', BLOCKCACHE => 'true'}
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/find-size-of-each-table-in-the-cluster-tp4078899p4078927.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Re: Re: Could not initialize all stores for the region

[Ted Yu]

Can you check server log on node 106 around 19:19:20 to see if there was
more clue ?

bq. being informed by the events which happens during their absent somehow?

Did you mean after nodeA came back online ?

Cheers

On Thu, Mar 31, 2016 at 9:57 AM, Zheng Shen <zhengshe...@outlook.com> wrote:

> Hi Ted,
>
> Thank you very much for your reply!
>
> We do have mutliple HMaster nodes, one of them is on the offline node
> (let's call it nodeA). Another is on node which is alwasy online (nodeB).
>
> I scanned the audit log, and found that during nodeA offline, the nodeB
> HDFS auditlog shows:
>
> hdfs-audit.log:2016-03-31 19:19:24,158 INFO FSNamesystem.audit:
> allowed=true ugi=hbase (auth:SIMPLE) ip=/192.168.1.106 cmd=delete
> src=/hbase/archive/data/default/vocabulary/2639c4d082646bb4a4fa2d8119f9aaef/cnt/2dc367d0e1c24a3b848c68d3b171b06d
> dst=null perm=null proto=rpc
>
> where (192.168.1.106) is the IP of nodeB.
>
> So it looks like nodeB deleted this file during nodeA's offline. However,
> should'nt services on nodeA (like HMaster and namenode) being informed by
> the events which happens during their absent somehow?
>
> Although we have only 5 nodes in this cluster, we do perform HA on every
> levels of HBase service stack. So yes, there are multiple instances of
> every services as long as it's possible or necessay (e.g. we have 3
> HMaster, 2 name node, 3 journal node)
>
> Thanks,
> Zheng
>
> 
> zhengshe...@outlook.com
>
> From: Ted Yu<mailto:yuzhih...@gmail.com>
> Date: 2016-04-01 00:00
> To: user@hbase.apache.org<mailto:user@hbase.apache.org>
> Subject: Re: Could not initialize all stores for the region
> bq. File does not exist: /hbase/data/default/vocabulary/
> 2639c4d082646bb4a4fa2d8119f9aaef/cnt/2dc367d0e1c24a3b848c68d3b171b06d
>
> Can you search in namenode audit log to see which node initiated the delete
> request of the above file ?
> Then you can search in that node's region server log to get more clue.
>
> bq. hosts the HDFS namenode and datanode, Cloudera Manager, as well as
> HBase master and region server
>
> Can you separate some daemons off this node (e.g. HBase master) ?
> I assume you have second HBase master running somewhere else. Otherwise
> this node becomes the weak point of the cluster.
>
> On Thu, Mar 31, 2016 at 7:58 AM, Zheng Shen <zhengshe...@outlook.com>
> wrote:
>
> > Hi,
> >
> > Our Hbase cannot performance any write operation while the read operation
> > are fine. I found the following error from regision server log
> >
> >
> > Could not initialize all stores for the
> >
> region=vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.
> >
> > Failed open of
> >
> region=vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.,
> > starting to roll back the global memstore size.
> > java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> > File does not exist:
> >
> /hbase/data/default/vocabulary/2639c4d082646bb4a4fa2d8119f9aaef/cnt/2dc367d0e1c24a3b848c68d3b171b06d
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:559)
> > at
> >
> >
> > Opening of region {ENCODED => 19faeb6e4da0b1873f68da271b0f5788, NAME =>
> >
> 'vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.',
> > STARTKEY => '576206_6513944', ENDKEY => '599122_6739914'} failed,
> > transitioning from OPENING to FAILED_OPEN in ZK, expecting version 22
> >
> >
> > We are using Cloudera CDH 5.4.7, the HBase version is 1.0.0-cdh_5.4.7,
> > with HDFS HA enabled (one of the namenode is running on the server being
> > shutdown). Our HBase cluster expereienced an expected node shutdown today
> > for about 4 hours. The node which is shutdown hosts the HDFS namenode and
> > datanode, Cloudera Manager

Re: Could not initialize all stores for the region

[Ted Yu]

bq. File does not exist: /hbase/data/default/vocabulary/
2639c4d082646bb4a4fa2d8119f9aaef/cnt/2dc367d0e1c24a3b848c68d3b171b06d

Can you search in namenode audit log to see which node initiated the delete
request of the above file ?
Then you can search in that node's region server log to get more clue.

bq. hosts the HDFS namenode and datanode, Cloudera Manager, as well as
HBase master and region server

Can you separate some daemons off this node (e.g. HBase master) ?
I assume you have second HBase master running somewhere else. Otherwise
this node becomes the weak point of the cluster.

On Thu, Mar 31, 2016 at 7:58 AM, Zheng Shen  wrote:

> Hi,
>
> Our Hbase cannot performance any write operation while the read operation
> are fine. I found the following error from regision server log
>
>
> Could not initialize all stores for the
> region=vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.
>
> Failed open of
> region=vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.,
> starting to roll back the global memstore size.
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> File does not exist:
> /hbase/data/default/vocabulary/2639c4d082646bb4a4fa2d8119f9aaef/cnt/2dc367d0e1c24a3b848c68d3b171b06d
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:559)
> at
>
>
> Opening of region {ENCODED => 19faeb6e4da0b1873f68da271b0f5788, NAME =>
> 'vocabulary,576206_6513944,1459420417369.19faeb6e4da0b1873f68da271b0f5788.',
> STARTKEY => '576206_6513944', ENDKEY => '599122_6739914'} failed,
> transitioning from OPENING to FAILED_OPEN in ZK, expecting version 22
>
>
> We are using Cloudera CDH 5.4.7, the HBase version is 1.0.0-cdh_5.4.7,
> with HDFS HA enabled (one of the namenode is running on the server being
> shutdown). Our HBase cluster expereienced an expected node shutdown today
> for about 4 hours. The node which is shutdown hosts the HDFS namenode and
> datanode, Cloudera Manager, as well as HBase master and region server (5
> nodes in totally in our small clusder).  During the node shuting down,
> beside the services running that that node, the other HDFS namenode,
> failover server, and 2 of 3 journal node are also down. After the node is
> recovered, we restarted the whole CDH cluster, and then it ends like this
> one...
>
> The HDFS checking "hdfs fsck" does not report any corrupted blocks.
>
> Any suggesion about where we should look into for this problem?
>
> Thanks!
> Zheng
>
> 
> zhengshe...@outlook.com
>

Re: find size of each table in the cluster

[Ted Yu]

bq. data is distributed on node servers,

Data is on hdfs, i.e. the Data Nodes.

bq. it gets propagated to all data nodes,

If I understand correctly, the -du command queries namenode.

bq. Is this size compressed or uncompressed?

Can you show us the table description (output of describe command in hbase
shell) ?

On Thu, Mar 31, 2016 at 8:38 AM, marjana  wrote:

> Thanks all on your replies.
> This is clustered env, with 2 master nodes and 4 data nodes. Master nodes
> have these components installed (as shown in Ambari UI):
> active hbase master
> history server
> name node
> resource manager
> zookeeper server
> metrics monitor
>
> Node server has these components:
> Data Node
> region server
> metrics monitor
> node manager
>
> So I looked on my node server for the hbase.rootdir, and it points to my
> hdfs://hbasmaserserver:8020//apps/hbase/data.
> Now this is confusing to me as I thought data is distributed on node
> servers, where region servers are.
> I sshed to my masterserver and looked under this dir and did see all my
> tables in my default namespace. Example:
> $ hadoop fs -du -s -h /apps/hbase/data/data/default/RAWHITS_AURORA-COM
> 2.0 G  /apps/hbase/data/data/default/RAWHITS_AURORA-COM
>
> So when I run this command on hbmaster, it gets propagated to all data
> nodes, correct? Is this size compressed or uncompressed?
>
> Many thanks!
> Marjana
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/find-size-of-each-table-in-the-cluster-tp4078899p4078919.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Re: Region Server Crash On Upsert Query Execution

[Ted Yu]

The attachments you mentioned did go through.

For #2, please adjust:

hbase.client.scanner.timeout.period
hbase.rpc.timeout

Both have default value of 60 seconds.

If possible, please pastebin server log snippet before it crashed.

Thanks

On Thu, Mar 31, 2016 at 3:04 AM, Amit Shah  wrote:

> A couple of observations
>
> 1. I could see GC pauses in the logs but I do not think that could be
> causing the jvm exists. I have configured the region server heap to be 2
> GB. The jconsole indicates that it hardly reaches 1.5 GB. Kindly find some
> graphs attached.
>
> 2. On another run the phoenix client failed with a socket timeout
> operation. The error is pasted here - http://pastebin.com/mAWwiE2J
> Are there client timeouts for each region server? It looks like the
> default is 1 min. How can I increase them?
>
> Thanks,
> Amit.
>
> On Thu, Mar 31, 2016 at 2:39 PM, Samir Ahmic 
> wrote:
>
>> Hi Amit,
>>
>> Check regionserver logs, usual suspects, are log running GC and HDFS
>> client
>> related issues. Check for compaction queue.
>>
>> Regrads
>> Samir
>>
>> On Thu, Mar 31, 2016 at 10:48 AM, Amit Shah  wrote:
>>
>> > Hi,
>> >
>> > We have been experimenting hbase (version 1.0) and phoenix (version 4.6)
>> > for our OLAP workload. In order to precalculate aggregates we have been
>> > executing an upsert phoenix query that aggregates raw data (over 10 mil
>> > records) to generate an OLAP cube.
>> >
>> > While executing the query, one of the region servers in a cluster of 3
>> RS
>> > crashes. I am trying to figure out what could be causing the region
>> server
>> > to crash.
>> > The server shows high disk operations before the jvm crashed. Kindly
>> find
>> > the disk and other stats attached.
>> >
>> > Any suggestions on where could I look into would be helpful.
>> >
>> > The upsert query that was executed is
>> >
>> > upsert into AGENT_TER_PRO
>> >
>> (AGENT_ID,TERRITORY_ID,PRODUCT_ID,SUM_TOTAL_SALES,SUM_TOTAL_EXPENSES,SUM_UNIT_CNT_SOLD,AVG_PRICE_PER_UNIT)
>> > select /*+ INDEX(TRANSACTIONS  AG_TER_PRO2) */
>> >  AGENT_ID,TERRITORY_ID,PRODUCT_ID, sum(TOTAL_SALES)
>> > SUM_TOTAL_SALES,sum(TOTAL_EXPENSES)
>> SUM_TOTAL_EXPENSES,sum(UNIT_CNT_SOLD)
>> > SUM_UNIT_CNT_SOLD,AVG(PRICE_PER_UNIT)  AVG_PRICE_PER_UNIT  from
>> > TRANSACTIONS   group by AGENT_ID,TERRITORY_ID,PRODUCT_ID;
>> >
>> > Thanks,
>> > Amit.
>> >
>> >
>>
>
>

Re: Leveraging As Much Memory As Possible

[Ted Yu]

For #2, I did a quick search in Phoenix code base for bucket cache - I
didn't find match.

Maybe ask on Phoenix mailing list.

On Wed, Mar 30, 2016 at 11:07 PM, Amit Shah <amits...@gmail.com> wrote:

> Thanks Ted and Anoop for the replies. A couple of follow up questions
>
> 1. I could see the bucket cache and on heap memory stats reflected on hbase
> region server UI <http://i.imgur.com/4uwaWuC.png>, what are ways to
> monitor
> and see if its getting utilized?
> 2. We are using phoenix as a sql layer above hbase. Are there any phoenix
> configurations that I could benefit in leveraging memory?
>
> Thanks,
> Amit.
>
> On Thu, Mar 31, 2016 at 10:44 AM, Anoop John <anoop.hb...@gmail.com>
> wrote:
>
> > Ya having HBase side cache will be a better choice rather than HDFS
> > cache IMO.   Yes u r correct...  You might not want to give a very
> > large size for the heap. You can make use of the off heap BucketCache.
> >
> > -Anoop-
> >
> > On Thu, Mar 31, 2016 at 4:35 AM, Ted Yu <yuzhih...@gmail.com> wrote:
> > > For #1, please see the top two blogs @ https://blogs.apache.org/hbase/
> > >
> > > FYI
> > >
> > > On Wed, Mar 30, 2016 at 7:59 AM, Amit Shah <amits...@gmail.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> I am trying to configure my hbase (version 1.0) phoenix (version -
> 4.6)
> > >> cluster to utilize as much memory as possible on the server hardware.
> We
> > >> have an OLAP workload that allows users to perform interactive
> analysis
> > >> over huge sets of data. While reading about hbase configuration I came
> > >> across two configs
> > >>
> > >> 1. Hbase bucket cache
> > >> <
> > >>
> >
> http://blog.asquareb.com/blog/2014/11/24/how-to-leverage-large-physical-memory-to-improve-hbase-read-performance
> > >> >
> > >> (off heap) which looks like a good option to bypass garbage
> collection.
> > >> 2. Hadoop pinned hdfs blocks
> > >> <
> > http://blog.cloudera.com/blog/2014/08/new-in-cdh-5-1-hdfs-read-caching/>
> > >> (max locked memory) concept that loads the hdfs blocks in memory, but
> > given
> > >> that hbase is configured with short circuit reads I assume this config
> > may
> > >> not be of much help. Instead it would be right to increase hbase
> region
> > >> server heap memory. Is my understanding right?
> > >>
> > >> We use HBase with Phoenix.
> > >> Kindly let me know your thoughts or suggestions on any more options
> > that I
> > >> should explore
> > >>
> > >> Thanks,
> > >> Amit.
> > >>
> >
>

Re: find size of each table in the cluster

[Ted Yu]

bq. hbase version is 1.1.1.2.3

I don't think there was ever such a release - there should be only 3 dots.

bq. /hbase is the default storage location for tables in hdfs

the root dir is given by hbase.rootdir config parameter.

Here is sample listing:

http://pastebin.com/ekF4tsYn

Under data, you would see:

drwxr-xr-x   - hbase hdfs  0 2016-03-22 20:26
/apps/hbase/data/data/default
drwxr-xr-x   - hbase hdfs  0 2016-03-14 19:13
/apps/hbase/data/data/hbase

hbase is system namespace.

Under default (or your own namespace), you would get table dir. Here is a
sample:

drwxr-xr-x   - hbase hdfs  0 2016-03-22 20:26
/apps/hbase/data/data/default/elog_pn_split

On Wed, Mar 30, 2016 at 7:26 PM, Stephen Durfey  wrote:

> I believe the easiest way would be to run 'hadoop dfs -du -h /hbase'. I
> believe /hbase is the default storage location for tables in hdfs. The size
> will be either compressed or uncompressed, depending upon if compression is
> enabled.
>
>
>
>
>
>
> On Wed, Mar 30, 2016 at 6:32 PM -0700, "marjana" 
> wrote:
>
>
>
>
>
>
>
>
>
>
> Hello,
> I am new to hBase, so sorry if I am talking nonsense.
>
> I am trying to figure out a way how to find the total size of each table in
> my hBase.
> I have looked into hbase shell commands. There's "status 'detailed'", that
> shows storefileSizeMB. If I were to add all of these grouped by tablename,
> would that be the correct way to show MB used per table?
> Is there any other (easier/cleaner) way?
> hbase version is 1.1.1.2.3, HDFS: 2.7.1
> Thanks
> Marjana
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/find-size-of-each-table-in-the-cluster-tp4078899.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>
>
>
>
>

Re: Leveraging As Much Memory As Possible

[Ted Yu]

For #1, please see the top two blogs @ https://blogs.apache.org/hbase/

FYI

On Wed, Mar 30, 2016 at 7:59 AM, Amit Shah  wrote:

> Hi,
>
> I am trying to configure my hbase (version 1.0) phoenix (version - 4.6)
> cluster to utilize as much memory as possible on the server hardware. We
> have an OLAP workload that allows users to perform interactive analysis
> over huge sets of data. While reading about hbase configuration I came
> across two configs
>
> 1. Hbase bucket cache
> <
> http://blog.asquareb.com/blog/2014/11/24/how-to-leverage-large-physical-memory-to-improve-hbase-read-performance
> >
> (off heap) which looks like a good option to bypass garbage collection.
> 2. Hadoop pinned hdfs blocks
> 
> (max locked memory) concept that loads the hdfs blocks in memory, but given
> that hbase is configured with short circuit reads I assume this config may
> not be of much help. Instead it would be right to increase hbase region
> server heap memory. Is my understanding right?
>
> We use HBase with Phoenix.
> Kindly let me know your thoughts or suggestions on any more options that I
> should explore
>
> Thanks,
> Amit.
>

Re: hbase version and hadoop version

[Ted Yu]

Please refer to http://hbase.apache.org/book.html#maven.release (especially
4. Build the binary tarball.)

Pass the following on command line:

-Dhadoop-two.version=2.7.2

FYI

On Wed, Mar 30, 2016 at 7:41 AM, Micha  wrote:

> Hi,
>
>
> hbase ships with hadoop jars in the libs directory which are older
> (2.5.2) than the actual hadoop version (2.7.2).
> So after upgrading hadoop to 2.7.2, should the jars in the libs
> directory of hbase (1.1.2) be replaced with the ones which come with
> hadoop?
>
>
> Thanks,
>  Michael
>

Re: Back up HBase tables before hadoop upgrade

[Ted Yu]

You can also snapshot each of the 647 tables.
In case something goes unexpected, you can restore any of them.

FYI

On Wed, Mar 30, 2016 at 6:46 AM, Chathuri Wimalasena 
wrote:

> Hi All,
>
> We have production system using hadoop 2.5.1 and HBase 0.94.23. We have
> nearly around 200 TB of data in HDFS and we are planning to upgrade to
> newer hadoop version 2.7.2. In HBase we have roughly 647 tables. Before the
> upgrade, we want to back up HBase tables in case of data loss or corruption
> while upgrade. We are thinking of using export and import functionality to
> export each table. Is there any other recommended way to back up hbase
> tables ?
>
> Thanks,
> Chathuri
>

Re: processing in coprocessor and region splitting

[Ted Yu]

bq. calculating another new attributes of a trade

Can you put the new attributes in separate columns ?

Cheers

On Fri, Mar 25, 2016 at 12:38 PM, Daniel Połaczański <dpolaczan...@gmail.com
> wrote:

> The data is set of trades and the processing is some kind of enrichment
> (calculating another new attributes of a trade). All attributes are needed
> (the original and new)
>
> 2016-03-25 18:41 GMT+01:00 Ted Yu <yuzhih...@gmail.com>:
>
> > bq. During the processing the size of the data is doubled.
> >
> > This explains the frequent split :-)
> >
> > Is the original data needed after post-processing (maybe for auditing) ?
> >
> > Cheers
> >
> > On Fri, Mar 25, 2016 at 10:32 AM, Daniel Połaczański <
> > dpolaczan...@gmail.com
> > > wrote:
> >
> > > I am testing different solutions (POC).
> > > The region size currenlty is 32MB (I know it should be >= 1GB, but we
> are
> > > testing different solutions with smaller amount of the data ). So
> > > increasing region size is not a solution. Our problems can happen even
> > when
> > > a region will be 1 GB. We want to proces the data with coprocessor and
> > > hadoop map reduce. I can not have one big Region because I want
> sensible
> > > degree of paralerism (with Map Reduce and coprocessors).
> > >
> > > Increasing region size + pre-splitting  is not an option as well
> because
> > I
> > > know nothing about keys(random long).
> > >
> > > During the processing the size of the data is doubled.
> > >
> > > And yes, coprocessor rewrites a lot of the data written into the table.
> > The
> > > whole record is serialized to avro and stored in one column (storing
> > single
> > > attribute in single column we will try in the next POC)
> > >
> > > it is not a typical big data project where we can allow former analysis
> > of
> > > the data:)
> > >
> > > 2016-03-25 17:38 GMT+01:00 Ted Yu <yuzhih...@gmail.com>:
> > >
> > > > What's the current region size you use ?
> > > >
> > > > bq. During the processing size of the data gets increased
> > > >
> > > > Can you give us some quantitative measure as to how much increase you
> > > > observed (w.r.t. region size) ?
> > > >
> > > > bq. I was looking for some "global lock" in source code
> > > >
> > > > Probably not a good idea using global lock.
> > > >
> > > > I am curious, looks like your coprocesser may rewrite a lot of data
> > > written
> > > > into the table.
> > > > Can client side accommodate such logic so that the rewrite is
> reduced ?
> > > >
> > > > Thanks
> > > >
> > > > On Fri, Mar 25, 2016 at 8:55 AM, Daniel Połaczański <
> > > > dpolaczan...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > > I have some processing in my coprocesserService which modifies the
> > > > existing
> > > > > data in place. It iterates over every row, modifies and puts it
> back
> > to
> > > > > region. The table can be modified by only one client.
> > > > >
> > > > > During the processing size of the data gets increased -> region's
> > size
> > > > get
> > > > > increased -> region's split happens. It makes that the processing
> is
> > > > > stopped by exception NotServingRegionException (because region is
> > > closed
> > > > > and splited to two new regions so it is closed and doesn't exist
> > > > anymore).
> > > > >
> > > > > Is there any clean way to block Region's splitting?
> > > > >
> > > > > I was looking for some "global lock" in source code but I haven't
> > found
> > > > > anything helpfull.
> > > > > Another idea is to create custom RegionSplitPolicy and explicilty
> set
> > > > some
> > > > > Flag which will return false in shouldSplit(), but I'm not sure yet
> > if
> > > it
> > > > > is safe.
> > > > > Could you advise?
> > > > > Regards
> > > > >
> > > >
> > >
> >
>

Re: processing in coprocessor and region splitting

[Ted Yu]

bq. During the processing the size of the data is doubled.

This explains the frequent split :-)

Is the original data needed after post-processing (maybe for auditing) ?

Cheers

On Fri, Mar 25, 2016 at 10:32 AM, Daniel Połaczański <dpolaczan...@gmail.com
> wrote:

> I am testing different solutions (POC).
> The region size currenlty is 32MB (I know it should be >= 1GB, but we are
> testing different solutions with smaller amount of the data ). So
> increasing region size is not a solution. Our problems can happen even when
> a region will be 1 GB. We want to proces the data with coprocessor and
> hadoop map reduce. I can not have one big Region because I want sensible
> degree of paralerism (with Map Reduce and coprocessors).
>
> Increasing region size + pre-splitting  is not an option as well because I
> know nothing about keys(random long).
>
> During the processing the size of the data is doubled.
>
> And yes, coprocessor rewrites a lot of the data written into the table. The
> whole record is serialized to avro and stored in one column (storing single
> attribute in single column we will try in the next POC)
>
> it is not a typical big data project where we can allow former analysis of
> the data:)
>
> 2016-03-25 17:38 GMT+01:00 Ted Yu <yuzhih...@gmail.com>:
>
> > What's the current region size you use ?
> >
> > bq. During the processing size of the data gets increased
> >
> > Can you give us some quantitative measure as to how much increase you
> > observed (w.r.t. region size) ?
> >
> > bq. I was looking for some "global lock" in source code
> >
> > Probably not a good idea using global lock.
> >
> > I am curious, looks like your coprocesser may rewrite a lot of data
> written
> > into the table.
> > Can client side accommodate such logic so that the rewrite is reduced ?
> >
> > Thanks
> >
> > On Fri, Mar 25, 2016 at 8:55 AM, Daniel Połaczański <
> > dpolaczan...@gmail.com>
> > wrote:
> >
> > > Hi,
> > > I have some processing in my coprocesserService which modifies the
> > existing
> > > data in place. It iterates over every row, modifies and puts it back to
> > > region. The table can be modified by only one client.
> > >
> > > During the processing size of the data gets increased -> region's size
> > get
> > > increased -> region's split happens. It makes that the processing is
> > > stopped by exception NotServingRegionException (because region is
> closed
> > > and splited to two new regions so it is closed and doesn't exist
> > anymore).
> > >
> > > Is there any clean way to block Region's splitting?
> > >
> > > I was looking for some "global lock" in source code but I haven't found
> > > anything helpfull.
> > > Another idea is to create custom RegionSplitPolicy and explicilty set
> > some
> > > Flag which will return false in shouldSplit(), but I'm not sure yet if
> it
> > > is safe.
> > > Could you advise?
> > > Regards
> > >
> >
>

Re: processing in coprocessor and region splitting

[Ted Yu]

What's the current region size you use ?

bq. During the processing size of the data gets increased

Can you give us some quantitative measure as to how much increase you
observed (w.r.t. region size) ?

bq. I was looking for some "global lock" in source code

Probably not a good idea using global lock.

I am curious, looks like your coprocesser may rewrite a lot of data written
into the table.
Can client side accommodate such logic so that the rewrite is reduced ?

Thanks

On Fri, Mar 25, 2016 at 8:55 AM, Daniel Połaczański 
wrote:

> Hi,
> I have some processing in my coprocesserService which modifies the existing
> data in place. It iterates over every row, modifies and puts it back to
> region. The table can be modified by only one client.
>
> During the processing size of the data gets increased -> region's size get
> increased -> region's split happens. It makes that the processing is
> stopped by exception NotServingRegionException (because region is closed
> and splited to two new regions so it is closed and doesn't exist anymore).
>
> Is there any clean way to block Region's splitting?
>
> I was looking for some "global lock" in source code but I haven't found
> anything helpfull.
> Another idea is to create custom RegionSplitPolicy and explicilty set some
> Flag which will return false in shouldSplit(), but I'm not sure yet if it
> is safe.
> Could you advise?
> Regards
>

Re: Inconsistent scan performance

[Ted Yu]

Crossing region boundaries which happen to be on different servers may be.

On Thu, Mar 24, 2016 at 5:49 PM, James Johansville <
james.johansvi...@gmail.com> wrote:

> In theory they should be aligned with *regionserver* boundaries. Would
> crossing multiple regions on the same regionserver result in the big
> performance difference being seen here?
>
> I am using Hortonworks HBase 1.1.2
>
> On Thu, Mar 24, 2016 at 5:32 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > I assume the partitions' boundaries don't align with region boundaries,
> > right ?
> >
> > Meaning some partitions would cross region boundaries.
> >
> > Which hbase release do you use ?
> >
> > Thanks
> >
> > On Thu, Mar 24, 2016 at 4:45 PM, James Johansville <
> > james.johansvi...@gmail.com> wrote:
> >
> > > Hello all,
> > >
> > > So, I wrote a Java application for HBase that does a partitioned
> > full-table
> > > scan according to a set number of partitions. For example, if there are
> > 20
> > > partitions specified, then 20 separate full scans are launched that
> cover
> > > an equal slice of the row identifier range.
> > >
> > > The rows are uniformly distributed throughout the RegionServers. I
> > > confirmed this through the hbase shell. I have only one column family,
> > and
> > > each row has the same number of column qualifiers.
> > >
> > > My problem is that the individual scan performance is wildly
> inconsistent
> > > even though they fetch approximately a similar number of rows. This
> > > inconsistency appears to be random with respect to hosts or
> regionservers
> > > or partitions or CPU cores. I am the only user of the fleet and not
> > running
> > > any other concurrent HBase operation.
> > >
> > > I started measuring from the beginning of the scan and stopped
> measuring
> > > after the scan was completed. I am not doing any logic with the
> results,
> > > just scanning them.
> > >
> > > For ~230K rows fetched per scan, I am getting anywhere from 4 seconds
> to
> > > 100+ seconds. This seems a little too bouncy for me. Does anyone have
> any
> > > insight? By comparison, a similar utility I wrote to upsert to
> > > regionservers was very consistent in ops/sec and I had no issues with
> it.
> > >
> > > Using 13 partitions on a machine that has 32 CPU cores and 16 GB heap,
> I
> > > see anywhere between 3K ops/sec to 82K ops/sec. Here's an example of
> log
> > > output I saved that used 130 partitions.
> > >
> > > total # partitions:130; partition id:47; rows:232730 elapsed_sec:6.401
> > > ops/sec:36358.38150289017
> > > total # partitions:130; partition id:100; rows:206890 elapsed_sec:6.636
> > > ops/sec:31176.91380349608
> > > total # partitions:130; partition id:63; rows:233437 elapsed_sec:7.586
> > > ops/sec:30772.08014764039
> > > total # partitions:130; partition id:9; rows:232585 elapsed_sec:32.985
> > > ops/sec:7051.235410034865
> > > total # partitions:130; partition id:19; rows:234192 elapsed_sec:38.733
> > > ops/sec:6046.3170939508955
> > > total # partitions:130; partition id:1; rows:232860 elapsed_sec:48.479
> > > ops/sec:4803.316900101075
> > > total # partitions:130; partition id:125; rows:205334
> elapsed_sec:41.911
> > > ops/sec:4899.286583474505
> > > total # partitions:130; partition id:123; rows:206622
> elapsed_sec:42.281
> > > ops/sec:4886.875901705258
> > > total # partitions:130; partition id:54; rows:232811 elapsed_sec:49.083
> > > ops/sec:4743.210480206996
> > >
> > > I use setCacheBlocks(false), setCaching(5000).  Does anyone have any
> > > insight into how I can make the read performance more consistent?
> > >
> > > Thanks!
> > >
> >
>

Re: Inconsistent scan performance

[Ted Yu]

I assume the partitions' boundaries don't align with region boundaries,
right ?

Meaning some partitions would cross region boundaries.

Which hbase release do you use ?

Thanks

On Thu, Mar 24, 2016 at 4:45 PM, James Johansville <
james.johansvi...@gmail.com> wrote:

> Hello all,
>
> So, I wrote a Java application for HBase that does a partitioned full-table
> scan according to a set number of partitions. For example, if there are 20
> partitions specified, then 20 separate full scans are launched that cover
> an equal slice of the row identifier range.
>
> The rows are uniformly distributed throughout the RegionServers. I
> confirmed this through the hbase shell. I have only one column family, and
> each row has the same number of column qualifiers.
>
> My problem is that the individual scan performance is wildly inconsistent
> even though they fetch approximately a similar number of rows. This
> inconsistency appears to be random with respect to hosts or regionservers
> or partitions or CPU cores. I am the only user of the fleet and not running
> any other concurrent HBase operation.
>
> I started measuring from the beginning of the scan and stopped measuring
> after the scan was completed. I am not doing any logic with the results,
> just scanning them.
>
> For ~230K rows fetched per scan, I am getting anywhere from 4 seconds to
> 100+ seconds. This seems a little too bouncy for me. Does anyone have any
> insight? By comparison, a similar utility I wrote to upsert to
> regionservers was very consistent in ops/sec and I had no issues with it.
>
> Using 13 partitions on a machine that has 32 CPU cores and 16 GB heap, I
> see anywhere between 3K ops/sec to 82K ops/sec. Here's an example of log
> output I saved that used 130 partitions.
>
> total # partitions:130; partition id:47; rows:232730 elapsed_sec:6.401
> ops/sec:36358.38150289017
> total # partitions:130; partition id:100; rows:206890 elapsed_sec:6.636
> ops/sec:31176.91380349608
> total # partitions:130; partition id:63; rows:233437 elapsed_sec:7.586
> ops/sec:30772.08014764039
> total # partitions:130; partition id:9; rows:232585 elapsed_sec:32.985
> ops/sec:7051.235410034865
> total # partitions:130; partition id:19; rows:234192 elapsed_sec:38.733
> ops/sec:6046.3170939508955
> total # partitions:130; partition id:1; rows:232860 elapsed_sec:48.479
> ops/sec:4803.316900101075
> total # partitions:130; partition id:125; rows:205334 elapsed_sec:41.911
> ops/sec:4899.286583474505
> total # partitions:130; partition id:123; rows:206622 elapsed_sec:42.281
> ops/sec:4886.875901705258
> total # partitions:130; partition id:54; rows:232811 elapsed_sec:49.083
> ops/sec:4743.210480206996
>
> I use setCacheBlocks(false), setCaching(5000).  Does anyone have any
> insight into how I can make the read performance more consistent?
>
> Thanks!
>

Re: Unexpected region splits

[Ted Yu]

Actually there may be a simpler solution:

http://pastebin.com/3KJ7Vxnc

We can check the ratio between online regions and total number of regions
in IncreasingToUpperBoundRegionSplitPolicy#shouldSplit().

Only when the ratio gets over certain threshold, should splitting start.

FYI

On Thu, Mar 24, 2016 at 12:39 PM, Ted Yu <yuzhih...@gmail.com> wrote:

> Currently IncreasingToUpperBoundRegionSplitPolicy doesn't detect when the
> master initialization finishes.
>
> There is also some missing piece where region server notifies the
> completion of cluster initialization (by looking at RegionServerObserver).
>
> Cheers
>
> On Thu, Mar 24, 2016 at 3:50 AM, Bram Desoete <bra...@ngdata.com> wrote:
>
>>
>>
>>
>> Pedro Gandola <pedro.gandola@...> writes:
>>
>> >
>> > Hi Ted,
>> >
>> > Thanks,
>> > I think I got the problem, I'm using
>> *IncreasingToUpperBoundRegionSplitPolicy
>> > (default)* instead *ConstantSizeRegionSplitPolicy* which in my use case
>> is
>> > what I want.
>> >
>> > Cheers
>> > Pedro
>> >
>> > On Mon, Feb 15, 2016 at 5:22 PM, Ted Yu <yuzhihong@...> wrote:
>> >
>> > > Can you pastebin region server log snippet around the time when the
>> split
>> > > happened ?
>> > >
>> > > Was the split on data table or index table ?
>> > >
>> > > Thanks
>> > >
>> > > > On Feb 15, 2016, at 10:22 AM, Pedro Gandola <pedro.gandola@...>
>> > > wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > I have a cluster using *HBase 1.1.2* where I have a table and a
>> local
>> > > index
>> > > > (using *Apache Phoenix 4.6*) in total both tables have *300 regions*
>> > > > (aprox: *18 regions per server*), my*
>> hbase.hregion.max.filesize=30GB
>> > > *and
>> > > > my region sizes are now *~4.5GB compressed (~7GB uncompressed)*.
>> However
>> > > > each time I restart a RS sometimes a region gets split. This is
>> > > unexpected
>> > > > because my key space is uniform (using MD5) and if the problem was
>> my
>> > > > *region.size
>> > > >> * *hbase.hregion.max.filesize *I would expect to have all the
>> regions or
>> > > > almost all splitting but this only happens when I restart a RS and
>> it
>> > > > happens only for 1 or 2 regions.
>> > > >
>> > > > What are the different scenarios where a region can split?
>> > > >
>> > > > What are the right steps to restart a region server in order to
>> avoid
>> > > these
>> > > > unexpected splits?
>> > > >
>> > > > Thank you,
>> > > > Cheers
>> > > > Pedro
>> > >
>> >
>>
>>
>>
>> Thanks Pedro for giving your solution.
>>
>> i see the same issue during Hbase restarts. unexpected region splits.
>> i believe it is because the *IncreasingToUpperBoundRegionSplitPolicy* is
>> basing
>>  his calculation on the amount of ONLINE regions.
>> but while the RS is starting only a couple of regions are online YET.
>> so the policy things it would be no problem to add another region
>> since 'there are only a few'.
>> (while there are actually already are 330 for that RS for that phoenix
>> table...
>> yes i know i need to merge regions.
>> but this problem got out of hand unnoticed for some time now here)
>>
>> could HBase block split region decision until it is fully up and running?
>>
>> Hbase 1.0.0 logs. (check mainly the last line)
>>
>> Mar 24, 11:06:41.494 AM INFO
>> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher
>> Flushed, sequenceid=69436099, memsize=303.3 K, hasBloomFilter=true, into
>> tmp
>> file
>>
>> hdfs://ns/hbase/data/default/CUSTOMER/60af2857a7980ce4f1ac602dd83e05a6/.tmp/
>> 0fd4988f24f24d5d9887c542182efccc
>> Mar 24, 11:06:41.529 AM INFO
>> org.apache.hadoop.hbase.regionserver.HStore
>> Added hdfs://-ns/hbase/data/default/CUSTOMER/
>> ff4ecd56e6b06f228404f05f171f8282/0/1d05cf9cac4c46008e47e3578e7a18d6,
>> entries=235, sequenceid=22828972, filesize=5.5 K
>> Mar 24, 11:06:41.561 AM INFO
>> org.apache.hadoop.hbase.regionserver.HStore
>> Completed compaction of 3 (all) file(s) in s of CUSTOMER,\x0A0+\xF6\
>> xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.
>> into 730489295b8c42afaec4a3b8bc38c915(size=1.4 M),
>> total size for store is 1.4 M. This selection was in queue for
>> 0sec, and took 0sec to execute.
>> Mar 24, 11:06:41.561 AM INFO
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread
>> Completed compaction: Request = regionName=CUSTOMER,
>> \x0A0+\xF6\xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.,
>> storeName=s, fileCount=3, fileSize=1.7 M, priority=7,
>> time=1456532583179472;
>> duration=0sec
>> Mar 24, 11:06:41.562 AM DEBUG
>>
>> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy
>> ShouldSplit because IB size=3269370636, sizeToCheck=2147483648,
>> regionsWithCommonTable=2
>>
>> i will also revert back to the ConstantSizeRegionSplitPolicy
>>
>> Regards,
>>
>>
>>
>>
>

Re: Unexpected region splits

[Ted Yu]

Currently IncreasingToUpperBoundRegionSplitPolicy doesn't detect when the
master initialization finishes.

There is also some missing piece where region server notifies the
completion of cluster initialization (by looking at RegionServerObserver).

Cheers

On Thu, Mar 24, 2016 at 3:50 AM, Bram Desoete <bra...@ngdata.com> wrote:

>
>
>
> Pedro Gandola <pedro.gandola@...> writes:
>
> >
> > Hi Ted,
> >
> > Thanks,
> > I think I got the problem, I'm using
> *IncreasingToUpperBoundRegionSplitPolicy
> > (default)* instead *ConstantSizeRegionSplitPolicy* which in my use case
> is
> > what I want.
> >
> > Cheers
> > Pedro
> >
> > On Mon, Feb 15, 2016 at 5:22 PM, Ted Yu <yuzhihong@...> wrote:
> >
> > > Can you pastebin region server log snippet around the time when the
> split
> > > happened ?
> > >
> > > Was the split on data table or index table ?
> > >
> > > Thanks
> > >
> > > > On Feb 15, 2016, at 10:22 AM, Pedro Gandola <pedro.gandola@...>
> > > wrote:
> > > >
> > > > Hi,
> > > >
> > > > I have a cluster using *HBase 1.1.2* where I have a table and a local
> > > index
> > > > (using *Apache Phoenix 4.6*) in total both tables have *300 regions*
> > > > (aprox: *18 regions per server*), my* hbase.hregion.max.filesize=30GB
> > > *and
> > > > my region sizes are now *~4.5GB compressed (~7GB uncompressed)*.
> However
> > > > each time I restart a RS sometimes a region gets split. This is
> > > unexpected
> > > > because my key space is uniform (using MD5) and if the problem was my
> > > > *region.size
> > > >> * *hbase.hregion.max.filesize *I would expect to have all the
> regions or
> > > > almost all splitting but this only happens when I restart a RS and it
> > > > happens only for 1 or 2 regions.
> > > >
> > > > What are the different scenarios where a region can split?
> > > >
> > > > What are the right steps to restart a region server in order to avoid
> > > these
> > > > unexpected splits?
> > > >
> > > > Thank you,
> > > > Cheers
> > > > Pedro
> > >
> >
>
>
>
> Thanks Pedro for giving your solution.
>
> i see the same issue during Hbase restarts. unexpected region splits.
> i believe it is because the *IncreasingToUpperBoundRegionSplitPolicy* is
> basing
>  his calculation on the amount of ONLINE regions.
> but while the RS is starting only a couple of regions are online YET.
> so the policy things it would be no problem to add another region
> since 'there are only a few'.
> (while there are actually already are 330 for that RS for that phoenix
> table...
> yes i know i need to merge regions.
> but this problem got out of hand unnoticed for some time now here)
>
> could HBase block split region decision until it is fully up and running?
>
> Hbase 1.0.0 logs. (check mainly the last line)
>
> Mar 24, 11:06:41.494 AM INFO
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher
> Flushed, sequenceid=69436099, memsize=303.3 K, hasBloomFilter=true, into
> tmp
> file
>
> hdfs://ns/hbase/data/default/CUSTOMER/60af2857a7980ce4f1ac602dd83e05a6/.tmp/
> 0fd4988f24f24d5d9887c542182efccc
> Mar 24, 11:06:41.529 AM INFO
> org.apache.hadoop.hbase.regionserver.HStore
> Added hdfs://-ns/hbase/data/default/CUSTOMER/
> ff4ecd56e6b06f228404f05f171f8282/0/1d05cf9cac4c46008e47e3578e7a18d6,
> entries=235, sequenceid=22828972, filesize=5.5 K
> Mar 24, 11:06:41.561 AM INFO
> org.apache.hadoop.hbase.regionserver.HStore
> Completed compaction of 3 (all) file(s) in s of CUSTOMER,\x0A0+\xF6\
> xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.
> into 730489295b8c42afaec4a3b8bc38c915(size=1.4 M),
> total size for store is 1.4 M. This selection was in queue for
> 0sec, and took 0sec to execute.
> Mar 24, 11:06:41.561 AM INFO
> org.apache.hadoop.hbase.regionserver.CompactSplitThread
> Completed compaction: Request = regionName=CUSTOMER,
> \x0A0+\xF6\xD8,1457121856469.183f6134683e0213ccb15558a56f7c02.,
> storeName=s, fileCount=3, fileSize=1.7 M, priority=7,
> time=1456532583179472;
> duration=0sec
> Mar 24, 11:06:41.562 AM DEBUG
>
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy
> ShouldSplit because IB size=3269370636, sizeToCheck=2147483648,
> regionsWithCommonTable=2
>
> i will also revert back to the ConstantSizeRegionSplitPolicy
>
> Regards,
>
>
>
>

Re: HBase & SSD

[Ted Yu]

Please take a look at the attachment to HBASE-14457

Cheers

On Tue, Mar 22, 2016 at 1:56 PM, Parsian, Mahmoud 
wrote:

> If anyone is using HBase with SSDs, please let us know the performance
> improvements.
>
> Thank you!
> Best regards,
> Mahmoud
>

Re: sporadic hbase "outages"

[Ted Yu]

bq. a small number will take 20 minutes or more

Were these mappers performing selective scan on big regions ?

Can you pastebin the stack trace of region server(s) which served such
regions during slow mapper operation ?

Pastebin of region server log would also give us more clue.

On Tue, Mar 22, 2016 at 10:57 AM, feedly team  wrote:

> Recently we have been experiencing short downtimes (~2-5 minutes) in our
> hbase cluster and are trying to understand why. Many times we have HLog
> write spikes around the down times, but not always. Not sure if this is a
> red herring.
>
> We have looked a bit farther back in time and have noticed many metrics
> deteriorating over the past few months:
>
> The compaction queue size seems to be growing.
>
> The flushQueueSize and flushSizeAvgTime are growing.
>
> Some map reduce tasks run extremely slowly. Maybe 90% will complete within
> a couple minutes, but a small number will take 20 minutes or more. If I
> look at the slow mappers, there is a high value for the
> MILLIS_BETWEEN_NEXTS counter (these mappers didn't run data local).
>
> We have seen application performance worsening, during slowdowns usually
> threads are blocked on hbase connection operations
> (HConnectionManager$HConnectionImplementation.processBatch).
>
>
> This is a bit puzzling as our data nodes' os load values are really low. In
> the past, we had performance issues when load got too high. The region
> server log doesn't have anything interesting, the only messages we get are
> a handful of responseTooSlow messages
> Do these symptoms point to anything or is there something else we should
> look at? We are (still) running 0.94.20. We are going to upgrade soon, but
> we want to diagnose this issue first.
>

Re: java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method equals.(Ljava/lang/Object;)Z

[Ted Yu]

In the dependency tree from your first email, I don't see any protobuf 2.6
dependency.

Would protobuf 2.6 give you any advantage ?

Cheers

On Mon, Mar 21, 2016 at 8:37 AM, yeshwanth kumar <yeshwant...@gmail.com>
wrote:

> what if i use protobuf version 2.6,
> is it supported?
>
> please let me know
>
> -Yeshwanth
> Can you Imagine what I would do if I could do all I can - Art of War
>
> On Fri, Mar 18, 2016 at 10:31 PM, yeshwanth kumar <yeshwant...@gmail.com>
> wrote:
>
> > Thank you Ted,
> > Thank You Sean, for the detailed explanation.
> >
> > i will revert to protbuf version 2.5
> >
> >
> >
> > -Yeshwanth
> > Can you Imagine what I would do if I could do all I can - Art of War
> >
> > On Fri, Mar 18, 2016 at 8:37 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >
> >> Thanks Sean for the explanation.
> >>
> >> Yeshwanth:
> >> Looks like you're using Spark as well.
> >>
> >> org.spark-project.protobuf:protobuf-java:jar:2.4.1-shaded:compile
> >>
> >> Note that more recent Spark releases use protobuf 2.5.0 as well:
> >>
> >> 2.5.0
> >>
> >> FYI
> >>
> >> On Fri, Mar 18, 2016 at 6:11 PM, Sean Busbey <bus...@cloudera.com>
> wrote:
> >>
> >> > The long version of Ted's answer is that your example brings in
> protobuf
> >> > version 3, which is known to be incompatible with some hbase internal
> >> > optimizations.
> >> >
> >> > The specific error you get is exactly that incompatibility: we extend
> a
> >> > protobuf class so that we can avoid an unnecessary byte array copy.
> >> > Unfortunately the stuff we override is marked final in protobuf v3
> which
> >> > results in the class verification error.
> >> >
> >> > --
> >> > Sean Busbey
> >> > On Mar 18, 2016 19:38, "Ted Yu" <yuzhih...@gmail.com> wrote:
> >> >
> >> > > HBase is built with this version of protobuf:
> >> > >
> >> > > 2.5.0
> >> > >
> >> > > On Fri, Mar 18, 2016 at 5:13 PM, yeshwanth kumar <
> >> yeshwant...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > i am using  HBase 1.0.0-cdh5.5.1
> >> > > >
> >> > > > i am hitting this exception when trying to write to Hbase
> >> > > >
> >> > > > following is the stack trace
> >> > > >
> >> > > > Exception in thread "main" java.lang.VerifyError: class
> >> > > > com.google.protobuf.HBaseZeroCopyByteString overrides final method
> >> > > > equals.(Ljava/lang/Object;)Z
> >> > > > at java.lang.ClassLoader.defineClass1(Native Method)
> >> > > > at java.lang.ClassLoader.defineClass(Unknown Source)
> >> > > > at java.security.SecureClassLoader.defineClass(Unknown Source)
> >> > > > at java.net.URLClassLoader.defineClass(Unknown Source)
> >> > > > at java.net.URLClassLoader.access$100(Unknown Source)
> >> > > > at java.net.URLClassLoader$1.run(Unknown Source)
> >> > > > at java.net.URLClassLoader$1.run(Unknown Source)
> >> > > > at java.security.AccessController.doPrivileged(Native Method)
> >> > > > at java.net.URLClassLoader.findClass(Unknown Source)
> >> > > > at java.lang.ClassLoader.loadClass(Unknown Source)
> >> > > > at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> >> > > > at java.lang.ClassLoader.loadClass(Unknown Source)
> >> > > > at
> >> > >
> >> org.apache.hadoop.hbase.util.ByteStringer.(ByteStringer.java:44)
> >> > > > at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:995)
> >> > > > at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:138)
> >> > > > at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1579)
> >> > > > at
> >> >

Re: Unable to find cached index metadata

[Ted Yu]

Have you posted this question on Phoenix mailing list ?

Looks like you may get better answer there since the exception is related
to Phoenix coprocessor.

Thanks

On Mon, Mar 21, 2016 at 3:51 AM, Pedro Gandola 
wrote:

> Hi,
>
> I'm using *Phoenix4.6* and in my use case I have a table that keeps a
> sliding window of 7 days worth of data. I have 3 local indexes on this
> table and in out use case we have aprox: 150 producers that are inserting
> data (in batches of 300-1500 events) in real-time.
>
> Some days ago I started to get a lot of errors like the below ones. The
> number of errors was so large that the cluster performance dropped a lot
> and my disks read bandwidth was crazy high but the write bandwidth was
> normal. I can ensure that during that period no readers were running only
> producers.
>
> ERROR [B.defaultRpcServer.handler=25,queue=5,port=16020]
> > parallel.BaseTaskRunner: Found a failed task because:
> > org.apache.hadoop.hbase.DoNotRetryIOException: *ERROR 2008 (INT10): ERROR
> > 2008 (INT10): Unable to find cached index metadata.*
> >  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > *Index update failed*
> > java.util.concurrent.ExecutionException:
> > org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR
> > 2008 (INT10): Unable to find cached index metadata.
> >  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > Index update failed
> > Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008
> > (INT10): ERROR 2008 (INT10): Unable to find cached index metadata.
> >  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > Index update failed
> > Caused by: java.sql.SQLException: ERROR 2008 (INT10): Unable to find
> > cached index metadata.  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > INFO  [B.defaultRpcServer.handler=25,queue=5,port=16020]
> > parallel.TaskBatch: Aborting batch of tasks because Found a failed task
> > because: org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008
> (INT10):
> > ERROR 2008 (INT10): Unable to find cached index metadata.
> >  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > Index update failed
> > ERROR [B.defaultRpcServer.handler=25,queue=5,port=16020]
> *builder.IndexBuildManager:
> > Found a failed index update!*
> > INFO  [B.defaultRpcServer.handler=25,queue=5,port=16020]
> > util.IndexManagementUtil: Rethrowing
> > org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR
> > 2008 (INT10): Unable to find cached index metadata.
> >  key=4276342695061435086
> >
> region=BIDDING_EVENTS,\xFEK\x17\xE4\xB1~K\x08,1458435680333.ee29454d68f5b679a8e8cc775dd0edfa.
> > Index update failed
>
>
> I searched for the error and I made the following changes on the server
> side:
>
>- *phoenix.coprocessor.maxServerCacheTimeToLiveMs *from 30s to 2min
>- *phoenix.coprocessor.maxMetaDataCacheSize* from 20MB to 40MB
>
> After I changed these properties I restarted the cluster and the errors
> were gone but disks read bandwidth was still very high and I was getting
> *responseTooSlow* warnings. As a quick solution I created fresh tables and
> then the problems were gone.
>
> Now, after one day running with new tables I started to see the problem
> again but I think this was during a major compaction but I would like to
> understand more the reasons of these problems.
>
> - What are the major consequences of these errors? I assume that index data
> is not written within the index table, right? Then, why was the read
> bandwidth of my disks so high even without readers and after changed those
> properties?
>
> - Is there any optimal or recommended value for the above properties or am
> I missing some tunning on other properties for the metadata cache?
>
> Thank you,
> Pedro
>

Re: Cant connect to hbase thrift

[Ted Yu]

If you are running Linux / OSX on localhost, you can use 'ps' command to
search for the thrift server.

You can also check any process is listening on port 9090.

FYI

On Sun, Mar 20, 2016 at 5:20 AM, ram kumar  wrote:

> Hi,
>
> I can't able to connect to Hbase Thrift
>
> thrift.transport.TTransport.TTransportException: Could not connect to
> localhost:9090
>
> Is there a way to check thrift status, whether it is running?
>
> Thanks
>

Re: Thrift Server: How to increase "max worker threads" and "max queued requests"

[Ted Yu]

See the following from hbase-default.xml

  
hbase.thrift.minWorkerThreads
16
...
  
hbase.thrift.maxWorkerThreads
1000
...
  
hbase.thrift.maxQueuedRequests
1000

On Thu, Mar 17, 2016 at 1:05 AM, Daniel  wrote:

> Hi, I find that the Thrift server will stop responding (the request hangs
> until timeout) when the number of concurrent requests reaches several
> hundred.
>
> I guess the problem is related to "max worker threads" and "max queued
> requests", according to the following console output on Thrift start:
>
> 2016-03-17 12:05:08,514 INFO  [main] thrift.ThriftServerRunner: starting
> TBoundedThreadPoolServer on /0.0.0.0:9090 with readTimeout 6ms;
> min worker threads=16, max worker threads=1000, max queued requests=1000
>
> I'd like to know how to increase "max worker threads" and "max queued
> requests", but cannot find them in the documentation.
>
> Thanks for any hint.
>
> Daniel

Re: Hbase in memory rest running in jenkins

[Ted Yu]

maven, git (which I think you may have setup already)

winutils.exe is out of the picture.

On Wed, Mar 16, 2016 at 7:56 AM, Gaurav Agarwal <gaurav130...@gmail.com>
wrote:

> No Jenkins is running on Unix box then what need to be done
> On Mar 16, 2016 8:12 PM, "Ted Yu" <yuzhih...@gmail.com> wrote:
>
> > bq. Do I need to configure winutils.exe
> >
> > Assuming the Jenkins runs on Windows, winutils.exe would be needed.
> >
> > Cheers
> >
> > On Wed, Mar 16, 2016 at 5:19 AM, Gaurav Agarwal <gaurav130...@gmail.com>
> > wrote:
> >
> > > Hello
> > >
> > > I used hbasetestingutility to run test cases on Windows and configure
> > > winutils.exe there in the class path.
> > >
> > > But when I have to run same test case on Jenkins where no hadoop is
> > > installed .Do I need to configure winutils.exe there or what needs to
> be
> > > done please share the info
> > >
> > > Thanks
> > >
> >
>

Re: How to increase Thrift maximum concurrent connection number?

[Ted Yu]

I think what observed (16 concurrent connections) was due to the following
parameter:

hbase.thrift.minWorkerThreads

whose default value is 16.

On Fri, Mar 18, 2016 at 11:06 PM, Daniel  wrote:

> Hi, my Thrift API server seems to allow at most 16 concurrent connections.
> Is there a way to raise this limit? I cannot find a clue in the
> documentation.
>
> Thanks a lot.
> Daniel

Re: Kerberos auth using java exception

[Ted Yu]

>From the code, looks like you were trying to authenticate as user test.
But from the attached log:

bq. getLoginUser :: root (auth:SIMPLE)

FYI

On Thu, Mar 17, 2016 at 8:33 PM, Saurabh Malviya (samalviy) <
samal...@cisco.com> wrote:

>
>
> Hi,
>
>
>
> I am trying to write java code snippet to authenticate hbase using
> Kerberos, But I am getting attached exception. I tried all kind of config
> through code but no luck.
>
>
>
> -Saurabh
>
> ---Java Code
>
>
>
> HbaseGetPropertyValues properties = new
> HbaseGetPropertyValues();
>
>  properties.getPropValues();
>
>
>
>Configuration hBaseConfig =
> HBaseConfiguration.create();
>
>
>
>hBaseConfig.setInt("timeout",
> Integer.parseInt(properties.TIMEOUT));
>
>
> hBaseConfig.set("hbase.zookeeper.quorum",
> properties.HBASE_ZOOKEEPER_QUORUM);
>
>//hBaseConfig.set("hbase.master",
> "hdfs-hbase-m1.insight-test-1:6");
>
>
> hBaseConfig.set("hbase.zookeeper.property.clientport",
> properties.HBASE_ZOOKEEPER_PROPERTY_CLIENTPORT);
>
>
> hBaseConfig.set("zookeeper.znode.parent",
> properties.ZOOKEEPER_ZNODE_PARENT);
>
>
>
>
> //hBaseConfig.set("hadoop.security.authentication", "kerberos");
>
>
> hBaseConfig.set("hbase.security.authentication", "kerberos");
>
>
>
>
>  hBaseConfig.set("hbase.master.kerberos.principal",
> "test/INSIGHT@INSIGHT");
>
>
> hBaseConfig.set("hbase.regionserver.kerberos.principal",
> "test/INSIGHT@INSIGHT");
>
>
> hBaseConfig.set("hbase.master.keytab.file", "/root/test.keytab");
>
>
> hBaseConfig.set("hbase.regionserver.keytab.file", "/root/test.keytab");
>
>
>
>//runLoginAndRenewalThread(hBaseConfig,
> "/root/test.keytab", "test/INSIGHT@INSIGHT");
>
>
>
>
> UserGroupInformation.setConfiguration(hBaseConfig);
>
>
> User.login(hBaseConfig,"hbase.master.keytab.file",
> "hbase.regionserver.kerberos.principal", null);
>
>
> //UserGroupInformation.loginUserFromKeytab("test/INSIGHT@INSIGHT
> ","/root/test.keytab");
>
>UserGroupInformation ugi =
> UserGroupInformation.loginUserFromKeytabAndReturnUGI("test/INSIGHT@INSIGHT
> ","/root/samalviy.keytab");
>
>UserGroupInformation.setLoginUser(ugi);
>
>
>
>System.out.println("getLoginUser ::
> "+ugi.getLoginUser());
>
>System.out.println(hBaseConfig);
>
>
>
>
> HBaseAdmin.checkHBaseAvailable(hBaseConfig);
>
>Connection connection =
> ConnectionFactory.createConnection(hBaseConfig);
>

Re: is it a good idea to disable tables not currently hot?

[Ted Yu]

Frank:
Can you take a look at the following to see if it may help with your use
case(s) ?

HBASE-15181 A simple implementation of date based tiered compaction

Cheers

On Fri, Mar 18, 2016 at 9:58 AM, Frank Luo  wrote:

> There are two reasons I am hesitating going that route.
>
> One is that most of tables are fairly small. Going to 10GB will force
> tables to shrink to some nodes but not evenly distributed around the
> cluster, hence discouraging parallelism. But I think I can manage this
> issue if the second is resolved.
>
> The second issue, which I have battled with for two years now, is that I
> am doing online puts, which occasionally triggers compacts when a region is
> heavily inserted, and whenever it happens, all subsequent read/write are
> all on hold and I can see time out error on the client side. A typical
> compact runs for 4 minutes now and I have to increase timeout on a number
> of places to accommodate that. So if I increase the size to 10 GB, will
> compact time double?
>
> -Original Message-
> From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org]
> Sent: Friday, March 18, 2016 11:34 AM
> To: user
> Subject: Re: is it a good idea to disable tables not currently hot?
>
> So you can safely increase your maximum region size to 10GB, which will
> divide the number of regions by 2. When you will be on 1.1.2 you can also
> do online merge to reduce this number of regions. The might help too.
>
> JMS
>
> 2016-03-18 12:32 GMT-04:00 Frank Luo :
>
> > 0.98 on hdp 2.2 currently.
> >
> > Soon will be on hdp2.3.4, which has HBase 1.1.2.
> >
> > -Original Message-
> > From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org]
> > Sent: Friday, March 18, 2016 11:29 AM
> > To: user
> > Subject: Re: is it a good idea to disable tables not currently hot?
> >
> > Hi Frank,
> >
> > It might be doable.
> >
> > What HBase version are you running?
> >
> > JMS
> >
> > 2016-03-18 12:25 GMT-04:00 Frank Luo :
> >
> > > No one has experience disabling tables?
> > >
> > > -Original Message-
> > > From: Frank Luo [mailto:j...@merkleinc.com]
> > > Sent: Thursday, March 17, 2016 4:51 PM
> > > To: user@hbase.apache.org
> > > Subject: is it a good idea to disable tables not currently hot?
> > >
> > > We have a multi tenants environment and each client occupies x
> > > number of hbase regions. We currently have about 500 regions per
> > > region server and I understand the guideline is less than 200. So we
> > > need to reduce the region counts. Increasing region file size is no
> > > more an option because we are already at 5G and I don’t want to go
> higher.
> > >
> > > Due to our unique use cases, all clients are running for a few hours
> > > in a day, then being quiet for the rest of time. So I am thinking
> > > whether it is a good idea to disable all quiet tables and only
> > > enable them when they are ready to run. Does anyone have experience on
> that?
> > >
> > > One thing I worry about is the Balancer. I am pretty sure the
> > > balancer will be confused when regions come and go. And I cannot
> > > afford not to have it running in case of region server crashes and
> > > come back. So doesn’t anyone have good ideas how to handle it?
> > >
> > > I already doing compact myself so that is not an issue.
> > >
> > > Another related question, if a region is enabled but not active
> > > read/write, how much resources it takes in terms of region server?
> > >
> > > Thanks!
> > >
> > > Frank Luo
> > >
> >
> > Merkle was named a leader in Customer Insights Services Providers by
> > Forrester Research <
> > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag
> > ency/awards-recognition/merkle-named-leader-forrester?utm_source=email
> > footer_medium=email_campaign=2016MonthlyEmployeeFooter
> > >
> >
> > Forrester Research report names 500friends, a Merkle Company, a leader
> > in customer Loyalty Solutions for Midsize Organizations<
> > http://www.merkleinc.com/who-we-are-customer-relationship-marketing-ag
> > ency/awards-recognition/500friends-merkle-company-named?utm_source=ema
> > ilfooter_medium=email_campaign=2016MonthlyEmployeeFooter
> > >
> > This email and any attachments transmitted with it are intended for
> > use by the intended recipient(s) only. If you have received this email
> > in error, please notify the sender immediately and then delete it. If
> > you are not the intended recipient, you must not keep, use, disclose,
> > copy or distribute this email without the author’s prior permission.
> > We take precautions to minimize the risk of transmitting software
> > viruses, but we advise you to perform your own virus checks on any
> > attachment to this message. We cannot accept liability for any loss or
> > damage caused by software viruses. The information contained in this
> > communication may be confidential and may be subject to the
> attorney-client privilege.
> >
> Merkle was named a leader 

Re: Hbase in memory rest running in jenkins

[Ted Yu]

bq. Do I need to configure winutils.exe

Assuming the Jenkins runs on Windows, winutils.exe would be needed.

Cheers

On Wed, Mar 16, 2016 at 5:19 AM, Gaurav Agarwal 
wrote:

> Hello
>
> I used hbasetestingutility to run test cases on Windows and configure
> winutils.exe there in the class path.
>
> But when I have to run same test case on Jenkins where no hadoop is
> installed .Do I need to configure winutils.exe there or what needs to be
> done please share the info
>
> Thanks
>

Re: is it a good idea to disable tables not currently hot?

[Ted Yu]

bq. By default memsotre is 40%. Here it's 24%

bq. Memstore.lowerLimit=0.24

J-M:
Looks like you misread the config Frank listed.

On Fri, Mar 18, 2016 at 12:36 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:

> By default memsotre is 40%. Here it's 24%. There is a lot you might want to
> look at on your cluster and usecase :(
>
> 1) You might have long pause GCs causing issues. Think about offheap cache
> and reduce heap to less than 20GB
> 2) Way to many regions. Think about your usecases and tables design to
> reduce that. Increase region size to 10GB.
> 3) Increase your memstore to 40%. If your usecase is mostly puts and you
> have issues with that, increase it.
> 4) Take a look at your flush size. It's useless to increase it to 256MB if
> you are already flushing only few KBs at a time.
> 5) etc. :(
>
> JMS
>
> 2016-03-18 15:26 GMT-04:00 Frank Luo <j...@merkleinc.com>:
>
> > Anil/Jean,
> >
> > Thanks for the tips. Very helpful.
> >
> > To answer your question. I just checked, the Region server's heap is 32G,
> > instead of 36G as I previously stated, but it is in the same range and I
> do
> > see long pause on GC.
> >
> > I think the reason it was set to a high value was that we used to have
> > 2000 regions per server, before we increased region file size from the
> > default to compressed 5G.
> >
> > So what should be the heap right size given 5G file size, 400 region per
> > server on an 80 node cluster?
> >
> > At this time, I think memstore related setting are all defaults from HDP.
> >
> > Flush.size = 128M
> > Memstore.lowerLimit=0.24
> > Memstore.upperLimit=0.25
> > Hstore.blocking.storefiles=10
> >
> > -Original Message-
> > From: anil gupta [mailto:anilgupt...@gmail.com]
> > Sent: Friday, March 18, 2016 12:37 PM
> > To: user@hbase.apache.org
> > Subject: Re: is it a good idea to disable tables not currently hot?
> >
> > @Frank, regarding write amplification:
> > 1. What is your flush size? default is 128 MB. You should increase your
> > "hbase.hregion.memstore.flush.size" so that you dont run over the limit
> of
> > store files.
> > 2. Have a look at "hbase.regionserver.global.memstore.lowerLimit".
> > 3. Your heap size is also too big. Maybe you also run into GC issues.
> Have
> > you checked your GC logs?
> > 4. IMO, writes getting blocks at 9 files might be very less for a big
> > Region Server. So, you can also consider increasing that.
> >
> > On Fri, Mar 18, 2016 at 10:22 AM, Frank Luo <j...@merkleinc.com> wrote:
> >
> > > Ted,
> > >
> > > Thanks for sharing. I learned something today.
> > >
> > > But I guess it doesn't apply to my case.  It is true that I only run a
> > > client for a few hours in a day, but the data is not date based.
> > >
> > > -Original Message-
> > > From: Ted Yu [mailto:yuzhih...@gmail.com]
> > > Sent: Friday, March 18, 2016 12:10 PM
> > > To: user@hbase.apache.org
> > > Subject: Re: is it a good idea to disable tables not currently hot?
> > >
> > > Frank:
> > > Can you take a look at the following to see if it may help with your
> > > use
> > > case(s) ?
> > >
> > > HBASE-15181 A simple implementation of date based tiered compaction
> > >
> > > Cheers
> > >
> > > On Fri, Mar 18, 2016 at 9:58 AM, Frank Luo <j...@merkleinc.com> wrote:
> > >
> > > > There are two reasons I am hesitating going that route.
> > > >
> > > > One is that most of tables are fairly small. Going to 10GB will
> > > > force tables to shrink to some nodes but not evenly distributed
> > > > around the cluster, hence discouraging parallelism. But I think I
> > > > can manage this issue if the second is resolved.
> > > >
> > > > The second issue, which I have battled with for two years now, is
> > > > that I am doing online puts, which occasionally triggers compacts
> > > > when a region is heavily inserted, and whenever it happens, all
> > > > subsequent read/write are all on hold and I can see time out error
> > > > on the client side. A typical compact runs for 4 minutes now and I
> > > > have to increase timeout on a number of places to accommodate that.
> > > > So if I increase the size to 10 GB, will compact time double?
> > > >
> > > > -Original Message-
> > > > Fr

Re: why Hbase only split regions in one RegionServer

[Ted Yu]

In Jack's case, even if the table was pre-split, after loading some data,
if balancer didn't run, the regions would still be out of balance.

We should help Jack find out the cause for imbalance of regions.

On Wed, Mar 16, 2016 at 4:17 AM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:

> +1 with what Heng said. I think we should just deprecate the ability to not
> pre-split a table ;) It's always good to pre-split it based on your key
> design...
>
> 2016-03-16 0:17 GMT-04:00 Heng Chen <heng.chen.1...@gmail.com>:
>
> > bq. the table I created by default having only one region
> >
> > Why not pre-split table into more regions when create it?
> >
> > 2016-03-16 11:38 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
> >
> > > When one region is split into two, both daughter regions are opened on
> > the
> > > same server where parent region was opened.
> > >
> > > Can you provide a bit more information:
> > >
> > > release of hbase
> > > whether balancer was turned on - you can inspect master log to see if
> > > balancer was on
> > >
> > > Consider pastebinning portion of master log.
> > >
> > > Thanks
> > >
> > > On Tue, Mar 15, 2016 at 4:43 PM, jackwang <wangjiajie...@gmail.com>
> > wrote:
> > >
> > > > I was writing 300GiB data to my Hbase table user_info, the table I
> > > created
> > > > by
> > > > default having only one region. When the writing was going I saw one
> > > region
> > > > became two regions and more late on it became 8 regions. But my
> > confusion
> > > > is
> > > > that the 8 regions were kept in the same RegionServer.
> > > >
> > > > Why Hbase didn't split the regions to different RegionServer. btw, I
> > had
> > > 10
> > > > physical RegionsServers in my Hbase cluster, and the region size I
> set
> > is
> > > > 20GiB, Thanks!
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://apache-hbase.679495.n3.nabble.com/why-Hbase-only-split-regions-in-one-RegionServer-tp4078497.html
> > > > Sent from the HBase User mailing list archive at Nabble.com.
> > > >
> > >
> >
>

Re: java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method equals.(Ljava/lang/Object;)Z

[Ted Yu]

HBase is built with this version of protobuf:

2.5.0

On Fri, Mar 18, 2016 at 5:13 PM, yeshwanth kumar 
wrote:

> i am using  HBase 1.0.0-cdh5.5.1
>
> i am hitting this exception when trying to write to Hbase
>
> following is the stack trace
>
> Exception in thread "main" java.lang.VerifyError: class
> com.google.protobuf.HBaseZeroCopyByteString overrides final method
> equals.(Ljava/lang/Object;)Z
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(Unknown Source)
> at java.security.SecureClassLoader.defineClass(Unknown Source)
> at java.net.URLClassLoader.defineClass(Unknown Source)
> at java.net.URLClassLoader.access$100(Unknown Source)
> at java.net.URLClassLoader$1.run(Unknown Source)
> at java.net.URLClassLoader$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at org.apache.hadoop.hbase.util.ByteStringer.(ByteStringer.java:44)
> at
>
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:995)
> at
>
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:138)
> at
>
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1579)
> at
>
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1293)
> at
>
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1125)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:369)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
> at
>
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:198)
> at
>
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.doMutate(BufferedMutatorImpl.java:141)
> at
>
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:105)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1117)
> at
>
> com.yotabites.fileIO.WriteToHbase.readZipFilesToHbase(WriteToHbase.java:440)
> at com.yotabites.fileIO.WriteToHbase.main(WriteToHbase.java:475)
>
> i am using protocol buffers 3.0 is that the issue?
>
> following is the dependency tree
>
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ Sample---
> [INFO] com.sample.sample:0.0.1-SNAPSHOT
> [INFO] +- junit:junit:jar:4.12:compile
> [INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:compile
> [INFO] +-
> com.googlecode.protobuf-java-format:protobuf-java-format:jar:1.4:compile
> [INFO] +- com.google.protobuf:protobuf-java:jar:3.0.0-beta-2:compile
> [INFO] +- org.apache.logging.log4j:log4j-core:jar:2.3:compile
> [INFO] |  \- org.apache.logging.log4j:log4j-api:jar:2.3:compile
> [INFO] +- org.apache.hadoop:hadoop-client:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  +- org.apache.hadoop:hadoop-common:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] |  |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
> [INFO] |  |  +- commons-collections:commons-collections:jar:3.2.2:compile
> [INFO] |  |  +- commons-configuration:commons-configuration:jar:1.6:compile
> [INFO] |  |  |  +- commons-digester:commons-digester:jar:1.8:compile
> [INFO] |  |  |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
> [INFO] |  |  |  \-
> commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
> [INFO] |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
> [INFO] |  |  +- org.apache.curator:curator-client:jar:2.7.1:compile
> [INFO] |  |  +- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile
> [INFO] |  |  \- org.apache.commons:commons-compress:jar:1.4.1:compile
> [INFO] |  | \- org.tukaani:xz:jar:1.0:compile
> [INFO] |  +- org.apache.hadoop:hadoop-hdfs:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.4:compile
> [INFO] |  |  +- xerces:xercesImpl:jar:2.9.1:compile
> [INFO] |  |  |  \- xml-apis:xml-apis:jar:1.3.04:compile
> [INFO] |  |  \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
> [INFO] |  +-
> org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  +-
> org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  |  +-
> org.apache.hadoop:hadoop-yarn-client:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  |  \-
> org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  |  \-
>
> org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  +- org.apache.hadoop:hadoop-yarn-api:jar:2.6.0-cdh5.5.1:compile
> [INFO] |  +-
> 

Re: java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method equals.(Ljava/lang/Object;)Z

[Ted Yu]

Thanks Sean for the explanation.

Yeshwanth:
Looks like you're using Spark as well.

org.spark-project.protobuf:protobuf-java:jar:2.4.1-shaded:compile

Note that more recent Spark releases use protobuf 2.5.0 as well:

2.5.0

FYI

On Fri, Mar 18, 2016 at 6:11 PM, Sean Busbey <bus...@cloudera.com> wrote:

> The long version of Ted's answer is that your example brings in protobuf
> version 3, which is known to be incompatible with some hbase internal
> optimizations.
>
> The specific error you get is exactly that incompatibility: we extend a
> protobuf class so that we can avoid an unnecessary byte array copy.
> Unfortunately the stuff we override is marked final in protobuf v3 which
> results in the class verification error.
>
> --
> Sean Busbey
> On Mar 18, 2016 19:38, "Ted Yu" <yuzhih...@gmail.com> wrote:
>
> > HBase is built with this version of protobuf:
> >
> > 2.5.0
> >
> > On Fri, Mar 18, 2016 at 5:13 PM, yeshwanth kumar <yeshwant...@gmail.com>
> > wrote:
> >
> > > i am using  HBase 1.0.0-cdh5.5.1
> > >
> > > i am hitting this exception when trying to write to Hbase
> > >
> > > following is the stack trace
> > >
> > > Exception in thread "main" java.lang.VerifyError: class
> > > com.google.protobuf.HBaseZeroCopyByteString overrides final method
> > > equals.(Ljava/lang/Object;)Z
> > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > at java.lang.ClassLoader.defineClass(Unknown Source)
> > > at java.security.SecureClassLoader.defineClass(Unknown Source)
> > > at java.net.URLClassLoader.defineClass(Unknown Source)
> > > at java.net.URLClassLoader.access$100(Unknown Source)
> > > at java.net.URLClassLoader$1.run(Unknown Source)
> > > at java.net.URLClassLoader$1.run(Unknown Source)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(Unknown Source)
> > > at java.lang.ClassLoader.loadClass(Unknown Source)
> > > at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> > > at java.lang.ClassLoader.loadClass(Unknown Source)
> > > at
> > org.apache.hadoop.hbase.util.ByteStringer.(ByteStringer.java:44)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:995)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:138)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1579)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1293)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1125)
> > > at
> > >
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:369)
> > > at
> > >
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:198)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.doMutate(BufferedMutatorImpl.java:141)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:105)
> > > at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1117)
> > > at
> > >
> > >
> >
> com.yotabites.fileIO.WriteToHbase.readZipFilesToHbase(WriteToHbase.java:440)
> > > at com.yotabites.fileIO.WriteToHbase.main(WriteToHbase.java:475)
> > >
> > > i am using protocol buffers 3.0 is that the issue?
> > >
> > > following is the dependency tree
> > >
> > > [INFO]
> > > [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ Sample---
> > > [INFO] com.sample.sample:0.0.1-SNAPSHOT
> > > [INFO] +- junit:junit:jar:4.12:compile
> > > [INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:compile
> > > [INFO] +-
> > >
> com.googlecode.protobuf-java-format:protobuf-java-format:jar:1.4:compile
> > > [INFO] +- com.google.protobuf:protobuf-java:jar:3.0.0-beta-2:compile
> > > [INFO] +- org.apache.logging.log4j:log4j-core:jar:2.

Re: why Hbase only split regions in one RegionServer

[Ted Yu]

When one region is split into two, both daughter regions are opened on the
same server where parent region was opened.

Can you provide a bit more information:

release of hbase
whether balancer was turned on - you can inspect master log to see if
balancer was on

Consider pastebinning portion of master log.

Thanks

On Tue, Mar 15, 2016 at 4:43 PM, jackwang  wrote:

> I was writing 300GiB data to my Hbase table user_info, the table I created
> by
> default having only one region. When the writing was going I saw one region
> became two regions and more late on it became 8 regions. But my confusion
> is
> that the 8 regions were kept in the same RegionServer.
>
> Why Hbase didn't split the regions to different RegionServer. btw, I had 10
> physical RegionsServers in my Hbase cluster, and the region size I set is
> 20GiB, Thanks!
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/why-Hbase-only-split-regions-in-one-RegionServer-tp4078497.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Re: How to change the base data directory with HBaseTestingUtility

[Ted Yu]

Please take a look at:

  private Path getBaseTestDir() {

String PathName = System.getProperty(

BASE_TEST_DIRECTORY_KEY, DEFAULT_BASE_TEST_DIRECTORY);

where:

  public static final String BASE_TEST_DIRECTORY_KEY =

  "test.build.data.basedirectory";

FYI

On Mon, Mar 14, 2016 at 3:06 PM, Nkechi Achara 
wrote:

> Hi,
>
> I am having issues running the HbaseTestingUtility within IntelliJ IDE, and
> I can see the following error is probably a result of the file name being
> too long:
>
> 16/03/14 22:45:13 WARN datanode.DataNode: IOException in
> BlockReceiver.run():
> java.io.IOException: Failed to move meta file for ReplicaBeingWritten,
> blk_1073741825_1001, RBW
>   getNumBytes() = 7
>   getBytesOnDisk()  = 7
>   getVisibleLength()= 7
>   getVolume()   =
>
> C:\Users\user1\Documents\work\Repos\hadoop-analys\reporting\mrkts-surveillance\target\test-data\9654a646-e923-488a-9e20-46396fd15292\dfscluster_6b264e6b-0218-4f30-ad5b-72e838940b1e\dfs\data\data1\current
>   getBlockFile()=
>
> C:\Users\user1\Documents\work\Repos\hadoop-analys\reporting\mrkts-surveillance\target\test-data\9654a646-e923-488a-9e20-46396fd15292\dfscluster_6b264e6b-0218-4f30-ad5b-72e838940b1e\dfs\data\data1\current\BP-429386217-192.168.1.110-1457991908038\current\rbw\blk_1073741825
>   bytesAcked=7
>   bytesOnDisk=7 from
>
> C:\Users\user1\Documents\work\Repos\hadoop-analys\reporting\mrkts-surveillance\target\test-data\9654a646-e923-488a-9e20-46396fd15292\dfscluster_6b264e6b-0218-4f30-ad5b-72e838940b1e\dfs\data\data1\current\BP-429386217-192.168.1.110-1457991908038\current\rbw\blk_1073741825_1001.meta
> to
>
> C:\Users\user1\Documents\work\Repos\hadoop-analys\reporting\mrkts-surveillance\target\test-data\9654a646-e923-488a-9e20-46396fd15292\dfscluster_6b264e6b-0218-4f30-ad5b-72e838940b1e\dfs\data\data1\current\BP-429386217-192.168.1.110-1457991908038\current\finalized\subdir0\subdir0\blk_1073741825_1001.meta
> at
>
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:615)
> at
>
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addBlock(BlockPoolSlice.java:250)
> at
>
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addBlock(FsVolumeImpl.java:229)
> at
>
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeReplica(FsDatasetImpl.java:1119)
> at
>
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeBlock(FsDatasetImpl.java:1100)
> at
>
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.finalizeBlock(BlockReceiver.java:1293)
> at
>
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1233)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: 3: The system cannot find the path specified.
>
> Any idea, how I can specify the base directory of Hbasetestingutility to
> not use this gargantuan starting directory?
>
> Thanks,
>
> Keech
>

Re: Testing HBase packaging changes

[Ted Yu]

On top of letting unit test suite pass,
running IntegrationTestBigLinkedList (and integration test related to the
feature you modify) regularly is a good practice such that the changes do
not introduce regression.

FYI

On Mon, Mar 14, 2016 at 2:15 PM, Bryan Beaudreault  wrote:

> Hello,
>
> We've been using CDH community for a few years now, but have reached the
> point where we want to be able to do some development and backporting
> upstream. We'd like to keep the base CDH packaging for consistency, but
> have come up with some automation around building CDH from source RPM, with
> our patches applied.
>
> I realize this starts to get into the territory of self-support, and we're
> ready for that in the areas we plan to work (mostly around
> reporting/monitoring, for now). But I'd like to know if anyone has any best
> practices for testing that your build of HBase is up to snuff?
>
> Are the built in unit tests enough? Do you run YCSB? PerformanceEval?
> Something else?
>
> While we will of course be careful with testing and stressing our own
> changes, I'm also just wanting to make sure that the we are hitting all the
> right places to catch artifacts from the build process or missed upstream
> issues.
>
> Thanks!
>

Re: Example of spinning up a Hbase mock style test for integration testing in scala

[Ted Yu]

You can inspect the output from 'mvn dependency:tree' to see if any
incompatible hadoop dependency exists.

FYI

On Mon, Mar 14, 2016 at 10:26 AM, Parsian, Mahmoud 
wrote:

> Hi Keech,
>
> Please post your sample test, its run log, version of Hbase , hadoop, …
> And make sure that hadoop-core-1.2.1.jar is not your classpath (causes
> many errors!).
>
> Best,
> Mahmoud
> From: Nkechi Achara >
> Date: Monday, March 14, 2016 at 10:14 AM
> To: "user@hbase.apache.org" <
> user@hbase.apache.org>, Mahmoud Parsian <
> mpars...@illumina.com>
> Subject: Re: Example of spinning up a Hbase mock style test for
> integration testing in scala
>
>
> Thanks Mahmoud,
>
> This is what I am using,  but as the previous reply stated, I  receiving
> an exception when starting the cluster.
> Thinking about it, it looks to be more of a build problem of my hbase mini
> cluster,  as I am receiving the following error:
>
> 16/03/14 12:29:00 WARN datanode.DataNode: IOException in
> BlockReceiver.run():
>
> java.io.IOException: Failed to move meta file for ReplicaBeingWritten,
> blk_1073741825_1001, RBW
>
>   getNumBytes() = 7
>
>   getBytesOnDisk()  = 7
>
>   getVisibleLength()= 7
>
>   getVolume()   =
> C:\Users\unknown\Documents\trs\target\test-data\780d11ca-27b8-4004-bed8-480bc9903125\dfscluster_d292c05b-0190-43b1-83b2-bebf483c8b3c\dfs\data\data1\current
>
>   getBlockFile()=
> C:\Users\unknown\Documents\trs\target\test-data\780d11ca-27b8-4004-bed8-480bc9903125\dfscluster_d292c05b-0190-43b1-83b2-bebf483c8b3c\dfs\data\data1\current\BP-1081755239-10.66.90.86-1457954925705\current\rbw\blk_1073741825
>
>   bytesAcked=7
>
>   bytesOnDisk=7 from
> C:\Users\unknown\Documents\trs\target\test-data\780d11ca-27b8-4004-bed8-480bc9903125\dfscluster_d292c05b-0190-43b1-83b2-bebf483c8b3c\dfs\data\data1\current\BP-1081755239-10.66.90.86-1457954925705\current\rbw\blk_1073741825_1001.meta
> to
> C:\Users\unknown\Documents\trs\target\test-data\780d11ca-27b8-4004-bed8-480bc9903125\dfscluster_d292c05b-0190-43b1-83b2-bebf483c8b3c\dfs\data\data1\current\BP-1081755239-10.66.90.86-1457954925705\current\finalized\subdir0\subdir0\blk_1073741825_1001.meta
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:615)
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addBlock(BlockPoolSlice.java:250)
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addBlock(FsVolumeImpl.java:229)
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeReplica(FsDatasetImpl.java:1119)
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.finalizeBlock(FsDatasetImpl.java:1100)
>
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.finalizeBlock(BlockReceiver.java:1293)
>
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1233)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: 3: The system cannot find the path specified.
>
> at org.apache.hadoop.io.nativeio.NativeIO.renameTo0(Native Method)
>
> at org.apache.hadoop.io.nativeio.NativeIO.renameTo(NativeIO.java:830)
>
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.moveBlockFiles(FsDatasetImpl.java:613)
>
> ... 7 more
>
> 16/03/14 12:29:00 INFO datanode.DataNode: Starting CheckDiskError Thread
>
> Thanks,
>
> Keech
>
> On 14 Mar 2016 6:10 pm, "Parsian, Mahmoud" > wrote:
> Hi Keech,
>
> You may use the org.apache.hadoop.hbase.HBaseCommonTestingUtility class to
> start a ZK, and an HBase cluster and then do your unit tests and
> integration.
> I am using this with junit and it works very well. But I am using Java
> only.
>
> Best regards,
> Mahmoud Parsian
>
>
> On 3/13/16, 11:52 PM, "Nkechi Achara" > wrote:
>
> >Hi,
> >
> >I am trying to find an example of how to spin up a Hbase server in a mock
> >or integration style, so I can test my code locally in my IDE.
> >I have tried fake-hbase and hbase testing utility and receive errors
> >especially when trying to start the cluster.
> >Has anyone got any examples in scala to do this?
> >
> >Thanks,
> >
> >Keech
>
>

Re: use of hbase client in application server

[Ted Yu]

For #1, single Hconnection should work.

For #2, can you clarify ? As long as the hbase-site.xml used to create
the Hconnection
is still valid, you can continue using the connection.

For #3, they're handled by the connection automatically.

For #4, the HTable ctor you cited doesn't exist in master branch.
You can control the following parameters for the ThreadPoolExecutor - see
HTable#getDefaultExecutor():

int maxThreads = conf.getInt("hbase.htable.threads.max", Integer.
MAX_VALUE);

if (maxThreads == 0) {

  maxThreads = 1; // is there a better default?

}

int corePoolSize = conf.getInt("hbase.htable.threads.coresize", 1);

long keepAliveTime = conf.getLong("hbase.htable.threads.keepalivetime",
60);

On Sun, Mar 13, 2016 at 3:12 AM, Shushant Arora 
wrote:

> I have a requirement to use long running hbase client in application
> server.
>
> 1.Do I need to create multiple HConnections or single Hconnection will
> work?
> 2. DO I need to check whether Hconnection is still active before using it
> to create Htable instance.
> 3.DO I need to handle region split and regionserver changes while using
> Hconnection or are they handled automatically.
> 4.Whats the use of thread pool in Htable instance.
> ExecutorService threadPool;
> HTable h = new HTable(conf, Bytes.toBytes("tablename"), threadPool);
>
>
> Thanks!
>

Re: Upgrade from 1.1.3 to 1.2.0 failed

[Ted Yu]

Shortly before the 'All datanodes are bad' error, I saw:

2016-03-09 11:57:20,126 INFO  [LeaseRenewer:hbase@aws-hbase-staging]
retry.RetryInvocationHandler: Exception while invoking renewLease of
class ClientNamenodeProtocolTranslatorPB over
staging-aws-hbase-2.aws.px/10.231.16.197:8020. Trying to fail over
immediately.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category WRITE is not supported in state standby
at 
org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:87)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:1774)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1313)

Looks like your hdfs was transitioning.

FYI


On Wed, Mar 9, 2016 at 7:25 AM, Ted Yu <yuzhih...@gmail.com> wrote:

> From the region server log, did you notice this:
>
> 2016-03-09 11:59:49,159 WARN  
> [regionserver/staging-aws-hbase-data-1.aws.px/10.231.16.30:16020] 
> wal.ProtobufLogWriter: Failed to write trailer, non-fatal, continuing...
> java.io.IOException: All datanodes 
> DatanodeInfoWithStorage[10.231.16.30:50010,DS-a49fd123-fefc-46e2-83ca-6ae462081702,DISK]
>  are bad. Aborting...
>   at 
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1084)
>   at 
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:876)
>   at 
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:402)
>
>
> Was hdfs stable around that time ?
>
> I am more interested in the failure of fresh 1.2.0 installation.
>
> Please pastebin server logs for that incident.
>
>
> On Wed, Mar 9, 2016 at 6:19 AM, Michal Medvecky <medve...@pexe.so> wrote:
>
>> You can check both logs at
>>
>> http://michal.medvecky.net/log-master.txt
>> http://michal.medvecky.net/log-rs.txt
>>
>> First restart after upgrade happened at 10:29.
>>
>> I did not find anything useful.
>>
>> Michal
>>
>> On Wed, Mar 9, 2016 at 2:58 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>
>> > Can you take a look at data-1 server log around this time frame to see
>> > what happened ?
>> >
>> > Thanks
>> >
>> > > On Mar 9, 2016, at 3:44 AM, Michal Medvecky <medve...@pexe.so> wrote:
>> > >
>> > > Hello,
>> > >
>> > > I upgraded my single hbase master and single hbase regionserver from
>> > 1.1.3
>> > > to 1.2.0, by simply stopping both, upgrading packages (i download
>> binary
>> > > packages from hbase.org) and starting them again.
>> > >
>> > > It did not come up; master is stuck in assigning hbase:meta:
>> > >
>> > > 2016-03-09 11:28:11,491 INFO
>> > > [staging-aws-hbase-3:16000.activeMasterManager]
>> master.AssignmentManager:
>> > > Processing 1588230740 in state: M_ZK_REGION_OFFLINE
>> > > 2016-03-09 11:28:11,491 INFO
>> > > [staging-aws-hbase-3:16000.activeMasterManager] master.RegionStates:
>> > > Transition {1588230740 state=OFFLINE, ts=1457522891475, server=null}
>> to
>> > > {1588230740 state=OFFLINE, ts=1457522891491,
>> > > server=staging-aws-hbase-data-1.aws.px,16020,1457522034036}
>> > > 2016-03-09 11:28:11,492 INFO
>> > > [staging-aws-hbase-3:16000.activeMasterManager]
>> > > zookeeper.MetaTableLocator: Setting hbase:meta region location in
>> > ZooKeeper
>> > > as staging-aws-hbase-data-1.aws.px,16020,1457522034036
>> > > 2016-03-09 11:42:10,611 ERROR [Thread-65] master.HMaster: Master
>> failed
>> > to
>> > > complete initialization after 90ms. Please consider submitting a
>> bug
>> > > report including a thread dump of this process.
>> > >
>> > > Did I miss something?
>> > >
>> > > I tried downgrading back to 1.1.3 but later realized this is not
>> > supported
>> > > (and does not work of course).
>> > >
>> > > Thank you
>> > >
>> > > Michal
>> >
>>
>
>

Re: Upgrade from 1.1.3 to 1.2.0 failed

[Ted Yu]

>From the region server log, did you notice this:

2016-03-09 11:59:49,159 WARN
[regionserver/staging-aws-hbase-data-1.aws.px/10.231.16.30:16020]
wal.ProtobufLogWriter: Failed to write trailer, non-fatal,
continuing...
java.io.IOException: All datanodes
DatanodeInfoWithStorage[10.231.16.30:50010,DS-a49fd123-fefc-46e2-83ca-6ae462081702,DISK]
are bad. Aborting...
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1084)
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:876)
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:402)


Was hdfs stable around that time ?

I am more interested in the failure of fresh 1.2.0 installation.

Please pastebin server logs for that incident.


On Wed, Mar 9, 2016 at 6:19 AM, Michal Medvecky <medve...@pexe.so> wrote:

> You can check both logs at
>
> http://michal.medvecky.net/log-master.txt
> http://michal.medvecky.net/log-rs.txt
>
> First restart after upgrade happened at 10:29.
>
> I did not find anything useful.
>
> Michal
>
> On Wed, Mar 9, 2016 at 2:58 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > Can you take a look at data-1 server log around this time frame to see
> > what happened ?
> >
> > Thanks
> >
> > > On Mar 9, 2016, at 3:44 AM, Michal Medvecky <medve...@pexe.so> wrote:
> > >
> > > Hello,
> > >
> > > I upgraded my single hbase master and single hbase regionserver from
> > 1.1.3
> > > to 1.2.0, by simply stopping both, upgrading packages (i download
> binary
> > > packages from hbase.org) and starting them again.
> > >
> > > It did not come up; master is stuck in assigning hbase:meta:
> > >
> > > 2016-03-09 11:28:11,491 INFO
> > > [staging-aws-hbase-3:16000.activeMasterManager]
> master.AssignmentManager:
> > > Processing 1588230740 in state: M_ZK_REGION_OFFLINE
> > > 2016-03-09 11:28:11,491 INFO
> > > [staging-aws-hbase-3:16000.activeMasterManager] master.RegionStates:
> > > Transition {1588230740 state=OFFLINE, ts=1457522891475, server=null} to
> > > {1588230740 state=OFFLINE, ts=1457522891491,
> > > server=staging-aws-hbase-data-1.aws.px,16020,1457522034036}
> > > 2016-03-09 11:28:11,492 INFO
> > > [staging-aws-hbase-3:16000.activeMasterManager]
> > > zookeeper.MetaTableLocator: Setting hbase:meta region location in
> > ZooKeeper
> > > as staging-aws-hbase-data-1.aws.px,16020,1457522034036
> > > 2016-03-09 11:42:10,611 ERROR [Thread-65] master.HMaster: Master failed
> > to
> > > complete initialization after 90ms. Please consider submitting a
> bug
> > > report including a thread dump of this process.
> > >
> > > Did I miss something?
> > >
> > > I tried downgrading back to 1.1.3 but later realized this is not
> > supported
> > > (and does not work of course).
> > >
> > > Thank you
> > >
> > > Michal
> >
>

Re: Upgrade from 1.1.3 to 1.2.0 failed

[Ted Yu]

Can you take a look at data-1 server log around this time frame to see what 
happened ?

Thanks

> On Mar 9, 2016, at 3:44 AM, Michal Medvecky  wrote:
> 
> Hello,
> 
> I upgraded my single hbase master and single hbase regionserver from 1.1.3
> to 1.2.0, by simply stopping both, upgrading packages (i download binary
> packages from hbase.org) and starting them again.
> 
> It did not come up; master is stuck in assigning hbase:meta:
> 
> 2016-03-09 11:28:11,491 INFO
> [staging-aws-hbase-3:16000.activeMasterManager] master.AssignmentManager:
> Processing 1588230740 in state: M_ZK_REGION_OFFLINE
> 2016-03-09 11:28:11,491 INFO
> [staging-aws-hbase-3:16000.activeMasterManager] master.RegionStates:
> Transition {1588230740 state=OFFLINE, ts=1457522891475, server=null} to
> {1588230740 state=OFFLINE, ts=1457522891491,
> server=staging-aws-hbase-data-1.aws.px,16020,1457522034036}
> 2016-03-09 11:28:11,492 INFO
> [staging-aws-hbase-3:16000.activeMasterManager]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as staging-aws-hbase-data-1.aws.px,16020,1457522034036
> 2016-03-09 11:42:10,611 ERROR [Thread-65] master.HMaster: Master failed to
> complete initialization after 90ms. Please consider submitting a bug
> report including a thread dump of this process.
> 
> Did I miss something?
> 
> I tried downgrading back to 1.1.3 but later realized this is not supported
> (and does not work of course).
> 
> Thank you
> 
> Michal

Re: Hbase procedure

[Ted Yu]

The procedure in this context is different from RDBMS stored procedure.
In flush(), you would see:

execProcedure("flush-table-proc", tableName.getNameAsString(),

  new HashMap());

In MasterFlushTableProcedureManager.java :

  public static final String FLUSH_TABLE_PROCEDURE_SIGNATURE =
"flush-table-proc";

The above is returned by getProcedureSignature().

See the javadoc for MasterProcedureManager to get some idea of what
globally barriered procedures are.


FYI

On Wed, Mar 2, 2016 at 3:52 AM, Gaurav Agarwal 
wrote:

> Hello
>
> I was exploring namespace in hbasse and suddenly saw there is something
> called as hbaseadmin.execProcedure(...);
> The comment is it is a globally barrier distributed stored procedure.
>
> Can I take some one help me to find Is this similar to rdbms stored proc?
>
> Or any inputs where can be used in client application.
>

Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)

[Ted Yu]

If there is no objection, I will create a backport JIRA tomorrow and attach 
patch. 

Thanks

> On Feb 29, 2016, at 8:14 PM, Andrew Purtell <andrew.purt...@gmail.com> wrote:
> 
> Thanks Jon. This is very helpful information for those of us who don't have 
> visibility to these users. Answers my question. 
> 
> 
>> On Feb 29, 2016, at 6:56 PM, Jonathan Hsieh <j...@cloudera.com> wrote:
>> 
>> The feature is definitely not abandoned -- we have few sizable customers at
>> PB scale that I can recall off the top of my head that have been using it
>> for over 8-12 months in the version backported to CDH (we backported an
>> early version back in oct/14 (CDH5.2), and updated with the recent more
>> recent changes in roughly may/15 (CDH5.4).   These are used primarily to
>> store documents -- think PDF files. They are pretty happy -- the customer
>> reported that it was slightly slower on the write side (10-15%) than a
>> competing system but significantly faster on the read side (3x-4x
>> throughput).
>> 
>> Over the holidays we shook out a semi-rare data loss issue with mob
>> compaction that had this as the root cause[1] -- (since pointers are only
>> stored in the "normal hfiles" the volume of datas on this case was fairly
>> large).
>> 
>> We've been working on trying to get at least one of them to present at
>> hbasecon, but I'm not reviewing submissions this year and don't know if
>> they made it or not.
>> 
>> Jon.
>> 
>> [1] https://issues.apache.org/jira/browse/HBASE-15035
>> 
>> On Fri, Feb 26, 2016 at 12:27 PM, Andrew Purtell <apurt...@apache.org>
>> wrote:
>> 
>>> ​I think we need at least one success story or one very interested user
>>> with a real project on the line to justify a backport. Otherwise it's a
>>> feature without any users - technically, abandoned. ​
>>> 
>>>> On Fri, Feb 26, 2016 at 10:11 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>>>> 
>>>> I am interested in hearing about user experience with MOB feature as
>>> well.
>>>> 
>>>> In my opinion, this feature is a nice addition to branch-1.
>>>> 
>>>> On Wed, Feb 10, 2016 at 1:45 PM, Andrew Purtell <apurt...@apache.org>
>>>> wrote:
>>>> 
>>>>> +user@
>>>>> 
>>>>> Is there anyone using the MOB feature in trunk for anything who can
>>>> comment
>>>>> on how well it's been working out? Intel folks maybe?
>>>>> 
>>>>> On Thu, Jan 21, 2016 at 7:46 AM, Sean Busbey <bus...@apache.org>
>>> wrote:
>>>>> 
>>>>>> The last time MOB on branch-1 came up, folks were concerned that it
>>>>>> wasn't stable enough in master yet. Is that still the case?
>>>>>> 
>>>>>> Can we get a [DISCUSS] flagged thread to see what, if anything, folks
>>>>>> would like to see gate inclusion in branch-1?
>>>>>> 
>>>>>> On Tue, Jan 19, 2016 at 7:31 PM, Jonathan Hsieh <j...@cloudera.com>
>>>>> wrote:
>>>>>>> +1 to 1.2 being feature complete corrently.  There has already
>>> been a
>>>>>>> release candidate and folks are burning down the blockers currently
>>>> to
>>>>>> prep
>>>>>>> for the next RC.
>>>>>>> 
>>>>>>> I like the idea of mob and sparkonhbase for 1.3.  I'm more
>>>> comfortable
>>>>>> with
>>>>>>> sparkonhbase -- it is a new module and thus not as invasive.
>>>>>>> 
>>>>>>> Jon.
>>>>>>> 
>>>>>>> On Tue, Jan 19, 2016 at 4:55 PM, Andrew Purtell <
>>>>>> andrew.purt...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Pretty sure Sean expressed 1.2 is feature complete and I'd support
>>>>> that.
>>>>>>>> Can we wait for 1.3 for MOB ? Can look at Spark connector then
>>> too.
>>>>>>>> 
>>>>>>>>> On Jan 19, 2016, at 4:52 PM, Ted Yu <yuzhih...@gmail.com>
>>> wrote:
>>>>>>>>> 
>>>>>>>>> Looks like 1.2.0 RC is in near future.
>>>>>>>>> 
>>>>>>>>> I wonder if it is time to revive this thread (due to customer
>>>>>> interest).
>

Re: [Error] : while registering Hbase table with hive

[Ted Yu]

Can you give us some more information ?

release of hbase
release of Hive

code snippet for registering hbase table

On Sun, Feb 28, 2016 at 7:21 PM, Divya Gehlot 
wrote:

> Hi,
> I trying to register a hbase table with hive and getting following error :
>
>  Error while processing statement: FAILED: Execution Error, return code 1
> > from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException:
> > MetaException(message:org.apache.hadoop.hive.serde2.SerDeException Error:
> > the HBase columns mapping contains a badly formed column family, column
> > qualifier specification.)
>
>
>
> May I know what could be the possible reason ?
>
>
> Thanks,
> Divya
>

Re: Modifying multiple hbase tables

[Ted Yu]

bq. do I have to manually go through each table

I think so.

bq. Is there maybe a GUI manager capable of such task?

No. Please use Admin API or shell command.

Cheers

On Fri, Feb 26, 2016 at 2:15 PM, ginsul  wrote:

> Hi,
>
> Due to space issue we have decided to start using snappy compression on our
> data and cap the number of versions stored. However, our database at this
> point has over 50 tables.
>
> Is there a less painful way to modify compression and version attributes
> across multiple tables or do I have to manually go through each table? Is
> there maybe a GUI manager capable of such task?
>
> Having done that, is it possible to force compression of an already
> existing
> data?
>
> Cheers,
> Gin
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/Modifying-multiple-hbase-tables-tp4078128.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)

[Ted Yu]

I am interested in hearing about user experience with MOB feature as well.

In my opinion, this feature is a nice addition to branch-1.

On Wed, Feb 10, 2016 at 1:45 PM, Andrew Purtell <apurt...@apache.org> wrote:

> +user@
>
> Is there anyone using the MOB feature in trunk for anything who can comment
> on how well it's been working out? Intel folks maybe?
>
> On Thu, Jan 21, 2016 at 7:46 AM, Sean Busbey <bus...@apache.org> wrote:
>
> > The last time MOB on branch-1 came up, folks were concerned that it
> > wasn't stable enough in master yet. Is that still the case?
> >
> > Can we get a [DISCUSS] flagged thread to see what, if anything, folks
> > would like to see gate inclusion in branch-1?
> >
> > On Tue, Jan 19, 2016 at 7:31 PM, Jonathan Hsieh <j...@cloudera.com>
> wrote:
> > > +1 to 1.2 being feature complete corrently.  There has already been a
> > > release candidate and folks are burning down the blockers currently to
> > prep
> > > for the next RC.
> > >
> > > I like the idea of mob and sparkonhbase for 1.3.  I'm more comfortable
> > with
> > > sparkonhbase -- it is a new module and thus not as invasive.
> > >
> > > Jon.
> > >
> > > On Tue, Jan 19, 2016 at 4:55 PM, Andrew Purtell <
> > andrew.purt...@gmail.com>
> > > wrote:
> > >
> > >> Pretty sure Sean expressed 1.2 is feature complete and I'd support
> that.
> > >> Can we wait for 1.3 for MOB ? Can look at Spark connector then too.
> > >>
> > >> > On Jan 19, 2016, at 4:52 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > >> >
> > >> > Looks like 1.2.0 RC is in near future.
> > >> >
> > >> > I wonder if it is time to revive this thread (due to customer
> > interest).
> > >> >
> > >> > As far as I can tell, the Mob related tests have been passing in the
> > >> recent
> > >> > past.
> > >> >
> > >> > Thanks
> > >> >
> > >> >> On Wed, Oct 28, 2015 at 2:07 PM, Andrew Purtell <
> apurt...@apache.org
> > >
> > >> wrote:
> > >> >>
> > >> >> I haven't heard an user answer in the affirmative to wanting it.
> > >> >>
> > >> >> I'll volunteer to RM 1.3, whenever we need it. Premature to have
> that
> > >> >> discussion without 1.2 even out the door yet, though.
> > >> >>
> > >> >>
> > >> >>
> > >> >> On Wed, Oct 28, 2015 at 1:01 PM, Stephen Jiang <
> > syuanjiang...@gmail.com
> > >> >
> > >> >> wrote:
> > >> >>
> > >> >>> Actually, it is actively changing in master branch on MOB feature
> > made
> > >> me
> > >> >>> think about: if we ever want to port MOB feature to branch-1, now
> > is a
> > >> >> good
> > >> >>> time.  We can commit changes in both branches; otherwise, we
> > probably
> > >> >> would
> > >> >>> miss some commits when we port MOB to branch-1 in a late time.
> > >> >>>
> > >> >>> I am more thinking about 1.3 release (certainly not 1.2), which we
> > >> still
> > >> >>> have some time to stabilize and allow interesting party to play
> with
> > >> the
> > >> >>> feature and give feedback.
> > >> >>>
> > >> >>> Thanks
> > >> >>> Stephen
> > >> >>>
> > >> >>> PS. given the features we discussed in 2.0.0 in the last community
> > >> >> meeting,
> > >> >>> I think it would not release earlier than 1.3 :-), unless we
> > >> >> intentionally
> > >> >>> not find a release manager for 1.3.
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>> On Wed, Oct 28, 2015 at 9:09 AM, Sean Busbey <
> bus...@cloudera.com>
> > >> >>> wrote:
> > >> >>>
> > >> >>>> It's practically November. Matteo, are you up for a thread on
> > target
> > >> >>>> dates for 2.0.0 to start RCs?
> > >> >>>>
> > >> >>>>
> > >> >>>> On Wed, Oct 28, 2015 at 11:02 A

Re: Some problems in one accident on my production cluster

[Ted Yu]

bq. RegionStates: THIS SHOULD NOT HAPPEN: unexpected {
ad283942aff2bba6c0b94ff98a904d1a state=SPLITTING_NEW

Looks like the above wouldn't have happened if you are using 0.98.11+

See HBASE-12958

On Wed, Feb 24, 2016 at 6:39 PM, Heng Chen <heng.chen.1...@gmail.com> wrote:

> I pick up some logs in master.log about one region
> "ad283942aff2bba6c0b94ff98a904d1a"
>
>
> 2016-02-24 16:24:35,610 INFO  [AM.ZK.Worker-pool2-t3491]
> master.RegionStates: Transition null to {ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068}
> 2016-02-24 16:25:40,472 WARN
>  [MASTER_SERVER_OPERATIONS-dx-common-hmaster1-online:6-0]
> master.RegionStates: THIS SHOULD NOT HAPPEN: unexpected
> {ad283942aff2bba6c0b94ff98a904d1a state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068}
> 2016-02-24 16:34:24,769 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:39:24,768 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:44:24,768 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:45:37,749 DEBUG [FifoRpcScheduler.handler1-thread-10]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:49:24,769 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:54:24,768 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 16:59:24,768 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 17:04:24,769 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be032ba11ca6ba252ef={ab07d6fbcef39be032ba11ca6ba252ef
> state=SPLITTING_NEW...
> 2016-02-24 17:09:24,768 DEBUG
> [dx-common-hmaster1-online,6,1433937470611-BalancerChore]
> master.HMaster: Not running balancer because 2 region(s) in transition:
> {ad283942aff2bba6c0b94ff98a904d1a={ad283942aff2bba6c0b94ff98a904d1a
> state=SPLITTING_NEW, ts=1456302275610,
> server=dx-common-regionserver1-online,60020,1456302268068},
> ab07d6fbcef39be03

Re: Some problems in one accident on my production cluster

[Ted Yu]

bq. two regions were in transition

Can you pastebin related server logs w.r.t. these two regions so that we
can have more clue ?

For #2, please see http://hbase.apache.org/book.html#big.cluster.config

For #3, please see
http://hbase.apache.org/book.html#_running_multiple_workloads_on_a_single_cluster

On Wed, Feb 24, 2016 at 3:31 PM, Heng Chen  wrote:

> The story is I run one MR job on my production cluster (0.98.6),   it needs
> to scan one table during map procedure.
>
> Because of the heavy load from the job,  all my RS crashed due to OOM.
>
> After i restart all RS,  i found one problem.
>
> All regions were reopened on one RS,  and balancer could not run because of
> two regions were in transition.   The cluster got in stuck a long time
> until i restarted master.
>
> 1.  why this happened?
>
> 2.  If cluster has a lots of regions, after all RS crash,  how to restart
> the cluster.  If restart RS one by one, it means OOM may happen because one
> RS has to hold all regions and it will cost a long time.
>
> 3.  Is it possible to make each table with some requests quotas,  it means
> when one table is requested heavily, it has no impact to other tables on
> cluster.
>
>
> Thanks
>

Re: HBase Schema

[Ted Yu]

Switching to user@ mailing list.

It is hard to read the image even after zooming.

Can you provide a clearer image ?

On Mon, Feb 22, 2016 at 10:34 AM, SaCvP125 
wrote:

> Hi experts :)
>
> I'm trying to insert some date to hbase but I wanna to insert into a
> specific column (with a big number of columns). In image I'm show the
> schema
> that I want for the Hbase.
>
> 
>
> Is this possible to have in Hbase?
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/HBase-Schema-tp4077996.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

Re: Hbase testing utility

[Ted Yu]

Which hbase / hadoop release are you using ?

Can you give the complete stack trace ?

Cheers

On Mon, Feb 22, 2016 at 3:03 AM, Gaurav Agarwal 
wrote:

> > I am trying to hbase testing utility to start minicluster on my local but
> getting some exception
> > Java.Lang.unsatisfiedLinkError:
> org.apache.hadoop.ok.native.NativeIO$windows.access
> >
> > Please let me know what needs to be done
>

Re: Two questions about the maximum number of versions of a column family

[Ted Yu]

Thanks for sharing, Stephen.

bq. scan performance on the region servers needing to scan over all that
data you may not need

When number of versions is large, try to utilize Filters (where
appropriate) which implements:

  public Cell getNextCellHint(Cell currentKV) {

See MultiRowRangeFilter for example.


Please see hbase-shell/src/main/ruby/shell/commands/alter.rb for syntax on
how to alter table. When "hbase.online.schema.update.enable" is true, table
can stay online during the change.

Cheers

On Sun, Feb 21, 2016 at 8:20 AM, Stephen Durfey  wrote:

> Someone please correct me if I am wrong.
> I've looked into this recently due to some performance reasons with my
> tables in a production environment. Like the books says, I don't recommend
> keeping this many versions around unless you really need them. Telling
> HBase to keep around a very large number doesn't waste space, that's just a
> value in the table descriptor. So, I wouldn't worry about that. The
> problems are going to come in when you actually write out those versions.
> My tables currently have max_versions set and roughly 40% of the tables
> are due to historical versions. So, one table in particular is around 25
> TB. I don't have a need to keep this many versions, so I am working on
> changing the max versions to the default of 3 (some cells are hundreds or
> thousands of cells deep). The issue youll run into is scan performance on
> the region servers needing to scan over all that data you may not need (due
> to large store files). This could lead to increased scan time and
> potentially scanner timeouts, depending upon how large your batch size is
> set on the scan.
> I assume this has some performance impact on compactions, both minor and
> major, but I didn't investigate that, and potentially on the write path,
> but also not something I looked into.
> Changing the number of versions after the table has been created doesn't
> have a performance impact due to just being a metadata change. The table
> will need to be disabled, changed, and re-enabled again. If this is done
> through a script the table could be offline for a couple of seconds. The
> only concern around that are users of the table. If they have scheduled job
> runs that hit that table that would break if they try to read from it while
> the table is disabled. The only performance impact I can think of around
> this change would be major compaction of the table, but even that shouldn't
> be an issue.
>
>
> _
> From: Daniel 
> Sent: Sunday, February 21, 2016 9:22 AM
> Subject: Two questions about the maximum number of versions of a column
> family
> To: user 
>
>
> Hi, I have two questions about the maximum number of versions of a column
> family:
>
> (1) Is it OK to set a very large (>100,000) maximum number of versions for
> a column family?
>
> The reference guide says "It is not recommended setting the number of max
> versions to an exceedingly high level (e.g., hundreds or more) unless those
> old values are very dear to you because this will greatly increase
> StoreFile size." (Chapter 36.1)
>
> I'm new to the Hadoop ecosystem, and have no idea about the consequences
> of a very large StoreFile size.
>
> Furthermore, it is OK to set a large maximum number of versions but insert
> only a few versions? Does it waste space?
>
> (2) How much performance overhead does it cause to increase the maximum
> number of versions of a column family after enormous (e.g. billions) rows
> have been inserted?
>
> Regards,
>
> Daniel
>
>
>
>

Re: Regarding Hbase Thrift Authentication

[Ted Yu]

Which version of hbase are you using ?

Have you taken a look at:

http://hbase.apache.org/book.html#security.client.thrift

Cheers

On Fri, Feb 19, 2016 at 3:11 AM, prabhu Mahendran 
wrote:

> Hi,
>
> I have try to access Hbase Thrift authentication in Secured Cluster by this
> referal link(https://issues.apache.org/jira/browse/HBASE-11960).Now i got
> Following Exception occurs.
>
>
> log4j:WARN No appenders could be found for logger
> (org.apache.thrift.transport.TSaslTransport).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> Exception in thread "main" java.security.PrivilegedActionException:
> org.apache.thrift.transport.TTransportException: GSS initiate failed
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at thriftclienttest.DemoClient.main(DemoClient.java:84)
> Caused by: org.apache.thrift.transport.TTransportException: GSS initiate
> failed
> at
>
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
> at
>
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at thriftclienttest.DemoClient.run(DemoClient.java:134)
> at thriftclienttest.DemoClient.access$000(DemoClient.java:56)
> at thriftclienttest.DemoClient$1.run(DemoClient.java:88)
> at thriftclienttest.DemoClient$1.run(DemoClient.java:85)
> ... 3 more
> Java Result: 1
>
>
> Please anyone help me to solve this.
>
>
> Best,
> Mahendran V.M
>

Re: Store Large files on HBase/HDFS

[Ted Yu]

For #1, please take a look
at 
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java

e.g. the following methods:

  public DFSInputStream open(String src) throws IOException {

  public HdfsDataOutputStream append(final String src, final int buffersize,

  EnumSet flag, final Progressable progress,

  final FileSystem.Statistics statistics) throws IOException {


Cheers

On Wed, Feb 17, 2016 at 3:40 PM, Arun Patel  wrote:

> I would like to store large documents (over 100 MB) on HDFS and insert
> metadata in HBase.
>
> 1) Users will use HBase REST API for PUT and GET requests for storing and
> retrieving documents. In this case, how to PUT and GET documents to/from
> HDFS?What are the recommended ways for storing and accessing document
> to/from HDFS that provides optimum performance?
>
> Can you please share any sample code?  or a Github project?
>
> 2)  What are the performance issues I need to know?
>
> Regards,
> Arun
>

Re: Error : starting spark-shell with phoenix client jar

[Ted Yu]

If you cannot wait for the next HDP release with fix for PHOENIX-2608,
please consider rebuilding Phoenix with patch from PHOENIX-2608 applied.

Cheers

On Thu, Feb 18, 2016 at 6:19 PM, Divya Gehlot <divya.htco...@gmail.com>
wrote:

> Thanks Ted for getting me to the root cause of the issue .
> I am using hortonworks distribution  HDP2.3.4.. How can I upgrade ?
> Could you provide me the steps ?
>
>
>
>
>
> On 18 February 2016 at 20:39, Ted Yu <yuzhih...@gmail.com> wrote:
>
>> This was likely caused by version conflict of jackson dependencies between
>> Spark and Phoenix.
>> Phoenix uses 1.8.8 while Spark uses 1.9.13
>>
>> One solution is to upgrade the jackson version in Phoenix.
>> See PHOENIX-2608
>>
>>
>> On Thu, Feb 18, 2016 at 12:31 AM, Divya Gehlot <divya.htco...@gmail.com>
>> wrote:
>>
>> > Hi,
>> > I am getting following error while starting spark shell with phoenix
>> > clients
>> > spark-shell  --jars
>> > /usr/hdp/current/phoenix-client/phoenix-4.4.0.2.3.4.0-3485-client.jar
>> > --driver-class-path
>> > /usr/hdp/current/phoenix-client/phoenix-4.4.0.2.3.4.0-3485-client.jar
>> > --master yarn-client
>> >
>> > StackTrace :
>> >
>> > >  INFO TimelineClientImpl: Timeline service address:
>> > >
>> >
>> http://ip-xxx-xx-xx-xxx.ap-southeast-1.compute.internal:8188/ws/v1/timeline/
>> > > java.lang.NoSuchMethodError:
>> > >
>> >
>> org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
>> > > at
>> > >
>> >
>> org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
>> > > at
>> > >
>> >
>> org.apache.hadoop.yarn.util.timeline.TimelineUtils.(TimelineUtils.java:50)
>> > > at
>> > >
>> >
>> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:172)
>> > > at
>> > >
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> > > at
>> > >
>> org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:108)
>> > > at
>> > >
>> >
>> org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
>> > > at
>> > >
>> >
>> org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
>> > > at
>> org.apache.spark.SparkContext.(SparkContext.scala:523)
>> > > at
>> > >
>> >
>> org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017)
>> > > at $iwC$$iwC.(:9)
>> > > at $iwC.(:18)
>> > > at (:20)
>> > > at .(:24)
>> > > at .()
>> > > at .(:7)
>> > > at .()
>> > > at $print()
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > at
>> > >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > > at
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > at java.lang.reflect.Method.invoke(Method.java:606)
>> > > at
>> > >
>> >
>> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
>> > > at
>> > >
>> >
>> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
>> > > at
>> > > org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
>> > > at
>> > org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
>> > > at
>> > org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
>> > > at
>> > >
>> org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
>> > > at
>> > >
>> >
>> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
>> > > at
>> org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
>> > > at

Re: Error : starting spark-shell with phoenix client jar

[Ted Yu]

This was likely caused by version conflict of jackson dependencies between
Spark and Phoenix.
Phoenix uses 1.8.8 while Spark uses 1.9.13

One solution is to upgrade the jackson version in Phoenix.
See PHOENIX-2608


On Thu, Feb 18, 2016 at 12:31 AM, Divya Gehlot 
wrote:

> Hi,
> I am getting following error while starting spark shell with phoenix
> clients
> spark-shell  --jars
> /usr/hdp/current/phoenix-client/phoenix-4.4.0.2.3.4.0-3485-client.jar
> --driver-class-path
> /usr/hdp/current/phoenix-client/phoenix-4.4.0.2.3.4.0-3485-client.jar
> --master yarn-client
>
> StackTrace :
>
> >  INFO TimelineClientImpl: Timeline service address:
> >
> http://ip-xxx-xx-xx-xxx.ap-southeast-1.compute.internal:8188/ws/v1/timeline/
> > java.lang.NoSuchMethodError:
> >
> org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
> > at
> >
> org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
> > at
> >
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.(TimelineUtils.java:50)
> > at
> >
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:172)
> > at
> > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> > at
> > org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:108)
> > at
> >
> org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
> > at
> >
> org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
> > at org.apache.spark.SparkContext.(SparkContext.scala:523)
> > at
> >
> org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017)
> > at $iwC$$iwC.(:9)
> > at $iwC.(:18)
> > at (:20)
> > at .(:24)
> > at .()
> > at .(:7)
> > at .()
> > at $print()
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> >
> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
> > at
> >
> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1340)
> > at
> > org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
> > at
> org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
> > at
> org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
> > at
> > org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
> > at
> >
> org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
> > at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
> > at
> >
> org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:125)
> > at
> >
> org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)
> > at
> > org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)
> > at
> >
> org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)
> > at
> > org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
> > at
> >
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
> > at
> >
> org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)
> > at
> org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
> > at
> >
> org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)
> > at
> > org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)
> > at
> >
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
> > at
> >
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> > at
> >
> org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
> > at
> >
> scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
> > at org.apache.spark.repl.SparkILoop.org
> > $apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
> > at
> org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
> > at org.apache.spark.repl.Main$.main(Main.scala:31)
> > at 

Re: disable major compaction per table

[Ted Yu]

Please see the following for description on data block encoding :
http://hbase.apache.org/book.html#compression

If you have further questions on data block encoding, please start another
thread.

On Thu, Feb 18, 2016 at 3:16 AM, Shushant Arora <shushantaror...@gmail.com>
wrote:

> Thanks!
>
> does hbase compress repeated values in keys and  columns :location say
> (ASIA). will that be repeated with each key or hbase snappy compression
> will handle that.
>
> same applies for repeated values of a column?
>
> Thanks!
>
> On Wed, Feb 17, 2016 at 7:14 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > bq. hbase.hregion.majorcompaction = 0 per table/column family
> >
> > I searched code base but didn't find relevant test case for the above.
> > Mind giving me some pointer ?
> >
> > Thanks
> >
> > On Tue, Feb 16, 2016 at 5:38 PM, Vladimir Rodionov <
> vladrodio...@gmail.com
> > >
> > wrote:
> >
> > > 1.does major compaction in hbase runs per table basis.
> > >
> > > Per Region
> > >
> > > 2.By default every 24 hours?
> > >
> > > In older versions - yes. Current  (1.x+) - 7 days
> > >
> > > 3.Can I disable automatic major compaction for few tables while keep it
> > > enable for rest of tables?
> > >
> > > yes, you can. You can set
> > >
> > > hbase.hregion.majorcompaction = 0 per table/column family
> > >
> > > 4.Does hbase put ,get and delete are blocked while major compaction and
> > are
> > > working in minor compaction?
> > >
> > > No, they are not.
> > >
> > > -Vlad
> > >
> > > On Tue, Feb 16, 2016 at 4:51 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > >
> > > > For #2, see http://hbase.apache.org/book.html#managed.compactions
> > > >
> > > > For #3, I don't think so.
> > > >
> > > > On Tue, Feb 16, 2016 at 4:46 PM, Shushant Arora <
> > > shushantaror...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi
> > > > >
> > > > > 1.does major compaction in hbase runs per table basis.
> > > > > 2.By default every 24 hours?
> > > > > 3.Can I disable automatic major compaction for few tables while
> keep
> > it
> > > > > enable for rest of tables?
> > > > >
> > > > > 4.Does hbase put ,get and delete are blocked while major compaction
> > and
> > > > are
> > > > > working in minor compaction?
> > > > >
> > > > > Thanks
> > > > >
> > > >
> > >
> >
>

Re: HBase : Transaction queries

[Ted Yu]

There is also Omid:
https://github.com/yahoo/omid/wiki

On Wed, Feb 17, 2016 at 7:07 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:

> Hi Divya,
>
> HBase doesn't support transactions like rollback and commits. It has row
> level atomicity only.
>
> Some frameworks have been build on top of HBase to provide this kind of
> features but with a big impact on the performances.
>
> You can take a look at (And I might be missing some):
> - Themis
> - Tephra
> - Phoenix (That is using one of the 2 above)
> - SpliceMachine (http://www.splicemachine.com/product/how-it-works/)
> - Trafodion (I think they changed the name...)
>
> HTH,
>
> JMS
>
> 2016-02-17 22:03 GMT-05:00 Divya Gehlot :
>
> > Hi,
> >
> > I am new bee to Hbase and recently  went through documentation and
> videos
> > on HBase
> > Now got curious about , How Hbase handles transactions such rollback and
> > commit.
> > Would really appreciate the inputs and guidance .
> >
> >
> > Thanks,
> > Divya
> >
>

Re: disable major compaction per table

[Ted Yu]

bq. hbase.hregion.majorcompaction = 0 per table/column family

I searched code base but didn't find relevant test case for the above.
Mind giving me some pointer ?

Thanks

On Tue, Feb 16, 2016 at 5:38 PM, Vladimir Rodionov <vladrodio...@gmail.com>
wrote:

> 1.does major compaction in hbase runs per table basis.
>
> Per Region
>
> 2.By default every 24 hours?
>
> In older versions - yes. Current  (1.x+) - 7 days
>
> 3.Can I disable automatic major compaction for few tables while keep it
> enable for rest of tables?
>
> yes, you can. You can set
>
> hbase.hregion.majorcompaction = 0 per table/column family
>
> 4.Does hbase put ,get and delete are blocked while major compaction and are
> working in minor compaction?
>
> No, they are not.
>
> -Vlad
>
> On Tue, Feb 16, 2016 at 4:51 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > For #2, see http://hbase.apache.org/book.html#managed.compactions
> >
> > For #3, I don't think so.
> >
> > On Tue, Feb 16, 2016 at 4:46 PM, Shushant Arora <
> shushantaror...@gmail.com
> > >
> > wrote:
> >
> > > Hi
> > >
> > > 1.does major compaction in hbase runs per table basis.
> > > 2.By default every 24 hours?
> > > 3.Can I disable automatic major compaction for few tables while keep it
> > > enable for rest of tables?
> > >
> > > 4.Does hbase put ,get and delete are blocked while major compaction and
> > are
> > > working in minor compaction?
> > >
> > > Thanks
> > >
> >
>

Re: disable major compaction per table

[Ted Yu]

For #2, see http://hbase.apache.org/book.html#managed.compactions

For #3, I don't think so.

On Tue, Feb 16, 2016 at 4:46 PM, Shushant Arora 
wrote:

> Hi
>
> 1.does major compaction in hbase runs per table basis.
> 2.By default every 24 hours?
> 3.Can I disable automatic major compaction for few tables while keep it
> enable for rest of tables?
>
> 4.Does hbase put ,get and delete are blocked while major compaction and are
> working in minor compaction?
>
> Thanks
>