Re: [ANNOUNCE] New HBase committer Yutong Xiao(肖禹同)

Congratulations Yutong, as a long time user and fan of HBase, you are appreciated! -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you

Re: [ANNOUNCE] New HBase committer Yutong Xiao(肖禹同)

Congratulations Yutong, as a long time user and fan of HBase, you are appreciated! -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you

Re: Hbase with AWS EMR

will likely want to just copy the relevant config dirs ( HADOOP_CONF_DIR, YARN_CONF_DIR, HBASE_CONF_DIR or wherever hbase_site.xml is, etc.) to wherever you are establishing your client from, and set them as env variables/get them into your classpath before starting the client -- Iain Wright This email

Re: Rolling upgrade with coprocessors

@Albert, Just a heads up -- in my experience, when replacing a coprocessor jar stored in HDFS (using an identical jar name), the new jar will not be loaded (even if you disable/enable the table). Tracked here: https://issues.apache.org/jira/browse/HBASE-9046 -- Iain Wright This email message

Re: Hbase Row key lock

HBase uses MVCC to deal with concurrent writes, this is a good explanation: https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged

Re: Cache of region boundaries are out of date.

you'll likely want to hash out this problem and confirm it before forcing the skip Be sure to drop them the version info for your stack, frequency of the problem, if you can reproduce consistently, when it started, etc. -- Iain Wright This email message is confidential, intended only for the recip

Re: Does Replication Affect Write Performance On The Main Cluster

Wish I had an answer for you.. This is relevant to our interests as well as I planned to run spark jobs on our slave cluster. If we don't hear conclusively on the list I will do some benchmarking in a dev environment and attempt to reproduce Best, Iain Sent from my iPhone > On Jul 15, 2016,

Re: HBase as a Multi-Model Data Store

Neat, thanks for sharing -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate

Re: Put performance test

You could use YCSB and a custom workload (i don't see a predefined workload for 100% puts without reads) https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads HBase also has a utility for running some evaluations via MR or a thread based client: $ ./hbase

Re: Opinions wanted: new site skin

Agreed, most recent version looks great on iphone 6s+: http://i.imgur.com/k4enJMB.png http://i.imgur.com/MYprWgQ.png Maybe Anil was referring to the first proposed update: http://i.imgur.com/jNmgjOQ.png -- Iain Wright This email message is confidential, intended only for the recipient(s

Re: [ANNOUNCE] YCSB 0.4.0 Release

Very cool. On Oct 22, 2015 6:53 PM, "Robert J. Moore" wrote: > On behalf of the development community, I am pleased to announce the > release of YCSB 0.4.0. > > Highlights: > > * Default measurement changed from histogram to hdrhistogram. > * Users who want

Re: Ramifications of minimizing use of .tmp directories / renames in HBase when using S3 as backing store

Hi Anthony, Just curious, you mention your access pattern is mostly reads. Is it random reads, M/R jobs over a portion of the dataset, or other? Best, -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information

Re: Ramifications of minimizing use of .tmp directories / renames in HBase when using S3 as backing store

I see - If it was only M/R, Hive, or a similar reporting/analytics workload without the low latency get's/read requirement, I was going to suggest writing to S3 directly and using spark+hivecontext. Cheers, -- Iain Wright This email message is confidential, intended only for the recipient(s

Re: How to know the root reason to cause RegionServer OOM?

What log is this seen in? Can you paste the log line? Do you mean /var/log/messages? On May 12, 2015 7:44 PM, David chen c77...@163.com wrote: A RegionServer was killed because OutOfMemory(OOM), although the process killed can be seen in the Linux message log, but i still have two following

Re: What companies are using HBase to serve a customer-facing product?

: - https://www.youtube.com/user/HadoopSummit - http://hbasecon.com/archive.html All the best, -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you

Re: Upgrading a coprocessor

Hi Hayden, We ran into the same thing ended up going with a rudimentary cp deploy script for appending epoch to the cp name, placing on hdfs, and disabling/modifying hbase table/enabling Heres the issue for this: https://issues.apache.org/jira/browse/HBASE-9046 - -- Iain Wright This email

Re: Trouble using copyTable to seed existing table to replication peer

allowed nodemanagers to localize resources from HDFS. Rookie mistake -- hopefully this thread saves someone else a couple of hours someday -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from

Re: Trouble using copyTable to seed existing table to replication peer

Anyone have any advice on this? I'm going to cross post to the hadoop users group since this seems to be a YARN related issue Best,

Re: Trouble using copyTable to seed existing table to replication peer

...@apache.org wrote: Do any map-reduce jobs work on this cluster ? On Fri, Sep 26, 2014 at 5:13 PM, iain wright iainw...@gmail.com wrote: Forgot to include the actual copy table command: [hbase@master2 bin]$ ./hbase org.apache.hadoop.hbase.mapreduce.CopyTable --starttime=1409128964 --peer.adr

Trouble using copyTable to seed existing table to replication peer

org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1411771844660_0004, with delay of 10800 seconds Thank you in advance for any advice -- Iain Wright This email message is confidential, intended only

Re: Trouble using copyTable to seed existing table to replication peer

is working well btw for new data. Thanks, -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose

Re: Shout-out for Misty

As an admin constantly referring to these docs, Thank you! -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do

Re: Are coprocessor really bad?

Pretty sure it depends on your implementation. Its powerful but you have the ability to shoot yourself in the foot. One example is, we tried watching post put on a table to increment values in another table. This ended up causing deadlocks when regionservers tried to pull an RPC to write to each

Re: Some regions keep using an older version of coprocessor.

Filed bug report for confirmation: https://issues.apache.org/jira/browse/HBASE-9046 Happy to supply any additional info as requested Best, iain -- Iain Wright Cell: (562) 852-5916 http://www.labctsi.org/ This email message is confidential, intended only for the recipient(s) named above

Re: Some regions keep using an older version of coprocessor.

deploy script which appends an epoch to their jar on each deployment so there is no cache involved. It's hacky, but yeah, using a different co-processor name and uploading that to HDFS/re-enabling table should solve your problem. pasted the script below for your reference. Cheers, iain wright

Re: how many severs in a hbase cluster

Hi Mohammad, I am curious why you chose not to put the third ZK on the NN+JT? I was planning on doing that on a new cluster and want to confirm it would be okay. -- Iain Wright Cell: (562) 852-5916 http://www.labctsi.org/ This email message is confidential, intended only for the recipient(s

flume+hbase+zookeeper, can't hang on to zookeeper session

)] Session establishment complete on server hbasemaster0.hadoop.domain.com/10.10.11.30:2181, sessionid = 0x138a0620045bdbe, negotiated timeout = 6000 Thanks for the continued help from the community on getting us going w/flume Cheers, -- Iain Wright Cell: (562) 852-5916 http://www.labctsi.org