hbase + mapreduce

Hello: I'm working in a proyect, and i'm using hbase for storage the data, y have this method that work great but without the performance i'm looking for, so i want is to make the same but using mapreduce. public ArrayList findZ(String z) throws IOException { ArrayList rows = new Arra

hbase + mapreduce

Hello: I'm working in a proyect, and i'm using hbase for storage the data, y have this method that work great but without the performance i'm looking for, so i want is to make the same but using mapreduce. public ArrayList findZ(String z) throws IOException { ArrayList rows = new Arra

Hbase MapReduce

Dear all, I wanted to ask a question.. Do Hbase Aggregate Functions such as rowcount, getMax, get Average use MapReduce to execute those functions? Thanks :D

Re: hbase + mapreduce

Here you have several examples: http://hbase.apache.org/book/mapreduce.example.html http://sujee.net/tech/articles/hadoop/hbase-map-reduce-freq-counter/ http://bigdataprocessing.wordpress.com/2012/07/27/hadoop-hbase-mapreduce-examples/ http://stackoverflow.com/questions/12215313/load-data-into

HBASE mapReduce stoppage

We are bulk loading 1 billion rows into hbase. The 1 billion file was split into 20 files of ~22.5GB. Ingesting the file to hdfs took ~2min. Ingesting the first file to hbase took ~3 hours. The next took ~5hours, then it is increasing. By the sixth or seventh file the ingestion just stops (mapRedu

HBase & MapReduce & Zookeeper

Hi folks, i'm running in an interesting issue: we have a zookeeper cluster running on 3 servers we run mapreduce jobs using org.apache.hadoop.conf.Configuration to pass parameters to our mappers the string based (key/value) approach is imho not the most elegant way, i would prefer to however p

Re: Hbase MapReduce

Regards, Dalia. You have to use MapReduce for that. In the HBase in Practice´s book, there are lot of great examples for this. On 11/24/2012 12:15 PM, Dalia Sobhy wrote: Dear all, I wanted to ask a question.. Do Hbase Aggregate Functions such as rowcount, getMax, get Average use MapReduce to ex

Re: Hbase MapReduce

Hi, but you do not need to us M/R. You could also use coprocessors. See this site: https://blogs.apache.org/hbase/entry/coprocessor_introduction -> in the section "Endpoints" An aggregation coprocessor ships with hbase that should match your requirements. You just need to load it and eventuall

Re: Hbase MapReduce

Do you think it would be a good idea to temper the use of CoProcessors? This kind of reminds me of when people first started using stored procedures... Sent from a remote device. Please excuse any typos... Mike Segel On Nov 24, 2012, at 11:46 AM, tom wrote: > Hi, but you do not need to us M/

RE: Hbase MapReduce

ail.com > Subject: Re: Hbase MapReduce > Date: Sat, 24 Nov 2012 12:05:45 -0600 > To: user@hbase.apache.org > > Do you think it would be a good idea to temper the use of CoProcessors? > > This kind of reminds me of when people first started using stored > procedures... > &

RE: Hbase MapReduce

e.org" , Date: 11/24/2012 01:33 PM Subject: RE: Hbase MapReduce It 's weird that hbase aggregate functions don't use MapReduce, this means that the performance will be very poor. Is it a must to use coprocessors? Is there a much easier way to improve the functions'

Re: Hbase MapReduce

m: michael_se...@hotmail.com Subject: Re: Hbase MapReduce Date: Sat, 24 Nov 2012 12:05:45 -0600 To: user@hbase.apache.org Do you think it would be a good idea to temper the use of CoProcessors? This kind of reminds me of when people first started using stored procedures... Sent from a remote device

HBase MapReduce problem

Dear all, I have a simple HBase MapReduce application and try to run it on a 12-node cluster using this command: HADOOP_CLASSPATH=`bin/hbase classpath` ~/hadoop-1.1.2/bin/hadoop jar .jar org.test.WordCount HBase version is 0.95.0. But i got this error: java.lang.RuntimeException

Re: HBASE mapReduce stoppage

Hi Dilon, Sounds like your table was not pre-split from the behavior that you are describing, but when you say that you are bulk loading the data using MR is this a MR job that does Put(s) into HBase or just generating HFiles (if using importtsv you have both options) that are later on bulk loaded

Re: HBase & MapReduce & Zookeeper

Hi there- re: "that we have to reuse the Configuration object" You are probably referring to this... http://hbase.apache.org/book.html#client.connections ... yes, that is general guidance on client connection.. re: "do i have to create a pool of Configuration objects, to share them synchron

Re: HBase & MapReduce & Zookeeper

Hi Doug, thanks a lot for reply, it's clear, that there is a parameter for maxClientCnxns, which is 10 by default of course i could increase it to s.th. big. but like i said, the old connections are still there, and i cannot imagine, that this is a correct behaviour, to let them open (establish

Re: HBase & MapReduce & Zookeeper

Configuration is not Comparable. Its instance identity that is used comparing Configurations down in the guts of HConnectionManager in 0.90.x hbase so even if you reuse a Configuration and tweak it per job, as far as HCM is concerned its the 'same'. Are you seeing otherwise? St.Ack On Tue, Jul

Re: HBase & MapReduce & Zookeeper

Hi St.Ack, thanks for your reply but funally i miss the point, what would be the options to solve our issue? andre

Re: HBase & MapReduce & Zookeeper

Can you reuse Configuration instances though the "configuration" changes? Else in your Mapper#cleanup, call HTable.close() then try HConnectionManager.deleteConnection(table.getConfiguration()) after close (could be issue with executors used by multi* operations not completing before delete of con

Re: HBase & MapReduce & Zookeeper

Hi Stack, just to make clear, actually the connections to the zookeeper being kept are not on our mappers (tasktrackers) but on the client, which schedules the MR job i think, the mappers are just fine, as they are andre Stack wrote: Can you reuse Configuration instances though the "configu

Re: HBase & MapReduce & Zookeeper

Then similarly, can you do the deleteConnection above in your client or reuse the Configuration client-side that you use setting up the job? St.Ack On Wed, Jul 20, 2011 at 12:13 AM, Andre Reiter wrote: > Hi Stack, > > just to make clear, actually the connections to the zookeeper being kept are >

Re: HBase & MapReduce & Zookeeper

Hi St.Ack, actually calling HConnectionManager.deleteConnection(conf, true); does not close the connection to the zookeeper i still can see the connection established... andre Stack wrote: Then similarly, can you do the deleteConnection above in your client or reuse the Configuration client

Re: HBase & MapReduce & Zookeeper

Andre: So you didn't see the following in client log (HConnectionManager line 1067) ? LOG.info("Closed zookeeper sessionid=0x" + Long.toHexString(this.zooKeeper.getZooKeeper().getSessionId())); HConnectionManager.deleteConnection(conf, true) is supposed to close zk connection in 0

Re: HBase & MapReduce & Zookeeper

unfortunatelly there was no such LOG entry... :-( our versions: hadoop-0.20.2-CDH3B4 hbase-0.90.1-CDH3B4 zookeeper-3.3.2-CDH3B4 either the map HConnectionManager.HBASE_INSTANCES does not contain the connection for the current config, or HConnectionImplementation.zooKeeper is null but the zooke

Re: HBase & MapReduce & Zookeeper

This seems to be cdh related. >> either the map HConnectionManager.HBASE_INSTANCES does not contain the connection for the current config You need to pass the same conf object. In trunk, I added the following: public static void deleteStaleConnection(HConnection connection) { See http://zhihon

Re: HBase & MapReduce & Zookeeper

Hi Ted, thanks for the reply, at the moment i'm hust wondering, why the client creates a zookeeper connection at all all the client has to do, is to schedule a MR job, which is done by connecting to the jobtracker and to provide all the needed stuff, config, some extra resources in the distrib

Re: HBase & MapReduce & Zookeeper

My guess is that it needs to ask the master for the regions so it can make the splits used by mapper tasks (to find master, needs to ask zk, etc.). Check it out yourself under the mapreduce package? St.Ack On Wed, Jul 20, 2011 at 3:06 PM, Andre Reiter wrote: > Hi Ted, > thanks for the reply, >

Re: HBase & MapReduce & Zookeeper

this issue is still not resolved... unfortunatelly calling HConnectionManager.deleteConnection(conf, true); after the MR job is finished, does not close the connection to the zookeeper we have 3 zookeeper nodes by default there is a limit of 10 connections allowed from a single client so after

Re: HBase & MapReduce & Zookeeper

Try getting the ZooKeeperWatcher from the connection on your way out and explicitly shutdown the zk connection (see TestZooKeeper unit test for example). St.Ack On Thu, Jul 28, 2011 at 6:01 AM, Andre Reiter wrote: > this issue is still not resolved... > > unfortunatelly calling HConnectionManager

Re: HBase & MapReduce & Zookeeper

10 connection maximum is too low. It has been recommended to go up to as many as 2000 connections in the list. This doesn't fix your problem but is something you should probably have in your configuration. ~Jeff On 7/28/2011 10:00 AM, Stack wrote: Try getting the ZooKeeperWatcher from the c

Re: HBase & MapReduce & Zookeeper

To: user@hbase.apache.org Sent: Thu, July 28, 2011 12:10:16 PM Subject: Re: HBase & MapReduce & Zookeeper 10 connection maximum is too low. It has been recommended to go up to as many as 2000 connections in the list. This doesn't fix your problem but is something you should pro

Re: HBase & MapReduce & Zookeeper

i guess, i know the reason, why HConnectionManager.deleteConnection(conf, true); does not work for me in the MR job im using TableInputFormat, if you have a look at the source code in the method public void setConf(Configuration configuration) there is a line creating the HTable like this : .

Re: HBase & MapReduce & Zookeeper

Yes, that's the connection leak. Use deleteAllConnections(true), and it will close all open connections. - Ruben From: Andre Reiter To: user@hbase.apache.org Sent: Thu, July 28, 2011 4:55:52 PM Subject: Re: HBase & MapReduce & Zookeeper i gues

Re: HBase & MapReduce & Zookeeper

s. > > - Ruben > > > > > > From: Andre Reiter > To: user@hbase.apache.org > Sent: Thu, July 28, 2011 4:55:52 PM > Subject: Re: HBase & MapReduce & Zookeeper > > i guess, i know the reason, why  HConnectionManager.deleteConnection(conf, > true); does

Re: HBase & MapReduce & Zookeeper

hi Ruben, St.Ack thanks a lot for your help! finally, the problem seems to be solved by an pretty sick workaround i did it like Bryan Keller described in this issue: https://issues.apache.org/jira/browse/HBASE-3792 @Ruben: thanks for the urls to that issues cheers andre

Hbase Mapreduce jobs Dashboard

Hi All, When I run Hadoop mapreduce jobs, the job statistics and status is displayed in jobtracker/task tracker. But when I use HBase mapreduce it doesn't. Is there any hbase mapreduce dashboard available or am I missing something? Thanks & Regards Jimson K James "T

Re: HBase & MapReduce & Zookeeper

I had the same issue. The problem for me turned out to be that the hbase.zookeeper.quorum was not set in hbase-site.xml in the server that submitted the mapreduce job. Ironically, this is also the same server that was running hbase master. This defaulted to 127.0.0.1 which was where the task

Re: HBase MapReduce problem

Why do you use 0.95 which was a developer release ? See http://hbase.apache.org/book.html#d243e520 Cheers On Fri, Jan 24, 2014 at 8:40 AM, daidong wrote: > Dear all, > > I have a simple HBase MapReduce application and try to run it on a > 12-node cluster using

Re: HBase MapReduce problem

#d243e520 > > Cheers > > > On Fri, Jan 24, 2014 at 8:40 AM, daidong wrote: > > > Dear all, > > > > I have a simple HBase MapReduce application and try to run it on a > > 12-node cluster using this command: > > > > HADOOP_CLASSPATH=`

Re: HBase MapReduce problem

Hi, I am also getting the same problem. I am using 0.96 version. May I know how to fix it? Thanks

Re: HBase MapReduce problem

Hi Ted, I am using HBase 0.96 version. But I am also getting the below error message 14/02/03 10:18:32 ERROR mapreduce.TableOutputFormat: org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for after 35 tries. Exception in thread "main" java.lang.RuntimeExcepti

Re: HBase MapReduce problem

Murali: Are you using 0.96.1.1 ? Can you show us the command line you used ? Meanwhile I assume the HBase cluster is functional - you can use shell to insert data. Cheers On Mon, Feb 3, 2014 at 8:33 PM, Murali wrote: > Hi Ted, > >I am using HBase 0.96 version. But I am also getting the be

Re: HBase MapReduce problem

Hi Ted Thanks for your reply. I am using HBase version 0.96.0. I can insert a record using shell command. I am running the below command to run my MapReduce job. It is a word count example. Reading a text file from hdfs file path and insert the counts to HBase table. "hadoop jar hb.jar com.h

Re: HBase MapReduce problem

See the sample command in http://hbase.apache.org/book.html#trouble.mapreduce : HADOOP_CLASSPATH=`hbase classpath` hadoop jar On Mon, Feb 3, 2014 at 9:33 PM, Murali wrote: > Hi Ted > > Thanks for your reply. I am using HBase version 0.96.0. I can insert a > record using shell command. I am

Re: HBase MapReduce problem

Hi Ted, I am trying your solution. But I got the same error message. Thanks

Re: HBase MapReduce problem

Did you create the table prior to launching your program ? If so, when you scan hbase:meta table, do you see row(s) for it ? Cheers On Feb 4, 2014, at 12:53 AM, Murali wrote: > Hi Ted, > > I am trying your solution. But I got the same error message. > > Thanks > > > > > >

HBase MapReduce in Kerberized cluster

I'm attempting to write a Crunch pipeline to read various rows from a table in HBase and then do processing on these results. I am doing this from a cluster deployed using CDH 5.3.2 running Kerberos and YARN. I was hoping to get an answer on what is considered the best approach to authenticate to

Re: Hbase Mapreduce jobs Dashboard

cker/task tracker. But when I use HBase mapreduce it > doesn't. > > Is there any hbase mapreduce dashboard available or am I missing > something? > > > > > > Thanks & Regards > > Jimson K James > > > > "The Quieter You Become The More You A

RE: Hbase Mapreduce jobs Dashboard

hbase.apache.org Subject: Re: Hbase Mapreduce jobs Dashboard HBase doesn't have it's own MapReduce system, it uses Hadoop's. How are you launching your jobs? On Mon, Sep 12, 2011 at 2:32 AM, Jimson K. James wrote: > Hi All, > > > > When I run Hadoop mapreduce jobs, t

Re: Hbase Mapreduce jobs Dashboard

> Sent: Monday, September 12, 2011 4:47 PM > To: user@hbase.apache.org > Subject: Re: Hbase Mapreduce jobs Dashboard > > HBase doesn't have it's own MapReduce system, it uses Hadoop's. How > are you launching your jobs? > > On Mon, Sep 12, 2011 at 2:32 AM, Jim

Re: Query regarding HBase Mapreduce

? 2) I am not sure what you mean by "HBase mapreduce on small files". If you are using MapReduce with HBase as a source, you are not dealing with files directly. If you are using HBase as a sink, then the lots of small files is a problem which is orthogonal to the use of HBase. I don

Re: Query regarding HBase Mapreduce

Hi amit I am starting with Hbase and MR so my opinion ismore about what I read than real world. However the documentation says Hadoop will deal better with a set of large files than a lot of small ones. regards amit bohra writes:

Re: Query regarding HBase Mapreduce

handle billions of files. 2012/10/25 amit bohra > Hi, > > We are working on processing of lot of small files. For processing them we > are using HBase Mapreduce as of now. Currently we are working with files in > the range for around few millions, but over the period of time it wo

something wrong with hbase mapreduce

Hi,i found a problem in my hbase mapreduce case. when first running mapreduce TableMapReduceUtil runs ok. and i use hbase shell to delete some data from the table that mapreduce one . then ran mapreduce to insert some new data. no thing data changed, mapreduce didnt work. after that i drop

A question about HBase MapReduce

Hello! I've read Lars George's blog http://www.larsgeorge.com/2009/05/hbase-mapreduce-101-part-i.html where at the end of the article, he mentioned "In the next post I will show you how to import data from a raw data file into a HBase table and how you eventually process the da

input split for hbase mapreduce

Hi If I am running mapreduce on hbase tables what will be the input to mapper function Thanks

Re: HBase MapReduce in Kerberized cluster

bq. it has been moved to be a part of the hbase-server package I searched (current) 0.98 and branch-1 where I found: ./hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java FYI On Wed, May 13, 2015 at 11:45 AM, Edward C. Skoviak < edward.skov...@gmail.com> wrote: > I'

Re: HBase MapReduce in Kerberized cluster

> I searched (current) 0.98 and branch-1 where I found: ./hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java Looking at both 0.98[1] and 0.98.6[2] on github I see TokenUtil as part of hbase-server. Is it necessary for us to add this call to TokenUtil to all MR jobs

Re: HBase MapReduce in Kerberized cluster

Please take a look at HBASE-12493 User class should provide a way to re-use existing token which went into 0.98.9 FYI On Thu, May 14, 2015 at 8:37 AM, Edward C. Skoviak wrote: > > I searched (current) 0.98 and branch-1 where I found: > > ./hbase-client/src/main/java/org/apache/hadoop/hbase/sec

hbase mapreduce running though command line

Hi, i want to run mapreduce program to insert data to tables in hbase. my cluster has 3 machines. If i want to run that program through command line, where can i do so..? should i do ${Hadoop_Home}/bin/hadoop jar MyJavaProg.jar java_mainclass_file source destn here MyJavaProg.jar is the jar of my

Re: something wrong with hbase mapreduce

i found that if i didnt control timestamp of the put mapreduce can run, otherwise just one time mapreduce. the question is i scan by timestamp to get my data so to put timestamp is my scan thing. any ideas ? thanks. 2010/12/1 梁景明 > Hi,i found a problem in my hbase mapreduce case. >

Re: something wrong with hbase mapreduce

t; > any ideas ? thanks. > > 2010/12/1 梁景明 > >> Hi,i found a problem in my hbase mapreduce case. >> >> when first running mapreduce TableMapReduceUtil runs ok. >> >> and i use hbase shell to delete some data from the table that mapreduce one >>

Re: something wrong with hbase mapreduce

amp to get my data > > so to put timestamp is my scan thing. > > > > any ideas ? thanks. > > > > 2010/12/1 梁景明 > > > >> Hi,i found a problem in my hbase mapreduce case. > >> > >> when first running mapreduce TableMapReduceUtil run

Re: something wrong with hbase mapreduce

gt;> > the question is i scan by timestamp to get my data >> > so to put timestamp is my scan thing. >> > >> > any ideas ? thanks. >> > >> > 2010/12/1 梁景明 >> > >> >> Hi,i found a problem in my hbase mapreduce case. >

Re: something wrong with hbase mapreduce

1, 2010, at 9:24, 梁景明 wrote: > >> > >> > i found that if i didnt control timestamp of the put > >> > mapreduce can run, otherwise just one time mapreduce. > >> > the question is i scan by timestamp to get my data > >> > so to put timestam

Re: something wrong with hbase mapreduce

t;> >> And finally let us know here :) >> >> Lars >> >> On Thu, Dec 2, 2010 at 2:48 AM, 梁景明 wrote: >> > 0.20.6 >> > >> > 2010/12/2 Lars George >> > >> >> What version of HBase are you using? >> >> >

Re: something wrong with hbase mapreduce

it was deleted from the shell > >> 4. Run a major compaction of the table on the shell (e.g. > >> "major_compact ") > >> 5. Re-run the MR job > >> 6. Check if the value is there again. > >> > >> And finally let us know here :) >

Re: something wrong with hbase mapreduce

gt; >> >> Try this: >> >> >> >> 1. Do the MR job >> >> 2. Do the delete from the shell >> >> 3. Check that it was deleted from the shell >> >> 4. Run a major compaction of the table on the shell (e.g. >> >> "major_compact "

HBase MapReduce Job with Multiple Scans

a "built in" way to run a mapreduce job that has multiple scans as input. I found the following related post which points me to creating my own map reduce "InputFormat" type by extending HBase's "TableInputFormatBase" and overriding the "getSplits()" meth

Re: A question about HBase MapReduce

:35 AM, "Florin P" wrote: >Hello! > >I've read Lars George's blog >http://www.larsgeorge.com/2009/05/hbase-mapreduce-101-part-i.html where >at the end of the article, he mentioned "In the next post I will show you >how to import data from a raw data >fi

HBase MapReduce with setup function problem

Dear all, I am writing a MapReduce application processing HBase table. In each map, it needs to read data from another HBase table, so i use the 'setup' function to initialize the HTable instance like this: @Override public void setup(Context context){ Configuration conf = HBaseConf

Re: input split for hbase mapreduce

Please take a look at map() method of Mapper classes in the code base. e.g. hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/GroupingTableMapper.java On Tue, May 9, 2017 at 2:51 AM, Rajeshkumar J wrote: > Hi > >If I am running mapreduce on hbase tables what will be

FILE_BYTES_READ counter missing for HBase mapreduce job

Hi, Basically I have a mapreduce job to scan a hbase table and do some processing. After the job finishes, I only got three filesystem counters: HDFS_BYTES_READ, HDFS_BYTES_WRITTEN and FILE_BYTES_WRITTEN. The value of HDFS_BYTES_READ is not very useful here because it shows the size of the .META

Fwd: HBase Mapreduce cannot find Map class

-- Forwarded message -- From: air Date: 2011/7/28 Subject: HBase Mapreduce cannot find Map class To: CDH Users import java.io.IOException; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; import org.apache.hadoop.conf.Configured

Re: HBase Mapreduce cannot find Map class

Maybe job.setJarByClass() can solve this problem. On Thu, Jul 28, 2011 at 7:06 PM, air wrote: > -- Forwarded message -- > From: air > Date: 2011/7/28 > Subject: HBase Mapreduce cannot find Map class > To: CDH Users > > > import java.

Re: hbase mapreduce running though command line

You don't need the conf dir in the jar, in fact you really don't want it there. I don't know where that alert is coming from, would be nice if you gave more details. J-D On Fri, Dec 9, 2011 at 6:45 AM, Vamshi Krishna wrote: > Hi, > i want to run mapreduce program to insert data to tables in hbas

Re: hbase mapreduce running though command line

i tried to run the program from eclipse, but during that , i could not see any job running on the jobtracker/tasktracker web UI pages. i observed that on the eclipse localJobRunner is executing , so that job is not submitted to the whole cluster, but its executing on that name node machine alone. S

Re: HBase MapReduce Job with Multiple Scans

; related post which points me to creating my own map reduce "InputFormat" > type by extending HBase's "TableInputFormatBase" and overriding the > "getSplits()" method: > > > http://stackoverflow.com/questions/4821455/hbase-mapreduce-on-multiple-scan-

Re: HBase MapReduce Job with Multiple Scans

> > mapreduce job that has multiple scans as input. I found the following > > related post which points me to creating my own map reduce "InputFormat" > > type by extending HBase's "TableInputFormatBase" and overriding the > > "getSplits()

Re: HBase MapReduce Job with Multiple Scans

apperJob" > > only > > > supports a single scan there doesn't appear to be a "built in" way to > > run a > > > mapreduce job that has multiple scans as input. I found the following > > > related post which points me to creating my o

Re: HBase MapReduce Job with Multiple Scans

ncreases.) > > > > > > > > Given that the HBase provided "TableMapReduceUtil.initTableMapperJob" > > > only > > > > supports a single scan there doesn't appear to be a "built in" way to > > > run a > > > > mapreduce job

HBase MapReduce - Using mutiple tables as source

Hi, While writing a MapReduce job for HBase, can I use multiple tables as input? I think TableMapReduceUtil.initTableMapperJob() takes a single table as parameter. For my requirement, I want to specify multiple tables and scan instances. I read about MultiTableInputCollection in the document ht

Re: HBase MapReduce with setup function problem

Have you considered using MultiTableInputFormat ? Cheers On Mon, Jan 27, 2014 at 9:14 AM, daidong wrote: > Dear all, > > I am writing a MapReduce application processing HBase table. In each map, > it needs to read data from another HBase table, so i use the 'setup' > function to initialize t

Re: HBase MapReduce with setup function problem

Dear Ted, Thanks very much for your reply! Yes. MultiTableInputFormat may work here, but i still want to know how to connect a hbase table inside MapReduce applications. Because i may need also write to tables inside map function. Do you know why previous mr application does not work? Because the

Re: HBase MapReduce with setup function problem

I agree that we should find the cause for why initialization got stuck. I noticed empty catch block: } catch (IOException e) { } Can you add some logging there to see what might have gone wrong ? Thanks On Mon, Jan 27, 2014 at 11:56 AM, daidong wrote: > Dear Ted, Thanks very much

issure about DNS error in running hbase mapreduce

i use hadoop-dns-checker check the dns problem ,seems all ok,but when i run MR task in hbase,it report problem,anyone have good idea? # ./run-on-cluster.sh hosts1 CH22 The authenticity of host 'ch22 (192.168.10.22)' can't be established. RSA key fingerprint is f3:4a:ca:a3:17:08:98:c2:0a:

Re: FILE_BYTES_READ counter missing for HBase mapreduce job

Addition info: The mapreduce job I run is a map-only job. It does not have reducers and it write data directly to hdfs in the mapper. Could this be the reason why there's no value for file_bytes_read? If so, is there any easy way to get the total input data size? Thanks Haijia On Thu, Sep 5,

Re: HBase MapReduce - Using mutiple tables as source

Hello Amlan, Issue is still unresolved...Will get fixed in 0.96.0. Regards, Mohammad Tariq On Mon, Aug 6, 2012 at 5:01 PM, Amlan Roy wrote: > Hi, > > > > While writing a MapReduce job for HBase, can I use multiple tables as input? > I think TableMapReduceUtil.initTableMapperJob() takes

Re: HBase MapReduce - Using mutiple tables as source

Hi, Isn't that the case that you can always initiate a scanner inside a map job (referring to another table from which had been set into the configuration of TableMapReduceUtil.initTableMapperJob(...) ) ? Hope this serves as temporary solution. On 08/06/2012 02:35 PM, Mohammad Tariq wrote:

Re: HBase MapReduce - Using mutiple tables as source

Hi Amlan, I think if you share your usecase regarding two tables as inputs, people on the mailing list may be able to help you better. For example, are you looking at joining the two tables? What are the sizes of the tables etc? Best Regards, Sonal Crux: Reporting for HBase

RE: HBase MapReduce - Using mutiple tables as source

. What is the best solution available in 0.92.0 (I understand the best solution is coming in version 0.96.0). Regards, Amlan -Original Message- From: Ioakim Perros [mailto:imper...@gmail.com] Sent: Monday, August 06, 2012 5:11 PM To: user@hbase.apache.org Subject: Re: HBase MapReduce - Using

Re: HBase MapReduce - Using mutiple tables as source

I > understand the best solution is coming in version 0.96.0). > > Regards, > Amlan > > -Original Message- > From: Ioakim Perros [mailto:imper...@gmail.com] > Sent: Monday, August 06, 2012 5:11 PM > To: user@hbase.apache.org > Subject: Re: HBase MapReduce -

RE: HBase MapReduce - Using mutiple tables as source

. Thanks, Best Regards, Wei Wei Tan Research Staff Member IBM T. J. Watson Research Center 19 Skyline Dr, Hawthorne, NY 10532 w...@us.ibm.com; 914-784-6752 From: "Amlan Roy" To: , Date: 08/06/2012 09:05 AM Subject: RE: HBase MapReduce - Using mutiple tables as so

Re: HBase MapReduce - Using mutiple tables as source

On Mon, Aug 6, 2012 at 3:22 PM, Wei Tan wrote: > I understand that this is achievable by running multiple MR jobs, each > with a different output table specified in the reduce class. What I want > is to scan a source table once and generate multiple tables at one time. > Thanks, > There is nothin

Re: HBase MapReduce - Using mutiple tables as source

Its available just as a patch on trunk for now. You wont find it in 0.92.0 ./zahoor On 06-Aug-2012, at 5:01 PM, Amlan Roy wrote: > https://issues.apache.org/jira/browse/HBASE-3996

HBase mapreduce job crawls on final 25% of maps

After trying to get help with distcp on hadoop-user and cdh-user mailing lists, I've given up on trying to use distcp and exporttable to migrate my hbase from .92.1 cdh4.1.3 to .98 on cdh5.3.0 I've been working on an hbase map reduce job to serialize my entries and insert them into kafka. Then I p

HBase mapreduce job: unable to find region for a table

I am running a very simple MR HBase job (reading from a tiny HBase table and outputs nothing). I run it on a pseudo-distributed HBase cluster on my local machine which uses a pseudo-distributed HDFS (on local machine again). When I run it, I get the following exception: Unable to find region for

A solution for data skew issue in HBase-Mapreduce jobs

Hi, all, I submit a new patch to fix the data skew issue in HBase-Mapreduce jobs. Would you please take a look at this new patch and give me some advice? https://issues.apache.org/jira/browse/HBASE-12590 Example: yeweichen2...@gmail.com

Re: HBase mapreduce job crawls on final 25% of maps

Excuse my double post. I thought I deleted my draft, and then constructed a cleaner, more detailed, more readable mail. On Tue, Apr 12, 2016 at 10:26 PM, Colin Kincaid Williams wrote: > After trying to get help with distcp on hadoop-user and cdh-user > mailing lists, I've given up on trying to us

Re: HBase mapreduce job crawls on final 25% of maps

I've noticed that I've omitted scan.setCaching(500);// 1 is the default in Scan, which will be bad for MapReduce jobs scan.setCacheBlocks(false); // don't set to true for MR jobs which appear to be suggestions from examples. Still I am not sure if this explains the significant request sl

  1   2   >