Re: performance of block cache
It works now by configuring the $JAVA_HOME explicitly. The JAVA_HOME is configured as $JAVA_HOME by default. Now I configure it to the complete path of my JDK explicitly. A little strange here, the $JAVA_HOME is already set in the shell environment, why do I still need to configure it again explicitly... 2014-09-15 14:58 GMT+08:00 牛兆捷 : > java -d64 version works well in the shell. > > 2014-09-15 11:59 GMT+08:00 牛兆捷 : > >> I use hbase-0.98-5-hadoop2 and modify the default heap size of region >> server in hbase-env.sh as below (keep all the other parameters in the file >> default): >> >> export HBASE_REGIONSERVER_OPTS="-Xmn200m >> -XX:CMSInitiatingOccupancyFraction=70 -Xms1024m -Xmx8000m" >> >> The error occurs when I start hbase cluster: >> >> 10.1.255.246: Invalid maximum heap size: -Xmx8000m >> 10.1.255.246: The specified size exceeds the maximum representable size. >> 10.1.255.246: Could not create the Java virtual machine. >> >> The jvm I use is 64 bit : >> >> java version "1.6.0_39" >> Java(TM) SE Runtime Environment (build 1.6.0_39-b04) >> Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode) >> >> Why 8G setting exceeds the maximum representable size. >> >> 2014-09-15 11:39 GMT+08:00 Nick Dimiduk : >> >>> The scripts automate use of the tool PerformanceEvaluation that ships >>> with >>> HBase, so in that way it runs against a cluster directly. It depends on >>> having independent configuration directories set up for each test >>> config. There's probably too much custom-to-my-environment stuff in >>> there, >>> but I hope I included enough diffs that you can work it out in your >>> deployment. Let me know if you have any more questions. >>> >>> -n >>> >>> On Sunday, September 14, 2014, 牛兆捷 wrote: >>> >>> > Hi, Nick >>> > >>> > Can your perf_blockcache performance testing script can be applied to >>> hbase >>> > cluster directly? >>> > If not, what kind of things should I take care? >>> > >>> > 2014-08-22 7:06 GMT+08:00 Nick Dimiduk >> >>> > >: >>> > >>> > > I'm familiar with Stack's work too, but thanks for pointing it out :) >>> > > >>> > > On Wed, Aug 20, 2014 at 8:19 PM, 牛兆捷 >> > >>> > wrote: >>> > > >>> > > > Hi Nick: >>> > > > >>> > > > Yes, I am interested in it. I will try first. >>> > > > >>> > > > Btw, this site (http://people.apache.org/~stack/bc/) also does the >>> > > similar >>> > > > performance evaluation. >>> > > > You can have a look if you are interested in. >>> > > > >>> > > > >>> > > > 2014-08-21 1:48 GMT+08:00 Nick Dimiduk >> > >: >>> > > > >>> > > > > Hi Zhaojie, >>> > > > > >>> > > > > I'm responsible for this particular bit of work. One thing to >>> note in >>> > > > these >>> > > > > experiments is that I did not control explicitly for OS caching. >>> I >>> > ran >>> > > > > "warmup" workloads before collecting measurements, but because >>> the >>> > > amount >>> > > > > of RAM on the machine is fixed, it's impact of OS cache is >>> different >>> > > with >>> > > > > different based on the amount of memory used by HBase. Another, >>> as >>> > Todd >>> > > > > pointed out on an earlier thread, is that my trend lines are >>> probably >>> > > > > optimistic/misleading. >>> > > > > >>> > > > > Something I was driving for was to understand how well the >>> different >>> > > > > implementations before as they're managing more and more memory. >>> I'd >>> > > like >>> > > > > to get some insight into how we might be able to take advantage >>> of >>> > > 100's >>> > > > or >>> > > > > even 1000's of GB of memory when the time comes. That's part of >>> why >>> > > > there's >>> > > > > so many variables. >>> > > > > >>> > > > > I scripted out the running of the tests, all of my >>> configurations are >>> > > > > available in the associated github repo [0], and all of the data >>> > points >>> > > > are >>> > > > > available as a csv. If you're interested in experimenting >>> yourself, >>> > > > please >>> > > > > let me know how I can help. >>> > > > > >>> > > > > Cheers, >>> > > > > Nick >>> > > > > >>> > > > > [0]: https://github.com/ndimiduk/perf_blockcache >>> > > > > >>> > > > > >>> > > > > On Wed, Aug 20, 2014 at 6:00 AM, 牛兆捷 >> > > wrote: >>> > > > > >>> > > > > > the complete blog link is: >>> > > > > > http://zh.hortonworks.com/blog/blockcache-showdown-hbase/ >>> > > > > > >>> > > > > > >>> > > > > > 2014-08-20 11:41 GMT+08:00 牛兆捷 >> >>> > >: >>> > > > > > >>> > > > > > > Hi all: >>> > > > > > > >>> > > > > > > I saw some interesting results from Hortonworks blog (block >>> cache >>> > > > > > > < >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> http://zh.hortonworks.com/wp-content/uploads/2014/03/perfeval_blockcache_v2.pdf >>> > > > > > > >>> > > > > > > ). >>> > > > > > > >>> > > > > > > In this result, the ratio of memory footprint to database >>> size is >>> > > > held >>> > > > > > > fixed while >>> > > > > > > the absolute values are increased. >>> > > > > > > >>> > > > > > > In my mind, the performance should becomes worse for larger >>> ratio >>
Re: Custom Filter on hbase Column
Thanks Anoop.I did that but the only method that was getting called in my filter was public byte[] toByteArray() ,even though I over ride transformcell. Thanks, Nishanth On Thu, Sep 11, 2014 at 10:51 PM, Anoop John wrote: > And u have to implement > transformCell(*final* Cell v) > in your custom Filter. > > JFYI > > -Anoop- > > On Fri, Sep 12, 2014 at 4:36 AM, Nishanth S > wrote: > > > Sure Sean.This is much needed. > > > > -Nishan > > > > On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey > wrote: > > > > > I filed HBASE-11950 to get some details added to the book on this > > topic[1]. > > > > > > Nishanth, could you follow that ticket and give feedback on whatever > > update > > > ends up proposed? > > > > > > [1]: https://issues.apache.org/jira/browse/HBASE-11950 > > > > > > On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu wrote: > > > > > > > See http://search-hadoop.com/m/DHED4xWh622 > > > > > > > > On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S > > > > > wrote: > > > > > > > > > Hey All, > > > > > > > > > > I am sorry if this is a naive question.Do we need to generate a > proto > > > > file > > > > > using proto buffer compiler when implementing a filter.I did not > see > > > that > > > > > any where in the documentation.Can some one help please? > > > > > > > > > > On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S < > > nishanth.2...@gmail.com> > > > > > wrote: > > > > > > > > > > > Thanks Dima and Ted.Yes I need to return the first the 1000 > > > > > > characters.There is no matching involved. > > > > > > > > > > > > -Nishan > > > > > > > > > > > > On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu > > > wrote: > > > > > > > > > > > >> In Nishanth's case, the 5K message is stored in one KeyValue, > > right > > > ? > > > > > >> > > > > > >> If only the first 1000 Characters of this message are to be > > > returned, > > > > > >> a new KeyValue > > > > > >> needs to be composed and returned. > > > > > >> > > > > > >> Cheers > > > > > >> > > > > > >> On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak < > > dspi...@cloudera.com > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > Hi Nishanth, > > > > > >> > > > > > > >> > Take a look at > http://hbase.apache.org/book/client.filter.html > > . > > > > If I > > > > > >> > understand your use case correctly, you might want to look at > > > > > >> > RegexStringComparator to match the first 1000 characters of > your > > > > > column > > > > > >> > qualifier. > > > > > >> > > > > > > >> > -Dima > > > > > >> > > > > > > >> > On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S < > > > > nishanth.2...@gmail.com > > > > > > > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Hi All, > > > > > >> > > > > > > > >> > > I have an hbase table with multiple cfs (say c1,c2,c3).Each > of > > > > this > > > > > >> > column > > > > > >> > > family has a column 'message' which is about 5K.What I need > > to > > > do > > > > > is > > > > > >> to > > > > > >> > > grab only the first 1000 Characters of this message when I > do > > a > > > > get > > > > > on > > > > > >> > the > > > > > >> > > table using row Key.I was thinking of using filters to do > > this > > > on > > > > > >> hbase > > > > > >> > > sever side.Can some one help me on how to go about this. > > > > > >> > > > > > > > >> > > > > > > > >> > > Thanks, > > > > > >> > > Nishan > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sean > > > > > >
Re: Custom Filter on hbase Column
Take a look at StoreScanner#next():
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
switch(qcode) {
case INCLUDE:
case INCLUDE_AND_SEEK_NEXT_ROW:
case INCLUDE_AND_SEEK_NEXT_COL:
Filter f = matcher.getFilter();
if (f != null) {
// TODO convert Scan Query Matcher to be Cell instead of KV
based ?
kv = KeyValueUtil.ensureKeyValue(f.transformCell(kv));
}
Cheers
On Mon, Sep 15, 2014 at 8:27 AM, Nishanth S wrote:
> Thanks Anoop.I did that but the only method that was getting called in my
> filter was public byte[] toByteArray() ,even though I over ride
> transformcell.
>
>
> Thanks,
> Nishanth
>
> On Thu, Sep 11, 2014 at 10:51 PM, Anoop John
> wrote:
>
> > And u have to implement
> > transformCell(*final* Cell v)
> > in your custom Filter.
> >
> > JFYI
> >
> > -Anoop-
> >
> > On Fri, Sep 12, 2014 at 4:36 AM, Nishanth S
> > wrote:
> >
> > > Sure Sean.This is much needed.
> > >
> > > -Nishan
> > >
> > > On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey
> > wrote:
> > >
> > > > I filed HBASE-11950 to get some details added to the book on this
> > > topic[1].
> > > >
> > > > Nishanth, could you follow that ticket and give feedback on whatever
> > > update
> > > > ends up proposed?
> > > >
> > > > [1]: https://issues.apache.org/jira/browse/HBASE-11950
> > > >
> > > > On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu wrote:
> > > >
> > > > > See http://search-hadoop.com/m/DHED4xWh622
> > > > >
> > > > > On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S <
> nishanth.2...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hey All,
> > > > > >
> > > > > > I am sorry if this is a naive question.Do we need to generate a
> > proto
> > > > > file
> > > > > > using proto buffer compiler when implementing a filter.I did not
> > see
> > > > that
> > > > > > any where in the documentation.Can some one help please?
> > > > > >
> > > > > > On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S <
> > > nishanth.2...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Dima and Ted.Yes I need to return the first the 1000
> > > > > > > characters.There is no matching involved.
> > > > > > >
> > > > > > > -Nishan
> > > > > > >
> > > > > > > On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu
> > > > wrote:
> > > > > > >
> > > > > > >> In Nishanth's case, the 5K message is stored in one KeyValue,
> > > right
> > > > ?
> > > > > > >>
> > > > > > >> If only the first 1000 Characters of this message are to be
> > > > returned,
> > > > > > >> a new KeyValue
> > > > > > >> needs to be composed and returned.
> > > > > > >>
> > > > > > >> Cheers
> > > > > > >>
> > > > > > >> On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak <
> > > dspi...@cloudera.com
> > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi Nishanth,
> > > > > > >> >
> > > > > > >> > Take a look at
> > http://hbase.apache.org/book/client.filter.html
> > > .
> > > > > If I
> > > > > > >> > understand your use case correctly, you might want to look
> at
> > > > > > >> > RegexStringComparator to match the first 1000 characters of
> > your
> > > > > > column
> > > > > > >> > qualifier.
> > > > > > >> >
> > > > > > >> > -Dima
> > > > > > >> >
> > > > > > >> > On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S <
> > > > > nishanth.2...@gmail.com
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > > Hi All,
> > > > > > >> > >
> > > > > > >> > > I have an hbase table with multiple cfs (say
> c1,c2,c3).Each
> > of
> > > > > this
> > > > > > >> > column
> > > > > > >> > > family has a column 'message' which is about 5K.What I
> need
> > > to
> > > > do
> > > > > > is
> > > > > > >> to
> > > > > > >> > > grab only the first 1000 Characters of this message when I
> > do
> > > a
> > > > > get
> > > > > > on
> > > > > > >> > the
> > > > > > >> > > table using row Key.I was thinking of using filters to do
> > > this
> > > > on
> > > > > > >> hbase
> > > > > > >> > > sever side.Can some one help me on how to go about this.
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > Thanks,
> > > > > > >> > > Nishan
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sean
> > > >
> > >
> >
>
Adding 64-bit nodes to 32-bit cluster?
Hi, We are thinking of adding some new 64-bit servers to run RegionServers to our 32-bit HBase cluster. Anything we should worry about or pay extra attention to? Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/
Re: Adding 64-bit nodes to 32-bit cluster?
Well, many HBase internal limits are based on the architecture so that should impact you right away in resource utilization in the RSs, so a heap or a setting that worked fine for your 32-bit RSs might need to be tuned for a 64-bit environment. Probably might be easier to run a 32-bit JVM in the 64-bit boxes and phase out the old 32-bit nodes after you test for a while in a couple of the 64-bit nodes new settings. cheers, esteban, -- Cloudera, Inc. On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > We are thinking of adding some new 64-bit servers to run RegionServers to > our 32-bit HBase cluster. Anything we should worry about or pay extra > attention to? > > Thanks, > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ >
Re: Adding 64-bit nodes to 32-bit cluster?
Hi Esteban, Sorry, I meant to point that out in my original email - yeah, heap sizes, Xmx, and such will be different for 32-bit and 64-bit servers, but I was wondering if there is anything in HBase that could complain if, say, a region written on a 32-bit server moves to a 64-bit server or anything else of that sort that is HBase (or Hadoop) specific. Thanks, Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Mon, Sep 15, 2014 at 5:19 PM, Esteban Gutierrez wrote: > Well, many HBase internal limits are based on the architecture so that > should impact you right away in resource utilization in the RSs, so a heap > or a setting that worked fine for your 32-bit RSs might need to be tuned > for a 64-bit environment. Probably might be easier to run a 32-bit JVM in > the 64-bit boxes and phase out the old 32-bit nodes after you test for a > while in a couple of the 64-bit nodes new settings. > > cheers, > esteban, > > > -- > Cloudera, Inc. > > > On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > Hi, > > > > We are thinking of adding some new 64-bit servers to run RegionServers to > > our 32-bit HBase cluster. Anything we should worry about or pay extra > > attention to? > > > > Thanks, > > Otis > > -- > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > Solr & Elasticsearch Support * http://sematext.com/ > > >
Re: Adding 64-bit nodes to 32-bit cluster?
Only where we touch the native Hadoop libraries I think. If you have specified compression implemented with a Hadoop native library, like snappy or lzo, and have forgotten to deploy 64 bit native libraries, and move to this 64 bit environment, you won't be able to open the affected table(s) until native link issues are resolved. On Mon, Sep 15, 2014 at 2:27 PM, Otis Gospodnetic wrote: > Hi Esteban, > > Sorry, I meant to point that out in my original email - yeah, heap sizes, > Xmx, and such will be different for 32-bit and 64-bit servers, but I was > wondering if there is anything in HBase that could complain if, say, a > region written on a 32-bit server moves to a 64-bit server or anything > else of that sort that is HBase (or Hadoop) specific. > > Thanks, > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > > On Mon, Sep 15, 2014 at 5:19 PM, Esteban Gutierrez > wrote: > >> Well, many HBase internal limits are based on the architecture so that >> should impact you right away in resource utilization in the RSs, so a heap >> or a setting that worked fine for your 32-bit RSs might need to be tuned >> for a 64-bit environment. Probably might be easier to run a 32-bit JVM in >> the 64-bit boxes and phase out the old 32-bit nodes after you test for a >> while in a couple of the 64-bit nodes new settings. >> >> cheers, >> esteban, >> >> >> -- >> Cloudera, Inc. >> >> >> On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < >> otis.gospodne...@gmail.com> wrote: >> >> > Hi, >> > >> > We are thinking of adding some new 64-bit servers to run RegionServers to >> > our 32-bit HBase cluster. Anything we should worry about or pay extra >> > attention to? >> > >> > Thanks, >> > Otis >> > -- >> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> > Solr & Elasticsearch Support * http://sematext.com/ >> > >> -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: Adding 64-bit nodes to 32-bit cluster?
Do we have kind of native compression in PB? Or not at all? Because if so, it might be an issue. I run a 0.94 cluster with a mix of 32 and 64 with Snappy enabled. But never tried 0.96 or more on it... 2014-09-15 18:36 GMT-04:00 Andrew Purtell : > Only where we touch the native Hadoop libraries I think. If you have > specified compression implemented with a Hadoop native library, like > snappy or lzo, and have forgotten to deploy 64 bit native libraries, > and move to this 64 bit environment, you won't be able to open the > affected table(s) until native link issues are resolved. > > > On Mon, Sep 15, 2014 at 2:27 PM, Otis Gospodnetic > wrote: > > Hi Esteban, > > > > Sorry, I meant to point that out in my original email - yeah, heap sizes, > > Xmx, and such will be different for 32-bit and 64-bit servers, but I was > > wondering if there is anything in HBase that could complain if, say, a > > region written on a 32-bit server moves to a 64-bit server or > anything > > else of that sort that is HBase (or Hadoop) specific. > > > > Thanks, > > Otis > > -- > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > On Mon, Sep 15, 2014 at 5:19 PM, Esteban Gutierrez > > > wrote: > > > >> Well, many HBase internal limits are based on the architecture so that > >> should impact you right away in resource utilization in the RSs, so a > heap > >> or a setting that worked fine for your 32-bit RSs might need to be tuned > >> for a 64-bit environment. Probably might be easier to run a 32-bit JVM > in > >> the 64-bit boxes and phase out the old 32-bit nodes after you test for a > >> while in a couple of the 64-bit nodes new settings. > >> > >> cheers, > >> esteban, > >> > >> > >> -- > >> Cloudera, Inc. > >> > >> > >> On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < > >> otis.gospodne...@gmail.com> wrote: > >> > >> > Hi, > >> > > >> > We are thinking of adding some new 64-bit servers to run > RegionServers to > >> > our 32-bit HBase cluster. Anything we should worry about or pay extra > >> > attention to? > >> > > >> > Thanks, > >> > Otis > >> > -- > >> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >> > Solr & Elasticsearch Support * http://sematext.com/ > >> > > >> > > > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) >
Re: Adding 64-bit nodes to 32-bit cluster?
bq. Do we have kind of native compression in PB? I don't think so. On Mon, Sep 15, 2014 at 4:28 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Do we have kind of native compression in PB? Or not at all? Because if so, > it might be an issue. > > I run a 0.94 cluster with a mix of 32 and 64 with Snappy enabled. But never > tried 0.96 or more on it... > > 2014-09-15 18:36 GMT-04:00 Andrew Purtell : > > > Only where we touch the native Hadoop libraries I think. If you have > > specified compression implemented with a Hadoop native library, like > > snappy or lzo, and have forgotten to deploy 64 bit native libraries, > > and move to this 64 bit environment, you won't be able to open the > > affected table(s) until native link issues are resolved. > > > > > > On Mon, Sep 15, 2014 at 2:27 PM, Otis Gospodnetic > > wrote: > > > Hi Esteban, > > > > > > Sorry, I meant to point that out in my original email - yeah, heap > sizes, > > > Xmx, and such will be different for 32-bit and 64-bit servers, but I > was > > > wondering if there is anything in HBase that could complain if, say, a > > > region written on a 32-bit server moves to a 64-bit server or > > anything > > > else of that sort that is HBase (or Hadoop) specific. > > > > > > Thanks, > > > Otis > > > -- > > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:19 PM, Esteban Gutierrez < > este...@cloudera.com > > > > > > wrote: > > > > > >> Well, many HBase internal limits are based on the architecture so that > > >> should impact you right away in resource utilization in the RSs, so a > > heap > > >> or a setting that worked fine for your 32-bit RSs might need to be > tuned > > >> for a 64-bit environment. Probably might be easier to run a 32-bit JVM > > in > > >> the 64-bit boxes and phase out the old 32-bit nodes after you test > for a > > >> while in a couple of the 64-bit nodes new settings. > > >> > > >> cheers, > > >> esteban, > > >> > > >> > > >> -- > > >> Cloudera, Inc. > > >> > > >> > > >> On Mon, Sep 15, 2014 at 1:37 PM, Otis Gospodnetic < > > >> otis.gospodne...@gmail.com> wrote: > > >> > > >> > Hi, > > >> > > > >> > We are thinking of adding some new 64-bit servers to run > > RegionServers to > > >> > our 32-bit HBase cluster. Anything we should worry about or pay > extra > > >> > attention to? > > >> > > > >> > Thanks, > > >> > Otis > > >> > -- > > >> > Monitoring * Alerting * Anomaly Detection * Centralized Log > Management > > >> > Solr & Elasticsearch Support * http://sematext.com/ > > >> > > > >> > > > > > > > > -- > > Best regards, > > > >- Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet > > Hein (via Tom White) > > >
Re: Adding 64-bit nodes to 32-bit cluster?
On Mon, Sep 15, 2014 at 4:28 PM, Jean-Marc Spaggiari wrote: > Do we have kind of native compression in PB? Protobufs has its own encodings, the Java language bindings implement them in Java. -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Copying data from 94 to 98 ..
Hello, I'm trying to copy data between Hbase clusters on different versions. I am using : /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -chuser hbase -chgroup hadoop -snapshot msg_snapshot -mappers 50 -copy-from hftp://src-cluster:50070/hbase -copy-to hdfs:/dest-cluster:8020/hbase Till now, based on various tips from the mailing list, I have modified the source cluster data dir paths to mimic the 98 convention (archive, table data paths, etc). This helped in jumping some roadblocks but not all. This is what I see now : 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed java.io.IOException at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) ... .. Caused by: java.io.IOException: read=-1, wanted=4 at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) at org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java .. Fails while trying to read refernce hfile. Is this something folks have done before and/or is possible to do? I'd really like to do this without having to upgrade my source cluster or downgrade my dest cluster. I'm using 94.6 on source cluster and 98.1 on dest cluster. Cheers, -Gautam.
Re: Copying data from 94 to 98 ..
To be clear I am running this command on the dest cluster. -G. On Mon, Sep 15, 2014 at 4:58 PM, Gautam wrote: > Hello, > I'm trying to copy data between Hbase clusters on different > versions. I am using : > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > -chuser hbase > -chgroup hadoop > -snapshot msg_snapshot > -mappers 50 > -copy-from hftp://src-cluster:50070/hbase > -copy-to hdfs:/dest-cluster:8020/hbase > > > Till now, based on various tips from the mailing list, I have modified the > source cluster data dir paths to mimic the 98 convention (archive, table > data paths, etc). This helped in jumping some roadblocks but not all. > > This is what I see now : > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > java.io.IOException > at > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > at > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > at > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > at > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > ... > .. > Caused by: java.io.IOException: read=-1, wanted=4 > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > at > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > .. > > > Fails while trying to read refernce hfile. Is this something folks have > done before and/or is possible to do? I'd really like to do this without > having to upgrade my source cluster or downgrade my dest cluster. > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > Cheers, > -Gautam. > -- "If you really want something in this life, you have to work for it. Now, quiet! They're about to announce the lottery numbers..."
Re: Adding 64-bit nodes to 32-bit cluster?
Yeah, what Andrew said you need to be careful to deploy the right codecs on the right architecture. Otherwise I don't remember any issue mixing RSs with 32/64-bit platforms only the heap sizing and some JVM tuning perhaps. esteban. -- Cloudera, Inc. On Mon, Sep 15, 2014 at 4:34 PM, Andrew Purtell wrote: > On Mon, Sep 15, 2014 at 4:28 PM, Jean-Marc Spaggiari > wrote: > > Do we have kind of native compression in PB? > > Protobufs has its own encodings, the Java language bindings implement > them in Java. > > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) >
Re: Copying data from 94 to 98 ..
bq. 98.1 on dest cluster Looking at the history for SnapshotManifestV1, it came with HBASE-7987 which went to 0.99.0 Perhaps you're using a distro with HBASE-7987 ? On Mon, Sep 15, 2014 at 4:58 PM, Gautam wrote: > Hello, > I'm trying to copy data between Hbase clusters on different > versions. I am using : > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > -chuser hbase > -chgroup hadoop > -snapshot msg_snapshot > -mappers 50 > -copy-from hftp://src-cluster:50070/hbase > -copy-to hdfs:/dest-cluster:8020/hbase > > > Till now, based on various tips from the mailing list, I have modified the > source cluster data dir paths to mimic the 98 convention (archive, table > data paths, etc). This helped in jumping some roadblocks but not all. > > This is what I see now : > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > java.io.IOException > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > at > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > ... > .. > Caused by: java.io.IOException: read=-1, wanted=4 > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > at > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > at > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > .. > > > Fails while trying to read refernce hfile. Is this something folks have > done before and/or is possible to do? I'd really like to do this without > having to upgrade my source cluster or downgrade my dest cluster. > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > Cheers, > -Gautam. >
Re: Copying data from 94 to 98 ..
Yep, looks like the CDH distro backports HBASE-7987. Having said that, is there a transition path for us or are we hosed :-) ? In general, what's the recommended way to achieve this, at this point I feel i'm going around the system to achieve what I want. If nothing else works with export snapshot should I just downgrade to 94, export snapshot and then upgrade to 98? Is the upgrade migration path different from what export snapshot does (i'd imagine yes)? Cheers, -Gautam. On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu wrote: > bq. 98.1 on dest cluster > > Looking at the history for SnapshotManifestV1, it came with HBASE-7987 > which went to 0.99.0 > > Perhaps you're using a distro with HBASE-7987 ? > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam wrote: > > > Hello, > > I'm trying to copy data between Hbase clusters on different > > versions. I am using : > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > -chuser hbase > > -chgroup hadoop > > -snapshot msg_snapshot > > -mappers 50 > > -copy-from hftp://src-cluster:50070/hbase > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > Till now, based on various tips from the mailing list, I have modified > the > > source cluster data dir paths to mimic the 98 convention (archive, table > > data paths, etc). This helped in jumping some roadblocks but not all. > > > > This is what I see now : > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > java.io.IOException > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > ... > > .. > > Caused by: java.io.IOException: read=-1, wanted=4 > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > at > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > .. > > > > > > Fails while trying to read refernce hfile. Is this something folks have > > done before and/or is possible to do? I'd really like to do this without > > having to upgrade my source cluster or downgrade my dest cluster. > > > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > > > > Cheers, > > -Gautam. > > > -- "If you really want something in this life, you have to work for it. Now, quiet! They're about to announce the lottery numbers..."
Re: Copying data from 94 to 98 ..
94 and 98 differs in directory layout so 98 is not able to read 94 layout unless you run the migration tool which is basically moving all the data in a "default" namespace directory e.g. /hbase/table -> /hbase/data/default/table /hbase/.archive/table -> /hbase/archive/default/table Matteo On Mon, Sep 15, 2014 at 6:17 PM, Gautam wrote: > Yep, looks like the CDH distro backports HBASE-7987. Having said that, is > there a transition path for us or are we hosed :-) ? In general, what's the > recommended way to achieve this, at this point I feel i'm going around the > system to achieve what I want. If nothing else works with export snapshot > should I just downgrade to 94, export snapshot and then upgrade to 98? Is > the upgrade migration path different from what export snapshot does (i'd > imagine yes)? > > Cheers, > -Gautam. > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu wrote: > > > bq. 98.1 on dest cluster > > > > Looking at the history for SnapshotManifestV1, it came with HBASE-7987 > > which went to 0.99.0 > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam wrote: > > > > > Hello, > > > I'm trying to copy data between Hbase clusters on different > > > versions. I am using : > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > -chuser hbase > > > -chgroup hadoop > > > -snapshot msg_snapshot > > > -mappers 50 > > > -copy-from hftp://src-cluster:50070/hbase > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > Till now, based on various tips from the mailing list, I have modified > > the > > > source cluster data dir paths to mimic the 98 convention (archive, > table > > > data paths, etc). This helped in jumping some roadblocks but not all. > > > > > > This is what I see now : > > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > > java.io.IOException > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > ... > > > .. > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > at > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > > at > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > > .. > > > > > > > > > Fails while trying to read refernce hfile. Is this something folks have > > > done before and/or is possible to do? I'd really like to do this > without > > > having to upgrade my source cluster or downgrade my dest cluster. > > > > > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > > > > > > > Cheers, > > > -Gautam. > > > > > > > > > -- > "If you really want something in this life, you have to work for it. Now, > quiet! They're about to announce the lottery numbers..." >
Tuning and Sizing Rules of Thumb
Hi all, I'd like to solicit your help in providing current rules of thumb and "best practices" for a section of the HBase Reference Guide. I've done some research and started a JIRA at https://issues.apache.org/jira/browse/HBASE-11791. I'd really appreciate it if you guys could take some time and weigh in. I'd like to get away from some of the "it depends" answers to these kinds of questions. Thanks in advance! Thanks, Misty
Re: Copying data from 94 to 98 ..
Thanks for the reply Matteo. This is exactly what I did. I modified the source cluster's dir structure to mimic that of the 98 cluster. I even got as far as it trying to look through the reference files. I end up with this exception : 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed java.io.IOException at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) ... .. Caused by: java.io.IOException: read=-1, wanted=4 at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) at org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java .. This and Ted's reply about HBASE-7987 leads me to believe that the export tool from my distro is in capable of working around the regionManifest file requirement. I'm now left with the option of downgrading my dest cluster to 94, copying data and then upgrading using the upgrade migration tool. Wanted to know if others have tried this or there are other things I can do. If not, i'l just go ahead and do this :-) Cheers, -Gautam. On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi wrote: > 94 and 98 differs in directory layout > so 98 is not able to read 94 layout unless you run the migration tool > which is basically moving all the data in a "default" namespace directory > e.g. > /hbase/table -> /hbase/data/default/table > /hbase/.archive/table -> /hbase/archive/default/table > > Matteo > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam wrote: > > > Yep, looks like the CDH distro backports HBASE-7987. Having said that, is > > there a transition path for us or are we hosed :-) ? In general, what's > the > > recommended way to achieve this, at this point I feel i'm going around > the > > system to achieve what I want. If nothing else works with export snapshot > > should I just downgrade to 94, export snapshot and then upgrade to 98? Is > > the upgrade migration path different from what export snapshot does (i'd > > imagine yes)? > > > > Cheers, > > -Gautam. > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu wrote: > > > > > bq. 98.1 on dest cluster > > > > > > Looking at the history for SnapshotManifestV1, it came with HBASE-7987 > > > which went to 0.99.0 > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam > wrote: > > > > > > > Hello, > > > > I'm trying to copy data between Hbase clusters on different > > > > versions. I am using : > > > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > > -chuser hbase > > > > -chgroup hadoop > > > > -snapshot msg_snapshot > > > > -mappers 50 > > > > -copy-from hftp://src-cluster:50070/hbase > > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > > > > Till now, based on various tips from the mailing list, I have > modified > > > the > > > > source cluster data dir paths to mimic the 98 convention (archive, > > table > > > > data paths, etc). This helped in jumping some roadblocks but not all. > > > > > > > > This is what I see now : > > > > > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export > failed > > > > java.io.IOException > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > > ... > > > > .. > > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > > > .. > > > > > > > > > > > > Fails while trying to read refernce hfile. Is this something folks > have > > > > done before and/or is possible to do? I'd really like to do this > > without > > > > having to upgrade my source cluster or downgrade my dest cluster. > > > > > > > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > > > > > > > > > > Cheers, > > > > -Gautam. > > > > > > > > > > > > > > > -- > > "If you really want something in this life, you have to wor
Re: Copying data from 94 to 98 ..
can you post the full exception and the file path ? maybe there is a bug in looking up the reference file. It seems to not be able to find enough data in the file... Matteo On Mon, Sep 15, 2014 at 10:08 PM, Gautam wrote: > Thanks for the reply Matteo. > > This is exactly what I did. I modified the source cluster's dir structure > to mimic that of the 98 cluster. I even got as far as it trying to look > through the reference files. > > I end up with this exception : > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > java.io.IOException > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > at > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > at > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > ... > .. > Caused by: java.io.IOException: read=-1, wanted=4 > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > at > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > at > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > .. > > > This and Ted's reply about HBASE-7987 leads me to believe that the export > tool from my distro is in capable of working around the regionManifest file > requirement. I'm now left with the option of downgrading my dest cluster to > 94, copying data and then upgrading using the upgrade migration tool. > Wanted to know if others have tried this or there are other things I can > do. If not, i'l just go ahead and do this :-) > > > Cheers, > -Gautam. > > > > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi > wrote: > > > 94 and 98 differs in directory layout > > so 98 is not able to read 94 layout unless you run the migration tool > > which is basically moving all the data in a "default" namespace directory > > e.g. > > /hbase/table -> /hbase/data/default/table > > /hbase/.archive/table -> /hbase/archive/default/table > > > > Matteo > > > > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam wrote: > > > > > Yep, looks like the CDH distro backports HBASE-7987. Having said that, > is > > > there a transition path for us or are we hosed :-) ? In general, what's > > the > > > recommended way to achieve this, at this point I feel i'm going around > > the > > > system to achieve what I want. If nothing else works with export > snapshot > > > should I just downgrade to 94, export snapshot and then upgrade to 98? > Is > > > the upgrade migration path different from what export snapshot does > (i'd > > > imagine yes)? > > > > > > Cheers, > > > -Gautam. > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu wrote: > > > > > > > bq. 98.1 on dest cluster > > > > > > > > Looking at the history for SnapshotManifestV1, it came with > HBASE-7987 > > > > which went to 0.99.0 > > > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam > > wrote: > > > > > > > > > Hello, > > > > > I'm trying to copy data between Hbase clusters on > different > > > > > versions. I am using : > > > > > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > > > -chuser hbase > > > > > -chgroup hadoop > > > > > -snapshot msg_snapshot > > > > > -mappers 50 > > > > > -copy-from hftp://src-cluster:50070/hbase > > > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > > > > > > > Till now, based on various tips from the mailing list, I have > > modified > > > > the > > > > > source cluster data dir paths to mimic the 98 convention (archive, > > > table > > > > > data paths, etc). This helped in jumping some roadblocks but not > all. > > > > > > > > > > This is what I see now : > > > > > > > > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export > > failed > > > > > java.io.IOException > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > > > ... > > > > > .. > > > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > > > > at > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.r
Re: Copying data from 94 to 98 ..
14/09/15 23:34:59 DEBUG snapshot.SnapshotManifestV1: Adding reference for file (4/4): hftp:// master42.stg.com:50070/hbase/.hbase-snapshot/msg_snapshot/84f60fc2aa7e96df91e6289e6c19dc25/c/afe341e4149649578c5861e32494dbec 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed java.io.IOException at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitReferencedFiles(SnapshotReferenceUtil.java:108) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:479) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.access$200(ExportSnapshot.java:89) at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportSnapshotInputFormat.getSplits(ExportSnapshot.java:600) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1107) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1124) at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:178) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:976) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:976) at org.apache.hadoop.mapreduce.Job.submit(Job.java:582) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:612) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:751) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:905) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:975) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:979) Caused by: java.io.IOException: read=-1, wanted=4 at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) at org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java:204) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.buildManifestFromDisk(SnapshotManifestV1.java:179) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:131) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:127) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) On Mon, Sep 15, 2014 at 10:18 PM, Matteo Bertozzi wrote: > can you post the full exception and the file path ? > maybe there is a bug in looking up the reference file. > It seems to not be able to find enough data in the file... > > Matteo > > > On Mon, Sep 15, 2014 at 10:08 PM, Gautam wrote: > > > Thanks for the reply Matteo. > > > > This is exactly what I did. I modified the source cluster's dir structure > > to mimic that of the 98 cluster. I even got as far as it trying to look > > through the reference files. > > > > I end up with this exception : > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > java.io.IOException > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > ... > > .. > > Caused by: java.io.IOException: read=-1, wanted=4 > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > at > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > .. > > > > > > This and Ted's reply about HBASE-7987 leads me to believe that the export > > tool from my distro is in capable of working around the regionManifest > file > > requirement. I'm now left with the option of downgrading my dest cluster > to > > 94, copying data and then upgrading using the upgrade migration t
Re: Copying data from 94 to 98 ..
While you continue on the snapshot approach, have you tried to Export the table in 0.94 to hdfs, and then Import the data from hdfs to 0.98? On Sep 15, 2014 10:19 PM, "Matteo Bertozzi" wrote: > can you post the full exception and the file path ? > maybe there is a bug in looking up the reference file. > It seems to not be able to find enough data in the file... > > Matteo > > > On Mon, Sep 15, 2014 at 10:08 PM, Gautam wrote: > > > Thanks for the reply Matteo. > > > > This is exactly what I did. I modified the source cluster's dir structure > > to mimic that of the 98 cluster. I even got as far as it trying to look > > through the reference files. > > > > I end up with this exception : > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > java.io.IOException > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > ... > > .. > > Caused by: java.io.IOException: read=-1, wanted=4 > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > at > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > .. > > > > > > This and Ted's reply about HBASE-7987 leads me to believe that the export > > tool from my distro is in capable of working around the regionManifest > file > > requirement. I'm now left with the option of downgrading my dest cluster > to > > 94, copying data and then upgrading using the upgrade migration tool. > > Wanted to know if others have tried this or there are other things I can > > do. If not, i'l just go ahead and do this :-) > > > > > > Cheers, > > -Gautam. > > > > > > > > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi < > theo.berto...@gmail.com> > > wrote: > > > > > 94 and 98 differs in directory layout > > > so 98 is not able to read 94 layout unless you run the migration tool > > > which is basically moving all the data in a "default" namespace > directory > > > e.g. > > > /hbase/table -> /hbase/data/default/table > > > /hbase/.archive/table -> /hbase/archive/default/table > > > > > > Matteo > > > > > > > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam > wrote: > > > > > > > Yep, looks like the CDH distro backports HBASE-7987. Having said > that, > > is > > > > there a transition path for us or are we hosed :-) ? In general, > what's > > > the > > > > recommended way to achieve this, at this point I feel i'm going > around > > > the > > > > system to achieve what I want. If nothing else works with export > > snapshot > > > > should I just downgrade to 94, export snapshot and then upgrade to > 98? > > Is > > > > the upgrade migration path different from what export snapshot does > > (i'd > > > > imagine yes)? > > > > > > > > Cheers, > > > > -Gautam. > > > > > > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu wrote: > > > > > > > > > bq. 98.1 on dest cluster > > > > > > > > > > Looking at the history for SnapshotManifestV1, it came with > > HBASE-7987 > > > > > which went to 0.99.0 > > > > > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam > > > wrote: > > > > > > > > > > > Hello, > > > > > > I'm trying to copy data between Hbase clusters on > > different > > > > > > versions. I am using : > > > > > > > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > > > > -chuser hbase > > > > > > -chgroup hadoop > > > > > > -snapshot msg_snapshot > > > > > > -mappers 50 > > > > > > -copy-from hftp://src-cluster:50070/hbase > > > > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > > > > > > > > > > Till now, based on various tips from the mailing list, I have > > > modified > > > > > the > > > > > > source cluster data dir paths to mimic the 98 convention > (archive, > > > > table > > > > > > data paths, etc). This helped in jumping some roadblocks but not > > all. > > > > > > > > > > > > This is what I see now : > > > > > > > > > > > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export > > > failed > > > > > > java.io.IOException > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.
Re: Copying data from 94 to 98 ..
Jerry, Can you elaborate on what you mean by "export table to hdfs"? I initially tried running the export on src cluster (-copy-to hdfs://dest/hbase ), it complains while trying to write the data to dest cluster (due to the hdfs protocol version mismatch). Then I tried running export on dest cluster (-copy-from hftp://src/hbase). On Mon, Sep 15, 2014 at 10:36 PM, Jerry He wrote: > While you continue on the snapshot approach, have you tried to Export the > table in 0.94 to hdfs, and then Import the data from hdfs to 0.98? > On Sep 15, 2014 10:19 PM, "Matteo Bertozzi" > wrote: > > > can you post the full exception and the file path ? > > maybe there is a bug in looking up the reference file. > > It seems to not be able to find enough data in the file... > > > > Matteo > > > > > > On Mon, Sep 15, 2014 at 10:08 PM, Gautam > wrote: > > > > > Thanks for the reply Matteo. > > > > > > This is exactly what I did. I modified the source cluster's dir > structure > > > to mimic that of the 98 cluster. I even got as far as it trying to look > > > through the reference files. > > > > > > I end up with this exception : > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > > java.io.IOException > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > at > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > ... > > > .. > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > at > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > > at > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > > .. > > > > > > > > > This and Ted's reply about HBASE-7987 leads me to believe that the > export > > > tool from my distro is in capable of working around the regionManifest > > file > > > requirement. I'm now left with the option of downgrading my dest > cluster > > to > > > 94, copying data and then upgrading using the upgrade migration tool. > > > Wanted to know if others have tried this or there are other things I > can > > > do. If not, i'l just go ahead and do this :-) > > > > > > > > > Cheers, > > > -Gautam. > > > > > > > > > > > > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi < > > theo.berto...@gmail.com> > > > wrote: > > > > > > > 94 and 98 differs in directory layout > > > > so 98 is not able to read 94 layout unless you run the migration tool > > > > which is basically moving all the data in a "default" namespace > > directory > > > > e.g. > > > > /hbase/table -> /hbase/data/default/table > > > > /hbase/.archive/table -> /hbase/archive/default/table > > > > > > > > Matteo > > > > > > > > > > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam > > wrote: > > > > > > > > > Yep, looks like the CDH distro backports HBASE-7987. Having said > > that, > > > is > > > > > there a transition path for us or are we hosed :-) ? In general, > > what's > > > > the > > > > > recommended way to achieve this, at this point I feel i'm going > > around > > > > the > > > > > system to achieve what I want. If nothing else works with export > > > snapshot > > > > > should I just downgrade to 94, export snapshot and then upgrade to > > 98? > > > Is > > > > > the upgrade migration path different from what export snapshot does > > > (i'd > > > > > imagine yes)? > > > > > > > > > > Cheers, > > > > > -Gautam. > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu > wrote: > > > > > > > > > > > bq. 98.1 on dest cluster > > > > > > > > > > > > Looking at the history for SnapshotManifestV1, it came with > > > HBASE-7987 > > > > > > which went to 0.99.0 > > > > > > > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > > > > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam > > > > > wrote: > > > > > > > > > > > > > Hello, > > > > > > > I'm trying to copy data between Hbase clusters on > > > different > > > > > > > versions. I am using : > > > > > > > > > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > > > > > -chuser hbase > > > > > > > -chgroup hadoop > > > > > > > -snapshot msg_snapshot > > > > > > > -mappers 50 > > > > > > > -copy-from hftp://src-cluster:50070/hbase > > > > > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > > > > > > > > > > > > > Till now, based on various tips from the mailing list, I have > > > > modified > > > > > > the > > > > > > > source cluster data dir paths to mimic the 98 convention > > (archive, > > > > > table > > > > >
Re: Copying data from 94 to 98 ..
ok, so you have an empty reference file under .snapshot/SnapshotName/... what you can do is find all the reference files and replace them with the version in /hbase/.archive you can find reference files because are in the form "name.regionName" (they are the only files with a dot in the middle) so.. something like cp /hbase/.archive/tableName/region/myfile.parent /hbase/.snapshot/snapshotName/region/myfile.parent Matteo On Mon, Sep 15, 2014 at 11:00 PM, Gautam wrote: > Jerry, > Can you elaborate on what you mean by "export table to hdfs"? I > initially tried running the export on src cluster (-copy-to > hdfs://dest/hbase ), it complains while trying to write the data to dest > cluster (due to the hdfs protocol version mismatch). Then I tried running > export on dest cluster (-copy-from hftp://src/hbase). > > On Mon, Sep 15, 2014 at 10:36 PM, Jerry He wrote: > > > While you continue on the snapshot approach, have you tried to Export the > > table in 0.94 to hdfs, and then Import the data from hdfs to 0.98? > > On Sep 15, 2014 10:19 PM, "Matteo Bertozzi" > > wrote: > > > > > can you post the full exception and the file path ? > > > maybe there is a bug in looking up the reference file. > > > It seems to not be able to find enough data in the file... > > > > > > Matteo > > > > > > > > > On Mon, Sep 15, 2014 at 10:08 PM, Gautam > > wrote: > > > > > > > Thanks for the reply Matteo. > > > > > > > > This is exactly what I did. I modified the source cluster's dir > > structure > > > > to mimic that of the 98 cluster. I even got as far as it trying to > look > > > > through the reference files. > > > > > > > > I end up with this exception : > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export > failed > > > > java.io.IOException > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > > ... > > > > .. > > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:115) > > > > at > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > > > .. > > > > > > > > > > > > This and Ted's reply about HBASE-7987 leads me to believe that the > > export > > > > tool from my distro is in capable of working around the > regionManifest > > > file > > > > requirement. I'm now left with the option of downgrading my dest > > cluster > > > to > > > > 94, copying data and then upgrading using the upgrade migration tool. > > > > Wanted to know if others have tried this or there are other things I > > can > > > > do. If not, i'l just go ahead and do this :-) > > > > > > > > > > > > Cheers, > > > > -Gautam. > > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi < > > > theo.berto...@gmail.com> > > > > wrote: > > > > > > > > > 94 and 98 differs in directory layout > > > > > so 98 is not able to read 94 layout unless you run the migration > tool > > > > > which is basically moving all the data in a "default" namespace > > > directory > > > > > e.g. > > > > > /hbase/table -> /hbase/data/default/table > > > > > /hbase/.archive/table -> /hbase/archive/default/table > > > > > > > > > > Matteo > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam > > > wrote: > > > > > > > > > > > Yep, looks like the CDH distro backports HBASE-7987. Having said > > > that, > > > > is > > > > > > there a transition path for us or are we hosed :-) ? In general, > > > what's > > > > > the > > > > > > recommended way to achieve this, at this point I feel i'm going > > > around > > > > > the > > > > > > system to achieve what I want. If nothing else works with export > > > > snapshot > > > > > > should I just downgrade to 94, export snapshot and then upgrade > to > > > 98? > > > > Is > > > > > > the upgrade migration path different from what export snapshot > does > > > > (i'd > > > > > > imagine yes)? > > > > > > > > > > > > Cheers, > > > > > > -Gautam. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu > > wrote: > > > > > > > > > > > > > bq. 98.1 on dest cluster > > > > > > > > > > > > > > Looking at the history for SnapshotManifestV1, it came with > > > > HBASE-7987 > > > > > > > which went to 0.99.0 > > > > > > > > > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > > > > > > > > > On Mon, S
