Hi Manjeet, I am sorry that I misunderstood your question.
The hbase book http://hbase.apache.org/book.html#_upgrade_paths describes: "You must stop your cluster, install the 1.x.x software, run the migration described at Executing the 0.96 Upgrade <http://hbase.apache.org/book.html#executing.the.0.96.upgrade> (substituting 1.x.x. wherever we make mention of 0.96.x in the section below), and then restart. Be sure to upgrade your ZooKeeper if it is a version less than the required 3.4.x." This is what I mean about "the environment ready for the upgrade". Since the HBase 1.2.1 cluster is the all new cluster, this is the different situation from yours. If the contents of /data/ExportedFiles have been put to the HDFS on HBase 1.2.1 cluster, try the below command: sudo -u hdfs hbase -Dhbase.import.version=0.94 org.apache.hadoop.hbase.mapreduce.Import test_table /data/ExportedFiles instead of yours. Best Regards. 2017-10-26 13:27 GMT+08:00 Manjeet Singh <[email protected]>: > Furthermore, clarity why I used scp command is : > > I have copy source cluster files to destination cluster by using scp > command and put them into destination cluster HDFS (It's because of two > different version of Haddop destination cluster hadoop is 1.2.1 and > destination is having Hadoop 2.0 ) First I get HDFS files to local linux > and use scp command to put them into destination cluster. > > Thanks > Manjeet Singh > > On Thu, Oct 26, 2017 at 10:26 AM, Manjeet Singh < > [email protected]> > wrote: > > > Hi Yung, > > > > First thanks for reply > > The link provided by you is for upgrading the Hbase version and problem > > statement is different > > Problem is when I am trying to export hbase data from one cluster to > > another cluster in same N/W, but with a different hbase version i.e. > > 0.94.27 (source cluster hbase) and another is destination cluster hbase > > version is 1.2.1. > > So this link shall be refer > > http://hbase.apache.org/0.94/book/ops_mgt.html#export > > > > > > for the second point which I forget to mention in mail, I did copy > contents > > of /data/ExportedFiles > > in destination cluster which is having HBase 1.2.1 but not with > > distcp instead of I used scp command > > and when I am trying to import data I am getting below error > > > > 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : > > attempt_1505781444745_0070_m_000003_0, Status : FAILED > > Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read > 121347 > > at org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue( > > SequenceFile.java:2306) > > at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordRead > > er.nextKeyValue(SequenceFileRecordReader.java:78) > > at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nex > > tKeyValue(MapTask.java:556) > > at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue > > (MapContextImpl.java:80) > > at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.ne > > xtKeyValue(WrappedMapper.java:91) > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask. > java:787) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:422) > > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro > > upInformation.java:1693) > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > > > > > > > > can you please elaborate more about "Is the environment ready for the > > upgrade?" > > > > Thanks > > Manjeet Singh > > > > > > > > On Thu, Oct 26, 2017 at 8:32 AM, Yung-An He <[email protected]> wrote: > > > >> Hi, > >> > >> Have you seen the reference guide > >> <http://hbase.apache.org/book.html#_upgrade_paths> to make sure that > the > >> environment is ready for the upgrade? > >> Perhaps you could try to copy the contents of /data/ExportedFiles to the > >> HBase 1.2.1 cluster using distcp before import data instead of using > >> "hdfs://<IP>:8020/data/ExportedFiles" directly. > >> Then create the table on the HBase 1.2.1 cluster using HBase Shell. > Column > >> families must be identical to the table on the old one. > >> Finally, import data from /data/ExportedFiles on the HBase 1.2.1 > cluster. > >> > >> > >> Best Regards. > >> > >> 2017-10-24 1:27 GMT+08:00 Manjeet Singh <[email protected]>: > >> > >> > Hi All, > >> > > >> > Can anyone help? > >> > > >> > adding few more investigations I have move all files to the > destination > >> > cluster hdfs and I have run below command:- > >> > > >> > sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import > test_table > >> > hdfs://<IP>:8020/data/ExportedFiles > >> > > >> > I am getting below error > >> > > >> > 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : > >> > attempt_1505781444745_0070_m_000003_0, Status : FAILED > >> > Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read > >> 121347 > >> > at > >> > org.apache.hadoop.io.SequenceFile$Reader. > getCurrentValue(SequenceFile. > >> > java:2306) > >> > at > >> > org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader. > >> > nextKeyValue(SequenceFileRecordReader.java:78) > >> > at > >> > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader. > >> > nextKeyValue(MapTask.java:556) > >> > at > >> > org.apache.hadoop.mapreduce.task.MapContextImpl. > >> > nextKeyValue(MapContextImpl.java:80) > >> > at > >> > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context. > >> > nextKeyValue(WrappedMapper.java:91) > >> > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) > >> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask. > java: > >> 787) > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > >> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild. > java:164) > >> > at java.security.AccessController.doPrivileged(Native Method) > >> > at javax.security.auth.Subject.doAs(Subject.java:422) > >> > at > >> > org.apache.hadoop.security.UserGroupInformation.doAs( > >> > UserGroupInformation.java:1693) > >> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java: > 158) > >> > > >> > > >> > > >> > > >> > can anyone suggest how to migrate data? > >> > > >> > Thanks > >> > Manjeet Singh > >> > > >> > > >> > > >> > > >> > > >> > Hi All, > >> > > >> > I have query regarding hbase data migration from one cluster to > another > >> > cluster in same N/W, but with a different version of hbase one is > >> 0.94.27 > >> > (source cluster hbase) and another is destination cluster hbase > version > >> is > >> > 1.2.1. > >> > > >> > I have used below command to take backup of hbase table on source > >> cluster > >> > is: > >> > ./hbase org.apache.hadoop.hbase.mapreduce.Export SPDBRebuild > >> > /data/backupData/ > >> > > >> > below files were genrated by using above command:- > >> > > >> > > >> > drwxr-xr-x 3 root root 4096 Dec 9 2016 _logs > >> > -rw-r--r-- 1 root root 788227695 Dec 16 2016 part-m-00000 > >> > -rw-r--r-- 1 root root 1098757026 Dec 16 2016 part-m-00001 > >> > -rw-r--r-- 1 root root 906973626 Dec 16 2016 part-m-00002 > >> > -rw-r--r-- 1 root root 1981769314 Dec 16 2016 part-m-00003 > >> > -rw-r--r-- 1 root root 2099785782 Dec 16 2016 part-m-00004 > >> > -rw-r--r-- 1 root root 4118835540 Dec 16 2016 part-m-00005 > >> > -rw-r--r-- 1 root root 14217981341 Dec 16 2016 part-m-00006 > >> > -rw-r--r-- 1 root root 0 Dec 16 2016 _SUCCESS > >> > > >> > > >> > in order to restore these files I am assuming I have to move these > >> files in > >> > destination cluster and have to run below command > >> > > >> > hbase org.apache.hadoop.hbase.mapreduce.Import <tablename> > >> > /data/backupData/ > >> > > >> > Please suggest if I am on correct direction, second if anyone have > >> another > >> > option. > >> > I have tryed this with test data but above command took very long time > >> and > >> > at end it gets fails > >> > > >> > 17/10/23 11:54:21 INFO mapred.JobClient: map 0% reduce 0% > >> > 17/10/23 12:04:24 INFO mapred.JobClient: Task Id : > >> > attempt_201710131340_0355_m_000002_0, Status : FAILED > >> > Task attempt_201710131340_0355_m_000002_0 failed to report status for > >> 600 > >> > seconds. Killing! > >> > > >> > > >> > Thanks > >> > Manjeet Singh > >> > > >> > > >> > > >> > > >> > > >> > > >> > -- > >> > luv all > >> > > >> > > > > > > > > -- > > luv all > > > > > > -- > luv all >
