I'm updating the document for modifying metadata in a selective way, but
the jenkins has a problem today. Here is the to-add part in
https://kylin.apache.org/docs/howto/howto_backup_metadata.html:
## Restore metadata selectively (Recommended)
If only changes a couple of metadata files, the administrator can just pick
these files to restore, without having to cover all the metadata. Compared
to the full recovery, this approach is more efficient, safer, so it is
recommended.
Create a new empty directory, and then create subdirectories in it
according to the location of the metadata files to restore; for example, to
restore a Cube instance, you should create a "cube" subdirectory：
{% highlight Groff markup %}
mkdir /path/to/restore_new
mkdir /path/to/restore_new/cube
{% endhighlight %}
Copy the metadata file to be restored to this new directory:
{% highlight Groff markup %}
cp meta_backups/meta_2016_06_10_20_24_50/cube/kylin_sales_cube.json
/path/to/restore_new/cube/
{% endhighlight %}
At this point, you can modify/fix the metadata manually.
Restore from this directory:
{% highlight Groff markup %}
cd $KYLIN_HOME
./bin/metastore.sh restore /path/to/restore_new
{% endhighlight %}
Only the files in the folder will be uploaded to Kylin metastore.
Similarly, after the recovery is finished, click Reload Metadata button on
the Web UI to flush the cache.
kdcool6932 <kdcool6...@yahoo.com.invalid> 于2018年10月29日周一 下午7:54写道：
> Thanks guys,, really appreciate the prompt response.@ShaoFeng,Yes we have
> the data in hive(or we can load that if needed). And we will be rebuilding
> those segments. Do we have any mail thread or document or blog to refer to
> for manully editing and restoring metadata. That would actually be great
> help, as we often get into situations like this as we don't want to take
> risk for 3 plus years (120+TB) data in Hbase for Kylin.Again,, really
> appreciate the help provided.Thanks,Ketan Sent from my Samsung Galaxy
> smartphone.
> -------- Original message --------From: ShaoFeng Shi <
> shaofeng...@apache.org> Date: 29/10/2018 12:36 pm (GMT+05:30) To: dev <
> dev@kylin.apache.org> Subject: Re: Merge Job in inconsistent state It is
> a known issue; the auto-merge was triggered on each segment change.Maybe,
> Kylin should not trigger the auto-merge on canceling/deleting
> ajob/segment?But you can keep that error job/segment, it won't impact on
> the query. Theonly thing is an error job there.Do you know the root cause
> of "No input paths specified in job"? Did youdelete some folders from
> HDFS?If you have the source data in Hive, you can rebuild those segments;
> Youcan take a backup of the metadata, and then dump metadata in local
> disk,copy that cube's json to a clean folder, edit it to delete these
> segment,and then restore metadata from the clean folder (same structure,
> e.g/cube/yourcube.json). After restored, build the segments for the
> missingdate range.Chao Long <wayn...@qq.com> 于2018年10月29日周一 下午2:12写道：> Hi
> Ketan,> As this merge job is an automatically triggered job, so it start
> again> when you discard it. If you don't want this job to be triggered
> again, you> can remove the "Auto Merge" related configuration on the cube
> design page> until the problem is resolved or fixed(if it's a bug).>>>
> This is the merging Segment[20181005080000_20181012170000]. And the>
> exception occurred during the merging job, so it's in an incorrect
> state.> Segment: 20181005080000_20181012170000 //**This segment Table
> was> deleted from Hbase (somehow, we don’t have the reason)**//> Start
> Time: 2018-10-05 08:00:00> End Time: 2018-10-12 17:00:00> Source
> Count: 0> HBase Table: KYLIN_CFLY2CKMCU> Region Count: 3> Size:
> less than 1 MB>>> To identify the root cause, you may provide more log
> around the error> message.>>>>>> ------------------ 原始邮件
> ------------------> 发件人: "ketan dikshit"<kdcool6...@yahoo.com.INVALID>;>
> 发送时间: 2018年10月28日(星期天) 凌晨0:25> 收件人: "dev"<dev@kylin.apache.org>;>> 主题:
> Merge Job in inconsistent state>>>> Hi Team,> We are using Kylin 2.3.1, And
> in the merge Job(which gets automatically> triggered), we are getting this
> error in Merge Cuboid Data Step;>> java.io.IOException: No input paths
> specified in job> at>
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:239)>
> at>
> org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)>
> at>
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)>
> at org.apache.kylin.engine.mr>
> .common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:622)>
> at>
> org.apache.kylin.storage.hbase.steps.HBaseMROutput2Transition$HBaseMergeMROutputFormat.configureJobOutput(HBaseMROutput2Transition.java:166)>
> at org.apache.kylin.engine.mr>
> .steps.MergeCuboidJob.run(MergeCuboidJob.java:82)> at
> org.apache.kylin.engine.mr>
> .common.MapReduceExecutable.doWork(MapReduceExecutable.java:130)>
> at>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>
> at>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)>
> at>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>
> at>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)>
> at>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)>
> at>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)>
> at java.lang.Thread.run(Thread.java:745)>> result code:2>> As it is a merge
> job, whenever I discard this job,, it again starts> automatically.> So we
> need a way(if there is one), of permanently discarding this job> somehow.>>
> Also in case this can be done by changing the metadata for the cube>
> segment, do let me know how it can be done(even if I have to delete/remove>
> the segments for these below days)> We are open for all options (except
> dropping the cube as it has more than> 1 year data, and is crucial for
> organisation). We are stuck on this for> some days now, help would really
> be appreciated.>> Merge Thresholds:> 8 (Hours)> 1 (Days)> 7 (Days)> 15
> (Days)>> Merge Job, (start and end):> MERGE CUBE - XXXX -
> 20181005080000_20181012170000>> Here are list of segments;> Segment:
> 20181005080000_20181006080000> Start Time: 2018-10-05 08:00:00> End Time:
> 2018-10-06 08:00:00> Source Count: 14899048> HBase Table: KYLIN_R1MUK56K71>
> Region Count: 1> Size: 860 MB>> Segment: 20181005080000_20181012170000
> //**This segment Table was deleted> from Hbase (somehow, we don’t have the
> reason)**//> Start Time: 2018-10-05 08:00:00> End Time: 2018-10-12
> 17:00:00> Source Count: 0> HBase Table: KYLIN_CFLY2CKMCU> Region Count: 3>
> Size: less than 1 MB>> Segment: 20181006080000_20181008000000> Start Time:
> 2018-10-06 08:00:00> End Time: 2018-10-08 00:00:00> Source Count: 24455686>
> HBase Table: KYLIN_0KH6PHTEM2> Region Count: 1> Size: 1.0498 GB>> Segment:
> 20181008000000_20181009000000> Start Time: 2018-10-08 00:00:00> End Time:
> 2018-10-09 00:00:00> Source Count: 14882090> HBase Table: KYLIN_V1CC4LDSIR>
> Region Count: 1> Size: 598 MB>> Segment: 20181009000000_20181010000000>
> Start Time: 2018-10-09 00:00:00> End Time: 2018-10-10 00:00:00> Source
> Count: 16245847> HBase Table: KYLIN_4A44K2VJEU> Region Count: 1> Size: 628
> MB>> Segment: 20181010000000_20181010080000> Start Time: 2018-10-10
> 00:00:00> End Time: 2018-10-10 08:00:00> Source Count: 5213022> HBase
> Table: KYLIN_EHO316VC7M> Region Count: 1> Size: 397 MB>> Segment:
> 20181010080000_20181010090000> Start Time: 2018-10-10 08:00:00> End Time:
> 2018-10-10 09:00:00> Source Count: 865722> HBase Table: KYLIN_I9LEJ2JDZ8>
> Region Count: 1> Size: 181 MB>> Segment: 20181010090000_20181010100000>
> Start Time: 2018-10-10 09:00:00> End Time: 2018-10-10 10:00:00> Source
> Count: 859127> HBase Table: KYLIN_9IBX3W4UNL> Region Count: 1> Size: 180
> MB>> Segment: 20181010100000_20181010110000> Start Time: 2018-10-10
> 10:00:00> End Time: 2018-10-10 11:00:00> Source Count: 855752> HBase Table:
> KYLIN_HRDJ16B3O8> Region Count: 1> Size: 179 MB>> Segment:
> 20181010110000_20181010120000> Start Time: 2018-10-10 11:00:00> End Time:
> 2018-10-10 12:00:00> Source Count: 849363> HBase Table: KYLIN_6BFHFA5LU1>
> Region Count: 1> Size: 178 MB>> Segment: 20181010120000_20181010130000>
> Start Time: 2018-10-10 12:00:00> End Time: 2018-10-10 13:00:00> Source
> Count: 851162> HBase Table: KYLIN_H41KZXUIRN> Region Count: 1> Size: 177
> MB>> Segment: 20181010130000_20181010140000> Start Time: 2018-10-10
> 13:00:00> End Time: 2018-10-10 14:00:00> Source Count: 836481> HBase Table:
> KYLIN_8RXPI7T0PA> Region Count: 1> Size: 173 MB>> Segment:
> 20181010140000_20181010150000> Start Time: 2018-10-10 14:00:00> End Time:
> 2018-10-10 15:00:00> Source Count: 780337> HBase Table: KYLIN_7L3WHR3ZQY>
> Region Count: 1> Size: 164 MB>> Segment: 20181010150000_20181010160000>
> Start Time: 2018-10-10 15:00:00> End Time: 2018-10-10 16:00:00> Source
> Count: 723669> HBase Table: KYLIN_RM0ICHV5EP> Region Count: 1> Size: 155
> MB>> Segment: 20181010160000_20181011170000> Start Time: 2018-10-10
> 16:00:00> End Time: 2018-10-11 17:00:00> Source Count: 17476745> HBase
> Table: KYLIN_Y8ZOSIWNJP> Region Count: 1> Size: 941 MB>> Segment:
> 20181011170000_20181012170000> Start Time: 2018-10-11 17:00:00> End Time:
> 2018-10-12 17:00:00> Source Count: 15485276> HBase Table: KYLIN_RWTQZFY6J4>
> Region Count: 1> Size: 887 MB>> Thanks,> Ketan@Exponential-- Best
> regards,Shaofeng Shi 史少锋
--
Best regards,
Shaofeng Shi 史少锋
