[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253450#comment-16253450
]
Dmitry commented on KYLIN-2849:
-------------------------------
Duplicate segments can appear when something wrong happens with job. In our
case there were a job error. And after its discarding we got two segments with
the same names. The second segment has status = NEW:
{
"uuid": "4a6f9098-d95c-4cca-8865-6efe2fc1e2f8",
"name": "20171030000000_20171211000000",
"storage_location_identifier": "KYLIN_VTYLURM5N4",
"date_range_start": 1509321600000,
"date_range_end": 1512950400000,
"source_offset_start": 0,
"source_offset_end": 0,
"status": "NEW",
"size_kb": 0,
"input_records": 0,
"input_records_size": 0,
"last_build_time": 0,
"last_build_job_id": null,
"create_time_utc": 1510412403197,
"cuboid_shard_nums": {},
"total_shards": 0,
"blackout_cuboids": [],
"binary_signature": null,
"dictionaries": null,
"snapshots": null,
"rowkey_stats": []
}
And when we try to delete it, we get an error:
{{"msg": "Cannot delete segment '20171030000000_20171211000000' as its status
is not READY. Discard the on-going job for it.".}}
But there are no any jobs running. And there are no HBase table called
KYLIN_VTYLURM5N4. Also it's not possible to REFRESH this segment. It says about
Segments overlapping.
I guess it's related to KYLIN-2962 issue.
It would be great to be able to delete such segments from GUI or via RestfulAPI.
Billy Liu, what data should be removed from metadata to fix this issue?
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---------------------------------------------------------------------------
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
> Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
> Reporter: 翟玉勇
> Assignee: Dong Li
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/20170822000000_20170823000000";
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '20170822000000_20170823000000' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '20170822000000_20170823000000' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
