I am sorry for not making a point for my question
I used a Global Dictionary cloumn when I build a cube .but I can't find
anything with this Global Dictionary infomation related with my cube in HBase
table 'kylin_metadata'
I searched all rowkeys in HBase table 'kylin_metadata' and filter out rowkeys
prefix with '/cube' ,'/model_desc' ,'/table' related with my cube name or my
model name
Nothing with Global Dictionary infomation can be found.
than I searched HDFS files in directory /kylin/kylin_metadata and find
nothing sadly.
I deleted the Global Dictionary rowkey in HBase table before and I resume the
job to build this cube
in kylin webUI job throw Error tall me ' No resource found at --
/dict/KYLINLABEL.USER_INFO/ROWKEY/bbcf19eb-1af9-4b03-b64b-40a5d423bcd6.dict'
but /dict/KYLINLABEL.USER_INFO/ROWKEY/bbcf19eb-1af9-4b03-b64b-40a5d423bcd6.dict
has been deleted by me.
finally I accoding to the stack infomation and find code in engine-mr module
which is source code in kylin 1.6.0
in the class AbstracHadoopJob have a method
```
attachKylinPropsAndMatedata(CubeInstance cube,Configuration conf) throw
Exception{
MetadataManager metaMgr = MetadataManager.getInstance(cube.getConfig());
// write cube / model_desc / cube_desc / dict / table
ArrayList<String> dumpList = new ArrayList<String>();
dumpList.add(cube.getResourcePath());
dumpList.add(cube.getDescriptor().getModel().getResourcePath());
dumpList.add(cube.getDescriptor().getResourcePath());
for (String tableName : cube.getDescriptor().getModel().getAllTables())
{
TableDesc table = metaMgr.getTableDesc(tableName);
dumpList.add(table.getResourcePath());
List<String> dependentResources =
SourceFactory.getMRDependentResources(table);
dumpList.addAll(dependentResources);
}
for (CubeSegment segment : cube.getSegments()) {
dumpList.addAll(segment.getDictionaryPaths());
}
attachKylinPropsAndMetadata(dumpList, cube.getConfig(), conf);
}
```
the problem is happend with dumpList which contains Global Dictionary paths ,so
I try to understand how dumpList be filled with data. thanks to your
annotations .I searched all rowkeys in HBase table 'kylin_metadata'and filter
out that which starts with prefix '/cube' or '/model_desc' or '/cube_desc' or
'/table' and so on .but finally I could not find any infomation with my cube
used Global Dictionary
I don't kown whether or not it clearly with my question?
[email protected]
From: ShaoFeng Shi
Date: 2017-01-06 09:53
To: dev
CC: user
Subject: Re: where kylin global dict store?
Kylin metadata is stored in HBase by default; but if a file is too large, it
will be saved in HDFS in the "/kylin/kylin_metadata" folder; You can do a
search there.
2017-01-06 8:52 GMT+08:00 zy <[email protected]>:
I think you could find the more content about kylin from the internet..i give
you as follow..
http://blog.csdn.net/yu616568/article/details/50570536
http://www.mamicode.com/info-detail-1015006.html
http://www.chinahadoop.cn/search?q=kylin
------------------ Original ------------------
From: "[email protected]"<[email protected]>;
Date: 2017年1月5日(星期四) 晚上11:21
To: "dev"<[email protected]>; "user"<[email protected]>;
Subject: where kylin global dict store?
Hi all :
I wan't to kown where kylin global dictionary stroe?
I found my cube infomation in HBase kylin table.for example:
in HBase console shell
get "kylin_metadata","/cube/dmp_user_info_cube.json"
i Get this (this data is formated )
-----------------------------------------------------------
{
"uuid": "f5db317d-32a9-4b14-8c56-837d3f9a3fae",
"last_modified": 1483607960443,
"version": "1.6.0",
"name": "dmp_user_info",
"owner": "ADMIN",
"descriptor": "dmp_user_info",
"cost": 50,
"status": "DISABLED",
"segments": [
{
"uuid": "9b187fdb-28ca-48f0-a0df-fe68cef05981",
"name": "20161001000000_20161220000000",
"storage_location_identifier": "KYLIN_3WV3VKHD99",
"date_range_start": 1475280000000,
"date_range_end": 1482192000000,
"source_offset_start": 0,
"source_offset_end": 0,
"status": "NEW",
"size_kb": 0,
"input_records": 0,
"input_records_size": 0,
"last_build_time": 0,
"last_build_job_id": null,
"create_time_utc": 1483607736083,
"cuboid_shard_nums": { },
"total_shards": 0,
"blackout_cuboids": [ ],
"binary_signature": null,
"dictionaries": {
"KYLINLABEL.USER_INFO/SEX":
"/dict/KYLINLABEL.USER_INFO/SEX/e01c7973-7dd7-411a-9e6d-79613e4fb2c9.dict",
"KYLINLABEL.USER_INFO/BIRTHDAY":
"/dict/KYLINLABEL.USER_INFO/BIRTHDAY/39270014-1c39-4d18-905d-43935d658a41.dict",
"KYLINLABEL.USER_INFO/NATIVE_PLACE":
"/dict/KYLINLABEL.USER_INFO/NATIVE_PLACE/c964fe32-66e9-4fa3-9902-fad28167b46d.dict",
"KYLINLABEL.USER_INFO/LOCATION":
"/dict/KYLINLABEL.USER_INFO/LOCATION/2155116f-6bd0-44cb-b14c- .dict"
},
"snapshots": null,
"index_path": null,
"rowkey_stats": [
[
"SEX",
3,
1
],
[
"BIRTHDAY",
9193,
2
],
[
"NATIVE_PLACE",
2268,
2
],
[
"LOCATION",
15,
1
]
]
}
],
"create_time_utc": 1483605652669,
"size_kb": 0,
"input_records_count": 0,
"input_records_size": 0
}
-------------------------------------
I find all thing begin with /cube or /cube_desc or /model_desc or /table or
/table_ext related to my cube and I can't find anything with Global Dictionary
with my cube
Help me !
[email protected]
--
Best regards,
Shaofeng Shi 史少锋
