well,the timestamp column was bigint such as 1507547479434 in hive table, when
I define the endtime to build the cube ,I found the timestamp 1507547479434 was
converted to '20171009' and the log show that kylin loaded data from hive with
condition "WHERE (USER_REG.REG_TIME < 20171009)",so the Intermediate Flat
Hive Table was null. I want to know could kylin derive other time values like
“year_start”, “day_start” from the bigint timestamp in hive as it does in kafka
table? or we must change the bigint timestamp into data format such as
"2017-10-09" in hive?
At 2017-10-09 22:04:56, ShaoFeng Shi <[email protected]> wrote:
>Hi Miao,
>
>What's the error as you said: "kylin failed to load data from hive tables"?
>
>In my opinion, it is not recommended to use timestamp as the partition
>column, since it is too fine granularity. Usually, the cube is partitioned
>by day/week/month; in some cases, it is by the hour; In streaming case, it
>might partition by the minute; But no case by timestamp. I put some
>comments about this in this document:
>https://kylin.apache.org/docs21/tutorial/cube_streaming.html
>
>2017-10-09 14:27 GMT+08:00 崔苗 <[email protected]>:
>
>> Hi,
>> we want to use tables in kafka as fact tables and tables in MySql as
>> lookup tables,so we put all the tables into hive and want to join them as
>> cubes.
>>
>> the time column in fact table was timestamp, so does kylin2.1 support
>> timestamp for cube partition?
>> I find this :https://issues.apache.org/jira/browse/KYLIN-633 ,
>>
>> it seems kylin already supprt Timestamp for cube partition,but when we
>> define timestamp as partition , kylin failed to load data from hive tables.
>>
>>
>> thanks in advanced for your reply.
>>
>>
>>
>>
>>
>
>
>--
>Best regards,
>
>Shaofeng Shi 史少锋
