Re: Questions / Clarifications for 1.5.x

[ShaoFeng Shi]

Hi Abhilash, welcome back;

Kylin 1.5 delivers a new plugable architecture, with many enhancements
especially on performance, like fast cubing algorithm, sharded storage
engine, etc. The facts 1) to 4) are unchanged in between I think; For the
issues like memory usage, you can open JIRA to us or submit a patch if you
have solved it;

1) Using Kylin's rest API to trigger a cube rebuild/refresh when the data
is changed in Hive;
2) No change I think; JIRA is welcomed;
3) What do you mean by "reload" here? If new segment is ready and query
comes, its dim snapshots will be loaded; as the it is immutable, a
"reload"/"refresh" is not needed;
4) The cap on snapshot is still there, you can customize the max size in
kylin.properties;
5) When deploy into cluster, the query servers are identical in role, that
means all servers can server all cubes;
6) You can play the sample cube shipped with kylin binary, which is
partitioned by a date column, then you can try the incremental build
function: https://kylin.apache.org/docs15/tutorial/kylin_sample.html



2016-06-29 13:56 GMT+08:00 Abhilash L L :

> Hello,
>
>We plan to upgrade to Kylin 1.5.x series from 1.2 series. We want to ask
> the team a few questions / clarification before proceeding. Since we are
> looking at upgrading, please do let us know the behaviour for the latest
> 1.5.x release.
>
>
> 1) Updating facts
> Eg:
> Lets say im building a cube for jira tickets. List of tickets is my fact
> table. List of ticket status is my dimension table. Now if a ticket is
> updated from 'open' to 'in progress' how to tell kylin this change ?
>
> 2) Multiple dimensions in memory
> When we were on 1.2 versions, all the dimensions were kept in memory. So
> memory always keeps increasing. The in memory dimensions were not swapped
> in / out, causing tomcat OOM issues. Is this still the same in 1.5.x ?
>
> 3) Do these dimension snapshots get reloaded on a cube refresh / data
> appended ?
>
> 4) Maximum size for dimension
> There was an older thread talking about 300mb max snapshot size. Does this
> limitation still hold? Do very high cardinality dimension still build the
> trees on one node ?
>
>
> 5) Multiple query servers
> When there are multiple query servers, will one query server serve only one
> cube or all servers serve all cubes. If its one cube only in one server,
> how does kylin handle a server going down.
>
>
> 6) Incremental build
> Is there a document on how incremental build works ?  We want to understand
> limitations / assumptions for this.
>
> Regards,
> Abhilash
>



-- 
Best regards,

Shaofeng Shi

Questions / Clarifications for 1.5.x

[Abhilash L L]

Hello,

   We plan to upgrade to Kylin 1.5.x series from 1.2 series. We want to ask
the team a few questions / clarification before proceeding. Since we are
looking at upgrading, please do let us know the behaviour for the latest
1.5.x release.


1) Updating facts
Eg:
Lets say im building a cube for jira tickets. List of tickets is my fact
table. List of ticket status is my dimension table. Now if a ticket is
updated from 'open' to 'in progress' how to tell kylin this change ?

2) Multiple dimensions in memory
When we were on 1.2 versions, all the dimensions were kept in memory. So
memory always keeps increasing. The in memory dimensions were not swapped
in / out, causing tomcat OOM issues. Is this still the same in 1.5.x ?

3) Do these dimension snapshots get reloaded on a cube refresh / data
appended ?

4) Maximum size for dimension
There was an older thread talking about 300mb max snapshot size. Does this
limitation still hold? Do very high cardinality dimension still build the
trees on one node ?


5) Multiple query servers
When there are multiple query servers, will one query server serve only one
cube or all servers serve all cubes. If its one cube only in one server,
how does kylin handle a server going down.


6) Incremental build
Is there a document on how incremental build works ?  We want to understand
limitations / assumptions for this.

Regards,
Abhilash