+1 the new Kylin version should target Hadoop 3
Best Regards!
-
Luke Han
On Mon, Mar 2, 2020 at 4:41 PM Billy Liu wrote:
> +1. Let's move to Hadoop3
>
> With Warm regards
>
> Billy Liu
>
>
> ShaoFeng Shi 于2020年2月27日周四 下午10:07写道：
>
>> Hi Yang,
>>
>> The main difference between 2.6 and 3.0 is the new real-time OLAP
>> feature. Hadoop 2 users can select either of them, depends on whether they
>> need the real-time feature.
>>
>> After 3.0, the next major features would be the Flink cube engine
>> (planned in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>>
>> When the parquet storage is released, as the dependency on HBase can be
>> dropped, then we assume the API issue will easier than today. We can
>> re-evaluate the possibility to support Hadoop 2.
>>
>> So I think the impact on today's Hadoop 2 users is acceptable. Not
>> mention that they still can manually compile that.
>>
>>
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofeng...@apache.org
>>
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscr...@kylin.apache.org
>> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>>
>>
>>
>>
>> Li Yang 于2020年2月27日周四 上午7:37写道：
>>
>>> The proposal means Kylin 3.0 will be the last major version that supports
>>> Hadoop 2.
>>>
>>> What will be recommended version for Hadoop 2 users after this? I feel
>>> the
>>> latest stable version of 2.6 is better than 3.0.
>>>
>>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>>> However we shall also think about what it means for Hadoop 2 users.
>>> Questions like below shall also be answered.
>>>
>>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>>> sound right here.)
>>> - How that version/branch will be maintained?
>>>
>>> +1 in general
>>>
>>> Regards
>>> -Yang
>>>
>>>
>>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang
>>> wrote:
>>>
>>> > +1
>>> >
>>> >
>>> > > 2020年2月26日 下午3:48，ShaoFeng Shi 写道：
>>> > >
>>> > > Hello, Kylin users and developers,
>>> > >
>>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of
>>> HBase
>>> > 1 and 2 are incompatible, we need to keep different branches for them.
>>> And
>>> > in each release, we need to build separate packages and do a round of
>>> > testing for them separately. Furthermore, Cloudera's API difference
>>> with
>>> > the Apache release makes the situation worse; We need to build 4 binary
>>> > packages for reach release. That has spent much of our manual effort
>>> and
>>> > computing resources.
>>> > >
>>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>>> > production use; And we see more and more users are starting to use the
>>> new
>>> > versions. We think it is time for Kylin to totally upgrade to the new
>>> > version. So that we can focus more on Kylin itself, instead of
>>> environments.
>>> > >
>>> > > Here is my proposal:
>>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a
>>> close
>>> > version);
>>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous
>>> releases;
>>> > > 3) We will re-evaluate the need for building binary packages for
>>> > Cloudera release. (we may raise another discuss)
>>> > >
>>> > > Please let us know your comments. And please also understand with the
>>> > limited resource we couldn't support multiple Hadoop versions...
>>> > >
>>> > > Thanks!
>>> > >
>>> > > Best regards,
>>> > >
>>> > > Shaofeng Shi 史少锋
>>> > > Apache Kylin PMC
>>> > > Email: shaofeng...@apache.org
>>> > >
>>> > > Apache Kylin FAQ:
>>> https://kylin.apache.org/docs/gettingstarted/faq.html
>>> > > Join Kylin user mail group: user-subscr...@kylin.apache.org
>>> > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>>> > >
>>> > >
>>> >
>>> >
>>>
>>
