>> This means adding a new built-in stack which uses the Bigtop
>> repository, and not developing Mpack on the Bigtop side, right?
>> If so, "bigtop mpack" sounds confusing a bit.
Looks you are confused with bigtop stack in ambari with bigtop mpack. What I
mean is, As bigtop mpack which is currently in-progress in the bigtop community
will take some more time develop and mature it, which can be used to trunk.
>>Sounds good to me. During the 2.7.x releases, users can use the HDP
>>stack by default. In addition, they can also add the Bigtop stack
>>through the Mpack, as Zhiguo mentioned in [1].
>>Since 2.8 or 3.0, we will drop HDP and users can use the built-in
>>Bigtop stack by default. It also allows us to drop the Mpack on the
>>Bigtop side.
Yes, this what mentioned. Bigtop mpack will be long term approach which we'll
work together with bigtop community and will drop HDP.
>> Regarding the built-in Bigtop stack developed on trunk, the current
>> BIGTOP stack in Ambari [2] may be a good start point to be upgraded.
>> I ensured that Ambari 2.7.6 was successfully built with the
>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too old to
>> use for now.
Yes, this can be. But Existing cluster upgrades can be challenge right.(where
those will be deployed with HDP-2.6.5,HDP-3.1.0 and HDP-3.1.5).?
On 20/06/22, 8:02 AM, "Kengo Seki" <sek...@apache.org> wrote:
Thank you so much for writing the draft and commenting on it, Brahma and
Viraj!
Firstly, a minor comment:
> Trunk Code : Parallel we can work for the trunk code, this we might give
as 2.8.
> * Working on bigtop mpack
This means adding a new built-in stack which uses the Bigtop
repository, and not developing Mpack on the Bigtop side, right?
If so, "bigtop mpack" sounds confusing a bit.
> hdp-select can stay on branch-2.7 and it can be moved to bigtop for trunk
branch. Thoughts?
Sounds good to me. During the 2.7.x releases, users can use the HDP
stack by default. In addition, they can also add the Bigtop stack
through the Mpack, as Zhiguo mentioned in [1].
Since 2.8 or 3.0, we will drop HDP and users can use the built-in
Bigtop stack by default. It also allows us to drop the Mpack on the
Bigtop side.
Regarding the built-in Bigtop stack developed on trunk, the current
BIGTOP stack in Ambari [2] may be a good start point to be upgraded.
I ensured that Ambari 2.7.6 was successfully built with the
`-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too old to
use for now.
[1]:
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cd8a59968d77942985dab08da52651626%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637912891362493977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9HBlpQcv%2Bun2VrDGL%2F21MJTMIJz2%2Bs7ufkfFytIorPE%3D&reserved=0
[2]:
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cd8a59968d77942985dab08da52651626%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637912891362493977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nqRlKJ%2F%2FyTcqkmWk%2FDdYCmApMtIa0izybsnw26lPb%2BQ%3D&reserved=0
Kengo Seki <sek...@apache.org>
On Mon, Jun 20, 2022 at 9:26 AM Viraj Jasani <vjas...@apache.org> wrote:
>
> Thanks for the nice draft, Brahma!
>
> IMHO, as we narrow down to 2.7.7 release, it might be still worth sticking
> to HDP i.e. plugin the new stack for HDP for 2.7 release line. For 2.8.0
> (or 3.0.0), we can add entire new stack (BDP sounds good one) and start
off
> with 2.8.0/3.0.0 stack version.
>
> hdp-select can stay on branch-2.7 and it can be moved to bigtop for trunk
> branch. Thoughts?
>
> Similarly, we can start-off with "python 3 only" support in trunk, whereas
> 2.7 release line can continue with python 2 to reduce complexities for
> maintenance releases on 2.7.
>
> All other points look good. Once again, thank you for drafting the roadmap
> and starting with this thread.
>
>
> On Sun, Jun 19, 2022 at 10:07 AM Battula, Brahma Reddy
> <bbatt...@visa.com.invalid> wrote:
>
> > Hi All,
> >
> > As Ambari back now, let’s start discussing roadmap. I will be closely
> > tracking for Jira, GitHub(with Jenkins) and wiki to work normal.
> > Based on the conclusion of this thread will update wiki[1].
> >
> > As 2.7.6 is released recently, we need to release 2.7.7
> >
> >
> > 1. Stack Management. There are two approaches for this. Open to
discuss
> > on following..
> > * Entirely new stack
> >
> > i.
> > Define new stack name. (BDP: BigDataPlatform, BDS:BigDataStack,
> > OSS:OpenSourceStack,TDP: TuskerDataPlatform…)
> >
> > ii.
> > Define stack versions
> > (zk-3.5.9,Hadoop-3.2.3,hive-3.1.3,tez-0.10.1,spark-3.2..)
> >
> > iii.
Based
> > above two we need to have select in bigtop code.
> >
> > * Using the existing for current release.
> >
> > i.
> > Plugin the new stack into HDP self
> >
> > ii.
> > Still, we can have hdp select
> >
> >
> >
> > I prefer the planA but planB takes less time. Users can easily adopt it
> > upgrade their stack.
> >
> >
> >
> > Note: Ideally bigtop mpack can be long term approach as stack id
decoupled
> > now, but currently it’s not ready for latest stack (upgrades can be
risk).
> > We’ll work in trunk for same. If this works shortly, we can choose.
> >
> >
> >
> > 1. Upgrade :
> > * Most of users will be deployed with HDP
> > versions(HDP-2.6.5,HDP-3.1.0), we might need to support the upgrade to
2.7.7
> > 2. Other work
> > * Any critical/blocker CVE’s Fixes
> > * CVEs Fixes : we’ll try to target some CVE fixes
> > * Review the open PR’s and Jira’s
> >
> >
> >
> > Trunk Code : Parallel we can work for the trunk code, this we might give
> > as 2.8.
> >
> > * Removal of python2 support
> > * Removal of hard dependency on HDP
> > * Working on bigtop mpack
> >
> > Can add more based on your inputs…
> >
> >
> > Any thoughts on this..? Please feel free to add anything which I missed.
> >
> >
> >
> > Reference:
> >
> > 1.
> >
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fpages%2Fviewpage.action%3FpageId%3D30755705&data=05%7C01%7Cbbattula%40visa.com%7Cd8a59968d77942985dab08da52651626%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637912891362493977%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6leb0PfMOrmqscqUjV%2FUdc8nhJOEGogKUFrsg7XR7AU%3D&reserved=0
> >
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org
For additional commands, e-mail: dev-h...@ambari.apache.org
