I agree with "1. Fix some security problems by upgrading Spring Framework and others".Security is important and we can use this as a ramp-up as the community is gradually bringing everything in Ambari back on track, like PR review, CI jobs,...
For other things like "frontend refactor", it would be good to have more inputs on existing issues/limitations before we decide to "refactor". For the mpack part, my understanding is that "management packs" is a framework that allows different stack vendors to provide a compatible stack that can be maintained and operated by Ambari. There could be minor differences between different vendors. So the definition like path/repo/name etc should be the vendor's decision instead of being defined by Ambari. I would suggest leaving the mpack related work to a certain community, for example, Bigtop, and Ambari just review and accept their PR just for the user's convenience. Just my 2 cents. And happy to learn any comments. 吴治国 <wzg547228...@163.com> 于2022年8月16日周二 17:50写道： > Hi community, > > In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5 > for users at first, and keep all big changes in Ambari2.8.0, e.g. > 1、Fix some security problems by upgrading Spring Framework and others > 2、Frontend refactor > 3、Upgrade python version > etc. > > All these changes will take long time, which may be bad for the release > cycle of Ambari itself. > So I'm considering we can move Mpack development to Ambari side, and 2.8.0 > is all about Bigtop stack, move other big changes to next major > version(except the security problem, which absolutely need to be fixed in > next release). > > But we need to make final decision for the things we discussed before, > like, > 1、Add prefix to installation path(/usr/bigtop) > 2、Which repo to store the packages(with prefix in installation path)? > 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select, > bigtop-select are all ok for me)? > Any other things I forgot? > > What’s your opinion? > > Best Regards, > Zhiguo Wu > > On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote: > > > > IMO, First way is not heavier when we consider all the efforts for mpack > with upgrades. And first one is easiest to adopt now. > > > > We'll start discussing the bigtop mailing list further. > > > > > > On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote: > > > > Thanks for the details, Brahma and Kengo. > > > > If I understand it correctly, there are two approaches under > discussion: > > > > 1、Prepare to release 2.7.7, in this version, we won't use HDP > components anymore, we'll replace it with the community version provided by > Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the > installation path so hdp-select can be reused. (Due to component version > changes, I don't know if the code under /stacks needs to be updated) > > 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change > Ambari's code (if the code under /stacks can be reused in the first way, > maybe it also works in this way?) > > > > Isn't the workload of the first way heavier? > > > > > > About the meeting > > > > I thought maybe email discussion would be better? > > The discussion can be open for a few days and all interested people > in the community can participate, not just a few of us. > > And due to time zone and language issues, many people are hard to > attending the meetings, email discussions may be more suitable for these > people. > > > > What do you think? > > > > Best Regards, > > Zhiguo Wu > > > > > On Jun 20, 2022, at 23:15, Battula, Brahma Reddy > <bb...@visa.com.INVALID> wrote: > > > > > >>> Assuming the understanding above is correct, my new concerns are: > > > > > > Yes, Hope we'll in same page now. Still we can discuss in weekly > call. > > > > > >>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's > trademark? > > >>> If so, should we give a new stack name to it, as you mentioned as > > >>> planA in [3]? > > >>> (or it makes upgrading HDP cluster difficult? I don't understand > it so much) > > > > > > Yes, it may be. Hence I introduced the planA and preferred it. one > more idea(bad) came in my mind that can we treat "HDP: Hadoop Data > Platform", not sure whether it should be ok or not(and we might not > delivering as distro's). > > > > > >>> * It may be a bit tough work to fix Bigtop's build scripts so > that its > > >>> artifacts match Ambari's requirements. > > >>> (package versioning, file layouts, included and not included > files, > > >>> file permissions, user/group creation, etc.) > > > > > > Only paths matter right everything else can be re-used from the > hdp-select. > > > > > > > > > On 20/06/22, 8:13 PM, "Kengo Seki" <sek...@apache.org < > ma...@apache.org>> wrote: > > > > > > Thanks for the elaboration Brahma, but I'm a bit confused. > > > At this point, my understanding is as follows. Are these correct? > > > > > > * In the 2.7.x line, we'll still use a stack definition based on > HDP's one [1]. > > > > > > * But regarding RPM and DEB packages, we'll build them from source > by leveraging > > > Bigtop's build scripts and publish them on the ASF distribution > site > > > or somewhere. > > > repoinfo.xml [2] will also be fixed to point that repository. > > > > > > Assuming the understanding above is correct, my new concerns are: > > > > > > * Can we still call it as "HDP"? Doesn't it infringe Cloudera's > trademark? > > > If so, should we give a new stack name to it, as you mentioned as > > > planA in [3]? > > > (or it makes upgrading HDP cluster difficult? I don't understand > it so much) > > > > > > * It may be a bit tough work to fix Bigtop's build scripts so that > its > > > artifacts match Ambari's requirements. > > > (package versioning, file layouts, included and not included files, > > > file permissions, user/group creation, etc.) > > > I'm not sure if it's faster than developing Mpack. > > > > > > [1]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0 > > > > > [2]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0 > > > > > [3]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0 > > > > > > > > Kengo Seki <sek...@gmail.com <ma...@gmail.com>> > > > > > > On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy > > > <bbatt...@visa.com.invalid <ma...@visa.com.invalid>> wrote: > > >> > > >> We'll not use the HDP Packages (Only selects we'll use from hdp, > which can be plugged in bigtop). Still we'll use opensource packages and > bigtop to create the RPM's and selects. > > >> Just we can use the source tar balls opensource components(e.g > Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish > in following repo. Or from the apache github repo's. > > >> > > >> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0 > > > > >> > > >> Planning to have sync up call on this week, once it's finalize, > we can summarise here the approach. What do you think..? > > >> > > >> > > >> On 20/06/22, 4:06 PM, "Kengo Seki" <sek...@apache.org < > ma...@apache.org>> wrote: > > >> > > >> Thanks for the reply, Brahma! > > >> > > >>> We'll use following repo, how other components in Apache are > hosted. > > >> > > >> Using that site for distribution has no problem. My concern is, > do we > > >> have the source tree for building HDP packages and does it still > work > > >> without the artifacts behind the paywall? > > >> According to ASF's release policy [1], we must publish the > > >> corresponding source releases whenever we publish binary > artifacts, > > >> such as RPM and DEB packages for HDP. > > >> > > >> [1]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0 > > > > >> > > >> Kengo Seki <sek...@apache.org <ma...@apache.org>> > > >> > > >> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy > > >> <bbatt...@visa.com.invalid <ma...@visa.com.invalid>> wrote: > > >>> > > >>> Hi Kengo Seki, > > >>> > > >>> Thanks for pitching here.. We'll use following repo, how other > components in Apache are hosted. Any other suggestions..? > > >>> > > >>> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0 > > > > >>> > > >>> > > >>> > > >>> On 20/06/22, 2:32 PM, "Kengo Seki" <sek...@apache.org < > ma...@apache.org>> wrote: > > >>> > > >>> Oh, I've just noticed that I submitted an additional question > into the > > >>> wrong thread mistakenly. > > >>> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0> > is > > >>> originally intended as a reply to this thread. Sorry for the > > >>> confusion. > > >>> > > >>> Kengo Seki <sek...@apache.org <ma...@apache.org>> > > >>> > > >>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sek...@apache.org < > ma...@apache.org>> wrote: > > >>>> > > >>>> Thank you so much for writing the draft and commenting on it, > Brahma and Viraj! > > >>>> Firstly, a minor comment: > > >>>> > > >>>>> Trunk Code : Parallel we can work for the trunk code, this we > might give as 2.8. > > >>>>> * Working on bigtop mpack > > >>>> > > >>>> This means adding a new built-in stack which uses the Bigtop > > >>>> repository, and not developing Mpack on the Bigtop side, right? > > >>>> If so, "bigtop mpack" sounds confusing a bit. > > >>>> > > >>>>> hdp-select can stay on branch-2.7 and it can be moved to > bigtop for trunk branch. Thoughts? > > >>>> > > >>>> Sounds good to me. During the 2.7.x releases, users can use the > HDP > > >>>> stack by default. In addition, they can also add the Bigtop > stack > > >>>> through the Mpack, as Zhiguo mentioned in [1]. > > >>>> Since 2.8 or 3.0, we will drop HDP and users can use the > built-in > > >>>> Bigtop stack by default. It also allows us to drop the Mpack on > the > > >>>> Bigtop side. > > >>>> > > >>>> Regarding the built-in Bigtop stack developed on trunk, the > current > > >>>> BIGTOP stack in Ambari [2] may be a good start point to be > upgraded. > > >>>> I ensured that Ambari 2.7.6 was successfully built with the > > >>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too > old to > > >>>> use for now. > > >>>> > > >>>> [1]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0 > > > > >>>> [2]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi > [message truncated...] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org > For additional commands, e-mail: dev-h...@ambari.apache.org > >
