Thank you for proposing this, Viraj and Brahma.
This proposal may feel sudden to someone, so let me explain some context,
mainly to the Bigtop community.
Ambari went to Attic once due to inactivity,
but some of its users and developers are reviving it now.
>From the Bigtop community, Roman and Evans undertook the role of
initial PMC members,
and Jun, Yuqi, Ganesh, Masatake, and I are also going to support it.
The default stack of Ambari was HDP, but it has gone behind paywall,
so we have to add an alternative stack in the next release of Ambari.
Viraj and Brahma are planning to leverage Bigtop packages for that purpose,
and that stack is tentatively named BDP (BigData Platform).
We would also like to take over a feature called hdp-select,
which enables users to switch the current stack easily with symlink.
For example, assuming that the following stack is installed:
/usr/hdp/SOME_VERSION/hadoop/...
/spark/...
/zookeeper/...
/etc/hadoop, /var/lib/hadoop, /usr/lib/hadoop are symlinks of
/usr/hdp/SOME_VERSION/hadoop/etc, /usr/hdp/SOME_VERSION/hadoop/var/lib,
/usr/hdp/SOME_VERSION/hadoop/usr/lib respectively.
Then the new stack is installed under /usr/hdp/NEW_VERSION,
users can run hdp-select to switch the symlinks to the new directories
for upgrading the stack.
(Is my understanding correct? Correct me if I'm wrong, Viraj and Brahma)
hdp-select was provided as a RPM/DEB package just like other packages
HDP provided,
so it is user-friendly to provide bdp-select (the BDP version of
hdp-select) as well.
This is the background of Viraj's proposal.
But there is a problem here to leverage Bigtop packages in this manner.
Bigtop respects Linux's Filesystem Hierarchy Standard,
so it doesn't install files into a single place (/usr/hdp/VERSION in
the example above),
but directly into /etc, /usr, /var and so on.
So if we accept bdp-select, we also have to develop a new feature to
change install directories of the packages,
keeping the current default for avoiding user-facing incompatibility.
Otherwise bdp-select itself is useless.
But it will be a tough work, because installation paths are hard-coded
in the RPM specs and DEB packaging scripts now.
Based on the above, how about creating a new branch for adding hdp-select
and developing the feature to switch installation paths?
After finishing the work on that branch and ensuring that it works expectedly,
we can merge it into master, I think.
Kengo Seki <sek...@apache.org>
On Thu, Jun 30, 2022 at 4:21 PM Battula, Brahma Reddy
<bbatt...@visa.com.invalid> wrote:
>
> Thanks @Viraj Jasani to start discussing on this.
>
> IMO, For upcoming ambari release(2.7.7) , we should have select other than
> HDP so that people can start using the ambari without HDP.
>
> I am +1(non-binding) on this approach to have selects in bigtop code. May be
> for long term , we can think for tarballs or mpacks which can go in mabari
> trunk.
>
>
> On 29/06/22, 10:22 AM, "Viraj Jasani" <vjas...@apache.org> wrote:
>
> Hi Ambari/Bigtop dev,
>
> As per the new roadmap of Apache Ambari, we would like to propose moving
> certain scripts (previously known as hdp-select and conf-select) to Bigtop
> so that their rpm installation could be managed independently.
> These scripts are a basic necessity in the Ambari framework for the
> installation of various Bigdata packages. The only major changes they
> would
> receive is when we onboard new services and components to Ambari, else
> they
> usually do not receive updates. In the past, we used to get hdp-select rpm
> downloaded and installed from HDP repositories.
>
> We have still not officially finalized the new replacement name for
> hdp-select, it would likely be bdp-select (bdp: BigData Platform).
> However,
> the purpose of bdp-select and conf-select remains the same.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org
For additional commands, e-mail: dev-h...@ambari.apache.org
