I make mpacks all the time, but tend to push them as extensions because the stack architecture is byzantine.
- It is java that deploys python code. As such, the structure behaves like java, which makes it a pain in the rear to track down simple python functions. Sometimes the code ends up in common-services, sometimes it stays in the current version of stacks. Some stack components only use code in their latest version, some have code files strung through the folders of 8 previous versions and 3 stacks (looking at you LLAP). - The install code doesn't know where it wants to be. sometimes its in the stack, sometimes it is in ambari. This means that a version of ambari tends to be hard coded to 1-2 versions of a stack. - Changing javascript for a stack isn't possible. This means that stack components are hard coded into the ember.js with a bunch of if statements. - Making additions to a stack feature is a major process (like adding hbase thrift to an existing hbase installation) - Upgrades require a yum file regardless of installation method. so learning how to create dummy rpms is necessary. I'd be willing to chip in and maintain the stack python code but the stack architecture really needs to be revisited. On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com> wrote: > Matt, > > > > Yeah, my thought was to start with whatever the most recent HDP/HDF stack > definition as a starting point. It just so happens that I have a > backburner project to do this already. I have been using Hortonworks for a > long time, but have found recently that I needed to install newer versions > of Apache Druid and Apache Storm, which would require me to do a new > stack…Full disclosure: I have NEVER mucked with stacks and am not a Python > guy…I write in Scala/Java predominantly….that being said, I would be happy > to collaborate on this if anyone feels that this would be worthwhile and > useful to the broader community. > > > > -- M. Aaron Bossert > > > > *From: *Matt Andruff <matt.andr...@gmail.com> > *Reply-To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Date: *Monday, May 11, 2020 at 1:46 PM > *To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Subject: *Re: life after Hortonworks > > > > Cloudera no longer uses ambari. They stuck with Cloudera manager in their > release of CDP.(CDH+HDP=CDP) > > > > https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=9t291neBjp4%2F2XnAvU34geDz9QgB%2BNAISHRyjvcvcog%3D&reserved=0> > > > > I don't think this means that Ambari is dead. I do think it means as > stated that the community will need to take on packaging a stack and > building rpms.(or at least packaging the stack) > > > > The legacy code for stack of HDP 2.6 stream is out there already in the > repo so it's just some work to create rpms of whatever Ambari wants to > release. > > > > I assume the stacks section is so poorly documented is because Hortonworks > was doing the work of packaging. I'm not sure the level of effort needed > to make a stack work but it seems like we could start with the last HDP > build (3.1.4) and keep moving forward. > > > > > > On Mon., May 11, 2020, 13:08 Stephen Boesch, <java...@gmail.com> wrote: > > I am reading between the lines that ambari is no longer a strategic > platform. Would someone please provide a link/reference to a Cloudera press > release or blog describing this and maybe related decisions/roadmaps? thx! > > > > Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert < > aa...@punchcyber.com>: > > For what it is worth, I have written blueprints before, but never stacks. > The documentation and tutorials for ambari stacks and blueprints are > horribly out of date, incomplete, or flat out missing. Perhaps that could > be an initial task for the community to undertake so that those of us who > were using the Hortonworks suite of tools and were comfortable with Ambari > can sever the cord, as it were....relying on commercial companies to > support open source tools once their objectives have changed is rarely a > good thing. > > > > Get Outlook for iOS > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7Y415fNQH3d4y67LbSIYGhLJxaCLWPGi6qcOutlSLGY%3D&reserved=0> > ------------------------------ > > *From:* Ganesh Raju <ganesh.r...@linaro.org> > *Sent:* Monday, May 11, 2020 12:57:50 PM > *To:* user@ambari.apache.org <user@ambari.apache.org> > *Subject:* Re: life after Hortonworks > > > > Here is more Apache Bigtop info > > > > releases > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=Ee0QKqwFQ%2FlBtha%2Bkw8enOtkr4PMD9PYWvCDyrvjqS0%3D&reserved=0> > > mailinglist > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=TcKPHYO%2Buhi%2BgLEXwjQtMWKb0nbuwinJUjiKDly2iVA%3D&reserved=0> > > list of components > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=tMDI6ARheCagibWzd4liiGVgLruvIfTnVztQuaExVK0%3D&reserved=0> > > > > Thanks > > Ganesh Raju > > > > > > On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ganesh.r...@linaro.org> > wrote: > > Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 > other components. > > > > Ganesh > > > > On Mon, May 11, 2020 at 9:56 AM Matt Andruff <matt.andr...@gmail.com> > wrote: > > I kept digging and I found that ambari supports stacks (that we have all > seen before). You can write your own custom stack > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=4KdHTtYYw%2B9AN6jH0j8GN%2FTaG7kT57byTJph9Ss%2B5uw%3D&reserved=0> > meaning you could write the "glue" what apache projects hortonworks had > written to roll out a set of services. I guess this is how they got around > being open source. I guess I'll start looking into making my own stack > from scratch. If anyone already has one or know of one, please let me know. > > > > On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ganesh.r...@linaro.org> > wrote: > > Apache Bigtop now has mpack and works with Ambari. You could try that. > > > > Thanks, > > Ganesh > > > > On Mon, May 11, 2020 at 9:15 AM Matt Andruff <matt.andr...@gmail.com> > wrote: > > Hey I was just poking around today trying to figure out if ambari still > works to install a cluster. > > > > I recently was able to install a Cloudera Manager installation. > > > > I was about to start an Ambari installation but when I poke the repos they > all seem to need a user/password. > > > > Is there a non-hortonworks(cloudera) hadoop that can be still be installed > with Amabari? > > > > If there is documentation somewhere just let me know, I couldn't find the > answer googling as Hortonworks(cloudera) docs are all that come up. > > > > -- > > Live life > Laugh Often > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0> > t > > > > > -- > > Live life > Laugh Often > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0> > t > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0> > t > >
