@Aaron Bossert - For that type of change (Ansible) you probably need to write up a design paper. (I have seen this done, and I'm not sure it's required but seems to be common practice) Sounds like ansible would be a good fit.
@Jeremy - thanks for the insights, I might ping you if I get stuck trying to get into it. I certainly wish there was a clean standard for installation locations. On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com> wrote: > Jeremy, > > > > I would be happy to tackle this with you and whoever else is willing…that > being said, I have always had a love/hate relationship with Ambari…the > stacks have always been intimidating to me….you mention revisiting the > architecture….do you have any ideas for what might be better? In my > opinion, I have always wanted to replace the guts of ambari with a system > that is underpinned by Ansible…seems much more user-friendly and MUCH > easier to configure and extend for those who are not comfortable with doing > their own Java to Python and creating RPM’s and the like….What are your > thoughts? > > > > > > -- M. Aaron Bossert > > > > *From: *jeremy montgomery <alericmcke...@gmail.com> > *Reply-To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Date: *Tuesday, May 12, 2020 at 9:14 AM > *To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Subject: *Re: life after Hortonworks > > > > I make mpacks all the time, but tend to push them as extensions because > the stack architecture is byzantine. > > - It is java that deploys python code. As such, the structure behaves > like java, which makes it a pain in the rear to track down simple python > functions. Sometimes the code ends up in common-services, sometimes it > stays in the current version of stacks. Some stack components only use > code in their latest version, some have code files strung through the > folders of 8 previous versions and 3 stacks (looking at you LLAP). > - The install code doesn't know where it wants to be. sometimes its > in the stack, sometimes it is in ambari. This means that a version of > ambari tends to be hard coded to 1-2 versions of a stack. > - Changing javascript for a stack isn't possible. This means that > stack components are hard coded into the ember.js with a bunch of if > statements. > - Making additions to a stack feature is a major process (like adding > hbase thrift to an existing hbase installation) > - Upgrades require a yum file regardless of installation method. so > learning how to create dummy rpms is necessary. > > I'd be willing to chip in and maintain the stack python code but the stack > architecture really needs to be revisited. > > > > On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com> > wrote: > > Matt, > > > > Yeah, my thought was to start with whatever the most recent HDP/HDF stack > definition as a starting point. It just so happens that I have a > backburner project to do this already. I have been using Hortonworks for a > long time, but have found recently that I needed to install newer versions > of Apache Druid and Apache Storm, which would require me to do a new > stack…Full disclosure: I have NEVER mucked with stacks and am not a Python > guy…I write in Scala/Java predominantly….that being said, I would be happy > to collaborate on this if anyone feels that this would be worthwhile and > useful to the broader community. > > > > -- M. Aaron Bossert > > > > *From: *Matt Andruff <matt.andr...@gmail.com> > *Reply-To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Date: *Monday, May 11, 2020 at 1:46 PM > *To: *"user@ambari.apache.org" <user@ambari.apache.org> > *Subject: *Re: life after Hortonworks > > > > Cloudera no longer uses ambari. They stuck with Cloudera manager in their > release of CDP.(CDH+HDP=CDP) > > > > https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0> > > > > I don't think this means that Ambari is dead. I do think it means as > stated that the community will need to take on packaging a stack and > building rpms.(or at least packaging the stack) > > > > The legacy code for stack of HDP 2.6 stream is out there already in the > repo so it's just some work to create rpms of whatever Ambari wants to > release. > > > > I assume the stacks section is so poorly documented is because Hortonworks > was doing the work of packaging. I'm not sure the level of effort needed > to make a stack work but it seems like we could start with the last HDP > build (3.1.4) and keep moving forward. > > > > > > On Mon., May 11, 2020, 13:08 Stephen Boesch, <java...@gmail.com> wrote: > > I am reading between the lines that ambari is no longer a strategic > platform. Would someone please provide a link/reference to a Cloudera press > release or blog describing this and maybe related decisions/roadmaps? thx! > > > > Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert < > aa...@punchcyber.com>: > > For what it is worth, I have written blueprints before, but never stacks. > The documentation and tutorials for ambari stacks and blueprints are > horribly out of date, incomplete, or flat out missing. Perhaps that could > be an initial task for the community to undertake so that those of us who > were using the Hortonworks suite of tools and were comfortable with Ambari > can sever the cord, as it were....relying on commercial companies to > support open source tools once their objectives have changed is rarely a > good thing. > > > > Get Outlook for iOS > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0> > ------------------------------ > > *From:* Ganesh Raju <ganesh.r...@linaro.org> > *Sent:* Monday, May 11, 2020 12:57:50 PM > *To:* user@ambari.apache.org <user@ambari.apache.org> > *Subject:* Re: life after Hortonworks > > > > Here is more Apache Bigtop info > > > > releases > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0> > > mailinglist > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0> > > list of components > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0> > > > > Thanks > > Ganesh Raju > > > > > > On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ganesh.r...@linaro.org> > wrote: > > Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 > other components. > > > > Ganesh > > > > On Mon, May 11, 2020 at 9:56 AM Matt Andruff <matt.andr...@gmail.com> > wrote: > > I kept digging and I found that ambari supports stacks (that we have all > seen before). You can write your own custom stack > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0> > meaning you could write the "glue" what apache projects hortonworks had > written to roll out a set of services. I guess this is how they got around > being open source. I guess I'll start looking into making my own stack > from scratch. If anyone already has one or know of one, please let me know. > > > > On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ganesh.r...@linaro.org> > wrote: > > Apache Bigtop now has mpack and works with Ambari. You could try that. > > > > Thanks, > > Ganesh > > > > On Mon, May 11, 2020 at 9:15 AM Matt Andruff <matt.andr...@gmail.com> > wrote: > > Hey I was just poking around today trying to figure out if ambari still > works to install a cluster. > > > > I recently was able to install a Cloudera Manager installation. > > > > I was about to start an Ambari installation but when I poke the repos they > all seem to need a user/password. > > > > Is there a non-hortonworks(cloudera) hadoop that can be still be installed > with Amabari? > > > > If there is documentation somewhere just let me know, I couldn't find the > answer googling as Hortonworks(cloudera) docs are all that come up. > > > > -- > > Live life > Laugh Often > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0> > t > > > > > -- > > Live life > Laugh Often > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0> > t > > > > -- > > IRC: ganeshraju@#linaro on irc.freenode.ne > <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0> > t > > -- Live life Laugh Often
