@aaron - i don't think that the server/agent piece needs to be messed
with. It is more puppet than ansible, but it works pretty well. It has a
few issues that need to be worked out, but nothing insurmountable.
Agent/Server (pieces that need work)
- Ability to invalidate the agent cache.
- Multithreading - Currently alerts/restarts work on a concept of
threads and timeouts. You can end up with too many concurrent tasks/alerts
and things get backlogged waiting on the timeouts. However, if you up the
threads you run into a lot of race conditions with auto-restart and alerts
firing concurrently. ----> main piece here is it needs a prioritized
queuing system locally.
- Order of restarts and hierarchy - Right now this is hard coded at the
server level within a component and somewhat for full restarts based on the
order on the left hand bar.
- Need the ability to modify the full restart sequence
- Need the ability to state whether a restart is blocking, blocking
only when component is on the same server, and non-blocking.
- Need the ability to chain 2 different components together.
Example, if i restart a kerberized hiveserver2 i can have it
restart hue as
well after it is restarted. Otherwise hue will error out since
the spnego
hashes are different.
I'll make some edits to the confluence page explaining how the
stacks/mpacks/extensions work.
On Tue, May 12, 2020 at 10:51 AM Matt Andruff <matt.andr...@gmail.com>
wrote:
> @Aaron Bossert - For that type of change (Ansible) you probably need to
> write up a design paper. (I have seen this done, and I'm not sure it's
> required but seems to be common practice) Sounds like ansible would be a
> good fit.
>
> @Jeremy - thanks for the insights, I might ping you if I get stuck trying
> to get into it. I certainly wish there was a clean standard for
> installation locations.
>
>
>
> On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com>
> wrote:
>
>> Jeremy,
>>
>>
>>
>> I would be happy to tackle this with you and whoever else is willing…that
>> being said, I have always had a love/hate relationship with Ambari…the
>> stacks have always been intimidating to me….you mention revisiting the
>> architecture….do you have any ideas for what might be better? In my
>> opinion, I have always wanted to replace the guts of ambari with a system
>> that is underpinned by Ansible…seems much more user-friendly and MUCH
>> easier to configure and extend for those who are not comfortable with doing
>> their own Java to Python and creating RPM’s and the like….What are your
>> thoughts?
>>
>>
>>
>>
>>
>> -- M. Aaron Bossert
>>
>>
>>
>> *From: *jeremy montgomery <alericmcke...@gmail.com>
>> *Reply-To: *"user@ambari.apache.org" <user@ambari.apache.org>
>> *Date: *Tuesday, May 12, 2020 at 9:14 AM
>> *To: *"user@ambari.apache.org" <user@ambari.apache.org>
>> *Subject: *Re: life after Hortonworks
>>
>>
>>
>> I make mpacks all the time, but tend to push them as extensions because
>> the stack architecture is byzantine.
>>
>> - It is java that deploys python code. As such, the structure
>> behaves like java, which makes it a pain in the rear to track down simple
>> python functions. Sometimes the code ends up in common-services,
>> sometimes
>> it stays in the current version of stacks. Some stack components only use
>> code in their latest version, some have code files strung through the
>> folders of 8 previous versions and 3 stacks (looking at you LLAP).
>> - The install code doesn't know where it wants to be. sometimes its
>> in the stack, sometimes it is in ambari. This means that a version of
>> ambari tends to be hard coded to 1-2 versions of a stack.
>> - Changing javascript for a stack isn't possible. This means that
>> stack components are hard coded into the ember.js with a bunch of if
>> statements.
>> - Making additions to a stack feature is a major process (like adding
>> hbase thrift to an existing hbase installation)
>> - Upgrades require a yum file regardless of installation method. so
>> learning how to create dummy rpms is necessary.
>>
>> I'd be willing to chip in and maintain the stack python code but the
>> stack architecture really needs to be revisited.
>>
>>
>>
>> On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>
>> wrote:
>>
>> Matt,
>>
>>
>>
>> Yeah, my thought was to start with whatever the most recent HDP/HDF stack
>> definition as a starting point. It just so happens that I have a
>> backburner project to do this already. I have been using Hortonworks for a
>> long time, but have found recently that I needed to install newer versions
>> of Apache Druid and Apache Storm, which would require me to do a new
>> stack…Full disclosure: I have NEVER mucked with stacks and am not a Python
>> guy…I write in Scala/Java predominantly….that being said, I would be happy
>> to collaborate on this if anyone feels that this would be worthwhile and
>> useful to the broader community.
>>
>>
>>
>> -- M. Aaron Bossert
>>
>>
>>
>> *From: *Matt Andruff <matt.andr...@gmail.com>
>> *Reply-To: *"user@ambari.apache.org" <user@ambari.apache.org>
>> *Date: *Monday, May 11, 2020 at 1:46 PM
>> *To: *"user@ambari.apache.org" <user@ambari.apache.org>
>> *Subject: *Re: life after Hortonworks
>>
>>
>>
>> Cloudera no longer uses ambari. They stuck with Cloudera manager in their
>> release of CDP.(CDH+HDP=CDP)
>>
>>
>>
>> https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0>
>>
>>
>>
>> I don't think this means that Ambari is dead. I do think it means as
>> stated that the community will need to take on packaging a stack and
>> building rpms.(or at least packaging the stack)
>>
>>
>>
>> The legacy code for stack of HDP 2.6 stream is out there already in the
>> repo so it's just some work to create rpms of whatever Ambari wants to
>> release.
>>
>>
>>
>> I assume the stacks section is so poorly documented is because
>> Hortonworks was doing the work of packaging. I'm not sure the level of
>> effort needed to make a stack work but it seems like we could start with
>> the last HDP build (3.1.4) and keep moving forward.
>>
>>
>>
>>
>>
>> On Mon., May 11, 2020, 13:08 Stephen Boesch, <java...@gmail.com> wrote:
>>
>> I am reading between the lines that ambari is no longer a strategic
>> platform. Would someone please provide a link/reference to a Cloudera press
>> release or blog describing this and maybe related decisions/roadmaps? thx!
>>
>>
>>
>> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
>> aa...@punchcyber.com>:
>>
>> For what it is worth, I have written blueprints before, but never
>> stacks. The documentation and tutorials for ambari stacks and blueprints
>> are horribly out of date, incomplete, or flat out missing. Perhaps that
>> could be an initial task for the community to undertake so that those of us
>> who were using the Hortonworks suite of tools and were comfortable with
>> Ambari can sever the cord, as it were....relying on commercial companies to
>> support open source tools once their objectives have changed is rarely a
>> good thing.
>>
>>
>>
>> Get Outlook for iOS
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0>
>> ------------------------------
>>
>> *From:* Ganesh Raju <ganesh.r...@linaro.org>
>> *Sent:* Monday, May 11, 2020 12:57:50 PM
>> *To:* user@ambari.apache.org <user@ambari.apache.org>
>> *Subject:* Re: life after Hortonworks
>>
>>
>>
>> Here is more Apache Bigtop info
>>
>>
>>
>> releases
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0>
>>
>> mailinglist
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0>
>>
>> list of components
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0>
>>
>>
>>
>> Thanks
>>
>> Ganesh Raju
>>
>>
>>
>>
>>
>> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ganesh.r...@linaro.org>
>> wrote:
>>
>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
>> other components.
>>
>>
>>
>> Ganesh
>>
>>
>>
>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <matt.andr...@gmail.com>
>> wrote:
>>
>> I kept digging and I found that ambari supports stacks (that we have all
>> seen before). You can write your own custom stack
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0>
>> meaning you could write the "glue" what apache projects hortonworks had
>> written to roll out a set of services. I guess this is how they got around
>> being open source. I guess I'll start looking into making my own stack
>> from scratch. If anyone already has one or know of one, please let me know.
>>
>>
>>
>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ganesh.r...@linaro.org>
>> wrote:
>>
>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>
>>
>>
>> Thanks,
>>
>> Ganesh
>>
>>
>>
>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <matt.andr...@gmail.com>
>> wrote:
>>
>> Hey I was just poking around today trying to figure out if ambari still
>> works to install a cluster.
>>
>>
>>
>> I recently was able to install a Cloudera Manager installation.
>>
>>
>>
>> I was about to start an Ambari installation but when I poke the repos
>> they all seem to need a user/password.
>>
>>
>>
>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>> installed with Amabari?
>>
>>
>>
>> If there is documentation somewhere just let me know, I couldn't find the
>> answer googling as Hortonworks(cloudera) docs are all that come up.
>>
>>
>>
>> --
>>
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>>
>>
>> --
>>
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>
> --
> Live life
> Laugh Often
>
