Thank you Matt,

I haven't seen 634 before. Should I merge my tickets as sub-tasks
addressing some of the points your bring up there?

Also, I have a note about elasticsearch_config_path which is set in
metron-deployment/roles/metron_streaming/defaults/main.yml. It seems
like it is not used anywhere in the code base.

Please let me know how I should proceed with two tickets I have created
that are relevant to yours.

- Dima

On 01/01/2017 04:43 AM, Matt Foley wrote:
> Hi Dima,
> Great to have the how-to doc in the wiki where it belongs.  Now we have a doc 
> to edit as we improve the install process :-)
>
> Did you look at https://issues.apache.org/jira/browse/METRON-634 before 
> opening METRON-642?  Please see my comments in the Jira for METRON-642.
> Thanks,
> --Matt
>
>
>
> On 12/30/16, 11:44 PM, "Dima Kovalyov" <[email protected]> wrote:
>
>     Hey,
>     
>     I wanted to finish what I've started with document for Metron with HDP
>     2.5, so I have migrated document (with minor text fixes and
>     clarifications) to here:
>     
> https://cwiki.apache.org/confluence/display/METRON/Metron+with+HDP+2.5+bare-metal+install
>     Old google doc was replaced with the link to this article.
>     
>     I also, created number of pull requests to fix minor bugs here and there
>     and created these two tickets: METRON-641 and METRON-642.
>     Please let me know if I did something out of proper procedure.
>     
>     Also, I agree that we should eventually strip HDP related steps from the
>     document, so in the end it will be like:
>     1. Build Mpack
>     2. Add to Ambari
>     3. Assigned Masters and Slave
>     4. PROFIT
>     But since we are where we are, let's leave it like that and fix all the
>     bugs first.
>     
>     p.s. have a happy holidays everyone
>     
>     - Dima
>     
>     On 12/16/2016 04:21 AM, Matt Foley wrote:
>     > I seem to have found the difficulty.  It will NOT show up on any system 
> that has /bin/java defined, which may account for why other folks with 
> Centos7 test systems aren’t seeing the behavior.
>     >
>     > On my Centos7 test system, it so happens that /bin/java is not defined, 
> even though $JAVA_HOME is correctly defined, and “$JAVA_HOME/bin” is in the 
> PATH.  In Centos7, when services launch through the (new in 7) systemctl 
> process, it drops all inherited environment variables and starts over fresh.  
> Although the systemd launch script 
> /usr/lib/systemd/system/elasticsearch.service does read in the 
> /etc/sysconfig/elasticsearch as an “EnvironmentFile”, it does not include 
> JAVA_HOME.
>     >
>     > When, eventually, the user-level launcher script at 
> /usr/share/elasticsearch/bin/elasticsearch gets invoked, JAVA_HOME is still 
> undefined.  But it looks for $JAVA_HOME/bin/java, so if “/bin/java” is linked 
> in the file system, then it’s good!  But if not, the launcher script dies.  
> Regrettably that launcher script, even though it is fairly complex, does not 
> write to any log file, and its stdout was closed long ago by the 
> service-level launcher.  So I had to hack it to see what it was doing.
>     >
>     > The solution is to simply write JAVA_HOME={{java64_home}} into the 
> elastic-sysconfig template.
>     >
>     > BTW, while munging thru code I reached the conclusion that 
> elastic-env.sh is basically orphaned.  Does anyone know of scripts that 
> source it? (Of course elastic-env.xml is still important, I’m only asking 
> about the elastic-env.sh file templated from it.)
>     >
>     > Thanks,
>     > --Matt
>     >
>     >
>     > On 12/14/16, 2:41 PM, "Matt Foley" <[email protected]> wrote:
>     >
>     > No, node.data and node.master are both correctly set to true (with 
> Ambari’s agreement/participation) in the elasticsearch.yml file in CONF_DIR, 
> and this is being correctly picked up by ES when launched interactively.  I 
> really think this is in the service management stuff in 
> /etc/init.d/elasticsearch and /usr/share/elasticsearch/bin/elasticsearch .  
> Remains to be proven, of course…
>     >
>     > The reason I think ES isn’t even being successfully launched by 
> systemd, is there is zero logging anywhere, except in ambari where it shows 
> nothing but a successful service launch.  No files created in 
> /var/log/elasticsearch, which all scripts agree is the value of LOG_DIR, 
> despite permissions set to “drwxr-xr-x. elasticsearch elasticsearch”
>     >
>     > Thanks,
>     > --Matt
>     >
>     > On 12/14/16, 2:07 PM, "David Lyle" <[email protected]> wrote:
>     >
>     >     Aha! There's your problem. :)
>     >     
>     >     Kidding aside, that is weird. I would expect the ES instance to 
> come up and
>     >     go status red right away, not up and die.
>     >     
>     >     I did have a horrible, horrible hack that made all that work, it 
> involved
>     >     modifying the stored es templates to both have node.master and 
> node.data
>     >     set to true in
>     >     /var/lib/ambari-server/resources/common-services/ELASTICSEARCH (from
>     >     memory, path may be a bit off). It occurred to me that an easy 
> incremental
>     >     step toward to full METRON-608 would be to simply expose the 
> templates to
>     >     the config pages...
>     >     
>     >     -D...
>     >     
>     >     
>     >     On Wed, Dec 14, 2016 at 4:59 PM, Matt Foley 
> <[email protected]> wrote:
>     >     
>     >     > David,
>     >     > No, it’s in my METRON-608 single-node deployment  :-)
>     >     >
>     >     > On 12/14/16, 12:46 PM, "David Lyle" <[email protected]> wrote:
>     >     >
>     >     >     Hi Matt and Jon,
>     >     >
>     >     >     FWIW, Metron with the MPack has been tested extensively on 
> CentOS 7.
>     >     > Works
>     >     >     like a champ. The issue right now is that sensor install is a 
> CentOS
>     >     >     6/Ansible proposition.
>     >     >
>     >     >     I haven't seen the issue with ES that you're experiencing, 
> Matt. Is
>     >     > that in
>     >     >     a 1 Master 3 Data node ES config or something else?
>     >     >
>     >     >     -D...
>     >     >
>     >     >
>     >     >     On Wed, Dec 14, 2016 at 3:29 PM, Matt Foley 
> <[email protected]>
>     >     > wrote:
>     >     >
>     >     >     > I hope we will try to support Centos7. Many of my company’s
>     >     > customers are
>     >     >     > requiring new installs to use RHEL 7 or Centos 7 rather 
> than 6.  In
>     >     >     > addition, we have the benefit that Centos 6 RPMs basically 
> always
>     >     > run fine
>     >     >     > in Centos7 (barring heavy-duty low-level system 
> manipulations that
>     >     > Metron,
>     >     >     > Kibana, and ES just don’t use), ES doesn’t distinguish 
> between them,
>     >     > and
>     >     >     > Kibana install support for Centos7 is already in place in 
> the
>     >     > Mpack.  And
>     >     >     > of course the Hadoop Stack runs fine in 7.
>     >     >     >
>     >     >     > For what it’s worth, I’ve been working with the Mpack 
> installation on
>     >     >     > Centos7 for the last few weeks, and it works fine except 
> for a
>     >     > persistent
>     >     >     > issue where the Elasticsearch service works fine if launched
>     >     > interactively
>     >     >     > but terminates immediately if launched with same arguments 
> as a
>     >     > service.
>     >     >     > Hope to find the cause in the next couple days.  (If anyone 
> knows
>     >     > why,
>     >     >     > would love to hear.  The daemonized launch scripts come 
> from ES, not
>     >     > from
>     >     >     > Metron, so “should just work”.)
>     >     >     >
>     >     >     > Thanks,
>     >     >     > --Matt
>     >     >     >
>     >     >     > On 12/13/16, 6:46 PM, "[email protected]" <[email protected]> 
> wrote:
>     >     >     >
>     >     >     >     For now, I'm not sure what the solution is, but I would 
> think
>     >     > choosing
>     >     >     > one
>     >     >     >     specific list of required/supported software (including 
> OS) and
>     >     >     > documenting
>     >     >     >     that thoroughly is the right start.  Something like what
>     >     > currently
>     >     >     > exists
>     >     >     >     for the vagrant side of things.
>     >     >     >
>     >     >     >     That said, long term I would love to see broader 
> support and
>     >     >     > automatically
>     >     >     >     generating documentation derived from the code.  Isn't 
> CentOS 6
>     >     > still
>     >     >     >     preferred over 7?  That's what I've been working on 
> solely for
>     >     > that
>     >     >     >     reason.
>     >     >     >
>     >     >     >     Jon
>     >     >     >
>     >     >     >     On Tue, Dec 13, 2016, 21:07 Matt Foley 
> <[email protected]> wrote:
>     >     >     >
>     >     >     >     > In my work on METRON-608, I’ve found a lot of small 
> but
>     >     > significant
>     >     >     > bugs
>     >     >     >     > in the existing Mpack (version 0.3.0).  These bugs 
> integrate
>     >     > with a
>     >     >     > lot of
>     >     >     >     > the oddball tweaks specified in the draft Install 
> docs I saw.
>     >     > I
>     >     >     > would like
>     >     >     >     > to submit a PR for these bugs, but they must be 
> accompanied by
>     >     >     > changes in
>     >     >     >     > the Install doc.
>     >     >     >     >
>     >     >     >     > How do we want to manage this?  Should we version the 
> Install
>     >     > doc?
>     >     >     > Or
>     >     >     >     > have multiple sections for the different versions?  
> Or have
>     >     >     > footnotes or
>     >     >     >     > sidebar comments about which version certain 
> paragraphs do and
>     >     > do
>     >     >     > not apply
>     >     >     >     > to?
>     >     >     >     >
>     >     >     >     > I’m inclined to use sidebars within the document, 
> because the
>     >     >     > dependencies
>     >     >     >     > aren’t just on which versions of HDP, Ambari, and 
> Metron you
>     >     > use.
>     >     >     > They
>     >     >     >     > also depend on Python 2.6 vs 2.7, and Centos 6 vs 7.
>     >     >     >     >
>     >     >     >     > Thanks,
>     >     >     >     > --Matt
>     >     >     >     >
>     >     >     >     >
>     >     >     >     > On 12/13/16, 4:13 PM, "[email protected]" 
> <[email protected]>
>     >     > wrote:
>     >     >     >     >
>     >     >     >     >     Sorry about the delay.  To be honest I delayed 
> once I saw
>     >     > the
>     >     >     >     > management UI
>     >     >     >     >     PR because I was going to wait for that to be 
> merged into
>     >     > master
>     >     >     >     > before I
>     >     >     >     >     did my mpack install.  My initial thought is that 
> there
>     >     > may need
>     >     >     > to be
>     >     >     >     > some
>     >     >     >     >     sort of a merger between what you provided and the
>     >     > Hortonworks
>     >     >     > blog
>     >     >     >     > post,
>     >     >     >     >     as I would prefer a single, comprehensive post 
> over
>     >     >     > fragmentation.
>     >     >     >     >
>     >     >     >     >     Jon
>     >     >     >     >
>     >     >     >     >     On Tue, Dec 13, 2016, 18:29 Dima Kovalyov <
>     >     >     > [email protected]>
>     >     >     >     > wrote:
>     >     >     >     >
>     >     >     >     >     > Thank you Jon,
>     >     >     >     >     >
>     >     >     >     >     > Just wondering if you have finished polishing 
> the
>     >     > document? I
>     >     >     > will
>     >     >     >     > have
>     >     >     >     >     > some time this week to polish and publish it if 
> there is
>     >     >     > anything
>     >     >     >     > left
>     >     >     >     >     > to format.
>     >     >     >     >     >
>     >     >     >     >     > - Dima
>     >     >     >     >     >
>     >     >     >     >     > On 12/01/2016 04:41 PM, [email protected] wrote:
>     >     >     >     >     > > While I can't delegate access to the wiki, I 
> am
>     >     > working on
>     >     >     >     > migrating the
>     >     >     >     >     > > documentation over.  I have been modifying the
>     >     > document for
>     >     >     >     > clarity and
>     >     >     >     >     > > updating up some of the formatting, so please 
> give me
>     >     > some
>     >     >     > time to
>     >     >     >     > do so.
>     >     >     >     >     > >
>     >     >     >     >     > > Jon
>     >     >     >     >     > >
>     >     >     >     >     > > On Thu, Dec 1, 2016 at 9:33 AM Dima Kovalyov <
>     >     >     >     > [email protected]>
>     >     >     >     >     > > wrote:
>     >     >     >     >     > >
>     >     >     >     >     > >> Hello,
>     >     >     >     >     > >>
>     >     >     >     >     > >> Who can provide me with an access to add new 
> page in
>     >     > this
>     >     >     >     > Installation
>     >     >     >     >     > >> space below?
>     >     >     >     >     > >> 
> https://cwiki.apache.org/confluence/display/METRON/
>     >     >     > Installation
>     >     >     >     >     > >> I want to add up-to-date article on how-to 
> install
>     >     > Metron
>     >     >     > using
>     >     >     >     > Ambari
>     >     >     >     >     > >> mpack.
>     >     >     >     >     > >> Please advise.
>     >     >     >     >     > >>
>     >     >     >     >     > >> My username is "dima" registered with 
> electronic mail
>     >     >     > dimdroll
>     >     >     >     > hosted by
>     >     >     >     >     > >> gmail.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Right now I have it posted here:
>     >     >     >     >     > >> https://goo.gl/HWGwpj
>     >     >     >     >     > >> But instead of maintaining it there it will 
> be
>     >     > convenient
>     >     >     > for me
>     >     >     >     > to have
>     >     >     >     >     > >> it in confluence instead of gdocs as I am 
> using
>     >     > confluence
>     >     >     > in our
>     >     >     >     >     > internal
>     >     >     >     >     > >> project as well, so I can basically copy 
> formatted doc
>     >     >     > between
>     >     >     >     > each
>     >     >     >     >     > other.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Thank you.
>     >     >     >     >     > >>
>     >     >     >     >     > >> - Dima
>     >     >     >     >     > >>
>     >     >     >     >     > >> On 11/30/2016 04:57 PM, 
> [email protected]<mailto:
>     >     > Zeolla
>     >     >     > @GMail.com>
>     >     >     >     >     > wrote:
>     >     >     >     >     > >> Probably the best long term place for this 
> would be
>     >     > here<
>     >     >     >     >     > >> 
> https://cwiki.apache.org/confluence/display/METRON/
>     >     >     > Installation>,
>     >     >     >     >     > >> potentially with a new mpack article (to
>     >     > differentiate from
>     >     >     > the
>     >     >     >     > ambari
>     >     >     >     >     > >> install which is already there, but dated).  
> I can
>     >     > help with
>     >     >     >     > migrating
>     >     >     >     >     > it
>     >     >     >     >     > >> to the wiki - I will run through your 
> documentation
>     >     > later
>     >     >     > today
>     >     >     >     > and
>     >     >     >     >     > request
>     >     >     >     >     > >> edit permissions if I find anything worth 
> updating.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Would you be OK if I put your doc in the 
> wiki at some
>     >     > point?
>     >     >     >     >     > >>
>     >     >     >     >     > >> Jon
>     >     >     >     >     > >>
>     >     >     >     >     > >> On Wed, Nov 30, 2016 at 9:51 AM Dima 
> Kovalyov <
>     >     >     >     > [email protected]
>     >     >     >     >     > >> <mailto:[email protected]>> wrote:
>     >     >     >     >     > >> As multiple users asked I have published 
> document for
>     >     >     > installing
>     >     >     >     > Metron
>     >     >     >     >     > on
>     >     >     >     >     > >> Centos 6 using Ambari 2.4 with HDP 2.5:
>     >     >     >     >     > >> https://goo.gl/HWGwpj
>     >     >     >     >     > >> It may look ugly as this is direct export 
> from
>     >     > internal
>     >     >     >     > confluence with
>     >     >     >     >     > >> cosmetic changes and our internal details 
> removed.
>     >     > Let me
>     >     >     > know if
>     >     >     >     > there
>     >     >     >     >     > is
>     >     >     >     >     > >> a better place to publish it.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Feel free to request edit permissions, I 
> would like
>     >     > the
>     >     >     > document
>     >     >     >     > to be
>     >     >     >     >     > >> up-to-date, simplified and corrected if 
> there are any
>     >     >     > errors. The
>     >     >     >     > goal
>     >     >     >     >     > for
>     >     >     >     >     > >> this document is to provide installation 
> guidance in a
>     >     >     > manner
>     >     >     >     > that Linux
>     >     >     >     >     > >> administrator with basic knowledge could 
> perform
>     >     >     > installation
>     >     >     >     > without
>     >     >     >     >     > >> opening any other links or searching for 
> commands,
>     >     > step by
>     >     >     > step.
>     >     >     >     >     > >> Thank you.
>     >     >     >     >     > >>
>     >     >     >     >     > >> p.s. feel free to let me know if you will 
> encounter
>     >     > any
>     >     >     > errors as
>     >     >     >     > I've
>     >     >     >     >     > >> seen them bunch.
>     >     >     >     >     > >>
>     >     >     >     >     > >> - Dima
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >> On 11/30/2016 04:45 PM, 
> [email protected]<mailto:
>     >     > Zeolla
>     >     >     > @GMail.com>
>     >     >     >     >     > wrote:
>     >     >     >     >     > >> I also have written up a method to retrieve 
> and setup
>     >     > the
>     >     >     > required
>     >     >     >     >     > >> packages in my script<
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> https://github.com/JonZeolla/Development/blob/master/Bash/
>     >     >     > setupMetron.sh>
>     >     >     >     >     > >> which could probably be cleaned up and 
> incorporated.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Jon
>     >     >     >     >     > >>
>     >     >     >     >     > >> On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <
>     >     >     > [email protected]
>     >     >     >     > <mailto:
>     >     >     >     >     > >> [email protected]>> wrote:
>     >     >     >     >     > >> In regards to your comment about a "full 
> package
>     >     > version
>     >     >     > dump"
>     >     >     >     > there is
>     >     >     >     >     > at
>     >     >     >     >     > >> least a start at `metron-deployment/scripts/
>     >     >     > platform-info.sh`.
>     >     >     >     > If we
>     >     >     >     >     > can
>     >     >     >     >     > >> enhance that, let's do it.
>     >     >     >     >     > >>
>     >     >     >     >     > >> On Wed, Nov 30, 2016 at 4:04 AM, shoggi <
>     >     > [email protected]
>     >     >     > <mailto:
>     >     >     >     >     > >> [email protected]>> wrote:
>     >     >     >     >     > >> Hi
>     >     >     >     >     > >>
>     >     >     >     >     > >> Just to wrap this up. I've tried to deploy 
> it on HDP
>     >     > 2.5
>     >     >     > but I
>     >     >     >     > got the
>     >     >     >     >     > >> exact same problem. What I did was removing 
> all
>     >     > components
>     >     >     > for
>     >     >     >     > 2.4.3.0
>     >     >     >     >     > and
>     >     >     >     >     > >> then created a new cluster based on 2.5. I 
> left it at
>     >     > that
>     >     >     > as the
>     >     >     >     > most
>     >     >     >     >     > >> likely issue could be with my base install. 
> May that
>     >     > be
>     >     >     > wrong
>     >     >     >     > version of
>     >     >     >     >     > >> something or even a missing dev package.
>     >     >     >     >     > >>
>     >     >     >     >     > >> As my work also coincided with the 
> announcement by
>     >     > Casey I
>     >     >     > went
>     >     >     >     > ahead
>     >     >     >     >     > and
>     >     >     >     >     > >> installed 0.3.0 as a dev installation 
> instead. That
>     >     > worked
>     >     >     > great
>     >     >     >     > as the
>     >     >     >     >     > >> dev-full-platform guide is very complete.
>     >     >     >     >     > >>
>     >     >     >     >     > >> As a general feedback, in order to get more 
> people
>     >     >     > interested and
>     >     >     >     > not
>     >     >     >     >     > >> having noobs like me come up with redundant 
> questions
>     >     > - At
>     >     >     > this
>     >     >     >     > stage
>     >     >     >     >     > also
>     >     >     >     >     > >> sorry for not providing a meaningful 
> subject, I
>     >     > simply was
>     >     >     > too
>     >     >     >     > excited
>     >     >     >     >     > :)
>     >     >     >     >     > >>
>     >     >     >     >     > >> There is much to win with a small extension 
> of the
>     >     >     > installation
>     >     >     >     > guide. I
>     >     >     >     >     > >> don't think it needs much more prose around 
> but a full
>     >     >     > package
>     >     >     >     > version
>     >     >     >     >     > dump
>     >     >     >     >     > >> with every new release would help greatly to 
> get
>     >     > things
>     >     >     > going.
>     >     >     >     > For the
>     >     >     >     >     > >> things which need to be compiled in, the 
> versioning
>     >     >     > information
>     >     >     >     > should
>     >     >     >     >     > be
>     >     >     >     >     > >> enough. I am saying this, as everyone starts 
> with a
>     >     >     > different base
>     >     >     >     >     > system.
>     >     >     >     >     > >> Some have the chance to start from scratch, 
> some have
>     >     > a
>     >     >     > system
>     >     >     >     > running
>     >     >     >     >     > and
>     >     >     >     >     > >> build something on top and then there are the
>     >     > experienced
>     >     >     > Metron
>     >     >     >     > users,
>     >     >     >     >     > who
>     >     >     >     >     > >> have many base requirements.. they tend to 
> forget.
>     >     > And I
>     >     >     > believe
>     >     >     >     > at the
>     >     >     >     >     > end
>     >     >     >     >     > >> it always fails just because of some very 
> tiny small
>     >     >     > difference.
>     >     >     >     >     > >>
>     >     >     >     >     > >> If there is a space somewhere available, 
> this could
>     >     > even be
>     >     >     > a
>     >     >     >     > spot,
>     >     >     >     >     > where
>     >     >     >     >     > >> people can say, "hey, I got this running on 
> XYZ and
>     >     > here is
>     >     >     > the
>     >     >     >     > package
>     >     >     >     >     > >> version dump". A simple table with 'Metron 
> version',
>     >     > 'OS
>     >     >     > release',
>     >     >     >     >     > 'Package
>     >     >     >     >     > >> list file', 'Version Details of every other 
> needed
>     >     >     > to-be-compiled
>     >     >     >     >     > package',
>     >     >     >     >     > >> is already a big help. This really could 
> supplement
>     >     > the
>     >     >     > existing
>     >     >     >     > guide
>     >     >     >     >     > >> found here:
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > https://cwiki.apache.org/confluence/display/METRON/
>     >     >     > Metron+Installation+on+an+Ambari-Managed+Cluster
>     >     >     >     >     > >> .
>     >     >     >     >     > >>
>     >     >     >     >     > >> But then, I could be alone and others did not
>     >     > experience
>     >     >     > the same
>     >     >     >     > and
>     >     >     >     >     > >> simply got it to run, I don't know. Anyway, 
> I would be
>     >     >     > happy to
>     >     >     >     > help if
>     >     >     >     >     > >> help is needed.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Cheers
>     >     >     >     >     > >> Shoggi
>     >     >     >     >     > >>
>     >     >     >     >     > >> On Sun, Nov 27, 2016 at 3:00 PM, shoggi <
>     >     > [email protected]
>     >     >     > <mailto:
>     >     >     >     >     > >> [email protected]>> wrote:
>     >     >     >     >     > >> Hi Simon
>     >     >     >     >     > >>
>     >     >     >     >     > >> Thank you for your very quick response. I 
> installed
>     >     > it on
>     >     >     > HDP
>     >     >     >     > 2.4.3.0.
>     >     >     >     >     > >>
>     >     >     >     >     > >> I had a 0.2.0 instance running and was very 
> conscious
>     >     > about
>     >     >     >     > versioning
>     >     >     >     >     > the
>     >     >     >     >     > >> various components. Somehow I had the 
> understanding
>     >     > of not
>     >     >     >     > touching 2.5
>     >     >     >     >     > as
>     >     >     >     >     > >> of yet. I give that a try.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Thanks heaps !
>     >     >     >     >     > >>
>     >     >     >     >     > >> Shoggi
>     >     >     >     >     > >>
>     >     >     >     >     > >> On Sun, Nov 27, 2016 at 2:52 PM, Simon 
> Elliston Ball <
>     >     >     >     >     > >> [email protected]<mailto:simon@
>     >     >     > simonellistonball.com>>
>     >     >     >     > wrote:
>     >     >     >     >     > >> Hi Shoggi,
>     >     >     >     >     > >>
>     >     >     >     >     > >> Can I ask which version of HDP you used as a 
> base. The
>     >     >     > article is
>     >     >     >     > a
>     >     >     >     >     > little
>     >     >     >     >     > >> out of date in that is calls for 2.4, which 
> the 0.3.0
>     >     >     > release is
>     >     >     >     > based
>     >     >     >     >     > on
>     >     >     >     >     > >> 2.5. There was a major version difference in 
> storm
>     >     > between
>     >     >     > the
>     >     >     >     > two,
>     >     >     >     >     > which
>     >     >     >     >     > >> seems likely to have caused the problem 
> you're seeing.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Simon
>     >     >     >     >     > >>
>     >     >     >     >     > >> Sent from my iPad
>     >     >     >     >     > >>
>     >     >     >     >     > >> On 27 Nov 2016, at 13:48, shoggi 
> <[email protected]
>     >     > <mailto:
>     >     >     >     >     > [email protected]>>
>     >     >     >     >     > >> wrote:
>     >     >     >     >     > >>
>     >     >     >     >     > >> Hi all
>     >     >     >     >     > >>
>     >     >     >     >     > >> I followed the installation guide to deploy 
> Metron via
>     >     >     > Ambari
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> https://community.hortonworks.com/articles/60805/deploying-
>     >     >     > a-fresh-metron-cluster-using-ambari-serv.html
>     >     >     >     >     > >> .
>     >     >     >     >     > >>
>     >     >     >     >     > >> I ended up with all but the Metron Components
>     >     > (Enrichment,
>     >     >     >     > Parsing,
>     >     >     >     >     > >> Parsers) to start up. The closest to 
> pinpoint the
>     >     > issue is,
>     >     >     > when
>     >     >     >     > I try
>     >     >     >     >     > to
>     >     >     >     >     > >> start any new parser topology such as with 
> bro:
>     >     >     >     >     > >>
>     >     >     >     >     > >> 
> /usr/metron/0.3.0/bin/start_parser_topology.sh -k
>     >     >     >     > x.localdomain:6667 -z
>     >     >     >     >     > >> x.localdomain:2181 -s bro
>     >     >     >     >     > >>
>     >     >     >     >     > >> The error I get is:
>     >     >     >     >     > >>
>     >     >     >     >     > >> Exception in thread "main" java.lang.
>     >     > BootstrapMethodError:
>     >     >     >     >     > >> java.lang.NoClassDefFoundError:
>     >     > org/apache/storm/Config
>     >     >     >     >     > >>
>     >     >     >     >     > >> Or in more details:
>     >     >     >     >     > >>
>     >     >     >     >     > >> Running: /usr/jdk64/jdk1.8.0_77/bin/java 
> -client
>     >     >     > -Ddaemon.name=
>     >     >     >     >     > >> -Dstorm.options= 
> -Dstorm.home=/usr/hdp/2.4.3.0-
>     >     > 227/storm
>     >     >     >     >     > >> -Dstorm.log.dir=/var/log/storm
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/
>     >     >     > lib:/usr/hdp/current/storm-client/lib
>     >     >     >     >     > >> -Dstorm.conf.file= -cp
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.
>     >     >     > 3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > clojure-1.6.0.jar:/usr/hdp/
>     >     >     >     >     > >> 2.4.3.<http://2.4.3
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> .>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/
>     >     >     > lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-
>     >     >     > auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/
>     >     >     >     >     > >> 2.4.3.<http://2.4.3
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> .>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/jackson-core-2.3.1.jar:/usr/hdp/
>     >     >     >     >     > >> 2.4.3.<http://2.4.3
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > .>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/
>     >     >     > usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/
>     >     >     > hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.
>     >     >     > v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.
>     >     >     > 
> 94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/
>     >     >     >     >     > >> 2.4.3.<http://2.4.3
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > 
> .>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > minlog-1.2.jar:/usr/hdp/
>     >     >     >     >     > >> 2.4.3.<http://2.4.3
>     >     >     >     >     >
>     >     >     >     > 
> .>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-
>     >     >     > 227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-
>     >     >     > 227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-
>     >     >     > 227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/
>     >     >     > lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/
>     >     >     > storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/
>     >     >     > zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-
>     >     >     > uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/
>     >     >     > hdp/2.4.3.0-227/storm/bin
>     >     >     >     >     > >> -Dstorm.jar=/usr/metron/0.3.0/
>     >     >     > lib/metron-parsers-0.3.0-uber.jar
>     >     >     >     >     > >> 
> org.apache.metron.parsers.topology.ParserTopologyCLI
>     >     > -k
>     >     >     >     >     > x.localdomain:6667
>     >     >     >     >     > >> -z x.localdomain:2181 -s bro
>     >     >     >     >     > >> Exception in thread "main" java.lang.
>     >     > BootstrapMethodError:
>     >     >     >     >     > >> java.lang.NoClassDefFoundError:
>     >     > org/apache/storm/Config
>     >     >     >     >     > >> at
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > org.apache.metron.parsers.topology.ParserTopologyCLI$
>     >     >     > ParserOptions.<init>(ParserTopologyCLI.java:199)
>     >     >     >     >     > >> at
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > org.apache.metron.parsers.topology.ParserTopologyCLI$
>     >     >     > ParserOptions.<clinit>(ParserTopologyCLI.java:47)
>     >     >     >     >     > >> at
>     >     >     >     >     > >>
>     >     >     >     >     >
>     >     >     >     > org.apache.metron.parsers.topology.ParserTopologyCLI.
>     >     >     > main(ParserTopologyCLI.java:266)
>     >     >     >     >     > >> Caused by: java.lang.NoClassDefFoundError:
>     >     >     > org/apache/storm/Config
>     >     >     >     >     > >> ... 3 more
>     >     >     >     >     > >> Caused by: java.lang.ClassNotFoundException:
>     >     >     >     > org.apache.storm.Config
>     >     >     >     >     > >> at java.net.URLClassLoader.
>     >     > findClass(URLClassLoader.java:
>     >     >     > 381)
>     >     >     >     >     > >> at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:
>     >     > 424)
>     >     >     >     >     > >> at 
> sun.misc.Launcher$AppClassLoader.loadClass(
>     >     >     > Launcher.java:331)
>     >     >     >     >     > >> at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:
>     >     > 357)
>     >     >     >     >     > >> ... 3 more
>     >     >     >     >     > >>
>     >     >     >     >     > >> Must be something wrong with my config, 
> possibly I
>     >     > have
>     >     >     > overlooked
>     >     >     >     >     > >> something but can't put my finger on it. Any 
> help is
>     >     >     > appreciated.
>     >     >     >     >     > >>
>     >     >     >     >     > >> Many thanks
>     >     >     >     >     > >> Shoggi
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >>
>     >     >     >     >     > >> --
>     >     >     >     >     > >> Nick Allen <[email protected]<mailto:nic
>     >     > [email protected]>>
>     >     >     >     >     > >> --
>     >     >     >     >     > >>
>     >     >     >     >     > >> Jon
>     >     >     >     >     > >>
>     >     >     >     >     > >> Sent from my mobile device
>     >     >     >     >     > >>
>     >     >     >     >     > >> --
>     >     >     >     >     > >>
>     >     >     >     >     > >> Jon
>     >     >     >     >     > >>
>     >     >     >     >     > >> Sent from my mobile device
>     >     >     >     >     > >>
>     >     >     >     >     > >> --
>     >     >     >     >     > > Jon
>     >     >     >     >     > >
>     >     >     >     >     > > Sent from my mobile device
>     >     >     >     >     > >
>     >     >     >     >     >
>     >     >     >     >     > --
>     >     >     >     >
>     >     >     >     >     Jon
>     >     >     >     >
>     >     >     >     >     Sent from my mobile device
>     >     >     >     >
>     >     >     >     >
>     >     >     >     >
>     >     >     >     > --
>     >     >     >
>     >     >     >     Jon
>     >     >     >
>     >     >     >     Sent from my mobile device
>     >     >     >
>     >     >     >
>     >     >     >
>     >     >
>     >     >
>     >     >
>     >     
>     >
>     >
>     >
>     >
>     >
>     
>     
>     
>
>
>


Reply via email to