David, how much RAM do you have in the test system where you run full dev? On 2/25/17, 8:46 AM, "David Lyle" <dlyle65...@gmail.com> wrote:
+1 (binding) Checked signatures Tests passed Full dev ran sucessfully-ish (bit more below, tl;dr- it's ugly, but much better after METRON-671) Data flowed into the index/dashboard/hdfs On full-dev: Monit got into a state where it started killing topologies because the Storm cli was non-responsive due to memory pressure. That caused cascading failures. I let it be for about 20 minutes, and it seemed to calm down. I then killed pcap-service and Monit to make some headroom. I haven't seen this in my work with METRON-671 as only sensor probes and pcap-service are managed by Monit. Once the topologies are monitored by Ambari, we no longer use the Storm CLI for status so that frees up some additional memory. -D... On Sat, Feb 25, 2017 at 10:42 AM, David Lyle <dlyle65...@gmail.com> wrote: > I'm running it up now. I don't think it should affect the release, but > it's extremely important that full-dev work. It's how quick-dev is created. > Any notion that it's been deprecated is incorrect. > > Fwiw, I've been working with it for the last few weeks as part of > METRON-671 and it's been (knocking loudly on wood) reliable. > I think we should leave full-dev in as the release verification step, > particularly in the case that a release changes the Hadoop versions. > There's a bit of a chicken-egg problem when we bump versions and quick dev > will lag. > > I'll update with run results shortly. > > -D... > > > On Sat, Feb 25, 2017 at 9:24 AM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > >> I see this with full dev on my laptop pretty much every time. >> I think it is a resource issue. I see not enough memory errors trying to >> start things. >> >> >> >> On February 25, 2017 at 09:15:15, Ryan Merriman (merrim...@gmail.com) >> wrote: >> >> When I go to Ambari to ensure the services are all up, HDFS is down. I >> tried it 4 or 5 times and got the same result each time. I've seen others >> validate with quick dev so I assumed full dev was not used anymore. I'll >> spin it up this morning and get a more detailed error. >> >> Is anyone else able to validate it in full dev? >> >> > On Feb 25, 2017, at 7:21 AM, Casey Stella <ceste...@gmail.com> wrote: >> > >> > What exactly are the errors that you saw, Ryan? >> >> On Sat, Feb 25, 2017 at 07:31 David Lyle <dlyle65...@gmail.com> wrote: >> >> >> >> Is there any reason full dev shouldn't be working? >> >> >> >>> On Fri, Feb 24, 2017 at 9:19 PM, Casey Stella <ceste...@gmail.com> >> wrote: >> >>> >> >>> Sounds like a good idea to me; thanks Ryan! >> >>>> On Fri, Feb 24, 2017 at 21:11 Ryan Merriman <merrim...@gmail.com> >> wrote: >> >>>> >> >>>> +1 binding >> >>>> >> >>>> Verified the signature >> >>>> Passed maven tests >> >>>> Started quick-dev, verified data in ES, kibana, and checked the >> >>> topologies >> >>>> for errors (bro topology has parsing errors but I think a couple bad >> >>>> messages in bro data set is normal) >> >>>> Tested REPL >> >>>> RPMs built fine >> >>>> >> >>>> The recommended build validation wiki page ( >> https://cwiki.apache.org/ >> >>>> confluence/display/METRON/Verifying+Builds >> >>>> <https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds >> >) >> >>>> has some mistakes. This did >> >>>> not run successfully in full-dev-platform and the HDFS paths look >> like >> >>> they >> >>>> are old. I am happy to update the wiki page if everyone agrees these >> >> are >> >>>> legitimate mistakes. >> >>>> >> >>>> On Fri, Feb 24, 2017 at 9:22 AM, Justin Leet <justinjl...@gmail.com> >> >>>> wrote: >> >>>> >> >>>>> +1 (non-binding) >> >>>>> >> >>>>> Verified signature >> >>>>> Ran build and tests in maven >> >>>>> Ran up in quick-dev and saw data flow through topologies into the UI >> >>>>> Ensured the REPL spun up and performed some basic tasks >> >>>>> Built rpms >> >>>>> >> >>>>> Justin >> >>>>> >> >>>>>> On Thu, Feb 23, 2017 at 11:18 AM, Casey Stella <ceste...@gmail.com >> > >> >>>>> wrote: >> >>>>> >> >>>>>> This is a call to vote on releasing Apache Metron 0.3.1-RC5 >> >>> incubating >> >>>>>> >> >>>>>> Full list of changes in this release: >> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3. >> >>>>>> 1-RC5-incubating/CHANGES >> >>>>>> >> >>>>>> The tag/commit to be voted upon is apache-metron-0.3.1-rc5- >> >>> incubating: >> >>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-metron. >> >>>>>> git;a=shortlog;h=refs/tags/apache-metron-0.3.1-rc5-incubating >> >>>>>> >> >>>>>> The source archive being voted upon can be found here: >> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3. >> >>>>>> 1-RC5-incubating/apache-metron-0.3.1-rc5-incubating.tar.gz >> >>>>>> >> >>>>>> Other release files, signatures and digests can be found here: >> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3. >> >>>>>> 1-RC5-incubating/ >> >>>>>> >> >>>>>> The release artifacts are signed with the following key: >> >>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-metron. >> >>>>>> git;a=blob;f=KEYS;h=8381e96d64c249a0c1b489bc0c234d >> >>>>> 9c260ba55e;hb=refs/tags/ >> >>>>>> apache-metron-0.3.1-rc5-incubating >> >>>>>> >> >>>>>> The book associated with this RC is located at >> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3. >> >>>>>> 1-RC5-incubating/book-site/index.html >> >>>>>> >> >>>>>> Please vote on releasing this package as Apache Metron 0.3.1-RC5 >> >>>>> incubating >> >>>>>> >> >>>>>> When voting, please list the actions taken to verify the release. >> >>>>>> >> >>>>>> Recommended build validation and verification instructions are >> >> posted >> >>>>> here: >> >> https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds >> >>>>>> >> >>>>>> >> >>>>>> This vote will be open for at least 72 hours. >> >>>>>> >> >>>>>> [ ] +1 Release this package as Apache Metron 0.3.1-RC5 incubating >> >>>>>> >> >>>>>> [ ] 0 No opinion >> >>>>>> >> >>>>>> [ ] -1 Do not release this package because... >> >> >> > >