Could we use the IntegrationTest components to do this?
On October 13, 2016 at 14:04:54, zeo...@gmail.com (zeo...@gmail.com) wrote: +1 Ryan and Otto's comments. I also strongly think we need to make a demo environment easier, but that should be different than quick-dev. Jon On Thu, Oct 13, 2016 at 1:15 PM Otto Fowler <ottobackwa...@gmail.com> wrote: > - create scripts/utilities to easily run a topology locally in an IDE > instead of in the VM > > > ^^^^^^^^ THIS. > > > On October 13, 2016 at 12:36:45, Ryan Merriman (merrim...@gmail.com) > wrote: > > Working with the quick-dev vagrant VM recently left a lot to be desired. > All forthcoming comments are made under the assumption that this VM is > intended for development purposes. If that is not true, I think we should > consider adding a VM for this purpose (or Docker containers?). Here are > the issues I ran into that I think can be improved: > > - had to upgrade VirtualBox from 5.0.16 to 5.0.20 > - had to update to the latest metron/hdp-base Vagrant box > - takes forever to spin up > - VM is constrained for resources making it unstable > - spent a large amount of time troubleshooting sensors (no raw messages > in Kafka) > - no easy way to debug topologies > > Fortunately I think we can make this a much better experience without a > major effort. Here are my ideas to do this: > > - update the prereqs for VirtualBox > - add a check for the appropriate base box version (Jira has already > been created https://issues.apache.org/jira/browse/METRON-497) > - don't install any sensors and replace them with a data generator that > just loops through sample data and emits to Kafka (could also be used to > replay and troubleshoot edge cases) > - everything in monit is off by default except for ES or other critical > services > - create scripts/utilities to easily run a topology locally in an IDE > instead of in the VM > - improved documentation with examples of how to run and troubleshoot > topologies > > Is this a worthwhile effort? I think this would also give users an easier > path to demonstrate or tour Metron's capabilities. Are there any other > improvements people would like to see? Should we wait for Docker? > Thoughts? > > Ryan Merriman > -- Jon
