Bram and Konrad, I'm personally interested in your perspective on multi-node testing.
In my experience with our stack of around 100 modules, I've found that single node testing is marginally better than spec testing in that it tells you if a service actually starts. However, my project has found that multi-node testing is the only way to know if most services (outside of node-only services) actually function properly. For instance, we found that perfectly valid rsyslog configurations would cause the daemon to start and appear to be functioning properly but that remote queueing, forwarding to multiple hosts, etc... was horribly broken. This was only possible to troubleshoot with multi-node environments via beaker (without writing something like beaker ourselves from scratch). I understand that we're the minority but I think that it's a gap that has significant real-world implications across the board. Note, we do use InSpec for compliance testing and have actually integrated it with Beaker in the simp-beaker-helpers gem. Thanks, Trevor On Tue, Apr 21, 2020 at 2:51 PM Konrad Scherer <konrad.sche...@windriver.com> wrote: > On 4/21/20 3:33 AM, Bram Vogelaar wrote: > > I too find it excruciating to use, I have described it in the past as > having a rage inducing cli, and the lack of > > documentation makes it hard to use. Which has driven me to use > test-kitchen for doing puppet acceptance testing. (with > > the added benefit of being able to use inspec profiles). > > In my world view the uptake has been very small outside the vox > pupuli and puppet supported modules because of this. > > After Litmus was released i have never considered Beaker again for any > real testing needs, since test-kitchen is very > > very single node only by design. > > My experience was similar. I tried to figure out beaker but was unable to > make progress due to the lack of documentation > and examples (almost two years ago now). With test-kitchen and > kitchen-puppet I was able to do my testing and it is my > preferred runtime testing framework. I also found the integration with > InSpec very useful. > > -- > Konrad Scherer, MTS, Linux Products Group, Wind River > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/e3571a4a-036c-5cea-7ae7-c92e8c3d08eb%40windriver.com > . > -- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 x788 -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CANs%2BFoWjv88hxUsxbudn0ZWVHRWmayuuGzy3OyMuBhd2PkBA1A%40mail.gmail.com.
