----- Original Message ----- > On Oct 24, 6:20 pm, "R.I.Pienaar" <r...@devco.net> wrote: > > ----- Original Message ----- > > > For MCollective, start reading about subcollectives[1] and Stomp > > > failover pools [2]. There's no reason why each of your Puppet > > > Masters > > > can't also be Stomp servers. ActiveMQ has some nice abilities to > > > send > > > messages between ActiveMQ servers that I started reading up on > > > (useful for when you start segregating dev and prod) but then I > > > ran into > > > scalability issues with ActiveMQ so replaced it with RabbitMQ and > > > haven't looked at what you can do with that yet. > > > > at how many nodes did you have scalability issues with ActiveMQ? > > did you > > do any tuning on it? > > 191 nodes to be exact, MCollective 1.3.1 and ActiveMQ 5.5 on a KVM VM > with 4GB RAM, 2GB allocated to ActiveMQ. Two of us spent the morning > tweaking ActiveMQ for better memory usage but weren't achieving much. > With all nodes connected but no actual Stomp messages ever sent, JMX > was telling me the Java heap was from 900M to 1.3G used, that's > before MCollective has even done anything. Broadcast just one message to the > estate and that would fill the heap and the broker would die. > ActiveMQ didn't appear to be thread bound so [1] didn't help too much nor did > experimenting with various systemUsage values, and tips from [2] gave > little improvement. > > I needed serious improvement as I was looking to connect another 300 > nodes within a week. One of the developers here nodded and smiled > when I mentioned ActiveMQ memory issues and said to try RabbitMQ, which I > got working as a drop in replacement for ActiveMQ in under half an > hour with a 10th of the memory footprint. I may eventually go back to > ActiveMQ as the fine grained user access control seems a little more > feature rich than RabbitMQ does on the surface.
Odd, that sounds suspiciously like some kind of messaging loop or something. Guess without recreating it's hard to say. But def have bigger networks than that and they just work - physical kit though, ActiveMQ seems quite unhappy about dodgy clocks like you would find on VMWare instances. It's the old painful story of memory tuning on Java though, its too complex but the jvm does at least give you the data you need. the symptom of using all memory prior to anything being done is normal though -- R.I.Pienaar -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
