*ON THE SERVER SIDE* I dug into the server logs and get the following in my php-scalarizr_messaging.log:
Jul 07 11:07:39 -07:00 - Scalr\System\Zmq\Cron\Task\ScalarizrMessaging@23619 - ERROR - Cannot handle message 'HostInit' (message_id: 0d58b6be-d0a1-48af-9532-133bbaad76c1) from server '52.40.180.82' (server_id: 567174e9-0d52-4e06-b9ca-675f6be3a4e6). Could not find the service "environment" in the DI container.(/opt/scalr-server/embedded/scalr/app/src/Scalr/DependencyInjection/BaseContainer.php:91) and when I did a grep "Could not find the service" php-scalarizr_messaging.log I got a lot of error messages from my testing today with different messages that could not be handled (HostInit, RebootFinish, WinHostdown, etc.). On Thursday, July 7, 2016 at 11:32:38 AM UTC-6, Jay Farschman wrote: > > After upgrading to 5.11.22 (Community Edition), I am unable to spin up new > instances. The instances become stuck in various states. For example, if I > spin up 3 instances I'll get 2 stuck in "pending" status, and 1 will make > it to "Initializing". Very rarely, perhaps one time in 10 and instance will > properly complete all of the required steps and show a status of "Running". > In all cases the instances start in Amazon Ec2 and I can log in. > > > > # scalarizr -v > Scalarizr 4.7.14 > > # ps -ef |grep scal > root 1354 1 0 16:30 ? 00:00:00 > /opt/scalarizr/embedded/bin/python /usr/bin/scalr-upd-client --daemonize > --pid-file /var/run/scalr-upd-client.pid --log-file > /var/log/scalarizr_update.log > root 1415 1 0 16:30 ? 00:00:11 > /opt/scalarizr/embedded/bin/python /usr/bin/scalarizr -c /etc/scalr > > *ON A SERVER STUCK PENDING* > The scalarizr_debug.log hangs at: > 2016-07-07 16:29:27,764+00:00 - DEBUG - scalarizr.messaging.p2p.consumer - > Waiting message acknowledge event: HostInit > > *ON A SERVER STUCK INITALIZING* > The scalarizr_debug.log gets past this because it received and decodes a > 'control' message. It does a number of other things, and then shortly after > completing a dpkg-query it starts attempts to start some message handlers > and the logs stop. > > 1167 2016-07-07 16:30:48,102+00:00 - DEBUG - > scalarizr.messaging.p2p.consumer - Starting message subhandler thread: > BeforeHostUpHandler > 1168 2016-07-07 16:30:48,102+00:00 - DEBUG - > scalarizr.messaging.p2p.consumer - Waiting message subhandler thread: > BeforeHostUpHandler > 1169 2016-07-07 16:30:48,102+00:00 - DEBUG - > scalarizr.messaging.p2p.consumer - Starting message handler > > I am struggling a bit with reasons this could happen, but I am leaning > toward a load issue causing a timeout. > > top - 17:29:08 up 130 days, 15:58, 2 users, load average: 2.62, 3.27, > 3.31 > > Tasks: 232 total, 3 running, 229 sleeping, 0 stopped, 0 zombie > > %Cpu(s): 7.1 us, 5.5 sy, 0.0 ni, 79.4 id, 0.3 wa, 0.0 hi, 0.0 si, > 7.7 st > > KiB Mem: 3854816 total, 3436444 used, 418372 free, 103544 buffers > > KiB Swap: 0 total, 0 used, 0 free. 213720 cached > Mem > > I'm currently testing with Ubuntu Images loaded with the 4.7.14 agent. > I'll try a different operating system and perhaps some different agent > versions. > > -- You received this message because you are subscribed to the Google Groups "scalr-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to scalr-discuss+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
