Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-13 Thread Jonathan Lozada De La Matta
tou ahould check out AWX On Thu, Dec 13, 2018 at 12:19 AM pradeep.hk wrote: > Got it. Thanks for your time. > My requirement is to be able to sequence together tasks(to create a > workflow) and after some prototyping, I found ansible playbook to meet that > requirement. > But then, scalability

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread pradeep.hk
Got it. Thanks for your time. My requirement is to be able to sequence together tasks(to create a workflow) and after some prototyping, I found ansible playbook to meet that requirement. But then, scalability is turning out to be an issue. If we could daemonize the ansible process so that it

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread Kai Stian Olstad
On Wednesday, 12 December 2018 12:32:30 CET Kai Stian Olstad wrote: > If you use Ansible dynamic inventory you can easily change the inventory > depending on the dynamic demand. You don't even need to use dynamic inventory, this will do the same as your script but let ansible do the forking.

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread Kai Stian Olstad
On 12.12.2018 11:22, pradeep.hk wrote: The requirement is to be able to launch a playbook on receiving a request (ie on demand). So, if 100 requests are received, it will result in 100 playbooks being executed parallely. You mentioned - Ansible uses about 50MB of memory per playbook. Is there

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread pradeep.hk
The requirement is to be able to launch a playbook on receiving a request (ie on demand). So, if 100 requests are received, it will result in 100 playbooks being executed parallely. You mentioned - Ansible uses about 50MB of memory per playbook. Is there something that can be done to optimize

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread pradeep.hk
Thanks a lot. That worked On Wednesday, December 12, 2018 at 3:09:32 PM UTC+5:30, Kai Stian Olstad wrote: > > On 12.12.2018 07:53, pradeep.hk wrote: > > I have a requirement to run lots of playbooks parallely on localhost. > > When I run 100 playbooks parallely (a simple one that executes the

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread pradeep.hk
Few things: (1)As mentioned in my previous reply, I even ran the date command 100 times but no change in memory usage. In addition, I also tried a empty loop to keep the process running (2)It is my requirement to run the playbook on the localhost in response to a request to execute a job (there

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread Kai Stian Olstad
On 12.12.2018 07:53, pradeep.hk wrote: I have a requirement to run lots of playbooks parallely on localhost. When I run 100 playbooks parallely (a simple one that executes the a shell command - date) , I see that it consumes lot of resources (memory ~ 7GiB and CPU ~ 200%) Is this on expected

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-12 Thread Karl Auer
That script does not run 100 copies of date. It runs 100 copies of an empty loop which may well be optimised out of existence. I'm not saying that Ansible is not the problem, but you do need to start with a fair comparison. Also, your playbook is running 100 times on one system; you may find it

Re: [ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-11 Thread Karl Auer
what happens if you leave Ansible out and just fire off 100 copies of date? On Wed, Dec 12, 2018 at 5:54 PM pradeep.hk wrote: > I have a requirement to run lots of playbooks parallely on localhost. > When I run 100 playbooks parallely (a simple one that executes the a shell > command - date) ,

[ansible-project] High memory usage when running multiple ansible playbooks on localhost

2018-12-11 Thread pradeep.hk
I have a requirement to run lots of playbooks parallely on localhost. When I run 100 playbooks parallely (a simple one that executes the a shell command - date) , I see that it consumes lot of resources (memory ~ 7GiB and CPU ~ 200%) Is this on expected lines ? here is what I am trying to run