Yes, this appears to be similar to the bug SVG pointed out, which I've tracked down to being related to the way python queues use pickle to serialize dictionary data (the resulting size of the data can be 200x than it was in the dictionary before). I'm currently working on a solution to this, and hope to include it in the next beta round.
James Cammarata Director, Ansible Core Engineering github: jimi-c On Fri, Oct 9, 2015 at 11:23 AM, Serge van Ginderachter < se...@vanginderachter.be> wrote: > In my experience, OOM's happen when the inventory is large, especially > when having lots of variables in the inventory, combined with lots (several > hundreds) of hosts. > > Not very scientific, but to get a rough idea on the size of your > inventory, could you show the output of > > > $ time (ansible all -m debug -a var=hostvars |wc -l ; ansible all > --list-hosts |wc -l) > 19788298 > 1046 > > real 2m3.222s > user 6m54.500s > sys 0m7.772s > > > Mine would yield 19.788.298 hostvars lines for 1046 hosts. > > That used to be a lot more, before I made some optimisations on a dynamic > inventory script that calculated a bunch of variables: > > $ ansible all -m debug -a var=hostvars |wc -l ; ansible all --list-hosts > |wc -l > 95924259 > 1037 > (that last job took 17 minutes to run, on an i7 quadcore laptop ) > > > > Possibly, see also https://github.com/ansible/ansible/issues/12239 > > > > > On 9 October 2015 at 16:48, Robert Lupinek <darthl...@gmail.com> wrote: > >> Ok, >> >> On the RHEL 7 host my versions are as follows: >> >> 1. ansible =ansible 1.9.3 >> 2. PythoPython = 2.7.5 >> >> On the RHEL 6 host my versions are as follows: >> >> 1. ansible = ansible 1.9.2 >> 2. Python = Python 2.6.6 >> >> I will report back once I can start monitoring malloc. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ansible Project" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to ansible-project+unsubscr...@googlegroups.com. >> To post to this group, send email to ansible-project@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/ansible-project/977bc078-311d-4ab5-9796-5bf9e108c228%40googlegroups.com >> <https://groups.google.com/d/msgid/ansible-project/977bc078-311d-4ab5-9796-5bf9e108c228%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to ansible-project+unsubscr...@googlegroups.com. > To post to this group, send email to ansible-project@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/CAEhzMJAVMxbeDkmuGFfXTmriM4Q-jZODa3ztxnXRVK6eh1-1Ww%40mail.gmail.com > <https://groups.google.com/d/msgid/ansible-project/CAEhzMJAVMxbeDkmuGFfXTmriM4Q-jZODa3ztxnXRVK6eh1-1Ww%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To post to this group, send email to ansible-project@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAMFyvFhJkonE-9ZALufgHVc_YpGPYJfPQha8V-zFPx_oHr-gMw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
