Thank you Toshio. The use of shell has worked perfectly. Much appreciated.
On Tuesday, November 26, 2019 at 6:34:52 PM UTC, Toshio Kuratomi wrote: > > What happens if you use shell instead of command? > > I'm not 100% certain but I think what's happening is that since you're > using the ansible command module instead of the shell module, it is > running the equivalent of: > cat '/var/lib/pgsql/temp/truncate_sql' > '/var/lib/pgsql/temp/postcodesio-*' '|' 'psql' 'postcodesiodb' > > That's sending the entire output of the first couple files to stdout > which the command module is then buffering to send back to the > controller. > > If you use the shell module then the pipe will actually be interpreted > by the shell and so it will do what you want. (If I'm wrong about why > the MemoryError is happening, there might be another problem in > addition but I am hopeful that it's all tied together). > > -Toshio > > On Tue, Nov 26, 2019 at 9:12 AM jonin <sebastia...@gmail.com <javascript:>> > wrote: > > > > Hi, > > > > I have created the following task to be run on a remote host: > > > > - hosts: nodes > > remote_user: root > > tasks: > > - name: Step Seven - remove all current postcode data and then > re-populate the database with most recent data > > become_user: postgres > > become: yes > > command: "cat /var/lib/pgsql/temp/truncate_sql > /var/lib/pgsql/temp/postcodesio-* | psql postcodesiodb" > > > > but it is failing with the below memory issue. A google makes me suspect > it is because I am trying to make ansible handle too large a file (a > postcode database) which it is loading into RAM. : > https://stackoverflow.com/questions/41367278/how-to-fix-memory-error-in-ansible/51320108. > > The suggestion is to use the synchronise module or rsync directly as a > command. For either I am struggling to visualise the syntax / structure to > do this. Any help much appreciated. > > > > TASK [Step Seven - remove all current postcode data and then re-populate > the database with most recent data] > ***************************************************************** > > fatal: [vm015018.bskyb.com]: FAILED! => {"changed": false, > "module_stderr": "Shared connection to vm015018.bskyb.com closed.\r\n", > "module_stdout": "Traceback (most recent call last):\r\n File > \"/tmp/ansible_ar0FKd/ansible_module_command.py\", line 213, in > <module>\r\n main()\r\n File > \"/tmp/ansible_ar0FKd/ansible_module_command.py\", line 207, in main\r\n > module.fail_json(msg='non-zero return code', **result)\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2294, in fail_json\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2273, in _return_formatted\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2227, in jsonify\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 782, in jsonify\r\n File \"/usr/lib64/python2.7/json/__init__.py\", > line 250, in dumps\r\n sort_keys=sort_keys, **kw).encode(obj)\r\n File > \"/usr/lib64/python2.7/json/encoder.py\", line 207, in encode\r\n chunks > = self.iterencode(o, _one_shot=True)\r\n File > \"/usr/lib64/python2.7/json/encoder.py\", line 270, in iterencode\r\n > return _iterencode(o, 0)\r\nMemoryError\r\n", "msg": "MODULE FAILURE", > "rc": 1} > > to retry, use: --limit > @/home/sco94/mwe-playbooks/playbooks/patching/postcodes.retry > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Ansible Development" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to ansibl...@googlegroups.com <javascript:>. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-devel/6ff52565-cbdd-4dca-96ee-85382530d7b5%40googlegroups.com. > > > > -- You received this message because you are subscribed to the Google Groups "Ansible Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-devel/56b962d6-b670-4954-9837-779ecc6d9197%40googlegroups.com.
