On Sat, Apr 15, 2017 at 3:06 PM, Michael Klement <[email protected]> wrote:
> I'm aware of that workaround, but I thought you'd want to know that > env_parallel is not fully robust and fails in obscure ways. > Is env_parallel deprecated? The man page doesn't seem to say so. env_parallel is pretty new and thus not the same production quality as parallel. > 2017-04-14 20:53 GMT+02:00 Michael Klement <[email protected]>: : >> v3.2.57: only stderr output: >> >> -bash: /usr/bin/which: Argument list too long >> -bash: -n: command not found Given your attached file, you are most likely being hit by the environment being too big: >From man env_parallel: Environment space By default env_parallel will export all environment variables, arrays, aliases, functions and shell options (see details for the individual shells below). But this only works if the size of the current environment is smaller than the maximal length of a command and smaller than half of the max if running remotely. E.g. The max size of Bash's command is 128 KB, so env_parallel will fail if set | wc -c is bigger than 128 KB. Technically the limit is in execve(1) which IPC::open3 uses. Bash completion functions are well-known for taking up well over 128 KB of environment space and the primary reason for causing env_parallel to fail. Instead you can use --env to specify which variables, arrays, aliases and functions to export as this will only export those with the given name. And earlier: Due to the problem with environment space (see below) the recommended usage is: # Run once to record the "empty" environment env_parallel --record-env alias myalias=echo myvar=it myfunc() { myalias $1 $myvar works.; } # Use --env _ to only transfer the names not in the "empty" environment env_parallel --env _ -S localhost myfunc ::: Yay, But even tough it is documented your bug report is valuable: env_parallel should detect this error and give you a sensible error message instead. As the above is the recommended usage, maybe --env _ should be default if ~/.parallel/ignored-vars exist? /Ole
