Re: [Open Babel] running Open Babel in parallel / distributed mode
It would be nice to see some explicit examples of how Open Babel might be used in this way, using one or all of these tools. - Noel On 4 February 2014 00:52, Francois Berenger beren...@riken.jp wrote: On 02/04/2014 12:14 AM, Maciek Wójcikowski wrote: You can also use xargs. Yes, xargs with the -P option, but the command lines are not trivial then. Pozdrawiam, | Best regards, Maciek Wójcikowski mac...@wojcikowski.pl mailto:mac...@wojcikowski.pl 2014-02-03 16:10 GMT+01:00 Igor Filippov igor.v.filip...@gmail.com mailto:igor.v.filip...@gmail.com: How is it different from GNU parallel? http://www.gnu.org/software/bash/manual/html_node/GNU-Parallel.html It should be quite similar in functionality. Igor On Mon, Feb 3, 2014 at 1:37 AM, Francois Berenger beren...@riken.jp mailto:beren...@riken.jp wrote: Hello, I do this almost everyday so I think I should share it with this list. In case you need to execute many Open Babel commands and don't want to wait, you can execute them in parallel on a multi-core computer. Of course, the commands should be independent, for example processing different datasets. Let's say the commands are in a file called for_par.sh. I developped a tool called PAR years ago that can do this: par -i for_par.sh -v -o log It will use all cores of the computer, display a completion percentage and store all output messages in the file log. If your user can connect to several computers e.g. via SSH then you can even run commands in a distributed manner. I use it daily on Linux but know some people used it on Mac OS X as well. The project is there: https://savannah.nongnu.org/projects/par The paper is freely available there: http://bioinformatics.oxfordjournals.org/content/26/22/2918.long -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss
Re: [Open Babel] running Open Babel in parallel / distributed mode
On 2/4/14, 6:28 PM, Noel O'Boyle wrote: It would be nice to see some explicit examples of how Open Babel might be used in this way, using one or all of these tools. Let's say you have a file commmands.sh: --- obabel some_file1 -Osome_other_file1.other_format ... obabel some_file2 -Osome_other_file2.other_format --- With PAR, you run them like this in parallel: par -i commands.sh -v -o log # all your cores will be used by default to check the logs (in case you are really careful about what you are doing, e.g. when preparing datasets for scientific use), I recommend: sort -u log | less - Noel On 4 February 2014 00:52, Francois Berenger beren...@riken.jp wrote: On 02/04/2014 12:14 AM, Maciek Wójcikowski wrote: You can also use xargs. Yes, xargs with the -P option, but the command lines are not trivial then. Pozdrawiam, | Best regards, Maciek Wójcikowski mac...@wojcikowski.pl mailto:mac...@wojcikowski.pl 2014-02-03 16:10 GMT+01:00 Igor Filippov igor.v.filip...@gmail.com mailto:igor.v.filip...@gmail.com: How is it different from GNU parallel? http://www.gnu.org/software/bash/manual/html_node/GNU-Parallel.html It should be quite similar in functionality. Igor On Mon, Feb 3, 2014 at 1:37 AM, Francois Berenger beren...@riken.jp mailto:beren...@riken.jp wrote: Hello, I do this almost everyday so I think I should share it with this list. In case you need to execute many Open Babel commands and don't want to wait, you can execute them in parallel on a multi-core computer. Of course, the commands should be independent, for example processing different datasets. Let's say the commands are in a file called for_par.sh. I developped a tool called PAR years ago that can do this: par -i for_par.sh -v -o log It will use all cores of the computer, display a completion percentage and store all output messages in the file log. If your user can connect to several computers e.g. via SSH then you can even run commands in a distributed manner. I use it daily on Linux but know some people used it on Mac OS X as well. The project is there: https://savannah.nongnu.org/projects/par The paper is freely available there: http://bioinformatics.oxfordjournals.org/content/26/22/2918.long -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss
Re: [Open Babel] running Open Babel in parallel / distributed mode
Francois, could you please elaborate on the log argument? Because, that might be the only difference here. Using either gnu parallel or xargs standard output and error are FIFO, which makes them useless. Other than that I cant see any advantage, since parallel line would be: cat commands.sh | parallel -P 8 log 2 error.log (or 21 if someone prefers joint log) Pozdrawiam, | Best regards, Maciek Wójcikowski mac...@wojcikowski.pl 2014-02-04 11:58 GMT+01:00 Francois Berenger beren...@riken.jp: On 2/4/14, 6:28 PM, Noel O'Boyle wrote: It would be nice to see some explicit examples of how Open Babel might be used in this way, using one or all of these tools. Let's say you have a file commmands.sh: --- obabel some_file1 -Osome_other_file1.other_format ... obabel some_file2 -Osome_other_file2.other_format --- With PAR, you run them like this in parallel: par -i commands.sh -v -o log # all your cores will be used by default to check the logs (in case you are really careful about what you are doing, e.g. when preparing datasets for scientific use), I recommend: sort -u log | less - Noel On 4 February 2014 00:52, Francois Berenger beren...@riken.jp wrote: On 02/04/2014 12:14 AM, Maciek Wójcikowski wrote: You can also use xargs. Yes, xargs with the -P option, but the command lines are not trivial then. Pozdrawiam, | Best regards, Maciek Wójcikowski mac...@wojcikowski.pl mailto:mac...@wojcikowski.pl 2014-02-03 16:10 GMT+01:00 Igor Filippov igor.v.filip...@gmail.com mailto:igor.v.filip...@gmail.com: How is it different from GNU parallel? http://www.gnu.org/software/bash/manual/html_node/GNU- Parallel.html It should be quite similar in functionality. Igor On Mon, Feb 3, 2014 at 1:37 AM, Francois Berenger beren...@riken.jp mailto:beren...@riken.jp wrote: Hello, I do this almost everyday so I think I should share it with this list. In case you need to execute many Open Babel commands and don't want to wait, you can execute them in parallel on a multi-core computer. Of course, the commands should be independent, for example processing different datasets. Let's say the commands are in a file called for_par.sh. I developped a tool called PAR years ago that can do this: par -i for_par.sh -v -o log It will use all cores of the computer, display a completion percentage and store all output messages in the file log. If your user can connect to several computers e.g. via SSH then you can even run commands in a distributed manner. I use it daily on Linux but know some people used it on Mac OS X as well. The project is there: https://savannah.nongnu.org/projects/par The paper is freely available there: http://bioinformatics.oxfordjournals.org/content/26/ 22/2918.long -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231; iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231; iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net mailto:OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Best regards, Francois Berenger. -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231; iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss -- Managing the Performance of Cloud-Based
[Open Babel] Multimolecule canonical SMILES
Hi there, I have a number of multimolecular complexes that I'm converting into canonical SMILES. Some of these molecules are carbon monoxide, and I am noticing that sometimes they are written as [C]=O, and sometimes as O=[C]. Is the expected behavior of canonical SMILES - or should it instead be canonicalizing each molecule in the complex? Thanks, - Lee-Ping -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ OpenBabel-discuss mailing list OpenBabel-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openbabel-discuss
Re: [Open Babel] running Open Babel in parallel / distributed mode
On 02/04/2014 08:09 PM, Maciek Wójcikowski wrote:
Francois, could you please elaborate on the log argument? Because,
that might be the only difference here. Using either gnu parallel or
xargs standard output and error are FIFO, which makes them useless.
Other than that I cant see any advantage, since parallel line would be:
Here are the most useful PAR options:
# par
-i or -c is mandatory
Usage: parallel.py [options] {-i | -c} ...
Execute commands in a parallel and/or distributed way.
Options:
-h, --helpshow this help message and exit
-c SERVER_NAME, --client=SERVER_NAME
read commands from a server instead of a file
(incompatible with -i)
-i COMMANDS_FILE, --input=COMMANDS_FILE
/dev/stdin for example (incompatible with -c)
-o OUTPUT_FILE, --output=OUTPUT_FILE
log to a file instead of stdout
-s, --server accept remote workers
-v, --verbose enable progress bar
-w NB_LOCAL_WORKERS, --workers=NB_LOCAL_WORKERS
number of local worker threads, must be = 0,
default
is number of detected cores, very probably 0 if
your
OS is not Linux
The biggest difference I see is that parallel is a GNU project
and is packaged for several Linux distributions; so it can
be installed automatically. PAR could probably be packaged
as an easy_install package for Python.
It would be nice to see a benchmark of PAR versus GNU parallel.
Since I am the author of PAR, I don't switch to another software
because I can add features in mine as I wish. :)
cat commands.sh | parallel -P 8 log 2 error.log
(or 21 if someone prefers joint log)
Pozdrawiam, | Best regards,
Maciek Wójcikowski
mac...@wojcikowski.pl mailto:mac...@wojcikowski.pl
2014-02-04 11:58 GMT+01:00 Francois Berenger beren...@riken.jp
mailto:beren...@riken.jp:
On 2/4/14, 6:28 PM, Noel O'Boyle wrote:
It would be nice to see some explicit examples of how Open Babel
might
be used in this way, using one or all of these tools.
Let's say you have a file commmands.sh:
---
obabel some_file1 -Osome_other_file1.other___format
...
obabel some_file2 -Osome_other_file2.other___format
---
With PAR, you run them like this in parallel:
par -i commands.sh -v -o log # all your cores will be used by default
to check the logs (in case you are really careful about
what you are doing, e.g. when preparing datasets for scientific use),
I recommend:
sort -u log | less
- Noel
On 4 February 2014 00:52, Francois Berenger beren...@riken.jp
mailto:beren...@riken.jp wrote:
On 02/04/2014 12:14 AM, Maciek Wójcikowski wrote:
You can also use xargs.
Yes, xargs with the -P option, but the command lines are not
trivial then.
Pozdrawiam, | Best regards,
Maciek Wójcikowski
mac...@wojcikowski.pl
mailto:mac...@wojcikowski.pl mailto:mac...@wojcikowski.pl
mailto:mac...@wojcikowski.pl
2014-02-03 16:10 GMT+01:00 Igor Filippov
igor.v.filip...@gmail.com
mailto:igor.v.filip...@gmail.com
mailto:igor.v.filippov@gmail.__com
mailto:igor.v.filip...@gmail.com:
How is it different from GNU parallel?
http://www.gnu.org/software/__bash/manual/html_node/GNU-__Parallel.html
http://www.gnu.org/software/bash/manual/html_node/GNU-Parallel.html
It should be quite similar in functionality.
Igor
On Mon, Feb 3, 2014 at 1:37 AM, Francois Berenger
beren...@riken.jp mailto:beren...@riken.jp
mailto:beren...@riken.jp
mailto:beren...@riken.jp wrote:
Hello,
I do this almost everyday so I think I should
share it with this
list.
In case you need to execute many Open Babel
commands
and don't want to wait, you can execute them
in parallel
on a multi-core computer.
Of course, the commands should be independent,
for example
processing different datasets.
Let's say the commands are in a file called
for_par.sh.
I developped a tool called PAR years ago that
can do this:
par -i for_par.sh -v -o log
It will use all cores of the
