Re: [CentOS] parallel bash scripts
On 3/28/2012 8:53 PM, Ed Greshko wrote: On 03/29/2012 11:47 AM, Paul Allen Newell wrote: I apologize for taking up the bandwidth by asking, Yeah, it does some like a waste of bandwidth to ask questions to which you already know the answer and then have it read by "hundreds" of people that may feel they want to answer and helpbecause they really think you've asked a question to which you didn't already know the answer. Understood and accepted -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 03/29/2012 11:47 AM, Paul Allen Newell wrote: > I apologize for taking up the bandwidth by asking, Yeah, it does some like a waste of bandwidth to ask questions to which you already know the answer and then have it read by "hundreds" of people that may feel they want to answer and helpbecause they really think you've asked a question to which you didn't already know the answer. -- Do not condemn the judgment of another because it differs from your own. You may both be wrong. -- Dandemis -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 3/28/2012 8:43 PM, Ed Greshko wrote: On 03/29/2012 11:33 AM, Paul Allen Newell wrote: I've done enough googles to be pretty certain I was right (as your link does confirm), but I was asking for a confirm of what knew. Too much reading this thread of "2>&1" and making myself confused. That being said, your suggested link does give me one better than googling tcsh ... its nice to see the bourne and tcsh version in one place. And you have the power to test too echo "Hello"> x cat x /tmp/nosuchfilehere>& outfile Look to see what is in outfile Ed: I not only have tested, I've been using it for ages. As I said, I was real certain (I should have stated that as "through googling and testing"). I just wanted a confirm as I got scrambled reading the thread. I apologize for taking up the bandwidth by asking, Thanks, Paul -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 03/29/2012 11:33 AM, Paul Allen Newell wrote: > I've done enough googles to be pretty certain I was right (as your link does > confirm), but I was asking for a confirm of what knew. Too much reading this > thread > of "2>&1" and making myself confused. > > That being said, your suggested link does give me one better than googling > tcsh ... > its nice to see the bourne and tcsh version in one place. And you have the power to test too echo "Hello" > x cat x /tmp/nosuchfilehere >& outfile Look to see what is in outfile -- Do not condemn the judgment of another because it differs from your own. You may both be wrong. -- Dandemis -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 3/28/2012 8:27 PM, Ed Greshko wrote: On 03/29/2012 11:15 AM, Paul Allen Newell wrote: On 3/28/2012 4:39 PM, Mark LaPierre wrote: On Tue, Mar 27, 2012 at 6:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. Is anyone using tcsh and can tell me if I am correct in my understanding that "whatever>& whatever.out" catches stdout and stderr? You do know that a simple google of your question would provide you with the answer, right? :-) http://www.bo.infn.it/alice/alice-doc/mll-doc/usrgde/node18.html is just one of the many responses And in it you'd see & outfile # redirects stderr& stdout to outfile in csh, Tcsh Ed: I've done enough googles to be pretty certain I was right (as your link does confirm), but I was asking for a confirm of what knew. Too much reading this thread of "2>&1" and making myself confused. That being said, your suggested link does give me one better than googling tcsh ... its nice to see the bourne and tcsh version in one place. Thanks, Paul -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 03/29/2012 11:15 AM, Paul Allen Newell wrote: > On 3/28/2012 4:39 PM, Mark LaPierre wrote: >> >> >> On Tue, Mar 27, 2012 at 6:37 PM, Mark LaPierre wrote: >> >>> Check out the redirection at the end of each command. 1>&2 redirects >>> the standard out of your child command to the standard error which then >>> appears in the parent shell. > > Is anyone using tcsh and can tell me if I am correct in my understanding that > "whatever >& whatever.out" catches stdout and stderr? > > You do know that a simple google of your question would provide you with the answer, right? :-) http://www.bo.infn.it/alice/alice-doc/mll-doc/usrgde/node18.html is just one of the many responses And in it you'd see >& outfile # redirects stderr & stdout to outfile in csh, Tcsh -- Do not condemn the judgment of another because it differs from your own. You may both be wrong. -- Dandemis -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
On 3/28/2012 4:39 PM, Mark LaPierre wrote: On Tue, Mar 27, 2012 at 6:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. Is anyone using tcsh and can tell me if I am correct in my understanding that "whatever >& whatever.out" catches stdout and stderr? Thanks, Paul -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: [CentOS] parallel bash scripts
I solved a similar problem by installing gnu parallel on my system. It did everything that I wanted, and better than I would have coded. Ali On Tue, Mar 27, 2012 at 6:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. At the end the last & launches your command into a background shell and then moves on to launch the next command. The redirections don't care if the command ever terminates. The result is that both commands are launched and the parent shell terminates leaving the standard error attached to the terminal that the parent was launched in. On 03/27/2012 09:08 PM, bruce wrote: > marklap...@aol.com > > hey mark > > what you have, appears to be pretty close to what i had... except my > tests never ended... the loops are infinite... > > can i do a fpaste and have you take a look at what i have? > > -btuce > > > On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre wrote: >> On 03/27/2012 05:25 PM, bruce wrote: >>> >>> hi. >>> >>> got a couple of test bash scripts. >>> >>> dog.sh, cat.sh >>> each script runs the underlying php in an endless loop. >>> >>> I'm trying to figure out how to run the scripts in parallel, from the >>> same parent shell script. something like: >>> >>> test.sh >>> >>> >>> where dog.sh would be : >>> >>> while true >>> do >>>pgrep dog >>>if [ $? -ne 0 ] >>>then >>> /dog.php >>>fi >>> sleep 5 >>> done >>> >>> my current tests, run dog.sh, which runs the dog.php ... but the test >>> never gets to run cat.sh >>> >>> thoughts/comments... >>> >>> thanks >> >> Hey Bruce, >> >> Do you mean to run these subprograms in parallel or in series? >> >> cat.sh >> #! /bin/bash >> >> CAT=0 >> until [ $CAT -eq 10 ] >> do >> echo "Inside a dog it's too dark to read. $CAT" >> CAT=$[$CAT + 1] >> sleep 2 >> done >> >> >> dog.sh >> #! /bin/bash >> >> DOG=0 >> until [ $DOG -eq 10 ] >> do >> echo "Next to a dog a book is man's best friend. $DOG" >> DOG=$[$DOG + 1] >> sleep 2 >> done >> >> >> test.sh >> #! /bin/sh >> >> /home/mlapier/test/dog.sh 1>&2& >> /home/mlapier/test/cat.sh 1>&2& >> >> >> [mlapier@mushroom test]$ ./test.sh >> [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0 >> Inside a dog it's too dark to read. 0 >> Next to a dog a book is man's best friend. 1 >> Inside a dog it's too dark to read. 1 >> Next to a dog a book is man's best friend. 2 >> Inside a dog it's too dark to read. 2 >> Next to a dog a book is man's best friend. 3 >> Inside a dog it's too dark to read. 3 >> Next to a dog a book is man's best friend. 4 >> Inside a dog it's too dark to read. 4 >> Next to a dog a book is man's best friend. 5 >> Inside a dog it's too dark to read. 5 >> Next to a dog a book is man's best friend. 6 >> Inside a dog it's too dark to read. 6 >> Next to a dog a book is man's best friend. 7 >> Inside a dog it's too dark to read. 7 >> Next to a dog a book is man's best friend. 8 >> Inside a dog it's too dark to read. 8 >> Next to a dog a book is man's best friend. 9 >> Inside a dog it's too dark to read. 9 >> >> >> >> -- >> users mailing list >> users@lists.fedoraproject.org >> To unsubscribe or change subscription options: >> https://admin.fedoraproject.org/mailman/listinfo/users >> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines >> Have a question? Ask away: http://ask.fedoraproject.org ___ CentOS mailing list cen...@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list cen...@centos.org http://lists.centos.org/mailman/listinfo/centos -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On 03/27/2012 10:35 PM, bruce wrote: Hi mark. I have a series of php apps that are async, and modify different tbls within a mysql db. The php apps also perform other tasks as well. The process needs to be able to periodically run the php apps. The shell scripts are used to run the individual php apps. During testing, I wanted to be able to run the apps, in a background manner, as well as display the output for debugging. The system has ~5-10 php apps that will be running. The overall goal for all of this is part of a targeted web crawler, where the php apps perform backend processing to manage the data from the crawling process as it gets stored in the database. So the process should fire up all of the processes, and they should then run every X secs... At least thats the way I ultimately want it to work. But I'm unable to figure out how to do this, and at the same time, display the output for the debugging process... Hopefully I've done a better job of explaining some of what this is for.. On Tue, Mar 27, 2012 at 10:23 PM, Mark LaPierre wrote: I don't understand what you are trying to do. Do you want to run these commands in series? Run one, wait for it to complete, then start the other? If that's where you want to go then leave off the trailing ampersand but keep the redirection. The output of the first program will appear in the terminal due to the redirection. When the first program ends the second program will start. That one needs redirection of standard out to standard error too so that it's output will appear in the terminal as well. Solution with trailing ampersand is appropriate for endless loops. Solution without trailing ampersand is appropriate for serial operation. I put the loop counter in the demo program just to stop the thing without having to us ps and kill. Without the loop counter they will continue to print inter-twined messages forever. Does the program webservice_setup.sh start programs itself that you are trying to read the output from? Are they correctly redirected to send their standard output to standard error? On 03/27/2012 10:01 PM, bruce wrote: Hi mark. the following is part of what i have tried... and what isn't working. i changed the test shell scripts to no longer have infinite loops. they simply execute the php app one time. however, it still doesn't work. in the initial shell script, it calls a php app that does some mysql processes. the php app has a number of echo/print msgs to track what's going on for testing.. when running as a background process the shell script/php app isn't working. the cmdline sits.. when i select the "return" key.. the process is diplayed as being stopped... if i remove the background "&" the shell script runs... -- not working /apps/parseapp2/webservice_setup.sh1>&2& -- works /apps/parseapp2/webservice_setup.sh /apps/parseapp2/webservice_setup.sh --- echo "starting the osu webservice test" mysql -uroot1 -pfoo&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. At the end the last&launches your command into a background shell and then moves on to launch the next command. The redirections don't care if the command ever terminates. The result is that both commands are launched and the parent shell terminates leaving the standard error attached to the terminal that the parent was launched in. On 03/27/2012 09:08 PM, bruce wrote: marklap...@aol.com hey mark what you have, appears to be pretty close to what i had... except my tests never ended... the loops are infinite... can i do a fpaste and have you take a look at what i have? -btuce On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre wrote: On 03/27/2012 05:25 PM, bruce wrote: hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks Hey Bruce, Do you mean to run these subprograms in parallel or in series? cat.sh #! /bin/bash CAT=0 until [ $CAT -eq 10 ] do echo "Inside a dog it's too dark to read. $CAT" CAT=$[$CAT + 1] sleep 2 done dog.sh #! /bin/bash DOG=0 until [ $DOG -eq 10 ] do echo "Next to a dog a book is man's best friend. $DOG" DOG=$[$DOG + 1] sleep 2 do
Fwd: Re: parallel bash scripts
Hi mark. I have a series of php apps that are async, and modify different tbls within a mysql db. The php apps also perform other tasks as well. The process needs to be able to periodically run the php apps. The shell scripts are used to run the individual php apps. During testing, I wanted to be able to run the apps, in a background manner, as well as display the output for debugging. The system has ~5-10 php apps that will be running. The overall goal for all of this is part of a targeted web crawler, where the php apps perform backend processing to manage the data from the crawling process as it gets stored in the database. So the process should fire up all of the processes, and they should then run every X secs... At least thats the way I ultimately want it to work. But I'm unable to figure out how to do this, and at the same time, display the output for the debugging process... Hopefully I've done a better job of explaining some of what this is for.. On Tue, Mar 27, 2012 at 10:23 PM, Mark LaPierre wrote: I don't understand what you are trying to do. Do you want to run these commands in series? Run one, wait for it to complete, then start the other? If that's where you want to go then leave off the trailing ampersand but keep the redirection. The output of the first program will appear in the terminal due to the redirection. When the first program ends the second program will start. That one needs redirection of standard out to standard error too so that it's output will appear in the terminal as well. Solution with trailing ampersand is appropriate for endless loops. Solution without trailing ampersand is appropriate for serial operation. I put the loop counter in the demo program just to stop the thing without having to us ps and kill. Without the loop counter they will continue to print inter-twined messages forever. Does the program webservice_setup.sh start programs itself that you are trying to read the output from? Are they correctly redirected to send their standard output to standard error? On 03/27/2012 10:01 PM, bruce wrote: Hi mark. the following is part of what i have tried... and what isn't working. i changed the test shell scripts to no longer have infinite loops. they simply execute the php app one time. however, it still doesn't work. in the initial shell script, it calls a php app that does some mysql processes. the php app has a number of echo/print msgs to track what's going on for testing.. when running as a background process the shell script/php app isn't working. the cmdline sits.. when i select the "return" key.. the process is diplayed as being stopped... if i remove the background "&" the shell script runs... -- not working /apps/parseapp2/webservice_setup.sh1>&2& -- works /apps/parseapp2/webservice_setup.sh /apps/parseapp2/webservice_setup.sh --- echo "starting the osu webservice test" mysql -uroot1 -pfoo< /apps/parseapp2/setup_tblDb.sql mysql -uroot1 -pfoo< /apps/parseapp2/webservice/webservice_tbls.sql /apps/parseapp2/parse_tbl_setup.php echo "started the osu webservice test" thoughts??? thanks On Tue, Mar 27, 2012 at 9:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. At the end the last& launches your command into a background shell and then moves on to launch the next command. The redirections don't care if the command ever terminates. The result is that both commands are launched and the parent shell terminates leaving the standard error attached to the terminal that the parent was launched in. On 03/27/2012 09:08 PM, bruce wrote: marklap...@aol.com hey mark what you have, appears to be pretty close to what i had... except my tests never ended... the loops are infinite... can i do a fpaste and have you take a look at what i have? -btuce On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre wrote: On 03/27/2012 05:25 PM, bruce wrote: hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks Hey Bruce, Do you mean to run these subprograms in parallel or in series? cat.sh #! /bin/bash CAT=0 until [ $CAT -eq 10 ] do echo "Inside a dog it's too dark to read. $CAT" CAT=$[$CAT + 1] sleep 2 done dog.sh #! /bin/bash DOG=0 until [ $DOG -eq 10 ] do echo "Next to a dog a book is man's best friend. $DOG" DOG=$[$DOG + 1] sleep 2 done test.sh #! /bin/sh /home/mlapier/test/dog.sh 1
Re: parallel bash scripts
On 27Mar2012 17:56, Rick Stevens wrote: | On 03/27/2012 04:06 PM, bruce wrote: | The problem is that a script wants a stdin, stdout and stderr. No evidence so far that they use stdin. | If | you're going to run them in parallel, you can't very well have both | scripts outputting to the terminal. Sure you can. Why not? I do it all the time with multiple tails etc. | The correct thing is to do what Dave said. Use nohup and redirect stdout | and stderr to a log file. You can then "tail -f" each one whenever you | want. Background and disown is better, usually. I'm finding nohup more and more painful; always the same output file, stdout and stderr glommed together etc. | An alternative is to launch the scripts in detached screen | sessions: | | screen -d -m dog.sh | screen -d -m cat.sh | | This would launch each script in its own detached screen session | (essentially creates virtual terminals with shells and runs the script | in them) and the calling script doesn't wait for them to complete. You can get screen session names, too. Much easier to work with than the ids you get by default. % screen -ls There are screens on: 2635.GETMAIL(Detached) 17863.CP_VIDEO (Detached) 6557.OI (Detached) 1247.BACKUP (Detached) 2203.BEYONWIZ (Detached) 9125.EMERGE (Detached) 15806.mutt-28mar2012-16:23 Re_parallel_bash_scr (Attached) 7 Sockets in /tmp/screen-cameron. That last is actually the email editor I'm using right now. Can detach and pick up again later! But: dog.sh dog.out 2>dog.err & cat.sh cat.out 2>cat.err & disown %1 %2 is simple and basic. Adjust %1 and %2 to the job numbers you get; see the "jobs" command to get current background jobs. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ To understand recursion, you must first understand recursion. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
I don't understand what you are trying to do. Do you want to run these commands in series? Run one, wait for it to complete, then start the other? If that's where you want to go then leave off the trailing ampersand but keep the redirection. The output of the first program will appear in the terminal due to the redirection. When the first program ends the second program will start. That one needs redirection of standard out to standard error too so that it's output will appear in the terminal as well. Solution with trailing ampersand is appropriate for endless loops. Solution without trailing ampersand is appropriate for serial operation. I put the loop counter in the demo program just to stop the thing without having to us ps and kill. Without the loop counter they will continue to print inter-twined messages forever. Does the program webservice_setup.sh start programs itself that you are trying to read the output from? Are they correctly redirected to send their standard output to standard error? On 03/27/2012 10:01 PM, bruce wrote: Hi mark. the following is part of what i have tried... and what isn't working. i changed the test shell scripts to no longer have infinite loops. they simply execute the php app one time. however, it still doesn't work. in the initial shell script, it calls a php app that does some mysql processes. the php app has a number of echo/print msgs to track what's going on for testing.. when running as a background process the shell script/php app isn't working. the cmdline sits.. when i select the "return" key.. the process is diplayed as being stopped... if i remove the background "&" the shell script runs... -- not working /apps/parseapp2/webservice_setup.sh1>&2& -- works /apps/parseapp2/webservice_setup.sh /apps/parseapp2/webservice_setup.sh --- echo "starting the osu webservice test" mysql -uroot1 -pfoo< /apps/parseapp2/setup_tblDb.sql mysql -uroot1 -pfoo< /apps/parseapp2/webservice/webservice_tbls.sql /apps/parseapp2/parse_tbl_setup.php echo "started the osu webservice test" thoughts??? thanks On Tue, Mar 27, 2012 at 9:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. At the end the last& launches your command into a background shell and then moves on to launch the next command. The redirections don't care if the command ever terminates. The result is that both commands are launched and the parent shell terminates leaving the standard error attached to the terminal that the parent was launched in. On 03/27/2012 09:08 PM, bruce wrote: marklap...@aol.com hey mark what you have, appears to be pretty close to what i had... except my tests never ended... the loops are infinite... can i do a fpaste and have you take a look at what i have? -btuce On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierrewrote: On 03/27/2012 05:25 PM, bruce wrote: hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks Hey Bruce, Do you mean to run these subprograms in parallel or in series? cat.sh #! /bin/bash CAT=0 until [ $CAT -eq 10 ] do echo "Inside a dog it's too dark to read. $CAT" CAT=$[$CAT + 1] sleep 2 done dog.sh #! /bin/bash DOG=0 until [ $DOG -eq 10 ] do echo "Next to a dog a book is man's best friend. $DOG" DOG=$[$DOG + 1] sleep 2 done test.sh #! /bin/sh /home/mlapier/test/dog.sh 1>&2& /home/mlapier/test/cat.sh 1>&2& [mlapier@mushroom test]$ ./test.sh [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0 Inside a dog it's too dark to read. 0 Next to a dog a book is man's best friend. 1 Inside a dog it's too dark to read. 1 Next to a dog a book is man's best friend. 2 Inside a dog it's too dark to read. 2 Next to a dog a book is man's best friend. 3 Inside a dog it's too dark to read. 3 Next to a dog a book is man's best friend. 4 Inside a dog it's too dark to read. 4 Next to a dog a book is man's best friend. 5 Inside a dog it's too dark to read. 5 Next to a dog a book is man's best friend. 6 Inside a dog it's too dark to read. 6 Next to a dog a book is man's best friend. 7 Inside a dog it's too dark to read. 7 Next to a dog a book is man's best friend. 8 Inside a dog it's too dark to read. 8 Next to a dog a book is man's best friend. 9 Inside a dog it's too dark to read. 9 -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription o
Re: parallel bash scripts
On 03/27/2012 09:37 PM, Mark LaPierre wrote: Check out the redirection at the end of each command. 1>&2 redirects the standard out of your child command to the standard error which then appears in the parent shell. At the end the last & launches your command into a background shell and then moves on to launch the next command. The redirections don't care if the command ever terminates. The result is that both commands are launched and the parent shell terminates leaving the standard error attached to the terminal that the parent was launched in. On 03/27/2012 09:08 PM, bruce wrote: marklap...@aol.com hey mark what you have, appears to be pretty close to what i had... except my tests never ended... the loops are infinite... can i do a fpaste and have you take a look at what i have? -btuce On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre wrote: On 03/27/2012 05:25 PM, bruce wrote: hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks Hey Bruce, Do you mean to run these subprograms in parallel or in series? cat.sh #! /bin/bash CAT=0 until [ $CAT -eq 10 ] do echo "Inside a dog it's too dark to read. $CAT" CAT=$[$CAT + 1] sleep 2 done dog.sh #! /bin/bash DOG=0 until [ $DOG -eq 10 ] do echo "Next to a dog a book is man's best friend. $DOG" DOG=$[$DOG + 1] sleep 2 done test.sh #! /bin/sh /home/mlapier/test/dog.sh 1>&2& /home/mlapier/test/cat.sh 1>&2& [mlapier@mushroom test]$ ./test.sh [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0 Inside a dog it's too dark to read. 0 Next to a dog a book is man's best friend. 1 Inside a dog it's too dark to read. 1 Next to a dog a book is man's best friend. 2 Inside a dog it's too dark to read. 2 Next to a dog a book is man's best friend. 3 Inside a dog it's too dark to read. 3 Next to a dog a book is man's best friend. 4 Inside a dog it's too dark to read. 4 Next to a dog a book is man's best friend. 5 Inside a dog it's too dark to read. 5 Next to a dog a book is man's best friend. 6 Inside a dog it's too dark to read. 6 Next to a dog a book is man's best friend. 7 Inside a dog it's too dark to read. 7 Next to a dog a book is man's best friend. 8 Inside a dog it's too dark to read. 8 Next to a dog a book is man's best friend. 9 Inside a dog it's too dark to read. 9 -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
hey rick... haven't talked to you in a very long time! -- i tried the nohup, but couldn't get it to work because of the fact that my processes have infinite loops.. so my test never got beyond running the 1st test shell script that's running an infinite loop... i can do a paste of what i'm testing if you have a few minutes to take a look on this thanks bruce badoug...@gmail.com On Tue, Mar 27, 2012 at 8:56 PM, Rick Stevens wrote: > On 03/27/2012 04:06 PM, bruce wrote: >> >> dave.. >> >> thanks. >> >> but the script is a long running script, and i want to see the output >> as the script is running without having to hit the keyboard.. >> >> so doing a tail of the file while the process is running isn't what >> i'm looking for. >> >> anything else? > > > The problem is that a script wants a stdin, stdout and stderr. If > you're going to run them in parallel, you can't very well have both > scripts outputting to the terminal. > > The correct thing is to do what Dave said. Use nohup and redirect stdout > and stderr to a log file. You can then "tail -f" each one whenever you > want. An alternative is to launch the scripts in detached screen > sessions: > > screen -d -m dog.sh > screen -d -m cat.sh > > This would launch each script in its own detached screen session > (essentially creates virtual terminals with shells and runs the script > in them) and the calling script doesn't wait for them to complete. > > You can then "screen -r " to attach to those screen sessions. You > can get the list of screen sessions running simply by doing "screen -r" > without specifying a session ID. Use "ctrl-A, ctrl-D" to detach from > a screen session (but leave it running). See "man screen" for details. > > Oh, yeah, you may need to "yum install screen" if you don't already have > it. > >> On Tue, Mar 27, 2012 at 7:00 PM, Dave Ihnat wrote: >>> >>> On Tue, Mar 27, 2012 at 06:06:56PM -0400, bruce wrote: anyway i can redirect the err/out to the stdout.. instead of the nohut.out file?? >>> >>> >>> The usual way would be to do a tail -f on the output file, e.g., >>> >>> Â nohup cat.sh 2>&1>cat.log >>> Â tail -f cat.log >>> >>> >>> If you're using bash, IIRC, you could reap the background ID and then use >>> 'disown', I suppose. Â "$!" should give you that. >>> >>> Cheers, >>> -- >>> Â Â Â Â Dave Ihnat >>> Â Â Â Â President, DMINET Consulting, Inc. >>> >>> -- >>> users mailing list >>> users@lists.fedoraproject.org >>> To unsubscribe or change subscription options: >>> https://admin.fedoraproject.org/mailman/listinfo/users >>> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines >>> Have a question? Ask away: http://ask.fedoraproject.org > > > > -- > -- > - Rick Stevens, Systems Engineer, AllDigital ri...@alldigital.com - > - AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 - > - - > - We are born naked, wet and hungry. Then things get worse. - > -- > > -- > users mailing list > users@lists.fedoraproject.org > To unsubscribe or change subscription options: > https://admin.fedoraproject.org/mailman/listinfo/users > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines > Have a question? Ask away: http://ask.fedoraproject.org -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On 03/27/2012 05:25 PM, bruce wrote: hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks Hey Bruce, Do you mean to run these subprograms in parallel or in series? cat.sh #! /bin/bash CAT=0 until [ $CAT -eq 10 ] do echo "Inside a dog it's too dark to read. $CAT" CAT=$[$CAT + 1] sleep 2 done dog.sh #! /bin/bash DOG=0 until [ $DOG -eq 10 ] do echo "Next to a dog a book is man's best friend. $DOG" DOG=$[$DOG + 1] sleep 2 done test.sh #! /bin/sh /home/mlapier/test/dog.sh 1>&2 & /home/mlapier/test/cat.sh 1>&2 & [mlapier@mushroom test]$ ./test.sh [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0 Inside a dog it's too dark to read. 0 Next to a dog a book is man's best friend. 1 Inside a dog it's too dark to read. 1 Next to a dog a book is man's best friend. 2 Inside a dog it's too dark to read. 2 Next to a dog a book is man's best friend. 3 Inside a dog it's too dark to read. 3 Next to a dog a book is man's best friend. 4 Inside a dog it's too dark to read. 4 Next to a dog a book is man's best friend. 5 Inside a dog it's too dark to read. 5 Next to a dog a book is man's best friend. 6 Inside a dog it's too dark to read. 6 Next to a dog a book is man's best friend. 7 Inside a dog it's too dark to read. 7 Next to a dog a book is man's best friend. 8 Inside a dog it's too dark to read. 8 Next to a dog a book is man's best friend. 9 Inside a dog it's too dark to read. 9 -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On Wed, Mar 28, 2012 at 01:06, bruce wrote: > but the script is a long running script, and i want to see the output > as the script is running without having to hit the keyboard.. I'm not sure what you mean. Do you mean to say you want to run the script and start seeing the output with the same command? If that is the case, there are several solutions. I usually use these two: $ nohup myscript &> myscript.log & less +F myscript.log $ nohup myscript 2>&1 | tee myscript.log If you actually want to run both scripts in the same shell and want to follow both outputs, try the following variation of the first command: $ nohup myscript1 &> myscript1.log & nohup myscript2 &> myscript2.log & less +F myscript1.log myscript2.log Then you can stop following the first log by hitting Ctrl+c and move on to the next file with :n you can then start following with F. You can go back again by repeating the same except change :n to :p. Hope this helps. -- Suvayu Open source is the future. It sets us free. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On 03/27/2012 04:06 PM, bruce wrote: dave.. thanks. but the script is a long running script, and i want to see the output as the script is running without having to hit the keyboard.. so doing a tail of the file while the process is running isn't what i'm looking for. anything else? The problem is that a script wants a stdin, stdout and stderr. If you're going to run them in parallel, you can't very well have both scripts outputting to the terminal. The correct thing is to do what Dave said. Use nohup and redirect stdout and stderr to a log file. You can then "tail -f" each one whenever you want. An alternative is to launch the scripts in detached screen sessions: screen -d -m dog.sh screen -d -m cat.sh This would launch each script in its own detached screen session (essentially creates virtual terminals with shells and runs the script in them) and the calling script doesn't wait for them to complete. You can then "screen -r " to attach to those screen sessions. You can get the list of screen sessions running simply by doing "screen -r" without specifying a session ID. Use "ctrl-A, ctrl-D" to detach from a screen session (but leave it running). See "man screen" for details. Oh, yeah, you may need to "yum install screen" if you don't already have it. On Tue, Mar 27, 2012 at 7:00 PM, Dave Ihnat wrote: On Tue, Mar 27, 2012 at 06:06:56PM -0400, bruce wrote: anyway i can redirect the err/out to the stdout.. instead of the nohut.out file?? The usual way would be to do a tail -f on the output file, e.g.,  nohup cat.sh 2>&1>cat.log  tail -f cat.log If you're using bash, IIRC, you could reap the background ID and then use 'disown', I suppose.  "$!" should give you that. Cheers, --     Dave Ihnat     President, DMINET Consulting, Inc. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org -- -- - Rick Stevens, Systems Engineer, AllDigitalri...@alldigital.com - - AIM/Skype: therps2ICQ: 22643734Yahoo: origrps2 - -- - We are born naked, wet and hungry. Then things get worse. - -- -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
dave.. thanks. but the script is a long running script, and i want to see the output as the script is running without having to hit the keyboard.. so doing a tail of the file while the process is running isn't what i'm looking for. anything else? thanks On Tue, Mar 27, 2012 at 7:00 PM, Dave Ihnat wrote: > On Tue, Mar 27, 2012 at 06:06:56PM -0400, bruce wrote: >> anyway i can redirect the err/out to the stdout.. instead of the >> nohut.out file?? > > The usual way would be to do a tail -f on the output file, e.g., > > nohup cat.sh 2>&1 >cat.log > tail -f cat.log > > If you're using bash, IIRC, you could reap the background ID and then use > 'disown', I suppose. "$!" should give you that. > > Cheers, > -- > Dave Ihnat > President, DMINET Consulting, Inc. > -- > users mailing list > users@lists.fedoraproject.org > To unsubscribe or change subscription options: > https://admin.fedoraproject.org/mailman/listinfo/users > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines > Have a question? Ask away: http://ask.fedoraproject.org -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On Tue, Mar 27, 2012 at 06:06:56PM -0400, bruce wrote: > anyway i can redirect the err/out to the stdout.. instead of the > nohut.out file?? The usual way would be to do a tail -f on the output file, e.g., nohup cat.sh 2>&1 >cat.log tail -f cat.log If you're using bash, IIRC, you could reap the background ID and then use 'disown', I suppose. "$!" should give you that. Cheers, -- Dave Ihnat President, DMINET Consulting, Inc. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
hey dave... anyway i can redirect the err/out to the stdout.. instead of the nohut.out file?? it appears that the processes are running in the ps tb.. the nohut.out file also has input/data.. i'd like to be able to see the output of the processes scroll on the term/screen if possible thanks On Tue, Mar 27, 2012 at 5:58 PM, Dave Ihnat wrote: > On Tue, Mar 27, 2012 at 05:25:56PM -0400, bruce wrote: >> got a couple of test bash scripts. >> ... >> I'm trying to figure out how to run the scripts in parallel, from the >> same parent shell script. something like: > > Have you tried > > nohup cat.sh 2>&1& > nohup dog.sh 2>&1& > > Cheers, > -- > Dave Ihnat > -- > users mailing list > users@lists.fedoraproject.org > To unsubscribe or change subscription options: > https://admin.fedoraproject.org/mailman/listinfo/users > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines > Have a question? Ask away: http://ask.fedoraproject.org -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
On Tue, Mar 27, 2012 at 05:25:56PM -0400, bruce wrote: > got a couple of test bash scripts. > ... > I'm trying to figure out how to run the scripts in parallel, from the > same parent shell script. something like: Have you tried nohup cat.sh 2>&1& nohup dog.sh 2>&1& Cheers, -- Dave Ihnat -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
HI Mateusz Yeah, tried the basic "&" ... but the issue is the shell scripts run as infinite loops.. and therefore, the dog.sh doesn't really exit/complete. So I can't really run them sequentially. If I open up separate term/windows, then of course, I can manually run them (one in each window).. thanks On Tue, Mar 27, 2012 at 5:46 PM, Mateusz Marzantowicz wrote: > W dniu 27.03.2012 23:25, bruce pisze: >> hi. >> >> got a couple of test bash scripts. >> >> dog.sh, cat.sh >> each script runs the underlying php in an endless loop. >> >> I'm trying to figure out how to run the scripts in parallel, from the >> same parent shell script. something like: >> >> test.sh >> >> dog.sh > &2 >> cat.sh > &2 >> >> where dog.sh would be : >> >> while true >> do >> pgrep dog >> if [ $? -ne 0 ] >> then >> /dog.php >> fi >> sleep 5 >> done >> >> my current tests, run dog.sh, which runs the dog.php ... but the test >> never gets to run cat.sh >> >> thoughts/comments... >> >> thanks > Did you try: > > command1 & command2 > > where command1 and command2 are bash built in commands or scripts (in > your case dog.sh and cat.sh)? > > > Mateusz Marzantowicz > -- > users mailing list > users@lists.fedoraproject.org > To unsubscribe or change subscription options: > https://admin.fedoraproject.org/mailman/listinfo/users > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines > Have a question? Ask away: http://ask.fedoraproject.org -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
Re: parallel bash scripts
W dniu 27.03.2012 23:25, bruce pisze: > hi. > > got a couple of test bash scripts. > > dog.sh, cat.sh > each script runs the underlying php in an endless loop. > > I'm trying to figure out how to run the scripts in parallel, from the > same parent shell script. something like: > > test.sh > > dog.sh > &2 > cat.sh > &2 > > where dog.sh would be : > > while true > do > pgrep dog > if [ $? -ne 0 ] > then > /dog.php > fi > sleep 5 > done > > my current tests, run dog.sh, which runs the dog.php ... but the test > never gets to run cat.sh > > thoughts/comments... > > thanks Did you try: command1 & command2 where command1 and command2 are bash built in commands or scripts (in your case dog.sh and cat.sh)? Mateusz Marzantowicz -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
parallel bash scripts
hi. got a couple of test bash scripts. dog.sh, cat.sh each script runs the underlying php in an endless loop. I'm trying to figure out how to run the scripts in parallel, from the same parent shell script. something like: test.sh dog.sh > &2 cat.sh > &2 where dog.sh would be : while true do pgrep dog if [ $? -ne 0 ] then /dog.php fi sleep 5 done my current tests, run dog.sh, which runs the dog.php ... but the test never gets to run cat.sh thoughts/comments... thanks -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
