Re: [Toolserver-l] Cronjob starting cronsub
On Tue, Mar 1, 2011 at 12:49 AM, MZMcBride wrote: > Magnus Manske wrote: >> I understand that, and I didn't say lfs was better (it has plenty of >> its own quirks), or that SGE was a bad choice - it seems to run well >> enough, all things considered. It's just that things like "SGE might >> interpret comments in your scripts as commands and die" or "you can't >> run qsub from within qsub" are types of behaviour that are unusual to >> say the least, and the toolserver users are left to the mercy of these >> issues (I'd call them bugs) without warning or documentation. (Yes, >> there is a wiki pages that says it can interpret special comments in >> your script. That's somewhat different from "it will reinterpret your >> Perl script and fall over"). > > Having learned some lessons today, you should write improved documentation > on the wiki. :-) I might, once I've recovered from said "learning" experience... Magnus ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
Magnus Manske wrote: > I understand that, and I didn't say lfs was better (it has plenty of > its own quirks), or that SGE was a bad choice - it seems to run well > enough, all things considered. It's just that things like "SGE might > interpret comments in your scripts as commands and die" or "you can't > run qsub from within qsub" are types of behaviour that are unusual to > say the least, and the toolserver users are left to the mercy of these > issues (I'd call them bugs) without warning or documentation. (Yes, > there is a wiki pages that says it can interpret special comments in > your script. That's somewhat different from "it will reinterpret your > Perl script and fall over"). Having learned some lessons today, you should write improved documentation on the wiki. :-) MZMcBride ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article , Magnus Manske wrote: > Thankfully, Marcin was actually helpful in answering this. Your > attempt to distract from the actual issues at hand by "literal > interpretation" has been duly noted. I'm sorry you didn't find my post helpful, but I don't see any need for this sort of attitude. I'm trying to help you. > It's just that things like "SGE might interpret comments in your > scripts as commands and die" or "you can't run qsub from within qsub" > are types of behaviour that are unusual to say the least, and the > toolserver users are left to the mercy of these issues (I'd call them > bugs) without warning or documentation. Recursive qsub not working was not a known issue until you posted it on the list just now, so it's unreasonable to complain about lack of documentation -- there's no way we can document an issue we don't even know about. You are also the first person to have a problem with an unrelated #$ in a script. Now that we can see that might be an issue, the documentation can be updated. The documentation is a wiki, so perhaps *you* could update it if you don't think it's very good. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1r0EwACgkQIXd7fCuc5vIXxwCgiDpQKCFf14bnWJrDyAujDIkr P7gAn2TNI3NrIEOZUfm3/xvZ0pVMK48t =1TEW -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 3:24 PM, River Tarnell wrote: > In article , > Magnus Manske wrote: >> Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo >> meaning from my scripts (which predate qsub availability) anyway? > > Because it means you can include directives in your script indicating > how it should run: > > #! /bin/ksh > #$ -o $HOME/mytool.out > #$ -l h_rt=3:0:0 > #$ -t 1-10 > #$ -tc 3 > # rest of script... > > This is easier than having a separate script just to start that one. And more complicated than passing these parameters to qsub directly. >> Is there at least a qsub option to turn it off? > > Yes. Thankfully, Marcin was actually helpful in answering this. Your attempt to distract from the actual issues at hand by "literal interpretation" has been duly noted. >> In my "day job", I work with a large compute farm (lfs, ~10K cores), >> so I'm more than happy to work with, and migrate existing cronjobs to, >> this system, but it would help if it weren't a deliberate minefield... > > I didn't write SGE. I considered several existing batch scheduling > systems and SGE seemed to be the best/most useful. Of course that > doesn't mean every user will like every feature it has. I understand that, and I didn't say lfs was better (it has plenty of its own quirks), or that SGE was a bad choice - it seems to run well enough, all things considered. It's just that things like "SGE might interpret comments in your scripts as commands and die" or "you can't run qsub from within qsub" are types of behaviour that are unusual to say the least, and the toolserver users are left to the mercy of these issues (I'd call them bugs) without warning or documentation. (Yes, there is a wiki pages that says it can interpret special comments in your script. That's somewhat different from "it will reinterpret your Perl script and fall over"). Magnus ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article , Magnus Manske wrote: > Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo > meaning from my scripts (which predate qsub availability) anyway? Because it means you can include directives in your script indicating how it should run: #! /bin/ksh #$ -o $HOME/mytool.out #$ -l h_rt=3:0:0 #$ -t 1-10 #$ -tc 3 # rest of script... This is easier than having a separate script just to start that one. > If that is desired for some cases, why is that activated by default? With SGE, it's usually desirable for *all* cases (or at least the vast majority). The cronsub script makes some of the most common uses unnecessary (e.g. options controlling output file), so it's used less there. However, turning it off for cronsub while it was enabled for qsub seemed like it would be more confusing for users. > Is there at least a qsub option to turn it off? Yes. > Will that option work with cronsub? No. But you can invoke qsub yourself if you prefer that. (This is a known issue but it's somewhat difficult to fix because of how cronsub works.) > In my "day job", I work with a large compute farm (lfs, ~10K cores), > so I'm more than happy to work with, and migrate existing cronjobs to, > this system, but it would help if it weren't a deliberate minefield... I didn't write SGE. I considered several existing batch scheduling systems and SGE seemed to be the best/most useful. Of course that doesn't mean every user will like every feature it has. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 3:01 PM, Marcin Cieslak wrote: >>> Magnus Manske wrote: >> On Mon, Feb 28, 2011 at 11:43 AM, Ilmari Karonen wrote: >>> On 02/28/2011 01:00 PM, Magnus Manske wrote: OK, so I gave up on starting my perl script using cronsub, which seems to work only if I give the perl script a parameter (WTF?) >>> >>> Any parameter? I smell a line break encoding issue. >>> >>> Try making sure there are no carriage returns in your crontab or in any >>> of the scripts involved. >> >> Well, since that happens also when I start cronsub fron the shell, I doubt >> it... > > It seems like qsub dumps core when executed from inside of the SGE job. > > /sge62/default/spool/wolfsbane/job_scripts/210756[3]: 1253 Segmentation > Fault(coredump) > > I have filed https://jira.toolserver.org/browse/TS-963 about this. Really > strange. Maybe some safety mechanism to prevent infinite recursion gone bad? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
>> Magnus Manske wrote: > On Mon, Feb 28, 2011 at 2:28 PM, Marcin Cieslak wrote: Magnus Manske wrote: cronsub update_dupes /home/magnus/update_dupes.pl >>> qsub: Unknown option >>> >>> qsub: Unknown option >>> >>> qsub: Unknown option >> >> This is because you have in your script >> >> #$cmd = "./cron_sql.sh" unless -e $cmd ; >> >> and this is recognized by qsub(1) as the embedded qsub option. >> >> If you change it to: >> >> # $cmd = >> >> it works again. > > Thanks! > > Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo > meaning from my scripts (which predate qsub availability) anyway? If > that is desired for some cases, why is that activated by default? Is > there at least a qsub option to turn it off? Will that option work > with cronsub? To quote man qsub(1): -C prefix_string Available for qsub and qrsh with script submission (-b n). Prefix_string defines the prefix that declares a direc- tive in the job's command. The prefix is not a job attribute, but affects the behavior of qsub and qrsh. If prefix is a null string, the command will not be scanned for embedded directives. The directive prefix consists of two ASCII characters which, when appearing in the first two bytes of a script line, indicate that what follows is an Sun Grid Engine command. The default is "#$". The user should be aware that changing the first delim- iting character can produce unforeseen side effects. If the script file contains anything other than a "#" character in the first byte position of the line, the shell processor for the job will reject the line and may exit the job prematurely. If the -C option is present in the script file, it is ignored. However, please be warned that cronsub currently messes up custom qsub options. //Marcin ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
>> Magnus Manske wrote: > On Mon, Feb 28, 2011 at 11:43 AM, Ilmari Karonen wrote: >> On 02/28/2011 01:00 PM, Magnus Manske wrote: >>> >>> OK, so I gave up on starting my perl script using cronsub, which seems >>> to work only if I give the perl script a parameter (WTF?) >> >> Any parameter? I smell a line break encoding issue. >> >> Try making sure there are no carriage returns in your crontab or in any >> of the scripts involved. > > Well, since that happens also when I start cronsub fron the shell, I doubt > it... It seems like qsub dumps core when executed from inside of the SGE job. /sge62/default/spool/wolfsbane/job_scripts/210756[3]: 1253 Segmentation Fault(coredump) I have filed https://jira.toolserver.org/browse/TS-963 about this. Really strange. //Marcin ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 2:28 PM, Marcin Cieslak wrote: >>> Magnus Manske wrote: >> On Mon, Feb 28, 2011 at 11:37 AM, River Tarnell wrote: >>> In article , >>> Magnus Manske wrote: OK, so I gave up on starting my perl script using cronsub, which seems to work only if I give the perl script a parameter (WTF?), and instead altered it to be run manually, generating a shell script that could then start via cronsub. >>> >>> This should not be the case. Please provide the exact script and >>> command line you used and the error message. >> >>> cronsub update_dupes /home/magnus/update_dupes.pl >> qsub: Unknown option >> >> qsub: Unknown option >> >> qsub: Unknown option > > This is because you have in your script > > #$cmd = "./cron_sql.sh" unless -e $cmd ; > > and this is recognized by qsub(1) as the embedded qsub option. > > If you change it to: > > # $cmd = > > it works again. Thanks! Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo meaning from my scripts (which predate qsub availability) anyway? If that is desired for some cases, why is that activated by default? Is there at least a qsub option to turn it off? Will that option work with cronsub? In my "day job", I work with a large compute farm (lfs, ~10K cores), so I'm more than happy to work with, and migrate existing cronjobs to, this system, but it would help if it weren't a deliberate minefield... Cheers, Magnus ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
>> Magnus Manske wrote: > On Mon, Feb 28, 2011 at 11:37 AM, River Tarnell wrote: >> In article , >> Magnus Manske wrote: >>> OK, so I gave up on starting my perl script using cronsub, which seems >>> to work only if I give the perl script a parameter (WTF?), and instead >>> altered it to be run manually, generating a shell script that could >>> then start via cronsub. >> >> This should not be the case. Please provide the exact script and >> command line you used and the error message. > >> cronsub update_dupes /home/magnus/update_dupes.pl > qsub: Unknown option > > qsub: Unknown option > > qsub: Unknown option This is because you have in your script #$cmd = "./cron_sql.sh" unless -e $cmd ; and this is recognized by qsub(1) as the embedded qsub option. If you change it to: # $cmd = it works again. You could have also used -C '##' to change the command prefix to (for example) two hash signs, but giving options to qsub does not work with the current cronsub (https://jira.toolserver.org/browse/TS-958). //Marcin ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 1:59 PM, River Tarnell wrote: > In article , > Magnus Manske wrote: >> > cronsub update_dupes /home/magnus/update_dupes.pl >> qsub: Unknown option >> >> qsub: Unknown option >> >> qsub: Unknown option > > What is the output from this? > > $ ksh -x /opt/local/bin/cronsub update_dupes /home/magnus/update_dupes.pl + . /sge62/default/common/settings.sh + SGE_ROOT=/sge62 + export SGE_ROOT + + /sge62/util/arch ARCH=sol-amd64 + + /sge62/util/arch -m DEFAULTMANPATH=/usr/share/man + + /sge62/util/arch -mt MANTYPE=man + SGE_CELL=default + export SGE_CELL + SGE_CLUSTER_NAME=toolserver + export SGE_CLUSTER_NAME + unset SGE_QMASTER_PORT + unset SGE_EXECD_PORT + [ /sge62/man:/opt/ts/gnu/share/man:/opt/ts/share/man:/opt/SUNWspro/man:/usr/postgres/8.3/man:/opt/ts/mysql/5.1/share/man:/opt/ts/python/2.6/share/man:/opt/ts/perl/5.10/share/man:/opt/ts/ruby/1.8/share/man:/usr/share/man:/usr/sfw/share/man:/usr/cluster/man:/opt/jobserver/man = ] + MANPATH=/sge62/man:/sge62/man:/opt/ts/gnu/share/man:/opt/ts/share/man:/opt/SUNWspro/man:/usr/postgres/8.3/man:/opt/ts/mysql/5.1/share/man:/opt/ts/python/2.6/share/man:/opt/ts/perl/5.10/share/man:/opt/ts/ruby/1.8/share/man:/usr/share/man:/usr/sfw/share/man:/usr/cluster/man:/opt/jobserver/man + export MANPATH + PATH=/sge62/bin/sol-amd64:/sge62/bin/sol-amd64:/opt/local/bin:/opt/ts/gnu/bin:/opt/ts/bin:/opt/ts/mysql/5.1/bin:/opt/ts/perl/5.12/bin:/opt/ts/perl/5/bin:/opt/ts/python/2.7/bin:/opt/ts/php/5.3/bin:/opt/ts/ruby/1.8/bin:/opt/ts/mono/2.0/bin:/opt/ts/tcl/8.5/bin:/usr/ccs/bin:/sge62/bin/sol-amd64:/usr/bin:/usr/sbin:/usr/sfw/bin:/usr/postgres/8.3/bin:/usr/cluster/bin:/opt/jobserver/bin + export PATH + unset ARCH DEFAULTMANPATH MANTYPE + PATH=/usr/bin:/sge62/bin/sol-amd64:/sge62/bin/sol-amd64:/opt/local/bin:/opt/ts/gnu/bin:/opt/ts/bin:/opt/ts/mysql/5.1/bin:/opt/ts/perl/5.12/bin:/opt/ts/perl/5/bin:/opt/ts/python/2.7/bin:/opt/ts/php/5.3/bin:/opt/ts/ruby/1.8/bin:/opt/ts/mono/2.0/bin:/opt/ts/tcl/8.5/bin:/usr/ccs/bin:/sge62/bin/sol-amd64:/usr/bin:/usr/sbin:/usr/sfw/bin:/usr/postgres/8.3/bin:/usr/cluster/bin:/opt/jobserver/bin + sflag= + QUEUE=all.q + OPTS= + hrt= + getopts slh f + shift 0 + [[ 2 -ge 1 ]] + JOBNAME=update_dupes + shift + [ -z ] + [[ 1 -ge 1 ]] + SCRIPT=/home/magnus/update_dupes.pl + + mktemp TMPF=/tmp/tmp.5jaiw2 + chmod 700 /tmp/tmp.5jaiw2 + trap rm -f /tmp/tmp.5jaiw2 0 + cat + 1> /tmp/tmp.5jaiw2 0<< #! /bin/sh cd $HOME exec "$@" exit 1 + grep ^#\$ + 0< /home/magnus/update_dupes.pl 1>> /tmp/tmp.5jaiw2 + grep Full jobname:[ \t]*update_dupes$ + 1> /dev/null + qstat -r + qsub -q all.q -N update_dupes /tmp/tmp.5jaiw2 /home/magnus/update_dupes.pl + 1> /dev/null qsub: Unknown option qsub: Unknown option qsub: Unknown option + rm -f /tmp/tmp.5jaiw2 ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article , Magnus Manske wrote: > > cronsub update_dupes /home/magnus/update_dupes.pl > qsub: Unknown option > > qsub: Unknown option > > qsub: Unknown option What is the output from this? $ ksh -x /opt/local/bin/cronsub update_dupes /home/magnus/update_dupes.pl - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 11:43 AM, Ilmari Karonen wrote: > On 02/28/2011 01:00 PM, Magnus Manske wrote: >> >> OK, so I gave up on starting my perl script using cronsub, which seems >> to work only if I give the perl script a parameter (WTF?) > > Any parameter? I smell a line break encoding issue. > > Try making sure there are no carriage returns in your crontab or in any > of the scripts involved. Well, since that happens also when I start cronsub fron the shell, I doubt it... ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On 02/28/2011 01:00 PM, Magnus Manske wrote: > > OK, so I gave up on starting my perl script using cronsub, which seems > to work only if I give the perl script a parameter (WTF?) Any parameter? I smell a line break encoding issue. Try making sure there are no carriage returns in your crontab or in any of the scripts involved. -- Ilmari Karonen ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Mon, Feb 28, 2011 at 11:37 AM, River Tarnell wrote: > In article , > Magnus Manske wrote: >> OK, so I gave up on starting my perl script using cronsub, which seems >> to work only if I give the perl script a parameter (WTF?), and instead >> altered it to be run manually, generating a shell script that could >> then start via cronsub. > > This should not be the case. Please provide the exact script and > command line you used and the error message. > cronsub update_dupes /home/magnus/update_dupes.pl qsub: Unknown option qsub: Unknown option qsub: Unknown option ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article , Magnus Manske wrote: > OK, so I gave up on starting my perl script using cronsub, which seems > to work only if I give the perl script a parameter (WTF?), and instead > altered it to be run manually, generating a shell script that could > then start via cronsub. This should not be the case. Please provide the exact script and command line you used and the error message. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
On Sun, Feb 27, 2011 at 1:10 PM, River Tarnell wrote: > In article , > Marcin Cieslak wrote: >> >> Magnus Manske wrote: >> > I have a wrapper script that starts a handful of jobs on the >> > toolserver (via qsub). But, when I put that wrapper script as a >> > "normal" cronjob, I get some error messages about qsub not found. >> > Starting the wrapper via cronsub doesn't work either. > > You should first run cronsub from the command-line with the same > arguments. If that still doesn't work, post the entire command line and > any output from it. > > Once you have it working from the command-line then it should be fine > fromc ron. > >> Actually cronsub does a litle bit more than sourcing >> /sge62/default/common/settings.sh file, so I don't know >> why it's not working for you. Maybe you could share some output? > > cronsub makes it easier to submit longrun jobs, automatically sets the > default output file, runs the script from $HOME and prevents duplicate > jobs being submitted. Otherwise it's identical to qsub. OK, so I gave up on starting my perl script using cronsub, which seems to work only if I give the perl script a parameter (WTF?), and instead altered it to be run manually, generating a shell script that could then start via cronsub. Script ( /home/magnus/cronjobs/most_wanted_wrapper.sh ) : #!/bin/ksh qsub -N mw_dewiki_p -j Y -o /home/magnus/cronjobs/update_most_wanted.dewiki_p.out -l sqlprocs-s5=1 /home/magnus/cronjobs/update_most_wanted.pl dewiki_p qsub -N mw_enwiki_p -j Y -o /home/magnus/cronjobs/update_most_wanted.enwiki_p.out -l sqlprocs-s1=1 /home/magnus/cronjobs/update_most_wanted.pl enwiki_p qsub -N mw_frwiki_p -j Y -o /home/magnus/cronjobs/update_most_wanted.frwiki_p.out -l sqlprocs-s6=1 /home/magnus/cronjobs/update_most_wanted.pl frwiki_p qsub -N mw_plwiki_p -j Y -o /home/magnus/cronjobs/update_most_wanted.plwiki_p.out -l sqlprocs-s2=1 /home/magnus/cronjobs/update_most_wanted.pl plwiki_p Running this script, or any of its individual lines, directly from shell, works absolutely fine. However, on > cronsub mostwanted /home/magnus/cronjobs/most_wanted_wrapper.sh I get (in the error file): /home/magnus/cronjobs/most_wanted_wrapper.sh[2]: 7274 Segmentation Fault(coredump) /home/magnus/cronjobs/most_wanted_wrapper.sh[3]: 7289 Segmentation Fault(coredump) /home/magnus/cronjobs/most_wanted_wrapper.sh[4]: 7290 Segmentation Fault(coredump) /home/magnus/cronjobs/most_wanted_wrapper.sh[5]: 7298 Segmentation Fault(coredump) I tried replacing qsub with cronsub, only to be met with a flood of errors relating to all the options it doesn't know, including the sql slot reservation (I guess that's my punishment for trying to be a good citizen...). Removing basically all options, using only lines like this: cronsub -s mw_dewiki_p /home/magnus/cronjobs/update_most_wanted.pl dewiki_p in the wrapper script, again runs peachy from shell, but barfs coredumps again when run via cronsub. So, as far as I am concerned, it's broken. Cheers, Magnus ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
>> River Tarnell wrote: > Otherwise it's identical to qsub. I just found out that it does not accept arguments to qsub (like "-m e"), see https://jira.toolserver.org/browse/TS-958 //Marcin ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article , Marcin Cieslak wrote: > >> Magnus Manske wrote: > > I have a wrapper script that starts a handful of jobs on the > > toolserver (via qsub). But, when I put that wrapper script as a > > "normal" cronjob, I get some error messages about qsub not found. > > Starting the wrapper via cronsub doesn't work either. You should first run cronsub from the command-line with the same arguments. If that still doesn't work, post the entire command line and any output from it. Once you have it working from the command-line then it should be fine fromc ron. > Actually cronsub does a litle bit more than sourcing > /sge62/default/common/settings.sh file, so I don't know > why it's not working for you. Maybe you could share some output? cronsub makes it easier to submit longrun jobs, automatically sets the default output file, runs the script from $HOME and prevents duplicate jobs being submitted. Otherwise it's identical to qsub. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
>> Magnus Manske wrote: > Hi, > > I have a wrapper script that starts a handful of jobs on the > toolserver (via qsub). But, when I put that wrapper script as a > "normal" cronjob, I get some error messages about qsub not found. > Starting the wrapper via cronsub doesn't work either. cronsub should work. cron has a distinct environment as compared to the user shell (because it gets started at system boot and has to make up its environment). Here's what I tried: my crontab: 00 12 * * * $HOME/wrapper $HOME/wrapper script: #! /bin/ksh . /sge62/default/common/settings.sh qsub sgeenv $HOME/sgeenv script: #! /bin/sh #$ -m be env Seems to work fine. Actually cronsub does a litle bit more than sourcing /sge62/default/common/settings.sh file, so I don't know why it's not working for you. Maybe you could share some output? //Marcin ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Cronjob starting cronsub
Hi, I have a wrapper script that starts a handful of jobs on the toolserver (via qsub). But, when I put that wrapper script as a "normal" cronjob, I get some error messages about qsub not found. Starting the wrapper via cronsub doesn't work either. Anyone: What do I have to put in my .environment file (or whatever) so that my "normal" cronjob can start qsub/cronsub jobs? Thanks, Magnus ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
