I think I've finally resolved the issue, though I still don't fully understand why. This was my original test makefile:

x=$(shell date -d $(*F) +"%H%M%S")
%.slp:
    echo $(x) ; sleep 120 ; touch $@

The "fix" was to replace the double quotes "" in x=$(shell...) with single quotes. The new makefile is:


x=$(shell date -d $(*F) +'%H%M%S')
%.slp:
    echo $(x) ; sleep 120 ; touch $@

This "fixed" makefile runs the same with make and qmake. I'm sure it must have something to do with the difference between single and double quotes in bash, but I don't know what...

Thanks for the help Reuti!

Andrew Joplin


On 02/28/2014 05:58 PM, Reuti wrote:
Am 01.03.2014 um 00:17 schrieb Andrew Joplin:

Correct.  In qstat I see the qmake job itself on one node, and a single sh job that gets 
scheduled to a different node.  In the output of "ps -e f" on the 1st node 
(where qmake is running) I see a single qrsh process, and the corresponding sh process on 
the other node.

Can you explain what you mean by "the rule isn't parallized"?  I thought if it works in 
parallel with make, it's "parallelized"...
`make` 3.82 also works correctly when being called on the command line (BTW 
which version do you have there?).

Maybe the hooks put into `make` were not the same as for the older version and 
something got screwed up; and/or the remote-calls are not made correctly by 
`make`.

What is the output when you add -d:

  $ qmake -V -cwd -- -d -f Makefile -j 2 today.slp yesterday.slp

-- Reuti


Andrew Joplin
Space and Geophysics Lab
Applied Research Laboratories - The University of Texas at Austin
(512) 835-3050


On 02/28/2014 04:58 PM, Reuti wrote:
Am 28.02.2014 um 23:56 schrieb Andrew Joplin:

SoGE is using gmake 3.82 as well.  If this is an actual bug, do you have any 
suggestions regarding a workaround?  I've tried to eliminate my need to call 
shell functions in the makefile, but haven't figure out how to do away with 
them.
I don't think that the problem is the used shell function, but that the rule 
isn't parallelized. In:

$ ps -e f

on the node in question, resp.:

$ qstat

you see only one job running / being submitted?

-- Reuti


Thanks!

Andrew Joplin


On 02/28/2014 04:34 PM, Reuti wrote:
Am 28.02.2014 um 23:20 schrieb Andrew Joplin:

$ qmake --version
GNU Make 3.82
Built for x86_64-unknown-linux-gnu (distributed make
load balancing by Grid Engine
)
Copyright (C) 2010  Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

The version at the end of the qmake man page is:

OGS/Grid Engine 2011.11  $Date: 2007/07/19 09:04:30 $                QMAKE(1)

Is this an old bug that's been fixed?
No, a new one which was introduced I fear. I get (forgot to mention):

$ qmake --version
GNU Make version 3.78.1-distributed make
load balancing by Grid Engine


I'm not sure which version the SoGE fork is using, but it's worth to try:

https://arc.liv.ac.uk/trac/SGE

-- Reuti


Andrew Joplin


On 02/28/2014 03:55 PM, Reuti wrote:
Am 28.02.2014 um 20:37 schrieb Andrew Joplin:

I've come up with a simple example to illustrate this problem.  I think it's 
not so much an implicit rule problem as it is using a shell function within the 
makefile:

Makefile:

x=$(shell date -d $(*F) +"%H%M%S")
%.slp:
     echo $(x) ; sleep 30 ; touch $@

Running with make starts both targets simultaneously as it should:

$ make -f Makefile -j 2 today.slp yesterday.slpecho 133400 ; sleep 30 ; touch 
today.slp
133400
echo 133400 ; sleep 30 ; touch yesterday.slp
133400

But running with qmake only spawns a single qsh job:

$ qmake -V -cwd -- -f Makefile -j 2 today.slp yesterday.slp
qmake: *** cannot determine architecture from environment variable SGE_ARCH
            no default architecture set
echo 133523 ; sleep 30 ; touch today.slp
dynamic mode
For me it's working as intended (but I have still SGE 6.2u5).

What is the output of.

$ qmake --version

-- Reuti


Does the use of the shell function somehow prevent qmake from running in 
parallel?
Andrew Joplin

On 02/27/2014 05:35 PM, Andrew Joplin wrote:
Newbie again with a qmake question.  I have several analysis jobs that I run 
with qmake (each rule processes data and generates some output file).  Some of 
my makefiles have all explicit rules, and they run fine in dynamic allocation 
mode.  But a couple newer makefiles use a couple implicit rules, and it does 
run on our cluster, but only one target at a time.

The syntax I'm using is:

$ qmake -V -cwd -- -f Makefile -j 20 --keep-going

I've already tried running with make instead, and it uses as many threads as I 
tell it.  Any ideas?  I'll try to work up a simplified test makefile.



_______________________________________________
users mailing list

[email protected]
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users

Reply via email to