Hello,
I tried to use distcc to perform a distributed build of a parallelized
c++ code (via mpiCC). However, every object file generated with the
distributed compilation is of length 0 (no errors are generated by
distcc). The code compiles fine with distcc and g++ . In addition, it
also compiles fine with mpiCC directly invoked as a "serial" compiler.
The problem seem to be interactions between mpiCC and distcc.
Did someone actually manage to use mpiCC with distcc ? Here are some
relevant data :
1- using distcc-2.18.3, compiled from scratch
2- CENTOS (same as RHEL4) - kernel 2.6.9-5.0.5.ELsmp - x86_64
architecture. gcc version is 3.4.3 20041212 (Red Hat 3.4.3-9.EL4), and
mpiCC coming from mpichgm-1.2.6..14a
3- as stated, i'm trying to build a home made library which otherwise
works well (distcc with g++ works fine for that library)
4-with distcc and mpiCC, compiling a c++ source file leads to a zero
length .o file, with no error report. I think the problem lies either in
the transmission of the resulting object code (is mpiCC building
"special" object files ?) or in mpiCC failing to compile without warning.
5- command line for one of the source files :
/usr/local/distcc/bin/distcc /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC
-g -m64 -Wall -DDEBUG -DPARALLEL -D_HAVE_PARMETIS_
-DMPICH_SKIP_MPICXX -D_MPIPP_H -D_HAVE_PARMETIS_
-I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/mpich-gm-gnu/include
-I/usr/local/mpich-gm-gnu/include -I/usr/local/autopack-gm-gnu
-I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/parmetis-gm-gnu
-I/home/admin/develop/Solver/itl -I/home/admin/develop/Solver/mtl
-I/home/admin/develop/Trellis/AOMD/include
-I/home/admin/develop/Trellis/model/include
-I/home/admin/develop/Trellis/Util/include
-I/home/admin/develop/Util/Loki/include
-I/home/admin/develop/Xext/XextPara/include
-I/home/admin/develop/Xfem/Xfem/Xfem/../include -c
Xfem/IntegratorPartitionBoundary.cc -o
obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o
I attached a log file
Thanks for your help,
Eric
--
Eric Bechet
Research Associate
Institut GeM, Ecole Centrale de Nantes
1, rue de la Noe, 44321 Nantes Cedex 3, France
(tel/fax) +33 2 40 37 (25 82 / 25 73)
distcc[12800] (dcc_trace_version) distcc 2.18.3 x86_64-unknown-linux-gnu; built Jun 6 2005 10:13:22
distcc[12800] (dcc_recursion_safeguard) safeguard level=0
distcc[12800] (main) compiler name is "distcc"
distcc[12800] (dcc_scan_args) scanning arguments: /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC -m64 -g -m64 -Wall -DDEBUG -DPARALLEL -D_HAVE_PARMETIS_ -DMPICH_SKIP_MPICXX -D_MPIPP_H -D_HAVE_PARMETIS_ -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/mpich-gm-gnu/include -I/usr/local/mpich-gm-gnu/include -I/usr/local/autopack-gm-gnu -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/parmetis-gm-gnu -I/home/admin/develop/Solver/itl -I/home/admin/develop/Solver/mtl -I/home/admin/develop/Trellis/AOMD/include -I/home/admin/develop/Trellis/model/include -I/home/admin/develop/Trellis/Util/include -I/home/admin/develop/Util/Loki/include -I/home/admin/develop/Xext/XextPara/include -I/home/admin/develop/Xfem/Xfem/Xfem/../include -c Xfem/IntegratorPartitionBoundary.cc -o obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o
distcc[12800] (dcc_scan_args) found input file "Xfem/IntegratorPartitionBoundary.cc"
distcc[12800] (dcc_scan_args) found object/output file "obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o"
distcc[12800] compile from IntegratorPartitionBoundary.cc to IntegratorPartitionBoundary.o
distcc[12800] (dcc_get_hostlist) read hosts from environment
distcc[12800] (dcc_parse_hosts) found tcp token "node0"
distcc[12800] (dcc_parse_hosts) found tcp token "node1"
distcc[12800] (dcc_parse_hosts) found tcp token "node2"
distcc[12800] (dcc_parse_hosts) found tcp token "node3"
distcc[12800] (dcc_parse_hosts) found tcp token "node4"
distcc[12800] (dcc_parse_hosts) found tcp token "node5"
distcc[12800] (dcc_parse_hosts) found tcp token "node6"
distcc[12800] (dcc_parse_hosts) found tcp token "node7"
distcc[12800] (dcc_parse_hosts) found tcp token "node8"
distcc[12800] (dcc_parse_hosts) found tcp token "node9"
distcc[12800] (dcc_parse_hosts) found tcp token "node10"
distcc[12800] (dcc_parse_hosts) found tcp token "node11"
distcc[12800] (dcc_parse_hosts) found tcp token "node12"
distcc[12800] (dcc_parse_hosts) found tcp token "node13"
distcc[12800] (dcc_parse_hosts) found tcp token "node14"
distcc[12800] (dcc_parse_hosts) found tcp token "node15"
distcc[12800] (dcc_parse_hosts) found tcp token "node16"
distcc[12800] (dcc_parse_hosts) found tcp token "node17"
distcc[12800] (dcc_parse_hosts) found tcp token "node18"
distcc[12800] (dcc_parse_hosts) found tcp token "node19"
distcc[12800] (dcc_parse_hosts) found tcp token "node20"
distcc[12800] (dcc_parse_hosts) found tcp token "node21"
distcc[12800] (dcc_parse_hosts) found tcp token "node22"
distcc[12800] (dcc_parse_hosts) found tcp token "node23"
distcc[12800] (dcc_parse_hosts) found tcp token "node24"
distcc[12800] (dcc_parse_hosts) found tcp token "node25"
distcc[12800] (dcc_parse_hosts) found tcp token "node26"
distcc[12800] (dcc_parse_hosts) found tcp token "node27"
distcc[12800] (dcc_parse_hosts) found tcp token "node28"
distcc[12800] (dcc_parse_hosts) found tcp token "node29"
distcc[12800] (dcc_parse_hosts) found tcp token "node30"
distcc[12800] (dcc_parse_hosts) found tcp token "node31"
distcc[12800] (dcc_lock_host) got cpu lock on node0 slot 0 as fd4
distcc[12800] (dcc_strip_dasho) result: /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC -m64 -g -m64 -Wall -DDEBUG -DPARALLEL -D_HAVE_PARMETIS_ -DMPICH_SKIP_MPICXX -D_MPIPP_H -D_HAVE_PARMETIS_ -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/mpich-gm-gnu/include -I/usr/local/mpich-gm-gnu/include -I/usr/local/autopack-gm-gnu -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/parmetis-gm-gnu -I/home/admin/develop/Solver/itl -I/home/admin/develop/Solver/mtl -I/home/admin/develop/Trellis/AOMD/include -I/home/admin/develop/Trellis/model/include -I/home/admin/develop/Trellis/Util/include -I/home/admin/develop/Util/Loki/include -I/home/admin/develop/Xext/XextPara/include -I/home/admin/develop/Xfem/Xfem/Xfem/../include -c Xfem/IntegratorPartitionBoundary.cc
distcc[12800] (dcc_spawn_child) forking to execute: /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC -m64 -g -m64 -Wall -DDEBUG -DPARALLEL -D_HAVE_PARMETIS_ -DMPICH_SKIP_MPICXX -D_MPIPP_H -D_HAVE_PARMETIS_ -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/mpich-gm-gnu/include -I/usr/local/mpich-gm-gnu/include -I/usr/local/autopack-gm-gnu -I/usr/local/parmetis-gm-gnu/METISLib -I/usr/local/parmetis-gm-gnu -I/home/admin/develop/Solver/itl -I/home/admin/develop/Solver/mtl -I/home/admin/develop/Trellis/AOMD/include -I/home/admin/develop/Trellis/model/include -I/home/admin/develop/Trellis/Util/include -I/home/admin/develop/Util/Loki/include -I/home/admin/develop/Xext/XextPara/include -I/home/admin/develop/Xfem/Xfem/Xfem/../include -E Xfem/IntegratorPartitionBoundary.cc
distcc[12801] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12800] (dcc_spawn_child) child started as pid12801
distcc[12800] (dcc_strip_local_args) result: /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC -m64 -g -m64 -Wall -c Xfem/IntegratorPartitionBoundary.cc -o obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o
distcc[12800] exec on node0: /usr/local/mpich-gm-gnu/bin/mpiCC -w -fPIC -m64 -g -m64 -Wall -c Xfem/IntegratorPartitionBoundary.cc -o obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o
distcc[12800] (dcc_note_state) note state 2, file "IntegratorPartitionBoundary.cc", host "node0"
distcc[12800] (dcc_connect_by_addr) started connecting to 192.168.1.100:3632
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc[12800] (dcc_x_token_int) send DIST00000001
distcc[12800] (dcc_x_token_int) send ARGC0000000b
distcc[12800] (dcc_x_token_int) send ARGV00000021
distcc[12800] (dcc_x_token_int) send ARGV00000002
distcc[12800] (dcc_x_token_int) send ARGV00000005
distcc[12800] (dcc_x_token_int) send ARGV00000004
distcc[12800] (dcc_x_token_int) send ARGV00000002
distcc[12800] (dcc_x_token_int) send ARGV00000004
distcc[12800] (dcc_x_token_int) send ARGV00000005
distcc[12800] (dcc_x_token_int) send ARGV00000002
distcc[12800] (dcc_x_token_int) send ARGV00000023
distcc[12800] (dcc_x_token_int) send ARGV00000002
distcc[12800] (dcc_x_token_int) send ARGV00000034
distcc[12800] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc[12800] (dcc_collect_child) cpp child 12801 terminated with status 0
distcc[12800] (dcc_collect_child) cpp times: user 0.106983s, system 0.058991s, 2341 minflt, 0 majflt
distcc[12800] cpp Xfem/IntegratorPartitionBoundary.cc on localhost completed ok
distcc[12800] (dcc_x_file) send 1345887 byte file /tmp/distcc_22191412.ii with token DOTI
distcc[12800] (dcc_x_token_int) send DOTI0014895f
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 65221 bytes; retrying 1280666 @65221
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 147456 bytes; retrying 1133210 @212677
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 49152 bytes; retrying 1084058 @261829
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 81920 bytes; retrying 1002138 @343749
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 98304 bytes; retrying 903834 @442053
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 98304 bytes; retrying 805530 @540357
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 114688 bytes; retrying 690842 @655045
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 131072 bytes; retrying 559770 @786117
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 131072 bytes; retrying 428698 @917189
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 131072 bytes; retrying 297626 @1048261
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 131072 bytes; retrying 166554 @1179333
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_pump_sendfile) Notice: sendfile: partial transmission of 131072 bytes; retrying 35482 @1310405
distcc[12800] (dcc_select_for_write) select for write on fd5
distcc[12800] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12800] (dcc_compile_remote) client finished sending request to server
distcc[12800] (dcc_note_state) note state 5, file "(NULL)", host "node0"
distcc[12800] (dcc_r_token_int) got DONE00000001
distcc[12800] (dcc_r_result_header) got response header
distcc[12800] (dcc_note_state) note state 6, file "(NULL)", host "(NULL)"
distcc[12800] (dcc_r_token_int) got STAT00000000
distcc[12800] (dcc_r_token_int) got SERR00000000
distcc[12800] (dcc_r_token_int) got SOUT00000000
distcc[12800] (dcc_r_token_int) got DOTO00000000
distcc[12800] (dcc_r_file) received 0 bytes to file obj/x86_64_linux-mpich/IntegratorPartitionBoundary.o
distcc[12800] (dcc_r_file_timed) 0 bytes received in 0.000918s, rate 0kB/s
distcc[12800] 1345887 bytes from Xfem/IntegratorPartitionBoundary.cc compiled on node0 in 0.2974s, rate 4419kB/s
distcc[12800] (dcc_unlock) release lock fd4
distcc[12800] compile Xfem/IntegratorPartitionBoundary.cc on node0 completed ok
distcc[12800] elapsed compilation time 0.301206s
distcc[12800] (dcc_exit) exit: code 0; self: 0.000000 user 0.002999 sys; children: 0.106983 user 0.058991 sys
distcc[12800] (dcc_cleanup_tempfiles) deleted 1 temporary files
__
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
