On Wed, Apr 25, 2018 at 12:40 PM, Danyang Su
<danyang...@gmail.com <mailto:danyang...@gmail.com>> wrote:
Hi Matthew,
In the worst case, every node/cell may have different label.
Do not use Label for this. Its not an appropriate thing. If
every cell is different, just use the cell number.
Labels are for mapping a relatively small number of keys (like
material IDs) to sets of points (cells, vertices, etc.)
Its not a great data structure for a permutation.
However, I still do not believe these numbers. The old code does
a string comparison every time. I will setup a test.
Matt
Below is one of the worst scenario with 102299 nodes and
102299 different labels for test. I found the time cost
increase during the loop. The first 9300 loop takes least
time (<0.5) while the last 9300 loops takes much more time
(>7.7), as shown below. If I use larger mesh with >1 million
nodes, it runs very very slowly in this part. The PETSc is
configured with optimization on.
Configure options --with-cc=gcc --with-cxx=g++
--with-fc=gfortran --download-mpich --download-scalapack
--download-parmetis --download-metis --download-ptscotch
--download-fblaslapack --download-hypre
--download-superlu_dist --download-hdf5=yes
--download-ctetgen --with-debugging=0 COPTFLAGS="-O3
-march=native -mtune=native" CXXOPTFLAGS="-O3 -march=native
-mtune=native" FOPTFLAGS="-O3 -march=native -mtune=native"
istart iend progress CPU_Time time cost - old (sec)
time cost - new (sec)
0 9299 0 1524670045.51166
9300 18599 0.100010753 1524670045.99605 0.4843890667
0.497246027
18600 27899 0.200010747 1524670047.32635 1.330302
1.3820912838
27900 37199 0.300010741 1524670049.3066 1.9802515507
2.2439446449
37200 46499 0.400010765 1524670052.1594 2.852804184
3.0739262104
46500 55799 0.500010729 1524670055.90961 3.7502081394
3.9270553589
55800 65099 0.600010753 1524670060.47654 4.5669286251
4.7571902275
65100 74399 0.700010777 1524670066.0941 5.6175630093
5.7428796291
74400 83699 0.800010741 1524670072.53886 6.44475317
6.5761549473
83700 92998 0.900010765 1524670079.99072 7.4518604279
7.4606924057
92999 102298 1 1524670087.71066 7.7199423313
8.2424075603
old code
do ipoint = 0, istart-1
!c output time cost, use 1 processor to test
if (b_enable_output .and. rank == 0) then
if (mod(ipoint,iprogress) == 0 .or. ipoint ==
istart-1) then
!write(*,'(f3.1,1x)',advance="no") (ipoint+1.0)/istart
write(*,*) ipoint,
(ipoint+1.0)/istart,"time",MPI_Wtime()
end if
end if
call DMSetLabelValue(dmda_flow%da,"cid_lg2g",ipoint, &
ipoint+1,ierr)
CHKERRQ(ierr)
end do
new code
call DMCreateLabel(dmda_flow%da,'cid_lg2g',ierr)
CHKERRQ(ierr)
call DMGetLabel(dmda_flow%da,'cid_lg2g',label, ierr)
CHKERRQ(ierr)
do ipoint = 0, istart-1
!c output time cost, use 1 processor to test
if (b_enable_output .and. rank == 0) then
if (mod(ipoint,iprogress) == 0 .or. ipoint ==
istart-1) then
!write(*,'(f3.1,1x)',advance="no") (ipoint+1.0)/istart
write(*,*) ipoint,
(ipoint+1.0)/istart,"time",MPI_Wtime()
end if
end if
call DMLabelSetValue(label,ipoint,ipoint+1,ierr)
CHKERRQ(ierr)
end do
Thanks,
Danyang
On 2018-04-25 03:16 AM, Matthew Knepley wrote:
On Tue, Apr 24, 2018 at 11:57 PM, Danyang Su
<danyang...@gmail.com <mailto:danyang...@gmail.com>> wrote:
Hi All,
I use DMPlex in unstructured grid code and recently
found DMSetLabelValue takes a lot of time for large
problem, e.g., num. of cells > 1 million. In my code, I use
I read your code wrong. For large loop, you should not use
the convenience function. You should use
DMPlexCreateFromCellList ()
DMGetLabel(dm, name, &label)
Loop over all cells/nodes{
DMSetLabelValue
Replace this by DMLabelSetValue(label, point, val)
}
DMPlexDistribute
The code works fine except DMSetLabelValue takes a lot
of time for large problem. I use DMSetLabelValue to set
material id for all the nodes or cells so that each
subdomain has a copy of material id. Is there any other
functions that can be used more efficient, e.g. set
labels by array, not 1 by 1?
That should take much less time.
Thanks,
Matt
Thanks,
Danyang
--
What most experimenters take for granted before they begin
their experiments is infinitely more interesting than any
results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.caam.rice.edu/%7Emk51/>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to
which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/
<http://www.caam.rice.edu/%7Emk51/>