Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-09-07 Thread Bruno Turcksin 
Dustin,

2016-09-07 11:07 GMT-04:00 Dustin Kumor :
> I'm using the function "add_entries" only in combination with the
> DynamicSparsityPattern and not with the SparsityPattern class. Moreover, it
> seems that the DynamicSparsityPattern class doesn't even have a "copy_form"
> function. So I do not really get the point.
Yes, sorry. I didn't realize that you were copying a
DynamicSparsityPattern and not a SparsityPattern.

Best,

Bruno

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-09-07 Thread Dustin Kumor 
Dear Martin, dear Bruno,

I'm sorry for giving a feedback to your answers that late, but I have been 
really busy with another project and had no time to deal with this problem.

@Martin: After applying the patch I get the same run time results as you 
did. So thank you for figuring out and fixing the problem.

@ Bruno: Of course you are right. The correct name of the functions is 
indeed "make_biorthogonal_sparsity_pattern" and not 
"make_offdiagonal_sparsity_pattern". Sorry for confusing you, but I talked 
to so many people about this problem and it always arised when filling the 
offdiagonal blocks of my block matrix that I just mixed up the terms. 
Concerning your first hint, I'm not quite sure if I understand it 
correctly. I'm using the function "add_entries" only in combination with 
the DynamicSparsityPattern and not with the SparsityPattern class. 
Moreover, it seems that the DynamicSparsityPattern class doesn't even have 
a "copy_form" function. So I do not really get the point.

Best,
Dustin


-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-31 Thread Martin Kronbichler 

Hi Dustin,

Thanks for your file. This is indeed a bug in deal.II. Our algorithms 
for copying the sparsity pattern contain a part that scales 
quadratically in the number of rows in case most of the rows are empty, 
as is the case with some of your sparsity patterns. This is very bad if 
there are many rows. When using a patch, the run time in release mode of 
copy_sparsity_pattern goes from over 20s on my machine to 0.328s. A 
patch addressing this issue has been published on the deal.II site:


https://github.com/dealii/dealii/pull/3040

Best,
Martin


On 31.08.2016 13:20, Dustin Kumor wrote:

Dear all,

some additional information. Out of interest I switched back to the 
last release version of the deal.ii library 8.4.1 and made a run with 
this version. The problem is still there and even worse. I attached 
again the two log files for debug and release mode respectively.



Best
Dustin
--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en

---
You received this message because you are subscribed to the Google 
Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to dealii+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-31 Thread Bruno Turcksin 
Dustin,

I don't think that the add_entries was made to copy entire SparsityPattern. 
I think that you should use the function copy_from 
(http://dealii.org/8.4.1/doxygen/deal.II/classSparsityPattern.html#a96248eff3fbfa4270dfe21b0a4ea077b).
 
This function was made for what you are trying to do.

BTW the function is called make_biorthogonal_sparsity_pattern in the test 
not make_offdiagonal_sparsity_pattern.

Best,

Bruno

On Tuesday, August 30, 2016 at 9:42:26 AM UTC-4, Dustin Kumor wrote:
>
> Dear Martin,
>
> unfortunately I have to revise all my statements. Updating the deal.II 
> library to the most recent developement version did not solve the problem. 
> I made a stupid mistake while creating the file for the test case. The 
> function make_offdiagonal_sparsity_pattern () has been commented out in 
> the test case file. And with this line commented out I need the same time 
> to copy as you did. If I take into account this function the time to copy 
> the dynamicsparsitypattern becomes as big as before. I attached the 
> corrected testcase file to the post again and also the two log files one 
> for each run. Can you run the program on your machine with the modified 
> file again and figure out whether you get similiar results or not? If you 
> observe the same behavior we can be sure that the problem is related to my 
> function make_offdiagonal_sparsity_pattern () and the way I create the 
> sparsity pattern.
>
> Best,
> Dustin 
>
>>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-31 Thread Dustin Kumor 
Dear all,

some additional information. Out of interest I switched back to the last 
release version of the deal.ii library 8.4.1 and made a run with this 
version. The problem is still there and even worse. I attached again the 
two log files for debug and release mode respectively.


Best
Dustin

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
JobId thinkpad-lsx.site Wed Aug 31 11:14:21 2016
setup_intial_fe_distribution, wall time: 0.596235s.
DEAL::
DEAL::Memory used by block (0, 0) = 50144
DEAL::Memory used by block (0, 1) = 5904
DEAL::Memory used by block (0, 2) = 180504
DEAL::Memory used by block (1, 0) = 20909304
DEAL::Memory used by block (1, 1) = 344734528
DEAL::Memory used by block (1, 2) = 21246732
DEAL::Memory used by block (2, 0) = 261504
DEAL::Memory used by block (2, 1) = 424332
DEAL::Memory used by block (2, 2) = 86904
DEAL::
DEAL::Nonzero entries: 53353179
DEAL::
copy_sparsity_pattern, wall time: 3772.89s.
DEAL::   n_dofs dofhandler_m:   240
 n_dofs dofhandler_s:   871215
 n_dofs biorthogonal basis: 3615
 n_dofs all:875070

setup_system, wall time: 3801.87s.
Setup, wall time: 3802.46s.


+-+++
| Total wallclock time elapsed since start|  3.81e+03s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Setup   | 1 |  3.80e+03s |1.e+02% |
| copy_sparsity_pattern   | 1 |  3.77e+03s |   99.% |
| setup_intial_fe_distribution| 1 | 0.596s | 0.016% |
| setup_system| 1 |  3.80e+03s |1.e+02% |
+-+---+++

JobId thinkpad-lsx.site Wed Aug 31 12:27:00 2016
setup_intial_fe_distribution, wall time: 0.19369s.
DEAL::
DEAL::Memory used by block (0, 0) = 50144
DEAL::Memory used by block (0, 1) = 5904
DEAL::Memory used by block (0, 2) = 180504
DEAL::Memory used by block (1, 0) = 20909304
DEAL::Memory used by block (1, 1) = 344734528
DEAL::Memory used by block (1, 2) = 21246732
DEAL::Memory used by block (2, 0) = 261504
DEAL::Memory used by block (2, 1) = 424332
DEAL::Memory used by block (2, 2) = 86904
DEAL::
DEAL::Nonzero entries: 53353179
DEAL::
copy_sparsity_pattern, wall time: 1176.15s.
DEAL::   n_dofs dofhandler_m:   240
 n_dofs dofhandler_s:   871215
 n_dofs biorthogonal basis: 3615
 n_dofs all:875070

setup_system, wall time: 1179.94s.
Setup, wall time: 1180.14s.


+-+++
| Total wallclock time elapsed since start|  1.18e+03s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Setup   | 1 |  1.18e+03s |1.e+02% |
| copy_sparsity_pattern   | 1 |  1.18e+03s |1.e+02% |
| setup_intial_fe_distribution| 1 | 0.194s | 0.016% |
| setup_system| 1 |  1.18e+03s |1.e+02% |
+-+---+++

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-30 Thread Dustin Kumor 
Dear Martin,

unfortunately I have to revise all my statements. Updating the deal.II 
library to the most recent developement version did not solve the problem. 
I made a stupid mistake while creating the file for the test case. The 
function make_offdiagonal_sparsity_pattern () has been commented out in the 
test case file. And with this line commented out I need the same time to 
copy as you did. If I take into account this function the time to copy the 
dynamicsparsitypattern becomes as big as before. I attached the corrected 
testcase file to the post again and also the two log files one for each 
run. Can you run the program on your machine with the modified file again 
and figure out whether you get similiar results or not? If you observe the 
same behavior we can be sure that the problem is related to my function 
make_offdiagonal_sparsity_pattern 
() and the way I create the sparsity pattern.

Best,
Dustin 

>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#include 


struct FacePointData
{
unsigned int
  master_face_no,
  slave_face_no,
  master_q_point_index;
dealii::hp::DoFHandler<3>::cell_iterator
  master_cell,
  slave_cell;
std::vector
  slave_face_dofs,
  master_face_dofs,
  slave_local_face_dofs,
  master_local_face_dofs,
  biorthogonal_face_dofs;
};

class InterfaceData
{
  public:
InterfaceData ();

std::map&
biorthogonal_master_dof_connections ();

std::map&
biorthogonal_slave_dof_connections ();

void
initialise (unsigned int const n_interface_q_points);

std::map&
master_biorthogonal_dof_connections ();

std::vector>&
master_q_points ();

std::map&
master_slave_dof_connections ();

unsigned int
n_cells_master ();

unsigned int
n_cells_slave ();

unsigned int
ndofs_biorthogonal ();

std::vector&
q_point_data ();

void
setup_data (dealii::hp::DoFHandler<3> const& dofhandler_m,
dealii::hp::DoFHandler<3> const& dofhandler_s,
dealii::hp::QCollection<2> const& face_q_collection);

std::map&
slave_biorthogonal_dof_connections ();

std::map&
slave_master_dof_connections ();

  private:
void
add_interface_q_point_data (unsigned int const& index,
unsigned int const& slave_face_no,
dealii::hp::DoFHandler<3>::cell_iterator const& master_cell,
dealii::hp::DoFHandler<3>::cell_iterator const& slave_cell);

void
detect_contact_interface(dealii::hp::DoFHandler<3> const& dofhandler_m,
dealii::hp::DoFHandler<3> const& dofhandler_s,
dealii::hp::QCollection<2> const& face_q_collection,
std::vector::active_cell_iterator>& master_cells,
std::vector::active_cell_iterator>& slave_cells,
std::vector& slave_face_no,
unsigned int& master_counter,
unsigned int& slave_counter,
unsigned int& n_interface_q_points);

void
setup_interface_q_point_data(std::vector::active_cell_iterator> const& master_cells,
 std::vector::active_cell_iterator> const& slave_cells,
 std::vector const& slave_face_no,
 unsigned int const n_master_cells,
 unsigned int const n_slave_cells,
 dealii::hp::FECollection<3> const& fe_collection,
 dealii::hp::QCollection<2> const& face_q_collection);

void
setup_interface_dof_connections (unsigned int const ndofs_m,
 unsigned int const ndofs_s);

unsigned int
  ndofs_bi,
  ncells_m,
  ncells_s;
std::vector>
  m_q_points;
std::vector
  qpoint_data;
std::map
  m_bi_dof_conn,
  s_bi_dof_conn,
  bi_m_dof_conn,
  bi_s_dof_conn,
  m_s_dof_conn,
  s_m_dof_conn;
};

using namespace dealii;

InterfaceData::InterfaceData ()
  :
  ndofs_bi(0),
  ncells_m (0),
  ncells_s (0)
{
}

std::map&
InterfaceData::biorthogonal_master_dof_connections ()
{
  return bi_m_dof_conn;
}

std::map&
InterfaceData::biorthogonal_slave_dof_connections ()
{

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-16 Thread Dustin Kumor 
Dear Martin,

the update to the most recent development version solved the problem. Now 
time needed to finish the copy operation is in the expected range. Thanks a 
lot for your support.

Best,
Dustin

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
JobId thinkpad-lsx.site Tue Aug 16 12:52:39 2016
setup_intial_fe_distribution, wall time: 0.0151739s.
DEAL::Nonzero entries: 6275947
copy_sparsity_pattern, wall time: 0.033499s.
DEAL::   n_dofs dofhandler_m:   240
 n_dofs dofhandler_s:   131769
 n_dofs biorthogonal basis: 1089
 n_dofs all:133098

setup_system, wall time: 0.542771s.
Setup, wall time: 0.558091s.


+-+++
| Total wallclock time elapsed since start| 0.921s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Setup   | 1 | 0.558s |   61.% |
| copy_sparsity_pattern   | 1 |0.0335s |   3.6% |
| setup_intial_fe_distribution| 1 |0.0152s |   1.6% |
| setup_system| 1 | 0.543s |   59.% |
+-+---+++

JobId thinkpad-lsx.site Tue Aug 16 12:48:38 2016
setup_intial_fe_distribution, wall time: 0.0675499s.
DEAL::Nonzero entries: 6275947
copy_sparsity_pattern, wall time: 0.052773s.
DEAL::   n_dofs dofhandler_m:   240
 n_dofs dofhandler_s:   131769
 n_dofs biorthogonal basis: 1089
 n_dofs all:133098

setup_system, wall time: 3.62941s.
Setup, wall time: 3.69712s.


+-+++
| Total wallclock time elapsed since start|  5.29s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Setup   | 1 |  3.70s |   70.% |
| copy_sparsity_pattern   | 1 |0.0528s |   1.0% |
| setup_intial_fe_distribution| 1 |0.0675s |   1.3% |
| setup_system| 1 |  3.63s |   69.% |
+-+---+++

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-16 Thread Dustin Kumor 
Dear Martin,

I'm using 8.5.0-pre. But I haven't updated the code for a long time. I will 
do this first, then compile the library again and let you know about the 
results.

Best,
Dustin

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-15 Thread Martin Kronbichler 

Dear Dustin,

I had a look at your program on my machine with the most recent 
development version of deal.II. Here is the final timer output in 
release mode:


+-+++
| Total wallclock time elapsed since start| 1.02s ||
| |||
| Section | no. calls |  wall time | % of total |
+-+---+++
| Setup   | 1 | 0.618s |   60.% |
| copy_sparsity_pattern   | 1 | 0.0557s |   5.4% |
| setup_intial_fe_distribution| 1 | 0.0119s |   1.2% |
| setup_system| 1 | 0.606s |   59.% |
+-+---+++

As you can see, the copy operation is very cheap on my machine at 
0.0557s versus 20.2s on your machine. When looking at the profiles, I 
cannot see anything suspicious either. My primary suspecion right now is 
that you are on an old version of deal.II where we had bad code in that 
function. (But I don't remember having changed anything during the last 
year.) What version are you using?


Best,
Martin

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-08-15 Thread Dustin Kumor 
Dear Martin,

after one week on a conference and two additional weeks on holidays I am 
finally back in office and realised the test case. I attached the file to 
the post and also log files for runs for both in Debug and Release mode.

You can call DynamicSparsityPattern::n_nonzero_elements() to get the number 
> of nonzero entries in the dynamic sparsity pattern. This method also exists 
> in BlockSparsityPattern (or all sparsity patterns that inherit from 
> BlockSparsityPatternBase):
>
> https://dealii.org/developer/doxygen/deal.II/classBlockSparsityPatternBase.html
>
> What I'm trying to understand here is what kind of properties your problem 
> has - whether there are many nonzero entries per row and other special 
> things that could explain your problems.
>
> I just checked the 3D case of step-22 for the performance of 
> BlockSparsityPattern::copy_from(BlockDynamicSparsityPattern) and the 
> performance looks where I would expect it to be. It takes 1.19s to copy the 
> sparsity pattern for a case with 1.6m DoFs (I have some modifications for 
> the mesh compared to what you find online) on my laptop. Given that there 
> are 275m nonzero entries in that matrix and I need to touch around 4.4 GB 
> (= 4 x 275m x 4 bytes per unsigned int, once for clearing the data in the 
> pattern, once for reading in the dynamic pattern, once for writing into the 
> fixed pattern plus once for write-allocate on that last operation) of 
> memory here, I reach 26% of the theoretical possible on this machine (~14 
> GB/s memory transfer per core). While I would know how to reach more than 
> 80% of peak memory bandwidth here, this function is no way near being 
> relevant in the global run time in any of my performance profiles. And I'm 
> likely the deal.II person with most affinity to performance numbers.
>
>
I checked the number of nonzero elements. For 133,098 dofs I got 6,347,893 
nonzero elements in the BlockDynamicSparsityPattern and it took me 20.1535s 
in Release and 56.5698s in Debug mode respectively to copy them. 


> Is there anything else special about your configuration or problem as 
> compared to the cases presented in the tutorial? What deal.II version are 
> you using, what is the finite element? Any special constraints on those 
> systems?
>

The configuration is special in my case. I have got two triangulations to 
realize a domain which has more than one hanging node on a face. So I have 
two triangulations describing the same geometry and the geometry is 
separated in two parts. On the one part I have a quiet coarse mesh and on 
the other part I have fine mesh. With the first triangulation I use 
trilinear finite elements on the coarse part an FENothing on the finer part 
and with the other triangulation I do the same thing vice versa. On the 
interface I use Mortar basis functions to connect both parts in a weak 
sense. Therefore I have to figure out the interface part first, then find 
the corresponding degrees of freedom on the interface and with these 
information I create the sparsity patterns for the connection blocks on my 
own. This is why my block matrix has 3x3 blocks, the ones belonging to the 
first and second triangulation and the connecting one belonging to the 
Mortar part.
This is a rough overview concerning the setup situation. If we come to the 
conclusion, that this is an important fact, I can explain the whole idea in 
more detail. But this will be a lot of lines to write then.

Best regards
Dustin 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#include 


struct FacePointData
{
unsigned int
  master_face_no,
  slave_face_no,
  master_q_point_index;
dealii::hp::DoFHandler<3>::cell_iterator
  master_cell,
  slave_cell;
std::vector
  slave_face_dofs,
  master_face_dofs,
  slave_local_face_dofs,
  master_local_face_dofs,
  biorthogonal_face_dofs;
};

class InterfaceData
{
  public:
InterfaceData ();

std::map&
biorthogonal_master_dof_connections ();

std::map&
biorthogonal_slave_dof_connections ();

void
initialise (unsigned int const n_interface_q_points);

std::map&
master_biorthogonal_dof_connections ();

std::vector>&
master_q_points ();

std::map&
master_slave_dof_connections ();

unsigned int
n_cells_master ();

unsigned int
n_cells_slave ();

unsigned int
ndofs_biorthogonal ();

std::vector&
q_point_data ();

void

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-07-07 Thread Martin Kronbichler 

Dear Dustin,



How is your computational setup, i.e., how many nonzero entries do
you have in your matrix?

I'm not sure if I understand what you mean. Do you mean the number of 
nonzero entries in /SparseMatrix/ or in the /BlockSparsityPattern/ or 
in the dynamic one? How can I get this information?
You can call DynamicSparsityPattern::n_nonzero_elements() to get the 
number of nonzero entries in the dynamic sparsity pattern. This method 
also exists in BlockSparsityPattern (or all sparsity patterns that 
inherit from BlockSparsityPatternBase):

https://dealii.org/developer/doxygen/deal.II/classBlockSparsityPatternBase.html

What I'm trying to understand here is what kind of properties your 
problem has - whether there are many nonzero entries per row and other 
special things that could explain your problems.


I just checked the 3D case of step-22 for the performance of 
BlockSparsityPattern::copy_from(BlockDynamicSparsityPattern) and the 
performance looks where I would expect it to be. It takes 1.19s to copy 
the sparsity pattern for a case with 1.6m DoFs (I have some 
modifications for the mesh compared to what you find online) on my 
laptop. Given that there are 275m nonzero entries in that matrix and I 
need to touch around 4.4 GB (= 4 x 275m x 4 bytes per unsigned int, once 
for clearing the data in the pattern, once for reading in the dynamic 
pattern, once for writing into the fixed pattern plus once for 
write-allocate on that last operation) of memory here, I reach 26% of 
the theoretical possible on this machine (~14 GB/s memory transfer per 
core). While I would know how to reach more than 80% of peak memory 
bandwidth here, this function is no way near being relevant in the 
global run time in any of my performance profiles. And I'm likely the 
deal.II person with most affinity to performance numbers.


Thus my interest in what is particular about your setup.


Have you checked that you do not run out of memory and see a large
swap time?

 I'm quiet sure that this is not the case/problem since I used one of 
our compute servers with 64 GB memory. Moreover, at the moment the 
program runs with an additional global refinement, i.e. about 16 
million dofs and only 33% of the memory is used. Swap isn't used at all.
That's good to know, so we can exclude the memory issue. Does your 
program use multithreading? It probably does in case you do not do 
anything special when configuring deal.II; the copy operation is not 
parallelized by threads but neither are almost all other initialization 
functions, so it should not become such a disproportionate timing here. 
10h for 2.5m dofs looks insane. I would expect something between 0.5 and 
10 seconds, depending on the number of nonzeros in those blocks.


Is there anything else special about your configuration or problem as 
compared to the cases presented in the tutorial? What deal.II version 
are you using, what is the finite element? Any special constraints on 
those systems?


Unfortunately this can not be done that easy. I have to reorganize 
things and kill a lot of superflous code. But besides that, I have a 
lot of other work to do. May be I can provide you an example file at 
the end of next week.
Let us know when you have a test case. I'm really curious what could 
cause this huge run time.


Best,
Martin

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-07-07 Thread Dustin Kumor 
Dear Martin,

thank you for answering that quickly. 

How is your computational setup, i.e., how many nonzero entries do you have 
> in your matrix? 
>
I'm not sure if I understand what you mean. Do you mean the number of 
nonzero entries in *SparseMatrix* or in the *BlockSparsityPattern* or in 
the dynamic one? How can I get this information?
 

> Have you checked that you do not run out of memory and see a large swap 
> time?
>
 I'm quiet sure that this is not the case/problem since I used one of our 
compute servers with 64 GB memory. Moreover, at the moment the program runs 
with an additional global refinement, i.e. about 16 million dofs and only 
33% of the memory is used. Swap isn't used at all.

>  

> How do the run times behave when you choose a smaller problem size? (I 
> wonder if there is some higher than O(N) complexity somewhere.)
>
Even in this cases the time needed to copy is comparatively long. I 
attached a log file wherein the the times are listed for different numbers 
of dofs.

  It would be very helpful if you could provide us an example file that 
> only contains the setup phase so we can investigate the issue further.
>
Unfortunately this can not be done that easy. I have to reorganize things 
and kill a lot of superflous code. But besides that, I have a lot of other 
work to do. May be I can provide you an example file at the end of next 
week.

Best regards,
Dustin

>

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
JobId alliance Thu Jun 23 13:22:53 2016

*
*  Starting Simulation on Thu, 23.06.2016 at 13:22:53
**

Cycle::0 (Initial mesh)
Setup:: ...
Setup:setup_intial_fe_distribution:: ...
Setup:setup_intial_fe_distribution:Time:: < 1 sec
setup_intial_fe_distribution, wall time: 0.000339985s.
Setup:setup_system:: ...
Setup:setup_system:setup_data:: ...
Setup:setup_system:setup_data:detect_contact_interface:: ...
Setup:setup_system:setup_data:detect_contact_interface:Time:: < 1 sec
Setup:setup_system:setup_data:setup_interface_q_point_data:: ...
Setup:setup_system:setup_data:setup_interface_q_point_data:Time:: < 1 sec
Setup:setup_system:setup_data:setup_interface_dof_connections:: ...
Setup:setup_system:setup_data:setup_interface_dof_connections:Time:: < 1 sec
Setup:setup_system:setup_data:Time:: < 1 sec
Setup:setup_system:hanging_node_constraints:: ...
Setup:setup_system:hanging_node_constraints:Time:: < 1 sec
Setup:setup_system:interpolate_boundary_values:: ...
Setup:setup_system:interpolate_boundary_values:Time:: < 1 sec
Setup:setup_system:reinit_sparsity_pattern:: ...
Setup:setup_system:reinit_sparsity_pattern:Time:: < 1 sec
Setup:setup_system:make_sparsity_pattern_deal:: ...
Setup:setup_system:make_sparsity_pattern_deal:Time:: < 1 sec
Setup:setup_system:make_biorthogonal_sparsity_pattern:: ...
Setup:setup_system:make_biorthogonal_sparsity_pattern:Time:: < 1 sec
Setup:setup_system:copy_sparsity_pattern:: ...
Setup:setup_system:copy_sparsity_pattern:Time:: < 1 sec
Setup:setup_system:merge_and_close_constraints:: ...
Setup:setup_system:merge_and_close_constraints:Time:: < 1 sec
Setup:setup_system:Time:: < 1 sec
Setup:setup_system:: n_dofs dofhandler_m:   81
 n_dofs dofhandler_s:   135
 n_dofs biorthogonal basis: 45
 n_dofs all:261

setup_system, wall time: 0.00929093s.
Setup:: Number of active cells:   24
Setup:: Number of degrees of freedom: 216
Setup:Time:: < 1 sec
Setup, wall time: 0.0101869s.
Assembling::...
Assembling:Time:: < 1 sec
Assembling, wall time: 0.00511193s.
Solving::...
Solving:cg::Starting value 0.130903
Solving:cg::Convergence step 7 value 1.40409e-08
Solving:cg::Starting value 0.103671
Solving:cg::Convergence step 7 value 1.58635e-08
Solving:cg::Starting value 0.0645919
Solving:cg:cg::Starting value 0.00513355
Solving:cg:cg::Convergence step 7 value 1.15629e-09
Solving:cg:cg::Starting value 0.00403430
Solving:cg:cg::Convergence step 7 value 2.42599e-09
Solving:cg:cg::Starting value 0.00101849
Solving:cg:cg::Convergence step 7 value 1.41922e-10
Solving:cg:cg::Starting value 0.000834389
Solving:cg:cg::Convergence step 7 value 5.36044e-10
Solving:cg:cg::Starting value 0.000228587
Solving:cg:cg::Convergence step 7 value 9.29881e-11
Solving:cg:cg::Starting value 0.000295453
Solving:cg:cg::Convergence step 7 value 2.47793e-10
Solving:cg:cg::Starting value 0.000123562
Solving:cg:cg::Convergence step 7 value 3.51006e-11
Solving:cg:cg::Starting value 0.000169932
Solving:cg:cg::

Re: [deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-07-06 Thread Martin Kronbichler 

Dear Dustin,

With regard to a lot of tutorial programms I would say that this is 
the usual way to work with /DynamicSparsityPatterns/, isn't it?
Yes, the description you give is exactly the way I would choose for the 
sparsity pattern.


So back to the question. Running the program the copy operation takes 
a huge amount of time. Just giving you some figures. With

ndofs_m = 823875,
ndofs_s = 1635075,
ndofs_bi = 25155,
meaning about 2.5 million dofs at all

it takes about 10 hours and 50 minutes to copy the 
/BlockDynamicSparsityPattern /in contrast to an assembling time of 2 
minutes and 38 seconds. So the question is if it is normal the copying 
operation taking so much time?
No, copying the sparsity pattern should in general be very fast. The 
functions that do this should be reasonably optimized so that you mostly 
pay for the memory access. It could be that we missed something for the 
block case, though. How is your computational setup, i.e., how many 
nonzero entries do you have in your matrix? Have you checked that you do 
not run out of memory and see a large swap time? How do the run times 
behave when you choose a smaller problem size? (I wonder if there is 
some higher than O(N) complexity somewhere.)



And also whether there is way to increase the performance?
By the way the program was compiled in release mode.
It should be possible to do the whole copy operation in say 2x the time 
it takes to zero a sparse matrix. If it's a bug we will fix it. It would 
be very helpful if you could provide us an example file that only 
contains the setup phase so we can investigate the issue further.


Best,
Martin

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[deal.II] Question concerning BlockSparsityPattern.copy_from() member function

2016-07-06 Thread Dustin Kumor 
Dear all,

I have got a question about the time it takes to copy a 
*BlockDynamicSparsityPattern 
*into a* BlockSparsityPattern* using the *copy_from()*-function.
In my special case I have a vector-valued 3D problem with trilinear finite 
elements. The system matrix is 3x3 block matrix with non-zero blocks (0,0), 
(0,2), (1,1), (1,2), (2,0), (2,1).  I'm setting up the 
*BlockDynamicSparsityPattern 
*the following way:

BlockDynamicSparsityPattern dsp (3,3);
dsp.block(0,0).reinit (ndofs_m,  ndofs_m);
dsp.block(0,1).reinit (ndofs_m,  ndofs_s);
dsp.block(0,2).reinit (ndofs_m,  ndofs_bi);
dsp.block(1,0).reinit (ndofs_s,  ndofs_m);
dsp.block(1,1).reinit (ndofs_s,  ndofs_s);
dsp.block(1,2).reinit (ndofs_s,  ndofs_bi);
dsp.block(2,0).reinit (ndofs_bi, ndofs_m);
dsp.block(2,1).reinit (ndofs_bi, ndofs_s);
dsp.block(2,2).reinit (ndofs_bi, ndofs_bi);
dsp.collect_sizes();

After that I create the sparsity pattern for the (0,0) and the (1,1) block

DoFTools::make_sparsity_pattern(dofhandler_m,
dsp.block(0,0),
constraints_m,
false);

DoFTools::make_sparsity_pattern(dofhandler_s,
dsp.block(1,1),
constraints_s,
false);

and then for the off-diagonal blocks

   make_offdiagonal_sparsity_pattern (dsp);

The last thing is to copy the *BlockDynamicSparsityPattern *into a


* BlockSparsityPatternsparsity_pattern.copy_from(dsp);*With regard to a 
lot of tutorial programms I would say that this is the usual way to work 
with *DynamicSparsityPatterns*, isn't it?

So back to the question. Running the program the copy operation takes a 
huge amount of time. Just giving you some figures. With
ndofs_m = 823875,
ndofs_s = 1635075,
ndofs_bi = 25155,
meaning about 2.5 million dofs at all

it takes about 10 hours and 50 minutes to copy the *BlockDynamicSparsityPattern 
*in contrast to an assembling time of 2 minutes and 38 seconds. So the 
question is if it is normal the copying operation taking so much time? And 
also whether there is way to increase the performance?
By the way the program was compiled in release mode.

Best regards
Dustin

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.