Re: [deal.II] segmentation fault when calling make_vectorized_array

[Denis Davydov]

Thanks a lot for your help, Martin!

Cheers,
Denis

> On 13 Jul 2017, at 11:02, Martin Kronbichler  
> wrote:
> 
> Hi Denis,
> 
>> While on the topic of alignment, do I need to be more careful about using
>> 
>> template 
>> struct QPData
>> {
>>Table<2, VectorizedArray > values;
>> }
>> QPData qp_data;
>> 
>> as a member variable in my class? 
> 
> Table is fine because it is based on AlignedVector, so it 
> will use the correct alignment internally.
> 
> Best,
> Martin
> 
> -- 
> The deal.II project is located at http://www.dealii.org/ 
> 
> For mailing list/forum options, see 
> https://groups.google.com/d/forum/dealii?hl=en 
> 
> --- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/dealii/xccZy4KU7Zc/unsubscribe 
> .
> To unsubscribe from this group and all its topics, send an email to 
> dealii+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] segmentation fault when calling make_vectorized_array

[Martin Kronbichler]

Hi Denis,


> While on the topic of alignment, do I need to be more careful about using
>
> template
> structQPData
> {
>Table<2, VectorizedArray > values;
> }
> QPData qp_data;
>
> as a member variable in my class?

Table is fine because it is based on AlignedVector,
so it will use the correct alignment internally.

Best,
Martin

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] segmentation fault when calling make_vectorized_array

[Denis Davydov]

Hi Martin,

> On 13 Jul 2017, at 09:13, Martin Kronbichler  
> wrote:
> 
>> 
>> no, it's just a member variable VectorizedArray: 
>> 
>> VectorizedArray a;
> 
> But then the class/struct that holds this member variable is not properly 
> aligned, you probably have it inside an std::vector or allocated it with 
> 'new'. You will need to put it into an AlignedVector or similar type.

this is a matrix-free level operator which lives in MGLevelObject.
Indeed, internally it stores objects in std::vector of shared pointers.

> Alternatively you need to wrap the member that is of type VectorizedArray 
> into an t.

This worked, thanks a lot!


While on the topic of alignment, do I need to be more careful about using

template 
struct QPData
{
   Table<2, VectorizedArray > values;
}
QPData qp_data;

as a member variable in my class? 

> 
>> 
>>   >│0x773913cb ::Operator()+35> 
>>   vmovaps %ymm0,0xe0(%rbx)
> 
> This is the offending instruction. The compiler tries to store to the address 
> given by an offset of 0xe0 (224 bytes) to the value in %rbx, using an 
> 'aligned' store operation (vmovAps) that assumes 32 byte alignment. If you 
> print the value of %rbx (in gdb with 'print $rbx'), I'm pretty sure you get 
> an address that is divisible by 16 but not by 32. It is a pity that the 
> compiler / API of the intrinsics forces the compiler to 

indeed:

(gdb) print $rbx
$1 = 22305328

Thanks a lot for detailed explanation.

Kind regards,
Denis.

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] segmentation fault when calling make_vectorized_array

[Martin Kronbichler]

Hi Denis,




no, it's just a member variable VectorizedArray:

VectorizedArray a;


But then the class/struct that holds this member variable is not 
properly aligned, you probably have it inside an std::vector or 
allocated it with 'new'. You will need to put it into an AlignedVector 
or similar type. Alternatively you need to wrap the member that is of 
type VectorizedArray into an AlignedVector.




|
  >│0x773913cb ::Operator()+35>   vmovaps %ymm0,0xe0(%rbx)

|


This is the offending instruction. The compiler tries to store to the 
address given by an offset of 0xe0 (224 bytes) to the value in %rbx, 
using an 'aligned' store operation (vmovAps) that assumes 32 byte 
alignment. If you print the value of %rbx (in gdb with 'print $rbx'), 
I'm pretty sure you get an address that is divisible by 16 but not by 
32. It is a pity that the compiler / API of the intrinsics forces the 
compiler to use an aligned access instruction whenever there is a 
variable of type VectorizedArray but there's nothing we can change and 
we have to live with that...


Best,
Martin

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] segmentation fault when calling make_vectorized_array

[Denis Davydov]

Hi Martin,

On Wednesday, July 12, 2017 at 6:05:30 PM UTC+2, Martin Kronbichler wrote:
>
> Hi Denis,
>
> what is a? Is it an element of an array? If yes, is the array of type 
> AlignedVector (or derived from that). 
>

no, it's just a member variable VectorizedArray: 

VectorizedArray a;

 

> You get this kind of error when there is a load or store to an address 
> that is not aligned by the size of the vectorized array, 32 bytes in your 
> case. The data types VectorizedArray make the compiler insert 'aligned 
> load/store' type of operations, which cannot always be guaranteed.
>
> Can you try to look at the problem in a debugger, e.g. gdb and post the 
> offending assembler line and a few before that to see the context?
>

here they are:

0x773913b1 ::Operator()+9> 
   mov%rdi,%rbx 
   │0x773913b4 ::Operator()+12>   callq  0x771d1f90 
<_ZN6dealii19MatrixFreeOperators4BaseILi3ENS_13LinearAlgebra11distributed6VectorIfEEEC2Ev@plt
   │0x773913b9 ::Operator()+17>   mov0x9ecd60(%rip),%rax# 0x77d7e120 
 
   │0x773913c0 ::Operator()+24>   lea0x10(%rax),%rax  
   │0x773913c4 ::Operator()+28>   mov%rax,(%rbx)  
   │0x773913c7 ::Operator()+31>   vxorps %xmm0,%xmm0,%xmm0
  >│0x773913cb ::Operator()+35>   vmovaps %ymm0,0xe0(%rbx)
   │0x773913d3 ::Operator()+43>   vmovaps 0x64ffc5(%rip),%ymm0# 0x779e13a0
   │0x773913db ::Operator()+51>   vmovaps %ymm0,0x100(%rbx)   
   │0x773913e3 ::Operator()+59>   mov-0x8(%rbp),%rbx 
   │0x773913e7 ::Operator()+63>   leaveq   
   │0x773913e8 ::Operator()+64>   retq

(I did a few replacement to ease the readability). Hopefully you can make 
sense out of it. 

Regards,
Denis. 

-- 
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [deal.II] segmentation fault when calling make_vectorized_array

[Martin Kronbichler]

Hi Denis,

what is a? Is it an element of an array? If yes, is the array of type 
AlignedVector (or derived from that).


You get this kind of error when there is a load or store to an address 
that is not aligned by the size of the vectorized array, 32 bytes in 
your case. The data types VectorizedArray make the compiler insert 
'aligned load/store' type of operations, which cannot always be guaranteed.


Can you try to look at the problem in a debugger, e.g. gdb and post the 
offending assembler line and a few before that to see the context?


Best,
Martin


On 12.07.2017 17:41, Denis Davydov wrote:

Hi all,

With the current master (182e974) I get a weird segmentation fault for 
*float* numbers when calling


|
 a =make_vectorized_array(0.);
|

inside a constructor of my class where

|
VectorizedArraya;
|

I see this only on a cluster (Intel Xeon Ivy Bridge), deal.II is 
compiled with *-march=native* with GCC 4.8.5 in Debug mode (obviously 
segmentation fault is there in Release as well).


Unfortunately, I wiped the old (working) installation and can't check 
it now,

but I think this used to work with *-march=native* a few months back.

Relevant config tests are:

|
--PerformingTestDEAL_II_HAVE_SSE2
--PerformingTestDEAL_II_HAVE_SSE2 -Success
--PerformingTestDEAL_II_HAVE_AVX
--PerformingTestDEAL_II_HAVE_AVX -Success
--PerformingTestDEAL_II_HAVE_AVX512
--PerformingTestDEAL_II_HAVE_AVX512 -Failed
--PerformingTestDEAL_II_HAVE_OPENMP_SIMD
--PerformingTestDEAL_II_HAVE_OPENMP_SIMD -Failed
|

If I try to replicate what *make_vectorized_array* does, then it works 
fine whereas calling it gives segfault:


|
constvalue_type u =0.;
VectorizedArrayresult;
  result =u;
// ^^ copy paste of make_vectorized_array
  a =u;
VectorizedArraydummy =make_vectorized_array(0.);// <- 
quick test for doubles
  a =make_vectorized_array(0.);<=Segmentationfault 
here!

|

I am puzzled with this... Any ideas where to dig?

Cheers,
Denis
--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en

---
You received this message because you are subscribed to the Google 
Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to dealii+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see 
https://groups.google.com/d/forum/dealii?hl=en
--- 
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to dealii+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.