Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 11:50 PM, Jürgen Hestermann  
> wrote:
> 
> I am trying to show the memory allocation for the 10x10 array as a "graphic":
> 
> arr --> arr[0],arr[1],arr[2],arr[3],arr[4],arr[5],arr[6],arr[7],arr[8],arr[9]
>  |  |  |  |  |  |  |  | |  |
>  V  V  V  V  V  V  V  V V  V
>arr[0,0],   .  .  .  .  .  . .arr[9,0],
>arr[0,1],   .  .  .  .  .  . .arr[9,1],
>arr[0,2],   .  .  .  .  .  . .arr[9,2],
> arr[0,3], arr[9,3],
> arr[0,4], arr[9,4],
> arr[0,5], arr[9,5],
> ......
> 
> arr is a single pointer (that points to arr[0]).
> arr[0] to arr[9] are (10) pointers located in a continuous memory block each 
> pointing to
> arr[0,0],
> arr[1,0],
> arr[3,0], and so on...
> arr[0,0] is a single integer.
> arr[0,0] to arr[0,9] are (10) integers located in a continuous memory block.
> The same applies for
> arr[1,0] to arr[1,9],
> arr[2,0] to arr[2,9],
> arr[3,0] to arr[3,9], and so on.

Thanks that helps. Indeed this is not what I need and I’m not even taking 
advantage of the resizable elements so I better not use dynamic arrays for my 
matrix.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 10:10 PM, Sven Barth via fpc-pascal 
>  wrote:
> 
> While your statement regarding allocation might be true you must not
> forget that a dynamic array consists of a meta data block (length,
> reference count) that is located directly in front of the data block. So
> even if the memory blocks would be allocated consecutively then there'd
> still be the meta data blocks inbetween.
> 
> If you already know that your dynamic arrays only have a specific size
> (for matrices used in games that should usually be the case) then you're
> better off with using a static array:
> 
> === code begin ===
> 
> type
>  TMatrix = array[0..2, 0..2, 0..2] of LongInt;
> 
> === code end ===
> 
> There you can use FillChar() as much as you want as that is indeed a
> single memory block containing 9 LongInt elements.

Yeah after all this talk, I’m going to use array[0..0, 0..0, 0..0] and allocate 
the memory myself to avoid overhead and confusion. Thanks for explaining 
everything to me but this time going low level makes the most sense.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Jürgen Hestermann]

Am 2017-04-04 um 16:54 schrieb Ryan Joseph:
>> var arr : array of array of Integer;
>> begin
>> SetLength(arr, 10, 10);
> “then the first array stores a pointer to each sub array.”
> Could you illustrate this is code?
> I don’t think I’m understanding this exactly like it’s represented in memory.
> There’s only one “sub array” in this 2x2 array so how does that look in 
memory?

Why 2x2?
SetLength(arr,10,10) creates a 10x10 array.

I am trying to show the memory allocation for the 10x10 array as a "graphic":

arr --> arr[0],arr[1],arr[2],arr[3],arr[4],arr[5],arr[6],arr[7],arr[8],arr[9]
  |  |  |  |  |  |  |  | |  |
  V  V  V  V  V  V  V  V V  V
arr[0,0],   .  .  .  .  .  . .arr[9,0],
arr[0,1],   .  .  .  .  .  . .arr[9,1],
arr[0,2],   .  .  .  .  .  . .arr[9,2],
arr[0,3], arr[9,3],
arr[0,4], arr[9,4],
arr[0,5], arr[9,5],
......

arr is a single pointer (that points to arr[0]).
arr[0] to arr[9] are (10) pointers located in a continuous memory block each 
pointing to
arr[0,0],
arr[1,0],
arr[3,0], and so on...
arr[0,0] is a single integer.
arr[0,0] to arr[0,9] are (10) integers located in a continuous memory block.
The same applies for
arr[1,0] to arr[1,9],
arr[2,0] to arr[2,9],
arr[3,0] to arr[3,9], and so on.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

On 04.04.2017 16:27, Ryan Joseph wrote:
>>> Does SetLength on a single level dynamic array not even allocate a 
>>> continuous block of memory?
>>
>> Yes, it does (as explained in all the other mails).
>> A (dynamic) array of integer will be allocated as a single block by 
>> SetLength.
>> So if you only have one level of a dynamic array as in
>>
>> var MyArray : array of integer;
>>
>> then SetLength(MyArray,1000) will allocate a single block of 1000 integers.
>> But with
>>
>> var MyArray : array of array of integer;
>>
>> SetLength(MyArray,1000);
>>
>> will allocate a single block of 1000 pointers (to an array of integer each).
>> Then, SetLength(MyArray[0],1000) will allocate one (!) block of 1000 
>> integers.
>> SetLength(MyArray[1],1000) will allocate another block of 1000 integers and 
>> so on….
> 
> I was allocating using SetLength(MyArray, 3, 3, 3) for a 3x3x3 matrix for 
> example. Maybe it depends on the memory manager and the state of heap but if 
> you called early in the programs execution it should be allocate all that 
> memory in one block I would think. 

Addendum: If you look at fpc_dynarray_setlength() again then you'll see
at line 289 that it's calling itself recursively for nested arrays.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

On 04.04.2017 16:27, Ryan Joseph wrote:
>>> Does SetLength on a single level dynamic array not even allocate a 
>>> continuous block of memory?
>>
>> Yes, it does (as explained in all the other mails).
>> A (dynamic) array of integer will be allocated as a single block by 
>> SetLength.
>> So if you only have one level of a dynamic array as in
>>
>> var MyArray : array of integer;
>>
>> then SetLength(MyArray,1000) will allocate a single block of 1000 integers.
>> But with
>>
>> var MyArray : array of array of integer;
>>
>> SetLength(MyArray,1000);
>>
>> will allocate a single block of 1000 pointers (to an array of integer each).
>> Then, SetLength(MyArray[0],1000) will allocate one (!) block of 1000 
>> integers.
>> SetLength(MyArray[1],1000) will allocate another block of 1000 integers and 
>> so on….
> 
> I was allocating using SetLength(MyArray, 3, 3, 3) for a 3x3x3 matrix for 
> example. Maybe it depends on the memory manager and the state of heap but if 
> you called early in the programs execution it should be allocate all that 
> memory in one block I would think. 

While your statement regarding allocation might be true you must not
forget that a dynamic array consists of a meta data block (length,
reference count) that is located directly in front of the data block. So
even if the memory blocks would be allocated consecutively then there'd
still be the meta data blocks inbetween.

If you already know that your dynamic arrays only have a specific size
(for matrices used in games that should usually be the case) then you're
better off with using a static array:

=== code begin ===

type
  TMatrix = array[0..2, 0..2, 0..2] of LongInt;

=== code end ===

There you can use FillChar() as much as you want as that is indeed a
single memory block containing 9 LongInt elements.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Mattias Gaertner]

On Tue, 4 Apr 2017 16:46:02 +0200
Sven Barth via fpc-pascal  wrote:

>[...]
> SetLength() allocates a single block of memory,

To avoid misunderstanding: SetLength(a,dim1,dim2) allocates one block
for the dim1 array and then for each element another block.

Mattias
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 10:07 PM, Jürgen Hestermann  
> wrote:
> 
> Am 2017-04-04 um 15:40 schrieb Ryan Joseph:
> > I’m glad I asked because of arrays of pointers is bad news for performance.
> 
> I don't think that you will notice a performance issue with dynamic arrays 
> (though it highly depends on the sizes and levels you use...)

As I understand it accessing continuous blocks of memory in tight loops 
increases the chance that it will be in cache L1/L2 cache. It’s probably 
trivial for my use but I’m making some game code which is highly performance 
sensitive and I’m trying to learn some best practices when writing data types. 
I watched a couple presentations from a compiler engineer from Google (works on 
clang and LLVM) that explained how this worked and it’s something worth 
learning and paying attention to imo.

> 
> > Does SetLength on a single level dynamic array not even allocate a 
> > continuous block of memory?
> 
> Yes, it does (as explained in all the other mails).
> A (dynamic) array of integer will be allocated as a single block by SetLength.
> So if you only have one level of a dynamic array as in
> 
> var MyArray : array of integer;
> 
> then SetLength(MyArray,1000) will allocate a single block of 1000 integers.
> But with
> 
> var MyArray : array of array of integer;
> 
> SetLength(MyArray,1000);
> 
> will allocate a single block of 1000 pointers (to an array of integer each).
> Then, SetLength(MyArray[0],1000) will allocate one (!) block of 1000 integers.
> SetLength(MyArray[1],1000) will allocate another block of 1000 integers and 
> so on….

I was allocating using SetLength(MyArray, 3, 3, 3) for a 3x3x3 matrix for 
example. Maybe it depends on the memory manager and the state of heap but if 
you called early in the programs execution it should be allocate all that 
memory in one block I would think. 

Another bottle neck of not using arrays of pointers is you need to perform some 
math if you want to know which element 2,2,2 is. I’m probably getting in over 
my head now. :)


Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

On 04.04.2017 16:54, Ryan Joseph wrote:
> 
>> On Apr 4, 2017, at 9:46 PM, Sven Barth via fpc-pascal 
>>  wrote:
>>
>> SetLength() allocates a single block of memory, cause array access is
>> ordinary pointer arithmetic. However if you have an array of array then
>> the first array stores a pointer to each sub array.
>>
>> E.g. the following would be valid, too:
>>
>> === code begin ===
>>
>> var
>>  arr: array of array of Integer;
>> begin
>>  SetLength(arr, 10, 10);
> 
> “then the first array stores a pointer to each sub array.”
> 
> Could you illustrate this is code? I don’t think I’m understanding this 
> exactly like it’s represented in memory. There’s only one “sub array” in this 
> 2x2 array so how does that look in memory?

Let's look at a smaller array, let's say 3 x 4, then it would look like
this:

arr = @arr[0], @arr[1], @arr[2]

arr[0] = 0, 0, 0, 0
arr[1] = 0, 0, 0, 0
arr[2] = 0, 0, 0, 0

Essentially you'd have four arrays, namely the outer array arr which
basically contains pointers to the other arrays, and the inner arrays
which are each three separate four element arrays. So all four of these
arrays could reside at completely different locations of the heap.

Is this clearer?

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

On 04.04.2017 15:40, Ryan Joseph wrote:
> 
>> On Apr 4, 2017, at 7:17 PM, Sven Barth via fpc-pascal 
>>  wrote:
>>
>> If you want continuous memory areas you need to use static arrays or develop 
>> your own dynamic data structure that uses array properties.
>>
>>
> 
> I’m glad I asked because of arrays of pointers is bad news for performance. 
> Does SetLength on a single level dynamic array not even allocate a continuous 
> block of memory? I could use GetMem and array[0..0] but it seems like dynamic 
> arrays should do basically that anyways if they’re not nested.

You could use something along these lines instead:

=== code begin ===

program tarrtest;

{$mode objfpc}
{$modeswitch advancedrecords}

type
  generic TTwoDimArray = record
  private
fData: array of T;
{ Note: Length1 and Length2 are not initialized by default, but you
could use trunk's management operators for that }
fLength1,
fLength2: LongInt;
function GetElement(Index1, Index2: LongInt): T; inline;
procedure SetElement(Index1, Index2: LongInt; aValue: T); inline;
  public
{ using SetLength() would lead to us needing to use
  "System.SetLength()" for the array which in turn would complain
  about usage of the static symtable; that's a problem that yet
  needs to be solved inside generics, for non-generics that would
  work however }
procedure AdjustLength(aLength1, aLength2: LongInt);
property Length1: LongInt read fLength1;
property Length2: LongInt read fLength2;
property Element[Index1, Index2: LongInt]: T read GetElement write
SetElement; default;
  end;

{ TTwoDimArray }

function TTwoDimArray.GetElement(Index1, Index2: LongInt): T;
begin
  { ToDo: Length check }
  Result := fData[Index1 * fLength1 + Index2];
end;

procedure TTwoDimArray.SetElement(Index1, Index2: LongInt; aValue: T);
begin
  { ToDo: Length check }
  fData[Index1 * fLength1 + Index2] := aValue;
end;

procedure TTwoDimArray.AdjustLength(aLength1, aLength2: LongInt);
begin
  SetLength(fData, aLength1 * aLength2);
  fLength1 := aLength1;
  fLength2 := aLength2;
end;

var
  arr: specialize TTwoDimArray;
  i: LongInt;
begin
  arr.AdjustLength(10, 5);
  i := arr[3, 2];
  arr[7, 1] := i;
end.

=== code end ===

Regards,
Sven

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 9:46 PM, Sven Barth via fpc-pascal 
>  wrote:
> 
> SetLength() allocates a single block of memory, cause array access is
> ordinary pointer arithmetic. However if you have an array of array then
> the first array stores a pointer to each sub array.
> 
> E.g. the following would be valid, too:
> 
> === code begin ===
> 
> var
>  arr: array of array of Integer;
> begin
>  SetLength(arr, 10, 10);

“then the first array stores a pointer to each sub array.”

Could you illustrate this is code? I don’t think I’m understanding this exactly 
like it’s represented in memory. There’s only one “sub array” in this 2x2 array 
so how does that look in memory?

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

On 04.04.2017 15:40, Ryan Joseph wrote:
> 
>> On Apr 4, 2017, at 7:17 PM, Sven Barth via fpc-pascal 
>>  wrote:
>>
>> If you want continuous memory areas you need to use static arrays or develop 
>> your own dynamic data structure that uses array properties.
>>
>>
> 
> I’m glad I asked because of arrays of pointers is bad news for performance. 
> Does SetLength on a single level dynamic array not even allocate a continuous 
> block of memory? I could use GetMem and array[0..0] but it seems like dynamic 
> arrays should do basically that anyways if they’re not nested.

SetLength() allocates a single block of memory, cause array access is
ordinary pointer arithmetic. However if you have an array of array then
the first array stores a pointer to each sub array.

E.g. the following would be valid, too:

=== code begin ===

var
  arr: array of array of Integer;
begin
  SetLength(arr, 10, 10);
  SetLength(arr[3], 5);
  arr[6] := Nil;
  SetLength(arr[8], 15);
end.

=== code end ===

Regards,
Sven

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Jürgen Hestermann]

Am 2017-04-04 um 15:40 schrieb Ryan Joseph:
> I’m glad I asked because of arrays of pointers is bad news for performance.

I don't think that you will notice a performance issue with dynamic arrays 
(though it highly depends on the sizes and levels you use...)

> Does SetLength on a single level dynamic array not even allocate a continuous 
block of memory?

Yes, it does (as explained in all the other mails).
A (dynamic) array of integer will be allocated as a single block by SetLength.
So if you only have one level of a dynamic array as in

var MyArray : array of integer;

then SetLength(MyArray,1000) will allocate a single block of 1000 integers.
But with

var MyArray : array of array of integer;

SetLength(MyArray,1000);

will allocate a single block of 1000 pointers (to an array of integer each).
Then, SetLength(MyArray[0],1000) will allocate one (!) block of 1000 integers.
SetLength(MyArray[1],1000) will allocate another block of 1000 integers and so 
on

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 7:17 PM, Sven Barth via fpc-pascal 
>  wrote:
> 
> If you want continuous memory areas you need to use static arrays or develop 
> your own dynamic data structure that uses array properties.
> 
> 

I’m glad I asked because of arrays of pointers is bad news for performance. 
Does SetLength on a single level dynamic array not even allocate a continuous 
block of memory? I could use GetMem and array[0..0] but it seems like dynamic 
arrays should do basically that anyways if they’re not nested.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Jürgen Hestermann]

Am 2017-04-04 um 15:18 schrieb Jürgen Hestermann:
> var MyArray : array of array of integer;
>
> you can do:
>
> SetLength(MyArray,3);
> SetLength(MyArray[0],2);
> SetLength(MyArray[1],3);
> SetLength(MyArray[2],4);
>
> So MyArray[0] points to an array of 2 integers,
> MyArray[1] points to an array of 3 integers and
> MyArray[2] points to an array of 4 integers.

The syntax of dynamic and static arrays are the same
although they should differ (because the involved pointers).
Therefore it is confusing and misleading.

For dynamic arrays
MyArray[0] should be MyArray^[0] and
MyArray[0,1] should be MyArray^[0]^[1]
and so on...

This would make it possible to distinguish between the pointer
(MyArray) and the data it points to (MyArray^).
Also MyArray^[0]^[1] would be a pointer while
MyArray^[0]^[1]^ would be the (static) array of integers.

But as you seldom access the pointers themself (they
are managed by the compiler) and because this automatic
derefencing of pointers for managed types has been defined
decades ago we have to live with this now...

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Jürgen Hestermann]

Am 2017-04-04 um 13:55 schrieb Ryan Joseph:
> Doesn’t iterating the array default the purpose of FillChar?
> The goal was the most efficient way clear the array with zero’s.
> Even if the array if nested 3 levels deep (anArray[x][y][z])
> it should (I hope) be a contiguous block of memory that was allocated 
(correct me if I wrong please).

No. If you have a dynamic array of multiple levels it
does not generate a continuous block for all levels.
Only the last level (if it is an array of integer or other non-managed type) is 
just that.
A (dynamic) array of array if an array of pointers.
Only for static arrays all levels are a continuous block.

The first level of a multiple level (dynamic) array is an array of pointers 
(array of array).
So you have a continuous block of pointers on the first level.

Each (!) second level array also is an array of pointers (array of array).

Only the last level is an array of integer (or whatever).

Therefore, for a 3x3x3 array you have 3 pointers on the first level.
Each of these pointers (MyArray[x]) again points to an array of 3 pointers.
And on the last level (MyArray[x,y]) each pointer points to an array of integer.
So you have 3x3=9 continuous blocks of 3 integers which can be located anywhere 
in memory.

BTW:
Each level of nested dynamic arrays can have individuell size.
On a 2-dimensional array

var MyArray : array of array of integer;

you can do:

SetLength(MyArray,3);
SetLength(MyArray[0],2);
SetLength(MyArray[1],3);
SetLength(MyArray[2],4);

So MyArray[0] points to an array of 2 integers,
MyArray[1] points to an array of 3 integers and
MyArray[2] points to an array of 4 integers.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

Am 04.04.2017 13:55 schrieb "Ryan Joseph" :
>
>
> > On Apr 4, 2017, at 4:58 PM, Howard Page-Clark via fpc-pascal <
fpc-pascal@lists.freepascal.org> wrote:
> >
> > You can always use FillChar and its kin on specific 'nested' arrays
like this
> >
> > type
> >  TIntArray = array of Integer;
> >  TIntIntArray = array of TIntArray;
> >  TIntIntIntArray = array of TIntIntArray;
> >
> >  procedure FillArray(const anArray: TIntIntIntArray; aValue: DWord);
> >  var
> >x, y: integer;
> >  begin
> >for x:=0 to High(anArray) do
> >  for y:=0 to High(anArray[x]) do
> >FillDWord(anArray[x][y][0], Length(anArray[x][y]), aValue);
> >  end;
>
> Doesn’t iterating the array default the purpose of FillChar? The goal was
the most efficient way clear the array with zero’s. Even if the array if
nested 3 levels deep (anArray[x][y][z]) it should (I hope) be a contiguous
block of memory that was allocated (correct me if I wrong please).
>

As I wrote that isn't the case. Each dynamic array is allocated
independently (and thus you could also resize each subelement
independently).

If you want continuous memory areas you need to use static arrays or
develop your own dynamic data structure that uses array properties.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Ryan Joseph]

> On Apr 4, 2017, at 4:58 PM, Howard Page-Clark via fpc-pascal 
>  wrote:
> 
> You can always use FillChar and its kin on specific 'nested' arrays like this
> 
> type
>  TIntArray = array of Integer;
>  TIntIntArray = array of TIntArray;
>  TIntIntIntArray = array of TIntIntArray;
> 
>  procedure FillArray(const anArray: TIntIntIntArray; aValue: DWord);
>  var
>x, y: integer;
>  begin
>for x:=0 to High(anArray) do
>  for y:=0 to High(anArray[x]) do
>FillDWord(anArray[x][y][0], Length(anArray[x][y]), aValue);
>  end;

Doesn’t iterating the array default the purpose of FillChar? The goal was the 
most efficient way clear the array with zero’s. Even if the array if nested 3 
levels deep (anArray[x][y][z]) it should (I hope) be a contiguous block of 
memory that was allocated (correct me if I wrong please). 

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

Am 04.04.2017 12:52 schrieb "Mark Morgan Lloyd" <
markmll.fpc-pas...@telemetry.co.uk>:
>
> On 02/04/17 10:00, Jonas Maebe wrote:
>
>> Allocating new memory via setlength also clears the memory (+ the
>> overhead of allocating the memory).
>
>
> Jonas, is it still the case that if SetLength() results in existing data
being moved that the original- which might be e.g. an unencrypted password-
isn't cleared?

The reallocation is delegated to the memory manager, thus SetLength() can
not know what is really done with the data area.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Jürgen Hestermann]

Am 2017-04-04 um 12:51 schrieb Mark Morgan Lloyd:
> On 02/04/17 10:00, Jonas Maebe wrote:
>> Allocating new memory via setlength also clears the memory (+ the
>> overhead of allocating the memory).
> Jonas, is it still the case that if SetLength() results in existing data 
being moved that the original- which might be e.g. an unencrypted password- isn't 
cleared?

Only formerly not existing elements are overwritten via fillchar.
Already existing data is not changed of course (otherwise you would
loose them).
If you have a (dynamic) array of 5 elements and you extend it
to 7 elements via Setlength(MyArray,7) then only the last 2 (new) elements
are cleared (because otherwise they would contain garbage).

This is (and must be) independend from moving data or just extending the 
existing array.
If the existing elements are moved (because the allocated memory cannot be 
extended)
then they are not cleared (but moved).
Only formerly not existing elements are filled with zeros and
they are never moved because they did not exist before the SetLength command.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Mark Morgan Lloyd]

On 02/04/17 10:00, Jonas Maebe wrote:


Allocating new memory via setlength also clears the memory (+ the
overhead of allocating the memory).


Jonas, is it still the case that if SetLength() results in existing data 
being moved that the original- which might be e.g. an unencrypted 
password- isn't cleared?


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Howard Page-Clark via fpc-pascal]

On 04/04/17 05:25, Ryan Joseph wrote:

Is it possible use FillChar on a multidimensional arrays?

arr: array of array of array of integer.
SetLength(arr, 3, 3, 3);
FillChar(arr[0], (3*3*3)*sizeof(integer), false);

I’m just getting crashes.


You can always use FillChar and its kin on specific 'nested' arrays like 
this


type
  TIntArray = array of Integer;
  TIntIntArray = array of TIntArray;
  TIntIntIntArray = array of TIntIntArray;

  procedure FillArray(const anArray: TIntIntIntArray; aValue: DWord);
  var
x, y: integer;
  begin
for x:=0 to High(anArray) do
  for y:=0 to High(anArray[x]) do
FillDWord(anArray[x][y][0], Length(anArray[x][y]), aValue);
  end;
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Setting record values

[Sven Barth via fpc-pascal]

Am 04.04.2017 05:25 schrieb "Ryan Joseph" :
>
> Thanks for the tips, I appreciate it.
>
> This is all pretty trivial but it’s kind of annoying that using an inline
class function is more efficient than a constructor despite having
identical functionality. It's tempting to remove the constructors now and
replace them with inline functions but it seems like the compiler should be
smarter and make this optimization for me.

At least for classes and objects the constructor is a bit more complicated
(especially the former ones), so that may be the reason that nothing is
done there for record constructors either. I'll check whether I can at
least get them to paricipate in auto inlining so that they would at least
be automatically inlined at higher optimization levels.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Array clearing

[Sven Barth via fpc-pascal]

Am 04.04.2017 06:55 schrieb "Ryan Joseph" :
>
>
> > On Apr 2, 2017, at 11:02 PM, Sven Barth via fpc-pascal <
fpc-pascal@lists.freepascal.org> wrote:
> >
> > can be easily seen by looking at the implementation of SetLength() in
> > $fpcdir/rtl/inc/dynarr.inc, fpc_dynarray_setlength().
> > Especially since line 220 (at least in the current revision) contains a
> > call to FillChar(), the whole ordeal can only be *more* complex than
> > FillChar().
>
> That’s good to know about SetLength.
>
> Is it possible use FillChar on a multidimensional arrays?
>
> arr: array of array of array of integer.
> SetLength(arr, 3, 3, 3);
> FillChar(arr[0], (3*3*3)*sizeof(integer), false);
>
> I’m just getting crashes.

That does only work with static arrays as dynamic arrays are in reality
managed pointers, thus you override the pointers themselves with your
FillChar. Not to mention that you're overwriting the memory behind the
outermost array as that only has a size of Length(arr) * SizeOf(Pointer).

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal