Re: [fpc-devel] Aligned dynamic arrays
> On Mar 30, 2019, at 5:02 PM, Jonas Maebe wrote:
>
> How is this done in the Metal C headers?
Looking this now it appears the padding may be put in the actual vector. Maybe
those macros put it in there? The fields of the struct need to be aligned on 16
bytes (4 floats) so. Does c++ even allow padding struct fields or is this kind
of fringe usage? I thought the compiler already did this so it could just add
more?
typedef struct
{
// Positions in pixel space
// (e.g. a value of 100 indicates 100 pixels from the center)
vector_float2 position;
// Floating-point RGBA colors
vector_float4 color;
} AAPLVertex;
typedef simd_float2 vector_float2;
/*! @abstract A vector of two 32-bit floating-point numbers.
* @description In C++ and Metal, this type is also available as
* simd::float2. The alignment of this type is greater than the alignment
* of float; if you need to operate on data buffers that may not be
* suitably aligned, you should access them using simd_packed_float2
* instead. */
typedef __attribute__((__ext_vector_type__(2))) float simd_float2;
/*! @abstract A vector of four 32-bit floating-point numbers.
* @description In C++ and Metal, this type is also available as
* simd::float4. The alignment of this type is greater than the alignment
* of float; if you need to operate on data buffers that may not be
* suitably aligned, you should access them using simd_packed_float4
* instead. */
typedef __attribute__((__ext_vector_type__(4))) float simd_float4;
>
>> If you saw the original bug report we’re trying to make a custom alignment
>> (of 4k) for the Metal framework. I propose the syntax because it solves the
>> problem of how to assign alignment and there is a precedent for it (i.e.
>> $align).
>
> No, I missed that. In that case, a vector type won't help. The best way then
> is, as has been mentioned elsewhere in this thread, a getmem variant that
> supports allocating memory on an aligned boundary. Similar to how unix
> platforms have posix_memalign.
>
> This falls outside the scope of direct support in the language, so a solution
> like Anthony's is better in this case if you want to encapsulate it
> completely.
I already started on it before I hit the snag and it’s not an unreasonably
intrusive thing to allocate the extra padding like I discussed earlier, but it
would probably require introducing a compiler directive (see $dynarrayalign
below) and adding alignment information to tarraydef. I’ll post what I finished
on GitHub if anyone wants to see how invasive the changes are.
There’s a good argument both ways and I guess it comes down to maintaining the
extra complexity in the compiler. I’m personally happy to use the record since
it’s feasible in 3.1.1 but I understand users who like the compiler syntax,
which is cleaner and probably faster.
So what should I do going forward? Should I abandon the idea or do any of the
other core team members want to me pursue this further?
{$push}
{$dynarrayalign 4096}
type
TIntArray = array of integer;
{$pop}
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Oh and might as well add dot product and cross product for vectors as well > ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On a related note, since a lot of the discussion here seems related to vector types, I seem to recall someone recently published an mmx, sse1/2/3/4, simd set of routines related to vector operations. I don't use much asm other than just being familiar with the basics of ia32 from long ago, but I do know there have been many enhancements for the instruction sets I mentioned which specifically greatly increase both the speed and parallelism of vector operation, including their combination with matrix math manipulation and vector transforms. I also know that the people in the mono project enhanced their compiler with special simd vector optimizations: https://www.mono-project.com/news/2016/12/20/system-numeric-vectors/ Having said that, might it be possible if we could incorporate official vector and matrix type and optimizations with the compiler? I think everyone would benefit from officially supported types of this nature that could have more eyes on them to ensure their operations are best optimized for the various types of computers and architectures free pascal supports. I can think of quite a few operations that would benefit from everyone contributing either their own ideas of the best optimizations or reporting problems or suggestions. Here is what I would suggest as a start. Use 4 component float (single and perhaps additionally double precision) vectors (X, Y, Z, W), 4x4 matrix type to match. For vectors we need these operations: add, subtract, multiply, divide, normalize, transpose For matrix types we need: identity, scale, rotate, translate, sheer, multiply (matrix * matrix), transform (vector * matrix), transpose, invert That's just a start, but these operations ought to have official simd support IMO Thanks ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On 30/03/2019 14:59, Ryan Joseph wrote:
On Mar 30, 2019, at 9:10 AM, Jonas Maebe wrote:
FPC always aligns data to its alignment as specified by the platform ABI.
{$align x} can be used to limit this alignment to a lower number. It cannot be
used to increase it.
This caused us quite a bit of problems with the Metal framework because we
needed padding fields also.
How is this done in the Metal C headers?
If you saw the original bug report we’re trying to make a custom alignment (of
4k) for the Metal framework. I propose the syntax because it solves the problem
of how to assign alignment and there is a precedent for it (i.e. $align).
No, I missed that. In that case, a vector type won't help. The best way
then is, as has been mentioned elsewhere in this thread, a getmem
variant that supports allocating memory on an aligned boundary. Similar
to how unix platforms have posix_memalign.
This falls outside the scope of direct support in the language, so a
solution like Anthony's is better in this case if you want to
encapsulate it completely.
Can you give some examples of the vector type? I don’t exactly know what you
guys are referring to.
A vector type is basically a (fairly small) array of integer or floating
point types. All current desktop (and many embedded) CPUs have special
execution units that operate on all elements of such vectors in
parallel, which means you can get better performance.
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
I always get nervous of mode switches because it's hard to remember them
or realise they exist, and it feels a little hacky, like it's a kind of
compatibility setting more than anything else. Like at one point, I
preferred there was an explicit way to set a type's alignment so there's no
ambiguity or risk of bugs if a different setting is used or otherwise
overridden. e.g.
type TVector4f = packed record
x, y, z, w: Single;
end align 16;
Just my thought.
Kit
On Sat 30/03/19 17:31 , Mattias Gaertner via fpc-devel
fpc-devel@lists.freepascal.org sent:
On Sat, 30 Mar 2019 12:57:48 -0400
Ryan Joseph wrote:
> > On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel
> > wrote:
> >
> > I guess you mean auto dereferencing.
> > {$ModeSwitch AutoDeref}
>
> Yeah I just found this by looking around in the compiler. Mind.
> Blown. No idea that existed!
And some people still ask for more modeswitches...
Mattias
___
fpc-devel maillist - fpc-devel@lists.freepascal.org [3]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[4]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Links:
--
[1] mailto:r...@thealchemistguild.com
[2] mailto:fpc-devel@lists.freepascal.org
[3] mailto:fpc-devel@lists.freepascal.org
[4] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On Sat, 30 Mar 2019 12:57:48 -0400
Ryan Joseph wrote:
> > On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel
> > wrote:
> >
> > I guess you mean auto dereferencing.
> > {$ModeSwitch AutoDeref}
>
> Yeah I just found this by looking around in the compiler. Mind.
> Blown. No idea that existed!
And some people still ask for more modeswitches...
Mattias
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel
> wrote:
>
> I guess you mean auto dereferencing.
> {$ModeSwitch AutoDeref}
Yeah I just found this by looking around in the compiler. Mind. Blown. No idea
that existed!
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On Sat, 30 Mar 2019 10:03:12 -0400
Ryan Joseph wrote:
> > On Mar 30, 2019, at 9:55 AM, Jonas Maebe
> > wrote:
> >> You are not required to dereference pointers to write to them.
> >> var
> >> P: PPoint;
> >> begin
> >> P := AlignedArray[0];
> >> P.X := 3; // can be okay
> >> AlignedArray[0].Y := 4; // can be okay as well
> >
> > That only works in {$mode delphi}
>
> Oh that’s why! How does this work in Delphi mode without pointers and
> why wasn’t it added to ObjFPC mode?
I guess you mean auto dereferencing.
{$ModeSwitch AutoDeref}
> That would have saved me some
> headache in the past if I knew this.
Mattias
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 30, 2019, at 9:55 AM, Jonas Maebe wrote:
>
>> You are not required to dereference pointers to write to them.
>> var
>> P: PPoint;
>> begin
>> P := AlignedArray[0];
>> P.X := 3; // can be okay
>> AlignedArray[0].Y := 4; // can be okay as well
>
> That only works in {$mode delphi}
Oh that’s why! How does this work in Delphi mode without pointers and why
wasn’t it added to ObjFPC mode? That would have saved me some headache in the
past if I knew this.
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> 1) In you example you are writing this with normal arrays: > > A[0].X := 0; > > And in my implementation it's exactly the same. Regarding writing the entire > type directly > > A[0]^ := V; > > If using the caret symbol ^ or an Item property is a deal breaker, then this > can be simplified using a custom smart pointer type similar to how the > aligned array type was implemented resulting in the same syntax as in the > first example. Yes, returning temporary smart pointer would do the trick. -1 syntax incompatibility then. I see this as a definite improvement. > 2) As I've implemented them aligned array is compatible with dynamic arrays. > They implicitly convert between the two with no loss of data. If you truly > want to share the data within each, then you can define functions to work on > referencing the data types they hold, allowing the same function to serve > either type. Well. Converting arrays is too slow to be usable, imho. Think about textures, vertex buffers, etc. > > For example: > > procedure VectorTransform(Matrix: TMat4; FirstVertex: PVec3; Count: Integer); > > and then > > Matrix.Rotate(12.5, 0, 0); > VectorTransform(Matrix, AlignedData[0], AlignedData.Length); > VectorTransform(Matrix, @DynamicArray[0], Length(AlignedData)); > > Or you can even overload VectorTransform if you don't want to see @ symbols. > > procedure VectorTransform(Matrix: TMat4; FirstVertex: var TVec3; Count: > Integer); overload; It's all right and will eventually work. But you miss much larger problem - that will require a lot of boilerplate/conversion/manual work. Think about mixing 3 graphical libraries using different array/matrix/vertex types. Instead of re-using each others code, one would eventually write 4th library. FPC community is too small to afford such useless combinatorics. > > 3) Regarding Hi, Lo, Length, I prefer to put these functions on types. For > example > > for I := A.Lo to A.Hi do A[I].X := I; > > Is that such a deal breaker? Yes. Generics / generic functions is a big deal. That would need two different functions for each different _array_ type. -- Regards, Denis Golovan ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 30, 2019, at 9:10 AM, Jonas Maebe wrote:
>
> FPC always aligns data to its alignment as specified by the platform ABI.
> {$align x} can be used to limit this alignment to a lower number. It cannot
> be used to increase it.
This caused us quite a bit of problems with the Metal framework because we
needed padding fields also. Is this something that could be added or would it
be a disaster to add padding to record fields? It sounds like there is already
a mechanism we could extend but I’m not sure.
>
>> Why not use a similar directive for array alignment? That would solve the
>> problem I discovered with insert/concat.
>> {$push}
>> {$align-array 4096}
>> type
>> TIntArray = array of integer;
>> {$pop}
>
> Arrays are aligned to the alignment requirements of their components. Again,
> a vector type would solve this, since vectors have their own alignment
> requirements in the ABIs.
If you saw the original bug report we’re trying to make a custom alignment (of
4k) for the Metal framework. I propose the syntax because it solves the problem
of how to assign alignment and there is a precedent for it (i.e. $align).
Can you give some examples of the vector type? I don’t exactly know what you
guys are referring to.
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On 30/03/2019 14:42, Anthony Walter wrote:
You are not required to dereference pointers to write to them.
var
P: PPoint;
begin
P := AlignedArray[0];
P.X := 3; // can be okay
AlignedArray[0].Y := 4; // can be okay as well
That only works in {$mode delphi}
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
You are not required to dereference pointers to write to them. var P: PPoint; begin P := AlignedArray[0]; P.X := 3; // can be okay AlignedArray[0].Y := 4; // can be okay as well ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
1) In you example you are writing this with normal arrays: A[0].X := 0; And in my implementation it's exactly the same. Regarding writing the entire type directly A[0]^ := V; If using the caret symbol ^ or an Item property is a deal breaker, then this can be simplified using a custom smart pointer type similar to how the aligned array type was implemented resulting in the same syntax as in the first example. 2) As I've implemented them aligned array is compatible with dynamic arrays. They implicitly convert between the two with no loss of data. If you truly want to share the data within each, then you can define functions to work on referencing the data types they hold, allowing the same function to serve either type. For example: procedure VectorTransform(Matrix: TMat4; FirstVertex: PVec3; Count: Integer); and then Matrix.Rotate(12.5, 0, 0); VectorTransform(Matrix, AlignedData[0], AlignedData.Length); VectorTransform(Matrix, @DynamicArray[0], Length(AlignedData)); Or you can even overload VectorTransform if you don't want to see @ symbols. procedure VectorTransform(Matrix: TMat4; FirstVertex: var TVec3; Count: Integer); overload; 3) Regarding Hi, Lo, Length, I prefer to put these functions on types. For example for I := A.Lo to A.Hi do A[I].X := I; Is that such a deal breaker? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On 30/03/2019 14:07, Ryan Joseph wrote:
I never got $align to work for records.
FPC always aligns data to its alignment as specified by the platform
ABI. {$align x} can be used to limit this alignment to a lower number.
It cannot be used to increase it.
Why not use a similar directive for array alignment? That would solve the
problem I discovered with insert/concat.
{$push}
{$align-array 4096}
type
TIntArray = array of integer;
{$pop}
Arrays are aligned to the alignment requirements of their components.
Again, a vector type would solve this, since vectors have their own
alignment requirements in the ABIs.
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 30, 2019, at 12:56 AM, Anthony Walter wrote:
>
> So the default property indexer returns references and not values. If you
> want values specifically you can use the Item property indexer.
>
> This way we can either access fields directly on the items withing the array
> like so:
>
> A[0].X := SomeValue;
>
Yes, but it’s a pointer so we need to dereference with ^.
A[0]^.X := 1;
That’s the one difference which we can’t resolve now. Basically we would need a
“var” result introduced into the language. C++ has that feature so maybe it’s
not crazy to implement in Pascal.
int& getter() {
return i;
}
function getInt: integer; var;
begin
result := i;
end;
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Exactly, alignment might be well attached to array or array element type. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 29, 2019, at 5:30 PM, denisgolovan wrote:
>
> Also I'd like to get an idea how this functionality is to play with existing
> alignment directives for records.
I never got $align to work for records. Isn’t that supposed to pad the fields
to 16 bytes? Doesn’t work for me.
{$push}
{$align 16}
type
TVertex = record
pos: TVec2;
col: TVec2;
end;
{$pop}
Why not use a similar directive for array alignment? That would solve the
problem I discovered with insert/concat.
{$push}
{$align-array 4096}
type
TIntArray = array of integer;
{$pop}
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Hi Jonas Vector type in FPC is too good to be true. I didn't dare to dream about it so far :) BR, Denis ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Hi Anthony > The more that I think about it, the more I am believe that using my approach > is probably the best solution. Hear me out please. > > First, it would be fairly easy to implement copy on write logic with a custom > record class. > > Second, because it's Pascal code, it's entirely possible to fully customize > how it operates now and into the future, without the need to modify the > compiler. Suppose later you decide that also want the ability allocate the > storage memory of using the same array interface, but with a vertex buffer > object as the mechanism for allocating storage. Maybe at a later time you > want to allow an arrays type to back a memory mapped file. > > In all cases, user code would be the proper avenue to implement these > features. Unless these is some special feature unique to an intrinsic type > that makes working with a user code type significantly more difficult, why > bother adding more complexity to the compiler and more complexity rigidly > into the built in types? > > Now that we have management operators, type helpers, generics, > implicit/explicit conversions, and operator overloading, why not use what we > already have and make it better? Can you, or anyone for that matter, find a > real problem with a type or other construct that user code solution can solve > trivially with approximately the same syntax? If so then yes I am all for > addling to the language and compiler. In all other cases when can use what > the nice compiler and language developers have created for us and do everyone > a favor by reporting and helping to fix the bugs those features might still > have. Yes. I am all for custom/user implementation in case it can be used everywhere together/interchangeably with native dynamic arrays. However FPC typing does not seem to cope with that (currently?). 1. e.g. primitive types (integers, floats) and records still need different handling in your TAlignedArray: A[0]^:=0; A.Item[0]:=0; vs A[0].X:=0; Standard arrays support A[0]:=0 and A[0].X:=0 via their compiler magic. 2. TAlignedArray memory layout can't be made compatible with dynamic arrays. Thus no existing functions accepting dynamic arrays would work with TAlignedArray and vice versa. And that's a huge deal as it just greatly increases fragmentation and messes with code re-use between libraries. Imho, Pascal is too good to follow into C/C++ custom arrays hell. 3. Does not support standard way of dealing with array like @A[0], High(A), Low(A), Length(A), etc. Think about generic functions accepting both standard dynamic array and your custom array/vector. Another minor issue is that your TAlignedArray is semantically more like Rust's Vec. It lacks public SetLength, but has Push/Pop/Clear. Thus it behaves differently to FPC array. -- Regards, Denis Golovan ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On 29/03/2019 22:30, denisgolovan wrote: Could we discuss more options here? Since this is mostly about vectors, I think the best approach is to add an actual vector type to the compiler (Vector Pascal may serve as inspiration on how to integrate it in the language). Anything from dynamic arrays to records to variables could then automatically align them to whatever is required by the target platform. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Denis, The more that I think about it, the more I am believe that using my approach is probably the best solution. Hear me out please. First, it would be fairly easy to implement copy on write logic with a custom record class. Second, because it's Pascal code, it's entirely possible to fully customize how it operates now and into the future, without the need to modify the compiler. Suppose later you decide that also want the ability allocate the storage memory of using the same array interface, but with a vertex buffer object as the mechanism for allocating storage. Maybe at a later time you want to allow an arrays type to back a memory mapped file. In all cases, user code would be the proper avenue to implement these features. Unless these is some special feature unique to an intrinsic type that makes working with a user code type significantly more difficult, why bother adding more complexity to the compiler and more complexity rigidly into the built in types? Now that we have management operators, type helpers, generics, implicit/explicit conversions, and operator overloading, why not use what we already have and make it better? Can you, or anyone for that matter, find a real problem with a type or other construct that user code solution can solve trivially with approximately the same syntax? If so then yes I am all for addling to the language and compiler. In all other cases when can use what the nice compiler and language developers have created for us and do everyone a favor by reporting and helping to fix the bugs those features might still have. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] Aligned dynamic arrays
Hi all Sorry for possible starting a new thread as I've just managed to subscribe to the list. Could we discuss more options here? Maybe separate aligned array is a better alternative? I mean a new type fully backward compatible with dynamic arrays, but with additional requirements (more specialized that is)? Like some interface (similar to memory manager record) for reallocating, deallocating, copy-on-write handling, etc. Ideally some state would be allowed as well, thus interface type. This way dynamic arrays might be more flexible. e.g. think of mmapped arrays on disk implementation support (requires extra state like file handles, etc.). I reckon it would require more changes to compiler though. Also I'd like to get an idea how this functionality is to play with existing alignment directives for records. -- Regards, Denis Golovan ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Inside the type I have: TAlignedArray = record public type TReference = ^T; TValue = T; ... procedure Push(const Item: TValue); function Pop: TValue; property Length: Integer read FLength write SetLength; property Reference[Index: Integer]: TReference read GetReference; default; property Item[Index: Integer]: TValue read GetItem write SetItem; ... end; So the default property indexer returns references and not values. If you want values specifically you can use the Item property indexer. This way we can either access fields directly on the items withing the array like so: A[0].X := SomeValue; Or if you need fast access to quickly populate the values, support we want to write 3 component vertex data, you could write: type TVertexBuffer = type TAlignedArray; var Buffer: TVertexBuffer; V: TVertexBuffer.TReference; begin Buffer.Length := VertexCount(ModelStream); V := Buffer[0]; for I := 1 to Buffer.Length do begin V.X := ReadX(ModelStream); V.Y := ReadY(ModelStream); V.Z := ReadZ(ModelStream); Inc(V); end; ... end; Which probably ends up being faster than continually accessing array dynamic values by index. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 29, 2019, at 9:15 PM, Anthony Walter wrote: > > Ryan, actually ... > > A.Length := 1; > A[0].X := 100; > > ... does work in my implementation. Really, how? [] property uses a getter which copies the array right? I’mm not aware of how to do this in Pascal without pointers. Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Ryan, actually ... A.Length := 1; A[0].X := 100; ... does work in my implementation. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 29, 2019, at 5:25 PM, Sven Barth via fpc-devel
> wrote:
>
> Ignoring something is not an answer. At least there'd need to be a runtime
> error.
>
> Thinking about it maybe it would be better to follow Anthony's idea with the
> record and not try to adjust/extend a language mechanism that wasn't geared
> for that.
Biggest problems with records is that we can’t make arrays of records because
the [] property copies the record with the getter (my other email on this never
got posted). If there was a feasible way to fix that problem then dynamic
arrays would be nearly obsolete anyways (probably still faster though).
var
A: TAlignedArray;
begin
A.Length := 1;
A[0].x := 100; // Doesn’t work, writes to a copy and never mutates the
underlying structure.
Is there no other way to specify alignment? It would be ugly but a compiler
directive could be used.
var
{$push}
{$dynamic-array-align 4096}
a: array of integer;
{$pop}
begin
SetLength(a,10); // just use normal SetLength now
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Op 3/28/2019 om 11:40 PM schreef Anthony Walter: Here is a brief follow up. I am working on other projects at the moment, but I am confident this a simple solution. Please tell me if this somehow will not fit you needs. My needs are simple: - auto clean like dynamic arrays. - preferably transparent with all existing mechanisms. - performance is important so no getters and setters (the units are pixels, and getters and setters in such cases slow down too much). - I mostly align for SSE/AVX not DMA, so my alignment is typically 64byte, I'd rather not use special functions that don't allocate via the pascal suballocator, that would only increase fragmentation However currently I have stowed the alignment requirement away in a few classes which are manually specialized versions of a generic one for different pixel size derive/specialize (*), so while I'm an alignment user, I'm not the best motivation to pull it in language. (*) 8-bit grayscale and 24-bit RGB have manually specialized classes, the rest uses an generic, which optimizes not as well in Delphi, but good enough for all but the most extreme speed. The 24-bit RGB case is also due to bugs in older (<=XE6) Delphis that don't deal well with pointer math of 3-byte types. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Am 29.03.2019 um 17:53 schrieb Ryan Joseph: First minor snag, fpc_dynarray_insert, fpc_dynarray_concat can allocate new arrays like SetLength can. Do we need to make aligned variants for these also? Using array + operators there is no possibility to set alignment. var a, b: array of integer; begin a += [1]; // no way to set alignment for a insert(1, b, 0); // no way to set alignment for b end. Some options: 1) ignore the problem 2) make extra syntax like: a: array of integer; alignment = 64; 3) make another function to allocate an empty array that sets alignment SetAlignment(a, 64); An empty array is Nil, changing that would lead to a whole different can of worms. 4) for insert/concat make variants like InsertAligned() and ignore + operators for aligned arrays Ignoring something is not an answer. At least there'd need to be a runtime error. Thinking about it maybe it would be better to follow Anthony's idea with the record and not try to adjust/extend a language mechanism that wasn't geared for that. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays / Maybe implement in mem-mgr?
> On Mar 29, 2019, at 1:21 PM, Martin Frb wrote: > > Question: Should the alignment be "user data" or "type info". > Does it need to be determined at runtime, and set to different values (for > the same type/variable) at runtime? I’m just doing what Sven said I could do, which is store the info in the dynamic array header and add a “SetLengthAligned” intrinsic. Maybe the new problem changes things though. As for the memory manager I’m not sure that helps because of how the dynamic arrays are already implemented. It would have to be extended to return an aligned block that had x number of bytes directly before it. No idea if that’s feasible for memory managers. Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays / Maybe implement in mem-mgr?
On 29/03/2019 17:53, Ryan Joseph wrote: First minor snag, fpc_dynarray_insert, fpc_dynarray_concat can allocate new arrays like SetLength can. Do we need to make aligned variants for these also? Using array + operators there is no possibility to set alignment. 1) ignore the problem 2) make extra syntax like: a: array of integer; alignment = 64; 3) make another function to allocate an empty array that sets alignment SetAlignment(a, 64); 4) for insert/concat make variants like InsertAligned() and ignore + operators for aligned arrays Question: Should the alignment be "user data" or "type info". Does it need to be determined at runtime, and set to different values (for the same type/variable) at runtime? If not, i.e. if a specific variable always needs 4k alignment, would it not be more logical to make it part of the type? type TFoo = array [x..y] of byte aligned 4096; Of course that does not change the above problem. The (current or fixed) alignment needs to be stored, so concat can retrieve it and honour it. Stored either in the header or the RTTI type info Question 2: Would it not be more efficient, if the alignment would happen in the memory manager? Then no over-allocation would be needed. And the memory that is skipped in front of the array could be used for other data. The memory manager would need 2 new methods: AllocMemAtAllignment FreeMemWithAllignment So the array still needs to know, it was allocated with Allignment and call the correct FreeMemWithAllignment. As a default for all memory manager AllocMemAtAllignment could do the over-allocate, and return the correct pointer, with the mem-managers align header. Individual MemManagers can then be optimized. pseudo code for AllocMemAtAllignment mem := alloc(size+align+sizeof(MemMgrAlignHeader)) result := getNextAlignedAddr(mem+sizeof(MemMgrAlignHeader)) (result-sizeof(MemMgrAlignHeader)) ^.RealAllocAddr := mem; The result looks then like mem // padding result - header: RealAllocAddr result // At least size bytes So pretty much what would have happened in the array code. But moved to the mem-manager. So it can be: a) re-used for other types or even called by usercode b) a mem-manager that can get the aligned address without padding, can be optimized, and return a normal block of memory (does not even need the MemMgrAlignHeader) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
First minor snag, fpc_dynarray_insert, fpc_dynarray_concat can allocate new arrays like SetLength can. Do we need to make aligned variants for these also? Using array + operators there is no possibility to set alignment. var a, b: array of integer; begin a += [1]; // no way to set alignment for a insert(1, b, 0); // no way to set alignment for b end. Some options: 1) ignore the problem 2) make extra syntax like: a: array of integer; alignment = 64; 3) make another function to allocate an empty array that sets alignment SetAlignment(a, 64); 4) for insert/concat make variants like InsertAligned() and ignore + operators for aligned arrays Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Gareth,
There is a bounty for this? I wonder if my solution is acceptable.
Also, my solution, or any for that matter, could be enhanced to make the
alignment size configurable per each array aligned array. That is, memory
alignment starting position and page size could be defined by each array.
This would allow for more flexibility and would be trivial enough to adapt
to what I've provided.
Finally, these methods could be added to the type, making it like a super
array or list:
{ Returns the lower bounds of the list }
function Lo: Integer;
{ Returns the upper bounds of the list }
function Hi: Integer;
{ Reverses the items in the list }
procedure Reverse;
{ Swap two items by index in the list }
procedure Exchange(A, B: Integer);
{ Adds and item to the end of the list }
procedure PushRange(const Collection: array of T);
{ Remove an item randomly from the list }
function PopRandom: T;
{ Return a copy of the list with items passing through a filter }
function Filter(Func: TFilterFunc): TArrayList;
{ Return the first item matching a condition }
function FirstOf(Func: TFilterFunc): T;
{ Removes an item by index from the list and decreases the count by one
}
procedure Delete(Index: Integer);
{ Sort the items using a comparer }
procedure Sort(Order: TSortingOrder = soAscend; Comparer: TCompare =
nil);
{ Attempt to find the item using DefaultCompare }
function IndexOf(const Item: T): Integer;
{ Join a the array into a string using a separator }
function Join(const Separator: string; Convert: TConvertString =
nil): string;
{ Returns true if ther are no items in the list }
property IsEmpty: Boolean read GetIsEmpty;
{ First item in the list }
property First: T read GetFirst write SetFirst;
{ Last item in the list }
property Last: T read GetLast write SetLast;
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
I wonder if there's any incentive to have such a feature in-built in the compiler. The bounty on #34031 has probably expired by now. If not, then your unit would be a great addition. Page-aligned memory is probably something that warrants further discussion with the core team. Gareth aka. Kit On Fri 29/03/19 05:01 , "Anthony Walter" sys...@gmail.com sent: Here is the full implementation of aligned arrays, let me know if it works for you. https://www.getlazarus.org/arrays/ [1] Here is another example: uses AlignArrays; procedure Test;var VertexData: TAlignedArray; V: PVec3; I: Integer;begin VertexData.Length := 100; // page sized and aligned memory allocated here V := VertexData[0]; // get a reference to the first item for I := 0 to V.Length - 1 do begin V^ := ComputeVertex(I); // write to vertex data faster via a pointer Inc(V); end; for V in VertexData do // array like enumerators are supported PrintVertex(V); V.Item[6] := ComputeVertex(Random(100)); // you can write directly using this syntax V[7]^ := ComputeVertex(Random(100)); // or write by dereferencing the default indexer end; // there is no need to free the memory, it is taken care of for you by finalize Links: -- [1] https://www.getlazarus.org/arrays/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Here is the full implementation of aligned arrays, let me know if it works for you. https://www.getlazarus.org/arrays/ Here is another example: uses AlignArrays; procedure Test; var VertexData: TAlignedArray; V: PVec3; I: Integer; begin VertexData.Length := 100; // page sized and aligned memory allocated here V := VertexData[0]; // get a reference to the first item for I := 0 to V.Length - 1 do begin V^ := ComputeVertex(I); // write to vertex data faster via a pointer Inc(V); end; for V in VertexData do // array like enumerators are supported PrintVertex(V); V.Item[6] := ComputeVertex(Random(100)); // you can write directly using this syntax V[7]^ := ComputeVertex(Random(100)); // or write by dereferencing the default indexer end; // there is no need to free the memory, it is taken care of for you by finalize ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Just to add a minor thing... while usually you don't have to align memory any more coarsely than 16 bytes for XMM, 32 bytes for YMM and 64 bytes for ZMM, the 4 kB granularity is useful for paging, especially if you're reading or writing to a swap file. An entire library of functions for such memory management and alignment would definitely be beneficial. Using FastMM4 if at all possible is even better. And that record definition is pretty nice, I have to say! Gareth aka. Kit On Thu 28/03/19 22:40 , Anthony Walter sys...@gmail.com sent: Here is a brief follow up. I am working on other projects at the moment, but I am confident this a simple solution. Please tell me if this somehow will not fit you needs. Create a new type that is compatible with Array where Array = type array of T.Define implcit conversions between the to types and similar methodsCreate a Push and Pop methods, well as a read write Length property and two indexers. Indexer named Reference is default and provides a memory reference to TnIndexer named Item is available and provides a copy of Tn So we would have ... TAlignedArray = record type TReference = ^T; type TValue = T; FPage: Pointer; FLength: Integer; procedure SetLength(Value: Integer); function GetReference(Index: Integer): TReference; function GetItem(Index: Integer): TValue; procedure SetItem(Index: Integer; const Value: TValue); public procedure Push(const Item: TValue); procedure Pop: TValue; property Length: Integer read FLength write SetLength; property Reference[Index: Integer]: TValue read GetReference; default; property Item[Index: Integer]: TValue read GetValue write SetValue; Examples usages: type TVertexArray = TAlignedArray; // and later var Vertices: TVertexArray;begin Vertices.Length = 1000; // okay, request a page aligned memory Vertices[0].X := X; // okay, we are working with references Vertices.Item[1] := Vec3(X, Y, Z); // okay, we are setting a value Vertices.Length = 0; // clean up When the vertex array need to grow, it uses the POSIX posix_memalign (or the _aligned_malloc on Windows) to request a 4 KB aligned and sized page adequate to contain all the items. The newish allocate and release mechanisms can further be used to simply the management of the TAlignedArray type. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org [1] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel [2]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel Links: -- [1] mailto:fpc-devel@lists.freepascal.org [2] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Here is a brief follow up. I am working on other projects at the moment, but I am confident this a simple solution. Please tell me if this somehow will not fit you needs. Create a new type that is compatible with Array where Array = type array of T. Define implcit conversions between the to types and similar methods Create a Push and Pop methods, well as a read write Length property and two indexers. Indexer named Reference is default and provides a memory reference to Tn Indexer named Item is available and provides a copy of Tn So we would have ... TAlignedArray = record type TReference = ^T; type TValue = T; FPage: Pointer; FLength: Integer; procedure SetLength(Value: Integer); function GetReference(Index: Integer): TReference; function GetItem(Index: Integer): TValue; procedure SetItem(Index: Integer; const Value: TValue); public procedure Push(const Item: TValue); procedure Pop: TValue; property Length: Integer read FLength write SetLength; property Reference[Index: Integer]: TValue read GetReference; default; property Item[Index: Integer]: TValue read GetValue write SetValue; Examples usages: type TVertexArray = TAlignedArray; // and later var Vertices: TVertexArray; begin Vertices.Length = 1000; // okay, request a page aligned memory Vertices[0].X := X; // okay, we are working with references Vertices.Item[1] := Vec3(X, Y, Z); // okay, we are setting a value Vertices.Length = 0; // clean up When the vertex array need to grow, it uses the POSIX posix_memalign (or the _aligned_malloc on Windows) to request a 4 KB aligned and sized page adequate to contain all the items. The newish allocate and release mechanisms can further be used to simply the management of the TAlignedArray type. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Am 28.03.2019 um 15:17 schrieb Ryan Joseph:
Also I don't think that a simple "do alignment or not" will be sufficient. What
kind of alignment do you pick? What if the user needs more alignment? So it would
probably be best to add a new SetLengthAligned of which the second argument is the
alignment followed by the lengths.
That’s correct, there’s an alignment parameter also, not just a boolean. The
header may need to include the desired alignment also for copy operations but
I’m not sure yet.
Since the offset is going to be part of the header now (and possibly the
desired alignment) does it matter that this will affect all dynamic arrays?
It’s probably only 4 bytes but still worth considering.
I'm not a fan of introducing a separate header for this. And abusing
some bits of the existing fields as flags just means that other parts of
dynarr.inc have to be touched even though we avoided that by ordering it
as [padding][header][data].
In dynarr.inc you then add a new setlength variant that takes an additional
alignment parameter while the old one without alignment calls the new one with
alignment 1.
Note: the compiler can call the new one for both variants, but the old one is
required for bootstrapping, so you could surround that with {$if not
defined(ver3_0) and not defined(ver3_2)}.
You mean if the compiler is < 3.0 then don’t parse the new SetLength variant?
No, I mean it like this:
=== code begin ===
procedure int_dynarray_setlength_aligned(var p : pointer;pti : pointer;
dimcount : sizeint;dims : pdynarrayindex;alignment :
longint);[external name 'FPC_DYNARR_SETLENGTH_ALIGNED'];
{$if defined(ver3_0) or defined(ver3_2)}
procedure fpc_dynarray_setlength(var p : pointer;pti : pointer;
dimcount : sizeint;dims :
pdynarrayindex);[Public,Alias:'FPC_DYNARR_SETLENGTH']; compilerproc;
begin
int_dynarray_setlength_aligned(p,pti,dimcount,dims,1);
end;
{$endif}
=== code end ===
If the compiler always calls the aligned variant (on call less) then the
one without alignment is only required for bootstrapping with 3.0.4 or 3.2.
Regards,
Sven
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Am 28.03.2019 um 18:41 schrieb Ryan Joseph: Now I’m using “cd /rtl; make all FPC=/path/to/compiler” to build the RTL but this is obviously slow and unnecessary. Is there a quicker way to build just the unit which contains dynarr.inc and have all the objects files to be put in the correct location? I don't see where this is slow... Even on my old PowerBook that time doesn't hinder me that much. That said, you could try "make ../units/-/system.ppu" in the RTL directory of your target OS (e.g. for Linux in the rtl/linux directory and for Mac OS X in the rtl/darwin directory). Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Ryan, this is just a thought. Do you need the flat memory address to start on an even 4 KB boundary or are you okay with the memory address starting at any DWORD address and spanning an even 4 KB after that address? In the latter case I have an easy solution for you. If the former I am unsure how to allocate memory at a specific address but could create a simple enough solution which may waste some space at the start yet give the array a starting address of a number divisible evenly by 4 KB. Let me know and I'll write an implementation which I am pretty sure you'll find either solution more than adequate. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On Thu, 28 Mar 2019, 20:34 Ryan Joseph, wrote: > Now I’m using “cd /rtl; make all FPC=/path/to/compiler” to build the RTL > but this is obviously slow and unnecessary. Is there a quicker way to build > just the unit which contains dynarr.inc and have all the objects files to > be put in the correct location? > Dynarr.inc gets pulled into system.pp via system.inc. So the whole RTL should be rebuilt anyway. Except of course if you only want to test your changes without requiring anything else from the RTL. Then you can look at the compiler command generated by make, system is the first unit to get compiled, and copy that. There may of course be other ways too. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Now I’m using “cd /rtl; make all FPC=/path/to/compiler” to build the RTL but this is obviously slow and unnecessary. Is there a quicker way to build just the unit which contains dynarr.inc and have all the objects files to be put in the correct location? Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 28, 2019, at 4:17 AM, Sven Barth via fpc-devel
> wrote:
>
> The order should be
>
> [padding] [header] [data]
>
> with the offset being part of the header. This way you reduce the changes
> needed to only those that allocate/free an array.
Yeah that’s probably the better idea because it’s less intrusive.
> Also I don't think that a simple "do alignment or not" will be sufficient.
> What kind of alignment do you pick? What if the user needs more alignment? So
> it would probably be best to add a new SetLengthAligned of which the second
> argument is the alignment followed by the lengths.
That’s correct, there’s an alignment parameter also, not just a boolean. The
header may need to include the desired alignment also for copy operations but
I’m not sure yet.
Since the offset is going to be part of the header now (and possibly the
desired alignment) does it matter that this will affect all dynamic arrays?
It’s probably only 4 bytes but still worth considering.
>
> In dynarr.inc you then add a new setlength variant that takes an additional
> alignment parameter while the old one without alignment calls the new one
> with alignment 1.
>
> Note: the compiler can call the new one for both variants, but the old one is
> required for bootstrapping, so you could surround that with {$if not
> defined(ver3_0) and not defined(ver3_2)}.
You mean if the compiler is < 3.0 then don’t parse the new SetLength variant?
Regards,
Ryan Joseph
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
Ryan Joseph schrieb am Do., 28. März 2019,
03:01:
> Here’s the basic structure:
>
> [header][padding][offset][head]
>
> [head] being the start of the elements (the pointer to the first element
> of the array) and aligned to the requested amount. Keep in mind the head is
> the pointer which is passed around so we need to subtract back from that in
> order to access the header, which contains the array high value and the
> refcount.
>
The order should be
[padding] [header] [data]
with the offset being part of the header. This way you reduce the changes
needed to only those that allocate/free an array.
> Look at the dynarr.inc in the /rtl directory to see what I’m talking
> about. Pretty hard to explain without seeing how Sven did things. :)
>
Most of that isn't my code. I only added Insert, Delete and Concat.
> >
> > The comment:
> > As for extending the array header, maybe introduce a new data type
> "aligned array".
> > So normal arrays do not have that field in there header.
>
> The plan was to make a “SetLengthAligned” or add an extra parameter to
> “SetLength”, i.e, SetLength(arr,100,true). I have no preference, what ever
> the compiler team wants.
>
Extending SetLength is not a good idea as that can take multiple lengths
for multi dimensional arrays.
Also I don't think that a simple "do alignment or not" will be sufficient.
What kind of alignment do you pick? What if the user needs more alignment?
So it would probably be best to add a new SetLengthAligned of which the
second argument is the alignment followed by the lengths.
In dynarr.inc you then add a new setlength variant that takes an additional
alignment parameter while the old one without alignment calls the new one
with alignment 1.
Note: the compiler can call the new one for both variants, but the old one
is required for bootstrapping, so you could surround that with {$if not
defined(ver3_0) and not defined(ver3_2)}.
Regards,
Sven
>
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
> On Mar 27, 2019, at 9:13 PM, Martin Frb wrote: > > 2 Ideas, well 1 Idea, 1comment... > > The Idea: > Would it not be easier to: > - allocate mem, with extra space for align, unless memmanager can handle > align. > - return the alligned pointer for the array, so all array ops will work > exactly as they currently do > - prefix/extend the array header by a field "OffsetToMemAlloc" (that is the > reverse, of what I understand your padding would do, but still can be a > positive number, the code knows to subtract it where needed) > > This field would allow to correctly get the memory for re-alloc/free > operations. I think we’re basically talking about the same thing but it’s hard to know without seeing code. The biggest challenge is that in order to get the array header you need to subtract from the “head" (the start of the elements). Right now it’s easy because you just subtract the size of the header but if the head is now aligned and 100’s of bytes away from the start of the header then you need to subtract that amount also. In fact technically speaking the offset field is adjacent to the start of elements and could be many bytes away from the actual header. Here’s the basic structure: [header][padding][offset][head] [head] being the start of the elements (the pointer to the first element of the array) and aligned to the requested amount. Keep in mind the head is the pointer which is passed around so we need to subtract back from that in order to access the header, which contains the array high value and the refcount. Look at the dynarr.inc in the /rtl directory to see what I’m talking about. Pretty hard to explain without seeing how Sven did things. :) > > The comment: > As for extending the array header, maybe introduce a new data type "aligned > array". > So normal arrays do not have that field in there header. The plan was to make a “SetLengthAligned” or add an extra parameter to “SetLength”, i.e, SetLength(arr,100,true). I have no preference, what ever the compiler team wants. Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Aligned dynamic arrays
On 28/03/2019 00:43, Ryan Joseph wrote: I would like to attempt to implement this feature request (https://bugs.freepascal.org/view.php?id=34031 for my boss) which has the SetLength intrinsic return aligned memory. We need this for the Metal framework which requires all vertex buffer memory to aligned on 4k boundaries so we can’t use dynamic arrays in their current form (see the bug report for example code). I’ve looked at the dynamic array implementation (dynarr.inc) and it appears that I could over allocate and extend the header (tdynarray) to include an extra field which stores the amount of padding until the actual array elements start (which is aligned). It would complicit the code to some extent however because the current design relies on lots of pointer math. Is this ok that I add this? Marco said in the comments there was a need so I assume this is a go but I wanted to ask first. 2 Ideas, well 1 Idea, 1comment... The Idea: Would it not be easier to: - allocate mem, with extra space for align, unless memmanager can handle align. - return the alligned pointer for the array, so all array ops will work exactly as they currently do - prefix/extend the array header by a field "OffsetToMemAlloc" (that is the reverse, of what I understand your padding would do, but still can be a positive number, the code knows to subtract it where needed) This field would allow to correctly get the memory for re-alloc/free operations. The comment: As for extending the array header, maybe introduce a new data type "aligned array". So normal arrays do not have that field in there header. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] Aligned dynamic arrays
I would like to attempt to implement this feature request (https://bugs.freepascal.org/view.php?id=34031 for my boss) which has the SetLength intrinsic return aligned memory. We need this for the Metal framework which requires all vertex buffer memory to aligned on 4k boundaries so we can’t use dynamic arrays in their current form (see the bug report for example code). I’ve looked at the dynamic array implementation (dynarr.inc) and it appears that I could over allocate and extend the header (tdynarray) to include an extra field which stores the amount of padding until the actual array elements start (which is aligned). It would complicit the code to some extent however because the current design relies on lots of pointer math. Is this ok that I add this? Marco said in the comments there was a need so I assume this is a go but I wanted to ask first. Regards, Ryan Joseph ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
