Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 30, 2019, at 5:02 PM, Jonas Maebe  wrote:
> 
> How is this done in the Metal C headers?

Looking this now it appears the padding may be put in the actual vector. Maybe 
those macros put it in there? The fields of the struct need to be aligned on 16 
bytes (4 floats) so. Does c++ even allow padding struct fields or is this kind 
of fringe usage? I thought the compiler already did this so it could just add 
more?

typedef struct
{
// Positions in pixel space
// (e.g. a value of 100 indicates 100 pixels from the center)
vector_float2 position;

// Floating-point RGBA colors
vector_float4 color;
} AAPLVertex;

typedef simd_float2 vector_float2;

/*! @abstract A vector of two 32-bit floating-point numbers.
 *  @description In C++ and Metal, this type is also available as
 *  simd::float2. The alignment of this type is greater than the alignment
 *  of float; if you need to operate on data buffers that may not be
 *  suitably aligned, you should access them using simd_packed_float2
 *  instead.  */
typedef __attribute__((__ext_vector_type__(2))) float simd_float2;

/*! @abstract A vector of four 32-bit floating-point numbers.
 *  @description In C++ and Metal, this type is also available as
 *  simd::float4. The alignment of this type is greater than the alignment
 *  of float; if you need to operate on data buffers that may not be
 *  suitably aligned, you should access them using simd_packed_float4
 *  instead.  */
typedef __attribute__((__ext_vector_type__(4))) float simd_float4;

> 
>> If you saw the original bug report we’re trying to make a custom alignment 
>> (of 4k) for the Metal framework. I propose the syntax because it solves the 
>> problem of how to assign alignment and there is a precedent for it (i.e. 
>> $align).
> 
> No, I missed that. In that case, a vector type won't help. The best way then 
> is, as has been mentioned elsewhere in this thread, a getmem variant that 
> supports allocating memory on an aligned boundary. Similar to how unix 
> platforms have posix_memalign.
> 
> This falls outside the scope of direct support in the language, so a solution 
> like Anthony's is better in this case if you want to encapsulate it 
> completely.

I already started on it before I hit the snag and it’s not an unreasonably 
intrusive thing to allocate the extra padding like I discussed earlier, but it 
would probably require introducing a compiler directive (see $dynarrayalign 
below) and adding alignment information to tarraydef. I’ll post what I finished 
on GitHub if anyone wants to see how invasive the changes are.

There’s a good argument both ways and I guess it comes down to maintaining the 
extra complexity in the compiler. I’m personally happy to use the record since 
it’s feasible in 3.1.1 but I understand users who like the compiler syntax, 
which is cleaner and probably faster.

So what should I do going forward? Should I abandon the idea or do any of the 
other core team members want to me pursue this further?

{$push}
{$dynarrayalign 4096}
type
TIntArray = array of integer;
{$pop}


Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Anthony Walter
Oh and might as well add dot product and cross product for vectors as well

>
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Anthony Walter
On a related note, since a lot of the discussion here seems related to
vector types, I seem to recall someone recently published an mmx,
sse1/2/3/4, simd set of routines related to vector operations. I don't use
much asm other than just being familiar with the basics of ia32 from long
ago, but I do know there have been many enhancements for the instruction
sets I mentioned which specifically greatly increase both the speed and
parallelism of vector operation, including their combination with matrix
math manipulation and vector transforms. I also know that the people in the
mono project enhanced their compiler with special simd vector optimizations:

https://www.mono-project.com/news/2016/12/20/system-numeric-vectors/

Having said that, might it be possible if we could incorporate official
vector and matrix type and optimizations with the compiler? I think
everyone would benefit from officially supported types of this nature that
could have more eyes on them to ensure their operations are best optimized
for the various types of computers and architectures free pascal supports.
I can think of quite a few operations that would benefit from everyone
contributing either their own ideas of the best optimizations or reporting
problems or suggestions.

Here is what I would suggest as a start. Use 4 component float (single and
perhaps additionally double precision) vectors (X, Y, Z, W), 4x4 matrix
type to match.

For vectors we need these operations:
add, subtract, multiply, divide, normalize, transpose

For matrix types we need:
identity, scale, rotate, translate, sheer, multiply (matrix * matrix),
transform (vector * matrix), transpose, invert

That's just a start, but these operations ought to have official simd
support IMO

Thanks
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Jonas Maebe

On 30/03/2019 14:59, Ryan Joseph wrote:




On Mar 30, 2019, at 9:10 AM, Jonas Maebe  wrote:

FPC always aligns data to its alignment as specified by the platform ABI. 
{$align x} can be used to limit this alignment to a lower number. It cannot be 
used to increase it.


This caused us quite a bit of problems with the Metal framework because we 
needed padding fields also.


How is this done in the Metal C headers?


If you saw the original bug report we’re trying to make a custom alignment (of 
4k) for the Metal framework. I propose the syntax because it solves the problem 
of how to assign alignment and there is a precedent for it (i.e. $align).


No, I missed that. In that case, a vector type won't help. The best way 
then is, as has been mentioned elsewhere in this thread, a getmem 
variant that supports allocating memory on an aligned boundary. Similar 
to how unix platforms have posix_memalign.


This falls outside the scope of direct support in the language, so a 
solution like Anthony's is better in this case if you want to 
encapsulate it completely.



Can you give some examples of the vector type? I don’t exactly know what you 
guys are referring to.


A vector type is basically a (fairly small) array of integer or floating 
point types. All current desktop (and many embedded) CPUs have special 
execution units that operate on all elements of such vectors in 
parallel, which means you can get better performance.



Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread J. Gareth Moreton
 I always get nervous of mode switches because it's hard to remember them
or realise they exist, and it feels a little hacky, like it's a kind of
compatibility setting more than anything else.  Like at one point, I
preferred there was an explicit way to set a type's alignment so there's no
ambiguity or risk of bugs if a different setting is used or otherwise
overridden.  e.g.

 type  TVector4f = packed record
     x, y, z, w: Single;
   end align 16;

 Just my thought.

 Kit

 On Sat 30/03/19 17:31 , Mattias Gaertner via fpc-devel
fpc-devel@lists.freepascal.org sent:
 On Sat, 30 Mar 2019 12:57:48 -0400 
 Ryan Joseph  wrote: 

 > > On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel 
 > >  wrote: 
 > > 
 > > I guess you mean auto dereferencing. 
 > > {$ModeSwitch AutoDeref} 
 > 
 > Yeah I just found this by looking around in the compiler. Mind. 
 > Blown. No idea that existed! 

 And some people still ask for more modeswitches... 

 Mattias 
 ___ 
 fpc-devel maillist - fpc-devel@lists.freepascal.org [3] 
 http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[4]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel 

 

Links:
--
[1] mailto:r...@thealchemistguild.com
[2] mailto:fpc-devel@lists.freepascal.org
[3] mailto:fpc-devel@lists.freepascal.org
[4] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Mattias Gaertner via fpc-devel
On Sat, 30 Mar 2019 12:57:48 -0400
Ryan Joseph  wrote:

> > On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel
> >  wrote:
> > 
> > I guess you mean auto dereferencing.
> > {$ModeSwitch AutoDeref}  
> 
> Yeah I just found this by looking around in the compiler. Mind.
> Blown. No idea that existed!

And some people still ask for more modeswitches...

Mattias
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 30, 2019, at 12:53 PM, Mattias Gaertner via fpc-devel 
>  wrote:
> 
> I guess you mean auto dereferencing.
> {$ModeSwitch AutoDeref}

Yeah I just found this by looking around in the compiler. Mind. Blown. No idea 
that existed!

Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Mattias Gaertner via fpc-devel
On Sat, 30 Mar 2019 10:03:12 -0400
Ryan Joseph  wrote:

> > On Mar 30, 2019, at 9:55 AM, Jonas Maebe 
> > wrote: 
> >> You are not required to dereference pointers to write to them.
> >> var
> >>   P: PPoint;
> >> begin
> >>   P := AlignedArray[0];
> >>   P.X := 3; // can be okay
> >>   AlignedArray[0].Y := 4;  // can be okay as well  
> > 
> > That only works in {$mode delphi}  
> 
> Oh that’s why! How does this work in Delphi mode without pointers and
> why wasn’t it added to ObjFPC mode? 

I guess you mean auto dereferencing.
{$ModeSwitch AutoDeref}

> That would have saved me some
> headache in the past if I knew this.

Mattias
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 30, 2019, at 9:55 AM, Jonas Maebe  wrote:
> 
>> You are not required to dereference pointers to write to them.
>> var
>>   P: PPoint;
>> begin
>>   P := AlignedArray[0];
>>   P.X := 3; // can be okay
>>   AlignedArray[0].Y := 4;  // can be okay as well
> 
> That only works in {$mode delphi}

Oh that’s why! How does this work in Delphi mode without pointers and why 
wasn’t it added to ObjFPC mode? That would have saved me some headache in the 
past if I knew this.

Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread denisgolovan


> 1) In you example you are writing this with normal arrays:
> 
> A[0].X := 0;
> 
> And in my implementation it's exactly the same. Regarding writing the entire 
> type directly
> 
> A[0]^ := V;
> 
> If using the caret symbol ^ or an Item property is a deal breaker, then this 
> can be simplified using a custom smart pointer type similar to how the 
> aligned array type was implemented resulting in the same syntax as in the 
> first example.

Yes, returning temporary smart pointer would do the trick. -1 syntax 
incompatibility then.
I see this as a definite improvement.

> 2) As I've implemented them aligned array is compatible with dynamic arrays. 
> They implicitly convert between the two with no loss of data. If you truly 
> want to share the data within each, then you can define functions to work on 
> referencing the data types they hold, allowing the same function to serve 
> either type.

Well. Converting arrays is too slow to be usable, imho. Think about textures, 
vertex buffers, etc.

> 
> For example:
> 
> procedure VectorTransform(Matrix: TMat4; FirstVertex: PVec3; Count: Integer);
> 
> and then
> 
> Matrix.Rotate(12.5, 0, 0);
> VectorTransform(Matrix, AlignedData[0], AlignedData.Length);
> VectorTransform(Matrix, @DynamicArray[0], Length(AlignedData));
> 
> Or you can even overload VectorTransform if you don't want to see @ symbols.
> 
> procedure VectorTransform(Matrix: TMat4; FirstVertex: var TVec3; Count: 
> Integer); overload;

It's all right and will eventually work. 

But you miss much larger problem - that will require a lot of 
boilerplate/conversion/manual work.
Think about mixing 3 graphical libraries using different array/matrix/vertex 
types. Instead of re-using each others code, one would eventually write 4th 
library.
FPC community is too small to afford such useless combinatorics.

> 
> 3) Regarding Hi, Lo, Length, I prefer to put these functions on types. For 
> example
> 
> for I := A.Lo to A.Hi do A[I].X := I;
> 
> Is that such a deal breaker?

Yes. Generics / generic functions is a big deal.
That would need two different functions for each different _array_ type.

-- 
Regards,
Denis Golovan
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 30, 2019, at 9:10 AM, Jonas Maebe  wrote:
> 
> FPC always aligns data to its alignment as specified by the platform ABI. 
> {$align x} can be used to limit this alignment to a lower number. It cannot 
> be used to increase it.

This caused us quite a bit of problems with the Metal framework because we 
needed padding fields also. Is this something that could be added or would it 
be a disaster to add padding to record fields? It sounds like there is already 
a mechanism we could extend but I’m not sure.

> 
>> Why not use a similar directive for array alignment? That would solve the 
>> problem I discovered with insert/concat.
>> {$push}
>> {$align-array 4096}
>> type
>>   TIntArray = array of integer;
>> {$pop}
> 
> Arrays are aligned to the alignment requirements of their components. Again, 
> a vector type would solve this, since vectors have their own alignment 
> requirements in the ABIs.

If you saw the original bug report we’re trying to make a custom alignment (of 
4k) for the Metal framework. I propose the syntax because it solves the problem 
of how to assign alignment and there is a precedent for it (i.e. $align).

Can you give some examples of the vector type? I don’t exactly know what you 
guys are referring to.

Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Jonas Maebe

On 30/03/2019 14:42, Anthony Walter wrote:

You are not required to dereference pointers to write to them.

var
   P: PPoint;
begin
   P := AlignedArray[0];
   P.X := 3; // can be okay
   AlignedArray[0].Y := 4;  // can be okay as well


That only works in {$mode delphi}


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Anthony Walter
You are not required to dereference pointers to write to them.

var
  P: PPoint;
begin
  P := AlignedArray[0];
  P.X := 3; // can be okay
  AlignedArray[0].Y := 4;  // can be okay as well
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Anthony Walter
1) In you example you are writing this with normal arrays:

A[0].X := 0;

And in my implementation it's exactly the same. Regarding writing the
entire type directly

A[0]^ := V;

If using the caret symbol ^ or an Item property is a deal breaker, then
this can be simplified using a custom smart pointer type similar to how the
aligned array type was implemented resulting in the same syntax as in the
first example.

2) As I've implemented them aligned array is compatible with dynamic
arrays. They implicitly convert between the two with no loss of data. If
you truly want to share the data within each, then you can define functions
to work on referencing the data types they hold, allowing the same function
to serve either type.

For example:

procedure VectorTransform(Matrix: TMat4; FirstVertex: PVec3; Count:
Integer);

and then

Matrix.Rotate(12.5, 0, 0);
VectorTransform(Matrix, AlignedData[0], AlignedData.Length);
VectorTransform(Matrix, @DynamicArray[0], Length(AlignedData));

Or you can even overload VectorTransform if you don't want to see @
symbols.

procedure VectorTransform(Matrix: TMat4; FirstVertex: var TVec3; Count:
Integer); overload;

3) Regarding Hi, Lo, Length, I prefer to put these functions on types. For
example

for I := A.Lo to A.Hi do A[I].X := I;

Is that such a deal breaker?
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Jonas Maebe

On 30/03/2019 14:07, Ryan Joseph wrote:


I never got $align to work for records.


FPC always aligns data to its alignment as specified by the platform 
ABI. {$align x} can be used to limit this alignment to a lower number. 
It cannot be used to increase it.



Why not use a similar directive for array alignment? That would solve the 
problem I discovered with insert/concat.

{$push}
{$align-array 4096}
type
   TIntArray = array of integer;
{$pop}


Arrays are aligned to the alignment requirements of their components. 
Again, a vector type would solve this, since vectors have their own 
alignment requirements in the ABIs.



Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 30, 2019, at 12:56 AM, Anthony Walter  wrote:
> 
> So the default property indexer returns references and not values. If you 
> want values specifically you can use the Item property indexer.
> 
> This way we can either access fields directly on the items withing the array 
> like so:
> 
> A[0].X := SomeValue;
> 

Yes, but it’s a pointer so we need to dereference with ^.

A[0]^.X := 1;

That’s the one difference which we can’t resolve now. Basically we would need a 
“var” result introduced into the language. C++ has that feature so maybe it’s 
not crazy to implement in Pascal.

int& getter() {
  return i;
}

function getInt: integer; var;
begin
  result := i;
end;

Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread denisgolovan
Exactly, alignment might be well attached to array or array element type.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Ryan Joseph


> On Mar 29, 2019, at 5:30 PM, denisgolovan  wrote:
> 
> Also I'd like to get an idea how this functionality is to play with existing 
> alignment directives for records.

I never got $align to work for records. Isn’t that supposed to pad the fields 
to 16 bytes? Doesn’t work for me.

{$push}
{$align 16}
type
  TVertex = record
pos: TVec2;
col: TVec2;
  end;
{$pop}

Why not use a similar directive for array alignment? That would solve the 
problem I discovered with insert/concat.

{$push}
{$align-array 4096}
type
  TIntArray = array of integer;
{$pop}

Regards,
Ryan Joseph

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread denisgolovan
Hi Jonas

Vector type in FPC is too good to be true.
I didn't dare to dream about it so far :)

BR,
Denis
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread denisgolovan
Hi  Anthony

> The more that I think about it, the more I am believe that using my approach 
> is probably the best solution. Hear me out please.
> 
> First, it would be fairly easy to implement copy on write logic with a custom 
> record class.
> 
> Second, because it's Pascal code, it's entirely possible to fully customize 
> how it operates now and into the future, without the need to modify the 
> compiler. Suppose later you decide that also want the ability allocate the 
> storage memory of using the same array interface, but with a vertex buffer 
> object as the mechanism for allocating storage. Maybe at a later time you 
> want to allow an arrays type to back a memory mapped file.
> 
> In all cases, user code would be the proper avenue to implement these 
> features. Unless these is some special feature unique to an intrinsic type 
> that makes working with a user code type significantly more difficult, why 
> bother adding more complexity to the compiler and more complexity rigidly 
> into the built in types?
> 
> Now that we have management operators, type helpers, generics, 
> implicit/explicit conversions, and operator overloading, why not use what we 
> already have and make it better? Can you, or anyone for that matter, find a 
> real problem with a type or other construct that user code solution can solve 
> trivially with approximately the same syntax? If so then yes I am all for 
> addling to the language and compiler. In all other cases when can use what 
> the nice compiler and language developers have created for us and do everyone 
> a favor by reporting and helping to fix the bugs those features might still 
> have.

Yes. 
I am all for custom/user implementation in case it can be used everywhere 
together/interchangeably with native dynamic arrays. 
However FPC typing does not seem to cope with that (currently?).

1. e.g. primitive types (integers, floats) and records still need different 
handling in your TAlignedArray:

A[0]^:=0; 
A.Item[0]:=0;
vs
A[0].X:=0;

Standard arrays support A[0]:=0 and A[0].X:=0 via their compiler magic.

2. TAlignedArray memory layout can't be made compatible with dynamic arrays. 
Thus no existing functions accepting dynamic arrays would work with 
TAlignedArray and vice versa. 
And that's a huge deal as it just greatly increases fragmentation and messes 
with code re-use between libraries.
Imho, Pascal is too good to follow into C/C++ custom arrays hell.

3. Does not support standard way of dealing with array like @A[0], High(A), 
Low(A), Length(A), etc. 
Think about generic functions accepting both standard dynamic array and your 
custom array/vector.

Another minor issue is that your TAlignedArray is semantically more like Rust's 
Vec. 
It lacks public SetLength, but has Push/Pop/Clear. Thus it behaves differently 
to FPC array.

-- 
Regards,
Denis Golovan
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Jonas Maebe

On 29/03/2019 22:30, denisgolovan wrote:

Could we discuss more options here?


Since this is mostly about vectors, I think the best approach is to add 
an actual vector type to the compiler (Vector Pascal may serve as 
inspiration on how to integrate it in the language). Anything from 
dynamic arrays to records to variables could then automatically align 
them to whatever is required by the target platform.



Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


[fpc-devel] Low-cost encoding reduces PPU size by 10%

2019-03-30 Thread J. Gareth Moreton
 Hi everyone,

 Sorry if the title sounds like a clickbait you find everywhere on the
Internet, and I apologise again for using FPC as my kind of personal
research ground!
 I've just submitted a patch that uses a cheap, customised encoding system
to minimise the number of bytes required to store fields in PPU files that
are used to store indices, data lengths and offsets... integer values that
are frequently small but nonetheless have a large domain (e.g. LongInt or
QWord).  I've submitted a number of patches here -
https://bugs.freepascal.org/view.php?id=35299 - and the PPU file size is
about 10% smaller.  For large projects like Lazarus, this amounts to over
10 megabytes.
 I admit that storage space isn't a premium on most systems these days, but
for things like embedded systems, then every little helps.

 The encoding system I've named "Varlen" and the technical details can be
found here: http://wiki.freepascal.org/Varlen_Encoding [1]
 The basic idea is that values close to zero take fewer bytes to store than
values near the top of the QWord domain.  Values between 0 and 127, for
example, only require a single byte as opposed to 4 or 8, making it highly
suitable for offsets, counts and length fields, especially if they are
frequently equal to zero.  Conversely, Varlen is not suitable for things
like pointers and hashes, whose literal values can be very large.
 One thing not mentioned but pointed out by another developer is that,
unlike something like LEB128, there is no way for a Varlen to encode an
endless byte stream, so if a PPU file gets corrupted, there's no way it
will put the compiler into an infinite loop or a dangerous resource
starvation situation.

 What do you guys think?
 Gareth aka. Kit
  

Links:
--
[1] http://wiki.freepascal.org/Varlen_Encoding
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Aligned dynamic arrays

2019-03-30 Thread Anthony Walter
Denis,

The more that I think about it, the more I am believe that using my
approach is probably the best solution. Hear me out please.

First, it would be fairly easy to implement copy on write logic with a
custom record class.

Second, because it's Pascal code, it's entirely possible to fully customize
how it operates now and into the future, without the need to modify the
compiler. Suppose later you decide that also want the ability allocate the
storage memory of using the same array interface, but with a vertex buffer
object as the mechanism for allocating storage. Maybe at a later time you
want to allow an arrays type to back a memory mapped file.

In all cases, user code would be the proper avenue to implement these
features. Unless these is some special feature unique to an intrinsic type
that makes working with a user code type significantly more difficult, why
bother adding more complexity to the compiler and more complexity rigidly
into the built in types?

Now that we have management operators, type helpers, generics,
implicit/explicit conversions, and operator overloading, why not use what
we already have and make it better? Can you, or anyone for that matter,
find a real problem with a type or other construct that user code solution
can solve trivially with approximately the same syntax? If so then yes I am
all for addling to the language and compiler. In all other cases when can
use what the nice compiler and language developers have created for us and
do everyone a favor by reporting and helping to fix the bugs those features
might still have.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


[fpc-devel] Aligned dynamic arrays

2019-03-30 Thread denisgolovan
Hi all

Sorry for possible starting a new thread as I've just managed to subscribe to 
the list.

Could we discuss more options here?

Maybe separate aligned array is a better alternative? 

I mean a new type fully backward compatible with dynamic arrays, but with 
additional requirements (more specialized that is)?
Like some interface (similar to memory manager record) for reallocating, 
deallocating, copy-on-write handling, etc. 
Ideally some state would be allowed as well, thus interface type.
This way dynamic arrays might be more flexible. e.g. think of mmapped arrays on 
disk implementation support (requires extra state like file handles, etc.).
I reckon it would require more changes to compiler though.

Also I'd like to get an idea how this functionality is to play with existing 
alignment directives for records.

-- 
Regards,
Denis Golovan
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel