Re: [fpc-pascal] Creating capturers
Hairy Pixels schrieb am Sa., 4. Nov. 2023, 15:48: > > > > On Nov 4, 2023, at 4:22 PM, Sven Barth > wrote: > > > > Then don't assign them every "frame". If you just keep them around then > they aren't more expensive than a virtual method call. > > And any other mechanism would involve the heap as well, because that's > how anonymous functions that capture variables work and thus anything that > wants to store them must play by the same rules. There won't be any changes > there. > > > > The need for a universal function pointer type really isn't about the > function references even though they work for that, albeit with unneeded > overhead. > > Lets say you have a sort function which takes a compare callback > parameter. You can make the callback type "is nested" which works on global > and nested/anonymous functions but not methods. What if the user wants to > provide the compare function as a method because they're calling it in a > class which shares the code elsewhere? As the writer of the function you'd > need to make 2 versions of the function, one "is nested" and "of object". > > The writer of the sort function shouldn't need to know what kind of > function it will be passed, just that a comparison callback is provided, > right? > Then you simply make the callback function type a function reference and be done. The user can pass in any kind of function they want and *nearly* every type of function-like pointer. And if you're really worried about performance when providing such a class then simply provide overloads woth different function-like pointer types, the compiler will pick the most suitable one then. It's really not that hard as an implementer of a class. Regards, Sven > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
> On Nov 4, 2023, at 4:22 PM, Sven Barth wrote: > > Then don't assign them every "frame". If you just keep them around then they > aren't more expensive than a virtual method call. > And any other mechanism would involve the heap as well, because that's how > anonymous functions that capture variables work and thus anything that wants > to store them must play by the same rules. There won't be any changes there. > The need for a universal function pointer type really isn't about the function references even though they work for that, albeit with unneeded overhead. Lets say you have a sort function which takes a compare callback parameter. You can make the callback type "is nested" which works on global and nested/anonymous functions but not methods. What if the user wants to provide the compare function as a method because they're calling it in a class which shares the code elsewhere? As the writer of the function you'd need to make 2 versions of the function, one "is nested" and "of object". The writer of the sort function shouldn't need to know what kind of function it will be passed, just that a comparison callback is provided, right? Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
Hairy Pixels schrieb am Sa., 4. Nov. 2023, 01:42: > > > > On Nov 3, 2023, at 8:31 PM, Sven Barth > wrote: > > > > If you need a catch all type, then you should use function references. > There is no need to invent yet another type. > > The only thing it can not handle is the assignment of nested function > pointers (in contrast to nested functions directly), because they can only > ever be used in a function that is below the one where the nested function > was assigned which is not a guaranteed property for function references. > > > > not always because it creates a heap allocated interfaces for every > function they're used in, even if it doesn't escape the scope. Maybe this > was fine for Delphi but they can't be used for anything real time outside > of GUI apps. > Then don't assign them every "frame". If you just keep them around then they aren't more expensive than a virtual method call. And any other mechanism would involve the heap as well, because that's how anonymous functions that capture variables work and thus anything that wants to store them must play by the same rules. There won't be any changes there. Regards, Sven > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
> On Nov 3, 2023, at 8:31 PM, Sven Barth wrote: > > If you need a catch all type, then you should use function references. There > is no need to invent yet another type. > The only thing it can not handle is the assignment of nested function > pointers (in contrast to nested functions directly), because they can only > ever be used in a function that is below the one where the nested function > was assigned which is not a guaranteed property for function references. > not always because it creates a heap allocated interfaces for every function they're used in, even if it doesn't escape the scope. Maybe this was fine for Delphi but they can't be used for anything real time outside of GUI apps. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
Hairy Pixels schrieb am Fr., 3. Nov. 2023, 12:37: > > > > On Nov 3, 2023, at 2:08 PM, Sven Barth > wrote: > > > > By default the purpose of anonymous functions assigned to function > references *is* that they can escape their scope. This will not change, > because they were after all introduced for Delphi-compatibility and there > won't be any designations that the value can't escape. If the compiler > can't figure it out by its own (see above) then tough luck; use a feature > better suited for that. > > > > Ok I misinterpreted what you previously said then. > > I think Swift does this because they only have a single function pointer > type which serves all purposes while FPC has 4 distinct types. > > IMO FPC needs this also since it's not possible for libraries to know > which kind of function the caller may provide. Behind the scenes the > different types are required (global, nested, method, reference) but there > should be a way to unify these into single type. How to store the capturer > and manage its memory would be an open question though. > If you need a catch all type, then you should use function references. There is no need to invent yet another type. The only thing it can not handle is the assignment of nested function pointers (in contrast to nested functions directly), because they can only ever be used in a function that is below the one where the nested function was assigned which is not a guaranteed property for function references. Regards, Sven > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
> On Nov 3, 2023, at 2:08 PM, Sven Barth wrote: > > By default the purpose of anonymous functions assigned to function references > *is* that they can escape their scope. This will not change, because they > were after all introduced for Delphi-compatibility and there won't be any > designations that the value can't escape. If the compiler can't figure it out > by its own (see above) then tough luck; use a feature better suited for that. > Ok I misinterpreted what you previously said then. I think Swift does this because they only have a single function pointer type which serves all purposes while FPC has 4 distinct types. IMO FPC needs this also since it's not possible for libraries to know which kind of function the caller may provide. Behind the scenes the different types are required (global, nested, method, reference) but there should be a way to unify these into single type. How to store the capturer and manage its memory would be an open question though. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
Hairy Pixels schrieb am Fr., 3. Nov. 2023, 02:00: > > > > On Nov 2, 2023, at 1:44 PM, Sven Barth > wrote: > > > > Now for nested as well as anonymous routines the compiler determines > whether a capturer is required at the point that the nested or anonymous > routine is assigned to a function reference (cause otherwise it can be > handled as a nested function or even a function or method pointer). This > requirement is detected during the parsing of the routine. > > On second thought I had some questions about this part. From what I see > FPC always allocates the interface when reference pointers are used but > what you write suggests maybe the compiler will demote the type to > something more optimized (like a nested function). We talked about this at > length before but I don't think anything was decided yet. > > for example: > > procedure DoThis; > var > proc: reference to procedure; > begin > proc := procedure > begin > end; > proc(); > end; > > doesn't need the heap allocated instance because the function reference > never escapes the scope and it has no captured variables so it could be > demoted to a global function even of nested. > As soon as an anonymous function gets assigned to a function reference it *must* be converted to a method of the capturer and an interface to that must be assigned to the function reference. Because that's what function references are: interfaces. If you don't want that then call the anonymous function directly without assigning it to anything or assign it to a nested function pointer and call that. In *theory* the compiler could detect that "proc" doesn't leave the scope and that it's value is "constant" (namely a function) and thus could optimize that, but that requires quite some improvements to the optimizer. > In fact Swift requires you to mark closures types as "@escaping" if they > can escape the current scope so that the compiler can be optimize them. > This is the feature I was thinking FPC needs. > By default the purpose of anonymous functions assigned to function references *is* that they can escape their scope. This will not change, because they were after all introduced for Delphi-compatibility and there won't be any designations that the value can't escape. If the compiler can't figure it out by its own (see above) then tough luck; use a feature better suited for that. Regards, Sven > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
> On Nov 2, 2023, at 1:44 PM, Sven Barth wrote: > > Now for nested as well as anonymous routines the compiler determines whether > a capturer is required at the point that the nested or anonymous routine is > assigned to a function reference (cause otherwise it can be handled as a > nested function or even a function or method pointer). This requirement is > detected during the parsing of the routine. On second thought I had some questions about this part. From what I see FPC always allocates the interface when reference pointers are used but what you write suggests maybe the compiler will demote the type to something more optimized (like a nested function). We talked about this at length before but I don't think anything was decided yet. for example: procedure DoThis; var proc: reference to procedure; begin proc := procedure begin end; proc(); end; doesn't need the heap allocated instance because the function reference never escapes the scope and it has no captured variables so it could be demoted to a global function even of nested. In fact Swift requires you to mark closures types as "@escaping" if they can escape the current scope so that the compiler can be optimize them. This is the feature I was thinking FPC needs. Can you clarify this? Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
> On Nov 2, 2023, at 1:44 PM, Sven Barth wrote: > > The most important piece of knowledge is that in a routine with nested (or > anonymous) routines the nested (or anonymous) routines aren't compiled down > to machine code right away. The compiler first parses the whole hierarchy > until it's on the global/method level again and only *then* it compiles the > outermost routine followed by the nested ones. > > Now for nested as well as anonymous routines the compiler determines whether > a capturer is required at the point that the nested or anonymous routine is > assigned to a function reference (cause otherwise it can be handled as a > nested function or even a function or method pointer). This requirement is > detected during the parsing of the routine. > > If the routine requires a capturer then after the whole hierarchy has been > parsed, then all the captured variables will be moved from the local symtable > of the routine to the capturer or corresponding capture variables for > parameters will be introduced and then the whole nodetree will be walked to > adjust usages of the involved symbols. > > So, no, there is no copy involved (except for parameter values right at the > start of a routine where a parameter is captured) as the whole variables are > relocated into the capturer and they're only ever accessed from there. Ok that makes sense to move the symbols to another table before the function ends. Thanks for your detailed answer. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Creating capturers
Am 01.11.2023 um 06:18 schrieb Hairy Pixels via fpc-pascal: I'm curious how the capturer is created in the case of anonymous functions. I know the function needs to create a capturer when nested functions are declared but what happens if there is an anonymous function which is declared in the code section? I think the compiler will only know about the requirement to create a capturer at this point but local variables have already been assigned so do they need to do an extra copy to the capturer when it's created? With nested function this is not a problem since the capturer can be created before any locals are assigned and thus not copying is needed, if I understand correctly. The most important piece of knowledge is that in a routine with nested (or anonymous) routines the nested (or anonymous) routines aren't compiled down to machine code right away. The compiler first parses the whole hierarchy until it's on the global/method level again and only *then* it compiles the outermost routine followed by the nested ones. Now for nested as well as anonymous routines the compiler determines whether a capturer is required at the point that the nested or anonymous routine is assigned to a function reference (cause otherwise it can be handled as a nested function or even a function or method pointer). This requirement is detected during the parsing of the routine. If the routine requires a capturer then after the whole hierarchy has been parsed, then all the captured variables will be moved from the local symtable of the routine to the capturer or corresponding capture variables for parameters will be introduced and then the whole nodetree will be walked to adjust usages of the involved symbols. So, no, there is no copy involved (except for parameter values right at the start of a routine where a parameter is captured) as the whole variables are relocated into the capturer and they're only ever accessed from there. Regards, Sven ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] Creating capturers
I'm curious how the capturer is created in the case of anonymous functions. I know the function needs to create a capturer when nested functions are declared but what happens if there is an anonymous function which is declared in the code section? I think the compiler will only know about the requirement to create a capturer at this point but local variables have already been assigned so do they need to do an extra copy to the capturer when it's created? With nested function this is not a problem since the capturer can be created before any locals are assigned and thus not copying is needed, if I understand correctly. == var gProc: reference to procedure; procedure DoThis; var a: array[0..2] of integer; begin // Compiler does not yet know the function requires a capturer a := [1,2,3]; // DoThis requires a capturer now since the function reference is assigned to an anonymous (nested) function. // Does it copy the array (and other local variables) or does the compiler always make a capturer for all functions if the "anonymousfunctions" mode switch is enabled? gProc := procedure begin writeln(Length(a)); end; end; Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
