Re: [fpc-devel] Good timing metric test program?
Gareth, First of all, thanks for working on compiler optimizations. I think it's very important. As of benchmark - there is a simple 3D ray tracer benchmark written on several languages including Pascal. AFAIR Pascal version almost as fast as Java one and slightly faster than Javascript. There is forum topic about it with source code: http://forum.lazarus.freepascal.org/index.php/topic,35700.0.html There are also some optimizations made by hand e.g. loop unrolling etc. It may help to analyze optimizer results. --- Best Regards, George 25.02.2019, 17:54, "J. Gareth Moreton" : > Given my recent work with the peephole optimizer, one thing that sprung to > mind is that I don't have a project that tests for performance gains in a > 'real world' program, where little optimisations add up over time. Given > that my x86-64 optimizer overhaul is rather substantial and makes a lot of > improvements when it comes to conditional jumps and code efficiency, is there > a benchmark that could be used to show the performance improvement compared > to the trunk? There are small ones that test individual components, but > nothing substantially large that I'm aware of. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
Speaking of the optimiser overhaul, what are timings like for others? Gareth aka. Kit ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
Thanks George, I've finished and debugged my optimizer overhaul, although performance varies. It is predominantly faster than the existing peephole optimizer, but not always as fast as I'd like (no more than a few seconds). I figure I might have introduced one or two bottlenecks during my debugging. I'll probably need some kind of profiling tool to figure out where the slowdowns are, but I'm just glad that I've got it working at last! If people are willing to make -O1 and -O2 slightly worse when it comes to optimisation, or at least -O1, then I can make it faster still, but one of my philosophies with this overhaul was not to make the compiled code any worse on -O1 and -O2... only equal or better. Gareth aka. Kit On Tue 26/02/19 12:46 , "George Bakhtadze" armorcava...@yandex.com sent: Gareth, First of all, thanks for working on compiler optimizations. I think it's very important. As of benchmark - there is a simple 3D ray tracer benchmark written on several languages including Pascal. AFAIR Pascal version almost as fast as Java one and slightly faster than Javascript. There is forum topic about it with source code: http://forum.lazarus.freepascal.org/index.php/topic%2C35700.0.html [1]">http://forum.lazarus.freepascal.org/index.php/topic,35700.0.html There are also some optimizations made by hand e.g. loop unrolling etc. It may help to analyze optimizer results. --- Best Regards, George 25.02.2019, 17:54, "J. Gareth Moreton" : > Given my recent work with the peephole optimizer, one thing that sprung to mind is that I don't have a project that tests for performance gains in a 'real world' program, where little optimisations add up over time. Given that my x86-64 optimizer overhaul is rather substantial and makes a lot of improvements when it comes to conditional jumps and code efficiency, is there a benchmark that could be used to show the performance improvement compared to the trunk? There are small ones that test individual components, but nothing substantially large that I'm aware of. Links: -- [1] http://forum.lazarus.freepascal.org/index.php/topic%2C35700.0.html ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
Gareth, I like very much what you do about compiler optimizations. From my point of view, execution speed is the most valuable, but of course 17% of compilation speed increase is worth it! If you want to have a big test case, and potentially find regressions, you may try the TestSQL3 project of our Open Source mORMot. It makes millions of checks, and there are timing of individual test cases available in the console log, for comparison. See https://github.com/synopse/mORMot/blob/master/SQLite3/TestSQL3.lpi How to setup mORMot is detailed in https://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_125 If you find only some part of it interesting for your performance tests (perhaps the SQLite3 tests, or the encryption performance written in asm won't benefit from FPC optimization work), just ask and I could extract some test cases for your purpose. Great work! Arnaud ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
Well, compiling Lazarus is what I've been doing to test the compiler's speed, and I've got some promising results: https://bugs.freepascal.org/view.php?id=34628#c114453 Though the speed of the runs varies a lot depending on what my system is doing, especially when I switch back and forth between my code and the unmodified trunk, I get about a 15% speed gain in the compiler and a small size saving too, mostly due to overhauled jump optimisations. When it comes to the metric test program, the best comparison I can think of are those fancy benchmark programs used to test graphics cards and spit out a score. Compiling Lazarus is good and all, but you can't easily determine if its compiled code is any more efficient than before, outside of painstakingly studying the disassembly side-by-side with the control case. Saying all that, it might be an incentive to design such a test program that does a number of different operations like multiplying a vector array by a matrix (this would a good test case for vectorisation), generating prime numbers using a Sieve of Eratosthenes (would test array polling) and converting integers into different bases (tests to see how well the compiler can deal with div and mod instructions, especially as, currently, the compiler isn't smart enough to combine the operations if the two appear together, since the DIV instruction returns both the quotient and the remainder simultaneously... even when dividing by a constant, which gets optimised into a multiplication using some trickery with how MUL works on x86 processors, if you try to compute the remainder right afterwards, it will do the multiplication trick again, multiply the resultant quotient by the divisor, and subtract the result from the original number). Of course, lots of those already exist as individual test cases, but I need something more extensive because a lot of optimisations, like those that are designed to decrease the chance of pipeline stalls (I added one in my optimiser overhaul, that turns "mov %reg1,%reg2; mov %reg2,%reg3" to "mov %reg1,%reg2; mov %reg1,%reg3" - I was able to slip it in effectively for free because another optimisation checks for the same arrangement, but only if %reg2 is discarded afterwards, not if it's used again later), are very hard to measure in a small test and need to be a part of an extensive bench test before the benefits start to show. Sometimes I get people asking why I'm bothering trying to find the smallest of savings in size and execution speed - or in my own programming, writing mathematical functions like the aforementioned matrix multiplication in raw assembly language for the same benefit - since it's so much time and effort for very little again. Truthfully... I enjoy the challenge! And I'm driven further because I can pass on the benefits to others. I do a lot of playing around with mathematics, and when it comes to number crunching, especially for things that can take weeks to complete (e.g. Lucas-Lehmer Primality Testing), even a small saving can multiply into an entire day of saved time. I grew up with Turbo Pascal and then Delphi 2.0 as a pre-teen, and being more of an algorithmic programmer nowadays, I want to be able to say about FreePascal: "This is a good language for time-critical functions". Just a little ambition! Gareth aka. Kit On Mon 25/02/19 18:41 , "Sven Barth" pascaldra...@googlemail.com sent: J. Gareth Moreton schrieb am Mo., 25. Feb. 2019, 19:14: The compiler isn't a valid case because the input source is different (because of the very changes made to said compiler). It needs to be a project that doesn't share anything with the compiler (except the run-time libraries), so the source code is exactly the same so that when it is built, it runs the same no matter which version of the compiler it was built with. I'm viewing it as a bit of a scientific experiment, where only a single variable is changed, namely the compiler used. The compiled program should produce exactly the same output and otherwise behave the same way, so that any time metrics reflect only how long it takes to complete and hence is reflective only of the quality of the machine code, not what the program is doing... if that makes any sense. You could always build an unmodified compiler with your modified one ;) Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
J. Gareth Moreton schrieb am Mo., 25. Feb. 2019, 19:14: > The compiler isn't a valid case because the input source is different > (because of the very changes made to said compiler). It needs to be a > project that doesn't share anything with the compiler (except the run-time > libraries), so the source code is exactly the same so that when it is > built, it runs the same no matter which version of the compiler it was > built with. > > I'm viewing it as a bit of a scientific experiment, where only a single > variable is changed, namely the compiler used. The compiled program should > produce exactly the same output and otherwise behave the same way, so that > any time metrics reflect only how long it takes to complete and hence is > reflective only of the quality of the machine code, not what the program is > doing... if that makes any sense. > You could always build an unmodified compiler with your modified one ;) Regards, Sven > ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
On 25.02.2019 19:16, Ondrej Pokorny wrote: On 25.02.2019 18:12, J. Gareth Moreton wrote: The compiler isn't a valid case because the input source is different (because of the very changes made to said compiler). It needs to be a project that doesn't share anything with the compiler (except the run-time libraries), so the source code is exactly the same so that when it is built, it runs the same no matter which version of the compiler it was built with. I'm viewing it as a bit of a scientific experiment, where only a single variable is changed, namely the compiler used. The compiled program should produce exactly the same output and otherwise behave the same way, so that any time metrics reflect only how long it takes to complete and hence is reflective only of the quality of the machine code, not what the program is doing... if that makes any sense. Gareth aka. Kit Build Lazarus? What I meant: build compiler with and without your changes and then compare build times for Lazarus. (So actually you compare the compiler, just like Marco suggested.) Ondrej ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
On 25.02.2019 18:12, J. Gareth Moreton wrote: The compiler isn't a valid case because the input source is different (because of the very changes made to said compiler). It needs to be a project that doesn't share anything with the compiler (except the run-time libraries), so the source code is exactly the same so that when it is built, it runs the same no matter which version of the compiler it was built with. I'm viewing it as a bit of a scientific experiment, where only a single variable is changed, namely the compiler used. The compiled program should produce exactly the same output and otherwise behave the same way, so that any time metrics reflect only how long it takes to complete and hence is reflective only of the quality of the machine code, not what the program is doing... if that makes any sense. Gareth aka. Kit Build Lazarus? Ondrej ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
The compiler isn't a valid case because the input source is different (because of the very changes made to said compiler). It needs to be a project that doesn't share anything with the compiler (except the run-time libraries), so the source code is exactly the same so that when it is built, it runs the same no matter which version of the compiler it was built with. I'm viewing it as a bit of a scientific experiment, where only a single variable is changed, namely the compiler used. The compiled program should produce exactly the same output and otherwise behave the same way, so that any time metrics reflect only how long it takes to complete and hence is reflective only of the quality of the machine code, not what the program is doing... if that makes any sense. Gareth aka. Kit On Mon 25/02/19 18:08 , Marco van de Voort f...@pascalprogramming.org sent: Op 2019-02-25 om 14:52 schreef J. Gareth Moreton: > > Given my recent work with the peephole optimizer, one thing that > sprung to mind is that I don't have a project that tests for > performance gains in a 'real world' program, where little > optimisations add up over time. Given that my x86-64 optimizer > overhaul is rather substantial and makes a lot of improvements when it > comes to conditional jumps and code efficiency, is there a benchmark > that could be used to show the performance improvement compared to the > trunk? There are small ones that test individual components, but > nothing substantially large that I'm aware of. > Build the compiler? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org [1] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel [2]">http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel Links: -- [1] mailto:fpc-devel@lists.freepascal.org [2] http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Good timing metric test program?
Op 2019-02-25 om 14:52 schreef J. Gareth Moreton: Given my recent work with the peephole optimizer, one thing that sprung to mind is that I don't have a project that tests for performance gains in a 'real world' program, where little optimisations add up over time. Given that my x86-64 optimizer overhaul is rather substantial and makes a lot of improvements when it comes to conditional jumps and code efficiency, is there a benchmark that could be used to show the performance improvement compared to the trunk? There are small ones that test individual components, but nothing substantially large that I'm aware of. Build the compiler? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] Good timing metric test program?
Hi everyone, Given my recent work with the peephole optimizer, one thing that sprung to mind is that I don't have a project that tests for performance gains in a 'real world' program, where little optimisations add up over time. Given that my x86-64 optimizer overhaul is rather substantial and makes a lot of improvements when it comes to conditional jumps and code efficiency, is there a benchmark that could be used to show the performance improvement compared to the trunk? There are small ones that test individual components, but nothing substantially large that I'm aware of. Gareth aka. Kit ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
