Re: ftbench: last updates

Thu, 21 Sep 2023 13:56:32 -0700 

Hi,
I have created a new branch.
Here you can see it: 
https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-2023-ahmet-final

-Made documentation and comment line (will continue).
-trailing whitespaces cleared
-more verbose commit messages
-formatted the code.

In this version of the code, i just didnt splitted iterations into chunks, got 
same results with it.

Best,
Goksu
goksu.in
On 18 Sep 2023 10:34 +0300, Werner LEMBERG <w...@gnu.org>, wrote:
>
> Hello Ahmet,
>
>
> > -I have changed the * and the sentence
>
> Thanks. Unfortunately, I was unclear that '*x*' in my e-mail is meant
> as Markdown syntax and not to be taken verbatim. In other words, 'x'
> should be typeset in italics, similar to a mathematical variable.
> Sorry for that, and please fix.
>
> Something different: Please don't use `\r` in messages for newlines.
> Use `\n` instead, which is standard (file `testing.mk`).
>
> > > I already changed the working way of the timing. I only start the
> > > benchmark at beginning and stop at the end.
> >
> > i mean, it times chunks, not single iteration. timer starts at the
> > beginning of the chunk and stop at the end (then divide the results
> > size of a chunk). because of it does not time single iteration, it
> > is already a bulk test.
>
> Ok.
>
> > > BTW, I suggest that you add another sentence, explaining *why*
> > > there are two values at all.
> >
> > actually, i didnt get the reason well but it may differ even with
> > same flags. i need help in this case.
>
> Alexei, please have a look. It seems that only(?) the 'Stroke' test
> is affected...
>
> > the code seems producing more accurate results after splitting the
> > results into chunks. are results seem satisfactory in your machine?
>
> Alas, they aren't. Running your code on my GNU/Linux machine, I get
> percental differences up to 14%.
>
> BTW, I've also just compiled `valgrind` from its git repository (to be
> sure that it exactly fits my 'libc' library version) and run
>
> ```
> valgrind --trace-children=yes make baseline
> valgrind --trace-children=yes make benchmark
> ```
>
> expecting to get identical results, because `valgrind` uses a CPU
> emulator. However, to my great surprise, they are *not* identical,
> and sometimes the values differ even enormously. Has anybody an idea
> why this happens? AFAICS from valgrind's source files, the
> `clock_gettime` function *is* caught.
>
> Note that with valgrind the creation of the webpage fails; I get
>
> ```
> Traceback (most recent call last):
> File "/home/wl/ahmet/tohtml.py", line 310, in <module>
> main()
> File "/home/wl/ahmet/tohtml.py", line 67, in main
> generate_total_results_table(html_file, BASELINE_DIR, BENCHMARK_DIR)
> File "/home/wl/ahmet/tohtml.py", line 192, in generate_total_results_table
> diff = ((baseline - benchmark) / baseline) * 100
> ZeroDivisionError: float division by zero
> ```
>
> Attached are the two directories from valgrind runs. Comparing this
> with non-valgrind values I think there is a fundamental interaction
> problem with the `bench` program: It seems that `bench` controls the
> number of loops *in addition* to what you specify via the command
> line. Unfortunately, I don't have time right now to check this by
> myself.
>
> Too bad that you didn't try `valgrind` by yourself earlier, as I've
> asked you repeatedly to do ...
>
> > Also, While there are less than 10 days for final evaluation, there
> > are points that are not completed:
> >
> > * meson
> > * cmake
> > * documentation
> >
> > because of our focus a bit changed, didnt worked on them
> > much. Should I complete them all? Is there a priority?
>
> Please forget meson and cmake, this is not important at this stage.
> What I now ask is to create a new branch, with *all* of your changes
> recreated as logical and (relatively small) incremental commits,
> rebased to 'master'. Ideally every commit is compilable. If not
> (which should be an exception, and there should be a good explanation
> for that), please add a remark in the commit message.
>
> * Try to adjust the formatting to be as near to the remaining code of
> FreeType.
>
> * The commit messages should also be in FreeType style, which is much*
> *more verbose than what you currently do.
>
> * Avoid trailing whitespace – whatever editor you use, *please* change
> the settings so that this gets automatically removed. Right now, I
> see zillions of trailing whitespace in file `tohtml.py`.
>
> * Hin-Tak is right: Documentation is of paramount importance so that
> you (or someone else) can continue the work.
>
> Have a look at previous GSoC students' (remote) branches to see how
> such a recreated branch should look like, for example
> `GSoC-2019-nikhil` or `GSoC-2017-ewaldhew`.
>
>
> Werner

Freetype Benchmark Results

Warning: Baseline and Benchmark have the same commit ID!

Info

InfoBaselineBenchmark
Parameters-c 1000 -w 100-c 1000 -w 100
Commit IDe61380fee61380fe
Commit Date2023-09-21 15:34:38 +03002023-09-21 15:34:38 +0300
Branchgsoc-2023-ahmet-finalgsoc-2023-ahmet-final

* Average time for single iteration. Smaller values are better.

* If a value in the 'Iterations' column is given as 'x | y', values x and y give the number of iterations in the baseline and the benchmark test, respectively.

Total Results

TestIterations* Baseline (µs) * Benchmark (µs)Difference (%)
Load100000 7.2 7.02.4
Load_Advances (Normal)100000 6.4 6.21.7
Load_Advances (Fast)100000 0.0 0.0-4.8
Load_Advances (Unscaled)100000 0.0 0.0-0.7
Render100000 9.7 9.34.1
Get_Glyph100000 7.0 6.83.2
Get_Char_Index94000 0.0 0.03.0
Iterate CMap1000 2.1 2.04.3
New_Face1000 51.8 49.15.3
Embolden100000 7.8 7.63.4
Stroke56642 | 57701 36.3 35.61.9
Get_BBox100000 5.8 5.71.3
Get_CBox100000 7.0 6.81.6
New_Face & load glyph(s)100000 1.1 1.10.6
Total duration for all tests:81 s

Results for Roboto_subset.ttf

TestIterations * Baseline (µs) * Benchmark (µs) Difference (%)
Load120000 4.94.90.6
Load_Advances (Normal)120000 4.24.20.6
Load_Advances (Fast)120000 0.00.0-21.4
Load_Advances (Unscaled)120000 0.00.0-7.7
Render120000 7.67.42.9
Get_Glyph120000 4.94.9-0.2
Get_Char_Index94000 0.00.00.0
Iterate CMap1000 2.02.00.3
New_Face1000 43.542.81.5
Embolden120000 5.35.30.6
Stroke60720 | 60600 33.033.0-0.1
Get_BBox120000 1.21.2-2.2
Get_CBox120000 5.04.91.3
New_Face & load glyph(s)120000 0.90.91.2
Total duration for the font:14 s

Results for Arial_subset.ttf

TestIterations * Baseline (µs) * Benchmark (µs) Difference (%)
Load95000 9.78.412.9
Load_Advances (Normal)95000 8.77.99.1
Load_Advances (Fast)95000 0.00.00.0
Load_Advances (Unscaled)95000 0.00.00.0
Render95000 11.910.610.7
Get_Glyph95000 9.28.112.1
Get_Char_Index94000 0.00.010.0
Iterate CMap1000 2.32.09.8
New_Face1000 62.252.216.1
Embolden95000 10.18.615.1
Stroke51585 | 55670 38.836.07.4
Get_BBox95000 7.97.54.3
Get_CBox95000 8.68.06.3
New_Face & load glyph(s)95000 1.21.20.8
Total duration for the font:19 s

Results for TimesNewRoman_subset.ttf

TestIterations * Baseline (µs) * Benchmark (µs) Difference (%)
Load95000 9.910.0-1.0
Load_Advances (Normal)95000 9.08.90.6
Load_Advances (Fast)95000 0.00.0-3.6
Load_Advances (Unscaled)95000 0.00.0-7.7
Render95000 12.912.62.2
Get_Glyph95000 9.810.0-1.3
Get_Char_Index94000 0.00.00.0
Iterate CMap1000 2.02.01.6
New_Face1000 54.654.01.1
Embolden95000 11.011.1-0.4
Stroke41895 | 41990 47.847.70.2
Get_BBox95000 9.69.60.2
Get_CBox95000 9.910.0-1.2
New_Face & load glyph(s)95000 1.41.5-1.1
Total duration for the font:20 s

Results for Tahoma_subset.ttf

TestIterations * Baseline (µs) * Benchmark (µs) Difference (%)
Load95000 6.06.4-6.8
Load_Advances (Normal)95000 5.15.3-2.8
Load_Advances (Fast)95000 0.00.00.0
Load_Advances (Unscaled)95000 0.00.00.0
Render95000 8.38.11.7
Get_Glyph95000 6.05.82.1
Get_Char_Index94000 0.00.03.7
Iterate CMap1000 2.02.02.8
New_Face1000 51.949.54.5
Embolden95000 6.76.61.8
Stroke62985 | 63935 31.831.31.5
Get_BBox95000 5.45.30.4
Get_CBox95000 5.95.81.2
New_Face & load glyph(s)95000 1.11.11.6
Total duration for the font:14 s

Results for Verdana_subset.ttf

TestIterations * Baseline (µs) * Benchmark (µs) Difference (%)
Load95000 5.45.41.6
Load_Advances (Normal)95000 4.74.9-4.4
Load_Advances (Fast)95000 0.00.00.0
Load_Advances (Unscaled)95000 0.00.09.7
Render95000 7.87.81.1
Get_Glyph95000 5.45.40.4
Get_Char_Index94000 0.00.00.0
Iterate CMap1000 2.12.06.6
New_Face1000 47.046.90.3
Embolden95000 6.06.3-4.8
Stroke66025 | 66310 30.330.20.4
Get_BBox95000 4.94.90.3
Get_CBox95000 5.45.4-0.2
New_Face & load glyph(s)95000 1.11.10.9
Total duration for the font:14 s
Freetype Benchmark

Reply via email to