I updated the string_alloc branch with support for static strings
(the string data itself is inside the data section). In that branch its
used for program constants, functions, efun names, etc. It saves quite a
bit of heap memory. The statistics look like this:
array_bytes 64 bytes 20.1 kB
free_block_bytes 16.9 kB 800.1 kB
malloc_block_bytes -52 kB 2.5 MB
malloc_bytes -68.8 kB 1.2 MB
marker_bytes -30.0 kB 216 bytes
num_arrays 1 254
num_malloc -119 116
num_short_pike_strings -815 1347
num_static_pike_strings 1067 1067
num_strings 3 3137
short_pike_string_bytes -240.3 kB 0 bytes
string_bytes 6.2 kB 522.3 kB
This table displays the difference in output of Debug.memory_usage, the
thirds column is the output for the string_alloc branch, the second the
difference between pike 8.0 and string_alloc. The difference should be
even bigger with more modules loaded, this is the output of
pike -e 'write("%O\n", Debug.memory_usage());'
Not sure where to go with this. The benchmark results are still pretty
mixed, so this is mainly saving memory right now.
arne
On Mon, 18 Aug 2014, Arne Goedeke wrote:
There was another topic that was mentioned briefly. It was a proposal by
Per from one year earlier about string allocation.
Currently, strings (headers + content) are allocated in one chunk. For short
strings (up to length of the header) they are allocated by the block
allocator,
otherwise using malloc. This has performance advantages when handling
short strings. On the other hand, keeping the header (which is modified
frequently) close to the content (which is constant) can have serious
disadvantages for performance.
The proposal was to split up header and string content into separate
allocations. I had a branch lying around which does this split. I took
the time to rebase it onto current 8.0. You can find it under
arne/string_alloc.
The current version allocates short strings using the same block
allocator used for the headers which usually results in the string
content being located directly after the headers. All other string have
their content allocated using malloc.
Benchmarks comparing this new branch to 8.0 have mixed results, but that
is to be expected. See below for the results, positive changes correspond
to the new branch being faster.
On top of this branch it would be possible to add more flags than
STRING_IS_SHORT to allow handling more types of allocation, like mmap
and constant strings. Think Stdio.read_file without memcpy.
Feedback welcome.
arne
-----------------------------------------------------------------
Test Result Change
-----------------------------------------------------------------
Ackermann . . . . . . . . . . . . . . . . . . . . . 33M/s -0.9%
Adding element to array (global) . . . . . . . . 5250k/s -0.6%
Adding element to array (local) . . . . . . . . . 5741k/s -12.2%
Adding element to array (private global) . . . . . 5201k/s -1.3%
Append array . . . . . . . . . . . . . . . . . . . 27M/s -0.7%
Append mapping (+) . . . . . . . . . . . . . . . . 49k/s -0.1%
Append mapping (|) . . . . . . . . . . . . . . . . 53k/s -12.8%
Append multiset . . . . . . . . . . . . . . . . . . 114k/s 5.4%
Array & String Juggling . . . . . . . . . . . . . . 80k/s 0.6%
Array Copy . . . . . . . . . . . . . . . . . . . . 41M/s 11.0%
Array Zero . . . . . . . . . . . . . . . . . . . . 258k/s 0.8%
Binary Trees . . . . . . . . . . . . . . . . . . . 933k/s -1.1%
Clone null-object . . . . . . . . . . . . . . . . . 10M/s -1.5%
Clone object . . . . . . . . . . . . . . . . . . 5900k/s -0.5%
Compile . . . . . . . . . . . . . . . . . . . 97k lines/s 3.7%
Compile & Exec . . . . . . . . . . . . . . . 95k lines/s 1.5%
Foreach (arr,global) . . . . . . . . . . . . . . . 66M/s -4.1%
Foreach (arr,local) . . . . . . . . . . . . . . . . 178M/s -0.9%
Foreach (arr;local;global) . . . . . . . . . . . . 40M/s -2.0%
Foreach (arr;local;local) . . . . . . . . . . . . . 59M/s -0.5%
GC . . . . . . . . . . . . . . . . . . . . . . . . 1496/s 1.8%
Insert in array . . . . . . . . . . . . . . . . . . 51M/s 1.2%
Insert in mapping . . . . . . . . . . . . . . . . 8913k/s 0.7%
Insert in multiset . . . . . . . . . . . . . . . 3461k/s 4.0%
Loops Nested (global) . . . . . . . . . . . . . . . 32M/s 0.3%
Loops Nested (local) . . . . . . . . . . . . . . . 37M/s 0.6%
Loops Nested (local,var) . . . . . . . . . . . . . 37M/s -0.0%
Loops Recursed . . . . . . . . . . . . . . . . . . 17M/s 1.4%
Matrix multiplication (100x100) . . . . . . . . 2.19 GF/s 0.6%
Read binary INT128 . . . . . . . . . . . . . . . . 172k/s 25.7%
Read binary INT16 . . . . . . . . . . . . . . . . . 15M/s 17.1%
Read binary INT32 . . . . . . . . . . . . . . . . . 11M/s -1.6%
Replace (parallel) . . . . . . . . . . . . . . . . 10k/s 0.2%
Replace (serial) . . . . . . . . . . . . . . . . . 16k/s -0.1%
Simple arithmentics (globals) . . . . . . . . . . . 94M/s 1.3%
Simple arithmentics (private global) . . . . . . . 117M/s 1.2%
Simple arithmetics (locals) . . . . . . . . . . . . 147M/s 1.0%
Sort equal integers . . . . . . . . . . . . . . . . 71M/s 0.2%
Sort ordered integers . . . . . . . . . . . . . . . 87M/s 0.9%
Sort unordered integers . . . . . . . . . . . . . . 14M/s 1.7%
Sort unordered objects . . . . . . . . . . . . . . 565k/s 8.7%
String Creation . . . . . . . . . . . . . . . . . 2585k/s 1.6%
String Creation (existing) . . . . . . . . . . . 6327k/s 2.2%
String Creation (wide) . . . . . . . . . . . . . . 583k/s -2.9%
Tag removal u. Parser.HTML . . . . . . . . . . . 4232k/s -10.0%
Tag removal u. Regexp.PCRE . . . . . . . . . . . . 442k/s 1.7%
Tag removal u. array_sscanf . . . . . . . . . . . 6110k/s 4.3%
Tag removal u. division . . . . . . . . . . . . . . 817k/s -2.4%
Tag removal u. search . . . . . . . . . . . . . . . 985k/s 4.0%
Tag removal using a loop . . . . . . . . . . . . . 181k/s 2.4%
Tag removal using sscanf . . . . . . . . . . . . . 431k/s -1.2%
Upper/lower case shift 0 . . . . . . . . . . . . . 123M/s 0.9%
Upper/lower case shift 1 . . . . . . . . . . . . . 60M/s 0.7%
call_out handling . . . . . . . . . . . . . . . . . 185k/s 1.2%
call_out handling (with id) . . . . . . . . . . . 3324k/s -6.7%
-----------------------------------------------------------------
0.8%
-----------------------------------------------------------------
