04.06.2020 20:11, Adriano dos Santos Fernandes wrote:
Hi!
To compile a statement a lot of Nodes are allocated and they are rarely
freed during compilation.
Most of them lives until the statement is destroyed.
With each node there is the memory manager overhead (16 bytes in release
build), that exists there to make individual node deallocation possible.
A waste of space in this case, as nodes are de-allocated as a whole with
the pool.
I did some tests to see how many nodes were created and how much memory
a statement pool uses:
-- 39 nodes, 31728 bytes allocated
select * from rdb$database;
-- 109 nodes, 91904 bytes allocated
select * from rdb$database
union all
select * from rdb$database
union all
select * from rdb$database
union all
select * from rdb$database
union all
select * from rdb$database;
-- 163 nodes, 120496 bytes allocated
execute block
as
declare n integer;
begin
select rdb$relation_id from rdb$database into n;
-- select repeated 20 more times
end!
So multiplying number of nodes by 16 we have around 2% of waste in these
tests.
Can't say it is a big win - to save 2% of memory, sorry.
I'm thinking on using something like a memory arena so a large block is
allocated (from the pool) and nodes allocation uses the arena, not
needing a per-node overhead.
It's not necessary to have a single large block. Many could be created
and we start with not very large to not waste space for small statements.
We could also estimate initial block size from the BLR size.
And inserting a protected delete operator in Node we avoid them being
explicitly called (with will crash).
>
What do you think?
It is doable of course, but if you want to really save memory (more then 2%)
you could find more ways to do it. For example - literal nodes. There is a lot
of same literals used in many statements such as NULL, 0, 1, "" and so on. And
there is no sense to allocate it again and again and again.
Next. To create compiled statement ready for execution from SQL statement
text,
engine:
a) generates DSQL tree
b) generates BLR code using DSQL tree
c) generates final compiled statement from BLR from (b).
If I'm not mistaken, nodes created at step (a) are allocated from statement pool
and not deleted until statement is destroyed. Is it possible to allocate them
from separate pool (dsql scratch pool, probably ?) and to free it after step
(b) ?
Next. Many nodes have two set of fields: for DSQL processing and for BLR (JRD)
processing. Obviously, DSQL fields is not used at all after step (b). Splitting
such nodes by two could save some memory, I think.
Also, many nodes is replaced when statement is parsed. We could try to re-use
such nodes, probably.
Next. Many optimizer internal structures are allocated from statement pool and
then really deleted. But memory allocated for it is not reused and wasted.
Moving
such objects into separate temporary pool allows to reduce memory allocation
(not
usage!) by 7-10% in some tests.
There is also inefficient memory allocation for a statement's impure area I'm
investigating currently. It is about parameters, variables and probably some
other
nodes.
Hope it helps,
Vlad
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
