Hi,
Apologies if this is the wrong place to ask this question but it seemed
like the most apt.
We are working on operational processes for online schema changes with
MariaDB 10.1.16 and have encountered memory issues which are proving
difficult to reliably solve.
If anyone on this list could explain to me in detail how to calculate
the memory footprint of modifying a large table with DDL statements I
would be very grateful.
Our test case is a 3-node galera cluster on which we want to alter a
~160GB table that has . Each node has 64GB ram and (originally) no swap
space allocated. Even with the innodb buffer pool lowered to 16GB we
consistently reached an OOM condition trying to add a column to the
table. If we lower the innodb buffer pool to 16GB *and* provision 16GB
of swap, the DDL completes. With the innodb buffer pool left at 50GB
and allocated swap of 16GB, same OOM condition.
Since this is an operational procedure we are testing the process under
heavy load, but the node that is applying the DDL is first placed into
RSU mode so the writesets simply build up in the gcache while it's
executing (and obviously traffic from the production environment would
be directed away from the node while this is happening). We are also
increasing the gcache to 800GB on the node as part of this process to
ensure that we have plenty of time for it to complete.
I'm fairly sure that the gcache is simply a continuous write-to-disk
process rather than anything involving significant memory, so *hope*
that's not a contributing factor here. I am however quite surprised
that executing DDL on a table can be so memory-intensive.
Our next test run will be with the innodb buffer pool set to 0 on the
node executing the DDL with no swap allocated to see if it can complete
purely in-memory. Logically it should be able to if this is a memory
constraint rather than a bug.
But fundamentally we need to be able to determine with confidence how
much memory we need to successfully execute DDL on a table of {n} size.
Thanks,
Mark
_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to : [email protected]
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
