> On 19 Apr 2017, at 12:37, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: > > On 18/04/17 13:45, Stas Kelvich wrote: >> Hi, >> >> currently logical replication worker uses ApplyContext to decode received >> data >> and that context is never freed during transaction processing. Hence if >> publication >> side is performing something like 10M row inserts in single transaction, then >> subscription worker will occupy more than 10G of ram (and can be killed by >> OOM). >> >> Attached patch resets ApplyContext after each insert/update/delete/commit. >> I’ve tried to reset context only on each 100/1000/10000 value of >> CommandCounter, >> but didn’t spotted any measurable difference in speed. >> >> Problem spotted by Mikhail Shurutov. >> > > Hmm we also do replication protocol handling inside the ApplyContext > (LogicalRepApplyLoop), are you sure this change is safe in that regard?
Memory is bloated by logicalrep_read_* when palloc happens inside. I’ve inserted context reset in ensure_transaction() which is only called in beginning of hande_insert/update/delete/commit when data from previous call of that function isn’t needed. So probably it is safe to clear memory there. At least i don’t see any state that can be accessed later in this functions. Do you have any specific concerns? Stas Kelvich Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
