On Fri, 12 Sep 2025 14:09:47 +0800 Zhang Mingli <[email protected]> wrote:
> Hi, all > > On Aug 30, 2025 at 04:04 +0800, Yugo Nagata <[email protected]>, wrote: > > > > Anyway, I will start rebasing the patches, reorganizing the patch set, > > and applying fixes made in pg_ivm [2]. > > First of all, we would like to express our sincere gratitude for your > continuous efforts and contributions to the IVM feature. > Our AQUMV (Answer Query Using Materialized Views) functionality in Apache > Cloudberry is built directly upon your foundational work. > > > We recently encountered an issue in one of our customer's production > environments. > The problem occurs in the function CreateIvmTriggersOnBaseTablesRecurse(), > where a Bitmapset relidsis used to record rte->relid. > > ``` > case T_RangeTblRef: > { > int rti = ((RangeTblRef *) node)->rtindex; > RangeTblEntry *rte = rt_fetch(rti, qry->rtable); > > if (rte->rtekind == RTE_RELATION && !bms_is_member(rte->relid, *relids)) > { > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, > TRIGGER_TYPE_BEFORE, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, > TRIGGER_TYPE_BEFORE, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, > TRIGGER_TYPE_BEFORE, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, > TRIGGER_TYPE_BEFORE, true); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, > TRIGGER_TYPE_AFTER, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, > TRIGGER_TYPE_AFTER, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, > TRIGGER_TYPE_AFTER, ex_lock); > CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, > TRIGGER_TYPE_AFTER, true); > > *relids = bms_add_member(*relids, rte->relid); > } > } > ``` > However, the Bitmapset structure is typically designed to store small > integers (such as relation index), whereas rte->relidrepresents the Oid of a > relation. > For instance, when a table is created immediately after initializing a new > cluster, its Oid might be 17019. Storing such a value in a Bitmapset consumes > approximately 0.2MB of memory when creating an IVM. > > (gdb) p bmsToString(((Bitmapset *) relids)) > $13 = 0x5643d7bbb070 "(b 17019)" > (gdb) p *((Bitmapset *) relids) > $14 = {nwords = 266, words = 0x5643d79d1bd8} > > This memory usage becomes even more significant when dealing with larger Oids. > Moreover, a more critical issue arises when the Oid exceeds 0x7FFFFFFF. Since > Oid is an unsigned integer, but the Bitmapset expects signed integers, an > assertion failure will occur in such cases. > > > We have encountered similar requirements in our own scenarios and eventually > switched to using a HTAB implementation. Hope this proves useful for your > work. Sorry for the late reply. While reviewing the discussion around reworking this, I noticed that I never replied to this comment. That makes sense. I'll update the patch accordingly. Regards, Yugo Nagata -- Yugo Nagata <[email protected]>
