On Tue, Jul 29, 2025 at 6:30 AM David Rowley <dgrowle...@gmail.com> wrote: > On the whole, I don't really see this as a flaw in the Memoize code. > We've plenty of other cases in the planner that produce inferior plans > due to lack of enough detail or accuracy of table statistics, so I'm > not planning on rushing to look into a fix. I will keep it in mind, > however.
Yeah, I agree that this issue isn't limited to the Memoize node; it's a more general problem. The optimizer can sometimes choose plans that are suboptimal by orders of magnitude due to inaccurate statistics. One way to improve this is by improving the accuracy of statistics, but that can be very difficult or even impossible in some cases, especially when dealing with attribute correlations or highly skewed data distributions. Another possible direction is to support runtime plan correction or feedback loops. We've always followed a "plan-first, execute-next" approach so far. But perhaps we could extend that by monitoring plan execution and triggering re-optimization when the executor detects that actual result sizes or other runtime statistics differ significantly from the estimates. In recent years, there have been more and more papers and research on adaptive query processing. It might be worth considering how PostgreSQL could support such techniques in the future. Thanks Richard
