Re: how to limit statement memory allocation

2021-03-10 Thread Radoslav Nedyalkov
On Tue, Mar 9, 2021 at 8:13 PM Tom Lane wrote: > Radoslav Nedyalkov writes: > > On Tue, Mar 9, 2021 at 6:53 PM Tom Lane wrote: > >> The only thing I'm aware of that could consume unbounded memory > >> on the server side is hash aggregation. (v13 has improved that > >> situation, but I'm guessi

Re: how to limit statement memory allocation

2021-03-09 Thread Tom Lane
Radoslav Nedyalkov writes: > On Tue, Mar 9, 2021 at 6:53 PM Tom Lane wrote: >> The only thing I'm aware of that could consume unbounded memory >> on the server side is hash aggregation. (v13 has improved that >> situation, but I'm guessing you are running some older version.) >> The planner won'

Re: how to limit statement memory allocation

2021-03-09 Thread Radoslav Nedyalkov
On Tue, Mar 9, 2021 at 6:53 PM Tom Lane wrote: > Radoslav Nedyalkov writes: > > On Tue, Mar 9, 2021 at 6:03 PM Tom Lane wrote: > >> Perhaps the accumulation is happening on the client side? libpq doesn't > >> have any provision for spilling a result set to disk. > > > Ah, I named it result set

Re: how to limit statement memory allocation

2021-03-09 Thread Tom Lane
Radoslav Nedyalkov writes: > On Tue, Mar 9, 2021 at 6:03 PM Tom Lane wrote: >> Perhaps the accumulation is happening on the client side? libpq doesn't >> have any provision for spilling a result set to disk. > Ah, I named it result set wrongly perhaps. > These are queries , part of a larger ETL

Re: how to limit statement memory allocation

2021-03-09 Thread Radoslav Nedyalkov
On Tue, Mar 9, 2021 at 6:03 PM Tom Lane wrote: > Radoslav Nedyalkov writes: > > Occasionally we get bad queries on our db that consume a lot of memory. > > These typically are full joins by mistake or just too large result sets. > > My understanding is these should go to a temp file but apparen

Re: how to limit statement memory allocation

2021-03-09 Thread Tom Lane
Radoslav Nedyalkov writes: > Occasionally we get bad queries on our db that consume a lot of memory. > These typically are full joins by mistake or just too large result sets. > My understanding is these should go to a temp file but apparently memory > allocation is preferred. Perhaps the accumu

how to limit statement memory allocation

2021-03-09 Thread Radoslav Nedyalkov
Hi all, Occasionally we get bad queries on our db that consume a lot of memory. These typically are full joins by mistake or just too large result sets. My understanding is these should go to a temp file but apparently memory allocation is preferred. Last time a statement hit 150GB RAM and did no