Re: [HACKERS] Analyzing foreign tables memory problems

2012-05-15 Thread Noah Misch
On Mon, May 14, 2012 at 09:21:20AM +0200, Albe Laurenz wrote: Noah Misch wrote: Just thinking out loud, we could provide an extern Datum AnalyzeWideValue; and direct FDW authors to use that particular datum. It could look like a toasted datum of external size WIDTH_THRESHOLD+1 but bear

Re: [HACKERS] Analyzing foreign tables memory problems

2012-05-14 Thread Albe Laurenz
Noah Misch wrote: 1) Expose WIDTH_THRESHOLD in commands/vacuum.h and add documentation so that the authors of foreign data wrappers are aware of the problem and can avoid it on their side. This would be quite simple. Seems reasonable. How would the FDW return an indication that a

Re: [HACKERS] Analyzing foreign tables memory problems

2012-05-13 Thread Noah Misch
On Wed, May 02, 2012 at 12:20:39PM +0200, Albe Laurenz wrote: 1) Expose WIDTH_THRESHOLD in commands/vacuum.h and add documentation so that the authors of foreign data wrappers are aware of the problem and can avoid it on their side. This would be quite simple. Seems

Re: [HACKERS] Analyzing foreign tables memory problems

2012-05-02 Thread Albe Laurenz
I wrote: Noah Misch wrote: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows and assumed to be all different.

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Tom Lane
Albe Laurenz laurenz.a...@wien.gv.at writes: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows and assumed to be

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Albe Laurenz
Tom Lane wrote: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows and assumed to be all different. This works

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Tom Lane
Albe Laurenz laurenz.a...@wien.gv.at writes: Tom Lane wrote: I'm fairly skeptical that this is a real problem, and would prefer not to complicate wrappers until we see some evidence from the field that it's worth worrying about. If I have a table with 10 rows and

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Simon Riggs
On Mon, Apr 30, 2012 at 3:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Albe Laurenz laurenz.a...@wien.gv.at writes: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Noah Misch
On Mon, Apr 30, 2012 at 12:27:45PM +0200, Albe Laurenz wrote: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Albe Laurenz
Simon Riggs wrote: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows and assumed to be all different. This

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Albe Laurenz
Tom Lane wrote: I'm fairly skeptical that this is a real problem, and would prefer not to complicate wrappers until we see some evidence from the field that it's worth worrying about. If I have a table with 10 rows and default_statistics_target at 100, then a sample of 3 rows will be

Re: [HACKERS] Analyzing foreign tables memory problems

2012-04-30 Thread Albe Laurenz
Noah Misch wrote: During ANALYZE, in analyze.c, functions compute_minimal_stats and compute_scalar_stats, values whose length exceed WIDTH_THRESHOLD (= 1024) are not used for calculating statistics other than that they are counted as too wide rows and assumed to be all different. This