Re: [GENERAL] SQL - planet redundant data

2005-09-17 Thread Jim C. Nasby
On Sun, Sep 11, 2005 at 11:00:02PM -0800, Poul Jensen wrote: Tom Lane wrote: No, tableoid is sort of a virtual column ... it doesn't exist on disk. When you query it you get a value fetched from the internal data structure representing the table. So virtual columns are possible - THIS

Re: [GENERAL] SQL - planet redundant data

2005-09-12 Thread Poul Jensen
Thank you for your input! Individual replies follow below. ## Chris Travers wrote: Ok. Imagine two huge huge tables: file_dataand additional_data create_table file_data ( file_id serial primary key, station_id text, ); create table

Re: [GENERAL] SQL - planet redundant data

2005-09-12 Thread John D. Burger
Hmm, in fact if the redundant values you're worried about come in long stretches (e.g., temperature is the same for many observations in a row), I suppose you could do the same thing - map a constant value to the range of observation IDs for which it holds. This gets back to having many

Re: [GENERAL] SQL - planet redundant data

2005-09-12 Thread Poul Jensen
Thank you, John! I misunderstood you the first time, but I now see we have the same thing in mind. So you'd have most of your data in a main table: create table observations ( obsIDintegerprimary key,-- Maybe a BIGINT temperaturefloat, etc. ); and

Re: [GENERAL] SQL - planet redundant data

2005-09-12 Thread Brent Wood
That is exactly what I want, and now I finally see how to do it (I think!). However, it is a considerable amount of work to set this up manually, plus, it has been a headache realizing how to get there at all. I'm hoping that one or more of the developers think it would be a good idea for

Re: [GENERAL] SQL - planet redundant data

2005-09-09 Thread John D. Burger
To store the detailed records the SQL novice would construct one table pr. file and exclude any constant columns since these are redundant (given in summary table). These detailed tables would then have different column schemas - (mn,tmp,wind) for ~58%, (hr,mn,tmp,wind) for ~40%,

Re: [GENERAL] SQL - planet redundant data

2005-09-09 Thread Jim C. Nasby
On Thu, Sep 08, 2005 at 07:58:55PM -0800, Poul Jensen wrote: This novice must be missing a sneaky way to avoid massive redundancy and still maintain easy access. I've been suggested to look at inheritance and foreign keys. Foreign keys I don't see how to use, but I could make What you seem

[GENERAL] SQL - planet redundant data

2005-09-08 Thread Poul Jensen
I have ~500,000 data files each containing ~1,000 records that I want to put into a database for easy access. Fictive example for illustration: File w. meteorological data from a given station. stat_id | yr | d_o_y | hr | mn | tmp | wind -|--|---|||--|--

Re: [GENERAL] SQL - planet redundant data

2005-09-08 Thread Bruno Wolff III
On Thu, Sep 08, 2005 at 19:58:55 -0800, Poul Jensen [EMAIL PROTECTED] wrote: This novice must be missing a sneaky way to avoid massive redundancy and still maintain easy access. I've been suggested to look at inheritance and foreign keys. Foreign keys I don't see how to use, but I could

Re: [GENERAL] SQL - planet redundant data

2005-09-08 Thread Tom Lane
Poul Jensen [EMAIL PROTECTED] writes: One thought: PostgreSQL adds to every table a system column tableoid containing a constant value. Is that value really stored 1,000 times for a 1,000-row table? Or...? No, tableoid is sort of a virtual column ... it doesn't exist on disk. When you query