Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-11-15 Thread Igor Romanchenko
On Thu, Nov 15, 2012 at 1:34 PM, Chitra Creta wrote: > Thanks for your example Chris. I will look into it as a long-term solution. > > Partitioning tables as a strategy worked very well indeed. This will be my > short/medium term solution. > > Another strategy that I would like to evaluate as a s

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-11-15 Thread Chitra Creta
Thanks for your example Chris. I will look into it as a long-term solution. Partitioning tables as a strategy worked very well indeed. This will be my short/medium term solution. Another strategy that I would like to evaluate as a short/medium term solution is archiving old records in a table bef

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-21 Thread Chris Travers
On Sun, Oct 21, 2012 at 5:46 AM, Chitra Creta wrote: > Thank you all. > > Ryan, would you mind sharing your one-time function to move it? > > Merlin, what are your suggestions to improve query performance? > > Shaun, thank you. I will look into facts and dimensions should all else > fail. > > Chr

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-21 Thread Chitra Creta
Thank you all. Ryan, would you mind sharing your one-time function to move it? Merlin, what are your suggestions to improve query performance? Shaun, thank you. I will look into facts and dimensions should all else fail. Chris, would you mind giving me an example of what you mean by your log, a

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-17 Thread Chris Travers
On Fri, Oct 12, 2012 at 7:44 AM, Chitra Creta wrote: > Hi, > > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many queries are

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-16 Thread Shaun Thomas
On 10/12/2012 09:44 AM, Chitra Creta wrote: 4. Creation of daily, monthly, yearly summary tables that contains aggregated data specific to the statistics required If the partitioning doesn't help you, I strongly urge you to build one or more Fact tables with appropriate Diminsions. If you don

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-16 Thread Merlin Moncure
On Fri, Oct 12, 2012 at 10:00 AM, Lonni J Friedman wrote: > On Fri, Oct 12, 2012 at 7:44 AM, Chitra Creta wrote: >> Hi, >> >> I currently have a table that is growing very quickly - i.e 7 million >> records in 5 days. This table acts as a placeholder for statistics, and >> hence the records are m

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-16 Thread Ryan Kelly
On Tue, Oct 16, 2012 at 09:26:09PM +1100, Chitra Creta wrote: > Thank you all for your suggestions. Since all of you recommended the > Partition option, I decided to try it out myself. > > I found a function that created partition tables for every month in the > last two years. A trigger was also

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-16 Thread Chitra Creta
Thank you all for your suggestions. Since all of you recommended the Partition option, I decided to try it out myself. I found a function that created partition tables for every month in the last two years. A trigger was also added to the parent table to ensure that every insert into it from hence

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread John R Pierce
On 10/12/12 7:44 AM, Chitra Creta wrote: 1. Purge old data 2. Reindex 3. Partition 4. Creation of daily, monthly, yearly summary tables that contains aggregated data specific to the statistics required if most of your queries read the majority of the tables, indexing will be of little hel

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Ryan Kelly
On Sat, Oct 13, 2012 at 01:44:02AM +1100, Chitra Creta wrote: > Hi, > > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many qu

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Ondrej Ivanič
Hi, On 13 October 2012 01:44, Chitra Creta wrote: > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many queries are run on thi

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Lonni J Friedman
On Fri, Oct 12, 2012 at 7:44 AM, Chitra Creta wrote: > Hi, > > I currently have a table that is growing very quickly - i.e 7 million > records in 5 days. This table acts as a placeholder for statistics, and > hence the records are merely inserted and never updated or deleted. > > Many queries are

[GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Chitra Creta
Hi, I currently have a table that is growing very quickly - i.e 7 million records in 5 days. This table acts as a placeholder for statistics, and hence the records are merely inserted and never updated or deleted. Many queries are run on this table to obtain trend analysis. However, these queries