Re: Datawarehousing help
[EMAIL PROTECTED] wrote: I was hoping she would figure out the DW hard stuff, then write the book explaining everything with lots of big, colorful pictures...(big grin). If I were to write the book, and she be the tech editor, then she would drive down to Philly and kick myand shove the already burning manuscript down my throat. no, she would gently and emphatically point out your mistakes and suggest corrections to be made RIGHT NOW!;-) i haven't had a chance to do any DW stuff, just the data cleaning that goes before it. and that's a project in and of itself.;-) -- -- Bill Shrek Thater ORACLE DBA [EMAIL PROTECTED] You gotta program like you don't need the money, You gotta compile like you'll never get hurt, You gotta run like there's nobody watching, It's gotta come from the heart if you want it to work. Never violate the Prime Directory! C:\ -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: bill thater INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: Datawarehousing help
Hello Rachel I am working with SAS on the mainframe (os390) and it works fine. We published a RFP for ETL tool and they are one of the candidates. Their tool looks good on paper but we are still evaluating papers and do not have hands on experience. We do have a SAS/oracle application on NT but I was not involved in the programming side. I checked with the programmer and he told me that in the little testing he did ( system still in test) SAS worked quickly and nicely with oracle, including graphs. SAS has a nice graph generating option. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Monday, May 06, 2002 4:13 PM Yechiel, have you used their tools? We are trying to decide whether or not to use them, so if anyone has had recent experience with them, I'd appreciate your thoughts on ease of use, understandability, quality of the product, etc Thanks Rachel |+--- || | || | || adaryechiel@h| || otmail.com | || | || 05/05/2002 | || 07:23 AM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: Re: Datawarehousing help| | Hello Dennis SAS has progressed a little in the last years and now offer a complete DW solution, including ETL tools. You can use their tools also to populate and query oracle. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Saturday, May 04, 2002 2:48 AM Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often
Re: Datawarehousing help
I worked on a project with an Oracle 7.3.4 database and a SAS OLAP tool. SAS built a datacube using the Oracle database but then the OLAP queries went against the datacube. It worked but we had some very knowledgeable SAS users. Yechiel Adar adaryechiel To: Multiple recipients of list ORACLE-L @hotmail.com[EMAIL PROTECTED] Sent by: rootcc: Subject: Re: Datawarehousing help 05/07/2002 08:58 AM Please respond to ORACLE-L Hello Rachel I am working with SAS on the mainframe (os390) and it works fine. We published a RFP for ETL tool and they are one of the candidates. Their tool looks good on paper but we are still evaluating papers and do not have hands on experience. We do have a SAS/oracle application on NT but I was not involved in the programming side. I checked with the programmer and he told me that in the little testing he did ( system still in test) SAS worked quickly and nicely with oracle, including graphs. SAS has a nice graph generating option. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Monday, May 06, 2002 4:13 PM Yechiel, have you used their tools? We are trying to decide whether or not to use them, so if anyone has had recent experience with them, I'd appreciate your thoughts on ease of use, understandability, quality of the product, etc Thanks Rachel |+--- || | || | || adaryechiel@h| || otmail.com | || | || 05/05/2002 | || 07:23 AM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: Re: Datawarehousing help| | Hello Dennis SAS has progressed a little in the last years and now offer a complete DW solution, including ETL tools. You can use their tools also to populate and query oracle. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Saturday, May 04, 2002 2:48 AM Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made
RE: Datawarehousing help
won't be me... why don't YOU write one and then talk to me about the joys of authorship (says the woman going blind looking a page proofs that are totally messed up) |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 06:13 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Someone's got to pick up Marlene's slack... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 5:44 PM To: Multiple recipients of list ORACLE-L nuh uh, for two reasons, the first and foremost being, there already IS one the second is that I have no plans to write any new books |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 04:55 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel
RE: Datawarehousing help
you people are soo funny. Writing a book takes time, hard work and more energy than I care to commit to the project.. especially on a subject with which I have zero experience |+--- || | || | || ksmith2@myfir| || stlink.net | || | || 05/06/2002 | || 10:38 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Oh, good idea! When's it being published Rachel? -Original Message- Chris Sent: Monday, May 06, 2002 1:55 PM To: Multiple recipients of list ORACLE-L Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform
RE: Datawarehousing help
Come on, Rachel! Zero Experence has never stopped anyone from publishing. Just look at academia and MCSE study guides... Scott Shafer San Antonio, TX 210-581-6217 -Original Message- From: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Sent: Tuesday, May 07, 2002 8:19 AM To: Multiple recipients of list ORACLE-L Subject: RE: Datawarehousing help you people are soo funny. Writing a book takes time, hard work and more energy than I care to commit to the project.. especially on a subject with which I have zero experience -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
nuh uh, for two reasons, the first and foremost being, I was paying attention when you spoke about the joys of authorship. the second is that I was paying attention when you spoke about the joys of authorship... .. . .. ... .. .. .. . -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 07, 2002 9:09 AM To: Multiple recipients of list ORACLE-L won't be me... why don't YOU write one and then talk to me about the joys of authorship (says the woman going blind looking a page proofs that are totally messed up) |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 06:13 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Someone's got to pick up Marlene's slack... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 5:44 PM To: Multiple recipients of list ORACLE-L nuh uh, for two reasons, the first and foremost being, there already IS one the second is that I have no plans to write any new books |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 04:55 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking
RE: Datawarehousing help
it stops me. I refuse to deliberately look like an idiot... I do enough damage inadvertently |+--- || | || | || [EMAIL PROTECTED]| || ms.osd.mil | || | || 05/07/2002 12:58 PM | || Please respond to| || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Come on, Rachel! Zero Experence has never stopped anyone from publishing. Just look at academia and MCSE study guides... Scott Shafer San Antonio, TX 210-581-6217 -Original Message- From: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Sent: Tuesday, May 07, 2002 8:19 AM To: Multiple recipients of list ORACLE-L Subject: RE: Datawarehousing help you people are soo funny. Writing a book takes time, hard work and more energy than I care to commit to the project.. especially on a subject with which I have zero experience -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: Datawarehousing help
Rachel, Are you licensed for Oracle Designer? We put the estimated row counts for each table into Designer and it produces a nice report showing sizing estimates per table or index, tablespace, and database. Pretty painless. You still need to add on extra space for archives, exports, backups, etc. but at least you can get a sizing estimate for the tables, themselves. I'm not familiar with SAS. Cherie Machler Oracle DBA Gelco Information Network Rachel_Carmichael@Son ymusic.com To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent by:cc: [EMAIL PROTECTED]Subject: Datawarehousing help 05/03/02 05:08 PM Please respond to ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc. help? Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too
Re: Datawarehousing help
Yechiel, have you used their tools? We are trying to decide whether or not to use them, so if anyone has had recent experience with them, I'd appreciate your thoughts on ease of use, understandability, quality of the product, etc Thanks Rachel |+--- || | || | || adaryechiel@h| || otmail.com | || | || 05/05/2002 | || 07:23 AM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: Re: Datawarehousing help| | Hello Dennis SAS has progressed a little in the last years and now offer a complete DW solution, including ETL tools. You can use their tools also to populate and query oracle. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Saturday, May 04, 2002 2:48 AM Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Friday, May 03, 2002 5:08 PM To: Multiple recipients of list ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated
Re: Datawarehousing help
Rachel, I haven't used their warehouse stuff and it has been a few years since I had to deal with people doing statistical analysis with SAS (by the way very good for that so if the reason considering this is the need for heavy duty statistical analysis that is a big plus) but the one thing that I remember from someone having SQL problems with SAS might be worth taking a look at. SAS's implementation of SQL was more restricted than Oracle's. You might want to check on what current differences are and if they are relavent to your situation. I can't remember the specifics but think it was not a matter of can't do it but rather have to do it another way. Pat have you used their tools? We are trying to decide whether or not to use them, so if anyone has had recent experience with them, I'd appreciate your thoughts on ease of use, understandability, quality of the product, etc Thanks Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Pat Hildebrand INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Rachel, If a DW is built and that users do not have access to a part of it in an ad hoc fashion, you gonna have a lot of political meetings They should have some data marts for their usage and keep most of them off the raw data. Regarding SAS tools, I've used SAS more than 10 years ago in math classes... when it was only a statistical tool. --- [EMAIL PROTECTED] a écrit : Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | | | | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help | | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size
RE: Datawarehousing help
A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More
RE: Datawarehousing help
Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't
RE: Datawarehousing help
Just wanted to reiterate the grain recommendation. The growth rate of our data warehouse increased app. 20x when the business side changed their mind from monthly to daily on our largest fact table. They did this one week after we got the monthly table into production as per their original requirements. Had to redo all the tablespace structures to make it easier save historical data to tape and drop it from the database. We now have 13 tablespaces each of which will be holding one month's worth of data. -Original Message- Sent: Friday, May 03, 2002 10:18 PM To: Multiple recipients of list ORACLE-L Excellent dude. -Original Message- WILLIAMS Sent: Friday, May 03, 2002 5:48 PM To: Multiple recipients of list ORACLE-L Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Miller, Jay INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated
RE: Datawarehousing help
I've always been a cynic about storage -- too much is never enough. |+-- || | || | || JayMiller@tdwate| || rhouse.com | || | || 05/06/2002 04:43| || PM | || Please respond | || to ORACLE-L | || | |+-- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Just wanted to reiterate the grain recommendation. The growth rate of our data warehouse increased app. 20x when the business side changed their mind from monthly to daily on our largest fact table. They did this one week after we got the monthly table into production as per their original requirements. Had to redo all the tablespace structures to make it easier save historical data to tape and drop it from the database. We now have 13 tablespaces each of which will be holding one month's worth of data. -Original Message- Sent: Friday, May 03, 2002 10:18 PM To: Multiple recipients of list ORACLE-L Excellent dude. -Original Message- WILLIAMS Sent: Friday, May 03, 2002 5:48 PM To: Multiple recipients of list ORACLE-L Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Miller, Jay INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: Datawarehousing help
[EMAIL PROTECTED] wrote: Cool!! Here comes Oracle Data Warehousing 101... um knowing the way the goddess feels about book writing, i wouldn't go there if i were you.;-) unless, of course, you're going to write the book for her to edit.;-) -- -- Bill Shrek Thater ORACLE DBA [EMAIL PROTECTED] You gotta program like you don't need the money, You gotta compile like you'll never get hurt, You gotta run like there's nobody watching, It's gotta come from the heart if you want it to work. On a clear disk you can seek forever. - Denning -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: bill thater INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Hi Jared - I am a little confused about this - 1. A DW may be a true 'warehouse' of enterprise data from which DM may be built. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. So DM first or DW first? The first statement seems to suggest DW, but the second seems to suggest DM. 2. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. So DW is not star-schema but DM is? Can you elaborate a little on how DW resemble an OLTP. I am very interested in data warehousing and please let me know if you have any good pointers. TIA Dennis Meng Database Administrator Focal Communications Corp. Jared.Still@r adisys.com To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent by: cc: root@fatcity.Subject: RE: Datawarehousing help com 05/06/02 01:23 PM Please respond to ORACLE-L A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know
RE: Datawarehousing help
nuh uh, for two reasons, the first and foremost being, there already IS one the second is that I have no plans to write any new books |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 04:55 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which
RE: Datawarehousing help
I was hoping she would figure out the DW hard stuff, then write the book explaining everything with lots of big, colorful pictures...(big grin). If I were to write the book, and she be the tech editor, then she would drive down to Philly and kick myand shove the already burning manuscript down my throat. I'll pass... -Original Message- Sent: Monday, May 06, 2002 5:24 PM To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] wrote: Cool!! Here comes Oracle Data Warehousing 101... um knowing the way the goddess feels about book writing, i wouldn't go there if i were you.;-) unless, of course, you're going to write the book for her to edit.;-) -- -- Bill Shrek Thater ORACLE DBA [EMAIL PROTECTED] You gotta program like you don't need the money, You gotta compile like you'll never get hurt, You gotta run like there's nobody watching, It's gotta come from the heart if you want it to work. On a clear disk you can seek forever. - Denning -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: bill thater INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Grabowy, Chris INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Someone's got to pick up Marlene's slack... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 5:44 PM To: Multiple recipients of list ORACLE-L nuh uh, for two reasons, the first and foremost being, there already IS one the second is that I have no plans to write any new books |+--- || | || | || cgrabowy@fcg.| || com | || | || 05/06/2002 | || 04:55 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical
RE: Datawarehousing help
Whether you start designing a full blown DW or start with some DM's really depends on a number of things: 1. experience 2. money 3. time Kimball et all suggest you start with projects that can be completed in 90 days, and assemble your DW piece meal. If you're starting from scratch, you may have to work backwards on it to give your users some experience ( as well as your self ). I have had the good fortune of working on 2 DW's with very experienced folks, and learned a heck of a lot in the process. The best pointers I can give at the moment are to buy Kimball's books, and find a good consultant that really knows how to design DW. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 02:23 PM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Hi Jared - I am a little confused about this - 1. A DW may be a true 'warehouse' of enterprise data from which DM may be built. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. So DM first or DW first? The first statement seems to suggest DW, but the second seems to suggest DM. 2. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. So DW is not star-schema but DM is? Can you elaborate a little on how DW resemble an OLTP. I am very interested in data warehousing and please let me know if you have any good pointers. TIA Dennis Meng Database Administrator Focal Communications Corp. Jared.Still@r adisys.com To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent by: cc: root@fatcity.Subject: RE: Datawarehousing help com 05/06/02 01:23 PM Please respond to ORACLE-L A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data
RE: Datawarehousing help
Oh, good idea! When's it being published Rachel? -Original Message- Chris Sent: Monday, May 06, 2002 1:55 PM To: Multiple recipients of list ORACLE-L Cool!! Here comes Oracle Data Warehousing 101... -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 4:18 PM To: Multiple recipients of list ORACLE-L Right now I'm collecting information.. I don't KNOW what this will be.. other than a learning experience of course. That which does not kill us makes us strong, right? rachel, anticipating great strength |+--- || | || | || Jared.Still@r| || adisys.com | || | || 05/06/2002 | || 02:23 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | A DW is not simply a collection of data marts. A DW may be a true 'warehouse' of enterprise data from which DM may be built. Extracts go to the DW, DW is used to build DM. A DW may in fact very much resemble an OLTP database, with a temporal component thrown in to track changes to data over time. Users are not (generally) allowed acces to the DW. This is a full blown DW architecture though, and you may only wish to start with some DM to get your feet wet, or maybe that's all that is actually needed. Jared [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 05/06/2002 06:53 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:RE: Datawarehousing help Dennis, Forgetting about normalization won't be a problem, I've always been more practical than by the book. As for amounts of data being collected, I can see them wanting data aggregated hourly. I greatly doubt the tech people will allow adhoc queries, they seem to do things right here. What will happen is that they will be contacted by marketing with an I need this new report NOW request, but tech will generate it. But *my* problem is that the data warehouse will supposedly be only a small part of what I'm responsible for, I don't think they understand the scope of what they are asking for, as yet. They will, I'll make sure of it. Right now, as this is a new internal group, I'm still collecting information on which databases I will be responsible for. Then I just have to remember that when I set deadliines, I am prone to underestimation. :) Rachel |+--- || | || | || DWILLIAMS@lif| || etouch.com | || | || 05/03/2002 | || 08:48 PM | || Please | || respond to | || ORACLE-L | || | |+--- | || | To: [EMAIL PROTECTED] | | cc: (bcc: Rachel Carmichael) | | Subject: RE: Datawarehousing help| | Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people
Re: Datawarehousing help
Hello Dennis SAS has progressed a little in the last years and now offer a complete DW solution, including ETL tools. You can use their tools also to populate and query oracle. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Saturday, May 04, 2002 2:48 AM Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Friday, May 03, 2002 5:08 PM To: Multiple recipients of list ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc. help? Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: DENNIS
RE: Datawarehousing help
Yechiel - I did not mean to imply that SAS had not improved since the '80s. They would be out of business otherwise. And of course every DW vendor is full solution vendor. Just read their brochures if you don't believe me. My point was that if you understand a company's roots, then often a lot of their quirks start to make sense. My point was that SAS has a VERY strong mathematical foundation, which may help set your understandings. The features you have listed Rachel can get off their brochures. Can you provide any more ideas? My guess is that they might be very strongly positioned to perform data mining. Can you confirm that? Dennis Williams DBA Lifetouch, Inc. [EMAIL PROTECTED] -Original Message- Sent: Sunday, May 05, 2002 6:23 AM To: Multiple recipients of list ORACLE-L Hello Dennis SAS has progressed a little in the last years and now offer a complete DW solution, including ETL tools. You can use their tools also to populate and query oracle. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Saturday, May 04, 2002 2:48 AM Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Friday, May 03, 2002 5:08 PM To: Multiple recipients of list ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected,
Datawarehousing help
Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc. help? Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: Datawarehousing help
Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Friday, May 03, 2002 5:08 PM To: Multiple recipients of list ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc. help? Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: DENNIS WILLIAMS INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of
RE: Datawarehousing help
Excellent dude. -Original Message- WILLIAMS Sent: Friday, May 03, 2002 5:48 PM To: Multiple recipients of list ORACLE-L Rachel - I always find it helpful to understand something if I know the origins. I worked with SAS several years ago. At that time it was a statistical analysis package. A scientist or engineer could load a set of test data into it and perform various arithmetic and statistical analyses. Today most of that can be done with Oracle or MS Excel. My point is that I would expect it to be heavily biased toward mathematical capabilities. Like Data Mining, which is all statistics. Learn what that term means. To learn Data Warehousing, I would encourage you to just do some Googling and find good tutorials. An excellent newslist is dwlist. Instructions: For help with list commands, send a message to mailto:[EMAIL PROTECTED] with the word help in the body of the message. The magazine http://www.intelligententerprise.com/ has some excellent information. I would search for Ralph Kimball. He is one of the leading figures in the DW arena. Look for some of his earliest columns on the magazine site. He also answers questions on dwlist from time to time. The main change you need yourself is to forget normalization. DBAs that can't get past that point don't last long in the DW field. In the early days the DW people would patiently explain the reasons to a DBA, but today there are enough DBAs that have made the leap that a hard-headed normalization bigot just isn't tolerated. It is much easier to just ask for a replacement DBA. The reason normalization isn't adhered to in DW is that users will be creating their own queries and they can't understand 10-table joins with outer joins, etc. A DW is usually loaded and then queried. Our DW is loaded each weekend and then queried all week. So a DW is deliberately denormalized and contains redundant data for ease of use. OLTP databases have no concept of time. A DW is all about time. To reconstruct what the situation is at various points of time, the DW has loads of historical data. For example, marketing people need to be able to reconstruct the amount of business they did with a customer over a period of time last year and compare it with the same period this year. So between denormalization and tons of detailed historical data, DWs are normally BIG! Fortunately they are usually read-only. For Oracle, you want Enterprise Edition with the partitioning option. And study Oracle Materialized Views. In schema, a DW is usually a central fact table and 4-6 dimension tables. Less than 4 dimensions and you don't need a DW. More than 6 and marketing people can't understand the model. Normally the fact table is much larger than the others, but not always. One of Wal-Mart's dimension tables is each person in the U.S. Just size each of those tables, and you've got your size. Growth is easy to predict. Ralph Kimball warns that often people will get the grain wrong. They will size it for data summarized at the weekly level, then after it is built they will realize that isn't going to cut it and need a daily level. You must start almost from scratch and get 7 times the disk capacity. That is the fun side of being a DW DBA. Your cynical instincts will still serve you well, just get them away from normalization and worry about getting the grain right. Okay, I've rambled along here too long. Hope that gets you off on the right foot. -Original Message- [mailto:[EMAIL PROTECTED]] Sent: Friday, May 03, 2002 5:08 PM To: Multiple recipients of list ORACLE-L Okay, my background is OLTP, but we are looking at a data warehousing project here any and all help appreciated! Specifically: 1) does anyone have any experience with a product called SAS Datawarehousing Administrator (or SAS)? 2) how do I go about doing rough estimates of sizing needs, assuming I will get rough numbers of information being collected, growth rates, length of history to keep, etc. help? Rachel -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: DENNIS WILLIAMS INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists
