Theory:> As I recall, there is a 'distributed' or 'part-file?' (the wrong term) mechanism whereby one can divide a large file into several physical files based on specified attributes. I'm not sure if it has to be part of the record ID.
Could it be that this configuration might be efficient? Caveat:> Of course this would be depend on how many total records, how many branches, what kind of processing happens against this file, record size, disk spindles/speeds/caching etc. Challenge:> Why don't some of the fips (file internals people - an acronym) suggest a few basic guidelines, formula, algorithyms, or else propose a bench-marking scheme to determine what configuration would be most efficient. Postulate:> I believe this is a common situation and one where non-U2 peoples miss some of the benefits available which are unique to U2 databases. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brian Leach Sent: Tuesday, August 10, 2004 5:39 AM To: [EMAIL PROTECTED] Subject: RE: [U2] Question on Unidata tables access speeds No he shouldn't. If that selection is pulling 2.5 million records, then that means at least 1.25 million entries against each branch - indexing 1+ million IDs against a single key is not exactly going to be efficient. Brian -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin King Sent: 09 August 2004 18:07 To: [EMAIL PROTECTED] Subject: RE: [U2] Question on Unidata tables access speeds AFAIK = As Far As I Know You should be indexed on BRANCH and/or LEASE_TYPE, and based on the data a cascading select may a better way to get at the information quicker. I'm no expert on the index optimizer, so cascading selects may not be necessary, but it sure seems logical that if you index even on BRANCH and then select like this (below) you might get better responses: SELECT filename WITH BRANCH = "0616" "0844" SELECT filename WITH LEASE_TYPE # "RA" Also, are BRANCH and LEASE_TYPE actual data fields or translates from another file? If these are translates it could seriously impact your select performance. -Kevin [EMAIL PROTECTED] http://www.PrecisOnline.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Monday, August 09, 2004 10:30 AM To: [EMAIL PROTECTED] Subject: Re: [U2] Question on Unidata tables access speeds I am doing select with BRANCH = "0616" "0844" AND LEASE_TYPE # "RA" Also What is AFAIK, how do we set it up for multiple connections ? thanks "Brian Leach" <[EMAIL PROTECTED] To: [EMAIL PROTECTED] o.uk> cc: Sent by: Subject: Re: [U2] Question on Unidata tables access speeds [EMAIL PROTECTED] er.u2ug.org 08/09/2004 10:18 AM Please respond to u2-users That might explain why one query takes so long, but not the time for the four queries (unless he is out of resource). UniData can run multiple queries concurrently, but AFAIK only one per connection. So I guess the question is how DataStage is connecting to UniData, as it looks like DataStage is scheduling these sequentially. Brian On Mon, 9 Aug 2004 10:35:01 -0400 <[EMAIL PROTECTED]> wrote: >You may want to try FILE.STAT and see if your file is properly sized. >If not, then use >the RESIZE command. > >Just my $0.0000001 worth. > >Grant > >-----Original Message----- >From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] >Sent: Monday, August 09, 2004 10:09 AM >To: [EMAIL PROTECTED] >Subject: RE: [U2] Question on Unidata tables access speeds > > >It is indexed on the key column, even then if I run the run the job (query) >4 times in parallel, I am not getting advange of the parallel query. Is >Unidata not good for the parallel query ?, or is there any tuning method ? >thanks > > > > > > "Kevin King" > > > <[EMAIL PROTECTED] To: > [EMAIL PROTECTED] > > cc: > > Sent by: Subject: RE: [U2] Question on >Unidata tables access speeds > [EMAIL PROTECTED] > > > er.u2ug.org > > > > > > > 08/08/2004 06:09 PM > > > Please respond to > > > u2-users > > > > > > > > > > >Is anything indexed? Sounds like you could probably benefit from >adding some indexes on fields used for the selection. That would make a >BIG difference. > >-----Original Message----- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On Behalf Of >[EMAIL PROTECTED] >Sent: Sunday, August 08, 2004 3:57 PM >To: [EMAIL PROTECTED] >Subject: [U2] Question on Unidata tables access speeds > > Hi , >Project data warehouse, Sources are Unidata tables, tool Datastage. >Performance issue, >while dumping in to hash file(Allowing stage write cash). It is >searching >5.5 million >rows and fetching 2.6 million rows and taking 2.5 hours. There is only >one key column in the ouput. When I remove select caluse(I mean select >all rows) and put that filter in my tool, it is ok, but not great. >After that When I run parallel jobs , I mean select the data by 4 jobs >for different ranges(all at the same time), it is taking >4 times than normal time . In other words, I am not getting performance >like Oracle or other data bases. I am new to Unidata, is there any >method where I can get good access speeds on Unidata.Any help will be >highly appreciated. > >thanks >pbr > > >--------------------------------------------------------------------- - >-------- >Electronic Privacy Notice. This e-mail, and any attachments, contains >information that is, or may be, covered by the Electronic >Communications Privacy Act, 18 U.S.C. 2510-2521, and is also >confidential and proprietary in nature. If you are not the intended >recipient, please be advised that you are legally prohibited from >retaining, using, copying, distributing, or otherwise disclosing this >information in any manner. Instead, please reply to the sender that you >have received this communication in error, and then immediately delete >it. Thank you in advance for your cooperation. >===================================================================== = >======== >------- >u2-users mailing list >[EMAIL PROTECTED] >To unsubscribe please visit http://listserver.u2ug.org/ >------- >u2-users mailing list >[EMAIL PROTECTED] >To unsubscribe please visit http://listserver.u2ug.org/ > > > > > >--------------------------------------------------------------------- -- >------- >Electronic Privacy Notice. This e-mail, and any attachments, contains information that >is, or may be, covered by the Electronic Communications Privacy Act, 18 U.S.C. >2510-2521, and is also confidential and proprietary in nature. If you >are not the >intended recipient, please be advised that you are legally prohibited >from retaining, >using, copying, distributing, or otherwise disclosing this information >in any manner. >Instead, please reply to the sender that you have received this communication in error, >and then immediately delete it. Thank you in advance for your cooperation. >===================================================================== == >======= >------- >u2-users mailing list >[EMAIL PROTECTED] >To unsubscribe please visit http://listserver.u2ug.org/ >------- >u2-users mailing list >[EMAIL PROTECTED] >To unsubscribe please visit http://listserver.u2ug.org/ > >_____________________________________________________________________ __ >_ This email was checked by MessageLabs SkyScan before entering >Microgen. ______________________________________________________________________ __ This email was checked on leaving Microgen for viruses, similar malicious code and inappropriate content by MessageLabs SkyScan. DISCLAIMER This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information. In the event of any technical difficulty with this email, please contact the sender or [EMAIL PROTECTED] Microgen Information Management Solutions http://www.microgen.co.uk ------- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ ---------------------------------------------------------------------- -------- Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply to the sender that you have received this communication in error, and then immediately delete it. Thank you in advance for your cooperation. ====================================================================== ======== ------- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ ________________________________________________________________________ This email was checked by MessageLabs SkyScan before entering Microgen. ________________________________________________________________________ This email was checked on leaving Microgen for viruses, similar malicious code and inappropriate content by MessageLabs SkyScan. DISCLAIMER This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information. In the event of any technical difficulty with this email, please contact the sender or [EMAIL PROTECTED] Microgen Information Management Solutions http://www.microgen.co.uk ------- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list [EMAIL PROTECTED] To unsubscribe please visit http://listserver.u2ug.org/
