It was a one-shot request. Building the index would take more time than the current process. If the request for dating the data were there, that approach would be used.
Thanks ----- Original Message ----- From: "Baakkonen, Rodney" <[EMAIL PROTECTED]> To: <u2-users@listserver.u2ug.org> Sent: Thursday, October 06, 2005 9:05 AM Subject: RE: [U2] Fw: More U2 programming hints > You could always build an index by date. > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Mark Johnson > Sent: Thursday, October 06, 2005 7:36 AM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] Fw: More U2 programming hints > > > I'll add another observation. > One file on a client's system (D3, proper mod) contains over 7 million > records. It's 1.25 GB. The client wanted to know what the oldest record was. > > I don't dare "SORT BY DATE" so I wrote a small basic program to BASIC SELECT > and keep the lowest number date field. Out of pure habit I used BASIC SELECT > as I know that at TCL a COUNT of this file takes around 20 minutes. So I can > imagine a EXECUTE SELECT would have to wait at least 20 minutes before > evaluating the first record. > > I don't know everything about select buffers, but something tells me that > the 20 minutes (from COUNT) would be much longer as it has to prepare the > active list of 7 million elements. > > As before, I can't imagine the EXECUTE SELECT making up for the lost time > going through the file from beginning to end *and* preparing an isolated > entity (active list) to READNEXT from. > > Please no flames on my exposed word called 'habit'. We all have habits, both > good and bad and only when proven that my habits are bad can I consider > changing them. Actual proof, not guesses and/or first-training. > > Thanks > Mark Johnson > ----- Original Message ----- > From: "Allen E. Elwood" <[EMAIL PROTECTED]> > To: <u2-users@listserver.u2ug.org> > Sent: Tuesday, October 04, 2005 5:45 PM > Subject: RE: [U2] Fw: More U2 programming hints > > > > This is the way it was explained to me way back in '88. The internal > select > > is slower on the whole file, but immediate in response. It works the same > > as LIST. If I list a file with 2,000,000 records I get immediate > response. > > > > If I want to process an entire file, then external select is slower on > > response, i.e. I have to wait for 2 million records to be selected before > > processing begins, but is quicker in processing all records. > > > > The internal is slower due to the system having to stop what it's doing, > > find the next group, break out the individual ID's from that group, and > then > > return it to the program - over and over again as it makes it's way > through > > the file. > > > > hth! > > > > Allen > > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] Behalf Of Stevenson, > > Charles > > Sent: Tuesday, October 04, 2005 14:24 > > To: u2-users@listserver.u2ug.org > > Cc: Louis Windsor > > Subject: RE: [U2] Fw: More U2 programming hints > > > > > > This is a bit disconcerting. > > BASIC SELECT should be faster than EXECUTE "SELECT..." > > Maybe the smart people can weigh in on this: > > > > > From: Louis Windsor > > > > > > A few years ago we used the BASIC SELECT FILE as opposed to > > > the EXECUTE "SELECT FILE". > > > > > > We updated UniVerse (don't ask from what version to what > > > version as I don't remember) and overnight ALL our programs > > > ran five or six times longer. > > > > Completely contrary to my experience and counter-intuitive, too. > > > > > We were told (by VMark) that the BASIC SELECT now selected > > > each group but it could be optioned to work the "old" way. > > > > Hmmm, do I vaguely, hazily remember something about that? Maybe on this > > list? Maybe in release notes? No uvconfig option jumps out at me. > > I don't think flavor would matter, or $OPTIONS [-]VAR.SELECT. > > $OPTIONS FSELECT would slow the BASIC SELECT down to approximately the > > same as EXECUTE "SELECT...", but not make it slower. > > Louis, do you, perchance, use $OPTIONS FSELECT? Maybe buried in a > > $include file common to every program? > > > > > I wrote a conversion program to change ALL BASIC SELECTs to > > > executed SELECTs in the source and recompiled and that is the > > > way we have done it ever since. > > > > > > I don't know if things are different now but we have grown to > > > prefer EXECUTEd selects as selection criteria can be included. > > > > Louis, can you run a simple benchmark and see if this is still true? > > Or show us an example of your own? > > > > INTERNAL: > > OPEN "[really big file]" TO F ELSE STOP > > CRT 'I1', TIMEDATE(), SYSTEM(9) > > SELECT F > > CRT 'I2', TIMEDATE(), SYSTEM(9) > > LOOP WHILE READNEXT ID > > READ REC FROM F, ID ELSE NULL > > REPEAT > > CRT 'I3', TIMEDATE(), SYSTEM(9) > > > > EXECUTED: > > OPEN "[really big file]" TO F ELSE STOP > > CRT 'E1', TIMEDATE(), SYSTEM(9) > > EXECUTE "SELECT [really big file]" > > CRT 'E2', TIMEDATE(), SYSTEM(9) > > LOOP WHILE READNEXT ID > > READ REC FROM F, ID ELSE NULL > > REPEAT > > CRT 'E3', TIMEDATE(), SYSTEM(9) > > > > (Run each a couple times, to allow for i/o differences in loading data > > buffer cache.) > > > > There should be virtually no elapsed time between I1:I2 above, but long > > elapsed time between E1:E2. > > I expect I2:I3 to approximately equal E2:E3. > > > > > > Let me explain why this is counter-intuitive. > > > > Normally, the BASIC SELECT statement itself does not actually do any > > select on the file. It merely sets things up behind the scenes so that > > subsequent READNEXTs each get the next id from the file opened to > > F.FILE, ("next" meaning as stored on disk). > > UV keeps track of where it is in the file, unbeknownst to you. Sorta > > like it keeps track of where it is for REMOVE or attribute-level > > EXTRACTs. > > > > > > > > Exceptions to internal being faster than executed: > > > > 1.SSELECT FILEVAR (i.e., 2 S's, SortSelect). > > You gotta read the whole file First to sort the keys. > > (and it's an alpha-type sort, even for numeric keys.) > > > > 2. $OPTIONS FESLECT > > Makes SELECT FILEVAR populate @SELECTED and to do so means traversing > > the file. > > > > 3. Louis Windsor. Poor bloke, they're out to get him. > > > > > > Chuck Stevenson > > ------- > > u2-users mailing list > > u2-users@listserver.u2ug.org > > To unsubscribe please visit http://listserver.u2ug.org/ > > ------- > > u2-users mailing list > > u2-users@listserver.u2ug.org > > To unsubscribe please visit http://listserver.u2ug.org/ > ------- > u2-users mailing list > u2-users@listserver.u2ug.org > To unsubscribe please visit http://listserver.u2ug.org/ > ------- > u2-users mailing list > u2-users@listserver.u2ug.org > To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
