Certainly the dimensioned array over dynamic would be beneficial in this situation whenever possible. Unfortunately, this only applies to your custom code Dave, as the SB+ stuff and the stuff written by Prelude is going to continue using the dynamic array access. In your custom code, you might be able to eek out some gains by separating the multivalued fields you're using out to single attributes, update the single attributes, and then replace the attributes in the record from the single attribute variables.
Whether the file is dynamic or static will certainly play a part, with dynamic files almost always performing worse than static hashed files. In my opinion [flame suit on] the only reason to use dynamic files in Unidata is to breach the 2G limit. There is no appreciable performance reason for using dynamic files, unless of course you simply want the application to perform worse than it would with a static hashed file. Especially with split load at 10% - you're going to split the group when it's 10% full of keys. That said, I can't really think of anything better except to not use a dynamic file. -K On Fri, Jun 1, 2012 at 12:47 PM, David L. Wasylenko <d...@pickpro.com> wrote: > File size looks small. > Very generic sizing rules: > Pick a modulo of the smaller of these two numbers: > > * The item count within the file (731,825) > * The total byte count divided by the group size (2,197,829,057 / > 16,384 = 134,144) > > Then find a prime number above that, or at least not divisible by 2, 3, 5, > 7... > > I get 134,147 for a reasonable modulo. > The current mod is 67,907 which is about half the calculated modulo. > > 50% of your data is in hash-overflow, causing a linear search within each > group at least 50% of the time. > This can be a large part of the issue. > > A dimensioned array instead of a dynamic array would help your type of > processing as well. > > > > ... david ... > > David L. Wasylenko > President, Pick Professionals, Inc > w) 314 558 1482 > d...@pickpro.com > > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org [mailto: > u2-users-boun...@listserver.u2ug.org] On Behalf Of Dave Laansma > Sent: Friday, June 01, 2012 12:09 PM > To: U2 Users List > Subject: Re: [U2] Monthly Multivalue Inserts and Speed > > FILE.STAT results: > > MOD OV HTY ITEMS BYTES MNI/G MXI/G MNB/I MXB/I > 67801 0 64 731825 2197829057 0 27 19 14300 > > ----- ------- ------- > 67801 731825 2197829057 > > Sincerely, > David Laansma > IT Manager > Hubbard Supply Co. > Direct: 810-342-7143 > Office: 810-234-8681 > Fax: 810-234-6142 > www.hubbardsupply.com > "Delivering Products, Services and Innovative Solutions" > > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of David L. > Wasylenko > Sent: Friday, June 01, 2012 12:07 PM > To: U2 Users List > Subject: Re: [U2] Monthly Multivalue Inserts and Speed > > What's the record count and total byte count? > > ... david ... > > David L. Wasylenko > President, Pick Professionals, Inc > w) 314 558 1482 > d...@pickpro.com > > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dave Laansma > Sent: Friday, June 01, 2012 9:41 AM > To: U2 Users List > Subject: Re: [U2] Monthly Multivalue Inserts and Speed > > Results of the ANALYZE.FILE: > > Dynamic File name = WHSE.STAT > Number of groups in file (modulo) = 67801 > Minimum groups of file = 67801 > Hash type = 0, blocksize = 16384 > Split load = 10, Merge load = 5 > Split/Merge type = KEYONLY > > Sincerely, > David Laansma > IT Manager > Hubbard Supply Co. > Direct: 810-342-7143 > Office: 810-234-8681 > Fax: 810-234-6142 > www.hubbardsupply.com > "Delivering Products, Services and Innovative Solutions" > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of David L. > Wasylenko > Sent: Friday, June 01, 2012 10:32 AM > To: U2 Users List > Subject: Re: [U2] Monthly Multivalue Inserts and Speed > > 3k isn't large. > #1 thing to look at is file size. > > Change the file-type to dynamic. It will not give the best performance > compared to monitoring file size of a static file type, but over the > long-term, avoids sizing issues that are the #1 reason for bad performance. > > Can you perform an "ANALYZE.FILE {filename}" and post the result? > > ... david ... > > David L. Wasylenko > President, Pick Professionals, Inc > w) 314 558 1482 > d...@pickpro.com > > > -----Original Message----- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dave Laansma > Sent: Friday, June 01, 2012 9:30 AM > To: U2-Users@listserver.u2ug.org > Subject: [U2] Monthly Multivalue Inserts and Speed > > I have a file that is taking a very long time to update, seemingly longer > and longer each month. Here is briefly how the attributes are organized, a > relatively simple example: > > > > Attribute 1 is multivalued with our G/L Period, i.e. 12-01 vm 12-02 vm > 12-03 etc. > > Attributes 2 through 30 are dependent on attribute 1 and contain a variety > of dollar amounts that are accumulated for the respective period in > attribute 1. > > > > At the end of each month, a new G/L Period is inserted into the > multivalued position of attribute 1 as well as each of the respective > multivalued positions in attributes 2 through 30. Nothing complicated. > > > > My question is this and is aimed largely at the technicians at Rocket ... > > > > If, at the beginning of the year, I were to insert each of the 12-periods > into their respective multivalues in attribute 1 and insert respective > <null> multivalues into attributes 2 through 30, thus performing all of the > 'inserts' at once, will the processing each month run faster as numbers are > simply accumulated into attributes 2 through > 30 and no inserting of new multivalues will have to be performed? > > > > A couple nuggets of information: > > > > This is only an example. In the actual file, these records contain many > other attributes and the records overall are relatively large, averaging > lengths of about 3,000 according to FILE.STAT > > These are not my files so reorganizing them is not an option. > > > > The heart of this question is, as a (numeric) number is accumulated in an > attribute, multivalue or subvalue, what causes the record to 'stretch' in > length, thus presumably impacting performance? > > > > I would like to think that numeric information is stored in a more > 'compact' method than strings, thus using less space within the database > ... but what I 'like' and what 'is' are often quite different. > > > > Thanks! > > > > Sincerely, > > David Laansma > > IT Manager > > Hubbard Supply Co. > > Direct: 810-342-7143 > > Office: 810-234-8681 > > Fax: 810-234-6142 > > www.hubbardsupply.com <http://www.hubbardsupply.com> > > "Delivering Products, Services and Innovative Solutions" > > > > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
