This is a very rough cut, but if you divide data+id bytes by 2048, you'll be in the ballpark.
so... 46Mb + 5Mb = 51Mb/2048=24902 I'd pad that a bit (you don't need tightly packed files), possibly as much as 50% & maybe go up to a MIN MOD of a prime near 36K. I'm assuming that disk space isn't a constraint as we're dealing with a few Mb. If you don't know what the data size will be, my admittedly subjective formula would be to divide the anticipated number of records by 10 & go with a number in that range, assuming no other issues (like the # of id's is going to be 40 Billion & it's a 32-bit file). In this case, you'd end up with an oversized file (MIN MOD around 72K) , but you can always adjust that if it's a problem. I don't mind oversizing nearly as much as undersizing.> Subject: [U2][UV] Dynamic File MINIMUM.MODULUS Calculation> Date: Thu, 29 Nov 2007 10:13:53 -0500> From: [EMAIL PROTECTED]> To: u2-users@listserver.u2ug.org> > I continue to struggle with accurate calculation of MINIMUM.MODULUS> values for dynamic files. Static hashed files do not seem to be such a> challenge for me. > > Here is an example where, once again, I was woefully shy...> > -----------------------------------------------------------------------> File name .................. CLM.RUN.STRIP.PROVDATA> Pathname ................... CLM.RUN.STRIP.PROVDATA> File type .................. DYNAMIC> Hashing Algorithm .......... GENERAL> No. of groups (modulus) .... 19618 current ( minimum 7867, 0 empty,> 8869 overflowed, 4612 badly> )> Number of records .......... 719305> Large record size .......... 75 bytes> Number of large records .... 6614> Group size ................. 2048 bytes> Load factors ............... 80% (split), 50% (merge) and 80% (actual)> Total size ................. 69978112 bytes> Total size of record data .. 46331730 bytes> Total size of record IDs ... 5114218 bytes> Unused space ............... 18528068 bytes> Total space for records .... 69974016 bytes> > File name .................. CLM.RUN.STRIP.PROVDATA> Number per group ( total of 19618 groups> )> Average Minimum Maximum StdDev> Group buffers .............. 1.74 1 15 1.19> Records .................... 36.67 10 68 12.15> Large records .............. 0.34 1 4 0.58> Data bytes ................. 2361.69 350 30359 2270.12> Record ID bytes ............ 260.69 70 491 86.71> Unused bytes ............... 944.44 24 2068 650.69> Total bytes ................ 3566.83 2048 30720 0.00> > > Number per record ( total of 719305> records )> Average Minimum Maximum StdDev> Data bytes ................. 64.41 34 28700 353.25> Record ID bytes ............ 7.11 2 20 1.14> Total bytes ................ 71.52 36 28720 354.13> -----------------------------------------------------------------------> > I cannot seem to figure out the relationship of the current modulus to> the record sizes/counts/group size factors. When I do the math I always> come up short.> > Can someone offer some suggestions for accurate calculation of> MINIMUM.MODULUS?> > Thanks.> > Perry Taylor> ZirMed, Inc.> > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.> -------> u2-users mailing list> u2-users@listserver.u2ug.org> To unsubscribe please visit http://listserver.u2ug.org/ _________________________________________________________________ Connect and share in new ways with Windows Live. http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007 ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
