Chris,
I second the thought that, because of the splitting and merging of groups, it
can be a waste of effort to overwork the "sizing" of a dynamic file.
One problem with your TEST_FILE below is that the Large Record Size is spec'ed
at less than 50% of the group size. Each record that is larger than the "large
record size" is given at least one full sized buffer in the overflow file, so a
record of 2037 bytes, in your example, would occupy 4096 bytes of space. The
ID and pointer is left in the primary data group. It appears that your records
average 250 bytes, so this probably is not a large factor, but that would also
suggest that you stick to a GROUP.SIZE of 1 (2048 bytes) rather than 2. Btw,
each of your 576 large records probably counts towards the "overflowed badly"
column, though, from an access point of view, the group might be in optimal
shape.
The effective modulo of a dynamic file is based on the space used by the
not(large records), but the " Total size of record data" includes the full
buffer size of the overflow records, I believe, and so should not be used to
compute the total size of your data. For record sizes like you have, I would
compute the total of the ids+records, add about 10% for overhead, divide by the
group size (2048, if you use the default), multiply by 1.25 (allow for the 80%
splitting factor), and then set the minimum modulus to the next larger prime
number.
In the example below, you can see 50 large records in a single group of a
dynamic file, but only the id's are in the primary buffer. If you do the math,
you will find that each 1001 record is using up a 4096 byte overflow buffer.
File name .................. BIGD
Pathname ................... BIGD
File type .................. DYNAMIC
Hashing Algorithm .......... GENERAL
No. of groups (modulus) .... 1 current ( minimum 1, 0 empty,
1 overflowed, 1 badly )
Number of records .......... 50
Large record size .......... 1000 bytes
Number of large records .... 50
Group size ................. 4096 bytes
Load factors ............... 80% (split), 50% (merge) and 30% (actual)
Total size ................. 217088 bytes
Total size of record data .. 205466 bytes
Total size of record IDs ... 534 bytes
Unused space ............... 2896 bytes
Total space for records .... 208896 bytes
LIST BIGD TOTAL EVAL "LEN(@ID)" TOTAL EVAL "LEN(@RECORD)" DET.SUP 18:03:29 07-
02-12 PAGE 1
LEN(@ID).. LEN(@RECORD)
========== ============
134 50050
50 records listed.
Note that if I stuck to the defaults and used sequential ids, I would have
saved more than 1/2 of the disk space, but still have used 150% of the total
id+record size.
File name .................. BIGD
Pathname ................... BIGD
File type .................. DYNAMIC
Hashing Algorithm .......... GENERAL
No. of groups (modulus) .... 31 current ( minimum 1, 3 empty,
4 overflowed, 0 badly )
Number of records .......... 50
Large record size .......... 1628 bytes
Number of large records .... 0
Group size ................. 2048 bytes
Load factors ............... 80% (split), 50% (merge) and 80% (actual)
Total size ................. 79872 bytes
Total size of record data .. 50709 bytes
Total size of record IDs ... 91 bytes
Unused space ............... 24976 bytes
Total space for records .... 75776 bytes
Rick Nuckolls
Lynden Inc
-----Original Message-----
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Austin
Sent: Monday, July 02, 2012 2:07 PM
To: u2-users@listserver.u2ug.org
Subject: Re: [U2] RESIZE - dynamic files
The dynamic file I'm working with is below. What do 'overflowed' and 'badly'
refer to under MODULUS? Is the goal of the RESIZE to eliminate that
overflow? Any ideas what I should change to achieve this?
File name .................. TEST_FILE
Pathname ................... TEST_FILE
File type .................. DYNAMIC
File style and revision .... 32BIT Revision 12
Hashing Algorithm .......... GENERAL
No. of groups (modulus) .... 82850 current ( minimum 24, 104 empty,
26225 overflowed, 1441 badly )
Number of records .......... 1157122
Large record size .......... 2036 bytes
Number of large records .... 576
Group size ................. 4096 bytes
Load factors ............... 80% (split), 50% (merge) and 80% (actual)
Total size ................. 449605632 bytes
Total size of record data .. 258687736 bytes
Total size of record IDs ... 19283300 bytes
Unused space ............... 171626404 bytes
Total space for records .... 449597440 bytes
Thanks,
Chris
> From: cjausti...@hotmail.com
> To: u2-users@listserver.u2ug.org
> Date: Mon, 2 Jul 2012 14:55:21 -0500
> Subject: [U2] RESIZE - dynamic files
>
>
> I was wondering if anyone had instructions on RESIZE with a dynamic file? For
> example I have a file called 'TEST_FILE'
> with the following:
>
> 01 ANALYZE.FILE TEST_FILE
> File name .................. TEST_FILE
> Pathname ................... TEST_FILE
> File type .................. DYNAMIC
> File style and revision .... 32BIT Revision 12
> Hashing Algorithm .......... GENERAL
> No. of groups (modulus) .... 83261 current ( minimum 31 )
> Large record size .......... 3267 bytes
> Group size ................. 4096 bytes
> Load factors ............... 80% (split), 50% (merge) and 80% (actual)
> Total size ................. 450613248 bytes
>
> How
> do you calculate what the modulus and separation should be? I can't use
> HASH.HELP on a type 30 file to see the recommended settings
> so I was wondering how best you figure out the file RESIZE.
>
> Thanks,
>
> Chris
> _______________________________________________
> U2-Users mailing list
> U2-Users@listserver.u2ug.org
> http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
