The resize SHOULDN'T (I don't think) have had any effect. Reducing the split percent will have made all of your groups smaller. That will have had various side-effects - reading and writing individual records in overflowed groups will be quicker, fewer records will be in overflowed groups, and splitting overflowed groups will be quicker because they're smaller.
That's still quite a result though - your file will have grown about 15% and it's lopped 90% off the run time! Cheers, Wol -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin Smith Sent: 29 November 2007 16:10 To: u2-users@listserver.u2ug.org Subject: RE: [U2] Badly overflowed Dynamic files I don't know if changing my the split load from 80% to 70% or if running a "RESIZE [filename] DYNAMIC" command (or both) did the trick, but one of the batches that has been taking up to two hours to complete finished the other night in under 10 minutes. Thanks a ton. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Anthony Youngman Sent: Tuesday, November 27, 2007 10:22 AM To: u2-users@listserver.u2ug.org Subject: RE: [U2] Badly overflowed Dynamic files Overflowed groups take longer to read, so you want to reduce the number of such groups. Is the size of this file roughly constant, or does it just slowly grow? If it's roughly constant, try setting a MINIMUM.MODULO at around 900,000. It'll hopefully break up a lot of your badly overflowed groups. Or change it away from a dynamic file to a hash-type that better suits your keys. If it grows steadily, try REDUCING your split figure - maybe 70%. Again, that will increase the number of groups, and hopefully reduce your number of overflowed groups. Lastly, I was going to suggest looking at your block size and large records, but I notice you don't have any. Cheers, Wol ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
