I did not originally notice the OP stated they were a Unidata platform. My distributed 
file comments were related to UniVerse. However, when designing a UniVerse distributed 
file, it is wise to pick the number of part files that will keep each part file size 
*well below* the 2GB limit. Based on how uniform of a distribution your part file 
algorithm yields, and the total amount of data that needs to be stored, I'd set the 
goal of each part file to be about 1GB (or less) in size. By doing this, you'd have 
another gigabyte (or more) of available space in each part file to accommodate growth. 
If in doubt, I'd recommend using a larger number of part files rather than fewer of 
them for a given distributed file. 

For administration ease, I like to use distributed files where I can resize 
(static/hashed files) each part file individually on an as-need basis. It's much 
easier to find weekend time to resize one or more 1GB files (or smaller), than to find 
a single big enough weekend time slot to resize a single 64bit file that contains 30GB 
of data. Additionally, the various individual part files can be spread over multiple 
disk spindles, so disk I/O can be optimized.
HTH,
ken
===
In a message dated 2/10/2004 12:24:18 PM Eastern Standard Time, [EMAIL PROTECTED] 
writes:
> 
>   Subj:  Re: 2 gig limits
>   Date:  2/10/2004 12:24:18 PM Eastern Standard Time
>   From:  [EMAIL PROTECTED]
>   To:  U2 Users Discussion List <[EMAIL PROTECTED]>
>   Cc:  [EMAIL PROTECTED]
>   Reply-To:  U2 Users Discussion List <[EMAIL PROTECTED]>
>   Sent from the Internet (Details
> 
> Dave, 
> 
> If those 2GB files are static files then there is a serious problem in
> your future.  If they are dynamic then they can grow beyond 2GB.
> FILE.STAT 'filename' at the colon prompt will tell you what the file is.

-- 
u2-users mailing list
[EMAIL PROTECTED]
http://www.oliver.com/mailman/listinfo/u2-users

Reply via email to