[EMAIL PROTECTED] wrote:
Hi,
How come that the time granularity of the backup processing chain
does not get finer as the systems get faster ?
What do you understand by time granularity?
I see a fifo as a method to smoothen out peaks and gaps in a
input function and to bring the output nearer to the input
average.
The intensity of peaks and gaps resp. the deviation from the
average can be characterized by the time span in which one
may expect that those irregularities compensate each other.
The product of this time span and the average speed determines
the size which is needed for an effective buffer.
This time span is what i mean with "time granularity".
If the overall system gets faster, i would expect this
characteristic timespan to get shorter. But it seems to
stay within the range of several seconds.
Since the average speed grew and the time staid more or
less the same, the fifo size had to grow.
The only logical explanation is that the characteristics
of the input function have changed while the system
became faster.
I think the problem is that they haven't changed... disks are larger,
CPUs faster, latency less, but as drive sizes have jumped by two orders
of magnitude (4GB=>400GB for ~$150US) seek times have not gotten faster
by more than about 2:1 over what was used almost a decade ago. So people
have far more small files, needing far more seeks to get the data, and
the ability to get to those files is simply not up to the job.
Faster hardware isn't cost effective, I have servers using 15k RPM
drives with LVD320 interface and ~4ms seek, and each drive cost more
than a complete discount computer! RAID helps, overlapping file open and
data read (or batching them) helps, but in truth the low cost of large
disk and better filesystems which don't get unusably slow with thousands
of small files have caused many people to HAVE a ton of small files,
which really hurt when doing backup, or grepping through all of them, or
similar.
Other views which lead me to the same result:
My considerations about the benefits and effectivity of a fifo
always dealt with relative speeds of input and output. Never
with absolute speed.
Thus one would expect that if both input and output speed
increase in the same proportion, then the effectivity should
stay the same. But it obviously doesn't.
A simple thought experiment:
Imagine a movie of an old backup run which is shown at double
speed. The report messages about buffer size and buffer fill would
stay exactly the same.
If everything would have gotten faster by technical progress then
we would have exactly that highspeed movie situation. But we haven't.
But oneproblem is that current drives have less internal RAM compared to
older drives if you take the drive speed into account.
True. But were the drive buffers sufficient 4 years ago ?
If they weren't very effective in the past then their relative
diminishment would not be significant now.
I'm still riddling.
What effect did change the shape of our input functions ?
You have my thoughts, I have a limited number of tests to support my
conclusions, feel free to present data supporting a better (more easily
improved) scenario.
--
bill davidsen <[EMAIL PROTECTED]>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
