Bob Friesenhahn wrote:
On Wed, 24 Jun 2009, Marcelo Leal wrote:
Hello Bob,
I think that is related with my post about "zio_taskq_threads and TXG
sync ":
( http://www.opensolaris.org/jive/thread.jspa?threadID=105703&tstart=0 )
Roch did say that this is on top of the performance problems, and in
the same email i did talk about the change from 5s to 30s, what i
think makes this problem worst, if this txg sync interval be "fixed".
The problem is that basing disk writes on a simple timeout and
available memory does not work. It is easy for an application to
write considerable amounts of new data in 30 seconds, or even 5
seconds. If the application blocks while the data is being comitted,
then the application is not performing any useful function during that
time.
Current ZFS write behavior make it not very useful for the creative
media industries even though otherwise it should be a perfect fit
since hundreds of terrabytes of working disk (or even petabytes) are
normal for this industry. For example, when data is captured to disk
from film via a datacine (real time = 24 files/second and 6MB to 50MB
per file), or captured to disk from a high-definition video camera,
there is little margin for error and blocking on writes will result in
missed frames or other malfunction. Current ZFS write behavior is
based on timing and the amount of system memory and it does not seem
that throwing more storage hardware at the problem solves anything at
all.
I wonder whether a filesystem property "streamed" might be appropriate?
This could act as hint to ZFS that the data is sequential and should be
streamed direct to disk.
--
Ian.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
