Re: Growisofs input cache --> Patch
Joerg Schilling wrote: Bill Davidsen wrote: In Run 1 there were several buffer underruns which slowed the DVD recorders down. In Run 2 the buffer was always at 100% (except for the end of course) :-). This seems reasonable, what were the performance numbers for the other system activity? I'm surprised at the underruns, cdrecord has internal fifo, and I thought you did, too. With a hacked cdrecord (around a50) the burn ran almost eight seconds slower, regardless of burn size, and never dropped below 92% full at the drive, and 70% or so in the fifo. Hacked how? Late reply... hacked to use O_DIRECT and some simple monitoring of data count, etc. Clearly writing data with O_DIRECT can be 3-5% slower than using system buffers, but the performance of the other applications was better, no video pauses, all response felt good, generally a better overall behavior. -- Bill Davidsen "Woe unto the statesman who makes war without a reason that will still be valid when the war is over..." Otto von Bismark -- To UNSUBSCRIBE, email to cdwrite-requ...@other.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@other.debian.org
Re: Growisofs input cache --> Patch
Bill Davidsen <[EMAIL PROTECTED]> wrote: > > In Run 1 there were several buffer underruns which slowed the DVD recorders > > down. In Run 2 the buffer was always at 100% (except for the end of > > course) :-). > > > > This seems reasonable, what were the performance numbers for the other > system activity? I'm surprised at the underruns, cdrecord has internal > fifo, and I thought you did, too. With a hacked cdrecord (around a50) > the burn ran almost eight seconds slower, regardless of burn size, and > never dropped below 92% full at the drive, and 70% or so in the fifo. Hacked how? Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Growisofs input cache --> Patch
On Tuesday 09 December 2008 23:56:47 Andy Polyakov wrote: > > In the mean time I created a patch for growisofs which I hope will be > > included in the future (see attachment). > > *If* such option will be added, it would go as -use-the-force-luke > option, not straight one. A. I would even write a patch to use the Star-Wars-option if you would integrate it. But aside from the discussion about O_DIRECT, I would really like to see growisofs using standard command line parameters like every other Unix application. Btw. the same is true for cdrecord... Also I do not see a reason why not to document the Star-Wars-options like 'tty', for example. It would have helped me much when developping turbojet... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Growisofs input cache --> Patch
On Tuesday 09 December 2008 21:48:25 Bill Davidsen wrote: > David Weisgerber wrote: > > Tested: 4x2742 MB > > > > Run 1 (w/o -no-directio): 5:37 min (for the slowest DVD recorder) > > Mem: 2060284k total, 593336k used, 1466948k free,15972k buffers > > > > Run 2 (w -no-directio): 4:38 min (for the slowest DVD recorder) > > Mem: 2060284k total, 1877136k used, 183148k free, 224k buffers > > --- > > > > In Run 1 there were several buffer underruns which slowed the DVD > > recorders down. In Run 2 the buffer was always at 100% (except for the > > end of course) :-). > > This seems reasonable, what were the performance numbers for the other > system activity? I'm surprised at the underruns, cdrecord has internal > fifo, and I thought you did, too. With a hacked cdrecord (around a50) > the burn ran almost eight seconds slower, regardless of burn size, and > never dropped below 92% full at the drive, and 70% or so in the fifo. The underruns are not suprising. I am burning directly from a NFS share via Gigabit Ethernet and the server has a consumer class 1 TB hard drive. Even if I only run with 8x speed, it consumes 4 * 8 * 1.3 MB/s = 41.6 MB/s... The first run needed 337s, this means it had an average data transfer rate of 8.1 MB/s for each burner. This means the 4 burners altogether sucked 32.5 MB/s in average. That's a good value for a consumer class network in my opinion... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Growisofs input cache --> Patch
> In the mean time I created a patch for growisofs which I hope will be > included > in the future (see attachment). *If* such option will be added, it would go as -use-the-force-luke option, not straight one. A. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Growisofs input cache --> Patch
David Weisgerber wrote: On Saturday 06 December 2008 12:35:14 Thomas Schmitt wrote: Hi, David Weisgerber wrote: So I will create a patch which adds such a switch to growisofs Andy Polyakov wrote earlier: There is a way to open image without O_DIRECT and without modifying source code: growisofs -Z /dev/cdrom=/dev/fd/0 < image.iso. Ta-dah... In my role as a frontend programmer for growisofs i would first try whether "Ta-dah" is a viable workaround. Otherwise I would need to distribute a patched version of growisofs with turbojet... If you rely on a growisofs novelty then you have to urge the users to install newest dvd+rw-tools. "Ta-dah" - if possible - would allow a much better decoupling of your software and the system installed tools. As my software is not so widely spread at the moment, I'd like to go the clean way. In addition, I think that using the pipe will create another buffer which I want to avoid. This problem is all about performance. In the mean time I created a patch for growisofs which I hope will be included in the future (see attachment). I have run some tests with it which showed me *very* good results: --- Tested: 4x2742 MB Run 1 (w/o -no-directio): 5:37 min (for the slowest DVD recorder) Mem: 2060284k total, 593336k used, 1466948k free,15972k buffers Run 2 (w -no-directio): 4:38 min (for the slowest DVD recorder) Mem: 2060284k total, 1877136k used, 183148k free, 224k buffers --- In Run 1 there were several buffer underruns which slowed the DVD recorders down. In Run 2 the buffer was always at 100% (except for the end of course) :-). This seems reasonable, what were the performance numbers for the other system activity? I'm surprised at the underruns, cdrecord has internal fifo, and I thought you did, too. With a hacked cdrecord (around a50) the burn ran almost eight seconds slower, regardless of burn size, and never dropped below 92% full at the drive, and 70% or so in the fifo. -- Bill Davidsen <[EMAIL PROTECTED]> "Woe unto the statesman who makes war without a reason that will still be valid when the war is over..." Otto von Bismark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Growisofs input cache --> Patch
On Saturday 06 December 2008 12:35:14 Thomas Schmitt wrote:
> Hi,
>
> David Weisgerber wrote:
> > So I will create a patch which adds such a switch to growisofs
>
> Andy Polyakov wrote earlier:
> > There is a way to open image without O_DIRECT and without modifying
> > source code: growisofs -Z /dev/cdrom=/dev/fd/0 < image.iso. Ta-dah...
>
> In my role as a frontend programmer for
> growisofs i would first try whether "Ta-dah"
> is a viable workaround.
>
> > Otherwise I would need to
> > distribute a patched version of growisofs with turbojet...
>
> If you rely on a growisofs novelty then
> you have to urge the users to install
> newest dvd+rw-tools.
> "Ta-dah" - if possible - would allow a
> much better decoupling of your software
> and the system installed tools.
As my software is not so widely spread at the moment, I'd like to go the clean
way. In addition, I think that using the pipe will create another buffer
which I want to avoid. This problem is all about performance.
In the mean time I created a patch for growisofs which I hope will be included
in the future (see attachment). I have run some tests with it which showed me
*very* good results:
---
Tested: 4x2742 MB
Run 1 (w/o -no-directio): 5:37 min (for the slowest DVD recorder)
Mem: 2060284k total, 593336k used, 1466948k free,15972k buffers
Run 2 (w -no-directio): 4:38 min (for the slowest DVD recorder)
Mem: 2060284k total, 1877136k used, 183148k free, 224k buffers
---
In Run 1 there were several buffer underruns which slowed the DVD recorders
down. In Run 2 the buffer was always at 100% (except for the end of
course) :-).
diff -crB dvd+rw-tools-7.1/growisofs.1 dvd+rw-tools-7.1-new/growisofs.1
*** dvd+rw-tools-7.1/growisofs.1 2008-12-04 17:25:48.0 +0100
--- dvd+rw-tools-7.1-new/growisofs.1 2008-12-08 12:33:34.0 +0100
***
*** 7,12
--- 7,13
[\fB\-dvd\-compat\fP]
[\fB\-overburn\fP]
[\fB\-speed=1\fP]
+ [\fB\-no\-directio\fP]
\-[\fBZ|M\fP]
.I /dev/dvd
.I
***
*** 68,73
--- 69,82
recordings are commonly performed at ~1/2 of advertised speed, because
of defect management being in effect.
.TP
+ .BI \-no\-directio
+ This option will turn on the file system cache. On Linux 2.6 systems,
+ growisofs will use the DIRECT_IO flag in order to bypass the file
+ system cache. This will enhance the overall system performance if an
+ ISO image is recorded only once. In situations where one image is
+ recorded multiple times use this option to let the system cache the
+ file in the memory.
+ .TP
.BI
More options can be found in the manpage for \fBmkisofs\fP.
diff -crB dvd+rw-tools-7.1/growisofs.c dvd+rw-tools-7.1-new/growisofs.c
*** dvd+rw-tools-7.1/growisofs.c 2008-12-04 17:25:48.0 +0100
--- dvd+rw-tools-7.1-new/growisofs.c 2008-12-08 15:47:43.0 +0100
***
*** 597,603
static char *in_device=NULL,*out_device=NULL,*argv0;
intdvd_compat=0, test_write=0, no_reload=0, mmc_profile=0,
!wrvfy=0, no_opc=0, spare=0;
double speed_factor=0.0;
char *ioctl_device;
int_argc;
--- 596,602
static char *in_device=NULL,*out_device=NULL,*argv0;
intdvd_compat=0, test_write=0, no_reload=0, mmc_profile=0,
!wrvfy=0, no_opc=0, spare=0, no_directio=0;
double speed_factor=0.0;
char *ioctl_device;
int_argc;
***
*** 3056,3061
--- 3055,3064
"is not permitted.\n",argv[0]),
exit(FATAL_START(EINVAL));
}
+ else if (!strcmp(opt,"-no-directio")) {
+ no_directio = 1;
+ continue;
+ }
else if (!strncmp(opt,"-use-the-force-luke",19))
{ char *s=strchr (opt,'='),*o;
***
*** 3151,3157
else if (argv[i][1] == '?' || !strcmp(opt,"-help"))
{ PRINT_VERSION (argv[0]);
printf ("- usage: %s [-dvd-compat] [-overburn] [-speed=1] \\\n"
! " -[ZM] /dev/dvd \n",argv[0]);
printf (" for see 'mkisofs %s'\n",opt);
exit (FATAL_START(EINVAL));
}
--- 3154,3160
else if (argv[i][1] == '?' || !strcmp(opt,"-help"))
{ PRINT_VERSION (argv[0]);
printf ("- usage: %s [-dvd-compat] [-overburn] [-speed=1] \\\n"
! " [-no-directio] -[ZM] /dev/dvd \n",argv[0]);
printf (" for see 'mkisofs %s'\n",opt);
exit (FATAL_START(EINVAL));
}
***
*** 3211,3217
if (sscanf(in_image,"/dev/fd/%u",&imgfd) == 1)
imgfd = dup (imgfd); /* validate descriptor */
#ifdef OPEN_DIRECTIO
! else if ((imgfd = OPEN_DIRECTIO(in_image,O_RDONLY))<0)
#else
else
#endif
--- 3214,3220
if (sscanf(in_image,"/dev/fd/%u",&imgfd) == 1)
imgfd = dup (imgfd); /* validate descriptor */
#ifdef OPEN_DIRECTIO
! else if (1 == no_directio || (imgfd = OPEN_DIRECTIO(in_image,O_RDONLY))<0)
#else
else
#endif
