We can dump(8) more than 2TB
After writing 2TB (INT_MAX * TP_BSIZE), dump(8) stops reporting progress because the blockswritten variable has wrapped around to negative. It needs to be a larger type like the tapesize variable; see optr.c:timeest(). This only affects the terminal chatter. The actual dump functionality is fine. ok? Index: dump.h === RCS file: /cvs/src/sbin/dump/dump.h,v retrieving revision 1.19 diff -u -p -r1.19 dump.h --- dump.h 24 May 2014 21:49:09 - 1.19 +++ dump.h 12 Jun 2014 11:28:35 - @@ -73,7 +73,7 @@ int etapes; /* estimated number of tape intnonodump; /* if set, do not honor UF_NODUMP user flags */ intnotify; /* notify operator flag */ -intblockswritten; /* number of blocks written on current tape */ +off_t blockswritten; /* number of blocks written on current tape */ inttapeno; /* current tape number */ time_t tstart_writing; /* when started writing the first tape block */ long xferrate; /* averaged transfer rate of all volumes */ Index: main.c === RCS file: /cvs/src/sbin/dump/main.c,v retrieving revision 1.50 diff -u -p -r1.50 main.c --- main.c 31 May 2014 08:28:13 - 1.50 +++ main.c 12 Jun 2014 11:29:07 - @@ -59,7 +59,7 @@ #include pathnames.h intnotify = 0; /* notify operator flag */ -intblockswritten = 0; /* number of blocks written on current tape */ +off_t blockswritten = 0; /* number of blocks written on current tape */ inttapeno = 0; /* current tape number */ intdensity = 0;/* density in bytes/0.1 */ intntrec = NTREC; /* # tape blocks in each tape record */ -- Christian naddy Weisgerber na...@mips.inka.de
Re: We can dump(8) more than 2TB
Ted Unangst: -intblockswritten; /* number of blocks written on current tape */ +off_t blockswritten; /* number of blocks written on current tape */ time_t tstart_writing; /* when started writing the first tape block */ longxferrate; /* averaged transfer rate of all volumes */ I'm not sure off_t is the right semantic type. Perhaps just use int64_t? I picked off_t for consistency with tapesize, but we could use int64_t for both. Not sure about the other off_t's there either. (And maybe change xferrate? I'm not sure why that needs to change size based on machine, and I can probably assemble an i386 system capable of wrapping it. int64_t again.) I can't get more than 66 MB/s out of my system, so good luck breaking 2 TB/s now, but yes, we can proactively fix that. -- Christian naddy Weisgerber na...@mips.inka.de
Re: We can dump(8) more than 2TB
On 12 June 2014 15:59, Christian Weisgerber na...@mips.inka.de wrote: Ted Unangst: -intblockswritten; /* number of blocks written on current tape */ +off_t blockswritten; /* number of blocks written on current tape */ time_t tstart_writing; /* when started writing the first tape block */ longxferrate; /* averaged transfer rate of all volumes */ I'm not sure off_t is the right semantic type. Perhaps just use int64_t? I picked off_t for consistency with tapesize, but we could use int64_t for both. Not sure about the other off_t's there either. (And maybe change xferrate? I'm not sure why that needs to change size based on machine, and I can probably assemble an i386 system capable of wrapping it. int64_t again.) I can't get more than 66 MB/s out of my system, so good luck breaking 2 TB/s now, but yes, we can proactively fix that. -- Christian naddy Weisgerber na...@mips.inka.de I would go with int64_t for blockswritten and anything else that is counting anything other that characters. Ken
Re: We can dump(8) more than 2TB
Date: Thu, 12 Jun 2014 15:14:29 -0400 From: Ted Unangst t...@tedunangst.com On Thu, Jun 12, 2014 at 21:07, Christian Weisgerber wrote: After writing 2TB (INT_MAX * TP_BSIZE), dump(8) stops reporting progress because the blockswritten variable has wrapped around to negative. It needs to be a larger type like the tapesize variable; see optr.c:timeest(). This only affects the terminal chatter. The actual dump functionality is fine. int notify; /* notify operator flag */ -intblockswritten; /* number of blocks written on current tape */ +off_t blockswritten; /* number of blocks written on current tape */ int tapeno; /* current tape number */ time_t tstart_writing; /* when started writing the first tape block */ longxferrate; /* averaged transfer rate of all volumes */ I'm not sure off_t is the right semantic type. Perhaps just use int64_t? int64_t is what we use in struct stat. That said, there is existing off_t abuse in dump.
Re: We can dump(8) more than 2TB
Date: Thu, 12 Jun 2014 15:14:29 -0400 From: Ted Unangst t...@tedunangst.com On Thu, Jun 12, 2014 at 21:07, Christian Weisgerber wrote: After writing 2TB (INT_MAX * TP_BSIZE), dump(8) stops reporting progress because the blockswritten variable has wrapped around to negative. It needs to be a larger type like the tapesize variable; see optr.c:timeest(). This only affects the terminal chatter. The actual dump functionality is fine. int notify; /* notify operator flag */ -int blockswritten; /* number of blocks written on current tape */ +off_tblockswritten; /* number of blocks written on current tape */ int tapeno; /* current tape number */ time_ttstart_writing; /* when started writing the first tape block */ long xferrate; /* averaged transfer rate of all volumes */ I'm not sure off_t is the right semantic type. Perhaps just use int64_t? int64_t is what we use in struct stat. That said, there is existing off_t abuse in dump. I seperately pointed out to naddy that blocksperfile is also mishandled; it will also overflow. It is printed with a %ld somewhere, so fix that too.
Re: We can dump(8) more than 2TB
New diff.
* Move all off_t variables that don't look like file sizes to
int64_t.
* Switch blockswritten to int64_t, so it won't wrap at 2TB.
* Same for blocksthisvol (deraadt@).
* Switch xferrate (tedu@) and blocksperfile from long to uint64_t.
* Since blocksperfile can be set with -B, move numarg() from long
to long long and don't mark small integer constant arguments with
'L'.
Index: dump.h
===
RCS file: /cvs/src/sbin/dump/dump.h,v
retrieving revision 1.19
diff -u -p -r1.19 dump.h
--- dump.h 24 May 2014 21:49:09 - 1.19
+++ dump.h 12 Jun 2014 21:02:32 -
@@ -65,15 +65,15 @@ int pipeout;/* true = output to standa
ino_t curino; /* current inumber; used globally */
intnewtape;/* new tape flag */
intdensity;/* density in 0.1 units */
-off_t tapesize; /* estimated tape size, blocks */
-off_t tsize; /* tape size in 0.1 units */
+int64_ttapesize; /* estimated tape size, blocks */
+int64_ttsize; /* tape size in 0.1 units */
intunlimited; /* if set, write to end of medium */
-off_t asize; /* number of 0.1 units written on current tape */
+int64_tasize; /* number of 0.1 units written on current tape
*/
intetapes; /* estimated number of tapes */
intnonodump; /* if set, do not honor UF_NODUMP user flags */
intnotify; /* notify operator flag */
-intblockswritten; /* number of blocks written on current tape */
+int64_tblockswritten; /* number of blocks written on current tape */
inttapeno; /* current tape number */
time_t tstart_writing; /* when started writing the first tape block */
long xferrate; /* averaged transfer rate of all volumes */
@@ -96,10 +96,11 @@ voidtimeest(void);
/* mapping routines */
union dinode;
-off_t blockest(union dinode *dp);
-void mapfileino(ino_t, off_t *, int *);
-intmapfiles(ino_t maxino, off_t *tapesize, char *disk, char * const *dirv);
-intmapdirs(ino_t maxino, off_t *tapesize);
+int64_tblockest(union dinode *dp);
+void mapfileino(ino_t, int64_t *, int *);
+intmapfiles(ino_t maxino, int64_t *tapesize, char *disk,
+ char * const *dirv);
+intmapdirs(ino_t maxino, int64_t *tapesize);
/* file dumping routines */
void ufs1_blksout(int32_t *blkp, int frags, ino_t ino);
Index: main.c
===
RCS file: /cvs/src/sbin/dump/main.c,v
retrieving revision 1.50
diff -u -p -r1.50 main.c
--- main.c 31 May 2014 08:28:13 - 1.50
+++ main.c 12 Jun 2014 21:19:18 -
@@ -59,12 +59,12 @@
#include pathnames.h
intnotify = 0; /* notify operator flag */
-intblockswritten = 0; /* number of blocks written on current tape */
+int64_tblockswritten = 0; /* number of blocks written on current
tape */
inttapeno = 0; /* current tape number */
intdensity = 0;/* density in bytes/0.1 */
intntrec = NTREC; /* # tape blocks in each tape record */
intcartridge = 0; /* Assume non-cartridge tape */
-long blocksperfile; /* output blocks per file */
+int64_tblocksperfile; /* output blocks per file */
char *host = NULL; /* remote host (if any) */
intmaxbsize = 64*1024; /* XXX MAXBSIZE from sys/param.h */
@@ -75,7 +75,7 @@ struct disklabel lab;
*/
static int sblock_try[] = SBLOCKSEARCH;
-static long numarg(char *, long, long);
+static long long numarg(char *, long long, long long);
static void obsolete(int *, char **[]);
static void usage(void);
@@ -121,11 +121,11 @@ main(int argc, char *argv[])
break;
case 'B': /* blocks per output file */
- blocksperfile = numarg(blocks per file, 1L, 0L);
+ blocksperfile = numarg(blocks per file, 1, 0);
break;
case 'b': /* blocks per tape write */
- ntrec = numarg(blocks per write, 1L, 1000L);
+ ntrec = numarg(blocks per write, 1, 1000);
if (ntrec maxbsize/1024) {
msg(Please choose a blocksize = %dKB\n,
maxbsize/1024);
@@ -139,7 +139,7 @@ main(int argc, char *argv[])
break;
case 'd': /* density, in bits per inch */
- density = numarg(density, 10L, 327670L) / 10;
+ density = numarg(density, 10, 327670) / 10;
if (density = 625 !bflag)
ntrec = HIGHDENSITYTREC;
break;
@@ -149,7 +149,7 @@ main(int argc, char *argv[])
break;
case 'h':
- honorlevel
Re: We can dump(8) more than 2TB
On Thu, Jun 12, 2014 at 21:30, Christian Weisgerber wrote: New diff. * Move all off_t variables that don't look like file sizes to int64_t. * Switch blockswritten to int64_t, so it won't wrap at 2TB. * Same for blocksthisvol (deraadt@). * Switch xferrate (tedu@) and blocksperfile from long to uint64_t. * Since blocksperfile can be set with -B, move numarg() from long to long long and don't mark small integer constant arguments with 'L'. I myself am not a dump fiend, but it all looks reasonable.
