Re: 9 track tapes and block sizes
On Monday, October 5, 2020 at 8:25, Chuck Guzis via cctech wrote:
> So increasing the mask for block length wouldn't seem to be a problem,
> assuming that SIMH could support it.
SIMH could be written to support it (I'm the maintainer-by-default of the
tape handling portion of SIMH). I don't know of any existing simulators
that depend on the length being exactly 24 bits. The tape handling library
defines an integer type to hold tape record lengths (it's a 32-bit unsigned
value), and simulators are supposed to declare tape record length variables
of this type. So extending the record length field to 28 bits shouldn't
cause problems.
The current library assumes that anything that's not one of the three
defined metadata markers (EOF, EOM, or erase gap) is a tape record length,
so there's no error recovery for undefined markers. This could be changed,
however, so that unrecognized markers would be ignored, i.e., transparently
skipped when reading or writing, as long as those markers contained the
length of the record to be skipped.
For example, if the upper four bits were dedicated to the marker type field
and the length field expanded to 28 bits, then we could have something
like:
Type Length Meaning
---
0 0 tape mark
0 >0 "good" data record
1 >0 \
... | undefined records (reserved for SIMH)
7 >0 /
8 >0 "bad" data record
9 >0 \
... | user-defined records
D >0 /
E anyuser-defined single-word marker
F FFF end of medium
F FFE erase gap
Fother undefined single word marker (reserved for SIMH)
The type E single-word marker and record types 9-D (that must contain
record lengths at both ends of the record) would be transparently ignored
by SIMH. These could be used for tape information or other uses devised by
the community.
-- Dave
Re: 9 track tapes and block sizes
On Mon, Oct 5, 2020 at 9:25 AM Chuck Guzis via cctech wrote: > On 10/4/20 10:51 PM, J. David Bryan via cctech wrote: > > On Sunday, October 4, 2020 at 16:00, Chuck Guzis via cctalk wrote: > > > >> A 16MB tape block is impossibly large in any case. > > > > The HP 3000 mag tape diagnostic attempts to write a single record from > BOT > > to EOT, which unfortunately fails under simulation due to the 16 MB > > limitation. In hindsight, it would have been better to accommodate > record > > lengths corresponding to the highest density and longest reel length, > which > > I think would need 28 bits. Four bits for metadata identifier would > still > > have been be good enough, and one of those should have been dedicated to > > "private data" that would appear invisible to programs running under > > simulation (and could be used to include information about the tape > image > > with the tape image). > > That's interesting. Looking at my own code for .TAP files, I see the > following: > > #define TAP_FILEMARK0x0 // 0 = filemark > #define TAP_EOM 0x // -1 = end of medium > #define TAP_ERASE_GAP 0xfffe // -2 = erase gap > #define TAP_ERROR_FLAG 0x8000 // error flag bit > #define TAP_LENGTH_MASK 0x00ff // mask for length > > So increasing the mask for block length wouldn't seem to be a problem, > assuming that SIMH could support it. There may be other high-order bit > meanings assigned, but I've not run into them. > The other way to handle it, should we not be able to steal bits which should be plan A, is to have a metadata type that says 'append this to the prior record' which would remove the limit entirely. Warner
Re: 9 track tapes and block sizes
On 10/5/20 2:06 PM, Warner Losh wrote: > The other way to handle it, should we not be able to steal bits which > should be plan A, is to have a metadata type that says 'append this to > the prior record' which would remove the limit entirely. There's a good idea! Instead of putting headers on an all-tape record, just archive the raw data and call it that. Even though I can handle a long block this way (assuming my SDHC card can keep up, I've never run across this. My controller does double-buffering while reading. --Chuck
Re: 9 track tapes and block sizes
On 10/4/20 10:51 PM, J. David Bryan via cctech wrote: > On Sunday, October 4, 2020 at 16:00, Chuck Guzis via cctalk wrote: > >> A 16MB tape block is impossibly large in any case. > > The HP 3000 mag tape diagnostic attempts to write a single record from BOT > to EOT, which unfortunately fails under simulation due to the 16 MB > limitation. In hindsight, it would have been better to accommodate record > lengths corresponding to the highest density and longest reel length, which > I think would need 28 bits. Four bits for metadata identifier would still > have been be good enough, and one of those should have been dedicated to > "private data" that would appear invisible to programs running under > simulation (and could be used to include information about the tape image > with the tape image). That's interesting. Looking at my own code for .TAP files, I see the following: #define TAP_FILEMARK0x0 // 0 = filemark #define TAP_EOM 0x // -1 = end of medium #define TAP_ERASE_GAP 0xfffe // -2 = erase gap #define TAP_ERROR_FLAG 0x8000 // error flag bit #define TAP_LENGTH_MASK 0x00ff // mask for length So increasing the mask for block length wouldn't seem to be a problem, assuming that SIMH could support it. There may be other high-order bit meanings assigned, but I've not run into them. --Chuck
Re: 9 track tapes and block sizes
> On Oct 5, 2020, at 11:50 AM, Jon Elson via cctalk > wrote: > > On 10/04/2020 11:30 PM, Chuck Guzis via cctalk wrote: >> The problem arose if the program either failed to keep up (e.g. competing >> programs demanding the CPU) or if an ECS transfer occurred. Where I got >> involved was with a 4-CPU configuration sharing 4MW of ECS, using the ECS >> for swap space. ECS transfers completely swamp the read/write pyramid, so >> the PP gets locked out and can't service the drive. Recovery was doing a >> backward-read to the BOT or preceding IRG. Lather, rinse, repeat. It was fun >> to watch and we never solved that one to anyone's satisfaction. > I can't imagine any way to fix this except to put a buffer memory in the PP > big enough to hold a few records of tape data. Probably played hob with disk > transfers, too, except that wasn't as visible. > > Jon Actually, on a full memory 6000 series system (32 banks) there is more than enough bandwidth to service an ECS transfer and other things too. ECS runs at 10 MWords per second. The bandwidth per bank is 1 MW/s so with 32 banks that only takes 1/3rd of the system bandwidth. I don't know the 6400 memory controller, but the Stunt Box on the 6600 will happily schedule all the memory requests, whether from CPU, PPs, or ECS, and fairly issue them to the banks. It would be interesting to run a simulation for that. paul
Re: 9 track tapes and block sizes
On 10/04/2020 11:30 PM, Chuck Guzis via cctalk wrote: The problem arose if the program either failed to keep up (e.g. competing programs demanding the CPU) or if an ECS transfer occurred. Where I got involved was with a 4-CPU configuration sharing 4MW of ECS, using the ECS for swap space. ECS transfers completely swamp the read/write pyramid, so the PP gets locked out and can't service the drive. Recovery was doing a backward-read to the BOT or preceding IRG. Lather, rinse, repeat. It was fun to watch and we never solved that one to anyone's satisfaction. I can't imagine any way to fix this except to put a buffer memory in the PP big enough to hold a few records of tape data. Probably played hob with disk transfers, too, except that wasn't as visible. Jon
Re: 9 track tapes and block sizes
> On Oct 5, 2020, at 12:30 AM, Chuck Guzis via cctalk > wrote: > > On 10/4/20 8:58 PM, jim stephens via cctalk wrote: > >> I had not seen your earlier no tape gap mentions. > > The old CDC 6000 SCOPE 1LT driver. Since SCOPE user programs use > circular buffering, The PP overlay 1LT simply emptied the CM buffer and > wrote it, 12-bit word by 12-bit word to the drive controller, so long as > the CPU program kept the buffer filled. At least on NOS, that's not how long block handling works. It uses 1MT (the main tape driver) with 1LT as a helper in a separate PP. Each reads a portion of the long block into its PP memory, hand off to the other PP to continue reading while in parallel the first PP transfers the piece it read into central memory. This kind of ping-pong algorithm is needed because the block transfer operations (both I/O and memory copy) are blocking. They can be split up into individual word transfers but that isn't how the problem is handled in this case, and possibly that wouldn't be fast enough. paul
Re: 9 track tapes and block sizes
On Sunday, October 4, 2020 at 16:00, Chuck Guzis via cctalk wrote: > A 16MB tape block is impossibly large in any case. The HP 3000 mag tape diagnostic attempts to write a single record from BOT to EOT, which unfortunately fails under simulation due to the 16 MB limitation. In hindsight, it would have been better to accommodate record lengths corresponding to the highest density and longest reel length, which I think would need 28 bits. Four bits for metadata identifier would still have been be good enough, and one of those should have been dedicated to "private data" that would appear invisible to programs running under simulation (and could be used to include information about the tape image with the tape image). -- Dave
Re: 9 track tapes and block sizes
On 10/4/20 8:58 PM, jim stephens via cctalk wrote: > I had not seen your earlier no tape gap mentions. The old CDC 6000 SCOPE 1LT driver. Since SCOPE user programs use circular buffering, The PP overlay 1LT simply emptied the CM buffer and wrote it, 12-bit word by 12-bit word to the drive controller, so long as the CPU program kept the buffer filled. The problem arose if the program either failed to keep up (e.g. competing programs demanding the CPU) or if an ECS transfer occurred. Where I got involved was with a 4-CPU configuration sharing 4MW of ECS, using the ECS for swap space. ECS transfers completely swamp the read/write pyramid, so the PP gets locked out and can't service the drive. Recovery was doing a backward-read to the BOT or preceding IRG. Lather, rinse, repeat. It was fun to watch and we never solved that one to anyone's satisfaction. I don't know if 1LT survived in NOS. Long NRZI reads/writes are easy, as all that was needed was to keep a XOR of the frames as they're read or written for the LRC. PE and GCR tapes would be a bit thornier, I'd imagine. --Chuck
Re: 9 track tapes and block sizes
On 10/4/2020 8:47 PM, Chuck Guzis via cctalk wrote: On 10/4/20 8:12 PM, jim stephens via cctalk wrote: On 10/4/2020 4:00 PM, Chuck Guzis via cctalk wrote: On 10/4/20 1:50 PM, J. David Bryan via cctalk wrote: I don't believe that you're missing anything. When I process these files, I mask off the lower 24 bits as the block length. A 16MB tape block is impossibly large in any case. --Chuck Oil logging tapes write 9 tracks with no inter record gaps or file marks till EOT. I'm guessing you've not digitized any of those? A couple of years ago, one of the NASA logging units turned up for not much money. Big flight case. Basically consisted of a Kennedy incremental drive (i.e. stepper) brought out to several BNC connectors. My recollection is that the Kennedy 9000-series unit was NRZI and likely 800 bpi. Unlikely that you could get 16MB on a 10.5" reel with that. Probably unreadable on standard tape drives with limited formatter memory. I've also mentioned CDC 7-track "long tapes". I've certainly seen the 9-track analog telemetry tapes also, but they're a much different kettle of fish. --Chuck The Microdata 1600 with the NRZI interface could write an entire tape of normal "record" data with a Pertec drive. We wrote code to write IBM records on our system and make our 360 happy. One of the debug efforts had a bug in some experimental code that neglected to do the eot order to the controller, and the entire tape was one record. The system did retries on it till the system crashed, then sucked the entire tape into the vacuum columns. We weren't popular that day. The system I mentioned earlier was at company my partners worked at earlier. I had not seen your earlier no tape gap mentions. Thanks Jim
Re: 9 track tapes and block sizes
On 10/4/20 8:12 PM, jim stephens via cctalk wrote: > > > On 10/4/2020 4:00 PM, Chuck Guzis via cctalk wrote: >> On 10/4/20 1:50 PM, J. David Bryan via cctalk wrote: >>> >> I don't believe that you're missing anything. When I process these >> files, I mask off the lower 24 bits as the block length. A 16MB tape >> block is impossibly large in any case. >> >> --Chuck > Oil logging tapes write 9 tracks with no inter record gaps or file marks > till EOT. I'm guessing you've not digitized any of those? A couple of years ago, one of the NASA logging units turned up for not much money. Big flight case. Basically consisted of a Kennedy incremental drive (i.e. stepper) brought out to several BNC connectors. My recollection is that the Kennedy 9000-series unit was NRZI and likely 800 bpi. Unlikely that you could get 16MB on a 10.5" reel with that. Probably unreadable on standard tape drives with limited formatter memory. I've also mentioned CDC 7-track "long tapes". I've certainly seen the 9-track analog telemetry tapes also, but they're a much different kettle of fish. --Chuck
Re: 9 track tapes and block sizes
I'm surprised that they have not turned up since they are useful for wells when trying to enhance output. seismic data recovery from tape has been going on for a long time
Re: 9 track tapes and block sizes
On 10/4/2020 4:00 PM, Chuck Guzis via cctalk wrote: On 10/4/20 1:50 PM, J. David Bryan via cctalk wrote: I don't believe that you're missing anything. When I process these files, I mask off the lower 24 bits as the block length. A 16MB tape block is impossibly large in any case. --Chuck Oil logging tapes write 9 tracks with no inter record gaps or file marks till EOT. I'm guessing you've not digitized any of those? A friend built a custom machine which could be placed on site at the well site. The tapes would be run to the unit, and then the data stream transmitted to a US based location for analysis. Made necessary in Canada as they became very aggressive about transmission of such data. In the 70's and 80s when this was in use it was very difficult to transport half inch tapes across the border w/o being challenged because customs was tipped off that the tapes were all potentially banned material. I'm surprised that they have not turned up since they are useful for wells when trying to enhance output. Thanks Jim
Re: 9 track tapes and block sizes
On 10/4/20 1:50 PM, J. David Bryan via cctalk wrote: > On Friday, October 2, 2020 at 13:51, Al Kossow via cctech wrote: > >> Those would already be broken with Bob's use of large negative numbers >> for physical end of tape and 'bad block is here' (you don't get to know >> how big that bad block was, so that is hell with tapes with >> variable-length records, grumble..) > > I'm not sure I understand. SIMH metadata markers are treated in the tape > library code as unsigned values, so EOM and erase gap are seen as large > unsigned values. The format limits record lengths to 24 bits (so about 16 > MB maximum per record), reserving the upper 8 bits to indicate the type of > the marker, and the bad record marker is the top byte = 0x80. The record > length of a bad block is encoded in the lower 24 bits and indicates how big > the bad block was. > > What am I missing? I don't believe that you're missing anything. When I process these files, I mask off the lower 24 bits as the block length. A 16MB tape block is impossibly large in any case. --Chuck
Re: 9 track tapes and block sizes
On Friday, October 2, 2020 at 13:51, Al Kossow via cctech wrote: > Those would already be broken with Bob's use of large negative numbers > for physical end of tape and 'bad block is here' (you don't get to know > how big that bad block was, so that is hell with tapes with > variable-length records, grumble..) I'm not sure I understand. SIMH metadata markers are treated in the tape library code as unsigned values, so EOM and erase gap are seen as large unsigned values. The format limits record lengths to 24 bits (so about 16 MB maximum per record), reserving the upper 8 bits to indicate the type of the marker, and the bad record marker is the top byte = 0x80. The record length of a bad block is encoded in the lower 24 bits and indicates how big the bad block was. What am I missing? -- Dave
Re: 9 track tapes and block sizes
On Sat, 2020-10-03 at 08:33 -0700, Chuck Guzis via cctalk wrote: > > In particular, consider a government project where several hundred > millions of 1970s dollars were spent by the government, yet almost > nothing other than a few papers survives. Those involved with > intimate > knowledge are inexorably dying off as the community ages out. The > lessons of "what did we learn form all of this?" will be gone > forever. > > Sometimes it seems that we spend as many resources in forgetting as > we > spend trying to remember. > I couldn't agree more...and it's not just governments (at all levels) but companies as well. In the mid-90's I worked on the IBM Microkernel project (was one of the original 6 people who started it). It eventually grew to 100's of people and morf'd into Workplace OS. I still have some of the printed documentation from that project but have long since lost a set of CDs that contained not only the PDFs for those documents (the source was in FrameMaker) but also all of the IBM microkernel source *and* build environment and tools. And that was only a part of the project...there were all of the personality neutral software as well as the various OS personalities (including AIX and OS/2). I seriously doubt if any of that survived in any form because of the way that the project was shutdown. The last estimate of the cost to IBM of the project was over $2,000,000,000 (in 1995 dollars). To my knowledge not much survived. What a waste. TTFN - Guy
Re: 9 track tapes and block sizes
On 10/3/20 3:20 AM, Al Kossow via cctalk wrote: > On 10/2/20 7:38 PM, Chuck Guzis via cctalk wrote: >> In fact, is there any standard for floppy disk metadata container files? >> > > Digtial archivists seem to be using > LoC BagIt, which essentially is a zip file of a directory > > https://en.wikipedia.org/wiki/BagIt > > I'd have to check with our digital archivist to see what metadata files > are included in CHM's archival bags. > > Invention of your own container will be doomed to failure due to lack > of adoption. The Bag-it scheme is interesting in that it can be used to hold data and metadata in an XML-style container, be the object a floppy disk or a spear point, but where are the standards for digital media for *what* must be preserved? It seems to me that professional digital archivists concern themselves with preserving content, but have few resources to determine context. The number of tapes that I've handled that bear only a tape number at best is alarming. Various government warehouses still have acres of the stuff. Labels dry up and fall off and aren't replaced (that goes for floppy media also). Nobody is left to interpret what the bits mean, much less the context in which they were created. In that sense, my hat's off to the "living history" transcribers who preserve context, but not necessarily content. A personal concern as I'm forming stronger relationships with the medical community in my dotage is that vast amounts of information will inevitably be lost in any case. In particular, consider a government project where several hundred millions of 1970s dollars were spent by the government, yet almost nothing other than a few papers survives. Those involved with intimate knowledge are inexorably dying off as the community ages out. The lessons of "what did we learn form all of this?" will be gone forever. Sometimes it seems that we spend as many resources in forgetting as we spend trying to remember. --Chuck
Re: 9 track tapes and block sizes
On 10/3/20 3:20 AM, Al Kossow via cctalk wrote: On 10/2/20 7:38 PM, Chuck Guzis via cctalk wrote: In fact, is there any standard for floppy disk metadata container files? Digtial archivists seem to be using LoC BagIt, which essentially is a zip file of a directory https://en.wikipedia.org/wiki/BagIt I'd have to check with our digital archivist to see what metadata files are included in CHM's archival bags. Invention of your own container will be doomed to failure due to lack of adoption. we use archivamatica for archiving https://www.archivematica.org/en/docs/archivematica-1.6/user-manual/transfer/bags/
Re: 9 track tapes and block sizes
On 10/2/20 7:38 PM, Chuck Guzis via cctalk wrote: In fact, is there any standard for floppy disk metadata container files? Digtial archivists seem to be using LoC BagIt, which essentially is a zip file of a directory https://en.wikipedia.org/wiki/BagIt I'd have to check with our digital archivist to see what metadata files are included in CHM's archival bags. Invention of your own container will be doomed to failure due to lack of adoption.
Re: 9 track tapes and block sizes
On Fri, Oct 2, 2020, 8:38 PM Chuck Guzis via cctech wrote: > In fact, is there any standard for floppy disk metadata container files? > > I'm not aware of any. > Teledisk? >
Re: 9 track tapes and block sizes
In fact, is there any standard for floppy disk metadata container files? I'm not aware of any. --Chuck
Re: 9 track tapes and block sizes
> On 10/2/20 1:42 PM, Eric Smith via cctalk wrote:
>
> >/Of course, that scheme breaks programs that use tape images but don't
> >/>/expect enormous or "negative" record lengths. /
> Those would already be broken with Bob's use of large negative numbers for
> physical end of
> tape and 'bad block is here' (you don't get to know how big that bad block
> was, so that is
> hell with tapes with variable-length records, grumble..)
0x is end-of-medium. I've encountered only a few tapes with this
marker.
0xFFFE is an erase gap. I've never encountered one, perhaps because
I treat all 0xFFnn as EOM, even though they are reserved.
0x is a tapemark.
The above three are dataless.
0x80nn is a bad block (Medium Error) of size nn > 0. Woe to me
when my tape drive returns a zero-length block for those, so sometimes I
cobble up a BAD0 label and surround it with these Medium Errors.
Heroics can ask my tape drive what happened (to distinguish it from a
normal tapemark).
I could suggest that other values for the first byte might be used for
metadata, but I think there are too many readers and existing images to
let that work. I just put metadata in the same directory, but it can
get misplaced easily, and nothing limits a directory to just one tape
image. Not optimal.
>
> But for all the people who get clean reads off their tapes, none of this is
> an issue.
> I wish I was so lucky.
I can't believe that anyone never gets a clean read from any of their
tapes; most of mine have been clean. But not all of them...
--
Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
Nature abhors straight antennas, clean lenses, and empty storage.
"Delete! Delete! OK!" -Dr. Bronner on disk space management
Card-sorting, Joel. -Crow on solitaire
Re: 9 track tapes and block sizes
On Fri, Oct 2, 2020, 9:37 PM Chuck Guzis wrote: > On 10/2/20 8:08 PM, Warner Losh wrote: > > > > > > On Fri, Oct 2, 2020, 8:38 PM Chuck Guzis via cctech > > mailto:cct...@classiccmp.org>> wrote: > > > > In fact, is there any standard for floppy disk metadata container > files? > > > > I'm not aware of any. > > > > > > Teledisk? > > > > > Heh, but no--there's no way to include a photo and other supporting > metadata--just a few lines of commentary. > Oh, that kind of meta data. Derp.i was thinking track, sector number, head info... Warner >
Re: 9 track tapes and block sizes
On 10/2/20 8:08 PM, Warner Losh wrote: > > > On Fri, Oct 2, 2020, 8:38 PM Chuck Guzis via cctech > mailto:cct...@classiccmp.org>> wrote: > > In fact, is there any standard for floppy disk metadata container files? > > I'm not aware of any. > > > Teledisk? > Heh, but no--there's no way to include a photo and other supporting metadata--just a few lines of commentary. --Chuck
Re: 9 track tapes and block sizes
On 10/2/20 1:42 PM, Eric Smith wrote: > On Fri, Oct 2, 2020 at 1:27 PM Chuck Guzis via cctalk > mailto:cctalk@classiccmp.org>> wrote: > > Actually, they're neither. I append the metadata after the EOI marker > on mine. Doesn't seem to bother the emulators. > > > Some programs (but probably very few) that use various so-called .tap > files assume that they can seek to the EOF (of the container file) and > read records backwards (supported by lengths being at both ends of > blocks in the container). Tacking metadata on the end breaks that. I'm > not criticizing, mind you. It's just something that people may need to > be aware of. Metadata records tacked on after the EOI marker are appended with the appropriate preamble-postamble length indicators, so this doesn't break the scheme (much). --Chuck
Re: 9 track tapes and block sizes
On 10/2/20 1:42 PM, Eric Smith via cctalk wrote: Of course, that scheme breaks programs that use tape images but don't expect enormous or "negative" record lengths. Those would already be broken with Bob's use of large negative numbers for physical end of tape and 'bad block is here' (you don't get to know how big that bad block was, so that is hell with tapes with variable-length records, grumble..) But for all the people who get clean reads off their tapes, none of this is an issue. I wish I was so lucky.
Re: 9 track tapes and block sizes
On Fri, Oct 2, 2020 at 1:27 PM Chuck Guzis via cctalk wrote: > Actually, they're neither. I append the metadata after the EOI marker > on mine. Doesn't seem to bother the emulators. > Some programs (but probably very few) that use various so-called .tap files assume that they can seek to the EOF (of the container file) and read records backwards (supported by lengths being at both ends of blocks in the container). Tacking metadata on the end breaks that. I'm not criticizing, mind you. It's just something that people may need to be aware of. The .tap file formats are horrible. Originally there was at least the virtue of simplicity, but because they've diverged we don't even have that. Al wrote: > Bordynuik's at least had some provisions for reporting a bad block, > as I'm sure yours does. > Aeons ago when I was doing stuff with tape images, I made a proposal for representing bad blocks of either known or unknown length. I don't know whether anything other than some of my own unpublished code adopted that particular proposal. IIRC, I proposed using a record length of 0x to designate any kind of "metadata" record, with the real length of the metadata record in the next word in on both ends (to still support backwards reading), and the third word at the beginning only being a tag of what kind of metadata it was, etc. Of course, that scheme breaks programs that use tape images but don't expect enormous or "negative" record lengths. I contributed to the morass by still calling my files .tap files.
Re: 9 track tapes and block sizes
On 10/2/20 12:26 PM, Chuck Guzis via cctalk wrote: This is basically the problem with all of the container file formats that I've seen. They seem to think that the data alone is sufficient to describe an object. Quite often, a simple paper label is more informative than a bunch of bits the joys of tacking bags onto legacy container formats .tpc begat Wilson .tap which begat Supnik .tap and Bordynuik .tap none of which can be identified without heuristics during reading Bordynuik's at least had some provisions for reporting a bad block, as I'm sure yours does.
Re: 9 track tapes and block sizes
On 10/2/20 11:51 AM, Paul Koning wrote: > > >> On Oct 2, 2020, at 1:46 PM, Chuck Guzis via cctalk >> wrote: >> >> On 10/1/20 11:40 PM, Warner Losh via cctalk wrote: >>> On Fri, Oct 2, 2020, 12:05 AM Tom Hunter via cctalk >>> wrote: >>> I have never figured out why Bob Supnik defined the magnetic tape containers (TAP files) with the one byte padding for odd length records. This seems very odd (pun intended). :-) Even on a machine which couldn't write 32 bit numbers (the record lenght) on odd boundaries you could write the 32 bit number as 4 individual bytes. Does anyone know the reason? >> >> On the .TAP files that I provide to customers, I ignore the 16-bit >> granularity and supply odd-length records as appropriate. > > You can certainly create tape container files like that, but those are not > TAP format. Instead, they are E11 format. You should call them by their > proper name. Actually, they're neither. I append the metadata after the EOI marker on mine. Doesn't seem to bother the emulators. This is basically the problem with all of the container file formats that I've seen. They seem to think that the data alone is sufficient to describe an object. Quite often, a simple paper label is more informative than a bunch of bits--indeed, it may represent the Rosetta Stone when it comes down to figuring out what's what. Even the diagnostic options are very weak. For example, on many platforms, it's possible to identify which (vertical) frames contain parity or other errors. There's no option for reporting this level of detail in any of the container files that I've seen. I append a log of the reading process as part of the metadata. Similarly, I've handled tapes where the density changes between files. Where to report this? Most container files don't even make a provision for reporting parity (NRZI tapes). On 7 track, the encoding schemes between even and odd parity can be quite different. --Chuck
Re: 9 track tapes and block sizes
> On Oct 2, 2020, at 1:46 PM, Chuck Guzis via cctalk > wrote: > > On 10/1/20 11:40 PM, Warner Losh via cctalk wrote: >> On Fri, Oct 2, 2020, 12:05 AM Tom Hunter via cctalk >> wrote: >> >>> I have never figured out why Bob Supnik defined the magnetic tape >>> containers (TAP files) with the one byte padding for odd length records. >>> This seems very odd (pun intended). :-) >>> Even on a machine which couldn't write 32 bit numbers (the record lenght) >>> on odd boundaries you could write the 32 bit number as 4 individual bytes. >>> Does anyone know the reason? > > On the .TAP files that I provide to customers, I ignore the 16-bit > granularity and supply odd-length records as appropriate. You can certainly create tape container files like that, but those are not TAP format. Instead, they are E11 format. You should call them by their proper name. paul
Re: 9 track tapes and block sizes
On 10/1/20 11:40 PM, Warner Losh via cctalk wrote: > On Fri, Oct 2, 2020, 12:05 AM Tom Hunter via cctalk > wrote: > >> I have never figured out why Bob Supnik defined the magnetic tape >> containers (TAP files) with the one byte padding for odd length records. >> This seems very odd (pun intended). :-) >> Even on a machine which couldn't write 32 bit numbers (the record lenght) >> on odd boundaries you could write the 32 bit number as 4 individual bytes. >> Does anyone know the reason? On the .TAP files that I provide to customers, I ignore the 16-bit granularity and supply odd-length records as appropriate. My own take is that the original was intended for DEC 16/32 bit hardware. There are other systems that make it impossible to write an odd-length record. More interesting are the 36-bit systems writing 7 track tapes (e.g. Univac 1100), where a tape record has to be a multiple of 18 bits/3 "bytes" long. An ANSI label record, for example, is 81 "bytes". Similar situations exist for 9 track tapes written on nominally 6-bit systems. --Chuck
Re: 9 track tapes and block sizes
On 10/1/20 11:40 PM, Warner Losh wrote:
>
>
> On Fri, Oct 2, 2020, 12:05 AM Tom Hunter via cctalk
> mailto:cctalk@classiccmp.org>> wrote:
>
> I have never figured out why Bob Supnik defined the magnetic tape
> containers (TAP files) with the one byte padding for odd length
> records.
> This seems very odd (pun intended). :-)
>
My theory is that this was a time when the controller interface (either
to the tape unit or to the host or both) was 16 bits wide.
>
> Even on a machine which couldn't write 32 bit numbers (the record
> lenght)
> on odd boundaries you could write the 32 bit number as 4
> individual bytes.
> Does anyone know the reason?
>
>
> RMS did this too if nothing else, it was in the water at Digital.
> But it would have been faster to access than unaligned buffers...
Recently, Al forwarded me a tape image from Chuck Guzis. It was claimed
to be a NOS 1.4 tape, and that's what it turned out to be. However, it
was one of those tapes that requires new code in my tools to read
smoothly. While I routinely see tapes with a single padding byte, this
one had many cases requiring two padding bytes, and some requiring
three! Thus this tape image is not SIMH-compliant. Also, most NOS files
had their own ANSI labels; the files were mostly just text programs
without their own names, so the ANSI labels helped, but this is unusual
for NOS.
Summary:
73 HDR1 labels encountered. +++
73 EOF1/EOV1 labels encountered. +++
29 single-padded blocks +++.
406 double-padded blocks +++.
24 triple-padded blocks +++.
--
Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
Nature abhors straight antennas, clean lenses, and empty storage.
"Delete! Delete! OK!" -Dr. Bronner on disk space management
Card-sorting, Joel. -Crow on solitaire
Re: 9 track tapes and block sizes
On Fri, Oct 2, 2020, 12:05 AM Tom Hunter via cctalk
wrote:
> I have never figured out why Bob Supnik defined the magnetic tape
> containers (TAP files) with the one byte padding for odd length records.
> This seems very odd (pun intended). :-)
> Even on a machine which couldn't write 32 bit numbers (the record lenght)
> on odd boundaries you could write the 32 bit number as 4 individual bytes.
> Does anyone know the reason?
>
RMS did this too if nothing else, it was in the water at Digital. But
it would have been faster to access than unaligned buffers...
Warner
Cheers
> Tom Hunter
>
> On Fri, Sep 18, 2020 at 9:17 AM Jeff Woolsey via cctech <
> cct...@classiccmp.org> wrote:
>
> > > Acoustically, the best tapes were the short-record "stranger" tapes.
> > > All sorts of interesting noise. I could tell from across the room when
> > > someone was running the tape section of the Navy audit tests for COBOL
> > > just by the sounds.
> > >
> > MALET was also pretty good, reading and writing a bunch of blocks that
> > were one frame longer or shorter than the last. Loud rising or falling
> > tone in the noisy computer room.
> >
> > --
> > Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
> > Nature abhors straight antennas, clean lenses, and empty storage.
> > "Delete! Delete! OK!" -Dr. Bronner on disk space management
> > Card-sorting, Joel. -Crow on solitaire
> >
> >
>
Re: 9 track tapes and block sizes
I have never figured out why Bob Supnik defined the magnetic tape
containers (TAP files) with the one byte padding for odd length records.
This seems very odd (pun intended). :-)
Even on a machine which couldn't write 32 bit numbers (the record lenght)
on odd boundaries you could write the 32 bit number as 4 individual bytes.
Does anyone know the reason?
Cheers
Tom Hunter
On Fri, Sep 18, 2020 at 9:17 AM Jeff Woolsey via cctech <
cct...@classiccmp.org> wrote:
> > Acoustically, the best tapes were the short-record "stranger" tapes.
> > All sorts of interesting noise. I could tell from across the room when
> > someone was running the tape section of the Navy audit tests for COBOL
> > just by the sounds.
> >
> MALET was also pretty good, reading and writing a bunch of blocks that
> were one frame longer or shorter than the last. Loud rising or falling
> tone in the noisy computer room.
>
> --
> Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
> Nature abhors straight antennas, clean lenses, and empty storage.
> "Delete! Delete! OK!" -Dr. Bronner on disk space management
> Card-sorting, Joel. -Crow on solitaire
>
>
Re: 9 track tapes and block sizes
Al Kossow wrote: >> I'm kind of maintaining that. > where? Here: https://github.com/brouhaha/tapeutils > and since your fscking around inside of it, have you added the > Bordynuik extensions in the ToTS tape images? No, but I certainly will if yout tell me what it is.
Re: 9 track tapes and block sizes
On 9/18/20 8:44 AM, Al Kossow via cctalk wrote: On 9/18/20 8:17 AM, Lars Brinkhoff wrote: I'm kind of maintaining that. where? Are you feeding the changes back to Eric or have you come up with your own no one knows about it except you version? the issues will be in the tap library from John Wilson and since your fscking around inside of it, have you added the Bordynuik extensions in the ToTS tape images?
Re: 9 track tapes and block sizes
On 9/18/20 8:17 AM, Lars Brinkhoff wrote: I'm kind of maintaining that. where? Are you feeding the changes back to Eric or have you come up with your own no one knows about it except you version? the issues will be in the tap library from John Wilson
Re: 9 track tapes and block sizes
Al Kossow wrote: >> I usually use tapeutils: >> https://github.com/brouhaha/tapeutils > > I should bug Eric about this, but the .tap files that library creates > doesn't have the Supnik SIMH extensions In case Eric doesn't have time to make updates, bug me instead. I'm kind of maintaining that.
Re: 9 track tapes and block sizes
On 9/18/20 5:21 AM, Patrick Finnegan via cctalk wrote: I usually use tapeutils: https://github.com/brouhaha/tapeutils I should bug Eric about this, but the .tap files that library creates doesn't have the Supnik SIMH extensions I use it for all of the SCSI recoveries that I do on Linux machines. It also works with 1/4" cartridges but you want to keep the block size at 512 so that if there is a bad physical block it will only return that one block as bad instead of however many blocks the read attempts to return.
Re: 9 track tapes and block sizes
I usually use tapeutils: https://github.com/brouhaha/tapeutils Pat On Fri, Sep 18, 2020 at 1:40 AM shad via cctalk wrote: > dear all, > thanks for the useful informations! > So now a question comes to mind... > what is the best utility for Linux to be used to read and archive tapes? > > Thanks > Andrea > >
Re: 9 track tapes and block sizes
dear all, thanks for the useful informations! So now a question comes to mind... what is the best utility for Linux to be used to read and archive tapes? Thanks Andrea
Re: 9 track tapes and block sizes
> On 17/09/2020 07:38, Dave Wade G4UGM via cctalk wrote:
> >/The docs for SIMH .TAP files are here:-
> >/>//>/http://simh.trailing-edge.com/docs/simh_magtape.pdf />//>/be careful
> >as there are also non-SIMH .tap formats />//>/In the IBM Mainframe emulation
> >world there is also .AWS, an IBM format />/introduced with its P390
> >Microchannel Mainframe card />/and .HET a Hercules extension to .AWS which
> >allows compression />//
> vtapeutils is quite handy for translating between (or listing) a few of
> the emulated tape formats. It supports AWS, TPC (2 byte record headers)
> and TAP (Simh - 4 byte record headers).
>
> https://sourceforge.net/projects/vtapeutils/
>
> I've used TPC on RSX-11M and VMSTPC on VAX/VMS to read 9-track tapes.
> Then I used vtapeutils to convert from TPC to TAP format for Simh.
>
> Matt
>
I have over 1000 TAP images in my collection, so I thought I'd give this
a whirl. It's handy for a quick summary of what the TAP image is.
However, documentation is a bit light, as is its robustness for corrupt
TAP images (of which I have plenty), although it seems forgiving of
unpadded blocks (unlike another, stricter tool I could mention).
For unlabelled tapes, it lists block count and min and max block size
for each file. Nice. But it does not do that for labelled tapes; instead
they get the labels that enclose each dataset decoded, and the number of
blocks is reported and checked. A warning is printed if the actual
block count differs. Sometimes this warning is incorrect, reporting 0
or a large negative number (-17342528 usually) of blocks. It is usually
correct for those of my tapes that have duplicate blocks (which I don't
expect it to detect/report otherwise). The 0s might be extraneous
EOF1/EOV1 labels. The negatives might be extraneous tapemarks. Or
stuff past EOI.
It would be nice if it printed the name of the TAP file in its report.
Built from source (edited Makefile to add $(CC) ). I didn't try vtapecp.
--
Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
Nature abhors straight antennas, clean lenses, and empty storage.
"Delete! Delete! OK!" -Dr. Bronner on disk space management
Card-sorting, Joel. -Crow on solitaire
Re: 9 track tapes and block sizes
> Acoustically, the best tapes were the short-record "stranger" tapes.
> All sorts of interesting noise. I could tell from across the room when
> someone was running the tape section of the Navy audit tests for COBOL
> just by the sounds.
>
MALET was also pretty good, reading and writing a bunch of blocks that
were one frame longer or shorter than the last. Loud rising or falling
tone in the noisy computer room.
--
Jeff Woolsey {{woolsey,jlw}@jlw,first.last@{gmail,jlw}}.com
Nature abhors straight antennas, clean lenses, and empty storage.
"Delete! Delete! OK!" -Dr. Bronner on disk space management
Card-sorting, Joel. -Crow on solitaire
Re: 9 track tapes and block sizes
> On Sep 17, 2020, at 2:11 PM, Chuck Guzis via cctalk > wrote: > > On 9/17/20 8:56 AM, Jon Elson via cctalk wrote: > >> This is not necessarily true. Many systems can handle "VBS" (Variable >> Block Sequential) tape files. >> But, yes, fixed block size is more common. > > "Hybrid" files are quite common, where all blocks are the same size, but > for the last one. Or, in the case of some PDP 11 tapes, there's a short > record containing file name, etc. followed by the file data in > uniformly-sized blocks, but for the last one in the file. Even there you might find some oddities. A RSTS distribution tape, for recent releases, begins with some DOS-11 formatted data, i.e., 14 byte file header and 512 byte data. That is followed by ANSI labelled data, so 80 byte file labels and data in a VMS-style backupset, 2048 byte blocks I think. paul
Re: 9 track tapes and block sizes
On 9/17/20 8:56 AM, Jon Elson via cctalk wrote: > This is not necessarily true. Many systems can handle "VBS" (Variable > Block Sequential) tape files. > But, yes, fixed block size is more common. "Hybrid" files are quite common, where all blocks are the same size, but for the last one. Or, in the case of some PDP 11 tapes, there's a short record containing file name, etc. followed by the file data in uniformly-sized blocks, but for the last one in the file. Prior to uniform conventions, tape files were anything that the originator could dream up with regards to block sizes. Very often, the correspondence was 1 record = 1 block. Even the convention of 80 character ANSI tape labels ran afoul of some 36 bit systems, where label records were 81 bytes. --Chuck
Re: 9 track tapes and block sizes
On 09/17/2020 12:29 AM, shad via cctalk wrote: Hello, I have a question about 9 track tapes and block sizes. What I know is that tape is subdivided in files by means of marks, and each file is subdivided in blocks of equal size. This is not necessarily true. Many systems can handle "VBS" (Variable Block Sequential) tape files. But, yes, fixed block size is more common. Programs like tar use a specific block size to create files on tape. However files can have different block sizes like bootloader file, installation dumps and root file system copy on 2.11BSD. Now suppose you find and unknown tape you want to preserve: using dd you could easily 1:1 copy tape files to hard disk files using a SCSI drive and Linux. But: how you know which block size is on the tape? OK, the tape formatter will know, and signal the system when reading. So, you would issue a read command to the OS, giving a block size at least as big as the biggest block you expected to see. The formatter/interface would report the number of bytes in that block. Now, when putting a tape image into a disk file, that gets more complicated. There are various "tape container file" schemes where the block size is encoded into the data in some form. If the tape just contains a single file, it may not matter as the data is just a stream of bytes. But, if the tape contains multiple files, then it is pretty important to preserve the record lengths and file marks. I'm a lot more familiar with IBM and DEC tape formats, and had my own container file format to process disk images of tapes. Unix systems support all these options in one way or another, but tar tapes are just a stream of bytes, and I don't think the block size has much effect. Jon
Re: 9 track tapes and block sizes
On 9/17/20 6:19 AM, Paul Koning via cctalk wrote: > CDC also supports "long blocks" in which the I/O for a single block is done > in pieces, so blocks can be read or written that are longer than what you > would think is the limit from the device limits. I'm not sure how long the > longest "long block" is. When testing out mods to 1LT, the longest record that I ever successfully read and wrote was about half a reel at 556 bpi. Of course, the data was meaningless. This was on a Cyber 74 (6600) using a 657 drive as the target and a derivative of SCOPE 3.1.6 (Special Systems hack). Acoustically, the best tapes were the short-record "stranger" tapes. All sorts of interesting noise. I could tell from across the room when someone was running the tape section of the Navy audit tests for COBOL just by the sounds. --Chuck
Re: 9 track tapes and block sizes
> On Sep 17, 2020, at 1:29 AM, shad via cctalk > wrote: > > Hello, > I have a question about 9 track tapes and block sizes. > What I know is that tape is subdivided in files by means of marks, and each > file is subdivided in blocks of equal size. As others have said, no, "equal size" is not a valid assumption. Even for operating systems that use constant data block sizes, the file labels have a different block size. Consider DEC DOS-11 format, which is used by a number of PDP-11 operating systems. That has a 14 byte file label followed by (usually) 512 byte data records. A very common format is ANSI labeled files. These have several 80 byte file labels, a tape mark, data records that can be any size up to "very large" and may or may not be all the same, then another tape mark and another set of 80 byte labels. Other operating systems may be much stranger. Looking at a CDC 6000 series NOS "deadstart" (system boot") tape, the record sizes of the first 12 records are 486, 930, 936, 825, 945, 960, 930, 951, 141, 486, 1266, 3816. And the largest record size is 3846; the shortest is 6 bytes. Yes, that goes against the "rule" which is really just an IBM convention that blocks smaller than 18 bytes are "noise". CDC also supports "long blocks" in which the I/O for a single block is done in pieces, so blocks can be read or written that are longer than what you would think is the limit from the device limits. I'm not sure how long the longest "long block" is. paul
Re: 9 track tapes and block sizes
On 17/09/2020 07:38, Dave Wade G4UGM via cctalk wrote: > The docs for SIMH .TAP files are here:- > > http://simh.trailing-edge.com/docs/simh_magtape.pdf > > be careful as there are also non-SIMH .tap formats > > In the IBM Mainframe emulation world there is also .AWS, an IBM format > introduced with its P390 Microchannel Mainframe card > and .HET a Hercules extension to .AWS which allows compression > vtapeutils is quite handy for translating between (or listing) a few of the emulated tape formats. It supports AWS, TPC (2 byte record headers) and TAP (Simh - 4 byte record headers). https://sourceforge.net/projects/vtapeutils/ I've used TPC on RSX-11M and VMSTPC on VAX/VMS to read 9-track tapes. Then I used vtapeutils to convert from TPC to TAP format for Simh. Matt
Re: 9 track tapes and block sizes
shad via cctalk wrote:
> Hello,
> I have a question about 9 track tapes and block sizes.
> What I know is that tape is subdivided in files by means of marks, and each
> file is subdivided in blocks of equal size.
> Programs like tar use a specific block size to create files on tape.
> However files can have different block sizes like bootloader file,
> installation dumps and root file system copy on 2.11BSD.
> Now suppose you find and unknown tape you want to preserve: using dd you
> could easily 1:1 copy tape files to hard disk files using a SCSI drive and
> Linux.
> But: how you know which block size is on the tape?
>
> Thanks
> Andrea
I've sometimes created my own simh .tap to Boottape converter (an later
found out that such a thing exists on the net) to create some boottapes
for my pdp11's using a tandberg 8GB quarter inch drive. It worked
(on Freebsd). I've installed 2.11BSD and on a KA630 Quasijarus Unix,
If you set the blocksize to variable you get back the read byte count
from the read() system call und you have to preserve that information to
the new copied tape (that's what the .tap formats are made for).
I know it is the other way around (.tap->tape) but this is my old q&d
tape write program, you hae made the other half yourselves...
#include
#include
#include
#include
#include
#include
#include
#define MAXBLOCK 50
extern int errno;
main()
{
FILE * ifp;
FILE * ofp;
unsigned char buf[MAXBLOCK*512];
unsigned char * bufp;
unsigned long foffs=0;
struct mtop mtio;
int blnum,mark,mt;
unsigned int blen, tlen;
mark=0;
if((ifp=fopen("tapefile","r"))==NULL)
{
fprintf(stderr,"Cant open input file\n");
exit(-1);
}
if(system("mt -f /dev/sa0 blocksize 0")<0)
{
fprintf(stderr,"Can't set variable blocksize on
/dev/sa0\n");
exit(-1);
}
if((mt=open("/dev/sa0", O_CREAT|O_TRUNC|O_WRONLY,0666))<0)
{
fprintf(stderr,"Can't open tape device\n");
exit(-1);
}
blnum=0;
l0:
while(1)
{
if(fread(&blen, sizeof(uint32_t),1,ifp)!=1)
{
fprintf(stderr,"Can't read blocklen\n");
fclose(ifp);
close(mt);
exit(-1);
}
foffs+=sizeof(uint32_t);
if(blen==0)
{
if(mark)
{
fprintf(stderr,"EOT detected\n");
fclose(ifp);
mtio.mt_op = MTWEOF;
mtio.mt_count = 2;
if (ioctl(mt, MTIOCTOP, &mtio) < 0)
fprintf(stderr, "MTIOCTOP err:
%d\n", errno);
close(mt);
exit(0);
foffs+=sizeof(uint32_t);
if(blen==0)
{
if(mark)
{
fprintf(stderr,"EOT detected\n");
fclose(ifp);
mtio.mt_op = MTWEOF;
mtio.mt_count = 2;
if (ioctl(mt, MTIOCTOP, &mtio) < 0)
fprintf(stderr, "MTIOCTOP err:
%d\n", errno);
close(mt);
exit(0);
}
else
{
fprintf(stderr,"tapemark detected
foffs= %08x\n",
blen,foffs);
mark=1;
mtio.mt_op = MTWEOF;
mtio.mt_count = 1;
if (ioctl(mt, MTIOCTOP, &mtio) <
0)
fprintf(stderr, "MTIOCTOP err:
%d\n", errno);
goto l0;
}
}
else
mark=0;
fprintf(stderr,"foffs %08x, blen bl %d =
%d:%04x\n",foffs,bl
Re: 9 track tapes and block sizes
Dave Wade wrote: > The docs for SIMH .TAP files are here:- > > http://simh.trailing-edge.com/docs/simh_magtape.pdf > > be careful as there are also non-SIMH .tap formats Haha, yes very much so. For the fun of it, people like to mix and match these options: - Records padded to even length or not. - Little or big endian record length metadata. - 9-track or 7-track data (the latter may have a parity bit).
RE: 9 track tapes and block sizes
> -Original Message- > From: cctalk On Behalf Of Dennis Boone via > cctalk > Sent: 17 September 2020 06:53 > To: cctalk@classiccmp.org > Subject: Re: 9 track tapes and block sizes > > > What I know is that tape is subdivided in files by means of marks, > and each > file is subdivided in blocks of equal size. > > Er, no. The blocks aren't necessarily of equal size. Unix people who are used to > tar often seem to have this mindset, but the general case is that records can be > of varying size. On labelled tapes the blocks are almost certainly different sizes. Usually one or more 80-charater labels followed by a tape mark followed by data blocks > > > Now suppose you find and unknown tape you want to preserve: using dd > > you could easily 1:1 copy tape files to hard disk files using a SCSI > drive and > Linux. > > DON'T DO THAT. If you use dd, you're throwing away information. > Specifically, you're throwing away knowledge of the block size. Most of the > conventional unix utilities don't care. Many other things do. In many cases, it's > difficult or impossible to reconstruct the block sizes from the content, but even > if it was, it's terrible archival practice. > > There are file formats for containing tape image data. The most common one > is probably the simh .tap format. These all preserve block lengths, tape marks, > indications of errors in reading the original, etc. Many fail to provide a means > to embed metadata, but you can put that in separate adjacent files. > The docs for SIMH .TAP files are here:- http://simh.trailing-edge.com/docs/simh_magtape.pdf be careful as there are also non-SIMH .tap formats In the IBM Mainframe emulation world there is also .AWS, an IBM format introduced with its P390 Microchannel Mainframe card and .HET a Hercules extension to .AWS which allows compression note all the above formats contain info that allows the file to be read block by block, both forwards and backwards, from any position on the tape. > > But: how you know which block size is on the tape? > > Generally speaking, do a read of a blocksize as large or larger than the max on > the tape, and the system will hand you the full record, and the actual number > of bytes read. If you're writing C or scripting code, the unix read() call does > this. From the command line, you can do it with dd - specify a large block size > and a count of 1, and it'll tell you what it actually got as it exits. > > For 9 track, few systems could write blocks larger than 32k or 64k, so those are > decent guesses for "large" there. If it's DLT or something more modern, then > the largest possible block might be a lot larger. The system reading the tape > may impose a limit based on available buffer space. You should able to > iteratively determine the largest size it will accept. > Long block mean fewer inter-block gaps so are often a choice for archives, especially on 6250BPI 9-track tape. If you are lucky and the tape contains labels these usually have info about the block and record size. http://www.setgetweb.com/p/i5/volum.htm Of and yes tape data often has records embedded in the block without terminating , or characters. So it was common to use tapes with 800 byte blocks, with 10 card images in each block, or 1200 byte blocks with ten 120 byte printer lines in each block. For commercial data you may even find multiple record types, so when I worked in insurance, we would have a customer record with one or more policy records. You will also find variable length records and blocks where there are length fields in the blocks. Of course that was a long time ago... .. also on 9-track tapes its possible to read off the end of the tape. > Many of the quarter-inch cartridge formats actually don't support block sizes > other than 512 bytes. If they were used on systems that expected to be able to > write larger and/or variable records, the system hardware or software may > have implemented a logical blocking layer on top of the > 512 hardware layer. If you're reading one of these and don't have the original > hardware/software to decode it, you'll have to figure out how to decipher it > yourself. > > De Hope this helps Dave
Re: 9 track tapes and block sizes
On 9/16/20 10:29 PM, shad via cctalk wrote: > Hello, > I have a question about 9 track tapes and block sizes. > What I know is that tape is subdivided in files by means of marks, and each > file is subdivided in blocks of equal size. > Programs like tar use a specific block size to create files on tape. > However files can have different block sizes like bootloader file, > installation dumps and root file system copy on 2.11BSD. > Now suppose you find and unknown tape you want to preserve: using dd you > could easily 1:1 copy tape files to hard disk files using a SCSI drive and > Linux. > But: how you know which block size is on the tape? Variable length blocks can be handled using the mt(1) command on linux. "defblksize 0" generally will do this, but this varies between distros. Some use the "setblk 0" parameter. Not all hardware and not all drivers support variable block size. As far as 9 track maximum block size, I've seen 132K blocks--and on old CDC 6000 7-track systems, the 1LT driver would permit a block to be as long as a reel of tape, provided the application reading or writing could keep up. 1LT was a bag of worms. ECS accesses could torpedo a transfer; it made the DD60 flicker like crazy. It was a great source of PSRs. --Chuck
Re: 9 track tapes and block sizes
> What I know is that tape is subdivided in files by means of marks, > and each file is subdivided in blocks of equal size. Er, no. The blocks aren't necessarily of equal size. Unix people who are used to tar often seem to have this mindset, but the general case is that records can be of varying size. > Now suppose you find and unknown tape you want to preserve: using dd > you could easily 1:1 copy tape files to hard disk files using a SCSI > drive and Linux. DON'T DO THAT. If you use dd, you're throwing away information. Specifically, you're throwing away knowledge of the block size. Most of the conventional unix utilities don't care. Many other things do. In many cases, it's difficult or impossible to reconstruct the block sizes from the content, but even if it was, it's terrible archival practice. There are file formats for containing tape image data. The most common one is probably the simh .tap format. These all preserve block lengths, tape marks, indications of errors in reading the original, etc. Many fail to provide a means to embed metadata, but you can put that in separate adjacent files. > But: how you know which block size is on the tape? Generally speaking, do a read of a blocksize as large or larger than the max on the tape, and the system will hand you the full record, and the actual number of bytes read. If you're writing C or scripting code, the unix read() call does this. From the command line, you can do it with dd - specify a large block size and a count of 1, and it'll tell you what it actually got as it exits. For 9 track, few systems could write blocks larger than 32k or 64k, so those are decent guesses for "large" there. If it's DLT or something more modern, then the largest possible block might be a lot larger. The system reading the tape may impose a limit based on available buffer space. You should able to iteratively determine the largest size it will accept. Many of the quarter-inch cartridge formats actually don't support block sizes other than 512 bytes. If they were used on systems that expected to be able to write larger and/or variable records, the system hardware or software may have implemented a logical blocking layer on top of the 512 hardware layer. If you're reading one of these and don't have the original hardware/software to decode it, you'll have to figure out how to decipher it yourself. De
9 track tapes and block sizes
Hello, I have a question about 9 track tapes and block sizes. What I know is that tape is subdivided in files by means of marks, and each file is subdivided in blocks of equal size. Programs like tar use a specific block size to create files on tape. However files can have different block sizes like bootloader file, installation dumps and root file system copy on 2.11BSD. Now suppose you find and unknown tape you want to preserve: using dd you could easily 1:1 copy tape files to hard disk files using a SCSI drive and Linux. But: how you know which block size is on the tape? Thanks Andrea
