Re: mt and tar fail on LTO-5 drives
On Aug 21 19:20, bartels wrote: > On 08/21/2013 11:51 AM, Frank Fesevur wrote: > >2013/8/20 Corinna Vinschen: > >> This bug is present since 2004 and nobody noticed it. I guess that > >> means there aren't many people out there actually partitioning their > >> tape drives... > >FYI: we use cygwin tar on a daily base to backup one server to > >LTO2-tapes, but I have never partitioned a tape. Wasn't even aware it > >could be done. > > Partitioning was first introduced on LTO with LTO-5, accommodating LTFS. Oh, right. I didn't know that, but the SCSI references for LTO drives are pretty clear. So, change the list to DDS, SLR, VXA-2, the latter of which is the only one I know which supports select partitioning. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpYYMwLAJoXQ.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On 08/21/2013 11:51 AM, Frank Fesevur wrote: 2013/8/20 Corinna Vinschen: This bug is present since 2004 and nobody noticed it. I guess that means there aren't many people out there actually partitioning their tape drives... FYI: we use cygwin tar on a daily base to backup one server to LTO2-tapes, but I have never partitioned a tape. Wasn't even aware it could be done. Partitioning was first introduced on LTO with LTO-5, accommodating LTFS. - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 21 11:51, Frank Fesevur wrote: > 2013/8/20 Corinna Vinschen: > > This bug is present since 2004 and nobody noticed it. I guess that > > means there aren't many people out there actually partitioning their > > tape drives... > > FYI: we use cygwin tar on a daily base to backup one server to > LTO2-tapes, but I have never partitioned a tape. Wasn't even aware it > could be done. Thanks for your info. The other alternative would have been that nobody uses Cygwin with tapes at all, which would have been kind of frustrating... :} Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpiOY0dHeMyM.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
2013/8/20 Corinna Vinschen: > This bug is present since 2004 and nobody noticed it. I guess that > means there aren't many people out there actually partitioning their > tape drives... FYI: we use cygwin tar on a daily base to backup one server to LTO2-tapes, but I have never partitioned a tape. Wasn't even aware it could be done. Regards, Frank -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 20 16:44, Corinna Vinschen wrote: > On Aug 20 16:06, bartels wrote: > > On 08/20/2013 03:32 PM, Corinna Vinschen wrote: > > >On Aug 20 13:57, bartels wrote: > > >>) > > >> > > >> > > >>Thanks, I think we can close this one! > > >Glad to read that. You could do me a favor, though. At the moment I > > >have no tape drive on any Windows machine for testing. Can you please > > >test the new `partseek' command in mt 2.4.0 which I announced a couple > > >of minutes ago and see if it does what it advertises in `man mt'? > > > > > > > > > > Sure. How long does it take to propagate to the mirrors? > > All I see is 2.3.3.1 > > Depends on the mirror. > > > And would that be experimental, or current? > > 2.4.0 is current now. Never mind. With a lot of help by my collegues big in virtualization, I managed to set up a Windows VM with access to the SCSI tape drive in the host machine. The partseek command works as advertized, apparently. However, while testing I found that the mkpartition command didn't work due to a bug in Cygwin. So you can use partitioned LTO tapes, but you can't partition LTO tapes(*) with Cygwin up to and including the current release 1.7.24. I fixed that in CVS so that will work with the next Cygwin release. Corinna (*) Actually, what doesn't work is creating two partitions on tape drives supporting initiator partitions (like DDS, LTO, SLR). What does work is creating a single partition on such drives, or to create one or two partitions on drives only supporting fixed partitions (like older SLR). This bug is present since 2004 and nobody noticed it. I guess that means there aren't many people out there actually partitioning their tape drives... -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpLUpoOr4Q6X.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On Aug 20 16:06, bartels wrote: > On 08/20/2013 03:32 PM, Corinna Vinschen wrote: > >On Aug 20 13:57, bartels wrote: > >>) > >> > >> > >>Thanks, I think we can close this one! > >Glad to read that. You could do me a favor, though. At the moment I > >have no tape drive on any Windows machine for testing. Can you please > >test the new `partseek' command in mt 2.4.0 which I announced a couple > >of minutes ago and see if it does what it advertises in `man mt'? > > > > > > Sure. How long does it take to propagate to the mirrors? > All I see is 2.3.3.1 Depends on the mirror. > And would that be experimental, or current? 2.4.0 is current now. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpX3jXIDkJME.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On 08/20/2013 03:32 PM, Corinna Vinschen wrote: On Aug 20 13:57, bartels wrote: ) Thanks, I think we can close this one! Glad to read that. You could do me a favor, though. At the moment I have no tape drive on any Windows machine for testing. Can you please test the new `partseek' command in mt 2.4.0 which I announced a couple of minutes ago and see if it does what it advertises in `man mt'? Sure. How long does it take to propagate to the mirrors? All I see is 2.3.3.1 And would that be experimental, or current? - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 20 13:57, bartels wrote: > On 08/20/2013 11:05 AM, Corinna Vinschen wrote: > >>...or the *partition* is at its end. Something occured to me this > >>morning. Are you using used tapes, rather than fresh ones? Are the > >>tapes you're using partitioned, by any chance, maybe because LTFS > >>partitions the tape in two partitions, one for the files and one for > >>metadata? > >> > >>This would explain the low capacity you see in mt status output. If the > >>tape is partitioned, the capacity returned by GetTapeParameters is not > >>the size of the entire tape, but the size of the current partition(*). > >>And partition 0 is probably the metadata partition. > >> > >>This also explains why you get a supposedly early ERROR_END_OF_MEDIA. > >>The partition is just not bigger. > >> > >>Try this: > >>[...] > > $ mt -f /dev/nst0 setpart 0 > > $ mt -f /dev/nst0 status 2 > > $ mt -f /dev/nst0 setpart 1 > > $ mt -f /dev/nst0 status 2 > > > >I wrote the setpart and mkpart commands before mt on Linux had them and > >naturally they are now using different strings. I'll have another look > >into mt to make it more compatible with mt on Linux. > > > > > Well, Corinna, you really saved the day. I missed the partition stuff > completely. > > $ mt -f /dev/nst0 status 2 > drive type : 56 (STK 9840) > tape capacity: 1419369472 KB remaining: 1324161024 KB > current file : -1 active partition :1 > current block: -1 cur logical block: 195351 > General status bits on (109): > ONLINE IM_REP_EN HW_ECC > min block size :2 max block size : 524288 > def block size : 131072 cur block size :0 > density code : 58 (Ultrium LTO-5) > > > Thanks, I think we can close this one! Glad to read that. You could do me a favor, though. At the moment I have no tape drive on any Windows machine for testing. Can you please test the new `partseek' command in mt 2.4.0 which I announced a couple of minutes ago and see if it does what it advertises in `man mt'? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgp9zZQTifme7.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On 08/20/2013 11:05 AM, Corinna Vinschen wrote: On Aug 20 10:29, Corinna Vinschen wrote: On Aug 19 21:10, bartels wrote: On 08/19/2013 07:12 PM, Corinna Vinschen wrote: It would be interesting to see the OS error codes. If you run tar under strace, the trace output should contain a line like [...] write: Win32 error or [...] close: Win32 error The value of is what I'm curious about. Here is the error: 15692 21364024 [main] tar 5700 mtinfo_drive::error: write: Win32 error 1100 ERROR_END_OF_MEDIA. The OS really thinks the medium is at its end... ...or the *partition* is at its end. Something occured to me this morning. Are you using used tapes, rather than fresh ones? Are the tapes you're using partitioned, by any chance, maybe because LTFS partitions the tape in two partitions, one for the files and one for metadata? This would explain the low capacity you see in mt status output. If the tape is partitioned, the capacity returned by GetTapeParameters is not the size of the entire tape, but the size of the current partition(*). And partition 0 is probably the metadata partition. This also explains why you get a supposedly early ERROR_END_OF_MEDIA. The partition is just not bigger. Try this: $ mt -f /dev/nst0 setpartition 0 $ mt -f /dev/nst0 status 2 $ mt -f /dev/nst0 setpartition 1 $ mt -f /dev/nst0 status 2 Make that $ mt -f /dev/nst0 setpart 0 $ mt -f /dev/nst0 status 2 $ mt -f /dev/nst0 setpart 1 $ mt -f /dev/nst0 status 2 I wrote the setpart and mkpart commands before mt on Linux had them and naturally they are now using different strings. I'll have another look into mt to make it more compatible with mt on Linux. Well, Corinna, you really saved the day. I missed the partition stuff completely. $ mt -f /dev/nst0 status 2 drive type : 56 (STK 9840) tape capacity: 1419369472 KB remaining: 1324161024 KB current file : -1 active partition :1 current block: -1 cur logical block: 195351 General status bits on (109): ONLINE IM_REP_EN HW_ECC min block size :2 max block size : 524288 def block size : 131072 cur block size :0 density code : 58 (Ultrium LTO-5) Thanks, I think we can close this one! - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 20 10:29, Corinna Vinschen wrote: > On Aug 19 21:10, bartels wrote: > > On 08/19/2013 07:12 PM, Corinna Vinschen wrote: > > > > > >It would be interesting to see the OS error codes. If you run tar under > > >strace, the trace output should contain a line like > > > > > > [...] write: Win32 error > > > > > >or > > > > > > [...] close: Win32 error > > > > > >The value of is what I'm curious about. > > > > Here is the error: > > > > 15692 21364024 [main] tar 5700 mtinfo_drive::error: write: Win32 error 1100 > > ERROR_END_OF_MEDIA. The OS really thinks the medium is at its end... > > ...or the *partition* is at its end. Something occured to me this > morning. Are you using used tapes, rather than fresh ones? Are the > tapes you're using partitioned, by any chance, maybe because LTFS > partitions the tape in two partitions, one for the files and one for > metadata? > > This would explain the low capacity you see in mt status output. If the > tape is partitioned, the capacity returned by GetTapeParameters is not > the size of the entire tape, but the size of the current partition(*). > And partition 0 is probably the metadata partition. > > This also explains why you get a supposedly early ERROR_END_OF_MEDIA. > The partition is just not bigger. > > Try this: > > $ mt -f /dev/nst0 setpartition 0 > $ mt -f /dev/nst0 status 2 > $ mt -f /dev/nst0 setpartition 1 > $ mt -f /dev/nst0 status 2 Make that $ mt -f /dev/nst0 setpart 0 $ mt -f /dev/nst0 status 2 $ mt -f /dev/nst0 setpart 1 $ mt -f /dev/nst0 status 2 I wrote the setpart and mkpart commands before mt on Linux had them and naturally they are now using different strings. I'll have another look into mt to make it more compatible with mt on Linux. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpFBYt2tAjSG.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On Aug 19 21:10, bartels wrote:
> On 08/19/2013 07:12 PM, Corinna Vinschen wrote:
> >
> >It would be interesting to see the OS error codes. If you run tar under
> >strace, the trace output should contain a line like
> >
> > [...] write: Win32 error
> >
> >or
> >
> > [...] close: Win32 error
> >
> >The value of is what I'm curious about.
>
> Here is the error:
>
> 15692 21364024 [main] tar 5700 mtinfo_drive::error: write: Win32 error 1100
ERROR_END_OF_MEDIA. The OS really thinks the medium is at its end...
...or the *partition* is at its end. Something occured to me this
morning. Are you using used tapes, rather than fresh ones? Are the
tapes you're using partitioned, by any chance, maybe because LTFS
partitions the tape in two partitions, one for the files and one for
metadata?
This would explain the low capacity you see in mt status output. If the
tape is partitioned, the capacity returned by GetTapeParameters is not
the size of the entire tape, but the size of the current partition(*).
And partition 0 is probably the metadata partition.
This also explains why you get a supposedly early ERROR_END_OF_MEDIA.
The partition is just not bigger.
Try this:
$ mt -f /dev/nst0 setpartition 0
$ mt -f /dev/nst0 status 2
$ mt -f /dev/nst0 setpartition 1
$ mt -f /dev/nst0 status 2
> >>I suppose there is nothing for cygwin to do, other
> >>than adding some items to the 'density' list:
> >> {0x44, "Ultrium LTO-3"},
> >mt already knows LTO-3 and 4.
>
> Which version is that? I have mt V2.3.2 and do not see it in the code.
The current version is 2.3.3. I'll release a 2.3.4 soon.
Corinna
(*)
http://msdn.microsoft.com/en-us/library/windows/desktop/aa362526%28v=vs.85%29.aspx
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
pgphEbRT_IlTt.pgp
Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On 8/19/2013 3:53 PM, bartels wrote: On 08/19/2013 08:44 PM, Guy Harrison wrote: Thanks Guy, but these troubles are only seen with tar / mt. Both drives work fine with LTFS. I know I am grasping at straws, but I wonder if manually setting the tape length with tar's --tape-length option would be of any use here ... Not sure if you can fool it into continuing ... Regards -- Eliot Moss -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 08:44 PM, Guy Harrison wrote: This *might* be useful.. http://www-01.ibm.com/support/docview.wss?uid=swg21393009 ..particularly.. http://www-01.ibm.com/software/sysmgmt/products/support/IBM_TSM_Supported_Devices_for_AIXHPSUNWIN.html Please bear in mind the page is for TSM (Tivoli Storage Manager) so even if you find your drive in there it still may mean you need a driver. It is often the case when an IBM driver is required that both the driver and drive firmware must match. I'm going to duck out now except to say one more thing. When I've had to install TSM on a windows machine & it's gone wrong it has always been the driver (or getting windows to use the right driver) at fault. Thanks Guy, but these troubles are only seen with tar / mt. Both drives work fine with LTFS. - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 07:12 PM, Corinna Vinschen wrote:
It would be interesting to see the OS error codes. If you run tar under
strace, the trace output should contain a line like
[...] write: Win32 error
or
[...] close: Win32 error
The value of is what I'm curious about.
Here is the error:
15692 21364024 [main] tar 5700 mtinfo_drive::error: write: Win32 error 1100
I suppose there is nothing for cygwin to do, other
than adding some items to the 'density' list:
{0x44, "Ultrium LTO-3"},
mt already knows LTO-3 and 4.
Which version is that? I have mt V2.3.2 and do not see it in the code.
- Bartels
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Monday 19 August 2013 17:40:31 bartels wrote: > On 08/19/2013 06:31 PM, Eliot Moss wrote: > > First of all, it's not clear to me whether it is a Microsoft problem > > or a device driver problem. I would see what's known about the > > behavior of the devices and their drivers with the specific Windows > > version you have. Of course it could be that other tools don't need to > > ask those same questions, for some reason, so that they end up working > > fine .. > > Good thinking. > > Running W7 64 professional. > > It detects the HP drive and automatically installs some driver. > This driver shows the same behaviour as the HP driver. > > And then there is the IBM driver. Not sure if W7 installs a default > driver for that. Same story. This *might* be useful.. http://www-01.ibm.com/support/docview.wss?uid=swg21393009 ..particularly.. http://www-01.ibm.com/software/sysmgmt/products/support/IBM_TSM_Supported_Devices_for_AIXHPSUNWIN.html Please bear in mind the page is for TSM (Tivoli Storage Manager) so even if you find your drive in there it still may mean you need a driver. It is often the case when an IBM driver is required that both the driver and drive firmware must match. I'm going to duck out now except to say one more thing. When I've had to install TSM on a windows machine & it's gone wrong it has always been the driver (or getting windows to use the right driver) at fault. Guy > What does that tell you? > > - Bartels > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 07:06 PM, Eliot Moss wrote: Not so much, yet. Presumably HP and IBM offer some kind of backup utility with the drive software, Sure: both drives work fine using LTFS. Plenty of bytes there. No problem at all. - Bartels. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 07:12 PM, Corinna Vinschen wrote: To be really sure, you could add debug output to Cygwin's mtinfo_drive::get_mp function and rebuild the DLL: diff -u -p -r1.90 fhandler_tape.cc Gladly. I would like to close this one way or the other. Which package and which dll are you talking about? My knowledge of cygwin internals is nearly zero. - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 8/19/2013 1:13 PM, Corinna Vinschen wrote: On Aug 19 18:40, bartels wrote: On 08/19/2013 06:31 PM, Eliot Moss wrote: In theory, typical SCSI/SAS tapes should use the default tape.sys driver, IIRC. There's nothing special with them, using standard SCSI/SAS commands. Thank you, Corinna; if that's so, then my suggestions around testing various drivers would not be useful. It could then be an OS internal problem or a problem with the driver -- but both are of Microsoft manufacture ... Regards -- Eliot -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 19 18:40, bartels wrote: > On 08/19/2013 06:31 PM, Eliot Moss wrote: > > > >First of all, it's not clear to me whether it is a Microsoft problem > >or a device driver problem. I would see what's known about the behavior > >of the devices and their drivers with the specific Windows version you > >have. Of course it could be that other tools don't need to ask those > >same questions, for some reason, so that they end up working fine .. > > Good thinking. > > Running W7 64 professional. > > It detects the HP drive and automatically installs some driver. > This driver shows the same behaviour as the HP driver. > > And then there is the IBM driver. Not sure if W7 installs a default driver > for that. > Same story. In theory, typical SCSI/SAS tapes should use the default tape.sys driver, IIRC. There's nothing special with them, using standard SCSI/SAS commands. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpRCN2KFHvFU.pgp Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On Aug 19 18:21, bartels wrote:
> On 08/19/2013 12:19 PM, Corinna Vinschen wrote:
> >On Aug 17 20:35, bartels wrote:
> >>Hello People,
> >>
> >>I have here two SAS connected LTO-5 drives: one IBM and one HP.
> >>
> >>Both drives work work fine, but sadly mt does not.
> >>The size reported by mt is a meager 35 GB, instead of the expected 1.5TB
> >>
> >>I have tried both an older 32 bit and the 'current' 64 bit cygwin: same
> >>result.
> >>
> >>Writing to tape works fine with tar, but the tape is quickly considered
> >>'full'.
> >>Is there any hope of fixing this? LTO-6 is already out there.
> >I don't know. Cygwin uses the Win32 tape API. The OS function
> >GetTapeParameters returns the capacity and the # of remaining bytes as 8
> >bytes LARGE_INTEGER value. The size of LT)-5 or LTO-6 should fit easily
> >into that.
> >
> >I just checked that the value is stored within Cygwin as 8 byte long
> >long value, so no problem there. The mt tool prints the value as %lld
> >value, so it should print it correctly as 8 byte value. From what I can
> >see, the wrong value *seems* to be returned by the OS.
> >[...]
> >But the bottom line is, I have no way to test and debug that, since I
> >don't have access to an LTO-5 drive, nor do I have a Windows machine
> >with SAS controller. However, since Cygwin as well as the mt tool are
> >Open Source, maybe you can have a look and debug this issue?
>
> Thanks for your time.
>
> I did have a look.
> The size reported to mt by the os seems to be the root cause: it is
> 38_247_858_176 bytes, instead of 1.5TB
Really? Can you safely confirm that this is the value returned by the
GetTapeParameters call in mtinfo_drive::get_mp()? If it's returned by
mt, it's not a safe bet since the data could be screwed up due some
as yet unknown Cygwin or mt bug.
To be really sure, you could add debug output to Cygwin's
mtinfo_drive::get_mp function and rebuild the DLL:
diff -u -p -r1.90 fhandler_tape.cc
--- fhandler_tape.cc19 Aug 2013 10:24:37 - 1.90
+++ fhandler_tape.cc19 Aug 2013 16:40:23 -
@@ -101,6 +101,8 @@ mtinfo_drive::get_mp (HANDLE mt)
{
DWORD len = sizeof _mp;
TAPE_FUNC (GetTapeParameters (mt, GET_TAPE_MEDIA_INFORMATION, &len, &_mp));
+ if (!lasterr)
+system_printf ("Capacity %D Remaining %D", _mp.Capacity, _mp.Remaining);
return error ("get_mp");
}
This prints the raw values returned by the OS to your terminal.
> My tar blocksize for the earlier test was 64 KB, within spec.
> HP has a max blocksize 512 KB, which seems to work fine.
> Still, it stops on an error after 36 GB (just as it does with -b 2048):
>
> + tar -f /dev/nst0 -c -b 524288 blah
> tar: /dev/nst0: Cannot write: No space left on device
> tar: Error is not recoverable: exiting now
>
> mt reports matching capacity, remaining and block:
>
> $ mt -f /dev/nst0 status 2
> drive type : 56 (STK 9840)
> tape capacity: 38247858176 B
> tape capacity: 37351424 KB remaining : 2105344 KB
> current file : -1 active partition :0
> current block: -1 cur logical block:73373
> General status bits on (10b):
> ONLINE IM_REP_EN HW_ECC HW_COMP
> min block size :2 max block size : 524288
> def block size : 131072 cur block size :0
> density code : 58 (Ultrium LTO-5)
>
> The IBM drive shows similar numbers and behaviour, slightly
> different error msg ('Cannot close' instead of 'Cannot write'):
> + tar -f /dev/nst1 -c -b 2048 blah
> tar: /dev/nst1: Cannot close: No space left on device
> tar: Exiting with failure status due to previous errors
It would be interesting to see the OS error codes. If you run tar under
strace, the trace output should contain a line like
[...] write: Win32 error
or
[...] close: Win32 error
The value of is what I'm curious about.
> I suppose there is nothing for cygwin to do, other
> than adding some items to the 'density' list:
> {0x44, "Ultrium LTO-3"},
mt already knows LTO-3 and 4.
> {0x58, "Ultrium LTO-5"},
Thanks, I add that to CVS.
> How do I best report this to Microsoft?
Assuming this is really a OS bug and not some error in Cygwin we just
didn't figure out yet, reporting to Microsoft requires to open a support
case. A testcase with plain OS tape API functions and without any
Cygwin functions in between would be required, otherwise it will be
immediately blamed on Cygwin. And some endurance is required to get
over the first support line.
> Any chance they fix it, you reckon?
I would be overly optimistic to say yes here... a careful "maybe"
is ok, though.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
pgpxzDQXFNTID.pgp
Description: PGP signature
Re: mt and tar fail on LTO-5 drives
On 8/19/2013 12:40 PM, bartels wrote: On 08/19/2013 06:31 PM, Eliot Moss wrote: First of all, it's not clear to me whether it is a Microsoft problem or a device driver problem. I would see what's known about the behavior of the devices and their drivers with the specific Windows version you have. Of course it could be that other tools don't need to ask those same questions, for some reason, so that they end up working fine .. Good thinking. Running W7 64 professional. It detects the HP drive and automatically installs some driver. This driver shows the same behaviour as the HP driver. And then there is the IBM driver. Not sure if W7 installs a default driver for that. Same story. What does that tell you? Not so much, yet. Presumably HP and IBM offer some kind of backup utility with the drive software, that you can use to see whether the drive can accept more that so many Gbytes to write. Or you could use whatever simple backup utility comes with Windows to test. The question is whether *Windows* based software (not necessarily from Microsoft) can write more to the drive than mt manages to. That might be a useful clue. However, if other software *can* write more to the drive, the question remains "How do they do it?" given the apparent OS behavior. If there are two or three different drivers possible (Windows default, HP, IBM), it might be interesting to see if the information returned to cygwin from the Windows OS is the same for each driver ... Regards -- Eliot Moss -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 06:31 PM, Eliot Moss wrote: First of all, it's not clear to me whether it is a Microsoft problem or a device driver problem. I would see what's known about the behavior of the devices and their drivers with the specific Windows version you have. Of course it could be that other tools don't need to ask those same questions, for some reason, so that they end up working fine .. Good thinking. Running W7 64 professional. It detects the HP drive and automatically installs some driver. This driver shows the same behaviour as the HP driver. And then there is the IBM driver. Not sure if W7 installs a default driver for that. Same story. What does that tell you? - Bartels -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 8/19/2013 12:21 PM, bartels wrote:
On 08/19/2013 12:19 PM, Corinna Vinschen wrote:
On Aug 17 20:35, bartels wrote:
Hello People,
I have here two SAS connected LTO-5 drives: one IBM and one HP.
Both drives work work fine, but sadly mt does not.
The size reported by mt is a meager 35 GB, instead of the expected 1.5TB
I have tried both an older 32 bit and the 'current' 64 bit cygwin: same result.
Writing to tape works fine with tar, but the tape is quickly considered 'full'.
Is there any hope of fixing this? LTO-6 is already out there.
I don't know. Cygwin uses the Win32 tape API. The OS function
GetTapeParameters returns the capacity and the # of remaining bytes as 8
bytes LARGE_INTEGER value. The size of LT)-5 or LTO-6 should fit easily
into that.
I just checked that the value is stored within Cygwin as 8 byte long
long value, so no problem there. The mt tool prints the value as %lld
value, so it should print it correctly as 8 byte value. From what I can
see, the wrong value *seems* to be returned by the OS.
Also, the write(2) function does not check for the remaining bytes, so I
wonder why tar should fail prematurely, unless there's a problem with
the block number. The OS function GetTapePosition returns the current
block number as LARGE_INTEGER, but Cywin stores it in a 32 bit int. So
the block number overflows after 4 billion blocks. But even with a
small blocksize of 512 bytes this would only occur at about 2 TB of
data, long after the end of the tape. Despite that this is more of a
theoretical problem, the mtop struct to pass parameters to ioctl(fd,
MTIOCTOP, ...) only allows 4 byte count values anyway, even on Linux.
Another potential problem is if you try to use blocksizes > 64K. I don't
know if that's still a problem in newer Windows versions, but with older
versions including Windows XP, the OS didn't handle blocksizes > 64K
correctly and we got spurious error messages. Something about this
should be in the mailing list archives of old.
But the bottom line is, I have no way to test and debug that, since I
don't have access to an LTO-5 drive, nor do I have a Windows machine
with SAS controller. However, since Cygwin as well as the mt tool are
Open Source, maybe you can have a look and debug this issue?
Thanks for your time.
I did have a look.
The size reported to mt by the os seems to be the root cause: it is
38_247_858_176 bytes, instead of
1.5TB
My tar blocksize for the earlier test was 64 KB, within spec.
HP has a max blocksize 512 KB, which seems to work fine.
Still, it stops on an error after 36 GB (just as it does with -b 2048):
+ tar -f /dev/nst0 -c -b 524288 blah
tar: /dev/nst0: Cannot write: No space left on device
tar: Error is not recoverable: exiting now
mt reports matching capacity, remaining and block:
$ mt -f /dev/nst0 status 2
drive type : 56 (STK 9840)
tape capacity: 38247858176 B
tape capacity: 37351424 KB remaining : 2105344 KB
current file : -1 active partition :0
current block: -1 cur logical block:73373
General status bits on (10b):
ONLINE IM_REP_EN HW_ECC HW_COMP
min block size :2 max block size : 524288
def block size : 131072 cur block size :0
density code : 58 (Ultrium LTO-5)
The IBM drive shows similar numbers and behaviour, slightly
different error msg ('Cannot close' instead of 'Cannot write'):
+ tar -f /dev/nst1 -c -b 2048 blah
tar: /dev/nst1: Cannot close: No space left on device
tar: Exiting with failure status due to previous errors
I suppose there is nothing for cygwin to do, other
than adding some items to the 'density' list:
{0x44, "Ultrium LTO-3"},
{0x58, "Ultrium LTO-5"},
How do I best report this to Microsoft?
Any chance they fix it, you reckon?
First of all, it's not clear to me whether it is a Microsoft problem
or a device driver problem. I would see what's known about the behavior
of the devices and their drivers with the specific Windows version you
have. Of course it could be that other tools don't need to ask those
same questions, for some reason, so that they end up working fine ...
Regards -- Eliot Moss
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On 08/19/2013 12:19 PM, Corinna Vinschen wrote:
On Aug 17 20:35, bartels wrote:
Hello People,
I have here two SAS connected LTO-5 drives: one IBM and one HP.
Both drives work work fine, but sadly mt does not.
The size reported by mt is a meager 35 GB, instead of the expected 1.5TB
I have tried both an older 32 bit and the 'current' 64 bit cygwin: same result.
Writing to tape works fine with tar, but the tape is quickly considered 'full'.
Is there any hope of fixing this? LTO-6 is already out there.
I don't know. Cygwin uses the Win32 tape API. The OS function
GetTapeParameters returns the capacity and the # of remaining bytes as 8
bytes LARGE_INTEGER value. The size of LT)-5 or LTO-6 should fit easily
into that.
I just checked that the value is stored within Cygwin as 8 byte long
long value, so no problem there. The mt tool prints the value as %lld
value, so it should print it correctly as 8 byte value. From what I can
see, the wrong value *seems* to be returned by the OS.
Also, the write(2) function does not check for the remaining bytes, so I
wonder why tar should fail prematurely, unless there's a problem with
the block number. The OS function GetTapePosition returns the current
block number as LARGE_INTEGER, but Cywin stores it in a 32 bit int. So
the block number overflows after 4 billion blocks. But even with a
small blocksize of 512 bytes this would only occur at about 2 TB of
data, long after the end of the tape. Despite that this is more of a
theoretical problem, the mtop struct to pass parameters to ioctl(fd,
MTIOCTOP, ...) only allows 4 byte count values anyway, even on Linux.
Another potential problem is if you try to use blocksizes > 64K. I don't
know if that's still a problem in newer Windows versions, but with older
versions including Windows XP, the OS didn't handle blocksizes > 64K
correctly and we got spurious error messages. Something about this
should be in the mailing list archives of old.
But the bottom line is, I have no way to test and debug that, since I
don't have access to an LTO-5 drive, nor do I have a Windows machine
with SAS controller. However, since Cygwin as well as the mt tool are
Open Source, maybe you can have a look and debug this issue?
Thanks for your time.
I did have a look.
The size reported to mt by the os seems to be the root cause: it is
38_247_858_176 bytes, instead of 1.5TB
My tar blocksize for the earlier test was 64 KB, within spec.
HP has a max blocksize 512 KB, which seems to work fine.
Still, it stops on an error after 36 GB (just as it does with -b 2048):
+ tar -f /dev/nst0 -c -b 524288 blah
tar: /dev/nst0: Cannot write: No space left on device
tar: Error is not recoverable: exiting now
mt reports matching capacity, remaining and block:
$ mt -f /dev/nst0 status 2
drive type : 56 (STK 9840)
tape capacity: 38247858176 B
tape capacity: 37351424 KB remaining : 2105344 KB
current file : -1 active partition :0
current block: -1 cur logical block:73373
General status bits on (10b):
ONLINE IM_REP_EN HW_ECC HW_COMP
min block size :2 max block size : 524288
def block size : 131072 cur block size :0
density code : 58 (Ultrium LTO-5)
The IBM drive shows similar numbers and behaviour, slightly
different error msg ('Cannot close' instead of 'Cannot write'):
+ tar -f /dev/nst1 -c -b 2048 blah
tar: /dev/nst1: Cannot close: No space left on device
tar: Exiting with failure status due to previous errors
I suppose there is nothing for cygwin to do, other
than adding some items to the 'density' list:
{0x44, "Ultrium LTO-3"},
{0x58, "Ultrium LTO-5"},
How do I best report this to Microsoft?
Any chance they fix it, you reckon?
- Bartels.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: mt and tar fail on LTO-5 drives
On Aug 17 20:35, bartels wrote: > Hello People, > > I have here two SAS connected LTO-5 drives: one IBM and one HP. > > Both drives work work fine, but sadly mt does not. > The size reported by mt is a meager 35 GB, instead of the expected 1.5TB > > I have tried both an older 32 bit and the 'current' 64 bit cygwin: same > result. > > Writing to tape works fine with tar, but the tape is quickly considered > 'full'. > Is there any hope of fixing this? LTO-6 is already out there. I don't know. Cygwin uses the Win32 tape API. The OS function GetTapeParameters returns the capacity and the # of remaining bytes as 8 bytes LARGE_INTEGER value. The size of LT)-5 or LTO-6 should fit easily into that. I just checked that the value is stored within Cygwin as 8 byte long long value, so no problem there. The mt tool prints the value as %lld value, so it should print it correctly as 8 byte value. From what I can see, the wrong value *seems* to be returned by the OS. Also, the write(2) function does not check for the remaining bytes, so I wonder why tar should fail prematurely, unless there's a problem with the block number. The OS function GetTapePosition returns the current block number as LARGE_INTEGER, but Cywin stores it in a 32 bit int. So the block number overflows after 4 billion blocks. But even with a small blocksize of 512 bytes this would only occur at about 2 TB of data, long after the end of the tape. Despite that this is more of a theoretical problem, the mtop struct to pass parameters to ioctl(fd, MTIOCTOP, ...) only allows 4 byte count values anyway, even on Linux. Another potential problem is if you try to use blocksizes > 64K. I don't know if that's still a problem in newer Windows versions, but with older versions including Windows XP, the OS didn't handle blocksizes > 64K correctly and we got spurious error messages. Something about this should be in the mailing list archives of old. But the bottom line is, I have no way to test and debug that, since I don't have access to an LTO-5 drive, nor do I have a Windows machine with SAS controller. However, since Cygwin as well as the mt tool are Open Source, maybe you can have a look and debug this issue? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpwUfLAERZWL.pgp Description: PGP signature
